About three decades ago, pixels where as big as toasters – and mostly green.

Today we have millions of them to deal with, and the terms anti-/aliasing are part of our daily live and every time an image is scaled, it gets interpolated to keep those bad “stairs” away from our eyes.

But what if the source image is aliased in the first place?

Checkout the demo of SmartAA!



While upsaling is the most common cause of aliasing, there are images that are naturally aliased – for example images from a 3D render engine.

Of course there are many ways to antialias cg images when generating them – for example by rendering to a higher resolution than targeted and downscaling the result (FSAA)- or by rendering and merging four slightly offset variations.

While this is as computational intensive as it sounds, modern GPU´s can handle this task rather easily as they have their buffers and hardwired magic to do so, but still it’s not uncommon to come along aliased images in the world of computer graphics.

So is all this material “useless” or can an image that has never seen any antialiasing, or has been upscaled with the nearest neighbour method, be reconstructed?

In fact all that modern imaging software does when rescaling an image is reconstructing its pixels to fit the new dimensions – most times with a bicubic filter.

SmartAA is a bicubic BC-spline reconstruction filter for PixelBender.

Its 3×3 kernel gathers all neighboring pixels of the one to reconstruct and weights them with the Mitchell-Netravali(german link) filter.

In essence, this means that each pixel is modified by the average color of its surrounding friends.
While “average” being the wrong term here as the influence of each pixel is determined by the spline curve of the Mitchell-Netravali filter.

This eliminates those hard, “stair-like” artifacts of the aliasing by visually “softening” or unsharping the image.

However this is not simply a destructive blurring effect but a reconstructive analysis of the source “signal”.

There are many known filters that do this hard job – it´s not easy to reconstruct information that was partly lost (or never there in the first place).

Thus all of them suffer of the same issues – the result always shows some artifacts.
Mostly it’s a visible unsharping, anisotropy or the so-called “ringing“.

In general bicubic reconstruction methods create the best results.

Among the various possible filter methods the windowed Sinc(Lanczos)- and Mitchell-Netravali filter are chosen the most as their output is considered to be the best mix of the inevitable unsharping, anisotropy and ringing.

Just for the record: there are “better” solutions, the so called “adaptive” or “smart” filters that try separate methods on edges, fills and other structures.
They are particularly useful when working with sequences of moving images (video) but again slow to compute.

SmartAA on the other hand is a quite simple implementation of the M.-N. filter and runs blazingly fast when run on the GPU – producing about 70 redraws per sec on a 1024*768 image.

Unfortunately it´s not nearly as fast on the CPU.
If you have a rather slow machine, you will have seen the demo struggling to keep a 24fps framerate, so its just! that bit too slow to to antialias the pixelated output of a flash based C64 emulator.

That would have been cool… would love to see something like that.

I kept it flash compatible anyway – still there might be applications.

Its a matter of personal reception, but to me the M.-N. filter looks better on videos as the inbuild “smooth”-method.
Anyway – flash video has been criticized enaugh for high cpu usage so lets not make things worse

Also: like any reconstruction filter it won´t work in any given scenario as there are some natural and some specific limitations.

So what can SmartAA do and what not? Here is a feature/limitation list:

Features:

- Works with Photoshop, After Effects, Flash
- Supports alpha channel
- Is real-time

Limitations:

- Does no up/down scaling – just reconstruction of anti aliasing
- Is scenario specific – for example text will often look strange
- works only with artifacts in the same pixel space.

Let me elaborate on the last point:

As SmartAA takes only neighbouring pixels into account it isn´t able to reconstruct aliasing effects that span over greater distances than 2 pixels.
For example if an image exists of 4*4 pixel blocks of the same color, those blocks would get rounded on the edges but will still be visible as rounded blocks.

But it works ideal on sources like aliased output from a render engine, or images that have been upscaled x2 with the nearest neighbour method.


Conclusion:

Aliasing should be fought before it ends up within an image as there are more sophisticated methods to keep it from emerging than to reconstruct it after the damage is done.

But sometimes it´s inevitable and if you are looking for a simple but effective way to get rid of that ugly blocks within an image or video – SmartAA will do the trick.


Download:

Grab the Shader here and use it with Photoshop (Pixel Bender Plugin needed), After Effects or Flash!


Hint:

Try to add a sharpen filter or better an unsharpen mask to the result to counter the blur effect.
Don´t forget to use the width/hight properties – SmartAA will work without but there will be artifacts on the upper- and lower right corner.

I hope this is helpful to anyone out there and would love to hear some comments or ideas.

Cheers!

Screen Space Ambient Occlusion or SSAO is a method to compute an approximated AO pass in real-time!

This is done by using the depth information of a 3D object or scene that has been stored from the Z-(depth) buffer of the renderer.

Go ahead and try the interactive demo or see a video example!

The technique was introduced 2007 at Siggraph by Crytek Studios and is becoming an often used feature within next generation live-3d engines or cg/video post applications.

After playing with the ray traced inside-out AO and having to realize that it´s not quite doable with Pixel Blender, I turned to SSAO and quickly got some nice results.

In short it´s all about importance sampling.

SmartSSAO looks up the depth information of 12 surrounding samples and checks if they get occluded to average the AO value.

The disadvantages lie in the nature of SSAO being “screen spaced” which means that it can´t take objects from the off into account, and the result has to be blurred to reduce the inevitable noise.
Also there is no ray bouncing, so classic AO is visually superior – but hey… who can argue with “real-time”?

View another video of the buffer and the solo SSAO pass without/with blurring.
Please note that those strange looking anomalies on the buddahs arms originate from the badly ranged zmap I produced in a hurry.


To sum it all up:

SSAO makes it possible to have AO in places where it normally isn´t possible – e.g. real-time engines (yes – even for Flash/Papervision), or within a video post process.

All that’s needed is the Zbuffer of a 3D rendering.

You can download the Shader here or get it packed with an AFX example.


Hint: As said before, the SSAO pass needs to be blurred in order to clear out the emerging noise – but don´t overdo it, as the blurring will “bleed” the result over to unwanted areas.
This could be targeted by a special SSAO blur filter that would again have to take the depth map into account.


Click here to see this screenshot as video

Update:

I feel like I should adress the X,Y-offset params of the filter which can be used to tweak the “shadow direction”.
“Shadow direction” in quotes as (SS)AO is all about indirect lightning so this is a real “tweak” or an unrealistic param – which I thought might come in handy in some cases.

I was asked if the shader was performant enaugh for flash – and even if it looks a bit ugly codewise as I had to get rid of a loop – it surely is.

So I updated the package and made this small interactive flex demo :

Be sure to try unchecking “color SSAO?” and “show SSAO?” to get an impression.

You can grab the source here.

Update 2:

The filter now features an additional sampling radius parameter which controls the dispersion and allows to adjust it to the desired level of detail – this should also make it more useable for papervision.

I hope this little experiment will be of use to somebody out there and would love to hear any suggestions.

Since some years the idea of harnessing the power of modern graphic cards for 2D/3D applications has become more and more popular.

Today we find GPU support in many CG applications like Photoshop or After Effects.

But first the fun part: Watch a video raytraced with PB and AFX!

The chaos group is even working on a GPU based real time version of Vray – a widely used modern GI raytracer.

In fact GPU shading is often more performant plus it quasi ‘adds’ an unused/inactive processor.

One could argue that Pixel Bender doesn´t quite fit the term as its mostly run in software mode (on the CPU) – for
example when executing a PB shader in Flash or rendering it with AFX… for now.

But PB is still young and the prospect of being able to run it within multiple adobe platforms (Flash,PS,AFX) is very compelling.

When I saw Tom Beddard´s PB port of Keenan Crane’s genius julia set tracer, I remembered that I had planned to get into PB GPU raytracing once I got around Mike Welsh´s PB raytracer.

So I spend some time with it, created a scene similar to ‘The Eye’ and added viewport controls as well as anti aliasing.

The result can be viewed here.

Rendertime was 3:10 on a Q6600 for 4700 frames.

And here is the modified shader.

I also fiddled around with adding an inside-out AO (ambient occlusion) pass but wasn´t able to get satisfying results.
It even almost fried my GPU in the process so in conclusion it’s doable but sadly not practical.

With CS4 Adobe introduced Pixel Bender to Flash, After FX and Photoshop. Pixel Bender allows to easily write and test filters witch are finally compiled to bytecode (Flash) or machine code (AFX).

For the new version of Smart Normal, its sobel-edge shader got portet to PB – making it blazing fast. Its even possible to use it for realtime shading.

Want to use Smart Normal directly within After Effects or Photoshop (CS4) ? Just grab it from Adobe Exchange and drag it into your plugin-directory.
(Photoshop needs the Pixel Bender Plugin).