In his TV series “Cosmos”, Carl Sagan travels the universe in a “ship of the imagination”.

Uninfluenced by the laws of physics, “perfect as a snowflake, organic as a dandelion seed”, this vessel carries him between galaxies, stars, and planets.

And now you can cruise the cosmos too!



Recently I watched cosmos again and remembered how that sequence fascinated me as kid.
The sheer scale of the distances involved blew my mind. It still does today.

The concept has been revisited by many popular movies and media (Man in Black, The Simpsons…) and is always the same – the camera enters the universe, our galaxy, solar system and finally stops on earth – sometimes entering the microcosm, like the DNA of a cell.

Years ago I experimented with something like a pyramidial image to archive this effect in flash, unfortunately the result didn´t create the feeling of depth that I had hoped for.

But flash has advanced in those years… as I wrote before, in the strange attractor post, Ralph Hauwert and Joa Ebert have published some astonishing code that renders nearly half a million particles in real-time!

First I decided that I would work with real data – otherwise this would become just another “random stars / noise flying at you” screensaver.

But after some research I learned that the actual distance of nearby stars (called “stellar parallax”) is not always known, so that most models and databases project stars onto the inside of a virtual sphere.

Finally I found the HYG database – “a subset of the data in three major catalogs: the Hipparcos Catalog,the Yale Bright Star Catalog (5th Edition), and the Gliese Catalog of Nearby Stars (3rd Edition)”.
It contains about 120.000 stars, many including their parallaxes and conveniently cartesian coordinates, so I saved the step of calculating xyz from right ascension, declination and distance – like I had to with other databases I tested.

The CSV file got stripped of all additional data aside from the positions with a simple parser – that got it down from 20 to 4MB – still too big for my taste.

So I added the Fzip library to Joa´s AS3 only version of Ralph´s experiment, parsed the position vectors, scaled them and got my first view on our galaxy – breathtaking.

Scaling, controls and a simple clipping where swiftly implemented and everything was set for the voyage to the stars…

Tribute – “A Glorious Dawn” displaced

While watching the result, an idea struck my mind:
As the z-axis movement of the camera or the zooming into the szene is done by simply scaling the particle positions, its obviously possible to change their parameters in realtime…

I remembered my displacement experiment with papervision and quickly added a color parameter to the particle class, loaded an flv video and compiled this:

“A Glorious Dawn” live rendered to a displaced point cloud

Conclusion

It´s possible to handle an awful lot of particles with flash, even more if haxe/alchemy – inlining gets involved.
As the displacement experiment shows, the dataset can even be dynamic!

And hey – these are real stars, so this is science!

Hint: turn up your gamma adjustment.

About three decades ago, pixels where as big as toasters – and mostly green.

Today we have millions of them to deal with, and the terms anti-/aliasing are part of our daily live and every time an image is scaled, it gets interpolated to keep those bad “stairs” away from our eyes.

But what if the source image is aliased in the first place?

Checkout the demo of SmartAA!



While upsaling is the most common cause of aliasing, there are images that are naturally aliased – for example images from a 3D render engine.

Of course there are many ways to antialias cg images when generating them – for example by rendering to a higher resolution than targeted and downscaling the result (FSAA)- or by rendering and merging four slightly offset variations.

While this is as computational intensive as it sounds, modern GPU´s can handle this task rather easily as they have their buffers and hardwired magic to do so, but still it’s not uncommon to come along aliased images in the world of computer graphics.

So is all this material “useless” or can an image that has never seen any antialiasing, or has been upscaled with the nearest neighbour method, be reconstructed?

In fact all that modern imaging software does when rescaling an image is reconstructing its pixels to fit the new dimensions – most times with a bicubic filter.

SmartAA is a bicubic BC-spline reconstruction filter for PixelBender.

Its 3×3 kernel gathers all neighboring pixels of the one to reconstruct and weights them with the Mitchell-Netravali(german link) filter.

In essence, this means that each pixel is modified by the average color of its surrounding friends.
While “average” being the wrong term here as the influence of each pixel is determined by the spline curve of the Mitchell-Netravali filter.

This eliminates those hard, “stair-like” artifacts of the aliasing by visually “softening” or unsharping the image.

However this is not simply a destructive blurring effect but a reconstructive analysis of the source “signal”.

There are many known filters that do this hard job – it´s not easy to reconstruct information that was partly lost (or never there in the first place).

Thus all of them suffer of the same issues – the result always shows some artifacts.
Mostly it’s a visible unsharping, anisotropy or the so-called “ringing“.

In general bicubic reconstruction methods create the best results.

Among the various possible filter methods the windowed Sinc(Lanczos)- and Mitchell-Netravali filter are chosen the most as their output is considered to be the best mix of the inevitable unsharping, anisotropy and ringing.

Just for the record: there are “better” solutions, the so called “adaptive” or “smart” filters that try separate methods on edges, fills and other structures.
They are particularly useful when working with sequences of moving images (video) but again slow to compute.

SmartAA on the other hand is a quite simple implementation of the M.-N. filter and runs blazingly fast when run on the GPU – producing about 70 redraws per sec on a 1024*768 image.

Unfortunately it´s not nearly as fast on the CPU.
If you have a rather slow machine, you will have seen the demo struggling to keep a 24fps framerate, so its just! that bit too slow to to antialias the pixelated output of a flash based C64 emulator.

That would have been cool… would love to see something like that.

I kept it flash compatible anyway – still there might be applications.

Its a matter of personal reception, but to me the M.-N. filter looks better on videos as the inbuild “smooth”-method.
Anyway – flash video has been criticized enaugh for high cpu usage so lets not make things worse

Also: like any reconstruction filter it won´t work in any given scenario as there are some natural and some specific limitations.

So what can SmartAA do and what not? Here is a feature/limitation list:

Features:

- Works with Photoshop, After Effects, Flash
- Supports alpha channel
- Is real-time

Limitations:

- Does no up/down scaling – just reconstruction of anti aliasing
- Is scenario specific – for example text will often look strange
- works only with artifacts in the same pixel space.

Let me elaborate on the last point:

As SmartAA takes only neighbouring pixels into account it isn´t able to reconstruct aliasing effects that span over greater distances than 2 pixels.
For example if an image exists of 4*4 pixel blocks of the same color, those blocks would get rounded on the edges but will still be visible as rounded blocks.

But it works ideal on sources like aliased output from a render engine, or images that have been upscaled x2 with the nearest neighbour method.


Conclusion:

Aliasing should be fought before it ends up within an image as there are more sophisticated methods to keep it from emerging than to reconstruct it after the damage is done.

But sometimes it´s inevitable and if you are looking for a simple but effective way to get rid of that ugly blocks within an image or video – SmartAA will do the trick.


Download:

Grab the Shader here and use it with Photoshop (Pixel Bender Plugin needed), After Effects or Flash!


Hint:

Try to add a sharpen filter or better an unsharpen mask to the result to counter the blur effect.
Don´t forget to use the width/hight properties – SmartAA will work without but there will be artifacts on the upper- and lower right corner.

I hope this is helpful to anyone out there and would love to hear some comments or ideas.

Cheers!

The flame wars are on!

CEOs, CTOs, platform evangelists and users are fighting the war of the worlds out there.
It´s HTML5 vs. Flash, open vs. closed, H264 vs. OGG, desktop vs. mobile, truth vs. lie.

Lets first calm down with this nice audio visualization before we dive in…



Since Steve´s presentation of the iPad and recent statements from him and Apple it´s clear that they are not going to support flash on their mobile devices.

This has triggered an ongoing debate on Flash, HTML5, browser plugins and web technology in general.

Unfortunately the whole discussion is rather uncivilized as there are a lot of people involved who make their living from those technologies and standards – and even more who use them on a daily basis.

So much has been written on this topic during the last two weeks… much was impulsive, exaggerated or plain wrong misinformation while few statements showed greater insight or were just watered down by irony and harsh language.





So I am not going to take part in this discussion with a detailed view on the technology involved or why one standard is superior to another etc.. sorry, but there will be no rant

Why?

Because this has already been done beautifully by others and because – it doesn´t matter.

What really matters is to get things done! To create content and to deliver it to the widest possible range of audience.

As a developer I don´t have the luxury of choosing my tools based on personal favor. I have to work with what’s best suited to achive the desired result – be it HTML, Flash or Silverlight or whatever may come along.

As a user however I have that luxury – I vote with every click and my wallet for and against technology that I consider to be worth my time or money.

So there is no point for these two groups of people to lash out against each other or certain technologies.
Of course this doesn´t apply to CEOs, CTOs or platform evangelists – it´s their job to influence the aforesaid groups on their opinion.

So what will all this mean for the web? Almost nothing.

The web has always been based on open standards while innovation was often driven by the “big boys” like IBM, MS, Apple and many others.
Finally this innovation gets adapted to open standards – as well as the economy assimilates ideas from the open source world.

And all this is healthy!

Sure – like in nature, it sometimes seems cruel and unforgiving, but whatever survives in the end may stay… just to be replaced by its successor if it doesn´t evolve.

And in the end everybody in this constantly changing ecosystem wins:

Companies get to sell their products and to rant about their competitors, developers get jobs instead of finally being replaced by generators, and users get more and more for their money.

So I am glad we live in a world with more than one operating system, programming language, browser, or pair of socks to choose from.

It´s the freedom “to choose” that makes the web/live so exciting.

Cheers!



PS: About the audio visualization experiment in the opener:

I wanted to do something with Five3D for a long time now and well – here it is.
You can load your own mp3´s via the right click menu.

Of course everybody remembers the scene from 2001 when Dave Bowman removes HAL´s memory banks.

While being shut down HAL sings the song ‘daisy’ and his voice continuously pitches down while his ‘mind fades’.

With this little test app you can sing like HAL!

…provided you installed the Flash Player 10.1 beta and have a microphone handy.

Yes 10.1 – as the new year is coming, so is the next subversion step of the flash player, bringing among other features video playback and p2p/streaming enhancements.
The beta2 has been released at the 17th December and brought also the long-awaited debug version.

But aside from the global error handling, the new feature I was looking forward to the most is the access to binary data coming from the microphone.

This will allow to develop a bunch of pretty interesting applications that make use of voice input – like voice recognition for example.

This little demo records data from the mic and plays it with an adjustable speed offset, pitching the sound.

You can download the source here.

Note: to be able to compile the project you need to download the updated “playerglobal.swc“.

Ever dreamed of having a chat with you own personal AI?

Well, we are still a bit behind Arthur C. Clarke´s dream of a self-aware AI like the HAL9000 but with AIML, Festival and Flash you can build your own talking chatbot in no time.

Have a chat with HAL first.

The Brain room:

So we are going to build ourselves the poor man’s version of a “Heuristically programmed ALgorithmic Computer” and of course we start with its brain.

The Program E – AIML Manager will function as the brain room and during the setup we install a set of data crystals (aka an AIML set).
Program E runs on PHP and parses and stores the actual “brain” within a mysql database.

So create a new database, install Program E to “mydir/programe” and upload the “talk2me.php” from the project package at the end of this article into “programe/src”.

Then upload and parse an AIML brain file into the db – call the new bot “HAL”.

The speech module:

We will use the Festival Text To Speech package to make HAL talk.
This needs a Unix based system and the installation is quite simple – refer to this tutorial if you encounter problems.

Festival comes with a “text2wave” script and a suitable default language.

We will also need the LAME mp3 encoder in order to convert those created wav files to a transferable mp3 because speed is critical.

Within the package you will find some PHP files including a “config.php”.
Upload and chmod them and modify the config to fit your servers directory structure.

The eye:

Now we setup an interface to Program E and our TTS system – HAL´s famous “eye”.

Open the “src/hal9000.fla” and the “src/Main.as” and change the variable “path” in line 31.

Upload the recompiled “index.swf” along with the HTML and JS file and have a first small talk with your chatbot.


Download the project here !



Note: This is the poor – AND the lazy man´s way to build a talking chatbot.
This could be implemented in many more elegant ways – it´s even possible to have a bidirectional voice communication with HAL by using an rmtp server and a speach recognition sdk.

Strange attractors are fractals that emerge at certain, sensitive parameters within the three-or more dimensional phase space.

They are are researched since the early 70s and strongly related to the chaos theory as they allow to watch the transition from chaos to order/geometry.

Good things first: see the result and try the chaotic attractor finder!

The 3D visualization of a strange attractor is quite resource intensive as points race around the attractors on chaotic trajectories – so it needs many iterations or particles to see a shape emerge from that chaos.

There are already some great flash experiments so I turned to Cinema 4D and PY4D as an advanced renderer would easily allow to raytrace and shadow the results – and to calculate even more particles of course.

This small PY4D script creates a strange attractor that Dr. Clifford Pickover had published 1990.

I managed to render a maximum of 50.000 particles with the R11 on my machine.

But with the new R11.5 release Maxon introduced render instances to C4D allowing now to calculate up to millions of particles – depending on the system.

I went with a number of 300.000 points, seeded the attractor parameters and rendered four transparent animations that could be combined to a 1.2 million particle video.

The preparation time for each frame was ~1min while PY4D took less than a second of that duration to perform five sinus/cosinus operations for each point.
Rendering itself took about five seconds per frame (CPU:Q6600).

If you are interested you can download the szenefile here.
(Remember to activate render instances in the PShape node of the XPresso tag).

Update:
The scene file has been updated to the R12.

When I learned that you can actually identify strange attractors using the Lyapunov exponent and found John Holder´s programm ‘lyapdemo.c’, I took a few minutes to port it to AS3.

You can use it to generate parameters for the PY4D script.

Gigapixel images – it´s all about resolution and detail – one gigapixel equals a billion pixels so we are talking about tons of data.
To be able to view such a picture, it hast to be sliced into tiny little bites – otherwise it would kill any present personal computer.
That technique is called pyramidial imaging, multiscale or deep scale and works quite fluid with Flash.

Click here to view a 86400*43200px image of our “Blue Marble“.

The uncompressed sourcefile was over 10GB big and took nearly a day to interpolate and cut it into 9 zoomlevels existing of 80.000 tiles.
The viewer uses the fantastic OpenZoom-API.

There are a bunch of motion detection and argumented reality experiments with flash out there – most involve a webcam and are able to track a moving area quite accurate.

I was curious if flash would be able to handle the task of tracking multiple objects/(blobs.

Take a look at the result and track some cars in the Czech Republic (live).
.

The example is using Justin Windle´s “MotionTracker” approach, Andrei Thomaz´s port of the blobDetection-library from v3ga and Josh Chernoff´s version of the MJPEG streaming-component by Doug Hughe .

Basically two images/frames are filtered and compared. The deviating parts are filtered to show up as white “blobs”. This area is grabbed via getColorBoundsRect() and divided to be searched for individual blobs.

The Result has a slight amount of lag, resulting from the cams framerate, MJPEG-decoding of the webcam stream and the fact that it has to be fed via a proxy.

Also it´s much easier to get accurate results, if the background is known and fixed (no day/night changes or changing “static” objects like parked cars etc.).
And it would take bigger image source to get a more percise tracking but without further optimisations the limit lies somewhere at 600*x.

So in conclusion realtime multi-object motion detection with flash is possible!

Update:

Big thanks to Og2t for bringing Kanpei Baba´s genious approach to my attention. That guy just thought one step ahead as he harnessed the power of the player itself by using floodFill() to mark found blobs.

This is a perfect example how easy it is to oversee a great pattern – even while already using it!
Its just the same idea with getColorBoundsRect() - of course you could iterate through the pixels with a simple algo to get the same result – but why if the runtime does it – and so much faster than the AVM ?

The experiment has been updated and eats virtually ‘no’ cpu-time in comparison to the original.

Update2:

The cam I used until now has been moved to face away from the road – the feed of the new one is till a but too small and lacks quality but its hard to find a relyable realtime stream that captures moving objects, shows a suitable envirionment AND offers the right resolution.

This little 3D experiment is called Metabubbles as it´s behavior and look are quite similar to those of real metaballs.

In fact it is just PV3D ViewportLayers with the blend-mode “add” that produce those gooey effects when balls with a similar z-depth pass each other – “old school”.

The cpu-load is quite high but the result is as psychedelic and relaxing as watching a lava-lamp…

Sit back and take a time out …

Ever wanted to keep an eye on your server without having to use the ssh terminal?

With Smart Stats you can – in realtime. It shows a live display of the most critical stats to monitor: cpu-load, memory consumption, top-processes, storage, network-traffic and active apache connections.

It exists of a very simple php-socketserver that executes tools like mpstat, top, iostat or netstat and wraps the output to something readable.

Have a look at the flashclient here or go ahead and download the package containing the sources and a readme.

With CS4 Adobe introduced Pixel Bender to Flash, After FX and Photoshop. Pixel Bender allows to easily write and test filters witch are finally compiled to bytecode (Flash) or machine code (AFX).

For the new version of Smart Normal, its sobel-edge shader got portet to PB – making it blazing fast. Its even possible to use it for realtime shading.

Want to use Smart Normal directly within After Effects or Photoshop (CS4) ? Just grab it from Adobe Exchange and drag it into your plugin-directory.
(Photoshop needs the Pixel Bender Plugin).

Once more a classic game gets revived with Papervision – Asteroids 3D.
Asteroids was one of the the first arcade shooters, developed 1979.

This version features dynamically generated asteroids (noise-displacement), “real” momentum and of course the famous hyperwarp.

As a decent 3D collision-detection drew to much performance, the game now uses dummy-objects and the simple and quick AABB-solution.

Try Asteroids 3D and blast some Asteroids!

Hint: Don´t forget your momentum when hyperwarping!

Again these two experiments were made do test the limits of Papervision and to explore the possibilities of flash 3D-games.

The first demo – “Imperial“ already looks a lot like LucasArts “Dark Forces” from 1995.
It loads MD2 models that include a small vertex-animation witch gets randomized.

“Safari“ shows a possible race-game.
The terrain works like the displacement experiment shown here earlyer.

Sokoban 3D – an adaptation of the classig strategy game from 1982.

It parses 100 levels (including the 50 original ones ) and allows the player to freely choose them.

The undo-function witch you might remember from the classic version is missing by design – to keep it a little chalanging.

Take a break and play Sokoban 3D
.

Real voxel-based engines haven´t been spottted often since the 90´s when they powered some pioneer-3D games.

Voxelworld isn´t exactly Novalogic´s “Comanche” but it draws a similar amount of pixels/voxels.

Sadly the SetPixels()-method isn´t fast enough to draw a bigger viewport.
This is a real bottleneck as flash would be able to calculate a quite high detailed voxel-landscape in realtime.

Take a flight ->