Get Adobe Flash player

The laughing man incident – defacing the web with OpenCV

15. März 2010 – 20:37

Tags: , , , ,

The laughing man incident - defacing the web with OpenCV

In the series GITS:SAC, a hacker called “the laughing man” defaces himself and other persons by digitally placing an animated logo over their face.

Now he is back and defaces the entire internet!

Look for yourself and check if your brain has already been hacked!


This experiment is really just an excuse to play around with hair cascades.

In 2001, Michael Jones and Paul Viola published their object detection framework, that introduced the integral image and cascade classifiers combined with a learning algorithm to achieve fast detection of facial features.

This method is still the quasi standard in face detection and an implementation of their paper is available within the Open Computer Vision library.

That code and even the whole library has been ported to flash and there are many great experiments out there – one by mr.doob even featuring the laughing man.
So this has already been done and explored with flash, but after my multiple object detection adventure I just had to do something “interesting” with the Viola-Jones detection.

However the idea to analyze and manipulate every image on any page is problematic.

Initially I was tempted to try this with JavaScript, even if I had some concerns regarding the performance – it should have been possible with modern browsers/engines … but sadly it isn’t.

The issue is the browser sandbox.

Sure, you can replace images within an html document on the same domain, but you can’t just load a page from a foreign domain into a frame and mess around with its contents. So in essence it´s the same cross domain policy that we know from the flash runtime and thus the solution to this is also the same: pull the desired content to your domain through a proxy.

Doing this with just streams or images is fairly simple and can be coded within a few lines, but fetching a complete page is a different story.
I did end up with a proxy that does this quite well but it struggles with redirects, certain JavaScripts and naturally flash, Silverlight or other plugin related content.

Still I find it quite reassuring to know that defacing a website isn´t an easy task.
After all, this topic is close to phishing and other black-hat stuff…

The detection part, on the other hand was simple – the actual OpenCV source was built quickly and works like a charm. Intel and other people where kind enough to contribute some of their research to the library – in form of xml lists with capable classifiers.

Because the server has to download first and then to compute all the images at once, you will notice a delay when loading a page with many images or with huge ones, although, the detection runs very fast.
Also I have it check every image twice – once with a frontal and then with a portrait classifier list.
The detection rate is satisfying – somewhat around 60-70% I would guess.

It would be possible to further improve that rate if one could adjust it to a fixed environment.

Conclusion

Sandboxes and cross domain policies are a good thing even if they are known to cause headaches to developers.
They protect us from frauds and make sure that others have to ask permission before they can use foreign content within their web apps.

Yet, given some time and motivation, they can be bypassed. So, always keep an open eye…

As for face detection: this isn´t just great fun to play around with but also has many interesting applications, spanning from camera tracking over to privacy (Google uses it to blur out faces in street view or to filter content for the image search) to security (face recognition).

I recommend OpenCV to anybody who wants to fool around with face/object detection or argumented reality
- and there´s much more…

Post a Comment