Multiple object motion detection
24. May 2009 – 02:41Tags: Blobdetection, MJPEG, Motion Tracking, Proxy, Webcam
There are a bunch of motion detection and argumented reality experiments with flash out there – most involve a webcam and are able to track a moving area quite accurate.
I was curious if flash would be able to handle the task of tracking multiple objects/(blobs.
>> Take a look at the result and track some cars in the Czech Republic (live).
.
The example is using Justin Windle´s “MotionTracker” approach, Andrei Thomaz´s port of the blobDetection-library from v3ga and Josh Chernoff´s version of the MJPEG streaming-component by Doug Hughe .
Basically two images/frames are filtered and compared. The deviating parts are filtered to show up as white “blobs”. This area is grabbed via getColorBoundsRect() and divided to be searched for individual blobs.
The Result has a slight amount of lag, resulting from the cams framerate, MJPEG-decoding of the webcam stream and the fact that it has to be fed via a proxy.
Also it´s much easier to get accurate results, if the background is known and fixed (no day/night changes or changing “static” objects like parked cars etc.).
And it would take bigger image source to get a more percise tracking but without further optimisations the limit lies somewhere at 600*x.
So in conclusion realtime multi-object motion detection with flash is possible!
Update:
Big thanks to Og2t for bringing Kanpei Baba´s genious approach to my attention. That guy just thought one step ahead as he harnessed the power of the player itself by using floodFill() to mark found blobs.
This is a perfect example how easy it is to oversee a great pattern – even while already using it!
Its just the same idea with getColorBoundsRect() - of course you could iterate through the pixels with a simple algo to get the same result – but why if the runtime does it – and so much faster than the AVM ?
The experiment has been updated and eats virtually ‘no’ cpu-time in comparison to the original.
Update2:
The cam I used until now has been moved to face away from the road – the feed of the new one is till a but too small and lacks quality but its hard to find a relyable realtime stream that captures moving objects, shows a suitable envirionment AND offers the right resolution.
25 Responses to “Multiple object motion detection”
Nice work.
Just one question, how did you get around the sandbox problem with my class?
- Josh Chernoff.
By Josh Chernoff on May 24, 2009
Hi Josh, first let me thank you for that MJPEG class – it works like a charm.
I used a very simple PHP proxy script to to pass through the webcam-feed:
http://www.filerack.de/as3/sp_tracking/proxy.phps
But that´s only half the story as the sandbox expects the host to return a crossdomain policy file on the request “policy-file-request” at port 843 before allowing any socketcontection.
(You can specify another port than the 843 default via loadPolicyFile())
So you need SSH-access to your host to be able to run a small socketserver such as this one:
http://www.smart-page.de/as3/sp_sandbox/sandbox.phps
…returning a policy file like this:
http://www.smart-page.de/as3/sp_sandbox/crossdomain.xml
Rather than including this in every socketserver you setup, its more simple to run a seperate one on port 843 just for that job.
Greetings from Aachen!
- Jan
By flashgordon - Jan Frischmuth on May 24, 2009
Really nice job!
By Henrique Vilela on Jul 15, 2009
Hey! Excellent example! I had a similar idea to track the cars on the parking site but it had evolved into face tracking experiment. I’ll come back to it. You can use much faster blob detection routine, just check my blog for clues. Bookmarked your site as well, great stuff here!
By Og2t on Jul 29, 2009
Hi Og2t,
first let me return that compliment – you have also been bookmarked
I am still stunned by Kanpei Baba´s approach and just can´t unterstand how everyone seams to have missed it… it´s so obvious!
By flashgordon on Jul 29, 2009
This method has also been discovered by Quasimondo and was shown (!) at Flash on the Beach 2007. It hasn’t spread as Mario hadn’t released the source.
I am really interested how you did get the live feed to work, would you share the source for that? Thanks!
By Og2t on Jul 31, 2009
Right, I saw that comment from Quasimondo.
Still its interesting that only those two bright minds came across this solution.
Practicly I already shared the source within my first comment.
Aside from the Proxy, all you need is Joshs port of Dougs MJPEG class.
If I have missed something or if you have any trouble / questions just drop me a mail.
I´ll be glad to help.
By flashgordon on Jul 31, 2009
do you have the source code for your work with multiple object detection? I like the way the kampei multiple detection works, but I need an AS3 version and I’m having some issues translating kampei’s AS2 to AS3.
By adam on Sep 25, 2009
Hi adam,
currently my tracker is a slightly tweeked mix of Kampeis approach (blob detection using flood fill) and some preprocessing/filtering technics that mostly follow Andrei.
I don´t want to take any credits away from the guys that initially came up with this so I´ll just point you to the Kampei port from Kynd that Og2t kindly brought to my attention:
http://www.kynd.info/dev/2009/05/flash-finding-objects-from-a-photo.html
I found the hardest task with motion detection is to find the right values for input preprocessing and to tune/limit the blob detection algo so it won´t crash browsers or fry cpu´s
If you need any help with the implementation just drop me a mail.
By flashgordon on Sep 25, 2009
Hi -
Could you please advise how to connect MJPEG stream properly? I am trying to do that without any motion detection.
The thing is that all I got is URL, which returns me MJPEG stream. This url is “http://81.198.212.174/axis-cgi/mjpg/video.cgi?resolution=640×480&compression=30″;.
Is that possible to stream it with Flash/Flex in the same approach you have been using?
Thanks in advance.
By Xpb7 on Oct 12, 2009
Hi Xpb7,
yes, it´s possible but you will need Josh´s streaming class, a php-proxy that fetches the feed (assuming that it doesn´t originate from your own server) and a php socket server that you will have to launch from your servers console.
You can find the link to the MJPEG streaming class within the post and I wrote a quick howto to the proxy and socketserver including script examples within the second comment.
By flashgordon on Oct 12, 2009
Sorry, I got that it is possible.
Now I am trying to solve sandbox security problem, using proxies as you advices.
By the way, why your example shows me nothing when I go to “http://www.smart-page.de/as3/sp_tracking/”?
By Xpb7 on Oct 12, 2009
Thanks for such a quick response!
I found your blog very usefull!
Is that ok I contact you here for help or should I better email you? I need a bit more help to make it done. I am good in as3, but not so good in PHP. What means “php socket server that you will have to launch from your servers console” from your last post?
Thanks a lot again!
By Xpb7 on Oct 12, 2009
Thanks for bringing this to my attention – the example will now work again.
Please feel free to contact me via mail if you need further help.
I will elaborate why a socket server is necessary as this information might be interesting for other readers too:
MJPEG streams are transferred via HTTP but handled a bit different than a normal image-download as they have another mime-type which basically tells the browser to keep the TCP connection open and to expect a continuous feed of new data (images).
With flash you have to open your own TCP connection to the stream, keep it alive and cut the incoming data into images.
That’s basically what Doug´s/Josh´s class does for you.
But as I wrote on my first comment…
Due to the flash players sandbox security policy, every time you try to open a socket-connection, flash query’s the port 843 on that url and expects that a crossdomain policy file is returned and allows the desired connection.
This is intended to prevent you from opening socket connections to services that you don´t own.
So you HAVE to run a socket server that listens on that query and returns the crossdomain file.
This service has to run on your server as an application – you could write it in C/Java/Pearl/Python… – or as I did in PHP.
Using the PHP CLI (Command Line Interface) SAPI you can run PHP scripts from your servers console (via telnet or SSH) – not as an on-demand script but as a program.
So you also need to have access to your servers console.
Executing the PHP socket server as a background task on linux will look something like this:
#nohup php -q sandbox.php &
Cheers!
By flashgordon on Oct 12, 2009
Hello, Tank you for the examples, and last post , i have used mjpeg class but in local , i have used this code:
import MJPEG;
//”192.168.101.51″ host of axis ip cam
var test:MJPEG = new MJPEG(“192.168.101.51″, “/axis-cgi/mjpg/video.cgi”, 80, “root”, “root”);
, i don’t undurstood exactly, to use this class in remote connection,I must replace “192.168.101.52″ to “proxy.phps” showed in the 2° post ? My question is , can you give me an example How can i integrate exactly MJPEG class to use in a online web application?
By fabrizio on Nov 11, 2009
Hi fabrizio,
Josh´s class needs the IP as first param, followed by the file/script to query and the port to open a socket on for this connection.
In my code this looks as follows:
motion = new MJPEG(“filerack.de”, “/as3/sp_tracking/proxy.php”, 80);
And please note that you have to rename the proxy.phps to *.php so your webserver will execute it instead of opening it as a syntax highlighted view.
Within this file setup your connection to your webcam and there you go
You should be able to get a connection if you test your flashfile local from within your IDE.
To make this run on a webserver you need to setup and execute the socketserver mentioned in the previous comments.
Good luck!
By flashgordon on Nov 11, 2009
thanks you very very much!, if i use your example in my localhost
i must replace in proxy.php
$fp = fsockopen (“62.16.100.204″, 80, $errno, $errstr, 30); with
“192.168.101.50″: is the ip of my cam
$fp = fsockopen (“192.168.101.50″, 80, $errno, $errstr, 30);
and in my fla i must replace
motion = new MJPEG(”filerack.de”, “/as3/sp_tracking/proxy.php”, 80);
with
motion = new MJPEG(”localhost”, “/proxy/proxy.php”, 80);
???
By fabrizio on Nov 12, 2009
I’m trying to get the snadbox.php to run on my media temple account. I can get in to the server via ssh and I have tried #nohup php -q sandbox.php & with no errors from ssh but I still get sandbox errors from flash. Could some one give a better example of how I would ssh my sandbox.php file to that it runs as a socket server. Thanks.
- Josh C
By Josh Chernoff on Dec 19, 2009
Hi Josh,
did you modify the variable $host in line 11 of the sandbox.php?
Maybe you should check back with the folks at media temple and ask them if you are allowed to open ports/that specific port as this is often the issue.
If they tell you that you have to use a port other than 843 – no problem…
Just change line 12 of the sandbox.php and don´t forget to tell flash where to send the request to ( via loadPolicyFile() ).
Hope this was of any help and wish you good luck with whatever intersting stuff you are working on
By flashgordon on Dec 19, 2009
Thanks for your help. I forgot to set the loadPolicyFile.
How did you go about setting that in the MJPEG.as file?
By Josh Chernoff on Dec 19, 2009
As I was able to open the default port I didn´t have to use loadPolicyFile(), but this should work fine:
//System.security.allowDomain(“*”); //if you are going across domains
System.security.loadPolicyFile(“xmlsocket://”+your_ip”+”:”+your_port);
By flashgordon on Dec 19, 2009
I still getting hung up by the sandbox.php.
I did set the vars for host and I left the port to 843. What could I do to test if that is working right?
By Josh Chernoff on Dec 19, 2009
First you should check if your host is responding to requests at that port by using a portscanner like http://www.t1shopper.com/tools/port-scanner/ for example.
If it is open, you can insert echo $pfile; after line 36 and echo $message; at line 48 of the sandbox.php, to print to the console what (and if) the socketserver and your flashclient are exchanging.
Nearly forgot to mention this – in order to see the output, you have to start the sandbox.php without nohub: #php sandbox.php.
Also don´t forget to always kill the process of the already running one before you (re)start the socketserver.
And last but not least – please make sure, that the crossdomain.xml resides at the same folder as the sandbox.php and holds the correct domain/ip.
By flashgordon on Dec 19, 2009
So I take it that if I ran that port scan on 843 and even if I did not have my socket server running I should be able to talk with that port, right?
if that is true then media temple does block that port and I will have to find another one I can use. Or maybe it’s that I have to have the socket server running correctly?
By Josh Chernoff on Dec 19, 2009
It depends on the systems configuration – ports will only listen to requests once they where opened by a service – for example php – or at least this should be the case as having unused ports open is a security issue.
So in short the answer is no – you should start the socketserver before scanning the port.
Also being able to scan the port doesn´t necessarily mean that your socketserver is working.
If executing the sandbox.php doesn´t throw
unable to bind address [98]: Address already in use…
or any other error you can be fairly sure that the sever is up and running and the port isn´t already occupied by another service.
But if it doesn´t reply to the portscan, the port is most likely being blocked by a firewall.
In that case you can try your luck and use another port – the last option is to ask media temple to open 843 or if you should use another one and which.
By flashgordon on Dec 19, 2009