Creating an example of video image tracking

Well, I suspect this question will get closed because it is a bit broad, but I've played around with image tracking and thought I'd show what I've done in case it's helpful.

I was interested in learning some physics, and have the following video:

To do the tracking, I smoothed out the images and converted them to black and white, which allows for the colored balls to who up on a black background. Note that I have already defined the symbol images which was the series of graphics that made up the video.

imagesbw = Binarize[MedianFilter[Image[ImageData[#][[All,All,1]]],4]]&/@images;

We lose the black ball, which is unfortunate, but my "physics question" was to see if I could predict the movement of the missing black ball from the information that is available. I started by collecting the position of the three visible balls using ComponentMeasurements which in my case finds four balls, one of which is the one in the upper right hand corner that doesn't move, so I'll ignore it. I can then get tracking information from the ComponentMeasurements output. Note I did play around with ImageFeatureTrack, but I didn't have much luck with it.

movement = ComponentMeasurements[#, "BoundingDiskCenter"] & /@ imagesbw;
tracking = 
 Graphics@Riffle[{Red, Orange, Blue, Red}, 
   Point[Cases[# /. movement, x_ /; Dimensions[x] == {2}]] & /@ 
    Range[4]]

There's some messiness in the data, so the line above deletes some of the points. Now I want to predict where the 8-ball is heading from the information in the point plot above. I approach this problem by assuming conservation of momentum assuming an elastic collision, ignoring friction and acceleration. There's a reason why my name isn't bobthephysicist.

There's probably no real reason to go through my ugly code to solve this problem, especially since I did so by surfing around Mathematica.SE so the solutions are already here. At the end of the day, I get the following graphic, which is a pretty decent fit, although I will admit I did need to incorporate a fudge factor in to the solution. My assumptions, as well as my not knowing the weights of the four objects, makes this task a bit daunting.

So in summary, tracking objects in Mathematica is possible, but it's not for the faint of heart. I think the process is summarized in the following steps:

• Import video as individual frames, possibly "downsampling" the video to make the data set manageable.
• Manipulating the images to remove extraneous information (colors, stationary objects, etc.)
• Use some tool such as MorphologicalComponents to identify the information of interest to you.
• Transform the output of the previous step into a usable format

That's my 2 cents -

What you are trying to do is called object tracking, and it is an active research area in computer vision. There are many algorithms for detecting and tracking multiple moving objects. I don't know of any examples in Mathematica, but here is one in MATLAB using background subtraction and Kalman filters.