The project shares code with my Sequence Grabber project and renders QuickTime movies using QTKit and CoreVideo. It also uses macros made up of multiple Core Image filters and defined in a simple XML file, each item in the popup menu is a macro of one or more Core Image filters that are linked together to produce the effect. I’ve also written a Python script to import Core Image Fun House data files and convert them into my “cimacro” files (this functionality will be rewritten in ObjC one day).

The latest version of the code (checked in but not shown in the demo movie) also features crude animation and allows the video source to be selected on the fly between a movie file and an iSight.

The easiest way to use the sequence grabber is just to add a CSequenceGrabberView to your project. The view supports start and stop actions so you can wire your user interface directly to the view. You can also access the underlying CSequenceGrabber object directly if needed. There are plenty of hooks in the code to add your custom functionality – CSequenceGrabber (and its children video and sound channel objects) sends many useful NSNotifications. I’m also going to make the code use Cocoa bindings as much as possible. That way you can use bindings and/or notifications as your need dictates.

I’ve improved the “QuadView” sample code to take advantage of Cocoa bindings support in CCoreImageView. One thing I’ve noticed is that the core image filtering just isn’t very fast. After two or three live video streams running through Core Image filters the frame rate drops rather dramatically. I will look into this further and hope to make big improvements if my code is actually the bottleneck. Or maybe it is time to just order that Quad G5 now that they’re available (one 2.5GHz core per video stream sounds like a great idea!).

If you are having trouble building the code – remember all projects require Xcode 2.1 (or newer, I am using Xcode 2.2 for development). Also you will need subversion to check out a full copy of the code.

Finally I’ve been really encouraged by the interest that my Sequence Grabber code has generated. I’m getting a few emails a week from people using it in their projects. I love hearing about the uses people are putting the code to and encourage anyone using it to append a comment to this blog posting.

1) Added code to write CoreImage images (CIImage objects) to a QuickTime movie (currently hardcoded to mpeg4) using QTKit. Now you execute Image Units against the video frames coming in from the camera and save the output to a movie. Pretty neat. At the moment it isn’t very efficient because I convert the frames to NSImages as an intermediate step (which I think takes the pixel data out of the GPU video ram and into RAM) – instead I think I can use a CoreVideo surface directly (but QTKit doesn’t support CoreVideo directly yet – I have more sample that i need to put online that enables QTKit and CoreVideo to work together). I also need to control the Sequence Grabber API a little better and try to prevent it from doing superfluous work (i.e. i need to put it into pure “preview” mode).

1.1) There are a couple of categories on QTMovie (part of QTKit) for conveniently creating movies and appending frames to it (note the timing code in the append frame code is wrong – which is why, if you watched the movie that accompanies the code I look like I’m dancing around like an idjit) and a category on CIImage for some handy dandy conversions (cropping, scaling, conversion to NSImage – thanks to Dan Wood’s code archive for most of that code).

2) Created a couple of specialised Image Units. This is really just an exercise in cikernel (really a subset of OpenGL Shading Language – that link is worth following btw) programming. I’m actually impressed with how easy it is to write cikernel code.

3) Actually that last item is a white lie. I didn’t create any Image Units – what I did do is create a CIFilter subclass that takes the cikernel code as an input parameter attribute and an array of user defineable input parameters that will get passed to the kernel from Cocoa. This allows you easily play around with cikernel code without having to create a separate target in Xcode and then dick around with using CIPlugin to load the Image Unit (why doesn’t CoreImage load all .plugins from my application plugins directory automatically?). It is pretty cheap and easy way of creating a Core Image filter without any administration work. See CCIGenericFilter.h/,m and CMotionDetector for more details

This is what happens when your cat wakes you up too darn early after peeing under the computer desk.

Sample movie (MP4/QuickTime 7 required, 64KB file): MotionDetection.mp4

No those aren’t mpeg encoding artefacts, those are my attempts to highlight the motion in the movie.

Source code: http://toxic-public.googlecode.com/svn/tags/BlogTag_20070927_729/Frameworks/ToxicMedia/Samples/SequenceGrabber (see MotionDetector subdirectory)

Have fun.

Screenshot: QuadScreenshot.png (rather large – but note the CPU bar in the dock.. Even with all three filters running it was barely denting the CPU).

See screenshot: SequenceGrabber_1.png.

Note that QuickTime movie saved to disk is not passed through CoreImage. Frames from the SequenceGrabber are exposed to client code through standard Cocoa delegate methods (well one method) so you can do whatever you want with the images (including add them to a QTKit movie).

The code is still in a work in progress but it is pretty functional right now. My aim is to keep it all quite simple (unlike some of the Apple sample code parts of it was based on) and usable when just dropped into a 3rd party project.

Source code: http://toxic-public.googlecode.com/svn/tags/BlogTag_20070927_729/Frameworks/ToxicMedia/ links.

Update. I’ve uploaded a new version of the SequenceGrabber source code that should compile cleanly on any Mac OS X 10.3 machine. Thanks to “jlecuyer” for pointing out the original version was broken.

Update: Information on new version here

Update: Link to source deleted, see my ToxicMedia page.