In my previous post I talk about the (rather pitiful) attempts at adding motion capture to my Sequence Grabber code. This post will document what additions I’ve made to the code under the covers.
1) Added code to write CoreImage images (CIImage objects) to a QuickTime movie (currently hardcoded to mpeg4) using QTKit. Now you execute Image Units against the video frames coming in from the camera and save the output to a movie. Pretty neat. At the moment it isn’t very efficient because I convert the frames to NSImages as an intermediate step (which I think takes the pixel data out of the GPU video ram and into RAM) – instead I think I can use a CoreVideo surface directly (but QTKit doesn’t support CoreVideo directly yet – I have more sample that i need to put online that enables QTKit and CoreVideo to work together). I also need to control the Sequence Grabber API a little better and try to prevent it from doing superfluous work (i.e. i need to put it into pure “preview” mode).
1.1) There are a couple of categories on QTMovie (part of QTKit) for conveniently creating movies and appending frames to it (note the timing code in the append frame code is wrong – which is why, if you watched the movie that accompanies the code I look like I’m dancing around like an idjit) and a category on CIImage for some handy dandy conversions (cropping, scaling, conversion to NSImage – thanks to Dan Wood’s code archive for most of that code).
2) Created a couple of specialised Image Units. This is really just an exercise in cikernel (really a subset of OpenGL Shading Language – that link is worth following btw) programming. I’m actually impressed with how easy it is to write cikernel code.
3) Actually that last item is a white lie. I didn’t create any Image Units – what I did do is create a CIFilter subclass that takes the cikernel code as an input parameter attribute and an array of user defineable input parameters that will get passed to the kernel from Cocoa. This allows you easily play around with cikernel code without having to create a separate target in Xcode and then dick around with using CIPlugin to load the Image Unit (why doesn’t CoreImage load all .plugins from my application plugins directory automatically?). It is pretty cheap and easy way of creating a Core Image filter without any administration work. See CCIGenericFilter.h/,m and CMotionDetector for more details