Nestled in a dimly lit corner of the AMD suite was a start-up company called Nuvixa. While much of the suite’s floor space was dedicated to pimping out fusion-based laptops and 6000 series video cards, Nuvixa was showing off some tech that can actually put all that compute goodness to practical use. Enter the presenter.
The software/hardware solution shown was created to solve the initial problem of students staring blankly at PowerPoint projections without paying attention to the professor and missing out on gestures and nuances they may be trying to convey. This evolved into a product that is suited for certain classroom environments, particularly distance learning, as well as corporate demonstration and presentation scenarios. So what is this product? We’re glad you asked.
Nuvixa’s Immersive Video Platform consists of a “ZD” camera; the marriage of an HD video camera, and a 3D infrared camera ala Kinect, (the “Z” refers to the depth dimension,) and also has a significant amount of software magic happening to superimpose the presenter over the presentation slides and allow him or her to interact with objects in some neat ways.
If you’ll recall, Apple introduced a gimmicky little feature in iChat back in 2007 which takes a picture of the background, then allows you to replace it with some other picture for a faux blue-screen effect. The super imposition of the presenter uses the same general concept of a fake blue screen but goes about it in a 3 dimensional way as opposed to Apple’s 2D solution. The infrared camera is used to detect how far away an object or person is and only work with objects at an adjustable, predefined range from the camera. In theory, and with a little bit more tweaking this should allow them to cut out the presenter with greater accuracy in a greater number of environments and lighting conditions. Cropping out the background has other important implications in terms of bandwidth. Since there is not as many pixels to be compressed, the encoding algorithm can compress the video stream much more efficiently, and we were told realize up to a 4X reduction in required bandwidth when streaming.
Once the presenter’s image has been effectively cut out it is superimposed over a presentation and the objects in the presentation can be made aware of the presenter’s location in the video frame. In a brief demo, by simply stepping left or right two planets being displayed would grow or shrink depending on how much screen real estate they had to work with on either side of the presenter. Other features include the ability to interact with objects by “grabbing” them and moving them around the screen, or performing some gesture such as waving an arm to uncover some hidden information on a slide.
The whole thing is based on OpenCL which handles the motion tracking, animation, and video compression. The software is designed to run on a broad range of computers by rendering what it can on the available hardware, and essentially ignoring the rest. This means that if the hardware is capable of encoding the video stream at 30FPS it will do so, but if it is slower the software will simply encode as much as it can at a reduced frame rate to maintain the live stream. The demo we saw was running on an unreleased Llano-based laptop at reasonable (but not quantified) frame rates, however being OpenCL based means it is hardware agnostic and need not necessarily run on a Fusion based platform as AMD would have you believe.
Overall while the demo and concept were rather impressive, there is much work that needs to be done in advance of Nuvixa’s planned 2H 2011 launch. The background elimination was working, but imperfect. Quite a bit of distracting fuzziness and cropping was occurring which runs afoul of the company’s mission to reduce distractions in the classroom by merging presenter and presentation. This is due in large part by the low resolution of the 3D, sorry, “ZD” camera which we are told was running at a 200×160 resolution (an odd resolution and perhaps 240×160 was meant.) Higher resolution spatial images should help clear that up but increasing the resolution increases the computing power required as well as the cost. Nuvixa told us they are in talks with several hardware manufactures to provide cameras for the final design, and are constantly tweaking their software as well as counting on the raw power of modern and upcoming computer systems to plow through the pixels.
Another concern we have for this technology is not one of hardware or software, but rather the human factor. It is going to take a monumental effort to get professors and other presenters trained and willing to use this technology and many silver-spoon tenured types will simply refuse. No demo was given of actually creating the presentation being used and any additional steps required are unknown at this time, but people resist change by nature. In order for this technology to be accepted in the mainstream, it will need to be virtually seamless, super responsive so that the presenter doesn’t make a fool of themselves waving an arm about trying to air-click a button, and have a level of polish that would make El Jobso himself proud. This is certainly possible given time, but the version 0.someting software being demoed is not quite there yet as of today.S|A