Wednesday, August 5, 2009

A Light Field and Microphone Array Teleconference Idea


Paul Mecklenburg and I were chatting, and the thought of really cheap cell phone cameras led us to day dreaming about a teleconference system.

Line up a dense array of cheap CCD sensors and microphones into a video conferncing wall you're projecting onto. Now you've got a light field, which permits rich virtual camera re-imaging (movement, rotation, zooming, depth of field). The microphones can be used to triangulate sound sources. Sources from e.g. table locations can be dampened (see "hush zone" in picture). Sources distant from mics can have an audio boost to account for volume fall off (see "audio boost"). The speaking locations can also be used to drive an automatic virtual camera which focuses on subjects doing the speaking.