Interactive 3D-AV

Interactive 3D-AV


Project Leaders:
     
  • Frank Melchior (Fraunhofer-Institut f. Digitale Medientechnologie, Ilmenau)
  • Dipl.-Inf. Jan P. Springer
  • Dipl.-Inf. (FH) Jan Hochstrate
  • Dr. Dieter Kemter
  • Prof. Dr. Bernd Fröhlich
    •  
 		 Project Participants:
     
  • Martin Scheffler
  • Christoph Sladeczek
  • Robert Dünger
  • Markus Hartleb
  • Moritz Fehr
 

Short Description

Im Rahmen dieses Projekts soll ein interaktives audio-visuelles System realisiert werden, das sich aus einem stereoskopischen Multi-Viewer-System zur Darstellung und einem Wellenfeldsynthesesystem zur Audiowiedergabe zusammensetzen soll. Audiovisuelle Objekte lassen sich mit diesen System für jeden Benutzer in qualitativ hochwertiger Weise darstellen. Benutzer haben die Möglichkeit mit dreidimensional arbeitenden Eingabegeräten die virtuellen Objekte und deren auditive Eigenschaften zu verändern und so interaktiv eine audio-visuelle Umgebung zu gestalten und erfahren.

Zur Realisierung dieses Projektes wird in Zusammenarbeit mit dem Fraunhofer-Institut für Digitale Medientechnologie ein Wellenfeldsynthese-System über die OSC-Schnittstelle (Open Sound Control) an das VR-System Avango angebunden. Diese Konfiguration erlaubt es, den Audio-Renderer aus Avango via UDP mit einem standardisierten Protokoll zu steuern.

The Setup

Two users are standing in front of a screen. Each user is wearing a pair of shutter glasses.

On the screen, the users see a model of a river landscape. The position of the users are tracked, so the projection is perspectically and spatialy correct.

The users can walk around in front of the screen and the projection has the appearance of a window into the 3D scene.

By using an input device, the users are able to throw stones into the water of a river running through the landscape. When a stone hits the water, it makes a big splashing noise and then the river is gurgling around the stone.

Behind the projection screen, an array of loudspeakers creates the sounds of the river gurgling around the stones. The loudspeakers are hooked up to the wave field synthesis (WFS) system. The sounds of the stones are spatially correct, which means that the users can move around the (virtual) source of the sound, and independent from the user perspective, the sound is perceived as coming from this location.

Viewing: For stereoscopic viewing, each user has to be provided with one separate view for each eye. In a two user projection system, four separate views are projected onto the same screen. These views are separated using a filter system based on shuttering and polarization. We implemented this system for two users, so four LCD projectors were needed.

To enable perspectively correct projection, the shutter glasses are tracked using the ARTrack optical tracking system.

For this, each pair of shuttering glasses are equiped with retroreflective spherical markers.

We also created another demo, where the user can push around billard balls with a tracked stick. This was a big hit at the Mediengang-exhibition at the Bauhaus University.

Documents