Project Non-Photorealistic Rendering for Volumetric Avatars

Prof. Dr. Bernd Fröhlich
M.Sc. Gareth Rendle
M.Sc. Adrian Kreskowski
B.Sc. Anton Lammert

Volumetric avatars are real-time 3D reconstructions that allow users of Virtual Reality (VR) systems to be accurately represented in a virtual environment. By fusing colour and depth contributions from RGBD cameras, one can produce volumetric avatar streams that allow rich communication between users in the virtual environment, conveying fluid full-body motions and gestures and facial expressions.

 

While photo-realism is a desirable characteristic of volumetric avatars in many use cases, there are some contexts in which detailed reconstructions may be unsuitable or unnecessary. For example, virtual environments are often simplified versions of real rooms, where photo-realistic avatars may appear incongruent, making the VR experience less plausible. Moreover, high-resolution textures increase bandwidth requirements when transmitting avatar streams. To address this, parts of the avatar could be textured with a reduced, cartoon-like colour palette.  

 

In these cases, a family of techniques known as Non-Photorealistic Rendering (NPR) techniques could prove useful. NPR techniques do not attempt to simulate the real world in as much detail as possible, but are instead designed to represent objects in an abstract manner. Depending on the context, their aim may be to improve understanding and recognition of surface features, to reveal hidden parts of an object, or to create visually consistent artistic styles. As part of the NPR4VR project in summer semester 2022, we investigated how different NPR effects can be applied to 3D geometry in stereoscopic viewing contexts.

In this project, we will:

• review NPR literature to find NPR techniques suitable for application to volumetric avatar streams
• implement a selection of NPR techniques in Unity, using hand-crafted shader pipelines and Unity's abstract graphics API, and apply them to real-time avatar data
• plan and execute a user study in VR to assess the perceived effect of the non-photorealistic volumetric avatar techniques

You will:

• Get hands-on experience with Unity
• Learn about NPR techniques
• Gain experience in carrying out user studies in VR

In this project, we will:

• review NPR literature to find NPR techniques suitable for application to volumetric avatar streams
• implement a selection of NPR techniques in Unity, using hand-crafted shader pipelines and Unity's abstract graphics API, and apply them to real-time avatar data
• plan and execute a user study in VR to assess the perceived effect of the non-photorealistic volumetric avatar techniques

You will:

• Get hands-on experience with Unity
• Learn about NPR techniques
• Gain experience in carrying out user studies in VR

• Solid programming skills in C# or C++ are required
• Experience in the field of real-time computer graphics and shader programming is helpful
• Previous experiences with graphics APIs such as OpenGL DirectX or Vulkan is helpful

If you are in doubt as to whether you fulfil the requirements, or if you have any further questions regarding the project, we are happy to have a discussion with you before or after the project fair on 10th of October, or after that via email to gareth.rendle[at]uni-weimar.de.

The final assessment of your work will be conducted based on the project contributions of every team member, including:

• Active participation in the project during and in between weekly meetings
• Design, implementation and evaluation of spatially and temporally consistent non-photorealistic rendering effects usable in a Cross-Platform context within Unity
• Intermediate talks
• Intermediate and final project presentations
• Documentation in form of a short paper

If you are excited about real-time graphics programming, non-photorealistic rendering and virtual reality, we would be happy to see you at the project fair!