Differentiable Inverse Rendering for Material Estimation

Project Description:

With this project, we would like to gain hands-on experience with the reconstruction of relightable 3D scenes. This involves not only the computation of scene geometry, but also the correct separation of lighting information and object material properties.   Together with polygonal (triangular) meshes, point clouds are among the most commonly used geometric primitives. Recently, 3D Gaussians [Kerbl et al., 2023], which can be understood as a volumetric extension of point-based surface geometry, have gained a lot of popularity in the fields of computer graphics and vision because of their capability of representing scenes in a near-photorealistic manner. Therefore, we want to focus our attention on exploring state-of-the-art relighting methods for point-based geometric representations including, but not limited to, 3D Gaussians. As a part of the practical experience, we would also generate our own point-based reconstructions of real-world objects varying in complexity with respect to their appearance (material properties). Finally, we aim to design and implement possible extensions of existing techniques to enable the relighting of objects with complex material properties.

Prerequisites:

  • successful completion of a Computer Graphics course
  • successful completion of a Photogrammetric Computer Vision course
  • solid programing skills in C/C++ or Python
  • desire for cooperative teamwork