Material-based Segmentation (SoSe 2024)
Project Description:
The material properties of the objects around us influence their visual appearance. Whether looking at a porcelain vase, or a cotton fabric, humans easily recognize which parts of a surface belong to the same material type. Differences in color and texture patterns, as well as shape discontinuities are strong cues for detecting regions of certain material. However, material properties can change within the boundaries of a single object (a stone statue partially covered with golden paint), while colors can be identical for multiple distinct objects (a white vase on top of a while table cloth).
Similar to the problem of semantic segmentation, the goal is to partition images into meaningful regions. The meaning of these regions, however, is not related to object boundaries or pure color similarities but to material properties instead. Depending on the lighting conditions, the viewing direction and the shape of a given object, the same material may have a very different appearance across the surface of the object.
In this project, we want to explore, implement, evaluate and compare different state-of-the-art algorithms for automated material-based scene segmentation. The result of such automated segmentation can later be used for high-quality material estimation of real-world objects with complex geometry and reflectance properties.
Challenges:
- unknown number of materials in the scene
- presence of spatially varying (inhomogeneous) materials
- global illumination effects such as interreflection, subsurface scattering, self-shadowing
- unknown lighting conditions
- lack of geometric information for the observed scene
Prerequisites:
- successful completion of either Deep Learning for Computer Vision or Image Analysis and Object Recognition
- solid programing skills