Für eine korrekte Darstellung dieser Seite benötigen Sie einen XHTML-standardkonformen Browser, der die Darstellung von CSS-Dateien zulässt.

News

Youtube Channel CG

You are here

  • English
  • Deutsch

Project group: Neural Post Processing of 3D rendered Scenes

 

Semester:

Winter Term 23/24 and summer term 24

SWS/ECTS: 15

Recommended for:

Master-Inf

Prerequisite

Knowledge of fundamental concepts in Computer Graphics and Machine/Deep Learning

Examination Type:

Study achievement

Goals:

The goal of this project group is to enhance the output of a 3D renderer. In standard rendering, adding more details to synthetic images usually requires exponentially more computational power.

[A scene from GTA V where the filter is applied only on the right side of the image] Richter, et al. 2021

[An image rendered into different styles] Gatys, et al. 2015

Approach:

In this project group, we want to leverage the power of neural networks as a post processor. This can be done to transfer the rendered output into different artistic styles or enhance the realism of the scene.
Additionally, it can also be used to apply various effects such as motion blur or depth of field. Aside from the quality improvements, a neural post processor can also be used to improve the rendering performance by applying super resolution or neural denoising on a low-sampled rendered output.

There are several potential challenges, such as maintaining the temporal coherence of the refined rendering output between different frames (e.g. for a moving camera), to ensure smooth transitions between the consecutive frames without visible artifacts.

References

Stephan R. Richter, Hassan Abu AlHaija and Vladlen Klotun, "Enhancing Photorealism Enhancement", 2021.
Leon A. Gatys, Alexander S. Ecker and Matthias Bethge, "A Neural Algorithm of Artistic Style", 2015.
Manu M. Thomas, Gabor Liktor, Cristoph Peters, Sungye Kim, Karthik Vaidyanathan and Angus G. Forbes, "Temporally Stable Real-Time Joint Neural Denoising and Supersampling", 2022.
Haozhi Huang, Hao Wang, Wenhan Luo, Lin Ma, Wenhao Jiang, Xiaolong Zhu, Zhifeng Li and Wei Liu, "Real-Time Neural Style Transfer for Videos", 2017.