Project Group: Interactive Ray Tracing Effects in Virtual Reality Applications

Topic

The new generation of NVIDIA RTX graphics cards, launched just days ago, promises ray tracing effects at interactive frame rates through ray tracing extensions that can be combined with traditional rasterization-based rendering. Such effects include area lights and soft shadows, indirect illumination, improved ambient occlusion, mirroring and metal-like materials, translucency and even caustics.

Of course, these effects introduce additional computational costs that will affect the achievable frame rate.

Virtual Reality applications have very hard constraints regarding the frame rate: they must render both left and right eye view at the display frame rate (e.g.~90 Hz for HTC Vive) to avoid simulator sickness and loss of immersion caused by missed frames.

The task of this project group is to implement a Virtual Reality game that can benefit massively from ray tracing effects (see below for details and ideas), and use the latest NVIDIA RTX graphics card hardware and driver extensions to add as many helpful visual effects as possible while still maintaining the required frame rate.

During the course of this project group, participants will

• Learn ray tracing basics
• Learn the new Vulkan API and its latest ray tracing extensions
• Learn about the constraints and requirements of Virtual Reality applications
• Use state of the art graphics and display hardware (NVIDIA RTX, HTC Vive)

The minimal goal that needs to be reached in order to pass is that a simple but functional Virtual Reality game runs on
the HTC Vive and / or our large scale stereo projection screen, with at least a few ray tracing based effects. The ideal result is an attractive Virtual Reality game that makes intelligent use of advanced ray tracing effects both for visual effects and for gameplay.

Participating in this project group requires team spirit, perseverance, and an interest in Computer Graphics. You need to have passed Computer Graphics I. Additionally, it is helpful to have participated in at least one other computer graphics course, e.g. Virtual Reality, Computer Graphics III, or Computer Graphics Practical Training.

Virtual Reality Game:

The game principle should be relatively straightforward to implement, but allow much room for the application of ray tracing effects, which is the main focus of this project group. For example, the 3D VR sceneray may be partly or even largely obstructed, but hidden objects may reveal themselves indirectly through ray tracing effects such as indirect illumination, indirect soft shadows, or visibility in partly mirroring surfaces or through translucent objects. From these clues, a user can then draw conclusions about the scene and act accordingly, e.g. to solve puzzles or to find a way forward.

One example for this principle idea is a 3D variant of the classic 2D brick breaker game principle (Arkanoid, Krakout, Break ’em All, ...; see screen shots above): the player has a pedal (in 3D: a translucent shield in front of him) and uses this to steer a moving ball, preventing it from leaving the playing area, and collecting bonus objects that the game generates. Goal of the game is to reduce a set of bricks in the playing area by making the ball touch each brick.  Different types of bricks have different sets of visual and behavioural properties; some may generate bonus objects when destroyed.  As in the typical 2D game, the 3D variant ignores gravity, i.e. the ball is always bounced from the shield towards the playing area and does not drop to the floor. Five walls (left, right, top, bottom, far end) limit the playing area.  The ball can leave the playing area by flying past the player, and is then lost (game over).

As the playing field can be modeled as a grid, with each grid cell being either empty or holding a specific type of brick in a specific state, and ball physics only need to handle collisions with non-moving objects, the basic game principle is relatively straightforward to implement.

Examples for helpful visual effects based on ray tracing: As bricks are stacked along the line of view, translucency effects can help the player to keep an overview of the bricks still in place. Some bricks, the ball itself, or bonus objects may act as area lights (causing soft shadows) depending on the current game situation. Bricks may consist of mirroring or metal-like materials. Indirect illumination or shadowing may give hints about the existence of bricks or bonus objects that are not directly visible.

If you are interested, please contact Martin Lambers.