Light Field Acquisition and Rendering
Hintergrund
In the first part of the PMDLumi project, we focused on developing new techniques for real-time rendering of light fields from 3D models. The standard approach of these techniques is based on a set of preacquired image-data. From these image data, new virtual views can be synthesized. Additional per pixel depth information, provided by a time-of-flight camera, is taken into account to improve image quality.
The light field data format as well as techniques for real-time acquisition and rendering of light fields have been developed using synthetic data. Synthetic data are currently extracted from a PMD-Simulator, which has also been developed by our group.
Our light field data format is based on a spherical approximation parameterizing camera space. For each sample position on the spherical arrangement a parabolic map of the opposite hemisphere, containing combined RGB and depth values per-pixel (RGBz), is stored. Both, the resolution of the camera parametrization as well as the image resolution of the parabolic map are adjustable. For camera space parametrization we provide hierarchical spherical arrangements containing either 12, 42, 162, 642 or even 2562 sample positions.
Our spherical light field renderer is based on a raycasting approach and runs on the GPU. It uses per-pixel depth information for efficient depth correction of rays. Image synthesis is then performed by rendering the smooth shaded faces of the polygonal camera sphere. For per-fragment interpolation an additional false-color is assigned to each vertex to establish per-pixel camera weights with individual polygons. Surface points are established very precisely for each fragment based on the parabolic maps stored with the nearby vertices.
Applying our raycasting approach, high-quality rendering results are achieved without any visible ghosting artifacts at high update rates of up to 53 frames per second. The rendering results show that simple objects can be reconstructed from as few as 12 light field samples. For more complex objects a sample count of 42 has proven to be sufficient in most cases.
This research project is partially funded by grant KO-2960-6/1,2 from the German Research Foundation (DFG).
Results
See our Light Field Archive for light field downloads and to access the light field renderer demo application.
The hierarchical representation of the spherical sample positions allows for the implementation of a continous level of detail for best rendering performance.
As the light field moves further away from the virtual viewpoint a coarser spherical approximation is chosen for light field synthesis. The implementation of LOD for
light field rendering facilitates the synchronous reconstruction of several light fields.
Based on the surface points established within the raycasting rendering algorithm correct per pixel depth values are determined based on the virtual viewpoints current
modelview matrix thus providing the possibility to smoothly composite light field renderings with different sources of 3D renderings.
LOD as well as correct depth evaluation for object composition are shown in the left image. The right image shows the spherical approximations
as they are refined on approaching the virtual viewpoint. The video is generated from a slightly different viewpoint, compared to the viewpoint used for LOD calculations
to improve visibility of the LOD switch.
(The polygonal Stanford Bunny model is shaded gray accompanied by the light fields of the Armadillo Model and the Bust model.)
Click on the images for a video of the scene.
The table shows the sample count per light field accompanied by the rendering performance and the total data size for a sample image resolution of 512 x 512 pixels. Videos on the right show real-time rendering results of our sphercial light field renderer. All of the original models are taken from the The Stanford 3D Scanning Repository .
See our Light Field Results for more videos of our rendering results.
Non of the rendering results available from this domain, neither images nor videos or light field data sets is to be used in any form without the explicit consent of Severin S. Todt and Christof Rezk-Salama .
| Sample Count | Render Performance | Data size | Video |
| 12 | 85 FPS | 3 MB |
|
| 42 | 78 FPS | 10 MB |
 Rightmost video shows light field rendering(left) compared with polygonal rendering of the original Stanford Bunny model(right) |
| 162 | 60 FPS | 41 MB |
|
| 642 | 52 FPS | 164 MB |
|
The images show rendering results based on our raycasting approach (click to enlarge):
1. Reconstruction of the Stanford Bunny model, sampled at a resolution of 512x512 pixels from 42 sample positions.
2. Same reconstruction, showing the spherical approximation applied to parametrize camera space.
3. Reconstruction of the Lucy model, at identical image resolution, but 162 light field samples.
4. Closeup of 3.
5. Reconstruction of simple Cat model. The synthetic image cleary shows the precision of our rendering technique. The concavity is reconstructed without artifacts.
Veröffentlichungen
| · | Lisa Brückbauer, Christof Rezk-Salama, Andreas Kolb |
| Relighting Spherical Light Fields With Polynomial Texture Mapping | |
| In VMV 2010: Vision, Modeling & Visualization, 2010 | |
| [bib] |
| · | S. Todt, C. Rezk-Salama, A. Kolb |
| Light Field Rendering for Games | |
| In Proc. Theory and Practice of Computer Graphics, 2008, pages 27--33, Best Student Paper Presentation Award | |
| [bib] [pdf] |
| · | Christof Rezk-Salama, Severin Todt and Andreas Kolb |
| Raycasting of Light Field Galleries from Volumetric Data | |
| In Computer Graphics Forum (Proc. EuroVis), 27(3), 2008, pages 839-846 | |
| [bib] [pdf] |
| · | S. Todt, M. Langer, C. Rezk-Salama, A. Kolb, K.D. Kuhnert |
| Spherical Light Field Rendering in Application for Analysis by Synthesis | |
| In Int. J. on Intell. Systems and Techn. and App. (IJISTA), Issue on Dynamic 3D Imaging, 5(1), 2008, pages 304 - 314 | |
| [bib] [pdf] |
| · | S. Todt, C. Rezk-Salama, A. Kolb |
| Fast (Spherical) Light Field Rendering with Per-Pixel Depth | |
| In Technical Report , Computer Graphics Group, University of Siegen, 2007 | |
| [bib] [pdf] |
| · | Severin S. Todt, C. Rezk-Salama, A. Kolb |
| Real-Time Fusion of Depth and Light Field Images | |
| In ACM SIGGRAPH Posters, 2005 | |
| [bib] [pdf] |