RESEARCH

2006

Digital Illumination for Augmented Studios

Zoom: Click on Image Virtual studio technology plays an important role for modern television productions. Blue-screen matting is a common technique for integrating real actors or moderators into computer generated sceneries. Augmented reality offers the possibility to mix real and virtual in a more general context. We propose a unifying technological approach for combining real studio content with computer-generated information. Digital light projection allows a controlled spatial, temporal, chrominance and luminance modulation of illumination – opening new possibilities for TV studios. Embedded imperceptible pattern projection allows integrating coded patterns into projected images that are used for in-shot camera tracking and depth acquisition. Images can be projected that are not recorded by the studio cameras to display direction information spatially anywhere within a studio environment. Projector-based and screen-based illumination makes an on-the-fly re-illumination of a studio possible without changing the physical light sources. Further applications can be imagined for advanced lighting in modern photo studios and stage performances.

Bimber, O., Grundhöfer, A., Zollmann, S., and Kolster, D. Digital Illumination for Augmented Studios Journal of Virtual Reality and Broadcasting, vol. 3, no. 8, 2006

Passive-Active Geometric Calibration for View-Dependent Projections onto Arbitrary Surfaces

Zoom: Click on Image Projecting images onto surfaces that are not optimized for projections becomes more and more popular. Such approaches will enable the presentation of graphical, image or video content on arbitrary surfaces. Virtual reality visualizations may become possible in everyday environments - without specialized screen material or static screen configurations. Upcoming pocket projectors will enable truly mobile presentations on all available surfaces of furniture or papered walls. The playback of multimedia content will be supported on natural stonewalls of historic sites without destroying their ambience through the installations of artificial projection screens. We present a hybrid technique for correcting distortions that appear when projecting images onto geometrically complex, colored and textured surfaces. It analyzes the optical flow that results from perspective distortions during motions of the observer and tries to use this information for computing the correct image warping. If this fails due to an unreliable optical flow, an accurate –but slower and visible– structured light projection is automatically triggered. Together with an appropriate radiometric compensation, view-dependent content can be projected onto arbitrary everyday surfaces. An implementation mainly on the GPU ensures fast frame rates.

Zollmann, S., Langlotz, T. and Bimber, O. Passive-Active Geometric Calibration for View-Dependent Projections onto Arbitrary Surfaces Workshop on Virtual and Augmented Reality of the GI-Fachgruppe AR/VR, pp. 181-191, 2006 Zollmann, S., Langlotz, T. and Bimber, O. Passive-Active Geometric Calibration for View-Dependent Projections onto Arbitrary Surfaces Journal of Virtual Reality and Broadcasting, vol. 4, no. 6, 2007 (re-print from Workshop on Virtual and Augmented Reality of the GI-Fachgruppe AR/VR 2006) Movie (~46MB)

Real-Time Adaptive Radiometric Compensation

Zoom: Click on Image Our new radiometric compensation algorithm considers the human visual perception properties to reduce visible artefacts resulting from the limited dynamic range and brightness of projectors. It preserves a maximum of luminance and contrast and is implemented entirely on the GPU. Real-time frame rates are achieved for supporting animated and interactive content. Initially our algorithm performs an off-line analysis of the projection surface’s geomety and reflectance. The image content is then analyzed to determine the average luminance values, the amount of high spatial frequencies, and a luminance threshold map. The threshold map stores information about the maximum non-perceivable luminance differences for each pixel. The radiometric compensation is carried out in two passes: In the first pass the intensity values are translated and scaled globally depending on the surface reflectance and the image content itself. The result is analyzed for clipping errors. These errors are then blurred with a Gaussian kernel. The applied sigma is inverse proportional to the amount of high spatial frequencies in the local image areas. In the final pass the image intensities are translated and scaled globally, but the luminance values are also adjusted locally depending on the defocused clipping errors. Time dependent adaptation factors are used for global and local transformations to avoid popping artifacts in animated and interactive content.

Grundhöfer, A. and Bimber, O. Real-Time Adaptive Radiometric Compensation Bauhaus-University Weimar, Technical Report #823, 2006 InIEEE Transactions on Visualization and Computer Graphics (TVCG), vol. 14, no. 1, pp. 97-108, 2008, (submitted: August 2006, accepted: February 2007, electronic version published: March 2007) Grundhöfer, A. and Bimber, O. Real-Time Adaptive Radiometric Compensation Siggraph'06 (Poster), 2006 Movie (~39MB, DivX Codec)

Low-Cost Stereoscopic Projection in Room Corners

Zoom: Click on Image A two-sided stereoscopic front-projection serves as an initial test platform for our sARc project. Instead of applying specialized projection screen the walls of an ordinary room corner is used for creating an immersive experience. This makes the system extremely inexpensive and portable. Two ceiling mounted 120Hz DPL projectors with front mounted wide angle lenses create a 6 x 2.2 meter image in a resolution of 1600x800 pixels. The image covers the user’s visual field and supports disparity based depth perception of the presented content. An optical tracking system ensures a correct projection from arbitrary perspectives. Quest 3D serves a presentation platform on the software side. A dedicated Quest channel has been developed that renders two stereo-pairs that are compressed into a single 1600x800 image. This image is streamed from an application PC to display PC that decomposes the sub-images and projects them consistently on the walls. Thereby the application and the calibrated projection of the content are independent from each other. The method for compensating secondary scattering described below is being carried out in the display PC before projecting the images. This project is in cooperation with the Faculty of Architecture, Bauhaus-University Weimar.

sARc Project Web-Site

Augmenting Large Scale Optical Holograms

Zoom: Click on Image Large scale optical holograms require large scale display technology for combining them with interactive graphical elements. Shuttered projection screens (SPS) can be used to sequentially display stereoscopic graphics (in the diffuse mode) and to reconstruct the holographic content (in the transparent mode). While the SPS is shuttered with 50Hz, the stereo pairs are synchronized and time-modulated at approximately 100Hz. Depth information of the holographic content are required to create consistent occlusion and illumination effects with the graphical content. A two-lens stereo camera system can be used for scanning the hologram partially. The different point clouds have to merge into a common coordinate system to form the whole surface. Due to the limited resolution of the range sensor, small gaps appear between the actual surface points. Instead of triangulating the points into a mesh of triangle primitives, the points remain unconnected. They are rendered as point primitives (splatted) with appropriate radii to fill the gaps of missing surface information. The splat size and resolution is adapted dynamically with respect to the observer’s position to ensure interactive frame rates.

Bimber, O. Merging Graphics and Holograms Journal of Holography and Speckle, vol.3, no. 2, pp. 73-79, 2006 Bimber, O. Augmenting Holograms IEEE Computer Graphics and Applications, 2006 Bimber, O. HoloGraphics: Combining Holograms with Interactive Computer Graphics Technical Report, 2006 HoloGraphics Project Web-Site Movie (~88MB, DivX Codec)