RESEARCH

2007

Mobile Phone Enabled Museum Guidance with Adaptive Classification

Zoom: Click on Image We present a new adaptive classification system for museum guidance tasks. It uses camera-equipped mobile phones for on-device object recognition in ad-hoc sensor networks and provides location and object aware multimedia content to museum visitors. Our approach is invariant against perspective, distance and illumination. It supports the scalable identification of single objects and multiple sub-objects, pervasive tracking, phone-to-sensor and phone-to-phone communication. It adapts to user behaviour and environmental conditions over time and achieves high recognition rates under realistic conditions. Our decentralized classification approach makes the system highly scalable to an arbitrarily large number of users since the heavy-weight training process is carried out off-line on the server while the lower-weight classification task is performed individually and in parallel by each mobile phone.

Bruns, E., Brombach, B. and Bimber, O. Mobile Phone Enabled Museum Guidance with Adaptive Classification IEEE Computer Graphics and Applications, vol. 28, no. 4, pp. 98-102, 2008 (submitted: November 2007, accepted: March 2008) Movie (~25MB) PhoneGuide Project Web-Site

Unsynchronized 4D Barcodes

Zoom: Click on Image We present a novel technique for optical data transfer between public displays and mobile devices based on unsynchronized 4D barcodes. We assume that no direct (electromagnetic or other) connection between the devices can exist. Time-multiplexed, 2D color barcodes are displayed on screens and recorded with camera equipped mobile phones. This allows to transmit information optically between both devices. Our approach maximizes the data throughput and the robustness of the barcode recognition, while no immediate synchronization exists. Although the transfer rate is much smaller than it can be achieved with electromagnetic techniques (e.g., Bluetooth or WiFi), we envision to apply such a technique wherever no direct connection is available. 4D barcodes can, for instance, be integrated into public web-pages, movie sequences, advertisement presentations or information displays, and they encode and transmit more information than possible with single 2D or 3D barcodes.

Langlotz, T. and Bimber, O. Unsynchronized 4D Barcodes International Symposium on Visual Computing, pp. 363-374, 2007 Langlotz, T. and Bimber, O. Unsynchronized 4D Barcodes Bauhaus-University Weimar, Technical Report #913, 2007 Movie: transmission rate of 800-2.400 characters/s, implemented by TU Graz (~20MB) PhoneGuide Project Web-Site First Public Presentations: Bauhhaus-University's open house, July 13th-16th 2006, click here for July 2006 online release Bauhhaus-University's open house, February 12th 2007, click here for February 2007 online release Bauhhaus-University's open house, July 12th-15th 2007, click here for February 2007 online release

Radiometric Compensation Through Inverse Light Transport

Zoom: Click on Image Radiometric compensation techniques allow seamless projections onto complex everyday surfaces. Implemented with projector-camera systems they support the presentation of visual content in situations where projection-optimized screens are not available or not desired - as in museums, historic sites, air-plane cabins, or stage performances. We propose a novel approach that employs the full light transport between projectors and a camera to account for many illumination aspects, such as interreflections, refractions, shadows, and defocus. Pre-computing the inverse light transport in combination with an efficient implementation on the GPU makes the real-time compensation of captured local and global light modulations possible.

Wetzstein, G. and Bimber, O. Radiometric Compensation through Inverse Light Transport Pacific Graphics, pp. 391-399, 2007 Wetzstein, G. and Bimber, O. A Generalized Approach to Radiometric Compensation Bauhaus-University Weimar, Technical Report #800, 2006 Wetzstein, G. and Bimber, O. Radiometric Compensation of Global Illumination Effects with Projector-Camera Systems Siggraph'06 (Poster), 2006 Movie (~40MB)

Imperceptible Calibration for Radiometric Compensation

Zoom: Click on Image We present a novel multi-step technique for imperceptible geometry and radiometry calibration of projector-camera systems. Our approach can be used to display geometry and color corrected images on non-optimized surfaces at interactive rates while simultaneously performing a series of invisible structured light projections during runtime. It supports disjoint projector-camera configurations, fast and progressive improvements, as well as real-time correction rates of arbitrary graphical content. The calibration is automatically triggered when mis-registrations between camera, projector and surface are detected.

Zollmann, S. and Bimber, O. Imperceptible Calibration for Radiometric Compensation EUROGRAPHICS (short), pp. 61-64, 2007 Zollmann, S. and Bimber, O. Imperceptible Calibration for Radiometric Compensation Bauhaus-University Weimar, Technical Report #859, 2007 Movie (~46MB, MPEG Codec)

Dynamic Adaptation of Projected Imperceptible Codes

Zoom: Click on Image We present a novel adaptive imperceptible pattern projection technique that considers parameters of human visual perception. A coded image that is invisible for human observers is temporally integrated into the projected image, but can be reconstructed by a synchronized camera. The embedded code is dynamically adjusted on the fly to guarantee its non-perceivability and to adapt it to the current camera pose. Linked with real-time flash keying, for instance, this enables in-shot optical tracking using a dynamic multi-resolution marker technique. A sample prototype is realized that demonstrates the application of our method in the context of augmentations in television studios.

Grundhöfer, A., Seeger, M., Häntsch, F., and Bimber, O. Dynamic Adaptation of Projected Imperceptible Codes IEEE International Symposium on Mixed and Augmented Reality (ISMAR'07), pp. 181-190, 2007 Grundhöfer, A., Seeger, M., Häntsch, F., and Bimber, O. Coded Projection and Illumination for Television Studios Bauhaus-University Weimar, Technical Report #843, 2007 Grundhöfer, A., Seeger, M., Häntsch, F., and Bimber, O. Dynamic Adaptation of Projected Imperceptible Codes Bauhaus-University Weimar, Technical Report #873, 2007 Movie (~45MB, DivX Codec)

Laser Pointer Tracking in Projector-Augmented Architectural Environments

Zoom: Click on Image We present a system that applies a custom-built pan-tilt-zoom camera for laser-pointer tracking in arbitrary real environments. Once placed in a building environment, it carries out a fully automatic self-registration, registrations of projectors, and sampling of surface parameters, such as geometry and reflectivity. After these steps, it can be used for tracking a laser spot on the surface as well as an LED marker in 3D space, using inter-playing fisheye context and controllable detail cameras. The captured surface information can be used for masking out areas that are critical to laser-pointer tracking, and for guiding geometric and radiometric image correction techniques that enable a projector-based augmentation on arbitrary surfaces. We described a distributed software framework that couples laser-pointer tracking for interaction, projector-based AR as well as video see-through AR for visualizations with the domain specific functionality of existing desktop tools for architectural planning, simulation and building surveying.

Kurz, D., Häntsch, F., Grosse, M., Schiewe, A., and Bimber, O. Laser Pointer Tracking in Projector-Augmented Architectural Environments IEEE International Symposium on Mixed and Augmented Reality (ISMAR'07), pp. 19-26, 2007 Kurz, D., Häntsch, F., Grosse, M., Schiewe, A., and Bimber, O. Laser Pointer Tracking in Projector-Augmented Architectural Environments Bauhaus-University Weimar, Technical Report #874, 2007 Movie (~45MB, DivX Codec)

360 Degrees Surround Smart Projection

Zoom: Click on Image In cooperation with Bennert Monumedia, we have installed a multi-projector system in the main tower of the Osterburg in Weida. Eight synchronized projectors apply our Smart Projector technology projecting corrected video content directly onto the available natural stone walls. This museum will offer an immersive perspective on the historical surrounding landscape to its visitors. For this project, we have developed a software that pre-corrects large format video footage to be projected onto the complex underlying surface. Therefore, no PCs are requited at the museum site itself. Eight low-cost media set-top boxes are used instead. The panoramic video frames are geometry corrected, radiometrically compensated and blended on a per pixel-basis.

City of Weida Bennert Monumedia