RESEARCH

2009

Projected Light Microscopy

Zoom: Click on Image We present an integrated and closed illumination technique for optical microscopy based on spatial light modulation at the object plane. The technique estimates the specimen modulation with uniform white light -- even though it is physically lit with an arbitrary image. From this estimation, we compute a new illumination pattern that provides improved visibility when observing the specimen through the oculars. This is achieved by enhancing the perceived local contrast through a second modulation of the derived illumination pattern by the specimen itself, and by reducing highlights caused by surface- and subsurface reflections. Real-time modulation estimation and illumination computations are performed continuously in a closed feedback loop to support movements of the specimen and interactive changes of the microscope's focus and magnification. We believe, that our illumination technique has potential in other fields, such as endoscopy. It can also be beneficial for digital microscopy, since images with a higher signal-to-noise ratio and reduced highlights can be captured. This can lead to more robust image analysis.

Bimber, O., Amano, T., Grundhoefer, A., Kurz, D., Kloeck, D., Thiele, S., Haentsch, F. Projected Light Microscopy ACM Siggraph (Talk), 2009 Movie (~67MB, DivX Codec)

Adaptive Coded Aperture Projection

Zoom: Click on Image We present an integrated and closed illumination technique for optical microscopy based on spatial light modulation at the object plane. The technique estimates the specimen modulation with uniform white light -- even though it is physically lit with an arbitrary image. From this estimation, we compute a new illumination pattern that provides improved visibility when observing the specimen through the oculars. This is achieved by enhancing the perceived local contrast through a second modulation of the derived illumination pattern by the specimen itself, and by reducing highlights caused by surface- and subsurface reflections. Real-time modulation estimation and illumination computations are performed continuously in a closed feedback loop to support movements of the specimen and interactive changes of the microscope's focus and magnification. We believe, that our illumination technique has potential in other fields, such as endoscopy. It can also be beneficial for digital microscopy, since images with a higher signal-to-noise ratio and reduced highlights can be captured. This can lead to more robust image analysis.

Grosse, M., Wetzstein, G., Grundhoefer, A., and Bimber, O. Coded Aperture Projection ACM Transactions on Graphics (to appear), 2009 (see video for details) Grosse, M., Wetzstein, G., Grundhoefer, A., and Bimber, O. Adaptive Coded Aperture Projection ACM Siggraph (Poster+Talk), 2009 Grosse, M. and Bimber, O. Coded Aperture Projection Emerging Display Technologies and Immersive Projection Technologies 2008 (EDT IPT08), 2008 Grosse, M. and Bimber, O. Coded Aperture Projection ACM Siggraph (Poster+Talk), 2008 Movie (~76MB, DivX Codec)

Subobject Detection through Spatial Relationships on Mobile Phones

Zoom: Click on Image We present a novel image classification technique for detecting multiple objects (called subobjects) in a single image. In addition to image classifiers, we apply spatial relationships among the subobjects to verify and to predict locations of detected and undetected subobjects, respectively. By continuously refining the spatial relationships throughout the detection process, even locations of completely occluded exhibits can be determined. Finally, all detected subobjects are labeled and the user can select the object of interest for retrieving corresponding multimedia information. This approach is applied in the context of PhoneGuide, an adaptive museum guidance system for camera-equipped mobile phones. We show that the recognition of subobjects using spatial relationships is up to 68% faster than related approaches without spatial relationships. Results of a field experiment in a local museum illustrate that unexperienced users reach an average recognition rate for subobjects of 85.6% under realistic conditions.

Brombach, B., Bruns, E. and Bimber, O. Subobject Detection through Spatial Relationships on Mobile Phones International Conference of Intelligent User Interfaces (IUI2009), 2009 Movie (~54MB, DivX Codec) PhoneGuide Project Web-Site

2008

Phone-to-Phone Communication for Adaptive Image Classification

Zoom: Click on Image We present a novel technique for adapting local image classifiers that are applied for object recognition on mobile phones through ad-hoc network communication between the devices. By continuously accumulating and exchanging collected user feedback among devices that are located within signal range, we show that our approach improves the overall classification rate and adapts to dynamic changes quickly. This technique is applied in the context of our PhoneGuide system -- a mobile phone based museum guidance framework that combines pervasive tracking and local object recognition for identifying a large number of objects in uncontrolled museum environments. We explain a technique that distributes the user feedback information during runtime through ad-hoc network connections between local devices. By doing so, we enforce cooperative classification improvements during the actual stay of the visitors. The general functionality of our technique has been tested with a small number of real devices in a museum. For proving its scalability, however, we have developed a simulator that evaluates our method for many hundred devices under several conditions. The simulation parameters have all been gathered in a museum, and are therefore realistic. We will show that ad-hoc phone-to-phone synchronization not only leads to higher overall classification rates, but also to quicker adaptations to dynamic changes during runtime.

Bruns, E. and Bimber, O. Phone-to-Phone Communication for Adaptive Image Classification International Conference on Advances in Mobile Computing & Multimedia (MoMM2008), pp. 276-281, 2008 Movie (~55MB, Quicktime Codec) PhoneGuide Project Web-Site

VirtualStudio2Go: Digital Video Composition for Real Environments

Zoom: Click on Image We synchronize film cameras and LED lighting with off-the-shelf video projectors. Radiometric compensation allows displaying keying patterns and other spatial codes on arbitrary real world surfaces. A fast temporal multiplexing of coded projection and flash illumination enables professional keying, environment matting, displaying moderator information, scene reconstruction, and camera tracking for non-studio film sets without being limited to the constraints of a virtual studio. The reconstruction of the scene geometry allows special composition effects, such as shadow casts, occlusions and reflections. This makes digital video composition more flexible, since static studio equipment, such as blue screens, teleprompters, or tracking devices, is not required. Authentic film locations can be supported with our portable system without causing a lot of installation effort. We propose a concept that combines all of these techniques into one single compact system that is fully compatible with common digital video composition pipelines, and offers an immediate plug-and-play applicability.

Grundhoefer, A. and Bimber, O. VirtualStudio2Go: Digital Videocomposition for Real Environments ACM Siggraph Asia, ACM Transactions on Graphics (TOG), vol. 27, no. 5, article 151, 2008 Movie (~60MB, DivX Codec) Supplementary Material (~60MB, ZIP) Augmented Studio Project Web-Site

Superimposing Dynamic Range

Zoom: Click on Image We present a simple and cost-efficient way of extending contrast, perceived tonal resolution, and color space of reflective media, such as paper prints, hardcopy photographs, or electronic paper displays. A calibrated projector-camera system is applied for automatic registration, radiometric scanning and superimposition. A second modulation of the projected light on the surface of such media results in a high dynamic range visualization. This holds application potential for a variety of domains, such as radiology, astronomy, optical microscopy, conservation and restoration of historic art, modern art and entertainment installations. In our experiments, we achieved contrast ratios of up of 45,000-60,000:1 with a peak luminance of more than 2,750 cd/m^2, could technically re-produce more than 620 perceptually distinguishable tonal values. Furthermore, we attained color space extensions of up to a factor of 1.4 (compared to a regular projection on white screens) or factor of 3.3 (compared to regular paper prints under environment light). Thereby, the hardcopy resolution can be several thousand DPI or several hundred LPI, while luminance and chrominance are modulated with a registration error of less than 0.3 mm. Thus, compared with most existing interactive HDR displays, we support near distance viewing at a contrast frequency of up to 7 cpd (given our current registration precision and assuming a viewing distance of 50 cm).

Bimber, O. and Iwai, D. Superimposing Dynamic Range ACM Siggraph Asia, ACM Transactions on Graphics (TOG), vol. 27, no. 5, article 150, December 2008 Bimber, O. and Iwai, D. Superimposing Dynamic Range ACM Siggraph (NewTech Demo), 2008 Bimber, O. and Iwai, D. Superimposing Dynamic Range Bauhaus-University Weimar, Technical Report #1358, 2008 Movie (~56MB, DivX Codec) Supplementary Images and Videos (~60MB, ZIP) Addendum: Implementation Details and Notes on Source Code (~30MB, PDF) Source Code (~210MB, ZIP) HDR Sample 1 (~2MB, EXR) HDR Sample 2 (~2MB, EXR) More HDR Samples in Supplementary Images and Videos Patent pending.

Coded Aperture Projection

Zoom: Click on Image In computer vision, optical defocus is often described as convolution with a filter kernel that corresponds to an image of the aperture being used by the imaging device. The degree of defocus correlates to the scale of the kernel. Convolving an image with the inverse aperture kernel will digitally sharpen the image and consequently compensate optical defocus. This is referred to as deconvolution or inverse filtering. In frequency domain, the reciprocal of the filter kernel is its inverse, and deconvolution reduces to a division. Low magnitudes in the Fourier transform of the aperture image, however, lead to intensity values in spatial domain that exceed the displayable range. Therefore, the corresponding frequencies are not considered, which then results in visible ringing artifacts if deconvolution is applied for projector defocus compensation. This is the main limitation of previous approaches, since in frequency domain the Gaussian PSF of spherical apertures does contain a large fraction of low Fourier magnitudes. Applying only small kernel scales will reduce the number of low Fourier magnitudes (and consequently the ringing artifacts) -- but will also lead only to minor focus improvements. To overcome this problem, we apply a coded aperture whose Fourier transform has less low magnitudes initially. This optical band-pass retains more frequencies and more image details are reconstructed. We integrate a coded aperture into an off-the-shelf projector to increase its depth of field. With the measured regional defocus on the screen surface, projected images are deconvolved with locally scaled aperture codes. This leads to better results than deconvolving with Gaussian PSFs in cases where regular spherical apertures are used. We achieve interactive frame-rates of up to 16 fps for an XGA resolution.

Grosse, M. and Bimber, O. Coded Aperture Projection Emerging Display Technologies and Immersive Projection Technologies 2008 (EDT IPT08), 2008 Grosse, M. and Bimber, O. Coded Aperture Projection ACM Siggraph (Poster+Talk), 2008 Grosse, M. and Bimber, O. Coded Aperture Projection Bauhaus-University Weimar, Technical Report #1302, 2008 Movie (~27MB, DivX Codec) sARc Project Web-Site

PlayReal: Spatial Augmented Reality Games

Zoom: Click on Image This student project investigates the potential of projector-camera systems as components for future game consoles. The geometry and radiometry of the real environment is automatically scanned, scene elements are identified, clustered and integrated into the game logic. Virtual game figures can then be projected stereoscopically into the real environment to enable an exciting augmented reality game experience. Artefacts that are due to modulation of projected light on the real environment's surfaces are compensated in real-time to achieve a certain level of consistency. Virtual objects can interact with real objects and vice versa. When colliding with virtual objects, for instance, real objects appear to shake through a special stereoscopic effect. A pan-tilt projector-camera unit will allow using large parts of an entire room as playground for spatial augmented reality games.

Movie (~42MB, in German)

Dynamic Bluescreens

Zoom: Click on Image Blue screens and chroma keying technology are essential for digital video composition. Professional studios apply tracking technology to record the camera path for perspective augmentations of the original video footage. Although this technology is well established, it does not offer a great deal of flexibility. For shootings at non-studio sets, physical blue screens might have to be installed, or parts have to be recorded in a studio separately. We present a simple and flexible way of projecting corrected keying colors onto arbitrary diffuse surfaces using synchronized projectors and radiometric compensation. Thereby, the reflectance of the underlying real surface is neutralized. A temporal multiplexing between projection and flash illumination allows capturing the fully lit scene, while still being able to key the foreground objects. In addition, we embed spatial codes into the projected key image to enable the tracking of the camera. Furthermore, the reconstruction of the scene geometry is implicitly supported.

Grundhöfer, A. and Bimber, O. Dynamic Bluescreens ACM Siggraph (Poster+Talk), 2008 Grundhöfer, A. and Bimber, O. Dynamic Bluescreens Bauhaus-University Weimar, Technical Report #1301, 2008 Movie (~50MB, DivX Codec) Augmented Studio Project Web-Site

Adaptive Training of Video Sets for Image Recognition on Mobile Phones

Zoom: Click on Image We present an enhancement towards adaptive video training for PhoneGuide, a digital museum guidance system for ordinary camera–equipped mobile phones. It enables museum visitors to identify exhibits by capturing photos of them. In this article, a combined solution of object recognition and pervasive tracking is extended to a client–server–system for improving data acquisition and for supporting scale–invariant object recognition. A static as well as a dynamic training technique are presented that preprocess the collected object data differently and apply two types of neural networks for classification. Furthermore, the system enables a temporal adaptation for ensuring a continuous data acquisition to improve the recognition rate over time. A formal field experiment reveals current recognition rates and indicates the practicability of both methods under realistic conditions in a museum.

Bruns, E. and Bimber, O. Adaptive Training of Video Sets for Image Recognition on Mobile Phones Journal of Personal and Ubiquitous Computing, 2008 (submitted: July 2007, accepted: February 2008) PhoneGuide Project Web-Site