RESEARCH

2005

Enabling Mobile Phones To Support Large-Scale Museum Guidance

Zoom: Click on Image Mobile phones have the potential of becoming a future platform for personal museum guidance. They enable full multimedia presentations and –assuming that the visitors are using their own devices– will significantly reduce acquisition and maintenance cost for museum operators. However, several technological challenges have to be mastered before this concept can be successful. One of them is the question of how individual museum objects can be intuitively identified before presenting corresponding information. We have developed an enhanced museum guidance system called PhoneGuide that uses widespread camera equipped mobile phones for on-device object recognition in combination with pervasive tracking. It provides additional location- and object-aware multimedia content to museum visitors, and is scalable to cover a large number of museum objects. In a field survey our system was able to identify 155 real museum exhibits from multiple perspectives with a recognition rate of 95% and a classification speed of less than one second per object. A coarse grid of only eight low-cost Bluetooth emitters distributed over two museum floors was used to achieve these results. Once an object has been recognized, related multimedia presentations such as videos, audio , text, computer graphics and images are displayed on the phone. Special thanks to the City Museum of Weimar and to CellIQ for their support.

Bruns, E., Brombach, B., Zeidler, T. and Bimber, O. Enabling Mobile Phones To Support Large-Scale Museum Guidance In IEEE Multimedia, vol. 14, no. 2, pp. 16-25, 2007 (submitted: October 2005, accepted: July 2006) Bruns, E., Brombach, B., Zeidler, T., and Bimber, O. Enabling Mobile Phones To Support Large-Scale Museum Guidance Technical Report, 2005

Compensating Indirect Scattering for Immersive and Semi-Immersive Projection Displays

Zoom: Click on Image Concavely shaped projection screens, such as CAVEs, two-sided workbenches, domes, or cylinders scatter a fraction of light to other screen portions. The amount of indirect illumination adds to the directly projected image and causes the displayed content to appear partially inconsistent and washed out. We have developed a reverse radiosity method that compensates first-level and higher-level secondary scattering effects in real-time. The images appear more brilliant and uniform when reducing the scattering contribution. A numerical solution is approximated with Jacobi iteration for a sparse-matrix linear equation system on the GPU. Efficient data structures allow packing the required data into textures which are processed by pixel shaders. Frame-buffer objects are used for a fast exchange of intermediate iteration results, and enable computations with floating point precision. Our algorithm’s result can be optimized for quality or performance.

Bimber, O., Grundhöfer, A., Zeidler, T., Danch, D., and Kapakos, P. Compensating Indirect Scattering for Immersive and Semi-Immersive Projection Displays In proceedings of IEEE Virtual Reality (IEEE VR'06), 2006 Bimber, O. New analytical solution in Projector-Based Augmentation Technical Report, 2006 Presentation (Macromedia Flash) sARc Project Web-Site Movie (~36MB)

Multi-Focal Projection

Zoom: Click on Image Many multi-projector rendering techniques exist that aim at creating a high consistency of image geometry, intensity and color. We proposed a concept and a solution for considering and optimizing a fourth image property – its focus. We describe a novel multi-focal projection concept that applies conventional video projectors and camera feedback. Multiple projectors with differently adjusted focal planes, but overlapping image areas are used. They can be either arbitrarily positioned in the environment, or can be integrated into a single projection unit. During an automatic one-time calibration process, structured light projection together with camera feedback allows to measure the relative focus value of every projector pixel on an arbitrary diffuse surface. Thereby, the focus values are geometrically and radiometrically corrected. If this is known, a final image with minimal defocus can be composed in real-time from individual pixel contributions of all projectors. Our technique is independent of the surfaces’ geometry, color and texture, of the environment light, as well as of the projectors’ parameters (i.e., position, orientation, luminance and chrominance).

Bimber, O. and Emmerling, A. Multi-Focal Projection: A Multi-Projector Technique for Increasing Focal Depth In IEEE Transactions on Visualization and Computer Graphics (TVCG), 2006 Bimber, O., Wetzstein, G., Emmerling, A., and Nitschke, C. Enabling View-Dependent Stereoscopic Projection in Real Environments In proceedings of International Symposium on Mixed and Augmented Reality (ISMAR'05), 2005 Presentation (~5MB) sARc Project Web-Site Movie (~42MB)

Enabling View-Dependent Stereoscopic Projection in Real Environments

Zoom: Click on Image With this work we take a first step towards an ad-hoc stereoscopic projection within real environments. We show how view-dependent image-based and geometric warping, radiometric compensation, and multi-focal projection enable a view-dependent visualization on ordinary (geometric complex, colored and textured) surfaces within everyday environments. All these techniques are accomplished at interactive rates and on a per-pixel basis for multiple interplaying projectors. Special display configurations for immersive or semi-immersive VR/AR applications that require permanent and artificial projection canvases might become unnecessary. Such an approach does not only offer new possibilities for augmented reality and virtual reality, but also allows merging both technologies. This potentially gives some application domains – like architecture – the possibility to benefit from the conceptual overlaps of AR and VR. Special thanks to the Faculty of Architecture, Bauhaus-University Weimar for their support.

Bimber, O., Wetzstein, G., Emmerling, A., and Nitschke, C. Enabling View-Dependent Stereoscopic Projection in Real Environments In proceedings of International Symposium on Mixed and Augmented Reality (ISMAR'05), pp. 14-23, 2005 Presentation (~5MB) sARc Project Web-Site Movie: Technology (~40MB) Movie: Head-tracked visualization inside an 11th century water reservoir (~7MB, DivX) Siggraph'05 Emerging Technologies (Supported by A.R.T., more3D , Bauhaus-University Weimar) Experiments inside an 11th century water reservoir in Peterborn/Erfurt (in cooperation with Bennert Group)

PhoneGuide: Museum Guidance Supported by On-Device Object Recognition on Mobile Phones

Zoom: Click on Image We present PhoneGuide – an enhanced museum guidance approach that uses camera-equipped mobile phones and on-device object recognition. Our main technical achievement is a simple and light-weight object recognition approach that is realized with single-layer perceptron neuronal networks. In contrast to related systems which perform computational intensive image processing tasks on remote servers, our intention is to carry out all computations directly on the phone. This ensures little or even no network connectivity and consequently decreases cost for online times. Our laboratory experiments and field surveys have shown that photographed museum exhibits can be recognized with a probability of over 90%. We have evaluated different feature sets to optimize the recognition rate and performance. Our experiments reviled that normalized color features are most effective for our method. Choosing such a feature set allows recognizing an object below one second on up-to-date phones. The amount of data that is required for differentiating 50 objects from multiple perspectives is les than 6KBytes. Special thanks to the Senckenberg Museum of Natural History, Frankfurt, to the Museum für Ur- und Frühgeschichte, Weimar, and to CellIQ for their support.

Föckler, P., Zeidler, T., Brombach, B., Bruns, E., and Bimber, O. PhoneGuide: Museum Guidance Supported by On-Device Object Recognition on Mobile Phones In proceedings of International Conference on Mobile and Ubiquitous Computing (MUM'05), 2005 Movie (Architecture Example) (~4MB) Movie (Senckenberg Museum - Content Provider) (~7MB) Movie (Senckenberg Museum - User) (~2MB)