Design of a natural interface for HMDs for CSI the Hague
In this project an HMD (head mounted display) was used to display augmented reality in 3D using a separate screen for each eye. Two cameras map the entire crime scene in 3D using state-of-the-art mapping and tracking software which is being developed at the TU Delft. This tool enables the forensic researcher to use built in tools such as thermal imaging, spectral imaging, and online databases to save details such as evidence and notes. Because the entire crime scene has been mapped and digitized, it can later be revisited to test hypotheses or witness statements.
This project’s goal was to design a natural interface which would be controlled with either gestures or a tangible object. Because the system, which is currently still being developed, could not yet stably use readily available objects, a product which could take on different forms was designed. The Fold consists of six panels that can be connected to each other in several different ways. The panels are held together by a shape connection combined with magnets. The panels can either be clicked into each other to form a tablet, hexagonal prism, stack, or several other configurations.
Using AR markers for tracking, menus with which the user can interact are augmented onto either the panels or the environment around the panels. Apart from the augmented menus, a HUD is always present with system information and a description of the option that is selected. Because the panels are hollow, real-world functionality can be added to them in the form of UV LED lights and lasers.
The thesis can be downloaded here
Parts of the thesis were also incorporated in a paper published by ACM:
Dragos Datcu, Thomas Swart, Stephan Lukosch, Zoltan Rusak, “Multimodal Collaboration for Crime Scene Investigation in Mediated Reality”, 14th ACM International Conference on Multimodal Interaction – ICMI, Santa Monica, California, USA, October 22-26, 2012.