Technical Session II
Date: Wednesday, November 23rd, 2016
Time: 10:30-12:00
Chair: Gabriel Zachmann, University of Bremen
Towards an A&VR molecular tangible interface based on a lightweight RGB markerless tracking of a modular physical model
Xavier Martinez, Nicolas Férey, Jean-Marc Vezien and Patrick Bourdot
Abstract: Physical models have always been used in the molecular science field as an understandable representation of complex molecules, particularly in chemistry. Even if physical models were recently completed by numerical molecular visualizations, they are still involved in research work and teaching. Indeed, they are more suitable than virtual objects for manipulating and building molecular structures, while in silico visualization proposes a wide range of molecular representations and rendering features. In this paper, we present a markerless tracking method to construct a molecular virtual representation from a flexible and modular physical model. Our approach is based on a single RGB camera, in interactive time to use the physical model as a tangible interface, and thus benefit from both physical and virtual representations. This method was designed to require only a light virtual and augmented reality hardware setup, such as a smartphone or HMD & mounted camera, providing a markerless molecular tangible interface usable in a classroom context or on a classical biochemistry researcher desktop. The approach proposes a fast image processing algorithm based on color blob detection to extract 2D atom positions of an userdefined conformation in each frame of a video. A tracking algorithm recovers a set of 2D projected atom positions as an input of the 3D reconstruction based on a Structure From Motion method. We tuned this method to robustly process a few key feature points and combine them within a global point cloud. Biological knowledge drives the final reconstruction, filling missing atoms to obtain the desired molecular conformation.
Spatial Augmented Reality environments design rules
Yahya Al-Meslemi and Frédéric Noël
Abstract: Augmented Reality finds an increasing interest for professional applications. A more specialized field concerned Spatial Augmented Reality (SAR) where virtual content is directly projected onto the real 3D world. Basically video-projectors are used to project virtual content on the top of physical objects. The principle is well known but the conditions to design an acceptable and efficient Spatial Augmented Reality environment are seldom well managed. The current paper reports some basic rules useful for the design of an efficient SAR system. Tracking and Visualization conditions are discussed to identify the parameters of a SAR system.
Augmented Reality in the Treatment of the Liver Tumors
Lucio Tommaso De Paolis
Abstract: Minimally Invasive Surgery is a surgery technique that provides evident advantages for the patients, but also some difficulties for the surgeons. In medicine, the Augmented Reality (AR) technology allows surgeons to have a sort of “X-ray” vision of the patient’s body and can help them during the surgical procedures. In this paper we present an application of Augmented Reality that could be used as support for a more accurate preoperative surgical planning and also for an image-guided surgery. The AR can support the surgeon during the needle insertion for the Radiofrequency Ablation of the liver tumours in order to guide the needle and to have an accurate placement of the surgical instrument within the lesion. The augmented visualization can avoid as much as possible to destroy healthy cells of the liver.
Social Platform for Heritage Awareness and Participation
Nikos Frangakis
Abstract: Pluggable Social Platform for Heritage Awareness and Participation (PLUGGY) will support citizens in shaping cultural heritage and being shaped by it. PLUGGY will enable them to share their local knowledge and everyday experience with others. The PLUGGY Social Platform will facilitate a continuing process for creating, modifying and safeguarding heritage where citizens will be prosumers and maintainers of cultural activities. It will be web based, easily accessed and will allow the development of shared identity and differentiation. PLUGGY Social Platform’s users will curate stories using the PLUGGY Curatorial Tool. Content will be both crowdsourced and retrieved from digital collections, allowing users to create links between seemingly unrelated facts, events, people and digitized collections, leading to new approaches of presenting cultural resources, and new ways of experiencing them. An Augmented Reality, a 3D sonic narration, a Geolocation and a collaborative Games applications will be developed to allow the full experience of the Virtual Museums created in the Social Platform of PLUGGY.
