This project explores the combination of Physical and Virtual Reality through the usage of Mixed Reality. Early explorations involved the usage of Spatial Augmented Reality in combination with Virtual Reality, two technologies with complementary characteristics that evolved separately in the past. Spatial Augmented Reality (SAR) augments the environment using projectors or screens, without the need of user instrumentation. By keeping a single unified frame of reference, it supports social interaction and natural perception of the space, but the augmentation is limited by physical constraints (e.g., it requires a surface to display information). Immersive head mounted displays on the other hand are not limited by the physical properties of the environment, yet they isolate the user from their environment.
By providing a unified frame of reference for both SAR and immersive displays, the users can select the visualization that is best suited for a given task. This enables both asymmetric collaboration between users, and back-and-forths for a single user.
These explorations were followed by the combination of additional modalities, in an incremental fashion. This way, one or more users can chose the desired modalities, and immerse themselves as much as the task requires. As a result, the virtual world can be framed in relationship with the physical one.
The combination of physical and virtual spaces could benefit multiple applications, particularly in the context of asymmetric collaboration. Complex industrial environments, such as the aerospace industry, already use Virtual Reality to ease the conception and iteration of ideas. Yet, the decisions involved are usually taken by experts discussing in physical meeting rooms. We explored the possibility of bridge these two spaces in order to ease their awareness and communication capabilities, by building on our previous work.
Currently, our efforts are focused on the understanding of the human capabilities to interact with these heterogeneous representations, with some very promising results.
Joan Sol Roo, Martin Hachet. One Reality: Augmenting How the Physical World is Experienced by Combining Multiple Mixed Reality Modalities. UIST’17, ACM Symposium on User Interface Software and Technology, Quebec, Canada. October 2017.
Damien Clergeaud*, Joan Sol Roo*, Martin Hachet, Pascal Guitton. Towards Seamless Interaction between Physical and Virtual Locations for Asymmetric Collaboration. VRST’17, 23rd ACM Symposium on Virtual Reality Software and Technology, Gothenburg, Sweden. November 2017. *Shared first co-authorship.
Joan Sol Roo, Martin Hachet. Towards a Hybrid Space Combining Spatial Augmented Reality and Virtual Reality. 3DUI’17, IEEE Symposium on 3D User Interfaces, March 2017, California, USA. 2017. http://3dui.org/. (Best tech-note award)