Seamless follows a multi-disciplinary approach in the scientific domains of virtual and augmented reality, human perception, human-computer interaction and human factors. The current revolution on the XR domain does not only require technological advancements, but to place the user at the center of this revolution. In order to support this revolution, and referring to the two aforementioned scenarios, three major scientific challenges (SC) are addressed:
Enable seamless transition between realities. Each reality presents its own limitations and features, which make them distinct both in terms of perception and interaction. A user must be able to transition across realities at will. Seamless aims to reduce the gap between perception and interaction modalities across realities in order to provide a continuous experience. Ultimately, ensuring that all realities can co-exist.
Enable seamless collaboration between realities. Different users laying in different realities should be able to collaborate as equals. By equals, we first refer to the perception/awareness of collaborators and the parallel realities. Users should be aware of other collaborators and should be able to share a common workspace. Second, we refer to the users’ interaction capabilities. Users must be able to interact and collaborate as equals, no matter the reality they are lying in or their individual characteristics, ultimately blurring the boundaries between realities during the collaboration process.
Enable seamless evaluation of user experience. The evaluation of user experience in VR/AR is multi-factorial, which requires the assessment of a wide range of parameters, from user performance, passing through user preferences, to the user’s mental state. Evaluation is a key step in any design process which allows to detect flaws and improve the system. Current evaluation methods strongly rely on the use of questionnaires or task-specific metrics, which only provide broad and imprecise evaluation of the user experience. The vision of Seamless is that evaluation of VR and AR systems should be able to provide precise information regarding the flaws of the system and should minimize the explicit collection of information from the user. A particular promising direction is the direct assessment of the user’s mental state through electroencephalography (EEG).
In order to address the three main scientific challenges, the research program of Seamless is structured through three main research axes following a bottom-up approach. Our bottom-up approach is user-centered, first considering the perceptual capabilities of users, then their interaction capabilities, and finally how the overall experience is perceived.
Research Axes
Model and enhance human perception among realities. The perception and awareness of our surroundings is required for any interaction. Virtual and augmented reality technology is highly heterogeneous and significantly alters how humans perceive their surroundings. In order to achieve seamless transition and collaboration among realities, fundamental research is needed in order to better understand human perception within and between different realities. On one hand, the mediation generates a perceptual degradation. Current XR technology is unable to provide the same cues for perceiving the virtual content as humans perceive real content. Thus, users will have a degraded/reduced perception of the virtual content which can hinder/limit their interaction capabilities. On the other hand, the mediation enables the control of the perceptual information that is available to the user, augmenting (e.g., displaying hidden or additional information) or diminishing it (e.g. removing distracting information). This research axis focuses on fundamental and applied research to better understand human perception in extended reality and how extended reality can be used to alter the perceptual information to enhance human capabilities.
Model and enhance interaction among realities. Interaction is strongly coupled with perception, not only because perception dictates available interactions, but also because interaction and perception can be modeled through a closed feedback loop. However, in a context of a highly heterogeneous reality-virtuality continuum, interaction modalities and capabilities highly differ within- and between-realities. For example, interaction methods that would be effective for VR contexts, might be non-optimal for AR contexts. Furthermore, in an application context in which users will constantly switch between realities, the challenges of seamless transition and collaboration raise. First, fundamental research is needed to model how users interact in such continuum when perceptual capabilities are degraded and/or augmented. Second, there is the need to propose novel interaction methods that ensure equivalent interaction capabilities along the interaction continuum. Finally, in the context of social XR, these methods should efficiently support collaboration.
Model and enhance user experience among realities. Interaction techniques leverage knowledge on user perception and user behavior in order to improve the interaction process, however, the evaluation of the system should not only be focused on purely performance metrics. In this respect, user experience is a fundamental aspect in any interactive system, which relates to the capacity of the users to use the system and their subjective experience while interacting with the system. However, the number of dimensions of user experience in VR and AR systems is larger than in traditional computer systems. This axis focuses on the research of such additional dimensions, notably, presence, embodiment, accessibility and learnability. The goal of this axis is to propose innovative methods to assess them, model them and predict them.
