Multisensory and Multimodal Interactive Maps for Visually Impaired People

Context

Visually impaired people face important challenges related to orientation and mobility. Accessible geographic maps are helpful for acquiring knowledge of an urban environment. Historically, raised-line paper maps with braille text have been used, but these maps possess significant limitations. For instance, only a small percentage of the visually impaired population reads braille. Recent technological advances have enabled the design of accessible interactive maps with the aim to overcome these limitations.

Accessible interactive map using multi-touch

We designed an accessible interactive map prototype based on the use of a multi-touch screen with a raised-line map overlay and speech output. In order to meet users’ needs, the design and development has been done following a participatory design cycle which actively involved visually impaired users all along the process.

So far the usability of accessible interactive maps had never been compared to the usability of raised-line maps. Therefore, we did not know whether interactive maps were worse or better solutions than traditional tactile paper maps with braille text for acquiring spatial knowledge in the absence of vision. Consequently, we evaluated usability of these two different map types for visually impaired people. Both maps were tested by 24 blind participants. The study showed that making the map interactive resulted in a higher efficiency and user satisfaction as compared to the tactile map. Improvement in spatial learning depended on users’ expertise and characteristics as well as the type of spatial knowledge (landmark, route, survey). This study demonstrates the importance of accessible interactive tactile maps for visually impaired people and their spatial cognition.
While the above mentioned accessible interactive map prototype demonstrated promising results for the spatial learning of visually impaired people, the content of the map remained limited to names of streets and buildings. One of the following steps was thus to design a prototype with more complex content. For this purpose, we integrated gestural interaction into the multi-touch map with tactile overlay.

Multisensory Maps for Visually Impaired People

In order to ensure their usability in the context of education and learning, it is important to evaluate educational technologies in the field. Thus, we deployed Mappie (an extension of the above mentioned accessible map) in a class of seven children and one caretaker during three months. In comparison with the previous map prototype, Mappie’s tactile overlay was coloured to be accessible not only by blind, but also by low vision and sighted people. Furthermore, a menu bar gave access to different types of spatial information (as an alternative to the above mentioned gestural interaction). Our formative study showed promising results and allowed insights in the design of accessible interactive maps, such as using unconventional modalities—olfactory and gustatory—to foster reflective learning, and using tangible objects to support storytelling.

Following this first study, we designed MapSense as an extension of Mappie. MapSense uses the same hardware and interaction techniques as Mappie, but additionally provides fourteen ”Do-It-Yourself” conductive tangibles. Some tangibles could be filled with scented material, such as olive oil, smashed raisins or honey, and thus creating a real multi-sensory experience. The map was explored during two classes of three hours separated by a week, taught conjointly by a locomotion trainer and a specialized teacher. We observed that the map and tangible objects triggered strong positive emotions and stimulated spatial learning as well as creativity of the visually impaired students.

To our knowledge the MapSense prototype is the first map to provide a multi-sensory experience that goes beyond audio-tactile output. We suggest that the possibilities of rapid prototyping provided by 3D printing are promising future venues for accessible interactive map experiences.

This work has been conducted as part of the ANR project Accessimaps during the PhD thesis of Emeline Brulé under the supervision of Gilles Bailly & Annie Gentes, in collaboration with the Elipse team at the IRIT research lab in Toulouse and Anke Brock (Inria-Potioc).

3D printed interactive maps for visually impaired people

With the emergence of 3D printing and low-cost microcontrollers, it is now easy to design affordable interactive small-scale models which are adapted to the needs of special education teachers. However, no study has previously been conducted to evaluate non-visual learning using interactive small-scale models. In collaboration with a specialized teacher, we designed a small-scale model and a raised-line map representing the evolution of the geography and history of a fictitious kingdom. The two conditions were compared in a study with 24 visually impaired students regarding the memorization of the spatial layout and historical contents. The study showed that the interactive small-scale model improved both space and text memorization as compared to the raised-line map with braille legend. In conclusion, we argue that affordable home-made interactive small scale models can improve learning for visually impaired students. Interestingly, they are adaptable to any teaching situation including students with specific needs.

This work has been done during the postdoc of Stéphanie Giraud (funded by a FIRAH scholarship) in collaboration with Christophe Jouffrais & Marc Macé (IRIT – Elipse, CNRS & University Toulouse).

Augmented reality maps for visually impaired people

As part of the VISTE project, we will design and implement an augmented reality prototype that will be used as spatial thinking training tool in special education schools. This prototype will make use of the PapARt technology, an OpenSource augmented reality framework.

Publications: