When Human-Computer Interaction Meets Neurotechnologies: What You Should Know about Brain-Computer Interfaces

We will give a course about brain-computer interfaces (BCI) at CHI 2017, on May 11th, 2017, in Denver, USA.

Summary

Brain-Computer Interfaces (BCIs) have brought new, exciting and promising perspectives of interaction. On the one hand, active BCIs enable users to control applications (such as assistive technologies or video games) using their brain activity alone. On the other hand, passive BCIs bring the possibility of adapting an application/interface based on users’ mental states. In this course, we first aim at introducing BCIs to the HCI community and to discuss how BCI-based applications could benefit HCI. Then, in a practical session, we will propose all participants to implement their own BCI, in a very simple way, using the free OpenViBE software. Finally, we will have discussions about what is possible or not with BCIs, what are their pros and cons.

Introduction

Brain-Computer Interfaces (BCIs) are systems that can translate the brain activity of a user — typically measured using ElectroEncephaloGraphy (EEG) — into a command for an interactive application. For instance, a BCI can enable a user to move a cursor to the left or right of a computer screen by imagining left or right hand movements respectively. Since they make computer control possible without any physical activity, EEG-based BCIs have promised to revolutionize many applications areas, notably to control assistive technologies (e.g. control of text input systems or wheelchairs) for motor-impaired users or as input devices for entertainment and Human-Computer Interaction (HCI). BCI technologies can also be used beyond communication and control, as a tool to monitor brain activity and mental states to assess interactive systems ergonomics or to design adaptive interfaces reacting to the user’s mental states. BCIs are thus increasingly used in the HCI community. However, BCI design is highly multidisciplinary, involving computer science, signal processing, cognitive neurosciences and psychology, among others. As such, fully understanding and using BCI can be difficult. This course aims at making this possible by providing the audience with an overview of BCI design, research and application.

Benefits

Such a course is a great opportunity to make innovative research challenges emerge and to create collaborations between the HCI and BCI communities. Indeed, the BCI community could greatly benefit from HCI researchers’ expertise in designing and evaluating their training protocols and applications while on the other hand, the HCI community will have a better idea of how they could use BCIs either as a new control modality or as an evaluation tool. At the end of the course, participants will know the different types of BCIs and what can be done with each of them. They will also be able to design and implement their own BCI using the open-source BCI platform OpenViBE. Finally, perceiving BCIs with their real capabilities — beyond science fiction and mainstream media reports — should help HCI professionals to be aware of the pitfalls to avoid and the ethical issues to consider when designing a BCI protocol.

Content

The course is composed of three sessions, where each session lasts around 80 minutes. The first session will provide an introduction to the main BCI types and applications, with basic signal processing techniques. The second session will be a hands-on one, covering practical BCI design and use, with an introduction to real-time processing of actual EEG recordings using OpenViBE. The last session will involve interactive discussions to introduce the audience with BCIs ethics, with their real capabilities, and with the way to conduct rigorous BCI research.

Session 1: An introduction to BCI

The first session will start with a brief history of BCIs and their purpose. We will then introduce “active BCIs” — controlling an application using brain-activity alone (e.g. Jeunet et al. 2015) — and “passive BCIs” — enhancing interaction by considering physiological data, such as monitoring workload (e.g. Frey et al. 2016). Multiple examples of the use of BCIs among various scientific communities will be given. Afterwards, the neuromarkers used for designing these BCIs will be explained. We will also describe the approaches for processing EEG signals and extracting relevant features. Finally we will mention some of the latest advances in BCI research, such as the use of instructional design to improve user training protocols.

Session 2: Practical BCI design and use with the OpenViBE software

This session will provide practical exercises for designing a BCI system. To do so, we will bring both medical grade EEG (a Brain Products LiveAmp), and consumer grade EEG (Emotiv and/or OpenBCI). We will show the audience how to record EEG signals with them (pros and cons), and illustrate the negative impact of muscle artifacts (notably facial or neck muscles) or ocular artifacts (i.e. blinks or eyes movements) that are concurrently recorded with EEG sensors. Such artifacts can pollute EEG signals and are one of the main confounding factors in EEG experiments.

Later on, we will initiate the audience to real-time EEG processing, analysis and visualization with OpenViBE. OpenViBE is a free and open-source software platform for the design, evaluation and use of real-time BCI systems. It can enable anyone to design their own BCI without writing a single line of code, simply by visually assembling boxes representing different EEG processing modules. As such, OpenViBE has been used all over the world by hobbyist, neuroscientists, artists, teachers, engineers or psychologists (and many others). We already organized two OpenViBE workshops during international conferences (International BCI conference 2014, International BCI meeting 2016).

During this session, we will show the audience how to visualize EEG signals in real-time, how to filter and process them, and how to design a very simple and basic BCI to detect eye opening or closing, or left-hand, right-hand or foot movement (both real and imagined movements). The ease-of-use of OpenViBE indeed enables the audience to realize such BCI in about an hour only.

Session 3: Discussion about BCI capabilities, pitfalls and related ethical considerations

As BCIs bring new interaction possibilities, they are sometimes associated with science fiction stories. This results in a wide media coverage as well as in strong reactions from the population, ranging from fear to be controlled/manipulated to extreme hope for patients to recover their impaired functions. To confront and possibly prevent this, researchers who develop and use BCIs have to be utterly aware of BCI real capabilities, of their pitfalls as well as of BCI-related ethical considerations (Haselager et al. 2009). In this way only, we will be able to lead useful research and to communicate properly about BCIs with the general public and patients. For this reason, in this third session, we will propose discussions aiming to guide participants to understand (1) real phenomena underlying BCI systems, (2) our boundaries and ethical responsibilities, leading to correct behaviour with end-users and the media. More specifically, we will propose group work on different themes, as follows:

• Brain activity recordings: choice of sensors – invasive or not, and their application. Principle of balance on risk ratio
• Cost of BCIs: BCI applications should be affordable by their end-users. Principle of accessibility.
• BCI enabling control: if a problem occurs while a patient is controlling a wheelchair using BCI, who is responsible? The patient, the engineer who conceived the BCI or the company who built the wheelchair? Principle of responsability.
• Experiments with patients: how should BCIs be presented to patients? How far should the experiment be described to them and their surroundings? Principle of informed consent form & management of the generated hope.

For each of these themes, participants will be provided with different documents which would stimulate discussions, e.g. scientific articles, newspaper articles, testimonies, videos. They will also be provided with a set of questions, hopefully inspiring/engaging them to form opinions. At the end of the session, each group will present their conclusions and discuss/argue with the other participants.

Intended Audience

The content of the course is suitable for students, industrials and researchers that are beginners in BCI research and use. People already familiar with BCI may nonetheless benefit from session 2 and 3 if they want to learn about the OpenViBE platform or reevaluate their practice.

Prerequisites: None.

Resources: Samples of course slides made by the authors for other courses are provided as additional material.

Instructor background

Fabien Lotte is a research scientist (with tenure) at Inria, and owns a PhD and an M.Sc. in Computer Science. He has more than 12 years of experience in BCI, and published about 90 peer-reviewed papers on the topic. He has taught about BCI in multiple international events (APSIPA 2010, BBCI neurotechnology summer schools 2012 & 2014, ICASSP 2014, IBRO Cognitive Neuroscience School 2016) and edited a book on BCI in 2016. https://sites.google.com/site/fabienlotte/

Jérémy Frey holds a PhD in Computer Science and a master in Cognitive Science. Over the last few years, he has been exploring how physiological computing could contribute to HCI, publishing in the very best conferences (CHI, UIST, TEI, INTERACT). He teaches computer science and has participated in workshops and scientific outreach activities in various international venues. http://phd.jfrey.info/

Camille Jeunet holds a PhD and a master degree in Cognitive Sciences. She is currently a post-doc at Inria Rennes, France and EPFL, Geneva, Switzerland. She is working on BCI user-training. In the last 3 years, she published a dozen peer reviewed papers on BCI, including 4 journals (impact factor > 3). She teaches cognitive sciences, BCI and ethics at the University, and took part in the development of a MOOC dedicated to scientific integrity. https://camillejeunet.wordpress.com/

Jelena Mladenović is a PhD student in Computer Science at Inria Bordeaux Sud-Ouest with a bachelor in mathematics and a master in Computational Biology and Biomedecine. She is working on computational modeling of user skills and states for improving BCI usability. http://jmladeno.net/

Léa Pillette recently obtained a M.Eng. from the National Institute of Technologies in Bordeaux and is currently starting a PhD. aiming to redefine formative feedback in BCI user training at Inria Bordeaux Sud-Ouest.