BrainConquest: Boosting Brain-Computer Communication with High Quality User Training

ERC Starting Grant, 2017-2022

PI: Fabien Lotte

Overview:

ERC Starting Grant (PI: Fabien Lotte)
Date: 2017-2022
Abstract:

Brain-Computer Interfaces (BCIs) are communication systems that enable users to send commands to computers through brain signals only, by measuring and processing these signals. Making computer control possible without any physical activity, BCIs have promised to revolutionize many application areas, notably assistive technologies, e.g., for wheelchair control, and man-machine interaction. Despite this promising potential, BCIs are still barely used outside laboratories, due to their current poor reliability. For instance, BCIs only using two imagined hand movements as mental commands decode, on average, less than 80% of these commands correctly, while 10 to 30% of users cannot control a BCI at all.

A BCI should be considered a co-adaptive communication system: its users learn to encode commands in their brain signals (with mental imagery) that the machine learns to decode using signal processing. Most research efforts so far have been dedicated to decoding the commands. However, BCI control is a skill that users have to learn too. Unfortunately how BCI users learn to encode the commands is essential but is barely studied, i.e., fundamental knowledge about how users learn BCI control is lacking. Moreover standard training approaches are only based on heuristics, without satisfying human learning principles. Thus, poor BCI reliability is probably largely due to highly suboptimal user training.

In order to obtain a truly reliable BCI we need to completely redefine user training approaches. To do so, I propose to study and statistically model how users learn to encode BCI commands. Then, based on human learning principles and this model, I propose to create a new generation of BCIs which ensure that users learn how to successfully encode commands with high signal-to-noise ratio in their brain signals, hence making BCIs dramatically more reliable. Such a reliable BCI could positively change man-machine interaction as BCIs have promised but failed to do so far.

Project members:
PhD students:

Léa Pillette 2017-2019 (partly funded by the ERC)
Jelena Mladenovic 2017-2019 (party funded by the ERC)
Aurélien Appriou 2017-2020
Camille Benaroch 2018-2021
Aline Roc 2019-2022

Post-docs:

Khadijeh Sadatnejad 2019-2021
Sébastien Rimbert 2021-2022
Cécilia Ostertag 2022

Engineers/Technicians:

Thibaut Monseigne 2019-2022
Pauline Dreyer 2021-2022

Visiting students:

Mehdi Bugallo 2018
Satyam Kumar 2018
Zachary Traylor 2022-2023

Master student interns:

Aline Roc 2018
Camille Benaroch 2018
Romain Sabau 2019
Alina Lushnikova 2020
David Trocellier 2020
Eidan Tzdaka 2020
Lena Kolodzienski 2020
Nibras Abo Alzahab 2021
Alper Er 2021
Smeety Pramij 2019 & 2021
Jordan Azzouguen 2022

Selected publications:

Journals:

R.N. Roy, M.F. Hinss, L. Darmet, S. Ladouce, E.S. Jahanpour, B. Somon, X. Xu, N. Drougard, F. Dehais, F. Lotte, “Retrospective on the first passive brain-computer interface competition on cross-session workload estimation”, Frontiers in Neuroergonomics: Neurotechnology and Systems Neuroergonomics, 2022 – pdf
C. Benaroch, C. Jeunet, F. Lotte, “When should MI-BCI feature optimization include prior knowledge, and which one?“, Brain-Computer Interfaces, vol. 9, no. 2, pp 115-128, 2022 – pdf
K. Sadatnejad, F. Lotte, “Riemannian channel selection for BCI with between-session non-stationarity reduction capabilities“, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022 – pdf
J. Mladenovic, J. Frey, S. Pramij, J. Mattout, F. Lotte, “Towards identifying optimal biased feedback for various user states and traits in motor imagery BCI“, IEEE Transactions on Biomedical Engineering, vol. 69, no. 3, pp. 1101-1110, 2022 – featured article – pdf
A. Appriou, L. Pillette, D. Trocellier, D. Dutartre, A. Cichocki, F. Lotte, “BioPyC, an Open-Source Python Toolbox for Offline Electroencephalographic and Physiological Signals Classification”. MDPI Sensors, vol. 21, no. 5740, 2021 – pdf
L. Pillette, B. N’Kaoua, R. Sabau, B. Glize, F. Lotte, “Multi-session influence of two modalities of feedback and their order of presentation on MI-BCI user training”, MDPI Multimodal Technologies and Interaction, vol. 5, no. 3, 2021 – pdf
C. Benaroch*, K. Sadatnejad*, A. Roc, A. Appriou, T. Monseigne, S. Pramij, J. Mladenovic, L. Pillette, C. Jeunet, F. Lotte (*: Authors contributed equally), “Long-term BCI training of a Tetraplegic User: Adaptive Riemannian Classifiers and User Training“, Frontiers in Human Neuroscience, section Brain-Computer Interfaces, vol. 15, 2021 – pdf
L. Pillette*, A. Roc*, B. N’Kaoua, F. Lotte (*: Authors contributed equally), “Experimenters Influence on Mental-Imagery based Brain-Computer Interface User Training“, International Journal of Human-Computer Studies, vol. 149, 102603, 2021 – pdf
A. Roc, L. Pillette, J. Mladenovic, C. Benaroch, B. N’Kaoua, C. Jeunet, F. Lotte, “A review of user training methods in brain computer interfaces based on mental tasks“, Journal of Neural Engineering, vol. 18, 011002, 2021 – pdf
L. Pillette, F. Lotte, B. N’kaoua, P.-A. Joseph, C. Jeunet, B. Glize, “Why we should systematically assess, control and report somatosensory impairments in BCI-based motor rehabilitation after stroke studies”, Neuroimage-Clinical, Elsevier, vol. 28, pp.102417, 2020 – pdf
A. Appriou, A. Cichocki, F. Lotte, “Modern machine learning algorithms to classify cognitive and affective states from electroencephalography signals“, IEEE System, Man & Cybernetics Magazine, vol. 6, no. 3, pp. 29-38, 2020 – pdf
L. Pillette, C. Jeunet, B. Mansencal, R. N’Kambou, B. N’Kaoua, F. Lotte, « A physical learning companion for Mental-Imagery BCI User Training », International Journal of Human-Computer Studies, vol. 136, pp. 102380, 2020 – pdf
F. Lotte*, C. Jeunet*, R. Chavarriaga*, L. Bougrain*, D. E. Thompson, R. Scherer, Md R. Mowla, A. Kübler, M. Grosse-Wentrup, K. Dijkstra, N. Dayan (*: Authors contributed equally), “Turning negative into positives! Exploiting ‘negative’ results in Brain-Machine Interface (BMI) research“, Brain-Computer Interfaces, vol. 6, no. 4, pp. 178-189, 2019 – pdf
F. Lotte, C. Jeunet, “Defining and Quantifying Users’ Mental Imagery-based BCI skills: a first step“, Journal of Neural Engineering, vol. 15, no. 4, 2018 – link – pdf – code
C. Jeunet, F. Lotte, J.-M. Batail, P. Philip, J.-A. Micoulaud-Franchi, “Using recent BCI literature to deepen our understanding of clinical neurofeedback: a short review”, Neuroscience, vol. 378, pp. 225-233, 2018 – pdf
F Lotte, L Bougrain, A Cichocki, M Clerc, M Congedo, A Rakotomamonjy, and F Yger, “A Review of Classification Algorithms for EEG-based Brain-Computer Interfaces: A 10-year Update“, Journal of Neural Engineering, vol. 15, no. 3, 2018 – featured article – Highly cited JNE paper 2018 (cf here) – link – pdf

Conferences:

S. Rimbert, D. Trocellier, F. Lotte, “Is Event-Related Desynchronization variability correlated with BCI performance?”. In Proc. IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence, and Neural Engineering (MetroXRAINE’22), 2022 – pdf
M.S. Yamamoto, F. Lotte, F. Yger, S. Chevallier, “Class-distinctiveness-based frequency band selection on the Riemannian manifold for oscillatory activity-based BCIs: preliminary results”, IEEE EMBS Engineering in Medecine and Biology Conference (EMBC’22), 2022 – pdf
J. Mladenović, F. Lotte, J. Mattout, J. Frey, “Simple Probabilistic Data-driven Model for Adaptive BCI Feedback“, Proc. NAT 2022 – 3rd Neuroadaptive Technology Conference, 2022 – pdf
S. Rimbert, F. Lotte, “How ERD modulations during motor imageries relate to users’ traits and BCI performances“, IEEE EMBS Engineering in Medecine and Biology Conference (EMBC’22), 2022 – pdf
A. Appriou, F. Lotte, “Tools for affective, cognitive and conative states estimation from both EEG and physiological signals”, Proceedings of the Third International Neuroergonomics Conference, 2021 – Young investigator award – pdf
J. Mladenović, J. Frey, J. Mattout, F. Lotte, “Biased feedback influences learning of Motor Imagery BCI”, 8th International BCI Meeting, 2021 – best student talk award – pdf
L. Bougrain, S. Rimbert, P.L.C. Rodrigues, G. Canron, F. Lotte, “Guidelines to use Transfer Learning for Motor Imagery Detection: an experimental study“, 10th International IEEE/EMBS Conference on Neural Engineering (NER), 2021 – pdf
K. Sadatnejad, F. Lotte, “Assessing a new form of BCI user learning”, 8th International BCI meeting, 2021 – pdf
S. Pramij, A. Roc, F. Lotte, “Freedom! Making a case for more options for users during training in BCI”, 8th International BCI meeting, 2021 – pdf
M.S. Yamamoto, K. Sadatnejad, R. M. Islam, F. Lotte, T. Tanaka. “Reliable outlier detection by spectral clustering on Riemannian manifold of EEG covariance matrix”, 8th International BCI meeting, 2021 – best student poster award – pdf
A. Appriou, J. Ceha, S. Pramij, D. Dutartre, E. Law, P.-Y. Oudeyer, F. Lotte, “Towards measuring states of epistemic curiosity through electroencephalographic signals”, IEEE Systems, Man & Cybernetics (IEEE SMC’20) conference, 2020 – pdf
E. Tzdaka, C. Benaroch, C. Jeunet, F. Lotte, “Assessing the Relevance of Neurophysiological Patterns to Predict Motor Imagery-Based BCI Users’ Performance“, IEEE Systems, Man & Cybernetics (IEEE SMC’20) conference, 2020 – pdf
M.S. Yamamoto, K. Sadatnejad, T. Tanaka, M.R. Islam, Y. Tanaka, F. Lotte, “Detecting EEG outliers for BCI on the Riemannian manifold using spectral clustering”. 42nd Annual International Conferences of the IEEE Engineering in Medicine and Biology Society (EMBC’2020), pp. 438-441, 2020 – pdf
K. Sadatnejad, A. Roc, L. Pillette, A. Appriou, T. Monseigne, F. Lotte, “Channel selection over Riemannian Manifold with non-stationarity consideration for brain-computer interfaces applications“, International Conference on Audio, Speech and Signal Processing (ICASSP’2020), pp. 1364-1368, 2020 – pdf
A. Roc, L. Pillette, B. N’Kaoua, F. Lotte, “Would Motor-Imagery based BCI user training benefit from more women experimenters?”, 8th International Graz Brain-Computer Interface Conference, 2019 – pdf
C. Benaroch, C. Jeunet, F. Lotte, “Are users’ traits informative enough to predict/explain their mental-imagery based BCI performances?”, 8th International Graz Brain-Computer Interface Conference, 2019 – pdf
S. Kumar, F. Yger, F. Lotte, “Towards Adaptive Classification using Riemannian Geometry approaches in Brain-Computer Interfaces“, IEEE International Winter Conference on Brain-Computer Interfaces, 2019 – pdf – code
L. Pillette, A. Appriou, A. Cichocki, B. N’Kaoua, F. Lotte, “Classification of attention types in EEG signals”, International BCI Meeting 2018 – pdf
F. Lotte, A. Cichocki, “Can transfer learning across motor tasks improve motor imagery BCI?”, International BCI Meeting 2018 – pdf
F. Lotte, A. Cichocki, “What are the best motor tasks to use and calibrate SensoriMotor Rhythm Neurofeedback and Brain-Computer Interfaces? A preliminary case study“, Real-time functional Imaging and Neurofeedback conference (RTFIN’2017), 2017 – pdf

Open-Source Software:

Open access data:

Dreyer Pauline, Roc Aline, Rimbert Sébastien, Pillette Léa, & Lotte Fabien. (2023). A large EEG database with users’ profile information for motor imagery Brain-Computer Interface research (To use (2)) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7554429