Hemisfer projects

Contents

HEMISFER: Hybrid Eeg-MrI and Simultaneous neuro-feedback for brain Rehabilitation

Project goal

The goal of HEMISFER is to make full use of neurofeedback paradigm in the context of rehabilitation and psychiatric disorders. The major breakthrough will come from the use of a coupling model associating functional and metabolic information from Magnetic Resonance Imaging (fMRI) to Electro-encephalography (EEG) to “enhance” the neurofeedback protocol. We propose to combine advanced instrumental devices (Hybrid EEG and MRI platforms), with new man-machine interface paradigms (Brain computer interface and serious gaming) and new computational models (source separation, sparse representations and machine learning) to provide novel therapeutic and neuro-rehabilitation paradigms in some of the major neurological and psychiatric disorders of the developmental and the aging brain (stroke, attention-deficit disorder, language disorders, treatment-resistant mood disorders, …). This project will be conducted through a very complementary set of competences over the different teams involved in HEMISFER (Visages Inserm U1228, HYBRID and PANAMA Teams from Inria/Irisa, EA 4712 team from University of Rennes I and ATHENA team from Inria Sophia-Antipolis).

Project’s summary 

Sub-projects

Hybrid EEG and fMRI platform for multi-modal neurofeedback

A Hybrid EEG-fMRI system for Neurofeedback (NF) is available at the Neurinfo platform (CHU Pontchaillou, Rennes France). It was developed by Marsel Mano and used by Lorraine Perronet for two NF studies.

In the following, its architecture and implementation will be shortly described. More details can be found in Mano et al., 2017. Moreover, since an equivalent system that operates bimodal EEG-fMRI NF is not commercially available at present, the authors have patented the architecture of the platform.

The NF platform at Neurinfo consists of different units: a EEG subsystem and fMRI unit for data collection, a NF unit for control and synchronization and a display to communicate with the subject.

The EEG subsystem is an MR compatible solution from Brain Products. The EEG signals are acquired with a 64-channel cap, equipped with an electrocardiogram electrode. The electrodes are connected with two amplifiers that, through optic fibers, transfer the signal to the NF Control Unit outside of the MR bore.

The fMRI subsystem is a Nordic-Neurolab (NNL) solution with a Siemens 3T MR scanner. The MR imaging is performed on a Siemens MR scanner with a 64 channels head coil that allows secure installation of the EEG cap. The NNL hardware solution is used for visual stimulation and synchronization between the MR console and the Control Unit.

The Control unit is the core of the platform and it is responsible of the synchonized EEG and fMRI signal acquisition, preprocessing and NF computation. The NF control unit is also responsible for running the experimental protocol: this includes starting/stopping the experiment, controlling all other objects’ behavior throughout the experiment, synchronization between different units of data acquisition and neurofeedback presentation, and finally saving all the experiment data.

The NF unit controls also the visualization: the instructions and the computed NF  are visualized though different animations that are showed to the subject by means of a MR compatible display positioned at the back of the MR bore and a rear facing mirror fixed at the top of the head coil.

Currently the plaftform is efficiently being used to perform NF experiments on volunteers and patients, however  its modular structure allows for introduction of other processing modules in view of future clinical applications.

  • e-poster from ISMRM 2017 conference:

E-poster

Hybrid EEG and fMRI neurofeedback

  • EEG-fMRI neurofeedback of a motor imagery task (poster from OHBM 2016):

 OHBM poster

  • Sample movie of one motor imagery neurofeedback task combining EEG (“Y” axis) and fMRI (“X” axis) (works best with Google Chrome)

Coupling EEG and fMRI

  • Poster
  • An original approach for coupling EEG and fMRI information in neurofeedback is under development. This approach intends to learn relevant information from NF-fMRI in EEG only. The idea, is to improve the NF-EEG quality with complementary information, and limit the use of the fMRI, which is exhausting for the patient, and costly.

The consortium

The consortium involves the following co-PIs and teams:

  • CominLabs representatives : Christian BARILLOT (DR CNRS, INSERM Visages U746, IRISA CNRS 6074),  Anatole LECUYER (DR Inria, HYBRID Team, INRIA/IRISA CNRS 6074), Rémi GRIBONVAL (DR Inria, PANAMA Team, INRIA,) Isabelle BONAN (PU-PH, INSERM Visages U1228, Rehabilitation Dept. CHU Rennes)
  • External partners : Dominique DRAPIER (PU-PH, EA 4712, University of Rennes I, Psychiatric Hospital of Rennes), Maureen CLERC (DR Inria, INRIA ATHENA team, Sophia-Antipolis)

Project’s members:

  • Current : Jean-Marie Batail, Elise BannierChristian Barillot, Isabelle Bonan, Simon Butet, Maureen ClercDominique DrapierRemi GribonvalAnatole Lecuyer, Stéphanie Leplaideur, Pierre Maurel, Claire Cury, Mathis Fleury, Giulia Lioi
  • Past members: Lorraine Perronnet, Jussi Lindgren,  Marsel ManoSaman Noorzadeh, Thomas Oberlin, Nicolas Raillard

Related Links

Supporting Institutions

Labex Cominlabs
Fondation pour la Recherche Médicale

Contact

Contact: Pierre.Maurel@irisa.fr

Publications from the project

PhD’s

Lorraine Perronnet. Combining electroencephalography and functional magnetic resonance imaging for neurofeedback. Medical Imaging. Université Rennes 1, 2017.

English. 〈NNT : 2017REN1S043〉. 〈tel-01661583〉

Abstract: NF is the process of feeding back real-time information to an individual about his/her ongoing brain activity, so that he/she can train to self-regulate neural substrates of specific behavioral functions. NF has been extensively studied for brain rehabilitation of patients with psychiatric and neurological disorders. However its effective deployment in the clinical armamentarium is being held back by the lack of evidence about its efficacy. One of the possible reason for the debated efficacy of current approaches could be the inherent limitations of single imaging modalities. Indeed, most NF approaches rely on the use of a single modality, EEG and fMRI being the two most widely used. While EEG is inexpensive and benefits from a high temporal resolution (millisecond), its spatial resolution (centimeters) is limited by volume conduction of the head and the number of electrodes. Also source localization from EEG is inaccurate because of the ill-posed inverse problem. In a complementary way, fMRI gives access to the self-regulation of specific brain regions at high spatial resolution (millimeter) but has low temporal resolution (second). Combined EEG-fMRI has proven much valuable for the study of human brain function, however it has rarely been exploited for NF purpose. In the context of NF, combining EEG and fMRI enables cross-modal paradigm evaluation and validation. But more interestingly it opens up avenues for the development of new NF approaches that would mix both modalities, either at the calibration phase or to provide a bimodal NF signal. Combined EEG-fMRI poses numerous challenges with regard to basic physiology, study design, data quality, analysis/integration and interpretation. These challenges are even greater if EEG and fMRI are both to be used simultaneously for online NF computation, because of the real-time constraint and the difficulty to find a task design compatible with EEG and fMRI’ diverging natures. The theoretical part of this PhD dissertation aims at identifying methodological aspects that differ between EEG-NF and fMRI-NF and at examining the motivations and strategies for combining EEG and fMRI for NF purpose. Among these combination strategies, we choose to focus on bimodal EEG-fMRI-NF as it seems to be one of the most promising approach and is mostly unexplored. The feasibility of this approach was recently demonstrated and opened an entire new field of investigation. First and foremost, we would like to address the following questions: what is the added value of bimodal NF over unimodal NF; are there any specific mechanisms involved when learning to control two NF signals simultaneously; how to integrate EEG and fMRI to derive a single feedback ? The experimental part of this PhD dissertation therefore focuses on the development and evaluation of methods for bimodal EEG-fMRI-NF. In order to conduct bimodal NF experiments, we start by building up a real-time EEG-fMRI platform. Then in a first study, we compare for the first time bimodal EEG-fMRI-NF with unimodal EEG-NF and fMRI-NF. Eventually, in a second study, we introduce and evaluate two integrated feedback strategies for EEG-fMRI-NF.https://tel.archives-ouvertes.fr/tel-01661583/file/PERRONNET_Lorraine.pdf BibTex

Other publications

Publications HAL de Christian,Barillot de Remi,Gribonval;Anatole,Lecuyer

2020

Journal articles

ref_biblio
Giulia Lioi, Claire Cury, Lorraine Perronnet, Marsel Mano, Elise Bannier, et al.. Simultaneous EEG-fMRI during a neurofeedback task, a brain imaging dataset for multimodal data integration. Scientific Data , 2020, 7 (1), pp.1-15. ⟨10.1038/s41597-020-0498-3⟩. ⟨hal-02865965⟩
Accès au texte intégral et bibtex
https://inria.hal.science/hal-02865965/file/s41597-020-0498-3.pdf BibTex
ref_biblio
Salomé Le Franc, Mathis Fleury, Mélanie Cogne, Simon Butet, Christian Barillot, et al.. Influence of virtual reality visual feedback on the illusion of movement induced by tendon vibration of wrist in healthy participants. PLoS ONE, 2020, 15 (11), pp.1-16. ⟨10.1371/journal.pone.0242416⟩. ⟨hal-03097386⟩
Accès au bibtex
BibTex
ref_biblio
Mathis Fleury, Giulia Lioi, Christian Barillot, Anatole Lécuyer. A Survey on the Use of Haptic Feedback for Brain-Computer Interfaces and Neurofeedback. Frontiers in Neuroscience, 2020, 1, ⟨10.3389/fnins.2020.00528⟩. ⟨hal-02459828v2⟩
resume
Neurofeedback (NF) and brain-computer interface (BCI) applications rely on the registration and real-time feedback of individual patterns of brain activity with the aim of achieving self-regulation of specific neural substrates or control of external devices. These approaches have historically employed visual stimuli. However, in some cases vision is unsuitable or inadequately engaging. Other sensory modalities, such as auditory or haptic feedback have been explored, and multisensory stimulation is expected to improve the quality of the interaction loop. Moreover, for motor imagery tasks, closing the sensorimotor loop through haptic feedback may be relevant for motor rehabilitation applications, as it can promote plasticity mechanisms. This survey reviews the various haptic technologies and describes their application to BCIs and NF. We identify major trends in the use of haptic interfaces for BCI and NF systems and discuss crucial aspects that could motivate further studies.
Accès au texte intégral et bibtex
https://hal.science/hal-02459828/file/fnins-14-00528.pdf BibTex
ref_biblio
Giulia Lioi, Simon Butet, Mathis Fleury, Elise Bannier, Anatole Lécuyer, et al.. A Multi-Target Motor Imagery Training Using Bimodal EEG-fMRI Neurofeedback: A Pilot Study in Chronic Stroke Patients. Frontiers in Human Neuroscience, 2020, 14, pp.1-13. ⟨10.3389/fnhum.2020.00037⟩. ⟨hal-02491848⟩
Accès au texte intégral et bibtex
https://inria.hal.science/hal-02491848/file/Lioi_2020_Frontiers_In_Neuroscience.pdf BibTex
ref_biblio
Claire Cury, Pierre Maurel, Rémi Gribonval, Christian Barillot. A sparse EEG-informed fMRI model for hybrid EEG-fMRI neurofeedback prediction. Frontiers in Neuroscience, 2020, 13, ⟨10.3389/fnins.2019.01451⟩. ⟨inserm-02090676v3⟩
resume
Measures of brain activity through functional magnetic resonance imaging (fMRI) or Electroencephalography (EEG), two complementary modalities, are ground solutions in the context of neuro-feedback (NF) mechanisms for brain-rehabilitation protocols. Though NF-EEG (real-time neurofeedback scores computed from EEG) have been explored for a very long time, NF-fMRI (real-time neurofeedback scores computed from fMRI) appeared more recently and provides more robust results and more specific brain training. Using simultaneously fMRI and EEG for multimodal neurofeedback sessions (NF-EEG-fMRI, real-time neurofeedback scores computed from fMRI and EEG) is very promising to devise brain rehabilitation protocols. However using fMRI is costly, exhausting and time consuming, and cannot be repeated too many times for the same subject. The original contribution of this paper concerns the prediction of multimodal NF scores from EEG recordings only, using a training phase where both EEG and fMRI synchronous signals, and therefore neurofeedback scores, are available. We propose a sparse regression model able to exploit EEG only to predict NF-fMRI or NF-EEG-fMRI in motor imagery tasks. We compare different NF-predictors steming from the proposed model. We show that one of the proposed NF-predictors significanlty improves over what EEG can provide alone (without the learning phase), and correlates at 0.74 in median with the ground-truth.
Accès au texte intégral et bibtex
https://inserm.hal.science/inserm-02090676/file/NF_EEG_fMRI_prediction_resub4BioRxiv.pdf BibTex

Conference papers

ref_biblio
Claire Cury, Giulia Lioi, Lorraine Perronnet, Anatole Lécuyer, Pierre Maurel, et al.. Impact of 1D and 2D visualisation on EEG-fMRI neurofeedback training during a motor imagery task.. ISBI 2020 – IEEE International Symposium on Biomedical Imaging, Apr 2020, Iowa City, United States. pp.1-4. ⟨inserm-02489459v2⟩
Accès au texte intégral et bibtex
https://inserm.hal.science/inserm-02489459/file/ISBI2020_resub_Final_2.pdf BibTex

Poster communications

ref_biblio
Giulia Lioi, Adolfo Veliz, Julie Coloigner, Quentin Duché, Simon Butet, et al.. The effect of neurofeedback training on effective connectivity assessed with dynamic causal modeling in stroke patients – a pilot study. WFNR 2020 – World Federation For NeuroRehabilitation, Oct 2020, Lyon, France. ⟨hal-03354319⟩
resume
Question A growing body of research suggest that aberrant interactions among cortical regions are crucially linked to motor rehabilitation after stroke (Guggisberg et al. 2019). Recent studies have shown the potential of Neurofeedback (NF) for stroke rehabilitation, however the effect of NF training upon functional interactions between motor areas is poorly understood. In a previous work (Lioi et al., 2020), we have tested bimodal EEG-fMRI NF for stroke rehabilitation: here we investigate the effect of NF training on motor networks. Methods Four right-handed chronic stroke patients (54 – 76 years, 2 females) with left hemiparesis took part to the study. The experimental protocol included 2 bimodal EEG-fMRI and 3 unimodal EEG NF sessions within a week. During each session, patients underwent 3 training runs alternating blocks of rest and motor imagery of the affected upper limb with NF. NF was displayed as a ball moving on a gauge proportionally to EEG and fMRI activities from regions of interest (ROIs) identified in the ipsilesional motor cortex. Representative ROIs time-series were extracted by selecting voxels that exceeded the NF contrast statistical threshold (p=0.05) within bilateral premotor, supplementary and primary motor cortices (PMC, SMA, M1). We used dynamic causal modeling (Zeidman et al., 2019) to assess causal influences between motor areas. The models were defined apriori on the base previous results (Grefkes et al., 2010) and included, respectively, 5 and 6 ROIs (Figure 1). We tested the effect of NF training on connection strengths with a Parametric Empirical Bayes second level analysis (Friston et al., 2016). Results In the model best explaining the difference between the first and the last training session (Figure 2) an increase in connectivity between ipsilesional motor ROIs was observed, which did not necessarily correspond to an increase in contralesional connectivity (Figure 2 B.). A general decrease in the strength of connection between hemispheres for PMC and M1 was also observed. This is of particular interest as an increase in ipsilesional connectivity between premotor and motor areas and a decrease in pathological transcallosal connections have been associated with improved motor performances in stroke patients (Grefkes & Fink, 2011) Conclusion These preliminary results on a small sample of stroke patients suggest that NF training of ipsilesional motor areas is associated to a reorganization of motor effective connectivity.
Accès au bibtex
BibTex
ref_biblio
Simon Butet, Quentin Duché, Giulia Lioi, Mathis Fleury, Emilie Leveque-Le Bars, et al.. Feasibility of a multi-session EEG-fMRI Neurofeedback training – preliminary results from a randomized controlled study in chronic stroke patients. WFNR 2020 – World Federation for NeuroRehabilitation, Oct 2020, Lyon, France. ⟨hal-03354325⟩
Accès au bibtex
BibTex

Preprints, Working Papers, …

ref_biblio
Lorraine Perronnet, Anatole Lécuyer, Marsel Mano, Mathis Fleury, Giulia Lioi, et al.. Learning 2-in-1: Towards Integrated EEG-fMRI-Neurofeedback. 2020. ⟨hal-02522245⟩
resume
Neurofeedback (NF) allows to exert self-regulation over specific aspects of one’s own brain activity by returning information extracted in real-time from brain activity measures. These measures are usually acquired from a single modality, most commonly electroencephalography (EEG) or functional magnetic resonance imaging (fMRI). EEG-fMRI-neurofeedback (EEG-fMRI-NF) is a new approach that consists in providing a NF based simultaneously on EEG and fMRI signals. By exploiting the complementarity of these two modalities, EEG-fMRI-NF opens a new spectrum of possibilities for defining bimodal NF targets that could be more robust, flexible and effective than unimodal ones. Since EEG-fMRI-NF allows for a richer amount of information to be fed back, the question arises of how to represent the EEG and fMRI features simultaneously in order to allow the subject to achieve better self-regulation. In this work, we propose to represent EEG and fMRI features in a single bimodal feedback (integrated feedback). We introduce two integrated feedback strategies for EEG-fMRI-NF and compare their early effects on a motor imagery task with a between-group design. The BiDim group (n=10) was shown a two-dimensional (2D) feedback in which each dimension depicted the information from one modality. The UniDim group (n=10) was shown a one-dimensional (1D) feedback that integrated both types of information even further by merging them into one. Online fMRI activations were significantly higher in the UniDim group than in the BiDim group, which suggests that the 1D feedback is easier to control than the 2D feedback. However subjects from the BiDim group produced more specific BOLD activations with a notably stronger activation in the right superior parietal lobe (BiDim > UniDim, p < 0.001, uncorrected). These results suggest that the 2D feedback encourages subjects to explore their strategies to recruit more specific brain patterns. To summarize, our study shows that 1D and 2D integrated feedbacks are effective but also appear to be complementary and could therefore be used in a bimodal NF training program. Altogether, our study paves the way to novel integrated feedback strategies for the development of flexible and effective bimodal NF paradigms that fully exploits bimodal information and are adapted to clinical applications.
Accès au texte intégral et bibtex
https://inria.hal.science/hal-02522245/file/Learning_2_in_1__towards_integrated_EEG_fMRI_neurofeedback_subBioRxiv.pdf BibTex

2019

Conference papers

ref_biblio
Claire Cury, Pierre Maurel, Giulia Lioi, Rémi Gribonval, Christian Barillot. Learning bi-modal EEG-fMRI neurofeedback to improve neurofeedback in EEG only. Real-Time Functional Imaging and Neurofeedback, Dec 2019, Maastricht, Netherlands. pp.1-2, ⟨10.1101/599589⟩. ⟨inserm-02368720⟩
resume
Introduction In neurofeedback (NF), a new kind of data are available: electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) acquired simultaneously during bi-modal EEG-fMRI neurofeedback. These two complementary techniques have only recently been integrated in the context of NF for brain rehabilitation protocols. Bi-modal NF (NF-EEG-fMRI) combines information coming from two modalities sensitive to different aspect of brain activity, therefore providing a higher NF quality [1]. However, the use of the MRI scanner is cumbersome and exhausting for patients. We present, a novel methodological development, able to reduce the use of fMRI while providing to subjects NF-EEG sessions of quality comparable to the bi-modal NF sessions [2]. We propose an original alternative to the ill-posed problem of source reconstruction. We designed a non-linear model considering different frequency bands, electrodes and temporal delays, with a structured sparse regularisation. Results show that our model is able to significantly improve the quality of NF sessions over what EEG could provide alone. We tested our method on 17 subjects that performed three NF-EEG-fMRI sessions each.
Accès au texte intégral et bibtex
https://inserm.hal.science/inserm-02368720/file/Abstract_rtFIN2019_2.pdf BibTex
ref_biblio
Mathis Fleury, Giulia Lioi, Christian Barillot, Anatole Lécuyer. The use of haptic feedback in Brain-Computer Interfaces and Neurofeedback. rtFIN 2019 – Real Time Functional Imaging and Neurofeedback, Dec 2019, Maastricht, Netherlands. ⟨hal-02387400⟩
Accès au bibtex
BibTex
ref_biblio
Giulia Lioi, Simon Butet, Mathis Fleury, Claire Cury, Elise Bannier, et al.. Bimodal EEG-fMRI Neurofeedback for upper motor limb rehabilitation: a pilot study on chronic patients. rtFIN 2019 – Real Time Functional Imaging and Neurofeedback, Dec 2019, Maastricht, Netherlands. pp.1-2. ⟨hal-02383532v3⟩
Accès au texte intégral et bibtex
https://inria.hal.science/hal-02383532/file/GiuliaLioi_rtFIN2019_v3%20%281%29.pdf BibTex
ref_biblio
Salomé Le Franc, Mathis Fleury, Mélanie Cogné, Simon Butet, Christian Barillot, et al.. Influence of visual feedback on the illusion of movement induced by tendon vibration of wrist in healthy subjects. SOFMER 2019 – 34ème congrès de la Société Français de Médecine Physique et de Réadaptation, Oct 2019, Bordeaux, France. ⟨hal-02415992⟩
resume
Illusion of movement induced by tendon vibration is a powerful approach to improve cortical excitability and can be useful for rehabilitation of neurological impairments. The aim of our study was to investigate whether a visual feedback of a moving hand congruent to the proprioceptive illusion induced by a tendon vibration of the wrist could increase the illusion of movement.
Accès au texte intégral et bibtex
https://inria.hal.science/hal-02415992/file/Abstract.pdf BibTex
ref_biblio
Giulia Lioi, Simon Butet, Mathis Fleury, Anatole Lécuyer, Isabelle Bonan, et al.. Efficacy of EEG-fMRI Neurofeedback in stroke in relation to the DTI structural damage: a pilot study. OHBM 2019 – 25th Annual Meeting of the Organization for Human Brain Mapping, Jun 2019, Rome, Italy. pp.1-4. ⟨hal-02265495⟩
Accès au texte intégral et bibtex
https://inria.hal.science/hal-02265495/file/OHBM%20abstract%201.pdf BibTex
ref_biblio
Claire Cury, Pierre Maurel, Rémi Gribonval, Christian Barillot. Can we learn from coupling EEG-fMRI to enhance neuro-feedback in EEG only?. OHBM 2019 – Annual Meeting Organization for Human Brain Mapping, Jun 2019, Rome, Italy. pp.1. ⟨inserm-02074623⟩
Accès au texte intégral et bibtex
https://inserm.hal.science/inserm-02074623/file/cury-claire-empenn.pdf BibTex
ref_biblio
Simon Butet, Giulia Lioi, Mathis Fleury, Anatole Lécuyer, Christian Barillot, et al.. A multi-target motor imagery training using EEG-fMRI Neurofeedback: an exploratory study on stroke. OHBM 2019- Orgaization for Human Brain Mapping, Jun 2019, Rome, Italy. pp.1-4. ⟨hal-02265496⟩
Accès au texte intégral et bibtex
https://inria.hal.science/hal-02265496/file/OHBM%20abstract%202.pdf BibTex
ref_biblio
Mathis Fleury, Giulia Lioi, Christian Barillot, Anatole Lécuyer. The use of haptic feedback in Brain-Computer Interfaces and Neurofeedback. CORTICO 2019 – Journée Jeunes Chercheurs en Interfaces Cerveau-Ordinateur et Neurofeedback, Mar 2019, Lille, France. ⟨hal-02387408⟩
Accès au texte intégral et bibtex
https://hal.science/hal-02387408/file/AbstractTemplate_rtFIN2019.pdf BibTex

2018

Poster communications

ref_biblio
Giulia Lioi, Mathis Fleury, Simon Butet, Anatole Lécuyer, Christian Barillot, et al.. Bimodal EEG-fMRI Neurofeedback for stroke rehabilitation. ISPRM 2018 -International Society of Physical and Rehabilitation Medicine, Jul 2018, Paris, France. ⟨inserm-01932954⟩
Accès au texte intégral et bibtex
https://inserm.hal.science/inserm-01932954/file/Poster_GiuliaLioi_ISPRM.pdf BibTex

2017

Journal articles

ref_biblio
Lorraine Perronnet, Anatole Lécuyer, Marsel Mano, Elise Bannier, Fabien Lotte, et al.. Unimodal Versus Bimodal EEG-fMRI Neurofeedback of a Motor Imagery Task. Frontiers in Human Neuroscience, 2017, 11, ⟨10.3389/fnhum.2017.00193⟩. ⟨hal-01519755⟩
resume
Neurofeedback is a promising tool for brain rehabilitation and peak performance training. Neurofeedback approaches usually rely on a single brain imaging modality such as EEG or fMRI. Combining these modalities for neurofeedback training could allow to provide richer information to the subject and could thus enable him/her to achieve faster and more specific self-regulation. Yet unimodal and multimodal neurofeedback have never been compared before. In the present work, we introduce a simultaneous EEG-fMRI experimental protocol in which participants performed a motor-imagery task in unimodal and bimodal NF conditions. With this protocol we were able to compare for the first time the effects of unimodal EEG-neurofeedback and fMRI-neurofeedback versus bimodal EEG-fMRI-neurofeedback by looking both at EEG and fMRI activations. We also propose a new feedback metaphor for bimodal EEG-fMRI-neurofeedback that integrates both EEG and fMRI signal in a single bi-dimensional feedback (a ball moving in 2D). Such a feedback is intended to relieve the cognitive load of the subject by presenting the bimodal neurofeedback task as a single regulation task instead of two. Additionally, this integrated feedback metaphor gives flexibility on defining a bimodal neurofeedback target. Participants were able to regulate activity in their motor regions in all NF conditions. Moreover, motor activations as revealed by offline fMRI analysis were stronger during EEG-fMRI-neurofeedback than during EEG-neurofeedback. This result suggests that EEG-fMRI-neurofeedback could be more specific or more engaging than EEG-neurofeedback. Our results also suggest that during EEG-fMRI-neurofeedback, participants tended to regulate more the modality that was harder to control. Taken together our results shed first light on the specific mechanisms of bimodal EEG-fMRI-neurofeedback and on its added-value as compared to unimodal EEG-neurofeedback and fMRI-neurofeedback.
Accès au texte intégral et bibtex
https://inria.hal.science/hal-01519755/file/fnhum-11-00193%20%281%29.pdf BibTex
ref_biblio
Marsel Mano, Anatole Lécuyer, Elise Bannier, Lorraine Perronnet, Saman Noorzadeh, et al.. How to Build a Hybrid Neurofeedback Platform Combining EEG and fMRI.. Frontiers in Neuroscience, 2017, 11, pp.140. ⟨10.3389/fnins.2017.00140⟩. ⟨inserm-01576500⟩
resume
Multimodal neurofeedback estimates brain activity using information acquired with more than one neurosignal measurement technology. In this paper we describe how to set up and use a hybrid platform based on simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI), then we illustrate how to use it for conducting bimodal neurofeedback experiments. The paper is intended for those willing to build a multimodal neurofeedback system, to guide them through the different steps of the design, setup, and experimental applications, and help them choose a suitable hardware and software configuration. Furthermore, it reports practical information from bimodal neurofeedback experiments conducted in our lab. The platform presented here has a modular parallel processing architecture that promotes real-time signal processing performance and simple future addition and/or replacement of processing modules. Various unimodal and bimodal neurofeedback experiments conducted in our lab showed high performance and accuracy. Currently, the platform is able to provide neurofeedback based on electroencephalography and functional magnetic resonance imaging, but the architecture and the working principles described here are valid for any other combination of two or more real-time brain activity measurement technologies.
Accès au texte intégral et bibtex
https://inserm.hal.science/inserm-01576500/file/Mano-Front%20Neurosci-2017.pdf BibTex

Conference papers

ref_biblio
Saman Noorzadeh, Pierre Maurel, Thomas Oberlin, Rémi Gribonval, Christian Barillot. Multi-modal EEG and fMRI Source Estimation Using Sparse Constraints. MICCAI 2017 – 20th International Conference on Medical Image Computing and Computer Assisted Intervention, Sep 2017, Quebec, Canada. ⟨10.1007/978-3-319-66182-7_51⟩. ⟨hal-01586495⟩
resume
In this paper a multi-modal approach is presented and validated on real data to estimate the brain neuronal sources based on EEG and fMRI. Combining these two modalities can lead to source estimations with high spatio-temporal resolution. The joint method is based on the idea of linear model already presented in the literature where each of the data modalities are first modeled linearly based on the sources. Afterwards, they are integrated in a joint framework which also considers the sparsity of sources. The sources are then estimated with the proximal algorithm. The results are validated on real data and show the efficiency of the joint model compared to the uni-modal ones. We also provide a calibration solution for the system and demonstrate the effect of the parameter values for uni-and multi-modal estimations on 8 subjects.
Accès au texte intégral et bibtex
https://inria.hal.science/hal-01586495/file/MICCAI17-583_final.pdf BibTex
ref_biblio
Marsel Mano, Elise Bannier, Lorraine Perronnet, Anatole Lécuyer, Christian Barillot. Hybrid EEG and fMRI platform for multi-modal neurofeedback. International Society of Magnetic Resonance in Medicine, ISMRM, Apr 2017, Honolulu, United States. pp.4550. ⟨inserm-01577442⟩
resume
Neurofeedback (NFB) relies on neurosignals for the estimation of brain activity. There exist a wide variety of NFB applications that use one type of neurosignals like fMRI or electroencephalography (EEG). Recently, the combination of two or more neurosignals has been receiving a lot of attention in the research community, but still very few multi-modal NFB applications exist. This is primarily because of the lack of commercial multi-modal NFB systems and the associated technical difficulties in building them. Here we are going to describe a bi-modal EEG and fMRI NFB platform that we have build in our lab. Our platform is designed to maximize modularity and parallel processing in order to be able to provide real-time NFB with high level of synchronization and minimal delays. We have successfully used our platform to conduct over 100 uni-modal and bi-modal NFB experiments with more than 30 healthy subjects.
Accès au texte intégral et bibtex
https://inserm.hal.science/inserm-01577442/file/Poster%20Hemisfer%20ISMRM%204550.2017.pdf BibTex

2016

Book sections

ref_biblio
Lorraine Perronnet, Anatole Lécuyer, Fabien Lotte, Maureen Clerc, Christian Barillot. Brain training with neurofeedback. Brain-Computer Interfaces 1, Wiley-ISTE, 2016. ⟨hal-01413424⟩
Accès au bibtex
BibTex
ref_biblio
Lorraine Perronnet, Anatole Lécuyer, Fabien Lotte, Maureen Clerc, Christian Barillot. Entraîner son cerveau avec le neurofeedback. Maureen Clerc; Laurent Bougrain; Fabien Lotte. Les interfaces cerveau-ordinateur 1, ISTE editions, pp.277-292, 2016, 978-1-78406-147-0. ⟨hal-01413408⟩
Accès au bibtex
BibTex

Patents

ref_biblio
Marsel Mano, Lorraine Perronnet, Anatole Lécuyer, Christian Barillot. Hybrid Eeg-MrI and Simultaneous neuro-feedback for brain Rehabilitation. France, Patent n° : PCT/EP2016/1652279. 2016. ⟨hal-01576711⟩
Accès au bibtex
BibTex

Poster communications

ref_biblio
Lorraine Perronnet, Anatole Lécuyer, Marsel Mano, Elise Bannier, Fabien Lotte, et al.. EEG-fMRI neurofeedback of a motor imagery task. Organization for Human Brain Mapping, Jun 2016, Genève, Switzerland. ⟨hal-01426182⟩
resume
EEG-fMRI-neurofeedback(NF) has been introduced for the first time by Zotev et al [1]. The authors hypothesized that bimodal EEG-fMRI-NF could be more efficient than unimodal EEG-NF or fMRI-NF performed alone. A recent study identified the fMRI signature of motor imagery during EEG-NF [3]. However to our knowledge EEG-fMRI-NF, EEG-NF and fMRI-NF have never been compared before. In the present work, we propose an EEG-fMRI-NF protocol of a motor imagery (MI) task and compare the cross-modal effects of EEG-NF, fMRI-NF and EEG-fMRI-NF. We hypothesized that: • EEG activations : EEG-NF ≥ EEG-fMRI-NF > fMRI-NF • fMRI activations : fMRI-NF ≥ EEG-fMRI-NF > EEG-NF As compared to [1] in which EEG and fMRI were represented with two separate gauges, our feedback metaphor integrates both EEG and fMRI signal in a single bi-dimensional feedback (a ball moving in 2D) in order for the subject to more easily perceive the NF training as one regulation task instead of two.
Accès au texte intégral et bibtex
https://inria.hal.science/hal-01426182/file/OHBM_poster_4133_lp.pdf BibTex
ref_biblio
Marsel Mano, Elise Bannier, Lorraine Perronnet, Anatole Lécuyer, Christian Barillot. Design of an Experimental Platform for Hybrid EEG-fMRI Neurofeedback Studies. 22nd Annual Meeting of the Organization for Human Brain Mapping (OHBM 2016), Jun 2016, Genève, Switzerland. . ⟨hal-01426072⟩
resume
Neurofeedback (NF) can be defined as the self-regulated change of a particular brain activity that is reflected in the change of a neural signal or a combination of neural signals such as EEG, fMRI, MEG, etc. There exist a variety of unimodal (i.e. EEG or fMRI) NF researches, but very few with multimodal NF applications. This is primarily because of the associated technical burdens. The purpose of this abstract is to give a technical description of the hybrid EEG-fMRI system that we have developed for our NF experiments as part of the project Hemisfer, including the hardware/software components and their roles.
Accès au texte intégral et bibtex
https://hal.science/hal-01426072/file/Hybrid%20EEG%C2%AD-fMRI%20neurofeedback%20of%20a%20motor%C2%ADimagery%20task.pdf BibTex

2015

Conference papers

ref_biblio
Thomas Oberlin, Christian Barillot, Rémi Gribonval, Pierre Maurel. Symmetrical EEG-FMRI Imaging by Sparse Regularization. 23rd European Signal Processing Conference (EUSIPCO 2015), Aug 2015, Nice, France. pp.1-5, ⟨10.1109/EUSIPCO.2015.7362708⟩. ⟨hal-01170889⟩
resume
This work considers the problem of brain imaging using simultaneously recorded electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). To this end, we introduce a linear coupling model that links the electrical EEG signal to the hemodynamic response from the blood-oxygen level dependent (BOLD) signal. Both modalities are then symmetrically integrated, to achieve a high resolution in time and space while allowing some robustness against potential decoupling of the BOLD effect. The novelty of the approach consists in expressing the joint imaging problem as a linear inverse problem, which is addressed using sparse regularization. We consider several sparsity-enforcing penalties, which naturally reflect the fact that only few areas of the brain are activated at a certain time, and allow for a fast optimization through proximal algorithms. The significance of the method and the effectiveness of the algorithms are demonstrated through numerical investigations on a spherical head model.
Accès au texte intégral et bibtex
https://hal.science/hal-01170889/file/oberlin_14011.pdf BibTex
ref_biblio
Elise Bannier, Marsel Mano, Stoermer Robert, Isabelle Corouge, Lorraine Perronnet, et al.. On the feasibility and specificity of simultaneous EEG and ASL MRI at 3T. Proceedings of ISMRM, May 2015, Toronto, Canada. ⟨inserm-01113276⟩
Accès au bibtex
BibTex
ref_biblio
Elise Bannier, Marsel Mano, Stoermer Robert, Isabelle Corouge, Lorraine Perronnet, et al.. Faisabilité et spécificités de l’ASL-EEG simultané à 3T. SFRMBM, Mar 2015, Grenoble, France. ⟨inserm-01113279⟩
Accès au bibtex
BibTex

Comments are closed.