A BCI relies on direct measures of brain activity, provides feedback to the user, is processed in real time and relies on intentional control. BCIs measure central nervous system (CNS) activity, converting it into artificial output in order to replace, restore, enhance, supplement or improve natural CNS output and changes the ongoing interactions between the CNS and the external and internal environment. BCI systems have applications in neurorehabilitation, assistive device technology, cognitive enhancement and increasing speed of human to computer communication. BCIs are used for communication or control of external prosthetic devices in people living with conditions such as spinal cord injury, Amyotrophic Lateral Sclerosis (ALS), Locked-in Syndrome (LIS) and Multiple Sclerosis (MS). BCIs can be used for functional electrical stimulation of muscles in a paralyzed person or of peripheral nerves to restore bladder function. BCIs can monitor brain activity during prolonged demanding tasks and detect lapses of attention and alert the person. BCIs are used in research to study CNS function.
Brain activity signal acquisition
Invasive electrodes entail risk due to surgery and show gradual degradation of recorded signals.
- Intracranial electroencephalography, also known as electrocorticography (ECoG)
- Single-neuron action potentials
- multi-unit activity (MUA)
- local field potentials (LFPs)
- Electroencephalography (EEG): Electrodes on the scalp measure electrical brain activity due to flow of electric currents during synaptic excitations of neural dendrites
- Functional magnetic resonance imaging (fMRI)
- Magnetoencephalography (MEG)
- Near-infrared spectroscopy (NIRS)
Implantable neural interface materials and components
- Neural dust: Electronic sensors placed into the cortex that are interrogated remotely using ultrasound and also powered by ultrasound
- Chronically implanted neural electrodes
- Neural electrodes both stimulate and measure nerve signals
- Microelectrodes, microelectrode arrays (MEA)
- Penetrating electrodes and non-penetrating electrodes
- Microelectromechanical systems (MEMS)
- Sharp glass electrodes and patch-clamp electrodes
- Vertical nanowire electrode arrays (VNEAs)
Intracortial brain computer interfaces (iBCIs)
iBCIs use microscale probes with 96 fine tipped microelectrodes in a 4x4 mm array that is inserted into the cortex. Sensors receive all-or-none output of single neurons, known as the action potential, as well as summed voltage fluctuations from small and large numbers of neurons, called field potentials.
- wireless implants
- transcutaneous systems
Motor imagery is the imagining of a movement rather than actual execution of movement. Motor imagery activates areas of the brain that are responsible for generating actual movement. Motor imagery techniques do not use external signals and are called endogenous BCIs. Steady-state visual evoked potential (SSVEP) and P300 BCIs are exogenous BCI paradigms that use external stimuli such as flickering LEDs or auditory beeps to evoke discriminative brain patterns.
Sensorimotor rhythms (SMR)
Within motor imagery, sensorimotor rhythms (SMR) have been used by patients with tetraplegia, spinal cord injury and Amyotrophic Lateral Sclerosis (ALS). In SMR imagination of kinesthetic movements of large body parts such as hands, feet and tongue can result in modulation of brain activity, such as event-related desynchronization (ERD) in mu (8-12 Hz) and beta rhythms (18-26 Hz) and relaxation results in event-related synchronization (ERS). EEG electrodes above the sensorimotor cortex record ERD and ERS modulations and can be used to control prosthetic devices and move the cursor on a computer. Training using the SMR paradigm can be up to several weeks. SMR can distinguish motor activities corresponding to large body parts but the decoded motor information does not include magnitude or direction kinematics parameters such as position, velocity or acceleration.
Imagined body kinematics (IBK)
Imagined body kinematics (IBK), sometimes referred to as natural imaginary movement, is a paradigm independent from SMR because of the different training and analysis protocols. IBK originated form invasive BCI technology but is extracted from low-frequency SMR signals. Subjects are asked to imagine the continuous movement of only one body part in multi-dimensional space and signals are decoded in the time domain. IBK requires less training time compared with SMR.
P300-based BCIs are communication tools where thought is used to input texts or commands, control smart home controls and for brain painting. The P300 is the one of the most studied event-related potentials (ERP). EEG signals of a specific event type are averaged to derive an ERP. In ERP P300 is a positive deflection with a time delay of about 250 ms to 750 ms after the onset of a rare and unexpected stimulus. P300 speller devices use a matrix of letters, numbers and symbols that flash in sequence and the subject is required to focus their attention on the intended character, which is determined by the speller based on its row and column. Visual P300 BCI can be used by most subjects easily with high accuracy but disadvantages include fatigue from the high level of attention and visual focus and inability of use for people with visual impairment.
Steady State Visual Evoked Potential (SSVEP)
SSVEP is also referred to as photic driving since responses are generated in the visual cortex. SSVEP requires highly accurate eye control. In response to flickering stimuli subjects shift their gaze and their attention. An EEG pattern is formed that is consistent with the flickering frequency of the stimulus on the central retina. Multiple flickering targets with distinct flickering frequencies are presented to the user and the intended target is determined by matching the pattern of EEG activity to the command associated with the particular frequency.
Hybrid BCI platforms
Hybrid BCI platforms combine EEG with one or more physiological measures such as heart rate by ECG, eye movement with electrooculography (EOG) or a hemodynamic signal recorded by functional near-infrared spectroscopy (fNIRS).
Covert and overt attention
A covert attention paradigm has a subject focus on a centrally located fixation point while following another point such as a cursor without overt eye movement whereas overt attention requires the subject to use overt eye movements. In both cases EEG signals are typically recorded from the posterior cortex.
Discrete movement intention
For discrete movement intention EEG signals are collected before the onset of movement even in subjects that are not able to physically execute an actual movement.
Auditory exogenous stimulation can be used to evoke auditory steady-state responses (ASSR). Rapid auditory stimuli has been shown to record ASSR maximum amplitude from the vertex of the scalp.
Under the somatosensory paradigm vibrotactile sensors are placed on the body and stimulated at different frequencies producing EEG signals. Somatosensory-based BCI systems have been used to assist patients with locked-in syndrome.
Preprocessing is needed to enhance signal-to-noise ratio and remove artifacts such as the part of EEG signals that come from muscular activity of the head and eye that are unrelated to the brain.
- Covariance Matrix Adaptation Evolution Strategy (CMA-ES)
After brain signals are preprocessed they are fed into one or more feature extraction algorithms which extract features in the time domain and frequency domain that encode messages or commands.
- Threshold crossings
- Multinunit activity
- Local field potentials
- Amplitude measures
- Band power
- Hjorth parameters
- Autoregressive models
- Spatial filters
- Short Term Fourier Transform (STFT)
- Auto Regressive Model (AR)
- Wavelet Transform (WT)
- Common Spatial Pattern (CSP)
Classification/ feature translation
In BCIs classification is the translation of features provided by the feature extractor to a category of brain patterns using classification algorithms.
- Linear Discriminant Analysis (LDA)
- Support Vector Machine (SVM)
- Non-linear methods such as neural networks
For training BCI users, virtual reality is a method of providing feedback. In gaming for the purposes of entertainment, virtual reality headsets used for gaming could include EEG sensors to potentially allow games to respond differently to the user depending on their mood or how they respond to particular elements of the game.
Optogenetics is a pre-clinical neuroscience research tool that has been suggested as an approach for neuroprosthetics and treatment of brain disorders. Optogenetics can be used for real-time control of genetically engineered brain neurons. Photosensitive proteins open and close membrane channels via light-inducible activation or suppression.
Optogenetics previously utilized optical fibers inserted into the skull, but wireless optogenetics technologies are being developed. . Optogenetic approaches could potentially use red or near infrared light, which has high tissue penetration, delivered by light emitting diodes (LED).
- RetroSense Therapeutics (aquired by Allergan in 2016) was sponsored by Retina Foundation of the Southwest to perform a clinical trail on human patients with retinitis pigmentosa. Channelrhodopsin-2 was delivered to retina cells in an AAV vector.
- Wireless optogenetic nanoscale device
- Deep brain electrical stimulation for Parkinson's disease (Circuit Therapeutics)
Devices and technology
- BrainGate Neural Interface, developed by Cyberkinetics with the Department of Neuroscience at Brown University developed for people with paralysis. Intellectual property is owned by the BrainGate company.
- BrainGate2, sucessor to BrainGate that is smaller than a contact lens and is surgically implanted into the ares of a disabled user’s motor cortex. Research on BrainGate2 is by the BrainGate Research Team which is funded from federal and philanthropic sources and separate from BrainGate the company.
- Cochlear implants
- Deep brain stimulators
- Responsive neurostimulation (RNS), utilizes intracranial electroencephalography (EEG) to detect seizures and delivers stimulation to cortical and subcortical brain structures for seizure control
- Visual cortical prosthesis, intended to restore visual function using electronic circuitry and electrical impulses
Research organizations and initiatives
- BNCI Horizon 2020 is a Coordination and Support Action funded within the European Commission’s Framework Programme 7 that aims to foster collaboration and communication among stakeholders such as research groups, companies, end users, policy makers and the general public.
- BrainGate research consortium includes researchers from Stanford, Brown, and Case Western Reserve University, investigating BrainGate2
DARPA and the Brain Initiative (USA)
The initiative is supported by several federal agencies, technology firms, academic institutions and scientists.
- Hand Proprioception and Touch Interfaces (HAPTIX), neural-interface, sense of touch for amputees
- Neural Engineering System Design (NESD), implantable neural interface, bio-electronics
- Neuro Function, Activity, Structure and Technology (Neuro-FAST)
- Next-Generation Nonsurgical Neurotechnology (N3), nonsurgical neural interfaces
- Reliable Neural-Interface Technology (RE-NET)
- Restoring Active Memory (RAM), implantable neural-interface memory prosthesis/memory aid
- Restoring Active Memory – Replay (RAM Replay)
- Revolutionizing Prosthetics program
- Systems-Based Neurotechnology for Emerging Therapies (SUBNETS), closed-loop diagnostic and therapeutic systems for neuropsychological illnesses
- BrainGate Co. is advancing the BrainGate Neural Interface system
- CTRL-labs, electromyography-based armband that reads nerve signal intentions to move
- Neuralink, developing implantable brain-computer interface (BCI) called the N1 sensor
- Kernel, developing hardware and software for implantable devices for people with neurological and degenerative diseases like epilepsy, dementia and Alzheimer’s disease
- Facebook - developing a skullcap to allow users to mentally type
- Metabrain, neural interfaces to speed up communication between humans and computers
- Truust Neuroimaging, neuroimaging, data processing and analysis
- Paradromics, developing implantable chip/neural interface for brains and computers to exchange data, communication device for people with paralysis
- Neuroloom, living electrodes made with nerve cells grown in microscopic needles interface with the retina to improve the ability to stimulate the retina and restore vision
- Blackrock Microsystems LLC, provides tools for neural engineering and neuroprosthetics
- Natus Medical, neuromonotoring products
- Emotiv, EEG brain monitoring
- NeuroSky, EEG
- Nihon Kohden, EEG
- Compumedics Limited, neurophysiology, cardiology, sleep disorders
- g.tec medical engineering, invasive and non-invasive BCIs
- Brain Products GmbH, neurophysiological hardware and software
- Advanced Brain Monitoring, B-Alert wireless EEG
- BrainCo, Inc., cognitive training and prosthetics
- MindMaze SA, rehabilitation, virtual reality medical products to neural recovery
- Neuroelectrics, EEG-based brain monitoring, brain stimulation, home therapy research
- Synchron, Inc., implantable neural interface, assistive technology for paralysis and neurological conditions
- NextMind, EEG-based BCI for mass market
- BIOS, BCI linked to AI to discover neural biomarkers and use AI-based neural treatments to treat chronic conditions
- NeuroPace, implantable BCI to treat neurological disorders, responsive neurostimulator (RNS) system for epilepsy
- Cadwell Industries, Inc, neurophysiology devices
- PlatoScience, tDCS for boosting cognition
- Neurosphere, neurofeedback enhanced meditation for enhanced well-being and productivity
- RxFunction, leg sensory neuroprosthesis for neuropathy and balance problems
- ANT Neuro, recording and analysis for EEG, EMG, MRI, TMS and MEG technology
- Neurable, neurotechnology hardwared and software, EEG-based, interpretation of human intent, control of devices, VR games
Non-invasive BCI for a mind-controlled robotic arm
Noninvasive neuroimaging enhances continuous neural tracking for robotic device control
Edelman, B. J., Meng, J., Suma, D., Zurn, C., Nagarajan, E., Baxter, B. S., C.C. Cline and He, B.
Carnegi Mellon University, University of Minnesota
Startup Neurable Unveils the World's First Brain-Controlled VR Game
Neurable's brain-computer interfaces enable hands-free control in virtual reality
A microelectrode array implanted into the primary motor cortex of a person with tetraplegia was used in a BCI system enabling the operation of e-mail, television, prosthetic hand and a robotic arm.
Farwell and Donchin demonstrated use of the P300 event-related potential to allow normal volunteers to spell words on a computer screen.
Dr. Jacques J. Vidal professor at University of California Los Angeles posed the question of whether observable brain signals could be carriers of information in person-computer communication or for controlling devices.
Signals from single cortical neurons demonstrated to control a meter needle
Operant Conditioning of Cortical Unit Activity
E.E. Fetz. Science 1969 Feb 28;163(3870):955-8.