The current state of electrocorticography-based brain–computer interfaces

View More View Less
  • 1 Departments of Neurosurgery,
  • 2 Physiology & Biomedical Engineering, and
  • 3 Neurology, Mayo Clinic, Rochester, Minnesota
Free access

Brain–computer interfaces (BCIs) provide a way for the brain to interface directly with a computer. Many different brain signals can be used to control a device, varying in ease of recording, reliability, stability, temporal and spatial resolution, and noise. Electrocorticography (ECoG) electrodes provide a highly reliable signal from the human brain surface, and these signals have been used to decode movements, vision, and speech. ECoG-based BCIs are being developed to provide increased options for treatment and assistive devices for patients who have functional limitations. Decoding ECoG signals in real time provides direct feedback to the patient and can be used to control a cursor on a computer or an exoskeleton. In this review, the authors describe the current state of ECoG-based BCIs that are approaching clinical viability for restoring lost communication and motor function in patients with amyotrophic lateral sclerosis or tetraplegia. These studies provide a proof of principle and the possibility that ECoG-based BCI technology may also be useful in the future for assisting in the cortical rehabilitation of patients who have suffered a stroke.

ABBREVIATIONS ALS = amyotrophic lateral sclerosis; BCI = brain–computer interface; ECoG = electrocorticography; EEG = electroencephalography; LIS = locked-in syndrome; WIMAGINE = Wireless Implantable Multichannel Acquisition system for Generic Interface with Neurons.

Brain–computer interfaces (BCIs) provide a way for the brain to interface directly with a computer. Many different brain signals can be used to control a device, varying in ease of recording, reliability, stability, temporal and spatial resolution, and noise. Electrocorticography (ECoG) electrodes provide a highly reliable signal from the human brain surface, and these signals have been used to decode movements, vision, and speech. ECoG-based BCIs are being developed to provide increased options for treatment and assistive devices for patients who have functional limitations. Decoding ECoG signals in real time provides direct feedback to the patient and can be used to control a cursor on a computer or an exoskeleton. In this review, the authors describe the current state of ECoG-based BCIs that are approaching clinical viability for restoring lost communication and motor function in patients with amyotrophic lateral sclerosis or tetraplegia. These studies provide a proof of principle and the possibility that ECoG-based BCI technology may also be useful in the future for assisting in the cortical rehabilitation of patients who have suffered a stroke.

ABBREVIATIONS ALS = amyotrophic lateral sclerosis; BCI = brain–computer interface; ECoG = electrocorticography; EEG = electroencephalography; LIS = locked-in syndrome; WIMAGINE = Wireless Implantable Multichannel Acquisition system for Generic Interface with Neurons.

For patients with functional limitations, the ability to use brain signals to control an assistive medical device would greatly improve quality of life. Patients with amyotrophic lateral sclerosis (ALS) or tetraplegia, for example, have significant impairments in communication and motor control. For patients with ALS, eye tracking may provide an option for these individuals to control a device, but this technology depends on light conditions and full ocular mobility, which may be limited in the late stages of ALS.1 Two recent studies in electrocorticography (ECoG)-based brain–computer interfaces (BCIs)—systems that record, amplify, and translate brain signals into computer commands for external devices—offer hope to these patients for improved self-management of physical limitations by restoring communication or motor control capabilities.2,3 In this review, we primarily focus on 2 conditions with high functional impairment and morbidity—ALS and tetraplegia—for which BCI technology has been implemented most extensively.

ALS is a neurodegenerative disease in which the death of motor neurons leads to progressive loss of muscle function; advanced cases result in complete peripheral paralysis (the locked-in syndrome [LIS]), which includes loss of speech. Compared with movements of the body, extraocular movements are spared until later in the disease course for patients with ALS who have LIS, allowing for use of eye-tracking systems. The prevalence of ALS in the US is 5 in 100,000 people;4 the median survival of patients with ALS is 2–3 years, with eventual death resulting from respiratory failure. Studies have shown that patients with LIS who can maintain self-initiated, open-ended communication through electronic devices believe themselves to have a higher quality of life than those who are limited to communicating by responding to questions from others with a yes/no code.5

Tetraplegia (also called quadriplegia) is the paralysis of all 4 extremities and the torso as a result of injury to the cervical spinal cord. Every year an estimated 250,000–500,000 individuals worldwide will have a spinal cord injury, most commonly caused by vehicular crashes, falls, or acts of violence.6 Nearly 50% of these individuals will have a spinal cord injury in the cervical region,7 and among individuals with a cervical spinal cord injury, 20% will be diagnosed with tetraplegia.2 In the US alone, there are more than 100,000 patients living with tetraplegia.

The development of BCIs as a new communication and control mode for patients with severe loss of communication ability or motor function began 30 years ago with electroencephalography (EEG) electrodes placed on the scalp. These used either changes in the raw deflection of the voltage associated with attention to visual stimuli in navigating a decision matrix8 or movement-associated band-limited changes in the power spectrum of the voltage to control 1-dimensional cursor movement on a computer screen.9 EEG sensors (electrodes) for recording brain activity for BCIs are noninvasive and widely available, they have a limited spatial resolution, and brain signals are relatively small compared to the noise, which reduces the single-trial decoding ability. At the opposite extreme are surgically placed microelectrodes that penetrate the cortex, measuring single neurons; these offer multidimensional control of BCI devices.10–12 In their current form, these high-fidelity microelectrode signals require externalized leads with connection to a research-grade experimental rig. Due to issues of infection, biological rejection, and signal instability, these electrodes are still at the stage of ongoing development.13–15 ECoG electrodes, placed subdurally on the arachnoidal surface of the brain, offer a middle-ground signal between microelectrodes and EEG (Fig. 1). Modern ECoG research began with a 1998 study in which electrodes over the sensorimotor cortex detected increases in high-frequency power during movement compared to rest.16,17 Currently, the use of subdural ECoG electrode arrays for BCIs provides a high signal-to-noise ratio and a localized cortical signal (typically, clinical ECoG arrays consist of electrodes 2.3 mm in diameter, placed 1 cm apart).

FIG. 1.
FIG. 1.

Electrode types that have been used for BCIs: EEG from scalp, ECoG from brain surface, and cortex-penetrating microelectrodes. Copyright Kai J. Miller. Published with permission.

The identification of a strong, reproducible brain signal in an individual can be used as the basis for driving a BCI. ECoG measures the electrical potential on the surface of the brain resulting from the sum of the local field potentials of the population of neurons directly underneath each electrode—approximately 500,000 for the most common clinical contact surface area of 5 mm.2,18 During active movement, the ECoG signal exhibits a superposition including 2 characteristic changes that occur in the cortical power spectrum and can be decoupled: 1) narrow-band power decreases at frequencies < 40 Hz (alpha/mu and beta rhythms) over large areas of the sensorimotor cortex;17,19,20 and 2) a broadband increase in power at frequencies > 50 Hz over focal cortical areas16,19,21–23 (Fig. 2). The ECoG signal has demonstrated broadband increases in power during activity, compared to rest, across a variety of cortical regions and behavioral tasks.24

FIG. 2.
FIG. 2.

Basic ECoG signal changes with movement. A: During movement, there is a widespread decrease in amplitude in the rhythmic phenomenon (“beta rhythm,” green), and spatially focal, somatotopic, broadband changes (pink). B: Schematic power spectrum to demonstrate the spectral shift between hand movement and rest at the most significant electrode. The intersection of rhythmic and broadband phenomena is typically in the 40- to 50-Hz range. C: The ECoG power spectrum (black line) is a superposition of 1-over-frequency–shaped broadband signal (pink) and rhythmic signal (green). D: The time series of broadband and rhythmic signals are shown during movement and rest. The broadband time series appears in the raw signal as a faster-moving random walk, whereas the brain rhythm can be visualized in the raw signal.

Most of what has been learned about ECoG signals for driving BCIs has been from patients with medically intractable epilepsy. These are individuals who undergo clinical ECoG for localization of the seizure focus prior to resection; after implantation they are monitored in the hospital for approximately 1 week while waiting for the onset of another seizure. During this time they may volunteer for studies that allow researchers to correlate strictly defined tasks with resulting ECoG signals that (because the placement of the ECoG array is known) can be correlated with the structure of the brain.23,25

ECoG-based BCIs may consist of the following components (Fig. 3):26 1) an implantable electrode array for real-time recording and digitization of the brain signal; 2) a subcutaneous amplifier and transmitter unit (with leads connected to the electrode array) for wireless transmission of the digitized signal; 3) a Bluetooth-enabled, signal processing computer for real-time analysis of the signal and translation into computer commands; and 4) a computerized device that receives these commands and performs an action that is now under the control of the patient (e.g., a tablet computer for communicating or Web browsing, an exoskeleton, a robotic arm, or a wheelchair). Implantable ECoG electrode arrays for clinical use are commercially available from a number of sources—e.g., NeuroPace, Medtronic, and PMT Corp.—as are some additional components of the BCI system. Current trends of improved electrodes, as well as implants that are more standardized and easier to place, are driving the ECoG-based BCI field forward.

FIG. 3.
FIG. 3.

Schematic artwork depicting the setup of the BCI from the Utrecht group.3 A: Signals are acquired from an implanted strip of ECoG electrodes. B: The brain signal is amplified by an implanted device in the chest. Basic signal processing is performed on the raw voltages to extract signal features (a high-frequency power measure reflecting broadband changes). Feature values are transmitted wirelessly to an external receiver. C: Using a tablet device, features are mapped onto control features for communication and navigation applications. Copyright Kai J. Miller. Published with permission.

Current Status of ECoG-Based BCI Studies

Current BCI devices are generally based on signals from 1 of 3 cortical areas: motor, vision, or speech. These demonstrations open possibilities for many other brain functions and other general BCI applications such as cortical rehabilitation.

Motor-Based Studies

In studies with epileptic patients, imagined motor movement (imagery) has been shown to induce ECoG-measured cortical activity change in primary motor areas of the brain at a level approximately 25% of that induced by the actual motor movement itself.27 Furthermore, when these individuals learned to use imagery to control a cursor on a computer screen in a feedback-providing manner, the cortical activity change induced by motor imagery surpassed that from the actual movement. Several groups showed initial proofs of principle in the research setting where a computer cursor could be controlled with motor imagery.28 These studies demonstrated that motor imagery has the potential to play an important role in driving motor-based BCIs, as the following 2 successful long-term ECoG-based BCI examples illustrate.

Interactional (Web Browsing, Communication)

In a proof-of-concept study, an ECoG-based BCI enabled an ALS patient with LIS to communicate independently via a typing program on a tablet computer.3 In October 2015, 2 commercially available, subdural electrode strips (Resume II, Medtronic) were implanted through 1-cm burr holes over the sensorimotor area. Leads connected to the electrodes were tunneled subcutaneously and connected to a commercial amplifier and transmitter device (Activa PC+S, Medtronic) placed subcutaneously below the left clavicle. When the BCI was in use, an antenna was placed on the chest over the amplifier/transmitter device; the antenna was connected by cable to a receiver that was in turn connected by cable to a tablet computer (Fig. 3). Previous studies had demonstrated that patients with quadriplegia were able to generate brain activity by attempting hand movement. Thus, during 28 weeks of training postimplantation, the ALS patient performed a series of increasingly challenging tasks on the computer screen by attempting to move her right hand, thereby generating a consistent ECoG signal: moving a cursor up and down to hit a target (to practice activating the motor cortex); moving a ball up and down at specific times (to regulate the timing and amplitude of the brain signal); and generating a “brain click” (mouse click) by maintaining an attempted hand movement for 1 second (to select a specific item from a grid). Spelling was performed with a commercial typing program by using brain clicks to select individual or grouped letters in sequential order. The patient was able to spell 1 letter every 52 seconds, which reduced to 33 seconds when word prediction was used. Even though spelling speed may not be very fast, the patient indicated that she was “satisfied/very satisfied” with the system and that the system was more adaptable compared to the eye tracker. On the 197th day postimplantation, the patient began independently using the system for communication in her home, and has continued to do so for more than 43 months, with usage increasing after she became fully locked in without eye movement. Furthermore, the ECoG signal and translation algorithms remained stable over this entire interval.29

Functional (Exoskeleton/Robotic Arms/Wheelchairs)

In a proof-of-concept study, an ECoG-based BCI enabled a tetraplegic patient to use brain signals to control a 4-limb exoskeleton.2 This study used the newly developed Wireless Implantable Multichannel Acquisition system for Generic Interface with Neurons (WIMAGINE) ECoG recorder,30 a 64-channel, biocompatible unit implanted epidurally to prevent intracranial infections. Bilateral WIMAGINE recorders were permanently placed over the upper-limb sensorimotor cortices; a helmet positioned an external antenna in front of each of the WIMAGINE units for wireless connection. The robotic neuroprosthesis (exoskeleton) was wearable, motorized, and equipped with a computer backpack for receiving, analyzing, and translating ECoG signals into mono-, bi-, and 3-dimensional movements of the upper limbs (Fig. 4). To perform a new exoskeleton movement, the patient created a mental image of the new movement, thereby increasing neural activity in the sensorimotor cortices that was, in turn, detected by the ECoG recorders and subsequently translated into computer commands to drive exoskeleton motors. Over a period of 2 years, the patient was able to use motor imagery to drive an exoskeleton to make 2-handed, multijoint movements of the upper limbs (using as many as 8 degrees of freedom) and to simulate walking (for a total distance of 145 m). Performance was not perfect and there were still some false positives across the different tasks (e.g., 5.6 per minute for the walking tasks with the exoskeleton). However, the same decoder could be used over at least 1.5 months, showing feasibility for day-to-day use without recalibration. The investigators anticipate that the results of this study, demonstrating that a tetraplegic patient could use brain signals to control a neuroprosthetic exoskeleton, can be extended in the future to enable tetraplegic patients to use brain activity to drive a wheelchair.

FIG. 4.
FIG. 4.

The BCI from the Grenoble team. A: An epidural WIMAGINE implant with wireless recharging ability is implanted into the space of a craniotomy over the hand sensorimotor region, bilaterally. B: Each WIMAGINE wireless recorder contains an 8 × 8 epidural ECoG array, with each electrode being 2 mm in diameter. There are 5 embedded reference electrodes. C and D: The signals are translated to control an exoskeleton that the patient is suspended in. Reprinted from The Lancet Neurology, Vol. 18, Benabid AL, Costecalde T, Eliseyev A, et al. An exoskeleton controlled by an epidural wireless brain-machine interface in a tetraplegic patient: a proof-of-concept demonstration, pp 1112–1122, copyright 2019, with permission from Elsevier.

Perceptual Decoding Studies (Vision)

ECoG arrays provide a new mechanism for assessing how the brain processes visual stimuli. To determine the relationship between object perception and neural activity in visual cortices, clinical ECoG arrays in 7 patients with epilepsy measured raw potential deflections and broadband changes associated with the subjects’ identification of a random series of house and face images.31 Results of the study indicated that the 2 measurements provided complementary information, and that analyzing both in combination was more accurate than either alone in predicting the timing and identification of each image perceived. Together, these measurements may provide a robust means of measuring how well the brain achieves the expected response to visual stimuli during good health as well as after injury or disease.

In a follow-up study, the subjects were asked to identify all of the faces when noise-degraded face and house images were randomly interspersed throughout the stream of nonnoisy images.32 When the noise level was increased, a decrease of cortical activity was measured by ECoG in a subset of known face-processing areas within the ventral temporal cortex. These results were consistent with face perception occurring in sequential steps within specific subregions of the cortex: in preperceptual regions (the pericalcarine cortex), basic shapes are built from simple image properties; in percept formation regions (posterior fusiform region/fusiform at large), basic shapes are converged with higher-order cognitive information such as current memories and present environment; and in postperceptual regions (fusiform at large), the perception is integrated into a larger context for the formation of new context-dependent memories.

The results of this study suggest that different forms of face perceptual deficits (prosopagnosia) after temporal lobe injury may be the result of selective injury to these specific functional subregions. Thus, ECoG-measured activity changes in the ventral temporal cortex during face perception from noisy images may be useful as a model system for studying the effect on cortical processing of limited input, as might occur following a brain injury such as a stroke.

Speech-Related Studies

Multiple steps have been taken toward building an ECoG-based BCI that will enable patients with ALS or stroke who are no longer able to communicate orally to use brain signals to drive a speech synthesizer that will speak for them. Decoding speech intention was accomplished by measuring cortical activity while distinct units of speech (phonemes) were recited by a patient with epilepsy in whom an ECoG microarray had been placed over the frontotemporal area during placement of the larger clinical array. The spacing of electrodes (3 mm apart for the microarray vs 10 mm for the clinical array) allowed for greater spatial resolution for the microarray, which was able to distinguish the subregions of the frontotemporal area associated with specific sets of phonemes.33

Another step was a determination of the individual sites within the ventral sensorimotor cortex that were associated with movement of speech articulators in the vocal tract (larynx, tongue, lips, and jaw). Cortical activity was measured while consonant–vowel syllables were recited by 3 patients with epilepsy in whom ECoG arrays had been implanted over either the left or right hemisphere.34 Results demonstrated that the coordinated, flexible effort of multiple articulators—associated with sites across the ventral sensorimotor cortex—was involved in generating speech.

Building on these earlier results, a speech synthesizer has recently been reported that is based on translating the neural activity associated with spoken sentences.35 Five patients with epilepsy in whom subdural ECoG arrays had been implanted long-term over the lateral surface of the brain recited hundreds of complete sentences while ECoG and microphone recordings were made simultaneously. One of these patients also recited 10 sentences numerous times, alternating between audible and silent (mimed) speech (for the latter, the appropriate mouth movements were made silently). The speech BCI used a 2-step process: in the brain-to-articulation step, the articulatory movements (kinematics) were extracted from the ECoG-recorded cortical activity (using the high-frequency amplitude and raw low-frequency ECoG signals as input); and in the articulation-to-acoustics step, the articulatory kinematics were transformed into speech acoustics. Listeners were able to understand the resulting synthesized speech, even when it was based on cortical activity generated by miming (as opposed to reciting) sentences. These results represent a step forward in the use of BCIs to restore spoken communication for patients with ALS or stroke.

Opportunities for Cortical Rehabilitation (Closed Loop Within the Brain)

One in four people worldwide aged 25 years and older are at risk for experiencing a stroke in their lifetime,36 from which many of these individuals will sustain a serious functional deficit. A large number of lives could be improved if there were a mechanism to rehabilitate cortical function in stroke victims and other patients who have sustained structural brain injury. Rehabilitation might be accomplished by the creation of implantable, closed-loop, ECoG-based devices to induce plasticity and actively remap cortical organization in the human brain after injury.

Evidence that the developing brain can reorganize the motor system after a stroke can be seen in adults with infantile hemiplegia who exhibit mirror movements, the unintentional symmetrical movement of a contralateral body part that accompanies the intentional movement of one part of the body.37 One case report describes a 19-year-old patient with epilepsy with left-sided hemiplegia after a perinatal hemispheric stroke who was under observation after implantation with a clinical ECoG array.38 This individual demonstrated the ability to greatly strengthen the grip in his left hand by simultaneously clenching his right hand. In studies to ascertain the reorganized physiology underlying this mirror movement–based augmentation, ECoG measurement of 12- to 30-Hz oscillations in the hand motor cortex detected significant changes in power accompanying movement of the hands either singly or together. In contrast to patients without motor deficits, these changes were not statistically different among the types of hand movement, suggesting that a common physiological mechanism was now being shared among the movements, due to interhemispheric reorganization in the hand motor cortex following perinatal stroke.

In the future, patients with chronic, severe limitations resulting from stroke, tumor resection, or trauma may be able to benefit from undergoing cortical rehabilitation with the assistance of an implantable, closed-loop ECoG device. The goal would be to increase the plasticity of the injured brain, i.e., its capacity to undergo the neural reorganization associated with learning. The approach would be based on Hebb’s rule and its research-based corollary, which (simplified) state that increased interaction between neurons increases the strength of their connectivity,39 and that this increase in strength occurs only if there is a timing delay between the neurons. Thus, the ECoG-based unit would be expected to have cortical recording and electrical stimulation functions as well as the capability of responding in a timely and autonomous fashion to varying brain activity. After implantation in an area of healthy, functionally related cortex, this device would record and decode brain surface activity and deliver paired stimulation (with a timing delay) to an area of at-risk (but still viable) cortex. It is anticipated that the long-term effect of the stimulation from the closed-loop ECoG device would be to induce neural plasticity in the brain area between the 2 electrodes, augmenting the residual connections and promoting synaptic growth and regeneration. In time, this would be expected to rehabilitate the brain at the edge of, and within, the injured area and restore lost brain function (Fig. 5).

FIG. 5.
FIG. 5.

Schematic of closed-loop neural prosthetic to augment brain function following injury. A: Depiction of 2 cortical regions (blue circles) connected by white matter pathways that have been injured following a stroke (stroke core is pink, stroke penumbra is light purple). B: An implanted device records and decodes brain surface activity and delivers paired stimulation between brain areas that will induce plasticity between the 2 regions. C: After prolonged use of the device, connections between the 2 cortical regions from residual white matter pathways are augmented or co-opted to restore function. Copyright Kai J. Miller. Published with permission.

Current Limitations of ECoG BCIs

The advances in motor, vision, and speech decoding provide substantial evidence that ECoG signals can provide new channels for motor control and communication. Moreover, studies have provided proofs of principle that ECoG-based BCIs are feasible in a few cases. Systems that increase the patient’s control of the system, such as multiple dimensions and faster decoding, can enhance the patient’s experience. Furthermore, brain signals can differ between patients with various etiologies,40 with some providing a more reliable control signal compared to others. It is therefore essential to advance the basic understanding of how brain signals change with various neurological diseases that limit function.

Currently, in published reports on fully implantable subdural ECoG devices (co-opted sense plus stimulation deep brain stimulation systems), these units have been limited to 1–2 channels (1 channel per 4-contact lead, with 2 leads) and have limited on-board feature extraction.3 Emerging devices, such as the WIMAGINE recorder, promise much higher channel counts with expanded signal processing capability.2 Whereas custom-made ECoG grids are currently available for short-term diagnostic and scientific use (cortec-neuro.com), chronically implantable electrode arrays are not. Customizable arrays are available, in which the shapes are aligned to the anatomy, and the contact size is optimized to the resolution of the functionality being assessed. Although reducing contact size to match functional representation is promising,33,35,41–44 there are obvious limitations in terms of how small of a neural scale nonpenetrating electrodes can probe, and also at what point electrode size is no longer considered ECoG. For probing structures not on the brain’s convex surface, depth electrodes (stereo-EEG electrodes) will have a conjugate role alongside ECoG.

Conclusions

Therapeutic BCIs are approaching clinical viability for restoring lost communication and motor function in patients with ALS or tetraplegia. ECoG-based BCI technology may also be useful in the future for assisting in the cortical rehabilitation of patients who have suffered a cortical injury such as stroke, trauma, or resected tumor.

Acknowledgments

Judith Gilbert assisted in the drafting of the manuscript. This publication was supported by CTSA Grant No. KL2 TR002379 from the National Center for Advancing Translational Science (NCATS; K.J.M. and N.P.S.). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH. K.J.M. is also supported by the Brain Research Foundation, with a Fay/Frank Seed Grant.

Disclosures

The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

Author Contributions

Conception and design: all authors. Drafting the article: all authors. Critically revising the article: Miller. Reviewed submitted version of manuscript: all authors. Approved the final version of the manuscript on behalf of all authors: Miller. Administrative/technical/material support: all authors. Study supervision: Miller.

References

  • 1

    Sharma R, Hicks S, Berna CM, Oculomotor dysfunction in amyotrophic lateral sclerosis: a comprehensive review. Arch Neurol. 2011;68(7):857861.

    • Search Google Scholar
    • Export Citation
  • 2

    Benabid AL, Costecalde T, Eliseyev A, An exoskeleton controlled by an epidural wireless brain-machine interface in a tetraplegic patient: a proof-of-concept demonstration. Lancet Neurol. 2019;18(12):11121122.

    • Search Google Scholar
    • Export Citation
  • 3

    Vansteensel MJ, Pels EGM, Bleichner MG, Fully implanted brain–computer interface in a locked-in patient with ALS. N Engl J Med. 2016;375(21):20602066.

    • Search Google Scholar
    • Export Citation
  • 4

    Oskarsson B, Gendron TF, Staff NP. Amyotrophic lateral sclerosis: an update for 2018. Mayo Clin Proc. 2018;93(11):16171628.

  • 5

    Rousseau M-C, Baumstarck K, Alessandrini M, Quality of life in patients with locked-in syndrome: evolution over a 6-year period. Orphanet J Rare Dis. 2015;10(1):88.

    • Search Google Scholar
    • Export Citation
  • 6

    World Health Organization. International perspectives on spinal cord injury. Accessed May 20, 2020. https://www.who.int/disabilities/policies/spinal_cord_injury/en/

    • Search Google Scholar
    • Export Citation
  • 7

    Fridén J, Gohritz A. Tetraplegia management update. J Hand Surg Am. 2015;40(12):24892500.

  • 8

    Farwell LA, Donchin E. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol. 1988;70(6):510523.

    • Search Google Scholar
    • Export Citation
  • 9

    Wolpaw JR, McFarland DJ, Neat GW, Forneris CA. An EEG-based brain-computer interface for cursor control. Electroencephalogr Clin Neurophysiol. 1991;78(3):252259.

    • Search Google Scholar
    • Export Citation
  • 10

    Santhanam G, Ryu SI, Yu BM, A high-performance brain-computer interface. Nature. 2006;442(7099):195198.

  • 11

    Sakellaridi S, Christopoulos VN, Aflalo T, Intrinsic variable learning for brain-machine interface control by human anterior intraparietal cortex. Neuron. 2019;102(3):694705.e693.

    • Search Google Scholar
    • Export Citation
  • 12

    Pandarinath C, Nuyujukian P, Blabe CH, High performance communication by people with paralysis using an intracortical brain-computer interface. eLife. 2017;6:e18554.

    • Search Google Scholar
    • Export Citation
  • 13

    Gulino M, Kim D, Pané S, Tissue response to neural implants: the use of model systems toward new design solutions of implantable microelectrodes. Front Neurosci. 2019;13:689.

    • Search Google Scholar
    • Export Citation
  • 14

    Goss-Varley M, Dona KR, McMahon JA, Microelectrode implantation in motor cortex causes fine motor deficit: implications on potential considerations to brain computer interfacing and human augmentation. Sci Rep. 2017;7(1):15254.

    • Search Google Scholar
    • Export Citation
  • 15

    Kim GH, Kim K, Lee E, Recent progress on microelectrodes in neural interfaces. Materials (Basel). 2018;11(10):1995.

  • 16

    Crone NE, Miglioretti DL, Gordon B, Lesser RP. Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. II. Event-related synchronization in the gamma band. Brain. 1998;121(pt 12):23012315.

    • Search Google Scholar
    • Export Citation
  • 17

    Crone NE, Miglioretti DL, Gordon B, Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. I. Alpha and beta event-related desynchronization. Brain. 1998;121(pt 12):22712299.

    • Search Google Scholar
    • Export Citation
  • 18

    Miller KJ, Sorensen LB, Ojemann JG, den Nijs M. Power-law scaling in the brain surface electric potential. PLoS Comput Biol. 2009;5(12):e1000609.

    • Search Google Scholar
    • Export Citation
  • 19

    Miller KJ, Leuthardt EC, Schalk G, Spectral changes in cortical surface potentials during motor movement. J Neurosci. 2007;27(9):24242432.

    • Search Google Scholar
    • Export Citation
  • 20

    Jasper H, Penfield W. Electrocorticograms in man: effect of voluntary movement upon the electrical activity of the precentral gyrus. Arch Psychiatr Nervenkr. 1949;183(1–2):163174.

    • Search Google Scholar
    • Export Citation
  • 21

    Miller KJ, Hermes D, Honey CJ, Human motor cortical activity is selectively phase-entrained on underlying rhythms. PLoS Comput Biol. 2012;8(9):e1002655.

    • Search Google Scholar
    • Export Citation
  • 22

    Miller KJ, Ojemann JG. A simple, spectral-change based, electrocorticographic brain–computer interface. In: Graimann B, Pfurtscheller G, Allison B, eds. Brain-Computer Interfaces: Revolutionizing Human-Computer Interaction. Springer; 2009:241258.

    • Search Google Scholar
    • Export Citation
  • 23

    Miller KJ. A library of human electrocorticographic data and analyses. Nat Hum Behav. 2019;3(11):12251235.

  • 24

    Miller KJ, Honey CJ, Hermes D, Broadband changes in the cortical surface potential track activation of functionally diverse neuronal populations. Neuroimage. 2014;85(pt 2):711720.

    • Search Google Scholar
    • Export Citation
  • 25

    Crone NE. Functional mapping with ECoG spectral analysis. Adv Neurol. 2000;84:343351.

  • 26

    Schalk G, McFarland DJ, Hinterberger T, BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans Biomed Eng. 2004;51(6):10341043.

    • Search Google Scholar
    • Export Citation
  • 27

    Miller KJ, Schalk G, Fetz EE, Cortical activity during motor execution, motor imagery, and imagery-based online feedback. Proc Natl Acad Sci U S A. 2010;107(9):44304435.

    • Search Google Scholar
    • Export Citation
  • 28

    Nam CS, Nijholt A, Lotte F. Brain–Computer Interfaces Handbook: Technological and Theoretical Advances. CRC Press; 2018.

  • 29

    Pels EGM, Aarnoutse EJ, Leinders S, Stability of a chronic implanted brain-computer interface in late-stage amyotrophic lateral sclerosis. Clin Neurophysiol. 2019;130(10):17981803.

    • Search Google Scholar
    • Export Citation
  • 30

    Sauter-Starace F, Ratel D, Cretallaz C, Long-term sheep implantation of WIMAGINE®, a wireless 64-channel electrocorticogram recorder. Front Neurosci. 2019;13:847.

    • Search Google Scholar
    • Export Citation
  • 31

    Miller KJ, Schalk G, Hermes D, Spontaneous decoding of the timing and content of human object perception from cortical surface recordings reveals complementary information in the event-related potential and broadband spectral change. PLoS Comput Biol. 2016;12(1):e1004660.

    • Search Google Scholar
    • Export Citation
  • 32

    Miller KJ, Hermes D, Pestilli F, Face percept formation in human ventral temporal cortex. J Neurophysiol. 2017;118(5):26142627.

  • 33

    Blakely T, Miller KJ, Rao RP, Localization and classification of phonemes using high spatial resolution electrocorticography (ECoG) grids. Paper presented at: 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; August 21–24, 2008; Vancouver, BC, Canada.

    • Search Google Scholar
    • Export Citation
  • 34

    Bouchard KE, Mesgarani N, Johnson K, Chang EF. Functional organization of human sensorimotor cortex for speech articulation. Nature. 2013;495(7441):327332.

    • Search Google Scholar
    • Export Citation
  • 35

    Anumanchipalli GK, Chartier J, Chang EF. Speech synthesis from neural decoding of spoken sentences. Nature. 2019;568(7753):493498.

  • 36

    Feigin VL, Nguyen G, Cercy K, Global, regional, and country-specific lifetime risks of stroke, 1990 and 2016. N Engl J Med. 2018;379(25):24292437.

    • Search Google Scholar
    • Export Citation
  • 37

    Woods BT, Teuber H-L. Mirror movements after childhood hemiparesis. Neurology. 1978;28(11):11521157.

  • 38

    Miller KJ, Abel TJ, Hebb AO, Ojemann JG. Reorganization of large-scale physiology in hand motor cortex following hemispheric stroke. Neurology. 2011;76(10):927929.

    • Search Google Scholar
    • Export Citation
  • 39

    Hebb DO. The Organization of Behavior: A Neuropsychological Theory. Psychology Press; 2005.

  • 40

    Freudenburg ZV, Branco MP, Leinders S, Sensorimotor ECoG signal features for BCI control: a comparison between people with locked-in syndrome and able-bodied controls. Front Neurosci. 2019;13:1058.

    • Search Google Scholar
    • Export Citation
  • 41

    Wang W, Degenhart AD, Collinger JL, Human motor cortical activity recorded with micro-ECoG electrodes, during individual finger movements. Paper presented at: 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society; September 3–6, 2009; Minneapolis, MN.

    • Search Google Scholar
    • Export Citation
  • 42

    Wang X, Gkogkidis CA, Iljina O, Mapping the fine structure of cortical activity with different micro-ECoG electrode array geometries. J Neural Eng. 2017;14(5):056004.

    • Search Google Scholar
    • Export Citation
  • 43

    Kellis S, Miller K, Thomson K, Decoding spoken words using local field potentials recorded from the cortical surface. J Neural Eng. 2010;7(5):056007.

    • Search Google Scholar
    • Export Citation
  • 44

    Siero JC, Hermes D, Hoogduin H, BOLD matches neuronal activity at the mm scale: a combined 7T fMRI and ECoG study in human sensorimotor cortex. Neuroimage. 2014;101:177184.

    • Search Google Scholar
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

Contributor Notes

Correspondence Kai J. Miller: Mayo Clinic, Rochester, MN. miller.kai@mayo.edu.

INCLUDE WHEN CITING DOI: 10.3171/2020.4.FOCUS20185.

Disclosures The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

  • View in gallery

    Electrode types that have been used for BCIs: EEG from scalp, ECoG from brain surface, and cortex-penetrating microelectrodes. Copyright Kai J. Miller. Published with permission.

  • View in gallery

    Basic ECoG signal changes with movement. A: During movement, there is a widespread decrease in amplitude in the rhythmic phenomenon (“beta rhythm,” green), and spatially focal, somatotopic, broadband changes (pink). B: Schematic power spectrum to demonstrate the spectral shift between hand movement and rest at the most significant electrode. The intersection of rhythmic and broadband phenomena is typically in the 40- to 50-Hz range. C: The ECoG power spectrum (black line) is a superposition of 1-over-frequency–shaped broadband signal (pink) and rhythmic signal (green). D: The time series of broadband and rhythmic signals are shown during movement and rest. The broadband time series appears in the raw signal as a faster-moving random walk, whereas the brain rhythm can be visualized in the raw signal.

  • View in gallery

    Schematic artwork depicting the setup of the BCI from the Utrecht group.3 A: Signals are acquired from an implanted strip of ECoG electrodes. B: The brain signal is amplified by an implanted device in the chest. Basic signal processing is performed on the raw voltages to extract signal features (a high-frequency power measure reflecting broadband changes). Feature values are transmitted wirelessly to an external receiver. C: Using a tablet device, features are mapped onto control features for communication and navigation applications. Copyright Kai J. Miller. Published with permission.

  • View in gallery

    The BCI from the Grenoble team. A: An epidural WIMAGINE implant with wireless recharging ability is implanted into the space of a craniotomy over the hand sensorimotor region, bilaterally. B: Each WIMAGINE wireless recorder contains an 8 × 8 epidural ECoG array, with each electrode being 2 mm in diameter. There are 5 embedded reference electrodes. C and D: The signals are translated to control an exoskeleton that the patient is suspended in. Reprinted from The Lancet Neurology, Vol. 18, Benabid AL, Costecalde T, Eliseyev A, et al. An exoskeleton controlled by an epidural wireless brain-machine interface in a tetraplegic patient: a proof-of-concept demonstration, pp 1112–1122, copyright 2019, with permission from Elsevier.

  • View in gallery

    Schematic of closed-loop neural prosthetic to augment brain function following injury. A: Depiction of 2 cortical regions (blue circles) connected by white matter pathways that have been injured following a stroke (stroke core is pink, stroke penumbra is light purple). B: An implanted device records and decodes brain surface activity and delivers paired stimulation between brain areas that will induce plasticity between the 2 regions. C: After prolonged use of the device, connections between the 2 cortical regions from residual white matter pathways are augmented or co-opted to restore function. Copyright Kai J. Miller. Published with permission.

  • 1

    Sharma R, Hicks S, Berna CM, Oculomotor dysfunction in amyotrophic lateral sclerosis: a comprehensive review. Arch Neurol. 2011;68(7):857861.

    • Search Google Scholar
    • Export Citation
  • 2

    Benabid AL, Costecalde T, Eliseyev A, An exoskeleton controlled by an epidural wireless brain-machine interface in a tetraplegic patient: a proof-of-concept demonstration. Lancet Neurol. 2019;18(12):11121122.

    • Search Google Scholar
    • Export Citation
  • 3

    Vansteensel MJ, Pels EGM, Bleichner MG, Fully implanted brain–computer interface in a locked-in patient with ALS. N Engl J Med. 2016;375(21):20602066.

    • Search Google Scholar
    • Export Citation
  • 4

    Oskarsson B, Gendron TF, Staff NP. Amyotrophic lateral sclerosis: an update for 2018. Mayo Clin Proc. 2018;93(11):16171628.

  • 5

    Rousseau M-C, Baumstarck K, Alessandrini M, Quality of life in patients with locked-in syndrome: evolution over a 6-year period. Orphanet J Rare Dis. 2015;10(1):88.

    • Search Google Scholar
    • Export Citation
  • 6

    World Health Organization. International perspectives on spinal cord injury. Accessed May 20, 2020. https://www.who.int/disabilities/policies/spinal_cord_injury/en/

    • Search Google Scholar
    • Export Citation
  • 7

    Fridén J, Gohritz A. Tetraplegia management update. J Hand Surg Am. 2015;40(12):24892500.

  • 8

    Farwell LA, Donchin E. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol. 1988;70(6):510523.

    • Search Google Scholar
    • Export Citation
  • 9

    Wolpaw JR, McFarland DJ, Neat GW, Forneris CA. An EEG-based brain-computer interface for cursor control. Electroencephalogr Clin Neurophysiol. 1991;78(3):252259.

    • Search Google Scholar
    • Export Citation
  • 10

    Santhanam G, Ryu SI, Yu BM, A high-performance brain-computer interface. Nature. 2006;442(7099):195198.

  • 11

    Sakellaridi S, Christopoulos VN, Aflalo T, Intrinsic variable learning for brain-machine interface control by human anterior intraparietal cortex. Neuron. 2019;102(3):694705.e693.

    • Search Google Scholar
    • Export Citation
  • 12

    Pandarinath C, Nuyujukian P, Blabe CH, High performance communication by people with paralysis using an intracortical brain-computer interface. eLife. 2017;6:e18554.

    • Search Google Scholar
    • Export Citation
  • 13

    Gulino M, Kim D, Pané S, Tissue response to neural implants: the use of model systems toward new design solutions of implantable microelectrodes. Front Neurosci. 2019;13:689.

    • Search Google Scholar
    • Export Citation
  • 14

    Goss-Varley M, Dona KR, McMahon JA, Microelectrode implantation in motor cortex causes fine motor deficit: implications on potential considerations to brain computer interfacing and human augmentation. Sci Rep. 2017;7(1):15254.

    • Search Google Scholar
    • Export Citation
  • 15

    Kim GH, Kim K, Lee E, Recent progress on microelectrodes in neural interfaces. Materials (Basel). 2018;11(10):1995.

  • 16

    Crone NE, Miglioretti DL, Gordon B, Lesser RP. Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. II. Event-related synchronization in the gamma band. Brain. 1998;121(pt 12):23012315.

    • Search Google Scholar
    • Export Citation
  • 17

    Crone NE, Miglioretti DL, Gordon B, Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. I. Alpha and beta event-related desynchronization. Brain. 1998;121(pt 12):22712299.

    • Search Google Scholar
    • Export Citation
  • 18

    Miller KJ, Sorensen LB, Ojemann JG, den Nijs M. Power-law scaling in the brain surface electric potential. PLoS Comput Biol. 2009;5(12):e1000609.

    • Search Google Scholar
    • Export Citation
  • 19

    Miller KJ, Leuthardt EC, Schalk G, Spectral changes in cortical surface potentials during motor movement. J Neurosci. 2007;27(9):24242432.

    • Search Google Scholar
    • Export Citation
  • 20

    Jasper H, Penfield W. Electrocorticograms in man: effect of voluntary movement upon the electrical activity of the precentral gyrus. Arch Psychiatr Nervenkr. 1949;183(1–2):163174.

    • Search Google Scholar
    • Export Citation
  • 21

    Miller KJ, Hermes D, Honey CJ, Human motor cortical activity is selectively phase-entrained on underlying rhythms. PLoS Comput Biol. 2012;8(9):e1002655.

    • Search Google Scholar
    • Export Citation
  • 22

    Miller KJ, Ojemann JG. A simple, spectral-change based, electrocorticographic brain–computer interface. In: Graimann B, Pfurtscheller G, Allison B, eds. Brain-Computer Interfaces: Revolutionizing Human-Computer Interaction. Springer; 2009:241258.

    • Search Google Scholar
    • Export Citation
  • 23

    Miller KJ. A library of human electrocorticographic data and analyses. Nat Hum Behav. 2019;3(11):12251235.

  • 24

    Miller KJ, Honey CJ, Hermes D, Broadband changes in the cortical surface potential track activation of functionally diverse neuronal populations. Neuroimage. 2014;85(pt 2):711720.

    • Search Google Scholar
    • Export Citation
  • 25

    Crone NE. Functional mapping with ECoG spectral analysis. Adv Neurol. 2000;84:343351.

  • 26

    Schalk G, McFarland DJ, Hinterberger T, BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans Biomed Eng. 2004;51(6):10341043.

    • Search Google Scholar
    • Export Citation
  • 27

    Miller KJ, Schalk G, Fetz EE, Cortical activity during motor execution, motor imagery, and imagery-based online feedback. Proc Natl Acad Sci U S A. 2010;107(9):44304435.

    • Search Google Scholar
    • Export Citation
  • 28

    Nam CS, Nijholt A, Lotte F. Brain–Computer Interfaces Handbook: Technological and Theoretical Advances. CRC Press; 2018.

  • 29

    Pels EGM, Aarnoutse EJ, Leinders S, Stability of a chronic implanted brain-computer interface in late-stage amyotrophic lateral sclerosis. Clin Neurophysiol. 2019;130(10):17981803.

    • Search Google Scholar
    • Export Citation
  • 30

    Sauter-Starace F, Ratel D, Cretallaz C, Long-term sheep implantation of WIMAGINE®, a wireless 64-channel electrocorticogram recorder. Front Neurosci. 2019;13:847.

    • Search Google Scholar
    • Export Citation
  • 31

    Miller KJ, Schalk G, Hermes D, Spontaneous decoding of the timing and content of human object perception from cortical surface recordings reveals complementary information in the event-related potential and broadband spectral change. PLoS Comput Biol. 2016;12(1):e1004660.

    • Search Google Scholar
    • Export Citation
  • 32

    Miller KJ, Hermes D, Pestilli F, Face percept formation in human ventral temporal cortex. J Neurophysiol. 2017;118(5):26142627.

  • 33

    Blakely T, Miller KJ, Rao RP, Localization and classification of phonemes using high spatial resolution electrocorticography (ECoG) grids. Paper presented at: 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; August 21–24, 2008; Vancouver, BC, Canada.

    • Search Google Scholar
    • Export Citation
  • 34

    Bouchard KE, Mesgarani N, Johnson K, Chang EF. Functional organization of human sensorimotor cortex for speech articulation. Nature. 2013;495(7441):327332.

    • Search Google Scholar
    • Export Citation
  • 35

    Anumanchipalli GK, Chartier J, Chang EF. Speech synthesis from neural decoding of spoken sentences. Nature. 2019;568(7753):493498.

  • 36

    Feigin VL, Nguyen G, Cercy K, Global, regional, and country-specific lifetime risks of stroke, 1990 and 2016. N Engl J Med. 2018;379(25):24292437.

    • Search Google Scholar
    • Export Citation
  • 37

    Woods BT, Teuber H-L. Mirror movements after childhood hemiparesis. Neurology. 1978;28(11):11521157.

  • 38

    Miller KJ, Abel TJ, Hebb AO, Ojemann JG. Reorganization of large-scale physiology in hand motor cortex following hemispheric stroke. Neurology. 2011;76(10):927929.

    • Search Google Scholar
    • Export Citation
  • 39

    Hebb DO. The Organization of Behavior: A Neuropsychological Theory. Psychology Press; 2005.

  • 40

    Freudenburg ZV, Branco MP, Leinders S, Sensorimotor ECoG signal features for BCI control: a comparison between people with locked-in syndrome and able-bodied controls. Front Neurosci. 2019;13:1058.

    • Search Google Scholar
    • Export Citation
  • 41

    Wang W, Degenhart AD, Collinger JL, Human motor cortical activity recorded with micro-ECoG electrodes, during individual finger movements. Paper presented at: 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society; September 3–6, 2009; Minneapolis, MN.

    • Search Google Scholar
    • Export Citation
  • 42

    Wang X, Gkogkidis CA, Iljina O, Mapping the fine structure of cortical activity with different micro-ECoG electrode array geometries. J Neural Eng. 2017;14(5):056004.

    • Search Google Scholar
    • Export Citation
  • 43

    Kellis S, Miller K, Thomson K, Decoding spoken words using local field potentials recorded from the cortical surface. J Neural Eng. 2010;7(5):056007.

    • Search Google Scholar
    • Export Citation
  • 44

    Siero JC, Hermes D, Hoogduin H, BOLD matches neuronal activity at the mm scale: a combined 7T fMRI and ECoG study in human sensorimotor cortex. Neuroimage. 2014;101:177184.

    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1300 1300 417
PDF Downloads 659 659 174
EPUB Downloads 0 0 0