Silicon Valley new focus on brain computer interface: hype or hope for new applications?

In the last year there has been increasing interest and investment into developing devices to interact with the central nervous system, in particular developing a robust brain-computer interface (BCI). In this article, we review the most recent research advances and the current host of engineering and neurological challenges that must be overcome for clinical application. In particular, space limitations, isolation of targeted structures, replacement of probes following failure, delivery of nanomaterials and processing and understanding recorded data. Neural engineering has developed greatly over the past half-century, which has allowed for the development of better neural recording techniques and clinical translation of neural interfaces. Implementation of general purpose BCIs face a number of constraints arising from engineering, computational, ethical and neuroscientific factors that still have to be addressed. Electronics have become orders of magnitude smaller and computationally faster than neurons, however there is much work to be done in decoding the neural circuits. New interest and funding from the non-medical community may be a welcome catalyst for focused research and development; playing an important role in future advancements in the neuroscience community.

Abbreviations and Acronyms

3D, Three-dimensional

BCI, Brain-Computer Interface

BMI, Brain-Machine Interface

CNS, Central Nervous System

CPU, Central Processing Unit

DPU, Decoding Processing Unit

ECoG, Electrocorticography

EEG, Electroencephalogram

fMRI, Functional Magnetic Resonance Imaging

PET, Positron Emission Tomography

PNS, Peripheral Nervous System

TPU, Tensor Processing Unit

Introduction

In the last year, there has been an explosion of interest by entrepreneurs looking to become actively involved in developing devices to interact with the central nervous system. These have included the likes of Elon Musk (Neuralink Inc. California USA), Mark Zuckerberg (Facebook Inc. California, USA), Bryan Johnson (Kernel. California, USA) as well as dedicated startups such as Paradromics (San Jose, California, USA) or Cortera (Berkeley, California, USA), and even DARPA (Defense Advanced Research Projects Agency. Virginia, USA), spurred on in part by the BRAIN initiative [1]. Each of these individuals and their respective companies share a particular focus in developing a robust brain-computer interface (BCI). We define BCI, for the purposes of this discussion, as a technological system designed to provide a stable mapping and modulation of activity within neural networks of the central nervous system. Therefore, at the very minimum, a working BCI will require both a physical interface to the brain (brain-machine interface; BMI) and computer systems that can process high bandwidth signals in real-time.

It is important to distinguish that there are very different engineering and neurological challenges between building BCIs for the peripheral nervous system (PNS) and central nervous system (CNS). In particular, space limitations for processing units, isolation of targeted structures, replacement of probes following failure, and delivery of nanomaterials in vivo [2, 3]; for the purpose of this commentary we will focus on the CNS as this is an area of particular interest by the entrepreneurs highlighted above.

Understanding the information transfer and processing of the nervous system is one of the most urgent challenges faced by the biomedical community, with a plethora of academic and clinical applications, including better understanding of aging, neurodegenerative diseases and interfaces for prosthetics and implants. For example, recent advances in chronic neural recording devices have facilitated the willful control of robotic prosthetic limbs for the treatment of paralysis [4] and improved seizure prevention with chronic telemetry in refractory epilepsy [5, 6]. There are many different kinds of potential BCIs that will each serve independent functions, however all systems must tackle three fundamental problems: how to accurately record information from relevant neural systems, how to decode such information, and how to stimulate and manipulate neuronal dynamics in an appropriate and meaningful way.

Neural engineering progress

The origins of neural engineering stretch back to early attempts to record activity chronically in the 1950s when electrodes were implanted into the cortex of rhesus monkeys to measure electrical activity in the central nervous system [7, 8]. Great innovations have been made in neural recording techniques, which have allowed the number of simultaneously recorded neurons to double approximately every 7 years [9], mimicking Moore’s law albeit at a much reduced rate [10]. Early clinical applications of BMIs centered on the restoration of perceptions to patients with sensory deficits. One of the pioneering studies was the work on potential cochlear implants in the 1970s that eventually reached life-changing reality in the 1980s for patients [11– 13].

In parallel to the development of the cochlear implant, researchers worked with the CNS by applying electrical current to the visual cortex of blind patients through grids of surface electrodes implanted over the visual cortex, thus developing visual prostheses [14, 15]. These systems allowed blind subjects to learn to recognize simple visual objects [16]. Neural engineering continued to improve with multi-channel neuronal recordings allowing owl monkeys [17] and later humans [4] to control two- and three-dimensional movements of a robot arm with multiple degree of freedom. Neuro-prosthetic research has undoubtedly benefited from these advances, but additional design parameters need to be included for effective long-term operation and clinical translation of neural interfaces.

While research in neural engineering has been steadily improving the bandwidth of BCI interfaces, the pace of this exponential increase falls far short of that seen in the silicon chip industry [9]. At current pace, the goal set by DARPA of recording from 10 6 neurons simultaneously would not be expected to be reached for around 80–100 years. Increasing interest and funding from members of Silicon Valley may prove to be a useful catalyst for the field and promote investigation of new applications of BCIs. For example, Facebook Inc. is investigating methods of non-verbal communication that will not require the virtual keyboards that are currently being used by patients with BrainGate [18].

Challenges

Despite advances in recent years, implementation of general purpose BCIs faces a number of constraints arising from engineering, computational, ethical, and neuroscientific factors. The future success of BCI is often imagined as a function of the capability to produce multi-electrode arrays with a greater and greater density of recording sites. Here, we outline several other challenges that must be overcome in parallel if BCI is to become of more than limited interest.

Perhaps the most immediate barrier to wider usage of BCI systems is the difficulty in implanting them. Non-invasive modalities, such as electroencephalogram (EEG) but also positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) lack the spatial resolution to record detailed activity at the level of the neuronal circuit, and so can only be used for very simple low bandwidth (typically binary choice) interfaces. There is no technology currently available that can record an action potential without the need for major surgery, although research into less invasive endovascular electrodes [19] and surface electrocorticogram (ECoG) devices is ongoing [20]. Furthermore, the quality of recordings obtained via implantable electrodes degrades over time due to a combination of gliosis [21, 22], neuronal depletion [23, 24], and degradation of the system itself [22, 25, 26]. This tends to limit recording times to a period of months or a few years at most, although the use of compliant materials [27] or soft ultra-thin wires [28, 29] designed to reduce mechanical shear has shown promise in reducing these effects.

By definition, detecting neuronal signals constitutes only one half of the BCI. These signals must then be able to be communicated to a computer via either a wired or wireless connection. This poses further challenges, necessitating a tunneled wire through the cranium. Wireless systems avoid this challenge, but create a host of new problems in turn including available bandwidth, safety, and the need for an implantable battery – which may last only a few months powering a large BCI system [30, 31]. To give a sense of the challenge here, we calculate that a 100,000-electrode system would require a communication protocol at least as fast as a Thunderbolt TM 3 connection (Apple, Inc. & Intel, Inc.), currently the fastest available consumer-level wired standard. The required bandwidth could be reduced drastically by on-chip processing, reducing the dimensionality of the data, but this in turn requires vastly more complex devices, limiting the number of electrodes per device, and greatly increasing its volume – a critical flaw in any proposed intracranial device. Furthermore, onboard processing of any kind poses serious and mostly unexplored challenges in terms of the energy dissipation required to maintain the device at body temperature so as not to cause thermal damage to the brain.

Current multi-electrode array systems offer up to around one thousand recording channels [32], in turn providing monitoring for hundreds of neurons from a single area [33], sufficient for the control of several univariate parameters. More general purpose BCI will require the sampling of tens if not hundreds of thousands of units, potentially from multiple cortical regions. This poses engineering and surgical challenges far beyond what is currently achievable.

Computational and data analysis challenges arise from the highly parallel nature of multiunit recordings. In general, there are four steps utilized to decode neural activity. Firstly, the signal must be filtered to remove extraneous noise. Secondly, spikes must be detected. Thirdly, these spikes must be ‘sorted’, typically by waveform, in order to be assigned to ‘units’ – putative single neurons. Lastly, the inferred population spike train must be decoded in order to provide a control signal. Whilst the first and second of these steps are essentially solved, for sufficiently high signal-to-noise systems, spike sorting is still an area of active research [34, 35], with no clear optimal solution, and often relies on semi-automated systems that require a great deal of human input to fine tune. Spike sorting may not be strictly necessary for the training of accurate decoders, as the raw spatiotemporal pattern of activity may suffice, but this may in turn reduce the dimensionality of the data.

Real-time processing of highly parallel recording systems remains a key challenge in the field. Promising technologies include a move away from general-purpose central processing units (CPUs) to application specific integrated circuits designed to perform a limited number of operations, such as Google’s tensor processing unit (TPU) or the graphical processing chips found in most computers. It is not unreasonable to suspect that the solution to decoding neural activity may lie in dedicated ‘decoding processing units’ (DPUs).

The physical scalability of BCI systems also poses a profound challenge. The brain is a three-dimensional (3D) structure. Unlike silicon wafers, manufacturing devices with a complex 3D structure and including integrated electronics poses a particular problem. Furthermore, current designs of multi-electrode arrays are typically not well suited to rapid scalability, requiring extensive redesign for each generation of device.

Even if this problem can be overcome, it may seem intuitive that more units result in greater bandwidth, however, the distributed nature of cortical processing has actually shown to result in a decreasing marginal value of each additional unit in terms of information retrieval [36]. Therefore, the common mantra that more units results in more information does not follow, at least not proportionally. We simply do not understand well enough the nature of distributed information representation and processing in the neocortex to be able to make more than a rudimentary estimate of what a particular sequence of activity might ‘mean’.

Conclusion

The literature has shown large decades of neuroscience research efforts in developing tools to probe the signaling complexity of the nervous system, with several clinical applications being developed. Although orders of magnitude smaller and computationally faster than neurons, our electronics cannot mimic the complexity of neural systems. Current understanding of the function of neural circuits could be compared to trying to understand the internet by means of a few dozen well-placed potentiometers in the data centers of service providers. This is not to disparage the efforts of neuroscientists, far from it, but rather to underscore that decoding neural circuits ranks among the deepest and most complex contemporary endeavors, and it will not be solved overnight by Silicon Valley enthusiasm and zeal alone. However, we consider that many of the engineering challenges outlines above are amenable to focused research and development, particularly those surrounding miniaturization and parallelization of recording systems. We support the interest of entrepreneurs in placing their focus on the neuroscience community, and we look forward to the future advancements that will undoubtedly be realized in the coming years.

Data availability

No data are associated with this article

This article is a very good summary of the state of play in BCI research as of 2019. It will be of interest to the BCI and general neuroscience communities.

Numerous funding sources with deep pockets, coming from very well established, and successful companies, start-ups, as well as the government signal that the time for BCI breakthroughs is clearly anticipated.

It may be appropriate to mention some of the already met needs of some users, such as the various P300 Speller systems in use by patients with motor neuron diseases. This population lacks any other communication means, and recent studies with ALS subjects’ use of BCI in their homes have shown reasonable success.

The authors have touched on the major and significant challenges. The challenge of safe, easily deployed high density electrodes, disposed in some 3D configuration inside the brain is extremely great. The reliable readout of these electrodes, and their connection to some external computing means is similarly difficult. Integrating the computational means with the sensors is clearly desirable but very difficult.

For fifty years this reviewer has seen technologies compared to that of the transistor, which grew from transistor radios with seven individual transistors, to 100 million transistors per square millimeter today. Approximately a 5e6 fold improvement.

Unfortunately no other technology has had a similar arc, and BCI is unlikely to be similarly blessed with inexpensive scalable improvements.

Some comment on the possibility of using Artificial Intelligence of the type currently in use to learn, master and dominate games of chess and GO.

Recommend indexing.

I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

The intention and the action performed to support the intention of the human-being is supported by the impeccable coordination of the peripheral nervous system (PNS) and the central nervous system (CNS). Any disruption in this coordination, for example dysfunction of afferent or efferent pathways or injury in the spinal cord or neurological disorders affecting the functioning of brain, affects the normal functioning of the human body. The individual becomes paralyzed and is unable to perform the simple day to day activities of walking or talking. Brain computer interfaces (BCIs) have been developed to help such individuals, where BCIs aim to bypass the dysfunctional pathways and interface the external mechanical or electrical devices with the functioning brain of an individual. Both non-invasive and invasive BCIs have been developed: in non-invasive BCIs non-invasive neuroimaging techniques are being used to acquire brain signals from the surface of the scalp while in invasive BCIs electrodes are placed on the cortical surface of the brain or inserted in the brain. The invasive technique where the electrodes known as microelectrodes are inserted in the brain records spike signals either from a single neuron, known as single unit activity (SUA) or from a group of neurons, known as local field potentials (LFPs).

In this article the authors have done a great job in summarizing the technical challenges faced by researchers during recording neural signals invasively and have discussed the different approaches developed to solve these problems. Given the recent media interest in the application of BCI as a tool for the means of communication and rehabilitation of paralysed people, several entrepreneurs have invested huge resources in developing BCI. The authors have applauded such efforts and have presented a bright outlook on the effect of such interests on the development of BCI for real world applications.

I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

This is a concise review by Mitrasinovic et al. concerning the evolution of Brain Computer Interfaces (BCIs), the current ‘state-of-the art’ of BCIs and the future challenges particularly in relation to electrode design, electrode placement and signal processing.

It is not immediately apparent to us why Silicon Valley is in the title. More should be made of this in the introduction. Specifically, in what areas do the authors believe that Silicon Valley can help advance BCIs to clinical reality? Are the private entrepreneurs mentioned in paragraph 1 based in Silicon Valley? A greater review of companies involved in BCI research may even be warranted. There are many research groups outside Silicon Valley where advances in computer and electronic engineering for BCIs are also taking place.

Nanomaterials are mentioned in paragraph 1 but with no explanation as to how these are used in BCIs or why they would replace materials already in use. The discussion on nanomaterials could be moved to the section discussing improvements in electrode design.

While recording electrode numbers have roughly doubled every seven years, is this the case for BCIs? The challenges in recording acutely from non-human primates with non-FDA approved electrodes is very different from the goal of BCIs in humans. At a minimum this difference should be highlighted to not give the readers an unrealistic view of progress.

The difference between peripheral and central nervous system is highlighted but what about the major differences between non-invasive EEG platforms and invasive implanted electrodes on the surface or penetrating the brain? This should also be highlighted. There is rapid development of non-invasive EEG recording interfaces with the brain which avoid the inconvenience, risks and costs of surgical implantation. Will advances in signal processing increase the accuracy of these non-invasive BCIs and lessen the applications or need for implanted BCIs? It is difficult to imagine how EEG interfaces could compete with the implanted BCIs on the basis of the volume, precision and reliability of the information being transferred.

PET and fMRI are mentioned as modes of BCI in addition to EEG.  These are not relevant to developing clinically- and commercially-relevant BCIs for ambulant individuals and we suggest that these modalities be deleted. At present the only way to activate neurons noninvasively is with transcranial magnetic stimulation (TMS).

Paragraph 2, page 4: The description of detecting neural signals only being “one half of a BCI” seems a little oversimplified. Not only do you need to record neural signals and decode them but next you need to use these signals to control an output such as a cursor on a screen or robotic limb and then also provide accurate feedback to the patient via electrical stimulation or other means. These challenges should be discussed.

Paragraph 2, page 4: We would submit that wired implanted BCIs with a connector penetrating the scalp have no future as a permanently implanted device because of infection risk and inconvenience. Implanted BCIs must become wireless if they are to have any physician or patient uptake. Wireless devices are already described, for example in Lowery et al. (2015 [1]), Rajangam et al. (2016 [2]) and Vansteensel et al. (2016 [3]). We agree there are challenges as the number of electrodes increase.

Paragraph 3, page 4: The number of electrodes required to adequately perform certain tasks is not known. Detailed vision, speech processing and fine motor control and the encoding and manipulation of memory would likely require significantly more electrodes than are currently available. However, vast increases in electrode numbers may not be required for all BCIs. The challenges of recording from ever increasing numbers of neurons has been laid out, but the challenges in basic neuroscience in understanding the basic coding of neurons in controlling movement should also be highlighted, for example, little is still known about the coding of control for grasping in the dorsal and ventral pre-motor cortices so this lack of knowledge affects our ability to extract information from these recorded populations of neurons.

Paragraph 4, page 4: The four steps that you outline to decode neural signals are focused on spike decoding, whereas the paragraphs before outline techniques that will not result in spike recordings, e.g. EEG, ECOG, endovascular devices. A broader description on decoding algorithms that includes the use of low frequency continuous signals such as the Local Field Potentials (LFP) possibly separated in step called “feature extraction” is needed. Before neural signals can be decoded, algorithms also need to be trained which is not a trivial problem for the target patients.

Paragraph 2, column 2, page 4: A large push in BCIs is getting the hardware necessary to be small enough and run on low power to allow patients to be mobile. In the description of parallel processing and Central Processing Units (CPUs), this challenge should be discussed.

The future design of BCIs using light or magnetic energy as an alternative to electricity could also be included in the discussion.

Mention could also be made of the surgical risk of implantation which includes haemorrhage, epilepsy and infection. The mitigation of risk needs to be factored in to the design of the devices and included in the informed consent process. It is important for physicians to work alongside engineers and scientists in the development of BCIs so that they are as safe and practical as possible.

A mention of the many ethical challenges such as informed consent, agency, stigma, equity, neural enhancement, privacy and security of data is important in a general review such as this. For example, there are major ethical challenges to apply BCIs in severely disabled individuals to allow communication or control of assist devices such as locked-in syndrome or advanced amyotrophic lateral sclerosis (ALS).

We have read this submission. We believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.