Chronic, wireless recordings of large-scale brain activity in freely moving rhesus monkeys.

Advances in techniques for recording large-scale brain activity contribute to both the elucidation of neurophysiological principles and the development of brain-machine interfaces (BMIs). Here we describe a neurophysiological paradigm for performing tethered and wireless large-scale recordings based on movable volumetric three-dimensional (3D) multielectrode implants. This approach allowed us to isolate up to 1,800 neurons (units) per animal and simultaneously record the extracellular activity of close to 500 cortical neurons, distributed across multiple cortical areas, in freely behaving rhesus monkeys. The method is expandable, in principle, to thousands of simultaneously recorded channels. It also allows increased recording longevity (5 consecutive years) and recording of a broad range of behaviors, such as social interactions, and BMI paradigms in freely moving primates. We propose that wireless large-scale recordings could have a profound impact on basic primate neurophysiology research while providing a framework for the development and testing of clinically relevant neuroprostheses.

INTRODUCTION

Single-unit recordings from cortical neurons in awake, behaving monkeys were pioneered by Edward Evarts in the 1960s[1], paving way for more than 5 decades of experimental studies focusing on the neurophysiology of the non-human primate’s brain in action. Until the late 1990s, technological limitations allowed researchers to sample just from one neuron at a time or, in rare situations, from a few neurons simultaneously. The introduction of chronic multielectrode implants[1-4] and the development of computer technologies for online information processing and analysis allowed several advancements in the field of primate neurophysiology, such as: recording simultaneously from many neurons for extended periods of time[5,6], extracting behavioral parameters from neural signals in real time[7], and using brain-derived signals to control external devices through brain-machine interfaces (BMIs)[8,9]. Altogether, these developments transformed chronic brain implants into one of the most pervasive experimental approaches employed by system neurophysiologists.

Since only tens of cortical neurons can be routinely sampled simultaneously in macaques (a miniscule fraction of the hundreds of millions of neurons that form the monkey cortex [10]) new neuronal recording methods are required to advance basic brain research, large-scale brain mapping, and clinical translation of BMIs[11]. Accordingly, significant improvements in brain recording technology are needed before BMIs can become clinically relevant for helping severely disabled patients regain mobility[11-13]. For example, our estimates indicate that a BMI aimed at restoring limb movements may require 5,000–10,000 neurons recorded simultaneously [14], whereas 100,000 neurons will be needed to drive a BMI in charge of producing full body movements [13].

Furthermore, tethered recordings in experimental animals have also limited the range of natural behaviors that can be studied, particularly in non-human primates. The transition to using a low-power, implantable, wireless interface is imperative for the success of experiments aimed at recording large-scale brain activity in behaving primates. In response to this need, several multichannel wireless recording systems have recently emerged[15-18]. However, to date, no system has been shown to be scalable in the number of recording channels.

Here we introduce an integrated paradigm for chronic, multichannel, multi-site, wireless large-scale recordings in freely roaming primates. We report the first volumetric recording probes with thousand-channel capacity, evidence of close to 5 years of continuous recordings, and the first scalable wireless recording interface validated in naturally behaving, unrestrained monkeys.

RESULTS

Chronic multichannel implants

Our results were obtained in eight adult rhesus monkeys (Table 1). Five monkeys received movable volumetric implants in multiple cortical areas of both cerebral hemispheres. Additionally, we present data from three rhesus monkeys (Monkeys I, G, Cl,) implanted with previous-generation microwire arrays, composed of fixed (non-adjustable) microelectrodes. We also show the latest version of our movable volumetric implants, called recording cubes (Fig. 1A). Each of these cubes is built by first creating an array of polyimide guiding tubes, spaced at 1 mm apart (4×10 or 10×10 arrangement). Each guiding tube accommodates bundles of 3–10 different length microwires (Fig. 1A). Each bundle includes a single leading microelectrode with a conical tip; the remaining microwires have cut angle tips. We call these implants volumetric because they record from a volume of cortical (or subcortical) tissue (Supplementary Fig. 1). The microelectrodes are made of stainless steel microwires, 30–50 µm in diameter, with polyimide insulation that leaves the tip exposed. The guiding tubes are fixed in a 3D printed plastic case, which also holds miniature screws for positioning the microelectrodes. The resulting recording cubes are light and compact: a fully assembled unit weighs 11.6 g and its surface area per channel equals ~0.22 mm2. A total of 4–8 recording cubes can be implanted per monkey (Supplementary Fig. 2).

Table 1

Implant Summary

Subject	ChannelsImplanted	Time withimplant(months)	MaxUnits isolated	Units permicrowire(global avg)	MaxSimultaneouschannels(wired)	MaxSimultaneouschannels(wireless)	References of studies
MonkeyO	1792	13	1,874	0.522	512	N/A	This work
MonkeyC	768	29	881	0.574	512	512	Ifft et al., 2013 [24]
MonkeyK	576	29	483	0.412	128	256	This work
MonkeyM	384	54	490	0.640	384	128	O’Doherty et al., 2011 [22]Shokur et al., 2013 [23]O’Doherty et al., 2012 [37]Ifft et al., 2011 [38]
MonkeyN	384	54	357	0.465	384	N/A	O’Doherty et al., 2011 [22]Shokur et al., 2013 [23]O’Doherty et al., 2012 [37]Ifft et al., 2011 [38]
MonkeyI	160	20	224	0.698	128	N/A	Fitzsimmons et al., 2009 [39]
MonkeyCl	160	12	77	0.240	128	N/A	Li et al., 2009 [29]Lebedev et al., 2008 [20]O’Doherty et al., 2009 [40]
MonkeyG	128	7	114	0.448	128	N/A	Li et al., 2009 [29]Lebedev et al., 2008 [20]O’Doherty et al., 2009 [40]

Figure 1

Recording Cubes and Primate Headcap

(a) Schematic drawings and photographs of actual microwire cubes, showing the wire density and adjustment mechanisms. The 3rd generation is a 10×10 array that is fitted with bundled microwires (right panel). (b) Photographs of final arrangements of connectors, showing a view from the top of the headcap of a monkey implanted with 2nd gen cubes (left) and a similar view of the VLSBA module on monkey implanted with 3rd gen cubes (middle). The right panel shows the wireless module mounted on the headcap. (c) Layered schematic drawing showing the modular headcap assembly, with the VLSBA connector organizer as the active module. (d) Schematic drawing showing the modular headcap wireless assembly on the right panel. Both headcaps are 3D printed.

During an implantation surgery, the guiding tube array is fixed in light contact with the cortical surface, without penetrating the brain. Several days later, the microwire bundles are advanced into the cortical tissue by rotating a set of microscrews. Depending on the design, each microscrew turn advances microelectrodes housed in one (Supplementary Fig. 3) or several guiding tubes (Fig. 1A). In our previous work with fixed arrays, the microelectrodes were inserted in the cortex during surgery, and although histology performed on animals with these arrays showed little damage to cortical layers (Supplementary Fig. 4), these implants were less durable than our moveable arrays[6]. Using movable arrays, we have learned that penetration with subsets of microelectrodes reduces dimpling of cortical surface and copes with the “bed of nails” effect, which often hinders penetrations with a large number of microelectrodes simultaneously in traditional arrays and silicon probes. In our implantation approach, each microelectrode bundle is lowered gradually over the course of several days. This allows microelectrodes to be positioned slower and also much deeper into the cortical tissue (i.e. layer VI). Additionally, since our chronic cortical recordings have lasted for close to 5 years in our monkeys (see below), it is likely that our implantation paradigm also maintains healthier cortical tissue.

After the recording cubes are fixed in place with dental cement, a custom designed 3D printed headcap is added to the implant to provide protective housing for the microelectrodes and electronics (Fig. 1C,D). The headcap is fitted in a modular manner. First, a conical base module is fixed on the top of the skull, which protects the implants and their connectors. Depending on the application, several additional modules can be attached to this base to accommodate electronics. For example, a wireless-compatible module houses radio transceivers and their power source (Fig 1D). Omnetics 36-pin connectors are employed to connect both external cables and the wireless headstages. For all monkeys except Monkey O, the connectors were fixed with dental cement in between the cubes (Fig 1B, left panel). For Monkey O, implanted with 1,792 microwires, a special module organized all connectors (Fig. 1C).

Multichannel recordings

Monkeys M and N were the first animals to be implanted with four recording cubes containing 96 microwires each (totaling 384 microwires per monkey). The cubes were placed bilaterally in the primary motor (M1) and primary somatosensory cortices (S1), (Supplementary Fig. 2). Then, using a more compact revision, six cubes were implanted bilaterally in the arm and leg representation areas of Monkey K’s M1 and S1, totaling 576 channels. Monkey C was implanted with eight of these recording cubes, bilaterally in supplementary motor area (SMA), M1, posterior parietal cortex (PPC), dorsal premotor cortex (PMd), and S1, totaling 768 channels (Supplementary Fig. 5). The latest revision allowed us to implant four 448-channel cubes, containing a total of 1,792 microelectrodes, in four cortical areas (448 microwires in each area, S1 and M1 in both hemispheres) of Monkey O. One month after the original implantation surgery, extracellular neuronal recordings were obtained from these implants through four 128-channel recording systems (Plexon) (Fig. 2A, B), for a total of 512 channels recorded simultaneously. A total of 1,874 single cortical units were isolated in Monkey O at the end of four consecutive daily recording sessions (each day recording 512 channels). 968 units were recorded in M1 (403, left hemisphere, 565 right hemisphere) and 906 units in S1 (254 in the left hemisphere, 652 in the right hemisphere) (Fig. 2B, C).

Figure 2

Large-scale activity Recordings

(a) Representative mean waveforms for 1,874 neurons over a center-out task. Intermittently colored to indicate array of origin. Green highlight: neurons from left hemisphere M1. Orange highlight: left hemisphere S1. Blue highlight: right hemisphere M1. Yellow highlight: right hemisphere S1. (b) Population raster plot depicts 1 second window of neuronal spiking activity for a single recording session (556 neurons). The vertical color key identifies the recorded areas. 128-channel Plexon system was used for each recorded area. (c) This panel shows unit isolation clustering using the first two principal components of a subset (50) of channels ) from Monkey O.

Long-term recordings

The longevity of our implants was assured through sterile surgery performed by an experienced neurosurgeon, regular (1 mm) spacing between implanted guiding cannulas, and slow insertion of microelectrodes into the brain. In this sample, Monkeys I, Cl and G had older generation fixed arrays, while Monkeys M, N, C, K and O had the movable recording cubes. Altogether, this approach has yielded high quality neuronal recordings from eight rhesus monkeys. To our knowledge, our recordings [6,13,14,19-24] supersede the longevity reported with silicon probes[25] or any other method thus far. This finding is further illustrated by a 4-year (2009 to 2013) sample of neuronal waveform traces, obtained from Monkey M and N (Fig. 3). At the time of writing, the recording duration in these animals has reached close to 5 years. Additionally, Monkeys K and C (both females), have been recorded for over 29 months. Monkey O (female), implanted with the most advanced arrays has been recorded for 13 months. Monkeys I, Cl, and G, were recorded for 20, 12 and 7 months respectively. Generally, the average yield (Fig. 3, Table 1) for stable implants remained around 0.5–1 units (over time) per microwire, a value that agrees with our previous results obtained using adaptions of this methodology in rodents [4] and human patients [26,27].

Figure 3

Recording Longevity

Neuronal recording yield over time measured by quantifying the average number of single units recorded per connector (32 channels). All the data from the 5 monkeys used in these experiments, and three other monkeys (T, I, and Cl), were used. Four-year sample waveforms, identified by date, from Monkey M’s left hemisphere M1 connector surround the figure showing mean sample waveforms. Highlighted area corresponds to the waveform’s standard deviation.

Large-scale data processing

Processing large-scale data sets is a computationally demanding task. To handle real time operations, we had to develop routines to process continuous streams of neural data and behavioral parameters. We also had to find a way to scale the number of neural channels. The first issue was solved with the development of our real-time BMI suite (Table 2) – a software package written in C++ for the Windows platform – that implemented all basic components needed to process large-scale activity and route it to control external devices (Supplementary Fig. 6). For the second issue of input scaling, we developed approaches for both tethered and wireless recordings. In our tethered recordings, that sample the full waveform of all channels, we developed networked spike acquisition software, which aggregated spikes acquired from multiple computers. In our wireless recordings, the system was designed to obtain waveform samples solely at the start of the session, when spike sorting parameters were generated. This allowed significant scaling to be possible due to the low bandwidth used.

Table 2

BMI Specifications

Time delay	<1 ms
Time synchronization accuracy	~0.3 ms
Bandwidth	130 ksps
Sampling rate	1–100kHz

Wireless recording system

Our wireless recording system was designed to match typical BMI needs (Table 3). This system consists of four components: digitizing headstages, a wireless transceiver, a wireless-to-wired bridge, and client software (Fig. 4, Supplementary Figs. 7, 8). Four such headstages are attached to a transceiver module for a total of 128 channels per transceiver unit (31.25 kHz sampling per channel, 1.3 kHz sampling of spike time onset). Multiple 128-channel modules can be stacked together to scale up the final system. Our system’s novel feature, bidirectional communication, enables spike sorting to be performed on the transceiver itself, thereby saving both power and wireless bandwidth. This is accomplished through key elements of the transceiver: the radio and the digital signal processor (DSP). The DSP performs signal conditioning, spike sorting, and radio control, to match the radio bandwidth with the 48 Mbps aggregate rate of data acquisition. Each transceiver is powered by a 3.7V 2000mAh Li-ion cell, and consumes approximately 2 mW per channel, or ~264 mW per 128 channel unit, allowing continuous operation for over 30 hours. The wireless-to-wired bridge takes incoming radio packets and bridges them with an Ethernet interface. Overall, this signal chain is optimized in both temporal and power domains, allowing for the increased number of channels. Additionally, because of the low bandwidth requirements, each transceiver uses a small slice of the ISM band. This allows significant scaling on the number of recording channels. We benchmarked our wireless system performance using four transmitters, or 512 channels simultaneously (Table 3), by measuring packet-bit drop errors (PER/BER) versus transmission range. Overall, the system performed within its optimal range (3m) with all 4 modules (Supplementary Fig. 9A). We then used this 512-channel system to record brain activity while Monkey C reached for grapes. Out of the 512 wirelessly recorded channels, we isolated a total of 494 units (103 M1 units, 102 S1 units, 98 SMA units, and 191 units in PPC). Cortical units exhibited typical firing rate modulations as the animals produced arm reaching movements (Supplementary Fig. 9b). For other experimental validation, two transceivers per monkey were used (i.e., 256 channels).

Table 3

Wireless Specifications

Radio band	ISM (2.54 GHz)
Channels	128 per module
Maximum channels	2560
Sampling rate	31.25 ksps/ch
Sorting	Templates
Templates per channel	2
Power consumption	264 mW
Battery life	>30 hrs
Input referred noise	4.9 µ Vrms
Analog gain	200
ADC resolution	12 bits
LSb of ADC	3.66 µV
Radio bandwidth	Out: 1.33 MbpsIn: 83.3 kbps
Dimensions	Headstages: 1.37 × 2.15 × 0.16 cmTransceiver: 4.3 × 2.1 × 0.34 cm

Figure 4

Large Scale Wireless System

Schematic diagram showing the information flow of the wireless recording system, beginning with spike waveforms to spikes transmitted across ISM radio band to the client. (a) Exploded diagram of wireless transceivers located inside the headcap. (b) Wireless bridges showing the main components of bidirectional communication and the connections to the client computer. (c) Photograph of the radio transceiver with scale. (d) Photograph of the wireless bridge, with scale. (e) Screenshot of the wireless client with four channels visible, demonstrating the PCA sorting method and graphical user interface.

Wireless BMI

The real-time operation of our wireless system in a typical BMI setup was tested in Monkey M (128 channels in one group of experiments and 256 in the other), Monkey C (256 and then 512 channels) and Monkey K (256 channels). These monkeys performed several BMI and behavioral tasks without any restraint. While inside a large Plexiglas enclosure (Fig. 5A), Monkeys C and M learned to use their brain activity to move a computer cursor to execute a classical center-out task. Initially Monkey C employed a joystick to perform the task manually. The monkey was then required to perform through a BMI, using a Wiener filter as the decoder[28]. During brain control with hand (BCWH), the joystick was still accessible to the monkey although it was disconnected from the cursor control. During brain control without hand (BCWOH), we removed the joystick from the setup, forcing the monkey to perform the task using its cortical activity alone (Supplementary Video 1). A total of 212 cortical units recorded from M1 (84 and 26 units in the left and right hemispheres, respectively), SMA (27 and 24 units) and left hemisphere S1 (51 units) were employed. Performance improved (reaching >80% correct trials) within 7 days (Fig. 5B). Population perievent time histograms (PETHs) for all recorded neurons, centered on target onset, show neuronal adaption to each task component (Fig. 5). Cortical ensembles exhibited similar patterns of modulation in BCWOH and HC, periods during which the animal moved the joystick. Different neuronal modulation patterns were observed in BCWOH, likely due to decreased body movements. In particular, many units were prominently inhibited prior to target onset. Additionally, excitatory neuronal modulations occurred during reaching for all modes of operation, but had lower amplitude and longer duration during BCWOH. This task was then replicated using a powered wheelchair as an end effector (Supplementary Video 2).

Figure 5

Wireless Brain Machine Interface

(a) Screenshot of monkey with wireless system performing a task using pure brain control. See Supplementary Video 1. (b) Performance metric increase in a single week of training. (c) Example average traces the y-coordinate of the cursor for trials with target at 0 degrees (North) of orientation, showing average constant movements despite variable postures during task interaction, as well as time-to-target. Peaks indicate reached target. (d) Peri-event time histograms centered on target onset for 212 units, grouped at each column with cursor control paradigms of basic hand control (HC) with joystick, brain control with joystick (BCWH), and sole brain control (BCWOH), respectively. Color key shows unit groupings per area, and include LS1, LM1, LSMA, RM1, and RSMA. Each unit was normalized per condition using z-score method.

To address the relevance of the number of recording channels for BMI decoding, we conducted an offline analysis known as neuron dropping curves[7] by randomly removing units from the training set of the decoder (Unscented Kalman Filter[29]). The performance of the decoder was measured as the correlation coefficient (r) between decoded and measured variable. We show neuron dropping curves for two monkeys (O and C) performing the center out task (Supplementary Fig. 10), and two monkeys (M and N) performing a bipedal locomotion task (see Supplemental Fig. 11A) on a treadmill. For all tasks and monkeys, model performance increased linearly as a function of the logarithm of the total number of neurons[14].

Wireless recordings in freely moving monkeys

We also explored the capacity of our wireless system to record from freely moving monkeys in an exploratory foraging task. Monkey K and Monkey C were allowed to freely move about an experimental room (Supplementary Video 3) while picking grapes from locations on the room floor and on suspended planks. Recordings included a total of 247 units from Monkey C (193 in M1, 64 in S1), and 156 units from Monkey K (107 in M1, 49 in S1). Our wireless system recorded reliably inside the 3m range of the recording equipment (Fig. 6A, Supplementary Video 2). Population PETHs aligned on the onset of different behaviors showed distinct neuronal modulation patterns (Fig. 6C). We then used several clustering methods (k-means, expectation-maximization, and support vector machines (SVM, linear kernel and radial basis function) to discretely classify several observed behaviors. All four clustering methods employed yielded good performance (Table 4) in correctly identifying six selected behaviors (Fig 6B) based on the combined electrical activity of cortical ensembles.

Figure 6

Freely Behaving Recordings and Future Arrays

(a) Screenshots of videos used for coding of behaviors. Two frames for three sample behaviors are shown. See Supplementary Video 3. (b) Plot of the first three principal components of neural activity for 6 different behaviors, color coded per behavior, shown as filled in circles. PCA data was used for SVM classification (see Table 5). (c) PETHs for behaviors from a single monkey, calculated from 107 M1 neurons centered around behavior onset. Legend for spectrograms placed on the bottom right corner to reduce figure clutter. Each unit was normalized for the entire session (all behaviors shown) using the z-score method. (d) 864 channel array assembly modules. Left panel shows the bottom view with guiding tube density. Right panel shows the top view with a depiction of the movable assemblies. (e) Photograph of fully assembled array with guiding tubes. Note the high density connector attached. (f) Connector and cap design. Left panel shows schematic for high density connectors. Right panel shows a redesign of the cap which should allow for the management of 10,000 channels without increasing the cap footprint. (g) Panel shows photograph of plastic cap mounted with a single connector. The fully assembled cap will host 12 connectors, each with 864 channel capacity.

Table 4

Clustering and classification performance

Algorithm	Performance
K-means	0.928
Expectation Maximization	0.886
SVM (linear kernel)	0.922
SVM (RBF kernel)	0.904

Further experimental validation was done in Monkey K while it performed an unrestrained locomotor task. The monkey walked both bipedally and quadrupedally on a treadmill while 176 units were wirelessly recorded from M1 (94 in the left hemisphere, 82 in right hemisphere). Population PETH analysis of left M1 neurons during right ankle swing phase revealed an inversion of firing patterns in a subpopulation of these cells (Supplementary Fig. 11).

Next generation implants

Continuous improvement of our arrays is necessary to further explore the relevance of large-scale recordings in BMIs and neurophysiology. We have introduced a manufacturing modification (Fig. 6D, E) that allows us to increase the number of microelectrodes per individual cannula in our recording cubes to up to 30 microwires. Using a new cap that accommodates close to 10,000 microwires (Figure 6F, G), we will be able to implant 8–12 cortical (subcortical) structures, extending from the frontal to the occipital lobes, in each hemisphere of an adult rhesus monkey.

DISCUSSION

To meet future experimental demands, chronic multielectrode implants should fulfill the following requirements: (i) produce minimal damage to neural tissue; (ii) maximize the number of simultaneously recorded neurons; (iii) sample from multiple brain areas; and (iv) maintain good recording quality for several years. Over the last 2 decades, our laboratory has progressively improved the performance of multichannel recording systems towards these goals through a series of technological developments and experimental tests in rodents, primates, and intra-operative human recordings[26,30,31]. Here we described a novel paradigm that allows wireless large-scale brain recordings to be carried out in freely roaming primates.

The first step of this new paradigm involved the introduction of volumetric recording probes, which can be used to sample from hundreds to thousands of neurons, distributed across multiple cortical areas per monkey. Using this new integrated methodology, we increased the number of cortical units recorded simultaneously per monkey in a single recording session (~500 units) [24], the total number of units isolated from a single animal (~2,000 units), and the longevity of chronic recordings (close to 5 years). To our knowledge, none of these milestones have been reached by another integrated system for chronic recordings in behaving monkeys. For comparison, the Utah silicon probe commonly used for BMI studies and other neurophysiological experiments in primates, provides fewer recording channels (~100), penetrates only the superficial cortical layers, does not allow adjustment of the microelectrode’s position after the implantation surgery, and cannot yield long-term cortical recordings beyond a few months[25,32].

The significant increase in the number of neurons recorded has not compromised the stability and reliability of our implants. Although we have not performed histology for the most recent implants, we have previously shown that implanted microwires produce minimal tissue damage[33]. This is further demonstrated by the fact that high quality neuronal data could be recorded from two monkeys for nearly 5 years and in another two subjects for nearly 3 years. While neuronal yield decreases during 1–3 months after the surgery, we showed that recordings stabilize at an average of 0.5–1 units per channel for many months or even years in most monkeys. A similar yield has been observed over the past 20 years in our laboratory in the recordings from rodents[4], owl monkeys[7] and human subjects[26,27].

Moreover, the performance of BMI decoders increases linearly with the logarithm of the cortical neuronal sample recorded simultaneously[14]. Therefore, increasing the number and long-term stability of multi-electrode recordings is imperative for the development of clinically relevant neuroprostheses driven by BMIs. Indeed, we have recently employed the technology described here to implement the first bimanual BMI, which utilized approximately 500 units to control a pair of virtual arm actuators [24]. By doubling or tripling the sample of neurons recorded simultaneously we envision that even richer BMI applications, such as a robotic exoskeleton aimed at restoring full body mobility, may become viable.

Previously, our group[17] and other laboratories[15,16,18] have described multichannel wireless recording technologies. While most of these systems improved the design, size, energy efficiency, and depth of implants of the components (Table 5), our current approach introduces several important innovations. The key innovation is its ability to maintain bidirectional wireless communication, an essential missing property in the literature. Bidirectional wireless communication is a crucial component for scaling up the number of channels recorded simultaneously, as it allows spike sorting performed on the implant and, therefore, reduces the amount of information transferred wirelessly. Using this approach, up to 512 channels of neuronal data were recorded in the present study. Overall, considering only the ISM band, we estimate that in the future up to twenty 128-channel transceivers could be used, each occupying 2MHz increments, for a total of 2,560 recording channels. Since two neurons could be isolated per channel, this technology could yield up to 5,120 neurons recorded simultaneously in a single monkey. However, a few bottlenecks, including size and power consumption, need to be addressed before reaching this benchmark.

Table 5

State-of-the-art wireless recording systems

Reference	No. ofChannels	Sampling(ksps)	Datadirection	Compression	Radio Band	Bandwidth	Noise(µ Vrms)	ADC (bit)	Power(mW)
Rizk et al., 2007 [41]	96	31.25	Downlink	Raw data	916.5 MHz	1 Mbps	--	12	100
Chestek et al., 2009 [15]	96	15.7	Downlink	Raw data	900MHz	345.6kbps	27.4	10	63.2
Sodagar et al.,2009 [42]	64	62.5	Downlink	Spike detection	4MHz	2 Mbps	8	8	14.4
Bonfanti et al., 2010[16]	64	20	Downlink	Spike waveform	400MHz	1.25Mbps	3	8	6.7
Roy & Wang, 2012 [43]	15	100	Downlink	Raw data	3.05 GHz	200 kbps	5.5	--	30
Borton et al., 2013 [18]	100	20	Downlink	Raw data	3.8 GHz	24 Mbps	8.6	12	90.6
Miranda et al., 2010 [44]	32	30	Downlink	Raw data	3.7–4.1 GHz	24 Mbps	3.2	12	142
This work	512*	31.25	Bidirectional	Onboard spikesorting	2.4 – 2.524GHz (ISM)	2Mbps OTA**/48Mbps aggregate	4.9	12	264

With four 128 channel transceivers used.

Over the air compressed data vs on unit data acquisition

Additionally, removing the need for hardware tethering is extremely advantageous for primate experiments, particularly when there is growing evidence that the context in which animals perform a given task dramatically influences the physiological properties of neural circuits[28]. For example, studies focused on how cortical circuits underlie natural behaviors such as tool utilization, social primate interactions and complex movements [34] are likely to benefit from our new recording paradigm. We have already started to investigate these kinds of questions using virtual reality [23,24] and continue development of new BMIs [35] such as the discrete classification of roaming behaviors shown in this study and, more recently, brain-to-brain interfaces[36].

In conclusion, use of a novel wireless large-scale recording approach as described here may prove ideal for studies of large-scale brain activity under a variety of normal and pathological conditions, as well as serving as both the neurophysiological and technological backbone for the development of future generations of BMIs and clinical neuroprostheses.

ONLINE METHODS

Animals

All animal procedures were performed in accordance with the National Research Council’s Guide for the Care and Use of Laboratory Animals and were approved by the Duke University Institutional Animal Care and Use Committee. Five adult rhesus macaque monkeys (Macaca mulatta, one male, four females) participated in the set of experiments (Monkeys M, N, C, K and O). Additionally, data from three monkeys (Monkey Cl, G, I, and T; two males, one female) was used in the longevity analyses. Histology data from one additional monkey (Monkey T, one female) was used in this report.

Surgery

Adult rhesus macaques were initially anesthetized with ketamine, and then intubated in the trachea. They were then placed in a stereotaxic apparatus and ventilated. General anesthesia was maintained with isoflurane throughout the surgery. A series of craniotomies were performed over the stereotaxically determined locations of cortical areas of interest. The dura matter was removed, and sulci landmarks inspected to confirm the location of cortical areas. Microwire cubes were brought in light contact with the pia matter and fixed to the bone with bone cement. Titanium screws were inserted in the skull to anchor the cement. Gelfoam coated in saline was inserted into the pia to sustain the surrounding tissue. After all recording cubes were implanted, a protective headcap was affixed to the screws and cemented to the skull. The cap encased the cubes and their connectors and provided housing for wireless components. The dura, dense connective tissue and, later, skull tissue grow back to capture and seal each of the microelectrode array/bundle penetration point. We have observed such tissue restoration in all implanted monkeys examined post-mortem. Because of these processes, the risk of infection growth around the microelectrode implants decreases significantly over the first few weeks after the implantation surgery.

Implantation

Once the animals recovered from the surgery (usually 1 week post operation), we began the insertion of the microelectrodes. Potential for infection is significantly reduced during the first post-surgery week by using antibiotic gel to cover the entire implant. Following the surgery and array implantation, microwire bundles were inserted slowly into the brain. Each microelectrode bundle is moved gradually over the course of several days. This was achieved by turning microscrews in the array assembly, which lowered subsets of microwires independently. Each quarter turn of the microscrew lowers the bundle by 53 micrometers. As expected, microelectrodes do evoke a tissue reaction over the first few weeks after implantation. This tissue reaction, which includes a local immunological response, is contained and does not lead to any significant tissue damage outside the microelectrode penetration rack.

Implants

Since the cortices of rhesus monkeys are convoluted, the implants were developed to be suited for both relatively shallow penetrations in cortical gyri as well as deeper penetrations in the sulci. As such, our recording cubes contain guiding tubes loaded separately for gyri and for sulci. For gyral penetrations, guiding tubes are fitted with 2–5 microelectrodes staggered at 0.3–0.5 mm. For sulcal penetrations, 5–10 microelectrodes are staggered at 0.5–1 mm. Using this approach, the density of microelectrodes within a cortical volume can be adjusted, depending on the depth of the targeted cortical area. Monkeys M and N were implanted with four 96-microwire recording cubes constructed of stainless steel 304 (30–50 µm in diameter). Each hemisphere received two cubes: one in the upper-limb representation area and one in the lower-limb representation area of sensorimotor cortex. Monkeys C and K were implanted with 128-microwire cubes (stainless steel 304, 30–50 µm in diameter). Monkey K received two cubes in M1 and one array in S1 for each hemisphere. Monkey C received two cubes in M1, one in S1 and another in SMA. Monkey O was implanted with four 400ch cubes (stainless steel 304, 30–50 µm in diameter), one in M1 and one in S1 for each hemisphere. Data from three previous monkeys (Monkeys I, Cl and G) was used for longevity analysis. Monkeys I, Cl, and G were implanted with 96 microwire arrays. Histology data from one additional monkey (Monkey T) was used in this report. Monkey T was implanted with three 32 microwire (stainless steel 304, 50 µm in diameter) fixed arrays.

Neuronal Isolation

After electrode insertion, we examined each microwire for units, and made adjustments to the cortical depth of the wire based on these results. After microwire positioning, we performed spike sorting for use in our experiments, monitoring the units on an almost daily basis (Supplementary Video 4 and 5). Units were sorted using standard template matching paradigms[45] available in our Plexon and wireless recording systems, which are standard and well used across the literature[12]. To assess whether recorded units were single or multi-units, we estimated refractory period using interspike intervals. Units that exhibited refractory periods greater than 1.6ms[9] were considered single units. Population plots included data from both single and multiunits so recorded units in these plots will be referred to simply as “neural units” or “units.”

Histology

Histology for microelectrode tracts was performed in one animal (Monkey T). The animal was deeply anesthetized with pentobarbital and transcardially perfused with 4% paraformaldehyde. Sections of representative areas of interest of the brain were selected, prepared, and stained with cresyl violet (Nissl method) in order to identify electrode tracks location. Slices were prepared using a HMH 505 E cryostat (Microm) with 50µm thickness.

BMI Suite

Our BMI suite is written in C++ and DirectX 9.0. It uses Lua bindings for configuration and scripting, and interfaces directly with Plexon using the C++ Plexon API. The BMI suite also interfaces with our custom designed wireless system using a Remote Procedural Call service written in the wireless client that serves firing rate of all neurons at every request (the BMI RPC calls at 100 Hz). Interfacing with other data streams is usually achieved through a UDP socket or, less often, a simple serial interface. Other signal input sources may be added in the future, using all existing algorithmic methods, without significant modification of the existing infrastructure due to the modularity of the design. Decoding algorithms operate using a common interface with the BMI software suite, allowing new routines to be implemented in a modular way. Algorithmic outputs can be combined flexibly at the user's behest using the software's GUI, and then passed to the experiment manager module. The experiment subroutine implements the behavioral task, handles stimuli, feedback, and experimental procedure logic. The experiment manager module uses a standardized interface, which allows different experiments to be switched in and out without affecting the rest of the software suite. Finally, experimental data is recorded through a unified threaded logging system, which automatically records all pertinent experimental variables.

Wireless Transceiver

An Analog Devices Blackfin BF532 DSP performs signal conditioning, sorting, and radio control on the transceiver, and is written in assembly language. The Blackfin processor uses sum absolute accumulate (SAA) instructions, typically utilized in MPEG video compression, in order to implement L1 norm template sorting. The ADCs output four samples every microsecond, and processing each proceeds in lockstep. First they are dithered to 14 bits via a low-pass pre-filter, high-passed at 250Hz via an integrator, then gained further from 0–128 (7.8 fixed-point) with a one-bit stochastic automatic gain control (AGC) block. This is followed by a one-bit normalized least mean squares (LMS) adaptive noise cancellation wherein the last 14 circular-buffered sampled channels are used to linearly predict the present channel. Then, this prediction is subtracted, and the error is saturated to one bit (plus sign) and used to update the weights. Only one weight is updated at a time, but as noise is largely uncorrelated with this rate (2.232 kHz), it does not affect the rejection ratio of > 40 dB. Noise-canceled samples are fed to eight poles of host-configurable IIR biquads, implemented as Direct Form-I with coefficient pre-scaling to prevent fixed-point saturation. The biquads are implemented with ping-pong register buffering and share taps between subsequent stages for maximum efficiency. Importantly, these biquads can be configured as oscillators, which has proved useful when debugging radio function.

On each headstage, a 32 channel amplifying and multiplexing chip (Intan RHA2132), feeds a 12-bit analog-to-digital converter (ADC, ADCS7476; 31.25 kHz sampling on each channel). Digital signals are isolated from the analog inputs with LDO regulators.

The blackfin processor has a special SAA instruction (sum absolute accumulate) that is intended to speed MPEG video compression by measuring the block difference between frames; here we use it to rapidly implement L1 norm template based sorting. As the SAA instruction operates on a four-sample 32-bit word, samples from each of the four headstages are shortened to eight bits and merged into one word and placed into a circular buffer; the sum of absolute differences of 16-sample stretches of waveform are then computed and compared to a threshold. If they are below the threshold a spike is marked as detected via a sticky bit in a radio FIFO. Two such templates are implemented per channel (templates A and B) for a total of 256 templates per unit. Note that there is no other threshold operation; template matching for both units per channel is computed for every incoming sample.

Wireless transmission uses a Nordic Semiconductor radio (nRF24L01+; 2.4GHz; 2 Mbps unidirectional bandwidth; 1.333 Mbps with overhead and direction switching). Signal conditioning occurs in synchrony to radio control; as there are no threads on the system; radio control state is implicitly encoded in the program counter. Radio data is sent as CRC-protected 32-byte packets at a rate of 5208.3 packets/s; the radio control code fills these packets with six samples from each of four continuously transmitted channels, plus eight bytes of template match. This equates to all 256 bits of template match being transmitted at 1.3 kHz. However, this packet leaves no space for synchronization or acknowledgment communication. Accordingly, given that L1 template matching is unable to accurately detect two simultaneous action potentials (templates A and B), we drop a few of these unlikely events via a look-up table compression, freeing up one extra bit per byte in the template-match fields. Four such bits are used to synchronize the transceiver and bridge, as every 16 outgoing packets the transceiver transitions to receive a command packet. The other four bits are used as an echo field to verify command packet reception. This radio protocol, including state transitions, utilizes 99.6% of available bandwidth with the Nordic radio chip. The whole transceiver software, which is written in assembly, fully processes a sample in less than 80 instructions, and consumes only 12.6 KB of L1 cache.

Wireless bridge

Rather than using polarized antennas, the bridge uses three identical Nordic radios with orthogonally arranged antennas; radio packets are accepted from any source provided they match the cyclic redundancy check (CRC). Successful reception of a data packet transitions the receiving radio to transmit mode, and begins bidirectional communication. There are four pairs of addresses and data, which allow all elements of the signal conditioning path to be varied on the transceiver, e.g. to set template, vary bandpass filter, disable AGC, or set which channels are being continuously transmitted. The bridge also features a full protocol stack, including IP, ICMP, DHCP, TCP, HTTP, and SLP; the last of which allows software discovery of bridges. Finally, the bridge features audio output, transceiver programming circuitry, and a power-over-Ethernet (PoE) module, so it can be remotely powered, e.g. within an animal room.

Wireless client

The sorting client is written in C/C++ using the GTK2 GUI toolkit with OpenGL and HLSL for graphics, presently on Debian GNU/Linux. The software allows further GUI control of all elements of the signal conditioning path, as outlined above. The client software allows for direct visualization of waveforms from four selected channels and all spike channels, while enabling spike sorting, data collection, and signal chaining and to interface with multiple bridges and transceivers. For spike sorting, two units can be sorted per channel, for a maximum of 256 units per transceiver. Unit isolation and sorting is performed with an algorithm based on principal component analysis (PCA): the user specifies a threshold to obtain waveform snippets, which are then represented by the first two principal components on a 2D plane. The user then marks the clusters of dots that represent neuronal waveforms in the PCA space

Unrestrained center out task

The task requires placing a computer cursor in the middle of the screen and, upon the appearance of a target, placing the cursor over the target for a juice reward. Each trial commenced with a circular target appearing in the center of the screen. The monkeys held the cursor within this target for a delay randomly drawn from a uniform distribution (from 500 to 2000 ms). After this delay, the central target disappeared and a circular target appeared on the screen at a distance of 7 cm from the center, in one of eight directions. The spacing between the objects was 0.1 cm. Before running this task inside the plexiglass enclosure, animals were trained while seated in the primate chair. The enclosure was large enough (1×1×1m) to allow the monkey freedom of movement. The monkey was free to perform the task at any time, and accommodated to the enclosure quickly. In only 3 days it started performing the manual task. Monkeys’ behavior was captured throughout each session by a video camera. For the long-term tasks, data collection was done by running an Ethernet PoE cable from a laboratory computer to the home cage and replacing the wireless system’s batteries every 24 hours.

Locomotion task

Three monkeys were involved in locomotion studies (M, N and K). They were trained to walk bipedally on a custom modified treadmill. A standard neck restraint was suspended from a linear actuator, which allowed bipedal support to be adjusted gradually while the monkey achieved bipedal posture and weight bearing. A juicer was placed in front of the macaque, and could be raised or lowered depending on the macaque’s height and the locomotor task (bipedal or quadrupedal walking). A video monitor mounted in front of the monkey displayed visual feedback for tasks, training and enrichment. Only Monkey K used wireless recordings and walked quadrupedally. Monkey M and N walked only bipedally and were recorded with our Plexon system, and their results were used in the dropping curve analysis.

Powered Wheelchair

A conventional motorized wheeled chair was modified for computer controlled movements. The human chair was dismounted, and the wheeled base was interfaced with a Robotics motor controller (RoboteQ Model VDC2450) based on a dual channel microprocessor. Computer control was established using a 900 MHz serial pipe (APC, Wireless RS232) to transmit serial commands from the computer to the controller at a 9600 baud rate.

Wheelchair task

Using the protocol for the center out task, data from the cursor in both BCWH and BCWOH paradigms was used as input into a differential drive control law that was used to drive the wheelchair through the room (3×3×4m). An additional cursor was shown on the screen, which represented the cart following the monkey’s cursor position. To provide this additional feedback, the cart’s coordinates were video-tracked and transformed unto the screen. Additional rewards were administered when the wheelchair cursor hit the target. The algorithm of cart control set the linear and angular velocities of the cart as functions of the monkey cursor position, while programmable boundaries kept the cart from striking the walls by setting linear velocity to 0 when outside the boundary of the working area. A sample video of the task is shown in Supplementary Video 2.

Free roaming experiments

Monkey K was videotaped and 256 cortical units were simultaneously recorded in this animal while it roamed freely in a room (3×3×4m). Offline, the monkey’s motions were hand labeled into 15 categories of behavior. Neuronal data was aligned with video recording, neurons that did not fire over 500 times in one hour were eliminated (leaving 107 neurons) and PETH analysis was performed with 2 second windows and 100ms bins. Population PETH data were calculated and analyzed as spectrograms. For discrete predictions, six behaviors were selected based on their saliency and ease of identification, as well as the amount of data available to them (>30 events per behavior).

Kinematics tracking

An in-house developed tracking system was used to track the three-dimensional position of the ankle, knee, hip, wrist, shoulder and head of monkeys, as well as the surface of the limbs. Both limbs were tracked using two units of this system.

Decoding algorithms

Historically, brain machine interface (BMI) research has used Weiner filters for online predictions of movement. Recently in our laboratory, we developed a more advanced algorithm for predicting limb movements, called the unscented Kalman filter[29]. This algorithm was used for the extraction of lower limb kinematic parameters from cortical activity, as well as cursor parameters from joystick tasks. The Weiner filter was also applied when indicated. For discrete classification, we used MATLAB implementations of EM, SVM and k-means.

Analysis

All data analyses were performed on MATLAB. No investigator blinding, nor animal randomization, was performed. PETHs were calculated taking a 1 sec time window centered on target onset in the joystick tasks, or swing onset for the left leg in the locomotor tasks, and averaging binned firing rates over all trials for each neuron. Neuron activity was normalized per neuron, per condition. Analysis of recordings over months and years of experimentation was performed by reading all experimental files and counting average number of units recorded per 32-channel array. Implementations of EM, SVM and k-means were available in MATLABs statistical toolbox.