Electrocorticography links human temporoparietal junction to visual perception.

Electrical stimulation of visual cortex can produce a visual percept (phosphene). We electrically stimulated visual cortex in humans implanted with subdural electrodes while recording from other brain sites. Phosphene perception occurred only if stimulation evoked high-frequency gamma oscillations in the temporoparietal junction (TPJ), a brain region associated with visual extinction and neglect. Electrical stimulation of TPJ modified the detectability of low-contrast visual stimuli.

Electrical stimulation of occipital lobe can produce the perception of phosphenes, bright spots in the visual field[1]. Phosphenes have been proposed to be a fundamental unit of visual perception, and may provide the building blocks for cortical prosthetics for the treatment of blindness[2]. In a previous study, we electrically stimulated identified human visual areas and found that only some areas supported phosphene perception[3]. The relationship between activity in visual cortex and visual perception is a matter of intense debate[4]. In order to search for the neural correlates of perception, in the present study we stimulated or recorded from 214 electrodes in 3 subjects.

In an initial screening step, individual electrodes were electrically stimulated and subjects verbally reported whether they perceived a phosphene. Across three subjects, 16 electrodes produced a phosphene (“percept electrodes”) and 128 electrodes did not produce a phosphene (“non-percept electrodes”); the remaining electrodes were not screened. Percept electrodes were concentrated over early visual areas near the occipital pole (Fig. 1a) consistent with previous reports[1,3]. Following screening, one percept electrode and the nearest non-percept electrode (both positioned on occipital cortex) in each subject were selected for experiment 1. Single 5 ms current pulses (1–1.5 second interpulse interval) were repeatedly delivered to these electrodes. The same stimulation current was used for the percept electrode and the non-percept electrode in each subject; the current was sufficient to always produce a phosphene in the percept electrodes. Subjects were instructed to remain alert but did not perform a behavioral task. Time-locked to delivery of the electrical pulses, neurophysiological data were collected from all non-stimulated electrodes. Neural oscillations in the gamma range (~ 30–200 hz), have been found to reflect neuronal spiking activity[5,6] and may serve as a general mechanism of information processing[7]. Comparing the gamma activity evoked by percept and non-percept electrode stimulation revealed a surprising pattern (Fig. 1b): a much greater response was observed in and around the temporoparietal junction (TPJ) for percept vs. non-percept stimulation. We selected the electrode closest to the TPJ in each subject for further analysis. When percept electrodes were electrically stimulated, a burst of high-frequency (60–150 hz) gamma activity was observed in the TPJ (Fig. 1c) beginning within 100 ms after stimulation onset and continuing for 200 ms. Non-percept electrode stimulation at the same current produced no such TPJ activity. To quantify this effect, we performed a two-factor ANOVA with stimulation electrode type (percept vs. non-percept) as the fixed factor, subject as the random factor and TPJ gamma response (relative to pre-stimulation baseline) for each stimulation pulse as the dependent measure. A significant effect of stimulation electrode type was observed (F1,237 = 64, p = 10−13). Across subjects, the gamma power increased 54% ± 6% (mean ± SEM) with percept electrode stimulation vs. −3% ± 3% with non-percept electrode stimulation. To determine if the effect was specific to high-frequency gamma power, we performed a similar ANOVA with TPJ low-frequency power (1 hz–30 hz) following stimulation as the dependent measure. A small difference was observed in low-frequency power (8% ± 5% vs.−7% ± 5%, F1,237 = 4, p = 0.04).

Figure 1

A. Percept electrodes (green) that produced a phosphene upon electrical stimulation and non-percept electrodes (red) that did not in three subjects. Subject 1 (s1) shows a posterior view of the right hemisphere; s2: posterior view of left hemisphere; s3: medial view of left hemisphere. Electrodes were implanted only in a single hemisphere for each subject (right for s1, left for s2 and s3). OP: occipital pole.

B. Maps showing the difference in gamma power between percept electrode stimulation and non-percept electrode stimulation in three subjects (same subjects as A). For each subject, one percept electrode and the nearest non-percept electrode in the implanted hemisphere was repeatedly stimulated, and the significance of post-stimulation difference in gamma power at each electrode (except for the stimulation electrodes) was calculated and mapped to the cortical surface. Black spheres show electrode locations. TPJ: temporoparietal junction.

C. TPJ response during electrical stimulation of occipital electrodes that did (left) or did not (right) produce a phosphene, averaged across subjects. Color scale indicates power at each frequency. Dark gray bar centered at t = 0 indicates stimulation artifact. Dashed white line at f = 30 Hz indicates boundary between two different frequency estimate techniques. Dashed black line indicates gamma band f-t window used to estimate power for single-trial analysis.

D. TPJ gamma responses for every trial during stimulation of a percept (left) or non-percept (right) electrode. Each horizontal line (raster) shows the power in a single trial over time, collapsed across 60–150 hz. Same color scale as C.

E. Receiver-operating curve analysis of single trial data.

To examine the consistency of the TPJ gamma power change, we plotted the TPJ gamma response to single trials (consisting of single 5 ms pulses) of occipital stimulation (Fig. 1d). Single trials of percept electrode stimulation resulted in high TPJ gamma power, while single trials of non-percept electrode stimulation did not. We constructed a receiver operating curve (ROC) to test whether it was possible to discriminate between percept and non-percept trials based on the TPJ gamma response (Fig. 1e). A high degree of discriminability was observed (mean d’ across subjects, 1.2). This suggests that the TPJ gamma activity carries information that an ideal observer could use in determining whether or not the subject perceived a phosphene.

The observation that TPJ gamma activity was present on trials in which percept electrodes were stimulated (and subjects perceived a phosphene) but not on trials in which non-percept electrodes were stimulated (and subjects did not perceive a phosphene) raises the possibility that TPJ gamma power might be causally related to visual perception. Another possibility is that TPJ gamma power was merely correlated with the location of electrical stimulation: high for stimulation of early visual areas (which tend to produce phosphenes) and low for late visual areas (which tend not to produce phosphenes)[3]. To distinguish these possibilities, we capitalized on the observation that electrical stimulation of percept electrodes over early visual areas does not always produce a phosphene: the likelihood of phosphene perception increases with the stimulation current[3]. Therefore, in experiment 2, we stimulated individual percept electrodes in the occipital lobe of each subject, but varied the stimulation current from trial to trial. At low stimulation currents, low levels of gamma power were observed in the TPJ; as the stimulation current increased, so did TPJ gamma power (Fig. 2a). To quantify this effect, a two-factor ANOVA was performed with stimulation current as the fixed factor, subjects as the random factor, and TPJ response (relative to pre-stimulation baseline) in each stimulation interval as the dependent measure. A significant effect of stimulation current was observed (F3, 1236 = 47, p = 10−28). To measure phosphene perception, subjects performed a two-interval forced-choice behavioral task that required them to report the interval containing electrical stimulation[8]. At high currents, performance was nearly perfect, indicating that a phosphene was always perceived; at low currents, performance was near chance, indicating no percept. The relationship with increasing stimulation currents was similar for TPJ gamma power (neurometric function) and for behavioral performance (psychometric function) with monotonic increases in both variables (Fig. 2b).

Figure 2

A. The average TPJ response during electrical stimulation of three percept electrodes in occipital lobe in s1 at varying stimulation currents (2–8 mA).

B. Psychometric (blue) and neurometric (red) functions for subjects s1, s2 and s3. The psychometric curve (left y-axis) shows the behavioral performance during the two-interval forced choice (2-IFC) task at different stimulation currents; performance was near chance (50%) at low currents and near ceiling (100%) at high currents (error bars show 75% confidence interval from the binomial distribution). The neurometric curve (right y-axis) shows the TPJ gamma power at the same currents (error bars show standard error of the mean).

C. TPJ response during percept electrode stimulation with near-threshold currents, averaged across subjects. Data averaged from trials in which subjects correctly (left) or incorrectly (right) discriminated which of two intervals contained electrical stimulation. Stimulation current was the same for correct and incorrect trials (4 mA for s1 and s2, 0.65 mA for s3).

D. TPJ response in the gamma band for single correct (left) and incorrect (right) trials of electrical stimulation at the same current. Each horizontal line (raster) shows the power in a single trial over time, collapsed across the gamma band.

The similarity between the psychometric and neurometric functions supported the idea of a link between TPJ responses and perception. The null hypothesis is that while increasing currents led to both improved discrimination and increased TPJ gamma power, these were independent processes. To test this hypothesis we examined trials in the two-interval forced choice task in which the identical near-threshold current was delivered to a percept electrode. As expect, this level of current produced a mix of correct trials (in which subjects correctly detected the stimulation interval, suggesting phosphene perception) and incorrect trials (in which they did not, suggesting no phosphene perception). If TPJ gamma power was dependent on the amount of stimulation current but not related to perception, we would expect no power difference between correct and incorrect trials (because the stimulation current was exactly the same in each trial). An ANOVA was performed with trial type as the random factor, subject as the fixed factor, and TPJ gamma power in the stimulation epoch relative to pre-stimulation baseline as the dependent measure. Across subjects, a significant effect of trial type was observed (F1,465 = 26, p = 10−6) with greater power in correct than incorrect trials (99% ± 5% vs. 42% ± 9%), demonstrating a relationship between TPJ gamma power and perception (Fig. 2c). It should be emphasized that the physical stimulation parameters in these two trial types were identical: the same electrode and same stimulation current. To examine the reliability of this effect, we examined individual correct and incorrect trials (Fig. 2d). An ROC analysis of the individual trial data revealed a significant ability to discriminate correct from incorrect trials based on the stimulated-evoked TPJ gamma power (mean d’ across subjects, 0.74).

These results suggested that the gamma oscillations in TPJ might be a neural signature of the phosphene percept. If this was the case, an ideal observer could perform the two-interval forced choice task by comparing the TPJ gamma power across the two intervals within a single trial. To test this idea, we compared the TPJ gamma activity between stimulation and non-stimulation intervals within individual trials. Within correct trials, there was a very large TPJ power difference between the stimulated and non- stimulated intervals (99% ± 5% vs. 19% ± 3%, t374 = 14, p = 10−36). Within incorrect trials, there was only a small TPJ power difference between the stimulated and non- stimulated intervals (42% ± 9% vs. 20% ± 6%, t93 = 2.2, p = 0.03). An ROC analysis confirmed that an ideal observer could do very well at distinguishing the two intervals in correct trials (d' = 1.1) but not in incorrect trials (d' = 0.3). This suggests that the electrical stimulation of visual cortex on incorrect trials did result in TPJ gamma oscillations, but that the amplitude of the oscillations was below the neural threshold for perception, leaving subjects unable to discriminate the two intervals.

If TPJ gamma oscillations are critical for visual perception, disrupting them would be expected to interfere with perception. Therefore, in experiment 3 we electrically stimulated the TPJ while subjects detected visually presented sine-wave gratings in a gaussian window (Gabor patches). A preliminary test examined whether TPJ stimulation in isolation produced a behavioral effect: for instance, if TPJ stimulation produced a phosphene, this could hinder perception of gratings in an uninteresting way. In our initial screening, subjects did not report a phosphene following TPJ stimulation. As an additional check, we also performed a more sensitive two-interval forced choice task in which subjects attempted to detect TPJ stimulation; subjects performed at chance level on this task (49%; 95% confidence interval, CI, from the binomial distribution of 32% to 65%). Next, we tested subject's ability to detect the location of a grating randomly presented in either the left or right hemifield on each trial. At high contrast, subjects easily detected the grating, performing at ceiling (99%, CI 95% to 100%). At threshold contrast, subjects detected the grating on 58% of trials (CI 50% to 66%). Next, we electrically stimulated the TPJ while subjects performed the task; stimulation and non-stimulation trials were randomly intermixed. Subjects continued to perform at ceiling levels (99%, CI 95% to 100%) for high-contrast gratings, demonstrating that TPJ stimulation did not interfere with the ability to perform the behavioral task. However, for threshold contrast gratings, a significant effect of stimulation was observed. Detection was better for gratings presented ipsilateral to the stimulated TPJ than for gratings presented contralateral to the stimulated TPJ (76% vs. 53%, p = 0.03 from binomial distribution, CIs 66% to 85% and 42% to 63%). Relative to no stimulation, performance improved when gratings were presented ipsilateral to the stimulated TPJ (76% vs. 58%, p = 0.05) but was not significantly different for gratings presented contralateral to the stimulated TPJ (53% vs. 58%, p = 0.6).

Previous work demonstrated that electrical stimulation of some sites in visual cortex, but not others, produces phosphenes[1,3]. In the present study, we combined electrical stimulation with electrical recording and found that subjects perceived a phosphene during electrical stimulation only when high-gamma power was recorded in the TPJ. TPJ activity during phosphene perception was observed both during passive stimulation (experiment 1) and while subjects performed a behavioral task (experiment 2) making it difficult to attribute the gamma activity to task performance. Our observation of visual perception related gamma activity in the TPJ is striking, because converging evidence suggests that the TPJ is critical for detecting behaviorally relevant stimuli [9]. The TPJ has been proposed as a neural generator for the P300 event-related potential, which is linked to target detection across sensory modalities[10]. In particular, damage to ventral regions of parietal lobe, especially the TPJ, may cause difficulties in orienting to a meaningful stimulus presented contralesionally either alone (spatial neglect) or with a simultaneous ipsilesional stimulus (spatial extinction)[11-13]. This suggests a possible parallel with our results in experiments 1 and 2. When electrical stimulation does not produce a phosphene, neural activity is produced at the electrode site, but it does not propagate through the cortical network to evoke TPJ activity, and hence fails to enter conscious awareness just as with neglected/extinguished visual stimuli. In contrast, when neural activity at the stimulation site does propagate to the TPJ, the activity enters conscious awareness and a phosphene is produced. Experiment 3 demonstrated that TPJ stimulation resulted in an altered ability to detect visual stimuli, with enhanced detection ipsilaterally and reduced detection contralaterally. These behavioral results are consistent with the hemispheric competition model of attentional control[14]. If the TPJ in one hemisphere is disrupted, it becomes less able to detect stimuli in the contralesional hemifield, but also decreases its transcallosal inhibition of the contralateral TPJ, producing an ipsilesional attentional bias that can actually improve detection performance for ipsilesional stimuli[15].

Online Methods

Informed consent was obtained and all procedures were approved by the Baylor College of Medicine Institutional Review Board or the Committee for the Protection of Human Subjects at the University of Texas Health Science Center at Houston. We studied three patients who had electrodes implanted for surgical treatment of epilepsy. Subject 1 was a 47 y.o. female, subject 2 was an 18 y.o. male, subject 3 was a 36 y.o. female. Subject 1 had electrodes implanted in the right hemisphere only; subjects 2 and 3 had electrodes implanted in the left hemisphere only. Electrodes that were determined to be near the epileptogenic region of cortex were excluded from the experiment.

Electrical Stimulation

For all experiments, the patient being studied was seated comfortably in their hospital bed. Because phosphenes are typically perceived as bright flashes, the patient sat with eyes open looking at an LCD display showing a black screen, to maximize detectability of phosphenes. A Bak isolated stimulator was used to deliver electrical stimulation under computer control[1].

Electrophysiological Recording and Data Analysis

Electrophysiological data was recorded using a 128-channel NeuroPort System from Blackrock Microsystems (Salt Lake City, Utah). Stimuli were delivered using Objective C programs running under a Macintosh. Electrophysiological data was acquired at 2 kHz and analyzed using Matlab and the FieldTrip toolbox[2]. Responses were filtered with Savitzky-Golay polynomials[3]. For responses below 30 Hz, a Hanning taper with a fixed window length was used. For responses above 30Hz, a multitaper filter was used. The ROC analysis was conducted using standard methods and the Matlab function perfcurve. Data from each subject was first analyzed independently. Time-frequency plots (Fig. 1c) were combined by simple averaging. To combine % power change data across subjects, repeated-measures ANOVAs were performed using the Matlab function "anovan". The dependent measure was the power in the electrode closest to the TPJ. Subject was used as a random factor and trial type (percept vs. non-percept or stimulation current level) was used as a fixed factor. Behavioral data from the two-interval forced choice task was analyzed using the Matlab function "binofit".

Neuroimaging

Before electrode implantation structural MR scans were obtained using an 8-channel parallel acquisition radio frequency coil on the whole-body 3 tesla scanner in the University of Texas Health Science Center at Houston Magnetic Resonance Imaging Center (Phillips Medical Systems, Bothell, WA). The structural scans included two repetitions of a magnetization-prepared 180 degrees radio-frequency pulses and rapid gradient-echo (MP-RAGE) sequence optimized for gray-white matter contrast with 1 mm thick sagittal slices and an in-plane resolution of 0.938 x 0.938 mm. Three-dimensional surface models of the subject’s brain were reconstructed using FreeSurfer[4,5].

Electrode Implantation and Localization

Following implantation surgery, the subject underwent whole-head computed tomography (CT). The electron-dense metal electrodes appeared as bright spheres in the CT. The center of each electrode in the ventral temporal strip was manually localized at the center of each sphere. Next, the CT scan was aligned to the pre-surgical structural MRI using the mutual information algorithm in the "3dAllineate" program in the AFNI package. Standard subdural recording electrodes were used (AdTech, Racine, WI). Each electrode consisted of a disc of platinum alloy covered in insulating silastic, except for a central 2.2 mm diameter region on the brain side of the electrode. To visualize the location of the electrode in the MR volume, a mask volume was created that consisted of a sphere of diameter 2.2 mm (centered on the electrode) using the AFNI program “3dcalc”. To visualize the location of the electrode on the cortical surface, this mask volume was mapped to the nearest nodes on the cortical surface using the AFNI program “3dVol2Surf”.

Screening

Individual electrodes were electrically stimulated and subjects verbally reported whether or not they perceived a phosphene, defined as a localized, brief visual percept, commonly described as a flash of light.

Experiment 1

In experiment 1, a single percept electrode and a single non-percept electrode were selected for repeated stimulation. For s1, a single 5 ms biphasic pulse of electrical stimulation at 8 mA was delivered for 60 trials at 1.2 second intertrial intervals (ITI); s2: 6 mA, 50 trials, 1 sec ITI; s3: 2 mA, 50 trials, 1.5 sec ITI. Data was collected from all electrodes that were not stimulated. The gamma power was calculated as the percent change in the window 60–150 hz and 100 ms to 300 ms post-stimulation relative to a pre-stimulation baseline consisting of the period 200 ms to 100 ms before stimulation. To create a map of the gamma power on the cortical surface (Fig. 1b), the t-statistic of the difference in the gamma power following stimulation (compared with pre-stimulation baseline) between percept and non-percept electrode stimulation was calculated for each non-stimulated electrode. Then, this t-statistic calculated for the electrode was applied to all cortical surface nodes in a sphere with radius of 5 mm centered on each electrode. Finally, spatially smoothing of the t-values on the cortical surface was applied with a full-width at half maximum of 1 mm.

Experiment 2

In experiment 1, passive stimulation at a single current was used. In experiment 2, subjects made a judgment during each trial of electrical stimulation, and the stimulation current was varied from trial to trial. Each trial contained two intervals, during only one of which a single 5 ms biphasic pulse of electrical stimulation (with a current that varied from trial to trial) was delivered. Subjects performed a two-interval forced choice task, determining which of the two intervals contained the electrical stimulation. The intervals were marked by spoken auditory cues (“one” or “two”), separated by an inter-interval period of 500 ms. Following both intervals, a tone indicated that the subject should respond by pressing one of two mouse buttons to signal their choice, and auditory feedback (a voice saying “good job” for correct trials or “try again” for incorrect trials) was delivered. If no response was received during the 2500 ms response window, other feedback (“please respond”) was delivered. An intertrial interval elapsed (1200 ms for s1, 1000 ms for s2, 1500 ms for s3) before the next trial began.

For experiment 2, a single TPJ electrode was analyzed for each subject. This electrode selection was based solely on anatomical criteria (and the results of experiment 1), meaning that the results of experiment 2 cannot be attributed to selection bias.

To increase trial numbers, and minimize stimulation at each individual site, multiple percept electrodes were stimulated for s1. Only one electrode was stimulated at a time, and comparisons between correct and incorrect trials were made only within a single electrode. For s1, three percept electrodes were stimulated. The stimulation currents were 2 mA, 4 mA, 6 mA, 8 mA and there were 130, 280, 130 and 130 trials at each current, respectively. For s2, one percept electrode were stimulated at 2, 4, 6, 8 mA (60, 100, 80, 59 trials). For s3, one percept electrode was stimulated at current 0.5 mA, 0.6 mA, 0.65 mA, 1.1 mA with 45, 100, 89 and 39 trials, respectively. A near-threshold current was selected as the current that gave closest to 75% behavioral performance. 75% was selected because it is midway between chance level (50%; no phosphenes) and perfect performance (100%; phosphenes always present) to provide a sufficient number of both correct and incorrect trials for analysis. This current was 4 mA for s1, 4 mA for s2 and 0.65 mA for s3. There were 226 correct and 54 incorrect trials at these currents for s1, 78/22 for s2 and 71/18 for s3.

Experiment 3

Two subjects performed experiment 3. In experiment 3, electrical stimulation at 200 hz was delivered to the TPJ. A control experiment was conducted for one subject. The subject attempted to detect the interval of TPJ stimulation using a two-interval forced choice paradigm[1,6] at stimulation current 1 mA for subject 1 (39 trials). For the main experiment, subjects viewed windowed sine-wave gratings (Gabor patches) of varying contrast. Electrical stimulation was delivered beginning at the onset of the visual stimulus (50 ms duration for subject 1, 100 ms for subject 2; 1 mA for subject 1, 2.5 mA for subject 2). Gratings were presented at 5 degrees eccentricity (50 ms duration for subject 1, 500 ms for subject 2). At high contrast, there were 114 trials without stimulation and 114 trials with stimulation. At threshold contrast, there were 170 trials without stimulation, 84 trials with ipsilateral stimulation and 89 trials with contralateral stimulation.