Fidelity of first-person reports following intracranial neuromodulation of the human brain: An empirical assessment of sham stimulation in neurosurgical patients.

Dear Editor,

Intracranial electrical stimulation (iES) has been used to map and perturb brain function in neurosurgical patients for nearly 150 years [1]. During so-called ‘functional mapping’ sessions with iES, brief pulses of electrical stimulation are followed by careful observation of elicited effects on both the body and mind. IES has great clinical utility when used to identify brain tissue involved in speech and other motor functions that should be spared during neurosurgical resection [2]. Over the past few decades, the clinical application of iES has moved beyond the relatively straightforward mapping of objectively-observable outcomes like motor movements, and into the modulation of subtle and subjective experiences like mood, anxiety, and obsessive-compulsive urges [3,4]. In parallel, there has been a rapid proliferation of non-invasive neuromodulation methodologies designed with similar aims in mind [5,6].

All efforts to neuromodulate unverifiable subjective states face shared challenges: the problem of demand characteristics and the potency of the placebo effect underscore the necessity for sham stimulations as a control condition in any such intervention. Although sham stimulation is routinely employed in both iES and other non-invasive neuromodulation interventions, to our knowledge there has been little empirical assessment of how effective this control actually is. For instance, one transcranial direct current stimulation (tDCS) study found that while the rate of reported side effects was significantly higher during veridical stimulation, nonetheless sham stimulation elicited considerable and complex effects, including impaired concentration, fatigue, pain, anxiety, and changes in mood and visual perception [7]. These findings suggest that patients have considerable difficulty differentiating spontaneous fluctuations in conscious experience from the effects actually elicited by non-invasive neuromodulation. This is especially problematic given that the content of false positive reports (e.g., pain, anxiety, and mood changes [7]) closely resembles the very symptoms neuromodulation is intended to mitigate.

Given these important concerns, more empirical work is needed to quantitatively explore the false positive (Type I error) rate of first-person reports following sham stimulation with invasive neuromodulation methods and to identify possible triggering factors for such false reports. The present study sought to address these aims by exploring iES functional mapping sessions accumulated from more than a decade of neurosurgical inpatients undergoing intracranial electroencephalography (iEEG) monitoring for intractable epilepsy at a U.S. tertiary care medical center. Our specific aim was to assess the fidelity of first-person reports following sham iES in a large sample of patients, with the broader goal of informing estimates of the reliability of neuromodulation of subjective experience in general.

Veridical iES was performed using standard procedures and parameters: bipolar stimulation was delivered using an alternating square wave current applied across two adjacent electrodes at 50 Hz, 2–10 mA current, and square-wave pulse width of 200e300 ms for a stimulation duration of 1–2 s. Following each iES pulse or sham stimulation, patients were asked standardized, open-ended questions about any experiences evoked (e.g., “Did you notice anything?” or “Any change?“, as in our recent work [8,9]), with occasional follow-up questions, as needed, to further clarify the character of effects. Further details of stimulation methods and parameters are described extensively in our prior work [8,9]. For each patient, we stimulated as many electrodes as possible, subject to practical (e.g., time constraints) and clinical (e.g., seizure onset) considerations. Stimulation was administered across all electrodes sequentially, in a single session typically lasting 45–90 minutes. The inter-stimulation interval was typically on the order of 10–20 seconds but varied with the duration and complexity of patients’ reports of the effects elicited by iES. For instance, when the patient reported ‘no change’ in their experience (null effects), the subsequent stimulation, at higher amplitude, was typically administered within a few seconds. However, when a complex effect was elicited, 1–2 minutes might elapse before the subsequent stimulation (either veridical or sham), in order to allow the patient to explain the effect in detail and allow the experimenter(s) to ask clarifying follow-up questions.

During sham stimulation, the experimenter behaved exactly as during veridical stimulation, adjusting settings on the stimulator and pressing the same buttons, followed by the same standardized questions about any changes in the patient’s experience – the only difference being that no current was actually delivered.

We identified 159 sham stimulations administered to 44 patients (M ±SD = 3.6 ±3.3 per patient). A total of 11 false positive reports were identified, for an overall Type I error rate of 6.9% (Fig. 1). Notably, the overwhelming majority of our sample (75%) never committed a single Type I error (Fig. 1); false positives were restricted to only 11 patients (25% of our sample), and no patient committed more than a single Type I error, even after numerous sham stimulations (for details of all false positive reports, see Table S1).

Fig. 1.

Quantitative assessment of false positive rates following sham intracranial electrical stimulation. Of the 44 patients in our sample (left panel), the majority (n = 33) never committed even a single type I error, and the remainder (n = 11) committed only one per patient. Considering all sham stimulations (n = 159; right panel), 93% yielded true negative reports. Among the few false positives we observed (n = 11), many might be excluded using a less conservative approach (see details of all false positives in Table S1). The actual false positive rate is therefore likely even lower than reported here.

As detailed in Table S1, we did not observe any cases of outright confabulation or otherwise complex reports in response to sham stimulation, nor any motor/behavioral effects that might be suggestive of a psychogenic movement disorder. Moreover, many false positives appear to have been lingering effects from recently applied veridical stimulation (Table S1).

Patients with refractory epilepsy often have mild to severe cognitive impairment; it is therefore possible that memory deficits, low intelligence, or misunderstanding of instructions could explain some false positive effects. To control for this possibility, we correlated patients’ IQ scores (in a subsample for which neuropsychological testing data were available; n = 32, Table S1) with their false positive rates, but found no significant relationship (r(30) = .18, p = .328).

To summarize, we found that neurosurgical patients were highly resilient to false positive reports following sham stimulation. The few false positive reports we did observe (Table S1) were often analogous to preceding veridical effects (e.g., Patients 19 and 20, after veridical olfactory effects had been elicited a moment before by stimulation to the orbitofrontal cortex, reported similar, lingering smells). Moreover, one Type I error directly followed veridical stimulation leading to induction of a mild seizure with accompanying aura experiences (see Patient 38, Table S1). All of these reports could well be a result of lingering effects from previous veridical stimulation or seizure auras, and hence could reasonably be discounted if a less conservative approach were adopted. Another patient reported the urge to urinate (Patient 43, Table S1), which might well have been a spontaneous urge that the patient mistakenly assumed to be the result of stimulation. This false positive, too, could reasonably be excluded under less stringent criteria. Overall, the high fidelity of patients’ first-person reports following iES is remarkable given that all patients had recently undergone neurosurgery (for electrode implantation), all patients had severe neurological disease (intractable epilepsy), and nearly all patients were on some form of analgesic medication at the time of testing (to control pain from the neurosurgery).

In conclusion, our findings strongly support the validity of prior research exploring first-person experiences elicited by iES and deep brain stimulation. Overall, patient reports of whether or not brain stimulation is modulating their subjective experience are highly reliable. Many iES neuromodulation interventions aimed at improving subjective well-being face ongoing challenges and controversy, however [10]. Further research assessing the fidelity of first-person reports following sham stimulation in larger samples of patients, and with non-invasive methods like tDCS, could further minimize the confounding effects of false positives and increase the overall effectiveness of neuromodulation interventions.