Neural dynamics of illusory tactile pulling sensations.

Directional tactile pulling sensations are integral to everyday life, but their neural mechanisms remain unknown. Prior accounts hold that primary somatosensory (SI) activity is sufficient to generate pulling sensations, with alternative proposals suggesting that amodal frontal or parietal regions may be critical. We combined high-density EEG with asymmetric vibration, which creates an illusory pulling sensation, thereby unconfounding pulling sensations from unrelated sensorimotor processes. Oddballs that created opposite direction pulls to common stimuli were compared to the same oddballs after neutral common stimuli (symmetric vibration) and to neutral oddballs. We found evidence against the sensory-frontal N140 and in favor of the midline P200 tracking the emergence of pulling sensations, specifically contralateral parietal lobe activity 264-320ms, centered on the intraparietal sulcus. This suggests that SI is not sufficient to generate pulling sensations, which instead depend on the parietal association cortex, and may reflect the extraction of orientation information and related spatial processing.

Introduction

Feeling a directional force on the skin, known as a pulling sensation, is vital in everyday life, allowing us to, for example, know our dance partner’s intention, or quickly learn the physical properties of a touched object (Johansson and Flanagan, 2009). Despite much progress in understanding peripheral tactile processing (Johansson et al., 1992a; Panarese and Edin, 2011; Pruszynski et al., 2018; Pruszynski and Johansson, 2014), little is known about how pulling sensations arise in the human brain.

On one view, pulling sensations are generated in the primary somatosensory cortex (SI) (Fortier-Poisson et al., 2016; Fortier-Poisson and Smith, 2016; Salimi et al., 1999). The firing pattern of subpopulations of cells in the monkey SI tracks the accelerations and decelerations of tangential forces and the frictional properties of held objects (Salimi et al., 1999). However, although such processing is necessary, it may not be sufficient for the pulling sensation. Instead, amodal circuitry in the frontal or parietal lobe may be required, as with other forms of tactile discrimination (Hernández et al., 2010; Mueller et al., 2019; Romo and Salinas, 2003). To date, separating sensory and amodal contributions to the pulling sensation has been difficult, and there is a lack of knowledge regarding the timing and neural circuitry underpinning the pulling sensation.

Within tactile neurophysiology, a distinction is made between mid-latency P50 and N140 potentials, which arise from a small number of sources, and later midline P200 and P3b potentials, which arise from multiple, functionally diverse sources (Allison et al., 1992). The N140, which originates in SII and the frontal lobe (Desmedt and Tomberg, 1989; Frot et al., 1999), is a reliable marker for tactile awareness (Auksztulewicz et al., 2012; Schröder et al., 2021) and texture processing (Genna et al., 2018). If pulling-related N140 enhancements are observed, it could indicate that early sensory elaboration performed by SII, along with top-down modulation from the frontal cortex, is critical to the pulling sensation.

Alternatively, if later potentials such as the P200 are selectively modulated by directional pulls, the process may depend on the integration of sensory and amodal frontoparietal inputs (Allison et al., 1992). The P200 appears to track a late stage of perceptual processing when stimulus features such as size, orientation, and direction are extracted (Luck and Hillyard, 1994). Feature extraction may facilitate the remapping of tactile inputs across spatial reference frames (Bufalari et al., 2014; Longo et al., 2012) and modalities (Harjunen et al., 2017), processes that are also indexed by the P200 (but cf. Sambo and Forster, 2009; Soto-Faraco and Azañón, 2013). These processes could be related to pulling sensations when the orientation of a pull must be extracted and converted to a useful reference frame for determining its direction. Nevertheless, precise control over the stimulus must be exerted before the function can be ascribed.

Three forms of unrelated neural activity typically occur alongside pulling-specific processes. Firstly, there is unrelated motor processing relating to postural corrections triggered by the sudden application of an external force (Johansson et al., 1992b). Secondly, there are somatosensory processes common to other types of tactile stimuli, generated by the physical characteristics of the stimulus, but unrelated to the pulling sensation itself (Birznieks et al., 2001). Finally, there are unrelated attentional processes (Nakajima and Imamura, 2000), initiated by the greater complexity and salience of pulling compared to non-pulling stimuli.

To disambiguate pure pulling sensations from their conjoined but unrelated motor, sensory and attentional processes, we here use an asymmetric vibration approach, which creates a strong, illusory sensation of the hand being pulled in a particular direction via a small handheld device, without active movement (Amemiya et al., 2005; Amemiya and Gomi, 2014, 2016; Amemiya and Maeda, 2008; Gomi et al., 2019; Tanabe et al., 2018; Tappeiner et al., 2009). Symmetric vibration was used as a control stimulus, which is closely matched in terms of stimulus complexity, but does not induce an illusory pulling sensation.

High-density EEG was recorded during a tactile oddball task in which uncommon target stimuli must be detected from a stream of common stimuli (Kida et al., 2003; Shinozaki et al., 1998; Spackman et al., 2007). Oddballs that created an illusory pulling sensation in the opposite direction to the common stimuli (asymmetric vibration) were compared to the same oddballs in the context of neutral common stimuli (symmetric vibration), and also to neutral oddball stimuli. These relative oddball effects meant we could isolate the brain activity specific to having directional expectations contradicted by directional pulling stimuli, and determine: 1) whether pulling sensations generate activity beyond SI, 2) the timing and identity of brain components indexing pulling sensations, and 3) what types of processing likely underpin pulling sensations.

Results

Opposite direction pulls easier to discriminate behaviorally

Participants reported that neutral stimuli (symmetric vibration) were subjectively similar to asymmetric vibration, aside from the absence of the pulling sensation, indicating that it was the pulling sensation itself that was used for discrimination (Figure 1). All participants were able to clearly feel the pulling sensation during the pre-test of two alternative forced choice tasks (2AFC). Mean discrimination was >80% (Figure 2A) and participants were equally good when comparing Left vs. Neutral to Right vs. Neutral (85.33% (SD = 7.62) vs. 81.45% (SD = 14.9); t(14) = −1.439, p = 0.172; Figure 3A left panel).

Figure 1

Task design and experimental setup

(A) Task design showing a single block in which Neutral (symmetric vibration) is the common stimulus and Left and Right pull (asymmetric vibration) are the randomly appearing Oddballs. Note that there were two other types of blocks, in which the Left and Right pulling stimuli acted as the common stimuli, with the oddballs being neutral and right, and neutral and left, respectively. Participants had to silently count all oddballs and report their count at the end of the block.

(B) Close up of the device used to generate asymmetric and symmetric vibration, showing top and cutaway view. Different acceleration profiles of the oscillating magnet were created by varying the current in the solenoids.

(C) Single trial accelerometer signals showing the recorded left-right acceleration of the vibrating device. Acceleration profiles are shown for left pull stimuli (top), right pull stimuli (middle) and neutral stimuli (bottom). Positive y axis values correspond to the leftward acceleration of the device (toward the midline of the participant’s body), negative values correspond to the rightward acceleration of the device (away from the midline).

(D) The three oddball conditions consisted of the “Opposite pull oddball” condition (right pull oddballs after left pull common and left pull oddballs after right pull common), the “Pull oddball after neutral” condition (left and right pull oddballs after neutral common), and the “Neutral oddball” condition (neutral oddballs after left and right common).

(E) Experimental setup showing participant holding the unattached vibrating device in their right hand using a pinch grip whilst high-density EEG was recorded.

(F) Diagram showing the relative location of the 129 electrodes. Electrodes from face, ears, and neck (shown in gray) were excluded from the main analysis owing to artifacts.

Figure 2

Behavioral, P50, and N140 ERP results

(A) Group mean pre-test pulling discrimination accuracy was high in all conditions, indicating that all experimental stimuli could clearly be perceived. There was no difference in participants’ ability to discriminate Left and Right pulls (asymmetric vibration) from Neutral stimuli (symmetric vibration), but the performance was better in a subset (n = 9) when discriminating left from right pulls as opposed from discriminating either of the pulling stimuli from the neutral stimulus. ∗p < 0.05, N.S. = Not Significant. Error bars show standard error of the mean (SEM).

(B) Group mean P50 amplitude and scalp maps (30-70ms) for all conditions. When comparing difference waves across the three oddball conditions over the contralateral parietal cortex there was no difference in mean amplitude. N.S. = Not Significant. Error bars show SEM.

(C) Group mean N140 amplitude and scalp maps (100-150ms) for all conditions.

(D) Difference waves from contralateral frontal electrodes show slightly attenuated N140 in the “Opposite pull oddball” and “Pull oddball after neutral” conditions (i.e. less negative for oddball than common conditions) and an enhanced N140 for the “Neutral oddball” condition (i.e. more negative for oddball than common condition). Difference wave N140 was significantly larger (more negative) for the “Neutral oddball” condition compared to the “Opposite pull oddball” and Pull oddball after neutral’ condition. ∗p < 0.05, N.S. = Not Significant. Error bars show SEM.

Figure 3

Instantaneous ERP amplitude across conditions

The amplitude (μV) of scalp activity across time in the two common stimulus conditions: “Common neutral” and “Common pull” (mean of left and right common), and in the three Oddball conditions: “Opposite pull oddball” (mean of left oddball after right common and right oddball after Left common), “Pull oddball after neutral” (mean of left oddball after neutral common and right oddball after neutral common), and “Neutral oddball” (mean of neutral oddball after left common and neutral oddball after right common).

During data collection, we realized that it would be informative to also have participants directly discriminate left and right pulls prior to EEG, which was conducted for a subset of participants (n = 9). Participants were better at discriminating Left vs. Right than discriminating Left/Right vs. Neutral (93.78% (SD = 7.03) vs. 86.22% (SD = 6.83); t(8) = −2.591, p = 0.032, Cohen’SD = 1.09; Figure 3A right panel). This finding was expected, given that Left/Right discrimination is similar to the “Opposite pull oddball” condition, which was associated with the strongest late brain response (see EEG results).

For the 2AFC task it was important to also examine the bias, meaning the percentage of trials during which participants make a particular response, because even if overall performance was acceptable, the presence of a strong bias could mean that one stimulus was not detected as reliably as the other. For example, if participants responded that a Pull was present (i.e. Left or Right) on significantly more than 50% of trials when judging Pull vs Neutral, it might indicate greater difficulty in identifying Neutral stimuli. However, this was not the case. We found no bias toward Left pulls vs. Neutral (47.6 vs. 52.4%; t(14) = 1.103, p = 0.288) or Right pulls vs. Neutral (53.07 vs. 46.93%; t(14) = −1.667, p = 0.118). Nor was there any Left vs. Right bias in the subset of participants (n = 9) who performed the additional Left vs. Right pull discrimination task (Left = 52.22% vs. Right = 47.78%; t(8) = −0.989, p = 0.352).

We performed additional Bayesian analysis (Dienes, 2014) to evaluate whether our Pull vs. Neutral null results were likely to be genuine. When comparing discrimination performance on the Left vs. Neutral task to the Right vs. Neutral task, weak support for the null was found (mean difference = 3.87, SE = 2.93; likelihood of theory = 0.044; likelihood of null = 0.057; Bayes factor = 0.77). Likewise, weak support for the null was found for the assessment of bias, when comparing Left pulls vs. Neutral (mean difference = 4.8, SE = 4.74; likelihood of theory = 0.039; likelihood of null = 0.05; Bayes factor = 0.77) and Right Pulls vs. Neutral (mean difference = 6.13, SE = 4.01; likelihood of theory = 0.036; likelihood of null = 0.031; Bayes factor = 1.17). Bayes factors <0.3 are considered to show acceptable support for the null (Dienes, 2014). As such, we cannot rule out the possibility that our null results were owing to noise. Nevertheless, participants were clearly able to discriminate both Pull and Neutral stimuli, and any differences in relative performance were likely small enough to be of negligible importance to the main oddball task.

During the main task, oddball counting error did not differ when comparing Left Common to Right Common blocks (5.49 (SD = 3.23) vs. 4.73 (SD = 2.51); Z = −0.483, p = 0.631) or when comparing mean Left/Right common blocks to Neutral common blocks (5.11 (SD = 2.34) vs 5.73 (SD = 2.34); Z = 1.079, p = 0.28). Thus, we did not observe any significant differences in oddball counting performance, consistent with participants showing similar levels of effort and attention across conditions.

The P200 is selectively enhanced by the pulling sensation

The “Opposite pull oddball” condition was associated with larger amplitude P200 responses than the “Pull oddball after neutral” condition (0.68 vs. 0.27μV; t (14) = 2.818, p = 0.014, Cohen’SD = 0.72; Figures 2 and 4; Table 1, Table 2 and 2) and the “Neutral oddball” condition (0.68 vs. 0.18μV; t (14) = 2.24, p = 0.042, Cohen’SD = 0.79). P200 latency was also shorter for the “Opposite pull oddball” condition compared to the “Pull oddball after neutral” condition (173.03ms vs. 193.91ms; t (12) = −2.388, p = 0.034, Cohen’SD = −0.85), but not when the Opposite pull oddball’ condition was compared to the “Neutral oddball condition” (t (13) = −0.799, p = 0.439).

Figure 4

P200 and P3b oddball effects across conditions

(A) Group average ERP oddball difference waves (subtraction of activity related to common stimuli) from central electrodes shown for the three oddball conditions. P200 (150-250ms) and P3b (250-500ms) windows are shown in gray.

(B) Group average difference wave scalp activity for the P200 and P3b in each of the three oddball conditions.

(C) Group average P200 amplitude was significantly larger in the “Opposite pull oddball” condition than both the other two oddball conditions, while P3b activity was larger in the “Opposite pull oddball” condition than the “Pull oddball after neutral” condition. ∗p < 0.05. Error bars show SEM.

Table 1

Onset latency and mean amplitude for P50, N140, P200, and P3b ERPs derived from difference waves for the three oddball conditions

	Latency (ms)				Mean amplitude (μV)
Condition	P50	N140	P200	P3b	P50	N140	P200	P3b
Opposite oddball	42.68 (10.9)	118.77 (12.1)	173.03 (23.88)	335.42 (40.43)	0.16 (0.38)	0.19 (0.46)	0.68 (0.63)	2.07 (1.45)
Pull oddball after neutral	38.02 (12.2)	115.49 (16.32)	193.91 (25.19)	382.84 (47.87)	0.02 (0.2)	0.19 (0.36)	0.27 (0.5)	1.25 (1.17)
Neutral Oddball	45.96 (9.96)	110.41 (15.87)	179.31 (19.91)	338.76 (47.16)	0.01 (0.25)	−0.17 (0.39)	0.18 (0.63)	1.7 (1.45)

Group (n = 15) mean and (SD) values shown. Note that the onset could not always be identified when using difference waves. So for P50 onset latency n = 13 in the Opposite oddball condition and n = 14 in the Neutral oddball condition, for N140 onset latency n = 13 in the Opposite oddball condition and n = 11 in the Pull after Neutral condition, and for P200 onset latency n = 13 in the Pull oddball after neutral condition and n = 14 in the Neutral oddball condition.

Table 2

Statistical comparison using paired sample t-test of onset latency and mean amplitude for P50, N140, P200, and P3b ERPs comparing the three oddball conditions

	Latency				Mean amplitude
Comparison	P50	N140	P200	P3b	P50	N140	P200	P3b
Oppo. vs. Pull after neu.	0.874 (0.399)	0.172 (0.867)	−2.388 (0.034∗)	−3.84 (0.002∗∗)	1.455 (0.168)	−0.099 (0.923)	2.818 (0.014∗)	2.499 (0.026∗)
Oppo. vs. Neu. Oddball	−0.296 (0.773)	1.024 (0.326)	−0.799 (0.439)	−0.412 (0.686)	1.164 (0.264)	2.595 (0.021∗)	2.240 (0.042∗)	1.104 (0.288)
Pull after neu. vs. Neu. Oddball	−1.496 (0.158)	0.531 (0.607)	1.684 (0.120)	5.221 (<0.001∗∗∗)	0.143 (0.889)	2.599 (0.021∗)	0.479 (0.639)	−2.118 (0.053)

Shown are t-values, with p values in parentheses (∗p < 0.05, ∗∗p < 0.01, ∗∗∗p < 0.001). DF = 14, except when onset latency could not be identified. P50 latency DF = 12 for Oppo. vs. Pull after neu. and DF = 13 for the other two comparisons. N140 latency DF = 8 for Oppo. vs. Pull after neu., DF = 12 for Oppo. vs. Neu. Oddball, and DF = 10 for Pull after neu. vs. Neu. Oddball. P200 latency DF = 12 for Oppo. vs. Pull after neu., DF = 13 for Oppo. vs. Neu. Oddball, DF = 11 for Pull after neu. vs. Neu. Oddball.

For the P3b the “Opposite pull oddball” condition was also associated with larger amplitude (2.07 vs. 1.25μV; t (14) = 2.499, p = 0.026, Cohen’SD = 0.62; Figure 4; Table 1, Table 2 and 2) and shorter latency (335.42ms vs. 382.84ms; t (14) = −3.84, p = 0.002, Cohen’SD = −1.07) responses compared to the “Pull oddball after neutral” condition. No significant differences in P3b amplitude or latency were observed when comparing the “Opposite pull oddball” condition with the “Neutral oddball” condition (Figure 4; Table 1, Table 2 and 2). This was likely because the P3b was larger than expected in the “Neutral oddball” condition. Indeed, for the P3b, compared to the “Pull after neutral oddball” condition, the “Neutral oddball” condition was associated with earlier ERP onset latencies (t (14) = 5.221, p < 0.001, Cohen’SD = 0.93) and a trend toward larger mean amplitudes (t(14) = −2.118, p = 0.053, Cohen’SD = 0.34; Figure 4; Table 1, Table 2 and 2).

Thus, it appears the P200 reflects the emergence of pulling sensation as its amplitude was selectively enhanced in the “Opposite Pull oddball” condition. We cannot completely exclude the P3b from contributing to the emergence of the pulling sensation. However, given its timing and the clearer P200 effects, it seems more likely that the P3b reflects processing related to the consequence of the pulling sensation, rather than its direct generation.

Mid-latency P50 and N140 do not directly index the pulling sensation

It was necessary to exclude brain potentials prior to the P200 as reflecting the pulling sensation. Note that, by showing that these brain potentials do not index the pulling sensation, we are not claiming that they are completely unrelated to the phenomenon. Earlier potentials undoubtedly reflect tactile processes that may directly lead to the emergence of the pulling sensation. However, such neural activity does not constitute the emergence of the pulling sensation itself and is probably common to many tactile stimuli.

To establish that a brain potential does not index the pulling sensation it is necessary to show: 1) that the potential in question is not significantly enhanced in the “Opposite pull oddball” condition relative to the “Pull oddball after neutral” condition, 2) That our experiment has sufficient power to find such differences if they, indeed, exist. In the case of the N140, both criteria were met (Figures 3C and 3D; Table 1, Table 2 and 2). The N140 did not differ when comparing “Opposite pull oddballs” to the “Pull oddball after neutral” condition (0.19 vs. 0.19μV; t (14) = −0.099, p = 0.923, Cohen’SD = −0.02) and there were no latency effects (Table 1, Table 2 and 2), meeting the first criterion. Moreover, for the “Neutral oddball” condition the N140 was actually significantly larger (i.e. more negative) than that observed for the “Opposite pull oddballs” condition (−0.17 vs. 0.19μV; t (14) = 2.595, p = 0.021, Cohen’SD = 0.84) and the “Pull oddball after neutral” condition (−0.17 vs. 0.19μV; t (14) = 2.599, p = 0.021, Cohen’SD = 0.98), thus meeting the second criterion. Our experiment was, therefore, sufficiently powered to find N140 effects, but these enhancements were related to the processing of neutral stimuli, rather than the pulling sensation.

We did not observe any differences in the P50 amplitude or latency across oddball conditions (Figure 3B.; Table 1, Table 2 and 2), consistent with the pulling sensation emerging much later in the somatosensory processing stream. Nevertheless, as all P50 oddball comparisons were non-significant, we cannot exclude the possibility that the null results were owing to noise in the data.

Pulling-related activity beyond SI, 264-320ms post-stimulus in the parietal lobe

The purpose of our main contrast, comparing the “Opposite pull oddball” condition and “Pull oddball after neutral” condition was to find brain activity specific to having a directional pulling expectation violated by a different directional pull. We analyzed the entire response window for significant clusters in an unbiased manner. This revealed a cluster of significant activity (264-320ms) that peaked over the left parietal cortex (280ms post-stimulus onset) and extended anteriorly to cover part of the left frontal lobe by ∼300ms post-stimulus onset (Figures 5A-5C). Adding our behavioral measures as covariates (Table S1) did not substantially change the results (Table 3), ruling out the possibility that the effects were artifacts of extremes of task performance.

Figure 5

Brain activity associated with the pulling sensation

(A–C) Results of SPM topographical analysis in sensor space when contrasting the “Opposite pull oddball” and “Pull oddball after neutral” conditions. A cluster of significant activity (p < 0.001 uncorrected, p < 0.05 FWE cluster threshold) was observed (264-320ms after stimulus onset), peaking at 280ms over the left parietal cortex and extending anteriorly. In table, x position is positive-going left to right, y position is positive-going from posterior to anterior.

(D) Group inversion (p < 0.05 uncorrected) suggested the significant scalp activity originated from the left parietal cortex. To clarify the location of this activity we conducted a separate group inversion using the Neutral common condition, in the P50 time window (40–64ms). This was overlaid on the previously identified pulling-related activity and showed a more anterior distribution, close to the SI hand area.

Table 3

Results of the SPM sensor space contrast comparing the “Opposite pull oddball condition” and “Pull oddball after neutral” condition, using behavioral measures as covariates

	Peak Level					Cluster Level
Behavioral covariate	x	y	ms	T value	pFWE corr.	No. Voxels	pFWE corr.
Pull vs. Neutral discrimination	−21	−52	280	6.59	0.036	757	0.030
Mean oddball count error	−21	−52	280	6.01	0.067	537	0.061
Left/Right common, oddball count error	−21	−52	280	6.10	0.062	568	0.055
Neutral common, oddball count error	−21	−46	280	6.41	0.043	897	0.020

Threshold was set at p < 0.001 uncorrected and only clusters that passed family-wise (FWE) cluster threshold of p < 0.1 were included, x position is positive-going left to right, y position is positive-going from posterior to anterior.

Next, we sought to determine whether the pulling-related activity was spatially distinct from early sensory activity. The results must be interpreted cautiously owing to the inverse problem and poor spatial acuity of the EEG signal (Grech et al., 2008). Group inversion of the pulling-related scalp activity (Figure 5D, red and green patches; Table 4) suggested an origin in the left parietal cortex, corresponding to the postcentral sulcus, superior parietal lobule (SPL), and the intraparietal sulcus (IPS). The SPM anatomy toolkit indicated that the cluster was centered on the IPS (Figure 5.; Table 4).

Table 4

Cluster locations for pull related activity (Opposite pull oddballs vs. Pull oddball after neutral) and Neutral common P50 activity, based on maximum probability maps

Contrast	Assignment based on Maximum Probability Map	Percent of Cluster volume in Area	Percent of Area activated by Cluster	Ratio
Pull related activity	Area 7A (SPL)	11.6	13.5	1.13
	Area 2	8	15.5	1.01
	Area hIP2 (IPS)	7.2	25.7	1.77
	Area hIP6 (IPS)	5.9	15.4	1.33
	Area hIP3 (IPS)	4.6	11	0.81
Neutral common P50	Area 2	15.8	30.6	1.23
	Area 3b	11.7	16.6	1.28
	Area 7A (SPL)	9.3	10.9	1.48
	Area 4p	6.6	24.9	1.55
	Area 3a	5.2	23.8	1.37

Estimates of activated areas were based on clusters of 2012 (p < 0.05) and 2018 (p < 0.0006) voxels, respectively. Only the five areas calculated to be the most likely origin for activity area shown, sorted according to percent of the cluster volume found in each area. Note that high ratio values indicate higher probabilities that the cluster had an origin in the specific brain area (see STAR Methods for details).

For comparison, we analyzed the approximate location of SI using the Neutral common condition P50 activity, as the P50 has been shown to have its main origin inside SI (Allison et al., 1992). This cluster was found to be located somewhat anterior, though partially overlapping with, our pulling-related cluster of activity (Figure 5D, blue and green patches; Table 4). The SPM anatomy toolkit indicated there was some P50 activity in the postcentral sulcus and SPL, as with the pulling-related activity. However, unlike the pulling-related activity, the P50 cluster was not strongly represented in the IPS, and instead was represented in the postcentral gyrus, consistent with the approximate location of the SI hand area (Holmes et al., 2019).

Thus, while the brain potentials that best reflect the pulling sensation, namely the P200 and to a lesser extent the P3b, are generated at multiple sensory, frontal, and parietal sites in the brain, our analysis indicates that it is parietal lobe activity that most closely correlates with the pulling sensation. The spatial location of this activity suggests that the pulling sensation necessitates activity beyond SI in parietal association areas.

Lack of significant activity when compared to neutral oddballs

We did not observe any significant clusters of activity when the “Neutral oddball” condition was compared to either of the other two oddball conditions using the SPM scalp analysis. This may in part have been owing to the large P3b generated by Neutral oddballs (Figure 4.) owing to their inherent uncertainty, a factor known to amplify long-latency ERPs (Furl and Averbeck, 2011; Kopp et al., 2016; Stern et al., 2010), which likely obscured any differences between the “Neutral oddball” and “Opposite pull oddball” conditions.

Discussion

Understanding how pulling sensations are generated has previously been confounded by correlated but unrelated sensory, motor, and attentional processes. Here we used asymmetric vibration-induced illusory pulling sensations embedded within an oddball task to negate many of these issues and reveal for the first time the brain activity particular to the sensation of a directional tactile pull. We found that the P200, and specifically activity 264-320ms in the parietal cortex, was related to the pulling sensation. Our results indicate that the pulling sensation does not emerge during early processing in SI or during later sensory elaboration involving SII and the PFC. But rather the pulling sensation depends on amodal processing beyond sensory regions, likely related to tactile feature extraction and spatial processing.

The pulling sensation necessitates activity beyond somatosensory cortex

Prior work on tangential forces indicated that activity in SI (Backlund Wasling et al., 2008; Fortier-Poisson et al., 2016; Salimi et al., 1999), combined with peripheral filtering (Birznieks et al., 2001; Pruszynski and Johansson, 2014), could be sufficient to generate the pulling sensation. Indeed, research using the cutaneous rabbit to separate tactile sensations from physical stimulation has found that activity in SI underpins such illusions (Blankenburg et al., 2006). However, our results suggest that although early processing in SI is undoubtedly necessary, it is unlikely to be sufficient for the pulling sensation to emerge.

The midline P200 best reflected the emergence of the pulling sensation. Topographically our P200 was similar to prior reports describing bilateral sensory, parietal, and lateral and medial frontal sources (Allison et al., 1992). This suggests that, like other forms of tactile discrimination, the pulling sensation may depend on a network of sensory and amodal regions (Bodegård et al., 2001; Mueller et al., 2019; Romo and Salinas, 2003). Moreover, when the entire response window was analyzed in an unbiased manner, pulling-related activity was found to partially overlap with SI, but was notably posterior, and was estimated to most likely originate in the IPS. Thus, while a network of regions including SI may contribute to the pulling sensation, the critical activity appears to be in the parietal association cortex.

The temporal dynamics of the pulling sensation

Determining when the pulling sensation emerges can inform theories of the somatosensory system (Johansson and Flanagan, 2009) and is of practical importance for researchers working on BCI, wearable haptics, and prostheses (Bensmaia and Miller, 2014), who need to decode environmental inputs within critical time-windows to aid perception and movement. Previously it was unclear if pulling sensations arise during time periods associated with mid-latency P50 and N140 potentials, or later midline P200 and P3b potentials.

The N140 has been associated with many somatosensory processes including tactile awareness (Forschack et al., 2020; Schröder et al., 2021), texture processing (Genna et al., 2016, 2017), endogenous and exogenous attention (Nakajima and Imamura, 2000), and spatial attention to locations on the body (Forster et al., 2016; Forster and Eimer, 2004). However, we found that N140 amplitude was not significantly different for our critical comparison between the “Opposite pull” and “Pull after neutral” oddball conditions. This non-significance could be ascribed to a lack of statistical power, but for the fact that Neutral oddballs showed significantly enhanced N140 amplitudes compared to these two oddball conditions. Thus, our experiment was sensitive to pulling-related N140 differences, but they were not present for the critical pulling-related comparison. In this context, the N140 may, therefore, reflect neural activity coding for the absence of an expected pulling sensation or activity in SII and the PFC related to frequency discrimination (Allison et al., 1992; de Lafuente and Romo, 2006; Valeriani et al., 2001), which may be enhanced for symmetric relative to asymmetric vibration.

Pulling-related P200 activity was enhanced in the “Opposite pull oddball” condition relative to the other oddball conditions. In the visual domain, where more research has been conducted, the P200 apparently tracks the activity of circuitry involved in feature detection, such as the extraction of orientation (Luck and Hillyard, 1994), direction (Grzeschik et al., 2016; Martin et al., 2010) and depth information (Omoto et al., 2010). The tactile P200 is modulated by multisensory integration of bodily processing (Harjunen et al., 2017), and tactile remapping across spatial reference frames (Bufalari et al., 2014; Longo et al., 2012). Tactile remapping is a complex process, operating across multiple timescales (Sambo and Forster, 2009; Soto-Faraco and Azañón, 2013), a key feature of which may be the transformation of sensory inputs into more amodal, spatial representations in the parietal lobe (Azañón et al., 2010; Heed et al., 2015; Ritterband-Rosenbaum et al., 2014). A similar re-coding from sensory to amodal representations may occur with the pulling sensation, dependent on, or concurrent with, the extraction of orientation and direction information from the stimulus.

Pulling-related activity was found 264-320ms post-stimulus. Such activity can be temporally dissociated from predominantly feedforward processing in SI and SII (<70ms), and early parietal activity (70-100ms) linked to the unusualness of the stimulus during oddball tasks (Huang et al., 2005). The timing is consistent with other somatosensory illusions, such as activity 200-300ms post-stimulus associated with the rubber hand illusion (Guterstam et al., 2019; Rao and Kayser, 2017). This time-window may, therefore, mark a point in the somatosensory processing stream when skin-centered processing interacts with spatial processing related to the body schema. It remains unclear if such processing should be considered part of the P200 or represents dissociable activity marking the P200-P3b transition, as some researchers consider the tactile P200 to extend beyond 300ms (Ballesteros et al., 2009).

After the pulling sensation has been generated, stimulus classification and memory updating can begin; processes indexed by the P3b (Polich, 2007). Earlier and larger P3b responses for opposite direction oddballs were probably observed because the extracted oddball pulling direction was maximally different from the common stimulus extracted direction (Miltner et al., 1989; Nakajima and Imamura, 2000), and as such were likely a consequence rather than a cause of the pulling sensation.

Toward a somatosensory association cortex account of the pulling sensation

Pulling-related activity originated in the parietal association cortex, from an area covering the postcentral sulcus, IPS, and SPL. Of note, the postcentral sulcus is considered a key somatosensory area for proprioception and vibration-based illusions (Chowdhury et al., 2020; Ehrsson et al., 2005; London and Miller, 2013; Soechting and Flanders, 1989). Meanwhile, the IPS is critical for orientation processing and anticipatory grip control (Ehrsson et al., 2003; Hadjikhani and Roland, 1998; Leoné et al., 2015; Van Boven et al., 2005) and the SPL is involved in spatial cognition (Colby and Goldberg, 1999; Sack, 2009), body location processing (Graziano et al., 2000) and transformations of sensory input into body-centered reference frames (Gallivan et al., 2009; Lacquaniti et al., 1995).

EEG scalp potentials have low spatial acuity and predominately reflect the output of pyramidal neurons. They must be interpreted cautiously and do not represent the totality of neural processes underlying the pulling sensation. Nevertheless, we speculate that activity in the postcentral sulcus might constitute a proprioceptive signal that the hand is stationary, a key subjective feature of the pulling sensation differentiating it from other illusory external force sensations (De Havas et al., 2017, 2018). It could be that the IPS contributes to the pulling sensation by extracting the orientation of the illusory force vector, while nearby populations of cells in the SPL are involved in mapping the extracted force vector in hand or externally centered coordinates. Further work is needed to test this account.

Limitations of the study

Comparing opposite direction oddballs to the same oddballs after neutral stimuli was designed to isolate activity specific to having directional pulling expectations violated by directional pulling sensations. We are, therefore, acknowledging an inherently predictive account of perception (Berthoz, 2000; Friston, 2005). However, it is difficult to fully exclude activity related to the process of comparison itself (Camalier et al., 2019; Garrido et al., 2009). Indeed, within a Bayesian framework, an argument could be made that “Opposite pull oddballs” produced greater parietal activity than the same oddballs in the context of neutral common stimuli, because in the latter condition the neutral common stimuli formed weaker priors leading to a smaller response when the priors were confounded by the oddball. This argument, however, is not wholly convincing because opposite pull oddballs produced a larger P200 response than neutral oddballs, despite having matched common stimuli, and thus the same priors. Additionally, neutral common stimuli produced slightly larger ERPs than pulling common stimuli, suggesting comparable or greater salience, and rendering weaker priors doubtful.

Conclusion

We found evidence that activity in SI is not sufficient to generate the pulling sensation. Rather, pulling sensations are associated with an enhanced P200 response, suggesting that detecting directional pulls depends on activity in a network composed of sensory and amodal parietal regions. Specifically, pulling-related activity was found 264-320ms post-stimulus in the parietal association cortex, localized to areas involved in processing proprioception, tactile orientation, and peripersonal space. This first step toward a spatiotemporally precise account of the pulling sensation will aid translational approaches related to delivering complex tactile feedback in prosthetic devices (Lucarotti et al., 2013; Raspopovic et al., 2014) and the development of handheld vibration devices for gaming, navigation and guiding the visually impaired (Amemiya and Sugiyama, 2010; Gomi et al., 2019; Takamuku et al., 2016), as well as furthering a more complete account of parietal lobe function.

STAR★Methods

Key resources table

Resource availability

Lead contact

Further information and requests for resources should be directed to the lead contact, Jack De Havas (jdehavas@gmail.com).

Materials availability

This study did not generate new unique reagents.

Experimental model and subject details

We recruited 15 participants (10 males, 5 females, mean age = 33.33 years, SD = 7.33 years). All participants were right handed. The sample size was chosen based on previous asymmetric vibration and somatosensory oddball EEG studies (Akatsuka et al., 2007; Amemiya and Gomi, 2016; Restuccia et al., 2009; Spackman et al., 2007). Experiments were undertaken with the understanding and written consent of each participant in accordance with the Code of Ethics of the World Medical Association (Declaration of Helsinki), and with the NTT Communication Science Laboratories Research Ethics Committee approval.

Method details

Experimental apparatus

Participants were seated at a table approximately 40cm from a computer monitor with their right forearm resting on an adjustable arm rest (Figure 1E.). View of the right arm was obscured by a dividing screen. Symmetric and asymmetric vibration stimuli were delivered by a small, coin-sized device (Amemiya et al., 2005; Amemiya and Gomi, 2014) covered with grip tape (sandpaper grit density = #400) that was held between index finger and thumb in a pinch grip (Figure 1B.). An accelerometer (356A03, PCB Piezotronics, Inc., New York, USA; sampling frequency = 4000Hz) was attached to the device. Accelerometer signals were displayed to the experimenter via an oscilloscope (TDS2004C, Tektronix, Inc., Oregon, USA), for the purposes of checking that the correct conditions were being administered at all times. Accelerometer signals were also recorded so that the precise stimulus onset time could be determined for every trial (Figure 1C.). Participants wore earplugs throughout the experiment to prevent auditory cues relating to the vibration conditions. Vibration onset timing, accelerometer recording, task instructions and fixation crosses were controlled via MATLAB (2017a) and Psychtoolbox (Brainard, 1997). Visual stimuli were displayed via a flat screen monitor (27-inch LCD, 1902 x 1080 pixels, 60 Hz refresh rate). EEG data were acquired via a 129 electrode net (HydroCel GES 300, MagstimEGI, Oregon, USA). Data were acquired at 1000Hz and Net Station EEG software (Magstim EGI, Oregon, USA).

Procedure

Vibration stimuli were generated by a solenoid actuator within a small device held in a pinch grip. Vibration was generated by a magnet anchored between a pair of springs surrounded by a solenoid (Figure 1B.). The magnet oscillated (100Hz) left and right in response to current passing through the solenoid. Leftward acceleration of the solenoid means the fingers receive a rightwards force (and vice versa). Current was delivered to the solenoid using a modified square wave. By varying the ratio of the positive and negative current duration, we generated either symmetric or asymmetric left-right acceleration profiles (Figure 1C) which were supplied to the finger skin surface. Under conditions of asymmetry, one force direction is rendered large and brief, while the other is small and prolonged. Due to nonlinearity in the perceptual system, only the larger of the forces is perceived despite the temporally-integrated forces in each direction being equal physically (Amemiya et al., 2005; Amemiya and Gomi, 2014). When symmetric vibration is used both force patterns are equal and then simple vibration is felt perceptually. Three forms of vibration were used throughout the experiment; asymmetric left (left pull), asymmetric right (right pull) and symmetric vibration, which is referred to as ‘Neutral’ (Figure 1C.).

First, an accuracy test was administered, in which participants received 100ms bursts of vibration and had to discriminate asymmetric (pulling) from symmetric (neutral) vibrations in a two alternative forced choice task. Responses were given with the left hand via keypad. Left and right pulling stimuli were tested in separate blocks (50 randomized trials per block, 25 per condition; block order counterbalanced). A subset of participants (n = 9) were also required to discriminate between left and right pulling stimuli under the same task conditions.

The main oddball task consisted of vibration stimuli delivered with a randomized ISI of 800ms–1100ms (Figure 1A.). Vibration stimulus duration was always 100ms. In each block, one vibration pattern (Left, Right or Neutral) was the common stimulus (80% of trials) and the other two were the oddballs (each 10% of trials; total oddball = 20%). Trials were pseudorandomized, such that the first trial of every block was a common stimulus and that every oddball was followed by a common stimulus. Each block consisted of 200 trials (common = 160, oddball A = 20, oddball B = 20). There were 15 blocks in total which were randomized and counterbalanced across participants (5 blocks for each of the 3 block types, defined according to the common stimulus, i.e. Left, Right and Neutral). Thus, in total there were 9 conditions, composed of three common stimuli conditions (Left, Right, Neutral, 800 trials per condition) and 6 oddball stimuli conditions (120 trials per condition). Oddball conditions were grouped into three conditions (Figure 1D.): ‘Opposite pull oddballs’ (Right oddballs during Left common and Left oddballs during Right common), ‘Pull oddball after neutral’ (Right oddballs during Neutral common and Left oddballs during Neutral common), and ‘Neutral oddball’ (Neutral oddballs during Left common and Neutral oddballs during Right common).

Participants were informed at the start of each block which stimulus was the common and which two were the oddball. They were instructed to pay attention to all stimuli and silently count the number of oddballs. At the end of each block they reported their estimate for the number of oddballs by responding to options presented on screen. Thus, they always simultaneously responded to two oddball conditions, helping to ensure that their effort levels were well controlled across conditions. Participants were naive to the purpose of the experiment when asked directly at the end of testing. The experiment lasted ∼2.5 h.

Quantification and statistical analysis

Behavioral data

Accuracy on the pre-test pulling direction two alternative forced choice (2AFC) discrimination task was determined for each participant by taking the sum of correctly identified pulling and neutral stimuli as a percentage of the total number of trials. Bias was calculated by taking the percentage of each response (i.e. how often they response Left or responded Neutral). Values significantly above 50% across participants indicated a response bias. Left and Right pulling conditions were calculated separately and compared via paired sample t-test. In the subset of participants (n = 9) who also completed a Left vs. Right pull discrimination block, we calculated the percentage correct in the same manner and compared this value to the mean of the Left vs. Neutral and Right vs. Neutral values via paired sample t-test. We also performed a Bayesian analysis (Dienes, 2014) using the mean difference between the Left/Right and Pull vs. Neutral conditions to specify the plausible expected effect size. This comparison was used because it represented the maximal difference between the three types of stimuli (i.e. Left, Right and Neutral) when they were combined into separate 2AFC tasks. Thus, this difference was a reasonable benchmark to determine whether Pull vs. Neutral null results were genuine. We compared performance (discrimination accuracy and bias) on the Pull vs. Neutral 2AFC tasks (normal distribution, 2-tailed). SE of the difference between means was adjusted to account for the low sample size (Dienes, 2014) using the formula: Adjusted SE = SE x (1 + (20/(df∗df))).

Oddball counting error was calculated for each block of the main task by taking the absolute of the estimated number of oddballs minus the actual number of oddballs. Oddball counting error was compared across participants via Wilcoxon signed-rank test. We compared blocks where the left pull was the common stimulus to blocks where right pull was the common stimulus, and we compared the average of these two blocks to blocks in which neutral was the common stimulus.

From the two behavioral tasks we extracted 4 variables that were to be used in covariate analyses with the EEG data: pre-test Pull vs. Neutral discrimination (mean of Left vs. Neutral and Right vs. Neutral block), mean oddball counting error (i.e. across all blocks), Left/Right common oddball counting error (mean of performance on blocks where left and right pull were the common stimulus), and Neutral common oddball counting error (Table S1).

EEG pre-processing

EEG data were pre-processed using EEGlab (Delorme and Makeig, 2004) and custom MATLAB (2017a) scripts. The data were down sampled to 250Hz for storage purposes. We re-referenced the data to the left and right mastoid electrodes and applied a bandpass (FIR 0.1–90Hz) and notch filter (48-52Hz). ICA components reflecting blinks, eye movements, heart, large EMG and electrical artifacts were then removed. Epochs were extracted (-200–700ms) and baselined (-100 – 0ms) for each participant. Electrodes from the face and side of the head below the ear were removed due to muscle activity artifacts in some participants, leaving a total of 93 electrodes, covering the entire scalp (Figure 1F.). We removed trials still displaying artifacts via whole brain threshold (+/−80μV) and by applying the ERPlab step function algorithm to frontal electrodes (window size = 200ms, step size = 50ms, threshold = 50μV). The mean percentage of trials rejected per participant was 9.62%. ERPs were averaged across conditions and smoothed using a low pass filter (second order Butterworth, cutoff 30Hz).

Traditional ERP analysis

We used a traditional ERP approach using ERPlab (Lopez-Calderon and Luck, 2014), in which we selected electrode locations and time windows based on previous research (Akatsuka et al., 2005; Allison et al., 1992; Kekoni et al., 1997; Shen et al., 2018), wide ERP windows were favored to avoid biasing conditions where the ERPs were flattened due to greater onset variability (Luck, 2014). Epochs were averaged for each of the 9 conditions and oddball difference waves were calculated by subtracting the activity of each stimulus when it was acting as the common stimulus from the activity of the same stimulus when it was acting as an oddball (Pulvermüller et al., 2006). Left and right directional versions of each oddball difference wave were averaged together to give the final experimental condition (Opposite pull oddball), and two other oddball conditions (‘Pull oddball after neutral’ and ‘Neutral oddball’), oddball difference waves.

Mean amplitude of the P50 (30 - 70ms), N140 (100 - 150ms), P200 (150–250ms) and P3b (250–500ms) event related potentials (ERPs) were quantified from the epoched and difference wave data for each condition. We calculated the onset latency of all ERPs by calculating the point where the signal reached 50% of the peak value within each time window. In line with previous literature, P50 analysis was based on the 6 electrodes surrounding P3, N140 analysis was based on the 5 electrodes surrounding F3, while P200 and P3b analysis was based on the 5 electrodes surrounding Cz. ERP measures were compared across conditions via paired sample t-tests. Our main comparison concerned the ERP responses in the ‘Opposite pull oddball’ condition compared to the ‘Pull oddball after neutral’ condition, however we also compared the ‘Opposite pull oddball’ condition to the ‘Neutral oddball’ condition and compared the ‘Pull oddball after neutral’ condition to the ‘Neutral oddball’ condition.

EEG analysis in surface space using SPM

To avoid the bias inherent to picking electrodes and time widows, we also used SPM 12 for M/EEG (Litvak et al., 2011) to analyze scalp data across the response window and then to perform source reconstructions of scalp activity. SPM controls for multiple comparisons using Random Field Theory (RFT), which is effective because of the temporal and spatial smoothness of EEG data (Kilner and Friston, 2010). Statistical parametric maps were created for each participant in each condition by interpolating from all electrodes into two-dimensional sensor space across the response window (0-500ms post stimulus onset), thus creating a 3D characterization of the ERP (16 mm × 16 mm x 0ms smoothing).

To determine if ‘Opposite pull oddball’ produced a larger response than the ‘Pull oddball after neutral’ conditions, it was only necessary to perform a paired t-test because both oddball conditions used the same common stimulus condition (i.e. common pull). This t-test was 1-tailed because our hypothesis only pertained activations that were larger in ‘Opposite pull oddball’ condition, not those that were larger in the ‘Pull oddball after neutral’ condition. We also ran the same t-contrast using our behavioral variables as covariates. However, we also wanted to check if there were any differences between the ‘Neutral oddball’ condition and the other two oddball conditions, for which we needed to use a partitioned error approach (random effects analysis), combined with two separate 2 × 2 within-subject’s ANOVAs with factors of oddball condition (Neutral oddball vs. Opposite pull oddball or Pull oddball after neutral) and stimulus type (oddball vs. common). For these analyses image files (SPM maps; NIFTI) were transformed into four sets of differential effects (overall effect, main effect of condition, main effect of type, condition × type interaction) for each participant (1st level contrasts), which were then entered into four separate one-sample t-tests (2nd level contrasts; for details see Franz et al., 2020). Of these contrasts, only the condition × type interaction was of interest because this contrast showed the effect of the oddball condition, whilst controlling for the common stimulus. For all scalp activity contrasts we used a threshold of p < 0.001 uncorrected and clusters were only included if they met the more stringent p < 0.05 family-wise cluster threshold.

EEG source localization

To locate the possible cortical origins of activity detected on the scalp we ran SPM 3D source reconstruction, using a group inversion approach (COH, 0-500ms, Hanning taper, 0-256Hz) to compensate for head anatomy and sensor noise variation (Litvak and Friston, 2008). An MNI template was used to construct the mesh, coregistration used the nasion and bilateral preauricular points as fiducials, and a forward model was created with the Boundary Elements Model (BEM). NIFTI (source-level) images (8mm smoothing) were extracted using a time window derived from the ‘Opposite pull oddball’ vs. ‘Pull oddball after neutral’ scalp analysis (264-320ms). To better refine the location of the activity, NIFTI images were subjected to a paired sample t-test with a general threshold set at p < 0.05 uncorrected, and selected the top cluster of activity (i.e. the cluster that contained the highest peak t-values). Due to the problem of circularity, this statistical test was used purely to better locate the already observed scalp effect (Oh et al., 2020), and to negate the issue of central attraction during source analysis, whereby at the group-level sources can tend to accumulate in biologically implausible central regions of the brain.

To better understand the location of our cluster of pulling-related activity, we compared it to the origin of the P50 generated in response to the Neutral common stimuli (an ERP independent of the main ‘Opposite pull oddball’ vs. ‘Pull oddball after neutral’ comparison), since the P50 is known to originate in SI (Allison et al., 1992). For this visualization we ran the same SPM group inversion but using a window of -100-100ms and contrasted (paired sample t-test) the baseline period (-100-0ms) with the P50 window (40-64ms) in the Neutral common condition, threshold at p < 0.0006 uncorrected. The threshold was chosen so that the top cluster contained approximately the same number of voxels (2018 voxels) as the ‘pulling-related activity’ cluster (2012 voxels).

The location of these clusters of brain activity was compared using the SPM anatomy toolbox (Eickhoff et al., 2005), which provides a list of brain areas ranked according to the likelihood that the observed activity originates within their probabilistically defined boundaries. We considered the top five areas to be representative of the cluster origin, given the spatial limitations of EEG. Ratios are calculated automatically by the toolbox for each area by dividing the mean probability at cluster location by the mean probability across the entire probability map of the brain. Higher values indicate location more toward the center of the area.

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Software and algorithms
MATLAB (2017a)	MathWorks	RRID: SCR_001622
PsychToolbox	http://www.psychtoolbox.net	RRID: SCR_002881
EEGLAB	http://sccn.ucsd.edu/eeglab/index.html	RRID: SCR_007292
ERPLAB	http://erpinfo.org/erplab	RRID: SCR_009574
SPM	https://www.fil.ion.ucl.ac.uk/spm/	RRID:SCR_007037
SPM Anatomy Toolbox	http://www.fz-juelich.de/ime/spm_anatomy_toolbox	RRID:SCR_013273