Enhanced perceptual processing of self-generated motion: Evidence from steady-state visual evoked potentials.

The sense of agency emerges when our voluntary actions produce anticipated or predictable outcomes in the external world. It remains unclear how the sense of control also influences our perception of the external world. The present study examined perceptual processing of self-generated motion versus non-self-generated motion using steady-state visual evoked potentials (SSVEPs). Participants continuously moved their finger on a touchpad to trigger the movements of two shapes (Experiment 1) or two groups of dots (Experiment 2) on a monitor. Degree of control was manipulated by varying the spatial relation between finger movement and stimulus trajectory across conditions. However, the velocity, onset time, and offset time of visual stimuli always corresponded to participants' finger movement. Stimuli flickered at a frequency of either 7.5 Hz or 10 Hz, thus SSVEPs of these frequencies and their harmonics provided a frequency-tagged measurement of perceptual processing. Participants triggered the motion of all stimuli simultaneously, but had greater levels of control over some stimuli than over others. Their task was to detect a brief colour change on the border(s) of one shape (Experiment 1) or of one group of dots (Experiment 2). Although control over shapes/dots was irrelevant to the visual detection task, we found stronger SSVEPs for stimuli that were under a high level of control, compared with the stimuli that were under a low level of control. Our results suggest that the spatial regularity between self-generated movements and visual input boosted the neural responses underlying perceptual processing. Our results support the preactivation account of sensory attenuation, suggesting that perceptual processing of self-generated events is enhanced rather than inhibited.

Introduction

Many events occurring in the environment are the consequences of human action. When an event temporally and spatially matches one's actions, it may provoke a subjective feeling of control over that event. The subjective feeling of controlling one's own movements, and through them, the external events, is often called the sense of control. The sense of control is an effective cue for self-recognition (Salomon et al., 2013; Tsakiris et al., 2010; Wen and Haggard, 2018). For example, when one watches a crowd of people on a live TV screen, the image of oneself might be noticed more readily because its movements match one's own. A previous study reported an effect of “self pop-out”: When people actively moved their limbs, self-recognition among four or six avatars was faster and was not influenced by the number of distractors, compared with a condition when their limbs were moved by someone else (Salomon et al., 2013). Self-generated motion, even when it is displayed on external objects, is considered to hold an advanced status in visual processing (Salomon et al., 2011).

The term ‘sense of agency’ is also used in the literature to represent the subjective feeling that one is the agent of actions and action consequences. The sense of control and the sense of agency are common in daily life. The sense of control is usually used to describe the feeling that emerges from movements and observations of proximal effects, while sense of agency may implicate more high-level factors, such as inference, expectation, and belief. The most widely accepted theory of how sense of control arises is the so-called comparator model. In this model, a prediction is generated from an efference copy of a motor command, and is compared with the actual sensory feedback; the sense of control arises from a match, and diminishes if they mismatch (Blakemore et al., 1999; Blakemore et al., 1998; Frith et al., 2000; Wolpert and Flanagan, 2001). Moreover, previous studies showed that predictions based on motor commands lead to a phenomenon known as sensory attenuation (Blakemore et al., 1998): people are less sensitive to a stimulus that is self-produced than one that is externally caused. Sensory attenuation has not only been found in somatosensory (Bays et al., 2005; Blakemore et al., 1999, 1998; Shergill, Bays, Frith, & Daniel M., 2003), but also in visual perception (Cardoso-Leite et al., 2010). For instance, Cardoso-Leite et al. (2010) reported that people were less sensitive in detecting a low-contrast Gabor patch when the orientation of the Gabor patch was congruent with a learned action, compared to when the orientation was incongruent with the action. Further, electrophysiological studies have shown that amplitude peaks of event-related potentials (ERPs, e.g., N1, P300) are smaller for self-produced than other-generated stimuli (Baess et al., 2011; Bednark and Franz, 2014; Gentsch et al., 2012; Gentsch and Schütz-Bosbach, 2011; Hughes and Waszak, 2011; Kühn et al., 2011; Poonian et al., 2013; Timm et al., 2013). The attenuation of self-produced effects cannot be explained by strategic allocation of attention (Timm et al., 2013), and the internal predictive mechanisms affecting perceptual processing still remain poorly understood.

In summary, on the one hand, previous studies on self-recognition showed that self-produced effects (e.g., motion) enjoy more attentional resources and a temporal advantage in processing (Gozli et al., 2016; Kumar et al., 2015; Salomon et al., 2013, 2011). This is consistent with the self-advantage effect found in many other cognitive fields, such as memory and visual recognition (Keenan et al., 1999; Rogers et al., 1977; Tacikowski and Nowicka, 2010; Whiteley et al., 2008). On the other hand, sensory attenuation has been widely reported for self-generated events (e.g., Bays et al., 2005; Blakemore et al., 1999, 1998; Cardoso-Leite et al., 2010; Shergill et al., 2003).

These two phenomena (i.e., the observed sensory attenuation and sensory enhancement for self-generated or self-related stimuli in different tasks) seem incompatible, but they may actually be two sides of the same coin: They can both be explained by a preactivation account (Roussel et al., 2013; Waszak et al., 2012). Specifically, the classic inhibition account of sensory attenuation suggests a reduction in response to the signal, due to cancellation against feedback predicted from the motor command (Fig. 1A, Bays and Wolpert, 2007; Blakemore et al., 1998; Wolpert and Ghahramani, 2000). In contrast, the preactivation hypothesis suggests that the preactivation of action consequences increases both baseline activation (Kühn et al., 2010) and the internal response of the signal (Fig. 1B, Roussel et al., 2013; Waszak et al., 2012). However, according to the Weber-Fechner law (Dehaene, 2003; Fechner, 1860; Gorea and Sagi, 2001), the increment in the baseline is larger than that in the response of the signal, ultimately resulting in a decrement in sensitivity of discrimination. The preactivation theory is supported by empirical evidence that action-congruent effects benefit from increased attentional allocation, compared with action-incongruent effects (Desantis et al., 2014; Gozli and Ansorge, 2016; Gozli et al., 2016). However, it remains unknown if our neural systems actually process self-produced effects to an enhanced level.

Fig. 1

Illustration of noise and signal distributions for the inhibition hypothesis (a) and preactivation hypothesis (b). According to the inhibition hypothesis, the internal response for the signal is inhibited by the sensory prediction, resulting in weaker response and less sensitivity in discrimination from the baseline. In contrast, according to the preactivation hypothesis, predicted sensory feedbacks result in enhanced activity for both the baseline and the internal response of the signal, with a larger effect in the baseline, resulting in reduced sensitivity in perceptual discrimination. Modified from “A preactivation account of sensory attenuation,” by Roussel, C., Hughes, G., & Waszak, F., 2013, Neuropsychologia, 51, 933-929.

The present study focuses on visual processing of self-generated motion. Self-generated motion usually contains continuous sensory input, and is an important cue for identifying the self in the external world (Bigelow, 1981; Sugiura et al., 2006). A previous study showed that objects whose movements were self-controlled were processed more accurately than objects that were not self-controlled (Salomon et al., 2011). In contrast to prior behavioural studies, the present study aims to examine the perceptual processing of self-generated motion using a neural measurement.

To measure the neural responses underlying the perceptual processing of continuous events, steady-state visual evoked potentials (SSVEPs) are a powerful measurement. SSVEPs are electroencephalogram (EEG) signals provoked by the neural response to repeated visual stimulation at a specific frequency (D. Regan, 1977). They are sensitive to both top-down attention (Morgan et al., 1996) and to the intensity of the stimulation e.g., contrast (Andersen et al., 2012; see Norcia et al., 2015 for a review). Compared with other widely used ERP techniques, SSVEP has a high signal-to-noise ratio, and is ideal for measuring level of neural processing of sensory input. To our knowledge, the SSVEP technique has not previously been used as an index of how self-generated action influences sensory processing.

Previous studies showed that SSVEPs, particularly the intermodulation responses (the sum or the difference of two flicker frequencies) (Cunningham et al., 2017), are increased by the grouping of multiple stimuli, such as Gestalt-grouped visual stimuli (Alp et al., 2016; Baker et al., 2011; Boremanse et al., 2013; Gundlach and Müller, 2013), bound motions (Aissani et al., 2011; Alp et al., 2017), and visual-auditory synchronization (Nozaradan et al., 2012). However, to the best of our knowledge, no previous study has examined whether the binding of a voluntary action and its corresponding effect enhances neural responses. The temporal binding of a voluntary action with its outcome is a well-known effect of the sense of control (Haggard et al., 2002), which could potentially produce enhanced neural activities for both the action and the effect. This view would predict an enhanced SSVEP for visual stimuli that are under control, consistent with the preactivation hypothesis rather than the inhibition hypothesis.

In the present study, participants moved their right index finger on a touchpad to simultaneously trigger the movements of two objects on a screen. The two objects' movements were identical in velocity, onset, and offset, corresponding to participants' finger movements, but differed in spatial relation to the participant's finger movements (Fig. 2). Specifically, the trajectory of one object was under better control (e.g., when the finger moved to the left, the object also moved to the left), while that of the other object was under poorer control (e.g., when the finger moved to the left, the object moved mainly downward and only slightly to the left). Importantly, participants' task was to detect a very short colour change on the border of one object. Therefore, this task was a perceptual task, and motion or level of control was irrelevant. We used frequency-tagging to measure SSVEPs related to the two objects when they were under relatively higher versus lower level of control. SSVEPs are sensitive to top-down attention (Morgan et al., 1996), and self-generated motion takes advantage in attentional allocation (Salomon et al., 2013, 2011). Therefore, in order to measure the influence of such top-down monitoring on controlled objects in our paradigm, we also designed a condition in which the two objects were under two different levels of control (high vs low) for the first 5 s, and then changed to the same median level of control. The logic is that people should be able recognise self-generated motion in the first 5 s, and, if they tend to intentionally monitor the object that is (or was) under stronger control, neural responses should remain stronger for the object that was under better control, even after the actual level of control shifted to match the other object. On the other hand, if visual processing depends only on instantaneous level of control, rather than prior control, neural responses should not differ between the two objects in the post-shift period when level of control are identical.

Fig. 2

The procedure of generating a 50/50 mixed movement from the participant's finger movement and a pre-recorded non self-movement. (A) is an example of the participant's finger movements. 1, 2, 3 indicate samples at a time sequence (the intervals depend how fast one moves the finger) (B) is an example of a pre-recorded movement generated by another individual in similar circumstances. To generate a mixed movement, the pre-recorded movements were first normalized to the magnitude of the finger movement (C). Then the normalized movements were mixed with the finger movements at a certain ratio (50/50 in this example) at each moment when the participant moved the finger (D). Finally, the mixed movement were normalized again to match the magnitude of the finger movements.

Experiment 1

Methods

Participants

Sixteen healthy volunteers were recruited from the university participant database. One participant was excluded because of a technical problem in EEG recording, resulting in a final sample size of 15 (mean age = 27.6, range 20–40, SD = 6.7, 8 females). We could not perform a power calculation because no previous studies, to our knowledge, had looked at the influence of spatial regularity between voluntary actions and visual stimuli on SSVEPs. However, our sample size is comparable to other cognitive SSVEP studies (e.g., Liu-Shuang et al., 2014; Rossion and Boremanse, 2011; Rossion et al., 2012). All the participants were right-handed, reported normal or corrected-to-normal visual acuity and motor ability, and reported no neurological abnormalities. The study was approved by the local ethics committee (University College London), and all participants provided informed consent prior to participation.

Task

Fig. 3 shows the timeline of the experimental trial. Participants pressed the space key on the keyboard to start the trial when they felt ready, and waited for the visual stimuli to appear. After 2 s, a 34.7-mm (132-pixel, visual angle: 4.0°) circle and a 31.6-mm (120-pixel, 3.6°) square appeared symmetrically at an equal distance of 36.8 ± 5.3 mm (140 ± 40 pixels, 4.2 ± 0.6°) from the centre of a 17-inch LCD screen (width × height: 338 mm × 270 mm, 1280 pixels × 1024 pixels, refresh rate = 60Hz). The slightly different sizes of the circle and square were designed to equalise the perception of shape size. The two shapes were presented in white on a black background, and flickered at different frequencies, either 7.5 (f1) or 10 Hz (f2). The on/off duty cycles were 50/50 for both frequencies. Participants started to move their right index finger on a touchpad as soon as they saw the two shapes. They were instructed to make a smooth and continuous motion, to pay equal attention to both of the shapes, and to detect a brief (400 ms) red flashing border (border width = 0.53 mm) on one of the shapes. The red border flickered at the same frequency as the object. Participants were told that the red border would not appear if the shapes were not moving. They were also instructed to continue moving their finger until the shapes disappeared from the screen, even after having seen the red border. The onset of border flashing occurred between 15 and 16 s with a probability of 10%, between 16 and 17 s with a probability of 10%, and between 17 and 19 s with a probability of 80% after the first onset of the finger movement. The motion of the stimulus was a mixture of the real-time finger movement and a pre-recorded movement at a certain ratio, depending on the level of control (Fig. 2). The pre-recorded movements were stored as (x, y) series, then as long as the participant's finger continued to move, the next data point from the stored (x, y) series was retrieved and used to update the object position on the screen. At each moment during the finger movement, the shift of the cursor position (NB: the cursor was invisible during the task) in x- and y-axes was mixed with a section randomly chosen from 10 000 pre-recoded continuous movements. The mixture of the two movements always involved a certain ratio (e.g., 85/15 in the condition of 85% control). Different sections of pre-recorded movements were applied to each object. Therefore even when two objects were under the same level of control (e.g., 50%), they moved in different directions. During the calculation of stimulus movement, the magnitude of finger movement was reduced to 1/10 to prevent excessive movement of the stimuli. The pre-recorded movement was normalized to a comparable magnitude of the finger movement. The onset, offset, and velocities of all the visual stimuli corresponded to the finger movement, regardless of the level of control. The two shapes disappeared from the screen 20 s after the first onset of the finger movement. Participants then pressed one of two labelled keys to indicate which shape's border changed during the trial. They were not given any feedback about their response. After their response, “next trial” was displayed on the screen for 500 ms, followed by the start screen of the next trial.

Fig. 3

Timeline of the task in Experiment 1. Participants pressed the space key to start the trial, and moved their right index finger on a touchpad to trigger the movement of two blinking shapes on the screen. They then reported the shape whose border colour briefly flashed to red during the trial. There were three different types of trial. In the ‘different control’ trials, participants had 85% control over one shape and 15% control over the other shape. In the ‘changing control’ trials, participants had 85% and 15% control over the two shapes, respectively, for the first 5 s, and then had 50% control over both shapes. In the ‘equal control’ condition, participants had 50% control over both shapes.

There were three different types of trial. In the different control trials, participants had 85% control over one shape and 15% control over the other shape. This condition was designed to compare the SSVEPs of self-generated vesus non-self-generated motion. In the changing control trials, participants had 85% and 15% control over the two shapes, respectively, for the first 5 s, and then had 50% control over both shapes. This condition was designed to investigate any persistent effects of having labelled an object as “in control”, assuming control could be detected within the first 5 s. The duration of 5 s was confirmed to be sufficient to differentiate a self-generated motion from a non-self-generated motion in a pilot investigation. In the equal control trials, participants had 50% control over both shapes throughout. Flickering frequencies (7.5, 10 Hz) and (initial) levels of control (15%, 85%) were counter-balanced between trials. The two objects flashed with equal probability. The assignment of shapes (square, circle) was randomized between trials.

Procedure

Participants were tested individually in a sound-attenuated, electrically shielded dark chamber. They sat approximately 50 cm from a 17-inch LCD monitor, wore an EEG electrode cap, and placed their left hand on a keyboard and right hand on a remote touchpad. All keys except the space key and two labelled response key were removed from the keyboard. The touchpad was attached to the desk, aligned with the central line of the monitor and the participant's body. After receiving instructions, participants practiced for six trials, containing 2 trials for each condition in a random order. Participants were instructed to fixate a cross presented in the centre of the screen throughout the trial, without moving their eyes between objects. The experiment contained 3 block, 30 trials per block, including 10 repeats of each condition. The trial order was randomized in each block. Participants took short breaks (3–5 min upon participants' request) between blocks. The experiment lasted for 90 min on average, including set-up of the EEG equipment.

EEG recording

The EEG was recorded with a BioSemi Active-Two amplifier system (BioSemi, Amsterdam, the Netherlands) from F7, F3, Fz, F4, F8, FCz, C5, C3, Cz, C4, C6, CPz, P7, P3, Pz, P4, P8, PO7, PO3, POz, PO4, PO6, O1, Oz, and O2 using Ag-AgCl active electrodes according to the international 10–20 system, mounted in a headcap. Six additional flat electrodes were attached to left-mastoid, right-mastoid, outer canthi of both eyes, and above and below the right eye. Conducting gel was applied to electrodes to ensure good contact to the skin. The offset potentials of all electrodes were kept below ±25 μV. EEG signals were recorded at a sampling frequency of 2048 Hz, and referenced online against the CMS and DRL electrodes of the BioSemi system.

EEG data processing

EEG signals were pre-processed using the EEGLAB toolbox (Delorme and Makeig, 2004) on Matlab R2016a (MathWorks, US). A 1–100 Hz band-pass filter (basic linear finite impulse response (FIR) filter, EEGLAB) was applied to remove slow drifts and high-frequency noises in EEG signals. EEG signals were then re-referenced to the average of the left- and right-mastoids, and segmented into epochs ranging 0–20 s after the onset of the first finger movement. Epochs containing large artefacts (±250 μV, 1.5% of the trials) were discarded, and independent component analysis was used to remove eye movement artefacts. Epochs from the ‘changing control’ condition were trimmed to include only the period after the onset of the change in control. Because Fourier transform results in a spectral resolution inversely proportional to the epoch duration, the epoch length of the ‘changing control’ condition was cut to 14 s which centered the spectral bins on the target frequencies of 7.5 and 10 Hz. A Discrete Fourier Transformation analysis was conducted for each epoch on electrode Oz. Signal-to-noise ratios (SNRs) were then calculated at a window of ±0.25 Hz (±5 frequency bins), and ±0.29 Hz (±4 frequency bins) for the 20-s epochs (‘different control’ and ‘equal control conditions’) and the 14-s epochs (‘changing control’ condition), respectively (Rossion and Boremanse, 2011; Rossion et al., 2012; Srinivasan et al., 1999).

Results

In the ‘different control’ condition, when the flashing shape was under 15% and 85% control, the average accuracy of detection of the border colour change was 95.3% (SD = 6.5%) and 95.0% (SD = 8.2%) respectively. In the ‘changing control’ condition, when the flashing shape was under 15% and 85% initial control, accuracy was 96.2% (SD = 7.6%) and 95.6% (SD = 5.3%) respectively. In the equal control condition, accuracy was 95.6% (SD = 5.1%). A one factor (‘different control’, ‘changing control’, vs ‘equal control’ conditions) repeated-measures ANOVA confirmed that the accuracy of detection did not differ among trial conditions (F (2, 28) = 0.03, p = .969, partial η2 = 0.002). Further, pairwise comparisons between the levels of control (or initial control) over the flashing shape did not reveal any effect of control on visual detection (a Bonferroni-adjusted p of .025 was used, for the ‘different control’ condition, t (14) = 0.11, p = .917; for the ‘changing control’ condition, t (14) = 0.38, p = .713).

Fig. 4, panels A and B, show the amplitude of each frequency in the ‘different control’ and ‘changing control’ conditions. The flicker frequencies of 7.5 and 10 Hz triggered clear and narrow-banded potentials at each fundamental frequency and at harmonics. Interesting, we also observed a large evoked potential at 17.5 Hz (f1 + f2), which was the sum frequency of the two objects. This might arise because the two objects occurred adjacently in the foveal visual field, and were somehow integrated or bound together (Aissani et al., 2011; Alp et al., 2016, 2017; Baker et al., 2011; Boremanse et al., 2013; M. P. Regan, He and Regan, 1995). However, because this was not the interest of the present study, we did not further analyse this sum frequency.

Fig. 4

The SSVEPs of the different control and changing control condition of Experiment 1. (A) and (B) show the amplitude of each frequency in each condition. (C) and (D) show the signal-to-noise ratios (SNRs) for the two types of trials in which the two frequencies (7.5 Hz, 10 Hz) were differently combined with objects that were under different levels of control. (E) and (F) show the SNRs relative to level of control. Error bars represent standard errors.

Because EEG signals have different background activities between different frequencies, and different peak amplitudes of SSVEP have been observed for the same stimuli presented in different frequencies (Srinivasan et al., 2006), we compared the SSVEP amplitude of the same frequency between different trials in which the frequency was associated with high versus low level of control. We divided the trials of each condition into two types. In some trials, the 7.5 Hz object was under 15% control while the 10 Hz object was under 85% control; this was reversed in the other trials. Fig. 4, panels C and D, show the SNRs of frequencies from 1.5 to 40 Hz in 0.5-Hz step for the two types of trials in the ‘different control’ and ‘changing control’ conditions, respectively. Fig. 4, panels E and F sorted the SNRs by level of control (or the initial level of control) for the fundamental frequencies (f1, f2) and the first harmonics (2f1, 2f2). Many previous studies show that the fundamental frequency and the harmonics do not show a simple linear relationship, and external factors such as attention have distinct effects on them (Kim et al., 2011; Langdon et al., 2011; Porcu et al., 2013; Saupe et al., 2009). Therefore, we included the “fundamental vs harmonic” as an independent factor in our analyses.

Regarding the ‘different control’ condition, we conducted a 2 × 2 × 2 (control (85% vs 15%) × flicker frequency (7.5 Hz vs 10 Hz) × “fundamental vs harmonic” repeated-measures ANOVA on the SNRs. Table 1 shows the analysis results. Importantly, the main effect of control was significant, showing that the object under 85% control had stronger SSVEP than that under 15% control. The main effects of flicker frequency and “fundamental vs harmonic” were also significant, showing that the SNRs of 7.5 Hz and 15 Hz were stronger than those of 10 Hz and 20 Hz, and the harmonics had larger SNRs than the fundamental frequencies. These two effects were probably due to large activation and noise in the alpha band. Although the raw SSVEP amplitude was highest for 10 Hz, the SNR at 10 Hz was relatively small compared to other frequencies. In addition, because the three-way interaction was significant, we conducted pairwise comparisons between the levels of control (15% vs 85%) for each frequency of interest (i.e., f1, f2, 2f1, 2f2). The Holm-Bonferroni method was used for four comparisons. The difference in SNRs between the different level of control was significant at 10 Hz (f2) and 15 Hz (2f1) (t (14) = 2.86 and 3.01, p = .013 and .009), but non-significant at 7.5 Hz (f1) and 20 Hz (2f2) (t (14) = 0.12 and 1.98, p = .909 and .068.

Table 1

Outcome of the ANOVAs conducted on the SNRs of the ‘different control’ condition and the ‘changing control’ condition in Experiment 1.

Factor	MSe	df	F	partial η2
Different control condition
Control	.138	1, 14	14.89*	.515
Flicker frequency	.888	1, 14	14.42*	.507
“Fundamental vs harmonic”	.888	1, 14	9.44*	.403
Control × flicker frequency	.151	1, 14	0.02	.002
Control × “fundamental vs harmonic”	.143	1, 14	2.48	.150
Flicker frequency × “fundamental vs harmonic”	2.382	1, 14	3.87	.217
Three-way interaction	.075	1, 14	9.65*	.408
Changing control condition
Control	.155	1, 14	3.58	.203
Flicker frequency	1.032	1, 14	11.46*	.450
“Fundamental vs harmonic”	.712	1, 14	8.60*	.380
Control × flicker frequency	.074	1, 14	4.55	.245
Control × “fundamental vs harmonic”	.101	1, 14	3.06	.180
Flicker frequency × “fundamental vs harmonic”	1.677	1, 14	5.06*	.265
Three-way interaction	.253	1, 14	.16	.012

*p < .05.

Further, in the ‘changing control’ condition, the participants only had different levels of control (15% and 85%) over the two shapes during the first 5 s, and then had equal levels of control (50%) in the following 10 s. If people continued to pay more attention to the object they had previously had more control over, we should have observed stronger SSVEP for this object even after the advantage in control over it disappears – reflecting a persistent effect of control. However, the same repeated-measures ANOVA on the SNRs of the ‘changing control’ condition did not confirm such top-down monitoring on control (Table 1). The main effect of initial level of control was nonsignificant, indicating that the effect of control observed in the ‘different control’ condition was mainly driven by online bottom-up processes. Additionally, the main effects of flicker frequency and “fundamental vs harmonic” were significant, showing the same pattern in EEG signals as the ‘different control’ condition.

Taken together, the above results suggest stronger neural responses for the object that was under a relatively high level of control than that which was under a relatively low level of control. In this task, the two objects usually moved to different positions on the screen during the trial, resulting in different retina positions. SSVEP is sensitive to retina position (Lin et al., 2012). In the ‘different control’ condition, for example, the object that was under a high level of control stayed significantly closer to the central fixation cross than the object that was under a low level of control (average distance from the centre of the shapes to the centre cross (respectively for high/low level of control): 142.5/154.3 pixel, 37.5/40.6 mm, 4.3/4.6°, t (14) = 4.81, p < .001, Cohen's d = 1.24). Hierarchical linear modelling (HLM) of individual results in the ‘equal control’ condition showed a significant negative correlation between the distance from the centre cross and the mean SNRs of the fundamental frequency and the first harmonic (the first level represents within-individual correlations, and the second level represents between-individual variations; for the within-individual correlation, t (874) = 4.25, p < .001). In order to exclude the influence of distance from the centre from the effect of control on SSVEPs, we implemented a structured equation modelling (SEM) for the results of the ‘different control’ condition (Fig. 5). In the model, the actual level of control (85% or 15%) over an object influences both the SSVEP (i.e., average SNR of the fundamental frequency and the first harmonic) and the distance from the centre. Simultaneously, the distance from the centre influences the SSVEP. If actual level of control still significantly influences the magnitude SSVEP after considering the influence of distance from the centre on SSVEP, this would support our conclusion that SSVEP is stronger for self-generated motion than non-self-generated motion, independent of retinal position factors. Modelling was conducted using IBM SPSS Amos 22. All trials from the ‘different control’ condition, from all participants, were pooled into the model. The SNRs and distance from the centre were normalized within each participant (by subtracting the mean and dividing the result by the standard deviation of each participant) to minimize the influence of individual difference. All the paths in the model were significant (see Fig. 5). These results confirm that control enhanced SSVEP directly, but also indirectly via the distance from the centre. However, the direct influence was much smaller than the indirect influence via the position of the object.

Fig. 5

The structure equation model (SEM) of the SSVEP and the coefficient of each path. All the paths shown in the model were significant, showing that actual level of control influenced SSVEP via both direct and indirect (via object position) paths.

Discussion

Experiment 1 was designed to measure the neural activity underlying the perceptual processing of an object when it was under a relatively high or low level of control. Participants moved their finger to trigger the movements of two visual objects simultaneously, which were tagged with differently flicking frequencies (7.5 vs 10 Hz). The participants were instructed to detect a brief colour change on one object's border, which did not require any attention on the motion or the level of control. The results showed stronger SSVEP for objects under a relatively high level of control, even after excluding the influence of object position. Further, the null finding of the ‘changing control’ condition suggested that such enhanced neural response was based on an online computation of the relation between one's movement and the visual input, rather than memory for whether the object had been under control in the past.

However, because the influence of object position was much stronger than the influence of level of control on SSVEP (Fig. 5), before further discussing our results, we decided to replicate our findings with a new paradigm that was able to exclude the influence of object position on SSVEPs in Experiment 2.

Experiment 2

Eighteen healthy right-handed volunteers were recruited from the same participant database as the previous experiment. All the participants reported normal or corrected-to-normal visual acuity, normal motor ability, and no neurological abnormalities. None had taken part in Experiment 1. Two participants were excluded from the analyses due to inadequate alertness during the task, resulting in a sample size of 16 (mean age = 24.3, ranged from 20 to 33, SD = 3.8, 9 females). The sample size of 16 was based on a power calculation with an effect size estimated from the results of Experiment 1 (α = .05, power (1 - β error) = .95) using G*Power 3 (Faul et al., 2007). The study was approved by the local ethics committee (University College London), and all participants provided informed consent prior to participant.

Task and procedure

Experiment 2 had the same conditions as Experiment 1, but used randomly distributed dots instead of distinct objects. The stimuli were 600 5.2-mm dots that were randomly distributed on the whole screen (width × height: 338 mm × 270 mm, all equipment was as in Experiment 1, Fig. 6). The minimum distance between dots was 10.4 mm (1.2°). The dots were either yellow (RGB 255 255 0) or blue (RGB 0 255 255). The yellow and blue dots were equal in number, and distributed asymmetrically around the centre of the screen. However, during the task only the dots that were positioned within 39 mm (4.5°) from the centre of the screen were presented to participants. Dots immediately appeared when they moved into this area, were immediately hidden when they moved out. Therefore, participants always saw yellow and blue dots as if through a central window. During each trial, when participants moved their finger on the touchpad, they had different levels of control (100%, 50%, or 0%, depending on the type of trial) over the motion of each group of dots. The motion of the dots with the same colour was the same, but differed from the dots of the other colour. When the dots were moved, the two groups of dots may have overlapped, but because they were both flickering, they were still visible even when partially obscured. The order of drawing the dots was randomized, to avoid the perception of depth. Participants were therefore able to perceive two distinct groups of moving dots using colour and coherent movements. Because the dots were randomly distributed and were always and only presented in the central area, the retina position of the two groups of dot did not differ.

Fig. 6

Timeline of the task in Experiment 2. The task was identical to Experiment 1, but used random dots in two colours instead of two distinct objects.

The task was similar to Experiment 1. Participants continuously moved their finger to trigger the movements of the dots, and detected a brief (400 ms) flash of one group of dots' border. All the dots in the same group flashed their borders together. In addition, because the two colours may have differed in contrast and luminance (we did not control these factors), we counter-balanced the combination of colour, along with flickering frequency and level of control. Each block in Experiment 2 contained 36 trials, including 12 trials for each condition. In addition, because the distributed motion was more difficult to perceive than the motion of a single object, we used 100% and 0% control as the high and low levels of control, respectively (cf. 85% and 15% in Experiment 1). The experimental conditions, procedure, EEG recording and data processing were all identical to Experiment 1.

In the ‘different control’ condition, when the flashing dots were under 0% and 100% control, the average accuracy of detection of the border colour change was 95.7% (SD = 7.1%) and 98.7% (SD = 2.9%) respectively. In the ‘changing control’ condition, when the flashing dots were under 0% and 100% initial control, accuracy was 96.1% (SD = 6.5%) and 98.5% (SD = 4.2%) respectively. In the equal control condition, accuracy was 96.4% (SD = 5.3%). The accuracy of visual detection did not differ among trial conditions (F (2, 28) = 0.98, p = .388, partial η2 = 0.061) or between the levels of control (or initial control) over the flashing dots (a Bonferroni-adjusted p of .025 was used, for the ‘different control’ condition, t (15) = 1.97, p = .067; for the ‘changing control’ condition, t (15) = 1.85, p = .084).

The SSVEP results of Experiment 2 replicated the findings of Experiment 1 (Fig. 7). However, the SSVEP amplitudes in Experiment 2 were smaller for both the fundamental frequencies and harmonics compared to Experiment 1. This was probably because of the weaker contrast and weaker intensity of the visual stimuli. Table 2 shows the results of the 2 × 2 × 2 (control (100% vs 0%) × flicker frequency (7.5 Hz vs 10 Hz) × “fundamental vs harmonic” repeated-measures ANOVA on the SNRs in the different control condition and the changing control condition. Regarding the ‘different control’ condition, we found significant main effects of control, flicker frequency, and “fundamental vs harmonic”. Dots under 100% control triggered stronger a neural response than when they were under 0% control. The SNR was stronger for 7.5 Hz than for 10 Hz flickers. The interaction between control and “fundamental vs harmonic” response was also significant. Post-hoc comparisons of the interaction between control and “fundamental vs harmonic” (a Bonferroni-adjusted p value of .025 was used) showed that the difference in SNRs between control conditions was significant at the fundamental frequencies (t (15) = 4.60, p < .001), but nonsignificant at the first harmonic frequencies (t (15) = 0.37, p = .714). Regarding the ‘changing control’ condition, the main effects of flicker frequency and “fundamental vs harmonic” were significant, but the main effect of control and all the interactions between factors were non = significant (Table 2).

Fig. 7

The SSVEPs of the ‘different control’ and ‘changing control’ conditions of Experiment 2. (A) and (B) show the amplitude of each frequency in each condition. (C) and (D) show the signal-to-noise ratios (SNRs) for the types of trials in which the two frequencies (7.5 Hz, 10 Hz) were differently combined with objects that were under different levels of control. (E) and (F) show the SNRs relative to the level of control. Error bars represent standard errors. The results replicated the finding of Experiment 1, showing that the neural response was stronger for the objects that were under relatively better control.

Table 2

Outcome of the ANOVAs conducted on the SNRs of the ‘different control’ condition and the ‘changing control’ condition in Experiment 2.

Factor	MSe	df	F	partial η2
Different control condition
Control	.043	1, 14	7.03*	.319
Flicker frequency	.382	1, 14	11.56*	.435
“Fundamental vs harmonic”	.556	1, 14	14.10*	.484
Control × flicker frequency	.059	1, 14	3.27	.179
Control × “fundamental vs harmonic”	.035	1, 14	5.22*	.258
Flicker frequency × “fundamental vs harmonic”	.193	1, 14	2.33	.135
Three-way interaction	.042	1, 14	1.23	.076
Changing control condition
Control	.037	1, 14	.225	.015
Flicker frequency	.357	1, 14	12.22*	.449
“Fundamental vs harmonic”	.394	1, 14	7.86*	.344
Control × flicker frequency	.075	1, 14	0.38	.024
Control × “fundamental vs harmonic”	.040	1, 14	0.64	.041
Flicker frequency × “fundamental vs harmonic”	.299	1, 14	2.25	.130
Three-way interaction	.073	1, 14	0.77	.049

*p < .05.

In summary, the results of Experiment 2 replicated the main findings of Experiment 1, showing that flickering visual stimuli triggered stronger neural responses when they were under a relatively high level of control than when they were under a relatively low level of control,. This effect seems to be the result of online computation, and receives minimum influence from memory for previous control. In addition, Fig. 8 shows the topography of SNR of the averaged response in the ‘different control’ condition, and the difference between 100% and 0% control. The topographies show that both the overall and control differential SSVEPs were localized to occipital electrodes (with the maximum value at Oz), indicating that the effect of control mainly occurred at a perceptual (visual) processing level.

Fig. 8

The topography of the SSVEP in the different control conditions of Experiment 2. (A) shows the average SNR of the fundamental frequency and the first harmonic of the dots that were under 100% control. (B) shows the average SNR of the fundamental frequency and the first harmonic of the dots that were under 0% control. The SSVEP was mainly localized to the occipital sites. (C) shows the difference in SSVEP between dots that were under 100% and 0% control. These results show that the difference in processing objects under higher and lower levels of control occurred mainly at visual sites.

General discussion

The aim of the present study was to examine neural responses underlying the perceptual processing of self-generated motion, compared with non-self-generated motion. We used SSVEPs to examine the extent of perceptual processing of the object whose motion was under a relatively high level of control, compared to when its motion was under a low level of control. Experiment 1 used two distinct objects, and Experiment 2 used two groups of randomly distributed dots. Results from both experiments showed stronger SSVEPs for stimuli whose movements were more correlated to participants' finger movements, than when their movements were less correlated to finger movements. Importantly, neither task (detection of border colour change in both experiments) required any top-down attention to the motion or fine control of the stimuli. Therefore, the advantage in perceptual processing for objects undergoing self-generated motion was probably based on an automatic computation of the relation between participants' finger movements and visual input, rather than strategic allocation of attention to specific objects, or explicit judgment about control.

Many recent studies of action control emphasise the sensory attenuation of action consequences, based on a comparator model (Blakemore et al., 1999, 1998; Frith et al., 2000; Wolpert and Flanagan, 2001). Somewhat in contrast, our results support a preactivation hypothesis of perceptual processing of action consequences (Roussel et al., 2013), by showing that self-generated motion enhanced perceptual processing, rather than inhibiting it. Indeed, work in both visual and auditory domains has revealed that predictability and regularity attract attention (Andreou et al., 2011; Bendixen, 2014; Bendixen et al., 2010; Chun and Jiang, 1999; Denham and Winkler, 2006) and increase brain activity (Barascud et al., 2016; Southwell et al., 2017). Neural activity of sensory inputs that are predictable from the efference copy of motor commands are usually attenuated (Baess et al., 2011; Bednark and Franz, 2014; Gentsch et al., 2012; Gentsch and Schütz-Bosbach, 2011; Hughes and Waszak, 2011; Kühn et al., 2011; Poonian et al., 2013; Timm et al., 2013). The preactivation hypothesis (Roussel et al., 2013) suggests that such attenuation is the result of increased neural activity in the baseline, due to prediction of a signal, rather than an inhibited response of the sensory input. When a task requires the detection of an action consequence, this requires discrimination between the baseline and the signal. Self-generated events probably show a disadvantage in perceptual processing compared to non-self-generated events, because the baseline is elevated by preactivation in anticipation of the signal. On the other hand, if a task involves the perceptual processing of an action consequence, without need for detection of a signal or discrimination from a baseline, it probably leads to an advantage for self-produced events. Most previous studies examining neural responses to self-produced events used brief stimuli and employed a detection or discrimination task. The present study is the first to shed light on the continuous perceptual processing of self-generated motion, without involving any related task, and demonstrated that the regularity between action and sensory input indeed triggers a stronger neural response.

The enhanced SSVEP for self-generated motion could be due to stronger attentional capture compared to non-self-generated motion. The regularity between self-generated movements and visual input may capture attention (Zhao et al., 2013). However, a previous study argued that the boost in brain activity of such regularities need not necessarily be attributed to the effect of attention (Southwell et al., 2017). In the present study, we also designed the ‘changing control’ condition to examine whether people intentionally monitored the object that they previously had control over. We did not find any effect of “previous status” of control on the SSVEPs. Therefore, the role of strategy in intentional monitor of an object under control was very small in our task, or the effect of it was minimal. However, attention could also be automatically and involuntarily captured, and enhance perceptual processing during an online computation of motor-visual regularity. The regular relation between one's action and events in the external world could be salient due to some bottom-up processes. Whether an event is salient or not depends on what it is compared with. Our task involved continuous motor action and visual feedback. In contrast, previous sensorimotor studies involved discrete action each triggering a single feedback event. A continuous steamed self-generated events may be more salient than comparable non-self-generated events. Conversely, a discrete self-generated event may be predicted by the initiation of a novel motor command, and may be less salient than a non-self-generated event.

Our definition of control is based on spatial regularity between movement and visual feedback. This may potentially produce perceptual grouping effects even though participants did not view the moving fingers directly. Previous studies using SSVEPs showed that the grouping of multiple frequency-tagged visual stimuli results in stronger neural responses (Aissani et al., 2011; Alp et al., 2016, 2017; Baker et al., 2011; Boremanse et al., 2013; Cunningham et al., 2017; Gundlach and Müller, 2013; Nozaradan et al., 2012). Moreover, in many previous studies, grouping had significant effects on intermodulation responses (i.e., the sum or the difference of two flicker frequencies, f1 ± f2) of visual stimuli, rather than the fundamental frequencies and harmonics (Aissani et al., 2011; Alp et al., 2016, 2017; Boremanse et al., 2013; Cunningham et al., 2017; Gundlach and Müller, 2013). However, no previous study examined the effect of inter-modal grouping between motor and visual modalities. We found that the sense of control, which could perhaps be considered a specific case of inter-modal grouping based on one's own actions, enhanced neural responses to the visual stimuli. We believe this represents an important new insight into the neural mechanism of control.

Additionally, we found that the effect of control (and central position) had significant effects at f2 and 2f1, but not at the other frequencies of interest (i.e., f1, 2f2). On the other hand, we found significant effects of control at the fundamental frequencies (f1, f2) rather than the first harmonics (2f1, 2f2). There were several differences in perceptual features between Experiment 1 and 2, including the stimulus position on the retina, “distributed vs centralised” motion, “superimposed vs distinct” motion, and size of flicker field. A previous study reported that feature-selective attention to one of two superimposed random dot kinematograms enhanced the SSVEP of the fundamental frequencies (Andersen et al., 2012). However, in the current literature it remains unclear how attention (e.g., top-down or bottom-up), perceptual features, or higher level cognitive processes affect SSVEPs of fundamental frequencies and harmonics (Kim et al., 2011; Langdon et al., 2011; Porcu et al., 2013; Saupe et al., 2009).

Lastly, in both experiments we observed the largest SSVEP amplitude at 10 Hz, which is in the alpha range (8–12 Hz). This might reflect the entrainment of the prominent oscillatory rhythm in alpha band in perceptual tasks (Norcia et al., 2015; Spaak et al., 2014; Srinivasan et al., 2006). Previous research reported that the human visual cortex shows the strongest response and resonance to 10 Hz visual flicker, indicating a selective preference of the neural oscillation frequency (Herrmann, 2001; D. Regan, 1966). However, because of prominent activity in the alpha band, the SNR of 10Hz was actually smaller than for the other frequencies. Because the overall activity in the alpha band is usually suppressed by attention (Klimesch, 2012) and steady-state evoked response is usually promoted by visual attention (Norcia et al., 2015), we suggest that the SNR of 10 Hz may be quite sensitive to perceptual processing linked to attention. Although it remains unclear how alpha entrainment and perceptual processing interact on SSVEPs, it is known that alpha entrainment does not just “hijack” perceptual processes, and SSVEPs are reliably increased by spatial attention, regardless of whether or not the flicker frequency is in the alpha band (Keitel et al., 2013; Keitel et al., 2014). Indeed, we found stronger effects of control at 10 Hz rather than 7.5 Hz in Experiment 1, and similar effects of control at both frequencies in Experiment 2, showing that the flicker frequency in the alpha band (10 Hz) was not essentially different from stimulation at other frequencies.

To summarize, we found that SSVEPs are stronger for visual stimuli whose spatial motion was more likely controlled by one's voluntary finger movements, compared to stimuli under less control. This occurred even when these features were task irrelevant. Further, the SSVEP in our task reflects the instantaneous degree of spatial coupling between movement and visual input, rather than any memory for past history of control. However, we cannot exclude the possibility of memory effect on SSVEPs in tasks requiring explicit semantic labelling and object representation. Our findings are consistent with a preactivation hypothesis (Roussel et al., 2013), which suggests that the neural activities of self-generated events are enhanced rather than inhibited. Our findings provide a basic understanding of how the brain processes action consequences, and inspires reconsiderations of the classical view on sensory attenuation. Our results also suggest that control has intrinsic cognitive value and automatically benefits from increased cognitive resources, compared with other events occurring in the environment that are not self-generated.