Brief period of monocular deprivation drives changes in audiovisual temporal perception.

The human brain retains a striking degree of plasticity into adulthood. Recent studies have demonstrated that a short period of altered visual experience (via monocular deprivation) can change the dynamics of binocular rivalry in favor of the deprived eye, a compensatory action thought to be mediated by an upregulation of cortical gain control mechanisms. Here, we sought to better understand the impact of monocular deprivation on multisensory abilities, specifically examining audiovisual temporal perception. Using an audiovisual simultaneity judgment task, we discovered that 90 minutes of monocular deprivation produced opposing effects on the temporal binding window depending on the eye used in the task. Thus, in those who performed the task with their deprived eye there was a narrowing of the temporal binding window, whereas in those performing the task with their nondeprived eye there was a widening of the temporal binding window. The effect was short lived, being observed only in the first 10 minutes of postdeprivation testing. These findings indicate that changes in visual experience in the adult can rapidly impact multisensory perceptual processes, a finding that has important clinical implications for those patients with adult-onset visual deprivation and for therapies founded on monocular deprivation.

Introduction

The brain possesses a remarkable degree of plasticity that enables it to reorganize to meet the demands of the changing environment (Pascual-Leone, Amedi, Fregni, & Merabet, 2005). This intrinsic property of the nervous system peaks during a defined period in development, the so-called critical or sensitive period (Berardi, Pizzorusso, & Maffei, 2000). During this period of plasticity, changes in sensory experience can engender morphological and functional changes that have enduring effects on sensory function.

Seminal work by Hubel and Wiesel discovered that early visual deprivation produced several changes in the structural and neurophysiological properties of neurons and circuits in the lateral geniculate nucleus and the visual cortex (Hubel & Wiesel, 1970; Wiesel & Hubel, 1963, 1965a, 1965b). They described that, after a few weeks to months of visual deprivation, there was a decrease in the number and overall activity of neurons in these brain regions that responded to the deprived eye, as well as changes in receptive field properties of these cells and a reorganization of ocular dominance columns in favor of the open eye (Hubel & Wiesel, 1970; Wiesel & Hubel, 1963). Importantly, these deprivation-induced changes endured after years of restoring normal sensory experience to the deprived eye (Wiesel & Hubel, 1965b). Furthermore, these changes were found to be more profound in response to monocular versus binocular deprivation, suggesting that an imbalance of input to the two eyes is a critical factor (Wiesel & Hubel, 1965a). In humans, early visual deprivation can occur through conditions such as untreated congenital cataracts and uncorrected refractive errors and is associated with permanent defects in performance of visual perceptual tasks, such as visual acuity, contrast detection, and shape and depth perception (Fine, Smallman, Doyle, & MacLeod, 2002; Fine et al., 2003; Levi, McKee, & Movshon, 2011; Ostrovsky, Andalman, & Sinha, 2006).

In recent years, the study of multisensory processing has received a great deal of attention. A growing body of compelling evidence supports the interaction and/or integration of information from different sensory modalities at various brain levels, especially in areas which were once considered to be classically “unisensory” (Foxe & Schroeder, 2005; Ghazanfar & Schroeder, 2006; Stein & Stanford, 2008). Integrating information from multiple senses has several benefits, including striking response gains at the neural level (Stein & Meredith, 1993; Stein & Stanford, 2008), as well as enhanced detection (Frassinetti, Bolognini, & Làdavas, 2002; Lovelace, Stein, & Wallace, 2003), discrimination (Ernst & Banks, 2002), localization (Zou, Müller, & Shi, 2012), and speeding response times (Diederich & Colonius, 2004) at the perceptual level. These neural and perceptual benefits depend on the physical characteristics of the combined stimuli, with the temporal proximity between the different cues that make up a multisensory stimulus being one of the important factors. Intuitively, natural events that originate from a common source are likely to arrive at the brain centers in close temporal proximity and, thus, integrating them may enhance perceptual saliency and performance. Indeed, the largest neural and perceptual gains are achieved when the two inputs making up a multisensory stimulus are close in time (Meredith, Nemitz, & Stein, 1987).

Extending this work into the psychophysical arena, a number of studies have focused on characterizing the window of stimulus onset asynchronies (SOAs) within which two stimuli are likely to be integrated or perceptually “bound” (Wallace & Stevenson, 2014). This window, termed the temporal binding window (TBW), is essentially a measure of multisensory temporal acuity with a larger window indicating poorer acuity and an increased likelihood that the brain will bind events that do not originate from a common source. Over the years, several studies have revealed some notable characteristics of the TBW. For example, the size of the TBW varies significantly across participants (Stevenson, Zemtsov, & Wallace, 2012). This individual variability has been linked with multisensory integrative abilities, such as individual susceptibility to the McGurk effect, scored as an illusory perception of /da/ or /ta/ when an observer fuses an auditory syllable /ba/ and a visual syllable /ga/ (Stevenson et al., 2012). In addition, the TBW size has been shown to be larger in clinical populations such as those with autism and schizophrenia (Foss-Feig et al., 2010; Stevenson et al., 2014; Stevenson et al., 2017; Noel, Stevenson, & Wallace, 2018). As with the adult plasticity seen in the visual system, multisensory temporal function has been shown to be highly malleable. For example, a short interval of perceptual training can significantly narrow the TBW (Powers, Hevey, & Wallace, 2012; Powers, Hillock, & Wallace, 2009).

Interestingly, altered visual experience has been shown to influence the maturation of multisensory integration during development. For example, work in animal models has shown that early visual deprivation impairs the ability of neurons to synthesize multisensory information (Carriere et al., 2007; Wallace, Perrault, Hairston, & Stein, 2004). In humans, imaging studies have revealed large-scale cross-modal reorganization of the visual cortex in patients who were visually deprived briefly during their early life (Collignon et al., 2015; Guerreiro, Putzar, & Röder, 2015). Perceptually, the effects of early visual deprivation in humans can also impair audiovisual interactions, most notably in the temporal domain (Chen, Lewis, Shore, & Maurer, 2017; Putzar, Goerendt, Lange, Rösler, & Röder, 2007; Putzar, Hötting, & Röder, 2010). For instance, patients who experienced visual deprivation owing to congenital cataracts showed wider TBWs when compared with age-matched typically developed controls during the performance of an audiovisual simultaneity judgment task (Chen et al., 2017). In addition, although monocularly deprived patients exhibited widened TBWs characteristic of an immature audiovisual integration system, binocularly deprived patients showed widened TBW that did not match any stage in typical development (Chen et al., 2017). Together, these findings indicate that normal visual experience is crucial for the development of both visual and multisensory (i.e., visual–nonvisual) functions.

Although the impact of atypical early visual experience on visual and multisensory functions have been well-established, the question of whether a short-term change in visual experience in adults can drive plastic changes in multisensory temporal function remains unanswered. Recent studies have shown that, despite the decreased plasticity in the brain after the closure of the critical period, the adult brain still retains marked plasticity that can impact perception and behavior (Berardi et al., 2000; Sabel, 2008). One example of such adult plasticity is the perceptual effect of short-term monocular deprivation. Occluding one eye with a translucent patch for 150 minutes boosts the perceptual dominance of the deprived eye during subsequent binocular rivalry and increases the apparent contrast of viewed gratings (Lunghi, Burr, & Morrone, 2011). According to Lunghi and her colleagues (Lunghi et al., 2011), this finding reflects a form of rapid homeostatic plasticity to visual deprivation and is mediated by an upregulation of contrast gain control mechanisms of the deprived eye in an attempt to optimize the attenuated or absent visual information. This enhanced performance in the deprived eye has been demonstrated across different visual tasks, such as phase combination, global motion coherence, and contrast matching and detection (Zhou, Clavagnier, & Hess, 2013), and is independent of the nature of deprivation (Chadnova, Reynaud, Clavagnier, & Hess, 2017; Kim, Kim, & Blake, 2017; Zhou et al., 2013). Given the evidence of the malleability of the TBW to effects of visual deprivation-induced plasticity during development, we hypothesized that a brief period of monocular deprivation will impact the TBW in adult humans through homeostatic plasticity. Such a result would reinforce the importance of adult visual experience in maintaining not only visual representations, but also in maintaining multisensory (e.g., audiovisual) representations.

Methods

Participants

Twenty undergraduates at Vanderbilt University were recruited to perform an audiovisual simultaneity judgment task using either their deprived or nondeprived eye before and after 90 minutes of monocular deprivation. Each participant was randomly assigned to either the deprived eye group, n = 10, 3 males, mean age 19.1 ± 3.0 years, or the nondeprived eye group, n = 10, 4 males, mean age 18.9 ± 1.2 years. All students had normal or corrected-to-normal vision and normal hearing. Normal vision was confirmed using a visual acuity task. Participants gave informed consent before being allowed to participate and received course credits for their participation. All recruitment and experimental procedures were approved by the Vanderbilt University Institutional Review Board. This work was carried out in accordance with the Declaration of Helsinki.

Stimulus and apparatus

The experimental stimuli were generated and presented using MATLAB (Math Works Inc., Natick, MA) software with the Psychophysics Toolbox Version 3 (Brainard & Vision, 1997; Pelli, 1997). The visual stimulus was displayed on a gamma-corrected monitor (21-inch Asus LCD) with a 120-Hz refresh rate and the auditory stimulus was presented binaurally through headphones (Sennheiser HD559). The visual stimulus was a gabor patch, sigma = 0.5°, which had a spatial frequency of 4 cycles per degree, subtended an angle of 6° and was presented at the center of fixation. The gabor patch was displayed at 20% contrast on a uniform gray background with a luminance of 10 cd/m2. The auditory stimulus comprised a white noise burst at approximately 75 dB and with a sampling frequency of 41 kHz. Both stimuli were presented for four frame periods equaling 33 ms. Although the intensity of the auditory stimulus was linearly ramped up and down, each for 25% of the stimulus duration, a similar effect was achieved with the visual stimulus by presenting the first and last frames at one-half of the actual contrast of the stimulus. A Minolta Chroma Meter CS-100 and a sound level meter were used to verify the luminance and sound intensity levels, respectively. The durations of all visual and auditory stimuli, as well as the SOAs, were confirmed using a Hameg 507 oscilloscope with a photovoltaic cell and microphone.

Procedure

Participants performed the experiment inside a WhisperRoom (SE 2000 Series) with their forehead and chin placed comfortably against a HeadSpot (University of Houston Optometry). The whisper room was only lit by the background luminance of the screen. Before the experiment began, each participant was randomly assigned to either the deprived eye group or the nondeprived eye group. The eye to be deprived was randomly chosen for each participant and counterbalanced across all participants. The deprived eye group performed the task with their deprived eye and the nondeprived eye group performed the task with their nondeprived eye before and after 90 minutes of monocular deprivation with an opaque eye patch (Figure 1A). Both the predeprivation and the postdeprivation sessions comprised two blocks of an unspeeded audiovisual simultaneity judgment task. A block took approximately 10 minutes to complete. During each block, the participant viewed a gabor patch and heard an auditory white noise burst after a brief fixation period lasting between 600 and 1000 ms (Figure 1B). The white circular dot that marked the fixation period was always present and participants were instructed to always fixate at it. The onsets of the two stimuli were separated by 13 parametrically varied SOAs (in ms): −500, −350, −200, −150, −100, −50, 0, 50, 100, 150, 200, 350, and 500. Negative and positive SOA values corresponded with auditory preceding vision and vision preceding auditory SOAs, respectively. Each SOA was presented 20 times (totaling 260 trials) in randomized fashion. After each presentation, participants accurately delivered their response by pressing “1” on the keyboard if the pair of audiovisual stimuli was synchronous or by pressing “2”, if the pair was asynchronous. Each participant was given two brief initial practice sessions, each consisting of nine SOAs and 10 trials per SOA. The rationale for this initial practice was to, first, exclude participants who could not perform adequately well on the task and, second, decrease familiarization effects during the main experiment. No feedback was provided on the correctness of their responses during the main experiment.

Figure 1.

Experimental procedure. (A) Participants were randomly assigned to either the deprived eye or nondeprived eye group. Participants in the deprived eye group performed the audiovisual temporal simultaneity judgment task with their deprived eye before and after 90 minutes of deprivation. For the nondeprived eye group, the deprivation phase started with the first predeprivation block of the nondeprived eye. The postdeprivation blocks were then performed using the nondeprived eye after 90 minutes from the start of the first predeprivation block. (B) On each block, participants judged the simultaneity of a gabor patch (33 ms) and an auditory white noise burst (33 ms). The onsets of the two stimuli were separated by a range of preselected SOAs between –500 ms and 500 ms. Negative and positive SOAs signified auditory leading and visual leading respectively. On each trial, there was a brief fixation period (600–1000 ms), followed by the stimulus presentation. Participants were then asked to respond by pressing the keyboard after which the next trial began automatically.

Analysis

Each participant completed two predeprivation and two postdeprivation blocks of the audiovisual simultaneity judgment task. Proportions of synchrony reports as a function of SOA were computed for each block and each participant. We then pooled (i.e., averaged) the data from the two predeprivation blocks into one combined predeprivation block and likewise, the two postdeprivation blocks into one postdeprivation block. These data were then fitted separately with a single-term Gaussian distribution model (through MATLAB fit.m) whose amplitude, mean and standard deviation were free to vary (Noel et al., 2018; Simon, Noel, & Wallace, 2017). Unlike the mean and the standard deviation parameters, the range of possible amplitude values was bound between 0 and 1. The mean and the standard deviation of the best fitting model were then taken as the point of subjective simultaneity (PSS) and the temporal window of simultaneity (TBW), respectively. The averaged r2 values across participants in both groups showed reasonable fits to the data for the combined predeprivation block (0.92 ± 0.01) and the combined postdeprivation block (0.91 ± 0.02). To determine whether the deprivation effect differed between the first and second postdeprivation blocks, the Gaussian distribution described elsewhere in this article was fitted to the data for the two postdeprivation blocks separately and their PSS and TBW measures were derived. Again, the averaged r2 values across participants for the two postdeprivation blocks were satisfactory: postblock 1 (0.89 ± 0.03) and postblock 2 (0.87 ± 0.04). Next, we computed the effect of deprivation on the performance measures for each participant by normalizing the postdeprivation measure (either combined or divided) to the baseline predeprivation measure. For the TBW, the relative change calculation ((post–pre)/pre × 100%) was used, whereas for the PSS an absolute change (post–pre) computation was used because of the inclusion of negative numbers.

The following statistical analyses were then carried out for each performance measure using the JASP software version 0.11 (JASP Team, 2018). First, an independent samples t-test was done to determine whether participants in both groups were comparable in their predeprivation performance. Second, a one-sample t-test against zero (baseline) was used to determine whether the computed deprivation effect (i.e., absolute or relative change) was statistically significant for each eye group. Third, we conducted an independent samples t-test to determine whether the absolute change in PSS differed between the deprived and nondeprived eye groups. For the TBW, a one-way analysis of covariance test was done to determine whether the relative change in TBW differed between the deprived and nondeprived eye groups while adjusting for the individual variability in baseline TBW. For the divided blocks, we conducted multiple t-tests (for four comparisons in total) to determine 1) whether the deprivation effect differed between the first and second postdeprivation blocks for each group and 2) whether the deprivation effect differed between the two groups for each postdeprivation block. Note that, in the latter scenario, a one-way analysis of covariance test was conducted to adjust for the between-participant differences in the predeprivation TBW. Correlation analyses were done using Pearson's correlation. All statistical analyses were two-tailed and significance level, alpha, was set to 0.05. In the case of multiple tests, alpha was adjusted with a Bonferroni-Holm correction. Furthermore, the statistical results were reported in both frequentist (i.e., p values) and Bayesian (i.e. Bayes factors) frameworks of hypothesis testing.

Results

The proportions of synchrony reports averaged across participants are plotted as a function of SOA for the deprived eye group (red) and the nondeprived eye group (blue) in Figure 2. Whereas Figure 2A displays the results between the averaged predeprivation blocks (dashed line) and the first postdeprivation block (solid line), Figure 2B shows the results for the averaged predeprivation blocks (dashed line) and the second postdeprivation block (solid line). For each performance measure (i.e., PSS and TBW), an independent samples t-test on the combined predeprivation blocks showed no statistically significant difference between the two eye groups, indicating that both groups were comparable at baseline, PSS: t(18) = 0.52, p = 0.611, BF10 = 0.437; TBW: t(18) = −1.55, p = 0.138, BF10 = 0.904.

Figure 2.

Mean proportion of synchrony reports. Proportion of synchrony reports averaged across participants in the deprived eye group (red) and the nondeprived eye group (blue) is plotted as a function of SOA for (A) the combined average of the predeprivation blocks (dashed line) and the first postdeprivation block (solid line) and for (B) the combined average of the predeprivation blocks (dashed line) and the first postdeprivation block (solid line).

No effect of deprivation on PSS

The PSS is the mean of the TBW and is defined as the SOA at which maximum perceived simultaneity is perceived. Essentially, the PSS measures which modality is given more weight in determining audiovisual temporal simultaneity. Interestingly, the PSS is not usually zero and may be shifted toward the auditory-leading (i.e., more visual weight or bias) or visual-leading (i.e., more auditory weight or bias) side, depending on stimulus-related factors such as stimulus duration and intensity (Boenke, Deliano, & Ohl, 2009; Jaśkowski, 1999; Sanford, 1971), task-related factors such as judging the onset versus the offset in an simultaneity–judgment task (Wen, Opoku-Baah, Park, & Blake, 2020), and attention-related factors such as being asked to attend to one modality (Schneider & Bavelier, 2003; Stelmach & Herdman, 1991; Zampini, Shore, & Spence, 2005). To determine the effect of monocular deprivation on the PSS, we computed the absolute change in the PSS before and after deprivation. Based on this computation, a negative change signified an increase in visual bias or weighting while a positive change signified a decrease in visual bias.

The effect of 90 minutes of monocular deprivation on the PSS is displayed in Figure 3. Our results showed that the absolute change in PSS was not statistically significant for either the deprived eye, DG: t(9) = −0.45, Bonferroni-Holm adjusted alpha = 0.05, p = 0.663, Cohen d = −0.142, BF10 = 0.337, or nondeprived eye, NG: t(9) = 1.68, Bonferroni-Holm adjusted alpha = 0.025, p = 0.128, Cohen d = −0.53, BF10 = 0.884, groups when data were pooled across the entire postdeprivation phase. Comparing the mean absolute change in PSS between the deprived and nondeprived eye groups showed no significant difference, t(18) = −1.59, p = 0.128, Cohen d = −0.713, BF10 = 0.943. Next, we determined whether the absolute change in PSS differed between the groups for each postdeprivation phase as well as between the postdeprivation phases for each group. Our results revealed that for both the first and second postdeprivation phases, there was no statistically significant difference between the mean absolute change in PSS for the deprived group and that of the nondeprived group, first: t(18) = –1.28, Bonferroni-Holm adjusted alpha = 0.0167, p = 0.215, Cohen d = −0.574, BF10 = 0.702; second: t(18) = −1.64, Bonferroni-Holm adjusted alpha = 0.0125, p = 0.118, Cohen d = −0.734, BF10 = 0.992. Also, the mean absolute change in PSS did not differ significantly between the first and second postdeprivation phases for the deprived group, t(9) = −0.71, Bonferroni-Holm adjusted alpha = 0.05, p = 0.499, Cohen d = −0.223, BF10 = 0.381, and for the nondeprived group, t(9) = −0.98, Bonferroni-Holm adjusted alpha = 0.025, p = 0.353, Cohen d = −0.309, BF10 = 0.457.

Figure 3.

Effects of deprivation on the PSS. (A) The mean absolute change in PSS after deprivation is plotted for the deprived (red) and nondeprived (blue) eye groups for data pooled across the entire post deprivation phase. Absolute change was computed as postdeprivation – predeprivation. (B) The mean absolute change in PSS after deprivation is plotted for the deprived (red) and nondeprived eye (blue) groups for the first and the second postdeprivation blocks. Circular dots represent individual participant data point within each eye group. The error bars represent ± standard error of the mean. ns p > 0.05, * p < 0.05, ** p < 0.01, *** p < 0.001.

Deprivation has contrasting effects on the TBW when indexed via the deprived and nondeprived eyes

As mentioned elsewhere in this article, the TBW is an index for multisensory temporal acuity with a narrow window signifying greater acuity and vice versa. To assess the impact of monocular deprivation on the size of the TBW, we computed the relative change in TBW between the postdeprivation and predeprivation phases when the data was pooled across the entire postdeprivation phase. In contrast with the lack of changes in PSS, deprivation resulted in a number of significant changes in the TBW (Figure 4). First, results from the combined postdeprivation block showed a significant decrease (i.e., negative relative change) in the TBW for the deprived eye group, DG: t(9) = −4.99, Bonferroni-Holm adjusted alpha = 0.025, p < 0.001, Cohen d = −1.578, BF10 = 52.012, and a significant increase (i.e., positive relative change) in the TBW for the nondeprived eye group, NG: t(9) = 2.53, Bonferroni-Holm adjusted alpha = 0.05, p = 0.032, Cohen d = 0.801, BF10 = 2.477, after deprivation (Figure 4A). To determine whether the two groups differed in their mean relative change in TBW, we conducted a one-way analysis of covariance test controlling for the individual differences in predeprivation TBW. Our analysis revealed a significant difference between the mean relative change in TBW for the deprived eye group versus that for the nondeprived eye, F(1, 17) = 16.47, p < 0.001, η2p = 0.492, BF10 = 107.06, while adjusting for predeprivation TBW.

Figure 4.

Effects of deprivation on the TBW. (A) The mean relative change in TBW after deprivation is plotted for the deprived (red) and nondeprived (blue) eye groups for data pooled across the entire post deprivation phase. Relative change was computed as (postdeprivation – predeprivation)/predeprivation × 100%. (B) The mean relative change in TBW after deprivation is plotted for the deprived (red) and nondeprived eye (blue) groups for the first and the second postdeprivation blocks. Circular dots represent individual participant data point within each eye group. The error bars represent ± standard error of the mean. ns p > 0.05, * p < 0.05, ** p < 0.01, *** p < 0.001.

Comparing the mean relative change in TBW between the deprived eye and nondeprived eye groups while adjusting for predeprivation TBW showed that the mean difference between the groups was significant during the first postdeprivation phase, F(1, 17) = 24.31, Bonferroni-Holm adjusted alpha = 0.0125, p < 0.001, η2p = 0.588, BF10 = 619.3, and nonsignificant during the second postdeprivation phase, F(1, 17) = 2.42, Bonferroni-Holm adjusted alpha = 0.025, p = 0.138, η2p = 0.125, BF10 = 2.167 (Figure 4B). In contrast, the mean relative change in TBW differed significantly between the first postdeprivation phase and second postdeprivation phase for the deprived eye group, DG: t(9) = –3.38, Bonferroni-Holm adjusted alpha = 0.0167, p = 0.008, Cohen d = −1.07, BF10 = 7.338. However, for the nondeprived eye group, there was no significant difference between the mean relative change in TBW for the first postdeprivation and second postdeprivation phases, NG: t(9) = −0.71, Bonferroni-Holm adjusted alpha = 0.05, p = 0.499, Cohen d = −0.223, BF10 = 0.381 (Figure 4B). These results emphasize two points: first, the differential impact of deprivation on the TBW of the deprived eye versus the nondeprived eye occurred only in the first deprivation phase and, second, the lack of significant difference observed during the second deprivation phase is attributable to the waning of the deprivation-induced effect observed in the deprived eye.

Relative change in TBW strongly correlates with the absolute change in PSS

Finally, we conducted a Pearson's correlation between the absolute change in PSS values pooled across all participants and postdeprivation blocks and the corresponding relative change in TBW values (Figure 5). Our analysis revealed a strong positive correlation between the absolute change in PSS values and the relative change in TBW values, r = 0.7, p < 0.001, log(BF10) = 10.607. This finding indicates that an enhancement in sensitivity to audiovisual temporal asynchrony (i.e., narrowing of the TBW), which was observed mostly for the deprived eye group, is associated with PSS shifts toward the auditory leading side (i.e., an increase in visual bias or weighting). In contrast, a decrease in temporal acuity or a widening of the TBW, which was observed mostly in the nondeprived eye group, is associated with PSS shifts toward the visual leading side (i.e., a decrease in visual bias or weighting).

Figure 5.

Correlation between the absolute change in PSS and the relative change in temporal window of simultaneity (TWS). Plotted on the x-axis the is relative change in TWS data pooled from participants across both eye groups (deprived eye: red; nondeprived eye group: blue) and the two postdeprivation blocks (first: circle; second: diamond) and plotted on the y-axis is the corresponding absolute change in PSS data. Positive and negative data points on the x-axis signify enhancement (narrowing of TWS) and depreciation (widening of TWS) of sensitivity to audiovisual temporal asynchrony, whereas positive and negative values on the y-axis signify more (negative shift in PSS) and less (positive shift in PSS) dominance (dom.) of the visual modality respectively. Our results revealed a strong positive correlation between relative change in TWS and the absolute change in PSS, r = 0.7, p < 0.001.

Discussion

Even after the closure of the critical period, the adult brain retains a marked degree of plasticity that can impact perception and behavior, and moreover, support recovery and rehabilitation after injury (Berardi et al., 2000; Sabel, 2008). Much of the evidence supporting this claim for the visual system has been provided through studies of perceptual learning (Goldstone, 1998; Huxlin, 2008) and, recently, through brief periods of monocular deprivation (Lunghi et al., 2011; Zhou et al., 2013). Adding to this body of knowledge, our study provides the first clear evidence that short-term monocular deprivation can induce changes in multisensory (i.e., audiovisual) temporal perception. Using an audiovisual simultaneity judgment task, we discovered that 90 minutes of monocular deprivation produced contrasting effects on the TBW (a measure of audiovisual temporal acuity), narrowing the TBW of participants who performed the task with their deprived eye and widening the TBW of those who used their nondeprived eye. Moreover, we observed that this effect was short lived, occurring only in the first phase of postdeprivation testing and took place within 10 minutes of the removal of deprivation.

During visual deprivation, the loss of visual information also impacts any visual-based multisensory processing (Carriere et al., 2007). In fact, the absence (often through the removal of visual information) or alteration (by manipulating the spatial and temporal properties of the cross-modal cues) of multisensory experience can influence normal multisensory function both before and after the critical period (Carriere et al., 2007; Odegaard, Wozny, & Shams, 2017; Stein, Stanford, & Rowland, 2014; Wallace et al., 2004). Although the deprivation effect observed in our study could be a result of compensation to the decrease in visually based multisensory information, we believe this is highly unlikely for two reasons. First, the monocular deprivation paradigm likely has very little impact on visually based multisensory experience, given the remaining information coming from the nondeprived eye. Second, the contrasting nature of the effects observed in the deprived and nondeprived eyes in our study are consistent with results that have been observed in purely visual studies of monocular deprivation (Lunghi et al., 2011; Lunghi et al., 2013; Zhou et al., 2013) and hence, points to the effects being driven in large measure by visual, as opposed to multisensory based compensatory mechanisms.

Monocular deprivation represents a powerful tool to study plasticity in the adult visual system. Indeed, evidence from past studies points to more detrimental effects of monocular as opposed to binocular deprivation on visual cortical organization and behavioral performance (Lewis, Maurer, & Brent, 1995; Wiesel & Hubel, 1965a). Mechanistically, monocular deprivation triggers various forms of homeostatic plasticity in an effort to restore the loss of visual drive and maintain the average levels of neural activity (Maffei, Nelson, & Turrigiano, 2004; Maffei & Turrigiano, 2008; Wang, Fontanini, & Maffei, 2012). These forms of homeostatic plasticity induced by monocular deprivation involve alterations in the balance of cortical excitation and inhibition via a complex interplay of excitation, inhibition, and intrinsic excitability (Maffei et al., 2004; Maffei & Turrigiano, 2008; Wang et al., 2012). In adult humans, short-term monocular deprivation has been associated with the downregulation of gamma aminobutyric acid (GABA) concentration in the primary visual cortex (Lunghi, Emir, Morrone, & Bridge, 2015). Using electroencephalographic recordings, other studies have also demonstrated that short-term monocular deprivation increases the amplitude of visual evoked potentials and neural oscillations believed to originate from the primary visual cortex when the deprived eye is stimulated and decreases these measures when the nondeprived eye is stimulated (Lunghi et al., 2015; Zhou, Baker, Simard, Saint-Amour, & Hess, 2015). Although these results emphasize an increase in overall neural activity after monocular deprivation, a recent 7T functional magnetic resonance imaging study has shown that besides an increase in the blood oxygen level-dependent signal in V1 for the deprived eye, monocular deprivation also shifts ocular dominance distributions in favor of the deprived eye (Binda et al., 2018). Collectively, these findings indicate that monocular deprivation induces plastic mechanisms geared at enhancing the visual information coming from the deprived eye.

Extending these findings to our current results, we believe that the changes observed in the TBW could arise from two main causes, which can occur separately or in tandem. Using the causal inference model, these two factors can be expressed in terms of changes in visual unisensory encoding and changes in the tendency to bind the audiovisual information (Beierholm, Quartz, & Shams, 2009; Körding et al., 2007; Magnotti, Ma, & Beauchamp, 2013). First, changes in the TBW as a result of changes in visual encoding after monocular deprivation is consistent with previous visual studies showing that monocular deprivation enhances the perception of the deprived eye while depressing that of the nondeprived eye (Lunghi et al., 2011; Zhou et al., 2013). In the multisensory domain, several studies have provided evidence that support the hypothesis that manipulating stimulus reliability, which occurs in the context of monocular deprivation, can influence multisensory interactions. For instance, Fister, Stevenson, Nidiffer, Barnett, and Wallace (2016) demonstrated that the proportion of perceived simultaneity of the paired audiovisual stimuli across SOA decreased faster for high than low saliency conditions. Although this study did not compute TBWs, the finding indicates that increasing stimulus effectiveness decrease the tolerance to audiovisual temporal asynchrony, which is comparable to narrowing of the TBW. Furthermore, Magnotti et al. (2013) demonstrated that decreasing the reliability (via blurring) of the visual stimulus during the performance of a simultaneity–judgment task widened the TBW. Studies using causal inference models have demonstrated that such effects emerge from changes in sensory noise, influencing the process of visual encoding (Beierholm et al., 2009; Magnotti et al., 2013).

Second, it is plausible that the changes observed in the TBW after monocular deprivation could arise from genuine cross-modal plasticity, reflecting changes in the tendency to bind the audiovisual stimulus pair. In a very interesting study, Lo Verde, Morrone, and Lunghi (2017) discovered using a binocular rivalry paradigm that, after monocular deprivation, the effect of visuo-haptic interaction on perceptual dominance disappeared for the deprived eye, which was potentiated in the visual domain, but was not affected for the nondeprived eye, which was weakened. In a separate control experiment where a postdeprivation effect was simulated by increasing the intensity of the visual stimulus for one eye and decreasing the intensity for the other eye, this unequal effect of monocular deprivation on cross-modal interaction for the deprived and nondeprived eyes disappeared, indicating that the effects on monocular deprivation were due to mechanisms that exceeded changes in contrast gain of the deprived eye (Lo Verde et al., 2017).

The idea that changes in the TBW can result from changes in binding tendency is consistent with studies that have demonstrated a relationship between TBW and intracortical excitation and inhibition balance—the underlying mechanism for homeostatic plasticity induced by monocular deprivation. For example, a recent study demonstrated that the size of the audio-tactile TBW measured across participants was associated with cortical excitation and inhibition balance indexed as glutamate/GABA concentrations in the left primary auditory cortex (Ferri et al., 2017), a region noted for integration of auditory and tactile information (Kayser, Petkov, Augath, & Logothetis, 2005; Schürmann, Caetano, Hlushchuk, Jousmäki, & Hari, 2006). Specifically, they showed that moderate excitation and inhibition balance was associated with the narrowest TBW sizes, whereas extreme (i.e., relatively high or low) excitation and inhibition balance was associated with wider TBW sizes (Ferri et al., 2017). Additionally, the link between changes in cortical excitation and inhibition balance mediating deprivation-induced homeostatic plasticity and the TBW can also be observed in clinical conditions or in perceptual mechanisms where excitation and inhibition balance is affected. For example, individuals with obesity, known to exhibit impaired homeostatic plasticity (suggestive of the presence of an altered excitation and inhibition balance) owing to a stronger GABAergic inhibition (Lunghi et al., 2019) also show widened TBW compared with healthy-weight controls (Scarpina et al., 2016). In addition, the TBW has been shown to be highly malleable to perceptual learning (Powers et al., 2009; Powers et al., 2012), which is known mechanistically to involve Hebbian plasticity, a form of experience-dependent plasticity that depends on the net intracortical excitation and inhibition balance like the homeostatic plasticity induced by monocular deprivation (see Binda et al., 2018, for a discussion). Together, these studies provide evidence that suggests that the TBW can be modulated to some extent by a direct influence of changes in cortical excitation and inhibition levels and or through changes in visual representations occurring as a result of changes in excitation and inhibition balance after monocular deprivation. To determine the relative contributions of these two mechanisms to the changes in TBW, further studies using neuroimaging techniques will be necessary to unravel the brain areas involved, the time course of these effects, and the changes in connectivity patterns induced by these effects.

Apart from changes in the TBW, we found that depriving participants monocularly for 90 minutes did not reveal any significant changes in the PSS within and between the eye conditions. The PSS is a central tendency of the TBW and represents a measure of perceptual weighting between the two modalities whose simultaneity is being judged. Mechanistically, it is known to stem from a combination of 1) the difference between the arrival time latencies and 2) the difference between the processing time of the two modalities (García-Pérez & Alcalá-Quintana, 2012). The lack of deprivation effect on the PSS seen in our study can be explained by prior electroencephalographic work that showed that monocular deprivation did not affect the latencies of C1, P1, and P2 components of the visual evoked potentials (Lunghi, Berchicci, Morrone, & Di Russo, 2015). Although the change in PSS was nonsignificant, we did observe a strong positive correlation between the absolute change in PSS and the relative change in the TBW when the data were pooled across the two postdeprivation blocks and participants in both eye groups. This finding implies that the two measures, that is, PSS and TBW, were driven by similar mechanisms underlying monocular deprivation, albeit less strongly for the PSS than for the TBW.

In conclusion, we have shown that a short perturbation in normal visual experience in adults through monocular deprivation results in significant, albeit transient, changes in multisensory temporal perception. These findings have implications on how the adult brain may react to visual impairment and how the clinical use of monocular deprivation as therapy may impact the restoration of balanced binocular function in adult patients with amblyopia. First, changes in visual experience in the adult can influence the way the visual system interacts with other sensory modalities, and thus such changes in multisensory perception may contribute to the overall compensatory behaviors observed in patients with adult-onset visual deprivation. Second, several studies have reported intact homeostatic plasticity among clinical populations such as amblyopia (Lunghi, Morrone, Secci, & Caputo, 2016; Zhou, Thompson, & Hess, 2013) and, consequently, have used monocular deprivation as therapy for improving visual perception in the amblyopic eye (Lunghi et al., 2019; Zhou et al., 2019). Our findings imply that, besides improving visual function in the amblyopic eye, monocular deprivation therapy may also ameliorate the impaired multisensory temporal processing observed in adult patients with amblyopia (Richards, Goltz, & Wong, 2017).