Disruption of information processing in schizophrenia: The time perspective.

We review studies suggesting time disorders on both automatic and subjective levels in patients with schizophrenia. Patients have difficulty explicitly discriminating between simultaneous and asynchronous events, and ordering events in time. We discuss the relationship between these difficulties and impairments on a more elementary level. We showed that for undetectable stimulus onset asynchronies below 20 ms, neither patients nor controls merge events in time, as previously believed. On the contrary, subjects implicitly distinguish between events even when evaluating them to be simultaneous. Furthermore, controls privilege the last stimulus, whereas patients seem to stay stuck on the first stimulus when asynchronies are sub-threshold. Combining previous results shows this to be true for patients even for asynchronies as short as 8 ms. Moreover, this peculiarity predicts difficulties with detecting asynchronies longer than 50 ms, suggesting an impact on the conscious ability to time events. Difficulties on the subjective level are also correlated with clinical disorganization. The results are interpreted within the framework of predictive coding which can account for an implicit ability to update events. These results complement a range of other results, by suggesting a difficulty with binding information in time as well as space, and by showing that information processing lacks continuity and stability in patients. The time perspective may help bridge the gap between cognitive impairments and clinical symptoms, by showing how the innermost structure of thought and experience is disrupted.

Introduction

We review here recent investigations regarding the visual perception of events in time in patients with schizophrenia. We examined the coding of the succession of distinct events in time, i.e. the ability to predict and follow the events flow. This ability is an integral part of our inner experience and our ability to interact with the outer world. We argue that these abilities are rooted in elementary mechanisms that allow us to distinguish and follow events in time on an unconscious level (< 20 ms), and which are impaired in patients with schizophrenia (Lalanne et al., 2012a, Lalanne et al., 2012b). Here we shall examine relationships between such elementary mechanisms and conscious experience, based on previous studies.

Time and predictive coding

Coding events efficiently in time is necessary in order to be connected with the outer world. It is necessary for encoding both predictable and new events. Predictive coding provides a theoretical framework to account for these abilities (Friston, 2008), by proposing that the brain triggers expectations about future sensory inputs. These can then be used to check whether actual sensory signals match expectations. If a match is found, sensory signals can be suppressed, whereas sensory information contrary to predictions will be relatively enhanced (Garrido et al., 2009). Predictable events thus bring about a suppression of information, whereas new events are detected by means of continuous updating of information. Here we focus on the regularity of this updating in time, on both unconscious and conscious levels. We also question how automatic updates are used and integrated into conscious, subjective experiences. It is not straightforward that the updating frequency is the same on automatic and subjective levels. In everyday life, new events can be both numerous and close in time, and the successive processing of events based on automatic updating could be misleading. For example, objects or people move behind information in the foreground or come out from side streets, windows are opened or closed, lights are switched on or off. Such events are successive, without, however, necessarily being logically related in time. A light might be switched on between successive appearances of the same moving object. If subjective perception were directly impacted by temporal updating mechanisms, the light would interrupt perception of the moving object, but this is not what happens. Furthermore, experimental evidence shows that our sensory system is not sensitive enough to capture the location of the moving object at exactly the same time as the light, resulting in the flash-lag effect, where the moving objet is perceived ahead of time in relation to the light (reviews in Hubbard, 2014, Shimojo, 2014). This suggests that the updating of information may not be totally accurate in time, and that additional processing affects our subjective perception. It thus raises questions about the relationship between the updating of information on an elementary and subjective levels. This is of particular relevance with respect to patients with schizophrenia, insofar as it has been proposed that they suffer from predictive coding impairments (Fogelson et al., 2014, Ford et al., 2014, Notredame et al., 2014). It is true that they display connectivity disorders (Friston, 1996, Uhlhaas and Singer, 2010), which might account for disturbances in recurrent loops subtending the constant updating of information processing and the detection of prediction errors (Fogelson et al., 2014). Patients with schizophrenia display a disturbed ability to detect deviants, e.g. a new and unexpected stimulus, and the amplitude of the EEG response to deviants is reduced (Umbricht and Krljes, 2005, for a review see Nagai et al., 2013). This would induce difficulties distinguishing between relevant and irrelevant information, and would cause patients to assign the wrong salience to events (Kapur, 2003, Nelson et al., 2013), possibly resulting in delusional beliefs (Schmack et al., 2013).

These difficulties may be explained in the context of predictive coding (Garrido et al., 2009). Several studies have suggested that some prediction aspects are impaired in patients (Ford et al., 2014, Franck et al., 2001, Frith, 2005, Neuhaus et al., 2013). However, the temporal dimension of this prediction has not been explored in patients (but see Schwartze et al., 2011 for evidence in healthy volunteers).

Since predictive coding is based on the continuous updating of information, any disturbance in how information is updated in time should impact predictive coding. Our results suggest not only that the frequency of updating is higher than previously believed in both controls and patients with schizophrenia, but also that the updating mechanisms are qualitatively impaired in patients.

Time events structure coding and schizophrenia

Distinguishing between two events in time requires that each event be considered to be ‘new’. If the second event is not detected as being new, it is either ignored or merged in time with the first, with the two events considered to be simultaneous. In the context of predictive coding, this means that an information update is needed to distinguish between events in time. Conversely, it means that our ability to distinguish between events in time might index the frequency of the updating mechanisms.

Many studies revealed a lower margin in our ability to distinguish subjectively between events in time, estimated at between 30 and 50 ms, irrespective of the sensory source. These results produced the concept of windows of time, or perceptual moments, within which all events are processed simultaneously (reviewed in Elliott et al., 2006, Elliott et al., 2007, Van Wassenhove, 2009, Wittmann, 2011). In the context of predictive coding, this means that information processing is updated every 50 ms. Interestingly, this time window is longer in patients, which suggests that updating is slower in patients with schizophrenia (Foucher et al., 2007, Giersch et al., 2009, Lalanne et al., 2012a, Schmidt et al., 2011). The time window is assessed using a simple paradigm involving two visual stimuli (e.g. two squares) shown on a computer screen1. They appear simultaneously or with a short stimulus onset asynchrony (usually between 0 and 100 ms) and participants judge whether the two stimuli are simultaneous or asynchronous. They respond by pressing a left response key for simultaneity and a right response key for asynchrony. Patients systematically require greater asynchronies than healthy participants before reporting that two stimuli are separated in time (Foucher et al., 2007, Giersch et al., 2009, Lalanne et al., 2012a, Schmidt et al., 2011). They have even more difficulty when they have to code the temporal order of the stimuli (Capa et al., 2014). Temporal order judgments were explored using exactly the same protocol as for asynchrony detection, but participants had to press the key on the same side of the second stimulus instead of deciding whether the stimuli are simultaneous or asynchronous. Control experiments have enabled us to rule out possible confounding factors like bias effects, eye movements, inter-hemispheric transfer or subjective judgments (review in Giersch et al., 2013).

However, our recent results challenge the assumption that all events are merged in time within 50 ms elementary time windows. They suggest, on the contrary, that events can be processed automatically as separate in time in the case of short delays of less than 20 ms, i.e. even when they are subjectively judged as being simultaneous. In the context of predictive coding, this suggests that updating mechanisms have a higher temporal resolution on the automatic level than on the subjective level. Below, we review evidence of such automatic mechanisms and their distortion in patients with schizophrenia.

Automatic updating of information within temporal windows

The exploration of implicit timing mechanisms was motivated by the mismatch between the mild clinical state of the patients involved, and their considerable impairments as regards subjectively distinguishing events in time. In some studies (with distractors, Giersch et al, 2009, or with multisensory information, Martin et al, 2013) patients with schizophrenia needed asynchronies of more than 100 to 200 ms to detect the stimuli were not simultaneous. We reasoned that if this were true in everyday life, it would cause major difficulties, which was not consistent with the mild clinical state of our outpatients. In our experiments, the instructions given included a direct, explicit question about the presence or absence of asynchrony. We wondered whether automatic processing in patients was more accurate than explicit responses suggested (Del Cul et al., 2006) and consequently used the Simon effect to investigate patients’ ability to code events in time independently of an explicit response. The Simon effect refers to the fact that responses are faster and more accurate when a visual stimulus is presented within the same perceptual hemifield as the responding hand (Hommel, 2011a, Hommel, 2011b, Van der Lubbe and Abrahamse, 2011). This effect was used to measure the implicit processing of events in time while avoiding the need for explicit instructions. As described above, two stimuli were displayed on the screen, one to the left and one to the right, and participants responded (‘simultaneity’ or ‘asynchrony’) by pressing the left or right response key, respectively. When both stimuli are displayed simultaneously, a Simon effect cannot occur, because the information displayed is equivalent on both sides of the screen, and participants cannot be biased to respond on any one side. Asymmetry only occurs when the stimuli are asynchronous, and in these conditions a Simon effect was observed (Lalanne et al., 2012a, Lalanne et al., 2012b). Healthy participants were biased to answer on the side of the second stimulus, independently of whether it was on the left or right. This means that responses differed according to the stimulus order. They were more frequent on the right when the second stimulus was on the right-hand side of the screen, and more frequent on the left when the second stimulus was on the left. Since physical information is identical on both sides, it is the temporal asynchrony that can be seen as responsible for the Simon effect. Importantly, the Simon effect was observed with short asynchronies (< 20 ms), even when the mean rate of ‘simultaneous’ responses was similar to the rate observed with perfect synchrony, i.e. when no asynchronies were detected (Lalanne et al., 2012b). It is when asynchronies are not detected that observation of a Simon effect provides an indication about the implicit processing of stimuli in time. When asynchronies cannot be detected, the stimuli are perceived subjectively in the same way as with perfect simultaneity. If this were also true on an automatic level, there should be no Simon effect. On the contrary, the fact that responses differ according to the stimulus order shows that part of the asynchrony has been processed implicitly, even though it was not captured by subjective judgments. Hence, the Simon effect observed with such asynchronies can be regarded as unconscious, and this effect might be interpreted as reflecting the updating of information on an automatic level. When subjects process a first stimulus, they are already prepared to process a following stimulus, and the results suggest that this update can occur unbeknown to the subject, in the case of delays lasting less than 20 ms.

Interestingly, Simon effects were also observed in patients with schizophrenia. For asynchronies above the threshold patients were biased towards the side of the second stimulus like healthy participants (Lalanne et al., 2012a). This contrasted with the results observed with the shortest asynchronies (below 20 ms), where patients were biased towards the side of the first stimulus, rather than the second. Like with the healthy controls, responses were more frequent on the left when the first stimulus was on the left, and on the right when the first stimulus was on the right, even when the asynchrony between the stimuli was not detected. This shows that part of the asynchrony has been automatically processed. We made sure patients did not display a general bias towards either the left or right (i.e. simultaneous or asynchronous), but in any case, such a spatial bias would not have yielded a bias depending on the stimulus order. In all the crucial result of our studies was that response rates depended on the stimulus temporal order, even for undetectable asynchronies (Lalanne et al, 2012b). The results show that patients distinguish between events in time on an implicit level. However, although patients did not merge events in time, the Simon effect was still on the opposite side in relation to controls, showing a qualitative difference in information processing with short asynchronies. Controls were biased towards answering on the side of the second stimulus, whereas patients were biased towards the side of the first stimulus. These results can be seen as evidence of impaired information updating in patients, who process stimuli in a discontinuous manner. Controls automatically follow events in time, and the high resolution of automatic updating mechanisms results not only in the processing of the first stimulus when it is isolated on the screen, but also in the ability to expect and subsequently process the second stimulus (Lalanne et al., 2012a, Lalanne et al., 2012b). Patients, on the other hand, remain stuck on the first event, i.e. at the updating stage. As such, these results would be consistent with disrupted predictive coding. At this stage, however, it is still unclear how these impairments relate to difficulties on a subjective level. This step is crucial for understanding the consequences of the impairments observed on a clinical level, and an attempt is made below to fill this gap in our knowledge.

Relationship between implicit and explicit coding in time

Results observed so far suggest a dissociation between patients’ implicit and explicit responses, since the Simon effect is on the side of the first stimulus in the case of short, undetected asynchronies, but on the side of the second stimulus when asynchronies are longer. Similarly, implicit and explicit responses have been shown to be dissociated when using multisensory stimuli (Martin et al., 2013). This raises the question of the relationship between impairments observed on an implicit and explicit level.

We combined two studies conducted and analyzed separately (Lalanne et al., 2012a and Experiment 1 in Lalanne et al., 2012b; method details can be found in these two papers). The demographic and clinical characteristics are detailed in Table 1 (Group 1).

Table 1

Demographic and clinical characteristics of the patients included in the reviewed studies.

					PANSS
		Sex ratio (M/F)	Years of education	Age	Total	Positive	Negative	Global	Disease duration (years)	Chlorpromazine equivalent
Group 1 (Lalanne et al., 2012a, Lalanne et al., 2012b) N = 30	Mean	18/12	12.0	36.0	68.0	15.3	20.2	32.4	12.0	317.5
	SD		1.9	6.4	16.8	3.7	6.6	9.4	10.3	285.1
Group 2 N = 28	Mean	20/8	12.9	36.2	73.5	17.0	20.1	36.3	13.2	234.3
	SD		2.3	9.1	20.7	5.6	7.2	11.1	7.8	150.3

In both studies, two squares were displayed in two of four possible locations, at one of the four corners of a virtual square in the middle of the screen. The only difference between the two experiments was the presence of connecters between squares in Lalanne et al. (2012b) (Fig. 1).

Fig. 1

Illustration of the procedure designed to explore simultaneity/asynchrony discrimination. Two squares are filled in, in gray, either simultaneously or asynchronously. These two squares were unconnected in Lalanne et al (2012a) (a), whereas half of them were connected in Lalanne et al. (2012b) (b). Participants are instructed to respond with the right key when they think that the squares are filled in asynchronously and the left key when they think that filling-in occurs simultaneously.

Performance did not differ significantly between experiments, which meant that we were able to combine the results together despite this difference. Most importantly, these were the two experiments in which we used SOAs of 8 ms only. We focus on this SOA since it is way below the threshold of subjective asynchrony detection and definitely unconscious. Since our aim was to explore the relationship between implicit and subjective asynchrony detection, we considered the conditions which had led to a Simon effect, i.e. to a difference in the rate of ‘simultaneous’ response as a function of the target order (targets located to the left and right of the middle of the screen in Lalanne et al., 2012a, and connected targets in the same locations in Lalanne et al., 2012b).

We conducted an ANOVA on the amplitude of the Simon effect at 8 ms, with group (patients vs. controls) and experiment as between-group factors. The Simon effect at 8 ms differed significantly between groups (F[1, 56] = 4.9, p < .05, η2 = 0.08), η2 = 0.1). There was no bias in controls (F < 1). In contrast, patients showed a significant bias to press to the side of the first square at 8 ms (9% more responses on the left side for left–right vs. right–left squares, F[1, 28] = 8.5, p < .01, η2 = 0.23, Fig. 2). This reflects a difference in the rate of ‘simultaneous’ responses according to stimulus order. It can equally be expressed as a decrease of the rate of ‘simultaneous’ responses in the right–left vs. left–right order. In fact, the comparison of response rates at 0 and 8 ms SOA in patients shows that in the right–left order, the rate of ‘asynchronous’ responses significantly increases at 8 relative to 0 ms (by 7%, F[1, 28] = 6.5, p < .05, η2 = 0.19).

Fig. 2

Amplitude of the bias (in %) to the side of the 1st or 2nd stimulus, when one stimulus is displayed on the right side of the screen and the other one on the left. A positive bias corresponds to a bias to the side of the first stimulus (in patients), whereas a negative bias corresponds to a bias to the side of the second stimulus (in healthy participants). See Lalanne et al. (2012a) and (2012b), for more detailed results.

The results therefore show that patients distinguish stimuli in time with surprising accuracy, at least on an implicit level. This is all the more surprising because they need larger asynchronies than controls in order to detect them explicitly. The question was whether the implicit ability to distinguish events in time facilitates and compensates for the difficulty detecting larger asynchronies on a subjective level. Correlations conducted on the above results suggest the opposite. In patients, the Simon effect was positively correlated with the rate of ‘simultaneous’ responses at SOAs above 50 ms (Table 2). The greater the tendency to press on the side of the first stimulus, the greater the difficulties detecting asynchronies on a subjective level. This correlation is not trivial, since at 8 ms the Simon effect reflects high temporal accuracy on an implicit level, whereas performance above threshold reflects the opposite on an explicit level. However, the correlation is consistent with our interpretation of the Simon effect in patients as reflecting a difficulty with implicitly following/predicting stimuli in time. The correlation further suggests a link between this implicit effect and the difficulty observed on a subjective level. The difficulty automatically following/predicting stimuli might play a causal role in the impairments in respect of distinguishing stimuli in time on a subjective level. No correlation was found with clinical symptoms.

Table 2

correlations between the Simon effect at 8 ms and the rate of simultaneous responses, in the data issued from Lalanne et al., 2012a, Lalanne et al., 2012b and (2012b).

30 patients with schizophrenia		R	p
Simon effect at 8 ms	Rate of ‘simultaneous’ response with:
	SOA 64 ms	0.41	0.025
	SOA 72 ms	0.40	0.028
	SOA 83 ms	0.55	0.002
	SOA 92 ms	0.46	0.011

We checked correlations in more recent data collected in a group of 28 patients. In this study we used a single paradigm instead of two different experiments (for demographic details, see Table 1, Group 2). Using the same experiment for all patients was expected to make correlations between performance and clinical symptoms more reliable. Only two locations were used for the targets, without any connecter. The minimal SOA was too long (24 ms) to examine the Simon effect as above. Hence we only correlated the ‘simultaneous’ response rate at the different SOA with clinical evaluations. We found that in patients the simultaneous response rates near threshold and above were significantly correlated with disorganization, as evaluated with item P2 of the PANSS (thought disorganization), and with the Lepine disorganization score (sum of items P2, N5, G10 and G11 in the PANSS, van Assche and Giersch, 2011; Table 3). The worse the ability to detect asynchronous events in time, the worse the disorganization. We found no other significant correlation with clinical symptoms. This effect is reminiscent of a correlation observed previously in a smaller group of patients (N = 20), when exploring temporal order judgment (Capa et al., 2014). In this study, there had been an (unpublished) correlation with the error rate for discriminating temporal order and item P2 (R = 0.65, p < .005 for the 48 ms SOA, and R = 0.54, p < 0.5 for the 72 ms SOA). These results are in keeping with the correlation found between perceptual and conceptual (clinical) disorganization (Silverstein and Keane, 2011, Uhlhaas and Silverstein, 2005, Van Assche and Giersch, 2011). It is as if the inability to update information and thus to go from one stimulus to another in time relates with a more general coordination failure.

Table 3

Correlations between clinical scores and the rate of simultaneous responses.

28 patients with schizophrenia		R	p
Disorganization score of Lepine P2/N5/G10/G11	Rate of ‘simultaneous’ response with:
	SOA 48 ms	0.48	0.010
	SOA 72 ms	0.47	0.011
	SOA 96 ms	0.40	0.033
	Threshold	0.50	0.007
Thought disorganization P2	Rate of ‘simultaneous’ response with:
	SOA 48 ms	0.52	0.004
	SOA 72 ms	0.47	0.011
	SOA 96 ms	0.60	0.001
	Threshold	0.56	0.002

Discussion and conclusion

Our results have shown that patients have difficulty distinguishing events in time on a subjective level, especially when they are asked to put events in order. On an implicit level, however, both controls and patients with schizophrenia have been shown to distinguish events in time automatically. Yet, patients still differ from controls on this implicit level. Controls are able to follow events automatically on an implicit level, whereas patients are stuck on the first event, without being able to go smoothly from the first to the second. In addition, we showed this to be true for asynchronies as short as 8 ms. At 8 ms the Simon effect was correlated with difficulties detecting asynchronies on a subjective level (Group 1), the latter being correlated with clinical disorganization (Group 2). The fact that in patients there is no merging in time at 8 ms shows that the updating mechanisms are sufficiently preserved for patients to catch 8 ms events. This is consistent with previous data (Herzog et al., 2004). However, unlike controls, patients do not process the second event, as if their ability to predict/follow the next event is disturbed when it is close in time. This might be related to other disturbances described in patients. For example, processing of the first event might not be stabilized fast enough for patients to follow information when the next event is close in time (Herzog et al., 2013). Similarly, this difficulty might have to do with deficient phasic responses of the magnocellular system (Dias et al., 2011, Knebel et al., 2011), although this would not explain why difficulties with explicitly detecting asynchronies are related to the Simon effect at 8 ms, or why patients’ ability to judge the order between stimuli more distant in time is impaired (Capa et al., 2014). This relationship between the updating mechanisms on an automatic and subjective level is consistent with our earlier proposal, according to which the Simon effect captures a core mechanism allowing us to relate events in time. This may involve specific neurobiological networks. In the context of an impairment in the cortico-cerebellar-thalamic-cortical circuit (Andreasen, 1999), high resolution systems like the cerebellum (Coull et al., 2011, Schwartze and Kotz, 2013) would be preserved, inasmuch as patients still distinguish information automatically in time when asynchronies are short. On the other hand, the integration of successive events enabling them to be linked would reflect abnormal integration of information beyond systems like the cerebellum.

If patients are unable to follow events on an unconscious level, the continuity of their perception, and ensuing train of thought would be disrupted. In turn, it would make it difficult to order events on a subjective level, especially as time order needs to be integrated with spatial organization. Accordingly, time disorders would complement the disorganization in space (Silverstein and Keane, 2011, Tschacher et al., 2006). Moreover, we have proposed that disorganization in time might impact patients’ ability to experience themselves as a continuous self (Martin et al., 2014). This hypothesis refers to the concept of minimal self, i.e. the fact that any perception, thought, or action is tacitly experienced in first-person mode. For example, when we say ‘I see the tree’, we are usually talking about the tree, without reflecting on the use of ‘I’. This non-reflexive use of ‘I’ illustrates the implicit (but not unconscious) presence of oneself in our mental activities. Cermolacce et al. (2007) and Sass and Parnas (2003) have described minimal self disorders in patients with schizophrenia and have observed that their experiences (thoughts, actions, perceptions) are not ‘given’ to them in a first-person perspective. Inasmuch as the minimal self is defined, or at least influenced, by our conscious perceptual experiences (see Mishara, 2007, for background theory on this point), it follows that disruptions in perceptions may affect the minimal self, i.e. the feeling of being here and now. Indeed, if patients are unable to follow events in time efficiently, and are thus not attuned to the outer world, it might be difficult for them to feel they are present here and now. Furthermore, we can hypothesize that temporal disruptions reveal discontinuities in patients’ inner thoughts (Northoff, 2015), in which case the fragmentation of their mental experience might directly affect their feeling of being one continuous self. These time disorders may thus constitute a key link between elementary and clinical disorders, and may provide an explanation for self-descriptions like the following (Fuchs, 2013):

“Time is also running strangely. It falls apart and no longer progresses. There arise only innumerable separate now, now, now … quite crazy and without rules or order. It is the same with myself. From moment to moment, various ‘selves’ arise and disappear entirely at random. There is no connection between my present ego and the one before”.