Patients with schizophrenia have a reduced neural response to both unpredictable and predictable primary reinforcers.

One prevalent theory of learning states that dopamine neurons signal mismatches between expected and actual outcomes, called temporal difference errors (TDEs). Evidence indicates that dopamine system dysfunction is involved in negative symptoms of schizophrenia (SZ), including avolition and anhedonia. As such, we predicted that brain responses to TDEs in dopamine midbrain nuclei and target areas would be abnormal in SZ. A total of 18 clinically stable patients with chronic SZ and 18 controls participated in an fMRI study, which used a passive conditioning task. In the task, the delivery of a small amount of juice followed a light stimulus by exactly 6 s on approximately 75% of 78 total trials, and was further delayed by 4-7 s on the remaining trials. The delayed juice delivery was designed to elicit the two types of TDE signals, associated with the recognition that a reward was omitted at the expected time, and delivered at an unexpected time. Main effects of TDE valence and group differences in the positive-negative TDE contrast (unexpected juice deliveries-juice omissions) were assessed through whole-brain and regions of interest (ROI) analyses. Main effects of TDE valence were observed for the entire sample in the midbrain, left putamen, left cerebellum, and primary gustatory cortex, bilaterally. Whole-brain analyses revealed group differences in the positive-negative TDE contrast in the right putamen and left precentral gyrus, whereas ROI analyses revealed additional group differences in the midbrain, insula, and parietal operculum, on the right, the putamen and cerebellum, on the left, and the frontal operculum, bilaterally. Further, these group differences were generally driven by attenuated responses in patients to positive TDEs (unexpected juice deliveries), whereas responses to negative TDEs (unexpected juice omissions) were largely intact. Patients also showed reductions in responses to juice deliveries on standard trials, and more blunted reinforcer responses in the left putamen corresponded to higher ratings of avolition. These results provide evidence that SZ patients show abnormal brain responses associated with the processing of a primary reinforcer, which may be a source of motivational deficits.

Introduction

In addition to positive, or psychotic symptoms, schizophrenia (SZ) is characterized by negative symptoms, reflecting deficits in areas such as motivation, emotional expression, and speech production. These negative symptoms have been shown to relate closely to functional impairments exhibited by patients (Green et al 2000; Tamminga et al 1998), particularly with regards to engagement in goal-directed behavior. The fact that patients also show deficits on experimental tasks of reinforcement learning suggests that dysfunctional reinforcement processing mechanisms may contribute to negative symptoms. This idea is further supported by evidence that brain dopamine (DA) systems, which are known to play a pivotal role in modulating reinforcement learning (Montague et al 2004), are disrupted in schizophrenia (Abi-Dargham et al 2000; Breier et al 1997; Laruelle et al 1996; Okubo et al 1997). As such, the purpose of our study was to investigate whether disrupted reward processing mechanisms contribute to deficits in motivated behavior in schizophrenia by examining the role of DA in reward processing.

One prevalent theory of DA system function is that DA cells signal, through phasic modulations of their firing rates, the registration of mismatches between reinforcer expectations and outcomes (Schultz et al 1997), known as “reward prediction errors,” or “temporal difference errors” (TDEs; if they pertain to the timing of reinforcement). Neuroimaging studies with human subjects have observed activation changes in both (presumably) dopaminergic midbrain nuclei (Aron et al 2004) and dopamine target structures (Berns et al 2001; McClure et al 2003; Seymour et al 2004) in association with the perception of prediction errors. McClure and colleagues (2003) found that an area of the left putamen showed significantly decreased activation, relative to baseline, when a juice reinforcer was omitted at the expected time (corresponding to a negative TDE). This area also showed increased activation when the reinforcer was unexpectedly delivered 10 seconds after the light cue (corresponding to a positive TDE).

The results of several studies (Jensen et al 2008; Murray et al 2007), in fact, suggest that SZ patients have specific impairment in the perception and representation of prediction errors. Murray et al. (2007) found differential activations between psychotic patients and controls in the dopaminergic midbrain in response to unexpected monetary outcomes in an operant learning paradigm. In learning to choose between stimuli predicting monetary outcomes, patients showed attenuated neural responses to outcomes that were better or worse than expected, whereas their responses to neutral outcomes were somewhat elevated, relative to those of controls. Corlett et al. (2007) found that prediction-error evoked fMRI activity in prefrontal cortex (PFC) was strongly predictive of clinical ratings of delusional thought content, as measured by the Brief Psychiatric Rating Scale (BPRS; Overall and Gorman 1962).

Study objectives and predictions

Multiple studies (Corlett et al 2007; Murray et al 2007) have identified abnormalities in prediction signaling in the context of higher-level cognitive tasks using a symbolic reinforcer (e.g., money). Determining whether or not there are alterations in responses prediction errors regarding a primary reinforcer (e.g., a food reinforcer) can make an important contribution to the interpretation of the above findings, by helping to establish whether these findings generalize to more elementary forms of learning. In order to address this issue, we implemented a version of the Pavlovian classical conditioning paradigm employed by McClure and colleagues (2003), which used a juice reinforcer, in conjunction with fMRI. This paradigm enabled us to separately investigate neural responses to positive and negative prediction errors in SZ patients. Furthermore, our study was specifically designed to examine the signaling of errors in predictions about the timing of a reinforcer (TDEs) in schizophrenia. Finally, because we hypothesized that faulty TDE processing is at the root of deficits in reward-driven learning and behavior, our goal was to investigate relationships between reinforcer responses and ratings of negative symptoms.

Based on evidence of both disrupted DA function in SZ (Abi-Dargham et al 2000; Breier et al 1997; Laruelle et al 1996; Okubo et al 1997) and deficits in reinforcement learning driven by both positive and negative feedback (Prentice et al in press; Waltz et al 2007; Waltz and Gold 2007), we hypothesized that patients would show abnormal brain correlates of both positive and negative TDEs. Based on our formulation that negative symptoms in SZ reflect a reduced ability of rewards and punishments to modulate learning and motivate behavior in SZ patients, we further hypothesized that patients would show systematic relationships between ratings of negative symptoms, such as avolition, and brain responses to the presence and absence of rewards.

Participants and Methods

Recruiting and Screening of Participants

Eighteen patients and 18 demographically-matched healthy controls participated in the study (Table 1). All participants provided written informed consent to a protocol approved by the institutional review boards at the National Institute on Drug Abuse and the University of Maryland School of Medicine. All participants were right-handed, as determined by the Edinburgh Handedness Inventory (Oldfield 1971).

Table 1

Subject Descriptive Information.

	NCs (N=18)		SZs (N=18)		p-value
	Mean	SD	Mean	SD
Demographic Information
Age	37.1	(9.0)	37.7	(10.0)	ns
Gender (M/F)	14/4		13/5		ns
Race (C/AA/O)	5/12/1		11/6/1		0.094
Neuropsychological Performance
WTAR - Scaled	107.2	(13.3)	101.7	(16.1)	ns
WASI - Full IQ	116.2	(11.3)	103.3	(16.5)	0.010
Chapman Anhedonia Scales
Physical Anhedonia	11.8	(6.2)	11.6	(7.8)	ns
Social Anhedonia	7.8	(6.1)	10.9	(5.8)	ns
Symptom Ratings
BPRS - Total			31.7	(8.9)
SANS - Total			23.5	(16.9)
CDS - Total			1.1	(2.2)

All patients were on stable antipsychotic medication regimens (no changes for four weeks), almost all with second-generation antipsychotic drugs (Supplementary Table 1). The diagnosis of schizophrenia or schizoaffective disorder in patients was confirmed using the SCID-I (First et al 1997), as was the absence of Axis I diagnoses in control participants. Control participants diagnosed with Axis II personality disorders (based on screening with the SIDP-R; Pfohl et al 1989) were also excluded. Participants in both experimental groups underwent medical screening, involving a medical history and physical exam, and subjects with evidence of any neurological or medical condition that might confound data interpretation were excluded (such as significant head injury, stroke or severe vascular problems, chronic untreated diabetes, etc.). Further exclusionary criteria included: pregnancy, current illegal drug use (both verified by urine screens), and admission of past substance dependence.

Procedures

Events prior to MRI scanning

Participants were instructed to abstain from alcohol for 24 hours prior to each visit, which was verified by a breathalyzer test prior to each session. Subjects were fluid-deprived for 3 hours prior to the actual MRI session to increase the value of the juice, which served as a primary reinforcer (see below).

Experimental task (TDE paradigm)

Participants underwent a classical conditioning task (Berns et al 2001; McClure et al 2003; see Figure 1). Prior to actual MRI scanning, participants were trained in a mock scanner to associate the receipt of a juice reward with the presentation of a light stimulus (a large yellow disc on a black background, which was displayed on a screen behind the subject’s head and viewed through a mirror attached to the head coil). During training, participants completed three runs of 26 ‘standard’ trials each, during which the light-juice interval was always 6 seconds. Following training, subjects performed the same task in the MRI scanner. During scanning, however, a number of ‘catch’ events were introduced. On catch trials the juice was not delivered at the ‘expected’ time point, but rather 4–7 seconds after the expected delivery. Based on McClure et al. (2003), we predicted that this type of catch event would induce a negative prediction error at the time when juice was not delivered as expected, and that a subsequent positive prediction error would be incurred by the ‘unexpected’ delivery of juice. The number of paired training events outnumbered the catch events by a ratio of approximately 3:1 in scanning sessions. The time between individual trials ranged from 4–12 seconds.

Figure 1

Classical Conditioning Task. Timing of events on normal and “catch” trials. On catch events, delivery of the juice reward occurs 10–13 second after the light cue, instead of the usual 6 seconds. During the training session (outside of the MRI scanner), subjects were presented only with the standard light-juice interval (6 s). During the MRI scanning session, catch trials were interspersed among the standard trials, representing approximately 1/4 of total trials.

Prior to training, participants chose one type of commercially available juice from three options: apple, grape, or fruit punch. Juice was delivered through small-bore IV tubing connected to syringes set into an MR-compatible syringe pump (Harvard Apparatus; Boston, Massachusetts). The end of the tubing was placed in the participant’s mouth, attached to a commercial sports mouth-guard to stabilize the tube. A 600-μl squirt of juice was delivered on each trial, at a rate of 1 ml/second (duration=0.6 s). Between runs of the TDE task, subjects rated the pleasantness of the juice stimuli, by moving a cursor on a visual-analog scale using a wheel manipulandum, with a rating of 8 representing extreme liking, and rating of 0 representing extreme aversion.

Interval Timing Test

The capacity of an individual to learn the association between the light cue and the juice reward is reliant upon their ability to accurately estimate the time at which they should (based on previous learning) expect receipt of the reward. To assess time estimation abilities, participants completed a short test of timing function, based on the work of Rao and colleagues (Harrington et al 2004; Hinton et al 2004), outside of the MRI scanner and following all other experimental phases. On each trial, subjects were presented with three temporal intervals, defined by four beeps, and asked to judge whether the third interval was shorter than, longer than, or the same as the two preceding intervals. The first two intervals were 6 s in all cases, while the third ranged from 4.5 to 7.5 s, in increments of 0.5 s. Thus, the timing paradigm assessed the ability to estimate a similar time interval as was used in the TDE/classical conditioning paradigm, albeit demanding much greater precision.

Other Psychological Assessments

Cognitive function was assessed in all participants using three standard measures: the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler 1999), the Wechsler Test of Adult Reading (WTAR; Wechsler 2001), and the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph et al 1998; Wilk et al 2005). In order to quantify the ability of study participants to experience pleasure both physically and in social contexts, we had all subjects complete the Scales for Physical and Social Anhedonia (Chapman et al 1976). Standard symptom ratings were obtained for all patients using the BPRS (Overall and Gorman 1962), the Scale for the Assessment of Negative Symptoms (SANS; Andreasen 1984), and the Calgary Depression Scale (CDS; Addington et al 1992).

Acquisition and Analysis of fMRI Data

MRI scanning

A 3-T Siemens Allegra scanner (Erlangen, Germany) acquired whole-brain functional EPI images for measurement of T2*-weighted BOLD effects (4-mm oblique axial slices, 30° axial to coronal; 64 × 64 matrix; FOV = 22 × 22 cm; TR = 2 s; TE = 27 ms; FA = 80°). In each scanning session, a whole-brain oblique axial T1-weighted structural image (MPRAGE) was acquired for anatomical reference (1-mm3 isotropic voxels; TR = 2.5 s; TE = 4.38 ms; FA = 80°).

Whole-brain analyses of MRI data

All preprocessing and first-level analyses of MRI data were performed using the AFNI software package (Analysis of Functional NeuroImages; Cox 1996). Preprocessing steps included volume-registration for motion correction, slice-timing correction, and temporal normalization. In order to generate statistical parametric maps for individual subjects, we used an approach similar to McClure et al. (2003) and D’Ardenne et al. (2008). We used three types of events as regressors in regression analyses: juice deliveries on standard trials, and positive and negative TDEs (juice deliveries and omissions on catch trials, respectively). Regressors were modeled by an idealized hemodynamic response function and its temporal derivative, time-locked to the event onsets. Additional regressors of no-interest included the six motion-correction curves.

For the purposes of this study, we examined responses only after learning because it provides the strongest method for studying TDE responses in isolation, while avoiding dependencies on exact learning-model parameters. That is, when learning is complete, positive and negative prediction errors evoked by changing the time of reward can be fully modeled without dependence on how effective TDEs were in driving learning.

Beta weights from the first-level analysis were spatially normalized to Talairach space and smoothed with a 4.2 mm FWHM Gaussian kernel prior to second-level (i.e. group) analyses. The main second-level analyses were two separate repeated-measures ANOVAs, performed to determine brain areas showing significant main effects of group, event type, and group x event type interactions: one with factors of group (patients vs. control) and TDE valence (positive vs. negative), and one with factors of group and trial-type (catch vs. standard, to compare responses to ‘unexpected’ and ‘expected’ deliveries of the reinforcer). Correction for multiple comparisons was accomplished using a Monte Carlo simulation, which determined a minimum cluster size required for a given voxel-wise threshold. This simulation determined that, for our chosen voxel-wise threshold (p<0.002), a minimum cluster size of 11 voxels (424 μl) was required to achieve an overall significance level of p < 0.05.

Analyses of group differences in regions of interest (ROI)

In order to compare the performance of the two groups in regions involved in the processing of the reinforcer, as well as prediction errors, we selected all significant clusters showing significant main effects of TDE valence and all clusters showing significant group X TDE valence interactions. This criterion yielded 11 ROIs, described below. Based on the results of McClure et al. (2003), which used the same paradigm to examine prediction-error-related brain activity in healthy volunteers, we selected two additional regions of interest a priori, in the left and right putamen by drawing spheres of 5 mm radius around their Talairach coordinates (±18, 2, 8). Thus, we performed further analyses on 13 total ROIs. Reported p-values based on this group analysis for a priori regions of interest were corrected for the number of comparisons made within each region.

Correlation analyses

We used Spearman correlation analyses to assess relationships among timing task performance, clinical ratings of avolition (to assess reductions in goal-directed behavior in schizophrenia), and BOLD responses to juice deliveries in the 11 ROIs identified functionally, and the 2 ROIs identified a priori. Clinical ratings of avolition were determined by summing the four items from the avolition subscale of the 25-item SANS.

Results

Brain regions distinguishing positive from negative TDEs

A two-way ANOVA with factors of group and TDE valence was performed to determine the brain regions distinguishing positive from negative TDEs in the entire sample (N=36; see Table 2a). This ANOVA produced five brain regions showing main effects of TDE valence, all of which resulted from activations to positive TDEs and deactivations to negative TDEs. Regions showing main effects of TDE valence included left putamen and the right midbrain, consistent with previous findings (D’Ardenne et al 2008; McClure et al 2003; Murray et al 2007; see Figure 2a–c). Large areas also emerged, centered on the left and right frontal and parietal operculum, extending into insular cortex. These areas correspond to primary gustatory cortex, as identified in previous studies (Lee et al 1998; Small et al 1999). For the purpose of ROI analysis, we divided each of the large areas into three components, based on their overlap with precentral gyrus, postcentral gyrus, and insula, as identified by the Talairach daemon (Lancaster et al 2000). Together with an additional cluster emerging in the declive of the cerebellum, this yielded 9 ROIs for further analysis. Note that a main effect of group, without respect to TDE valence, is difficult to interpret, because we expected (and observed) activations for positive TDEs, and deactivations for negative TDEs; thus, we do not report this effect.

Table 2

Brain regions showing significant main effects of TDE valence (positive vs. negative) or significant group × TDE valence interactions.

		(a) Main Effect of Event (Signif. Pos. - Neg. TDE Contrast)					(b) Group × TDE Valence Interaction (Group Diff. in Pos. - Neg. TDE Contrast)
Location(s)	L/R	ROI(s)	Vol (μl)	# Voxels	X, Y, Z	Mean F	ROI(s)	Vol (μl)	# Voxels	X, Y, Z	Mean F
Midbrain Putamen	R	1	2761	70	3, −19, −2	16.7
	L	2	2545	65	−23, −1, −4	15.9
Pre- and Postcentral Gyrus/Insula	R						10	516	13	22, 6, 19	15.3
	L	3,5,7	18642	474	−55, −6, 20	21.1	11	504	13	−51, −9, 45	15.6
Declive of Cerebellum	R	4,6,8	26383	671	54, −6, 20	21.7
	L	9	1480	38	−18, −66, −17	15.2

Figure 2

Regions emerging from whole-brain analyses. a–c. Regions showing differential responses to positive and negative TDEs within the entire sample (N=36). Midbrain contrasts are shown in panels a and b, while contrasts in putamen and insula are depicted in panels b and c. d–f. Areas showing group X TDE valence interactions. Panel d shows right putamen, while panels e and f show region in left precentral gyrus. In all coronal and transverse slices, radiological convention is used, depicting the left hemisphere on the right side of the image.

Two brain regions, the right putamen and in the left frontal operculum, showed significant group × TDE valence interactions (Table 2b, Figure 2d–f). In these two areas, controls showed BOLD activations to positive TDEs and deactivations to negative TDEs, whereas patients showed the opposite (aberrant) pattern (see Figure 3b–c, ROIs 10 and 11).

Figure 3

a. Contrast in MRI activity evoked by positive and negative TDEs in each group, in each ROI. b. Analysis of MRI activity evoked by positive TDEs, relative to baseline. c. Analysis of MRI activity evoked by negative TDEs, relative to baseline. d. Analysis of MRI activity evoked by standard juice deliveries, relative to baseline. In all panels, * indicates group difference significant at p < 0.05. MRI signal changes expressed in arbitrary units (A.U.).

ROI analyses of group differences in the positive – negative TDE contrast

With 11 regions emerging from whole-brain analyses, and 2 regions chosen a priori, we performed additional analyses on 13 total ROIs. In 9 of the 13 ROIs investigated, the two subject groups differed in the magnitude of the [positive – negative] TDE contrast (the difference trended toward significance in ROI 13, the right putamen; Figure 3a). This was true despite the fact that in nine of these regions (ROIs 1–9), the magnitude of the [positive – negative] TDE contrast was significantly different from zero within the patient group (all t’s > 2.75; see Figure 3a).

Further analyses revealed that group differences in the [positive – negative] TDE contrast were generally driven by attenuated responses to positive TDEs in the patient group (all t’s > 3.10; Figure 3b). Whereas controls showed significant activations to positive TDEs, relative to baseline, in all 13 ROIs, patients showed significant activations to positive TDEs only in the pre- and postcentral gyrus, bilaterally (ROIs 5–8). In response to negative TDEs, however, patients and controls did not differ significantly in their responses to negative TDEs in any of the ROIs (Figure 3c), with patients showing robust deactivations, relative to baseline, in nine regions (ROIs 1–9), including the midbrain and left putamen.

Together, these analyses indicate that group differences in BOLD signal modulations, due to the valence of prediction errors, generally result from attenuated responses in patients to unexpected administrations of the reinforcer. In order to determine whether patients’ responses to positive TDEs were selectively disrupted, or if patients showed more general dysfunction in neural processing of the reinforcer, we performed analyses comparing subjects’ responses to unexpected (catch) versus expected (standard) juice deliveries.

MRI responses to non-TDE events

Juice deliveries on standard vs. catch trials

A two-way ANOVA of group X trial-type was performed to compare unexpected deliveries of reward (positive TDEs) with expected reward delivery (null TDEs). This analysis revealed significant main effects of group in numerous areas, regardless of whether they occurred on standard or catch trials (Table 3). Controls showed more positive responses to juice delivery than patients, regardless of trial type, in all of these areas, which overlapped with many of the ROIs identified above, including midbrain, left and right putamen, left and right frontal operculum, and left insular cortex.

Table 3

Clusters showing main effects of group for juice delivery, regardless of trial type

Location(s)		Vol (μl)	# Voxels	X, Y, Z	Mean F
Midbrain/Basal Ganglia
Midbrain	L	966	25	−3, −19, −14	14.2
Putamen	L	9503	242	−23, −9, 9	15.2
	R	12418	316	26, −4, 5	15.1
Frontal Cortex
Precentral Gyrus	L	461	12	−61, −7, 22	14.0
	R	2683	68	53, −12, 32	14.0
Insula	L	1626	41	−57, −32, 20	14.7
Inf. Frontal Gyrus	L	490	12	−37, 5, 30	14.1
Mid. Frontal Gyrus	L	686	17	−31, −6, 43	13.3
Mid. Frontal Gyrus	R	596	15	36, 61, 13	13.3
	R	3483	89	48, 15, 26	15.2
Medial Frontal Gyrus	R	520	13	1, −25, 68	13.3
Cingulate Gyrus	L	939	24	−3, −29, 29	14.9
	R	3283	84	3, −1, 38	14.6
Paracentral Lobule	R	1718	44	1, −22, 45	14.8
Parietal Cortex
Postcentral Gyrus	L	4717	120	−40, −21, 55	14.6
	R	1366	35	49, −20, 55	15.2
Inf. Parietal Lobule	L	1063	27	−33, −34, 28	14.2
	R	3683	94	49, −51, 48	14.7
Sup. Parietal Lobule	L	907	23	−27, −65, 54	13.9
Supramarginal Gyrus	L	2964	75	−52, −53, 25	14.7
Precuneus	L	1349	34	−10, −74, 45	14.8
	R	516	13	4, −56, 48	15.0
Temporal Cortex
Sup. Temporal Gyrus	R	2034	52	47, −44, 18	13.5
Mid. Temporal Gyrus	L	867	22	−56, −34, −6	14.0
	R	1601	41	54, −38, −8	15.0
Cerebellum
Declive	L	2081	53	−29, −68, −15	15.1

Also in our previously identified ROIs, participants within each group showed similar responses to juice deliveries on standard and catch trials, with controls showing significant activations to both in all regions (relative to baseline; compare Figures 3b and 3d). In three of the 13 ROIs, in fact, BOLD responses to expected juice deliveries and unexpected juice deliveries correlated significantly in the patient group (see Supplementary Table 2).

No region showed a significant main effect of trial-type, and no brain region showed a significant group X trial-type interaction. Thus, we did not find evidence of an enhanced neural response to (presumably unexpected) juice deliveries on catch trials, relative to juice deliveries on standard trials.

MRI responses to the light cue

It should be noted that patients did not show attenuated activations, or deactivations to all event types; brain responses to the reward-predicting cue (the conditioned stimulus) were largely intact in SZ patients (see Supplementary Table 3; Supplementary Figure 1).

Behavioral and Correlation Analyses

Timing Task Performance

Patients and controls differed significantly in their performance on the interval timing task done outside the scanner [t(31)=3.38; p=0.002]. Whereas patients correctly estimated only 45.8% (SD=12.1%) of test intervals, controls correctly estimated 61.6% (SD=14.7%) of test intervals.

When we computed Spearman correlation coefficients for relationships between scores on the interval timing task and MRI responses to standard juice deliveries, we observed a single significant correlation: timing task performance in patients related systematically to responses to standard juice deliveries in the left frontal operculum/precentral gyrus (Table 4).

Table 4

Spearman Correlation between Standard Juice Responses and Behavioral Measures in Patients

ROI Source	Region of Interest	Interval Timing		Avolition
		Rho	p	Rho	p
WBA: ME of TDE Valence	(1) R Midbrain	−0.330	0.196	−0.439	0.068
	(2) L Putamen	−0.238	0.357	−0.380	0.120
	(3) L Insula	−0.217	0.403	−0.398	0.102
	(4) R Insula	−0.228	0.379	−0.462	0.053
	(5) L Precentral Gyrus	−0.184	0.479	−0.585	0.011
	(6) R Precentral Gyrus	−0.238	0.357	−0.413	0.088
	(7) L Postcentral Gyrus	−0.249	0.334	−0.504	0.033
	(8) R Postcentral Gyrus	−0.248	0.337	−0.418	0.085
	(9) L Declive	−0.470	0.057	−0.079	0.755
WBA: Group × TDE Valence Interaction	(10) R Putamen	0.280	0.277	0.007	0.977
	(11) L Precentral Gyrus	0.553	0.021	−0.217	0.388
A priori	(12) L Putamen	−0.172	0.510	−0.566	0.014
	(13) R Putamen	−0.246	0.342	−0.477	0.045

Abbreviations: WBA, whole-brain analysis; ME, main effect; TDE, temporal difference error

Anhedonia and Avolition

Exploratory analyses of relationships between BOLD data and rating of clinical symptoms revealed that avolition subscores from the SANS correlated negatively with MRI responses to standard juice deliveries in the left frontal and parietal operculum (pre- and postcentral gyrus; Table 4). Furthermore significant negative correlations were observed between avolition ratings and MRI responses to standard juice deliveries in the left and right a priori putamen ROIs, and six additional ROIs showed correlations of medium effect size (d > 0.3; Cohen 1992). In all of these ROIs, patients with the highest ratings of avolition showed the least positive BOLD responses to juice deliveries. Supplementary Figure 2 illustrates this pattern in the left frontal operculum and the left putamen ROIs.

By contrast, avolition ratings bore no relationship to self-reports of reinforcer enjoyment (Spearman’s rho = −0.023), which did not differ between patients and controls [mean rating of 5.8 (1.4) for patients vs. mean of 5.3 (1.1) for controls; t(34)=1.25; p>0.10]. Avolition ratings also bore no relationship to self-reports of physical (ρ = −0.287) and social anhedonia (ρ = −0.037) from the Chapman scales. Patients and controls also did not differ significantly on these measures [t(34)<1.6; p>0.10 in both cases; see Table 1].

Discussion

In this functional MRI study of prediction error-related activity, our data led us to draw the following conclusions: 1. Multiple components of reward circuits showed responses that distinguished between positive and negative TDEs in the entire sample of participants; 2. Patients with schizophrenia showed reduced contrasts in brain activity evoked by positive and negative TDEs in multiple brain areas, driven largely by attenuated responses to positive TDEs; 3. Patients showed attenuated responses in numerous brain regions to delivery of the juice reinforcer on standard trials, as well; and 4. Clinical ratings of avolition correlated significantly with brain activity evoked by standard juice delivery in the primary gustatory cortex and putamen. Each of these findings will be addressed in turn.

Contrasts between positive and negative TDEs

When combined across groups, neural activity distinguished between positive and negative temporal difference errors in multiple brain regions. Consistent with the results of McClure and colleagues (D’Ardenne et al 2008; McClure et al 2003), these areas included the midbrain and the left putamen. Both ventral and dorsal areas of the neostriatum have been identified as components of circuits for reward-based learning (Haber et al 2006; Voorn et al 2004), and a role for these areas in outcome processing, in particular, is supported by the results of several previous neuroimaging studies (Delgado et al 2005; Delgado et al 2000; McClure et al 2003). Our results provide further evidence that projections from the midbrain to the basal ganglia comprise a critical component of circuits for processing outcomes, and mismatches between expectations and outcomes.

Additionally, significant contrasts between unexpected juice delivery and omissions were observed, bilaterally, in the insula and the frontal and parietal operculum, consistent with the results of previous studies that have linked these areas to gustation (Lee et al 1998), general interoception (Craig 2003; Critchley et al 2004), and the processing of outcomes (Paulus et al 2005; Seymour et al 2004).

Group differences in responses to TDEs

Whole-brain analyses revealed that patients showed a reduced BOLD contrast between positive and negative TDEs in right putamen and the left frontal operculum. Additional ROI analyses revealed group differences in the [positive – negative] TDE contrast in the right midbrain, the left putamen, right primary gustatory cortex, and left cerebellum.

Evidence of abnormal prediction-error responses in schizophrenia suggests a possible factor underlying reinforcement learning deficits commonly observed in patients (Kemali et al 1987; Keri 2008; Schwartz et al 2003). Our finding of attenuated prediction-error-related activity in schizophrenia is consistent with the results of a recent neuroimaging study (Murray et al 2007), which found evidence of abnormal prediction error responses in the midbrain, among other brain regions, in SZ patients performing an operant learning paradigm. Several other studies have identified attenuated responses in the striatum in SZ patients (Kumari et al 2002; Reiss et al 2006), associated with the feedback-driven learning of procedures.

We observed group differences between patients and controls mainly for positive prediction errors (unexpected juice delivery), rather than negative prediction errors (unexpected juice omissions). This finding suggests that sensitivity to outcomes that are worse than expected (negative TDEs) might be relatively preserved in schizophrenia. The relatively intact response to negative TDEs in patients in the present study was somewhat unexpected, in light of the behavioral evidence pointing to impaired online error-correction behavior in schizophrenia (Prentice et al in press; Waltz and Gold 2007).

It is possible, however, that the type of the learning involved, specifically in terms of whether subjects need to explicitly represent feedback and use it to make rapid adjustments in behavior, may influence the extent to which brain signals associated with negative prediction errors are abnormal in schizophrenia. Despite relatively clear evidence of deficits in the ability to use feedback to make rapid adjustments in behavior, other behavioral results indicate that punishment-driven learning may be unimpaired in SZ patients if the learning task is of a probabilistic or procedural nature (Waltz et al 2007). The fact that the passive conditioning task in the current study has been shown to depend largely on striatal regions, implicated in procedural learning, might explain our finding of largely intact neural responses to negative TDEs in patients. It is possible, furthermore, that the negative TDE error signal is intact in schizophrenia, but that the ability of target brain areas to use error information to modulate response selection and learning is disrupted.

Responses to reinforcer deliveries on standard trials

Contrary to our expectations, patients also showed attenuated responses in numerous brain regions to delivery of the juice reinforcer on standard trials. Furthermore, we observed that neuronal activity evoked by juice delivery in the left primary gustatory cortex and bilateral putamen correlated significantly with clinical ratings of avolition. These findings, together with our observation that BOLD responses to expected and unexpected juice deliveries correlated significantly in three ROIs in the patient group, suggest that abnormal processing of the juice reinforcer was a contributing factor to the attenuated positive prediction error signals in SZ patients, and that reductions in goal-directed behavior frequently observed in schizophrenia patients may be influenced by abnormal processing of rewards.

Findings regarding the experience of rewards in schizophrenia are mixed. Evidence from behavioral studies (Cohen and Minor 2008; Gard et al 2007; Germans and Kring 2000) supports, in large part, the idea that patients with schizophrenia have intact self-reported experience of rewards. Indeed, patients in the present study reported finding the juice just as pleasant as controls did. These findings, however, appear to be contradicted by the results of multiple neuroimaging studies (Crespo-Facorro et al 2001; Paradiso et al 2003; Plailly et al 2006), which have found evidence of abnormal neural responses to pleasant stimuli in schizophrenia (especially primary reinforcers, such as pleasant odors). Furthermore, multiple studies have found evidence of reduced gray matter volume in components of reward processing circuitry, such as insula and ventral prefrontal cortex (Crespo-Facorro et al 2000; Davatzikos et al 2005).

One possible explanation for this apparent dichotomy is that, even if patients report normal hedonic experiences, the physiology underlying those hedonic experiences might still be abnormal. Several previous studies have found differential brain responses between SZ patients and controls, despite similar behavioral performance on learning tasks (Murray et al 2007; Zedkova et al 2006), as well as similar reports of emotional experience (Takahashi et al 2004). A disconnect between the reported experience of rewards, and the associated physiology, may partly explain the reduced ability of rewards and punishments to motivate behavior in schizophrenia. In short, patients who do not report a reduced experienced of pleasure might function like those who do, if the physiology underlying the experience of pleasure is abnormal. This view is supported by our finding that ratings of avolition in patients correlate most strongly with neural responses to the juice reinforcer, whereas avolition ratings bore no relationship to self-reports of reinforcer enjoyment.

It is possible that the strong response to juice delivery on standard trials, in components of reward circuits in controls, stemmed from the frequency of “catch” trials in our paradigm (roughly ¼ of total trials). A much lower proportion of nonstandard events has been used in some paradigms where infrequency was used to enhance the salience of an event (see, e.g., Zink et al 2006; Zink et al 2004). In our study, expectations may have changed across the course of the session, causing MRI responses to the reinforcer on standard and catch trials to become more similar.

Our observation of a significant correlation between timing task performance in patients, and neural responses to standard juice delivery in the left frontal operculum, suggests that it is also possible that group differences in interval timing abilities contributed to group differences in brain responses to TDEs. However, the fact that SZ patients showed largely intact responses to omissions of the reinforcer (negative TDEs) contradicts this, suggesting that patients had developed a relatively normal expectation as to the timing of standard juice deliveries. Thus, poor interval estimation is likely to make only a minor contribution to attenuated physiological responses to prediction errors in SZ patients.

Might the altered neurophysiological response to juice rewards reflect the impact of antipsychotic medications that attenuate dopamine signaling rather than an effect that can be considered to be a consequence of the illness? Previous studies have not produced a clear picture of what the impact of antipsychotic medications on reward processing might be. At least two studies using pharmacological challenge in normal volunteers have found that antipsychotic drugs modulate feedback-related brain activity (Abler et al 2007; Zirnheld et al 2004). On the other hand, results from studies of reward processing in patients with schizophrenia are mixed. Juckel and colleagues (2006a; 2006b) found that unmedicated SZ patients and patients on first-generation (typical) antipsychotics showed attenuated activity in ventral striatum, relative to controls. However, treatment with olanzapine appears to produce a relative normalization of reward anticipation BOLD response in the ventral striatum (relative to activity observed in patients treated with typical antipsychotics; Juckel et al 2006a; Schlagenhauf et al 2008). Thus, at least in the case of a symbolic reinforcer, some antipsychotic medications may actually enhance brain activity related to reward anticipation in SZ patients.

The design of the present study does not allow us to address this question in a straightforward fashion. In the absence of random assignment to drug type and dose, any post-hoc analysis of medication effects is confounded. That is, it is not possible to evaluate drug dose and type independently of the illness features and treatment history that led to the choice of that drug and dose. This may be a particular concern in the patient group studied here, where 1/3 of individuals were receiving clozapine – evidence of treatment resistance. Thus, in principle, our findings of abnormal reward processing physiology may be limited to treated patients. In order to answer the critical question of whether these abnormalities are intrinsic to the illness would require a very ambitious study design where patients are evaluated off medication and then again on medication, with the need to randomly assign drug type and dose in order to comprehensively address this issue.

The fact that our findings may only generalize to treated patients does not diminish the clinical importance of the results, because the great majority of patients with schizophrenia take antipsychotic medications. Such patients demonstrate altered reward processing, and the correlations between brain reward signal modulation and negative symptoms suggest that this physiological response is related to core features of the illness – features that are highly treatment resistant and are highly predictive of functional disability.