Learning with individual-interest outcomes in Autism Spectrum Disorder.

Recent work has suggested atypical neural reward responses in individuals with Autism Spectrum Disorder (ASD), particularly for social reinforcers. Less is known about neural responses to restricted interests and few studies have investigated response to rewards in a learning context. We investigated neurophysiological differences in reinforcement learning between adolescents with ASD and typically developing (TD) adolescents (27 ASD, 31 TD). FMRI was acquired during a learning task in which participants chose one of two doors to reveal an image outcome. Doors differed in their probability of showing liked and not-liked images, which were individualized for each participant. Participants chose the door paired with liked images, but not the door paired with not-liked images, significantly above chance and choice allocation did not differ between groups. Interestingly, participants with ASD made choices less consistent with their initial door preferences. We found a neural prediction-error response at the time of outcome in the ventromedial prefrontal and posterior cingulate cortices that did not differ between groups. Together, behavioural and neural findings suggest that learning with individual interest outcomes is not different between individuals with and without ASD, adding to our understanding of motivational aspects of ASD.

Introduction

The Social Motivation model of Autism Spectrum Disorder (ASD; Chevallier et al., 2012) proposes that social communication deficits and restricted and repetitive behaviours (American Psychiatric Association, 2013) arise from atypical functioning of motivational neurocircuitry. It has further been suggested that atypical reward processing in ASD is important to consider both for fundamental understanding of this condition and due to potential therapeutic implications (Dichter and Adolphs, 2012). Indeed, the modifier model of ASD proposes that some of the non-diagnostic symptoms that show inter-individual variability across individuals with ASD, such as motivation, may be important to consider precisely because of their importance to treatment response (Schiltz et al., 2018). Several recent studies have suggested atypical neural responses to social (Cox et al., 2015; Damiano et al., 2015; Kohls et al., 2013; Scott-Van Zeeland et al., 2010) and non-social (Damiano et al., 2015; Kohls et al., 2018, 2013; Schmitz et al., 2008) rewards, in ASD. Yet, few studies have investigated the neural response to rewards in ASD in the context of learning (Bellebaum et al., 2014; Schuetze et al., 2017; Solomon et al., 2011), despite the fact that learning to change behaviour is a core function of the brain’s reward and motivation system (Garrison et al., 2013; Liu et al., 2011).

In addition to building knowledge about the secondary features of ASD, studying reinforcement learning is important because it is a component of many behavioural therapies for ASD (Dawson and Burner, 2011; Virués-Ortega, 2010). Understanding the extent and nature of reward processing abnormalities, and associated differences in learning from rewards, is therefore highly clinically relevant. Indeed, the modifier model of ASD proposes that some of the non-diagnostic symptoms that show inter-individual variability across individuals with ASD, such as motivation, may be particularly important to consider precisely because of their relevance for treatment response (Schiltz et al., 2018).

The existing literature suggests that learning from feedback is not broadly impaired in ASD but may be atypical in specific contexts, for example depending upon reward type (Jones et al., 2013; Lin et al., 2012b; Scott-Van Zeeland et al., 2010), contingences (Minassian et al., 2007; Solomon et al., 2014, 2011), and requirements for flexibility (D’Cruz et al., 2013). A better understanding of when and why behavioural responses during learning are atypical may come from studying how the brain responds to reward feedback during learning.

Over the course of learning, feedback is used to update one’s estimate of the value of actions using a prediction error (PE) that reflects the difference between outcome and expectation (Rescorla and Wagner, 1972). Electrophysiology (Schultz et al., 1997) and human neuroimaging (Garrison et al., 2013) have implicated dopaminergic projections to the ventral striatum and medial prefrontal cortex in encoding a PE signal.

No studies to our knowledge have specifically examined PE signals during a reinforcement learning task in ASD, though atypical neural (Bellebaum et al., 2014; Cléry et al., 2013; Scott-Van Zeeland et al., 2010; Solomon et al., 2014) and physiological (Lawson et al., 2017) responses to feedback has been suggested in previous work. Hence, the objective of the study was to investigate whether PE signals in the context of learning from rewards are different in ASD participants compared to TD participants. Specifically, we focused on rewards that are related to participants’ likes and dislikes.

In the present study, we examined learning using a reinforcer that could be particularly motivating to individuals with ASD: images related to one’s own interests. A common symptom in ASD is circumscribed interests (CIs), or “intense preoccupations” (Smith et al., 2009) that are “abnormal in intensity or focus” (Turner-Brown et al., 2011). To investigate whether neural responses during learning are typical or atypical in ASD, we collected functional magnetic resonance imaging blood-oxygen-level dependent (fMRI BOLD) responses while participants with and without ASD performed a reinforcement learning task using individualized images related to their own interests as reinforcers. Based on previous findings of either typical (Dichter et al., 2012; Rivard et al., 2018) or enhanced (Cascio et al., 2014; Kohls et al., 2018) affective responses to special interest stimuli in ASD, we hypothesized similar or enhanced learning performance and PE neural responses in participants with ASD. Specifically, we hypothesized that ventro-medial prefrontal cortex (vmPFC; O’Doherty, 2004; Rushworth et al., 2011), posterior cingulate cortex

(pCC; Pearson et al., 2011) and nucleus accumbens (NAcc; Abler et al., 2006; Ernst et al., 2005; Hare et al., 2008) would show a PE response at the time of outcome that would be enhanced in youth with ASD. ROI analyses were focused on these regions as they have been frequently implicated in prediction error responses, including a previous study with a similar task design (Lin et al., 2012a). Importantly, we used images of items participants did not like as a control condition, and our hypotheses are therefore limited to an expectation regarding the difference between these two conditions.

We conducted this study in adolescents because they have the cognitive maturity to complete the task during fMRI and adolescents generally have heightened reward sensitivity (Foulkes and Blakemore, 2016; Galvan, 2010), which could provide interesting dynamic range for comparing between groups. This is also a group that is relatively understudied and underserved by community interventions. Ultimately our goal is to increase knowledge about potential strengths or deficits in reinforcement learning in ASD that could be useful in a therapeutic context.

Methods and materials

Participants

32 participants with ASD (four female; four left-handed, age range: 14–20) and 31 TD participants (eight female; two left-handed, age range: 14–20), were recruited from a larger online study assessing image preferences (Cho et al., 2017), i.e. all participants in this study also participated in the study of Cho et al. (2017) in order to assess their image preferences. Recruitment took place via advertisements and through service providers for ASD in Calgary, Alberta. Exclusion criteria were MRI contraindications, TD participants with a history of neurological or psychiatric disorders were also excluded. One TD participant was included who self-reported dyslexia. Seven participants with ASD reported one or more co-occurring diagnoses and six participants with ASD were taking one or more psychotropic medications (Table S1). All participants had normal or corrected-to-normal vision and no colour blindness.

All participants with ASD had a clinical diagnosis which was confirmed with the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2; Lord et al., 2012), administered by a research reliable rater. To assess extent of social symptoms, the Social Responsiveness Scale (SRS-2; Constantino and Gruber, 2005) was completed by parents. Four participants initially recruited for the ASD group did not meet clinical cut-offs and were excluded. One participant with ASD was excluded due to technical problems. This resulted in a final sample of 27 participants with ASD (four female, mean age: 16.4 years, SD: 2.1, age range: 14–20) and 31 TD participants (eight female, mean age: 16.5, SD: 2.1, age range: 14–20). Informed consent was obtained from participants over the age of 18 and assent with parental consent for participants younger than 18 years. Study procedures were approved by the University of Calgary Conjoint Health Research Ethics Board.

Cognitive assessment

General intelligence was assessed with the Wechsler Abbreviated Scale of Intelligence, Second Edition (WASI-II; Wechsler and Hsiao-pin, 2011). We used the Perceptual Reasoning Index (PRI) as a covariate in our analyses as it assesses nonverbal fluid abilities, which may more accurately capture intellectual functioning in ASD, relative to scores that include a verbal component (Mottron, 2004). Participants were not matched on full-scale IQ or PRI. Table 1 (Results section) describes participant characteristics.

Table 1

Participant characteristics for the whole and the fMRI samples. Except for participant numbers (N), means are shown with standard deviation in brackets. ADOS-2: Autism Diagnostic Observation Schedule, ADOS-2: Autism Diagnostic Observation Schedule Restrictive Repetitive Behaviours Subscale, F: female, FSIQ: Full-scale Intelligence Quotient, PRI: Perceptual Reasoning Index, SRS-2: Social Responsiveness Scale – 2nd Edition.

Whole Sample
	ASD	TD	Total
N (F)	27 (4)	31 (8)	58 (11)	χ2(1) = 0.15, p > 0.05
Age	16.4 (2.1)	16.5 (2.1)	16.5 (2.1)	t(56) = -0.19, p > 0.05
PRI	98.5 (17.2)	108.5 (12.9)	103.7 (15.8)	t(56) = -2.37, p < 0.001
FSIQ	90.5 (16.8)	109.5 (11.8)	100.5 (17.2)	t(56) = -4.73, p < 0.001
SRS-2	75 (8.2)	44.2 (6.4)	58.2 (17)	t(56) = 12.2, p < 0.001
ADOS-2	13.7 (4.2)
ADOS-2 RRB	3.4 (1.6)

Stimuli

FMRI data were collected while participants completed a reinforcement learning task that included individualized liked and not-liked images and a common set of ‘noise’ images as outcomes.

In an online survey (Cho et al., 2017), all participants of the current study were asked to list items they liked and did not like. As individuals were scanned for the fMRI study between 1 week and 12 months after the online survey participation, parents were asked during an intake interview to consult with their children and provide additional likes and dislikes. This was done to confirm participants’ survey responses and to capture the most accurate interests at the time of the study. We then used all depictable items from parents’ and children’s lists (Table S2) and conducted an online image search to create individualized stimuli sets for each participant. On average, participants saw 43 unique images they liked and 38 images they didn’t like, which did not differ between groups (F(1,56) = 2.9, p > 0.05). All participants saw at least one social image (i.e. depicting people) in their likes and dislikes categories. Due to true randomisation of the stimuli, seven participants (three ASD) saw one or more liked images twice and 10 participants (five ASD) saw one or more not-liked images twice. Six noise images chosen from an online image search depicted random patterns in different colours (red, yellow, light green, dark green, blue). All images were resized to 600 × 400 pixels and did not include any written text. Fig. 1 shows example stimuli.

Fig. 1

Example stimuli for one study participant. The top rows depict liked, the middle rows not-liked and the bottom rows noise stimuli. The same six noise images were used for all participants.

Image validation

We ensured that participants preferred images in their liked over not-liked set by asking them to rate liked and not-liked images randomly chosen from their set on a Likert-scale from 1 (lowest) to 10 (highest). In the interest of time, and because we did not plan to assess or remove individual images from the analysis, we chose to have them rate a subset of 10 images of each category rather than the entire set. As some participants indicated difficulty in assigning numeric values to the liking of images, partway through data collection we added a pairwise preference task, that allowed us to collect information about relative ranking of images. This image comparison task was completed in a subset of participants (12 ASD, 19 TD). For this task, participants were instructed to press a button on the left or right to indicate the image they preferred. On each trial, two unique images from their set were presented on the screen (18 liked vs. not-liked image pairs, 12 liked vs. noise image pairs and 12 not-liked vs. noise image pairs). Image preferences were analysed by calculating choice proportions for liked > not-liked, liked > noise and noise > not-liked. These proportions were compared against 50% chance using one-sample t-tests and between groups using independent samples t-tests. Statistical analyses of behavioural measures were conducted using IBM SPSS Statistics 24.

Experimental task

Our task was adapted from multi-armed bandit tasks commonly used in human (Lie et al., 2009) and animal (Morris et al., 2006; Rothenhoefer et al., 2017) research investigating behavioural and neural responses to probabilistic reward feedback. In particular, our task was adapted from Lin et al. (2012b, 2012a) by presenting coloured doors instead of slot machines to make it child friendly, and by adapting the outcomes to be interest-related images. The task design is illustrated in Fig. 2. Each trial started with a black fixation cross presented for a random duration drawn from a uniform distribution between one and three seconds. Next, two doors of different colours were presented side by side for two seconds. Participants were told to “Choose any door you like to look at a picture behind it” by pressing the right or left button on a two-button Lumina LU-400 response pad. The task is an implicit learning task as participants are not given any instructions about strategy. Their choice was reflected via a black outline around the chosen door, followed by a black fixation cross presented for a random duration drawn from a uniform distribution between one and three seconds. A liked, not-liked, or noise picture was then shown for two seconds. In the rare cases of missed responses (mean total = 2.03, SD = 2.9, mean ASD = 2.5, SD = 3.6, mean TD = 1.1, SD = 1.6; t(30) = 1.7, p > 0.05), no outcome picture was shown and the experimenter reminded participants to choose one of the doors via an intercom. Each trial took between eight and 12 s and participants completed 120 trials over two sessions of 60 trials (˜20 min). Inspection of choice behaviour showed that a subset of participants did not vary their responses across the task (3 ASD, 2TD), that is they chose a specific door (1 LPD, 4 NLPD) 100% of the time. As these participants appeared to approach the task differently from the rest of the sample, all analyses were repeated with and without these ‘perseverating’ participants.

Fig. 2

Task Design. Each trial started with a fixation cross at the centre of the screen for 1–3 seconds followed by the presentation of two differently coloured doors. Two seconds were provided for participants to choose one of these doors and their choice was indicated immediately by a black frame around the chosen door (not shown). Outcome images varied based on door contingencies (Table 2) and could be a liked image, a not-liked image or a noise image. Images were presented for two seconds.

Table 2

Door contingencies. Each door was associated with a specific outcome probability, with the exception of the grey door that was not selectable (LPD = Liked paired door, NLPD = Not-liked paired door).

LPD	NLPD	Neutral door	Noise door
80% liked image	20% liked image	33.3% liked image
20% not-liked image	80% not-liked image	33.3% not-liked image
		33.3% noise image	100% noise image

The task included five different doors (red, yellow, green, blue, and grey). Similar to Lin et al. (2012a), each of the four coloured doors was associated with specific outcome probabilities randomized across participants (Table 2).

The grey door was not selectable and was used to create forced-choice trials that ensured all participants experienced outcomes associated with each door. 40 forced-choice trials showed the grey door paired with the liked paired (LPD), not-liked paired (NLPD), neutral and noise door (10 trials each). 80 free-choice trials paired LPD or NLPD doors with the noise or neutral doors 20 times for each combination (Table 3). Door contingencies were chosen so that: 1) choice behaviour of the liked paired and not-liked paired doors could be examined independently of one another and 2) to maximize PE signals, for example, if a liked image is expected and a not-liked image is shown instead we assumed this would generate a larger PE signal than a noise image. The choice of trial number (40 per condition) was a trade-off between scan time and power. Presentation of each door was counterbalanced between the right and left sides of the screen. Before the scan, participants practiced a shorter form of the task that did not include any of the doors or images that were used during the actual task.

Table 3

Trials per condition. In free-choice trials, the liked paired doors (LPD) and not-liked paired doors (NLPD) were presented with the neutral and the noise door. In forced-choice trials, each door was presented together with the grey door to ensure some experience with each door.

# Trials	Door Pairing	Trial Type
40	LPD with Noise (20x)LPD with Neutral (20x)	Free-Choice
40	NLPD with Noise (20x)NLPD with Neutral (20x)
40	LPD with Grey (10x)NLPD with Grey (10x)Neutral with Grey (10x)Noise with Grey (10x)	Forced-Choice
Total = 120

Door rating

Before and after the ˜1 -h scan session, participants rated the coloured doors. While choice behaviour was our primary measure of learning in this study, a change in their ratings could serve as a secondary measure of how the value of the doors changed during the task. Participants were presented each door separately on the screen and asked to rate how much they liked the door on a Likert scale from 1 (lowest) to 10 (highest). Due to the experiment taking longer than scheduled for five participants (two ASD), they did not complete these ratings and therefore were excluded from analyses of these measures. To confirm that there was not a systematic preference for a certain door type before learning, we analysed the pre-ratings in a repeated measures ANOVA that included door type (four levels: liked paired, not-liked paired, neutral, noise) as a within-subjects factor and group as a between-subjects factor. To assess a change in ratings, we calculated the differences between pre- and post-ratings and entered these into a repeated measures ANOVA with door (four levels: liked paired, not-liked paired, neutral, noise) as a within-subjects factor and group as a between-subjects factor. We also assessed whether changes in ratings were significantly different from zero using one-sample t-tests.

Response time

In order to assess whether groups differed in their time to make decisions, we analysed response times in a repeated measures ANOVA that included trial type with two levels (free and forced choice trials) as a within-subjects factor and group as a between-subjects factor. We further assessed the effects of door type on response time by conducting two repeated measures ANOVAs using the forced choice trials and the free choice, separately. These analyses included door type with two levels (LPD and NLPD) as a within-subjects factor and group as a between subjects factor.

Choice behaviour

Our primary measure of learning was participants’ choice behaviour calculated as the number of times a door was chosen divided by the total number of choice trials during which that door was presented (excluding missed trials). We entered the values of our contrast of interest in a repeated measures ANOVA that included door type (LPD and NLPD) as a within-subjects factor and group as a between-subjects factor. We further binned trials into three sections (˜27 trials each) and entered the choice proportion for each section in a repeated measures ANOVA that included time (early, middle, late) as within-subjects factor and group as a between-subjects factor.

Since individuals with ASD may use different strategies during tasks that include probabilistic feedback (Solomon et al., 2014), we also investigated a measure of strategy. Specifically, we analysed participants’ choice behaviours based on previous outcomes of each door, i.e. propensity to choose a door again after it was followed by a liked image (“win-stay”) or choose a door again after a not-liked image (“lose-stay”). We calculated these scores as the number of times a door was chosen after it was associated with a liked or not-liked image in a previous trial divided by the total number of times that same door was available. These scores were entered in a repeated measures ANOVA that included two levels (Chose-Following-Liked-Outcome (i.e., win-stay), Chose-Following-Not-Liked-Outcome (i.e., lose-stay)) as a within-subjects factor and group as a between-subjects factor.

We also asked whether participants’ initial rating of a door was related to their tendency to choose that door in the learning task using a linear mixed model with mean-centred pre-ratings and group as fixed effects, participant as a grouping variable, and choice proportion as the outcome. To see whether choice association with pre-ratings varied across time, we ran a linear mixed model in each of the three bins.

We assessed whether learning and strategy were associated with intellectual functioning (PRI scores) and ASD symptoms (SRS-2 scores) using Pearson correlations. We further assessed relations with ADOS-2 scores within the ASD group.

Neuroimaging acquisition

MRI data were acquired on a 3 T GE MR750w (Waukesha, WI) scanner at the Alberta Children's Hospital using a 32-channel head coil. T2*-weighted gradient-echo echo-planar images (EPI) were acquired for each participant (Flip angle: 70 degrees, FOV: 22.0, TR: 2500 ms, TE: 30 ms, voxel size: 3.4 × 3.4 x 3.5 mm). To improve signal from the orbitofrontal cortex (OFC) and ventromedial prefrontal cortex (vmPFC), oblique axial slices were positioned +30 degrees relative to the anterior-posterior commissure line as described in (Deichmann et al., 2003). A whole-brain high-resolution T1-weighted MPRAGE image was acquired for each participant (Flip angle: 10 degrees, FOV: 24.0 cm, TI: 600, voxel size: 0.8 × 0.8 x 0.8 mm) while participants watched a video of their choice.

Neuroimaging preprocessing

We used the SPM12 (Wellcome Trust Centre for Neuroimaging, London, UK) toolbox in MATLAB (R2014b, MathWorks, Natick, MA, USA) for fMRI data preprocessing. Structural images were manually aligned to the anterior-posterior commissure (AC-PC) line before segmentation into grey matter, white matter and cerebrospinal fluid. Functional images were corrected for slice timing and realigned to the first functional volume acquired in the first run. Functional images were co-registered to the T1 structural image, normalized to Montreal Neurological Institute (MNI) space using parameters derived from the T1 segmentation and spatially smoothed with a Gaussian full-width-at-half-maximum 8 mm filter. The ART toolbox (Artifact Detection Tool, http://www.nitrc.org/projects/artifact_detect/) was used to identify fMRI volumes that exceeded a scan-to-scan motion threshold of 0.9 mm. These volumes were censored in functional analyses (3.6% of scans in total, 4.1% in the ASD group, 2.5% in the TD group, not significantly different between groups). Participant’s total number of censored scans was added as a covariate to the final model, to control for inter-individual variability in head motion. For three participants, more than 30% of volumes exceeded this threshold and were excluded from this analysis (two ASD). Three masks were created for our a priori regions of interest (ROI) using atlases within the WFU Pickatlas toolbox, version 3.0.5 (Maldjian et al., 2003) to perform small volume corrections. The vmPFC mask was created combining right and left frontal-med-orb masks from the aal atlas (Tzourio-Mazoyer et al., 2002), the pCC mask was created using the posterior cingulate mask from the TD labels atlas, the NAcc was created combining right and left nucleus accumbens masks from the IBASPM 71.

Neuroimaging analysis

FMRI data were analysed using a General Linear Model in SPM12. First level models included 7 regressors of interest (plus three modulators, Table 4). Two regressors modelled the start of free-choice trials, separately for trials including the LPD and NLPD. A third regressor modelled the start of all forced-choice trials. Four regressors modelled the time of outcome after the liked-paired, not liked-paired, neutral, and noise doors were chosen. Three of them (liked, not-liked and neutral) included a parametric modulator to reflect their PE; no PE was modelled for the noise door. The PE was calculated as the difference between reward magnitude (RM) of the presented outcome and the stimulus value (SV) of the chosen door.

Table 4

First-level model regressors and parametric modulators. Different reward magnitude (RM) for modulators was chosen to reflect that the not-liked outcome was on average less preferred than the noise outcome.

Regressor	Prediction Error Modulator
Onset of LPD during free-choice trials	n.a.
Onset of NLPD during free-choice trials	n.a.
Onset of all doors during forced-choice trials	n.a.
Onset of outcome images; LPD chosen	RM of liked outcomes = +1RM of not-liked outcomes = -1RM of noise outcomes = 0
Onset of outcome images; NLPD chosen	RM of liked outcomes = -1RM of not-liked outcomes = +1RM of noise outcomes = 0
Onset of outcome images; neutral door chosen	RM of liked outcomes = +1RM of not-liked outcomes = -1RM of noise outcomes = 0
Onset of outcome images; noise door chosen	n.a.

Similar to previous fMRI studies on reinforcement learning (Lin et al., 2012a), we have used the participants’ choice behaviour to infer the stimulus value as a time varying predictor based on the current value state of the door. Specifically, the SV of each door was calculated based on choice allocation to that door using a moving average of five trials (chosen to estimate the SV locally while mitigating the influence of individual choices). The SV ranged from 0 to 1 (0 = the given door was never chosen within the last five trials, 1 = the given door was chosen each time within the last five trials) and were adjusted to range from -1 to +1 to match the RM. Similar to previous fMRI studies on reinforcement learning (Lin, Adolphs, et al., 2012), we have used the participants’ choice behaviour to infer the stimulus value as a time varying predictor based on the current value state of the door.

All events were modelled as delta functions convolved with the canonical hemodynamic response function. The six motion realignment parameters, plus one composite displacement, as well as censored volumes, were added as nuisance regressors. Models were estimated with a high-pass filter of 128 s. At the first level, we contrasted the PE of the LPD and NLPD. This contrast was entered into a second-level analysis using one-sample t-tests to investigate activity across and within groups, and a two-sample t-test to compare ASD and TD groups. We included age, sex and censored volumes as covariates in the second-level analysis and repeated the analysis with handedness and PRI as additional covariates. First, we examined responses in our three a priori ROI (vmPFC, pCC and NAcc). Inferences were drawn using a height threshold of p < 0.001 with family wise error (FWE) correction p < 0.05 at the cluster-level. Whole-brain exploratory analyses were also run, with inference at p < 0.001 height threshold and p < 0.05 FWE corrected over the whole brain.

As previous work has suggested an association between learning and neural response to rewards (O’Doherty, 2004; Schönberg et al., 2007), we also assessed the relations between choice allocation and PE-BOLD response. To do this, we averaged beta values over voxels in the clusters that were significant in the whole-group PE-Liked > PE-Not-liked contrast and Pearson correlated the resulting values with the LPD choice proportion. As affective neural responses have been associated with ASD symptom severity (Kohls et al., 2018), we also correlated SRS-2 Total and SRS-2 Restricted, Repetitive Behaviours (RRB) scores with these same beta values. Further, we correlated beta values from significant clusters in the ASD group with ADOS-2 Total and ADOS-2 RRB scores. For behavioural correlations, significance was set at p < 0.05 after Bonferroni correcting for the number of clusters.

Results

Participant characteristics

Participant groups did not significantly differ in age (t(56) = -0.19, p > 0.05) or sex (χ2(1) = 0.15, p > 0.05). Participants with ASD scored significantly lower on PRI (t(56) = -2.37, p = 0.021, d = -0.63, 95% CI [-17.3 -1.5]) and FSIQ (t(56) = -4.725, p = < 0.001, d = -1.26, 95% CI [-25.8 -10.4]) of the WASI-II and significantly higher on SRS-2 Total (t(56) = 12.2, p < 0.001, d = 3.3, 95% CI [26.8 34.4]) compared to TD participants.

Individualized image sets

An overview of all liked and not-liked items reported and used in this study is listed in Supplementary Table S2. Items could be roughly categorized into: Animals, Activities, Entertainment (e.g. books, movies, TV shows, and games), Food, People and Other. 17 liked items and 13 not-liked items overlapped between groups (e.g., both a TD and ASD participant liked skiing and did not like cockroaches). 24 items were mentioned as both liked and not-liked (e.g., one participant liked moths while another did not). As interests varied within and between participants, we could not compare broader image categories, e.g. social vs. non-social between groups. Importantly, participants’ items always included both social and non-social images in both categories (liked and not-liked) such that an influence of the social aspect of images is rather unlikely. We show more detailed image categories in Table S2.

Image ratings showed a significant main effect of image type (F(1,42) = 104.9, p < 0.0001, ηp2 = 0.71, 95% CI [2.9 4.3]): as expected participants rated liked images (mean total = 7.1, SE = 0.3, mean ASD = 7.1, SE = 0.44, mean TD = 7.1, SE = 0.35) above not-liked images (mean total = 3.5, SE = 0.3, mean ASD = 3.51, SE = 0.5, mean TD = 3.41, SE = 0.25; Fig. 3). There was no significant effect of group (F(1,42) = 0.009, p > 0.05) or interaction between group and image type (F(1,42) = 0.02, p > 0.05).

Fig. 3

Image Validation. Mean and standard error in Likert-ratings for each participant. Participants below the diagonal line rated their liked images higher than their not-liked images.

A subset of participants also indicated pairwise preferences. A set of one-sample t-tests for image pairs (liked > not-liked, liked > noise, noise > not-liked) showed that liked images were preferred significantly above chance when they were presented with either not-liked (t(30) = 69.8, p < 0.0001, d = 25.5, 95% CI [91.9 97.4]) or noise images (t(30) = 26.9, p < 0.0001, d = 9.8, 95% CI [85.2 99.2]), and noise images were chosen significantly above chance when they were presented with not-liked images (t(30) = 10.8, p = 0.021, d = 3.9, 95% CI [51.9 76.1]). Groups did not differ for any pairings (liked > not-liked: t(29) = 0.12, p > 0.05, d = 0,045; mean total = 95.2, SE = 1.4, mean ASD = 94.9, SE = 3.0, mean TD = 95.0, SE = 1.4; liked > noise: t(29) = 0.39, 0,145; p > 0.05; mean total = 92.7, SE = 3.4, mean ASD = 93.9, SE = 5.3, mean TD = 91.7, SE = 4.8; noise > not-liked: t(29) = 1.6, p > 0.05, d = 0,594; mean total = 64.5, SE = 5.9, mean ASD = 82.6, SE = 5.9, mean TD = 57.0, SE = 7.9).

Door ratings

Participant ratings of the doors before the task did not significantly differentiate between the doors based on their assignment, or by group (LPD: mean total = 6.0, SE = 0.3; mean ASD = 6.1, SE = 0.4; mean TD = 5.8, SE = 0.4; NLPD: mean total = 6.3, SE = 0.3; mean ASD = 6.5, SE = 0.42; mean TD = 6.0, SE = 0.4; neutral door: mean total = 6.0, SE = 0.3; mean ASD = 6.2, SE = 0.5; mean TD = 5.8, SE = 0.4; noise door: mean total = 5.8, SE = 0.3; mean ASD = 5.7, SE = 0.43; mean TD = 5.8 (SE = 0.41). These pre-ratings showed no significant effect of door type (i.e., LPD, NLPD, neutral or noise) (F(3,153) = 0.63, p > 0.05), no significant effect of group (F(1,51) = 0.4, p > 0.05) or interaction between group and door type (F(3,153) = 0.2, p > 0.05). Participants liked the yellow door less than the other colours (F(3,153) = 15.8, p < 0.001; mean total yellow door = 4.9, SE = 0.3; mean ASD = 4.8, SE = 0.4; mean TD = 4.9, SE = 0.4). The change in ratings from pre- to post-task was not significantly different from zero for any door (LPD: t(52) = 1.36, p > 0.05; NLPD: t(52) = -1.57, p > 0.05; neutral: t(52) = -0.21, p > 0.05; noise: t(52) = 0.65, p > 0.05). Moreover the change in ratings did not significantly depend on door type (F(3, 153) = 1.37, p > 0.05) and this was not mediated by group (main effect: F(1,51) = 0.2, p > 0.05) or group by door interaction (F(3, 153) = 0.23, p > 0.05).

Participants responded significantly faster (F(1,56) = 126.24, p < 0.0001, ηp2 = 0.7, 95% CI [0.17 0.24]) to forced trials (mean total =670 ms, SD =111 ms; mean ASD =670 ms, SD = 113; mean TD =669 ms, SD = 112) compared to free choice trials (mean total =876 ms, SD =207 ms; mean ASD =880 ms, SD = 212; mean TD =871 ms, SD = 207). There was no significant effect of group (F(1,56) = 0.015, p > 0.05) or interaction between group and trial type (F(1,56) = 0.042, p > 0.05). For both forced-choice and free-choice trials, reaction time was not affected by door type (forced: (F(1,56) = 0.049, p > 0.05; free: F(1,56) = 0.014, p > 0.05), with no main effect of group (F(1,56) = 0.008, p > 0.05; F(1,56) = 0.146, p > 0.05) or interaction between group and door type (F(1,56) = 0.129, p > 0.05; F(1,56) = 0.03, p > 0.05).

Learning task choice behaviour

Participants chose the LPD significantly above 50% chance (t(57) = 3.12, p = 0.003, d = 0.8; mean total = 58.5%, SD = 20.7; mean ASD = 56.8%, SE = 4.0; mean TD = 59.9%, SE = 3.7), whereas the NLPD was chosen at chance level (t(57) = 0.6, p > 0.05, d = 0.2; mean total = 51.7%, SD = 21.8, mean ASD = 53.2%, SE = 4.2; mean TD = 50.4%, SE = 3.9; Fig. 4). A post-hoc one-sample t-test against chance level for each group separately showed that TD participants chose the LPD significantly above chance (t(30) = 2.51, p = 0.018, d = 0.45; mean = 59.9%, SD = 22.1) but not ASD participants (t(26) = 01.83, p > 0.05, d = 0.35; mean = 56.8%, SD = 19.4). A follow-up analysis excluding participants who perseverated in responding (3 ASD, 2 TD) showed significantly above chance selection of LPD but not NLPD in both groups (ASD LPD: t(23) = 2.05, p = 0.05, d = 0.42; mean = 56.9%, SD = 16.4; ASD NLPD: t(23) = -0.82, p > 0.05, d = -0.17, mean = 47.4%, SD = 15.3; TD LPD: t(28) = 2.01, p = 0.05, d = 0.37; mean = 57.9%, SD = 21.1; TD NLPD: t(28) = -0.85, p > 0.05, d = -0.16, mean = 47.1%, SD = 18.3).

Fig. 4

Participants’ choice for the liked paired (LPD) and not-liked paired door (NLPD). Coloured bars indicate group means.

A repeated measures ANOVA for choice behaviour with door type (LPD, NLPD) as a within-subject factor showed a trend-level effect of door type (F(1,56) = 3.7, p = 0.059, ηp2 = 0.1, 95% CI [-0.003 0.134]) with no main effect of, or interaction with, group (F(1,56) = 0.002, p > 0.05; F(1,56) = 0.74, p > 0.05). A post-hoc repeated measures ANOVA for choice behaviour with door type (LPD, NLPD) as a within-subject factor for each group individually showed a main effect of choice in the TD but not in the ASD group: TD participants chose the LPD significantly over the NLPD door (F(1,30) = 4.26, p = 0.048, ηp2 = 0.12, 95% CI [-0.001 0.189]). After removing perseverating participants we found a main effect of door type in both groups (ASD: F(1,23) = 7.15, p = 0.014, ηp2 = 0.24, 95% CI [0.021 0.167]), TD: (F(1,28) = 5.04, p = 0.03, ηp2 = 0.15, 95% CI [0.009 0.206]). A repeated measures ANOVA for choice behaviour with time (early, middle, late) as a withinsubject factor did not show a significant effect of time (F(1,55) = 0.07, p > 0.05), group (F(1,55) = 0.25, p > 0.05), or interaction between time and group (F(1,55) = 0.18, p > 0.05), suggesting that choice behaviour was relatively stable from the first 40 trials.

In additional analyses assessing the effect of age, a post-hoc Pearson correlation showed that age was not significantly correlated with choosing the LPD (r = 0.154, p > 0.05) or the NLPD (r = -0.108, p > 0.05). Further, in a post-hoc repeated measures ANOVA for choice behaviour with door type (LPD, NLPD) as a within-subject factor and age as a between-subject factor showed no effect of age (F(6,56) = 1.43, p > 0.05), or interaction between choice and group (F(6,56) = 0.914, p > 0.05).

Analysis of choice strategy showed a significant effect of previous outcome on door choice (F(1,56) = 6.01, p = 0.017, ηp2 = 0.1, 95% CI [0.01 0.09]); participants were more likely to choose a door if it had been paired with a liked outcome on the previous trial (mean: 55.6%, SD: 14.0) relative to a not-liked outcome (mean: 50.0%, SD: 10.7). There was no main effect of group (F(1,56) = 0.028, p > 0.05) or interaction between choice and group (F(1,56) = 0.47, p > 0.05).

Choice behaviour was not correlated with PRI (LPD: r = 0.023, p > 0.05, NLPD: r = 0.098, p > 0.05). Likewise, choice strategy was not correlated with PRI (chose follow-liked: r = 0.2, p > 0.05; chose follow-not-liked: r = 0.03, p > 0.05).

SRS-2 Total scores were not correlated with either of these learning measures (LPD: r = -0.002, p > 0.05; NLPD: -0.047, p > 0.05; chose follow-liked: r = -0.023; p > 0.05; chose follow-not-liked: r = 0.01, p > 0.05). Within the ASD sample, total ADOS-2 scores were not correlated with learning or strategy (LPD: r = -0.14, p > 0.05; NLPD: -0.19, p > 0.05; chose follow-liked: r = -0.3, p > 0.05; chose follow-not-liked: r = -0.12, p > 0.05).

We assessed the association between baseline ratings of the doors and subsequent choice, and found a significant main effect of pre-ratings (F(1,208) = 44.7, p < 0.001, 95% CI [0.04 0.09], β = 0.065) and a significant interaction between group and pre-ratings (F(1,208) = 8.0, p = 0.005, 95% CI [-0.07 -0.01], β = -0.039). Participants overall were more likely to choose a door they gave an initially high rating to, although participants with ASD were less likely to base their choice on their initial ratings of a door (Fig. 5).

Fig. 5

Influence of pre-ratings of a door on subsequent choice proportion of that door (ASD β = 0.026, TD β = 0.038).

Within each bin (early, middle, late) we found a significant main effect of pre-ratings (early: F(1,208) = 10.7, p = 0.001; middle: F(1,208) = 9.2, p = 0.003; late: F(1,208) = 7.0, p = 0.009). However, significant interactions between group and pre-ratings were not found early in the task (F(1,208) = 1.6, p > 0.05), appearing only in the middle and late segments (F(1,208) = 4.1, p = 0.04; F(1,208) = 4.8, p = 0.03). In other words, both participant groups started the task by choosing doors according to initial liking; however, participants with ASD were less influenced by their initial door ratings at baseline.

Likely due to the index finger resting on the left and the middle finger on the right button, participants chose the left door significantly more often than the right door (mean total left door = 51.4%, SD = 4.8; mean total right door = 48.6%, SD = 4.8; F(1,56) = 4.6, p = 0.036, ηp2 = 0.07 [0.002 0.053]). Importantly, there was no main effect of group or interaction of group and side of door (F(1,56) = 0.108, p > 0.05, ηp2 = 0.002).

Neuroimaging results

Using small volume correction on our a priori defined ROI (vmPFC, pCC, NAcc) for the contrast of liked vs. not-liked PE, we found significant activity in the pCC ([-6, -52, 16], p = 0.001, z = 4.51, kE = 316) and in the vmPFC ([-10, 48, -12], p < 0.0001, z = 4.73, kE = 417) for the combined group. In the TD group, we found significant activity in the pCC ([-6 -52 18], p = 0.001, z = 4.54, kE = 300) and vmPFC ([-4, 60, -8], p = 0.001, z = 4.31, kE = 247). In the ASD group, we found significant activity only in the vmPFC ([-10, 44, -14], p = 0.024, z = 4.34, kE = 42). No significant group differences were found in any ROI. No significant activations were found in the NAcc; however, we note that several participants had substantial signal dropout in this region.

Using whole-brain correction, in the combined sample (Fig. 6a) for the contrast of liked vs. not-liked PE, we found significant activity in the vmPFC ([-6, 44, -18], p < 0.0001, z = 4.97, kE = 923), the pCC ([-6, -52, 16], p = 0.009, z = 4.51, kE = 414), the left parahippocampal gyrus (lphg, [-30, -18, -16], p < 0.0001, z = 4.79, kE = 765), the right parahippocampal gyrus (rphg, [38, -40, -26], p = 0.005, z = 4.06, kE = 470) and the left medial temporal gyrus (lmtg, [-64, -10, -16], p = 0.008, z = 4.43, kE = 427). In the TD group (Fig. 6b), we saw significant activity in the vmPFC ([-4, 60, -8], p = 0.012, z = 4.31, kE = 389), pCC ([-6, -52, 18], p < 0.013, z = 4.54, kE = 386) and lmtg ([-60, -12, -18], p = 0.007, z = 5.41, kE = 436) as well as trend-level activity in the lphg ([-26 -28 -22], p = 0.057, z = 4.44, kE = 200). In the ASD group (Fig. 6c), we saw significant activity only in the vmPFC ([-14, 44, -14], p = 0.009, z = 4.73, kE = 413). No significant differences were found between the ASD and TD groups. Results did not change when age was added as a covariate and when perseverating participants were taken out of the analysis.

Fig. 6

Significant clusters in response to PE. Significance maps show the PE contrast for LPD versus NLPD in the combined groups (a), the TD group (b) and the ASD group (c). Images are thresholded at p < 0.001 uncorrected. Clusters surviving multiple comparison correction are described in the text. ASD = Autism Spectrum Disorder, lmtg = left medial temporal gyrus, LPD = liked paired doors, lphg = left parahippocampal gyrus, NLPD = Not-liked paired doors, pCC = posterior Cingulate Cortex, rphg = right parahippocampal gyrus, TD = typically developing.

Correlational analyses of extracted beta values from the peaks of all five significant clusters (vmPFC, pCC, lphg, rphg, lmtg) across the whole sample showed that none of the clusters correlated with learning (LPD choice %), social functioning (SRS-2 Total) or restricted and repetitive behaviours (SRS-2-RRB; all uncorrected p-values > 0.05). Within the ASD sample, none of the clusters correlated with total ADOS-2 or ADOS-2 RRB scores.

Discussion

In this study, we investigated behavioural and neural responses during a reinforcement learning task in adolescents with ASD relative to TD adolescents, using individualized images as outcomes. Participants learned to choose the liked paired, but not the not-liked paired, door above chance, and choice allocation did not differ between groups. Despite this, participants with ASD were more likely to make choices that did not concur with their initial ‘liking’ of the doors, suggesting a greater degree of flexibility, or sensitivity, in response to feedback. Greater sensitivity is particularly interesting given previous research suggesting increased value of circumscribed interests in ASD possibly making them stronger reinforcers during a learning task (Turner-Brown et al., 2011). Importantly, learning performance was not influenced by age.

In terms of BOLD response during learning, ROI as well as whole brain analyses showed significant activity during choice feedback in the pCC and vmPFC for the combined sample as well as the TD group. The ASD group showed significant activity only in the vmPFC. Furthermore, we found no group differences in vmPFC or PCC during choice feedback. In addition to more classic PE regions, we also saw activity in bilateral parahippocampal gyri which is in line with a recent study that found increased hippocampal activation during PE signals when comparing positive versus negative outcomes in adolescents but not adults (Davidow et al., 2016). These findings are interesting in a developmental context as it suggests categorical differences of the PE signal which is supported by the lack of a linear age effect in our study. Future research needs to explore the effects of age in a larger sample suited to investigate linear changes or differences between different age groups.

Together our results show largely intact behavioural and neural responses in ASD in a learning task when individualized interests images are used as reinforcers. Typical learning and reward responses towards interest images suggest that differences in the brain’s reward and motivation system in ASD may not be general to all learning contexts and types of reinforcers. This has implications for motivation-based theories of ASD, suggesting that motivational differences may best be understood as relatively specific.

In previous work, reinforcement learning in ASD has been described as slower (Solomon et al., 2011), more variable (Lin et al., 2012b; Yechiam et al., 2010), and less flexible (Solomon et al., 2015) relative to TD controls. Importantly though, not all studies have shown impaired learning performance in ASD (Barnes et al., 2008; Brown et al., 2010; Dichter et al., 2012; Scott-Van Zeeland et al., 2010; Solomon et al., 2011), and several factors appear to be influential such as reward contingencies (Minassian et al., 2007; Solomon et al., 2011) and the type of reward (e.g. social or non-social; (Dichter et al., 2012; Lin et al., 2012b; Scott-Van Zeeland et al., 2010). Although we did not assess learning under different contingencies or with social reinforcers, our study adds to this literature by showing that learning was not different between groups with relatively clear contingencies and reinforcers related to individuals’ interests. However, it is important to note that the effect sizes for the learning measure were small with our task and while our number of trials has shown within-group activation in previous work (Lin et al., 2012a) it may not have been sufficient to detect between-group effects. Further, a task that makes use of clinically assessed CIs might also engage learning in the ASD group to a greater extent and potentially bring out group differences not seen in this study. Yet, we also found that participants with ASD were more likely to make choices that diverged from their initial preferences. These findings suggest that interest-related reinforcers (not necessarily clinically defined CIs) may be particularly useful in overriding prepotent behaviours in this group.

Several recent studies have used neuroimaging to explore the possibility that CI symptoms may be related to hyper-responding in the brain’s reward system towards CI-related stimuli (Benning et al., 2016; Cascio et al., 2014; Kohls et al., 2018). Our study and two others (Rivard et al., 2018; Sasson et al., 2012) have instead found that affective responses in ASD are not significantly different towards interest-related stimuli compared to a TD control group, when contrasted with own interests as well as more general ‘frequently used ASD interests’ (Sasson et al., 2012). However, as we used interest-related images broadly defined and not all images were related to clinical-level CI symptoms, more refined experimental paradigms are needed to investigate affective responses to CIs. Furthermore, since participants did not rate each individual image stimuli and because there was a range in ratings of liked and not-liked images, it is unclear whether all of their images captured their true interests correctly which could have weakened possible effects. However, we have shown that on a randomly chosen subset of images the liked images were on average preferred over the non-liked. We also note that here and in the above studies, the TD and ASD groups did not differ on behavioural measures of the CI stimuli (i.e., Likert ratings in the present study). Behavioural paradigms sensitive to differences in value are needed to further the study of CI symptoms in ASD.

Adolescence is a period defined by increased risk taking and impulsivity (Casey et al., 2011, 2008) - behaviours linked to heightened sensitivity to motivational cues (Andersen et al., 2000; Braams et al., 2015). This increased sensitivity to rewards in adolescence is important to consider given that we found intact learning performance in youth with ASD whereas studies reporting slower and more variable learning were conducted in adults (Lin et al., 2012b; Solomon et al., 2011). Furthermore, previous research has shown that CIs are more common in younger individuals and decrease in intensity with age (Esbensen et al., 2009). Even though it has been shown that CIs have an intrinsic value for adults (Dichter et al., 2012) and children alike (Cascio et al., 2014; Foss-Feig et al., 2016), our findings need to be replicated in other age groups to be generalized across the lifespan.

While in the present study we used individualized images as reinforcers, several previous behavioural (Benning et al., 2016; Sasson et al., 2008), neuroimaging (Dichter et al., 2012) and eye-tracking (Sasson et al., 2012, 2011; Sasson and Touchstone, 2014) studies have used a set of images of commonly reported interests to investigate CI symptoms in ASD. A strength of using a common image set is that it enables straightforward replication of experiments and generalization of findings. However, an important weakness is that CIs are idiosyncratic by definition (Anthony et al., 2013) and it is therefore unclear how engaging these stimuli are across participants. Examining the interests reported here, we found that some interests were mentioned by both groups within the same valence category (e.g. hockey was liked by ASD and TD participants alike) and that within one group some interests were mentioned in different valence categories (e.g. basketball was liked and disliked by TD participants). These observations are in line with a larger online study from our lab in which we found that liking of interests reported by adolescents and young adults with and without ASD were largely similar between groups (Cho et al., 2017). The variety and overlap of interests between groups highlights the importance of personalizing stimuli rather than assuming agreement on general stimuli. However, it does limit the ability to compare responses towards specific categories (e.g., People vs. Entertainment) between groups as participants did not have equal representation of categories. Further, we did not use highly aversive image categories to contrast liked images (e.g., violent images) which might evoke stronger responses. Also, having participants rate the doors before the task, might have cued them towards a potential role of the value of the doors. While it was important to assess initial preferences of the colours of the door pre-task, it might be worth doing this differently in future studies to avoid any possible biases towards the door values. Another limitation of this study was that the nature of our tasks limited our sample to high-functioning individuals who understood task instructions and tolerated MRI scanning. This makes our results difficult to generalize for the broader autism spectrum. Furthermore, our sample was relatively small and consisted primarily of male participants, which precludes a separate analysis for females only.

In sum, we found that learning and fMRI BOLD responses were not different between adolescents with and without ASD, when personalized likes and dislikes were used as reinforcers. Intervention programs for ASD show varying success rates (Sallows and Graupner, 2005), which makes our findings particularly interesting as these programs make use of reinforcement learning strategies and sometimes integrate access to CIs to elicit learning. Greater study of reward system responses in ASD, particularly in a developmental context, are needed in order to improve and optimize existing therapies.

Conflict of interest

All authors declare no conflict of interests.