Behavioural flexibility of children with Autism Spectrum Disorder on a card-sorting task with varying task difficulty.

Inflexibility is taken to be a key characteristic of Autism Spectrum Disorder (ASD), although it is unclear which aspect of cognitive functioning is critical in this context. The current study investigated task-switching problems and inflexibility with a group of children with ASD, and a mental-aged matched control group. Participants (n = 50; mean age = 7 years) completed two card-sorting tasks, which involved learning to sort by either two or three possible dimensions, and then the sorting rule was switched although the number of dimensions required to sort the cards remained the same. Following the sorting rule change, the ASD group made more errors compared to controls. Errors were also related to task type (two or three dimensions), but this was not found to interact with ASD. If poor performance were solely dependent on executive function (working memory) problems in ASD, then a steeper decrease in performance with an increase in task difficulty for one group, compared to another group, would be expected. The current results suggest that task difficulty is an aspect of importance in set-shifting, but shifting is not differentially affected by this component.

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that presents with the individual experiencing persistent deficits in social-communication and social-interaction, and displaying inflexibility in behaviour, such as restricted and/or repetitive patterns of behaviours, interests, or activities (APA, 2013). A number of theoretical accounts have been advanced to explain the symptoms that present in those diagnosed with ASD, such as Weak Central Coherence Theory (Frith, 1989), Executive Dysfunction hypothesis (Russell, 1997), and the Theory of Mind deficit (Baron-Cohen et al., 1985). However, it is now generally accepted that a single underlying account of the range of presenting symptoms may be unrealistic (Frith and Happé, 1994), and greater focus has been given to explaining the particular symptom domains, such as restricted or repetitive behaviours, which are related to inflexibility in those with ASD (Kenworthy et al., 2010; Yerys et al., 2009).

Individuals with ASD can display poor performance compared to typically developing individuals on experimental tasks such as intra-dimensional/extra-dimensional shift (ID/EDS; Hughes et al., 1994; Yerys et al., 2009), and the Wisconsin Card Sorting Task (WCST; Geurts et al., 2009; Van Eylen et al., 2011; Westwood et al., 2016). In such tasks, a previously learned rule is altered, and a new rule must be learned to maintain performance. As individuals with ASD can take longer than comparison groups to perform according to the new rule (Reed et al., 2011; Van Eylen et al., 2011; Yerys et al., 2009).

However, while many set-shifting tasks observe impaired performance in those with ASD (Dichter et al., 2010; Reed et al., 2011; Yerys et al., 2009), there is some degree of variation in the findings both between and within tasks (see Geurts et al., 2009; Poljac et al., 2010; Robinson et al., 2009). An issue that could be relevant to inconsistent switching results is task difficulty (de Vries and Geurts, 2012; Geurts et al., 2009; Steele et al., 2007; Williams et al., 2005).

The literature with respect to the impact of task difficulty on performance of those with ASD, across a range, of tasks suggests a very varied pattern of outcomes. Minshew and Goldstein (2001) noted that, as task complexity increases, deficits in basic memory processes in those with ASD become greater. In contrast, Williams et al. (2005) noted that task difficulty was not the primary reason for the relative difficulty experienced by those with ASD on a range of tasks. Similarly, while Steele et al. (2007) noted that memory load differentially impacted spatial abilities for individuals with ASD; Ozonoff and Strayer (2001) noted no such effect. With respect to prospective memory, Mahy et al. (2014); see also Altgassen et al. (2009) noted that individuals with ASD were not differentially impaired by increases in task difficulty.

Switching performance for those with ASD has been suggested to be worse when the tasks used involve greater levels of difficulty (de Vries and Geurts, 2012). For example, when the rate at which the performance-governing rule is switched, or the arbitrariness of the rule to be learned is greater, then switching performance is worse (Dichter et al., 2010; Stoet and Lopez, 2011). In fact, switching deficits can disappear when simpler tasks, or tasks with minimal working memory demands, are employed (de Vries and Geurts, 2012; Maes et al., 2011; Poljac et al., 2010). These findings suggest that there might be a role for working memory components of executive function in switching tasks.

Unfortunately, there is little literature that would allow a judgement to be made as to whether the relative decrease in switching performance is related to task difficulty per se. The above switching experiments often vary task difficulty by altering the nature of the rule, rather than by altering the complexity of the same rule (Stoet and Lopez, 2011), introducing a potential confound. Those studies which have varied some dimension of the task to manipulate its difficulty, often vary factors external to the task, like the number of trials delivered on one rule before a swift to the next, rather than some aspect of the task itself – a ‘within-task’ variation of difficulty (Dichter et al., 2010; Poljac et al., 2010).

One way in which task difficulty has been investigated by varying the task itself is by increasing the number of dimensions or stimuli involved in the discrimination (see Williams et al., 2005, for an overview). The set-shifting procedure developed by Reed et al. (2011) offers an opportunity to examine the impact of task difficulty on switching when this is varied ‘within-task’. In this task, participants sort a set of cards, which display stimuli differing along a number of dimensions (e.g., type, colour, number). They receive feedback for sorting the cards according to a rule, and, when learned to criterion, the rule is changed. This also allows some validation that task difficulty is dependent on the number of dimensions, as, if task difficulty produces greater inflexibility there should be more errors when the cards contain three dimensions than when they contain only two (see Barch et al., 1997).

In addition to adding to the empirical knowledge regarding the relationship between shifting and task difficulty, analysis of task difficulty has theoretical relevance for the issue of the relationship between switching and executive dysfunction (e.g., Ozonoff et al., 1991). A lack of flexibility is often taken to reflect underlying problems with executive functioning (Hill, 2004; Hughes et al., 1994). Typically, executive function is defined as a set of cognitive abilities, including working memory, inhibition, planning, and flexibility (e.g., Hill, 2004; Miyake et al., 2000; Miyake and Friedman, 2012). Traditionally, card-sort tasks are seen primarily as flexibility tasks (Yerys et al., 2009), although they may also touch on other components of executive function, such as inhibition (e.g., Bialystok and Martin, 2004). However, these components of executive function are actually only modestly related to one another (Miyake et al., 2000), and flexibility and shifting are particularly weakly associated with other executive processes (e.g., Miyake et al., 2000 Miyake and Friedman, 2012).

It can be suggested that, if performance on a task depends on executive function, then a steeper decrease in performance, with an increase in task difficulty, would imply a differential problem with executive function for a group showing this pattern of results (e.g., Minshew and Goldstein, 2001). For example, if a group with ASD displayed poor performance on a task because an executive function problem made the demands of the task hard for them to master, then making that task even harder would make their performance decline more rapidly. In contrast, a group lacking such a cognitive problem, and whose performance was just dependent on task difficulty, would show a smaller decrease in performance as task difficulty increased. However, if the difference in performance between two groups declined at the same rate as task difficulty increased, then this might suggest that shifting is impaired, but that the shifting impairment is not influenced when task demands that might be expected to impact other aspects of executive function (such as working memory) are increased.

The current study examined flexibility of children with ASD in order to explore whether task difficulty affects performance, and, in doing so, extended previous findings that children with ASD require more trials to relearn a task following the change in the sorting rule in comparison to controls (Reed et al., 2011). Two performance measures were taken as it has been suggested that global performance might be more closely linked to working memory than perseveration errors (Lehto, 1996). If task difficulty differentially impacts the ability of those with ASD to set-shift, then this suggests this impairment is related to a working memory/executive function problem. However, if task difficulty does not differentially impact those with ASD, then some other mechanism may underlie behavioural inflexibility.

Method

Participants

Fifty male children (25 with ASD, and 25 typically developing children matched on verbal mental age) participated in the study. Mental-age has been suggested as important in this regard, but a group with learning disabilities (intellectual disabilities) without ASD was not included, as it is unclear how the profile of set-shifting performance is impacted in this population (cf. Barnard et al., 2008; Lanfranchi et al., 2010). Participants in the ASD group all had a diagnosis of Autism Spectrum Disorder, which was given by paediatricians (with at least 5 years' experience) who were independent from the current study, according to the DSM-IV or DSM-5 criteria and clinical judgment. None of the sample had a clinically-recognised comorbid diagnosis. The severity of the participants ASD was also assessed using the Gilliam Autism rating Scale (GARS-II) completed by the parents, and they were found to have an autism quotient of between 88 and 111, which indicates an ‘average’ severity of symptoms.

The mean mental age for the ASD group, as measured by the British Picture Vocabulary Scale, was 76.80 (±22.50) months, and the mean mental age for the control group was 73.36 (±8.36), t(48) = .48, p = .447, d = .22. The BPVS was employed as a measure of intellectual function, as previous work has shown that linguistic ability are very strong predictors of task performance in this population (Kelly et al., 2015). The mean chronological age for the ASD group was 98.56 (±11.14) months, and the mean chronological age for the control group was 71.27 (±7.02), t(48) = 10.47, p < .001, d = 3.03. It might be noted that the lower mental age compared to the chronological age for the ASD group could indicate a moderate intellectual impairment, despite the clinical diagnosis (which, of course, would be more extensive than the current non-diagnostic test).

Ethical approval for this research was provided by the Department of Psychology Ethics Committee, Swansea University. These experiments were conducted according to established ethical guidelines, and informed consent was obtained from the participants.

Materials

Gilliam Autism Rating Scale (GARS-II; Gilliam, 2006) comprises three subscales, each describing behaviours symptomatic of autism (Stereotyped Behaviours, Communication, and Social Interaction). The raw scores from these subscales can be converted into standard scores (mean = 10, standard deviation = 3). These subscales combined and converted to give an Autism Index, high scores meaning greater autistic severity (mean = 100 [average autistic severity], standard deviation = 15). The scale is appropriate for persons aged 3–22 years, and is completed by parents or professionals in about 10 minutes. Its internal reliability is 0.96, and it has high criterion validity with the Autism Behaviour Checklist (0.94).

British Picture Vocabulary Scale (BPVS; Dunn and Dunn, 1982) is a Picture Vocabulary Scale, which measures receptive language ability. The BPVS is standardized for use on children in the U.K. between 3 and 17 years old. It gives an age equivalent score for children's receptive language ability, and provides a measure of verbal IQ. It has an internal reliability of 0.93, and has a 0.59 correlation with the Reynell Comprehension Scale (Reed et al., 2011). The BPVS also has a correlation on 0.61 with the Leiter international performance scale, which provides a non-verbal IQ score, for people with significant communication disorders, and various types of learning disorders (Glenn and Cunningham, 2005).

Conditioning Materials: Two packs, of 128 cards each, were used in the study, each card was 6 cm × 4 cm, was printed on white card, and was laminated. The first pack was of the 2D cards; these cards varied along two dimensions, and could have one of four different shapes (square, cross, triangle, circle), and be one of four different colours (red, blue, yellow, brown). Each card contained two of the shapes that it pictured (e.g., two squares, or two circles). There were 8 examples of each of the colour-shape combinations in the pack. Thus, these cards could be sorted by shape or colour (see Reed et al., 2011). The 3D cards were similar to the 2D cards, but varied along three dimensions – not only in terms of colour and shape, but also in terms of the number of shapes that each card contained (two, three, four, or five), and, thus, could be sorted by shape, colour, or number. There were 2 examples of each of the colour-shape-number combinations in the pack. Examples of the cards can be seen in Fig. 1.

Fig. 1

Examples of the stimuli used for the 2D and 3D card tasks.

Procedure

Participants were tested at their school, in a quiet room that was free from distractions. The study involved two card-sorting tasks (one using the 2D cards, and one using the 3D cards). Each task involved both training and test phases (see Reed et al., 2011), and was the same for both the 2D and 3D conditions. The order in which participants received the 2D and 3D card task was counterbalanced.

Phase 1 (Training)

Each phase began with the same four key-cards being put next to each other, in a line in front, of the participant. Each card showed a different shape, and was of a different colour.

All participants were given the following instructions: “We are going to play a game. You have to sort these cards into four piles. The cards that are like each other should be put together in the same pile.”

It was also explained to participants that they could have a break, or stop playing the game, if they wished to do so, at any time. The experimenter then provided a demonstration of the task. The experimenter shuffled the cards for 10 s, and then placed them face down in a single pile on the table. The experimenter then turned over one card at a time from the top of the pile, and placed it on one of the other four exposed cards according to a rule – either colour or shape (counterbalanced across participants). The experimenter demonstrated how to sort the cards by the correct dimension (e.g., if they were required to sort by shape, the experimenter gave a demonstration of how to sort by shape; whereas, if they were required to first sort by colour, the experimenter gave a demonstration of how to sort the cards by colour). This continued until all of the cards from the pile were exposed and placed onto the four visible piles.

Following the demonstration, the participant was required to begin the task. Participants picked up one card at a time from the top of the pile of cards, and placed each card on one of the four piles. The rule determining which response was correct differed across the participants. For half of the participants, if they sorted the cards into piles based on colour, then that was correct; and for the remaining participants, if they sorted by shape, then that was considered correct. Number was never used as a sorting criterion for the 2D group. The rule by which the participants has to sort the cards determined the manner in which the experimenter had sorted the cards in the demonstration part of this phase (i.e., if the participant were assigned to sort the cards by colour, then they had witnessed the experimenter sort the cards in this manner). If a card was placed in the correct pile, the participant received verbal praise, e.g., “Yes, well done!”. If the participant sorted the cards by another variable, then this was considered to be incorrect, and the participant was told that they had made a mistake. If participants made a mistake, they were asked to continue to sort the rest of the cards, from the pile, and were required to do so until they had made 10 consecutive correct responses. If all the cards were used from the pile without the participant reaching criterion, then the cards were collected from the visible piles, reshuffled, and placed face downwards so that the task could continue until the participants reached 10 consecutive correct responses in a row. This phase typically took the participants approximately 5 min to complete.

Phase 2 (Test)

Following the completion of Phase 1, Phase 2 began immediately, and the sorting rule was changed without informing the participant, or reshuffling the cards. However, the change in the sorting rule was not explicitly signalled to the participant (e.g., by saying that the rule has now changed), but they now received feedback for their sorting responses in line with the new rule. As in Phase 1, the participant was required to choose one card, at a time, from the top of the pile, and place it on one of the four piles. A correct trial was when the participant had sorted the cards by the alternative dimension to that trained in previously. If they had previously sorted by colour, the rule would change to shape, and vice versa. Corrective feedback was given as described for Phase 1. Participants took around 5–10 min to complete Phase 2. A limit of two reshuffles was put in place to avoid unnecessary distress for those participants who could not master the task.

Following completion of the first card sorting task, the participants were tested for their mental age using the BPVS.

Results

Fig. 2 shows the mean number of trials needed to reach criterion in Phase 1 and 2, for the 3D and 2D conditions, for both groups. The ASD group required more trials during both phases for both conditions. In addition, the ASD group also required more trials during Phase 2 than the in Phase 1. These data were analysed by a three-factor mixed-model analysis of variance (ANOVA) with phase (1 versus 2) and condition (3D versus 2D) as within-subject factors, and group (ASD versus control) as a between-subjects factor. This analysis revealed statistically significant main effects of group, F(1,48) = 57.30, p < .001, η = .544, condition, F(1,48) = 62.08, p < .001, η = .564, and phase, F(1,48) = 86.20, p < .001, η = .642, and a statistically significant interaction between phase and condition, F(1,48) = 25.72, p < .01, η = .349. There were no other statistically significant interactions, all ps > .10. These data were further analysed by separate, two-factor mixed-model ANOVAs (group × condition) for each phase. The analysis for Phase 1 revealed a statistically significant main effect for group, F(1,48) = 26.11, p < .001, η = .352, and condition, F(1,48) = 3.94, p < .05, η = .076, but there was no significant interaction, F < 1. The analysis for Phase 2 revealed statistically significant main effects for group, F(1,48) = 37.75, p < .001, η = .440, and condition, F(1,48) = 48.80, p < .001, η = .504, but there was no significant interaction between the two factors, F < .1.

Fig. 2

Mean trials to criterion in learning the rule to sort the cards in Phase 1 and Phase 2 for the two tasks for both groups of participants.

Fig. 3 shows the mean number of errors made by participants in each condition, in each phase of the study. Participants in both groups made more errors in Phase 2, and the ASD group made more errors in comparison to the control group. These data were analysed by a three-factor mixed model ANOVA (phase × condition × group), which revealed statistically significant main effects of group, F(1,48) = 79.74, p < .001, η = .624, condition, F(1,48) = 29.40, p < .001, η = .380, and phase, F(1,48) = 399.44, p < .001, η = .893. There were statistically significant interactions between group and phase, F(1,48) = 73.74, p < .001, η = .606, and condition and phase, F(1,48) = 11.02, p < .01, η = .187, but no other interaction was statistically significant, all ps > .10. These data were further analysed by separate, two-factor mixed-model ANOVAs (group × condition) for each phase. The analysis for Phase 1 revealed a statistically significant main effect for group, F(1,48) = 9.88, p < .01, η = .171, and condition, F(1,48) = 7.86, p < .01, η = .141, but there was no significant interaction, F < 1. The analysis for Phase 2 revealed statistically significant main effects for group, F(1,48) = 120.26, p < .001, η = .715, and condition, F(1,48) = 31.21, p < .001, η = .394, but there was no significant interaction between the two factors, F < .1.

Fig. 3

Mean number of errors made in learning the rule to sort the cards in Phase 1 and Phase 2 for the two tasks for both groups of participants.

Fig. 4 shows the number of perseverative errors (continuing to respond as previously trained in Phase 1) made in each condition, by each group, in Phase 2. The ASD group made more preservative errors than controls, in both conditions, and both groups made more errors on the 3D task in comparison to the 2D task. These data were analysed by a two-factor mixed-model ANOVA (condition × group), which revealed statistically significant main effects of group, F(1,48) = 77.63, p < .001, η = .618, and condition, F(1,48) = 39.23, p < .001, η = .450, but the interaction between the two factors was not statistically significant, p > .10.

Fig. 4

Mean number of perseverative errors made in learning the rule to sort the cards in Phase 1 and Phase 2 for the two tasks for both groups of participants.

Table 1 presents the Pearson's correlations between the demographic variables (age and verbal mental age) and the numbers of errors made by the participants for the whole sample, and separately for the two groups. Inspection of these data reveals a significant negative effect of chronological age for the sample for most types of error – reflecting that the ASD group was older than the control group. There were few statistically reliable effects of age or mental verbal age for the two groups. There were higher negative correlations between both age and mental verbal age for the control than the ASD group, but these correlations were not statistically reliable.

Table 1

Pearson correlations between age and verbal mental age (BPVS months) and the various types of error in both phases.

Error	Sample		ASD		Control
	Age	BPVS	Age	BPVS	Age	Mental Age
Phase 1 2D Total	.068	−.086	−.114	−.078	−.388	−.271
Phase 1 3D Total	.508***	.153	.260	.046	.455	.373
Phase 2 2D Total	.616***	.077	.230	.066	−.273	−.263
Phase 2 3D Total	.744***	.051	.388	.044	−.160	−.422
Phase 2 2D Perseverative	.493***	−.048	.059	−.059	−.399	−.515**
Phase 2 3D Perseverative	.662***	−.044	.322	−.140	−.070	−.287

**p < .01; **p < .001.

Discussion

The current study examined whether flexibility was impaired in a sample of children with ASD in comparison to a group of matched typically developing controls, in the light of some divergent results in the literature (see Geurts et al., 2009; Poljac et al., 2010; Reed et al., 2011; Robinson et al., 2009). It also examined whether performance was differentially affected by task difficulty, as this has some theoretical importance for the source of this effect (Minshew and Goldstein, 2001). The ASD group made more preservative errors, and more total errors, in comparison to the comparison group of mental-age matched typically-developing children (see also Reed et al., 2011; Yerys et al., 2009). However, although participants made more errors in a condition with greater stimulus complexity, indicating greater task difficulty, this variable was not found to interact with ASD (see also Poljac et al., 2010).

That the present study replicated previous findings of deficits in flexibility for people with ASD (e.g., Hughes et al., 1994; Ozonoff et al., 1991). More specifically, the present study replicated the finding that, when a rule changes on a card-sorting task, children with ASD have more problems in learning the new rule than comparison children. However, although the present study did find that the introduction of an extra dimension to the stimuli made the task more difficult to learn, initially, it found no significant interaction of task difficulty with group. Thus, a potential interpretation is that while shifting is impaired, the shifting impairment is not influenced when working memory demands are increased in the task.

In this, the current results corroborate those reported by Poljac et al. (2010) who also found, using a different procedure, no interaction between task difficulty and group performance on a switching task. If performance were solely dependant on aspects of executive function, such as working memory, then a steeper decrease in performance with an increase in task difficulty for one group compared to another group would be expected (e.g., Minshew and Goldstein, 2001). As the difference in performance between two groups was independent of task difficulty level, then this might suggest that the problem is not in purely with working memory/executive function. Of course, that is not to say that working memory aspects of executive function does not play a role (de Vries and Geurts, 2012; Maes et al., 2011; Poljac et al., 2010), but, rather, that this is not the sole contribution.

If task difficulty level is, indeed, not relevant to set-shifting performance, then this has some implications theoretically for the mechanisms that may underlay this ability. In working memory tasks, such as the n-back task, or in planning tasks, such as the Tower of London, a steeper decrease in performance with an increase in task difficulty for one group as compared to another would suggest that there is a problem with working memory or planning. However, as no such differential impact was noted in the current study, then this type of cognitive flexibility may be the product of some other cognitive process. In this regard, the executive dysfunction theory infers that poor performance by individuals with ASD is the result of working memory dysfunction (e.g., Ozonoff et al., 1991). However, studies using direct measures of working memory have not found impairments in this domain for individuals with ASD (e.g., Ozonoff and Strayer, 2001). These studies challenge the view that individuals with ASD perform poorly on working memory tasks because of a deficiency in working memory per se. This suggestion could be further tested by using a well-established working memory task, such as the n-back digit span task, to differentiate the effect of a “pure” working memory from an executive function task.

It might be noted that working memory and executive function can show developmental changes (Luciana et al., 2005). The current study may not have found a difference in performance between the ASD and typically developing groups as the mental age tested was not in the sensitive part of the developmental range to show differences. This possibility could be explored using a wider range of ages, but if this were the case, then it would set some boundary conditions for the impacts of various aspects of tests in terms of their relationship to cognitive function. A further possibility is that task difficulty could be measured differently, the present study measured difficulty in terms of the number of errors made, and trials needed to reach criterion, as has been used previously (Barch et al., 1997). Another possible measure of task difficulty is time, i.e. how long it takes to learn each task (3D versus 2D) in phase 1. Future research in this area should continue to investigate whether task dimension (3D versus 2D) determines task difficulty. A similar study to the present study could be done, using a table-top design, and could measure task difficulty independently, by measuring how long participants take to learn the two tasks (3D versus 2D), during phase one.

Additionally, there are also limitations including potential sampling bias, small sample size, and the age and functioning level of the children, which could be addressed in further research. Although, according to the clinical diagnosis, the ASD had no comorbid problems such as intellectual impairments, their mental age as measured by the BPVS in the current study implies the possible presence of moderate intellectual delays. If this is correct, then this aspect of the sample differs from several prior studies using the ID/ED that employed samples without intellectual disability (Landa and Goldberg, 2005; Goldberg et al., 2005; Yerys et al., 2009), or samples with a large range of cognitive ability (e.g., Ozonoff et al., 1994). Such a difference in cognitive functioning (and measures used to assay cognitive function) may explain why some studies have found relationships with mental age and others have not. In this regard use of a nonverbal IQ test, as well as the verbal measure employed here, may also be useful. Based on the current mental-age scores, it may be that the ASD group also had some learning/intellectual Disabilities, which would be import to further explore in order to determine if the difference between the ASD group and the control group were related to the ASD or to a comorbid intellectual disability. Although this remains a possibility, it should be noted that the ASD group had been clinically assessed as having ASD.

In summary, the present data seems to suggest that there is a difficulty in set-shifting for individuals with ASD, but the finding that task difficulty does not impact this ability suggests that this deficit may not be related to working memory problems.

Declarations

Author contribution statement

Phil Reed: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.