Beyond group differences: Exploring the preliminary signals of target engagement of an executive function training for autistic children.

Understanding both for whom and how interventions work is a crucial next step in providing personalized care to children with autism spectrum disorder (ASD). Autistic children present with heterogeneity both within core ASD criteria and with respect to co-occurring mental health challenges, which may affect their ability to benefit from intervention. In a secondary data analysis of a randomized control trial evaluating an executive function (EF) training with 70 7- to 11-year-old autistic children, we explored: (1) whether co-occurring attention-deficit/hyperactivity disorder (ADHD) features or anxiety features at baseline moderated the extent to which children benefited from the EF training. In other words, we asked, "For whom is training effective?" We also explored: (2) the extent to which changes in a brain-based measure of target engagement predicted the clinical outcomes of the EF training. This is a step towards asking, "How is training effective?" We found that EF training improved behavioral inhibition only for children with clinically significant co-occurring ADHD features. Anxiety features, while prevalent, did not moderate EF training efficacy. Finally, for the EF training group only, there was a significant correlation between pre-to-post change in an EEG-based measure of target engagement, N2 incongruent amplitude during a flanker task, and change in repetitive behaviors, a behavioral outcome that was reported in the parent RCT to have improved with training compared to waitlist control. This study provides preliminary evidence that EF training may differentially affect subgroups of autistic children and that changes at the neural level may precede changes in behavior. LAY
SUMMARY: Understanding both for whom and how interventions work will help us provide personalized care to children with autism spectrum disorder (ASD). Autistic children present with many different strengths and challenges. Co-occurring mental health challenges may affect how much autistic children benefit from intervention. We analyzed secondary data from a rigorously designed pilot intervention study, a randomized control trial (RCT), that enrolled 70 7- to 11-year-old autistic children to assess whether a set of computer-based executive function (EF) training games improved their performance. Executive functions include being able to shift between tasks, inhibit a response, and keep information in working memory. In the current study, we explored: (1) whether children's co-occurring attention-deficit/hyperactivity disorder (ADHD) features or anxiety features, measured before the EF training began, affected how much they benefited from the EF training. In other words, we asked, "For whom is training effective?" We also explored: (2) whether children's brain-based changes in EF predicted their performance in everyday life (e.g., parent report on a survey). This is a step toward asking, "How is training effective?" We found that EF training improved children's inhibition ability, but only for children with clinically significant ADHD features. While many children in our sample also had anxiety features, we found that anxiety levels did not affect how well the EF training worked. Finally, for children who received the EF training, changes in a brain-based measure of conflict monitoring (i.e., being able to noticing differences in stimuli) predicted changes in children's repetitive behaviors. This study provides early evidence that EF training may be more effective for some autistic children than others, especially those with ADHD features.

INTRODUCTION

We are increasingly encouraged to conceptualize children's mental health challenges and response to intervention in terms of transdiagnostic characteristics, using frameworks such as the NIH Research Domain Criteria (RDoC; Beauchaine & Hinshaw, 2020). Co‐occurring mental health and developmental conditions are prevalent among children with autism spectrum disorder (ASD), who have higher rates of co‐occurring anxiety (Kirsch et al., 2020), attention‐deficit/hyperactivity disorder (ADHD; Supekar et al., 2017), and intellectual disability (ID; Maenner et al., 2020) than neurotypical children. These co‐occurring mental health conditions may affect the efficacy of interventions designed to support the core features common to ASD.

Executive function (EF) is often challenging for autistic children (Demetriou et al., 2018; Geurts et al., 2014; Happé et al., 2006; Kenworthy et al., 2008; Rosenthal et al., 2013). EF is the ability to manage complex and conflicting information while engaging in goal‐oriented behaviors (Welsh & Pennington, 1988) and includes inhibition, working memory, and set‐shifting or flexibility (Lehto et al., 2003; McAuley & White, 2011; Miyake et al., 2000; Miyake & Friedman, 2012). EF is related to a wide variety of characteristics for autistic children: social competence (Pellicano, 2013), restricted and repetitive behaviors (Faja & Nelson Darling, 2019; Geurts et al., 2014), academic achievement (St. John et al., 2018), and emotion regulation (Jahromi, 2017). Understanding how EF impacts these clinical outcomes is essential.

Brief training in EF skills, paced just beyond children's current level of competency, improves EF for both children with neurotypical development (Diamond & Lee, 2011) and autistic children (de Vries et al., 2015; Kenworthy et al., 2014; Yerys et al., 2019). There is emerging support for computer‐based EF training for autistic children. In children with both ASD and ADHD, an intervention targeting multitasking via an app (Project EVO) showed improvement in EF, ADHD symptoms, and social skills on a lab‐based task and a parent‐report measure (Yerys et al., 2019), while a different EF training for a sample of autistic children failed to find intervention effects (de Vries et al., 2015). Computer‐based interventions may reduce stress associated with face‐to‐face intervention, allow users to move at an individual pace, and be more feasibly implemented, engaging, and appealing (Bölte et al., 2010; Dichter et al., 2012)—key factors that make computer‐based intervention especially appropriate for autistic children.

The current study is a secondary data analysis of a parent randomized controlled trial (the “parent RCT”) that examined the efficacy of a computer‐based EF training for school‐age autistic children when combined with in‐person metacognition coaching (Faja et al., 2021). In this RCT, EF training was hypothesized to improve a set of cascading EF outcomes, modeled hierarchically on the way that an intervention might impact proximal to distal outcomes over time (Figure 1). EF growth was hypothesized to first present at the neural level, such that brain‐based evidence of growth in EF (i.e., the ability to monitor and differentiate conflicting information) would establish target engagement of neural processes for the EF training (“neural target”). Then, if the EF training had a robust enough effect, growth in EF outcomes would also present at the behavioral level, during lab‐based tasks that assess a range of specific EF skills similar to those targeted by the training (“behavioral targets”). Finally, the most pervasive and functionally important gains in EF would present at the level of generalized EF skills, as reported by parents on a questionnaire (“clinical outcomes”) (Figure 1). The parent RCT found that the EF training improved EF skills only at the neural level—the level of target engagement. At the more distal, clinical outcomes level, there were treatment effects for restricted and repetitive behaviors, but no effects related to EF behavior (Faja et al., 2022). This raised the possibilities that (1) the EF training was simply not delivered at the intensity needed to improve behavioral in addition to brain‐based EF outcomes, or that (2) the EF training improved EF at the behavioral level only for a subgroup of participants.

FIGURE 1

Conceptual diagram of RCT design and outcomes. EF, executive function; BRIEF, Behavior Rating Inventory of Executive Function; GEC, General Executive Composite; RBS‐R, Repetitive Behaviors Scale‐Revised.

Researchers have been urged to explore moderators of intervention effects for autistic children for at least the past decade (Klinger et al., 2020; Schreibman et al., 2011). Despite this, few studies have formally done so. Begeer et al. (2015) found that children's more passive social interaction styles blunted the efficacy of a social communication intervention. Lopata et al. (2020) found that while children's externalizing behaviors blunted social cognition outcomes, their adaptive skills enhanced ASD symptom reduction. More research is needed on child, family, and contextual characteristics that affect (i.e., blunt or enhance) how well interventions work. ADHD symptoms and anxiety symptoms have both been found to moderate (in this case, blunt) the efficacy of a behavioral parent training intervention for disruptive behavior in autistic 3‐ to 7‐year‐olds (Lecavalier et al., 2017). To our knowledge, no studies have examined moderators of EF training efficacy for autistic children.

We propose that children's co‐occurring ADHD and anxiety features may moderate, or affect the strength of, EF training efficacy. Autistic children with ADHD features (i.e., “ASD + ADHD”) may especially benefit from EF training, compared to autistic children without ADHD features. Children with ASD + ADHD may have a unique EF profile and etiology that would require distinct intervention supports. For example, children with ASD + ADHD do not always have an additive EF deficit; their performance on inhibitory EF tasks varies depending on task demands (Cremone‐Caira et al., 2021). Studies comparing EF profiles among children with ASD and ADHD directly have demonstrated that while both groups show EF impairment (Karalunas et al., 2018; Sinzig et al., 2008; Tye et al., 2014), those with ASD may display more profound and global EF deficits (Corbett et al., 2009; Verté et al., 2006). Considerably less‐well studied is EF development among groups of children with a diagnosis of ASD + ADHD. In a comprehensive review, Craig et al. (2016) showed that response inhibition, along with flexibility and planning, discriminated EF challenges among children with ASD + ADHD compared to those with only ASD or ADHD, suggesting that children with ASD + ADHD do indeed experience executive dysfunction and that it may manifest uniquely from ASD and ADHD, respectively. This evidence suggests EF problems are a transdiagnostic challenge spanning across diagnostic groups and that our EF training, because it targets EF challenges that may be more variable and pervasive in children with ASD + ADHD, may be more helpful for children with both conditions than for children with ASD alone.

Autistic children with co‐occurring anxiety may respond differently to intervention (Reaven, 2011). They may especially benefit from interventions that allow them to practice challenging or anxiety‐provoking tasks in a relatively nonthreatening or supportive space. For example, social skills interventions both improve social skills and reduce social anxiety, likely due to opportunities for practice or exposure (Schohl et al., 2014). Anxiety features are correlated with reduced EF in autism (Hollocks et al., 2014), and it is thought that anxiety may directly blunt EF skills like attention, shifting, and working memory (Corbetta & Shulman, 2002) and potentially inhibit learning. Metacognition coaching on emotion regulation has been provided in previous EF training trials (Macoun et al., 2020) and may enhance intervention effects specifically for children with anxiety because it allows children to focus on practicing EF skills without anxiety causing impairment in the moment. We hypothesized that our EF training, because it includes coaching and support to reduce performance anxiety and emotion dysregulation during intervention sessions, may be particularly helpful for autistic children with co‐occurring anxiety.

The parent RCT found that the EF training, compared to waitlist, improved performance on the hypothesized neural target: children's ability to monitor and differentiate conflicting information during a flanker task as measured by EEG (e.g., N2 incongruent amplitude) (Faja et al., 2022). It is not known whether the pre‐to‐post training change in this “neural target” is related to change in the behaviorally measured clinical outcomes of interest, including parent‐reported EF and repetitive behaviors (Figure 1). We hypothesize that EF training would affect clinical outcomes by first improving neural measures of EF, such that neural EF growth mediates the relation between receiving EF training and improved clinical outcomes.

Given the high occurrence of co‐occurring mental health conditions among autistic children, it is important to identify which subgroups of autistic children may respond best to inventions designed to enhance EF. The first goal of this secondary data analysis was to explore whether clinical characteristics predict which school‐age autistic children show benefits at the behavioral level from EF training. We hypothesized that children with co‐occurring ADHD or anxiety features might show the most benefit from the EF training on lab‐based behavioral target measures of EF, due to the unique and more impacted EF challenges in children with ASD and ADHD, and to the benefits of practice with challenging tasks and metacognitive coaching for children with anxiety. The second goal of this study was to explore whether the measure of neural target engagement in the parent RCT, change in brain‐based EF as indexed by N2 incongruent amplitude during a flanker task, either mediated the relation between treatment group randomization and more distal behavioral outcomes or was simply correlated with change in behavioral outcomes.

METHOD

Participants

The final study sample consisted of 70 participants with an existing autism spectrum disorder (ASD) diagnosis (7 females, 63 males). All participants were between the ages of 7 to 11 years and had low average to above average cognitive functioning (Full‐Scale IQ ≥ 80). Recruitment occurred from 2015 to 2017 via community sources, a hospital research registry, word‐of‐mouth, and clinical referrals. Participants were considered ineligible if they met any of the following exclusionary criteria: (1) colorblindness, (2) significant sensory or motor impairments that prevented completion of the experimental battery, (3) not fluent in English, (4) use of medications or diagnoses that affect the central nervous system (5) exposure to alcohol or controlled substances in utero, and (6) seizure history or the use of seizure medication. Other medication use (stimulant and nonstimulant) was nonexclusionary and did not differ by group (see Faja et al., 2022). The goal of the parent RCT was to determine whether EF training improved EF for autistic children “above and beyond” typical supports (e.g., medication use).

Procedure

Participants' caregivers completed an initial phone screen to assess eligibility based on demographic information, reported clinical characteristics, and adaptive functioning using the Vineland Adaptive Behavior Scales, Second Addition (Vineland‐2; Sparrow et al., 2005). Diagnostic and cognitive eligibility were evaluated during the first study visit under the supervision of a licensed psychologist. A diagnosis of ASD was confirmed according to DSM‐5 criteria (American Psychiatric Association, 2013) informed by the Autism Diagnostic Interview‐Revised (ADI‐R; Rutter et al., 2003), the Autism Diagnostic Observation Schedule, Second Addition (ADOS‐2; Lord et al., 2012), and clinical expertise. The Wechsler Abbreviated Scale of Intelligence‐2 (WASI‐2; Wechsler, 2011) was administered to confirm that children had a Full‐Scale IQ of 80 or above and were verbally fluent.

In the second and third study visits, children completed an experimental battery of electroencephalography (EEG) and behavioral EF and social cognition tasks (described below). Caregivers also completed questionnaires. Children were randomized to either an EF training group or a waitlist control group. Randomization was computer generated. Research staff running post‐training visits were blind to group assignment. Baseline procedures were repeated post‐training, which usually occurred within 2 weeks of the end of the training, as soon as possible for the family. Within the EF training group, all participants returned for post‐training visits. Within the waitlist group, one family withdrew at randomization and another was lost to follow‐up due to a family stressor.

The study was conducted at Boston Children's Hospital and approved by its Human Subjects Division; all parents provided consent and all children provided written assent to participate. The parent RCT, including the selection of outcome measures, was pre‐registered on ClinicalTrials.gov (NCT02361762). See Faja et al., 2022 for the CONSORT diagram as well as full information on study design, training content, fidelity, and acceptability, and EEG acquisition and processing procedures.

Intervention

EF training consisted of up to 10 1‐h sessions (roughly once a week), facilitated by a graduate student or research assistant. The training included four computer games designed to improve EF and attention for neurotypical preschool to school‐age children (Rueda et al., 2005). Games were minimally modified to optimize fit and interest for autistic children during a piloting phase. Two of the games targeted inhibition (interference suppression and reactive inhibition), two targeted visual working memory, and all required set‐shifting. These specific EF skills were selected as training content because of prior evidence of ASD‐specific EF dysfunction in these areas (Faja et al., 2022). The number of consecutive correct responses and overall accuracy determined participants' progress between levels of each game. A metacognitive coaching manual based on the principles of cognitive behavioral therapy was created to provide EF psychoeducation, increase metacognitive awareness, improve sustained attention and engagement, and support emotion regulation during challenging portions of the training by providing problem solving, validation of emotion, mindfulness of emotion, and other coping strategies to participants and their families. Coaching supported the generalization of skills and information learned during training. Specific examples of coaching strategies can be found in the Supplemental Materials of the parent RCT report (Faja et al., 2022). The full manual is available from the senior author. Each training session included approximately 10 min of play for each training game (total of 40 min), 15 min of metacognitive coaching (interspersed), and a 5‐min caregiver check‐in to share strategies learned.

Measures

Outcome measures

Outcome variables are organized hierarchically: neural targets, lab‐based behavioral targets, and clinical outcomes. See Figure 1.

Neural target

The primary brain‐based EF target was the mean amplitude of the N2 ERP component during incongruent trials (extracted between 300 and 400 ms; Faja et al., 2016; Samyn et al., 2014) of the flanker portion of the Child Attention Network Task (Rueda et al., 2004). The N2 ERP is an index of children's ability to monitor and differentiate conflicting information. The task included 120 total trials: 12 practice and 108 test trials (presented in random order). A beep lasting 150 ms combined with a fixation cross for 450 ms in the middle of the screen preceded each trial. Next, a target stimuli and flankers were displayed for 2000 ms. Congruent trials (50%) consisted of a central target animal flanked by two animals on each side facing the same way. For the incongruent trials (50%), the target and flankers faced opposite directions. Participants pressed a button to indicate which direction the target animal faced (50% left, 50% right) and received feedback immediately after selecting their response.

Lab‐based behavioral targets

Change Task

The Change task (De Jong et al., 1995) measures children's reaction time to complete a nondominant response after a period of giving dominant responses (i.e., reactive inhibition). After completing practice rounds, participants completed four test blocks with 75% trials prompting a “dominant” task (i.e., left/right button press) and 25% trials prompting a different, “change” task (i.e., space bar press). Change trials included stop signals equally occurring at 50, 200, 350, and 500 ms before the anticipated response of each participant. Each test block utilized the mean correct reaction time (RT) from the previous block to adjust for individual RT variability. The Stop Signal Reaction Time (SSRT), which estimates the latency required to inhibit a dominant response when a stop signal was presented (Band et al., 2003; Crone & van der Molen, 2004), was the dependent variable. Lower scores indicated better performance (i.e., inhibiting the dominant response and switching to the change response more efficiently).

Stroop Task

The Stroop task (Perlstein et al., 1998; Stroop, 1935) is a measure of children's ability to inhibit interference from non‐relevant information to complete a task (i.e., interference control). Participants first completed practice trials. Test trials were presented in pseudorandom order for three conditions: (1) congruent (25%), a color word displayed in the same color (e.g., red written in red); (2) incongruent (25%), a color word displayed in a different color (e.g., red written in blue); and (3) neutral (50%), a non‐color word in one of four colors (e.g., tiger written in red). Participants pressed a button to indicate the text color. The difference between percent trials correct for congruent and incongruent conditions was the dependent variable. Lower scores represented better performance (i.e., better suppression of interfering information).

Clinical outcomes

Behavior Rating Inventory of Executive Function

The Behavior Rating Inventory of Executive Function (BRIEF) (Gioia et al., 2000) is a caregiver‐report questionnaire that measures generalized EF skills that children display in everyday life. The Global Executive Composite (GEC) indexes EF skills most broadly, and was used as the dependent variable.

Repetitive Behavior Scale‐Revised

The Repetitive Behavior Scale‐Revised (RBS‐R) (Lam & Aman, 2007) is a caregiver‐report questionnaire that measures restricted and repetitive behaviors and interests. It was originally conceptualized as a secondary outcome given that EF training would likely first affect EF and then potentially generalize to other domains. We included RBS‐R in the present study because a significant treatment effect was found in exploratory analyses for the parent RCT, such that children who received EF training showed significantly reduced repetitive behaviors compared to children in the waitlist control group (Faja et al., 2021).

Hypothesized moderators

Child Behavior Checklist

The Child Behavior Checklist (CBCL) (Achenbach, 1999) is a normed caregiver‐report questionnaire that assesses behavioral and emotional problems. This study identified two subscales of interest for analysis as potential moderators: ADHD Problems and Anxiety Problems (referred to as ADHD features and anxiety features, respectively). The CBCL classifies children who obtain a T‐score of 70 or above as falling within the clinically significant range for that scale. For our study, we used the CBCL scale T‐scores as continuous moderating variables.

Analytic plan

To account for pre–post training change in each outcome variable, a residual score was derived such that each outcome residual represented post‐training values, controlling for pre‐training values. All models were also conducted with post‐training values as the dependent variable and pre‐training values as a covariate; results did not change. Pre–post residual scores for each outcome variable were therefore used to maximize power.

To assess moderation, multiple regression models for each behavioral target's pre–post residual score—Change task SSRT and Stroop task percent trials correct—were built separately for each potential moderator—ADHD features and anxiety features—for four total models. Training group (dummy coded), pretest ADHD or anxiety features, and a group‐by‐clinical features interaction term were included as predictors in each model. A significant interaction term indicated a moderated treatment group effect, in which case a Johnson‐Neyman region of significance (RoS) for the moderated effect was calculated using PROCESS (Hayes, 2017). An RoS determines the cut‐point on the moderator at which a significant effect of training (compared to waitlist) is present. The moderation and its RoS were visualized for interpretation using InterActive (McCabe et al., 2018).

To assess whether change in the neural target, brain‐based EF, mediated the relation between training group randomization and distal clinical outcomes, we used PROCESS (Hayes, 2017) to conduct a series of regressions and obtain a, the effect of training group randomization on brain‐based EF, and b, the effect of brain‐based EF on clinical outcomes controlling for training group assignment. The bias‐corrected confidence interval for the indirect effect, ab, was calculated using bootstrapping (10,000 samples) in PROCESS. Correlations (Pearson's r) assessed the relation between pre‐post change in brain‐based EF and clinical outcomes, separately by group.

RESULTS

Aim 1: Moderated training effects

The first aim of this study was to explore whether co‐occurring mental health symptoms predict which school‐age autistic children may most benefit from EF training. Specifically, we assessed whether ADHD features and anxiety features moderated the effect of the EF training on the proximal behavioral targets of the trial. EF training was hypothesized to affect these behavioral targets immediately post‐training by improving in performance on two lab‐based tasks (Change and Stroop tasks) that require behavioral EF skills directly targeted during the intervention (inhibition and flexibility) (Figure 1). See Table 1 for descriptive statistics and tests of group equivalence for variables of interest.

TABLE 1

Descriptive characteristics

	Waitlist M (SD)	Training M (SD)	t/F (df), p
N (70 total)	35	35	–
Sex	31 M; 4 F	32 M; 3 F	X 2 (1, N = 70) = 0.69, p = 0.50
Age in years	9.10 (1.34)	9.15 (1.38)	0.18 (68), 0.86
Race
Asian	3%	0%	–
Black	9%	3%	–
White	77%	88%	–
Biracial	11%	9%	–
Ethnicity	3% Hispanic	11% Hispanic	X 2 (1, N = 70) = 1.94, p = 0.16
ASD symptoms (SRS Total T‐score)	69.55 (9.30)	67.83 (10.08)	−0.73 (66), 0.47
Nonverbal IQ	102.74 (13.09)	108.54 (15.03)	1.72 (68), 0.09
Verbal IQ	102.40 (12.40)	106.60 (15.16)	1.23 (68), 0.21
Potential moderators: Pretest
ADHD features (CBCL)	63.74 (8.38)	60.83 (6.67)	−1.60 (67), 0.12
Anxiety features (CBCL)	63.38 (8.70)	62.49 (8.95)	−0.42 (67), 0.67
Outcomes of interest: Pretest
N2 Incongruent Amplitude	−2.50 (5.19)	0.65 (3.37)	2.61 (53), 0.01*
Stroop task (congruent–incongruent percent correct)	122.12 (137.66)	135.73 (147.73)	0.39 (65), 0.70
SSRT (ms)	232.56 (94.19)	234.75 (85.19)	0.10 (64), 0.92
BRIEF GEC	68.18 (10.20)	66.31 (11.98)	−0.69 (67), 0.49
RBS‐R Total	22.85 (14.11)	17.31 (10.64)	−1.85 (67), 0.07
Outcomes of interest: Post‐test
N2 incongruent amplitude	−3.60 (4.55)	−1.48 (3.79)	1.84 (50), 0.07
Stroop task (congruent–incongruent percent correct)	82.12 (175.57)	79.81 (163.78)	−0.55 (63), 0.96
SSRT (ms)	237.01 (95.41)	210.78 (76.96)	−1.22 (63), 0.23
BRIEF GEC	67.77 (10.42)	65.85 (9.96)	−0.75 (61), 0.46
RBS‐R total	27.03 (17.90)	13.61 (9.04)	−3.70 (58), 0.001*

Abbreviations: BRIEF GEC, Behavior Rating Inventory of Executive Function, Global Executive Composite; CBCL, Child Behavior Checklist; F, female; M, male; RBS‐R, Repetitive Behaviors Scale‐Revised; SRS, Social Responsiveness Scale; SSRT, stop‐signal reaction time.

p < 0.05.

Assessing moderation of ADHD features

Participants did not differ by group on pre‐test levels of ADHD features, t(67) = −1.60, p = 0.12. Fifteen of the 70 participants, 21% of the sample, presented with ADHD features in the clinically significant range (i.e., T‐score > = 70).

ADHD features significantly moderated the effect of the training intervention on stop‐signal reaction time (SSRT) during the change task, β = −0.33, p = 0.047 (Table 2, Figure 2). The region of significance (RoS) for this moderation was 70.69, indicating that the training group differed from the waitlist group on SSRT only for children with ADHD features of T = 70.69 or higher (Figure 3). A reasonable proportion (19%) of the sample fell within the RoS, which supported further interpretation (McCabe et al., 2018). Visualizing the interaction effect within the RoS, it was apparent that children in the training group had shorter SSRT latencies than children in the waitlist group within this RoS (Figure 2). That is, controlling for baseline SSRT, the children with elevated ADHD features (T = 70.69) who received EF training had post‐training SSRT latencies that were 98 ms faster, on average, than the untrained children with elevated ADHD features. Children's ADHD features did not significantly moderate the effect of training on the second behavioral target (Stroop).

TABLE 2

Children's ADHD features moderate the effect of GAMES intervention on stop‐signal reaction time (SSRT)

	B	SE	β	t (p)	95% CI
Intercept	2.83	15.76		0.18 (p = 0.858)	[−28.72, 34.39]
Group (0 = Waitlist, 1 = Training)	−16.60	22.29	−0.10	−0.75 (p = 0.459)	[−61.22, 28.02]
ADHD Features	4.27	1.91	0.37	2.23 (p = 0.029)	[0.44, 8.10]*
Group X ADHD Features	−6.05	2.98	−0.33	−2.03 (p = 0.047)	[−12.01, −0.85]*
	F (3, 58) = 2.20, R 2  = 0.10

Note: ADHD Features was centered prior to creation of interaction term and entry into model.

p < 0.05.

FIGURE 2

Participants' ADHD features significantly moderated the degree of group difference in pre‐to‐post SSRT change task, such that only children with clinically significant ADHD features (T‐score > 70.69) experienced a significant improvement (i.e., reduction) in their reaction time in the training group compared to the waitlist group. Y‐axis is residual SSRT (post‐training SSRT controlling for pretraining SSRT).

FIGURE 3

Marginal effects plot of region of significance (RoS) for moderation of ADHD features on treatment efficacy. The difference between training and waitlist groups in residual stop signal reaction time (SSRT) is significantly different than 0 only for participants with mean + 1.15 SD ADHD features (T = 70.69) and higher. Raw data are represented by rug plot dashes on x‐axis.

Assessing moderation of anxiety features

Participants did not differ by group on pretest levels of anxiety features, t(67) = −0.42, p = 0.67. Twenty‐two of the 70 participants, 30% of our sample, presented with anxiety features in the clinically significant range (i.e., T‐score > = 70). However, children's anxiety features did not significantly moderate the effect of EF training on the behavioral (Stroop, Change SSRT) target of interest. This suggests that the effect of the EF training on lab‐based behavioral targets did not vary based on children's pre‐training anxiety features.

Aim 2: The relation between brain‐based target engagement and clinical outcomes

An important way to learn about how children might benefit from training is to examine potential improvements in “target engagement,” which are measures closely tied to a theorized mechanism (e.g., brain‐based response), that are sensitive to treatment, and that relate to improvements in functional outcomes (i.e., clinically significant changes). The most rigorous way to demonstrate target engagement is through mediation, or analysis of indirect effects (Emsley et al., 2010). However, changes in the neural target, N2 incongruent amplitude, did not mediate the effect of randomization to the training group on either clinical outcome of interest (BRIEF GEC, RBS‐R), ps >0.05. As a follow‐up exploratory analysis, we examined correlations between pre–post residual change in N2 incongruent amplitude and the clinical outcomes (BRIEF GEC, RBS‐R) within each group separately.

N2 incongruent amplitude was not significantly correlated with BRIEF GEC within the training group, r(24) = −0.04, p = 0.84 or the waitlist group, r(21) = 0.07, p = 0.76. However, N2 incongruent amplitude was significantly correlated with RBS‐R, r(22) = 0.48, p = 0.02, such that, of the children who received EF training, children who had more negative N2 incongruent amplitudes from pre‐ to post‐training (i.e., an enlargement of the N2 amplitude) had fewer repetitive behaviors over that time period (Figure 4). An increased N2 amplitude (i.e., more neural resources activated) during the more challenging incongruent condition of the Flanker task is thought to reflect a more mature response, and this pattern is found in typical development (Pozuelos et al., 2019). N2 incongruent amplitude and RBS‐R were not significantly correlated within the waitlist group, r(20) = −0.20, p = 0.39. Further, the training and waitlist group correlation coefficients were significantly different from each other, Fisher's z = 2.17, p = 0.03. While this demonstrates that brain and behavior improvements are significantly linked for the training group but not the waitlist group, formal moderation would assess whether the brain‐behavior relation for the training group was significantly stronger for the training group as compared to waitlist. The moderation effect approached significance, such that the relation between N2 incongruent amplitude and RBS‐R trended towards being stronger for the training than the waitlist group, β = −0.36, p = 0.08.

FIGURE 4

Greater negative inflections in N2 incongruent amplitude from pre‐ to post‐training predict fewer repetitive behaviors for training compared to waitlist group. Gray lines indicate 95% confidence intervals of linear estimates. Pearson's r values for EF training (r(22) = 0.48, p = 0.02) compared to waitlist (r(20) = −0.20, p = 0.39) are significantly different, Fisher's z = 2.17, p = 0.03. However, training group did not significantly moderate the relation between N2 incongruent amplitude and RBS‐R score.

DISCUSSION

In their parent RCT analyses, Faja and colleagues reported that a computer‐based executive function (EF) training paired with metacognition coaching demonstrated: (a) improvement in a brain‐based measure of EF, a preliminary effect consistent with target engagement; and (b) improvement in an exploratory behavioral outcome, children's repetitive behaviors (2021). However, they did not find main effects of EF training on more distal outcomes: behavioral targets (lab‐based EF tasks) or the primary clinical outcome (parent‐report of generalized EF). This follow‐up secondary data analysis found that: (1) the effect of EF training on lab‐based behavioral EF is actually moderated by ADHD features such that EF training improved lab‐based behavioral EF (SSRT) only for children with clinically significant ADHD features. This study also reports preliminary evidence that: (2) within the EF training group only, changes in the neural EF target, N2 incongruent amplitude, were correlated with improvements in repetitive behaviors. Of note, mediation analyses did not provide support the effect of EF training on clinical outcomes indirectly through N2 incongruent amplitude (i.e., mediation), which would be necessary evidence for neural target engagement for this EF training.

Together with results from Project EVO (Yerys et al., 2019), our findings suggest computer‐based EF training may be more effective at changing behavior for autistic children with co‐occurring ADHD features than for autistic children without these features. ADHD is even more strongly associated with challenges in EF than ASD, which possibly gives children a “double dose” of unique inhibitory control difficulties that may be especially responsive to intervention (Corbett et al., 2009; Lee et al., 2021; Verté et al., 2006). Lee and colleagues found that children with ADHD had EF deficits in inhibition, initiation, and working memory, while children with ASD and ADHD had challenges with other aspects of EF such as set‐shifting and emotional control. Similarly, autistic children with ADHD symptoms were found to have a unique profile of inhibition abilities (i.e., worse interference control, reactive inhibition) compared to children with ASD alone (Cremone‐Caira et al., 2021). It seems apparent that there is vast heterogeneity in the types of EF deficits that autistic children have, and that EF training, especially for inhibition, might be particularly important to recommend for autistic children who also have ADHD features.

Future research should assess the effect of EF training on EF skills other than inhibition and flexibility before it is recommended for all autistic children. It could be that while EF training improves neural EF target engagement (i.e., conflict monitoring as indexed with N2) for all children (Faja et al., 2022), it only improves the specific behavioral EF skills that we assessed (inhibition, flexibility) for children with co‐occurring ADHD. Future research should explore these nuances, including whether children's age moderates the efficacy of EF training in autism and whether or not EF training should be targeted toward specific facets of EF depending on diagnostic profile. In addition, we observed some slowing of reaction time (SSRT) from pre‐ to post‐training in the waitlist group (only for children with some level of ADHD features). This was not observed for other parent RCT dependent variables and so is likely not due to motivation or practice difference and instead may be specific to the way that ADHD features affect reaction time measures for school‐age autistic children. More research is needed to understand volitivity over time in reaction time measures for this population, both with and without intervention.

We had hypothesized that this EF training might be particularly effective for autistic children with co‐occurring anxiety features because of the metacognition and emotion regulation components included in our EF training delivery. However, this hypothesis was not supported. The EF training games themselves were not designed to improve emotion regulation, but rather EF exclusively. While almost a third of our sample had clinically significant anxiety symptoms, we did not conduct a follow‐up clinical interview with our participants to determine a formal anxiety diagnosis. The EF training and metacognition coaching package may be more effective at improving EF for children with formal anxiety diagnoses rather than symptoms as reported on a questionnaire. Alternately, the emotion regulation coaching might not have been necessary for children with anxiety in the context of this EF training, which may have been experienced as fun and low‐threat. In contrast, in stressful, real‐world situations, emotion regulation coaching might especially enhance EF for autistic children with co‐occurring anxiety.

It is perhaps not surprising that anxiety and ADHD features differentially affected EF training outcomes in our study. While they did not assess formal moderation, Antshel et al. (2011) found that a social skills training was effective for autistic children with co‐occurring anxiety but not for autistic children with ADHD. The focus of the intervention likely impacts which children will find it most helpful, such that for children with ADHD features, some interventions (e.g., social skills training) may be less effective and some (e.g., EF training) may be more effective, because ADHD features may distract children in the former but be directly targeted in the latter. Future research should explore whether children with co‐occurring ADHD would best benefit from EF training before engaging in other interventions that may require them to demonstrate EF skills such as sustained attention in order to benefit (e.g., social skills training).

We found a significant correlation between enhanced N2 incongruent amplitude and reductions in repetitive behaviors, present only within the EF training group. EF deficits in areas such as inhibition and set‐shifting have been thought to underlie increased repetitive behaviors and restricted interests (Mosconi et al., 2009; Schmitt et al., 2017). One theory is that poor cognitive control, including EF, leads to heightened anxiety and/or arousal and, consequently, greater expression of restrictive interests and/or repetitive behaviors as a result of the need to regulate that arousal (Spiker et al., 2012). Faja and Nelson Darling (2019) found that school‐age autistic children who had better inhibition and cognitive flexibility tended to have fewer restricted and repetitive behaviors and interests. Longitudinal evidence also supports this link, with one study demonstrating that preschool EF skills predicted the severity of RRBIs in autistic children 3 years later (Pellicano, 2013). EF training may reduce repetitive behaviors by enhancing the function of brain systems involved in inhibitory control. However, we did not find a significant indirect effect, which would be the strongest evidence of target engagement by demonstrating that EF training improved repetitive behaviors by changing N2 amplitude. Given our sample size, we lacked statistical power to detect a smaller‐than‐medium indirect effect (Fritz & MacKinnon, 2007). In sum, a correlation between N2 and RBS‐R scores within the EF training group but not the waitlist group is not enough to suggest that N2 change causes improvements in repetitive behavior impairment; it is possible that this correlation is not driven by the training but some other unidentified third variable. However, demonstrating correlated change between treatment targets and outcomes is a first step towards learning the mechanisms by which EF training might work. It will be important to examine whether the EF training improves other quality‐of‐life reducing outcomes such as anxiety and whether it reduces only functionally impairing repetitive behaviors, as some RRBIs are thought to be soothing for autistic individuals and a part of their identity. Future work should continue to explore whether EF interventions lead to increased target engagement and clinical outcomes in sequence, which is in line with theories of both complex behavior change and developmental cascades (e.g., Masten & Cicchetti, 2010).

This study has several limitations. First, we did not recruit a sample of autistic children with formally diagnosed co‐occurring disorders. The scales we used, the CBCL ADHD Problems and Anxiety Problems scales, merely measure features and are not proxies for formal clinical diagnosis. CBCL scales have been commonly used, although not formally validated, with autistic samples (Factor et al., 2017; Kaat et al., 2014; Kanne et al., 2009 & McGrew et al., 2007). Further, a quarter to a third of our sample presented with co‐occurring ADHD and anxiety features, respectively, which approaches current prevalence estimates. Second, as a secondary data analysis, we were limited in statistical power by the sample size of the parent RCT, which may have prevented us from detecting a small‐to‐medium mediation of treatment effects. Third, our sample was non‐diverse with regards to race and ethnicity. We and others should prioritize the recruitment of representative samples and the integration of determinants of racial and ethnic inequity into future empirical work. Fourth, and most important, our research questions about EF training moderators and mediators were post‐hoc and highly exploratory in nature. Our findings need to be replicated. Despite this, any research that attempts to understand how and for whom interventions work will strengthen our ability to make a priori hypotheses and invest in adequately powered trials in the future.

In conclusion, this study contributes to our understanding of which autistic children are “ready to benefit” from executive function training. It also provides preliminary evidence of correlated pre‐ to post‐training change in brain‐based EF and behavioral outcomes. Future work should validate neural and proximal behavioral measures of target engagement to learn more about EF intervention efficacy. With further replication, these findings will help us make personalized recommendations to children and teens about which interventions will be most impactful to their ability to succeed and their quality of life.