Investigating functional brain network integrity using a traditional and novel categorical scheme for neurodevelopmental disorders.

BACKGROUND: Current diagnostic systems for neurodevelopmental disorders do not have clear links to underlying neurobiology, limiting their utility in identifying targeted treatments for individuals. Here, we aimed to investigate differences in functional brain network integrity between traditional diagnostic categories (autism spectrum disorder [ASD], attention-deficit/hyperactivity disorder [ADHD], typically developing [TD]) and carefully consider the impact of comorbid ASD and ADHD on functional brain network integrity in a sample adequately powered to detect large effects. We also assess the neurobiological separability of a novel, potential alternative categorical scheme based on behavioral measures of executive function.
METHOD: Five-minute resting-state fMRI data were obtained from 168 children (128 boys, 40 girls) with ASD, ADHD, comorbid ASD and ADHD, and TD children. Independent component analysis and dual regression were used to compute within- and between-network functional connectivity metrics at the individual level.
RESULTS: No significant group differences in within- or between-network functional connectivity were observed between traditional diagnostic categories (ASD, ADHD, TD) even when stratified by comorbidity (ASD + ADHD, ASD, ADHD, TD). Similarly, subgroups classified by executive functioning levels showed no group differences.
CONCLUSIONS: Using clinical diagnosis and behavioral measures of executive function, no differences in functional connectivity were observed among the categories examined. Despite our limited ability to detect small- to medium-sized differences between groups, this work contributes to a growing literature suggesting that traditional diagnostic categories do not define neurobiologically separable groups. Future work is necessary to ascertain the validity of the executive function-based nosology, but current results suggest that nosologies reliant on behavioral data alone may not lead to discovery of neurobiologically distinct categories.

Background

Autism spectrum disorder (ASD) and attention-deficit/hyperactivity disorder (ADHD) are behaviorally heterogeneous, prevalent neurodevelopmental disorders that are currently defined by a symptom-based classification (American Psychiatric Association, 2013) and display few clear links between diagnostic criteria and specific neurobiological alterations (Ameis, 2017; Mueller et al., 2017). These disorders exhibit shared deficits in executive function (Leno et al., 2017) and associated functional brain alterations (Di Martino et al., 2013; Chantiluke et al., 2014), which may be exacerbated by the high rates of comorbidity between ASD and ADHD (Leitner, 2014). These challenges limit the utility of the diagnostic and statistical manual of mental disorders' (DSM-5, American Psychiatric Association, 2013) criteria as predictors of etiology or treatment response (Clark et al., 2017). The NIH has proposed the Research Domain Criteria (RDoC) framework, which instead advocates for revisions to diagnostic systems that define neurobiologically separable categories for mental health disorders (Insel et al., 2010). Following these guidelines, a potential alternative nosology may be developed that is specifically tied to targeted treatments. In this study, we first assess the neurobiological separability of traditional diagnostic categories by evaluating differences in functional brain network integrity between children with ASD, ADHD and typically developing (TD) children. Comorbidity between ASD and ADHD is rarely considered in neuroimaging studies. Here, we explicitly examine functional network connectivity in children with comorbid ASD and ADHD separately from children with ASD (without ADHD), ADHD (without ASD), and TD children. Finally, we assess the neurobiological separability of a novel classification system based on behavioral measures of executive function as an alternative to the DSM-5 classification system.

The DSM-5 defines ASD according to behavioral symptoms of social/communication deficits and restricted and repetitive behaviors; ADHD is defined by either primarily inattentive or hyperactive symptoms, or a combination of both (American Psychiatric Association, 2013). Although there exists a large body of work implicating functional brain network alterations in ASD and ADHD (Ameis, 2017), to date, no biomarkers have been identified to supplement diagnosis of these disorders. Here, we focus on functional connectivity as an important marker for dysfunction in these neurodevelopmental disorders and operationalize neurobiological separability as group differences in functional connectivity in large-scale brain networks important for cognition.

Case-control studies comparing functional connectivity in ASD and ADHD to TD children have produced largely inconsistent results (Mash et al., 2018; Hull et al., 2016), although studies of children with ADHD appear to be more consistent than studies of ASD (Castellanos and Aoki, 2016). Both hyperconnectivity (Uddin et al., 2013; de Celis et al., 2014) and hypoconnectivity (Bos et al., 2014; Yoo et al., 2017) have been reported in children with ADHD and ASD. The neuroscience field is becoming increasingly aware of the lack of reproducible findings across studies, which may be the result of low sample sizes, inflated false-positive rates due to analytic choices, and heterogeneity inherent to groups of interest (Poldrack et al., 2017). In response, the field is calling for a focus on replication studies using adequately powered samples (Fletcher and Grafton, 2013). Here, we aim to resolve the discrepancy in prior studies characterizing brain networks of children with ASD and ADHD by comparing functional network integrity between groups in a large sample of children adequately powered to detect large differences between groups.

Few neuroimaging studies have directly compared functional network connectivity of children with ASD and ADHD, resulting in inconclusive findings of both common and distinct network alterations across case-control studies (Di Martino et al., 2013; Christakou et al., 2013). Further, only two prior functional neuroimaging studies considered the impact of comorbidity on diagnostic group differences by examining an ASD with comorbid ADHD group distinct from non-comorbid ASD and ADHD groups (Di Martino et al., 2013; Chantiluke et al., 2014). These studies demonstrated that children with ASD and comorbid ADHD exhibit functional brain network abnormalities that resemble alterations in children with ASD (without ADHD) and ADHD (without ASD) plus unique abnormalities specific to comorbidity between ASD and ADHD (Di Martino et al., 2013; Chantiluke et al., 2014). These prior results suggest that some brain network findings may be nonspecific across ASD and ADHD, and some may only hold for subgroups within a disorder (e.g., a comorbid group). The current study addresses both of these concerns in the context of two highly prevalent neurodevelopmental disorders. The first aim of this study is to test differences in functional connectivity between children with ASD, ADHD, comorbid ASD and ADHD, and TD children as an index of the neurobiological separability of these categories.

Inconsistent findings and non-specificity of functional network alterations in ASD and ADHD may also be attributed to the heterogeneity characteristic of these disorders. Individual differences among children within a DSM diagnostic category are increasingly recognized across the biological psychiatry field, with recent calls to account for this variability in research studies (Ameis, 2017; Corbett et al., 2009). One possible approach for parsing heterogeneity in these disorders is to define more homogeneous subgroups of children with ASD and ADHD. Based on DSM-5 criteria, there are no currently defined subgroups for ASD (American Psychiatric Association, 2013). Although ADHD has three DSM-defined subgroups (Inattentive, Hyperactive/Impulsive, and Combined), these are currently inadequate to capture the full range of symptoms (Lee et al., 2014) or to predict treatment response (Mueller et al., 2017). In addition to high levels of within-group variability, there is considerable overlap in both phenotypic and biological alterations in ASD and ADHD, one of the most striking similarities being common difficulties in executive function (Corbett et al., 2009). Shared alterations in structural (Dougherty et al., 2016) and functional (Di Martino et al., 2013) neural underpinnings of executive function across ASD and ADHD have likewise been reported. Importantly, Chantiluke et al. (2014) showed that children with “pure” ASD and ADHD exhibited little to no disorder-specific functional alterations during a temporal discounting task, whereas the comorbid group exhibited pronounced functional differences compared with all other groups (Chantiluke et al., 2014). This finding highlights the inadequacy of defining subgroups solely within a single disorder, and instead calls for the definition of subgroups that may cut across disorders.

To this end, we recently leveraged both theoretical (focusing on executive functions) and data-driven (using a subgrouping method called latent profile analysis) computational psychiatry approaches to develop a possible alternative categorization of neurodevelopmental disorders. In line with the RDoC framework, we focused on a neurocognitive construct that is linked to underlying neurobiology— executive function— to investigate mental health disorders (Insel et al., 2010). We demonstrated that behavioral measures of executive function can be used to define subgroups of children across various diagnostic groups: ASD, ADHD, comorbid ASD and ADHD, and TD children (Dajani et al., 2016). Three subgroups emerged, with “above average,” “average,” and “impaired” executive functions, which crossed traditional diagnostic boundaries. To achieve the ultimate goal of defining diagnostic categories that map onto neurobiologically distinct groups, here, we assess the neurobiological separability of the current DSM categories of ASD and ADHD, in addition to considering comorbidity, using functional network connectivity indices. Moreover, we assess the neurobiological separability of an alternative categorical scheme based on executive function subgroups (i.e., “above average,” “average,” and “impaired” subgroups). Specifically, we evaluated the separability of groups in within- and between-network functional connectivity of three major intrinsic connectivity networks (ICNs) important for cognition (Menon and Uddin, 2010): the frontoparietal network (FPN), salience network (SN), and DMN (Uddin, 2015).

We predicted that children with ADHD would exhibit reduced DMN connectivity and stronger DMN-FPN and DMN-SN coupling (Castellanos and Aoki, 2016) compared with TD children. We expected that children with ASD would exhibit hyperconnectivity within FPN, DMN, and SN (Uddin et al., 2013; Nomi and Uddin, 2015) compared with TD children. Inconsistent findings comparing ASD, ADHD, and ASD with comorbid ADHD groups precludes establishing well-founded a priori hypotheses specific to the comorbid ASD and ADHD group. Further, we anticipated that there would be a parametric increase of functional connectivity within the FPN and SN across executive function subgroups, such that the “impaired” subgroup would exhibit the lowest functional connectivity and the “above average” group would exhibit the highest functional connectivity (Reineberg et al., 2015).

Methods and materials

Participants

Participants aged 8 to 13 years (N = 168) included a subset of children used in our previous study (Dajani et al., 2016). Written informed consent was obtained from all legal guardians and written assent was obtained from all children. All procedures were approved by the Institutional Review Board at the Johns Hopkins School of Medicine and all methods were carried out in accordance with the approved guidelines.

Diagnostic and IQ measures

Community diagnoses of ASD were confirmed with the Autism Diagnostic Observation Schedule (ADOS-G, Lord et al., 2000, or ADOS-2, Lord et al., 2012, based on study enrollment date) and Autism Diagnostic Interview-Revised (ADI-R, Rutter et al., 2005). The Diagnostic Interview for Children and Adolescents IV (Reich et al., 1997) was used to confirm community ADHD diagnoses, determine whether children with ASD had comorbid ADHD, and for exclusionary purposes. Community diagnoses of ADHD were also confirmed with the Conners' Parent Rating Scales (CPRS-R:L, Conners, 1997, or CPRS-3, Conners, 2008, based on study enrollment date) and the ADHD Rating Scale IV, Home version (DuPaul et al., 1998) (Table 1). Executive functions used for the latent profile analysis were measured primarily with a parent-report (eight subscales of the BRIEF, Gioia et al., 2000), in addition to two laboratory measures, the Statue subscale of the NEPSY-II (Korkman et al., 2007) and the backward digit span of the WISC-IV. See supplementary information for more details.

Table 1

Diagnostic information.

	Primary diagnosis
	TD	ADHD	ASD
	n = 59	n = 53	n = 56
Secondary Dx (present), No. (%)	2 (3.2)	24 (44.4)	43 (76.8)
ADHD	0	–	34 (60.7)
Oppositional defiant	0	21 (39.6)	14 (25.0)
Simple Phobia	1 (1.7)	6 (11.3)	14 (25.0)
Generalized anxiety	0	0	7 (12.5)
Obsessive compulsive	0	0	5 (8.9)
Separation Anxiety	0	0	1 (1.8)
Dysthymia	0	0	1 (1.8)
CD, MDD, Mania, PD, Somata	0	0	0
ADHD measuresb, M (SD)
Conners Hyper/Impulsivec	48 (5.51)	71 (12.54)	66 (10.85)
Conners Inattentionc	45 (4.71)	73 (8.2)	(.)
Conners 3 Hyper/Impulsivec	45 (6.95)	76 (11.74)	80 (9.55)
Conners 3 Inattentionc	43 (8.69)	77 (9.57)	84 (4.85)
ADHD Hyperactivityd	0.22 (0.59) [0–3]	4.00 (2.89) [0–9]	3.76 (2.23) [0–8]
ADHD Inattentiond	0.20 (0.58) [0–3]	7.00 (1.91) [2–9]	5.76 (2.75) [0–9]
ASD measurese, M (SD)
ADI-R Af	–	–	20.57 (5.66)
ADI-R Bg	–	–	15.67 (4.70)
ADI-R Ch	–	–	6.26 (2.15)
ADOS-2 Social Affect	–	–	7.59 (3.28)
ADOS-2 RRBh	–	–	4.14 (1.73)
ADOS-G CSi	–	–	11.97 (3.20)
ADOS-G RRBj	–	–	3.00 (1.67)

Data is presented for full sample of eligible participants, N = 168.

Conduct Disorder, Major Depressive Disorder, Mania or Hypomania, Panic Disorder, and Somatization Disorder.

Reporting n = 130 for Conners (1997) and n = 32 with Conners-3 (2008); Conners missing data: n = 6; ADHD Home Rating Scale IV missing data: n = 4.

Conners Parent Rating Scales T-scores.

ADHD Home Rating Scale IV symptom counts.

ADI missing data: n = 2 ASD participants; Reporting n = 22 with ADOS-2 data and n = 34 with ADOS-G.

Autism Diagnostic Interview-Revised reciprocal social interaction.

Verbal communication.

Restricted and repetitive behaviors.

Communication and social interaction.

Stereotyped behaviors and restricted interests.

Subsamples

To compare the integrity of ICNs between groups delineated by primary diagnosis (ASD, ADHD, TD), three equally sized diagnostic groups (N = 129, three groups of n = 43) were randomly selected from the larger sample of 168 participants in order to produce unbiased group ICA results (hereafter, ‘Diagnostic group sample’). This design was adequately powered to detect a large effect for a one-way ANOVA (alpha = 0.05, f = 0.4, power = 99%). We were underpowered to detect medium (power = 71%) and small effects (power = 16%). Diagnostic groups were delineated by primary diagnosis; individuals may have had additional secondary diagnoses. For example, the “ASD” group included individuals with comorbid psychiatric diagnoses, including ADHD (n = 26, 61%). Some children in the “ADHD” group had comorbid psychiatric diagnoses, but this did not include comorbid ASD, given that ASD is commonly considered a primary and not secondary diagnosis (see Table 1 for more details). Diagnostic groups did not differ in age, sex, handedness, FSIQ, or head motion (mean framewise displacement [FD (Power et al., 2012)], translational and rotational motion, Table 2).

Table 2

‘Diagnostic group sample’ demographics.

	Diagnostic groups			P value
	TD	ADHD	ASD
	n = 43	n = 43	n = 43
Sex	31 M/12 F	31 M/12 F	34 M/9 F	0.69
Age	10.50 (1.02)	10.03 (1.25)	10.37 (1.45)	0.23
Range	[8.00–12.58]	[8.00–12.42]	[8.17–12.92]
Racea	6, 3, 7, 27	7, 0, 8, 28	1, 0, 4, 37	0.05
Ethnicityb, No.Hispanic/Latino	2	4	3	0.07
FSIQc	112.4 (11.53)	110.7 (11.33)	107.49 (10.16)	0.12
Range	[90–145]	[93–136]	[90–131]
Motiond	0.23 (0.12)	0.26 (0.14)	0.26 (0.15)	0.49
Handednesse, No. L,R	4, 1, 37	5, 0, 38	3, 0, 39	0.64

Numbers for each of the following racial categories presented in the following order: African American, Asian, Biracial, Caucasian.

FSIQ: WISC-IV full-scale IQ.

Mean framewise displacement.

Number of children with left, ambidextrous, right, handedness.

To address the issue of comorbidity in assessing diagnostic group differences, an additional group analysis was performed. Diagnostic groups were categorized as TD, ADHD, ASD (without ADHD), and ASD + ADHD (ASD with comorbid ADHD). Due to sample size limitations, group size for this analysis was limited to 22 (N = 88, ‘Diagnostic comorbid sample’), but this design was still adequately powered to detect a large effect (alpha = 0.05, f = 0.4, power = 88%). It was not adequately powered to detect medium (46%) or small effects (10%). Groups did not differ in age, sex, handedness, or head motion (Table S1). Due to sample size limitations, we were unable to match the diagnostic comorbid samples on FSIQ (p < .001).

A subset of the 168 participants eligible for this study was generated to ensure equal group sizes for each executive function (EF) subgroup (‘EF subgroup sample’, N = 129, three groups of n = 43). Similar to the ‘Diagnostic group sample’, this design was adequately powered to detect a large effect (alpha = 0.05, f = 0.4, power = 99%). The ‘EF subgroup sample’ was representative of the Dajani et al. (2016) sample in EF scores, age, FSIQ and the distribution of diagnostic categories within each EF subgroup. EF subgroups did not differ in age, sex, handedness or mean head motion, but did differ on FSIQ (p < .001) and reached a near-significant difference in mean FD for the raw data (p = .06, Table S2). Following preprocessing, there were no group differences in mean FD (p = .36, Fig. 1). The difference in FSIQ was expected given the subgroups were delineated based on EF, which tends to be highly correlated with IQ metrics (Arffa, 2007).

Fig. 1

Effect of rigorous motion correction.

Mean FD is in mm, displayed on raw data and data after preprocessing with ICA-AROMA.

Data acquisition

Resting state fMRI (rs-fMRI) data were acquired for participants on a Phillips 3 T scanner using an 8-channel head coil (TR = 2.5 s, flip angle = 70°, sensitive encoding acceleration factor = 2, 3 mm slices, voxel size = 2.7 × 2.7 × 3 mm). Most participants had a 156-volume dataset, but a subset had a shorter acquisition time of 128 volumes (156 volumes: n = 114, 128 volumes: n = 15 in the ‘EF subgroup sample’). High-resolution T1-weighted scans were also acquired to facilitate registration of the functional image to standard space (TR = 8.0 ms, TE = 3.7 ms, 1 mm isotropic voxels). Participants were asked to withhold stimulant medication (e.g., Adderall) the day before and on the day of MRI scanning, similar to prior neuroimaging studies comparing children with ASD and ADHD (Di Martino et al., 2013; Dennis et al., 2014). Non-stimulant medications were continued as prescribed (e.g., antidepressants, allergy medication, see Table S3 for detailed medication status information).

Experimental design and statistical analysis

Preprocessing

Participants with excessive in-scanner motion (> 5 mm of absolute maximum motion) were excluded from the study. To reduce motion artifacts, volumes in raw rs-fMRI data that contained motion spikes (>3 mm or degrees) at the beginning or end of the scan were deleted for 5 participants (volumes deleted were restricted to the first or last volumes of the scan to preserve the temporal continuity of the time series).

Standard preprocessing procedures included the following: First, structural images were brain extracted using FSL's BET tool. Using FEAT, fMRI data underwent motion correction, 4D intensity normalization, smoothing with a 6 mm kernel, and estimation of linear and non-linear warping parameters to normalize to the MNI152 2 mm template. Following the removal of motion-related signals in native space using ICA-AROMA (see supplemental information), warping parameters were applied to denoised functional images.

Dual regression analyses employed in this study require all participants to have equal rs-fMRI scan lengths, therefore all participants' rs-fMRI data were truncated to the shortest participant's scan length by removing volumes at the end of the scan, resulting in all scans including 121 volumes (5 min of data). Standard preprocessing was conducted in FSL 5.0.9. In addition, a state-of-the-art ICA-based denoising procedure (ICA-AROMA, Pruim et al., 2015) was used to remove motion-related artifacts in native space (see supplemental information).

Group ICA

We ran a group ICA using the FSL MELODIC v3.14 toolbox (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/MELODIC) with temporal concatenation to identify common spatial patterns across participants. Five components of interest were manually identified from the group ICA by two of the authors (DRD and LQU): right FPN (Smith et al., 2009), left FPN (Smith et al., 2009), SN (Seeley et al., 2007), and anterior and posterior DMN (Smith et al., 2009). Specifically, right and left FPN components were identified by the presence of lateralized dorsolateral PFC, ventrolateral PFC and lateral parietal cortices (Seeley et al., 2007). The SN was identified by the presence of anterior insula and dorsal ACC (Seeley et al., 2007). In children, the DMN tends to decompose into anterior and posterior components (Uddin et al., 2013; Nomi and Uddin, 2015). The anterior DMN was identified by the primary presence of ventromedial PFC and the posterior DMN was identified by the precuneus and posterior cingulate cortex (Smith et al., 2009). A separate group ICA model was run for each of the three subsamples (Fig. 2). For example, for the ‘diagnostic group’ sample, a group ICA was run including only participants in the ‘diagnostic group’ sample (n = 129). Separate group ICA models were processed for the ‘diagnostic comorbid’ and ‘EF subgroup’ samples (Fig. 3). This procedure was undertaken to ensure that group ICA maps represented the exact set of participants included in any one analysis, thereby improving the accuracy of individual-level estimates of spatial and temporal maps derived using dual regression. Networks were qualitatively highly similar across subsamples, as determined by visual inspection (Fig. S1).

Fig. 2

Study methods. Three subgrouping schemes were evaluated for differences in within- and between-network functional connectivity of frontoparietal, salience, and default-mode networks. For each subsample, distinct group ICA, dual regression, and permutation testing was performed.

Fig. 3

Fifteen-component ICA results for the ‘EF subgroup’ sample.

Components E (posterior DMN), F (L FPN), H (anterior DMN), I (R FPN), and K (SN) were used to assess group differences in network connectivity. One artifactual component emerged (component J).

Dual regression

Dual regression is a reliable technique which allows for the identification of group differences in the spatial and temporal features of ICNs common to the entire sample (Filippini et al., 2009; Zuo et al., 2010). Using FSL's dual regression command (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/DualRegression/UserGuide), individual-level spatial and temporal maps were constructed for each of the five components of interest generated from the group ICA. To test group differences in within-network connectivity, the normalized individual-level spatial maps were subjected to permutation testing using FSL's randomise tool for each of the five components of interest (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/Randomise/UserGuide, 5000 permutations). To test for group differences in between-network connectivity, the FSLNets package was implemented in MATLAB (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FSLNets). For each subsample, F-tests were conducted to ascertain the presence of group differences in within-network and between-network connectivity strength. Significance was determined using a threshold-free cluster enhancement of p < .05 (FWE-corrected) for both within- and between-network analyses, as in previous studies (Uddin et al., 2013; Nomi and Uddin, 2015).

Results

Effect of rigorous motion correction

A 2 × 1 repeated-measures ANOVA was used to assess whether motion (indexed by mean FD, Power et al., 2012) decreased as a function of preprocessing with ICA-AROMA by comparing mean FD for raw and preprocessed data. The ANOVA demonstrated a significant decrease in mean FD following preprocessing, F(1, 332) = 25.36, p < .001 (raw: M = 0.22 SD = 0.14, preprocessed: M = 0.04 SD = 0.02, Fig. 1).

Comparisons between diagnostic groups

No within- or between-network connectivity differences were found when comparing diagnostic groups (TD, ADHD, ASD; within-network: FWE-corrected p's > 0.26, between-network: FWE-corrected p's > 0.37). When comparing diagnostic groups and statistically controlling for the effects of EF subgroup, F-tests remained non-significant (within-network: FWE-corrected p's > 0.20, between-network: FWE-corrected p's > 0.11).

Comparisons between comorbid diagnostic groups

When diagnostic groups were broken down according to comorbidity (TD, ADHD [without ASD], ASD [without ADHD], ASD + ADHD), F-tests revealed no group differences in within-network (FWE-corrected p's > 0.30) or between-network connectivity (FWE-corrected p's > 0.14).

Comparisons between EF subgroups

Group differences in network connectivity strength were assessed across five ICNs between children with “above average,” “average,” or “impaired” EF. F-tests revealed no significant differences between EF subgroups in within-network nor between-network connectivity (within-network: FWE-corrected p's > 0.24, between-network: FWE-corrected p's > 0.29). Likewise, after controlling for diagnostic status, no differences in connectivity emerged (within-network: FWE-corrected p's > 0.25, between-network: FWE-corrected p's > 0.47).

Discussion

With the recent exponential increase in computational power and growing awareness of the limitations of current psychiatric diagnostic systems, there have been numerous calls to leverage the strengths of computational psychiatry approaches to develop a more parsimonious and neurobiologically valid nosology of mental health disorders (Huys et al., 2016; Chekroud et al., 2017). This study is the first to use ICA dual regression, a reliable data-driven approach to investigate differences in both within- and between-network functional connectivity between clinical groups, to assess the neurobiological separability of a traditional diagnostic classification system and a novel subgrouping system based on behavioral measures of executive function, while rigorously correcting for motion-related artifacts that are pervasive in pediatric psychiatric populations. We also carefully consider the impact of comorbidity between ASD and ADHD on brain network functional connectivity. Contrary to previous reports, results indicate that the current DSM categories for ASD and ADHD classify children into groups exhibiting negligible functional connectivity differences of major cognitive networks. Unpredictably, executive function subgroups displayed limited differences in functional connectivity of major cognitive networks, suggesting that a categorical nosology based solely on behavioral features may not map onto differences in functional connectivity.

Comparisons between traditional diagnostic categories (i.e., ASD, ADHD, and TD), which are based on observable symptoms according to DSM criteria, showed no differences in within- or between-network functional connectivity of the FPN, SN or DMN. Given several possible reasons for these null findings, we argue that TD, ASD and ADHD groups do not exhibit large differences in functional connectivity of major cognitive networks. We provide rationale demonstrating the improbability that null results were due to past functional connectivity findings being method-specific. Instead, previously published findings may simply represent false positives due to low sample size. Additionally, heterogeneity in sample composition across studies may have led to positive, but inconsistent, results that are ultimately not generalizable. These results do not preclude the existence of small- to medium-sized differences in functional connectivity between groups, but is unclear whether such differences can contribute to the identification of sensitive and specific diagnostic biomarkers for ASD and ADHD.

The results presented here contradict past findings of significant differences in functional connectivity between children with ASD, ADHD and TD children. It is unlikely that previous results demonstrating group differences are method-specific and do not extend to dual regression ICA-based analyses. On the contrary, multiple prior studies comparing TD children and children with ASD and ADHD using ICA-derived networks report connectivity differences between diagnostic groups. Specifically, studies of individuals with ADHD report reduced segregation of DMN-FPN networks in children, adolescents, and adults (Kessler et al., 2014; Sudre et al., 2017), but results from within-network connectivity studies are less coherent. For example, both hyperconnectivity (de Celis et al., 2014) and hypoconnectivity (Yoo et al., 2017) have been reported in ADHD for the FPN and the DMN. The ASD literature is similarly inconsistent, even when limited to studies using dual regression ICA. For example, there have been reports of both within-network hyperconnectivity (Uddin et al., 2013; Nomi and Uddin, 2015) and hypoconnectivity of the FPN (Bos et al., 2014) and DMN (Bos et al., 2014; Washington et al., 2014). Of note, our null results are in line with a previous study of high-functioning adults with ASD, which identified no meaningful differences in connectivity of any large-scale brain networks between ASD and control groups using ICA dual regression (Tyszka et al., 2014). Although results are not consistent across studies, they clearly demonstrate that positive results are possible when using ICA dual-regression to investigate network integrity in children with ASD and ADHD, ruling out the possibility that previously reported network connectivity differences in ASD and ADHD are method-specific.

Previously published findings demonstrating group differences in functional connectivity between TD, ASD, and ADHD groups may simply represent false positives. Accordingly, our null results may be explained by the improved methodology used in this study. Past studies using ICA dual regression to compare children with ASD or ADHD to TD children used small sample sizes (n = 20–26 per group) (Uddin et al., 2013; Yoo et al., 2017; Nomi and Uddin, 2015; Washington et al., 2014; Choi et al., 2013). It is well established that low sample sizes contribute to reduced power to detect true results. However, it is less appreciated that positive results reported from underpowered studies also have a greater likelihood of being false (Poldrack et al., 2017; Ioannidis, 2005). Further, the exorbitant number of researcher degrees of freedom available in fMRI analyses, including choice of preprocessing pipeline, analysis method, group matching procedures, thresholding procedures, and multiple comparison correction lead to inflated false-positive rates (Poldrack et al., 2017). Here, we nearly double the sample size of past studies and arrive at null results, which in combination with the fact that past results are inconsistent across studies, suggests that some previous results could be attributed to false positives (Ioannidis, 2005). In addition, we employ a rigorous motion correction procedure not used in any previous study of functional connectivity differences in ASD and ADHD. ICA-AROMA is a denoising approach that reduces the likelihood of group differences emerging solely due to differences in in-scanner motion. The strength of long-range functional connectivity is underestimated in cases of increased head motion (Van Dijk et al., 2012), which is rife in studies of youth with neurodevelopmental disorders (Yerys et al., 2009). Previous studies of within-network connectivity of the DMN in both children with ASD (Washington et al., 2014; Starck et al., 2013) and ADHD (Castellanos and Aoki, 2016) that did not use ICA-AROMA may have been influenced by residual effects of motion. Taken together, these results suggest that findings of reduced long-range connectivity in ASD and ADHD may be a byproduct of increased motion artifacts in these clinical groups.

An additional factor that may contribute to inconsistency in results across studies is heterogeneity in sample composition, including variability in IQ, comorbidity or medication status, which may have led to idiosyncratic results that are sample-specific and thus not generalizable to the greater clinical population of children with ASD or ADHD. Research into the impact of IQ levels on neuroimaging results are only beginning to emerge for the ASD literature (Reiter et al., 2019) and are virtually absent in the ADHD literature, where it is commonplace to forgo group matching on IQ (Choi et al., 2013). Similar trends are evident for understanding the impact of medication status on case-control neuroimaging studies (Hull et al., 2016; Rubia, 2018; Rubia et al., 2014). In general, comorbidity tends to correlate with greater symptom severity and disability (Lai et al., 2014). While it is possible that a particular moderating factor, or precise mixture of moderating factors, may produce reliable case-control functional connectivity differences in children, as of yet, there is not enough literature to identify them. Even if we were to identify these moderating factors, if a particular set of results only hold for a narrow subset of individuals, it may not be particularly meaningful and would not aid in progress towards the ultimate goal to identify a generalizable diagnostic biomarker of ASD and/or ADHD.

Two factors may be contributing to the limited separability of ASD and ADHD categories based on functional connectivity indices: high within-category heterogeneity and high between-category similarity. There is considerable evidence for the existence of subgroups within ASD (Feczko et al., 2018) and ADHD diagnostic categories (Fair et al., 2012a), which are often delineated with neuropsychological measures. Recent evidence has also emerged for neurobiological subgroups of ADHD based on network connectivity of the reward system (Costa Dias et al., 2015) and other large-scale brain networks (Gates et al., 2014). In addition, there is substantial overlap in ASD and ADHD categories in symptomatology (e.g., social skills deficits), behavioral domains beyond symptoms (e.g., executive dysfunction), and genetic factors (Rommelse et al., 2010) in addition to high rates of comorbidity between ASD and ADHD. Supporting the notion that ASD and ADHD categories demonstrate limited neurobiological validity, a recent, large-scale resting-state fMRI study reported that data-driven functional connectivity features were not uniquely associated with TD, ASD or ADHD categories, but instead were dimensionally shared to greater or lesser degrees with each of these groups (Kernbach et al., 2018). Evidence for both high within-group heterogeneity and between-group overlap reduces the validity of current DSM categories for ASD and ADHD, and thus may explain our lack of dissociability between disorders based on functional connectivity metrics.

Few neuroimaging studies consider the impact of comorbid ASD and ADHD on case-control findings, and instead rely on the primary diagnosis of children to classify patients. This leads to inherent heterogeneity in clinical groups studied, likely leading to inconsistencies in findings across the neuroimaging literature. The current study is one of the first functional neuroimaging studies to directly compare children with ASD, ADHD, comorbid ASD and ADHD, and TD children, and the first to do so using ICA dual regression combined with ICA-AROMA. Here, we do not find statistically significant differences between ASD and ADHD groups, even when accounting for comorbidity. Although these results suggest that there are not large differences in functional connectivity between ASD, ADHD, and comorbid groups, it does not rule out the possibility that medium- or small- effects were missed. Ideally, this analysis should be replicated in future studies with a larger sample size to confirm whether group differences in functional connectivity exist between children with comorbid ASD and ADHD and other clinical and typically developing populations (e.g., n = 45 per subgroup to detect medium effects).

There do exist a small number of neuroimaging studies that were adequately powered to detect medium- to large- effects who report differences in functional connectivity between TD children and children with ASD and ADHD (Di Martino et al., 2014a; Fair et al., 2012b). Here, we were adequately powered to detect large differences between diagnostic groups in between- and within-network functional connectivity. The present study had higher power to detect true effects than many prior studies, but still does not achieve the ideal power to detect medium or small effect sizes, therefore, we may have missed medium- or small-sized effects between groups in functional connectivity. We also acknowledge that ASD, ADHD, and TD groups may be neurobiologically separable based on other neuroimaging metrics such as white matter integrity and/or structural topology. But, years of research attempting to identify diagnostic biomarkers for these disorders have not been fruitful (Uddin et al., 2017), which weakens our confidence that medium- or small-sized differences between diagnostic groups will lead to the identification of sensitive and specific diagnostic biomarkers for ASD and ADHD.

Amidst the increasing sophistication of machine learning algorithms, increases in computational power and availability of large-scale datasets, researchers have yet to identify reliable diagnostic biomarkers for ASD and ADHD based on either functional or structural neuroimaging features (Uddin et al., 2017). A recent study that took advantage of both structural and functional neuroimaging data reported modest, but not clinically useful, accuracy of a complex machine learning algorithm attempting to classify individuals with ASD or ADHD versus TD individuals (Sen et al., 2018). Further, a recent review of classification studies of ASD and ADHD using neuroimaging techniques suggested that heterogeneity within and between disorders may be limiting researchers' (and machines') ability to distinguish disorders based on neurobiology (Uddin et al., 2017). In sum, it is not likely that neuroimaging features, whether they are functional or structural, will identify reproducible diagnostic biomarkers for ASD and ADHD. The present results and previous research comport with the idea that these traditional diagnostic categories are not neurobiologically separable, which contributes to their poor prediction of treatment response or prognosis (Cuthbert, 2014).

As an alternative nosology to DSM-5 classifications, we tested whether there existed differences in functional connectivity between “above average,” “average” and “impaired” EF subgroups (Dajani et al., 2016). Contrary to our hypotheses, we observed no differences in functional connectivity between these subgroups, suggesting that EF-defined subgroups are not distinct based on functional connectivity of networks important for cognition. This null result does not preclude the possibility that EF groups may be separable based on functional networks not examined here (e.g., sensory or subcortical networks) or structural neuroimaging metrics. Another possible explanation for this null result is that EF in children may be best assessed dimensionally rather than categorically (Barch, 2017). The primary goal of this study was to assess the validity of the traditional categorical nosology (i.e., the DSM-5) and an alternative categorical nosology (i.e., EF subgroups). In characterizing neurodevelopmental and psychiatric disorders, recent studies have revealed the importance of testing whether categorical, dimensional, and hybrid categorical-dimensional models best describe psychiatric disorders (Barch, 2017). Future studies would benefit from testing both symptom-based and executive function-based dimensional models to determine whether these may better characterize neurobiological differences between children.

Additionally, our results may have been influenced by our choice of measures for EF subgrouping, which were primarily based on a parent-report of EF symptoms (i.e., the BRIEF (Gioia et al., 2000)). We chose to focus on a parent-report of EF symptoms because of the strong psychometric properties of the measure and ease of obtaining the information in a clinical setting. Although using a parent-report of observable EF clearly has easily translatable implications for clinical practice, which is a current challenge for the field (Paulus et al., 2016), the choice to focus on a parent-report of behavior to characterize psychiatric disorders may be many steps removed from underlying large-scale brain network integrity. This leads to a rather simple explanation for these results – that behavior does not map one-to-one to functional network integrity.

There are numerous lines of evidence to suggest that brain-behavior relationships are not simply one-to-one (Pessoa, 2014). Varying types of functional network miswirings across development may manifest as a singular phenotype (Di Martino et al., 2014b), suggesting that distinct brain abnormalities may appear behaviorally as the same neurodevelopmental disorder. Likewise, disparate genetic etiologies may lead to similar behavioral profiles (Dougherty et al., 2016; Pelphrey, 2017). Diagnostic categories should necessarily define neurobiologically homogeneous groups to allow for the development of targeted treatments specific to a neurobiological signature of the disorder. This suggests that advances in mental health research necessarily rely on characterizing underlying neurobiological mechanisms of pathophysiology, and that psychiatry may be fundamentally limited as long as assessments are limited to observable phenomena (Pine, 2017).

Applying principles of computational psychiatry to clinical research has the potential to transform the mental health field from the current trial-and-error choice of treatments towards precision medicine (Haker et al., 2016). Current psychiatric diagnostic systems rely on observable behaviors to classify disease, with unknown links to underlying neurobiology (Huys et al., 2016). Biomarkers, on the other hand, may provide information that may be able to stratify current diagnostic categories or replace symptom-based classification systems completely. Here, we aimed to leverage the strengths of computational psychiatry methodology to propose an alternative categorical scheme based on behavioral measures of executive function, which we predicted would lead to distinct subgroups based on functional network connectivity. Contrary to predictions, we found that EF subgroups could not be distinguished based on within- or between-network connectivity metrics of major cognitive networks, suggesting that nosologies reliant on behavioral data alone may not lead to discovery of neurobiologically distinct categories, limiting their utility in predicting prognosis and efficacious treatments.

Limitations

Although the present study had numerous strengths including a larger sample than previous similar studies, the results reported here should be considered in light of several limitations. Although our sample was adequately powered to detect large group differences in functional connectivity, it is possible that the length of the timeseries of individuals' resting-state data (5 min) was not long enough to generate reliable indices of functional connectivity (Birn et al., 2013), possibly leading to null differences between groups. Children in the clinical samples had various psychiatric comorbidities aside from ASD and ADHD (most commonly, oppositional defiant disorder [ODD]). While this is expected for children with ADHD given the high rates of comorbidity with ODD (Angold et al., 1999), it may have introduced additional confounds that were not taken into account in this study. Future studies may consider the impact of different types and number of comorbid disorders on brain network integrity. One alternative explanation for our finding of no group differences in functional network integrity between EF subgroups is that another RDoC domain, such as social communication, may be better suited for developing an alternative nosology for children with neurodevelopmental disorders. In an effort to identify the most fruitful behavioral indices for biomarker identification, future studies may employ large-scale multivariate analyses, such as canonical correlation analysis (Xia et al., 2018), to identify the behavioral metrics that are correlated with neuroimaging metrics.

Conclusions

We present findings that traditional diagnostic categories of ASD and ADHD could not be distinguished from one another or from TD children based on within- and between-network functional connectivity of three major cognitive networks: the frontoparietal, salience, and default-mode networks. Likewise, EF subgroups did not reflect distinct subgroups based on functional connectivity. Our results suggest that ASD and ADHD categories may not be neurobiologically distinct based on functional connectivity. Previous reports of functional connectivity differences between groups may represent false positives, and it is essential that future studies include adequately powered samples to decrease this likelihood. In the context of the broader diagnostic neuroimaging biomarker literature, results suggest that there is limited validity for a categorical diagnostic scheme for neurodevelopmental disorders. Accordingly, future studies may seek to improve current diagnostic schemes by employing large-scale multivariate analyses applied to symptom measures and neurobiological variables to identify dimensional biomarkers of neuropsychiatric disease.