Quantifying patterns of brain activity: Distinguishing unaffected siblings from participants with ADHD and healthy individuals.

Attention-deficit/hyperactivity disorder (ADHD) is one of the most prevalent and heritable psychiatric disorders. While previous studies have focussed on mapping focal or connectivity differences at the group level, the present study employed pattern recognition to quantify group separation between unaffected siblings, participants with ADHD, and healthy controls on the basis of spatially distributed brain activations. This was achieved using an fMRI-adapted version of the Stop-Signal Task in a sample of 103 unaffected siblings, 184 participants with ADHD, and 128 healthy controls. We used activation maps derived from three task regressors as features in our analyses employing a Gaussian process classifier. We showed that unaffected siblings could be distinguished from participants with ADHD (area under the receiver operating characteristic curve (AUC) = 0.65, p = 0.002, 95% Modified Wald CI: 0.59-0.71 AUC) and healthy controls (AUC = 0.59, p = 0.030, 95% Modified Wald CI: 0.52-0.66 AUC), although the latter did not survive correction for multiple comparisons. Further, participants with ADHD could be distinguished from healthy controls (AUC = 0.64, p = 0.001, 95% Modified Wald CI: 0.58-0.70 AUC). Altogether the present results characterise a pattern of frontolateral, superior temporal and inferior parietal expansion that is associated with risk for ADHD. Unaffected siblings show differences primarily in frontolateral regions. This provides evidence for a neural profile shared between participants with ADHD and their healthy siblings.

Introduction

Attention-deficit/hyperactivity disorder (ADHD) is one of the most prevalent (Polanczyk et al., 2007, Simon et al., 2009) and heritable psychiatric disorders (Faraone and Biederman, 2005). Heritability estimates are around 75%, and relatives of participants with ADHD have about 25% risk to have ADHD themselves which is about four times higher than the population rate (Biederman et al., 1990). While unaffected siblings share some of the biological risk for disease with their affected siblings, they do not express this risk symptomatically. Unaffected siblings of participants with ADHD have shown patterns of cognitive and neural functioning intermediate to those observed in affected siblings and healthy controls (Durston et al., 2006, Greven et al., 2015, Slaats-Willemse et al., 2003) Specifically, unaffected siblings of participants with ADHD have shown impairments in response inhibition, as evidenced in impaired performance on cognitive tasks, as well as brain abnormalities in structures subserving response inhibition. Therefore, this group of individuals represents an interesting study population in exploring disorder mechanisms and a complex group for disorder classification as preformed in this study. Response inhibition deficits belong to the most prevalent deficits observed in children and adolescents with ADHD (Barkley, 1999, Castellanos et al., 2006, Hart et al., 2014a, Slaats-Willemse et al., 2003, van Rooij et al., 2015b). To probe deficits in response inhibition in the current study, we chose the Stop-Signal Task functional magnetic resonance imaging (fMRI) paradigm. This experimental paradigm has previously shown reductions in task related brain (de)activations in participants with ADHD compared to healthy controls (Hart et al., 2014a, Janssen et al., 2015, Lipszyc and Schachar, 2010, van Rooij et al., 2015b).

In contrast to studies that examined focal or connectivity differences between unaffected siblings and participants with ADHD (van Rooij et al., 2015a, van Rooij et al., 2015b), we quantified group separation on the basis of spatially distributed patterns of activity across the brain, which provides a unified measure of group separation that is more representative of the overall pattern of brain activity than any individual region. Pattern recognition is ideal for this purpose and aims to extract regularities in data, which can be used to predict group membership (Hastie et al., 2009). Early pattern recognition studies aimed to show that participants with ADHD could be distinguished from healthy controls based on different MRI modalities (Hart et al., 2014a, Hart et al., 2014b, Igual et al., 2012, Lim et al., 2013, Wang et al., 2011). These studies were usually small in size and the literature tends to show reduced classification performance with increased sample size (Wolfers et al., 2015). Larger studies capture more of the inherent heterogeneity of ADHD, in terms of its symptomatology and pathophysiology. Therefore, those studies are more indicative for the predictability of ADHD in clinical settings, as a heterogeneous group of patients approach clinics to seek treatment.

In studies on unaffected siblings of schizophrenia and autism spectrum disorders, researchers used neural patterns to distinguish siblings from their respective patient group and healthy controls (Fan et al., 2008, Segovia et al., 2014, Yu et al., 2013). However, until now no pattern recognition study has investigated unaffected siblings of participants with ADHD. In the present study we sought to: (i) precisely quantify the group separation between unaffected siblings, participants with ADHD and healthy controls in a large sample that accurately reflects the range of variation in the disease phenotype and (ii) map the nature of these differences to identify response inhibition related activation patterns, that underlay the shared genetic load between unaffected siblings and participants with ADHD. The present study is the largest study employing pattern recognition to investigate unaffected siblings of participants with ADHD, using a hallmark deficit of ADHD as biomarker, response inhibition (Barkley, 1999, Castellanos et al., 2006, Hart et al., 2014a, Slaats-Willemse et al., 2003, van Rooij et al., 2015b).

Methods

Participants

We used data from the NeuroIMAGE project, a large longitudinal clinical cohort consisting of individuals tested at two different sites in The Netherlands, the Vrije Universiteit in Amsterdam and the Donders Centre for Cognitive Neuroimaging in Nijmegen. We selected all individuals who performed the Stop-Signal Task. ADHD diagnosis was based on K-SADS (Birmaher et al., 2010) structured psychiatric interviews and Conners' questionnaires (Conners et al., 1998). The total sample consisted of 184 participants with ADHD, 103 unaffected siblings, and 128 healthy controls (Table 1). This sample is similar to the sample detailed in our previous publication (van Rooij et al., 2015b), with the exception that the current study excluded subjects if there was an inconsistent diagnosis based on either K-SADS or Conners' questionnaire. Ethics approval for this study was obtained from relevant ethics review boards, and informed consent/assent was signed by parents and their children. A comprehensive overview of recruitment, diagnostics, ethical approval, testing procedures, and quality control are provided in a separate methods publication (von Rhein et al., 2014).

Table 1

Demographic and clinical characteristics of complete sample.

	Participants with ADHD		Unaffected siblings		Healthy controls		Sig.
N	184		103		128
Males	128		41		60
Females	56		62		68
	Mean	SD	Mean	SD	Mean	SD
ADHD symptomsa	12.94	2.90	0.75	1.28	0.36	0.90	ADHD > siblings = controls
Age	17.24	3.27	17.12	4.06	16.36	3.24	ADHD = siblings = controls
Age range	8 ↔ 25		7 ↔ 27		9 ↔ 23
Estimated IQb	95.13	16.84	102.20	15.79	106.03	14.17	ADHD < siblings = controls
IQ range	55 ↔ 138		65 ↔ 144		58 ↔ 141
	Mean	SD	Mean	SD	Mean	SD
SSRT (ms)	270.3	61.91	252.52	49.32	258.83	52.65	ADHD > siblings = controls
ICV (ms)	0.211	0.052	0.18	0.047	0.17	0.041	ADHD > siblings = controls
Errors (n)	6.45	7.89	4.05	5.29	3.45	4.31	ADHD > siblings = controls
Current medication	107		4		0
Comorbid ODDc	34		24		1
Comorbid CDc	4		7		0
Comorbid RDc	27		12		0

Note: ADHD = Attention-deficit/hyperactivity disorder; ODD = Oppositional defiant disorder; CD = Conduct disorder; RD = Reading disability; SSRT = Stop-signal reaction time; ICV = Intra-individual component of variation; Errors = Number of errors on go-trials; Sig. = Nominal significant differences are listed in this column if this column is empty no significant differences could be reported.

ADHD diagnosis was based on K-SADS structured psychiatric interviews and Conners' questionnaires (Conners et al., 1998).

Estimated IQ was based on the block-design and vocabulary subtests of the Wechsler Intelligence Scale for Children (WISC) or Wechsler Adult Intelligence Scale (WAIS-III; Wechsler, 2012).

ODD, CD, and RD diagnosis was based on K-SADS structured psychiatric interviews (Kaufman et al., 1997).

Stop-Signal Task design

Response inhibition was measured using an fMRI-adapted version of the Stop-Signal Task (van Meel et al., 2007, van Rooij et al., 2015b), consisting of four blocks of 60 trials each. Participants were instructed to respond as quickly and accurately as possible to a go-signal (two-choice reaction time task) with a left or right button press on a button box, unless the go-signal was followed by a stop-signal (25% of trials), in which case participants were instructed to withhold their response. Participants who did not reach 70% accuracy on the go-trials were excluded prior to analyses (N = 5). The task was adapted to the performance of the participant, by varying the delay between go and stop-signal (stop-signal delay), in order to achieve 50% successful inhibition on stop-trials for all participants. The stop-signal delay was decreased from an initial 250 ms, by 50 ms after successful inhibition, and increased by 50 ms after failed inhibition. The main measure of response inhibition performance, the stop-signal reaction time (SSRT), was calculated by averaging the delay necessary for a participant to successfully inhibit his/her response in 50% of the stop-trials. Secondary outcome measures were the total number of omission and commission errors on go-trials (errors) and the intra-individual component of variation (ICV), calculated by dividing the reaction time variance by the mean reaction time (both calculated from reaction times on correct go trials).

Acquisition of functional MRI

Data were acquired at both sites on similar 1.5 Tesla Siemens MRI scanners (Siemens Sonata at VUmc; Siemens Avanto at Donders Centre for Cognitive Neuroimaging) using the same Siemens 8-channel head coil and the following protocol: The Stop-Signal Task was collected in four runs using a T2*-weighted echo planar imaging sequence (TR = 2340 ms, TE = 40 ms, FOV = 224 × 224 mm, 37 slices, voxel size = 3.5 × 3.5 × 3.5 mm, 94 volumes per run). To assist accurate normalization, participants were also scanned using a high resolution MPRAGE T1-weighted sequence (TR = 2730 ms, TE = 2.95 ms, TI = 1000 ms, flip angle = 7°, voxel size = 1 × 1 × 1 mm, matrix size = 256 × 256, FOV = 256 mm, 176 slices).

Processing of fMRI data

Functional MRI data were processed using FSL (FMRIB's Software Library, www.fmrib.ox.ac.uk/fsl; fMRI Expert Analysis Tool, version 6.0; Jenkinson et al., 2012, Smith et al., 2004, Woolrich et al., 2009). Preprocessing included removal of the first four volumes of each run, within-run motion correction to the middle volume, slice-timing correction, and spatial smoothing with a 6 mm Gaussian kernel, before residual motion correction was applied using ICA–AROMA. ICA-AROMA is an advanced motion correction tool that has been shown to outperform other motion correction procedures (Pruim et al., 2015a, Pruim et al., 2015b). The data from each run were registered to the participant's T1 anatomical image using linear, boundary-based registration implemented in FSL-FLIRT. For each participant a general linear model was fit, including successful stop, failed stop and successful go trials as regressors in addition to error trials, signal from cerebral spinal fluid and white matter, which were included as nuisance covariates. Task regressors were convolved with a double-gamma hemodynamic response function and data were high pass filtered with a cutoff of 0.01 Hz prior to estimation. The resulting single-subject regression coefficient images (‘beta maps’) were transformed to participant-level anatomical space (3 mm isotropic resolution) and combined across runs using a fixed effects model, using FSL-FEAT. This resulted in three participant-level activation maps, (1) successful stop, (2) failed stop, and (3) successful go, which were transformed to a neutral ‘midspace’, a procedure which neutralizes potential registration biases due to structural group and gender differences. The reader is referred to a prior publication for further details of the processing procedures (van Rooij et al., 2015b), where the only difference in the present manuscript was the addition of the advanced motion correction using ICA-AROMA.

Quantifying and mapping group separation with Gaussian process classifiers

Gaussian process classifiers (Rasmussen and Williams, 2006) were used to distinguish participants with ADHD from their unaffected siblings and healthy controls. Gaussian processes are best described as a distribution over functions, where inference proceeds by first computing the posterior distribution over functions according to the rules of probability. This is referred to as conditioning the prior distribution on the data. In the classification case, the posterior process is then passed through a sigmoid response function that maps the output to the unit interval, thereby providing a valid probability score for each prediction. These quantify predictive confidence and provide the primary advantage of Gaussian process classifiers over alternative approaches. Further details surrounding this approach have been published previously (Marquand et al., 2010). First, we estimated group separation on the basis of neuroimaging biomarkers. For this, we trained GPC models to make predictions based on the activation maps corresponding to the three task regressors, described in the fMRI processing section. The total number of features in these classifications was 224,781. Second, we estimated group separation on the basis of behavioural data, which provides a reference for the classifier above. For this, we trained a GPC model on the basis of data from the behavioural task. Specifically, we used the number of errors during the task, the ICV as well as the SSRT as features (see above). Each classifier was embedded within a leave-one-participant-out cross-validation procedure, and the measure of generalizability was the area under the receiver-operating characteristic curve (AUC). This measure has the advantage that it is not sensitive to a particular choice of decision threshold. Statistical significance was assessed by permutation testing for the AUC, taking into account the family structure within the sample. Specifically, instead of permuting the labels individually, we permuted the labels that belong to participants from the same family together. In that way, we ensured that the family structure was preserved, when the labels were shuffled.

Multiple-comparison correction for the AUCs was performed with the Bonferroni-Holm method (Holm, 2010) and 95% confidence intervals were reported and based on the modified Wald-method (Kottas et al., 2014). Note that these confidence intervals should be considered illustrative only. The primary measures we use to assess statistical significance are p-values derived from the permutation testing procedures described above, which fully account for the family structure in the data.

A common approach to visualizing the importance of each brain region to the classification is to visualize the classifier weights directly (Mourao-Miranda et al., 2005). However, the classifier weights are influenced by both the signal and noise in the data, which complicates interpretation. Therefore, forward maps (Haufe et al., 2014) were computed that provide a better indication of the differential activation pattern underlying the group separation. Most commonly, these maps are reported without applying a threshold, but it is clearly desirable to localise the most important differences. Therefore, we present a novel approach to thresholding forward maps based on fitting a mixture model. To achieve this, we fit a Gaussian-Gamma mixture to the image histograms that provide an explicit model for the null distribution plus positive and negative activations (Beckmann and Smith, 2004). For this, we used the implementation in the FSL-MELODIC software.1 After fitting this model, these maps can then be thresholded in two ways: (i) by an alternative hypothesis testing (AHT) procedure where voxels are declared significant if they have a probability pAHT > 0.5 of belonging to one of the alternative distributions (Beckmann and Smith, 2004) or (ii) by controlling the false discovery rate (FDR) against the explicitly modelled null distribution (Efron, 2004, Efron et al., 2001). Here this was done at the norminal rate of pFDR < 0.05. In our data, both approaches lead to similar conclusions (see Supplementary Fig. 1). All figures were visualized in Caret (Van Essen et al., 2001).

Supplementary Fig. 1

First column shows the forward maps of significant predictions based on successful stop activation maps. Second column shows Z-statistics thresholded at pAHT > 0.5 for the forward maps. Third column shows Z-statistics thresholded at pFDR < 0.05 the forward maps.

Sensitivity analyses were performed to increase the confidence in the analyses described above. Since the total sample showed a slight class imbalance with respect to gender and scan-site and in order to reduce nuisance variance, the sample was perfectly matched for gender as well as scan-site and optimally on age. We used optimal matching algorithms implemented in the R-package MatchIt to simultaneously match age across all groups (Ho et al., 2011; for information on the matched sample see Supplementary Table 1).

Results

Descriptive statistics: Stop-Signal Task

Unaffected siblings showed shorter stop-signal reaction times and lower error rates than the participants with ADHD (Table 1, Wald χ2 = 7.941, p < 0.005 and Wald χ2 = 10.701, p < 0.001, respectively), but did not differ from healthy controls in their reaction times and error rates (Table 1, Wald χ2 = 0.743, p = 0.389 and Wald χ2 = − 0.5954, p = 0.343, respectively). The intra-individual component of variation was lower in unaffected siblings as compared to the participants with ADHD (Table 1, Wald χ2 = 20.213, p < 0.001), and slightly higher than in healthy controls (Wald χ2 = 4.057, p = 0.044). These results are similar to those reported in an earlier study with an overlapping sample (van Rooij et al., 2015b). The addition of age, gender, IQ, medication status, or comorbid diagnoses to the model did not influence the reported group differences.

Quantifying and mapping group separation

The accuracies for discriminating groups are summarised in Fig. 1. Briefly, unaffected siblings could be distinguished from the participants with ADHD on the basis of successful stop activation maps (AUC = 0.65, p < 0.002, 95% Modified Wald CI: 0.59–0.71 AUC) and participants with ADHD could be distinguished from healthy controls on the basis of the same activation maps included as features to the classifications (AUC = 0.64, p < 0.001, 95% Modified Wald CI: 0.58–0.70 AUC). We also found nominally significant discrimination of unaffected siblings from healthy controls based on successful stop activation maps (AUC = 0.59, p < 0.030, 95% Modified Wald CI: 0.52–0.66 AUC) as well as nominally significant discrimination between participants with ADHD and healthy controls based on failed stop activation maps (AUC = 0.60, p < 0.019, 95% Modified Wald CI: 0.54–0.66 AUC, Fig. 1; for balanced accuracy, sensitivity and specificity measures in the complete and matched sample see Supplementary Table 2).

Fig. 1

Depicted are the results for all predictions in the complete and matched sample, the x-axis corresponds to the area under the receiver-operating characteristic curve (AUC). An area of 0.5 indicates no discrimination; ** indicates that the prediction remains significant after Bonferroni-Holm correction; * indicates that predictions are significant before multiple comparison correction.

Fig. 2 shows the forward maps for the successful stop activation maps for each group distinction without applying a threshold. In Supplementary Fig. 1, the same maps are shown without a threshold in the first column, with a threshold of pAHT > 0.5 in the second and pFDR < 0.05 in the third. The classifier discriminating participants with ADHD from healthy controls showed a frontolateral, superior temporal and inferior parietal pattern with positive coefficients favouring ADHD. The pattern that separated unaffected siblings from participants with ADHD showed high coefficients in frontolateral and inferior parietal areas favouring ADHD, but was for the remainder wide-spread in comparison. The nominally significant unaffected sibling versus healthy controls distinction, showed a pattern with high coefficients primarily in inferior frontolateral areas favouring unaffected siblings. In Supplementary Fig. 2 we show the fit of the mixture models to the three forward maps. The sensitivity analyses for which we perfectly matched the sample on gender and scan site and optimally on age showed a similar pattern of results as those described above, with exception of the successful-stop difference between ADHD and their unaffected siblings, all predictions improved in the matched sample (Fig. 1).

Fig. 2

Forward maps of significant predictions based on successful stop activation maps. I) Unaffected siblings versus healthy controls show a frontolateral pattern with positive coefficients favouring siblings. II) Unaffected siblings versus participants with ADHD show a widespread pattern with positive coefficients favouring ADHD predictions. III) Participants with ADHD versus healthy controls show a relatively clear frontolateral, interior-parietal and superior-temporal pattern with positive coefficients favouring ADHD predictions. The left images correspond to the left hemisphere, the right images to the right hemisphere.

Supplementary Fig. 2

Fit of the mixture models to the three forward maps depicted.

For the classifier trained to separate groups on the basis of the behavioural data, we showed that unaffected siblings could be distinguished from participants with ADHD (AUC = 0.66, p < 0.001) but not from healthy controls (AUC = 0.51, p > 0.05), based on behavioural scores described earlier. Participants with ADHD could be distinguished from healthy controls (AUC = 0.71, p < 0.001).

Discussion

In this study we showed that: (i) unaffected siblings of participants with ADHD could be distinguished from healthy controls and from participants with ADHD. Further, (ii) participants with ADHD could be reliably distinguished from controls. (iii) The predictions on behavioural data were approximately equally accurate, except for the distinction of unaffected siblings from healthy controls, which was not possible with behavioural data. The pattern of difference between participants with ADHD and healthy controls was characterised by positive bilateral frontolateral, superior temporal and inferior parietal coefficients favouring ADHD and frontolateral coefficients favouring unaffected siblings in comparison with healthy controls. This provides evidence for a neural profile shared between participants with ADHD and their healthy siblings.

The pattern of difference reported here, partially overlaps with regions in frontal and parietal areas reported in earlier studies of our NeuroImage sample (van Rooij et al., 2015a, van Rooij et al., 2015b). Looking at the thresholded forward maps (Supplementary Fig. 1), we see frontolateral areas with positive coefficients favouring participants with ADHD as well as unaffected siblings when contrasted with healthy controls. In comparison to our previous studies that examined focal or connectivity differences, we extend these findings by precisely quantifying group separations based on task activation maps and show that the pattern of difference that distinguishes the groups is characterized by a widespread profile.

The diagnostic accuracy we report is moderate in relation to earlier studies aiming to separate participants with ADHD from controls using small samples (Wolfers et al., 2015) but is comparable to studies that have employed large samples that capture more of the heterogeneity in the ADHD phenotype (Sabuncu and Konukoglu, 2014). The pattern recognition approach we employed allowed us to quantify the degree of separation between groups and therefore also the degree to which shared familial risk factors present in patients and unaffected siblings are expressed in patterns of brain activity. In line with earlier studies that identified patterns of shared risk between siblings of participants with autism and schizophrenia (Fan et al., 2008, Segovia et al., 2014, Yu et al., 2013), we could distinguish unaffected siblings from participants with ADHD and, although to a lesser degree, from healthy controls. The patterns associated with these distinctions showed an overlap, with the pattern associated with the distinction of participants with ADHD from healthy controls, primarily in frontolateral regions. Interestingly, a distinction based on behavioural data was not possible between unaffected siblings and healthy controls, indicating that unaffected siblings are not behaviourally different from healthy controls in response inhibition. However, they show a different neural pattern, which may be linked to compensatory brain processes in these unaffected individuals compared to their affected siblings.

As mentioned in the introduction, ADHD has mostly been classified in considerably smaller studies (Hart et al., 2014a, Hart et al., 2014b, Igual et al., 2012, Johnston et al., 2014, Lim et al., 2013, Peng et al., 2013, The ADHD Consortium, 2012, Wang et al., 2011, Wang et al., 2013, Zhu et al., 2008). In a classical analytic setting, p-values derived from measures of central tendency (e.g. a t-test) have an explicit dependency on the sample size, so the significance necessarily increases with increasing sample size, even though the effect size may not. In contrast, the predictive accuracy is a measure of class overlap that is governed by the distributions of the different classes and is largely independent of sample size, if properly assessed (e.g. using cross-validation). Therefore, the estimate of class overlap becomes more precise with increased sample size. This is important because the present study is the largest task-based fMRI study employing pattern recognition in ADHD and therefore may represent a benchmark for what is possible in terms of accuracies in representative cohorts of heterogeneous disorders. This heterogeneity may, for example, stem from sampling subjects at different ages and at different points on their developmental trajectory. Our results suggest that – like all psychiatric disorders – the heterogeneity of the ADHD phenotype presents a major challenge for identifying disease mechanisms and for finding biomarkers that predict diagnosis and disease course. For example, previous research shows that only a subset of participants with ADHD display behavioural alterations in response inhibition (Mostert et al., 2015a, Mostert et al., 2015b). Different participants with ADHD may have different symptom profiles and different underlying biological causes (Faraone et al., 2015). Therefore, finding methods to parse heterogeneity is a major research initiative. Clustering methods are most commonly used for this purpose and aim to partition patients into subgroups (Fair et al., 2012, Mostert et al., 2015b, van Hulst et al., 2014), but alternative methods such as normative modelling (Marquand et al., 2016a, Marquand et al., 2016b) may also be beneficial for understanding heterogeneity underlying psychiatric disorders.

In summary, the present results describe a pattern of frontolateral, superior temporal and inferior parietal expansion that is associated with risk for ADHD. Unaffected siblings show differences primarily in frontolateral regions. This provides evidence for a neural profile shared between participants with ADHD and their healthy siblings. In the future, pattern recognition techniques can be employed to break down heterogeneity in those groups. This may allow us to better understand brain mechanisms that protect participants who share familiar risk but are unaffected.

The following are the supplementary data related to this article.

Supplementary Table 1

Demographic and clinical characteristics of matched sample.

Supplementary Table 2

Balanced accuracy/ sensitivity/ specificity measures for the predictions described in the results section based on the three activation maps.

Financial disclosures

Barbara Franke has received a speaker fee from Merz. Jan K Buitelaar has been in the past 3 years a consultant to/member of advisory board of/and/or speaker for Janssen Cilag BV, Eli Lilly, Shire, Lundbeck, Roche and Servier. He is not an employee of any of these companies, and not a stock shareholder of any of these companies. He has no other financial or material support, including expert testimony, patents, and royalties. Pieter Hoekstra has been a member of the advisory board of Shire. The other authors do not report conflicts of interest. None of these, or any of the funding agencies have had any influence on the content of this manuscript.