Prediction of sleep side effects following methylphenidate treatment in ADHD youth.

OBJECTIVE: Sleep problems is the most common side effect of methylphenidate (MPH) treatment in ADHD youth and carry potential to negatively impact long-term self-regulatory functioning. This study aimed to examine whether applying machine learning approaches to pre-treatment demographic, clinical questionnaire, environmental, neuropsychological, genetic, and neuroimaging features can predict sleep side effects following MPH administration.
METHOD: The present study included 83 ADHD subjects as a training dataset. The participants were enrolled in an 8-week, open-label trial of MPH. The Barkley Stimulant Side Effects Rating Scale was used to determine the presence/absence of sleep problems at the 2nd week of treatment. Prediction of sleep side effects were performed with step-wise addition of variables measured at baseline: demographics (age, gender, IQ, height/weight) and clinical variables (ADHD Rating Scale-IV (ADHD-RS) and Disruptive Behavior Disorder rating scale) at stage 1, neuropsychological test (continuous performance test (CPT), Stroop color word test) and genetic/environmental variables (dopamine and norepinephrine receptor gene (DAT1, DRD4, ADRA2A, and SLC6A2) polymorphisms, blood lead, and urine cotinine level) at stage 2, and structural connectivities of frontostriatal circuits at stage 3. Three different machine learning algorithms ((Logistic Ridge Regression (LR), support vector machine (SVM), J48) were used for data analysis. Robustness of classifier model was validated in the independent dataset of 36 ADHD subjects.
RESULTS: Classification accuracy of LR was 95.5% (area under receiver operating characteristic curve (AUC) 0.99), followed by SVM (91.0%, AUC 0.85) and J48 (90.0%, AUC 0.87) at stage 3 for predicting sleep problems. The inattention symptoms of ADHD-RS, CPT response time variability, the DAT1, ADRA2A DraI, and SLC6A2 A-3081T polymorphisms, and the structural connectivities between frontal and striatal brain regions were identified as the most differentiating subset of features. Validation analysis achieved accuracy of 86.1% (AUC 0.92) at stage 3 with J48.
CONCLUSIONS: Our results provide preliminary support to the combination of multimodal classifier, in particular, neuroimaging features, as an informative method that can assist in predicting MPH side effects in ADHD.

Introduction

Attention deficit hyperactivity disorder (ADHD) is a neurodevelopmental disorder characterized by symptoms of inattention, hyperactivity, and impulsivity. Methylphenidate (MPH) is the most frequently prescribed first-line therapeutic agent for ADHD, and it is reportedly effective in approximately 70% of children with the disorder (Santosh and Taylor, 2000). MPH is generally well tolerated by the patients and severe adverse events are rare. A recent meta-analysis also demonstrated that MPH has favorable tolerability and good efficacy in child and adolescents and could be recommended as first-choice medication (Cortese et al., 2018). However, MPH is associated with a distinctive pattern of relatively common, less severe, but often impairing side effects that vary from one individual to another as in most psychiatric conditions (Cortese et al., 2013; Vitiello, 2008). From a clinical perspective, the goal of the initial titration of MPH treatment would be to establish the best therapeutic response at the lowest dose with the fewest side effects. Although there has been a considerable degree of interest in the predictors of therapeutic response to MPH on the grounds of optimizing individual treatment, limited work has been done in depth examining the predictors of MPH side effects. Given the importance of minimizing side effects in establishing the optimal therapeutic dose of MPH, it could be clinically useful to identify any biological/cognitive characteristics within the patients that determine the occurrence of adverse events.

In the current study, we aim to identify predictors of the occurrence of sleep problems, which is one of the most common side effects of MPH treatment in ADHD youth (Feldman and Reiff, 2014). Sleep side effects are important for several reasons that are directly relevant to optimizing clinical care. First, considerable data (both correlational and causal) indicates that sleep loss negatively impacts daytime attention and mood in youth (Baum et al., 2013; Beebe, 2011; Beebe et al., 2009, 2010). As such, sleep side effects may be notably counterproductive to optimal treatment response (Lee et al., 2011), emerge more frequently in stimulant-naïve group (Wigal et al., 2012) and a reason of discontinuation (Wigal et al., 2006). Second, several studies indicate that sleep problems precede the onset of mood disorders (e.g., Gregory et al., 2009), raising concern that this side effect could increase vulnerability to comorbid problems with mood in ADHD youth. Finally, increasing evidence suggests that sleep plays a critical role in aspects of brain development and learning (Dahl, 2007; Ednick et al., 2009; Tononi and Cirelli, 2006), with notable interplay between sleep and neural circuitry involved in attentional and emotional control (e.g. fronto-striatal circuitry) (Beebe et al., 2009; Yoo et al., 2007). These converging evidences suggest that sleep side effect followed by MPH treatment carry potential to negatively impact long-term self-regulatory functioning (Stein, 1999; Van der Heijden et al., 2007).

Shared features of neural systems governing attention and sleep/wake regulation may partially explain high rates of sleep problems as side effects of MPH treatment. The presumed therapeutic action of MPH involves the dopaminergic and noradrenergic neurotransmitter systems in prefrontal and striatal regions (Wilens, 2008). In particular, MPH increases dopamine and norepinephrine concentrations in the prefrontal cortex. Increases in dopamine and norepinephrine by MPH are thought to increase wakefulness and lead to enhanced performance on tasks requiring vigilance and mental awareness, as well as to produce decreased need for sleep or sleep disturbance (Huang et al., 2011; Wisor et al., 2001, Zalesky et al., 2010). Although the exact mechanism is unclear, many of the neurotransmitter systems (e.g., dopamine, norepinephrine, serotonin) and brain regions (prefrontal and striatal [caudate, putamen]) involved in the regulation of sleep/wakefulness (Lazarus et al., 2013; Vetrivelan et al., 2010) are also related to the regulation of attention/arousal and pharmacological mechanisms of psychostimulants (Owens et al., 2013). As such, indices and correlates of functioning in key neural systems (e.g. fronto-striatal) involved in both sleep/wake and attention regulation could help to identify those youth at highest risk for sleep side effects following MPH use.

Dopaminergic and noradrenergic genes have been implicated in modifying the therapeutic response to MPH (Wilens, 2008). The dopamine transporter gene (DAT1) and the dopamine D4 receptor gene (DRD4) are the most extensively studied (Arnsten, 2011; Froehlich et al., 2010). The alpha-2A adrenergic receptor gene (ADRA2A) and the norepinephrine transporter gene (SLC6A2) have also been the focus of recent studies (Mick and Faraone, 2008). Candidate genetic predictors of MPH side effects have been reported. For instance, Stein and colleagues demonstrated that side effects varied as a function of the DAT1 polymorphism (Stein et al., 2005). McGough and colleagues found that an increased risk of irritability was associated with dopamine gene variants (McGough et al., 2006). However, to date, there is no study examining objective biological markers that can predict sleep problems as a side effect of MPH treatment in ADHD.

In addition to genetic factors in ADHD, it is possible that environmental risk factors (e.g., lead, nicotine) and interplay between genetic predisposition and environmental exposure modulate neurocircuits and neurotransmitter systems related to action of MPH (Nigg et al., 2010). As for neuropsychological endophenotypes of ADHD, the continuous performance test (CPT) and the Stroop color word test (SCWT) have been employed to measure neurocognitive functioning and its changes with MPH treatment in ADHD (Kebir et al., 2009; Kim et al., 2013). Thus, these variables may have clinical utility as predictors of treatment response in ADHD. With regard to neuroimaging measures, diffusion tensor imaging (DTI) has emerged as a powerful technique for searching clinically relevant biomarkers or measuring response to treatment in psychiatric disorders (van Ewijk et al., 2012). DTI research into ADHD has yielded support to the frontostriatal model of the disorder (Casey et al., 2007; van Ewijk et al., 2012).

Machine learning is an area of artificial intelligence concerned with the construction and study of systems that can learn from data (Orru et al., 2012). Recent evidence indicates that the application of machine learning classification techniques to psychiatric data may allow prediction of treatment response or side effects at the individual level (Orru et al., 2012). It is hoped that these methods could inform and assist clinicians to make more effective clinical decisions prior to treatment and would lead to fewer unsuccessful trials. To our knowledge, there has been no study that applied machine learning approaches to predict side effects of MPH treatment in ADHD.

In this study we applied machine learning approaches using pre-treatment demographic, clinical questionnaire, environmental, neuropsychological, genetic, and neuroimaging information (or features) to predict the presence or absence of MPH side effects in ADHD youth. Based on the similarities in neurotransmitter systems and neural circuitry involved in both attention and sleep regulation, we hypothesized that the biological/cognitive correlates of dopamine/norepinephrine neurotransmitter systems and prefrontal-striatal neurocircuits in ADHD would show significant predictive potential for differentiating between the patients who will develop and not develop sleep problems as a side effect to MPH administration.

Methods

The present study included 83 ADHD subjects (9.5 ± 2.6 years, 65 boys) recruited from the Seoul National University Hospital in Korea. ADHD was diagnosed according to DSM-IV criteria using the Kiddie-Schedule for Affective Disorders and Schizophrenia-Present and Lifetime Version (K-SADS-PL) (Kaufman et al., 1997). ADHD patients with an intelligence quotient (IQ) below 70, a past or an ongoing history of either tic disorder, obsessive compulsive disorder, language disorder, learning disorder, convulsive disorder, pervasive developmental disorder, schizophrenia, bipolar disorder, or brain damage, a past history of taking stimulants or atomoxetine longer than 6 months, or a recent history of taking stimulants or atomoxetine over the last 4 weeks were excluded from the study (Hong et al., 2014). The study protocol was approved by the institutional review board for human subjects at the Seoul National University Hospital. Detailed information about the study was given to parents and children, and written informed consents were obtained prior to study entry. At baseline, the parents completed the ADHD Rating Scale-IV (ADHD-RS) (DuPaul et al., 1998) and Disruptive Behavior Disorder rating scale (DBD) (Silva et al., 2005), and the participants undertook the CPT (Greenberg and Waldman, 1993), SCWT (Golden, 1978), genetic/environmental testing, and DTI scans.

The participants were enrolled in an 8-week, open-label trial of MPH. Initial doses of MPH were maintained for 2 weeks, and the doses were adjusted at the 2nd and the 4th week of treatment. The doses were titrated upward until sufficient therapeutic effects were achieved, on the basis of the subjects’ and the parents’ reports of symptom improvement and adverse effects, and then the doses were maintained for the remainder of the 8 weeks. After enrollment at baseline, our study involved four visits at the 2nd, 4th, 6th and 8th weeks. At each visit, we interviewed both the participants and their parents using the Barkley Stimulant Side Effects Rating Scale (Barkley et al., 1990) to determine the presence/absence of side effects. This is a 17-item scale of commonly reported adverse events associated with MPH treatment in ADHD. The “insomnia or trouble sleeping” question was used to determine the presence or absence of sleep problems. If the answers from the participants and their parents were incongruent, we thoroughly examined the reason why one of them reported sleep problems while the other didn't. After discussion, the interviewer, a well-trained child and adolescent psychiatrist, decided the occurrence of sleep problems if difficulty in falling asleep were at least one day per week after starting MPH treatment. Determination of the sleep side effects was conducted at the 2nd week of the treatment to examine innate susceptibility to sleep side effects at initial dose of methylphenidate whose effects could be similar across participants. In addition, all subjects were retained at the 2nd week without reporting considerable suffering from side effects and deciding to drop-out from the current study.

Genomic DNA was extracted from whole blood lymphocytes using a G-DEXTM II Genomic DNA Extraction Kit (Intron, Korea). The DRD4 exon III VNTR polymorphism and the 40-base pair VNTR polymorphism located in the 3’-UTR of DAT1 were genotyped, as previously described (Hong et al., 2012). For the ADRA2A and SLC6A2, the detection of a single nucleotide polymorphism was based upon analysis of primer extension products generated from previously amplified genomic DNA, using a chip-based matrix-assisted laser desorption/ionization time-of-flight (MALDI-TOF) mass spectrometry platform (Sequenom, California, USA). The ADRA2A (MspI and DraI) and SLC6A2 (G1287A and A-3081T) polymorphisms were genotyped as previously described (Hong et al., 2012). As for environmental factors, we measured blood lead and urine cotinine based on the evidence of our prior ADHD research (Cho et al., 2010). For the lead measurement, a volume of 5 ml of venous blood was collected from each child in metal-free tubes and samples were assayed using previously described methods (Kim et al., 2010). We used urine cotinine as a biomarker for environmental tobacco smoke exposure and it was measured using cotinine direct ELISA kits (BioQuant, San Diego, CA, USA), as previously described (Cho et al., 2013).

The image acquisition and processing implemented herein was based on standard protocols and methods, and is identical to our recent analysis performed in the same cohort (Hong et al., 2014). In brief, for each individual, we seeded streamlines throughout all of white matter and reconstructed the connectome using Automated Anatomical Labeling (AAL) atlas (Tzourio-Mazoyer et al., 2002). The network-based statistic (NBS) (Zalesky et al., 2012, HYPERLINK " 2010) (http://www.nitrc.org/projects/nbs/) was used to identify regional brain networks showing a significant between-group difference in inter-regional connectivity strength. We identified networks that comprised axonal fiber bundles traversed by a different number of streamlines between ADHD patients and healthy controls (Hong et al., 2014). A tract-averaged FA value was extracted for each fiber bundle, by averaging the FA values over all voxels intersected by at least one streamline. We utilized the track-averaged FA value of the ADHD group in the current study. Excluding drop-outs and participants with severe head motion (absolute displacement >5 mm estimated by rmsdiff function built in FMRIB Software Library v6.0), neuroimaging data of 67 subjects (including 8 subjects with sleep problems) was available for stage 3 analysis. For this study, analysis focusing on direct frontostriatal connectivity was performed using the 10 frontal lobe structures (except the olfactory cortex, paracentral lobule, and supplementary motor area) and two major striatal regions (the caudate nucleus and putamen) (Tzourio-Mazoyer et al., 2002). Streamlines connecting between a frontal and a striatal ROIs were isolated, resulting in a total of 80 pair-wise connections (i.e., 40 within each hemisphere and 40 crossing the hemispheres). Mean FA was extracted from each existing pair-wise connection.

We used Waikato Environment for Knowledge Analysis (WEKA) (Hall et al., 2009), an open source machine learning framework, to apply and evaluate performance of various classification algorithms to differentiate between patients who developed and did not develop sleep side effects. In the dataset, the above-noted genetic, environmental, neuroimaging, neuropsychological, and clinical attributes were included (see Table 1). In addition, demographic and clinical attributes of age, gender, IQ, height and weight at baseline, and initial MPH dose were included for the dataset. We used Wrapper subset evaluation method which evaluates attribute sets by using a learning scheme and identifies features that optimize the prediction performance (Kohavi and John, 1997) Wrapper subset evaluation method was used for feature evaluation and forward greedy hill-climbing augmented with a backtracking algorithm to search and select the subset of feature space. To avoid the overfitting and to increase the generalization of results, Wrapper subset evaluation method was applied with ten-fold cross validation. This gave list of features along with how many times they were chosen (this varies from 0% to 100% time) out of ten folds. Features which were chosen in each fold of ten-fold carried a selection weight of 100% while features which were never selected had a selection weight of 0%. The final feature subset used in this paper was chosen from the feature space generated from each fold of ten-fold cross validation. Generated feature space for this paper mostly included the features which consistently performed well in each fold of ten-fold (30% or more) cross validation stage. Following this method made the results less prone to overfitting and more generalizable to new instances. Features selected have a computational importance with or without clinical relevance or importance; thus, an experienced clinician should decide how to interpret the selected features.

Table 1

Demographic, clinical, and neuropsychological characteristics, genotype frequencies, and lead and cotinine levels of the ADHD participants at baseline.

	Training dataset (n = =83)	Independent dataset (n = =36)
Age, mean (SD) years	9.5 (2.6)	8.5 (2.5)
Female, n (%)	18 (21.7%)	4 (11.1%)
IQ, mean (SD)	107 (14)	110.4 (15.9)
Handedness (right), n (%)a	74 (90.2%)	34 (94.4%)
CPT, mean (SD)
Omission errors	65.7 (20.8)	65.0 (20.6)
Commission errors	64.2 (16.9)	67.1 (20.8)
Response time variability	63.4 (17.6)	63.9 (16.2)
SCWT
Word test	45.2 (11.0)	38.9 (11.9)
Color test	45.0 (10.5)	43.4 (11.0)
Color-Word test	46.5 (11.7)	43.1 (13.2)
Interference	53.6 (11.3)	52.1 (8.7)
ADHD-RS, mean (SD)
Inattention	15.1 (5.7)	14.8 (5.7)
Hyperactivity-impulsivity	11.0 (5.9)	11.2 (6.5)
Total	26.1 (10.5)	26.0 (10.8)
ADHD subtypes, n (%)
Combined	44 (53.0%)	17 (47.2%)
Inattentive	32 (38.6%)	8 (22.2%)
Hyperactive-impulsive	1 (1.2%)	4 (11.1%)
Not otherwise specified	6 (7.2%)	7 (19.4%)
Comorbid disorders, n (%)
Oppositional defiant disorder	16 (19.3%)	4 (11.1%)
Anxiety disorder	2 (2.4%)	3 (8.3%)
Genotype
DAT1, n (%)a
With 10/10	66 (80.5%)	32 (88.9%)
Without 10/10	16 (19.5%)	4 (11.1%)
DRD4, n (%)a
With 4/4	42 (51.2%)	21 (58.3%)
Without 4/4	40 (48.8%)	15 (41.7%)
ADRA2A MspI, n (%)a
G/G	38 (46.3%)	16 (44.4%)
G/C+C/C	44 (53.7%)	20 (55.6%)
ADRA2A DraI, n (%)a
C/C	22 (26.8%)	9 (25.0%)
C/T+T/T	60 (73.2%)	27 (75.0%)
SLC6A2 G1287A, n (%)a
G/G	34 (41.5%)	21 (58.3%)
G/A+A/A	48 (58.5%)	15 (41.7%)
SLC6A2 A-3081T, n (%)a
A/A	29 (35.4%)	7 (19.4%)
A/T+T/T	53 (64.6%)	29 (80.6%)
Environmental measure
Lead (µg/dL), mean (SD)	1.5 (0.4)	1.4 (0.5)
Cotinine (µg/g), mean (SD)	0.7 (1.3)	0.9 (1.4)

ADHD, attention deficit hyperactivity disorder; ADHD-RS, ADHD rating scale; ADRA2A, alpha-2A adrenergic receptor gene; CPT, continuous performance test; DAT1, dopamine transporter gene; DRD4, dopamine D4 receptor gene; SCWT, Stroop color word test; SLC6A2, norepinephrine transporter gene.

ADHD (n = 82).

We applied three different machine learning algorithms: support vector machines (SVM), a decision tree algorithm (J48), and Logistic Ridge Regression (a regression-based approach which handles multicollinearity well) with nested ten-fold cross validation to compare the performance of various algorithms. Researchers recommended that it is better to use nested ten-fold cross validation for evaluating the performance of classifiers in case of small sample sizes (e.g., a sample size less than 250) (Hawkins et al., 2003). We chose SVM which is commonly used for modeling complex nonlinear hypothesis spaces when sample sizes are small. We used sequential minimal optimization (SMO) algorithm for training the SVM classifier (Keerthi et al., 2001). We used SVM with second order polynomial kernel and tuned the model for optimum cost parameter from 1 to 100 (details are available upon request). J48 is an implementation of C4.5 decision tree algorithm (Quinlan, 1996). in WEKA. J48 decision tree gives an outcome model which is easy to interpret and applicable in clinical practice. Classification accuracy and area under receiver operating characteristic (ROC) curve (AUC) of these algorithms were compared to find the best classifier for sleep side effects. Based on the cost and burden of obtaining various measures, we set up three stages (see below) and examined whether classification accuracy and AUC could be improved by addition of the measures with each stage. We think that examining tests in tranches based on increasing cost provides data to say whether or not different groups of tests are worth the additional cost.

Stage 1: demographics (age, gender, IQ, height/weight) and clinical information (ADHD-RS, DBD, initial MPH dose).

Stage 2: stage 1 + neuropsychological (CPT, SCWT) and genetic/environmental (DAT1, DRD4, ADRA2A, SLC6A2, lead, cotinine) measures.

Stage 3: stage 2 + neuroimaging measures (frontostriatal connectivity of DTI).

Finally, we tested reproducibility and robustness of our classifier models using an independent dataset. From the study titled “ADHD translational research center” (http://clinicaltrials.gov/show/NCT02623114), an independent group consisted of 36 ADHD subjects (8.5 ± 2.5 years, 32 boys) were recruited from the Seoul National University Hospital. All participants fulfilled the same inclusion and exclusion criteria described above. Demographic, clinical, neuropsychological, genetic/environmental and neuroimaging data were collected in the same manner. For the estimation of frontostriatal connectivity in the independent sample, we applied identical preprocessing methods in the previous research of our group (Hong et al., 2014), except NBS approach. Briefly, deterministic streamlines were generated within 80 pair-wise connections of fronto-striatal circuit. Streamlines selected as a key classifier in the training dataset were registered to FA map of each subject, then a tract-averaged FA value was extracted as features representing structural connectivity. After administration of MPH, sleep side effects at the 2nd week were rated from each subject included in the independent dataset. A set of classifiers in each stage was combined in a step-wise manner, then, applied for the prediction of sleep side effects at the 2nd week of treatment using all 3 machine learning algorithms.

Results

Demographic and clinical characteristics

The initial mean dosage of MPH was 16.0 ± 6.2 mg per day (range, 10–36 mg per day) in the training data. The mean overall ADHD-RS score decreased from 26.1 ± 10.5 at baseline to 17.9 ± 9.4 at the 2nd week of treatment. Of the DSM-IV subtypes of ADHD, the combined subtype was the most common (53.0%), followed by the inattentive (38.6%) subtype (Table 1). Among the 83 ADHD subjects, 10 of them developed sleep problems during the first 2-week of MPH trial.

In the independent data, initial mean dosage of MPH was 20.9 ± 9.3 mg (range, 5–45 mg per day), and ADHD-RS score at baseline was 26.0 ± 10.8. The most common subtype of ADHD was combined (47.2%), followed by inattentive subtype (22.2%). Sleep side effects emerged to 15 of 36 subjects (41.7%) in the independent sample.

Prediction of sleep side effects of MPH treatment

In the training dataset, all three machine learning algorithms well predicted the presence/absence of the sleep side effects after MPH treatment at stage 1 (SVM, accuracy 89.7% (AUC 0.83); J48, accuracy 85.9% (AUC 0.76); Logistic Ridge Regression, accuracy 85.9% (AUC 0.87), Table 2). Escalated AUC were found when combining neuropsychological and genetic/environmental measures at stage 2.

Table 2

Classification accuracy and area under receiver operating characteristic (ROC) curve (AUC) performance of the classifiers for predicting sleep problems.

	Support Vector Machine				J48				Logistic Ridge Regression
	Accuracy	Sensitivity	Specificity	AUC	Accuracy	Sensitivity	Specificity	AUC	Accuracy	Sensitivity	Specificity	AUC
Training dataset
Stage 1	89.7%	40.0%	97.1%	0.83	85.9%	20.0%	95.6%	0.76	85.9%	50.0%	91.2%	0.87
Stage 2	92.3%	40.0%	100%	0.87	83.3%	30.0%	91.2%	0.78	92.3%	70.0%	95.6%	0.92
Stage 3	91.0%	37.5%	98.3%	0.85	90.0%	25.0%	98.4%	0.87	95.5%	100%	94.9%	0.99
Independent dataset
Stage 1	58.3%	0.0%	100%	0.50	58.3%	0.0%	100%	0.50	58.3%	0.0%	100%	0.51
Stage 2	66.7%	40.0%	85.7%	0.63	72.2%	40.0%	95.2%	0.71	66.7%	40.0%	85.7%	0.66
Stage 3	66.7%	40.0%	85.7%	0.63	86.1%	86.7%	85.7%	0.92	69.4%	46.7%	85.7%	0.70

Stage 1: demographics and clinical information.

Stage 2: stage 1 + neuropsychological/genetic/environmental measures.

Stage 3: stage 2 + neuroimaging measures.

ADHD, attention deficit hyperactivity disorder; MPH, methylphenidate.

Logistic Ridge Regression classification accuracy was 95.5% (sensitivity 1.00; specificity 0.95; AUC 0.99) at stage 3 for predicting sleep problems at the 2nd week of treatment (Table 2). Wrapper subset evaluation method demonstrated the inattention symptoms of ADHD-RS, CPT response time variability, the DAT1, ADRA2A DraI, and SLC6A2 A-3081T polymorphisms, and the structural connectivities between the left middle frontal gyrus (orbital part) and left caudate, left inferior frontal gyrus (orbital part) and left putamen, right middle frontal gyrus and right putamen, left superior frontal gyrus (orbital part) and right caudate, and left medial orbitofrontal gyrus and right caudate as the most differentiating subset of features (Fig. 1). SVM and J48 classification accuracies at stage 3 were 91.0% (sensitivity 0.38; specificity 0.98; AUC 0.85), and 90.0% (sensitivity 0.25; specificity 0.98; AUC 0.87), respectively. Fig. 2 shows the AUCs of the classifiers. Examining Logistic Ridge Regression, the best performing algorithm, classification accuracy and AUC continued to improve between all stages and was best at stage 3 (Table 2).

Fig. 1

Key fronto-striatal tracts in prediction of sleep side effects following methylphenidate treatment. This figure is a deterministic streamline data from single subject which is visualized with the TrackVis Software. Tracts connecting (a) left middle frontal gyrus (orbital part) - left caudate and left inferior frontal gyrus (orbital part) - left putamen (b) left superior frontal gyrus (orbital part) - right caudate, and left medial orbitofrontal gyrus - right caudate, and (c) right middle frontal gyrus and right putamen were selected as differentiating features.MFG, middle frontal gyrus; SFG, superior frontal gyrus; OFG, orbitofrontal gyrus; Caud, Caudate; Puta, Putamen.

Fig. 2

Comparison of AUC performance of the classifiers at stage 3. (a) Training dataset (b) Independent dataset. ADHD, attention deficit hyperactivity disorder; AUC, area under receiver operating characteristic (ROC) curve.

Performance validation of key subset of features in the independent dataset

At stage 1, demographic and clinical features performed just above the chance level classification accuracy in all three machine learning algorithm (Table 2). However, step-wise combination of the differentiating features enhanced prediction performance at stage 2 and 3.

After combination of structural connectivity features, J48 algorithm predicted 86.1% (sensitivity 0.87; specificity 0.86; AUC 0.92) of the sleep side effect at stage 3. Logistic ridge regression and SVM achieved 69.4% (sensitivity 0.47; specificity 0.86; AUC 0.70) and 66.7% (sensitivity 0.40; specificity 0.86; AUC 0.63) accuracy, respectively at stage 3.

Discussion

To our knowledge, the present study is the first to apply machine learning approaches using demographic, clinical neuropsychological, genetic, environmental, and neuroimaging data together to predict MPH side effects in ADHD. As hypothesized, the indices and correlates of functioning in key neurotransmitter and neural systems involved in both sleep/wake and attention regulation showed significant predictive potential for sleep side effects following MPH treatment.

By using three different types of machine learning algorithm(SVM, Logistic Ridge Regression and J48 decision tree), the presence/absence of sleep side effects after MPH administration was successfully predicted in the present study. In the training dataset, all three algorithms showed superior prediction performance. A step-wise combination of neuropsychological and genetic/environmental measures (stage 2) as well as neuroimaging features (stage 3) further increased the classification accuracy and AUC. Current findings suggest that individuals with sleep side effects following MPH treatment might have differential profile across gene-brain-behavior. Differential treatment responses to MPH by DAT1, DRD4 and ADRA2A DraI genetic polymorphisms (Froehlich et al., 2011; Kim et al., 2010; Winsberg and Comings, 1999), CPT performances (Rapport et al., 1986), or frontostriatal dysconnectivity (Hong et al., 2015) have been found in ADHD subjects. Together with those evidences, current findings may account for why sleep side effects is present only in a subpopulation of ADHD patients.

Key features from training dataset could well predict presence/absence of sleep side effect after MPH treatment in the independent dataset. The best accuracy was achieved by the J48 algorithm, followed by logistic regression and SVM. Decision trees have a strength in incorporating the interaction effects between variables (Zhao and Zhang, 2008), while linear regression model and SVM do not take those into account. Given the evidence that clinical phenotype is inevitably inter-correlated with genetic or neurobiological changes (Prathikanti and Weinberger, 2005), superior performance of J48 may attribute to consideration of interactions among classifiers in the prediction of new dataset.

Sonuga-Barke and colleagues found that side effects related to sleep were not predictable from patients’ demographic and clinical characteristics, such as age, gender, height, weight, and psychiatric comorbidity (Sonuga-Barke et al., 2009). They suggested that these side effects may be predictable from correlates related to underlying mechanisms of action of MPH. Their finding and suggestion are in line with our results, which showed that the classification accuracy and AUC by Logistic Ridge Regression improved with the addition of neuropsychological/genetic/environmental measures to demographics and clinical information (85.9% to 92.3%, 0.87 to 0.92, respectively), and further improved with the addition of neuroimaging measures (92.3% to 95.5%, 0.92 to 0.99, respectively). Overall, our ability to predict sleep side effects using all variables was successful. However, given the current cost of the imaging and genetic studies and the relatively low risk of treatment with MPH, we do not think that this is of substantial immediate clinical utility.

It is important to note that the DAT1, ADRA2A DraI, and SLC6A2 A-3081T polymorphisms were included in the most differentiating subset of features. These polymorphisms were reported to be associated with treatment response to MPH (Kim et al., 2010; Hong et al., 2012; Park et al., 2013). Our prior research also suggested ADRA2A DraI as one of the key features in predicting MPH treatment responders (Kim et al., 2015). Considering prior evidence and pharmacogenetic action of ADRA2A, norepinephrine pathway could play an important role in both eliciting treatment response and adverse effects with regard to MPH administration in ADHD. It has been well recognized that the mode of action of MPH in ADHD treatment is in its blockade of not only the dopamine transporter (DAT) but also the norepinephrine transporter (NET) (Solanto, 1998; Spencer et al., 1996). In the frontal cortex, where DAT density is low and NET density is higher, it is unlikely that blockade of the DAT accounts for the elevation of dopamine levels in this region. Thus, it has been postulated that dopamine is transported by the NET in the frontal cortex, since dopamine has a higher affinity for the NET as compared with its affinity for the DAT (Madras et al., 2005). ADHD medications have one pharmacological effect in common, which is to elevate extracellular levels of dopamine and norepinephrine in the frontal cortex.

In the current study, serum lead and cotinine level were not selected as key predictors of MPH-induced sleep side effects. These environmental toxins have been associated with symptoms of ADHD in a number of literatures (Cho et al., 2013; Froehlich et al., 2009) as well as increased risk for sleep problems (Liu et al., 2015). However, evidence that environmental toxins may increase sensitivity to sleep side effects following MPH treatment has not yet been identified. Although we could not find meaningful relationship between exposure to environmental toxin and sleep side effects in the present study, potential mechanism of neurotoxicity induced by lead and cotinine, and their impact on pharmacologic side effects may need further investigation.

Collectively, we could successfully predict sleep side effects after MPH administration using several biological markers related with ADHD. However, we did not assess co-existent sleep problems among ADHD subjects at baseline. It has been suggested that subjective reports of sleep disturbances also could be present in medication-free children with ADHD (Konofal et al., 2010). Sleep problems in ADHD might be multifactorial (Owens, 2005), however, underlying pathophysiology is still less well understood. Further studies are needed to understand co-occurrence of sleep problems in medication-free ADHD subjects. Meanwhile, pharmacological actions of MPH on dopaminergic and noradrenergic neurotransmitters (Wilens, 2008), and associated brain circuits have been figured out clearly (Lazarus et al., 2013, Vetrivelan et al., 2010). Here, we aimed to examine the occurrence of sleep side effects particularly after MPH treatment based on biological underpinning.

The application of our findings to real-world clinical care will require continued research examining the interplay of MPH, sleep side effects, and treatment outcomes in ADHD. However, there are at least two viable clinical approaches that could be used to manage patients who are identified as vulnerable to sleep side effects, supporting the clinical utility of this investigation. Namely, clinicians could 1) track sleep and titrate medications to maximize therapeutic response while minimizing sleep side effects, and/or 2) provide behavioral sleep intervention or prevention strategies that have proven efficacy in psychiatric populations (Haynes et al., 2006; Troxel et al., 2011).

There are several limitations to this study that deserve comment. First, the sample size was small, and occurrence of sleep side effects were only found in 10 among 83 participants in the training data. Feature extraction can be affected by imbalance of classes, which might be resulted in the low sensitivity of prediction models. Our findings should be replicated in a larger sample. Second, this study was conducted at one university center in Korea and, thus, we are unable to make inference with regard to the generalizability across different research centers or ethnic groups for any of the successful predictors found in our results. Third, we were not able to collect objective data of sleep problems such as polysomnography findings of the patients before the start of MPH treatment. As noted above, we only examined the occurrence of sleep side effects after MPH use; pre-existent sleep problems at baseline were not assessed. Fourth, genetic polymorphisms included in our study were selected from several candidate gene association studies which had limited power and sample size. Further studies with genome-wide association design are needed to confirm whether SNPs associated with DAT or NET are involved in sleep side effects following MPH treatment. Finally, the pharmacological treatment for the ADHD subjects was with a single medication. The predictive potential for other ADHD medications requires further investigation. The specificity of the predictive markers of this study should be interpreted cautiously since there was no placebo treatment arm.

In summary, the results of this study demonstrate that sleep side effects following MPH treatment in ADHD can be identified at the individual level using a range of biological and cognitive measures including genetic, neuroimaging, and neuropsychological data. From a clinical perspective, our results provide preliminary support to the combination of multimodal classifier, in particular, neuroimaging features, as an informative method that can assist in predicting response and adverse events to pharmacological treatments in ADHD. Prediction of sleep side effects associated with MPH treatment would not only help decide alternative treatment options initially such as using non-stimulants (e.g., atomoxetine, clonidine), but also reduce unsuccessful trials and make preventive interventions including alarming the risk of sleep side effects, providing behavioral intervention for sleep improvement or prescribing sleep medications (e.g., melatonin). Still, cost of neuroimaging and genetic polymorphism tests are quite high, which limits immediate utilization in the clinic. Further studies that examine more extensive biological/cognitive correlates that may determine the occurrence of sleep problems in medicated ADHD patients are needed to provide a better understanding of the underlying pharmacological mechanisms of MPH and its role on sleep-wake regulation.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.