A Predictive Model Offor Attention Deficit Hyperactivity Disorder Based on Clinical Assessment Tools.

BACKGROUND: At present, clinicians diagnose that the clinical diagnosis of attention deficit hyperactivity disorder (ADHD) in children is mainly on the basis of the information provided by their parents, the behaviour of children in clinical clinics and the assessments of clinical rating scales and neuropsychological tests. Notably, no unified standard exists currently for analysing the results of various measurement tools for diagnosing ADHD. Therefore, clinicians interpret the results of clinical rating scales and neuropsychological tests entirely based on their clinical experience. METHODS AND
SUBJECTS: To provide guidance for clinicians on how to analyse the results of various clinical assessment tools when diagnosing ADHD, this study assessed children with ADHD and children in the control group using two clinical assessment scales-parent rating scale (PSQ) and Child Behavior Checklist (CBCL)-and one neuropsychological test (Integrated Visual and Auditory Continuous Performance Testing). The two-sample t-test (FDR correction) screened the parameters of the three assessment tools with significant inter-group differences. LibSVM was used to establish a classification prediction model for analysing the accuracy of ADHD prediction using parameters of the three assessment tools and weight values of each parameter for classification prediction.
RESULTS: A total of 19 parameters (16 from clinical rating scales, 3 from neuropsychological tests) with significant inter-group differences were screened. The accuracy of classification modelling was higher for the clinical rating scales (61.635%) than for the neuropsychological test (59.784%), whereas the accuracy of classification modelling was higher for the clinical rating scales combined with the neuropsychological test (70.440%) than for the former two parameters alone. The three parameters with the highest weight values were learning problem (0.731), hyperactivity/impulsivity (0.676) and activity capacity (0.569). The three parameters with the lowest weight values are integrated control force (0.028), visual attention (0.028) and integrated attention (0.034).
CONCLUSION: Our study findings indicate that the diagnosis of ADHD should be based on multidimensional assessment. For a more accurate diagnosis of ADHD, assessments and that more assessment parameters should be developed on the basis of different dimensions of physiology or psychology in the future to obtain a more accurate diagnosis of ADHD. Furthermore, the predictive model for ADHD may improve our understanding and help in optimisation of the treatment of such a condition.

Introduction

Attention deficit hyperactivity disorder (ADHD) is one of the most common diseases in child psychiatric clinics, with a prevalence rate of 7.2% in school-age children.1,2 The pathogenesis of ADHD involves deficits in attention, working memory and executive inhibition. Furthermore, a recent research found that children with ADHD exhibit a deficit both in automatic and controlled cognitive processes.3 ADHD is involved in a variety of neuropsychological mechanisms. Therefore, the symptoms of ADHD are highly heterogeneous and are influenced by multiple factors, including physiological, psychological and social factors. At present, clinicians diagnose ADHD in children mainly based on the information provided by their parents, the behaviour of children at clinics and the assessments of clinical rating scales and neuropsychological tests. Conners' parent rating scale (PSQ) and Child Behavior Checklist (CBCL) are two of the most used clinical rating scales for evaluating ADHD.4 In recent years, increasing attention has been focused on the development of objective evaluation tools (such as various neuropsychological tests) for managing ADHD. Previous study has demonstrated that Integrated Visual and Auditory Continuous Performance Testing (IVA-CPT) could accurately discriminate children with ADHD.5 At present, there is no unified standard tool for analysing the results of various measurement tools used for ADHD. Therefore, clinicians analyse the results of clinical rating scales and neuropsychological tests entirely based on their clinical experience.

Machine learning provides an opportunity to parse the mechanisms and symptoms of mental disorders with multi-factorial and complex impact factors.6 Support vector machine (SVM), a machine learning method with high prediction accuracy, was first proposed by Vapnik et al.7 Machine learning involves means the establishment of a data classification model by through analysing a known data set with multi-dimensional parameters to predict the classification of new unknown data sets classified in accordance with unknown laws.8 Among various machine learning techniques, SVM is best suited to solve pattern recognition of small samples with high-dimensional parameters.9 SVM has been successfully applied in the diagnosis of clinical diseases and the prediction of treatment effect. In 2015, Kim et al used SVM to predict the response of ADHD children with ADHD to methylphenidate with 84.6% accuracy.10 Meanwhile, several studies have adopted SVM in the diagnosis of ADHD.

In the present study, two clinical rating scales (PSQ and CBCL) and one neuropsychological test (IVP-CPTin diagnosing ADHD) were adopted as measuring tools. SVM was used to analyse the results of these measurements and establish a classification prediction model. The present study aimed to offer suggestions for clinicians on how to analyse the results of clinical scales and neuropsychological tests for ADHD.

Methods and Subjects

Subjects

From January 2017 to June 2018, school-aged children (aged 6–12 years) who were admitted to The Affiliated Hospital of Hangzhou Normal University for various learning problems were recruited. The diagnosis was conducted by an experienced psychiatrist in accordance with The Diagnostic and Statistical Manual of Mental Disorders (DSM-5) after a semi-structured interview from the ADHD module of Schedule for Affective Disorders and Schizophrenia in School-Age Children–Present and Lifetime Version (K-SADS-PL).11,12 The children diagnosed with ADHD were assigned to the ADHD group, whereas those who failed to meet any criteria for mental disorders were assigned to the control group. In this study, all children with brain damage, neurological disorders, genetic disorders, substance dependence, epilepsy, IQ of 70 or less or any other mental disorders were excluded. All the subjects volunteered to participate in the present study and the parents or legal guardians of these subjects provided written informed consent. All the protocols were approved by the Research Ethics Review Board of The Affiliated Hospital of Hangzhou Normal University. The present study was conducted in accordance with the Declaration of Helsinki. Finally, at the end of this study, 159 subjects (95 in the ADHD group and 64 in the control group) were recruited, and no significant difference was observed in the age and sex gender ratio between the two groups (Table 1).

Table 1

Demographic Characteristics of ADHD Group and Control Group

	ADHD Group (n = 95)	Control Group (n = 64)	P/F-value
Age	8.34 ± 1.41	8.17 ± 1.54	0.498
Gender (male/female)	57/20	49/14	0.694

Measurements of Clinical Rating Scales

SQ consists of 48 questions of four grades and was completed by the parents. Six rating parameters (impulsion/hyperactivity, conduct problem, ADHD index, cognitive problems, psychosomatic problems and anxiety) were selected for analysis.13 CBCL consists of three parts and 128 questions, and it was completed by the parents. Fifteen parameters (social function, activity ability, social ability, learning ability, behaviour problem, internalisation, externalisation, delinquent behaviour, social problems, thought problems, anxiety/depression, somatic complaints, attention problems, withdrawn and aggressive behaviour) were selected for the analysis.13

Measurement by Neuropsychological Test

IVA-CPT was selected to assess the functions of attention and self-control ability to auditory and visual stimuli4. When the visual or auditory stimuli were presented to the subjects, four cognitive variables, including omission number, error response number, reaction time and stability, were monitored. A total of 29 parameters were obtained by analysing these cognitive variables.

Statistical Methods

The results of PSQ, CBCL and IVA-CPT were analysed using a two-sample t-test (false discovery rate corrected, FDR) to screen out the parameters with a significant difference between the two groups, which was shown by mean ± standard deviation. In case of significant effects, Cohen’s d was computed for the effect size.3,14 Based on the screened parameters, LibSVM was used to develop the classification model. The four kernel functions of LibSVM (linear, polynomial, radial basis function (RBF) and sigmoid) were used for classification modelling. The specific steps are as follows: 1) training of the four classification models using the four kernel functions; 2) using the four classification models to classify the subjects; 3) using leave-one-out method to evaluate the accuracy, sensitivity (true positive rate) and specificity (true negative rate) of the four classification models; 4) applying Plotroc.py in LibSVM toolbox to draw a receiver operating characteristic (ROC) curve to evaluate the classification accuracy (a high area under the ROC curve (AUC) value suggests a good classification accuracy; 5) calculation of the weight of evaluation parameters based on the classification model with the highest accuracy. A high weight value suggests an important classification prediction.

Results

Comparative Analysis Between ADHD and Control Groups

The comparative analysis results showed significant differences in 16 parameters of PSQ and CBCL between the ADHD and control groups (p < 0.05, FDR corrected) (Table 2). Significant differences were observed in three parameters of IVA-CPT between the ADHD and control groups (p < 0.05, FDR corrected) (Table 3).

Table 2

Comparative Analysis of Clinical Rating Scales

	ADHD Group (n = 95)	Control Group (n = 64)	t	p	Cohen’s d
PSQ:
Impulsion/hyperactivity	1.4 ± 0.68	1.01 ± 0.56	3.708	0.002*	0.63
Conduct problem	2.76 ± 1.38	2.32 ± 1.36	1.918	0.09	0.32
ADHD index	3.27 ± 1.25	2.52 ± 1.2	3.613	0.002*	0.61
Cognitive problems	1.8 ± 0.59	1.33 ± 0.61	4.791	0.00009*	0.78
Psychosomatic problems	0.36 ± 0.39	0.24 ± 0.32	2.044	0.075	0.34
Anxiety	0.62 ± 0.52	0.42 ± 0.37	2.548	0.034*	0.44
CBCL:
Social function	14.9 ± 4.82	17.77 ± 4.4	−3.796	0.002*	0.62
Activity ability	3.08 ± 1.8	4.23 ± 1.95	−3.693	0.002*	0.61
Social ability	8.2 ± 3.36	9.65 ± 2.96	−2.805	0.025*	0.45
Learning ability	3.61 ± 1.16	3.89 ± 1.13	−1.589	0.151	0.24
Behaviour problem	51.05 ± 22.42	37.15 ± 18.71	4.019	0.001*	0.67
Internalisation	11.4 ± 6.97	8.43 ± 6.14	2.682	0.028*	0.45
Externalisation	14.14 ± 7.5	11.15 ± 6.57	2.496	0.037*	0.42
Social problems	5.12 ± 2.78	3.31 ± 2.55	4.193	0.001*	0.68
Thought problems	2.25 ± 1.75	1.37 ± 1.38	3.262	0.007*	0.56
Anxiety/depression	6.18 ± 4.35	4.17 ± 3.73	2.972	0.017*	0.50
Somatic complaints	2.4 ± 2.49	2.03 ± 2.22	0.963	0.337	0.16
Attention problems	10.27 ± 3.73	7.05 ± 3.55	5.36	0.00001*	0.88
Withdrawn	4.47 ± 3.15	3.14 ± 2.46	2.756	0.025*	0.47
Delinquent behaviour	3.05 ± 2.06	2.15 ± 1.74	2.747	0.025*	0.47
Aggressive behaviour	11.07 ± 6.09	9 ± 5.54	2.321	0.053	0.36

Note: *P < 0.05 (FDR corrected).

Table 3

Comparative Analysis of Neuropsychological Test Results

	ADHD Group (n = 95)	Control Group (n = 64)	t	p	Cohen’s d
Auditory comprehension	81.41 ± 14.93	87.08 ± 17.29	−2.264	0.053	0.35
Visual comprehension	85.71 ± 15.78	89.49 ± 18.09	−1.435	0.188	0.22
Auditory control	82.04 ± 21.27	89.51 ± 19.59	−2.126	0.066	0.37
Visual control	79.49 ± 21.85	87.43 ± 22.02	−2.292	0.053	0.36
Integrated control force	79.75 ± 18.14	87.89 ± 20.43	−2.599	0.031*	0.42
Auditory attention	84.47 ± 21.22	91.71 ± 19.72	−2.17	0.062	0.35
Visual attention	76.61 ± 23.28	85.71 ± 16.69	−2.662	0.028*	0.45
Integrated attention	79.08 ± 21.87	87.2 ± 19.95	−2.378	0.048*	0.39
Number of hyperactive events	10.59 ± 9.22	8.02 ± 8.02	1.975	0.082	0.30
Hyperactivity quotient	104.6 ± 11.51	108.11 ± 8.59	−2.068	0.073	0.35
Auditory caution	90.59 ± 21.1	95.94 ± 16.59	−1.655	0.136	0.28
Auditory coherence	81.19 ± 18.59	88.48 ± 17.28	−2.262	0.053	0.41
Auditory stamina	92.43 ± 18.74	94.26 ± 17.23	−0.643	0.567	0.10
Visual caution	89.27 ± 20.22	95.29 ± 21.01	−1.895	0.09	0.29
Visual consistency	81.52 ± 19.91	87.2 ± 16.97	−1.739	0.121	0.31
Visual persistence	89.23 ± 18.73	93.06 ± 16.19	−1.5	0.171	0.22
Auditory alertness	76.75 ± 28.6	82.65 ± 28.59	−1.308	0.231	0.21
Auditory attention	89.22 ± 19.28	93.22 ± 18.15	−1.116	0.307	0.21
Auditory velocity	105.02 ± 13.65	108.25 ± 14.13	−1.574	0.152	0.23
Visual alertness	82.97 ± 28.99	90.32 ± 20.64	−1.687	0.131	0.29
Visual attention	82.71 ± 18.62	88.74 ± 17.93	−1.892	0.09	0.33
Visual velocity	85.07 ± 17.26	90.65 ± 16.5	−2.184	0.062	0.33
Balance	126.07 ± 19.45	122.62 ± 18.03	1.109	0.307	0.18
Auditory agility	108.32 ± 19.06	107.55 ± 23.59	0.242	0.844	0.04
Visual agility	85.39 ± 19.31	85 ± 13.63	0.29	0.823	0.02
Auditory persistence	104.96 ± 19.21	105.23 ± 18.32	0.005	0.996	0.01
Visual persistence	99.04 ± 16.76	100.6 ± 16.78	−0.219	0.844	0.09
Auditory sensation/motility	102.45 ± 22.73	99.35 ± 26.75	0.65	0.567	0.12
Visual perception/motility	106.99 ± 24.82	113.15 ± 14.22	−1.981	0.082	0.30

Note: *P < 0.05 (FDR corrected).

LibSVM Classification Model

Firstly, 16 parameters of the clinical rating scales were selected for classification modelling. Among the four classification models using the four kernel functions, the prediction accuracy of linear classification model was the highest (61.635%), and its sensitivity and specificity were 71.579% and 46.875%, respectively (Table 4). The AUC of the linear classification model approximated 0.7839 (Figure 1). Secondly, three parameters obtained from IVA-CPT were selected for classification modelling. Among the four kernel functions, the classification prediction accuracy of the linear model was also the highest (59.748%), and its sensitivity and specificity were 100% and 0%, respectively (Table 4). The AUC of the linear model was about 0.6100 (Figure 1). Finally, 19 parameters (16 parameters of PSQ and CBCL and three parameters of IVA-CPT) were used in classification modelling. The classification prediction accuracy of the linear was the highest (70.44%), and its sensitivity and specificity were 74.737% and 64.063%, respectively (Table 4). The AUC amounted to 0.8222 (Figure 1).

Table 4

LibSVM Classification Model

	Accuracy %	Sensitivity %	Specificity %
16 parameters of clinical rating scales:
Linear	61.635	71.579	46.875
RBF	60.377	91.579	14.063
Polynomial	57.233	57.895	56.250
Sigmoid	59.748	100.000	0.000
3 parameters of neuropsychological test:
Linear	59.748	100.000	0.000
RBF	56.604	94.737	0.000
Polynomial	49.686	67.368	23.438
Sigmoid	59.748	100.000	0.000
19 parameters of clinical rating scales and neuropsychological test:
Linear	70.440	74.737	64.063
RBF	55.975	100.000	0.000
Polynomial	55.975	60.000	50.000
Sigmoid	59.748	100.000	0.000

Figure 1

ROC curve of LibSVM classification model.

In addition, the classification modelling analysis results of LibSVM showed the weight values of 19 parameters of the clinical rating scale and neuropsychological test for classification prediction (Table 5).

Table 5

Weight Values of the Parameters

Parameters	Weight Values
Clinical rating scale:
CBCL:
Social function	0.337
Activity ability	0.569
Social ability	0.258
Behaviour problem	0.074
Internalisation	0.302
Externalisation	0.144
Social problems	0.035
Thought problems	0.150
Anxiety/depression	0.256
Attention problems	0.165
Withdrawn	0.234
Delinquent behaviour	0.111
PSQ:
Impulsion/hyperactivity	0.676
ADHD index	0.484
Cognitive problems	0.731
Anxiety	0.149
Neuropsychological test:
Integrated control force	0.028
Visual attention	0.028
Integrated attention	0.034

Discussion

To date, clinical rating scales have been used as the primary assessment tools for ADHD, and the information required in these tools was provided by the primary caregiver of children based on their subjective memory. The information required in the neuropsychological tests, such as IVA-CPT, which have been developed in recent years, was provided by children themselves. Neuropsychological tests are relatively objective. However, it still remains uncertain as to how the parameters obtained by these evaluation tools must be analysed and which parameters are the most reliable ones. In this study, t-test was conducted to compare the 50 parameters obtained from PSQ, CBCL and IVA-CPT between the ADHD and control groups, and 19 parameters with significant differences were screened out (4 parameters of PSQ, 12 parameters of CBCL and three parameters of IVA-CPT). Then, 16 clinical rating scale parameters and 3 neuropsychological test parameters were used for SVM classification modelling. The accuracy of the classification model using 16 parameters of clinical rating scales is higher than that of the classification model using 3 parameters of the neuropsychological test. This finding implies that the clinical rating scales might be more accurate than the neuropsychological tests in the diagnosis of ADHD. Finally, the accuracy of the classification model using all 19 parameters of the two kinds of assessment methods was higher than that of the models using the parameters obtained from clinical rating scales or neuropsychological test alone. This result suggests that a comprehensive reference of the results of various assessment methods could improve the accuracy of ADHD diagnosis. In 2011, the prediction accuracy of 62.52% on ADHD diagnosis was achieved by a research team from Johns Hopkins University using machine learning methods and based on the data of the brain functional and structural magnetic resonance imaging from the ADHD-200 competition.15,16 In 2019, Kaur et al used SVM method to predict ADHD in accordance with electroencephalography data, and the prediction accuracy reached 93.3%, which was higher than the accuracy obtained in this study.17 This finding suggests that in the future, the diagnosis of ADHD should be a comprehensive reference for a variety of assessment results.

The weights of 19 parameters in the classification model were also analysed. The top three parameters with the highest weight value were “learning problem” and “Hyperactivity/impulsivity parameters” of PSQ and the parameters “activity ability” of CBCL. The possible reason is that most children with ADHD were taken to the clinic by their parents only after they start schooling and caused serious concern because of their deficit in learning ability caused by attention deficit or hyperactivity symptoms. Therefore, the parents and clinicians are more concerned about “learning problem”, “hyperactivity/impulsivity parameters” and “activity ability”. Thus, these parameters play the most important roles in the diagnosis of clinical ADHD. Notably, the three parameters of the neuropsychological test showed the lowest weighted value. This result indicates that the main reference for the current clinical diagnosis of ADHD is still the results of the clinical rating scale and the parents’ description of children’s behaviour, academic performance and social adaptability. The current clinical diagnostic criteria for ADHD (such as DSM-5) are also closely related to the symptoms reflected by the parameters of these clinical rating scales.

In conclusion, the findings suggest that the diagnosis of ADHD should be based on multidimensional assessment. For a more accurate diagnosis of ADHD, more assessment parameters should be developed from different dimensions of physiology or psychology in the future. Furthermore, the predictive model for ADHD may lead to a better understanding and optimisation of the treatment of such a condition.