Development of the Attention-Deficit Hyperactivity Disorder-After School Checklist.

OBJECTIVES: This study aimed to develop the attention-deficit hyperactivity disorder (ADHD)-After School Checklist (ASK) to evaluate the severity of ADHD symptoms and self-management ability in children and adolescents in South Korea. Additionally, we evaluated the reliability and validity of the scale.
METHODS: We developed the ASK to evaluate the effect of ADHD psychopathologies on self-management and interpersonal impulsivity. We investigated the reliability and validity of the scale with 1349 parents (male 56.9%; 1202 parents of non-ADHD children, 147 parents of children with ADHD) in Seoul and Gyeonggi Province, Korea.
RESULTS: According to the construct validity test using principal constant analysis with the varimax rotation method, two factors explained 60.7% of the cumulative variance in ASK scores. Cronbach's alpha for the whole scale was 0.71. There was no statistical difference between mean ASK scores at test and retest. Mean total ASK scores of the ADHD group were significantly higher than those of the non-ADHD group (p<0.001).
CONCLUSION: The ASK can be used as a reliable and valid tool to evaluate not only self-management capability of children and adolescents with ADHD in their academic and everyday life, but also their impulsiveness in interpersonal relationships. Copyright: © Journal of the Korean Academy of Child and Adolescent Psychiatry.

INTRODUCTION

Attention-deficit hyperactivity disorder (ADHD) is a disorder accompanied by a variety of behavioral problems and difficulties in self-management caused by inattention, hyperactivity, and impulsivity.1) In Korea, the number of children diagnosed with ADHD is increasing, with a 5-year average annual increase rate of 4.2% from 2009 to 2013.2) Moreover, since September 2016, certain drugs prescribed to adults with ADHD have been covered by insurance.

According to the Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM-5), core ADHD symptoms are largely categorized into inattention and hyperactivity-impulsivity.3) Inattention is exhibited through difficulty performing tasks, impatience, difficulty maintaining attention, difficulty organizing, etc. Hyperactivity is manifested in moving and running around excessively and inappropriately for the context, excessive movement of hands or feet, excessive talking, etc., while impulsivity is displayed in patterns of disruption or interference when others talk or act, difficulty delaying gratification, impetuous decision-making without consideration of long-term consequences, etc. ADHD symptoms not only reduce the child’s ability to adjust to school, maintain relationships with peers and teachers, observe rules, and achieve proficient academic performance, but also influence their life at home and extracurricular activities after school. Particularly after school, ADHD symptoms can cause conflicts with parents and family, and reduce children’s ability to self-manage, complete homework, and perform extracurricular activities. In summary, decreased functioning and behavioral problems due to ADHD symptoms have effects both in and out of school.4)

Most children and adolescents with ADHD attend hospital with a guardian (e.g., a family member). During diagnosis and treatment, information on their school life can sometimes be directly obtained from teachers, but in most cases information on ADHD symptoms, behaviors, and functioning at school is obtained from families. Thus, teachers’ observations of children’s school life are delivered through parents and the information is likely to be inaccurate.5) In contrast, direct observation of children’s behavior after school can be obtained from parents. Accordingly, information concerning children’s performance and behavioral patterns at home after school is more likely to be accurate than that concerning their school life.5) Despite the clinical need for accurate information on the symptoms and self-management of children and adolescents with ADHD after school, measurement scales to evaluate these behaviors are still not in use in Korea.

Currently, the measurement scales used to evaluate children with ADHD in Korea include the Korean version of the ADHD Rating Scale for Parents and Teachers,6) Conner’s Rating Scale for Parents and Teachers,7) and the Child and Adolescent Behavior Checklist.8) The Korean version of the ADHD Rating Scale for Parents and Teachers assesses behaviors based on the DSM-IV diagnostic criteria, and consists of nine items measuring inattention and nine measuring hyperactivity-impulsivity. The structure of this scale is very useful for classifying ADHD into three subtypes: predominantly inattentive, predominantly hyperactive-impulsive, and combined.9) Reliability and validity assessments6) of the Korean version of this scale as well as standardization testing9) have been conducted. Conner’s Rating Scale for Parents and Teachers is utilized to evaluate childhood behavioral problems, and is designed for parents and teachers to assess hyperactivity and other behavioral problems in children aged 3 to 17 years.10) Originally, this scale was developed with 93 items for assessment by parents and 39 items for assessment by teachers; however, a shorter form was later developed consisting of 48 and 28 items for parents and teachers, respectively. Many studies were conducted outside Korea on the reliability and validity of Conner’s Rating Scale,6-10) and the reliability and validity of its Korean version were tested by Park et al.7) The Child and Adolescent Behavior Checklist contains the social competence scale, to measure sociability and academic performance, and the behavioral symptom scale, to assess 119 behavioral problems. The Korean version of these two scales was standardized by Oh et al.,11) who also demonstrated their reliability and validity. These scales are relatively effective in assessing overall symptom severity, as they evaluate ADHD symptoms in widely ranging life domains. However, their limitation is that the same response scale is used for diverse behavioral patterns in a variety of contexts.12)

ADHD is a disorder in which self-management is difficult due to inattention and impulsivity. Accordingly, a primary treatment goal in ADHD is to increase the individual’s ability to self-manage to the level of their peers.13) To date, no instrument has been developed in Korea to evaluate self-management ability in children and adolescents with ADHD. Therefore, it is necessary to develop an assessment tool that can be used in clinical practice. The best approach to accurately assessing ADHD symptom severity and self-management ability is direct observation by a trained rater; however, it is difficult to evaluate or discern problems at home using this approach. Therefore, a measurement scale should be developed that can be conveniently used at home to reliably assess symptom severity and self-management ability in children and adolescents with ADHD. With such a scale, it would be possible to determine the need for psychiatric evaluation and intervention for low self-management ability, and evaluate the effects of therapeutic interventions on the improvement of both ADHD symptoms and self-management ability.

Accordingly, the purpose of the present study was to develop the ADHD-After School Checklist (ASK), a measurement scale for parents to assess ADHD symptom severity and after- school self-management ability of children and adolescents with ADHD, and examine its reliability and validity.

METHODS

Study participants

The present study was conducted with 1202 parents of children and adolescents aged 6 to 15 years living in Seoul or Gyeonggi Province and attending general kindergarten, elementary, or middle school; additionally, 152 parents of children and adolescents with ADHD in the same age range who had visited a pediatric psychiatry outpatient clinic in Seoul were included, from which 5 were excluded from the analysis due to incomplete responses. Therefore, data from 1349 parents were analyzed. The final sample comprised 767 male respondents (56.9%) and 582 female respondents (43.1%); 1202 (89.1%) were in the non-ADHD group and 147 (10.9%) in the ADHD group (Table 1).

Table 1

Demographic data of the study subjects

	Age (year)
	6	7	8	9	10	11	12	13	14	15
ADHD (n)
M	2	19	21	20	11	7	10	6	7	2
F	2	7	7	6	5	5	3	2	3	2
Total	4	26	28	26	16	12	13	8	10	4
Normal (n)
M	9	137	148	105	64	60	57	44	26	12
F	4	87	110	97	51	60	67	26	25	13
Total	13	224	258	202	115	120	124	70	51	25
Total (n)	17	250	286	228	131	132	137	78	61	29

ADHD: attention-deficit hyperactivity disorder, F: female, M: male

The diagnosis of ADHD was made by a pediatric psychiatrist based on the DSM-5 diagnostic criteria. Exclusion criteria were as follows: an overall IQ score of 70 or lower on Wechsler’s Intelligence Scale; a clear history of brain damage, other than ADHD, due to a genetic disorder or acquired brain injury (e.g., cerebral palsy); comorbidity with seizure disorder or another neurological disorder; diagnosis of autism spectrum disorder, schizophrenia, bipolar disorder, major depressive disorder, or other psychiatric disorder; and history of taking psychiatric medication.

Development of Attention Deficit Hyperactivity Disorder-After School Checklist

To develop the ASK, a pilot study was conducted with 144 parents of patients with ADHD who visited one of five pediatric psychiatry outpatient clinics in Seoul. In the pilot study, parents were asked the open-ended question “What is the most difficult aspect for you after your child comes home from school due to his/her ADHD symptoms?” The most frequent response was homework and self-studying, followed by daily life management, relationships with friends and siblings, relationship with parents, and extracurricular activities. The contents of the measurement scale were determined based on the pilot study findings, and the ASK was developed by adding response anchors for each item in order to measure severity.

The final version of the ASK consisted of a total of seven items, each measured on a 5-point scale. The 5 points on the scale for each item were anchored with specific criteria (Supplementary Table 1 in the online-only Data Supplement); the items were categorized into the domains of self-management and impulsivity in interpersonal relationships. The first three items in the domain of self-management assessed studying and task management: 1) delay in initiating a task, 2) motivation and avoidance, and 3) level of feedback. The remaining two items in the domain of self-management assessed the level of daily life management: 4) voluntary daily life management and 5) completion and delay in daily life management. The items in the domain of impulsivity in interpersonal relationships assessed 6) intrusion and 7) disruption and interference.

Procedure

The present study was approved by the Institutional Life Ethics Committee of Inje University Sanggye Paik Hospital (IRB No. SGPAIK 2014-11-012). Participants in the non-ADHD group were recruited through announcements on the Internet and social network sites. The parents who volunteered to participate in the study either completed the survey online or filled out a paper version mailed to them. To recruit participants in the ADHD group, the researcher visited pediatric psychiatry outpatient clinics and parents of children and adolescents diagnosed with ADHD who consented to participate in the study were recruited.

Data analysis

The data were analyzed with SPSS for Windows version 21.0 (IBM Corp., Armonk, NY, USA) and SAS for Windows version 9.4 (SAS Institute Inc., Cary, NC, USA). Validity was tested using a factor analysis package, and discriminant validity was tested with an independent group t-test on the scores of the non-ADHD and ADHD groups. With respect to reliability, Cronbach’s alpha was computed to assess internal reliability, and a paired t-test was utilized to assess test-retest reliability. Data regarding standardization were examined using descriptive statistics. Statistical significance was determined as p<0.05.

RESULTS

Validity assessment

A factor analysis was conducted to evaluate the validity of the seven ASK items. The results of a principal component analysis showed that the communality of item 3 was 0.4, a generally accepted lower limit, and all other items showed a communality of 0.5 or higher. Thus, all items were included in the final version of the ASK (Table 2).

Table 2

Factor analysis

	Communality	Rotated factor matrix
		1	2
Q1	0.686	0.819	0.123
Q2	0.612	0.751	0.220
Q3	0.400	0.549	0.314
Q4	0.515	0.714	0.071
Q5	0.579	0.707	0.282
Q6	0.699	0.251	0.798
Q7	0.755	0.134	0.859
Factor		Self-management	Impulsiveness
Rotation sums of squared loadings		2.626	1.620
Cumulative variance explained (%)		37.5	23.2

Extraction method: principal axis factoring, Rotation method: varimax with Kaiser normalization

Items were factor-analyzed into two groups: items 1 through 5, and items 6 and 7. The rotation sums of squared loadings were 2.626 for factor 1 (self-management) and 1.620 for factor 2 (impulsivity); the proportions of variance explained were 37.5% and 23.2% for factors 1 and 2, respectively. Based on these findings, the items could be categorized into two factors: items 1 through 5, and items 6 and 7. Items 1 through 5 assessed self-management, as intended during their development, and specifically measured parental assessment of their children’s management of study activity and everyday life (scheduling, personal hygiene, etc.). Items 6 and 7 referred to impulsivity in interpersonal relationships, measuring parental assessment of their children’s inability to control impulsivity in interpersonal relationships, and insistence on their own opinion or interference with that of others.

Reliability assessment

To assess the reliability of the ASK, we examined the correlations between each test item and individual factors of the scale as well as the total score, and calculated Cronbach’s alpha (an index of internal reliability). Cronbach’s alpha for all seven items was 0.71. Regarding the individual factors extracted in the factor analysis, factor 1 (items 1 through 5) showed a high reliability (Cronbach’s alpha=0.79), while factor 2 (items 6 and 7) showed acceptable reliability (Cronbach’s alpha=0.63).

To assess the temporal stability of the ASK, test-retest reliability was assessed with 20 participants at an interval of 2 weeks. The difference in scores between test and retest was not significant (t=0.382, p=0.644).

Standardization

The mean and standard deviation of the total ASK scores are shown in Table 3, according to gender and age. In both boys and girls, the ASK score was highest for 6-year-olds and lowest for 12-year-olds.

Table 3

Mean and SD of the ASK by age (n=1349)

	Age (year)
	6	7	8	9	10	11	12	13	14	15
Male  (n=767)	16.27±3.52	15.66±4.42	14.89±4.15	15.47±4.02	14.72±4.42	14.69±4.44	14.27±5.34	14.30±3.23	15.06±5.31	13.43±4.55
Female  (n=582)	15.67±5.28	14.40±4.03	13.68±4.51	13.66±3.45	12.80±4.17	13.00±3.98	11.86±3.44	11.89±3.10	12.64±5.01	13.00±4.50

Data are presented as mean±SD. ASK: attention-deficit hyperactivity disorder-After School Checklist, SD: standard deviation

The mean total ASK score of boys was 14.9 (SD=4.4), which was significantly higher (t=6.8, p<0.001) than that of girls, 13.3 (SD=4.1). Overall, the total ASK score showed a decreasing trend from age 6 through age 12 in both boys and girls (Fig. 1).

Fig. 1

Trajectories of the attention-deficit hyperactivity disorder- After School Checklist of general population by sex. SD: standard deviation.

Table 4 summarizes gender- and age-specific total score norms from ages 6 to 15 years that were computed based on the findings of the present study.

Table 4

Normative data of the ASK

Percentile	Age (year)
	6	7	8	9	10	11	12	13	14	15
Male
2.3 (-2 SD)	9.22	7.16	6.59	7.44	5.88	5.81	3.59	5.35	4.44	4.32
16 (-1 SD)	12.75	11.46	10.74	11.45	10.30	10.25	8.93	8.57	9.75	8.88
84 (1 SD)	19.80	20.04	19.04	19.49	19.14	19.13	19.61	15.03	20.37	17.98
98 (2 SD)	23.32	24.33	23.18	23.51	23.56	23.56	24.95	18.25	25.68	22.53
Female
2.3 (-2 SD)	5.11	6.34	4.66	6.75	4.46	5.05	4.97	5.70	2.62	3.99
16 (-1 SD)	10.39	10.37	9.17	10.21	8.63	9.03	8.41	8.80	7.63	8.50
84 (1 SD)	20.95	18.43	18.20	17.11	16.97	17.00	15.30	14.99	17.66	17.50
98 (2 SD)	26.22	22.46	22.71	20.57	21.15	20.98	18.75	18.08	22.67	22.01

ASK: attention-deficit hyperactivity disorder-After School Checklist, SD: standard deviation

Discriminant validity

To investigate whether the ASK could discriminate ADHD from non-ADHD, the differences in ASK scores between the ADHD and non-ADHD groups were examined and the results are displayed in Table 5. The total score of the ADHD group, without considering gender, was higher than that of the non-ADHD group by 4.8 points (t=14.0, p<0.001). For boys, the total score was higher by 4.2 points in the ADHD group vs. the non-ADHD group (t=10.0, p<0.001); for girls, it was higher by 5.5 points (t=9.3, p<0.001). A Mann-Whitney U test was conducted to examine differences in age-specific total scores between the ADHD and non-ADHD groups, and revealed that scores were significantly higher for the ADHD group at all ages, with the exception of 6-year-olds (Fig. 2).

Table 5

Differences of the total scores of the ASK between ADHD group and general population

	Age (year)
	6	7	8	9	10	11	12	13	14	15	Total
ADHD (n=147)	17.00±2.94	19.46±4.39	18.68±4.63	17.73±4.07	18.38±3.95	17.75±5.19	19.31±6.28	19.67±2.55	21.30±6.22	20.25±5.06	18.49±4.61
Normal (n=1202)	15.77±4.42	14.69±4.03	13.93±4.09	14.26±3.67	13.28±4.11	13.48±4.01	12.38±3.89	11.50±3.06	12.51±3.67	12.08±3.21	13.65±3.99
U value	20.5	1238.5[***]	1475[***]	1402[***]	325.5[***]	382.5[**]	271.5[***]	117.5[**]	54[***]	7[*]
t-value											14.0[***]

Data are presented as mean± standard deviation.

p<0.05,

p<0.01,

p<0.001. ADHD: attention-deficit hyperactivity disorder, ask: ADHD-After School Checklist

Fig. 2

Differences of the trajectories of the ADHD-After School Checklist between general population and ADHD group. *p< 0.05, **p<0.001. ADHD: attention-deficit hyperactivity disorder, SD: standard deviation.

DISCUSSION

In the present study, the ASK, a scale to assess ADHD symptom severity and self-management ability in Korean children and adolescents was developed for parental use at home with relative ease. Reliability analyses showed that the internal consistency was high for all items and between the items and each factor, and test-retest score differences were not significant, suggesting that the ASK is a reliable tool. Discriminant validity analysis showed that ASK scores were significantly different between the groups, demonstrating that the ASK can differentiate between ADHD and non-ADHD. Gender and age comparisons of ASK scores showed that boys scored significantly higher than did girls, and the scores tended to decrease with age.

The findings of the present study are in line with those of standardization studies of other scales that are most widely used at present.9) Regarding the Korean version of the ADHD Rating Scale for Parents and Teachers, boys scored significantly higher than did girls on the inattention and hyperactivity-impulsivity factors, and the same pattern of findings was reported in normative research on Conner’s Rating Scale for Parents and Teachers.14,15) A normative study conducted with 4- to 19-year-olds in the US utilizing the ADHD Rating Scale found that the scores on inattention and hyperactivity- impulsivity decreased with age,16) which is congruent with the findings of the present study. However, in the present study, the difference in ASK scores between the ADHD and non-ADHD groups was not significant for 6-year-old children. The mean ASK score of 6-year-olds in the non-ADHD group was 15.77, which is relatively higher than that of non-ADHD children of other ages (13.12). In an age as young as 6 years, difficulty in self-management and impulsivity in interpersonal relationships, which are measured by the ASK items, are quite common even in normally-developing children; therefore, it is believed that this may be the reason why no significant difference was observed in comparison to children with ADHD.

ADHD is a disorder manifesting problems in the frontal lobe functions related to planning, sustained attention, ability to organize behavior, inhibition of behavioral response, etc.17) Thus, individuals with ADHD display low ability in self-management and control, which eventually results in a low level of functioning in various areas such as daily life, studying, and interpersonal relationships.18) Although ADHD causes difficulty with self-management, measurement scales to assess children’s self-management ability have never been commonly utilized in Korea. An existing scale that is closest to measuring ADHD-related self-management ability is the Self-Control Rating Scale (SCRS),19) which evaluates children’s ability to control themselves. The scale consists of a total of 33 items rated by parents and teachers,20,21) Its validity was examined in Korea, and the scale was reported to be useful in assessing impulsivity.22) A study used the SCRS to evaluate the effect of behavioral therapy in children with intellectual developmental disability; however, the scale contains a relatively large number of items, and inter-rater reliability was low because specific guidelines were absent, even though severity is rated on a 7-point scale. Hence, it is not easy to use the SCRS in clinical practice. Comparatively, items in the ASK are simple, and specific response anchors are provided so that any rater can respond using consistent criteria. Moreover, the measurement items in ASK are grouped into studying and task performance management, daily life management, and impulsivity control in interpersonal relationships, making the scale very useful for assessing any of these specific areas. In addition, it is difficult to find an adequate instrument to assess children’s self-management ability. An alternative may be the Perceived Medical-Condition Self-Management Scale, but this cannot be used with children with ADHD because the items refer to disease management in adults suffering from a chronic disease.23)

It seems that few tools are available besides the ASK to assess impulsivity in interpersonal relationships in children with ADHD. Of course, some relevant items are included in instruments to assess the overall severity of ADHD symptoms such as the ADHD Rating Scale for Parents and Teachers,24) Conner’s Rating Scale for Parents and Teachers,25,26) and Child and Adolescent Behavior Checklist.27) These scales are relatively adequate for assessing overall ADHD severity28); however, it has never been tested whether their use is appropriate to assess impulsivity in interpersonal relationships, and future research is therefore necessary. Additionally, the Barratt Impulsiveness Scale29) may be used to evaluate impulsivity and other ADHD symptoms, but only in adults.

Items in the aforementioned scales were developed around the ADHD diagnostic criteria. As such, they are biased toward assessing ADHD symptoms and are therefore inadequate to evaluate the domain of self-management. Another instrument, the Vanderbilt ADHD Teacher/Parent Rating Scale, has the advantage of assessing in detail overall academic performance, behavior in the classroom, interpersonal relationships, and participation in organized activities, as well as ADHD symptoms.30) However, this instrument has as many as 47 items and does not provide specific response anchors for each point on the rating scale.

In summary, unlike the existing instruments frequently used in Korea to assess the severity of ADHD symptoms, the ASK has the following advantages: it consists only of seven items, so it can be quickly and easily administered; it can be used as a discriminant tool in large-scale studies with the general population; and from the perspective of clinical practice, it is easy for parents to use at home to regularly monitor the severity of ADHD symptoms and self-management ability of their children. Accordingly, future studies are expected to utilize the ASK to examine the effects of therapeutic interventions on improving self-management ability by administering the ASK before and after the intervention. However, because self-management ability can be assessed in a vast range of domains, the ASK may be limited to an overall assessment of the entire component of self-management. A future study should investigate this issue by comparing the ASK with a more detailed and comprehensive assessment of self-management. In addition, to understand the extent to which the ASK reflects the characteristics of patients with ADHD, the relationship between the ASK and assessments of various ADHD clinical characteristics and functioning (particularly, neuropsychological testing) should also be investigated.

Limitations of the present study are as follows. The data analyzed in the study were collected only in Seoul and Gyeonggi Province, and intra-rater reliability was not considered even though the diagnosis of ADHD was made by a psychiatrist. Moreover, although the ASK was developed and evaluated based on responses provided by parents, information concerning the parents was not collected. Lastly, a comparison was not made between the ASK and other existing scales for ADHD. In order to use the ASK to clinically evaluate patients in diverse respects and provide treatment, the practicality of the ASK should be investigated in future research by cross-validating it with standardized scales that are currently available. The ASK is expected to be used to monitor the prognosis or the effect of therapy in patients with ADHD, as its repeated administration is not burdensome. Consequently, it would be relatively easy to conduct a long-term follow-up study using this scale.

CONCLUSION

The findings of the present study demonstrate that the ASK can be easily used to assess the ability of children and adolescents with ADHD to self-manage their academic and daily lives, as well as their impulsivity in interpersonal relationships.

Supplementary Materials

The online-only Data Supplement is available with this article at https://doi.org/10.5765/jkacap.2018.29.2.47.