Validity and Reliability of Adult ADHD Self-Report Scale Thai Version (ASRS-V1.1 TH).

BACKGROUND: The adult ADHD Self-Report Scale Thai version (ASRS-V1.1) (18 items) is a questionnaire for screening adult ADHD. AIM: To test the validity and reliability of the 18-question ASRS-V1.1 Thai version (ASRS-V1.1 TH) as a screening tool for adult ADHD.
METHODS: The original 18-question ASRS-V1.1 version was translated into Thai. The process was composed of forward-translation, synthesis of the translation, and back translation. Cross cultural adaptation, field testing, and final adjustment were completed consecutively. The 18-question ASRS-V1.1 TH were sent to 1,500 parents of kindergarten and elementary school students in Bangkok, Thailand. The diagnostic interview was randomly selected for 50 parents from the positive result group and 50 parents from the negative result group. The clinical interview for confirming diagnosis was run by 3 psychiatrists who were blinded to the results and used DSM-5 ADHD criteria for diagnosis.
RESULTS: The 18-question ASRS-V1.1 TH had satisfactory internal consistency (Cronbach's alpha = 0.92: Cronbach's alpha = 0.87 for inattentive scale, Cronbach's alpha = 0.84 for hyperactive / impulsive scale). For testing the criteria validity, the questionnaire has an adequate. The AUC from the first 6 questions was 0.80 (95% CI: 0.68-0.92) while from the 18 questions was 0.71(95% CI: 0.55-0.86).
CONCLUSIONS: The 18-question ASRS-V1.1TH is a psychometrically reliable and valid measure for screening adult ADHD in Thai clinical samples, especially the first 6 questions of the questionnaire.

1. Introduction

ADHD is one of the most common developmental psychiatric disorders among children and often persists into adulthood.[ The prevalence of the ADHD persistence strongly depends on how it is defined. If ‘ADHD remission’ is defined as no longer meeting the full diagnostic criteria than the remission rate has been shown to be quite high (about 60% of subjects), despite the fact that 30% of those in ‘remission’ still met criteria for some ADHD symptoms and also reported a low level of functioning.[ Over the past twenty years, many studies about the prevalence of adult ADHD reported 2.9%-3.2% for full criteria diagnosis based on DSM-IV and 6.6%-16.4% for partial diagnosis.[

According to the WHO Disability Assessment, [ many adults with ADHD not only had psychiatric comorbidities (mood disorders, anxiety disorders, substance use disorders, intermittent explosive disorder) but also significant loss of basic functioning (self-care, mobility, cognition), occupational functioning (days off work, loss of productivity, loss of social functioning), and relational functioning (marital relationship, parent-child relationship, co-worker relationships).

Despite the fact that ADHD often persists into adulthood and has significant impact on the relationships, career, and even the personal safety of the individual,[ only 10.1% of population who are suspected to have ADHD were diagnosed and received treatment. This means that most of adults with ADHD have gone overlooked and untreated.

In Thailand, there are very few studies about adult ADHD. Only one study, focused on the prevalence of ADHD in parents of ADHD children, reported 16% of parents had a diagnosis of adult ADHD.[ The first reason for so few studies in this area is because the previous DSM diagnosis criteria for ADHD were developed only for children and need modification to address the disorder in adults. Secondly, no practical screening instruments for diagnosing adult ADHD were available in Thai especially self-report questionnaires which can screen potential adult ADHD from the general population. The preliminary objective of this study was to develop a Thai version instrument for screening adult ADHD.

WHO developed the Adult ADHD Self-Report Scale version 1.1 (ASRS-V1.1) in conjunction with revision of the WHO Composite International Diagnostic Interview (CIDI) for the WHO World Mental Health initial surveys in 2004. It is a standardized and well validated tool for assessing current ADHD symptoms in individuals aged 18 years or older,[ it has been translated into many languages and used in many countries. This tool has two forms which are in a 6-question and an 18 question form. The 6-question outperformed the 18-question in terms of sensitivity (68.7% v. 56.3%), specificity (99.5% v. 98.3%), total classification accuracy (97.9% v. 96.2%), and cohen’s kappa κ (0.76 v. 0.58).[ Although the 6-question form has been translated into Thai, it is still in the process of validation and many aspects of the 18-question form would provide more details about adult ADHD. Therefore, the objective of this study was to evaluate the reliability and validity of the 18-question ASRS-V1.1 - Thai version to assess symptoms and to use it as a screening instrument for adult ADHD.

2. Methods

2.1 Study design

This study was conducted in the kindergarten and the elementary schools which joined the school mental health network of the psychiatric department at Ramathibodi Hospital, Mahidol University, Bangkok. The study was approved by the Ethics Committee of the faculty of medicine, Ramathibodi Hospital, Mahidol University on January 8, 2013. All subjects provided their verbal and written informed consent prior to study participation.

2.2 Participants

Participants consisted of parents of the students in the classes which were randomly selected from each level of included schools. To be included in this study the participants had to be more than 18 years old and agree to participate in this study. There were no exclusion criteria. 1500 questionnaires were distributed to parents, 816 of them were returned (54.4%). There were 52 and 764 participants who had positive and negative screening results, respectively. 50 participants of each group were randomly selected for telephone interview (Figure 1).

Figure 1.

Flowchart of the study

2.3 Measurements

The 18-question ASRS-V1.1 is a symptom checklist instrument which consisted of 18 ADHD symptoms from the DSM-IV-TR criteria[ for children which have been adjusted for adult ADHD. The questionnaire included the 6-question ASRS-V1.1 as the first six items of the questionnaire. The 18-question version was divided into two groups of symptoms:

Inattentive group: questions number 1, 2, 3, 4, 7, 8, 9, 10, 11.

Hyperactive/impulsivity group: question numbers 5, 6, 12, 13, 14, 15, 16, 17, 18.

Each item has a 5-point scale in which 0 = never, 1 = seldom, 2 = sometimes, 3 = often, 4 = very often. This instrument was translated into Thai according to WHO World Mental Health initiative interview translation guidelines after obtaining permission from the copyright holder. The process of translation included forward translation from the original English to Thai, reaching consensus among the language experts on the forward translation, back translation into English, international harmonization by the authors, and completing and giving feedback by 10 parents. The last modification and adjustment were made for the final version. The WHO CIDI advisory committee provided permission to test the validity and reliability of the final version.

2.4 Procedure

The 18-question ASRS-V1.1 Thai version (ASRS-V1.1 TH) was administrated by 1,500 randomly selected parents between October and November 2014. The results from the questionnaires were divided into two groups: the positive result group and negative result group. The positive result group was the group of participants who responded with 4 or more checkmarks in the darkly shaded area of the first six questions (appendix 1). This method was chosen because in previous studies it had been found to have higher reliability and validity than using the sum score of the full 18-question version of the original ASRS-V1.1.[ Afterwards, from January to March 2015 fifty randomly selected parents from each group were interviewed by 3 psychiatrists who had experience in diagnosing adult ADHD and were blinded to the groupings.(figure 1). The clinical interview had four parts. First, interviewers used a semi-structured interview according to the symptoms from the ADHD criteria in the Diagnostic and Statistical Manual of Mental Disorders, Fifth edition (DSM-5, 2013), which were matched with current ADHD symptoms in the adults. Concurrent with DSM-5 criteria a clinical diagnosis of adult ADHD required at least five symptoms of either inattention or hyperactivity/impulsivity during the 6 months before the interview. Secondly, participants were assessed about childhood symptoms of ADHD by asking “Were these symptoms present prior to 12 years of age?”. Thirdly, information about the setting where these symptoms were present was assessed by asking “Have these symptoms occurred in more than two settings such as your home, workplace or other setting?” Finally, the impairment criteria were assessed by asking “Do these symptoms impact your daily life in areas such as working, parenting, or relationships with others?”

Participants who had five symptoms or more in the first part with symptoms occurring prior to age 12, in more than two settings and having an impact on their daily life were grouped into the adult ADHD positive group, the remaining were grouped into the adult ADHD negative group.

2.5 Data analysis

The collected data were analyzed using SPSS version 22.0. Descriptive statistics were used to report frequencies, percent, means, and standard deviations of demographic data. For reliability of the questionnaire, the internal consistency was analyzed with Cronbach’s alpha coefficient.

For validity of the questionnaire, the construct validity and criteria validity were assessed by performing exploratory factor analysis (axis rotation by promax method) and the indices of sensitivity and specificity respectively. The strength of association between the first six questions score and the full 18 questions with a clinical diagnosis was assessed by calculating the area under curve (AUC) of the receiver operating characteristic (ROC) and to determine the proper sensitivity and specificity of the questionnaire for the best cut-off score. Following a study by Swets and colleagues[, the predictive utility of ASRS-V1.1TH was analyzed by comparing the positive result of the first 6 questions and the total score from the 18 questions.

3. Results

3.1 Descriptive statistics

A total of 816 parents completed the 18-question ASRS-V1.1 TH from the 1,500 questionnaires which were sent out (54.4% completion rate). Respondents ages ranged from 18 to 72, mean (sd) age was 37.7 (8.36) years, and 454 (56.1%) of the respondents were female. Other demographic data are reported in table 1.

Table 1.

Demographic data of study participants (total and those contacted for follow-up)

General information	Total sample(N = 816)	contacted [a](N = 51)	not contacted [b](N = 49)
Mean age (sd) (Years)	37.7 (8.37)	37.2 (6.97)	36.0 (6.70)
Sex n(%)
male	354 (43.8)	29 (59.2)	13 (29.5)
female	454 (56.2)	20 (40.8)	31 (70.5)
Educational level n (%)
≤ Grade 6	95 (12.0)	7 (14.3.0)	3 (6.8)
≤ Grade 9	119 (15.0)	4 (8.2)	10 (22.7)
≤ Grade 12 or equivalent	273 (34.4)	24 (49.0)	16 (36.4)
bachelor’s degree	282 (35.5)	12 (24.5)	13 (29.5)
≥ bachelor’s degree	25 (3.1)	2 (4.1)	2 (4.5)
Occupation n (%)
unemployed	76 (9.6)	3 (6.3)	4 (9.1)
bureaucrat/state employee	117 (14.8)	7 (14.6)	9 (20.5)
company employee	400 (50.7)	26 (54.2)	18 (40.9)
part-time job/freelance	113 (14.3)	7 (14.6)	7 (15.9)
others	83 (10.5)	5 (10.4)	6 (13.6)
Residence in thailand n (%)
Metro Bangkok	526 (65.1)	31 (62.0)	28 (63.7)
Central region	55 (6.8)	4 (8.0)	1 (2.3)
Northern region	45 (5.6)	3 (6.0)	3 (6.8)
Northeastern region	142 (17.6)	8 (16.0)	10 (22.7)
Eastern region	15 (1.9)	1 (2.0)	0 (0.0)
Western region	6 (.7)	0 (0.0)	1 (2.3)
Southern region	19 (2.4)	3 (6.0)	1 (2.3)
Mean ASRSv1.1 TH sum score (SD)	21.20 (9.06)	27.73 (12.20)	27.34 (11.17)

a = randomly selected participants which can contacted by telephone

b = randomly selected participants which cannot contacted by telephone

The mean (sd) score of the 18-question version was 21.20 (9.06). The mean (sd) number of items of first six questions which were marked in the darkly shaded area was 1.36(1.28) (mean score and SD for each item were reported in table 2). There were 53 participants (6.8%) which had positive screening results according to the first six items on the questionnaire.

Table 2.

Mean and SD scores for each item, correlation between each item and the ASRS-V1.1 TH total score, and Cronbach’s alpha if item deleted

Items	Mean (SD)	Corrected item/total correlation	Cronbach’s alpha if item deleted
Item 1	1.12 (0.95)	0.429	0.77
Item 2	1.04 (0.88)	0.496	0.76
Item 3	1.30 (0.83)	0.436	0.77
Item 4	1.19 (0.89)	0.497	0.77
Item 5	1.58 (1.02)	0.455	0.74
Item 6	1.40 (1.02)	0.331	0.76
Item 7	1.26 (0.79)	0.513	0.77
Item 8	1.30 (0.88)	0.558	0.76
Item 9	1.06 (0.93)	0.482	0.78
Item 10	1.48 (0.97)	0.452	0.77
Item 11	1.38 (0.89)	0.522	0.77
Item 12	.68 (0.83)	0.415	0.74
Item 13	1.00 (0.88)	0.587	0.72
Item 14	1.04 (1.01)	0.474	0.74
Item 15	1.23 (1.00)	0.452	0.74
Item 16	1.01 (0.89)	0.462	0.73
Item 17	1.25 (0.99)	0.494	0.74
Item 18	0.83 (0.80)	0.461	0.74

3.2 Reliability and item analysis

The Cronbach’s alpha coefficient of the total 18-question ASRS-V1.1 TH was 0.86. It was 0.79 for the inattention subscale and 0.76 for the hyperactive/impulsive subscale.

Table 2 shows the correlation between each item and the ASRS-V1.1 TH total score. Each item had a correlation with the total score of more than 0.3, which means that each item was acceptable and didn’t need to be revised or dropped. The mean scores for all items and the α values (if item deleted) were also shown.

3.3 Validity analysis

3.3.1 Exploratory factor analysis

The sample was adequate for factor analysis (Kaiser– Meyer–Olkin [KMO] measure = 0.920; and χ² of Bartlett’s test of sphericity = 2989.35 [p < 0.0001]), the loading factor value of every question was > 0.4 and the 18-question ASRS-V1.1 TH was divided into 4 groups (table 3)

Table 3.

Test of construct validity using factor loading for each item of ASRSv1.1-TH

	Factor loading
	1	2	3	4
Item 1	0.692	0.222	0.321	0.142
Item 2	0.670	0.362	0.265	0.403
Item 3	0.354	0.309	0.285	0.785
Item 4	0.669	0.355	0.346	0.264
Item 5	0.384	0.265	0.670	0.197
Item 6	0.063	0.170	0.664	0.367
Item 7	0.665	0.426	0.353	0.247
Item 8	0.689	0.427	0.449	0.249
Item 9	0.487	0.358	0.575	0.131
Item 10	0.370	0.348	0.314	0.748
Item 11	0.469	0.419	0.509	0.420
Item 12	0.363	0.575	0.428	-0.027
Item 13	0.514	0.539	0.683	0.110
Item 14	0.392	0.422	0.627	0.091
Item 15	0.312	0.662	0.325	0.306
Item 16	0.300	0.744	0.323	0.256
Item 17	0.429	0.526	0.494	0.211
Item 18	0.365	0.726	0.272	0.231

The 1st factor consisted of questions number 1, 2, 4, 7, 8

The 2nd factor consisted of questions number 12, 15, 16, 17, 18

The 3rd factor consisted of questions number 5, 6, 9, 11, 13, 14

The 4th factor consisted of questions number 3, 10

3.3.2 Criteria validity

The criteria validity was assessed by calculating the sensitivity and specificity from the results of the psychiatric interview, comparing the positive and negative groups results. The possible cut-off point was estimated by plotting the receiving operation characteristic (ROC) curve. The area under curve (AUC) of ROC curve from the first 6 questions was 0.80 (95%CI: 0.68-0.92) (figure 2a). AUC from the full 18 questions was 0.71 (95%CI: 0.55-0.86) (figure 2b).

Figure 2.

Comparing area under curve (AUC) from plotting receiving operation characteristics (ROC) curve between the 6-question (2a) and 18-question (2b) version of ASRS-V1.1 TH

Sensitivity and specificity for the best cut-off point of the first six questions of ASRS-V1.1 TH were 90.91% and 62.50%, respectively. While the sensitivity for the best cut-off of the full 18 questions for an ADHD diagnosis was equal to the first six questions (90.91%), specificity was lower (45.00%). Positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), and negative likelihood ratio (NLR) for the best cut-off of the first six items of the 18-question ASRS-V1.1 TH were 40%, 96.15%, 2.42, and 0.15, respectively. While the PPV, NPV, PLR, NLR of the full 18 questions of the ASRS-V1.1 TH were 31.25, 94.74, 1.65 and 0.20 respectively. Sensitivity, specificity, PPV, NPV, PLR, NLR for the other cut-off point of first six questions and total 18 questions are shown in table 4.

Table 4.

Comparing sensitivity, specificity, PPV, NPV, PLR, and NLR of the 6-question and 18-question ASRS-V1.1 TH

Cut-off for the first six questions ASRS-V1.1 TH[a]	Sens. (%)	Spec. (%)	PPV (%)	NPV (%)	PLR	NLR
0 v 1-6	100.00	27.50	27.50	100.00	1.38	0.00
0-1 v 2-6	100.00	47.50	34.38	100.00	1.90	0.00
0-2 v 3-6	100.00	57.50	39.29	100.00	2.35	0.00
0-3 v 4-6[*]	90.91	62.50	40.00	96.15	2.42	0.15
0-4 v 5-6	27.27	90.00	42.86	81.82	2.73	0.81
0-5 v 6	9.09	97.50	50.00	79.59	3.64	0.93
Cut-off for total 18 questions ASRS-V1.1 TH[b]	Sens. (%)	Spec. (%)	PPV (%)	NPV (%)	PLR	NLR
0-23 v 24-72[*]	90.91	45.00	31.25	94.74	1.65	0.20
0-25 v 26-72	81.82	45.00	29.03	30.00	1.49	0.40
0-26 v 27-72	81.82	52.50	32.14	91.30	1.72	0.35
0-27 v 28-72	81.82	57.50	34.62	92.00	1.93	0.32
0-28 v 29-72	81.82	60.00	36.00	92.31	2.05	0.30
0-29 v 30-72	81.82	62.50	37.50	92.59	2.18	0.29

Abbreviations: NLR = negative likelihood ration, NPV = negative predictive value, PLR = positive likelihood ration, PPV = positive predictive value, Sens. = sensitivity, Spec. = specificity

anumber of responses in the darkly shade area of the first 6 questions

bthe total score from 18 questions

*optimal cut-off point from this study

AUC of ROC curve of the 18-question ASRS-V1.1 TH in the inattentive area for ADHD-I or ADHD-C diagnosis was 0.62 (95%CI: 0.46-0.79) and AUC of ROC curve in the hyperactive/impulsive area for ADHD-H or ADHD-C was 0.72 (95%CI: 0.55-0.89). Sensitivity and specificity of both the inattentive and hyperactive/impulsive areas are shown in table 5.

Table 5.

Sensitivity and Specificity of the 18-question ASRS-V1.1 TH for inattentive and hyperactive-impulsive domain

The 18-question ASRS-V1.1 TH
Cut off [a]	Inattentive		Cut off [a]	Hyperactive-impulsive
	Sensitivity (%)	Specificity (%)		Sensitivity (%)	Specificity (%)
13	89	46	10*	90	37
14*	89	51	11	80	44
15	56	51	12	70	46
16	56	54	13	70	56
17	44	56	15	70	63
18	44	64	16	70	68
19	22	72	17	60	71

a the total score from 18 questions

*optimal cut-point score from this study

4. Discussion

4.1 Main findings

The objective of this study was to test the validity and reliability of the 18-question ASRS-V1.1 TH for screening adult ADHD. The internal consistency using Cronbach’s coefficient was 0.86 which shows good internal consistency for this questionnaire. The internal consistency of the subscales are acceptable for both the inattentive section (α = 0.79) and hyperactive/impulsive section (α = 0.76). These results were higher than the original 6-question ASRS-V1.1TH (α = 0.63-0.72). These results are understandable given that the more items there are on the questionnaire the higher Cronbach’s alpha will be. However these results represented not only good internal consistency for the whole questionnaire but also acceptable internal consistency in both inattentive and hyperactive/impulsive subscale. Cronbach’s alpha did not increase after removing any of the items, meaning that each item was necessary.

Using factor analysis to test the construct validity, the loading factor value of every question was high (>0.4) which indicated high constructive validity of this questionnaire. The 18 questions of ASRS-V1.1TH were divided into 4 groups. The 1st Factor consisted of the inattentive subscale items. The 2nd Factor consisted of the hyperactive/impulsive subscale items. The 3rd Factor consisted of the hyperactive/impulsive subscale items except for items 9 and 11 which were in the inattentive subscale. Although items 9 and 11 have the highest loading on the 3rd factor, we found that they also have a high loading on the 1st factor as well. The reason may come from the fact that these items imply both impulsive and inattentive characters. Concordant with Barkley’s contention, inhibition problems can contribute to problems with inattention, distractibility, and working memory.[ For the last factor, all items were in the hyperactive/impulsive subscale. By using factor analysis of the 18-question ASRS-V1.1 TH, the questionnaire was divided into 4 factors instead of 2 factors according to inattentive and hyperactive/impulsive symptoms. The reason may be due to all items in 1st factor reflecting symptoms related to task difficulty, but items in 4th factor represented forgetulness symptoms. Items in 3rd factor reflected symptoms characteristic of restlessness and agitation, while 2nd factor represented both verbal and behavioral impulsive symptoms. These patterns of behavior may represent a semi-independent dimension of ADHD symptoms in adults, which might different from children. Also this factor analysis result was in concordance with other studies, which examine factor structure in adults. However, they found three factor structures (inattentive symptoms, verbal impulsivity and behavioral impulsivity with other hyperactive symptoms).[ When the factor analysis of the ASRS-V1.1 TH was analyzed with only inattentive items, we also found the same results.

From studying the AUC, the AUC value of the ASRS-V1.1 TH of the first six questions shows good concordance (0.80) while the AUC of the total 18 questions shows fair concordance (0.71). The results indicate better screening potential using only the first six questions compared to the total 18 questions, which is consistent with the results of the original ASRS-V1.1 study.[ However these AUC results were lower than results testing the original version.[

Testing of the criterion validity for the 18 question version of ASRS-V1.1TH showed that sensitivity was 90.91% and specificity was 45.0%. While sensitivity for the best cut-off of the first six questions ASRS-V1.1 TH was equal to the full 18 questions version (90.91%), specificity was higher (62.5%). These results indicate that using the 6-question version of the ASRS-V1.1 TH could be a better screening tool than the 18-question version. This result is in concordance with the results of Kessler and colleagues study using the original version.[ However the sensitivity and specificity of the 6-question version were different from this original version where the sensitivity was 68.7% and the specificity was 99.5%.[ However, when we changed the definition of the positive result group to be a group of participants who responded with 5 or more checkmarks in the darkly shade area of the first 6 questions, the sensitivity and the specificity turned out to be 27.27% and 90.00% respectively. This change can reduce false positive results and should be appropriate for use when the sample size is large or when clinical interview is not available but false negative results are a concern. The first strength of this study is the large number of respondents (816 parents). Even though the percentage of overall respondents was 54.4%, the sample is still quite large. Secondly, the reliability and validity of the original ASRS-V1.1 had already been tested and testing of the ASRS-V1.1 TH followed the WHO WMH Initiative Interview Translation Guidelines. Thirdly, adjusted DSM-5 criteria for adult ADHD was matched to criteria for adult ADHD that were used during the clinical interview process. Finally, all interviewers were blinded to the ASRS-V1.1 TH results.

4.2 Limitations

This study has several limitations. Firstly, there were 44 parents who did not attend the clinical interview due to missing data on the questionnaire (e.g. no contact details recorded), and the amount of time it took to complete the questionnaire created difficulty in collecting participants full data. However the demographic data and sum of all ASRSv1.1 TH questions score between the 2 groups were not different, except for gender with the number of male participants being significantly more than female participants (p = 0.004) (table1). Secondly, inter-rater and intra-rater reliability were not tested. However all 3 interviewers worked together in the process of questionnaire translation and had a consensus regarding the details of the semi-structured interview for adult ADHD questions. Thirdly, we did not have other informants (as were commonly used in other studies). Results from other studies have shown that self reporting is less accurate than including other’s reports along with self-reporting. [ Finally, comorbidity and other medical conditions were not assessed. These conditions may have interfered with our ADHD diagnosis results.[

4.3 Implications

The 18-question ASRS-V1.1TH is a psychometrically reliable and valid measure for screening ADHD in adults. Although ASRS-V1.1 TH can be used as a screening tool, the clinical interview with patients and other informants such as parents, spouse, colleagues are still important for diagnosis. Further studies should look at the other benefits from the 18-question ASRS-V1.1TH as a tool for the assessment of ADHD severity.