The Reliability and Validity of the Korean Version of the Autism-Spectrum Quotient.

OBJECTIVE: This study was conducted to investigate the reliability and validity of the Korean version of Autism-Spectrum Quotient (AQ).
METHODS: 20 participants with high-functioning autism (HFA) and 99 normal participants were recruited. All participants were completed the AQ and Empathy Quotient (EQ), and parents of the HFA group completed the parent-report AQ. For testing the reliability, we examined Cronbach's alpha, performed item analysis, and compared self versus parent report score of HFA participants. For testing the validity, we compared the difference of the score of AQ among HFA and control group using independent t-tests, and performed correlation analysis between AQ and EQ. The receiver operation characteristic curve analysis was performed to determine a cut-off.
RESULTS: The Korean version of the AQ exhibited adequate internal consistency, and in most items, the HFA group scored higher in comparison to the control group. It was demonstrated that AQ has good discriminant validity through the confirmation of the significant difference in the AQ score between two groups. The concurrent validity was established through the significant correlation between AQ and EQ in the HFA group. The best estimate cut-off score of AQ for screening was 23.
CONCLUSION: The Korean version of the AQ was determined as a reliable and valid instrument to assess HFA in Korean population.

INTRODUCTION

According to DSM-5, an essential diagnostic feature of the autistic spectrum disorder (ASD) is characterized by persistent impairment in social communication and social interaction, restricted and repetitive patterns of behavior, interest, or activities [1]. In DSM-IV, it was referred to as the triad of impairment [2]. The prevalence of ASD is approximately 1% in the world population [1]. However, in recent times, the prevalence is increasing globally [3], and epidemiological studies on Korean children have reported a prevalence of 2.64% [4]. This is due to various factors such as increased public awareness, expansion in the range of diagnostic criteria [5]. In most cases, individuals with ASD are unable to lead independent lives and experience a low quality of life [6,7]. Moreover, it also creates financial and personal burdens for families of persons with disabilities [8,9].

However, the diagnosis of ASD often constitutes a difficult task. Although, ASD can be diagnosed from early infancy and is known to possess a high level of diagnostic stability [10], sometimes, the diagnosis is delayed until late childhood or even early adulthood [11]. Furthermore, if the intelligence quotient of the patient is at least 70 or more, the condition is diagnosed as high functioning autism (HFA), that implies a relatively high cognitive and linguistic skills, rendering early detection or diagnosis difficult [12].

There are diverse diagnostic tools for ASD. For example, Autism Diagnostic Interview-Revised (ADI-R) [13] and Autism Diagnostic Observation Schedule-Generic (ADOS-G) [14] are widely used. These tools are known to effectively differentiate ASD, and various studies have demonstrated that the reliability and validity range from good to excellent [15]. However, these are time consuming for assessment as other diagnostic interviews. Looking at other tools, the Childhood Autism Rating Scale (CARS) is also a simple autistic rating scale widely used in clinical settings. However, it is not a self-report and is generally assessed through interviews with caregivers [16].

Due to the limitations of existing measures, Baron-Cohen [17] suggested a short self-rating scale to ascertain whether individuals with normal intelligence levels have the autistic phenotype or not. In particular, Baron-Cohen argued that autistic features need to be perceived quantitatively on a continuum. Thus, the Autism-spectrum Quotient (AQ) was developed to quantify autistic characteristics. This questionnaire consists of 50 items and contains five subscales that corresponding to the autism triad and the cognitive deficits of autism: social skill, attention-switching, attention to detail, communication, and imagination.

The psychometric properties of AQ were reported to be satisfactory. According to Baron-Cohen et al. [17], the test-retest and inter-rater reliability was good, and the internal consistency of the five subscales varied from moderate to high. AQ was confirmed as effective in distinguishing the HFA patients by means of the total and the subscale score, and assessing the response rate of each group to the 50 items [18,19]. Further, 80% of the patients with HFA or Asperger’s syndrome attained a score of 32 or higher, while only 2% of the participants from the control group achieved this score [17]. Concurrent validity was also confirmed through the examination of the relationship between the empathy scale and AQ [20]. Meanwhile, AQ studies of the general population show that first-degree relatives of ASD patients have a higher level of AQ [21]. Additionally, AQ was associated with genetic polymorphisms, patterns of brain activity and brain structure, and hormonal profile [22-24]. Due to the effectiveness of AQ, it has been translated into various languages, and validation studies were conducted through Japanese [25,26] and Dutch samples [27].

However, the reliability and validity of the AQ have not been examined in Korea. Moreover, in the Korean context, few studies regarding the screening tools for HFA exist, and tool such as the high-functioning Autism Spectrum Screening Questionnaire (ASSQ) [12] is designed for children. Therefore, the purpose of this study is to identify the reliability and validity of AQ, and to clarify whether AQ can effectively assess autistic characteristics when it is applied to the Korean population.

METHODS

Participants

The study was approved by the Institutional Review Board (IRB No. 2014-17) for human subjects at the National Center for Mental Health. This study encompassed two different samples. The first group consisted of adolescents and adult patients who visited the National Seoul Hospital and were diagnosed with autism spectrum disorder without intellectual disability, in other words, high-functioning autism (HFA). Patients’ diagnoses were assessed by psychiatrists based on DSM-5 diagnostic criteria. Out of the 20 HFA patients, 18 complete the Korean Wechsler Adult Intelligence Scale-IV (K-WAIS-IV). Further, 17 patients obtained an FSIQ or GAI score of over 70, and 1 attained a FSIQ score of 69. However, psychiatrists’ interviews confirmed that the patients’ level of adaptive functioning was close to normal. Two patients did not accomplish the K-WAIS-IV, since the interview with the psychiatrist confirmed that the level of intellectual functioning and adaptive functioning corresponds to the normal range. Subjects were excluded if the caregivers were unable to provide sufficient information, because they did not agree with the clinical study, or if the patient did not understand the content of the informed consent. The selected patients were administered the Korean version of AQ and the EQ, and their caregivers conducted the AQ-parent version. The second group was the control group that comprised university students. In total, 100 students were recruited, including 50 university students in Cheon-an, Chungcheongnam-do and 50 university students in Kyung-ju, Gyeongbuk province. The participants volunteered, and they were asked to fill the AQ and EQ. Participants were included in the analyses for this study only if they completed all items of the AQ as well as the EQ. One study subject from the control group was excluded due to missing data, leaving a total of 99 normal group participants in the analysis. Prior to the administration of these tools for the participants for completion, the researchers explained the objectives of the study to the participants to obtain reliable data.

Measurement instruments

Autism Spectrum Quotient

The Autism Spectrum Quotient (AQ) is a self-report questionnaire that measures autistic traits in intellectually normal adults and adolescents. It consists of a series of 50 questions pertaining to the lack of social skills, difficulty in attentionswitching, excessive attention to detail, lack of imagination, and communication difficulties. To eliminate the response bias of the respondents, 26 out of 50 items were presented as inverse items. For response to each question, the respondents were asked to apply a 4-point Likert scale from ‘definitely disagree’ to ‘definitely agree’. The scoring method followed Baron-Cohen et al. [17] The total score range is from 0 to 50, and if the obtained score is higher, it implies that the severity of autistic traits can be considered higher.

Item translation

The item translation proceeded as follows: Korean authors translated items of the AQ (in English) into Korean. Subsequently, the Korean items were evaluated by a psychiatrist for accurate correspondence with the original English items. The final version of the Korean AQ employs the same choice format as the English one.

Empathy quotient

The empathy quotient (EQ) is a self-report instrument developed by Baron-Cohen and Wheelwright to measure empathy in both healthy individuals and those with autism spectrum disorders. This scale consists of 40 items that assess empathy, and 20 filler items that were added to distract the participant from singular focus on empathy. Similar to the AQ scale, out of the 40 items, 19 items are presented as inverse items. For each question, the response can be selected based on a 4-point Likert scale, ranging from ‘strongly disagree’ to ‘strongly agree’. The scores ranged from 0 to 80, and higher scores signify higher empathy.

Statistical analysis

The statistical analysis was performed with SPSS 21.0 for windows (IBM Corp., Armonk, NY, USA). The frequency analysis and descriptive statistics were employed in the demographic analysis. Cronbach’s alpha coefficient was utilized to confirm the internal consistency of the AQ. The frequency analysis was used for item analysis (the percentage of each group’s score on each AQ item), and the correlation between self-versus parent report was used to confirm the reliability of the test. Validity was determined in terms of discriminant validity and concurrent validity. Independent t-test was employed to verify the discriminant validity of the AQ. To investigate the concurrent validity, the Pearson correlation analysis was conducted with AQ and EQ. The receiver operating characteristic (ROC) curve, the area under the curve, and the sensitivity and specificity of the AQ at different threshold scores were examined through the ROC curve analysis.

RESULTS

Demographic characteristics

In the control group, there were 54 males and 45 females, with a mean age of 23.68 (SD=3.64) years. In the HFA group, there were 17 males and 3 females, with a mean age of 20.15 (SD=7.24) years. There were 10 outpatients (50%) who visited child and adolescent psychiatry, 8 outpatients (40%) who visited clinics for adult developmental disorders, and 2 inpatients in closed ward for children and adolescents (10%).

Reliability

Internal consistency

The internal consistency of the Korean version of AQ was calculated using the data for the control and patient groups. Cronbach’s alpha coefficients were determined as follows: AQ total=0.85, social skill=0.81, attention switching=0.51, attention to detail=0.29, communication=0.77, imagination=0.53. These indicate that the internal consistency of the total Korean AQ scale is high, and the subscales vary in internal consistency.

Item analysis

An item analysis (percentage of each group’s scoring on each item) is presented in Table 1. The control group participants scored higher than participants with HFA only on 7 items out of 50 items (items 4, 6, 8, 30, 34, 40, and 49). This confirms the importance of these items for discriminating HFA group from the control group. Among 7 items, 3 items were included in the subscale ‘attention to detail’, and every 2 items were included in subscale ‘attention switching’ and ‘imagination’.

Table 1.

Item analysis for groups

Item	Group 1 (%)	Group 2 (%)
	HFA (N=20)	Controls (N=99)
1	55.0	20.2
2	65.0	56.6
3	25.0	19.2
4	65.0	68.7
5	75.0	66.7
6	55.0	63.6
7	45.0	36.4
8	35.0	47.5
9	40.0	33.3
10	70.0	18.2
11	75.0	23.2
12	65.0	61.6
13	55.0	30.3
14	55.0	39.4
15	65.0	34.3
16	75.0	48.5
17	50.0	12.1
18	45.0	28.3
19	30.0	8.1
20	55.0	10.1
21	65.0	24.2
22	90.0	26.3
23	65.0	17.2
24	55.0	10.1
25	95.0	88.9
26	95.0	30.3
27	55.0	24.2
28	65.0	21.2
29	50.0	37.4
30	25.0	64.6
31	35.0	11.1
32	80.0	32.3
33	80.0	18.2
34	45.0	53.5
35	65.0	22.2
36	45.0	26.3
37	70.0	24.2
38	80.0	26.3
39	80.0	33.3
40	55.0	64.6
41	50.0	25.3
42	70.0	24.2
43	60.0	47.5
44	75.0	26.3
45	55.0	25.3
46	65.0	36.4
47	60.0	21.2
48	80.0	34.3
49	25.0	54.5
50	50.0	40.4

HFA: high-functioning autism

Self versus parent report

To test if the self-report by patients with HFA might contribute to incorrect scores, all participants in the patient group were asked if a parent could also complete an AQ about them. Every parent agreed to do this. The parent version of the AQ does not include10 items out of the 50 in the self report (item 3, 5, 6, 8, 12, 20, 23, 27, 36, and 42), since it is difficult for parents to answer these questions. The mean difference in AQ score between the self-report and parent-report for the 40-item AQ was 0.5 points; patients scored themselves more highly than their parent’s. The correlation between the AQ score and the self-report versus the parent-report was not significant.

Validity

Discriminant validity

The mean total AQ and subscale scores and standard deviation from each group are provided in Table 2. When the control group is compared to the HFA group with the t-test of AQ total and subscale score, significant difference was found between the groups. The AQ total score of HFA group was significantly higher than that for the control group [t (117)=-6.90, p<0.001, d=1.74]. Also, the AQ subscale ‘social skill’ [t (117)=-6.25, p<0.001, d=1.56], ‘attention switching’ [t (117)=-3.96, p<0.001, d=0.99], ‘communication’ [t (117)=-6.61, p<0.001, d=1.49], and ‘imagination’ [t (117)=-4.18, p<0.001, d=0.93] yielded the same results. However, the ‘attention to detail’ subscale did not exhibit any significant difference between the control and HFA group [t (117)=-0.89, p=0.38, d=0.20].

Table 2.

Discriminative validity among HFA and control group

Case	Group 1	Group 2
	HFA (N=20)	Controls (N=99)
AQ total (M, SD)	28.65 (6.31)	17.18 (6.87)
AQ social skill (M, SD)	6.45 (2.37)	2.67 (2.48)
AQ attention switching (M, SD)	6.60 (1.82)	4.75 (1.92)
AQ attention to detail (M, SD)	4.65 (1.95)	4.28 (1.63)
AQ communication (M, SD)	6.00 (2.66)	2.42 (2.11)
AQ imagination (M, SD)	4.96 (2.28)	3.05 (1.76)

HFA: high-functioning autism, AQ: autism spectrum quotient

There was no difference between the mean AQ total and subscale scores of men and women with HFA group. In the control group, no difference between the mean AQ total and 3 subscale score was observed. However, there was a significant sex difference in ‘communication’ [t (97)=2.15, p<0.05, d=0.44] and ‘imagination’ [t (97)=3.15, p<0.05, d=0.64). In these two subscales, the control group males scored higher than the control group females.

Concurrent validity

To provide evidence for the concurrent validity of the AQ, the relationship between AQ scores and EQ was investigated, since the relationship between autistic traits and empathy has been widely documented in both adults and children [20,28]. In the control group, the relationship between AQ and EQ was not significant. For the HFA group, the relationship between AQ total and three subscale scores (social skill, attention switching, communication) and EQ was significant. They exhibited a significant negative correlation. The subscale ‘attention to detail’ showed a significant correlation with EQ, but it had a positive correlation. The correlation between the subscale ‘imagination’ and EQ was not significant (Table 3).

Table 3.

Correlation analysis between AQ and EQ

Scales		AQ total	AQ social skill	AQ attention switching	AQ attention to detail	AQ communication	AQ imagination
Group 1	EQ	-0.66[†]	-0.67[†]	-0.56[*]	0.53[*]	-0.70[†]	-0.32
HFA (N=20)
Group 2	EQ	-0.12	-0.16	-0.05	-0.04	-0.09	-0.06
Controls (N=99)

p<0.05,

p<0.01.

HFA: high-functioning autism, AQ: autism spectrum quotient, EQ: empathy quotient

Determining a useful cut-off point

The area under the ROC curve was 0.88 (std. err. 0.04, 95% CI 0.81–0.95) that signifies the accuracy of the AQ to fall in good range. The area under the ROC is indicative of the overall accuracy of a test, representing the probability that a randomly selected ‘true-positive’ individual will score higher on the test than a randomly selected ‘true-negative’ individual. Also, examination of the receiver operating characteristics for the total AQ score suggested that a threshold score of 23 resulted in the accurate classification of the maximum number of people. At this cut-off, the sensitivity is 0.75 and specificity is 0.85 (Table 4).

Table 4.

Detailed report of diagnostic statistics for the Autism Spectrum Quotient

Cut-off point	Sensitivity (%)	Specificity (%)
≥6	100.00	0.00
≥8	100.00	4.04
≥10	100.00	12.12
≥11	100.00	13.13
≥12	100.00	14.14
≥13	100.00	19.19
≥14	100.00	23.23
≥15	100.00	43.43
≥16	95.00	47.47
≥17	95.00	53.54
≥18	90.00	64.65
≥19	90.00	71.72
≥20	90.00	75.76
≥21	90.00	80.81
≥22	85.00	80.81
≥23	75.00	84.85
≥25	75.00	88.89
≥26	70.00	88.89
≥28	65.00	89.90
≥29	65.00	90.91
≥30	60.00	91.92
≥31	50.00	92.93
≥32	35.00	93.94
≥33	35.00	94.95
≥34	30.00	94.95
≥35	20.00	97.98
≥36	5.00	97.98
≥38	0.00	97.98
≥41	0.00	100.00

DISCUSSION

This study explores the reliability and validity of the Korean version of the AQ in adolescents and adults with HFA. The results of this study are summarized as follows.

First, in the reliability analysis, the Cronbach’s alpha in our study had a value of 0.85 that indicates the total score has a good internal consistency. The internal consistency of the four subscales, including ‘social skill’, ‘communication’, ‘attention switching’, and ‘imagination’, vary from moderate to high. Conversely, the subscale ‘attention to detail’ exhibited low internal consistency. In the previous studies, the internal consistency of the subscale was moderate to high, but a low level of Cronbach’s alpha has been rarely seen. Therefore, in future studies, it would be beneficial to analyze the factor structure of the AQ and identify the new factor structure.

In the item analysis, out of a total of 50 items, except for 7, the HFA group responded at a higher rate than the normal group. A relatively low internal consistency may have affected these results, since these exceptional 7 items were included in three subscales (‘attention switching’, ‘attention to detail’, and ‘imagination’).

When the self-report and parent-report AQ scores were compared, the HFA group showed a slightly higher score than their parents. This result is different from that of previous studies, in which parents tended to give higher scores than patients themselves [17,26]. There was no significant correlation between self-report and parent-report. However, there are many studies that demonstrate a low correlation between self-report and parent-report [29,30]. For example, in a study of Korean youth and caregivers, the Korean Child Behavior Checklist (K-CBCL) and Korean Youth Self-Report (K-YSR) were both administered, and the results exhibited low agreement for adolescents’ problem behaviors among adolescents themselves and their parents. However, these discrepancies do not indicate wrong evaluations.

Second, the validity analysis results are as follows. In the discriminant validity analysis, the mean of AQ total score was significantly different from the HFA group and control group. Further, in the four subscales, except one, the HFA group showed a significantly higher score than the control group. This suggests that the Korean version of AQ could clearly distinguish between HFA group and control group.

In the concurrent validity analysis, a significant relationship between AQ and EQ in the normal group was not found. Conversely, in the HFA group, there were significant negative correlations between AQ total score and EQ that is consistent with the result of previous studies. Further, 3 subscales of AQ exhibited significant correlation with EQ. Empathy implies the ability to understand the way in which others feel, what they are thinking, to understand their intentions, predict their behavior, and feel the emotions that they feel. Also, empathy is known to be lacking in patients with ASD [28]. Thus, these results indicate that AQ evaluates the autistic traits of the HFA group.

Moreover, AQ showed good ROC characteristics, and, at a cut-off of 23, it showed good sensitivity and specificity. These sensitivity and specificity levels are similar to those observed in previous studies [19]. The cut-off score is relatively low compared to previous studies, but considering that the most important thing is being able to identify the people who are most likely to have a clinical level of autistic traits, we believe that a threshold score of 23 would ensure that false negatives are limited and support AQ as a useful screening instrument for clinical practice.

Taken together, the AQ showed high reliability and validity, similar to the results from previous studies with the original AQ. Such consistency supports the cross validity of the Korean version of the AQ.

The implication of this study is that we validated the AQ in Korea where there are few tools to screen patients with HFA, and found that AQ has satisfactory reliability and validity. In HFA patients, many symptoms at the early stages of development tend to decrease gradually as they get older [31,32]. Their relatively high cognitive and language functions interfere in early detection or judgment by the parents or teacher. This leads to late in-depth assessment and therapeutic intervention, and late therapeutic interventions lead to poor prognosis in later life, such as impaired activities of daily living, difficulty in social adjustment, and difficulty in correcting problem behaviors. Because of this, even if the intellectual function is at an average level, it is rare that an individual can lead an independent life in the community. For example, an empirical study with Australian HFA patients revealed that despite their normal level of intelligence, patients face significant disadvantages in employment settings [33]. The poor psychosocial functioning and maladjustment of ASD patients increases the burden, including economic costs, for members of the patients’ families as well as the community and the nation as a whole [34]. Therefore, noticing autistic characteristics and making appropriate therapeutic interventions at an early age is important. In this regard, AQ could serve as a helpful measurement in the process of screening autistic spectrum disorder.

The limitations of this study and suggestions for future research are stated below. First, the retest was not performed in this study; therefore, we were unable to evaluate the test-retest reliability of AQ over time. For future research, it is recommended to retest and evaluate the reliability more precisely. Second, the total number of participants in the HFA group is 20 that is considerably smaller than the control group participants which were 99. In addition, male participants were far greater in number than female participants in the HFA group, whereas the sex ratio was almost the same in the control group. We should consider that the sex ratio of the ASD is about 4:1 for male and female. However, since the number of female subjects was too small, there is a possibility that the deviation of the AQ score between male and female patients was not clear. Also, since the study was conducted on patients who visited the same medical institution, there may be a problem in the representative value of the patient sample. Therefore, these group’s results should be interpreted with care. Future research should be conducted with larger samples, so that the results from this study can be reconfirmed. Third, the clinical characteristics of the HFA group, such as the severity of the ASD, the level of daily living functioning, and the education level of the HFA group was not obtained.

In conclusion, despite its several limitations, the Korean version of AQ appears to be a useful and reliable measure of screening autistic traits in Korean adolescents and adults with HFA. Given the worldwide burden of ASD and its increasing prevalence, validating diagnostic assessments for use in clinical and research settings is important. We believe that AQ is useful as a primary assessment tool to determine the need for thorough examination and therapeutic intervention for people who are vulnerable to ASD and who do not have intellectual disability.