Reliability and validity of the Korean version of the internet addiction test among college students.

We developed a Korean translation of the Internet Addiction Test (KIAT), widely used self-report for internet addiction and tested its reliability and validity in a sample of college students. Two hundred seventy-nine college students at a national university completed the KIAT. Internal consistency and two week test-retest reliability were calculated from the data, and principal component factor analysis was conducted. Participants also completed the Internet Addiction Diagnostic Questionnaire (IADQ), the Korea Internet addiction scale (K-scale), and the Patient Health Questionnaire-9 for the criterion validity. Cronbach's alpha of the whole scale was 0.91, and test-retest reliability was also good (r = 0.73). The IADQ, the K-scale, and depressive symptoms were significantly correlated with the KIAT scores, demonstrating concurrent and convergent validity. The factor analysis extracted four factors (Excessive use, Dependence, Withdrawal, and Avoidance of reality) that accounted for 59% of total variance. The KIAT has outstanding internal consistency and high test-retest reliability. Also, the factor structure and validity data show that the KIAT is comparable to the original version. Thus, the KIAT is a psychometrically sound tool for assessing internet addiction in the Korean-speaking population.

INTRODUCTION

Internet addiction is a new clinical entity defined as a maladaptive pattern of internet use causing clinically significant impairment or distress to affected individuals (1). Official diagnostic criteria for internet addiction do not exist yet, however, and the disorder has been considered as either impulse control disorder (1) or behavior addiction (2). The upcoming Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5) will include internet addiction in its appendix (3). The prevalence of internet addiction varies according to the methodology and population studied, but in some countries such as Korea, it is substantial; for example, it was estimated that 8.5% of the total population are currently affected by this disorder (4). It is therefore not difficult to understand why the Korean Government termed internet addiction a serious public health issue and established an independent government agency for policy making and for treatment of those suffering from the problem (5).

Internet addiction has been also designated pathologic internet use (6), compulsive internet use (7), and problematic internet use (8). Although there are some minor differences between the proposed diagnostic criteria, all share common elements such as excessive use of internet, withdrawal, tolerance, and negative consequences for interpersonal or personal well-being (9). Several tools have been developed and tested for their psychometric properties; these include the Internet Addiction Test (IAT) (10), Generalized Problematic Internet Use Scale (11), and Korea internet addiction scale (12). Among these, the IAT has been most widely used and well-tested for its psychometric properties (13). This 20-item Likert-type questionnaire was developed for screening and measuring levels of internet addiction. Each item is rated from 1 (rarely) to 5 (always) and total scores can range from 20 to 100. Although the norms and cut-off score of the IAT are not established, Young has suggested the score above 70 causes significant problems (10). Items of the IAT include compulsive behavior related to use of the internet, the occupational or academic difficulties, lack of competence at home, problems in interpersonal relations, and emotional problems (10).

The excellent psychometric properties of the original version are well documented in the literature (13), and good reliability and validity data have been reported for other language versions, thus suggesting the adaptability of the IAT to other cultures. These languages include Chinese (14), French (15), Italian (16), Portuguese (17), Finnish (18), German (19), and Malay (20). In Korea, two major translated versions have been used in (21, 22), and studies used them often with minor modifications depending on populations targeted. Psychometric data of the Korean versions are available including good internal consistencies (Cronbach alpha 0.79-0.94) and mixed results for factor structure (23). Criterion validities have not reported and test-retest reliability was shown in only one study (24); furthermore, during the development no process of back-translation was conducted, which may limit the cross-cultural adaptability of the original scale (25). Therefore, in this study, we developed a Korean version of the IAT (KIAT) through a process of forward and back translation and examined its reliability and validity in a sample of college students.

MATERIALS AND METHODS

Participants

The participants were undergraduate students from Kongju National University in Chungnam Province, Korea. Recruitment began with in-campus advertisements from three departments. Volunteering students had to sign a written informed consent and complete the questionnaire including demographic data, time spent on the internet, and psychological measures. The final sample was 279 participants. Of these, 177 (62.8%) were women, and the average age was 19.9 (SD = 2.7) yr. KIAT average score was 32.9 (SD = 9.4). About a half (51.4%) of the participants described themselves as moderate internet users, 36.2% as under-users, and 12.1% as excessive users. Daily work-related use of the internet was less than an hour for 83.0%, between one and two hours for 12.1%, and more than two hours for 4.3%. Seventy-two percents of the participants spent less than an hour daily for non work-related use, 20.2% between one and two hours, and 6.4% more than two hours. Non-random sample of participants (n = 174, 62.4%) were retested with the KIAT after two weeks.

Measures

Translation and back-translation

We obtained permission from Dr Kimberly Young to translate the IAT and use it in a psychometric study. The forward and backward translation process was done in accordance with a guideline for developing other language version of questionnaire (25), except for a pre-test. Three mental health professionals who were fluent in both Korean and English translated and created the initial draft, which was back translated by a professor majoring in English language, and, after carefully reviewing the back translation, a final version (KIAT) was produced. The preliminary research raised concern about validity of item 7, "How often do you check your e-mail before something else that you need to do?" as this is the only item concerning a specific use of the internet and the item was found to have poor factorial validity (26, 27). Thus we substituted the term, "email" with a more general one, "the internet."

Internet Addiction Diagnostic Questionnaire

The Internet Addiction Diagnostic Questionnaire (IADQ) was made from based on the criteria of DSM-IV pathological gambling (1). It consisted of eight questions for the diagnosis of internet addiction. Addiction was defined as answering "yes" to five or more of the eight items.

The Korea Internet addiction scale

The Korea Internet addiction scale (K-scale) is a self-questionnaire to measure tendency for Internet addiction (24). The original 40-item version was later condensed to form a 20-item short form (27). This Likert type scale has response set from 1 ("never") to 4 ("always"), thus total scores lie between 20 and 80. Excellent Cronbach's alpha values were found for the short form, used in this study, among elementary (0.89) and middle school students (0.91) (27).

The Patient Health Questionnaire-9

The Patient Health Questionnaire-9 (PHQ-9) is an assessment tool for screening and assessing the severity of depression (28). It consists of nine items based on DSM-IV diagnostic criteria for major depressive disorder, and asks respondents how often they experienced these problems during the previous two weeks. Four-point responses to each item range from 0 ("not at all") to 3 ("almost daily"), so that total scores are between 0 and 27. The Korean version used in this study had good reliability and validity (29). PHQ-9 was used to assess the convergent validity of the KIAT since a close association of depression with internet addiction has been consistently reported in the literature (30).

Statistical analysis

In order to estimate the internal consistency of the KIAT, Cronbach's alpha was calculated. We used Pearson's correlation analyses to determine the test-retest reliability, concurrent validity, and convergent validity. Principal component analysis with varimax rotation was conducted to determine the factor structure underlying the KIAT items.

All statistical tests were two-sided. Statistical significance was set at a value of P < 0.05. Statistical Analysis PASW statistics software version 18.0 (SPSS Inc., Chicago, IL, USA) was used for data entry and statistical analyses.

Ethics statement

The study protocol was approved by the institutional review board of Gongju National Hospital (IRB No. 2012-06). Written informed consent was obtained from all participants.

RESULTS

Reliability

Cronbach's alpha of the KIAT with 20 items was 0.91 and removal of individual items caused values to range between 0.90 and 0.91. Item-to-total scale correlations (Pearson r) were between 0.43 and 0.67, but it was 0.25 for item 4 (Table 1). Two-week test-retest reliability was substantial (r = 0.73) confirming temporal stability.

Table 1

Mean, corrected item-total correlation, and Cronbach's alpha of the KIAT

KIAT, Korean version of the Internet Addiction Test.

Factorial validity

Based on an eigenvalue-greater-than-one principle, our principal component analysis extracted four factors that accounted for 58.9% of the variance (Table 2). Factor I encompasses items describing internet over-use and failure to control time (Q1, Q5, Q7, Q17, Q14, and Q16). It also covers ensuing performance problems at work and school (Q2, Q6, and Q8). These were designated "Excessive internet use". Factor 2, "Dependence" involves social substitution (Q3 and Q19) and emotional dependence (Q11, Q12, and Q15). Factor 3, "Withdrawal" contains items about fear of being withdrawn (Q13 and Q18), and withdrawal symptoms (Q20). Final Factor 4, "Avoidance of reality" contains three items (Q4, Q9, and Q10).

Table 2

Principal component analysis and internal consistency of the Korean version of the Internet Addiction Test (n=279)

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization. Loadings greater than 0.3 are shown.

Concurrent and convergent validity

Table 3 summarizes the concurrent and convergent validity of the KIAT. The total scores of the KIAT were significantly correlated with other established measures of internet addiction (i.e., K-scale and IADQ) and with depressive symptoms. Level of depression, which is theoretically related to internet addiction, was also significantly related, thus, providing good support for convergent validity of the KIAT.

Table 3

Correlation between scores of the Internet Addiction Test and other scales

*Correlation is significant at the 0.01 level (2-tailed). KIAT, Korean version of the Internet Addiction Test; K-scale, Korea Internet addiction scale; IADQ, Internet Addiction Diagnostic Questionnaire; PHQ-9, The Patient Health Questionnaire-9.

DISCUSSION

In this study, we translated and adapted the IAT to the Korean language and found good reliability and validity of the translated version. First, the internal consistency was excellent (Cronbach's alpha > 0.90), this value is better than those that have been reported for the original version (13) but similar to other language versions (15, 17). And item-to-total correlations and Cronbach's alpha values with deletion of individual items showed that the internal consistency was generally stable. However, one exception was item 4; it had a low correlation, and overall internal consistency exceeded that of total items when the item was deleted. We therefore had to exclude the item for the factor analysis. Item 4 concerns newly formed social relations on the internet: "How often do you form new relationships with fellow on-line users?" We believe that our result reflects recent change in the internet environment where many young people now build their social relationships through social-networking service such as Facebook (31). The issue of the validity issue of item 4 was also raised in two recent factor analytic studies: one of Korean college students (26) and the other of US students (32). Therefore, item 4 nowadays has more relevance to an average pattern of internet use rather than being a construct for internet addiction. In line with change in pattern of internet use, we propose that the item 4 needs to be revised.

Our study is one of a few studies to investigate the test-retest reliability of the IAT. One Korean study using a different translation of the IAT reported two-week correlation of r = 0.85 among high school students (23). A recent German study reported similar two-week reliability of r = 0.83 among college students (19). Our study also confirmed the temporal stability of the KIAT among college students.

In our exploratory factor analysis, four factors were extracted. Others have proposed various factor solutions: one factor (15, 18), two factor (19, 31), three (33, 34), five (20), and six factors (13, 16, 17). These variations may be explained by differences in language versions (culture or translation), population studied (online sample or college students), and methods of factor extraction. Our finding of five factors is new but is in line with common elements in the instruments measuring internet addiction: 1) compulsive internet use and excessive time spent; 2) withdrawal symptoms; 3) using the internet for social comfort; 4) negative consequences (34).

The six-factor structure found in the first factor analytic study of the IAT by Widyanto and McMurran (13) is of limited significance as these authors recruit a small online sample of 86 participants of diverse backgrounds and nationality. Further studies failed to replicate this factor solution, although one Portuguese study (17) extract six factors from a group of university students, but the items clustered in each domain coincided only partially with the original version. Recent studies on larger samples of students support a fewer factors: Jelenchick et al. (32) identified two factors (dependent use and excessive use) among 215 US college students; Korkeila et al. (18) and Barkes et al. (19) supported two factor solution among university students. A recent study on Korean university students also found two-factor solution as the best fitting model for the IAT (34). This two-factor structure was similar to that identified in the US and Finnish study (18, 31). The items clustered as Factor 1 in our study are identical to "Excessive Use" and Factor 2, 3, 4 are items in "Dependent Use" in the study of Jelenchick et al. (32). Thus, although the number of factors in our exploratory factor analysis is larger than in these studies, our finding point to similarity to different language versions in factorial validity of the IAT.

Convergent validity of the KIAT was demonstrated by the significant correlation with depression, which is one of the most commonly reported symptomatic correlates of internet addiction (35). Other studies have reported convergent validity of the IAT with time of internet use and specific online activities (14), and with frequency of internet use (35). The concurrent validity of the KIAT was shown by demonstrating significant correlation with other established measures of internet addiction. Studies reported significant correlations of the IAT with the Compulsive Internet Use Scale and Chen Internet Addiction Scale (36).

Limitations of this study were as follows. First, participants in this study were students from a single university who volunteered through school advertisements. There needs to be a careful consideration for representativeness of this sample for the method of sampling was not randomized. Second, we did not investigate the detailed activities undertaken on the internet, which may have yielded insight into aspects of internet overuse. Third, as the KIAT is a self-administered scale, we cannot rule out effects of denial or minimization on the part of the respondents (37). Future study may benefit from combined use of questionnaires by spouses or parents. Finally, our study did not investigate the discriminant validity and diagnostic utility of the KIAT; for example, cut-off scores between normal and pathological internet users and comparison with clinical interviews for internet addiction disorder will be necessary. Our results need to be replicated with other populations including adolescents, community population, and those seeking mental health services. And to shed more light on factor structure of the KIAT, confirmatory factor analysis is required to confirm our finding and to compare with other factor solutions suggested from previous studies.

The significance of this study is as follows: first, we confirmed the test-retest reliability and concurrent validity of the KIAT, which has hardly examined in the literature. Second, although there existed two older Korean versions of the IAT, only our version was produced by backward translation, which is an important procedural element when one requires cross-cultural adaptation of a scale. Third, by changing the item 7 we were able to extract a more stable factor structure and achieve better construct validity. Thus, with respect to the revised version of the IAT, we recommend that "email" in item 7 should be reworded as "the internet" and that item 4 should be deleted or altered to reflect recent changes in the significance of social networks in the medium of the internet.

In conclusion, the KIAT had an excellent internal consistency and high test-retest reliability. It also has concurrent validity as shown by the significant correlation with other scales reflecting internet addiction. A four-factor structure, comparable to the original version, suggests adequate factorial validity of the KIAT. The KIAT is a sound psychometric measure that can be used for screening for, and research on, internet addiction among the Korean-speaking population.