Psychometric evaluation of the Persian Internet Disorder Scale among adolescents.

BACKGROUND AND AIMS: Given the growing epidemiological research interest concerning Internet addiction, brief instruments with a robust theoretical basis are warranted. The Internet Disorder Scale (IDS-15) is one such instrument that can be used to quickly assess the Internet addiction in an individual. However, only two language versions of the IDS-15 have been developed. This study translated the IDS-15 into Persian and examined its psychometric properties using comprehensive psychometric testing.
METHODS: After ensuring the linguistic validity of the Persian IDS-15, 1,272 adolescents (mean age = 15.53 years; 728 males) completed the IDS-15, Depression Anxiety Stress Scale (DASS), Internet Gaming Disorder Scale - Short Form (IGDS9-SF), and the Bergen Social Media Addiction Scale (BSMAS). Confirmatory factor analysis (CFA), Rasch models, regression analysis, and latent profile analysis (LPA) were carried out to test the psychometric properties of the Persian IDS-15.
RESULTS: Both CFA and Rasch supported the construct validity of the Persian IDS-15. Multigroup analysis in CFA and differential item functioning in Rasch indicated that male and female adolescents interpreted the IDS-15 items similarly. Regression analysis showed that the IDS-15 correlated with IGDS9-SF and BSMAS (ΔR2 = .12 and .36, respectively) is stronger than the DASS (ΔR2 = .03-.05). LPA based on IDS-15 suggests three subgroups for the sample. Significant differences in depression, anxiety, IGDS9-SF, and BSMAS were found among the three LPA subgroups.
CONCLUSION: The Persian IDS-15 has robust psychometric properties as evidenced by both classical test theory and Rasch analysis.

Introduction

Recent research has reported that the median percentage of Internet usage in 2015 was 54% across 21 emerging and developing countries (e.g., Malaysia and mainland China) and 87% across 11 advanced economies (e.g., the United States and Canada; Pew Research Center, 2016). Given the high prevalence of Internet use and smartphone use, research has also indicated that children and adolescents are at risk of addiction to Internet-related activities (Pontes, Szabo, & Griffiths, 2015; Yang, Chen, Huang, Lin, & Chang, 2017). Consequently, over the past two decades, Internet addiction has received growing interest and attention from healthcare professionals, including researchers and clinicians (Kuss, Griffiths, Karila, & Billieux, 2014; Pontes, Kuss, & Griffiths, 2015). Although Internet-related activities provide convenience and enjoyment to individuals, a large amount of research consistently indicated the negative impacts of Internet addiction among a small minority of individuals (e.g., Rücker, Akre, Berchtold, & Suris, 2015; Sariyska, Reuter, Lachmann, & Montag, 2015; Wartberg, Kriston, Kammerl, Petersen, & Thomasius, 2015; Weinstein, Dorani, Elhadif, Bukovaz, & Yarmulnik, 2015). For example, Sariyska et al. (2015) reported that Internet addiction is associated with attention-deficithyperactivity disorder and depression. Wartberg et al. (2015) reported that Internet addiction was associated with impaired family functioning, lowered life satisfaction, and problematic family interaction among adolescents. Weinstein et al. (2015) reported that Internet addiction was related to social anxiety among young adults. Rücker et al. (2015) suggested that Internet addiction was correlated with substance abuse, poor emotional well-being, and decreased academic performance among adolescents. Moreover, a recent nationally representative study by Macur and Pontes (2018) found that Slovenian adolescents presenting high risk of Internet addiction exhibited poorer levels of self-control and were generally less satisfied with their lives in comparison with adolescents presenting low risk of Internet addiction.

The fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) working group led by Petry acknowledged that the healthy discussion among experts and the available empirical evidence led the DSM-5 to include a subtype of Internet addiction, Internet gaming disorder (IGD; Griffiths et al., 2016; Petry et al., 2014). However, the research in the Internet addiction field lacks consensus in terms of definition and characterization of the phenomenon. Consequently, many different instruments assessing Internet addiction have been developed, often with inconsistent criteria (Petry, Zajac, & Ginley, 2018). The discrepancies among the Internet addiction instruments often relate to the diverse theoretical basis, different factorial structure and psychometric properties (Király, Nagygyörgy, Koronczai, Griffiths, & Demetrovics, 2014; Pontes & Griffiths, 2017). Another flaw of these instruments is that most of them (as noted by Kuss et al., 2014) are based on outdated criteria of pathological gambling and/or substance dependence in the DSM-IV (American Psychiatric Association [APA], 1994) rather than the more updated clinical criteria, such as those developed for IGD in the latest (fifth) edition of the DSM-5 (APA, 2013) and those for Gaming Disorder in the latest (11th) revision of the International Classification of Diseases (Petry et al., 2018). Therefore, Internet addiction instruments developed to date using DSM-IV criteria may not be directly capturing the concept of Internet addiction.

Researchers have suggested the appropriateness of applying the IGD criteria to assess Internet addiction (e.g., Pontes & Griffiths, 2017; Rumpf et al., 2015). To the best of the authors’ knowledge, only two instruments [the Internet Addiction Scale and the Internet Disorder Scale (IDS-15)] based on the DSM-5 IGD criteria have been developed to assess Internet addiction (Cho et al., 2014; Pontes & Griffiths, 2017). However, the Internet Addiction Scale (Cho et al., 2014) suffers from several weaknesses, including being lengthy (41 items), which is a disadvantage for time-limited research, as well as rather weak psychometric properties (e.g., low internal consistency of .49 and .65 reported by Cronbach’s α). In contrast, IDS-15 has only 15 items with much stronger psychometric properties (α = .79–.85; Pontes & Griffiths, 2017).

The IDS-15 has been validated only in English and Italian (Monacis, Sinatra, Griffiths, & de Palo, 2018; Pontes & Griffiths, 2017). Therefore, this study translated the IDS-15 into Persian due to the large population of Persian-speaking people globally (approximately 120 million people in Iran, Afghanistan, and Tajikistan). In addition, Internet-based behaviors may differ across different cultures. Studies examining the psychometric properties of the IDS-15 in other countries and languages are thus warranted. Moreover, to be confident that the IDS-15 is psychometrically robust, the psychometric properties should be tested repetitively, especially using a variety of statistical methods in different populations (or ethnicities). Because the two previous psychometric studies on IDS-15 mainly used classical test theory, this study used Rasch models to test the psychometric properties of the IDS-15 as a way of providing additional support to the limited literature concerning the IDS-15.

Because adolescence is a critical transition period (i.e., rendering them more vulnerable to addictive behaviors and other disorders), many adolescents experience extensive physical and psychological changes (Lin & Tsai, 2016) that lead to emotional distress (Strong, Tsai, Lin, & Cheng, 2016) and excessive Internet use (Skoog, Stattin, & Kerr, 2009). In addition, because of the significant physical, biological, cognitive, and social changes during adolescence, many mental health disorders first emerge during this period (Davidson, Grigorenko, Boivin, Rapa, & Stein, 2015). Therefore, adolescents are at high risk for developing Internet addiction. Fortunately, Internet addiction can be diagnosed early and early intervention may be provided, if there are brief and psychometrically robust instruments to screen those with potential problems. Given the aforementioned rationale, this study rigorously tested the psychometric properties of the IDS-15 among adolescents in a Middle East country (i.e., Iran). Two types of psychometric testing (classical test theory and Rasch) were applied to comprehensively and thoroughly examine the psychometric properties of the IDS-15. Given that males and females have different characteristics and patterns in using the Internet (Tsai, Strong, Chen, Lee, & Lin, 2018; Yang, Lin, Huang, & Chang, 2018), the psychometric testing also investigated whether the IDS-15 can be appropriately used for both genders. In addition, latent profile analysis (LPA) was conducted to cluster the participants into several latent subgroups across different levels of Internet addiction and to further assess the characteristics of the subgroups.

Methods

Participants and procedures

This study was conducted from September to December 2017. A multistage sampling approach was used to select the study sample from 58 high schools in Qazvin. Twenty-two high schools were randomly selected from a list of high schools provided by the Organization for Education at Qazvin. Two classes were then randomly selected in each school. Of the 1,483 adolescents who were initially approached, only 1,272 (85.7%) agreed to participate in the study.

With respect to the translation process, a standardized procedure was adopted with three stages (Beaton, Bombardier, Guillemin, & Ferraz, 2000; Khoshnevisan et al., 2012) to translate the English IDS-15 into Persian. First, the English IDS-15 was independently translated into two Persian forward translation versions by two bilingual translators whose native language was Persian. Second, a merged Persian version was synthesized from the two forward translations by the two aforementioned translators and a project manager. Third, the merged Persian version was independently back-translated into two English backward translations by two native English translators who were fluent in Persian. All translated versions (two forward translation versions, one merged Persian version, and two backward translation versions) were then reviewed by an expert committee comprising a psychologist, pediatrician, psychiatrist, psychometrician, public health specialist, health education specialist, and nurse to ensure semantic, idiomatic, experiential, and conceptual equivalencies. After the review from the committee, an interim Persian version was generated and pretested on a sample of Iranian adolescents (n = 34, 19 boys; mean age = 15.10 years).

Instruments

Internet Disorder Scale (IDS-15)

The IDS-15 consists of four subscales comprising escapism and dysfunctional emotional coping (EDEC; four items), withdrawal symptoms (WS; four items), impairments and dysfunctional self-regulation (IDSR; four items), and dysfunctional Internet-related self-control (DISC; three items). All the items are rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree) with higher scores on the IDS-15 indicating a higher likelihood of Internet addiction (Monacis et al., 2018). The English IDS-15 had satisfactory internal consistency (α = .79–.85), construct validity, and criterion-related validity (Pontes & Griffiths, 2017).

Depression Anxiety Stress Scale (DASS)

The DASS consists of three 7-item subscales comprising depression, anxiety, and stress. All items are rated on a 4-point Likert scale ranging from 0 (did not apply to me at all) to 3 (applied to me very much, or most of the time) with higher scores indicating higher levels of psychological distress (Lovibond & Lovibond, 1995). The Persian DASS had acceptable internal consistency (α = .84 for depression, .91 for anxiety, and .86 for stress) and supported the factorial structure reported in previous studies (Asghari, Saed, & Dibajnia, 2008; Lin, Broström, Nilsen, Griffiths, & Pakpour, 2017).

Internet Gaming Disorder Scale – Short Form (IGDS9-SF)

The IGDS9-SF assesses IGD using nine items and was developed based on the IGD criteria in the DSM-5 (Pontes & Griffiths, 2015). All the items are rated on a 5-point Likert scale ranging from 1 (never) to 5 (very often) with a higher score indicating a higher likelihood of IGD. The Persian IGDS9-SF has satisfactory internal consistency (α = .90), promising construct validity, and supported measurement invariant across gender (Wu et al., 2017).

Bergen Social Media Addiction Scale (BSMAS)

The BSMAS (Andreassen et al., 2016) was adapted from the Bergen Facebook Addiction Scale (Andreassen, Torsheim, Brunborg, & Pallesen, 2012) and consists of six items comprising six core addiction elements (salience, mood modification, tolerance, withdraw, conflict, and relapse) proposed by Griffiths (2005). All the items are rated on a 5-point Likert scale ranging from 1 (very rarely) to 5 (very often), and a higher score indicates a greater likelihood of addiction to social media (Andreassen et al., 2016). More recently, Bányai et al. (2017) reported that a score on the BSMAS of over 19 (out of 30) represents at-risk problematic use of social media. The Persian BSMAS has satisfactory internal consistency (α = .86), promising construct validity, and supported measurement invariance across gender (Lin et al., 2017).

Data analysis

Psychometric testing using classical test theory

Ceiling and floor effects were conducted for each subscale of the IDS-15 and the IDS-15 total score, and a percentage of <20% is deemed as acceptable (Jette, Warren, & Wirtalla, 2005). Internal consistency was performed using Cronbach’s α and a value >.7 indicates satisfactory reliability (Cheng, Luh, Yang, Su, & Lin, 2016). Corrected item-total correlation was calculated to demonstrate how an item adheres to its construct and a value of >.4 is deemed as acceptable (Wang, Wang, & Shee, 2007). Standard error of measurement was computed to assess how the observed scores were attributed from the measurement errors, and a low value is anticipated (Cheng et al., 2016). Test–retest reliability was assessed using intraclass correlational coefficient and a value >.7 represents being acceptable.

Construct validity was examined via confirmatory factor analysis (CFA) using diagonally weighted least squares means and variance estimation, and average variance extracted and composite reliability were additionally computed using the factor loadings and uniqueness values retrieved from the CFA; average variance extracted >0.5 and composite reliability >0.6 are the recommended cutoffs (Bagozzi & Yi, 1988; Fornell & Larcker, 1981). Two CFA structures were examined according to the suggestions from previous studies on IDS-15 (Monacis et al., 2018; Pontes & Griffiths, 2017). More specifically, (a) a second-order structure with four first-order factors (EDEC, WS, IDSR, and DISC) and one second-order factor (IDS) and (b) a first-order structure with the aforementioned four first-order factors, which are mutually correlated in the structure. Each IDS-15 item was then evaluated using both loading and significance levels, where a loading >0.4 (Brown, 2006) with statistical significance indicates an acceptable item. A series of fit indices were used to determine whether the structure of the IDS-15 was supported: comparative fit index (CFI = 0.90) and Tucker–Lewis index (TLI >0.95) indicate acceptable fit and excellent fit, respectively (Hu & Bentler, 1998); root mean square error of approximation (RMSEA) <0.05, <0.08, and >0.10 indicate close, reasonable, and poor fit, respectively (Browne & Cudeck, 1993); standardized root mean square residual (SRMR) <0.08 (Cheng et al., 2016; Lin et al., 2012). Concurrent validity was assessed using the associations between the IDS-15 and other psychometric instruments (DASS, IGDS9-SF, and BSMAS). More specifically, several regression models were constructed using IDS-15 as the dependent variable with one of the following independent variables: DASS depression subscale, DASS anxiety subscale, DASS stress subscale, IGDS9-SF, and BSMAS. Moreover, two confounders (age and gender) were included for all the regression models as suggested in a previous study (Pontes, 2017).

Multigroup CFA with three models was used to examine the measurement invariance of the IDS-15 in the scale level. The three models included a configural model (Model 1 – a first-order IDS-15 framework with all items loading on the underlying concept of Internet addiction), a metric invariance model (Model 2 – a model based on Model 1 to constrain all factor loadings being equal across gender), and a scalar invariance model (Model 3 – a model based on Model 2 to additionally constrain all item intercepts being equal across gender). Models 1 and 2 were compared for metric invariance and Models 2 and 3 for scalar invariance (Bagheri, Jafari, Tashakor, Kouhpayeh, & Riazi, 2014); ΔCFI >−0.01, ΔSRMR <0.01, and ΔRMSEA <0.015 suggested measurement invariance across groups (Chen, 2007).

Psychometric testing using Rasch analysis

Rasch partial credit model was used to report item difficulty with the unit of logit and to examine item validity using information-weighted fit statistic (infit) mean square (MnSq) and outlier-sensitive fit statistic (outfit) MnSq. The main reason for using a partial credit model rather than a graded response model is because the intervals between every two descriptors in the Likert-type scale were different in every item. A good fit of an item has both infit and outfit MnSq between 0.5 and 1.5 (Jafari, Bagheri, & Safe, 2012). A low MnSq indicates redundancy of an item, and a high MnSq indicates misfit of an item (Khan, Chien, & Brauer, 2013; Strong, Lin, Tsai, & Lin, 2018). Item and person reliability were generated and a value >.7 is suggested as acceptable (Chang, Wang, Tang, Cheng, & Lin, 2014).

In addition, differential item functioning (DIF) was conducted based on the Rasch models to test measurement invariance across gender at item level. More specifically, DIF tests the measurement invariance for each IDS-15 item. Hence, DIF clearly identify which items for males as compared with females are easier or harder to fulfill the item description (Lin, Ou, et al., 2018; Lin, Pakpour, et al., 2018). A commonly used cutoff of >0.5 was used in the DIF contrast (i.e., the difference of difficulty between two groups) for indicating substantial DIF (Shih & Wang, 2009).

LPA and comparisons among different subgroups

LPA is a person-centered approach that helps to identify latent subgroups of individuals within a population. LPA was used to classify the adolescents into categories based on the four self-reported level of the risk of Internet disorder using IDS-15 subscale scores. The optimal number of classes was selected based on five fit statistics: the Akaike information criterion (AIC), the Bayesian information criterion (BIC), the sample-size adjusted BIC (SSABIC), Lo–Mendell–Rubin (LMR) Likelihood Ratio Test (LRT), entropy and bootstrap LRT. Lower AIC, BIC, and SSABIC values suggest better-fitting models while the entropy measure ranges from 0 to 1 with higher scores demonstrating a better classification quality. In addition, the LMR in the LRT examines whether a model with k profiles provides a significant improvement in fit over a model with k−1 profiles. A non-significant value (i.e., p > .05) indicates that the model with one fewer class provides a more parsimonious fit to the data. Age and gender were examined if they were associated with each profile of membership. To examine sociodemographic characteristics as well as addiction scales as predictors of risk Internet addiction, Multinomial regression using the R3Step method was used. The data were analyzed using Mplus (version 7.4; Los Angeles, CA) and Winstep software (version 3.70.0.2; Winstep Software Technologies, Chicago, IL).

Ethics

The study procedures were carried out in accordance with the Declaration of Helsinki, and the study was approved by the Ethics committee of Qazvin University of Medical Sciences. All participants were informed about the study, fully understood the study purpose, and all provided informed consent. It should also be noted that parental consent was sought for those participants younger than 18 years of age. For this reason, all parents of adolescents younger than 18 years were encouraged to attend a session providing information about the study. Those adolescents whose parents gave their consent were contacted and asked to participate in the study after describing the study aims and procedures.

Results

Participant characteristics

The mean age of the participants was 15.53 years (SD = 1.2; n = 1,272; 728 males). Their average scores on the questionnaires were 46.97 for IDS-15 (SD = 9.78), 7.5 for DASS depression subscale (SD = 4.6), 8.1 for DASS anxiety subscale (SD = 5.0), 7.6 for DASS stress subscale (SD = 4.7), 12.6 for IGDS9-SF (SD = 3.8), and 13.6 for BSMAS (SD = 2.8) (see Table 1 for further details).

.

Participants’ characteristics (N = 1,272)

	Mean ± SD or n (%)
Age (years)	15.53 ± 1.2
Gender (male)	728 (57.2)
Family income
Poor (<$600)	136 (10.7)
Fair ($600–1,200)	814 (64.0)
Good (>$1,200)	320 (25.2)
Father’s educational year	7.8 ± 3.8
Mother’s educational year	6.1 ± 3.5
Internet Disorder Scale total score	46.97 ± 9.78
EDEC subscale score	4.14 ± 0.71
WS subscale score	2.62 ± 0.77
IDSR subscale score	3.67 ± 1.03
DISC subscale score	2.66 ± 0.84
DASS depression subscale score	7.5 ± 4.6
DASS anxiety subscale score	8.1 ± 5.0
DASS stress subscale score	7.6 ± 4.7
Internet gaming disorder	12.6 ± 3.8
Social media addiction	13.6 ± 2.8

Note. SD: standard deviation; EDEC: escapism and dysfunctional emotional coping; WS: withdrawal symptoms; IDSR: impairments and dysfunctional self-regulation; DISC: dysfunctional Internet-related self-control; DASS: Depression Anxiety Stress Scale.

Psychometric properties of the IDS-15 at item level

All items in the IDS-15 had acceptable factor loadings derived from the second-order CFA (0.64–0.86), adequate values in corrected item-total correlation (0.76–0.89), and satisfactory test–retest reliability (0.70–0.88). In addition, all items exhibited excellent fit in the Rasch analyses: infit MnSq between 0.81 and 1.23 and outfit MnSq between 0.80 and 1.34. No substantial DIF was displayed for all the items across gender (DIF contrasts = −0.50–0.44). Moreover, the difficulty of the item was between −1.69 and 1.27 logit (Table 2).

.

Psychometric properties of the Internet Disorder Scale (IDS-15) at the item level

	Analyses from classical test theory			Rasch analyses
Item no.	Factor loadinga*	Item-total correlation	Test–retest reliabilityb	Infit MnSq	Outfit MnSq	Difficulty	DIF contrast across genderc,d
EDEC-1	0.77	0.83	0.81	0.99	1.05	−1.69	0.00
EDEC-2	0.64	0.78	0.72	1.23	1.34	−1.05	−0.16
EDEC-3	0.80	0.83	0.78	0.85	0.90	−1.31	0.08
EDEC-4	0.76	0.82	0.77	0.99	1.00	−1.38	0.17
WS-1	0.65	0.86	0.82	0.94	0.97	1.20	−0.33
WS-2	0.68	0.89	0.88	1.01	1.08	1.27	−0.13
WS-3	0.71	0.88	0.79	0.95	1.02	0.98	−0.49
WS-4	0.70	0.87	0.73	0.86	0.92	0.88	−0.50
IDSR-1	0.80	0.76	0.74	1.08	1.08	−0.54	0.31
IDSR-2	0.83	0.78	0.70	1.15	1.13	−0.45	0.21
IDSR-3	0.86	0.79	0.86	0.92	0.89	−0.50	0.33
IDSR-4	0.83	0.78	0.81	1.07	1.03	−0.48	0.44
DISC-1	0.69	0.83	0.73	0.99	1.02	1.18	0.17
DISC-2	0.65	0.81	0.79	1.05	1.10	1.06	−0.06
DISC-3	0.80	0.83	0.80	0.81	0.80	0.82	0.35

Note. EDEC: escapism and dysfunctional emotional coping; WS: withdrawal symptoms; IDSR: impairments and dysfunctional self-regulation; DISC: dysfunctional Internet-related self-control; MnSq: mean square error; DIF: differential item functioning.

aBased on the second-order confirmatory factor analysis. bUsing intraclass correlation coefficient. cDIF contrast >0.5 indicates substantial DIF. dDIF contrast across gender = Difficulty for females − Difficulty for males.

All factor loadings were significant at p = .001.

Psychometric properties of the IDS-15 at scale level

No substantial ceiling effect (0%–17.9%) or floor effect (0%–3.9%) was found in the IDS-15 total score and the subscale scores. Reliability conducted using classical test theory (Cronbach’s α = .76–.90 and composite reliability = 0.76–0.90) and Rasch analyses (person separation reliability = 0.71–0.85 and item separation reliability = 0.71–0.98) was satisfactory. CFA showed that the fit statistics of the IDS-15 were excellent in CFI (0.96) and TLI (0.95) and acceptable in RMSEA [90% confidence interval (CI) = 0.054–0.064]. The CFA results support the structure of the IDS-15 (Figure 1). Average variance extracted values were also adequate (0.47–0.56) and standard errors of measurement were relatively low (Table 3).

.

Second-order confirmatory factor analysis with standardized factor loadings. EDEC: escapism and dysfunctional emotional coping; WS: withdrawal symptoms; IDSR: impairments and dysfunctional self-regulation; DISC: dysfunctional Internet-related self-control; IDS-15: Internet Disorder Scale

.

Psychometric properties of the Internet Disorder Scale (IDS-15) at the scale level

Psychometric testing	EDEC	WS	IDSR	DISC	IDS-15
Composite reliability	0.83	0.78	0.90	0.76	0.86
Average variance extracted	0.55	0.47	0.69	0.51	0.56
Ceiling effects (%)	17.90	14.40	0	0	0
Floor effects (%)	0.20	1.40	2.40	3.90	0
Internal consistency (Cronbach’s α)	.83	.90	.78	.76	.90
Person separation reliability	0.72	0.85	0.73	0.71	–
Item separation reliability	0.98	0.71	0.97	0.97	–
Standard error of measurement	0.32	0.33	0.36	0.41	3.10

Note. EDEC: escapism and dysfunctional emotional coping; WS: withdrawal symptoms; IDSR: impairments and dysfunctional self-regulation; DISC: dysfunctional Internet-related self-control.

Concurrent validity of the IDS-15 was supported by the significant associations between IDS-15 and DASS depression subscale (β = 0.114, p < .05), DASS anxiety subscale (β = 0.148, p < .01), DASS stress subscale (β = 0.120, p < .01), IGDS9-SF (β = 0.371, p < .001), and BSMAS (β = 2.308, p < .001). Moreover, the associations between IDS-15 and two Internet-related questionnaires (IGDS9-SF and BSMAS; ΔR2 = .12 and .36, respectively) were stronger than those between IDS-15 and DASS subscales (Table 4). Multigroup CFA demonstrated that male and female adolescents interpreted the IDS-15 items similarly (ΔCFI = −0.002 and −0.007, ΔSRMR = 0.001 and 0.004, ΔRMSEA = 0.003 and 0.002; Table 5).

.

Concurrent validity of the Internet Disorder Scale (IDS-15) using regression models with adjustment for age and gender

Criterion	β (p value)	ΔR2	Overall R2 (adjusted R2)
Depressiona	0.114 (<.05)	.03	.167 (.165)
Anxietya	0.148 (<.01)	.05	.274 (.241)
Stressa	0.120 (<.01)	.03	.324 (.301)
IGDS9-SF	0.317 (<.001)	.12	.474 (.470)
BSMAS	2.308 (<.001)	.36	.763 (.461)

Note. ΔR2: overall R2 − the R2 derived from age and gender; that is, indicating the explained variance of the criterion; IGDS9-SF: Internet Gaming Disorder Scale – Short Form; BSMAS: Bergen Social Media Addiction Scale.

Depression, anxiety, and stress were measured using Depression Anxiety Stress Scale.

.

Measurement invariance across gender on Internet Disorder Scale (IDS-15) through confirmatory factor analysis

Models and comparisons	Fit statistics
	χ2 (df)	Δχ2 (Δdf)	CFI	ΔCFI	SRMR	ΔSRMR	RMSEA	ΔRMSEA
M1: configural	769.46 (168)*		0.928		0.051		0.053
M2: plus all loadings constrained	790.55 (183)*		0.921		0.052		0.056
M3: plus all intercepts constrained	810.51 (198)*		0.920		0.056		0.058
M2−M1		21.09 (15)		−0.007		0.001		0.003
M3−M2		19.96 (15)		−0.002		0.004		0.002

Note. M1: Model 1, a configural model; M2: Model 2, a model based on M1 with all factor loadings constrained being equal across groups; M3: Mode 3, a model based on M2 with all item intercepts constrained being equal across groups; CFI: comparative fit index; SRMR: standardized root mean square residual; RMSEA: root mean square error of approximation.

p < .05.

LPA and comparisons among different subgroups

Table 6 presents the fit statistics for the two- to four-profile solutions. Consistent improvement was observed up to the three-profile solution and the four-profile solution decreased the classification quality. Although AIC, BIC, and SSABIC were lowest for the four-profile solution, the LMR value was not significant for the four-profile solution. Therefore, the three-profile solution was the model that best-fitted the data. More specifically, the LMR p value associated with the four-profile model was significant indicating that the three-profile model provides the best fit to the data. Given that the lines in Figure 2 did not intersect, the profile was named based on quantitative differences. In the three-class model, the average latent class probabilities for the most likely class were 0.92, 0.86, and 0.90 for the three latent classes, indicating good model fit and very strong probabilities of class assignment. The first profile (i.e., low Internet addiction risk) included 13.8% adolescents (n = 176), the second profile (i.e., medium Internet addiction risk) had 44.7% adolescents (n = 569), and the final profile (i.e., high Internet addiction risk) contained 41.4% adolescents (n = 527).

.

Latent profile analysis to identify subgroups of participants

	AIC	BIC	SSABIC	Entropy	LMR test (p value)
Two classes	11,332.729	11,399.658	11,358.363	0.818	1,192.523 (<.0001)
Three classes	11,034.981	11,127.651	11,070.474	0.748	299.372 (<.0001)
Four classes	10,926.217	11,044.629	10,971.570	0.713	115.531 (.115)

Note. The bold values indicate the values of the group we used for the final decided group number. AIC: Akaike information criterion; BIC: Bayesian information criterion; SSABIC: sample-size adjusted BIC; LMR test: Lo–Mendell–Rubin Likelihood Ratio Test.

.

Latent profile analysis with three subgroups (low, medium, and high Internet addiction). EDEC: escapism and dysfunctional emotional coping; WS: withdrawal symptoms; IDSR: impairments and dysfunctional self-regulation; DISC: dysfunctional Internet-related self-control

Table 7 shows the results of the multinomial logistic regression for examining associations between sociodemographic variables (independent variables) and Internet addiction profile (dependent variable). The analysis treated the high-risk addiction latent subgroup as the reference group. Factors associated with high-risk addiction included being female and older, higher father’s education, higher anxiety and stress, higher BSMAS total score, IGD weekly Internet usage, and total score of the IDS-15. Regarding the IDS-15 comparison among three subgroups, the first profile yielded the lowest Internet addiction mean score (IDS-15 = 32.83, 95% CI [32.19–33.47]), followed by the second (IDS-15 = 46.27, 95% CI [45.89–46.66]) and the third profiles (IDS-15 = 58.17, 95% CI [57.80–58.53]). In terms of subscales in the IDS-15, the adolescents in the three profiles reported the highest scores on the EDEC compared to three dimensions of the IDS-15 (Figure 2).

.

Predictors of membership in latent profile of Internet addiction risks

	Low addiction risk (n = 176)			Medium addiction risk (n = 569)
	b	SE	OR [95% CI]	b	SE	OR
Age (years)	−1.61	0.71	0.20 [0.05–0.82]	0.10	0.06	1.12 [0.93–1.54]
Gender (female)	1.64	0.31	5.17 [3.17–11.10]	0.38	0.16	1.47 [0.84–3.05]
Mother’s education year	−0.03	0.04	0.97 [0.89–1.06]	−0.02	0.02	0.98 [0.94–1.03]
Father’s education year	−0.11	0.04	0.89 [0.82–0.97]	−0.01	0.02	1.01 [0.97–1.06]
DASS depression score	−0.03	0.02	0.97 [0.93–1.01]	−0.06	0.02	0.94 [0.91–0.98]
DASS anxiety score	−0.13	0.02	0.88 [0.84–0.92]	−0.07	0.02	0.93 [0.90–0.96]
DASS stress score	−0.12	0.02	0.89 [0.85–0.92]	−0.05	0.01	0.96 [0.93–0.98]
BSMAS total score	−1.20	0.07	0.30 [0.26–0.35]	−0.46	0.04	0.63 [0.59–0.68]
Weekly Internet usage	−0.19	0.02	0.83 [0.80–0.86]	−0.08	0.01	0.92 [0.90–0.95]
IGDS9-SF total score	−0.17	0.04	0.85 [0.79–0.91]	−0.05	0.02	0.95 [0.91–0.99]
IDS-15 total score	−1.89	0.13	0.15 [0.12–0.20]	−0.88	0.07	0.42 [0.37–0.47]

Note. The latent profile of high addiction risk was set as the reference group. All odds ratios were adjusted for all variables included in the table. SE: standard error; OR: odds ratio; CI: confidence interval; BSMAS: Bergen Social Media Addiction Scale; IGDS9-SF: Internet Gaming Disorder Scale – Short Form; IDS-15: Internet Disorder Scale; DASS: Depression Anxiety Stress Scale.

Discussion

This study comprehensively and thoroughly assessed the psychometric properties of the Persian IDS-15, an instrument developed using the modified criteria of IGD in the DSM-5 (APA, 2013). Overall, the results obtained for the Persian IDS-15 at the factorial structure and measurement invariance levels were adequate across both item and scale levels. The psychometric properties of the Persian IDS-15 were ascertained using different psychometric frameworks (i.e., classical test theory and Rasch analysis), and the results evidenced the robustness of the IDS-15 in assessing Internet addiction among Iranian adolescents. Male and female participants interpreted the IDS-15 items similarly, and the results supported the use of IDS-15 across gender. Moreover, the LPA showed that the study’s participants could be classified into three latent subgroups based on participants’ risk for developing Internet addiction.

The CFA results were comparable to the previous two studies assessing the psychometric properties of IDS-15 (English version by Pontes & Griffiths, 2017 and Italian version by Monacis et al., 2018). However, the results of this study were more comparable to the findings from Pontes and Griffiths (2017) than those from Monacis et al. (2018). More specifically, Pontes and Griffiths (2017) reported satisfactory CFA without correlating any residual of the IDS-15 item and a three-subgroup finding in their LPA. Conversely, Monacis et al. (2018) presented their CFA with an unexpected covariance (the residual of Item 9 to that of Item 10) and a four-subgroup finding in the LPA. A possible explanation proposed by Monacis et al. (2018) was based on the composition of the recruited sample. In addition to the general populations regardless of their age, Monacis et al. recruited a subgroup from gaming halls. Given that the participants recruited from gaming halls are very likely to have different levels of Internet addiction from those who were recruited from other places, Monacis et al. additionally classified a subgroup as those with critical addiction risk. That is, the participants from gaming halls are likely to include actual addicts and may have also been affected by other behavioral addictions. As a result, the structure of IDS-15 may also rather be influenced by the specific subgroup.

Similar to the studies by Pontes and Griffiths (2017) and Monacis et al. (2018), this study found that the IDS-15 had promising reliability as demonstrated by internal consistency and composite reliability. The study also extended the satisfactory reliability results to test–retest reliability and separation reliability reported by Rasch, providing novel findings. Therefore, a variety of interested stakeholders (e.g., researchers, clinicians, and other healthcare providers) can be assured that the reproducibility of the IDS-15 is stable and that items are coherent to each other. In addition, the findings share similarity with those reported by Monacis et al. (2018) in terms of measurement invariance and concurrent validity. Findings from this study supported the notion that the IDS-15 is measurement invariant across gender, and the study extended the results to the item level; that is, no DIF items were displayed in the Rasch findings. The absence of DIF items and the presence of measurement invariance for the structure of the IDS-15 facilitate meaningful comparisons between groups (Gregorich, 2006; Lin, Ou, et al., 2018). From a statistical and clinical perspective, establishing invariance indicated that respondents in the comparison groups (e.g., male and female groups in this study) responded to the IDS-15 items in the same way (i.e., they share similar processes in the underlying constructs), which suggests that generic treatment and prevention programs are likely to be effective for the comparison groups. In contrast, when DIF occurs (i.e., invariance is not established), the comparison groups may need specifically tailored assessments, treatment, and prevention programs.

In terms of the characteristics of the three subgroups with different risks of Internet addiction, key differences were found in gender, emotional distress, weekly Internet usage, and other addictive behaviors (Internet gaming and social media use). The group at high risk of Internet addiction was generally male, prone to high emotional distress, engaging more Internet usage, and having other Internet-related addiction behaviors. The findings parallel to the results of the previous studies in which being male (e.g., Sasmaz et al., 2014), high weekly Internet usage (e.g., Bouna-Pyrrou, Mühle, Kornhuber, & Lenz, 2015), impaired emotional distress (e.g., Pontes, Caplan, & Griffiths, 2016; Weinstein et al., 2015), and addiction to other behaviors (e.g., Pontes, 2017; Rücker et al., 2015) were found to be associated with Internet addiction.

However, unlike the finding from Pontes and Griffiths (2017) that younger age was related to higher risk of Internet addiction, this study found that age was not significantly different between the three subgroups across different levels of addiction risk. A possible reason is because of the different age ranges. Because in the study by Pontes and Griffiths (2017), subgroups had a wider age range compared to that of this study (15.53 ± 1.2 vs. 33 ± 12.3), it is possible that the age effect was not obvious in the findings here. Nevertheless, this study showed a three-profile solution in the LPA, which corresponds to the LPA solution found by Pontes and Griffiths (2017). Additionally, the results demonstrated that more than 40% of the participants (527 out of 1,272) were classified as a high addiction risk by the LPA. Given that our study is focused on adolescent population, this finding may reflect the results from Pontes and Griffiths (2017) regarding the association between young age and higher risk of addiction. Indeed, other studies have found that adolescents frequently use Internet excessively (e.g., Strong, Lee, Chao, Lin, & Tsai, 2018; Tsai et al., 2018).

There are several limitations in the study. First, the study recruited only adolescents; therefore, the generalizability of our results cannot be applied to those in different life stages (e.g., adults and the elderly). Second, the data were self-report and subject to well-known biases such as memory recall biases and social desirability biases. Third, the adolescent sample data were only collected in one Iranian city and may not necessarily by representative of all Iranian adolescents. Fourth, the participants were recruited from schools; thus, the data were nested and therefore could have impacted on the findings. Finally, because the participants were Iranian, the study is unable to directly compare the IDS-15 between Western and Eastern countries (e.g., whether Western people are more Internet-addicted) or to examine the measurement invariance across different cultures. Researchers and other interested stakeholders (e.g., healthcare professionals) are reminded that using the IDS-15 to compare Western people to Eastern people in assessing Internet addiction requires caution. Based on the aforementioned potential limitations, the present authors would encourage future research to examine the psychometric properties of the IDS-15 across different age groups and different ethnicities that are underresearched such as the one investigated in this study.

Conclusions

This study showed that the IDS-15 had a clear structure of four distinct and interrelated domains, and all items were perfectly embedded in their belonging domain. The Persian IDS-15 also had robust psychometric properties as evidenced by both classical test theory and Rasch analysis. Moreover, no substantial DIF was displayed across gender in this study’s sample. Similarly, measurement invariance was supported for IDS-15 across gender. LPA clustered the participants into three subgroups, and the high-risk Internet addiction group had more males, more weekly Internet usage, higher DASS scores in anxiety and depression, and higher BSMAS and IGDS9-SF scores compared with medium and low Internet addiction groups.