Internet Gaming Disorder Among Slovenian Primary Schoolchildren: Findings From a Nationally Representative Sample of Adolescents.

Background and aims Since the inclusion of Internet Gaming Disorder (IGD) in the latest (fifth) edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) as a tentative disorder, a few psychometric screening instruments have been developed to assess IGD, including the 9-item Internet Gaming Disorder Scale - Short-Form (IGDS9-SF) - a short, valid, and reliable instrument. Methods Due to the lack of research on IGD in Slovenia, this study aimed to examine the psychometric properties of the IGDS9-SF in addition to investigating the prevalence rates of IGD in a nationally representative sample of eighth graders from Slovenia (N = 1,071). Results The IGDS9-SF underwent rigorous psychometric scrutiny in terms of validity and reliability. Construct validation was investigated with confirmatory factor analysis to examine the factorial structure of the IGDS9-SF and a unidimensional structure appeared to fit the data well. Concurrent and criterion validation were also investigated by examining the association between IGD and relevant psychosocial and game-related measures, which warranted these forms of validity. In terms of reliability, the Slovenian version IGDS9-SF obtained excellent results regarding its internal consistency at different levels, and the test appears to be a valid and reliable instrument to assess IGD among Slovenian youth. Finally, the prevalence rates of IGD were found to be around 2.5% in the whole sample and 3.1% among gamers. Discussion and conclusion Taken together, these results illustrate the suitability of the IGDS9-SF and warrants further research on IGD in Slovenia.

Introduction

Internet Gaming Disorder (IGD) was recently included as a tentative disorder in the latest (fifth) edition of the American Psychiatric Association’s (APA) Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2013) due to its widely reported negative impacts on both mental and physical health of a minority of individuals (Griffiths, 2015; Griffiths, Király, Pontes, & Demetrovics, 2015; Griffiths & Pontes, 2015; King, Gainsbury, Delfabbro, Hing, & Abarbanel, 2015; Kuss & Griffiths, 2012; Pontes & Griffiths, 2014). Although reliable prevalence rates of IGD across many populations are lacking due to the limitations in the assessment (Petry, Rehbein, Ko, & O’Brien, 2015), findings from studies including representative samples have reported IGD rates of approximately 8.5% in American youth aged 8–18 years (Gentile, 2009), 1.2% in German adolescents aged 13–18 years (Rehbein, Kliem, Baier, Mößle, & Petry, 2015), 5.5% in Dutch adolescents aged 13–20 and 5.4% in Dutch adults (Lemmens, Valkenburg, & Gentile, 2015), 4.3% in Hungarian adolescents aged 15–16 years (Király et al., 2014), 1.4% in Norwegian gamers (Wittek et al., 2015), and 1.6% in youth from seven European countries aged 14–17 years (Müller et al., 2015).

In terms of the definition and clinical diagnosis of the phenomenon, IGD comprises a behavioral pattern encompassing persistent and recurrent engagement with both online and offline games, leading to significant impairment or distress over a 12-month period. Accordingly, IGD may be indicated when a person endorses five or more out of the nine following criteria: (a) preoccupation with games; (b) withdrawal symptoms when gaming is taken away; (c) tolerance, resulting in the need to spend increasing amounts of time engaged in games; (d) unsuccessful attempts to control participation in games; (e) loss of interest in previous hobbies and entertainment as a result of, and with the exception of, games; (f) continued excessive use of games despite knowledge of psychosocial problems; (g) deceiving family members, therapists, or others regarding the amount of gaming; (h) use of games to escape or relieve negative moods; and (i) jeopardizing or losing a significant relationship, job, or education or career opportunity because of participation in games (APA, 2013).

Although IGD is not yet recognized as an official clinical entity (Petry & O’Brien, 2013), several new promising psychometric tools covering the nine IGD criteria have been developed following the formal recognition of this condition by the APA in May 2013 (e.g., Király, Sleczka, et al., 2015; Lemmens et al., 2015; Pontes & Griffiths, 2015; Pontes, Király, Demetrovics, & Griffiths, 2014). However, most of the new measures to assess IGD are arguably lengthy and lack cross-cultural validation. As a result of this, Pontes and Griffiths (2015) developed the Internet Gaming Disorder Scale – Short-Form (IGDS9-SF), a concise psychometric tool that fully captures the nine IGD criteria suggested by the APA via nine Likert-type items, and provides brevity as an attempt to overcome question fatigue in time-limited surveys. The IGDS9-SF was originally validated in a sample of 1,397 English-speaking gamers from 58 different countries and was reported to have adequate psychometric properties (Pontes & Griffiths, 2015). More recently, the IGDS9-SF was also validated in a sample of 509 Portuguese adolescents (Pontes & Griffiths, 2016) and the findings from this study further supported the cross-cultural validity and reliability of the IGDS9-SF as a suitable psychometric tool to assess IGD in different cultures.

Despite the relatively slow progress made in the best way to assess the construct, several researchers have urged for the need of unification in the field of assessment of IGD (see Griffiths, King, & Demetrovics, 2014; Petry et al., 2014; Pontes & Griffiths, 2014) and even though efforts have been made to bridge this gap (e.g., Petry et al., 2014), assessing IGD still lacks broad international consensus as noted by leading researchers in the field in recent debates (Griffiths et al., 2016). Furthermore, there is a general scarcity of robust cross-cultural studies employing the nine IGD criteria (Király, Griffiths, & Demetrovics, 2015; Petry & O’Brien, 2013). Petry et al. (2014) suggested that “establishing the psychometric properties of instruments assessing these nine [IGD] criteria should begin using a cross-cultural perspective.” (p. 6). Given this rationale, the aim of this study was threefold to: (a) examine the psychometric properties of the IGDS9-SF in a nationally representative sample of Slovenian primary school students; (b) provide reliable prevalence rates of IGD in Slovenia; and (c) provide empirical evidence supporting the usefulness of the IGDS9-SF to assess IGD using the APA’s framework with the aim of contributing to the unification of IGD assessment.

Methods

Participants and procedures

This study was part of the Slovenian project “For the Health of the Young” (the full name of the project was “Healthy lifestyle of children and youth through the empowerment of youth workers and the establishment of programs on a local level” http://eeagrants.org/project-portal/project/SI05-0007) which was supported by Norway Grants and conducted in partnership of National Institute of Public Health and Youth Association No Excuse. Recruitment of participants was based on a random probability sample selection with all primary school classes (eighth grade) in Slovenia serving as the primary sampling units. The stratification was based on 12 statistical regions of Slovenia according to the Nomenclature of Territorial Units for Statistics (i.e., NUTS 3) and population density. School principals of the randomly selected primary schools were contacted and invited to participate in the study. The questionnaires were administered by trained staff and completed by the students using desktop computers in the school setting in April and May 2015. Informed consent was obtained from parents/legal guardians and participating adolescents. Consequently, 1,095 questionnaires were administered and cases with severe missing values (i.e., ≥10%) were excluded from the study. The data were weighted according to the 12 Slovenian statistical regions to represent the general population of eighth graders. After cleaning the data, a final representative sample size of 1,071 eighth graders was obtained. Participants’ age ranged from 12 to 16 years (M = 13.44 years; SD = 0.59). Of those, 89% were born in 2001, and gender distribution was relatively even (i.e., 50.2% male).

Measures

Sociodemographic and educational variables

The survey assessed participants’ age, gender, and region of residency in Slovenia.

Game-related variables

These comprised a set of variables that assessed if participants had played any videogames in the last 12 months (yes/no); age they first played videogames; and time spent in gaming during weekdays and weekends (hr).

Subjective psychosocial status

This included the following questions: “How satisfied are you with your life?” (from 1 = “completely dissatisfied” to 10 = “totally satisfied”); and “How would you assess your mental health?” [1 = “bad;” 2 = “satisfactory;” 3 = “good;” 4 = “very good;” and 5 = “excellent”].

Internet Gaming Disorder Scale – Short-Form (IGDS9-SF)

To date, the IGDS9-SF is the shortest psychometric tool developed that assesses IGD based on the nine criteria suggested by the APA (2013). The IGDS9-SF assesses the severity of IGD and its detrimental effects by examining both online and/or offline gaming activities occurring over a 12-month period via nine questions that are answered using a 5-point scale: 1 = “never;” 2 = “rarely;” 3 = “sometimes;” 4 = “often;” and 5 = “very often.” The scores are obtained by summing the responses and total scores can range from 9 to 45, with higher scores indicating higher degree of gaming disorder. Due to the lack of diagnostic information regarding the IGDS9-SF, a more strict diagnostic approach of endorsement of five or more of the nine IGD criteria as assessed by the IGDS9-SF should be employed. Consequently, only items where participants responded “very often” should be considered as an endorsement of an individual IGD criterion (Pontes & Griffiths, 2015).

Data management, analytic strategy, and statistical analysis

Data management procedure further included analyzing the distribution (i.e., univariate normality) of the IGDS9-SF items using standard recommendations (i.e., Skewness > 3 and Kurtosis 9) (Kline, 2011) alongside screening for both univariate (i.e., scores ± 3.29 standard deviations from the IGDS9-SF z-scores) (Field, 2013) and multivariate outliers (using Mahalanobis distances and the critical value for each case based on the chi-square distribution values) within the sample. Statistical analyses included: (a) descriptive analysis of the main sample’s characteristics; (b) assessment of the construct validity of the IGDS9-SF by performing a confirmatory factor analysis (CFA); (c) concurrent validity analysis by examination of the Pearson’s correlation coefficients between the IGDS9-SF scores and the two subjective psychosocial measures; (d) criterion validity of the IGDS9-SF by examining the Pearson’s correlation coefficients between the IGDS9-SF scores and the number of hours spent in gaming on both weekdays and weekends alongside their coefficients of determination (R2); (e) analysis of the scale’s reliability using the Cronbach’s alpha, composite reliability, and factor determinacy coefficients of internal consistency; and (f) estimation of prevalence rates of IGD [and its confidence interval (CI)] among participants using a procedure for the estimation of CI of a proportion outlined by Newcombe (1998). All analyses were performed with the aid of the Mplus 7.2 (Muthén & Muthén, 2012) and IBM SPSS Statistics 20 (IBM Corp, 2011) statistical packages.

Ethics

The study procedures were carried out in accordance with the Declaration of Helsinki. The Institutional Review Board of Nottingham Trent University (United Kingdom) and the National Institute of Public Health (Slovenia) approved the study. All subjects were informed about the study and all provided written informed consent. Parental written informed consent was provided by parents.

Results

Descriptive statistics

Table 1 summarizes all relevant descriptive information collected from the sample. The majority of participants (53.3%) were either from Osrednjeslovenska region, n = 281, 26.3%, Podravska region (n = 154, 14.4%), or Savinjska region (n = 135, 12.6%). Moreover, 82.1% (n = 852) of the sample reported having played a videogame in the last 12 months with the mean age of 7 years (SD = 2.35) being the age participants remembered playing videogames for the first time. Overall number of hours spent in gaming on weekdays (Mean = 1.71 hr, SD = 2.19) and weekends (Mean = 2.49 hr, SD = 2.85) were relatively low.

Table 1.

Main sociodemographic characteristics, frequency, and history of gameplay among the sample (N = 1,071)

Sociodemographic characteristics
Age (years) (Mean, SD)	13.44 (0.59)a
Gender (male) (n, %)	537 (50.2)
Statistical region of residency (n, %)a
Osrednjeslovenska	281 (26.3)
Podravska	154 (14.4)
Savinjska	135 (12.6)
Gorenjska	112 (10.5)
Jugovzhodna Slovenia	79 (7.4)
Goriška	63 (5.9)
Pomurska	60 (5.6)
Obalno–kraška	49 (4.6)
Spodnjeposavska	41 (3.8)
Koroška	40 (3.8)
Zasavska	28 (2.6)
Notranjsko–kraška	27 (2.5)
Game-related variables
Played any game in the last 12 months (yes) (n, %)	852 (82.1)a
Age first played games (years) (Mean, SD)	7.36 (2.35)a
Time spent in gaming on weekdays (hr) (Mean, SD)	1.71 (2.19)a
Time spent in gaming on weekends (hr) (Mean, SD)	2.49 (2.85)a

Variable presenting with missing values.

Construct validity: Confirmatory Factor Analysis (CFA)

Construct validity of the IGDS9-SF was investigated by performing a CFA on the instrument’s nine indicators using maximum-likelihood estimation method with robust standard errors (MLR) to test the one-factor solution of the IGD construct as previously demonstrated (Pontes & Griffiths, 2015, 2016). Conventional fit indices and thresholds were used to examine the goodness of fit of the model under analysis: χ2/d.f. [1;4]; Root Mean Square Error of Approximation (RMSEA) [0.05;0.08], RMSEA 90% CI with its lower limit close to 0 and the upper limit below .08; probability level value of the test of close fit (Cfit) > .05; Standardized Root Mean Square Residual (SRMR) [0.05;0.08]; Comparative Fit Index (CFI); and Tucker–Lewis Fit Index (TLI) [.90;.95] (Bentler, 1990; Bentler & Bonnet, 1980; Hooper, Coughlan, & Mullen, 2008; Hu & Bentler, 1999). The CFA yielded the following results: χ2 (27) = 51.57, χ2/d.f. = 1.91; RMSEA = 0.030 [90% CI: 0.017–0.042], Cfit = .99; SRMR = 0.022; CFI = .99; and TLI = 98. The results of the CFA produced acceptable standardized item loadings (i.e., λ ≥ .50, p < .0001) (see Figure 1). In sum, the results obtained clearly demonstrated that the unidimensional solution presents an excellent fit to the data.

Figure 1.

Graphical summary of the Confirmatory Factor Analysis (CFA) results obtained from the nine items if Internet Gaming Disorder Scale – Short-Form (IGDS9-SF) (N = 1,071)

Concurrent validity

To achieve some degree of concurrent validity, a test should show statistically significant correlations with other measures to which it is theoretically related (Frick, Barry, & Kamphaus, 2010). In the context of the present research, concurrent validity was assessed by investigating the degree of association between IGDS9-SF total scores and participants’ self-reported satisfaction with life and overall mental health status because past studies have shown that IGD impairs satisfaction with life and mental health (Festl, Scharkow, & Quandt, 2013; Kim et al., 2016; Männikkö, Billieux, & Kääriäinen, 2015; Peng & Liu, 2010; Rehbein, Psych, Kleimann, Mediasci, & Mößle, 2010). The results from the Pearson’s correlation analysis between both self-rated life satisfaction (r[990] = –.11, R2 = .01, p < .0001) and self-rated mental health (r[998] = –.12, R2 = .01, p < .0001) and IGD, yielded statistically significant results in the expected direction, providing preliminary support for the IGDS9-SF’s concurrent validity.

Criterion-related validity

Criterion validation of the IGDS9-SF was conducted using the time spent in gaming on weekdays and on the weekends as the two external criteria of the instrument. The selection of such criteria should be based on variables that are indicators or measures of the same variable the instrument (i.e., IGDS9-SF) intends to measure (i.e., time spent in gaming) (Rubin & Babbie, 2009). Moreover, several studies have shown that IGD is commonly associated with greater frequency of gameplay (Fuster, Carbonell, Pontes, & Griffiths, 2015; Lemmens, Valkenburg, & Peter, 2009; Pápay et al., 2013). In line with the previous findings, the results of this analysis revealed a statistically significant and positive association between IGD and the number of hours spent in gaming both during weekdays (r[494] = .47, R2 = .22, p < .0001) and weekends (r[539] = .52, R2 = .27, p < .0001), lending further support to the measure’s criterion-related validity.

Reliability analysis

The internal consistency of the IGDS9-SF as given by the Cronbach’s alpha was excellent (α = .93) according to standard interpretation guidelines (George & Mallery, 2003) and could not be further improved by deletion of any item. In terms of composite reliability of the IGDS9-SF, the coefficient obtained was .93, which is higher than the threshold of .70 (Fornell & Larcker, 1981; Hair, Black, Babin, & Anderson, 2010). Finally, factor determinacy for the IGDS9-SF was .97, which is well over the conventional threshold of .80 (Muthén & Muthén, 2012). Together, these findings suggest that the IGDS9-SF possesses excellent internal consistency levels given the high coefficients obtained.

Prevalence rates of Internet Gaming Disorder (IGD) in Slovenia

Prevalence rates of IGD were estimated based on data from the whole sample (N = 1,071) and a gamer-only sample (i.e., those answering “yes” to the question asking if they had played any videogames during the past 12 months) (N = 852). According to the APA, IGD leads to clinically significant impairment or distress as indicated by at least five out of the nine proposed criteria for IGD (APA, 2013). In the context of this study, a criterion was considered endorsed by answering with “very often” to any of the nine criteria assessed by the IGDS9-SF. Hence, similar to APA’s recommendation, participants endorsing at least five out of the nine criteria in this study were operationally defined as potentially meeting a positive IGD diagnosis (Pontes & Griffiths, 2015). Accordingly, 2.5% (n = 26) [95% CI: 1.7–3.7%] of the whole sample was classed as potentially presenting with IGD, as opposed to 3.1% (n = 26) of the gamer-only sample [95% CI: 2.1–4.5%].

Discussion and Conclusion

The main aims of this study were to provide psychometric evidence regarding the validity and reliability of the IGDS9-SF alongside the prevalence rates of IGD in a representative sample of Slovenian eighth graders. To achieve these aims, the IGDS9-SF underwent rigorous psychometric scrutiny regarding its validity (i.e., construct, concurrent, and criterion-related validity) alongside an assessment of its internal consistency. In addition, the prevalence rates based on the data collected were estimated to allow cross-cultural comparisons between past and future studies assessing IGD.

In terms of construct validity, the results obtained suggested that the IGDS9-SF has a one-dimensional factor structure. Previous studies using the IGDS9-SF have also reported similar findings as to the test’s dimensionality (Pontes & Griffiths, 2015, 2016). Regarding the assessment of concurrent validity, although the strength of the correlation coefficients obtained were not particularly high, these results appear to be in line with the past research that found IGD to be associated with decreased satisfaction with life and poorer overall mental health (Festl et al., 2013; Ko, Yen, Chen, Chen, & Yen, 2005; Männikkö et al., 2015; Mentzoni et al., 2011) as the same was observed in this study as participants that tended to report less self-rated satisfaction with life and poorer mental health tended to exhibit higher levels of IGD.

This finding may be useful for clinicians because individuals’ subjective psychosocial status may be a relevant psychological marker to be used in clinical assessment of IGD. Criterion validation of the IGDS9-SF was also examined in this study, and as expected, higher levels of disordered gaming were associated with higher gaming frequency of gaming (i.e., increased number of hours spent in gaming) during both weekdays and weekends. These results appear to be consistent with previous studies that reported similar associations between IGD and time spent engaged in gaming (Fuster et al., 2015; Lemmens et al., 2009; Pontes & Griffiths, 2015; Pontes et al., 2014). Finally, as for the internal consistency of the IGDS9-SF, the results obtained support the claim that the IGDS9-SF is psychometrically reliable given that all three coefficients of reliability were excellent as they were above the recommended thresholds. Overall, the results obtained from the psychometric examination of the IGDS9-SF suggest that the present findings replicate those found in previous similar studies (Pontes & Griffiths, 2015, 2016).

Since there are currently no available data on IGD in Slovenia, one of the aims of this study was to provide robust prevalence rates of IGD in Slovenia utilizing a valid and reliable psychometric assessment tool in a sample of eighth graders. Findings demonstrated that prevalence rates of IGD were relatively low in the sample. More specifically, IGD rates were 2.5% in the whole eighth grade Slovenian population and 3.1% among Slovenian eighth grade gamers. These findings may stimulate cross-cultural research on IGD, since a very few studies have used random and representative samples to estimate the prevalence rates of IGD using the latest diagnostic framework provided by the APA. The prevalence figures obtained in this study are very similar to those reported in the previous studies using representative samples that found IGD prevalence rates to be around 1.4% in the Norwegian general population (Wittek et al., 2015), 3.6% in early adolescents from five European countries (i.e., Estonia, Germany, Italy, Romania, and Spain) (Strittmatter et al., 2015), 1.6% in European adolescents from seven European countries (i.e., Germany, Greece, Iceland, Netherlands, Poland, Romania, and Spain) (Müller et al., 2015), 3% in Dutch adolescents (Van Rooij, Schoenmakers, Vermulst, Van Den Eijnden, & Van De Mheen, 2011), and 4.3–6.7% in Hungarian adolescents (Király et al., 2014).

Although the findings reported were both strong and robust, there are potential limitations worth highlighting. First, the present results concern only Slovenian adolescents from the eighth grade and therefore cannot be generalized to the wider population. Second, satisfaction with life and overall mental health status were assessed with a single self-rated question rather than a validated instrument, which may be arguably problematic, since single items may not completely subsume a latent construct as well as a standardized psychometric tool. Third, self-reported data may be prone to various biases such as social desirability and memory recall biases among others. Regardless of these potential limitations, the present findings may be useful to the ongoing debate as to whether IGD merits recognition as a clinical entity. In addition, future studies could benefit from replicating these findings in studies using individuals clinically diagnosed with IGD as data on clinical samples are currently sparse. In addition to this, empirical studies using Item Response Theory (IRT) could be utilized to better understand the psychometric and diagnostic features of each of the nine IGD criteria. This would also contribute to current debates on the relevance and adequacy of such criteria (Griffiths et al., 2016). Finally, the present findings suggest that a minority of adolescents appear to have psychosocial problems that are associated with IGD. Therefore, it would be reasonable to develop new prevention and treatment policies to help mitigate IGD within the general population of adolescents as this condition may lead to serious and harmful health-related consequences.

Authors’ contribution

HMP has helped in the design of the study, analyzed the data, and drafted the manuscript. MM has provided methodological guidance on the design of the study throughout all stages, prepared the data for statistical analyzes, and contributed on the review process of the manuscript. MDG has helped designing the study, provided assistance on the interpretation of the results, and helped in reviewing the manuscript.

Conflict of interest

No competing financial interests exist.