Development and Verification of an Internet Game Literacy Scale.

BACKGROUND: Education on internet games for parents and internet game literacy are needed to prevent problematic internet game playing in Korea. We created an 18-item Internet Game Literacy Scale (IGLS). It is a valuable tool for assessing the positive and negative aspects of internet game play. We aimed to determine the validity of the IGLS and the cut-off for the tendency for internet gameplay.
METHODS: An online research company gathered data from 300 participants. Factor analysis, including Cronbach's α and consistency coefficient, exploratory factor analysis, and confirmatory factor analysis were conducted to verify the 18 items of the IGLS. Additionally, a K-means cluster analysis was performed to determine the cut-off values for positive and negative IGLS scores.
RESULTS: The 18 items of the IGLS were proven to be reliable, as evidenced by a high Cronbach's alpha (α = 0.892). The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.903, and Bartlett's test of sphericity was good (χ² = 1,623.314, P < 0.001). All 18 items were segregated into two factors, with nine items each. The eigenvalue of all 18 items was significant at > 0.4. In the analysis of the validity of the 18-item IGLS with confirmatory factor analysis (CFA) (maximum likelihood estimation, with an oblique method), the fit indices of the standard three-factor model reached acceptable standards. The cut-off point of the total score between the low positive and average positive groups was 23, and the cut-off point of the total score between the average positive and high positive groups was 30. The cut-off point of the total score between the low negative and the average negative groups was 24. The cut-off point of the total score between the average negative group and the high negative group was 32.
CONCLUSION: The study assessed the reliability and validity of the IGLS and suggested a cut-off for low, average, and high Internet game literacy degree with 300 Korean adults aged 21-49 years. The current results suggest that the IGLS has good internal consistency and a proper cut-off for positive and negative internet game literacy degrees.

INTRODUCTION

Internet games are the most popular entertainment applications among online activities. Furthermore, children and adolescents demonstrate high attachment to internet gameplaying.12 In a 2019 Korea Creative Content Agency (KOCCA) survey, about 70% of South Koreans stated that they were playing internet games. Of these, 60% of the survey participants indicated that they played mobile games, while 40% indicated that they played personal computer games.3 A total of 74.4% of Korean adolescents aged 13–19 years played internet games for leisure. Nevertheless, internet gameplaying has been linked to many negative consequences, including failure in school and social activities, job loss, and increased family conflicts.4

In Korea, there have been several problems with adolescents' internet gameplay in recent decades. To prevent problematic internet gameplaying in Korea, the “Game Shutdown System” was implemented on November 20, 2011, and was finally legalized on April 24, 2014.5 The system could ban children aged ≤ 16 years from playing internet games between midnight and 6:00 a.m.5 However, Sung6 reported that the game shutdown system was not effective in preventing internet gaming disorder in children and adolescents. According to critics of the Game Shutdown Law, the law violated children's civil rights. The government could not show that playing internet games was more harmful than watching television or movies, listening to music, or engaging in other indoor activities.7 In Korea, a selective shutdown system has been suggested to overcome the shortcomings of the Game Shutdown System.7 A selective game shutdown system where parents regulate their children's internet gameplay through controlling internet gaming data and login times may be implemented.67 Parents are given control on the regulation of the permission and duration of their children's internet gameplay; hence, their perception of internet games is fundamental in addressing internet gaming disorder in Korea.

Internet games for parents and internet game literacy are needed to reduce the strength of the selective game shutdown system and compensate for these shortcomings. Adolescent patterns of internet use and internet gameplay are closely associated with their parents' perception of internet gameplay.89 The experience of internet gameplay and intimacy for game cultures in adults could narrow the gaps in internet game culture and behaviors between adolescents and adults.10 However, there have been a few scoring systems to determine the degree of internet game literacy. In a focused group interview of KOCCA, 30 parents, teachers, and health caregivers reported that internet games could be a good material for leisure time; internet games could also contribute economically in the Korean industry. However, the players were at risk for problematic internet gameplay.11 Based on these results, the KOCCA 2018 version of the game literacy scale was developed and verified with 217 healthy adults. In an online survey of 1,000 healthy adults, Kim et al.12 classified four internet game perception styles: 1) “online game is an obstacle of study,” 2) “online game is just a game,” 3) “online game is a stain in life,” and 4) “online game is vaguely bad.”

Considering these surveys, we created an 18-item questionnaire called the Internet Game Literacy Scale (IGLS). We hypothesized that the IGLS would be a valuable tool for assessing the positive or negative perceptions towards internet gameplay. We aimed to demonstrate the validity of the IGLS and the cut-off for the tendency for internet gameplay.

METHODS

Participants

An online research company (Embrain®, Seoul, Korea) collected data from 300 participants to verify 18 items of the IGLS (Table 1). They sent invitation e-mail letters to 170,000 members of the Embrain company who had already agreed to participate in the research from January 18, 2021, to January 25, 2021. Their ages ranged from 20 to 49 years. The participants were classified into three groups: 20-29 years old, 30–39 years old, and 40–49 years old. The data of 100 participants (50 males and 50 females) in each age group were accepted in the following order: first answered, first accepted, in order to achieve equal distribution of sex and age among the participants (Table 2). Once the answers of 100 participants in each age group were gathered, no more data were collected in that age group. Each participant received $15 as compensation for the completion of the questionnaire.

Table 1

The items of the Internet Game Literacy Scale

Items of the Internet Game Literacy Scale			Strongly disagree	Disagree	Neutral	Agree	Strongly agree
Nine positive items
	1	Games are accessible and enjoyable for everyone.	1	2	3	4	5
	2	Gaming can make life vivacious.	1	2	3	4	5
	3	Gaming can induce creativity.	1	2	3	4	5
	4	Gaming can reduce stress.	1	2	3	4	5
	5	Gaming can induce concentration.	1	2	3	4	5
	6	Gaming can make someone feel worthy and valuable.	1	2	3	4	5
	7	Gaming can induce anger management and impulse control.	1	2	3	4	5
	8	People can develop friendships and social skills through games.	1	2	3	4	5
	9	Games can contribute to the national industry and economy.	1	2	3	4	5
Nine negative items
	1	People neglect general life activity because of gaming (i.e., traveling, family events etc.).	1	2	3	4	5
	2	Games induce negative habits (i.e., cursing, swearing etc.).	1	2	3	4	5
	3	Even after trying to, it is hard to quit gaming.	1	2	3	4	5
	4	Gaming can induce stress.	1	2	3	4	5
	5	Gaming can induce violence.	1	2	3	4	5
	6	Games can induce negative self-perception in real life.	1	2	3	4	5
	7	Games can induce impulsive behaviors.	1	2	3	4	5
	8	Games can create problems within friends and families (i.e., bullying, disrupting relationships).	1	2	3	4	5
	9	Games ruin ways of living in real life.	1	2	3	4	5

Table 2

General descriptions of the population in this research

General descriptions		Male (n = 150)	Female (n = 150)	Statistics
Age, yr		34.2 ± 8.7	35.8 ± 8.9	t = −1.5, P = 0.141
Education, yr		15.4 ± 3.7	15.2 ± 3.0	t = 0.65, P = 0.140
Income (1,000 Korean won/month)				χ2 = 0.062, P = 0.963
	< 2,000	35 (23.2)	36 (24.0)
	2,000–4000	62 (41.3)	60 (40.0)
	4,000–6000	25 (16.7)	25 (16.7)
	> 6,000	28 (18.7)	29 (19.3)
Jobs				χ2 = 121.91, P = 0.002
	Professionals	23 (15.1)	24 (16.1)
	Office workers	90 (60.0)	89 (59.5)
	Service providers	15 (10.0)	6 (4.0)
	Others	18 (12.0)	8 (5.3)
	No jobs	4 (2.7)	23 (15.3)
Total Internet using time, hour/day		4.9 ± 2.4	4.49 ± 2.7	t = 0.39, P = 0.223
	SNS	0.5 ± 0.5	0.6 ± 0.5	t = −1.12, P = 0.874
	Gaming	0.7 ± 0.7	0.6 ± 0.7	t = 1.04, P = 0.303
	Broadcasting	0.7 ± 0.9	0.5 ± 0.7	t = 1.90, P = 0.063
	Searching	0.4 ± 0.8	0.4 ± 0.6	t = 0.30, P = 0.762
	Shopping	0.1 ± 0.4	0.2 ± 0.5	t = −1.00, P = 0.321
	Others	0.1 ± 0.3	0.1 ± 0.1	t = −0.09, P = 0.924

Values are presented as mean ± standard deviation or number (%).

SNS = social network service.

Development of IGLS

The IGLS was designed to measure the general perception of internet games. Eighteen items were rated using a five-point Likert scale, with 1 representing “strongly disagree,” 2: “disagree,” 3: “neutral,” 4: “agree,” and 5: “strongly agree.” The IGLS consists of nine positive internet game literacy scales and nine negative internet game literacy scales (Table 1). The 18 items of the IGLS were extracted from 11 items (six positive internet game literacy and five negative internet game literacy) in the report of the focused group interview of KOCCA11 and seven items (three positive Internet game literacy and four negative Internet game literacy) from the online game perception type as described by Kim et al.12

Statistical analysis

The factor analysis for items of the IGLS included Cronbach's α and the consistency coefficient. In the analysis of the validity of the IGLS with the exploratory factor analysis (EFA), the Kaiser-Meyer-Olkin Measure (KMO) of sampling adequacy was applied using the extraction method with maximum likelihood, rotation method, and oblimin with Kaiser normalization.

To determine how well the original two-factor model fits the Korean adult sample, we performed a confirmatory factor analysis (CFA). We evaluated five fit indices to examine the model fit: χ2 test, comparative fit index, Tucker-Lewis index (TLI), root-mean-square error of approximation (RMSEA), and standardized root mean square residual (SRMR). The following standards were used to evaluate the model fit: CFI of > 0.90, TLI > 0.90,13 RMSEA ≤ 0.08, and SRMR < 0.08.141516171819

Considering the individual concept of internet game assessed with positive (negative) IGLS scores, a K-means cluster analysis was performed to classify all the participants into three groups: the low positive internet game concept (negative) group, general positive (negative) Internet game concept group, and high positive (negative) internet game concept group.

An independent t-test was used to compare the positive (negative) IGLS scores between the male and female participants. An analysis of variance test was used to compare the positive (negative) IGLS scores between study groups.

All statistical analyses were performed using SPSS version 24 (IBM Corp., Armonk, NY, USA). A result is statistically significant when the P value is < 0.05.

Ethics statement

The Institutional Review Board of Chung Ang University Hospital approved this study (IRB number-2090005434). In addition, the current study was conducted following the principles embodied in the Declaration of Helsinki for all investigations involving humans. Informed consent was obtained from all participants before commencing the study.

RESULTS

Description of the IGLS

The average scores of the positive scale items in the IGLS were 25.29 ± 5.29 (range: 12–38), and the average scores of the negative scale items in the IGLS were 29.13 ± 6.79 (range: 13–45). Both were normally distributed in terms of skewness (positive IGLS: −0.161 ± 0.141, negative IGLS: 0.044 ± 0.172) and kurtosis (positive IGLS: −0.122 ± 0.281, negative IGLS: −0.015 ± 0.281).

Analysis of the reliability of the IGLS

Analysis of the reliability of IGLS for the 18 items

The 18-item IGLS was proven to be reliable, with a high Cronbach's alpha (α = 0.892). Cronbach's alpha for the standardized items was also high (α = 0.891). When each item was eliminated from the IGLS one by one, all the Cronbach's α for the 18 eliminated items was above 0.850, which means that none of the items compromised reliability within the IGLS. The test-retest reliability of the 18 items was r = 0.78 (P < 0.001).

Analysis of the reliability of the IGLS for positive and negative items

The reliability of the IGLS for the nine positive items was proven with a high Cronbach's alpha (α = 0.835). Cronbach's alpha for the standardized items was also high (α = 0.836). When each item was eliminated from the IGLS with nine positive items one by one, all the Cronbach's α for the nine eliminated items was above 0.801, which means that none of the nine positive items compromised reliability within the IGLS (Table 3).

Table 3

Reliability of the Internet Game Literacy Scale

Questions	Scale mean if item deleted	Scale variance if item deleted	Corrected item-total correlation	Squared multiple correlations	Cronbach's α if item deleted
Pos 1	21.6281	26.285	0.467	0.278	0.827
Pos 2	22.2864	23.387	0.618	0.428	0.810
Pos 3	22.6080	23.280	0.628	0.470	0.809
Pos 4	21.7487	25.623	0.449	0.299	0.828
Pos 5	23.0302	22.575	0.684	0.531	0.801
Pos 6	23.1156	23.921	0.596	0.410	0.813
Pos 7	23.2312	23.280	0.620	0.464	0.809
Pos 8	22.6935	24.608	0.438	0.258	0.831
Pos 9	22.5528	24.693	0.419	0.217	0.834
Neg 1	27.3116	37.266	0.724	0.557	0.898
Neg 2	27.3417	37.792	0.702	0.550	0.899
Neg 3	27.3568	36.776	0.720	0.555	0.898
Neg 4	27.8291	39.314	0.651	0.456	0.903
Neg 5	27.6030	38.695	0.683	0.511	0.901
Neg 6	28.0955	39.218	0.616	0.423	0.905
Neg 7	27.3266	37.928	0.779	0.653	0.895
Neg 8	27.9196	38.690	0.608	0.388	0.906
Neg 9	27.4171	36.679	0.750	0.598	0.896

Pos: positive item, Neg: negative item

The reliability of the IGLS for the nine negative items was proven with a higher Cronbach's alpha (α = 0.910). Cronbach's alpha for the standardized items was also higher (α = 0.911). When each item was eliminated from the IGLS with nine negative items one by one, all the Cronbach's α for the nine eliminated items was above 0.895, which means that none of the items of the nine negative items compromised reliability within the IGLS (Table 3).

Analysis of the validity of the IGLS with the EFA

The KMO was 0.903, and Bartlett's test of sphericity was good (χ2 = 1,623.314, P < 0.001) (Table 4). All 18 items were segregated into two factors, with nine items each. The eigenvalue of all 18 items was significant at > 0.4. Under factor 1, nine negative items of the IGLS were segregated with 36.57% variance, 36.57% cumulative percentage, and 6.582 total eigenvalues (Table 4). The range of parameter estimates for factor 1 was 0.627 for negative item 8 and 0.791 for negative item 7. Under factor 2, nine positive items of the IGLS had 15.263% variance, 51.830 cumulative percentage, and 2.747 total eigenvalues. The range of parameter estimates for factor 2 was 0.751 for positive item 5 and 0.430 for positive item 4.

Table 4

Exploratory factor analysis of Internet Game Literacy Scale

Name of factor		Negative Internet Game Literacy Scale	Positive Internet Game Literacy Scale
Eigenvalue total		6.582	2.747
Percentage of variance		36.567	15.263
Cumulative percentage		36.567	51.830
Kaiser-Meyer-Olkin Measure of sampling adequacy			0.903
Bartlett's test of sphericity			Approximate χ2	1,623.314
			Degree of freedom	153
			Significance (P)	< 0.001
Pattern matrix
	Factor	1	2
	Pos 1		0.470
	Pos 2		0.621
	Pos 3		0.733
	Pos 4		0.430
	Pos 5		0.751
	Pos 6		0.656
	Pos 7		0.676
	Pos 8		0.470
	Pos 9		0.497
	Neg 1	0.739
	Neg 2	0.722
	Neg 3	0.757
	Neg 4	0.682
	Neg 5	0.731
	Neg 6	0.696
	Neg 7	0.791
	Neg 8	0.627
	Neg 9	0.731

Extraction method: maximum likelihood, rotation method: Oblimin with Kaiser Normalization.

Pos = positive item, Neg = negative item.

Analysis of the validity of the IGLS with the CFA

In the analysis of the validity of the 18-item IGLS with the CFA (maximum likelihood estimation, with an oblique method), the fit indices of the standard three-factor model reached acceptable standards, CFI = 0.933, TLI = 0.923, RMSEA = 0.062, and SPMR = 0.0594 (90% confidence interval, 0.049–0.075) (Fig. 1). In CFA, the range of factor loadings of the nine positive items was between 0.43 and 0.79, and they were correlated well with one another. The range of factor loadings of the nine negative items was between 0.63 and 0.83, and nine negative items correlated better with one another (Fig. 1).

Fig. 1

Analysis of the validity of the Internet Game Literacy Scale with the confirmatory factor analysis.

K-means cluster analysis of IGLS

We performed a K-means cluster analysis of 300 participants with positive and negative IGLS items. In the positive items of the IGLS, all participants were divided into three groups: low positive internet game concept group (n = 78, final centroid: 19), average positive internet game concept group (n = 173, final centroid: 26), and high positive internet game concept group (n = 49, final centroid: 33). The final Euclidean distance between the low positive internet game concept group and the average positive internet game concept group was 7.478. The final Euclidean distance between the average positive internet game concept group and the high positive internet game concept group was 6.692. The cut-off point of the total score between the low positive and average positive groups was 23, and the cut-off point of the total score between the average positive and high positive groups was 30.

In the negative items of the IGLS, all participants were divided into three groups: low negative internet game concept group (n = 58, final centroid: 19), average negative internet game concept group (n = 128, final centroid: 28), and high negative internet game concept group (n = 114, final centroid: 38). The final Euclidean distance between the low positive internet game concept group and the average positive internet game concept group was 8.106. Further, the final Euclidean distance between the average positive internet game concept group and the high positive internet game concept group was 8.279. The cut-off point of the total score between the low negative and average negative groups was 24. The cut-off point of the total score between the average negative group and the high negative group was 32.

Comparison of IGLS scores between male and female participants as well as between generations

Male participants showed higher positive IGLS scores (t = 3.48, P = 0.002) and lower negative IGLS scores (t = −3.66, P = 0.001) compared to female participants (Table 5). There was no significant difference in the positive IGLS scores between age groups (F = 2.3, P = 0.102). However, negative IGLS scores were higher in participants in their thirties and forties than in those in their twenties (F = 10.6, P < 0.001) (Table 5). Female participants in the forties age group showed the highest negative IGLS scores compared to female participants in the other age groups (Fig. 2). In participants in their twenties, there were no significant differences in the positive and negative IGLS scores between male and female participants (Fig. 2).

Table 5

Comparisons of Internet Game Literacy Scale scores between male and female participants

Scores by age group		Male participants	Female participants	Statistics
Positive Internet Game Literacy Scale
	Total	26.3 ± 4.7	24.3 ± 5.2	t = 3.48, P = 0.002
	20–29	26.4 ± 5.2	25.6 ± 5.8	t = 0.73, P = 0.473
	30–39	26.5 ± 4.6	24.2 ± 4.7	t = 2.51, P = 0.011
	40–49	25.9 ± 4.5	23.1 ± 4.8	t = 3.06, P = 0.003
Negative Internet Game Literacy Scale
	Total	27.7 ± 5.9	30.5 ± 7.2	t = −3.66, P = 0.001
	20–29	25.3 ± 6.1	27.2 ± 6.9	t = −1.41, P = 0.164
	30–39	28.1 ± 6.1	30.9 ± 5.6	t = −2.42, P = 0.023
	40–49	29.7 ± 4.8	33.5 ± 7.7	t = −3.05, P = 0.003

Fig. 2

Comparisons of Internet Game Literacy Scale scores between male and female participants.

(A) Positive Internet Game Literacy Scale scores. (B) Negative Internet Game Literacy Scale scores.

DISCUSSION

The current study assessed the reliability and validity of the IGLS and suggested a cut-off for low, average, and high positive (negative) Internet game literacy degree among 300 Korean adults aged 21–49 years. The results suggest that the IGLS has good internal consistency and a proper cut-off for positive and negative internet game literacy scores.

The internet consistency of the IGLS in all 18 items was high (Cronbach's α = 0.892). The reliability of nine positive 9 IGLS items and nine negative 9 IGLS items was also high (Cronbach's α = 0.836 and Cronbach's α = 0.910, respectively). All items of the IGLS had significant meanings in the EFA. Additionally, the IGLS showed an excellent model of fit in CFA in terms of positive and negative IGLS items. Results suggested that IGLS had good reliability and validity for assessing the internet game perception.

Parental monitoring and restriction for children are known to effectively control healthy internet gaming in children and adolescents.202122 However, Krossbakken et al.23 reported that the psychoeducational parental guide had no effect in preventing problematic internet gameplay in children with internet gaming disorder. These results suggest that parental guides should provide important information to those who already have problems rather than as a means of primary prevention.23 Before providing information, we believe that proper assessment tools for parental internet game concepts should be developed. In that aspect, the IGLS could suggest the educational content of internet game etiquette for parents and caregivers. However, future studies should develop a more detailed IGLS that also considers the genre of games and the age of users.

Furthermore, misunderstanding the concept of internet games could lead to negative stigma in internet games. It might provoke indiscreet criticisms for internet games, despite its benefits.2425 Indiscreet criticism of parents could lead to conflicts between parents and adolescents regarding internet gaming.26 The conflicts due to internet gaming could lead to successive game-related conflicts between parents and adolescents.2627 Thus, neutral concepts for internet games may be necessary for reducing conflicts between parents and children in monitoring children's internet gameplay. Two systematic review papers also suggested adolescents' healthy internet use habits using educational programs based on school and family.2829

In the K-means cluster analysis with 23 and 30 cut-off scores, 300 participants were classified into 78 low positive internet game concept group, 173 in the positive internet game concept group, and 49 in the high positive internet game concept group. In the K-means cluster analysis with 24 and 32 cut-off scores, 300 participants were classified into 58 low negative Internet game concept group, 128 in the positive Internet game concept group, and 114 in the high positive internet game concept group.

As shown in the current study, internet games were perceived more negatively among Korean adults. Due to game companies' attractive marketing for continued playing and negative consequences leading to health problems, internet games might be regarded negatively. Nonetheless, internet games play an integral role in daily life, from communication to entertainment and work.3031

Among the negative consequences, academic failure and irregular life patterns due to problematic internet games could lead to severe conflicts between parents and children in Korea.32 As shown in the current results, the negative perception scores of internet games among female participants in their forties who might have children aged from teenagers in Korea was the highest compared to other generations. Interestingly, participants in their twenties had no differences in negative IGLS scores between male and female participants. Korean female participants in their twenties may be exposed to internet games at an early stage of their life,33 and most of them had no children. For these reasons, we think that they can be perceive internet games positively, and few female participants in their twenties experienced negative consequences of internet gameplay between parents and children.

The current study had several limitations. First, the current study was conducted through an online survey with a relatively small sample size. In particular, members of online survey companies could have different characteristics, such as greater familiarity with active questionnaire surveys or digital activities. The study findings must be interpreted with caution considering the small sample size and inclusion biases. Second, the participants in the study were not assessed through a structured clinical interview. Psychological state and psychiatric comorbidities could affect the corresponding answers and analyses. Future studies should focus on the development of scales and cut-off scores, including psychological and psychiatric status. Finally, current data were not validated using other scales that are used to assess perception types of internet games. Future studies should compare the current IGLS with other scales.

In conclusion, the IGLS may be a reliable, valid, and relevant tool for assessing the perception of internet games. It can be used to assess negative and positive perception on internet gaming.