Psychometric validation of the Persian Bergen Social Media Addiction Scale using classic test theory and Rasch models.

Background and aims The Bergen Social Media Addiction Scale (BSMAS), a six-item self-report scale that is a brief and effective psychometric instrument for assessing at-risk social media addiction on the Internet. However, its psychometric properties in Persian have never been examined and no studies have applied Rasch analysis for the psychometric testing. This study aimed to verify the construct validity of the Persian BSMAS using confirmatory factor analysis (CFA) and Rasch models among 2,676 Iranian adolescents. Methods In addition to construct validity, measurement invariance in CFA and differential item functioning (DIF) in Rasch analysis across gender were tested for in the Persian BSMAS. Results Both CFA [comparative fit index (CFI) = 0.993; Tucker-Lewis index (TLI) = 0.989; root mean square error of approximation (RMSEA) = 0.057; standardized root mean square residual (SRMR) = 0.039] and Rasch (infit MnSq = 0.88-1.28; outfit MnSq = 0.86-1.22) confirmed the unidimensionality of the BSMAS. Moreover, measurement invariance was supported in multigroup CFA including metric invariance (ΔCFI = -0.001; ΔSRMR = 0.003; ΔRMSEA = -0.005) and scalar invariance (ΔCFI = -0.002; ΔSRMR = 0.005; ΔRMSEA = 0.001) across gender. No item displayed DIF (DIF contrast = -0.48 to 0.24) in Rasch across gender. Conclusions Given the Persian BSMAS was unidimensional, it is concluded that the instrument can be used to assess how an adolescent is addicted to social media on the Internet. Moreover, users of the instrument may comfortably compare the sum scores of the BSMAS across gender.

Introduction

Internet accessibility worldwide has increased markedly as a result of the rapid improvement in technology. This development has led to many studies examining the use of various modern online activities, such as social media and Internet gaming (Cheng & Li, 2014; Kuss & Griffiths, 2012; Kuss, Griffiths, Karila, & Billieux, 2014; Mazzoni & Iannone, 2014; Ryan, Chester, Reece, & Xenos, 2014). Although individuals are rarely addicted to Internet itself, activities engaged through the medium of Internet appear to elicit addictive behaviors (e.g., gambling, gaming, and networking) in a small minority of individuals (Griffiths, 2017; Griffiths & Szabo, 2013). Addiction to such Internet-related activities may result in serious and negative psychosocial impact (Griffiths, 2000). Griffiths (2000) has argued that all addictions comprise six core components (i.e., salience, mood modification, tolerance, withdrawal, conflict, and relapse) and that individuals with Internet addiction experience preoccupation and cognitive distortions, deterioration of socialized behaviors, and unpleasant feelings or physical discomfort.

The latest version of the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (i.e., DSM-5), has acknowledged the potential negative impact of addiction to Internet-related activities on the well-being of an individual by recognizing Internet gaming disorder (IGD) as a tentative disorder (American Psychiatric Association, 2013). However, gaming is just one of many potentially addictive activities that can be engaged in online. Social media addiction appears to be a growing mental health problem according to recently reported evidence (e.g., Griffiths, Kuss, & Demetrovics, 2014; Pantic, 2014; Ryan et al., 2014). Therefore, some research has focused on developing a psychometrically robust instrument to assess addiction to social media, such as the six-item Bergen Facebook Addiction Scale (Andreassen, Torsheim, Brunborg, & Pallesen, 2012) and the Facebook Intrusion Questionnaire (Elphinston & Noller, 2011).

Although the two aforementioned scales have demonstrated good psychometric properties, a major limitation is that they do not assess all possible social media, i.e., they assess only one specific social media platform (i.e., Facebook). Moreover, the focus of both scales is on the medium itself rather than on the activities that can be carried out in social media (e.g., playing games such as Farmville; chatting with friends on Facebook; checking the status of a celebrity on Facebook). Given that people rarely become addicted to the Internet or social media platform itself, developing an instrument with accurate terminologies and sentence wordings is needed (Griffiths, 2013).

The Bergen Social Media Addiction Scale (BSMAS) is a newly developed scale that overcomes the aforementioned drawbacks, because it assesses the use of social media activities more generally rather than on one specific platform (Andreassen et al., 2016). However, to the best of our knowledge, the psychometric properties of this scale have only been examined in an Italian sample (Monacis, De Palo, Griffiths, & Sinatra, 2017a) and a nationally representative Hungarian sample (Bányai et al., 2017). Therefore, further psychometric testing of the BSMAS is needed. Both Monacis et al. (2017a) and Bányai et al. (2017) primarily applied classic test theory (CTT) methods to assess the psychometric properties of the BSMAS. Although their studies comprehensively applied a variety of methods to examine psychometric robustness of the BSMAS, they did not use Rasch models in their analysis.

To be confident that an instrument is psychometrically robust, its psychometric properties should be tested utilizing different statistical methods and across different populations (or ethnicities). Therefore, using Rasch models to test the psychometric properties of the BSMAS would provide additional support to the limited literature. The use of Rasch analyses can provide further understanding of the properties of the BSMAS including assessment of: (a) an interval scale to illustrate the BSMAS score (i.e., Rasch score with the unit of logit); (b) the psychometric between-item properties of the BSMAS (i.e., the BSMAS items) and person (i.e., the BSMAS raters); (c) the information about whether an item is redundant or out-of-concept; and (d) the threshold order of the response scales of the BSMAS (i.e., the 5-point Likert scale; Chang, Wang, Tang, Cheng, & Lin, 2014; Tractenberg, 2010).

This study was conducted to test the psychometric properties of the BSMAS in a Middle East country (i.e., Iran) to identify whether the BSMAS can be applied to such an ethnicity (i.e., Persian). First, the BSMAS was translated into Persian for high-school students. In addition, psychometric properties under CTT (e.g., internal consistency, composite reliability, construct validity, criterion-related validity, and floor and ceiling effects) and those under Rasch analyses (e.g., separation item and person properties and categorical functioning of the response scale) of the Persian BSMAS followed by its measurement invariance (using CTT method) and differential item functioning (DIF; using Rasch method) were examined.

Methods

Participants and procedure

The participants of this study were recruited from high schools (students aged 14–19 years) in Qazvin (Iran) using a two-stage sampling method. Thirty high schools were randomly selected from a total of 54 high schools. Then, in each school, three classes were randomly selected to participate in the study. Of 3,075 adolescents who were approached, 2,676 agreed to participate (response rate = 87%). All participants used social media.

Instruments

In addition to a series of standardized questionnaires, a background information sheet was used to collect data regarding participant demographic characteristics, including age, gender, parental education, current smoking status (yes vs. no), and hours spent on social media per day.

BSMAS

The BSMAS, which is adapted from the Bergen Facebook Addiction Scale (Andreassen et al., 2012), consists of six items rated on a 5-point Likert scale ranging from 1 (very rarely) to 5 (very often). The BSMAS applies the six core addiction elements (salience, mood modification, tolerance, withdrawal, conflict, and relapse) proposed by Griffiths (2005) to assess the experience of using social media over the last year (Andreassen et al., 2016). A higher score of the BSMAS indicates stronger addiction to the social media, and a BSMAS score over 19 indicates an individual is at-risk of developing problematic social media use (Bányai et al., 2017). Moreover, the structure of the BSMAS has been identified as a one-factor solution with measurement invariance supported across gender in an Italian sample (Monacis et al., 2017a).

The BSMAS was translated into Persian according to guidelines described by Beaton, Bombardier, Guillemin, and Ferraz (2000) for translation and cross-cultural adaptation of self-report measures. Consequently, several steps were taken. Two bilingual translators whose mother tongue was Persian independently translated the English version of the BSMAS. The two translators then met and resolved any discrepancies between the two translated versions to produce an interim version. The interim Persian version was then back-translated into English by two translators. The translators were not aware of the original English version. An expert committee including a methodologist, psychiatrist, psychologist, and a nurse consolidated all the versions of the BSMAS and developed a penultimate Persian version of the BSMAS for field testing. Finally, the Persian BSMAS was pretested on a sample of 67 high-school adolescents.

Internet Gaming Disorder Scale – Short Form (IGDS-SF)

Based on the concept of IGD proposed in the DSM-5, the IGDS-SF was designed to have nine items using the criteria for IGD in the DSM-5 (Pontes & Griffiths, 2015). All the IGDS-SF items are rated on a 5-point Likert scale ranging from 1 (never) to 5 (very often), and a higher IGDS-SF score indicates a higher level of IGD. The one-factor structure of the IGDS-SF has been supported, and the structure was found to be invariant across gender (Wu et al., 2017). The Persian IGDS-SF has gone through a rigorous method to ensure its linguistic validity. In addition, psychometric results indicated that the Persian IGDS-SF has excellent reliability and validity in a sample of Iranian adolescents (Wu et al., 2017).

Depression Anxiety Stress Scale (DASS)

The DASS comprises 21 items and assesses three psychiatric symptoms (i.e., depression, anxiety, and stress). Each symptom contains seven items that are rated on a 4-point Likert scale ranging from 0 (did not apply to me at all) to 3 (applied to me very much or most of the time; Lovibond & Lovibond, 1995). The internal consistency of the DASS was acceptable (α = .84 for depression, .91 for anxiety, and .86 for stress), and the factorial structure of the Persian DASS was previously confirmed (Asghari, Saed, & Dibajnia, 2008). Because of the previously demonstrated relationship between psychological health and social media addiction (Andreassen et al., 2016), the DASS was used for examining the criterion-related validity of the BSMAS.

Attention deficit hyperactivity disorder (ADHD) Rating Scale

The ADHD Rating Scale is an 18-item checklist based on DSM-IV criteria (Pappas, 2006). All the items are rated on a 4-point Likert scale ranging from 0 (never or not observed) to 3 (often or always). In addition to a total score, the ADHD Rating Scale contains two subscales: The inattention subscale contains all the odd-numbered items, and the hyperactivity subscale contains all the even-numbered items. A higher score indicates more ADHD-related behavioral problems. The two-factor structure of the ADHD Rating Scale has been confirmed, including its Persian version (Ghanizadeh & Jafari, 2010; Mercier et al., 2016). Moreover, the Persian ADHD Rating Scale showed satisfactory reliability and validity (Ghanizadeh & Jafari, 2010). Because of the revealed relationship between ADHD-related behaviors and social media addiction (Andreassen et al., 2016), the ADHD Rating Scale was used for examining the criterion-related validity of the BSMAS. The ADHD Rating Scale was completed by parents.

Data analysis

Descriptive statistics were used to demonstrate the characteristics of the participants and a Pearson’s correlation matrix was used to illustrate the relationships among collected variables. For psychometric testing, both CTT and Rasch methods were used to comprehensively evaluate the psychometric properties of the BSMAS, including the reliability, validity, categorical functioning in response scale, and measurement invariance (or DIF). More specifically, CTT was to investigate the ceiling/floor effects (a percentage of <20% suggests acceptable; Jette, Warren, & Wirtalla, 2005), internal consistency (Cronbach’s α > .7 indicates satisfactory; Cheng, Luh, Yang, Su, & Lin, 2016), corrected item-total correlation (a value of >0.4 is acceptable; Wang, Wang, & Shee, 2007), standard error of measurement (SEM) that assesses how the observed scores were attributed from the measurement errors (a value of

In terms of the construct validity in CTT, confirmatory factor analysis (CFA) was used with diagonally weighted least squares (DWLS) estimator. A DWLS estimator was applied because it is suitable for Likert-type scales (Lin, Burri, Fridlund, & Pakpour, 2017; Lin, Oveisi, Burri, & Pakpour, 2017), such as the BSMAS. A series of fit indices were used: comparative fit index (CFI) and Tucker–Lewis index (TLI) > 0.9; root mean square error of approximation (RMSEA) and standardized root mean square residual (SRMR) < 0.08; non-significant χ2 indicates the support of construct validity (Cheng et al., 2016; Lin et al., 2012). Moreover, the factor loadings were retrieved and the uniqueness values from the CFA were used to compute average variance extracted (AVE) and composite reliability (CR) to additionally understand the construct validity and reliability, respectively, of the BSMAS; AVE > 0.5 and CR > 0.6 are the recommended cutoffs (Bagozzi & Yi, 1988; Fornell & Larcker, 1981).

Multigroup CFA was carried out for testing the measurement invariance of the BSMAS at the scale level. There were three models in the multigroup CFA: configural model (Model 1; i.e., a first-order BSMAS framework with all six items loading on the underlying concept of social media addiction); metric invariance model (Model 2; i.e., a model based on Model 1 to constrain all factor loadings being equal across gender); scalar invariance model (Model 3; i.e., a model based on Model 2 to additional constrain all item intercepts being equal across gender). Measurement invariance in the multigroup CFA was assessed by comparing Model 1 with 2 for metric invariance and Model 2 with 3 for scalar invariance (Bagheri, Jafari, Tashakor, Kouhpayeh, & Riazi, 2014), and the recommendations indicating measurement invariant were ΔCFI > −0.01, ΔSRMR < 0.01, and ΔRMSEA < 0.015 (Chen, 2007).

Criterion-related validity was assessed by constructing several regression models with BSMAS or IGDS-SF as a dependent variable. For the regression model using BSMAS as the dependent variable, there were seven independent variables (IGD, depression, anxiety, stress, inattention, hyperactivity, and daily hours spent on social media) with two confounders (age and gender). For the regression model using IGDS-SF as the dependent variable, there were six independent variables (depression, anxiety, stress, inattention, hyperactivity, and daily hours spent on social media) with two confounders (age and gender). The standardized coefficient (β) was used to demonstrate the criterion-related validity.

The strongest association was expected to be IGDS-SF, followed by daily hours spent on social media, and other psychological (e.g., anxiety and depression) or behavioral problems (e.g., inattention and hyperactivity). These criteria were used because prior studies have demonstrated that social media addiction is weakly correlated to psychological health and ADHD-related behavioral problems, whereas strongly correlated to IGD and daily hours spent on social media (Andreassen et al., 2016; Monacis, De Palo, Griffths, & Sinatra, 2017b). Moreover, it was anticipated that the regression models using BSMAS as the dependent variable would have similar results of those using IGDS-SF as the dependent variable.

In terms of Rasch analyses, a Rasch partial credit model was used to report the item difficulty, and to examine item validity, categorical functioning in the response scale, item and person separation reliability, and item and person separation index. The item difficulty was presented using an interval unit (i.e., logit, a higher logit represents a more difficult item). Item validity was tested using information-weighted fit statistic (infit) mean square (MnSq), and outlier-sensitive fit statistic (outfit) MnSq with a recommended range between 0.5 and 1.5 suggested a good fit (Jafari, Bagheri, & Safe, 2012). A lower value in MnSq indicates an item being more redundant, and a higher value indicates being more out-of-concept (Khan, Chien, & Brauer, 2013). Categorical functioning was used to understand whether the Likert-type response scale of BSMAS had all categories for each item located in their expected orders (Jafari, Bagheri, Ayatollahi, & Soltani, 2012). More specifically, participants who are more addicted to social media are more likely to select a higher point in the BSMAS item (i.e., these respondents would be expected to select “very often” instead of selecting “often”). Item and person reliability with a value >0.7 suggests good reproducibility in the order of item difficulty and person’s underlying ability (Chang et al., 2014). Item and person indices with a value >2 suggest that the BSMAS has the ability to separate individuals or items into more than two distinct groups (Kook & Varni, 2008).

In addition, DIF in the Rasch models was used to test measurement invariance across gender at item level. Unlike multigroup CFA that tests the measurement invariance based on the entire BSMAS structure, DIF tests the measurement invariance for each item. Therefore, DIF can clearly identify which items for males are easier or harder than females to fulfill the item description (Lin, Yang, Lai, Su, & Wang, 2017). The commonly used cutoff for indicating substantial DIF is the DIF contrast (i.e., the difference of difficulty between two groups) >0.5 DIF (Shih & Wang, 2009).

IBM SPSS 23.0 (IBM Corp., Armonk, NY) was used to perform descriptive statistics and the CTT analyses except CFAs. CFAs were analyzed using the lavaan package (Rosseel et al., 2017) in R software; the Rasch models (including the DIF analyses) were carried out using WINSTEPS 3.75.0.

Ethics

The study procedures were carried out in accordance with the Declaration of Helsinki, and the study was approved by the Ethics Committee of Qazvin University of Medical Sciences. All participants were informed about the study, fully understood the study purpose, and all provided an informed consent. It should also be noted that parental consent was sought for those participants younger than 18 years of age.

Results

Table 1 shows the characteristics of the participants (N = 2,676). They had a mean age of 15.54 years (SD = 1.21). Slightly more than half were males (56.5%), and over one-quarter of them were currently smokers (28.5%). On average, they spent 3.75 hr/day on social media (SD = 1.13). The prevalence of being at risk to social media addiction in the sample was 22.4% for whole sample (15.1% for females; 28.1% for males). The correlation coefficients among the variables are shown in Table 2. BSMAS was highly correlated with IGDS-SF (r = .734), and both BSMAS and IGDS-SF were moderately correlated with daily time spent on social media (r = .583 and .476, respectively). Depression was moderately correlated with anxiety (r = .519). All other correlations were low or negligible (r < .25).

.

Participants’ characteristics (N = 2,676)

	Mean ± SD or n (%)
Age (years)	15.54 ± 1.21
Gender (male)	1,511 (56.5)
Fathers’ educational year	7.74 ± 3.90
Mothers’ educational year	6.25 ± 3.59
Currently smoker (yes)a	763 (28.5)
Score in Bergen Social Media Addiction Scale	15.24 ± 4.83
Score in Internet Gaming Disorder Scale – Short Form	22.43 ± 7.91
Score in depressionb	7.70 ± 4.61
Score in anxietyb	8.39 ± 4.82
Score in stressb	7.79 ± 4.97
Score in inattentionc	3.58 ± 2.01
Score in hyperactivitiyc	5.06 ± 2.09
Total score in ADHD Rating Scalec	8.63 ± 2.89
Daily hours spent on social media (hours)	3.75 ± 1.13

Note. ADHD: attention deficit hyperactivity disorder; SD: standard deviation.

aFifteen participants did not report their smoking status. bMeasured using Depression Anxiety Stress Scale. cMeasured using ADHD Rating Scale.

.

Correlation matrix among tested variables

	r (p value)
	1. BSMAS	2. IGDS-SF	3. Depressiona	4. Anxietya	5. Stressa	6. Inattentionb	7. Hyperactivityb	8. Timec
1.	–	.734 (<.001)	.214 (<.001)	.167 (<.001)	.125 (<.001)	.222 (<.001)	.131 (<.001)	.583 (<.001)
2.		–	.206 (<.001)	.128 (<.001)	.088 (<.001)	.145 (<.001)	.108 (<.001)	.476 (<.001)
3.			–	.519 (<.001)	.068 (.001)	−.010 (.622)	.061 (.002)	.108 (<.001)
4.				–	.016 (.426)	.048 (.014)	.002 (.931)	.116 (<.001)
5.					–	.023 (.241)	.011 (.584)	.118 (<.001)
6.						–	−.001 (.970)	.118 (<.001)
7.							–	.004 (.817)

Note. BSMAS: Bergen Social Media Addiction Scale; IGDS-SF: Internet Gaming Disorder Scale – Short Form.

aMeasured using Depression Anxiety Stress Scale. bMeasured using ADHD (attention deficit hyperactivity disorder) Rating Scale. cDaily time spent on social media.

The satisfactory psychometric properties of the BSMAS were supported by both CTT and Rasch analyses. At the BSMAS item level (Table 3), the results demonstrated that all six items had quite strong factor loadings in the CFA (0.64–0.83) and had acceptable corrected item-total correlations (0.56–0.69). No items had an infit or outfit MnSq outside of the recommended range (infit MnSq = 0.88–1.28; outfit MnSq = 0.86–1.22), and no items displayed substantial DIF across gender (DIF contrast = −0.48 to 0.24). Moreover, the 4-point Likert scale was functioned in order, and respondents more addicted to social media selected higher points on each BSMAS item (Figure 1); the higher Rasch score indicates more severe in the BSMAS (Figure 2).

.

Psychometric properties of the Bergen Social Media Addiction Scale (BSMAS) and endorsement rates of social media addiction in item level

Item no.	Classical test theory		Rasch analyses				Endorsement rate (%)
	Factor loadingsa	Item-total correlation	Infit MnSq	Outfit MnSq	Difficulty	DIF contrastb	Whole sample	Males	Females
BSMAS1	0.79	0.68	0.94	0.92	0.17	0.13	20.8	26.1	14.0
BSMAS2	0.64	0.56	1.28	1.22	0.64	−0.48	16.1	20.0	11.2
BSMAS3	0.83	0.69	0.88	0.92	−0.14	0.13	26.0	33.0	17.1
BSMAS4	0.72	0.65	1.00	1.00	0.04	0.12	19.2	24.5	12.4
BSMAS5	0.69	0.64	1.03	0.99	−0.25	−0.14	23.9	28.5	17.9
BSMAS6	0.82	0.69	0.88	0.86	−0.46	0.24	31.4	39.3	21.1

Note. Infit: information-weighted fit statistic; Outfit: outlier-sensitive fit statistic; MnSq: mean square error; DIF: differential item functioning.

aBased on confirmatory factor analysis. bDIF contrast across gender = difficulty for females − difficulty for males; and a value >0.5 indicates substantial DIF.

.

Ordered categorical functioning in the response scale of Bergen Social Media Addiction Scale (BSMAS)

.

The relationship between Rasch score and the Bergen Social Media Addiction Scale (BSMAS) score

At the BSMAS scale level (Table 4), both ceiling (2.2%) and floor effects (4.7%) were negligible. The internal consistency was very good (α = 0.86). AVE (0.51) and CR (0.86) were higher than the suggested cutoff, and SEM (1.81) was lower than the SD/2 in the sample (i.e., 2.415). All the fit indices in the CFA were excellent except the significant χ2 test. The separation reliability shown in Rasch analyses was satisfactory in both item (0.99) and person (0.80). Although the person separation index was slightly lower than expected (1.99), the item separation index performed extremely well (11.80).

.

Psychometric properties of Bergen Social Media Addiction Scale at scale level

Psychometric testing	Value	Suggested cutoff
Classical test theory
Ceiling effects (%)	2.2	<20
Floor effects (%)	4.7	<20
Internal consistency (Cronbach’s α)	0.86	>0.7
Confirmatory factor analysis
χ2 (df)	86.53 (9)*	Non-significant
Comparative fit index	0.993	>0.9
Tucker–Lewis index	0.989	>0.9
RMSEA (90% confidence interval)	0.057 (0.046, 0.068)	<0.08
SRMR	0.039	<0.08
Average variance extracted	0.51	>0.5
Composite reliability	0.86	>0.6
Standard error of measurement	1.81	<SD/2a
Rasch analysis
Item separation reliability	0.99	>0.7
Item separation index	11.80	>2
Person separation reliability	0.80	>0.7
Person separation index	1.99	>2

Note. RMSEA: root mean square error of approximation; SRMR: standardized root mean square residual.

The SD/2 was 2.415 based on our results (see Table 1 for the SD information).

p < .001.

As the item level of BSMAS displayed no DIF among all items, the measurement invariance at the scale level was further examined. The multigroup CFA (Table 5) concurred with the results of Rasch analyses that male and female adolescents interpreted the BSMAS similarly in both metric and scalar invariances.

.

Measurement invariance across gender on social media through confirmatory factor analysis

Model and comparisons	Fit statistics
	χ2 (df) or Δχ2 (Δdf)	p value	CFI or ΔCFI	SRMR or ΔSRMR	RMSEA or ΔRMSEA
Models
M1: Configural	109.849 (18)	<.001	0.991	0.038	0.062
M2: Plus all loadings constrained	123.562 (23)	<.001	0.990	0.041	0.057
M3: Plus all intercepts constrained	152.385 (28)	<.001	0.988	0.046	0.058
Model comparisons
M2−M1	13.713 (5)	.02	−0.001	0.003	−0.005
M3−M2	28.823 (5)	<.001	−0.002	0.005	0.001

Note. M1 is a configural model; M2 is a model based on M1 to additionally constrain all factor loadings being equal across gender; M3 is a model based on M2 to additionally constrain all item intercepts being equal across gender. CFI: comparative fit index; SRMR: standardized root mean square residual; RMSEA: root mean square error of approximation.

The criterion-related validity of the BSMAS was tested with the criteria of IGDS-SF, DASS, ADHD Rating Scale, and daily hours spent on social media. After controlling for age and gender, it was found that all the aforementioned criteria were significantly correlated to BSMAS (p ≤ .01). The magnitudes were as expected. IGDS-SF showed the highest relationship (β = 0.542), followed by the daily time spent on social media (β = 0.281). It was also found that IGDS-SF shared similar results with the BSMAS (Table 6).

.

Criterion-related validity of the BSMAS using regression models with adjustment for age and gender

	BSMASa		IGDS-SFb
Criterion	β	p value	β	p value
IGDS-SF
Internet gaming disorder	0.542	<.001	–	–
Depression Anxiety Stress Scale
Depression	0.038	.01	0.147	<.001
Anxiety	0.039	.01	−0.014	.49
Stress	0.039	.002	0.019	.29
ADHD Rating Scale
Inattention	0.101	<.001	0.087	<.001
Hyperactivity	0.061	<.001	0.085	<.001
Self-report
Daily hours spent on social media	0.281	<.001	0.427	<.001

Note. BSMAS: Bergen Social Media Addiction Scale; ADHD: attention deficit hyperactivity disorder; IGDS-SF: Internet Gaming Disorder Scale – Short Form.

Total R2 (adjusted R2) = .624 (.623); the R2 (adjusted R2) of age and gender was .075 (.075).

Total R2 (adjusted R2) = .262 (.260); the R2 (adjusted R2) of age and gender was .040 (.039).

Discussion

This study tested the psychometric properties of the BSMAS in Iran to investigate whether the BSMAS can be applied in Persian ethnicity. We found that the Persian BSMAS had a one-factor structure invariant across male and female adolescents. Other psychometric properties found in the Persian BSMAS include satisfactory internal consistency and promising criterion-related validity, as well as minimal (nearly negligible) floor and ceiling effects. Moreover, this is the first study to use Rasch models to investigate the BSMAS, and the BSMAS shared robust properties using CTT psychometric testing and Rasch models. More specifically, the categorical functioning in the 5-point Likert scale was supported for all the BSMAS items. Therefore, the BSMAS scoring method is arguably appropriate (i.e., adding up the BSMAS item scores to yield a total score). Although the study did not aim to recruit adolescents with online addiction problems, the sample had a high prevalence rate of being at risk to social media addiction using the Persian BSMAS. The main reasons for this are likely to be the popularity of the smartphone use with Wi-Fi access (Pew Research Center, 2016), which provides adolescents with 24/7 exposure to social media. In addition, adolescents appear to have great knowledge in using the modern technology but appear to have poor control in resisting its use. However, it should be noted that BSMAS scores are indicative and the only way of assessing a genuine addiction to social media would be through a clinical diagnosis from an appropriate practitioner.

The results of the acceptable reliability and satisfactory validity were comparable with the previous version of the BSMAS, Bergen Facebook Addiction Scale (Andreassen et al., 2012; Pontes, Andreassen, & Griffiths, 2016). Moreover, the results concerning the factorial structure being one-factor are consistent with the previous Italian findings (Monacis et al., 2017a), which is the only study that has tested the psychometric properties of BSMAS. This study extended the evidence of the one-factor structure of the BSMAS from CTT findings to the Rasch models. Moreover, the results agree with those reported by Monacis et al. (2017a) that males and females read the BSMAS items using similar interpretations. Again, DIF examinations were used in the Rasch models to support the measurement invariance results of BSMAS at item level. However, there were two results slightly different from the findings by Monacis et al. (2017a): First, the results did not correlate the uniqueness of Item 1 to that of Item 2, whereas Monacis et al. (2017a) did. Second, this study found both metric and scalar invariances in the BSMAS across gender, whereas Monacis et al. (2017a) only found metric invariance across gender.

Because the characteristics of the participants were directly compared with those of Monacis et al. (2017a), two major differences of different countries and different ages were identified. The sample in this study solely included Iranian adolescents, whereas the sample of Monacis et al. (2017a) included Italian adolescents and young adults. Because different countries may have different access to or using habits concerning the social media, it is possible that the BSMAS structure is slightly different between centuries. In addition, the cognitive ability between adolescents and adults is different (Lin, Scheerma, Yaseri, Pakpour, & Webb, 2017), which implies that these different age groups may have different interpretations of BSMAS items. Nevertheless, the one-factor structure of the BSMAS was confirmed, and the only difference was whether Items 1 and 2 additionally shared similar concept outside of the BSMAS. Moreover, by carefully examining the results reported by Monacis et al. (2017a), results from this study demonstrated that their CFA without relating uniqueness is supported, if somewhat loose cutoff for RMSEA is applied. That is, their RMSEA (0.094) is lower than 0.1, which is an alternative suggestion for RMSEA (Lin, Fung, Nikoobakht, Burri, & Pakpour, 2017).

The scalar invariance was not supported in the Italian sample (Monacis et al., 2017a), if the ΔCFI and ΔRMSEA values were used. Their ΔCFI was −0.02 and ΔRMSEA was 0.027, and CFI was 0.97 and RMSEA was 0.08 for the scalar invariance model. However, some argue that if the constrained model shows acceptable model fit (i.e., CFI > 0.9 and RMSEA < 0.08), the measurement invariance still can be supported even the ΔCFI and ΔRMSEA are not satisfactory (Wu, Chang, Chen, Wang, & Lin, 2015). Moreover, an alternative of ΔCFI (≥0.02), which supports the measurement invariance for the findings of Monacis et al. (2017a), has been proposed (Limbers, Newman, & Varni, 2008, 2009; Lin et al., 2012).

Only weak associations were found between the BSMAS and the criteria of psychological and behavioral problems, although the associations were significant. Therefore, the findings somewhat agree with the reported correlations among social media addiction, psychological health, and ADHD-related behavioral problems (Andreassen et al., 2016). More specifically, the BSMAS score was found to correlate with the IGDS-SF score (Andreassen et al., 2016; Monacis et al., 2017b). However, more evidence from other populations is needed. For example, the participants in this study were apparently healthy adolescents, and they might not have mental health issues or ADHD-related behavioral problems. Such a sample might develop the aforementioned problems after they become addicted to social media. However, they might not have had such problems during the data collection period as they might not have been exposed to social media for long enough. It was postulated that stronger relationships may be found in the sample with mental health problems or with ADHD-related behaviors.

There are some limitations in the study. First, no data were collected as to when the participants started to use the social media, and this might affect the relationships between BSMAS score, DASS score, IGD-SF score, and ADHD Rating Scale score. Second, no data were collected regarding psychiatric diagnosis of the participants. Confounding effects from psychiatric disorders cannot be ruled out. Third, high-school students were only recruited for testing the psychometric properties of the BSMAS; the generalization of the results to people in other age groups (e.g., adults) is therefore restricted. The present authors would encourage future studies to explore the psychometric properties of the BSMAS in other age groups. Fourth, because this study only recruited one ethnicity (i.e., Persian), there was no comparison of the BSMAS between Western and Eastern countries, which could have important implications for healthcare professionals. Future studies are warranted to explore the measurement invariance or DIF issues across countries (or cultures) for the BSMAS.

Conclusions

This study showed the unidimensionality of the Persian BSMAS in a large sample of Iranian students. The Persian BSMAS also had robust psychometric properties, including the supported categorical functioning in the response scale. Moreover, both the measurement invariance and DIF-free items were supported for the Persian BSMAS. Based on the aforementioned results, healthcare professionals and providers can easily use the summated score of the Persian BSMAS to compare the level of social media addiction between genders.