Do evaluation and self-identification relate to self-reported and actual social media use?

Background and aims: Social media use can sometimes become excessive and damaging. To deal with this issue, scholars and practitioners have called for the development of measures that predict social media use. The current studies test the utility of evaluation and self-identification measures for predicting social media use. Method: Study 1 examined the relation between evaluation (n = 58) and self-identification (n = 56) measures on the one hand and several self-report measures of social media use on the other hand. Study 2 examined whether the main results of Study 1 could be replicated and whether evaluation (n = 68) and self-identification (n = 48) also relate to actual social media use. We probed evaluation and self-identification using implicit and explicit measures.
Results: Explicit evaluation and self-identification measures significantly correlated with several of the self-report measures of social media. Explicit evaluation also significantly correlated with several indices of actual social media use. Implicit measures did not relate to social media use. Discussion and conclusions: The current results suggest that researchers and practitioners could benefit from using explicit evaluation and self-identification measures when predicting social media use, especially an evaluation measure since this measure also seems to relate to actual social media use. Study 2 was one of the first to test the ecological validity of social media use measures. Although implicit measures could provide benefits for predicting social media use, the current studies did not show evidence for their predictive utility.

Introduction

For many of us, the use of social media has become a part of daily life (Kemp, 2020). However, this popularity of social media is not without risk. Increasing evidence suggests that higher levels of social media use might be related to reduced mental and physical health (see Andreassen, 2015; Keles, McCrae, & Grealish, 2020 for reviews). Given these relations, researchers and practitioners are looking for measures to predict social media use. Such measures could help researchers to test theories about social media use and practitioners to establish targeted prevention and intervention.

As inspiration for developing these measures, researchers often look at research on substance (over)use (e.g., alcohol, nicotine, and other drugs). In this research, measures of evaluation and self-identification have proven useful. Several meta-analyses have shown that evaluations of addictive substances (e.g., whether alcohol is liked or disliked) are related to substance (over)use (e.g., Cooke, Dahdah, Norman, & French, 2016; Topa & Moriano, 2010). Research also suggests that self-identification with an addictive substance (i.e., considering an addictive substance or behavior as an important part of one's identity) strongly relates to the use of that substance (Chen et al., 2021; Lindgren, Neighbors, Gasser, Ramirez, & Cvencek, 2017).

Notably, in the context of substance use, it is often considered that individuals might not be honest when reporting their behavior. Measures that assess evaluation and self-identification under conditions of automaticity (i.e., implicit measures) have been developed to overcome this problem. For instance, evaluative responding in (some) implicit measures, such as the Implicit Association Test (IAT; Greenwald, McGhee, & Schwartz, 1998), might be less controllable than evaluative responding in explicit (i.e., self-report) measures (e.g., Stieger, Goritz, Hergovich, & Voracek, 2011). Therefore, these implicit measures might be less sensitive to dishonest responding (Van Dessel et al., 2020). Accordingly, implicit measures of evaluation and self-identification have been developed that showed predictive validity in the context of substance use. For instance, a meta-analysis found that implicit evaluation scores were moderately correlated with nicotine, alcohol, and marijuana use (Rooke, Hine, & Thorsteinsson, 2008), and a review study indicated that implicit measures of self-identification consistently predict several substance-related outcomes (e.g., prospective substance use; Lindgren et al., 2017).

Interestingly, few studies have tested the relation between measures of evaluation or self-identification and social media use. A study by Ho, Lwin, and Lee (2017) showed that explicit (i.e., self-reported) evaluation of social media was associated with a measure assessing social media addiction in adolescents. Ho et al. also found that explicit (i.e., self-reported) self-identification with social media, amongst other predictors, had the strongest association with different measures of social media use in both adolescents and adults. A study by Pelling and White (2009) showed that explicit evaluations of social media predicted intentions to use social media. Explicit self-identification with social media predicted both intentions to use social media and social media use behavior itself. Two other studies have shown initial evidence for the predictive utility of implicit measures of evaluations for social media use, but produced small effects (r = 0.18 in Brailovskaia & Teichert, 2020; r = 0.22 in Turel & Serenko, 2020). To the best of our knowledge, the utility of implicit measures of self-identification for probing social media use has not yet been investigated.

The current studies aimed to systematically test whether social media use can be predicted by measures of evaluation of social media, as well as by measures of self-identification with social media. In social research, two types of measures are typically used to assess technology use, namely self-rated frequency estimates of technology use and self-rated experiences with regard to technology use (Ellis, Davidson, Shaw, & Geyer, 2019). Importantly, there is a lot of debate on how social media use should be operationalized (Cingel, Carter, & Krause, 2022), and research investigating which of these two types of measures best reflect actual technology use (and especially social media use) is scarce. We therefore chose to employ both types of measures. In Study 1, we thus tested the relation between implicit and explicit (i.e., self-report) measures of evaluation of and self-identification with social media on the one hand, and self-report measures of social media use on the other hand. The self-report measures of social media use consisted of one measure assessing frequency of social media use and three measures assessing experiences related to social media use. To assess experiences related to social media use we employed a social media addiction scale because, in technology use research, measures that assess addiction symptoms (e.g., withdrawal) are the most frequently used type of experience measure (see Andreassen & Pallesen, 2014 for an overview of existing instruments). Further, we also assessed self-evaluation and self-esteem in relation to social media use because these types of experiences have been argued to play an important role in social media use (see Cingel et al., 2022 for a recent review).

Importantly, however, preliminary evidence has shown that experience measures poorly relate to actual technology use and that frequency measures are only moderately related to objective use (Boase & Ling, 2013; Ellis et al., 2019). Therefore, in contrast to Study 1 which relied on self-reports to assess social media use, Study 2 tested whether evaluation and self-identification measures were predictive of actual social media use behavior. Notably, scholars rarely test whether the technology use scales they employ in their studies relate to actual behavior (e.g., De-Sola Gutiérrez, Rodríguez de Fonseca, & Rubio, 2016; Keles et al., 2020). This is a significant methodological limitation that has been acknowledged in the field of social media and smartphone use research (e.g., Ellis, Kaye, Wilcockson, & Ryding, 2018; Ellis et al., 2019; Keles et al., 2020). Indeed, the conclusions drawn from studies that exclusively rely on invalidated proxies of technology use might not be fully accurate, and practitioners that use these measures as diagnostic tools might be misinformed. Finally, Study 2 also tested whether results from Study 1 were replicable.

Study 1

Method

All (anonymized) data files, materials, study, and analytic scripts for Study 1 are publicly available on the Open Science Framework at https://osf.io/8yzfr/.

Participants

Participants were native Dutch-speaking undergraduates from Ghent University and participated in exchange for course credits. A total of 114 participants completed the study. No participants were excluded. One group of participants (n = 58) completed the evaluation measures, while another group of participants (n = 56) completed the self-identification measures.

Measures

Implicit evaluation and self-identification measures

Implicit evaluation of, and self-identification with, social media was measured using two IATs. Both measures followed the standard procedure of the IAT (Nosek, Greenwald, & Banaji, 2007). Participants were instructed to categorize stimuli as fast as possible using the “F” and “J” keys on the keyboard. Category labels were presented in the top left and right corner to aid classification. The stimuli of the evaluation IAT consisted of 16 names and logos of social media (e.g., the logo of Twitter) and traditional media (e.g., the logo of a Belgian newspaper) and 16 positively (e.g., “good”) and negatively (e.g., “bad”) valenced words. Stimuli of the self-identification IAT consisted of the same social and traditional media stimuli but included words referring to the self (e.g., “me”) and others (e.g., “they”) instead of valenced words. On each trial, a stimulus was presented in the center of the screen until the participants pressed one of the two keys. If the response was correct, the stimulus disappeared, and the next stimulus was presented 400 ms later. If the response was incorrect, the stimulus was replaced by a red “X” for 200 ms, and the next word appeared 400 ms after the red “X” disappeared. In the first block, participants practiced categorizing the social and traditional media stimuli for 16 trials. In the second block, participants practiced categorizing the valence stimuli (evaluation IAT) or the self/others stimuli (self-identification IAT). Next, participants completed 64 critical trials during which stimuli from all four categories were categorized. During these trials, traditional media stimuli and positively valenced stimuli (“self” stimuli for the self-identification IAT) shared the same response key, whereas social media and negatively valenced stimuli (“others” stimuli for the self-identification IAT) shared the other response key. Participants then practiced sorting social and traditional media stimuli with the response key assignment reversed for 32 trials. Finally, participants completed 64 critical trials during which stimuli from all four categories were categorized using the new response key assignment.

Scores for both IATs were calculated using the D4 (also known as the D600) scoring algorithm (Greenwald, Nosek, & Banaji, 2003). Reaction times on trials of the second critical block were subtracted from reaction times on trials of the first critical block, such that higher scores indicated more positive evaluation of and more self-identification with social media. Spearman-Brown corrected split-half reliability for both IATs was 0.89.

Explicit evaluation and self-identification measures

Explicit evaluation of social media was measured by asking participants to rate the extent to which they like social media, and explicit self-identification with social media was measured by asking participants to rate the extent to which they consider social media an important part of their identities. Responses were provided on rating scales that ranged from -50 (I dislike social media or social media is not an important part of my identity) to +50 (I like social media or social media is a very important part of my identity).

Social media addiction measure

Social media addiction was assessed using the Bergen Social Media Addiction Scale (BSMAS; Andreassen, Torsheim, Brunborg, & Pallesen, 2012). The BSMAS consists of six items (α = 0.73) that probe six basic symptoms of addiction (e.g., withdrawal) in the context of social media use. For instance, participants answered the question “how often during the last year have you tried to cut down on the use of social media without success?”. Responses were provided on a Likert scale ranging from one (very rarely) to five (very often). The scores for each item were transformed, with zero representing scores less than three on the Likert scale and one representing scores larger than or equal to three on the Likert scale. Total scores were obtained by summing the item scores.

Self-rated frequency of social media use measure

We measured frequency of social media use using three questions (α = 0.76) asking participants to what extent they agreed with different statements (e.g., “I often use social media”). Participants responded on Likert scales ranging from one (strongly disagree) to five (strongly agree). Total scores were obtained by summing the item scores.

Self-esteem and self-evaluation measures

Self-esteem in relation to social media was measured using six questions (e.g., “I feel more positive about myself when I gain likes, friendship-requests, and followers on social media”). The use of social media to deal with negative self-evaluation was assessed using two questions (e.g., “I feel an urge to post something on social media sites when I have negative thoughts about myself”). Responses were provided on Likert scales ranging from one (strongly disagree) to five (strongly agree). Total scores for each construct were obtained by summing the item scores. Cronbach's alphas were 0.86 and 0.67 for the self-esteem and self-evaluation items, respectively.

Procedure

Upon entering the research lab, participants provided informed consent and were seated in front of a computer screen. Participants then completed the evaluation or self-identification measures. The order of the implicit and explicit measures was counterbalanced between participants. Next, participants completed the questions regarding social media use. At the end of the study, all participants also received the IAT and explicit measure that they had not completed yet. This final measurement phase was included for exploratory reasons (i.e., to allow exploratory analyses with greater statistical power).

Statistical analysis

Descriptive statistics for the study variables are presented in Table 1. To rule out order effects, we first conducted one-way MANOVAs with task order (implicit or explicit measure first) as independent variable and implicit and explicit measure scores as outcome variables. Visual inspection of Q-Q plots and results from Shapiro-Wilk tests revealed that the data of nearly all variables did not follow a normal distribution (except for the data of the self-identification IAT and self-esteem variables). Therefore, we calculated (non-parametric) Spearman correlations (rather than Pearson correlations) between evaluation and self-identification measure scores on the one hand and all of the social media measure scores on the other hand.

Table 1.

Descriptive statistics for Study 1 variables per group

	Evaluation group (n = 58)	Self-identification group (n = 56)
Evaluation IAT, M (SD)	0.43 (0.45)	-
Explicit evaluation, M (SD)	18.74 (19.84)	-
Self-identification IAT, M (SD)	-	0.38 (0.42)
Explicit self-identification, M (SD)	-	8.00 (23.99)
BSMAS, M (SD)	2.84 (1.65)	3.02 (1.67)
Self-rated frequency, M (SD)	10.24 (2.36)	10.45 (2.44)
Self-esteem, M (SD)	17.29 (4.44)	17.04 (5.43)
Self-evaluation, M (SD)	3.31 (1.69)	3.64 (1.74)

Note. IAT = Implicit Association Test; BSMAS = Bergen Social Media Addiction Scale.

Ethics

The study procedures were carried out in accordance with the Declaration of Helsinki. All subjects were informed about the study, and all provided informed consent. The ethical committee of the Faculty of Psychology and Educational Sciences at Ghent University approved both studies.

Results and discussion

Preliminary analyses showed that task order (implicit or explicit measure first) did not have significant omnibus effects on implicit and explicit evaluation scores (Pillai's trace = 0.06, P = 0.21) or implicit and explicit self-identification scores (Pillai's trace = 0.02, P = 0.54), hence, the datasets could be analyzed without distinguishing between participants who completed the implicit measure first and participants who completed the explicit measure first.

Spearman correlations between all study variables are presented in Table 2. Evaluation IAT scores did not relate to any of the social media use measure scores (r s < 0.23, ps > 0.09). The explicit evaluation measure moderately correlated with self-rated frequency of social media use scores, r (56) = 0.44, P < 0.001, 95% CI = [0.21, 0.63], but not with any of the other outcome measures (r s < 0.14, ps > 0.27). Self-identification IAT scores also did not relate to any of the outcome measures (r s < 0.24, ps > 0.08). The explicit self-identification measure, on the other hand, was related to all four outcome variables. More specifically, moderate to strong correlations were observed with BSMAS scores, r (54) = 0.47, P < 0.001, 95% CI = [0.24, 0.65], self-rated frequency scores, r (54) = 0.40, P = 0.002, 95% CI = [0.16, 0.60], self-esteem scores, r (54) = 0.50, P < 0.001, 95% CI = [0.27, 0.67], and self-evaluation scores, r (54) = 0.42, P = 0.001, 95% CI = [0.18, 0.61].

Table 2.

Spearman correlations between Study 1 variables

	1.	2.	3.	4.	5.	6.	7.
1. Evaluation IAT
2. Explicit evaluation	0.23
3. Self-identification IAT	0.37***	0.18
4. Explicit self-identification	0.11	0.50***	0.07
5. BSMAS	0.06	−0.14	0.15	0.47***
6. Self-rated frequency	0.04	0.44***	0.23	0.40**	0.48***
7. Self-esteem	0.22	0.12	−0.10	0.50***	0.44***	0.31***
8. Self-evaluation	0.02	−0.13	0.01	0.42***	0.53***	0.29**	0.51***

Note. IAT = Implicit Association Test; BSMAS = Bergen Social Media Addiction Scale.

** P ≤ 0.01. *** P ≤ 0.001.

The correlations between the social media use measure scores and the correlations between the evaluation and identification measure scores are based on the complete sample (N = 111; note that three participants were excluded because they did not complete the second implicit and explicit measures). The other correlations are based on subsamples per measurement type (evaluation group, n = 58; self-identification group, n = 56).

Results from Study 1 showed initial evidence for utility of both the explicit self-identification and the explicit evaluation measure for the prediction of self-reported social media use. The explicit self-identification measure also significantly predicted scores related to addiction. Exploratory analyses including the complete sample (and thus with greater statistical power) produced similar results (see Table S1 in supplementary material). In Study 2, we attempted to replicate these results while also introducing measures of actual social media use.

Study 2

All (anonymized) data files, materials, study, and analytic scripts for Study 2 are publicly available on the Open Science Framework at https://osf.io/8yzfr/. The study design, sampling, and analysis plan of Study 2 were preregistered (see https://osf.io/295qa).

Study 2 was conducted online. An invitation to participate in the study, a link to the study, and inclusion criteria for participating, were posted in Facebook groups for undergraduates from Ghent University. A total of 205 participants started the study. For 37 participants, the study was terminated because they did not meet our preregistered inclusion criteria (e.g., being willing to have social media use tracked). The data from three participants were excluded because they had incomplete questionnaire and/or IAT data. The final sample with complete questionnaire and IAT data consisted of 165 participants. As in Study 1, one group of participants completed the evaluation measures (n = 97), while another group completed the self-identification measures (n = 68).

A total of 155 participants (correctly) installed the application to track social media use. The behavioral data from 37 participants were excluded because they did not have behavioral data from seven full days. Additionally, the behavioral data from two participants were excluded because they did not insert the (correct) code to link the behavioral data to the survey data. The final sample with complete survey and behavioral data consisted of 116 participants. Out of those 116 participants, 68 participants completed the evaluation measures and 48 participants completed the self-identification measures. At the end of the study, participants received a monetary reward (€10).

Measures and procedure

The study measures and procedure were identical to Study 1, with one exception. Before presenting the questions regarding social media use, we defined social media and gave examples of the social media applications that were tracked. This was done to avoid having participants interpret “social media” in different ways.

To test the relation between our measures of interest and behavioral measures of social media use, at the end of the survey, participants were asked to install the mobile DNA application on their phones, an application that tracks smartphone application use (imec-mict-UGent, 2019), and to keep it on their phone for eight days. To control for time differences of when the mobile DNA application was installed, we removed data from the first tracking day, resulting in behavioral data from seven full days for each participant. We used data from social media applications that met the following definition of social media: “websites and applications that enable users to create and share content or to participate in social networking” (Lexico, n.d.). The behavioral outcomes that we computed were daily averages of time spent using social media (minutes), the number of pickups (i.e., the number of times a social media application was opened) without receiving a notification, the number of pickups after receiving a notification, and the number of times that checking occurred (i.e., application usage lasting less than 15 s; Andrews, Ellis, Shaw, & Piwek, 2015). We also computed the total number of social media apps that had been used over the seven-day period.

We conducted the same analyses as in Study 1 to test the relation between the evaluation and self-identification measures and (1) self-report measures of social media use (for participants with complete survey data; n = 97 for the evaluation group and n = 68 for the identification group) and (2) behavioral measures of social media use (for participants with complete survey and behavioral data; n = 68 for the evaluation group and n = 48 for the identification group). As in Study 1, visual inspection of Q-Q plots and results from Shapiro-Wilk tests revealed that the data of nearly all variables did not follow a normal distribution (except for the data of both IAT and self-esteem variables). Therefore, as in Study 1, we calculated (non-parametric) Spearman correlations. Descriptive statistics for the study variables are presented in Table 3.

Table 3.

Descriptive statistics for Study 2 variables per group

	Evaluation group with complete self-report data (n = 97)	Self-identification group with complete self-report data (n = 68)	Evaluation group with complete self-report and behavioral data (n = 68)	Self-identification group with complete self-report and behavioral data (n = 48)
Evaluation IAT, M (SD)	0.35 (0.35)	-	0.32 (0.36)	-
Explicit evaluation, M (SD)	5.30 (0.99)	-	5.24 (0.99)	-
Self-identification IAT, M (SD)	-	0.38 (0.39)	-	0.33 (0.38)
Explicit self-identification, M (SD)	-	4.57 (1.52)	-	4.67 (1.51)
BSMAS, M (SD)	3.01 (1.54)	2.94 (1.36)	2.93 (1.50)	3.04 (1.30)
Self-rated frequency, M (SD)	10.45 (1.95)	9.94 (2.02)	10.29 (1.92)	10.02 (2.08)
Self-esteem, M (SD)	17.31 (4.48)	17.10 (5.61)	17.03 (4.11)	17.50 (5.44)
Self-evaluation, M (SD)	3.32 (1.60)	3.65 (1.73)	3.29 (1.44)	3.73 (1.71)
Time (minutes), M (SD)	-	-	113.80 (55.92)	123.50 (78.20)
Pickups, M (SD)	-	-	112.86 (98.60)	100.62 (74.76)
Pickups after notification, M (SD)	-	-	15.44 (13.32)	16.65 (20.07)
Checking, M (SD)	-	-	63.88 (69.97)	51.74 (46.75)
Number social media apps, M (SD)	-	-	6.38 (1.60)	5.94 (1.45)

Note. IAT = Implicit Association Test; BSMAS = Bergen Social Media Addiction Scale.

For exploratory purposes, we also computed Spearman correlations between the other self -report measures of social media use (i.e., social media addiction, self-reported frequency of social media use, self-esteem, and self-evaluation) and the behavioral measures of social media use (see Table S4 in supplementary material).

Results

Preliminary analyses showed that task order (implicit or explicit measure first) did not have significant omnibus effects on implicit and explicit evaluation scores (Pillai's trace = 0.02, P = 0.31) or implicit and explicit self-identification scores (Pillai's trace = 0.03, P = 0.38).

Correlations with self-report measures of social media use

Spearman correlations between evaluation measures, self-identification measures, and self-report measures of social media use are presented in Table 4. Evaluation IAT scores (Spearman-Brown corrected split-half reliability = 0.77) did not relate to any of the self-report measures of social media use (r s < 0.20, ps > 0.35). The explicit evaluation measure strongly correlated with self-rated frequency of social media use scores, r (95) = 0.63, P < 0.001, 95% CI = [0.49, 0.73], and moderately correlated with BSMAS scores, r (95) = 0.23, P = 0.02, 95% CI = [0.03, 0.41], and self-esteem scores, r (95) = 0.30, P = 0.003, 95% CI = [0.11, 0.47]. Self-identification IAT scores (Spearman-Brown corrected split-half reliability = 0.72) did not relate to any of the social media use measure scores (r s < 0.12, ps > 0.35). The explicit self-identification measure moderately correlated with self-rated frequency scores, r (66) = 0.49, P < 0.001, 95% CI = [0.29, 0.66] and self-esteem scores, r (66) = 0.30, P = 0.013, 95% CI = [0.06, 0.50].

Table 4.

Spearman correlations between Study 2 self-identification measures, evaluation measures, and self-report measures

	1.	2.	3.	4.	5.	6.	7.
1. Evaluation IAT
2. Explicit evaluation	0.10
3. Self-identification IAT	0.33***	0.03
4. Explicit self-identification	0.09	0.46***	−0.14
5. BSMAS	0.19	0.23*	0.08	0.24
6. Self-rated frequency	0.08	0.63***	0.06	0.49***	0.32***
7. Self-esteem	0.09	0.30**	−0.05	0.30**	0.40***	0.38***
8. Self-evaluation	−0.08	0.18	−0.11	0.21	0.37***	0.07	0.42***

Note. IAT = Implicit Association Test; BSMAS = Bergen Social Media Addiction Scale.

*P < 0.05.** P ≤ 0.01. *** P ≤ 0.001.

The correlations between the social media use measure scores and the correlations between the evaluation and identification measure scores are based on the complete sample (N = 165). The other correlations are based on subsamples per measurement type (evaluation group, n = 97; self-identification group, n = 68).

Correlations with behavioral measures of social media use

Spearman correlations between evaluation measures, self-identification measures, and behavioral measures of social media use are presented in Table 5. Evaluation IAT and self-identification IAT scores did not correlate with any of the behavioral measures (r s < 0.25, ps > 0.08). Small to moderate correlations were observed between the explicit evaluation measure on the one hand and number of pickups, r (66) = 0.30, P = 0.012, 95% CI = [0.07, 0.50], number of pickups after receiving a notification, r (66) = 0.29, P = 0.014, 95% CI [0.06, 0.50], and checking behavior, r (66) = 0.33, P = 0.006, 95% CI = [0.10, 0.53], on the other hand. The explicit self-identification measure did not correlate with any of the behavioral outcome measures (r s < 0.25, ps >0.09).

Table 5.

Spearman correlations between self-identification measures, evaluation measures, and behavioral measures

	1.	2.	3.	4.	5.	6.	7.	8.
1. Evaluation IAT
2. Explicit evaluation	0.10
3. Self-identification IAT	0.33***	0.03
4. Explicit self-identification	0.09	0.46***	−0.14
5. Time (minutes)	0.06	0.17	0.24	0.22
6. Pickups	0.03	0.30**	0.18	0.08	0.70***
7. Pickups after notification	0.08	0.29**	0.21	0.24	0.51***	0.54***
8. Checking	0.02	0.33**	0.13	0.09	0.59***	0.96***	0.58***
9. Number social media apps	−0.12	0.19	−0.05	0.08	0.34***	0.34***	0.32***	0.35***

Note. IAT = Implicit Association Test.

* P < 0.05. ** P ≤ 0.01 *** P ≤ 0.001.

The correlations between implicit and explicit measures on the one hand, and behavioral measures on the other hand are based on subsamples per measurement type with complete survey and behavioral data (evaluation group, n = 68; self-identification group, n = 48). The correlations between the behavioral measures are based on the complete sample with complete survey and behavioral data (N = 116). The correlations between the evaluation and identification measure scores are based on the complete sample with complete survey data (N = 165). The correlations between implicit and explicit measures are based on subsamples per measurement type with complete survey data (evaluation group, n = 97; self-identification group, n = 68).

Discussion

Results from Study 2 corroborate Study 1's results, suggesting utility of both the explicit evaluation and self-identification measures but no utility of the implicit measures for predicting self-reported social media use. In contrast to results from Study 1, the explicit evaluation measure of evaluation also correlated with measures of social media addiction and self-esteem scores in relation to social media. Explicit self-identification, however, did not significantly correlate with addiction scores .

Results of Study 2 also extended those of Study 1 by showing that the explicit evaluation measure was related to behavioral measures of social media use. It should be noted, however, these correlations were smaller compared to (some of) the correlations with the self-report measures of social media use. Moreover, while the explicit self-identification measure did relate to several self-report measures of social media use, it did not relate to actual social media use. This finding suggests that the predictive utility of explicit self-identification measures for social media use might be limited.

General discussion

Summary and interpretation of findings

Our results show evidence for the utility of explicit evaluation and self-identification measures to predict social media use. In both studies, explicit evaluation and self-identification measures showed moderate to strong correlations with several self-report measures of social media use. Results from Study 2 showed that the explicit evaluation measure also related to several behavioral measures of social media use.

The finding that explicit self-identification correlated with self-reports of social media use corroborates findings from previous studies on substance and social media use (e.g., Chen et al., 2021; Ho et al., 2017; Lindgren et al., 2017; Pelling & White, 2010). However, results from Study 2 did not show evidence for a relationship between explicit self-identification and actual social media use, suggesting that self-identification measures might be less useful for the assessment of social media use than previously assumed. Notably, the finding that explicit evaluation related to social media use is consistent with previous findings in research on substance use (e.g., Cooke et al., 2016; Topa & Moriano, 2010), but contradictory to findings from previous studies on social media use. For instance, A study by Ho et al. (2017) found that social media evaluation was significantly associated with a measure assessing social media addiction in adolescents, but not in adults. Moreover, the results from this study showed that social media evaluation was not associated with social media (over)use for both age groups. Other studies have found that social media evaluation predicts intentions to use social media, but does not predict social media use behavior itself (Baker & White, 2010; Pelling & White, 2009). Our findings suggest that evaluation might be more relevant for the prediction of social media use than previously observed, especially given that explicit evaluation also related to behavioral measures of social media use.

It is of note, however, that correlations between the explicit evaluation measure and the behavioral outcome measures were less strong than (some of the) correlations with self-report measures. This observation is not entirely surprising given that correlations between self-report and behavioral measures typically tend to be lower than correlations between self-report measures (e.g., because of less shared variance between the measures; Dang, King, & Inzlicht, 2020). Another possible explanation for this observation is that not all instances of social media use were properly recorded (e.g., for some smartphone types the application had problems tracking smartphone use).

As opposed to previous findings in research on substance use (e.g., Lindgren et al., 2017; Rooke et al., 2008) and research on social media use (Brailovskaia & Teichert, 2020; Turel & Serenko, 2020), the current studies did not show evidence for the utility of implicit evaluation and self-identification measures to predict social media use. One possible explanation for this observation is that not all IAT stimuli were relevant to participants (e.g., participants might not use all social media apps that were included as stimuli in the IATs). However, (a) previous studies have shown that the category labels (for the target categories) of the IAT are of higher importance for IAT performance than the specific stimuli that are used (e.g., De Houwer, 2001), and (b) the IAT that was used in the study by Brailovskaia and Teichert (2020) also included non-personalized social media stimuli, but the results did reveal significant relations with social media use. A second possible explanation is that the current studies were not powered enough to detect smaller effects. Indeed, previous studies that observed significant correlations between implicit measure scores and social media scores had larger sample sizes (N = 145 in Brailovskaia & Teichert, 2020; N = 220 in Turel & Serenko, 2020) and results revealed small effects (r = 0.18 in Brailovskaia & Teichert, 2020; r = 0.22 in Turel & Serenko, 2020). Also, highly powered recent studies in the context of substance use failed to find a relation between implicit measures and substance use (e.g., Cummins, Lindgren, & De Houwer, 2020). These findings support the idea that the correlation between implicit measures and social media use is indeed small, which would imply that we did not have enough power to detect this relation in our studies.

Implications

Our findings have implications for both researchers and practitioners. If researchers aim to study social media use, they might benefit from also including evaluation and self-identification measures of social media, especially an explicit evaluation measure since this measure seems to relate to actual social media use behavior. Indeed, Study 2 was one of the first studies to test the ecological validity of measures developed to assess social media use.

While there is a lot of controversy regarding the pathological nature of social media use, research suggests that social media use can become problematic, resulting in negative consequences (Carbonell & Panova, 2017; Griffiths, Lopez-Fernandez, Throuvala, Pontes, & Kuss, 2018). In response to these problems, programs aimed at reducing social media use have already been developed (e.g., Foothills at Red Oak Recovery, n.d.; Mind Box Psychology, n.d.; Thomas More, n.d.). Our results have implications for practitioners who want to predict or treat excessive social media use. First, practitioners, such as educators, could (eventually) use evaluation and self-identification measures as a screening or diagnostic tool to establish more targeted prevention or intervention. Secondly, changing self-identification with and evaluation of social media might be a relevant strategy for intervention. For example, previous studies have shown that changes in substance self-identification are associated with recovery from problematic substance use (Lindgren et al., 2017). Of course, verifying the causal relationship between excessive social media use on the one hand and evaluation and self-identification, on the other hand, is required before such an intervention could be considered viable.

It should be noted, however, that it would be premature to use these measures (on their own) in their current form for practical purposes given that (a) correlations between the explicit evaluation measures and behavioral measures of social media use were only small to moderate and (b) the explicit self-identification measure did not relate to actual social media use. Further research examining the (predictive) validity of these measures is necessary before they can be applied in real-world contexts.

Our result also suggest that researchers and practitioners should expect little from implicit measures of evaluation and self-identification in the context of social media use. Even if our failure to find evidence for the utility of implicit measures was due to a lack of power to detect a small effect, it would imply that the effect is small at best. This conclusion, however, should be treated with caution because our null results could also have been due to the specific stimuli that we used in our implicit measurement tasks.

Limitations and future research

The current studies are not without limitations. First, we probed explicit evaluation and self-identification using a single question (i.e., the extent to which individuals consider social media as something positive or as an important part of their identity). Future studies could include more questions for each construct because other aspects of evaluation and self-identification might be of importance. For example, previous studies have shown that the extent to which individuals identify themselves with addictive behavior (e.g., viewing oneself as a drinker) also strongly relates to substance use (e.g., Ramirez, Olin, & Lindgren, 2017). Second, the current study samples consisted exclusively of students. Future studies should test the generalizability of the current results, especially because previous studies have shown that social media use and its correlates can differ for adults and adolescents (e.g., Ho et al., 2017).

Finally, we recommend that researchers further test the validity of implicit measures as measures of social media use, employing well-powered studies with multiple variants of the implicit measures. As previously discussed, the use of implicit measures could be beneficial in the context of social media use because of response bias. While incorporating behavioral measures of social media use could also overcome this problem, researchers might not always have the resources to include such measures in their studies (e.g., computer coding skills; Ellis et al., 2018). For practitioners, the use of behavioral measures would probably be even more problematic because of ethical constraints. Although implicit measures can have added value in this context, one should note that also in other contexts, it has been argued that the predictive utility of implicit measures for behavior (e.g., substance use) might be overestimated and that explicit measures often outperform implicit measures when predicting behavior (e.g., Cummins et al., 2020; Oswald, Mitchell, Blanton, Jaccard, & Tetlock, 2013). If future studies confirm the current studies' findings, researchers and practitioners should direct their efforts towards the development and use of explicit (rather than implicit) measures of social media use.

Conclusions

The current study results suggest that explicit evaluation and self-identification measures have predictive utility for social media use, with more robust evidence for predictive utility of evaluation measures, given that this measure also correlated with several behavioral measures of social media use. Study 2 is one of the firsts to test the ecological validity of social media use measures by examining their relationship with actual social media use. These findings can have important implications for researchers and practitioners. While the use of implicit measures could have benefits in the context of social media use, the current studies did not show evidence for their predictive utility.

Funding sources

This work is supported by Ghent University [grant number BOF16/MET_V/002] to JDH and by the Scientific Research Foundation Flanders [grant number FWO19/PDS/041] to PVD.

Authors’ contribution

All authors were involved in developing the study concept and contributed to the design. Data collection and data-analyses were performed by FC. FC drafted the manuscript. PVD, JC, and JDH provided critical revisions. All authors had full access to all data and take responsibility for the integrity of the data and the accuracy of the data analysis. All authors approved the final version of the manuscript for submission.

Conflict of interest

The authors declare no conflict of interest.

Supplementary material

Supplementary data to this article can be found online at https://doi.org/10.1556/2006.2022.00042.