Factor structure and interpretation on the Thai-Social Media Engagement Scale (T-SMES).

Due to the fact that social media has become an integral part of daily life, particularly in the new normal society, there are few instruments for measuring social media usage with a cut-off score. As a reason, the objective of this study was to develop a Thai-Social Media Engagement Scale (T-SMES) that has been verified for validity and reliability as well as a cut-off score for interpretation. Data was collected from 403 participants through an online questionnaire. The findings of exploratory factor analysis (EFA) revealed that the T-SMES was extracted into three components; accounted for 66.44% of the total variance, including (1) feeling at ease and not missing out, (2) making it a habit, and (3) a sense of being attracted to and connected to others. The test items were satisfactory in terms of validity and reliability. For identifying high social media engagement, a cut-off score of 24 was found to be optimal (sensitivity = 80.9%, specificity = 72.2%, positive predictive rate (PPR) = 92.3%, negative predictive rate (NPR) = 47.9%, and accuracy rate = 79.2%). Overall, the findings suggest that the T-SMES is an empirically valid and reliable instrument for measuring social media engagement, with optimal cut-off scores that can be used in practice.

Introduction

Online behavior can be defined as a 'new normal,' referring to a way of being in today's world (Kuss and Griffiths, 2017), During the Covid-19 outbreak, it's vital to rely increasingly on online channels to reduce the likelihood of disease transmission through interpersonal contact. One of the most common online activities is utilizing social media, which refers to the creation, sharing, and collaboration of online content for a variety of purposes, such as conveying a story, sharing personal experiences, or sharing images or video clips created by oneself or shared by others via social networking sites such as Facebook, Instagram, Twitter, Line, Tik-Tok, Clubhouse and others. On social networking sites, users can form virtual communities where they can create public profiles and interact with others (Kuss and Griffiths, 2011, 2017). According to Ni et al. (2020), social media is important for sustaining social contacts as well as for browsing news and entertainment. Griffiths and Kuss (2017) and Alt (2018) stressed the need for adolescence to engage in social media in order to avoid missing out, keep updated, and communicate with others. Nowadays, social media is also one of the most effective marketing channels for reaching millions of people, resulting in trading and commercial value (Nepomuceno et al., 2020).

Since the internet and social media are freely available online activities, the number of users has increased significantly, particularly in the last few years when the world has encountered the Covid-19 outbreak. According to our most recent data, there have been 4.48 billion social media users worldwide in July 2021, accounting for about 57 percent of the total global population, and more than 9 out of 10 internet users utilize social media on a monthly basis (DataReportal, 2021a). In Thailand, 55 million people used social media, accounting for 78.7% of the total population, rising 3 million over the previous year. By 2025, Thailand's social media user number is expected to reach 62 million. The majority of users are between the ages of 25 and 34 (Digital Business Lab, 2021). The average daily internet usage is 11 h and 25 min, with social media being the most popular online activity (Electronic Transactions Development Agency, 2020).

According to the data above, those who spend a lot of time on social media are more likely to participate in excessive use, which can lead to behavioral and mental health problems. In the literature, excessive use of social media has been associated with decreased sleep quality, daily cognitive failures, eating problems as a result of a preoccupation with physical appearance and muscularity, and negative emotions including depression and anxiety (Xanidis and Brignell, 2016; Nguyen et al., 2020; Vidal et al., 2020; Imperatori et al., 2021; Skogen et al., 2021). The empirical evidence also suggested clinical symptom arisen from an excessive use of social media, predominantly rely on the clinical criteria of Substance-Related Addictions and Internet Gaming Disorder in DSM-5 (American Psychiatric Association, 2013; Tutgun-Ünal and Deniz, 2015; Regina et al., 2016; Shahnawaz and Rehman, 2020). On the other hand, excessive use of social media may not always consider a pathology if it does not cause personal distress, clinically significant functional impairment, or become a public health burden (Kardefelt-Winther et al., 2017). There is empirical evidence revealed the unlinked of non-pathological addictive behaviors and personal functions; for example, a study of over 17,000 adolescents from Ireland, the United States, and the United Kingdom found no significant associations between digital-screen engagement and well-being (Orben and Przybylski, 2019), as well as a study of 471 undergraduate students in the United States found no significant effect of social media use on impaired mental health functioning (Berryman et al., 2018). In Thailand, studies on internet or social media use and/or overuse have yielded mixed results (Tangmunkongvorakul et al., 2019; Wisessathorn, 2019; Mongkhon et al., 2021)

Assessments of social media usage are crucial for determining the severity of use. However, the tools available were developed using a wide range of conceptual frameworks, procedures, interpretations, and cut-off scores (Shahnawaz and Rehman, 2020) which can be summarized into three broad categories: (1) the generalized social media addiction scale is a tool for determining the severity of social media addiction in those who use it on a regular basis which is not restricted to any particular activity or application, for example, the Bergen Social Media Addiction Scale (BSMAS) (Andreassen et al., 2016), Social Media Disorder (SMD) Scale (Regina et al., 2016), Social Media Addiction Scale (SMAS) (Tutgun-Ünal and Deniz, 2015). (2) The specific social networking addiction scale is a tool for determining the severity of social media addiction based on usage of certain social media applications. The most common is Facebook, as evidenced by the Facebook Intrusion Questionnaire (Ryan et al., 2014) and the Bergen Facebook Addiction Scale (BFAS) (Andreassen et al., 2012) which has been translated into Thai-version and validated for psychometric properties (Phanasathit et al., 2015).

The tools in the first two categories, however, are mostly based on the Addictive Behavior concept, which is relevant to the clinical criteria of Substance-Related Addictions and Internet Gaming Disorder as defined by the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), which include preoccupation, mood modification, tolerance withdrawal, conflict, and relapse (American Psychiatric Association, 2013). The third category, the social media engagement scale, takes a different approach. These tools are based on the Engagement concept, which emphasizes that any behavioral involvement is the combination of three primary influencing factors: cognitive, affective, and behavioral - all of which interact and are tailored to the situation, similar to the ABC model of attitude (Throuvala et al., 2019; Kondrotienė and Bakanauskas, 2021). By incorporating engagement concepts into the construction of adolescent social media engagement evaluation instruments, Ni et al. (2020) created the Social Media Engagement Scale.

As previously stated, there are a variety of techniques for developing social media usage measurement tools. The majority of them take a similar approach, beginning with item development and factor structure testing using statistical methods such as exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), and then testing psychometric properties such as validity and reliability to reflect the standardization (Tutgun-Ünal and Deniz, 2015; Ni et al., 2020; Shahnawaz and Rehman, 2020). Only a few studies have attempted to interpret the findings by examining the cut-off score to differentiate between addicted, high-engaging, and non-addicted social media users (Regina et al., 2016; Bányai et al., 2017; Li et al., 2020; Luo et al., 2021).

In this study, therefore, the main objective is to establish a Thai-Social Media Engagement Scale (T-SMES) based on the engagement concept, which involves (1) item construction and testing the factor structure of the items to ensure the appropriateness for Thai people; (2) examining the quality of the T-SMES by analyzing the test validity using content validity, convergent validity and criterion validity, as well as analyzing the test reliability using internal consistency and intraclass correlation coefficient (ICC), and (3) analyzing the cut-off score leading to interpretation. The outcomes of the study would contribute to the development of an evidence-based academic standards tool that could be utilized in practice in Thailand with both normal and pathological social media users.

Literature review

Social media engagement

Social media, a type of computer-based technology, allows people to communicate across virtual networks. Social media engagement refers to any type of interaction or communication that occurs on social networking sites, such as tweets, retweets, likes, and replies on Twitter, or reactions to posts, shares, comments, impressions and hashtag on Facebook (Aswani et al., 2019). During disasters such as earthquakes (Ahn et al., 2021), hurricanes (Mirbabaie et al., 2020), and pandemics such as COVID-19, social media is efficient at conveying both formal and informal information. This virtual platform will help with emergency alerts, crisis response, and information gathering and dissemination, as well as promote public engagement by allowing individuals to act as both receivers and influencers of information (Mirbabaie et al., 2020; Obembe et al., 2021; Ridhwan and Hargreaves, 2021).

Several facets of social media engagement have been determined in the literature, depending on the issues being studied. Social media engagement in the fields of public health and mental health focuses on determining the optimal level of usage suitable for an individual’s development, receiving crucial information for safety, and ensuring that social media is not overused to impair physical and mental health (Andreassen et al., 2016; Bányai et al., 2017; Berryman et al., 2018; Mongkhon et al., 2021; Tuarob et al., 2022). In business and marketing, social media engagement emphasizes a longer-term relationship between users or between users and a brand. For businesses, engagement consisted of three components: intimacy, platforms, and metrics, all of which provided important information that could propel the company forward (Hallock et al., 2019; Mardjo, 2019). In information technology and computer sciences, social media engagement involves the investigation of factors that influence the spread of accurate information and misinformation on a daily basis. Misinformation could be in any form, be it willful misinformation, fictional discussions, non-verifiable information, rumors, or fake profiles. The source of misinformation propagation arises from (1) content attributes that are popular, high emotional response, or high hashtag frequency, (2) propagator’s personality attributes, and (3) network attributes (Allcott et al., 2019; Aswani et al., 2019; Kar and Aswani, 2021).

The intention of using social media

Apart from utilizing social media to communicate and update information in everyday life, the user's purpose at the time drives the user's intention to use it. For example, students' intentions to use social media for learning (Balakrishnan, 2017), tourists' intentions to plan and take vacations (Parra-López et al., 2011), consumers' intentions to purchase goods and services (Lal, 2017), employees' intentions to look for employment (Karacsony et al., 2020), political leaders' intentions to engage the public and mobilize policies in support of economic, social, political, or sustainable development goals (SDGs) (Grover et al., 2021), and patients' and health professionals' intentions to use social media for medical services (De Angelis et al., 2018). As a result of this trend, social media utilization will be higher than usual. However, it frequently appears to be a spam engagement in a short period of time. When users achieve their purpose, their behavior patterns revert to the regular until the other intention is driven again.

In addition to the user's purpose, personal factors such as perceived ease of use, perceived usefulness and benefit of usage, perceived connectivity, and trust toward members of the community have a direct effect on the intention of using social media (Parra-López et al., 2011; Wamba and Akter, 2016; Lal, 2017; Yuan et al., 2021; Werling and Barkela, 2021). Peer influence has an impact on the likelihood of using social media as a mediating variable (Trivedi et al., 2021).

Thailand's internet and social media usage situations

The situation of internet use in Thailand

In 2021, Thailand had a population of 69.88 million, of which 48.59 million (69.5%) were internet users, an increase of 3.4 million (+7.4%) between 2020 and 2021 (DataReportal, 2021b). According to a survey performed by the Electronic Transactions Development Agency (2020), Thais spend an average of 11 h and 25 min every day on the internet (up from 11 h and 33 min in 2019). Generations Y and Z spend half of their day on the internet. This was due in part to the Work from Home and e-learning measures, which boosted internet usage by more than 2 h over the previous year (average hourly usage for Gen Y was 12 h and 26 min, Gen Z was 12 h and 8 min, Gen X was 10 h and 20 min, and Baby Boomers was 8 h and 41 min). Students used the Internet the most (12 h 43 min a day), followed by self-employed/freelancers (11 h 28 min), private workers (11 h 10 min), business owners (10 h 21 min), and the unemployed (10 h 10 min). All online activities have increased in popularity since last year, especially social media, which has ranked No. 1 for the past eight years with 95.3% of active users, followed by 85% of those who use the internet for entertainment, 82% for information searching, and 77.8% for online communication, while online learning (e-learning) has returned to the top ten for the past three years, representing 56.8%.

The situation of social media use in Thailand, in comparison to other countries

Thailand is undeniably one of Southeast Asia's most active social media users (Digital Business Lab, 2021). When considering only social media aspects, Thais spend 2 h and 48 min a day on average, ranking 16th in the world and third in Southeast Asia, while the Philippines (4 h and 15 min per day) has the highest rate in the world and Malaysia (3 h and 1 min) is ranked 13th in the world and second in the region (Digital Marketing Institute, 2021). Facebook is Thailand's most popular social networking site. Second and third place, respectively, go to YouTube and Facebook's chat feature and Messenger (ICAT, 2022). The majority of Thais use Facebook for posting, commenting, chatting, and purchasing. In contrast to the most popular platforms in the world's top ten countries, YouTube and WhatsApp attract more visitors than Facebook, illustrating the popularity of video and messaging (Digital Marketing Institute, 2021).

According to Thai literature, social media has both positive and negative effects on Thai youths' values. Positive values could be found in education and knowledge seeking, gratitude, selflessness, public awareness, and being healthy. Some of the negative values generated by social media include excessive spending, multiple love, materialism, gambling, superstitions, and a lack of respect for the Thai language (Suksomsot and Pumpruek, 2021). As for internal factors such as consciousness, mental immunity, and technological skills, as well as external factors such as social support, well-known idols or influencers, peer influence, and social norms, encouraged media literacy to behave on social media with awareness (Boonyapitruksagoon and Poonpol, 2019).

The interest in social media information differs from generation to generation. According to a report published in Thailand, baby boomers favor traditional media such as television, but they are open to receiving news through online media. Gen X appears to be plagued by a fear of missing out. They are receptive to social media in order to stay up with social trends, particularly from the official agency, even if they do not trust them straightaway. For Gen Y, their values differ from previous generations in that they have more independence and challenge while despising coercion. When it comes to social media, Gen Y is well-versed. They prefer to share personal information with the public, but they are less vulnerable to identity theft due to their high media literacy (Limpanathawin and Cheyjunya, 2019). In comparison to earlier generations, Gen Z consumes the most online information, from both official and unofficial sources, and decides to spread it promptly, particularly when it comes to trending content, novelty, and eye-catching graphics (Euajarusphan, 2018; Nimngern, 2022).

As a result of the aforementioned studies, therefore, the T-SMES was designed as a psychological instrument based on the concept of social media engagement in the public health realm. Multiple metrics were used to elevate the T-SMES as a standard instrument with a cut-off score in interpretation in practice, allowing us to distinguish users who used social media optimally from those who overused it and were at risk of health concerns. The T-SMES items were designed on the basis of social media usage in Thailand, predominantly among Gen Y (between the ages of adolescence and 40 years old), and included instructions for respondents to answer in a manner that was consistent with their daily routine. Another strategy to reduce errors caused by the intention of using social media for a specific purpose was to include a set of items related to the purpose of social media usage in the 30 previous days. Social contact, news updates, commercial reviews, and academic activities were all examples of specific intended uses. If the ratings for multiple activities were relatively similar, it was possible to presume that this was the person's regular pattern or personal routine. However, recall bias is a possibility, which is typical in self-rating scale assessments.

Methods

Participants and procedure

The study included 403 participants (129 males and 274 females) between the ages of 18 and 40 (M = 22.62, SD = 5.813 years old). Data was obtained through online questionnaires from July to August 2021, while Thailand was facing with the second wave of the COVID-19 outbreak. As a result, the majority of the target population has been forced to work or study from home, which has provided a great opportunity to recruit research volunteers.

Sample size considerations, sampling methods, and inclusion criteria

An adequate sample size was determined using the statistical criteria that a rotational factor loading for a sample size of at least 300 would require at least .32 at an alpha level of .01 (two-tailed) (Tabachnick and Fidell, 2007). In this study, the factorability analysis and correlation matrix suggested an adequate sample size for the EFA analysis, with the measure of sampling adequacy (MSA) = .929; statistically significant Bartlett test of sphericity with chi-square = 4630.892 (df = 153), p-value < .001.

The sampling methods used in this study were based on purposeful sampling, with the inclusion criterion being that they were over the age of 18 and had access to social media sites such as Facebook, Instagram, Twitter, Line, Tik-Tok, and others. To comply with the questionnaire's focus on GenY's social media usage behavior, the exclusion criteria were applied to participants above the age of 40.

Data collection procedure and respondent rate

The data collection procedure began with the department's Facebook page publishing an online questionnaire link, which was subsequently shared with our networks at other educational institutions. In the questionnaire instructions, the participants were informed regarding the study objectives, criteria for inclusion of respondents, and the time required to complete the questionnaire (approximately 15–25 min). In addition, the data collection was undertaken in accordance with research ethics. Anonymity, confidentiality, and voluntariness have all been explained to the participants. Participants may discontinue their involvement at any time without penalty or loss of benefits.

In July 2021, an online questionnaire link was first disseminated with the intent of collecting data until the number of responses met the statistical requirements of sample size adequacy (n ≥ 300, as indicated by the rule of thumb). The obtained data exceeded the requirement two months later, in August 2021. A total of 429 participants responded to the questionnaire. However, 26 participants were over the age of 40, which exceeded the study's inclusion criteria, and they were all excluded. As a result, a total of 403 people (93.93%) remained for further analysis.

Sociodemographic characterization of the participant

Participants' sociodemographic characteristics and their behavior related to social media use are shown in Tables 1 and 2. Statistical differences indicated that females utilized social media for significantly longer lengths of time than males (t = .4.305, p-value < .001; M (females) = 8.40, SD = 4.511 h/day, M (males) = 6.52, SD = 3.775 h/day), despite the fact that there were no gender differences in the age at when the first social media account was created (t = .213, p-value = .832; M (females) = 14.06, SD = 3.963 years old, M (males) = 13.96, SD = 3.589 years old). It also showed a difference between gender and the purpose of social media use, with females using it significantly more than males to keep up with news (t = 2.390, p-value = .017; M (females) = 1.93, SD = .638, M (males) = 1.77, SD = .654) and commercial purposes (t = 2.208, p-value = .028; M (females) = 1.18, SD = .741, M (males) = 1.00, SD = .814), but no difference in usage for social interaction (t = .721, p-value = .472; M (females) = 1.79, SD = .585, M (males) = 1.75, SD = .718) and academic purposes (t = .709, p-value = .479; M (females) = 1.52, SD = .713, M (males) = 1.57, SD = .762) as shown in Table 2.

Table 1

The survey of sociodemographic characteristics and their behavior in relation to social media use.

	Male(n = 129)	Female(n-274)	Total(n = 403)
Age (years)	M = 22.95,SD = 5.467	M = 22.47,SD = 5.973	M = 14.03,SD = 3.843
Age when created the first social media account (years)	M = 13.96,SD = 3.589	M = 14.06,SD = 3.963	M = 7.82,SD = 4.380
Time spent on social media (hours/day)	M = 6.52,SD = 3.775	M = 8.40,SD = 4.511	M = 7.82,SD = 4.380
An active account on social media applications; f (%)
Facebook	128 (99.2%)	271 (98.9%)	399 (99%)
Line	122 (94.6%)	265 (96.7%)	387 (96%)
YouTube	116 (89.9%)	258 (94.2%)	374 (92.8%)
Instagram	97 (75.2%)	256 (93.4%)	353 (87.6%)
Tik-Tok	69 (53.5%)	201 (73.4%)	270 (67%)
Twitter	56 (43.4%)	181 (66.1%)	237 (58.8%)
Others	57 (44.2%)	138 (50.4%)	195 (48.4%)
Friends on social media; f (%)(Based on the applications used the most)
less than 100	12 (9.3%)	26 (9.5%)	38 (9.4%)
100–200	16 (12.4%)	27 (9.9%)	43 (10.7%)
201–300	7 (5.4%)	22 (8.0%)	29 (7.2%)
301–400	3 (2.3%)	11 (4.0%)	14 (3.5%)
401–500	2 (1.6%)	8 (2.9%)	10 (2.5%)
greater than 500	42 (32.6%)	108 (39.4%)	150 (37.2%)
missing value	47 (36.4%)	72 (26.3%)	119 (29.5%)

Table 2

The comparative analysis of sociodemographic characteristics and their behavior in relation to social media use (n = 403).

	FemaleM (SD)	MaleM (SD)	t	p-value
Age when created the first social media account (years)	14.06 (3.963)	13.96 (3.589)	.213	.832
Time spent on social media (hours/day)	8.40 (4.511)	6.52 (3.775)	4.305∗∗	<.001
Social media usage behaviors based on the purpose of use[responses from never (0) to very often (3)]
News updates	1.93 (0.638)	1.77 (0.654)	2.390∗,	.017
Social interaction	1.79 (0.585)	1.75 (0.718)	.721	.472
Academic purposes	1.52 (0.713)	1.57, (0.762)	.709	.479
Commercial purposes	1.18 (0.741)	1.00 (0.814)	2.208∗	.028

∗∗p-value < .01, ∗p-value < .05 to test the difference between female and male, with independent t-test analysis.

The score of social media usage behavior was interpreted as follows: 0–0.74 = never, 0.75–1.50 = occasionally, 1.51–2.24 = always, 2.25–3 = very frequently or on a regular basis.

Measures

All of the measures employed in this study were Thai versions, either recently developed or generally used in the country, to ensure that the participants fully understood the questions and to remove prejudice arising from linguistic or cultural variations that could affect the study conclusion.

Thai-Social Media Engagement Scale (T-SMES)

T-SMES was a newly developed instrument that measures the extent to which Thai people engage in social media, regardless of the specific applications they used. It consisted of 18 items on a four-point Likert scale (potential scores of 0–54) ranging from never (0) to very often (3).

The T-SMES development process is depicted in Figure 1. In the first step, the first version of the T-SMES was designed and quality tested in its basic form. Then, in the latter step, more advanced analysis was employed to verify the factor structure, validity, and reliability as well as provide a cut-off score for practical interpretation.

Figure 1

The T-SMES development process.

The first version of the T-SMES

The first step in creating a new standardized instrument is to create a set of items utilizing a variety of techniques, such as group discussion, key informant interviews, clinical observations, literature data review, or a combination of these (Streiner et al., 2015; Boateng et al., 2018). Following the theoretical guideline, the T-SMES items were created using a five-step procedure that included an initial evaluation of their quality as follows.

Item construction: The item was developed based on a review of the literature and observations of Thai people's use of social media. This preliminary data was synthesized to generate the initial set of T-SMES items.

Conceptual framework: There are two main approaches that serve as the fundamental concept of the T-SMES. (1) the concept of social media engagement in public health and mental health which focuses on detecting, preventing, and treating the negative health impacts of excessive use (Andreassen et al., 2016; Griffiths and Kuss, 2017). (2) the concept of "engagement" which refers to people's daily interactions with social media. According to the literature, engagement consists of three components: cognitive, affective, and behavioral (Ni et al., 2020). High engagement on social media may lead to addiction, which can cause physical and mental health problems. In this case, the initial set of T-SMES were developed by using both approaches outlined above, with the purpose of distinguishing between people who use social media optimally and those who overuse it and potentially affect their health problems. This stage resulted in the creation of 36 items.

Theoretical validity: Theoretical validity is used to assure a new instrument's quality by examining how accurately it represents the underlying construct and ensuring that items create valid measures. In this case, the initial set of T-SMES items was theoretically validated by analyzing their contents, grouping related data, and removing redundant items. Following theoretical validation, 18 items were removed due to content overlap, leaving just 18 items for further investigation.

Content validity: To ensure the content validity of the initial version of the T-SMES, seven experts from the fields of psychology, education, and information science evaluated the Item Objective Congruence Index (IOC). The guideline was that if the value of the IOC was less than .70, those items should be removed. In this case, all of the items met the criteria, with only two items requiring minor linguistic fluidity modifications

Finally, (5) internal consistency: To ensure the reliability of the initial version of the T-SMES, a Cronbach's alpha coefficient was determined in a pilot study with 30 participants (aged 18–40 years old). In this case, Cronbach’s alpha coefficient was .937 (M = 35.10, SD = 11.124), indicating high internal consistency.

Thai-internet dependency scale (T-IDS)

T-IDS was a screening instrument designed to evaluate the level of Internet usage among Thai people. It consisted of six components i.e., excessive and uncontrollable use, withdrawal and tolerance, loss of function and relationship, reducing negative mood, physical symptoms, and other related behaviors. This questionnaire included 32 items on a four-point Likert scale (potential scores of 32–128) ranging from never (1) to very often (4). The Cronbach's alpha coefficient in the pilot study was .979 (M = 69.87, SD = 24.309), which was similar to the previous study's findings (Wisessathorn, 2017), with a cut-off score of 54 indicating higher than normal use (Wisessathorn et al., 2019). In this study, T-IDS was utilized to test convergent validity with T-SMES. The high correlations between the two tests suggested that they had good convergent validity.

Thai general health Questionnaire-28 (Thai GHQ-28)

The Thai GHQ-28 was a mental health screening test developed by Goldberg in 1972 and translated into Thai by Nilchaikovit et al. (1996) to examine mental health in four areas i.e., somatic symptoms, anxiety and insomnia, social dysfunction, and severe depression. It had 28 questions on a four-point Likert scale (potential scores of 0–84), with responses ranging from never (0) to very often (3). In the pilot phase, the Thai GHQ-28 had a Cronbach's alpha coefficient of .934 (M = 22.33, SD = 15.484), representing high consistency. In this study, the Thai GHQ-28 was used to test criterion validity with T-SMES. The weak to moderate correlations between the two test scores were expected to indicate the extent to which a measure was associated to an outcome.

Social media usage

This questionnaire was designed to examine social media usage behavior based on the purpose of use which involved social interaction, news updates, commercial, and academic purposes. It consisted of 14 questions on a four-point Likert scale, with responses ranging from never (0) to very often (3). The internal consistency in the pilot study yielded a Cronbach's alpha coefficient ranging from .765 to .916, representing moderate to high consistency (Cronbach's alpha coefficient for social interaction (4 items) = .805 (M = 8.13, SD = 2.886), news updates (3 items) = .765 (M = 6.33, SD = 1.863), commercial (4 items) = .916 (M = 5.60, SD = 3.756), and academic (3 items) = .827 (M = 5.03, SD = 2.236).

Statistical analysis

Several types of statistical data analysis were used to evaluate the standardized of the T-SMES: EFA for testing T-SMES factor structure, Pearson correlation for testing convergent validity, criterion validity, and inter-class correlation, Cronbach's alpha for indicating internal consistency, and analysis of ROC curve, sensitivity, specificity and crosstab for interpreting T-SMES scores. Variations in participant characteristics were also investigated using univariate methods such as independent t-tests and chi-square.

Results

The T-SMES factor structure

Test for EFA assumptions

The T-SMES factor structure was investigated using EFA, which began with a factorability analysis to ensure that the correlation matrix was appropriate for use in the EFA. The Measure of Sampling Adequacy (MSA) and the Bartlett test of sphericity were used to determine factorability. In this study, MSA was .929, with values approaching 1 indicating that each variable can be perfectly predicted without error by the other variables (Hair et al., 2019), and a statistically significant Bartlett test of sphericity with chi-square = 4630.892 (df = 153), p-value < .001, indicating that at least some of the variables in the correlation matrix had significant correlations (Hair et al., 2019). Based on the findings of the above investigation, this correlation matrix was found to be appropriate for use in the EFA.

Identifying the variables (items)

The communality which reflects the variance accounted for by the common components was utilized to identify the items. Items with low communality (less than .40) were removed since the aim of factor analysis was to explain variance through common components (Yong and Pearce, 2013). In this study, all of the 18 items shown communality ranged between .554 and .786; therefore, they were all maintained for further analysis.

Identifying the number of components, extraction method and rotation method

The number of components was determined using an eigenvalue higher than one and a scree plot (as illustrated in Figure 2), as well as the evaluation of appropriate factors to explain a specified percentage of variance explained, which generally should be 60% or greater (Hair et al., 2019). The principal component analysis (PCA) extraction method was then used to estimate the factor loadings and total variance explained for each component. Some factor loadings, on the other hand, were spread across multiple components, resulting in cross-factor loadings. To solve the cross-factor loadings, oblique rotation with Promax and Kaiser normalization methods was utilized.

Figure 2

Scree plot of the T-SMES (18 items).

As a result, the 18-item T-SMES was extracted into three components, which accounted for 66.44 percent of the total variance. These three components, which included 47.03 percent in the first component, 12.80 percent in the second component, and 6.61 percent in the third component, were adequate to meet a specified percentage of variance explained (Hair et al., 2019). The factor loadings in each component were shown in Table 3.

Table 3

The factor loadings of the T-SMES (18 items).

		Component 1Feel at ease and not missing out	Component 2Make it a habit	Component 3Sense of being attracted to and connected to others
TSMES_13	I feel happier on social media than in real life	.927
TSMES_14	I'm worried if I can't use social media	.820
TSMES_12	I am bored if I can't use social media	.818
TSMES_11	I am more at ease on social media than I am in real life	.764
TSMES_16	Social media helps me not feel lonely	.731
TSMES_8	When I don't use social media, I feel out of trend or am missing something important	.494		.356
TSMES_17	Social media has given me a sense of self-existence	.481		.450
TSMES_1	I use social media daily		.888
TSMES_2	I use social media whenever I have time		.853
TSMES_4	I make it a habit to use social media to relax		.779
TSMES_3	Even though it's past bedtime, I use social media before bed		.778
TSMES_18	Social media gives me something to do as a time filler		.674
TSMES_15	Every time I turn on an Internet-connected device (such as mobile phone, iPad, Notebook), I always access social media		.615
TSMES_10	Using social media makes me have a good relationship with my family			.818
TSMES_6	On social media, I am more attracted to and influenced than in real life			.799
TSMES_5	Receiving attention or comments from others on social media makes me happy			.789
TSMES_7	It is important to me to receive support or encouragement from others on social media			.785
TSMES_9	Using social media makes me have a good relationship with my friends			.699
Eigenvalue		8.466	2.303	1.190
% of variance explained		47.03%	12.80%	6.61%
% of cumulative variance explained		47.03%	59.83%	66.44%

Factor loading (the values in the table) refers to the coefficient that depicts the association between the original variables and the components (Hair et al., 2019). If the factor loadings of the original variable have the greatest relationship with any of the components, those variables should be included in that component.

Eigenvalue represents the amount of variance accounted for by a factor, also known as the latent root (Hair et al., 2019).

% of variance explained refers to the percentage of variation that can be described by each component, computed as a percentage ratio of eigenvalues to total variation.

The cross-factor loadings in items 8 and 17 were shown in Table 3. The difference in factor loadings in component 1 and 3 for item 8 was .138, which was large enough to provide it to be loaded in the larger component (the first component). Despite the fact that the difference in factor loadings in component 1 and 3 for item 17 was only .031, the decision was made based on the content. Since this item's content was similar to those of other items in component 3, item 17 has been assigned to component 3. The component score covariance matrix was then produced, as shown in Table 4.

Table 4

Component score covariance matrix.

Component	1	2	3
1	2.286
2	1.623	1.586
3	2.951	2.012	4.043

The Component Score Covariance Matrix consists of the estimated covariance coefficient to measure the amount of variation between components in pairs. The higher the value, the greater the variation.

Names and meanings of T-SMES components:

The result revealed that the 18-item T-SMES was extracted into three components, each of which was named and defined as follows: the first component, which consisted of six items (items 13,14,12,11,16, and 8), was named 'feel at ease and not missing out,' referred to the feeling that social media makes you feel at ease, not lonely, and not missing out on important information. The second component, which consisted of six items (items 1,2,4,3,18, and 15), was named ‘make it a habit,' referred to using social media in a routine or habitual manner without recognizing. Lastly, the third component, consisted of six items (items 10,6,5,7,9, and 17), was named ‘a sense of being attracted to and connected to others’ referred to the feeling that social media aids in gaining attention, existing, and feeling connected with others. A high score in each component suggested a high level of social media engagement for those reasons.

Validity and reliability of the T-SMES

Test for validity: convergent validity and criterion validity

To verify convergent validity, the T-SMES score was taken in relation to the score of the T-IDS scale, which evaluated similar content. The high correlations between the two tests suggested that they had good convergent validity. According to the Pearson correlation test, the correlation coefficient between the T-SMES and T-IDS was .630 (p-value < .01), indicating satisfactory convergent validity. Meanwhile, using Thai GHQ-28 as a parameter, the criterion validity test predicted a weak to moderate correlation indicating the degree to which a measure was related to an outcome. The Pearson correlation coefficient between the T-SMES and the Thai GHQ-28 was .306 (p-value < .01), indicating satisfactory criterion validity as shown in Table 5.

Table 5

Test for validity and reliability of the T-SMES.

	No. of items	Cronbach’s alpha		ICC(95%CI)	Pearson coefficient (n = 403)
		(Pilot, n = 30)	(n = 403)		T-IDS	GHQ-28	T-SMES
T-IDS	32	.979	.977	N/A	1
GHQ-28	28	.934	.924	N/A	.483∗∗	1
T-SMES	18	.937	.933		.630∗∗	.306∗∗	1
Component 1	6	N/A	.900	.599 (.558–.639)	.640∗∗	.363∗∗	.912∗∗
Component 2	6	N/A	.883	.554 (.512–.596)	.418∗∗	.125∗	.764∗∗
Component 3	6	N/A	.885	.563 (.521–.605)	.531∗∗	.267∗∗	.873∗∗

∗∗p-value < .01, ∗p-value < .05.

Intraclass correlation coefficient (ICC) analyzed using a two-way mixed effects model with random persons effects and fixed measures effects, regardless of whether the interaction effect was present or not.

Test for reliability: internal consistency and intraclass correlation coefficient (ICC)

The internal consistency of the whole T-SMES was excellent in both the pilot and large cohort studies, according to Cronbach's alpha coefficient (pilot study = .937, large cohort = .933). Meanwhile, each component's reliability demonstrated high internal consistency (component 1 = .900, component 2 = .883 and component 3 = .885, respectively) and a moderate ICC (component 1 = .599, 95%CI = .558 to .639, component 2 = .554, 95%CI = .512 to.596, and component 3 = .563, 95%CI = .521 to.605, respectively). As a result, the T-SMES proved to be a reliable instrument, as indicated in Table 5.

Interpretation of the T-SMES score

The receiver operating characteristic curve (ROC curve), sensitivity, specificity, positive predictive rate (PPR), negative predictive rate (NPR) and accuracy rate were taken into account when interpreting scores from the T-SMES scale. However, since there are currently no clinical criteria for social media addiction or high levels of engagement, the T-IDS scale was used as the gold standard in the estimation, with T-IDS scores of 54 indicating excessive Internet use and the potential for psychological and behavioral problems.

At the T-IDS score of 54, the estimated area under the ROC curve was .820 (95%CI = .770 to.869, positive cases = 324, negative cases = 79) as shown in Figure 3, while the sensitivity, specificity, PPR, NPR and accuracy rate were shown in Table 6.

Figure 3

ROC curve analysis. The estimated area under the ROC curve was .820.

Table 6

Estimation of the cut-off score for the T-SMES.

Cut-off score	True positive(A)	False positive(B)	False negative(C)	True negative(D)	Sensitivity(%)	Specificity(%)	PPR(%)	NPR(%)	Accuracy(%)
21	280	34	44	45	86.4	57.0	89.2	50.6	80.7
22	271	31	53	48	83.6	60.8	89.7	47.5	79.2
23	265	28	59	51	81.8	64.6	90.4	46.4	78.4
24	262	22	62	57	80.9	72.2	92.3	47.9	79.2
25	257	20	67	59	79.3	74.7	92.8	46.8	78.4
26	246	18	78	61	75.9	77.2	93.2	43.9	76.2
27	237	18	87	61	73.1	77.2	92.9	41.2	74.0
28	231	18	93	61	71.3	77.2	92.8	39.6	72.5
29	224	16	100	63	69.1	79.7	93.3	38.7	71.2
30	210	13	114	66	64.8	83.5	94.2	36.7	68.5

Sensitivity = true positive (A)/true positive (A) and false negative (C).

Specificity = true negative (D)/true negative (D) and false positive (B).

PPR: positive predictive rate = true positive (A)/true positive (A) and false positive (B).

NPR: negative predictive rate = true negative (D)/true negative (D) and false negative (C).

Accuracy = true positive (A) and true negative (D)/ALL.

The findings in Table 6 shown that the cut-off score at score of 24 was acceptable for indicating high social media engagement (sensitivity = 80.9%, specificity = 72.2%, PPR = 92.3%, NPR = 47.9%, and accuracy rate = 79.2%).

The ROC curve was represented in Figure 3. The analysis of the ROC is considered through the curve and the area under the curve: (1) The ROC curve represents the relationship between sensitivity and specificity by displaying the average of a test's sensitivity across all possible specificity values, or vice versa. The curves that are closest to the top-left corner of the graph are the best to interpret. (2) The area under the ROC curve ranges from 0 to 1, with 0 denoting a completely incorrect test and 1 denoting a perfectly accurate test. A score of 0.8–0.9 is considered excellent, and a score of more than 0.9 is considered outstanding (Mandrekar, 2010). In this case. the ROC curve in Figure 3 was closer to the top-left corner with a T-IDS score of 54 and an area under the curve of .820, indicating excellent interpretation.

However, determining the optimum cut-off score appears to be delicate. Other analyses, such as prevalence or comparisons between individuals with non-high and high engagement in terms of time spent on social media (Luo et al., 2021) and/or social media use behaviors based on the purpose of use should be carried out to confirm the accuracy of the given cut-off score.

Table 7 indicated that at the T-SMES cut-off score of 24, there were 284 (70.5%) persons with high social media engagement (primarily women) who spent significantly more time on social media for all purposes than the non-high group.

Table 7

Prevalence and comparisons between non-high and high engagement groups based on the T-SMES cut-off score of 24.

	Non-highengagement(TSMES <23)	High engagement(TSMES ≥24)	Statistic-value	p-value
Prevalence; f (%)	119 (29.5%)	284 (70.5%)	χ2 = 67.556∗∗	<.001
Time spent on social media (hours/day); M (SD)	6.10 (3.597)	8.52 (4.481)	t = -5.627∗∗	<.001
The purpose of social media use; M (SD)(The higher the score, the higher the level of engagement)
Social interaction	1.39 (.523)	1.95 (.598)	t = -8.844∗∗	<.001
News updates	1.61 (.638)	2.00 (.618)	t = -5.646∗∗	<.001
Commercial	0.93 (.714)	1.21 (.778)	t = -3.352∗∗	.001
Academic	1.34 (.760)	1.61 (.702)	t = -3.413∗∗	.001

∗∗p-value < .01, ∗p-value < .05.

Discussion

Based on the fundamental concept of engagement, which is a multi-dimensional structure that integrates cognition, affection, and behavior (Kondrotienė and Bakanauskas, 2021; Otto, 2021), the newly established T-SMES defined social media engagement as a person's affective and behavioral responses toward social media usage. The T-SMES was found to have three factor structures as a result of the EFA analysis: (1) feeling at ease and not missing out, (2) making it a habit, and (3) a sense of being attracted to and connected to others, with components 1 and 3 classified in the affection domain, referring to the dynamic of emotions when the individual used social media. Component 2 was assigned to the behavior domain and referred to the use of social media in a habitual or unintentional manner, whereas no component was assigned to the cognition domain. Unlike the Social Media Engagement Scale for Adolescents (SMES-A) which employed the same concept as T-SMES, it was able to extract all components according to theory (Ni et al., 2020). Most current social media instruments, on the other hand, are based on the concept of addictive behavior, with components of clinical diagnostic criteria for mental illness, such as compulsive or addictive behavior, acting as a foundation. Some of the most well-known are the Bergen Social Media Addiction Scale (BSMAS) (Andreassen et al., 2016), Social Media Disorder (SMD) Scale (Regina et al., 2016), and Bergen Facebook Addiction Scale (BFAS) (Andreassen et al., 2012).

The T-SMES psychometric properties were evaluated for validity and reliability. According to the validity test, the T-SMES and T-IDS correlation revealed a moderate level of convergent validity. Despite the fact that the two measures were identical in content, the T-SMES was developed on the concept of engagement, whilst the T-IDS was developed on the concept of addictive behavior (Wisessathorn, 2017). The moderate correlations between the T-SMES and the T-IDS revealed that increased social media engagement was associated with increased Internet usage, however these behavior may not be as addictive as those that lead to psychological and behavioral problems. This moderate correlation was sufficient to prove that convergent validity may be tolerated (Tutgun-Ünal and Deniz, 2015). The criterion validity was subsequently measured by the correlation between T-SMES and the Thai GHQ-28, and it met our expectations indicating satisfactory criterion validity (Fung, 2019; Afe et al., 2020). According to the reliability test, the T-SMES had a high level of internal consistency and an adequate intraclass correlation coefficient, which was consistent with previous study findings (Regina et al., 2016; Ni et al., 2020). In light of the aforementioned findings, the T-SMES measurement model provided instrument qualities that were adequate for use in practice.

Considering the optimal cut-off score seems to be complicated. Most of them are compared to the gold standard, which is mainly a medical diagnosis. However, finding an absolute technique to serve as a gold standard proved difficult because there are currently no clinical criteria for identifying social media addiction or excessive use. As a result, some previous research used the latent profile analysis plus ROC approach to investigate cut-off scores (Bányai et al., 2017; Li et al., 2020; Luo et al., 2021).

Using a similar method, the T-SMES cut-off score was calculated using the area under the ROC curve, sensitivity, specificity, PPR, NPR, and accuracy rate. To confirm the decided cut-off score, the prevalence and comparison testing of participants with non-high engagement and high engagement in terms of time spent on social media and purpose of use were taken into account. In this investigation, a cut-off score of 24 was determined to be acceptable for detecting high social media engagement (sensitivity = 80.9%, specificity = 72.2%, PPR = 92.3 percent, NPR = 47.9%, and accuracy rate = 79.2%). At the cut-off score of 24, the sensitivity, PPR, and accuracy rate were all high (about 80% or greater), but the specificity was moderate, and the NPR was relatively low. This could imply that T-SMES was strong at detecting high social media engagement but not so effective at detecting non-high engagement, resulting in underestimation in the non-high case (Habibzadeh et al., 2016; Trevethan, 2017).

At the cut-off score of 24, it was found that there were 284 (70.5%) people with high social media engagement (primarily women), with an average time of 8.52 (SD = 4.481) hours per day. When compared to before the COVID-19 epidemic, the amount of time spent online appeared to be longer (UNICEF Thailand, 2022). The survey from the Electronic Transactions Development Agency (2020) revealed that due to the restrictions of traveling, social distancing, and work-from-home measures, Thais spent an average time online boosted by more than 2 h over the previous year in 2019. Furthermore, their online behavior has altered in response to the situation. In the COVID-19 situation, the use of social media applications appeared to be diverse, not only on Facebook but also on Tik-Tok and YouTube. The purpose of use has been broadened to include not only for social interaction and news update, but also business and online commerce. These findings may serve as preliminary data for a more in-depth investigation.

Limitation and future research

There are some limitations to implementing the findings, thus the following recommendations for future research are provided:

The sample size required for representativeness and generalization in the extraction methods:

The extraction method was the first step in determining the factor structure of the T-SMES scale. Questions concerning sample size adequacy arise regularly when utilizing an extraction method in EFA analysis. However, as the primary goal of EFA is to acquire strong data in order to evaluate the scale's factor structure, concerns about its representativeness and population generalization appear to be secondary. Strong data confirming the stability, reliability, and replicability of a factor solution is required for enhancing sample power and effect size. Not only the number of samples has to be taken into consideration, but also the strength of item loadings, the homogeneity of the communalities, and the number of items per factor. According to the literature, a small sample size in an EFA analysis appears to be acceptable (de Winter et al., 2009; Kyriazos, 2018).

These findings take into account the sample size and sample power. The factorability analysis of MSA and the Bartlett test of sphericity confirmed the adequacy of sample size and met statistical standards. Despite the fact that the major purpose of EFA is to strengthen the scale rather than to investigate representativeness and generalization, a large sample size enhances the analysis's robustness.

Thus, future research may employ large sample size analyses to standardize and validate the scale's psychometric properties. Examples include dimensionality testing, reliability testing with test-retest analysis, split-half analysis, and validity testing with predictive validity, differentiation by known groups, and discriminant validity. Future research to generate a normative score for the country's population is also suggested.

The cut-off scores:

Due to the limitations of using a non-medical gold standard for social media engagement, the T-SMES cut-off score appeared to be successful in recognizing high social media engagement but limited in detecting low engagement. This might be one of the aspects to consider while implementing the T-SMES. Future research might fill this gap by investigating the relationships between the T-SMES and other standardized measures to verify that it accurately recognizes all possible groups, thus boosting the T-SMES sensitivity and specificity.

Finally, (3) The content of T-SMES items:

Since the benefits of using social media in the pilot study were largely for updating news and maintaining interpersonal relationships rather than gaming, it might be one of the considerations in the interpretation of the T-SMES. The extent to which people participate in online gaming has theoretically based explanations ranging from controllable to uncontrollable and addictive. In the diagnostic criteria for mental health problems, uncontrolled online gaming is referred to as "Internet Gaming Disorder: IGD" (American Psychiatric Association, 2013). Thus, if information regarding online gaming is required in future research, it is recommended that T-SMES may be used in conjunction with other standard measurements such as the Thai Internet Gaming Disorder Scale 9-item (IGD Scale-9 TH).

Implications

Theoretical implications

The three-factor structure of the T-SMES supports the fundamental concept of social media engagement, which emphasizes the multi-dimensional structure of internal contributions. Two of the three components of the concept of social media engagement are included in the T-SMES, including affective and behavioral engagement, but not the cognitive engagement component. It's possible that the primary motivations for using social media during the COVID-19 epidemic in Thailand are affective (Factor 1: feeling at ease and not missing out; Factor 3: a sense of being attracted to and connected to others) and behavioral (Factor 2: making it a habit). At the time of the outbreak, the measures of lockdown and social distancing had been implemented. Social media could be the key function in achieving a sense of connectedness to others. Users have reported feeling comfortable with and being able to use most social networking applications on a regular basis since most of them are available on mobile phones. Because the more they utilize it, the more proficient they get. This could be one of the reasons people can utilize a variety of social media applications, be open to new sites like Tik-Tok, and expand their use to commercial and business activities.

Meanwhile, the cognitive engagement components appear to be missing from the T-SMES structure, which challenges the theory. Because cognitive engagement in social media relates to an individual's cognitive predisposition for positive social media use, a high degree of cognitive engagement indicates that the person is likely to participate in online social contact and avoid offline face-to-face communication (Ni et al., 2020). The absence of a cognitive component in the T-SMES structure might indicate that Thai people are still motivated to communicate face-to-face even though social media is utilized regularly during the COVID-19 pandemic, suggesting a new normal society for Thai culture.

Managerial implications

Despite the fact that social media is useful and necessary in everyday life, public health professionals are concerned about excessive use of social media, which may have negative effects on physical and mental health, personality, values, and socialization processes. As a result, determining a person's degree of social media use is challenging but crucial.

With the aforementioned goals in mind, the T-SMES is developed with the objective of becoming a standard psychological screening instrument with high test validity and reliability. There is also a cut-off point for interpreting the scores related to Thai’s online behavior. Policymakers and practitioners may find the T-SMES beneficial as follows:

In the public health sector, policymakers such as municipal or community health administrators might disseminate the T-SMES in the community to screen and monitor the effects of excessive social media usage, as well as mobilize a policy or strategy to promote ‘social media literacy’ among people of all ages. This T-SMES may be used by counselors, psychologists, and mental health professionals to monitor how people use social media on an individual basis, allowing them to tailor an intervention or counseling program to suit each individual's needs.

In the educational sector, T-SMES can be used by school administrators, teachers, or instructors to ensure that students are using social media responsibly. Students' usage of social media should be focused on functional purposes that support their studies and daily life without interfering with other areas, such as face-to-face socializing or forming an unrealistic virtual role model.

In the business sector, social media is a vital tool for enhancing business by promoting relationships between users or between consumers and a company via rapid and easy-to-access online media. As a result, the use of T-SMES may be beneficial in capturing consumer behavior in each component and, collectively, allow the company to gain consumer insight information in order to generate media that is enticing, eye-catching, and matched with the actual demands of the customer.

Conclusion

During the COVID-19 epidemic, Thai people appear to be spending more time on social media. They are also interested in a variety of social media applications that serve not only for news updates and social interaction but also for business and online commerce. On the other hand, there is concern that the increased time spent on social media may negatively affect people's health and well-being. To bridge this gap, an affordable and accessible screening instrument that is available in Thai is required for use in the country.

As a result, this study decided to fill the gap by developing a screening instrument called the "Thai-Social Media Engagement Scale or T-SMES" with the purpose of becoming one of the country's high-quality instruments with a cut-off score for interpretation. The factor structure, validity, and reliability of the T-SMES were tested and all of those qualities satisfied statistical standards, indicating that the T-SMES was acceptable for use in practice. T-SMES had a cut-off score that distinguished between people who used it optimally and those who used it excessively and provided guidelines for interpretation. These findings might be applied to a variety of sectors, including public health, education, and business as addressed above.

Declarations

Author contribution statement

Manika Wisessathorn: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Nuchchamon Pramepluem: Conceived and designed the experiments; Performed the experiments; Wrote the paper.

Sawian Kaewwongsa: Conceived and designed the experiments; Analyzed and interpreted the data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability statement

Data will be made available on request.

Declaration of interest’s statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.