The Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S): Measurement Invariant Evidence for Its Nine-Item Version in Taiwan, Indonesia, and Malaysia.

Background: As the number of COVID-19 cases grows worldwide, one solution to the global pandemic is vaccination. Unfortunately, the hesitancy of receiving vaccines is still high, particularly among younger age groups (eg, students). Because the hesitancy of receiving vaccines is an important issue, instruments have been developed to assess vaccine hesitancy. Moreover, the use of these instruments among specific groups such as students is of critical importance. Aim: The present study examined the psychometric properties of the nine-item MoVac-COVID19S (also known as the DrVac-COVID19S) including its measurement invariance among university students in three different countries (ie, Taiwan, Indonesia, and Malaysia).
Methods: A multi-country, web-based cross-sectional survey study was conducted among 1809 university students, recruited from the three countries from May to September 2021. The nine-item scale was translated into Traditional Chinese, Bahasa Indonesian, and Malay.
Results: The results of the confirmatory factor analysis indicated that the one-factor structure of the MoVac-COVID19S was fully supported among Indonesian and Malay participants (comparative fit index [CFI] = 0.991 and 0.998; Tucker-Lewis index [TLI] = 0.997 and 0.987; root mean sqaure error of approximation [RMSEA] = 0.040 and 0.071; and standardized root mean square residual [SRMR] = 0.014 and 0.039). Moreover, the four-factor structure was supported among Indonesian, Malay, and Taiwanese participants (CFI = 0.998, 0.998, and 0.985; TLI = 0.997, 0.996, and 0.973; RMSEA = 0.044, 0.038, and 0.091; and SRMR = 0.013, 0.018, and 0.049).
Conclusion: The MoVac-COVID19S has good construct validity among university students from three different countries (ie, Taiwan, Indonesia, and Malaysia). The four-factor structure of the MoVac-COVID19S was supported. Therefore, health-care providers may want to assess the four underlying constructs to better understand why a university student accepts or declines COVID-19 vaccine uptake in the three countries. Using the findings, government policymakers and health-care authorities can design appropriate programs to help decrease vaccine hesitancy.

Introduction

The recent outbreak of the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), also known as coronavirus disease 2019 (COVID-19)1 was declared as a public health emergency of international concern and a pandemic2 by the World Health Organization (WHO). At the time of writing (May 2022), the COVID-19 outbreak had affected 109 countries with over 515 million confirmed cases and over 6.24 million deaths.3 In the Asia Pacific region (where the present study was carried out), there had been over 113,000 confirmed cases and over 3400 deaths.4 One of the most efficient ways to help control the pandemic is to achieve herd immunity via vaccination.5,6

Coronavirus vaccines save millions of live each year. They work by preparing the body’s immune system to recognize and fight the virus they target.7 Most of the vaccines use the S-protein of SARS-CoV-2.8 The international scientific community worked quickly to develop vaccines to control SARS-CoV-29 to help prevent unnecessary deaths from COVID-19 infection. As the vaccines have been used worldwide, it is hoped that as many individuals as possible will be given them.10 The WHO also stated that vaccines are an efficient tool alongside preventive behaviors in the fight against the COVID-19 pandemic and a number of vaccines have proved successful.7 However, there are still many people who refuse to get vaccinated and the WHO has listed vaccine hesitancy as one of the top challenges in global health.11

The vaccine acceptance varies across countries according to a recent systematic review.12 Russia, several European countries, and countries in the Middle-east were the ones with the lowest COVID-19 vaccines acceptance rates. The hesitancy to get a COVID-19 vaccine also occurs among specific cohorts such as university students.13 A study in the United States of America (USA) found only half of the medical students surveyed indicated they would participate in a COVID-19 vaccine trial, and one-fifth of them were unwilling to get the vaccine even when it is approved by the Food and Drug Administration (FDA).13 In Asia, the willingness to get a COVID-19 vaccine appears to be lower than the willingness in America. Among the three countries where the present study was conducted (ie, Indonesia, Taiwan, and Malaysia), the prevalence rates of willingness to get a COVID-19 vaccine were relatively low: 62.04% in Indonesia, 56.21% in Taiwan, and 70.66% in Malaysia.14 Given that the vaccine hesitancy has been observed in these three countries,15,16 it is important for the authorities and other stakeholders to understand the psychological mechanisms underlying why COVID-19 vaccine uptake is relatively low in these three countries.

As the hesitancy of receiving vaccines has become an important issue,17 the assessment of students’ attitude toward vaccination uptake using validated and established instruments related to vaccination uptake is of critical importance.18 The Motors of COVID19 Vaccination Acceptance Scale (MoVac-COVID19S; also known as the Drivers of COVID-19 Vaccination Acceptance Scale [DrVac-COVID19S]) is an established instrument that has previously been used to assess vaccination uptake among different populations,19 including mainland Chinese University students,20 Taiwan university students,21 and Ghanaian university students.22 The factor structure of MoVAC-COVID19S has been examined and verified using confirmatory factor analysis (CFA). The MoVac-COVID19S was developed using the cognitive model of empowerment (CME) and was adapted from the CME-guided Motors of Influenza Vaccination Acceptance Scale (MoVac-Flu Scale).23 Utilizing the CME, the scale contains four important constructs in understanding an individual’s acceptance to getting a vaccine. These are values (whether individuals care about the purpose of vaccine uptake), impacts (whether individuals believe in the effects of vaccine uptake), knowledge (whether individuals know about facts related to vaccine uptake), and autonomy (whether individuals can get vaccinated whenever they want).24

There are two versions of the MoVac-COVID19S. One is the nine-item MoVac-COVID19S containing only positively worded items and the other is the 12-item MoVac-COVID19S containing both positively- and negatively worded items. Because the 12-item version contains three negatively worded items and nine positively worded items, there may be wording effects.21,22 Indeed, prior studies have found that psychological instruments with both positively and negatively worded items can jeopardize the scale’s factor structure; eg, Rosenberg Self-Esteem Scale;25 KINDer und Jugendliche zur Erfassung der gesundheitsbezogenen Lebensqualität (KINDL) Quality of Life Instrument (an instrument originates from Germany and with its full name indicates health-related quality of life for children and young people);26,27 University of California Los Angeles (UCLA) Loneliness Scale;28 and WHO Quality of Life Questionnaire-Brief version.29,30 Therefore, the nine-item MoVac-COVID19S is probably better to use than the 12-item version from a psychometric and methodological standpoint. Moreover, given that the SARS-CoV-2 is an issue of global concern, it is important to know if the validated MoVac-COVID19S can be used on populations other than those tested previously (ie, mainland Chinese, Taiwanese, and Ghanaians). Therefore, the present study examined the psychometric properties of the nine-item MoVac-COVID19S including its measurement invariance among university students in three different countries (ie, Taiwan, Indonesia, and Malaysia).

Methods

Study Design and Area

A multi-country, web-based cross-sectional study was conducted in three public universities from three different countries (ie, one each from Taiwan, Indonesia, and Malaysia). The universities were located in Southern Taiwan, West Java – Indonesia, and Southwest Kuala Lumpur – Malaysia.

Participant, Sample Size, and Data Collection

Participants were recruited through each university’s educational system database from May to September 2021 via convenience sampling (ie, invitations were sent out using students’ personal email addresses recorded in the database). Moreover, the present study aimed to recruit at least 500 participants in each country to have a sufficient sample size for confirmatory factor analysis (CFA). According to the rule of thumb in CFA, a sample size of 50 is very poor, 100 is poor, 200 is fair, 300 is good, 500 is very good, and 1000 is excellent.31 University students aged 18 years and above were eligible to participate. Ethical approval was obtained from the Institutional Review Boards from the National Cheng Kung University (Taiwan) (IRB ref: B-ER-11-220), Universitas Padjadjaran (Indonesia) (IRB ref:655/UN6.KEP/EC.2021), and University Malaya (Malaysia) (IRB ref: UM.TNC2/UMREC_1480) before data collection was conducted. All participants were informed about the study and were asked for their consent before completing the survey. Once the participants agreed to participate, the participants were asked to log in using their personal email address to avoid having any participants repeat the survey more than once.

Measure

MoVac-COVID19S

The MoVac-COVID19S was adapted from the MoVac Flu Scale originally developed by Vallée-Tourangeau.23 The MoVac-COVID19S was originally validated in an earlier study among mainland Chinese university students.20 This study reported that the MoVAC-COVID19S had high internal consistency in both the nine-item version and 12-item version. In the present study, the scale was translated into three different language including Bahasa Indonesian, Malay, and Traditional Chinese. The standard translation included forward translation, back translation, and committee consolidation. The translation procedure was verified by the professional translator, following international standardized guidelines.32 The number of items in all local language were the same with the original scale (English version).

Data Analysis

The demographic variables and the MoVac-COVID19S item score of the participants across the three regions (ie, Taiwan, Indonesia, and Malaysia) were analyzed using descriptive statistics, including means (and standard deviations) and frequencies (percentages). Then, two types of factor structures were analyzed using the CFA with the diagonally weighted least squares (DWLS) estimator on the MoVac-COVID19S. The DWLS estimator was used because it can tackle the issue of ordinal scale (eg, the five-point Likert scale in the MoVac-COVID19S) and reports accurate estimations in the model parameters.33 The two CFA factor structures were a one-factor structure (with only one latent concept of COVID-19 vaccine acceptance) and a four-factor structure (with four latent concepts according to the CME model: values, impacts, knowledge, and autonomy). Fit indices were used to examine whether the data derived from each region fitted well with the two different factor structures separately; the fit indices of comparative fit index (CFI), Tucker-Lewis index (TLI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR) were applied to check the data-model fit. Satisfactory data-model fit should have the CFI and TLI > 0.95 together with the RMSEA and SRMR < 0.08.34 However, when the model contains lower df (eg, the present study’s df for the one-factor structure was 25 and the df for the four-factor structure was 20), the RMSEA may be higher.35 Therefore, a proposed RMSEA < 0.1 is acceptable.36

After evaluating the factor structures for each region’s participants, measurement invariance was examined across the three regions’ participants using multigroup CFA (MGCFA). Again, the MGCFA adopted the DWLS estimator and examined the two factor structures. In the MGCFA, three nested models were defined for the present study: Model A was a configural model (ie, a factor structure that allowed the factor loadings and item threshold to be different across the three regions’ participants); Model B was a loading-constrained model (ie, a factor structure that constrained the factor loadings being equal but allowed item threshold to be different across the three regions’ participants); and Model C was a loading-and-threshold-constrained model (ie, a factor structure that constrained both factor loadings and item thresholds to be equal across the three regions’ participants). Later, fit indices derived from the MGCFA in the nested models were used to examine whether the two MoVac-COVID19S factor structures were equivalent across the three regions’ participants. More specifically, the CFI > 0.95, and RMSEA and SRMR < 0.08 in the configural model would indicate the three regions’ participants interpreted the MoVac-COVID19S in the same factor structure. The ΔCFI > −0.01; ΔRMSEA < 0.015; ΔSRMR < 0.03 between the configural and the loading-constrained models (ΔCFI = CFI value of the loading-constrained model minus CFI value of the configural model) would indicate the three regions’ participants interpreted the MoVac-COVID19S items in the same factor loadings. The ΔCFI > −0.01; ΔRMSEA < 0.015; ΔSRMR < 0.01 between the loading-constrained and loading-and-threshold-constrained models (ΔCFI = CFI value of the loading-constrained model minus CFI value of the configural model) would indicate the three regions’ participants interpreted the MoVac-COVID19S items in the same item thresholds.37 The statistical analyses in the present study were performed using IBM SPSS 24.0 or LISREL 8.7.

Results

The participants’ characteristics and their MoVac-COVID19S item scores are presented in Table 1. The three student samples – Taiwanese (n = 614), Indonesian (n = 671), and Malay (n = 524) – were relatively young (mean age = 24.60 years for Taiwanese [SD = 6.58]; = 23.58 years for Indonesian [SD = 7.15]; and = 22.28 years for Malay [SD = 3.17]) and primarily female (69.7% for Taiwanese; 80.5% for Indonesian; and 68.3% for Malay). All the participants had received higher education with over two-fifths of the Taiwanese participants having a postgraduate degree (42.2%). Most of the Indonesian students were majoring in a health-related program (73.0%) but this was not the case for Taiwanese students (39.1%) and Malay students (9.7%).

Table 1

Demographic Variables and Motors of COVID-19 Vaccination Acceptance Scale Item Scores Among Taiwanese, Indonesian, and Malay Samples

	Taiwan	Indonesia	Malaysia
N	614	671	524
Sex, N (%)
Female	428 (69.7%)	540 (80.5%)	358 (68.3%)
Male	186 (30.3%)	131 (19.5%)	166 (31.7%)
Age, M (SD)	24.60 (6.58)	23.58 (7.15)	22.28 (3.17)
Range	19–60	18–55	19–46
Educational level, N (%)
Bachelor	355 (57.8%)	553 (82.41%)	487 (92.9%)
Master	231 (37.6%)	62 (9.2%)	26 (5.0%)
Doctoral	28 (4.6%)	18 (2.7%)	11 (2.1%)
Education, N (%)
Non-health-related	374 (60.9%)	181 (27.0%)	473 (90.3%)
Health-related	240 (39.1%)	490 (73.0%)	51 (9.7%)
MoVac-COVID19S, M (SD)
Item 1	5.13 (1.46)	5.65 (1.55)	5.93 (1.23)
Item 2	5.58 (1.23)	5.60 (1.58)	5.68 (1.16)
Item 3	6.16 (0.99)	5.92 (1.59)	6.28 (1.07)
Item 4	5.63 (1.36)	5.54 (1.62)	5.90 (1.33)
Item 5	5.70 (1.19)	5.29 (1.65)	5.86 (1.16)
Item 6	6.23 (0.88)	5.74 (1.54)	6.14 (1.14)
Item 7	5.89 (1.05)	5.65 (1.56)	6.13 (1.09)
Item 8	6.46 (0.84)	5.25 (1.79)	4.93 (1.72)
Item 9	5.87 (1.10)	5.62 (1.59)	5.69 (1.27)

Abbreviations: M, Mean; SD, Standard deviation.

The CFA results (Table 2) indicated that the one-factor structure of the MoVac-COVID19S was fully supported among the Indonesian and Malay participants (CFI = 0.991 and 0.998; TLI = 0.997 and 0.987; RMSEA = 0.040 and 0.071; and SRMR = 0.014 and 0.039). However, the RMSEA (0.112) was too high among Taiwanese participants regarding the one-factor structure of the MoVac-COVID19S, although the other fit indices were satisfactory. The four-factor structure of the MoVac-COVID19S was again fully supported by the Indonesian and Malay participants (CFI = 0.998 and 0.998; TLI = 0.997 and 0.996; RMSEA = 0.044 and 0.038; and SRMR = 0.013 and 0.018). The RMSEA (0.091) for the Taiwanese participants regarding the four-factor structure of the MoVac-COVID19S was slightly high but acceptable. Moreover, all other fit indices were satisfactory for the Taiwanese participants regarding the four-factor structure of the MoVac-COVID19S (CFI = 0.985; TLI = 0.973; SRMR = 0.049). Therefore, the present data acceptably fitted the four-factor structure of the MoVac-COVID19S.

Table 2

Confirmatory Factor Analysis Results of the Motors of COVID-19 Vaccination Acceptance Scale Among Taiwanese, Indonesian, and Malay Samples

Model	Sample	χ2 (df)	CFI	TLI	RMSEA	SRMR
One-factor model	Taiwan	217.83 (25)	0.971	0.958	0.112	0.068
	Indonesia	52.12 (25)	0.998	0.997	0.040	0.014
	Malaysia	91.04 (25)	0.991	0.987	0.071	0.039
Four-factor model	Taiwan	121.09 (20)	0.985	0.973	0.091	0.049
	Indonesia	45.72 (20)	0.998	0.997	0.044	0.013
	Malaysia	35.05 (20)	0.998	0.996	0.038	0.018

Note: Excellent fit values are in bold; ie, CFI and TLI > 0.95; RMSEA and SRMR < 0.08.

Abbreviations: CFI, comparative fit index; TLI, Tucker–Lewis index; RMSEA, root mean square error of approximation; SRMR, standardized root mean square residual.

MGCFA further demonstrated that the one-factor structure was not invariant across the three regions’ participants. However, the four-factor structure was invariant across the three regions’ participants (Table 3). More specifically, only ΔRMSEA and ΔSRMR were found to be slightly higher than the suggested cutoffs (0.019 vs 0.015 for ΔRMSEA; 0.011 vs 0.01 for ΔSRMR) when evaluating the item threshold invariance.

Table 3

Measurement Invariance Results of the Motors of COVID-19 Vaccination Acceptance Scale Among Taiwanese, Indonesian, and Malay Samples

	Model A	Model B	Model C
One-factor model
χ2 (df)/ Δχ2 (df)	314.62 (75)	205.95 (16)	695.90 (16)
CFI/ ΔCFI	0.988	−0.010	−0.016
RMSEA/ ΔRMSEA	0.073	0.016	0.042
SRMR/ ΔSRMR	0.039	0.134	−0.030
Four-factor model
χ2 (df)/ Δχ2 (df)	269.10 (60)	16.22 (10)	198.43 (10)
CFI/ ΔCFI	0.987	0.000	−0.001
RMSEA/ ΔRMSEA	0.076	−0.004	0.019
SRMR/ ΔSRMR	0.011	0.014	0.011

Notes: Model A: configural model; Model B: loadings constrained equal across subsamples; Model C: loadings and thresholds constrained equal across subsamples. Excellent fit values are in bold; ie, CFI > 0.95; RMSEA and SRMR < 0.08. Supported measurement invariance values are in bold; ie, ΔCFI > −0.01; ΔRMSEA < 0.015; ΔSRMR < 0.03 (for factor loading) or < 0.01 (for item threshold).

Abbreviations: CFI, comparative fit index; RMSEA, root mean square error of approximation; SRMR, standardized root mean square residual.

Discussion

The present study evaluated the factor structure and measurement invariance of the nine-item MoVac-COVID19S among university students in three different countries (ie, Taiwan, Indonesia, and Malaysia). The participants recruited among these three countries were young adults and had relatively good sample sizes (ie, every country had the sample size at very good level for CFA results).31 More specifically, the participants were all university students with the majority studying on a health-related program. Both one-factor and four-factor structures of the MoVac-COVID19S were supported among Indonesian and Malaysian participants. However, only the four-factor structure was somewhat acceptable among the Taiwanese participants (ie, there was a slightly but somewhat acceptable RMSEA smaller than 0.136,38 among the Taiwanese sample). In the present study, the one-factor structure of the MoVac-COVID19S had a poor RMSEA among the Taiwanese sample.

Therefore, the one-factor structure in the present study was found to be non-invariant across three different countries, unlike the four-factor structure showing invariance across the countries. These findings were similar to the previous study investigating the invariance of the MoVac-COVID19S among Taiwanese and Mainland China students,21 where the four-factor structure (but not one-factor structure) of the MoVac-COVID19S (including both nine-item and 12-item versions) was invariant across Taiwanese and mainland Chinese university students. Therefore, the present study extended the findings of Yeh et al21 to another two countries (ie, Indonesia and Malaysia). In other words, the present study’s findings when combined with prior evidence demonstrate that the MoVac-COVID19S has a consistent four-factor structure corresponding to the CME.23,24 Therefore, when using MoVac-COVID19S to assess COVID-19 vaccine acceptance across different subsamples, the four-factor structure should be applied for accurately understanding the underlying constructs of COVID-19 vaccine acceptance. In other words, the invariant findings of present study on the four-factor structure of MoVac-COVID19S demonstrate that MoVac-COVID19S meaningfully compares the willingness of receiving COVID-19 vaccines among Taiwanese, Indonesian, and Malay students.

The four-factor structure of the MoVac-COVID19S has been supported by another study using another psychometric testing method (ie, Rasch analysis) among Taiwanese and Ghanaians.22 More specifically, Fan et al22 found that all the MoVac-COVID19S items (including those negatively worded ones) had satisfactory fit statistics in constructing the latent concepts of values, impacts, knowledge, and autonomy. Moreover, the MoVac-COVID19S items were found to be invariant across Taiwanese and Ghanaian participants. However, Fan et al22 did not test the nine-item MoVac-COVID19S. Therefore, the present study cannot be directly compared with the study findings of Fan et al (ie, the present study reported the psychometric evidence concerning the nine-item MoVac-COVID19S whereas Fan et al examined the 12-item MoVac-COVID19S). Nevertheless, the present study’s findings agree with Fan et al’s findings that MoVac-COVID19S should be treated as a four-factor structure corresponding to the CME.23,24

It is important to show that the CME is supported in the MoVac-COVID19S. Not only because the MoVac-COVID19S was developed using the CME20,21 but the guidance of a well-established theoretical framework can help health-care providers design effective programs to improve vaccine uptake.39,40 Using the present CME and MoVac-COVID19S as an example, if the MoVac-COVID19S finds that individuals have low scores on knowledge, government agencies and/or health-care providers should work on campaigns targeting COVID-19 vaccine knowledge improvement. When the knowledge is improved, it may be possible to increase the acceptance and willingness of COVID-19 vaccine uptake among individuals with poor knowledge concerning the COVID-19 vaccine. However, further empirical evidence on using the MoVac-COVID19S in identifying potential factors in improving COVID-19 vaccine acceptance is needed.

The present study had two key strengths. First, the validated instrument (ie, MoVac-COVID19S) was developed based on the CME, which indicates that the MoVac-COVID19S has a strong theoretical framework to support its ability in assessing COVID-19 vaccine uptake acceptance. Therefore, relevant stakeholders or health-care providers can use MoVac-COVID19S to understand the underlying psychological mechanism as to why an individual does not get COVID-19 vaccinated. Second, the present study was conducted across three countries and the measurement invariance of COVID-19S was examined. Therefore, the present study’s findings support the use of the MoVac-COVID19S and it is acceptable in assessing and comparing COVID-19 vaccination willingness across Taiwan, Indonesia, and Malaysia.

There are some limitations in the present study. First, the present study only recruited university students via convenience sampling. Therefore, the present study’s findings cannot necessarily be generalized. Given that most evidence concerning the psychometric evidence of the MoVac-COVID19S relates to university students,20–22 future studies using the MoVac-COVID19S on populations other than university students are warranted. Second, the present study only examined factor structure and invariance for the MoVac-COVID19S. Therefore, additional psychometric testing concerning the MoVac-COVID19S (eg, test-retest reliability, responsiveness, and concurrent validity) was not carried out among the present studied sample. Third, the characteristics of the university students in the three countries (ie, Taiwan, Indonesia, and Malaysia) were not comparable in terms of their age, gender distribution, educational level, and program enrolled upon in the university. Therefore, the measurement invariance testing between the three countries might somewhat be biased. However, given that different demographic features usually lead to differences in outcome measures, the present study’s measurement invariance findings are likely to be underestimated rather than overestimated (ie, the invariance findings could be concluded as being valid).

Conclusion

The present psychometric evaluation study found that the MoVac-COVID19S, an instrument modified from the MoVac-Flu Scale, contains good construct validity among university students in three different countries (ie, Taiwan, Indonesia, and Malaysia). The four-factor structure of the MoVac-COVID19S was supported and health-care providers may want to assess the four underlying constructs to better understand why a university student accepts or declines COVID-19 vaccine uptake in the three countries. Using the findings, government policymakers and health-care authorities can design appropriate programs to help decrease vaccine hesitancy.