Development and Psychometric Properties of the Health Belief Scales Toward COVID-19 Vaccine in Ho Chi Minh City, Vietnam.

BACKGROUND: Beliefs of an individual about health conditions can play an important role in contributing to their behavior concerning good and bad health practices. The aim of this study was to develop and assess a set of vaccination belief scales in relation to COVID-19 vaccines.
METHODS: A cross-sectional study was conducted between July 2020 and January 2021 using a systematic random sampling of 425 adult outpatients at two hospitals in Ho Chi Minh City, Vietnam. An instrument was designed based on the dimensional structure and wording of the health belief model (HBM), which was followed by the assessment of the instrument's internal consistency and the construct validity.
RESULTS: The 15-item instrument showed the content validity index (CVI) of scales reached a value of 1.0. Exploratory factor analysis was performed on the first sample (n = 170) and extracted 12 out of 15 draft items in a four-factor model (threat of disease, perceived benefits, perceived barriers, and cues to action) that accounted for 68.3% of the total variance. Cronbach's alpha coefficient of 0.765 showed as satisfactory. Following this, the confirmatory factor analysis in the remaining sample (n = 255) found a good fit between a four-factor model and a theoretical model of HBM with acceptable values of fit indices.
CONCLUSION: Beliefs scales for COVID-19 vaccination have been determined to be valid and reliable. They can be a helpful instrument for health educators to use for assessing immunization beliefs of individuals and the public where there is the need to implement new vaccines, such as the COVID-19 vaccine, before they are used more widely across the community.

Introduction

Coronavirus disease 2019 (COVID-19) first appeared around December 2019, and has been affecting people around the world with unforeseen consequences to all aspects of living and health, as well as negative impacts on economies.1 According to the World Health Organization (WHO), the data updated on January 13, 2021 showed that over 90,335,008 confirmed cases had been officially recorded in more than 223 countries or territories worldwide with more than 1,954,336 deaths.2 Vietnam has been marked as one of the countries that have successfully responded to the threat of the pandemic with a total of just 1521 confirmed cases and 35 deaths.3 Currently, the COVID-19 pandemic continues to adversely affect people worldwide and now we have seen the emergence of the SARS-CoV-2 variant recently in the UK and Africa, which leads to the hospitals and health-care systems being faced with a number of challenges.4–6 People who are most at risk of severe illness and death include those aged over 65 years and those with underlying diseases include hypertension, diabetes, cardiovascular illness, cancer and chronic respiratory disease.7 There is still no specific treatment for COVID-19, so performing all measures of physical prevention such as wearing masks, washing hands, maintaining social distance are deemed essential.8 Besides, the development of a safe and effective vaccine is considered a way to control the pandemic.9 As of January 2021, the WHO recorded over 235 potential vaccines against COVID-19 that are currently in pre-clinical and clinical process with some vaccines showing high efficacy, such as the Pfizer and BioNTech COVID-19 vaccine which may provide some early protection with the efficacy being 52% between the first and second doses and growing to 95% in seven days or more after the second dose. These vaccines have been approved widely in the European Union as well in Canada and the US.10–12 Also, the Oxford vaccine has been reported to be up to 90% effective in Phase III trial results.13 In Vietnam, the Government has been supporting the development process of a safe and effective vaccine. Noted is the vaccine named Nano Covax, which has been developed to Phase I and preparing for the process of Phase II clinical trials.14 At present, little is known about an instrument that has been established to measure the contributing factors in COVID-19 vaccination-related behaviors. The health belief model (HBM), which includes four factors of perceived susceptibility and perceived severity (beliefs regarding illness and beliefs regarding what the consequences will be of contracting illness), perceived benefits (individual’s beliefs about benefits that they will get from participating in health-related behaviors), barriers to vaccination (any barriers that might be standing in their way), and cues to action (exposure to information that encourages them to take action),15 has successfully been used for explaining why individuals adopt personal health behaviors toward vaccinations,16–18 and could be used to design and tackle ways to improve vaccination beliefs before a new vaccine is offered to the public. As a result, this study aims to develop and assess the instrument for COVID-19 vaccination beliefs based on HBM among people with chronic diseases in Ho Chi Minh City, including the use of exploratory (EFA) and confirmatory factor analysis (CFA) processes.

Methods

Design and Context

The study was performed between July 2020 and January 2021, the context being that the cases of COVID-19 infection and death have been increasing rapidly around the world.2 People of all ages can contract COVID-19, and older people and people with chronic diseases are at high risk of severe illness and death.19 The COVID-19 vaccines are in the process of phase II and III clinical trials. The evidence on the promotion of vaccination is generally beneficent in the context of the pandemic.10 However, the acceptance and uptake of new vaccines in the community will be an unprecedented challenge. The HBM has proved to be a useful tool for predicting vaccination behaviors in previous studies.16,17 As a result, the development of tools for measuring COVID-19 vaccination beliefs based on HBM are necessary.

Development of the Items Based on the Domains of HBM

First, a 15-item draft scale was designed based on the dimensional structure and wording of the HBM, combining the common keywords in Vietnamese of the hepatitis B vaccination scales which were developed in our previous study,20 and information about COVID-19 from the WHO.2 This combination created appropriate concept labels and reflected, completely, the content of each dimension. The four domains of HBM include: (i) Perceived threat (susceptibility and severity) which is the combining of two subscales of susceptibility and severity. For perceived susceptibility: the research showed that an individual does not believe that they are at risk and they will not change their behaviors. For example, individuals who do not think they will get the illness are less likely to get a vaccination. Two items were included in this domain, “I am worried about the likelihood of getting COVID-19 in the future”, and “Contracting COVID-19 is currently a high possibility for me”. The perceived severity, including how dangerous COVID-19 is, is shown in two items: “I could be severely ill if I contract COVID-19”, and “I am afraid to even think about getting ill with COVID-19”. (ii) Perceived benefits are what they know about the benefits that they will get from receiving the vaccination, including three items: “Vaccination will prevent me from contracting COVID-19”, “By being immunized, I will protect others from COVID-19”, and “I will feel less worried about the possibility of severe illness”. (iii) Barriers to vaccination are measured by four items: “I am afraid of the vaccination because the COVID-19 vaccine can cause Adverse Events Following Immunization (AEFIs)”, “The COVID-19 shot can be painful”, “COVID-19 infection can be self-limiting and it is unnecessary to receive the vaccination”, and “The cost of COVID-19 vaccine will be expensive”. (iv) Cues to action are external events that prompt a desire to perform a behavior change, which is measured by four items: “all people should be vaccinated to promote public health”, “I follow new information and debate about vaccines”, “I would like to receive information about vaccines from healthcare workers”, and “I will receive a COVID-19 vaccine if my healthcare workers recommend a vaccination”.

Adaptation Procedures

The 15-items draft scale was sent to a panel of five experts from the University of Medicine and Pharmacy at Ho Chi Minh City, who have worked in fields relating to the subject of the study for over 10 years, including two specialists in infectious diseases, one family doctor, one psychologist, and one expert on public health. The goal was to get their feedback on the quality of items and how well each item reflected the overall construct through a four-point Likert scale ranging from 1 point (cannot be used, not clear) to 4 points (very clear). Besides, these experts were also invited to recommend adaptations for better content of each item. Following the development of the 15 self-administered items, we proceeded with a pilot study, that was sent to 20 people with chronic diseases at District 2 Hospital. After completing a 15-item questionnaire, the participants were asked for their opinions about the wording of items correlating with the comprehensibility, by one question: Was there any difficulty in understanding each item? And they were also required to suggest modifications to improve clarity if it was needed. According to their contributions, the final items were completed, and the data in the pilot test will not be retained for further analysis.

Data Collection Procedures

A cross-sectional study was considered by systematic random sampling of 425 adults with chronic conditions including diabetes, heart disease and cancer which were diagnosed by physicians at outpatient departments of two hospitals in Ho Chi Minh City; however, patients who had cognitive problems, or missed out any items in the 15-item scales were excluded. A self-administered, structured questionnaire was used which consisted of items on the baseline characteristics of participants and 15-items of created belief scales towards the COVID-19 vaccination. The sample size was calculated based on the number of items in the questionnaire. To reach a sufficient sample size for psychometric study, it needed 10–20 observations per item in factor analysis.21 Data were randomly categorized into two separate samplings. The first sample (n = 170) was to perform consistency reliability analysis (CRA) and the EFA while the last sample (n = 225) was used to analyze confirmatory factor analysis (CFA).

Assessment of Psychometric Properties

STATA software 13.0 was used for analyzing data with the frequencies and percentages being calculated to describe the baseline characteristics of participants.

The content validity index (CVI) of items was to confirm the integrity of a construct.22,23 The CVI was estimated by the number of experts who gave a score of 3 or 4 to each item divided by the number of all experts. Because the total number in the expert panel was five or fewer, the CVI value was expected to reach 1.0.24

The 15-item draft scale measured perceived threat (4 items), benefits (3 items), vaccination barriers (4 items), and cues to action (4 items). Each item was estimated using a 5-point Likert scale ranging from 1 for strongly agree to 5 points for strongly disagree. The psychometric assessment was conducted to determine the construct validity including the reliability of the scales, exploratory factor analysis (EFA), and confirmatory factor analysis (CFA). A measure of reliability was internal consistency, which applied to the consistency among the variables in a summated scale. Firstly, we assess the correlation with each separate item, including the item-to-total correlation and the correlation among items. The results are acceptable when the item-to-total correlations are at a level of 0.30 or higher.25 The second measure is the reliability coefficient, which assesses the consistency of each item and total scales; Cronbach’s alpha of 0.70 or higher is acceptable.21

Kaiser-Meyer-Olkin (KMO) measures of sampling adequacy and Bartlett’s test of sphericity were performed. The KMO value of > 0.6 or higher and p<0.05 were evaluated to be statistically significant.21 The EFA was to determine the number of factors and percentages of explained variance,21 which extracted factors according to eigenvalues greater than 1.0 and used processes of principal components factor (PCF) and orthogonal rotation.21 Besides, the number of factors in the model was also assessed by using a Scree test.21 A value of factor item loadings at or above the 0.3 level was considered significant in factor analysis.26 The remaining sample (n = 255) was followed for CFA, which was to assess the adequacy of a proposed factor structure and the fit between an observed model and a theoretical model.

These indices were used to estimate, as a good fit, between an observed model and a theoretical model: the relative chi-square fit index <3, the comparative fit index (CFI) and Tucker-Lewis index (TLI), incremental fit index (IFI), Coefficient of Determination (CD)> 0.90, the Root Mean Square Error of Approximation (RMSEA) and Standardized Root Mean Squared Residual (SRMR) of <0.08. However, these are considered as acceptable fit indices: χ2/df < 5, CFI > 0.90, RMSEA < 0.10, SRMR = 0.05–0.10 and IFI > 0.90.27,28

Ethics Approval

Our research complied with the Declaration of Helsinki. All the participants in the study signed a consent form before participating, as well as being aware they could withdraw from the study at any time without penalties. The study was approved by the Ethical Committee of the University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam (protocol number 162/UMP- BOARD).

Results

Expert Committee Review About Content Validity

The 15-item draft scale was established, which showed that most experts agreed with the dimensional structure of the draft scales, and no items were deleted from the scales. The CVI of 15-items reached a value of 1.0. Nevertheless, some points needed to be added to gain more clarity and understanding, such as item 2, and item 6 should have added to it “near future”, “and not getting COVID-19”, respectively. After adapting these items, the last version was compiled. Results of a pilot test among people with chronic diseases also showed that there was no difficulty in understanding all items of the instrument, and they all confirmed that no further adaptations were needed for items. It took about 10–15 minutes to complete each response.

Baseline Characteristics of Participants

The demographic characteristics of the participants are presented in Table 1. The results indicate no significant differences between sex, age, occupation, education, and source of information about COVID-19 in the EFA and CFA samples (p>0.05). Most participants were female (67.8%), had a mean age of 52.9±5.6 years and 57.4% had a high school education and higher.

Table 1

Participant’s Characteristics

	Total(n = 425)	RA and EFA(n = 170)	CFA(n = 255)	p
Age (Mean±SD)	52.9 ± 15.6	53.3 ± 15.8	52.6 ± 15.4	0.681
Gender
Male	137 (32.2)	51 (30.0)	86 (33.7)	0.459
Female	288 (67.8)	119 (70.0)	169 (66.3)
Education
Primary education	51 (12.0)	17 (10.0)	34 (13.3)
Secondary education	130 (30.6)	55 (32.4)	75 (29.4)	0.750
Higher education	138 (32.5)	56 (32.9)	82 (32.2)
University/College	106 (24.9)	42 (24.7)	64 (25.1)
Occupation
Worker	87 (20.5)	30 (17.7)	57 (22.4)
Housewife	118 (27.8)	46 (27.1)	72 (28.2)
Officer/Business	98 (23.1)	42 (24.7)	56 (22.0)	0.221
Retired	105 (24.7)	41 (24.1)	64 (25.1)
Health worker	17 (4.0)	11 (6.5)	6 (2.4)
History of Illness
Yes	372 (87.5)	142 (83.5)	230 (90.2)	0.051
No	53 (12.5)	28 (16.5)	25 (9.8)
Source of information
TV	350 (82.4)	141 (82.9)	209 (82.0)	0.897
Social networks	247 (58.1)	98 (57.7)	149 (58.4)	0.920
Relatives	164 (38.6)	63 (37.1)	101 (39.6)	0.612
Websites of Hospital/Ministry of Health	90 (21.2)	34 (20.0)	56 (22.0)	0.716

Reliability Analysis

Table 2 shows data from the first sample (n = 170); after analyzing the internal consistency reliability, three items were deleted from the scales because of no relation to the total items (item-test correlation <0.3). The final 12-item version is shown in ; . The internal consistency with a standardized Cronbach’s alpha coefficient was 0.765. The relationship of each item to the scales was from 0.333–0.683, which indicated from moderate to strong.

Table 2

Cronbach’s α Analysis (n = 170)

Item	Sign	Original Scale			Modified Scale
		Item-Test Corr.	Item-Rest Corr.	α	Item-Test Corr.	Item-Rest Corr.	α
Perceived susceptibility
I am at high risk of COVID-19 infection	+	0.607	0.488	0.719	0.619	0.498	0.737
I think I will get COVID-19 in the near future	+	0.607	0.490	0.719	0.635	0.517	0.735
Perceived severity
I could be severely ill if I got COVID-19	+	0.497	0.378	0.731	0.541	0.423	0.747
I am afraid of even thinking about getting ill with COVID-19	+	0.354	0.212	0.748	0.392	0.246	0.766
Perceived benefits
Immunization will prevent me from contracting COVID-19	+	0.670	0.536	0.711	0.683	0.547	0.729
By being immunized and not getting COVID-19, I will protect others from COVID-19	+	0.622	0.492	0.717	0.645	0.514	0.734
By being immunized, I feel less worried about the possibility of severe illness from getting COVID-19	+	0.584	0.477	0.721	0.574	0.460	0.742
Perceived barriers
I am afraid that the COVID-19 vaccine can cause AEFIs	-	0.372	0.263	0.742	0.333	0.216	0.766
The COVID-19 shot can be painful	-	0.185	0.070	0.756	Excluded
COVID-19 infection can be self-limiting and an unnecessary vaccination	-	0.439	0.274	0.745	0.479	0.312	0.762
I think that the cost of the COVID-19 vaccine will be expensive	-	0.515	0.406	0.729	0.493	0.376	0.751
Cues to action
I think that all people should be vaccinated to promote public health	+	0.488	0.395	0.731	0.441	0.339	0.755
I follow new information and debates about vaccines	+	0.225	0.152	0.748	Excluded
I would like to receive information about vaccines from healthcare workers	+	0.169	0.080	0.753	Excluded
I will receive a COVID-19 vaccine if my healthcare workers recommend a vaccination	+	0.507	0.429	0.731	0.455	0.368	0.754
Total				0.747			0.765

Exploratory Factor Analysis

KMO coefficient of 0.731 and Bartlett’s test of the result was 692.6 (p <0.001) (Table 3). They showed that the sample size was adequate and appropriate for EFA. Four factors were extracted from the EFA with eigenvalues >1 and factor loadings >0.5, which accounted for 68.3% of the observed variance (Table 4). Also, the Scree test found the same results from four extracted factors (Figure 1), including the perceived threat of disease (susceptibility and severity including 4 items), benefits (3 items) and barriers of vaccination (3 items), and cues to action (2 items).

Table 3

Factor Postestimation of the HBM Scale (n = 170)

Factor Postestimation Value	Total	EFA	CFA
Kaiser-Meyer-Olkin coefficiency	0.725	0.731	0.706
Bartlett test
χ2	1583.2	692.6	926.4
Degrees of freedom	66	66	66
p-value	<0.001	<0.001	<0.001

Table 4

Exploratory Factor Analysis for HBM Scale (n = 170)

	Item	Factors 1	Factors 2	Factors 3	Factors 4
1	I am at high risk of COVID-19 infection	0.878	−0.005	−0.196	0.035
2	I think I will get COVID-19 in near future	0.861	0.105	−0.146	−0.029
3	I could be severely ill if I got COVID-19	0.810	0.122	0.166	0.081
4	I am afraid of even thinking about getting ill with COVID-19	0.699	0.015	0.387	0.159
5	Immunization will prevent me from contracting COVID-19	0.238	0.622	−0.145	0.350
6	By being immunized and not getting COVID-19, I will protect others from COVID-19	0.100	0.836	−0.133	0.153
7	By being immunized, I feel less worried about the possibility of severe illness from getting COVID-19	−0.039	0.758	−0.142	−0.040
8	I am afraid that the COVID-19 vaccine can cause AEFIs	0.069	−0.084	0.747	−0.087
9	COVID-19 infection can be self-limiting and unnecessary vaccination	−0.150	−0.370	0.606	−0.169
10	I think that the cost of COVID-19 vaccine will be expensive	−0.091	−0.215	0.725	−0.145
11	I think that all people should be vaccinated to promote public health	0.023	0.139	−0.027	0.883
12	I will receive a COVID-19 vaccine if my healthcare workers recommend a vaccination	0.069	0.053	−0.147	0.863
	Eigenvalue	3.452	2.384	1.337	1.022
	Explain variance	28.8%	19.9%	11.1%	8.5%
	Cumulative explain variance	68.3%

Figure 1

Scree plot of eigenvalues after factor analysis (n = 170).

Confirmatory Factor Analysis

As shown in Figure 2, data were analyzed from the second sample (n=255), model fits were found for four dimensions of HBM, which showed the strong relation of each factor with another and with standardized covariance values of items ranging from 0.067–0.56. Factor loadings of the 12 items were acceptable, ranging from 0.29–0.87. The squared multiple correlations were from 0.24–0.91 meaning that roughly 60% of them were high correlations (≥0.50), showing that the factors contributed to acceptable ability for a good explanation for the variation of items in the model. The model had a fair fit to the data, with χ2/df=3.53, RMSEA=0.094, CFI=0.878, TLI=0.833, SRMR=0.059, and CD=0.994. We followed the process of potential modifications that could better improve the fit of the four-factor model. A revised model in which covariations between the residual items “I am at high risk of COVID-19 infection” and “I think I will get COVID-19 in near future”, and items “I am afraid of even think about getting ill with COVID-19” and “I am afraid that the COVID-19 vaccine can cause AEFIs”, plus items “By being immunized and not getting COVID-19, I will protect others from COVID-19” and “By being immunized, I feel less worried about possibility of severe illness from getting COVID-19” were included and resulted in a considerable improvement of the good fit of the statistics, with χ2/df = 2.63, RMSEA = 0.078, CFI = 0.920, TLI = 0.908, SRMR = 0.056, and CD = 0.991 (Table 5). Therefore, the revised model contributed to a better fit to the data compared with the primary one, by an increase in CFI and TLI, a decrease in RMSEA, SRMR and value of the χ2/df. These data demonstrated that the scales had a sufficient factor of validity. Figure 2 indicated perfectly the standardized solution for the revised four-factor model.

Figure 2

Confirm factor analysis for HBM scale (n = 255).

Table 5

Goodness-of-Fit Statistics

Fit Statistic	Value	Expected Value
Likelihood ratio test for model vs saturated comparison
χ2/df	2.63
p-value	<0.001	>0.05
Population error
Root mean square error of approximation (RMSEA)	0.078	<0.08
RMSEA 90% CI, lower bound	0.061
RMSEA 90% CI, upper bound	0.096	<0.1
Probability RMSEA ≤ 0.05	0.005
Baseline comparison
Comparative fit index (CFI)	0.920	>0.9
Tucker-Lewis index (TLI)	0.908	Close to 1
Size of residuals
Standardized root mean squared residual (SRMR)	0.056	<0.08
Coefficient of determination (CD)	0.991	Close to 1

Discussion

To the best of our knowledge, there was no valuable and reliable instrument to assess beliefs in individuals regarding novel coronavirus vaccines based on the HBM. In the context, there was no specific treatment for COVID-19 whilst it has continued to spread widely around the world and patients aged over 65 years and those who live with underlying diseases are at increased risk of severe illness and death. Also, there still exists insufficient knowledge and practice regarding COVID-19 among chronically illpatients.29 Therefore, immunization is an important health-protective behavior for the improvement of public health and the development of an efficacious and safe COVID-19 vaccine, which is hoped to reduce morbidity and mortality from COVID-19.30 The research of Sanche et al. found that the threshold for COVID-19 herd immunity may be between 55% and 82% of the population.31 Nevertheless, beliefs toward vaccines were one of the major barriers to vaccine uptake and the achievement of herd immunity, especially for protecting the most vulnerable of the population. The HBM is being used widely to have a thorough insight into people’s health-related behaviors and it plays an important role as a predictor in vaccination uptake and has been used in many previous studies.32,33 Therefore, exploring significant HBM constructs in this study toward the new vaccination may be important for tailored interventions to enhance the acceptance of the vaccine when it becomes available. In this study, the instrument was designed based on the HBM theory combining the keywords in our previous study in relation to Hepatitis B vaccination in Vietnamese, followed by information about COVID-19 from the WHO.2,20 The final scale was modified according to the point of view of the specialists. All of the 15-item scales were based on four factors as the HBM theoretical framework was accepted by experts with the expected content validity and some minor wording changes to ensure the comprehensiveness of the items, and also checked by 20 adult patients. Therefore, the instrument was considered as readable for individuals. The final scales of 12-items showed good internal consistency reliability assessed using the Cronbach’s alpha coefficients for the overall scale was 0.765, which was considered satisfactory.34 This result was similar to our previous study20 and Rhodes et al.’s study about HBV vaccination35 as well as Kim’s study in HPV vaccination with Cronbach’s alpha of more than 0.7.16 Meanwhile, Pielak’s study assessed the HBM scale of measles immunization and showed that an overall Cronbach’s alpha was only 0.63, reflecting moderate internal consistency.36 Besides, the scales showed the item-to-total correlations are at an acceptable level (≥0.30).34 Four factors were extracted from the EFA with eigenvalues >1 and factor loadings >0.5, which indicated that the model explained 68.3% of the observed variances, which is a good level since Miller-Carpenter recommends that a scale should explain at least 50% of variance.26

In terms of construct validity, CFA showed that the model fitted the data well including acceptable indices of CFI, TLI, IFI, CD, SRMR, RMSEA. However, the 4-factor model was not satisfactory for the Χ 2 test (p<0.001) but had a good fit χ2/df = 2.63. This might be because the χ2 was impacted by the sample size.37,38 Factor loadings of the 12 items were acceptable ranging from 0.29–0.87. Squared multiple correlations were obtained from 0.24–0.91, showing that the factors contributed acceptable ability to good explanation for the variation of items in the model. This study was consistent with our previous study that assessed psychometric properties of attitude scales for Hepatitis B vaccination.20 Overall, it is demonstrated that the vaccination COVID-19 scales based on HBM had a sufficient factor validity by most of the satisfactory psychometric results of the tests.

Strengths and Limitations

The COVID-19 vaccines are in development so assessing using hypothetical vaccine information may be a little bit different from the real-life situation. Moreover, some trials showed that the COVID-19 vaccination was convincingly effective in small populations and healthy people so it is unknown whether the finding could be generalized to elderly and comorbid patients. Although the level of education can affect answering the questionnaire about beliefs, the majority of participants in our study had a high school education and higher; also, we proceeded with a pilot study to assess the comprehensibility of the questionnaire. Therefore, this is a novel and valid tool to assess an individual’s beliefs about the COVID-19 immunization.

Conclusion

Beliefs scales for COVID-19 vaccination have been determined to be valid and reliable, which can be a helpful instrument for health educators to use for assessing vaccination beliefs of individuals and the public where there is the need to implement new vaccines, such as the COVID-19 vaccine before they are used more widely across the community.