Development and validation of a questionnaire to assess knowledge, attitude, practices, and concerns regarding COVID-19 vaccination among the general population.

BACKGROUND AND AIMS: There seems to be hesitation in the general population in accepting COVID 19 vaccine because of associated myths and/or misinformation. This study is dedicated to develop and validate a tool to interpret vaccine acceptance and/or hesitancy by assessing the knowledge, attitude, practices, and concerns regarding the COVID vaccine.
MATERIAL AND METHODS: Mixed methods study design was used. In phase 1, the questionnaire was developed through literature review, focus group discussion, expert evaluation, and pre-testing. In phase 2, the validity of the questionnaire was obtained by conducting a cross-sectional survey on 201 participants. The construct validity was established via principal component analysis. Cronbach's alpha value was used to assess the reliability of the questionnaire.
RESULTS: The 39-item questionnaire to assess the knowledge, attitude, practices, and concerns regarding the COVID-19 vaccine was developed. The Cronbach's alpha value of the questionnaire was 0.86 suggesting a good internal consistency.
CONCLUSION: The developed tool is valid to assess the knowledge, attitude, practices and concerns regarding the COVID-19 vaccine acceptance and/or hesitancy. It has the potential utility for healthcare workers and government authorities to further build vaccine literacy.

Introduction

COVID-19 pandemic has affected many aspects of the people’s life including physical, social, emotional and behavioural wellbeing [[1], [2], [3], [4]]. People are desperate to get rid of this pandemic and mass vaccination seems to be a promising measure [5].

In India, the COVID-19 vaccine was launched on 16th January 2021 for healthcare and frontline workers [6]. A significant proportion of eligible candidates have not turned up for their second dose of vaccine [7]. Despite immense efforts made to develop a safe and effective vaccine, people are hesitant to accept the vaccine [8]. The vaccine’s acceptance is influenced by multiple factors such as their knowledge and perception on the perceived likelihood of the COVID-19 spread, perceived safety of vaccine, logistics, perceived efficacy of the vaccine and perceived risk etc. Thus, it is pivotal to interpret people’s knowledge, attitude, practices, and concerns regarding COVID-19 vaccine to improve its mass acceptance [9].

The studies conducted so far have assessed people’s opinion regarding the COVID-19 vaccine way before it was introduced in real time [[9], [10], [11]]. Besides, these studies have used semi-structured questionnaires that have not been validated [[9], [10], [11]]. Moreover, these survey tools have been administered in western population with different socio-demographic and cultural factors influencing the acceptance of COVID-19 vaccine. Initial studies done in this area have emphasized upon the need to develop a questionnaire to interpret views of Indian population regarding the vaccine [12]. Thus, we planned this study to develop a validated, concise and easy to administer questionnaire to assess the vaccine acceptance and/or hesitancy among the general population.

Methods

The questionnaire was developed and validated using a mixed methods study design [[13], [14], [15]]. The two-phased standardized methodology was implemented: Phase 1 (qualitative phase) for development of the questionnaire and Phase 2 (quantitative phase) for the validation of the questionnaire. The ethics approval was obtained from the ethics committee of All India Institute of Medical Sciences, New Delhi (IECPG-194/24.02.2021). The informed consent was taken either through the Google Form or telephonically for participants for which the investigators filled the Google Form.

Phase 1: Questionnaire development

The questionnaire was developed following a systematic methodology comprising literature review, focus group discussion (FGD), expert evaluation and pretesting (Table 1 ).

Table 1

Steps involved in questionnaire development and validation.

Step	Nature of activity	Methods	Number of items at the end of step	Addition or subtraction
I	Development of construct	Literature review	45	–
II	Development of construct	FGDs	57	Addition of 12 items
III	Item generation	Develop items	57	–
IV	Establishment of face and content validity	Expert validation	39	Deletion of 18 items
V	Establishment of Construct validity	Item analysis and Factor analysis	39	–

Literature review

A comprehensive literature review was carried out using search engines like Google Scholar and Pubmed to gain an understanding of the existing evidence on the knowledge, attitude, practice and concerns among people regarding COVID-19 vaccine. The keyword string (“Coronavirus vaccin∗” OR “COVID-19 vaccin∗” OR vaccin∗ OR “COVID-19 Immunisation”) AND (Knowledge∗ OR understand∗ OR literacy∗) AND (Attitude OR belief OR perception∗) AND (Practice∗ OR Behaviour∗) was used. This initial search resulted in 6428 related articles. After screening the titles, abstracts and full-texts, 17 articles were found to be relevant resulting in the generation of 45 items.

Focus Group Discussion (FGD)

The FGDs were carried out [12] with different target groups such as doctors, subject experts, paramedics, other hospital staff, middle-aged and elderly, working professionals, young adults and housewives. A diverse sample was selected using the principle of maximum diversity on the basis of different socio-demographic parameters such as age, gender, socio-economic and education status. Purposive sampling technique was used to identify the participants. Indian residents with age above 18 years, who could speak and understand Hindi and/or English were enrolled in the discussion. Eight FGDs were conducted involving five to six participants in each using video conferencing via Google Meet. On an average, each discussion lasted for 45–50 min till saturation of themes. A semi-structured discussion guide used by the moderator to understand the perspective on different aspects related to knowledge, attitude, practice and concerns regarding COVID-19 vaccine. All discussions were transcribed verbatim, translated to English and de-identified by removing any personal information. This resulted in 12 items in the construct.

The final construct after the extensive literature review and FGDs comprised 57 items. Due attention was paid to ensure that items were organised in an appropriate sequence avoiding overlapping. The questions were framed in simple language, with no ambiguity.

Expert evaluation for face and content validity of questionnaire

The developed questionnaire was evaluated for critical appraisal and content and face validity by the team of six experts from the departments of General Medicine, Community Medicine, Geriatric Medicine and Clinical Psychology. On this basis, 18 items were deleted (due to repetition and non-relevance) and 6 were re-worded (to enhance the clarity).

Phase 2: Questionnaire validation

In this phase, the data was collected from 13 March 2021 to 19 March 2021 through a web-based questionnaire via Google form. The questionnaire was administered to 201 participants aged 18 years and above. Participants were recruited via convenience sampling from diverse population groups in view of fulfilling maximum diversity.

Statistical analysis

Descriptive statistics was used to analyse demographics such as age, gender, education qualification, occupation and socio-economic status. The mean, standard deviation, median, quartile and range were calculated for quantitative parameters. The internal consistency was assessed using Cronbach’s alpha. The Cronbach’s alpha value of 0.7 or higher indicates good internal consistency [16]. The content validity and face validity were established through FGDs and expert evaluation. For construct validity, the exploratory factor analysis with varimax rotation was carried out to test the domain structure [17]. The Kaiser–Mayer–Olkin (KMO) measure was used to assess sample adequacy, and values of more than 0·5 show that the data are suitable for factor analysis. p values < 0.05 were considered as significant. The data was analyzed using IBM SPSS Statistics 24 software.

Results

Following the systematic methodology, the final version of the knowledge, attitude, practices and concerns (KAPC) questionnaire comprising 39 items (Box 1 ) is freely available for use. The first section of the questionnaire consists of the items related to socio-demographic profile and a question regarding getting COVID-19 vaccine. The second section of the questionnaire consists of the items regarding the knowledge about the COVID-19 vaccine and the source of information. Besides, the items have been included to assess the attitude, perception and concerns (drivers and barriers) of people regarding the COVID-19 vaccine.

Alt-text: Box 1

Socio-demographic profile of study participants

The socio-demographic information of 201 participants in the validation phase is depicted in Table 2 . The sample had a predominance of male participants (54.73%) with the mean age of 34.39 ± 13.67 years, residing in metropolitan areas (49.25%) and belonging to the middle socio-economic class (73.63%). Slightly more than three-fourth (79.10%) of the participants reported that they have not been vaccinated with COVID-19 vaccine.

Table 2

Demographic characteristics of participants (n = 201).

Socio-demographic variable	Frequency (n = 201)	Percentage (%)
Age (years)
18–35	134	66.67
35–60	54	26.87
>60	13	6.47
Gender
Male	110	54.73
Female	91	45.27
Type of residence
Metropolitan	99	49.25
City	72	35.82
Town	14	6.97
Village	16	7.96
Socio-economic Status
High	18	8.96
Middle	148	73.63
Low	35	17.41

Descriptive statistics of survey results

Findings depicted that slightly more than half of the participants (59.20%) knew that it is not legally mandatory to take COVID-19 vaccine. Participants had a fair idea about the eligibility of different target groups for the vaccination as nearly three-fourths of the participants knew that adults aged over 18 years (78.11%) and recovered COVID patients (75.12%) were eligible for the vaccine, however, only half of the participants (51.74%) were aware about the eligibility of people with comorbidities. Nearly half of the participants (49.25%) thought that the immunity against the COVID-19 virus could be achieved 14 days after the first dose of vaccination. Moreover, participants were significantly influenced by various sources of information such as healthcare workers (82.59%), family and friends (82.09%), government agencies (81.6%), news from TV/radio (77.11%) and social media platforms (69.35%).

Majority of the participants were willing to get vaccinated (59.21%) and would recommend it to their family and friends (65.67%). Various motivating factors influencing them to get vaccinated include perception that there is no harm in getting vaccinated and its benefits outweigh the risks involved. Moreover, participants were interested in getting vaccinated if their healthcare provider would recommend them and if their role models, senior doctors and scientists would get vaccinated. However, majority participants (70.15%) had a belief that the vaccine would eradicate COVID-19 virus. Apart from this, there were few concerns among people regarding the vaccine as the majority of the participants expressed their concern about the rapid development of the vaccine (64.68%) and the unforeseen future effects that might be associated with it (45.77%).

Validity of the questionnaire

The content validity and face validity of the questionnaire were established through a satisfactory level of agreement among panelists. The construct validity was established by using factor analysis. Correlation matrix was used to assess the degree of correlation. Kaiser-Meyer-Olkin value (0.826) and the Bartlett test of sphericity (Chi-squared, df = 666; P-value <0.001), were used to establish sampling adequacy. This was followed by factor analysis using principal component analysis and varimax rotation to examine domain structure. The scree plot provided an estimate of the number of tenable factors, however, the eigenvalue with the cut-off of 1 was used to determine the number of factors. This resulted in the retainment of 10 factors in the final questionnaire.

Reliability of the questionnaire

The internal consistency of the questionnaire was established by calculating Cronbach’s alpha coefficient. The reliability coefficient came out to be 0.86, suggesting a good internal consistency.

Discussion

The success of COVID-19 vaccination drive depends on the acceptance of the vaccine among masses. Experts find it crucial to determine the vaccine acceptance. Following a systematic methodology, we have developed a validated questionnaire that is concise and easy to comprehend. The questionnaire will enable us to interpret vaccine acceptance and/or hesitancy by assessing the knowledge, attitude, practices and concerns related to the COVID-19 vaccine.

The vaccine literacy influences the vaccine acceptance. In this questionnaire, the knowledge of people regarding the COVID-19 vaccine was assessed by asking various questions related to its legal mandatoriness, eligibility of different population groups and time span within which the vaccine could provide protective immunity against the virus. Further, we also evaluated various sources of information that might influence their decision regarding getting vaccinated. Along with this, the attitude of people towards the vaccine was assessed by interpreting their willingness to get vaccinated, extending recommendations to their family and friends and paying for the vaccine. Although, the willingness to get vaccinated is directly influenced by certain drivers and concerns related to COVID-19 vaccine, which is also evaluated in this questionnaire.

During this unprecedented pandemic, various studies have been conducted to assess the vaccine acceptance among people. However, in the study conducted by Lazarus et al. [9] and Harapan et al. [10], no validated tool has been used to assess knowledge, attitude and practices towards the COVID vaccine. In yet another study by Shekhar et al. [11], the attitude and concerns regarding the COVID-19 vaccine were assessed but no information has been provided about the validity of the questionnaire. We have developed this questionnaire to overcome these limitations.

There are several strengths associated with the questionnaire. First, it is a concise and easy to use questionnaire which can be used in a resource constrained setting with minimal participation burden. Second, it will help in quick assessment of vaccine acceptance and/or hesitancy among different population groups. Third, the findings obtained by using the questionnaire will provide government authorities and healthcare providers an insightful understanding about various drivers and barriers associated with the COVID-19 vaccine and gauge current levels of willingness among people to get vaccinated. However, the study holds a limitation. Predictive and concurrent validity could not be established as it required a long-term follow-up.

In conclusion, a reliable and validated questionnaire has been developed in this study which will enable us to assess the knowledge, attitude, practices, and concerns among people regarding the vaccine. This tool will highlight various drivers and barriers related to vaccine acceptance, thus, assisting the healthcare workers and government authorities to work towards a successful vaccination drive by building vaccine literacy.

Declarations

Not applicable.

Financial support and sponsorship

None.

Declaration of competing interest

The Author(s) declare(s) that there is no conflict of interest.