Trends in classifying vaccine hesitancy reasons reported in the WHO/UNICEF Joint Reporting Form, 2014-2017: Use and comparability of the Vaccine Hesitancy Matrix.

Since 2014, the World Health Organization (WHO) member states have been annually reporting vaccine hesitancy reasons, using the WHO/UNICEF Joint Reporting Form (JRF). The Vaccine Hesitancy Matrix (VHM), developed by a WHO strategic advisory group of experts, can serve as an important tool to categorize vaccine hesitancy reasons reported in the JRF. We aimed to describe the reasons for vaccine hesitancy reported globally from 2014 to 2017 to ascertain trends over time and understand the comparability of using the VHM to classify hesitancy reasons from 2014 to 2016 based on previously published literature. We conducted a quantitative content analysis to code and categorize vaccine hesitancy reasons reported in the JRF from 2014 to 2017. Vaccine hesitancy trends were consistent from 2014 to 2017, where vaccine hesitancy reasons were mainly related to "individual and group level influences" (59%) followed by "contextual influences" (25%), and "vaccine- or vaccination-specific issues" (16%). Comparability of our approach to categorize vaccine hesitancy to the previously published JRF data showed that results were mostly but not entirely consistent. Major differences in categorizing vaccine hesitancy were noted between two specific reasons - "experience with past vaccination" (under "individual and group influences") and "risk/benefit- scientific evidence" (under "vaccine and vaccination-specific issues"); this was usually due to lack of clear definitions in some sub-categories and generic responses reported in the JRF. The JRF hesitancy module may benefit from modifications to improve the data quality. Understanding global vaccine hesitancy is crucial and JRF can serve as an important tool, especially with the potential introduction of a COVID-19 vaccine.

INTRODUCTION

Substantial progress has been made over the past four decades in improving global coverage of routine childhood immunization through the Expanded Programme on Immunization (EPI). Between 1980 and 2018, the coverage of the third dose of diphtheria-tetanus-pertussis vaccine (DTP3) increased from around 25% to 85% globally; similarly, the coverage of the first dose of measles-containing vaccine increased from 18% to 85%.[1,2] Despite these improvements, trends over the past decade point to a stagnation in global immunization coverage.[3] Recent outbreaks of vaccine-preventable diseases (VPDs) around the globe, including in Europe, United States, and Africa, have been partly attributed to a rise in vaccine hesitancy; thereby, highlighting the role of vaccine hesitancy in caregivers’ decision to delay or refuse vaccines for their children.[4,5] The World Health Organization’s (WHO) Strategic Advisory Group of Experts on Immunization (SAGE) Vaccine Hesitancy Working Group defines vaccine hesitancy as a “delay in acceptance or refusal of vaccines despite the availability of vaccination services. Vaccine hesitancy is complex and context-specific, varying across time, place, and vaccines. It is influenced by factors, such as complacency, convenience, and confidence” [pp 4161].[6] The Working Group also developed the Vaccine Hesitancy Matrix (VHM),[7] which broadly categorizes the reasons for hesitancy into “contextual influences,” “individual and group influences,” and “vaccine- or vaccination-specific issues” that provide an overarching framework to understand vaccine hesitancy. Each of these three influences further elaborates on the specific reasons for vaccine hesitancy, which are referred to as sub-categories in this manuscript.[7]

To enable and improve monitoring of trends in global vaccine hesitancy, the SAGE Working Group developed indicators for vaccine hesitancy that were included as part of the global WHO/United Nations Children’s Fund (UNICEF) Joint Reporting Form (JRF) for the first time in 2014; since then, these have been reported by the member countries, annually. The JRF is a standardized questionnaire that is used to collect various immunization-related data, including vaccine hesitancy that is completed by the EPI program managers in WHO member states.[8] The vaccine hesitancy indicator captures “reasons for hesitancy to accept vaccines according to the national schedule,” using open-ended questions for the first, second, and third reasons for the hesitancy. In this analysis, we only focus on the trends noted in the three vaccine hesitancy reasons for every member state.

Prior studies have examined the JRF vaccine hesitancy data to quantify the underlying drivers.[9,10] Marti et al. described the reasons for vaccine hesitancy, using only the 2014 JRF data.[9] Lane et al. analyzed trends in global vaccine hesitancy, using 2014–2016 JRF data.[10] Both studies used quantitative content analysis[11] to code the open-ended reasons for the hesitancy reported in JRF into mutually exclusive categories based on the VHM. The coded responses were then described quantitatively. These studies provided crucial insights into vaccine hesitancy trends globally and point to the feasibility of global vaccine hesitancy surveillance, using standardized indicators. However, the comparability of employing quantitative content analysis to describe trends in hesitancy using the VHM has not yet been examined. Moreover, we did not find any other peer-reviewed literature for the 2017 JRF data. Therefore, we aimed to 1) describe the reasons for vaccine hesitancy reported globally in 2017 and compare the results to the previous 3 years (2014–2016) to ascertain trends over time and 2) understand the comparability of using the VHM to classify hesitancy reasons with previously published literature for JRF data from 2014 to 2016. Based on this analysis we provide recommendations for future effective and enhanced usage of VHM and JRF data to assess vaccine hesitancy globally.

METHODS

We conducted a quantitative content analysis, using an approach previously described by Lane et. al. to understand the reasons for vaccine hesitancy reported in the JRF.[10] First, we accessed the publicly available WHO website, hosting the JRF database, which was updated on 22 October 2018.[12] Second, we generated a sub-set of the data on the open-ended responses regarding the reasons for vaccine hesitancy, submitted by the member states from 2014 to 2017. Third, we imported the country-level data into Microsoft Excel 2016 (version 16). Each row in the database represented a unique record for each member state for each of the four years. This resulted in up to four rows of data for each member state. The columns in the dataset contained unique variables for the three vaccine hesitancy reasons reported for each of the four years. Two analysts (SK and BH) independently reviewed and coded the open-ended responses for the vaccine hesitancy reasons, using the VHM established categories (i.e. “individual and group influences,” “contextual influences,” and “vaccine and vaccination-specific issues”) and related sub-categories, classifying the drivers of vaccine hesitancy for 2014–2016 and 2017, respectively.[7] At the end of the coding process, we reviewed all the coding across the years; then we discussed any discrepancies with a supervisor (MFJ) and resolved those. A member state was considered as having no reported hesitancy if the JRF data contained answers stating “no hesitancy” for all three reasons in a year or if a member state reported “no hesitancy” for the first reason followed by “not applicable,” or provided no response for the second and/or the third reason for a given year.

The Excel database was then imported into Stata version 14 SE (StataCorp. 2015. Stata Statistical Software: Release 14. College Station, TX: StataCorp LP) for descriptive analysis. We calculated the frequencies and proportions for all categories and sub-categories of vaccine hesitancy and disaggregated the results by year. We obtained data from the previous analysis by Lane et al.[10] to assess the comparability of using the VHM to classify the reported hesitancy reasons in the JRF for 2014–2016. We compared the percentage of coding for each sub-category of hesitancy in relation to all responses for vaccine hesitancy (including the responses that did not align with VHM) reported for the current and previous analysis. This percentage was obtained by calculating the sum of frequencies for each sub-category for the 3 years (2014–2016) and dividing it by total number of responses for the 3 years (2014–2016).

RESULTS

Comparing the most recent data (2017) to the previous three years (2014–2016)

From 2014 to 2017, 79% to 83% of the 194 member states completed the vaccine hesitancy section of JRF each year by reporting one or more vaccine hesitancy reason(s). In 2017, a total of 451 open-ended responses were provided in the JRF to indicate up to three reasons for vaccine hesitancy per WHO member state (Table 1). For each year, a total of 582 responses were possible for the 194 members, with each member state reporting up to three vaccine hesitancy reason. For 2017, of the total responses, 108 were excluded from further analysis because they did not fit any of the VHM categories (N = 55), reported “not applicable” (N = 32), or reported “no hesitancy” (N = 21) in 2017. Of the remaining hesitancy responses (N = 343), 56% were “individual and group influences”, 29% were “contextual influences”, and 15% were “vaccine- or vaccination-specific issues”. The classifications of the responses were proportionally similar when the 2017 data were compared to that of the previous 3 years (Table 1). For 2017, among the responses that were coded using the VHM, the top three vaccine hesitancy reasons were “knowledge and awareness” (17.8%), “experience with past vaccination” (15.8%) that fall under “individual and group influences”, and “religion/culture/socioeconomic factors” (13.1%) which fall under “contextual influences”, as shown in Figure 1. In each of the four years, most responses were classified as individual and group influences, followed by contextual influences and vaccine- or vaccination-specific influences. Examples of how the responses were classified into the three broad categories and the sub-categories are provided in Table 2.

Table 1.

Distribution of the vaccine hesitancy reasons reported in the WHO/UNICEF Joint Reporting Form, coded based on the Vaccine Hesitancy Matrix, 2014–2017

	2014	2015	2016	2017	2014–2017
	n (%)	n (%)	n (%)	n (%)	n (%)
Member state response rate*	154 (79)	158 (81)	160 (82)	163 (83)	-
Category as per Vaccine Hesitancy Matrix (VHM)
Individual and group	177 (60)	187 (58)	198 (60)	192 (56)	754 (59)
Contextual	67 (23)	76 (24)	83 (25)	98 (29)	324 (25)
Vaccine- or vaccination-specific	52 (18)	57 (18)	49 (15)	53 (15)	211 (16)
Total valid responses	296 (100)**	320 (100)	330 (100)	343 (100)	1,289 (100)
Excluded responses
Not aligned with VHM	43	42	46	55	186
Not applicable	33	23	34	32	122
No hesitancy	45	43	45	21	154
Total excluded responses	121	108	125	108	462
All responses	417	428	455	451	1,751

*Number of member states out of the 194 member states that reported at least one or more hesitancy reason. ** Does not add to exact 100 due to rounding.

WHO = World Health Organization

UNICEF = United Nations Children’s Fund

Figure 1.

Vaccine hesitancy trends using the WHO/UNICEF Joint Reporting Form data on open-ended responses for 2017

Table 2.

Examples of the responses coded based on the Vaccine Hesitancy Matrix, using the data reported in the WHO/UNICEF Joint Reporting Form, 2014–2017

Categories and sub-categories of vaccine hesitancy influences based on the VHM*	Examples of the reasons for vaccine hesitancy given to the open-ended questions as reported by WHO member states in the JRF*
1. Individual and group influences
1.1. Experience with past vaccination 1.2. Beliefs, attitudes about health and prevention 1.3. Knowledge/awareness 1.4. Health systems and providers-trust and personal experience 1.5. Risk/benefit (perceived, heuristic) 1.6. Immunization as a social norm vs. not needed/harmful	Afraid of side effects; doubts of vaccine safety Personal beliefs/philosophical reasons, practice of homeopathic medication, natural lifestyle Low knowledge about vaccine preventable diseases, myths and misconceptions, rumors about side effects Past bad experience at health care facility, mistrust of health staff for fear of catching Ebola Virus Disease, low level of trust between parents and health care workers Lack of parental motivation/neglect, belief that there is no threat-no need, low perceived risk of vaccine preventable diseases Complacency, impact of anti-vaccine movements, the influence of peers
2. Contextual influences
2.1 Communication and media environment 2.2. Influential leaders, gatekeepers, and anti-vaccination lobbies 2.3. Historical influences 2.4. Religion/culture/gender/ socioeconomic 2.5. Politics/policies 2.6. Geographic barriers 2.7. Pharmaceutical industry	Disclosure of potential adverse events in the wrong way in the media; negative information on social networks; negative information about vaccines in the media, the Internet, and other publications Anti-vaccine movement, anti-vaccine campaign, impact of anti-vaccine campaign None reported Religious beliefs, pockets of religious groups, traditional cultural beliefs None reported Inaccessibility/security problem; geographic accessibility of services Vaccine manufacturers, vaccines are designed for pharmacists to profit, pressure from the pharma companies
3. Vaccine and vaccination-specific issues
3.1. Risk/benefit (scientific evidence) 3.2. Introduction of a new vaccine or new formulation 3.3. Mode of administration 3.4. Design of vaccination program/mode of delivery 3.5. Reliability and/or source of vaccine supply 3.6. Vaccination schedule 3.7. Costs 3.8. Role of health care professionals	The situation presented in 2014 with the girls of the Carmen de Bolívar [school-based HPV vaccination program in Colombia] (HPV) [vaccine], still exists in the group of parents and relatives of the country’s adolescents, remembering the risk of getting sick that was before the publicity made around the event, the fear of adverse effect following vaccination (esp. HPV). The fear of new vaccination from epidemic of Ebola virus disease, consent form needed for HPV vaccine The fear of needles, bad application techniques, afraid of injection Discouragement of parents linked to the long waiting line in vaccination posts, parents working and unable to attend, parents do not have enough time to take their children to receive vaccination Temporary dose shortages, the shortage of vaccine, doubts about the quality of vaccines Multiple injections in one session, being scared of three injections at one time, the vaccine schedule Illegal payment of vaccines, costs, vaccinators do not come to their homes for immunization False contradiction by specialist pediatrician, recommendation from HCWs, low awareness of health workers
4. Codes that did not align with the VHM categories	Obstacles, lack of ownership, missing vaccination card, poor demand, lack of interest, vaccine safety, safety

*VHM = Vaccine Hesitancy Matrix; JRF = Joint Reporting Form; HCW: Health care workers; Note: Verbatim responses noted above

Comparability of using VHM to code the hesitancy reasons reported in the JRF (2014-2016)

Comparison in coding for individual and group influences

The largest difference between the two analyses in the category of “individual and group influences” was noted in the percentage of responses classified under the sub-category of “experience with past vaccination.” The prior analysis from 2014 to 2016 classified 1.8% of their total responses under this sub-category, while in the current analysis, we classified 18.0% under this sub-category (Figure 2). The smallest difference in coding frequency was for the responses classified under the sub-category of “immunization as a norm” with 0.3% of the responses coded as such in both analyses.

Figure 2.

Percentage of responses for the two quantitative content analyses, using the WHO/UNICEF Joint Reporting Form data on open-ended responses coded as the individual and group influences for vaccine hesitancy, 2014–2016

Comparison in coding for contextual influences

For the category of “contextual influences,” we noted that the largest difference between our analysis and the Lane et al. analysis was in the percentage of responses classified under the sub-category of “influential leaders, gatekeepers, and vaccine lobbies” (6.8% in the prior analysis versus 2.7% in the current analysis) (Figure 3). The smallest difference in coding was for the responses classified under the sub-category of “communication and media environment” (approximately 4.6% of responses in this sub-category in both analyses). The “contextual influences” was the second most coded category for vaccine hesitancy in both analyses.

Figure 3.

Percentage of responses for the two quantitative content analyses, using the WHO/UNICEF Joint Reporting Form data on open-ended responses coded as the contextual influences for vaccine hesitancy, 2014–2016

Comparison in coding for vaccine- or vaccination- specific issues

As for the category of “vaccine- or vaccination-specific issues,” the largest difference in coding was in the sub-category of “risk/benefit (scientific evidence).” The percentage of coding for this category was between 23.7% in the prior analysis versus less than 1%, across the 3 years in the current analysis (Figure 4). The smallest difference in coding percentage in this category was for the responses classified under the sub-category of “reliability and/or source of vaccine” (1.3% responses in the prior analysis versus 1.5% responses in our analysis) for the 3 years.

Figure 4.

Percentage of responses for the two quantitative content analyses, using the WHO/UNICEF Joint Reporting Form data on open-ended responses coded as vaccine- or vaccination-specific influences for vaccine hesitancy, 2014–2016

The current analysis also classified higher proportion of responses as those that could not be categorized using VHM and used a more conservative approach in coding responses that were ambiguous or lacking context. In the prior analysis, 5.2% did not fit any of the VHM categories but the same was true for 12.2% of the responses in the current analysis (Figure 4).

DISCUSSION

We found consistent trends across the broader VHM categories related to vaccine hesitancy over the four years from 2014 to 2017. Individual- and group-level influences accounted for most vaccine hesitancy reasons. The cumulative reasons provided for vaccine hesitancy across the 194 member states suggest that there are a multitude of factors, influencing vaccination behaviors, which can inform underlying factors influencing vaccine hesitancy and facilitate the development of targeted intervention approaches.

Upon examining the comparability of using the VHM to classify the 2014–2016 JRF hesitancy data, we found many areas of agreement between the prior analysis by Lane et al. and our current analysis. The “individual and group influences,” such as lack of knowledge/awareness about vaccination services, beliefs/attitudes about health, and perceptions of vaccination risks and benefits were highlighted in both analyses. At the contextual level, both analyses showed agreement in the high frequency of hesitancy reasons linked to religious, cultural, and socioeconomic factors, as well as negative historical influences, the communication, and media environments. As for the “vaccine and vaccination-specific issues” category, both analyses showed a parallel pattern in the hesitancy responses related to the design of vaccination programs, the role of health-care professionals, and the lack of flexibility in the vaccination schedule.

When compared, the largest difference between the prior analysis and our analysis was in the responses we classified as part of the sub-category of “experience with past vaccination,” which Lane et al. classified under the sub-category of “risk/benefit (scientific evidence).” The reasons for these differences could be two-fold. First, in our reading, there is ambiguity in how the two sub-categories are described in the VHM. Moreover, most responses provided by the member states in the JRF were brief and only noted concerns about the side effects related to vaccines, without providing any specifics on the known scientific evidence about specific vaccines causing adverse events as suggested in the VHM description of “risk/benefit (scientific evidence).” Second, we also acknowledge that any other differences in comparability between the two analyses could result from the subjective interpretations in the coding process. Overall, we have further provided novel evidence of how the JRF data and VHM could be utilized through understanding the comparability of using VHM for categorizing vaccine hesitancy based on comparing our results with that from previous research by Lane et al.[10]

Our results taken together with findings from the prior analysis demonstrate the need to address the diverse behavioral and social factors that influence vaccination decisions and experiences at both the individual and contextual levels. Additionally, the current scientific literature strongly highlights a multitude of social, economic, and cultural factors, such as parental knowledge, education, socioeconomic status, religious beliefs, media environment, and quality of health systems that drive vaccine acceptance, delay, and refusal.[6,13-16] We found that the media environment coupled with influential leaders, gatekeepers, and vaccine lobbies were among the leading contextual influences reported by the member states; this calls for more effective strategies to address emerging and existing misinformation about vaccination services that often spread through social networks.[17,18] Furthermore, the diversity of responses provided as the reasons for vaccine hesitancy more broadly suggest a complex interplay of localized and context-specific drivers of vaccination demand that need to be addressed with more tailored approaches.[19,20]

Strengths and limitations

In the annual JRF data reported by WHO member states, the open-ended responses on the reasons for vaccine hesitancy reported were often very short, ambiguous and lacked contextual information. Consequently, it was difficult to interpret some of the responses. We were unable to classify over 10% of all responses based on VHM categorization owing to one or more of these reasons. Data quality issues in the JRF were compounded by the fact that only about a third of the reported hesitancy data were purportedly based on empirical assessments and the rigor and credibility of such assessments were unknown.[9,10] We acknowledge the independent coding for the current analysis for different years as a limitation; however, in our review all the codes for the four years together we found that the coding was largely consistent.

A major strength in using the VHM was that it gave us a comprehensive framework to categorize the diverse drivers of vaccination experiences and behaviors that may lead to vaccine hesitancy. However, we uncovered the overlapping descriptions of different sub-categories as the key limitation of VHM. For example, the sub-category on “influential leaders, gatekeepers, and vaccine lobbies” refers to “community leaders and influencers, including religious leaders” in some settings. This sub-category overlaps substantially with the sub-category on “religion/culture/gender/socio-economic” factors because they both refer to the potential negative influences of some religious leaders. Similarly, the “immunization as a social norm vs. not needed/harmful” sub-category is focused on how “vaccine acceptance or hesitancy is influenced by a peer group or social norms.” The interpretation of this sub-category may be conflated with the “anti-vaccine lobbies” sub-category.

Recommendations

The JRF provides a valuable and unique opportunity to collect annual global data, which could be used to help program planners, governments, and non-government organizations to understand emerging drivers of vaccination behaviors.[21] While the use of open-ended responses to capture reasons for hesitancy may have provided some details, the brevity of country-level responses suggests that pre-categorized response options of the commonly reported reasons would be a more efficient approach to improve data quality and reduce the time it takes to code these responses. A combination of multiple-choice and a few open-ended questions may yield more detailed vaccine hesitancy information. A structured format with prompts and follow-up instructions to elicit specific responses may also improve data quality. For instance, the JRF hesitancy module could first obtain information on the broader hesitancy reasons, possibly as a selection of one or more responses from pre-categorized options. This may be followed up by asking an open-ended question to gain a richer understanding of the reported issue(s). Follow-up questions could also elucidate if the hesitancy may be linked to a specific vaccine or all vaccines, population groups, or sub-groups for whom the hesitancy is predominant, and the regions of the country most affected by the hesitancy. More importantly, future JRF hesitancy modules should include a more detailed guidance on how to report the hesitancy data to improve data quality. Lastly, the results from our comparability analysis of the VHM should be considered in ongoing global efforts to standardize the measurement of behavioral and social drivers of vaccination.[22]

The need to understand the drivers of global vaccine hesitancy is perhaps more crucial than ever as the world gears up to introduce COVID-19 vaccines to bring the pandemic under control. The over-abundance of information (correct, mis-, or disinformation)[23] surrounding the COVID-19 pandemic has been characterized as part of the infodemic phenomenon that the world is experiencing. The current infodemic climate underscores the persistent complexities in how perceptions and attitudes toward vaccines and vaccinations are formed, and how such perceptions can be even more complex during health emergencies. Considerations should be given to modifying the JRF to collect snapshot data on country-level drivers of accepting or refusing new vaccines, which may be accomplished using ad-hoc modules with brief items. Using VHM to then classify drivers of vaccine hesitancy for new vaccine may help in vaccine introduction planning, preparedness and response through tailored communication and policies. Provision of such flexibility will help maximize the utility of the JRF to provide near real-time snapshots of vaccine-specific drivers of vaccination behaviors in the context of emerging health threats that require introduction of new vaccines.

Conclusion

Routine use of the JRF hesitancy data and VHM offers a feasible approach to getting annual global snapshots of the potential drivers of vaccination behaviors and experiences. The drivers of vaccine hesitancy reported in the JRF were diverse, but the trends revealed that the individual and group level influences, coupled with contextual factors are the major contributors. To realize its full potential, the JRF hesitancy module needs to be modified to improve the data quality, and its scope should be broadened with a shift toward a more comprehensive, standardized monitoring of the global demand for immunization services.

Click here for additional data file.