COVID-19 vaccine hesitancy in eight European countries: Prevalence, determinants, and heterogeneity.

We examine heterogeneity in COVID-19 vaccine hesitancy across eight European countries. We reveal striking differences across countries, ranging from 6.4% of adults in Spain to 61.8% in Bulgaria reporting being hesitant. We experimentally assess the effectiveness of different messages designed to reduce COVID-19 vaccine hesitancy. Receiving messages emphasizing either the medical benefits or the hedonistic benefits of vaccination significantly increases COVID-19 vaccination willingness in Germany, whereas highlighting privileges contingent on holding a vaccination certificate increases vaccination willingness in both Germany and the United Kingdom. No message has significant positive effects in any other country. Machine learning-based heterogeneity analyses reveal that treatment effects are smaller or even negative in settings marked by high conspiracy beliefs and low health literacy. In contrast, trust in government increases treatment effects in some groups. The heterogeneity in vaccine hesitancy and responses to different messages suggests that health authorities should avoid one-size-fits-all vaccination campaigns.

INTRODUCTION

Vaccination is a highly effective public health tool that has considerably reduced the global burden of infectious diseases (, ). Estimates suggest that mass immunization has substantially contributed to the 55% worldwide decline in the mortality of children less than 5 years of age since 1990 (). Between 2010 and 2015 alone, vaccination campaigns helped prevent 10 million deaths (). Similarly, vaccines against COVID-19 provide a vital instrument to help fight the pandemic, with both economic and health system advantages over nonpharmaceutical interventions.

However, vaccine hesitancy—the choice to delay or refuse available vaccines—poses a major obstacle to the effectiveness of ongoing COVID-19 immunization programs (–). Even before this pandemic, the World Health Organization declared vaccine hesitancy a “top 10 health threat” (). Resurgence in measles outbreaks and increased prevalence of other vaccine-preventable diseases illustrate the threat posed by low levels of vaccine acceptance (, ). A better understanding of vaccine hesitancy and its determinants is therefore crucial to help bring the COVID-19 pandemic under control. Some important insights can be drawn from previous studies that predate the pandemic. For example, one systematic review synthesized evidence from 470 primary studies on influenza vaccination. Here, predictors of vaccine hesitancy were found to include a perceived low risk posed by the disease, lack of health knowledge, misconceptions regarding the safety and efficacy of vaccines, and economic hardship, while the influence of sociodemographic variables (age and sex) remained more ambiguous (). Another systematic review of 28 studies found low socioeconomic status to be a key barrier to human papillomavirus vaccine uptake in Europe (). Furthermore, a systematic review of 20 studies examining the uptake of measles vaccines found that concerns about side effects, a perceived low risk of severe disease progression, mistrust toward health experts, and older age were associated with vaccine refusal, whereas the impact of education and income was inconsistent across studies (). Last, a systematic in-depth review of 27 qualitative studies examining parental views on routine childhood immunization identified four broad themes that explained reduced vaccination acceptance, namely, (i) general beliefs about (child) health that emphasize natural immunity; (ii) influence of social, cultural, and religious networks and communities (which can both reinforce or undermine vaccine acceptance); (iii) political views and attitudes, including distrust in political institutions, authorities, and experts; and (iv) access-related aspects such as financial constraints and general alienation from the health care system ().

A crucial question is whether the factors identified in studies of other vaccines can also explain hesitancy with regard to the COVID-19 vaccines and to what extent COVID-19 vaccine hesitancy is distinctive. Three potentially unique characteristics of the COVID-19 context deserve to be highlighted: First, the universal dominance of the virus in public discourse along with its enormous socioeconomic repercussions may increase individuals’ risk and threat perceptions. Second, multiple vaccines were developed and authorized at different times in different countries, which created confusion and might have contributed to vaccine hesitancy. Third, the COVID-19 vaccines were developed at an unprecedented speed, which may negatively affect the public’s trust in their quality and safety. Some recent studies have cited concerns about the fast development and testing of the vaccines as a key determinant of COVID-19 vaccine hesitancy (, ). At the same time, recent studies have highlighted important similarities between general and COVID-19–specific vaccine hesitancy. For example, mistrust toward national governments and health authorities (, –), fears about adverse side effects (, , ), and lower levels of education and income (, , , , ) were shown to lower the likelihood of COVID-19 vaccine take-up. In addition, recent studies found significant correlations between a measure of general attitudes toward vaccination—composed of subscales on confidence, complacency, convenience, risk calculation, and collective responsibility (referred to as the “5C model” of drivers of vaccine hesitancy) ()—and reported willingness to get vaccinated against COVID-19 (, ).

To promote vaccine acceptance across hesitant population segments, more effective public health campaigns are urgently needed for ensuring sufficient uptake of COVID-19 booster vaccinations and a potential future vaccine for specific variants. A better understanding of vaccine hesitancy is also crucial for promoting pandemic preparedness in the future. Experimental studies suggest that public health messaging and subtle information nudges can shape individuals’ health-related beliefs and decisions. For example, previous studies have shown how messages—which (i) contain simple reminders, (ii) present general facts about vaccination, (iii) highlight the reduced risk of severe illness, (iv) correct specific misinformation and conspiracy beliefs, or (v) emphasize the concept of herd immunity—can significantly increase general vaccine uptake (–). In light of this earlier behavioral research, similar communication strategies could be expected to effectively increase vaccine acceptance in the context of the COVID-19 pandemic. An emerging body of literature has tested the effectiveness of different communication strategies for addressing COVID-19 vaccine hesitancy. Two survey experiments conducted in the United Kingdom and the United States found that providing information about prosocial benefits and herd immunity effectively increased participants’ willingness to get vaccinated against COVID-19 (), whereas two other survey experiments from France and the United Kingdom reported null effects (, ). Four United States–based studies found that messages emphasizing the effectiveness and low risk of side effects associated with the COVID-19 vaccines significantly increased willingness to get vaccinated (, , ), while a similar United Kingdom–based study did not find any such effects (). A survey experiment conducted in the Netherlands found that debunking specific vaccination myths, such as “vaccination can cause autism,” resulted in more favorable attitudes toward COVID-19 vaccines ().

While evidence for the general effectiveness of different communication strategies is mixed, various messages highlighting altruistic motives (), personal health benefits (, , –), personal nonhealth benefits (), or societal benefits of mass vaccination (, ) significantly increased reported vaccine uptake in one or two countries each. However, evidence regarding the effectiveness of different messages across multiple countries remains scarce, and we lack an understanding of what explains the variability of results in previous studies.

To address these gaps in existing knowledge, we first identify the level of COVID-19 vaccine hesitancy across eight European countries, as well as similarities and differences in public perceptions of the four COVID-19 vaccines currently available in the European Union and the United Kingdom. Second, we use a mix of quantitative and qualitative analyses to understand the fears and concerns driving vaccine hesitancy. Third, using identically designed randomized controlled survey experiments across the eight countries, we test the effectiveness of several different messages in increasing the willingness to receive COVID-19 vaccines. Fourth, finding vast heterogeneity across countries, we seek to gain a more nuanced understanding of different responses to these messages through a machine learning analysis of heterogeneity and relate it to barriers already identified in the existing vaccine hesitancy literature.

RESULTS

Prevalence and determinants of hesitancy

A total of 10,122 unvaccinated participants across eight countries were recruited between 8 April and 2 July 2021. While our survey experiments were not implemented simultaneously in all eight countries, the time span of 3 months is similar to the data collection window of previous prominent survey experiments on vaccine hesitancy (). The prevalence of COVID-19 vaccine hesitancy disaggregated by participants’ gender is presented in Fig. 1 (for further details on respondents’ general attitudes toward COVID-19 vaccination, see figs. S12 to S18). These estimates take into account the COVID-19 vaccination rate at the time at which the surveys were fielded in each respective target country (see Table 1 and table S6 for eligibility criteria in each country) and can thus be interpreted as extrapolated population-level estimates of COVID-19 vaccine hesitancy.

Fig. 1.

COVID-19 vaccine hesitancy by gender.

Participants were considered vaccine hesitant if they reported being either unsure whether they would get vaccinated against COVID-19 or certain that they would not get vaccinated. Participants who had already received one or two vaccine shots were coded as not hesitant.

Table 1.

COVID-19 vaccination rates by country and age group.

The vaccination rates reported here are the averages for the time during which each survey was in the field. Data for the seven EU countries were drawn from https://vaccinetracker.ecdc.europa.eu/public/extensions/COVID-19/vaccine-tracker.html#uptake-tab, and data from the United Kingdom were drawn from https://coronavirus.data.gov.uk/details/vaccinations.

Country	Time of survey	First shot %	Both shots %	18–24 first	18–24 full	25–49 first	25–49 full	50–59 first	50–59 full	60–69 first	60–69 full	70–79 first	70–79 full	80+ first	80+ full
Germany	9 to 30 April 2021	25.3%	8%	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA
United Kingdom	17 June to 2 July 2021	83.0%	60.9%	39.1%	14.5%	65.32%	31.22%	86.65%	81.00%	91.19%	88.10%	94.90%	93.45%	95.00%	92.73%
Sweden	15 to 24 June 2021	56.8%	34.2%	11.5%	6.4%	31.4%	10.2%	79.9%	29.5%	88.4%	66.6%	95.0%	79.8%	94.4%	89.2%
Spain	15 to 21 June 2021	59.9%	36.5%	7.8%	5.4%	31.6%	12.2%	86.7%	53.4%	93.1%	29.9%	97.8%	96.1%	100.0%	100.0%
France	15 to 21 June 2021	60.2%	30.9%	37.8%	5.4%	46.5%	12.4%	66.6%	28.9%	76.2%	49.5%	88.6%	72.7%	79.1%	68.0%
Italy	15 to 21 June 2021	62.3%	31.4%	29.3%	8.8%	42.6%	15.6%	70.4%	29.2%	81.2%	41.2%	87.0%	47.0%	94.8%	87.6%
Bulgaria	15 to 21 June 2021	15.8%	13.4%	6.0%	4.6%	11.3%	9.6%	17.8%	15.4%	22.8%	19.6%	24.8%	21.1%	15.9%	13.2%
Poland	15 to 21 June 2021	52.5%	36.3%	33.5%	13.4%	41.3%	24.5%	54.7%	39.3%	65.9%	49.4%	79.3%	74.4%	61.0%	58.1%

The estimates point to considerable heterogeneity across countries. COVID-19 vaccine hesitancy was lowest in Spain, with 6.22% [95% confidence interval (CI), 5.02 to 7.42%] of women and 6.82% (95% CI, 5.55 to 8.10%) of men hesitant to receive the COVID-19 vaccine, compared with Bulgaria, showing the highest rates with 64.19% (95% CI, 62.51 to 67.20%) of women and 59.20% (95% CI, 56.19 to 62.50%) of men hesitant. In most countries, men were less skeptical of the COVID-19 vaccine than women.

Participants who indicated that they would only be willing to get vaccinated with some vaccines but not all were asked to specify which vaccines they would accept (see Fig. 2 and figs. S19 to S21). Across countries, vaccine-specific willingness to get vaccinated was highest for the BioNTech/Pfizer vaccine, ranging from 57.09% (95% CI, 51 to 63%) of respondents in Poland to 93.17% (95% CI, 92 to 95%) in Germany. While overall acceptance of the viral vector vaccines was consistently lower than it was for the mRNA vaccines, we did observe substantial differences in the perceptions surrounding the AstraZeneca vaccine. Notably, 31.92% (95% CI, 26 to 38%) of the conditionally willing respondents in the United Kingdom indicated that they would accept this vaccine, compared with only 2.6% (95% CI, 2 to 4%) in Germany.

Fig. 2.

Acceptance of different COVID-19 vaccines among unvaccinated participants with conditional willingness.

A summary of the sociodemographic characteristics of the unvaccinated participants who completed the full survey and participated in the survey experiment is provided in table S1, including a breakdown by target country. To understand the sample composition further, we provide census information on gender, age, and education (see table S2).

Findings from multivariate logistic regression analyses assessing demographic factors associated with vaccine hesitancy are summarized in Table 2. Women were more hesitant in five of the eight countries. However, they were not significantly more hesitant than men in Spain, Sweden, and Poland—suggesting that the results on a possible gender gap on COVID-19 vaccine hesitancy have important heterogeneity. In six of eight countries, vaccine hesitancy was significantly higher in older age groups [ranging from odds ratio (OR) = 1.12; 59% CI, 1.02 to 1.24; P < 0.01 in Poland to OR = 1.67; 95% CI, 1.48 to 1.89; P < 0.001 in Sweden and the United Kingdom). The trend was reversed in Germany, with a significant decrease in vaccine hesitancy among older age groups (OR = 0.80; 95% CI, 0.75 to 0.85; P < 0.001). These differences are most likely a result of age-based priority access to vaccinations as well as variation in the timing of our surveys. Specifically, the German survey was launched earlier, and only around 25% of the population had already been vaccinated (see Table 1), whereas the samples of unvaccinated participants in other countries may overrepresent vaccine-hesitant individuals in the older age groups as those willing had likely already received their vaccination at that point in time. This explanation is further supported by the differential representation of older participants in Germany compared with the other countries (above–65-year-old participants account for around 16% of the German sample, but for less than 2% in all other countries’ samples).

Table 2.

Determinants of vaccine hesitancy among those not yet vaccinated.

Coefficients are odds ratios (95% CI) based on logistic regressions. Standard errors are clustered at the regional level. Outcome is an indicator variable for vaccine hesitancy, which is equal to 1 if participants reported being either unsure whether they would get vaccinated or certain that they would not and 0 otherwise. We included the experimental group as an additional control to adjust for the possible impact of the information treatments messages on vaccine hesitancy (coefficient not shown here).

	Bulgaria	France	Germany	Italy	Poland	Spain	Sweden	United Kingdom
Female	1.37*(1.03–1.81)	1.44**(1.12–1.85)	1.29*(1.03–1.61)	1.43*(1.03–1.98)	1.24(0.97–1.58)	0.90(0.57–1.43)	1.04(0.69–1.58)	1.42**(1.09–1.86)
Age group	0.98(0.87–1.12)	1.18***(1.08–1.29)	0.80***(0.75–0.85)	1.52***(1.30–1.79)	1.12**(1.02–1.24)	1.48**(1.16–1.87)	1.67***(1.52–1.83)	1.67***(1.45–1.94)
Education	0.76*(0.60–0.96)	0.88(0.72–1.07)	0.69***(0.61–0.78)	0.74**(0.60–0.90)	0.91(0.74–1.12)	0.84(0.61–1.15)	0.80***(0.70–0.91)	0.75***(0.64–0.87)
Employment	0.83(0.58–1.19)	0.97(0.65–1.44)	1.05(0.92–1.21)	0.90(0.67–1.22)	1.21(0.92–1.58)	0.56**(0.38–0.83)	0.62***(0.50–0.78)	0.92(0.65–1.29)
N	1069	1101	2285	1087	1098	1102	1097	1205

*P < 0.05.

**P < 0.01.

***P < 0.001.

In five of eight countries, vaccine hesitancy was significantly associated with lower education. In Germany, for instance, the predicted probability of refusing the COVID-19 vaccine was 45.07% (95% CI, 39.95 to 50.20%; P < 0.001) for participants who had not completed secondary education, compared with 15.89% (95% CI, 12.98 to 18.79%; P < 0.001) for participants who held a university degree, all else equal. Participants who were employed reported significantly lower levels of COVID-19 vaccine hesitancy in Spain and in Sweden (OR = 0.56; 95% CI, 0.39 to 0.80; P < 0.001; and OR = 0.62; 95% CI, 0.45 to 0.86; P < 0.001, respectively), whereas respondents’ employment status did not significantly affect vaccine hesitancy in any other country. Surveys in all countries except for the United Kingdom have captured additional details on the impact of (i) general vaccine skepticism, (ii) perceived social norms regarding vaccination, (iii) perceived COVID-19 threat, (iv) COVID-19 risk group status, and (v) exposure to COVID-19 in individuals’ social environment on COVID-19 vaccine hesitancy. The findings from this analysis are presented in table S5.

Participants’ motives and reasons for their vaccine hesitancy, elicited from their free-text statements, can be grouped into five themes (see table S3). Fear of side effects constituted the most common theme, mentioned by 22% of vaccine-hesitant participants in Spain and up to 41% of vaccine-hesitant participants in Italy. While many participants listed side effects as a general concern, some responses offered stronger sentiments, referring to COVID-19 vaccines as “poison” and revealing a fear of experiencing lethal side effects. Some respondents were concerned about side effects linked to (i) preexisting medical conditions such as chronic diseases or allergies, (ii) certain characteristics of specific vaccines, including concerns about “genetic modification” introduced by mRNA vaccines or blood clot incidents associated with the AstraZeneca vaccine, and (iii) potential infertility or harm to an (unborn) child. We included additional quantitative analyses to examine whether pregnancy-related concerns might be a key correlate of the gender gap in vaccination willingness. However, with the exception of Germany, we did not find any empirical support for a higher gender gap between women and men in reproductive age (here defined as <35 years) as compared with women and men in older age groups (>45 years) (see fig. S22).

The lack of evidence regarding the long-term effects of the COVID-19 vaccines was listed as another major concern spurring vaccine hesitancy, cited by 17% of participants in Bulgaria and up to 44% of participants in France. Several respondents portrayed the current vaccination campaign as a large-scale human experiment (“We are all just guinea pigs”; see table S3) and expressed concerns about the speed with which the vaccines were developed. Relatedly, many respondents pointed to fears about detrimental middle- to long-term health impacts of the COVID-19 vaccines and the lack of scientific evidence on these long-term effects.

In addition, some participants cited low levels of trust in the quality and efficacy of COVID-19 vaccines as a key barrier to getting vaccinated (ranging from 6% of participants in the United Kingdom to 11% of participants in France), which was often linked to concerns about the vaccines’ potency against more recent variants of the coronavirus and uncertainty or concerns about whether and to what extent vaccinated individuals might still transmit the virus.

Another theme that emerged, listed by 7% of participants in Poland and by up to 14% of Swedish respondents, was the perception that COVID-19 does not represent a substantial health threat, thus rendering vaccination unnecessary. Here, many participants emphasized their own good health and argued that they were confident that their own immune system would be sufficiently capable of fending off the virus. Others reported a low perceived risk of contracting the virus due to limited social interactions or little mobility and referred to COVID-19 as simply “a flu” or even denied its existence altogether.

A final commonly cited barrier to vaccination was distrust toward the government, pharmaceutical companies, or “elites” in general, listed by 3% of participants in Poland and up to 12% of participants in Spain. More specifically, participants repeatedly suggested that profits for pharmaceutical companies were the primary purpose of the vaccination campaign, viewed the vaccination as a means of state control, and voiced concerns or fears about being experimented on, while some participants sympathized with conspiracy theories surrounding the vaccines or COVID-19.

Causal effects of messages

The outcomes of the survey experiment are presented separately for each country in Table 3 (and additionally in figs. S23 to S30). In Germany, three of four treatments significantly increased participants’ willingness to get vaccinated. Specifically, the odds of accepting the COVID-19 vaccine were 1.5 times higher for participants who were presented with the COVID-19 risk reduction message, relative to participants in the control group (OR = 1.46; 95% CI, 1.17 to 1.83; P = 0.001). Starting from a baseline acceptance level of 27% in the control group, the treatment effect thus corresponds to a six-percentage-point increase in respondents’ intention to vaccinate. Messages highlighting hedonistic benefits and the benefits of owning a vaccination passport were also associated with significantly higher odds of COVID-19 vaccination willingness: 1.43 (95% CI, 1.06 to 1.93; P = 0.020) and 1.44 (95% CI, 1.09 to 1.91; P = 0.010), respectively. Respondents exposed to the altruistic message were also somewhat more likely to indicate that they would accept a vaccination offer, but the difference to the control group was not statistically significant (OR = 1.31; 95% CI, 0.98 to 1.75; P = 0.063). Because of budget constraints, we excluded the altruistic message as the least effective treatment from the survey experiment in the seven remaining countries.

Table 3.

Treatment effects on COVID-19 vaccine willingness.

Coefficients are odds ratios (95% CI) based on logistic regressions. Standard errors are clustered at the regional level. All treatment arms compared to the control arm. All analyses control for stratification variables and region to account for potential variations in COVID-19 infection rates. The outcome is an indicator variable of vaccine willingness, which is equal to 1 if the respondents reported that they would definitely get vaccinated and 0 otherwise.

	Bulgaria	France	Germany	Italy	Poland	Spain	Sweden	United Kingdom
COVID-19 risk reduction	1.460.48–4.47	1.060.65–1.73	1.46**1.17–1.83	0.69*0.49–0.98	0.900.53–1.54	0.66**0.49–0.89	1.160.89–1.53	0.880.63–1.23
Vaccination certificate	1.580.67–3.73	0.870.46–1.63	1.44*1.09–1.91	0.970.71–1.34	0.970.61–1.54	0.970.96–1.36	0.920.66–1.27	1.51***1.11–2.05
Hedonistic benefits	1.250.44–3.56	1.450.86–2.45	1.43*1.06–1.93	0.780.52–1.16	0.960.60–1.53	1.000.63–1.59	0.860.65–1.13	1.190.77–1.86
Altruistic benefits	–	–	1.310.98–1.75	–	–	–	–	–
Control group mean	0.06	0.14	0.27	0.34	0.13	0.67	0.44	0.22
N	1069	1108	2323	1087	1104	1102	1097	1205

*P < 0.05.

**P < 0.01.

***P < 0.001.

There was vast heterogeneity across countries with regard to the impact of the three experimental messages on respondents’ willingness to get vaccinated. In the United Kingdom, the vaccination certificate message significantly increased the odds of intending to get vaccinated by 1.51 (95% CI, 1.11 to 2.05; P = 0.008) compared with the control group, corresponding to an increase from 22 to 28% of participants being willing to get vaccinated. In Bulgaria, Poland, France, Italy, and Sweden, none of the messages significantly improved participants’ reported vaccination intentions. Treatment effects even pointed in the opposite direction in some countries, thus revealing potential harmful effects of public messaging, although these effects were significant only in Spain and in Italy. While informing participants about the risk reduction effects associated with COVID-19 vaccines was found to have positive effects in Germany, the same message caused an adverse effect on vaccination intentions in two countries, leading to an eight-percentage-point drop in vaccination willingness relative to the control group in Spain (OR = 0.66; 95%, CI 0.49 to 0.89; P = 0.006) and to a six percentage-point drop in Italy (OR = 0.69; 95% CI, 0.49 to 0.98; P = 0.039).

We further report treatment effects for an alternative outcome specification, which additionally considers how many of the different COVID-19 vaccines a respondent would be willing to accept (AstraZeneca, Johnson & Johnson, BioNTech/Pfizer, and Moderna). The findings remain very similar (see table S4).

Heterogeneity in treatment effects

Heterogeneity in treatment effects both across and within countries was further assessed using a model-based recursive partitioning approach (, ), which included (i) individual socioeconomic characteristics: age, gender, education, and employment, and (ii) country-level characteristics: the level of misinformation/conspiracy beliefs held by the population, public trust in the government, the level of overall health literacy, and whether a curfew was in place at the time of the survey. We found evidence for substantial heterogeneity in the effect of all three messages, as illustrated by the model-based trees presented in figs. S31 to S33. Each split displayed in the trees indicates that the parameters of the treatment-outcome model are significantly different between the two respective subgroups (). An analysis of the observed splits and of the direction and size of treatment effects in the terminal nodes revealed that high levels of misinformation and conspiracy beliefs (to be found in the cases of Bulgaria, Poland, and Italy) were a key barrier to beneficial treatment effects for both the COVID-19 risk reduction and the hedonistic benefits message. For example, the subgroup of participants 45 years and older who lived in countries with a high prevalence of misinformation and a curfew in place at the time of the survey reported a 41% (OR = 0.59; 95% CI, 0.32 to 1.09; P = 0.092) lower likelihood of reporting being willing to get vaccinated after receiving the COVID-19 risk reduction message. The relevance of country-level misinformation was somewhat more ambiguous for the vaccination certificate message.

Further, our results revealed that high country-level trust in government (to be found in Sweden and Germany in this case) were associated with higher treatment effectiveness across the assessed messages. Specifically, the subgroup of less educated male participants living in countries where the population’s level of trust in government was high and where there was a low/medium level of misinformation as well as a curfew in place at the time of the survey were significantly more likely (OR = 1.80; 95% CI, 1.05 to 3.07; P = 0.032) to accept the COVID-19 vaccine after receiving the hedonistic benefits message, relative to the control group (see Table 4). An almost identical pattern emerged for the same subgroup of participants who were shown the vaccination certificate message; these participants were 94% (OR = 1.95; 95% CI, 1.15 to 3.29; P = 0.013) more likely to report being willing to get vaccinated against COVID-19 than the control group (see Table 4).

Table 4.

Model-based recursive partitioning approach: Heterogeneity in treatment effects on vaccine willingness.

CIs (95%) are Wald CIs. Shown are remaining nodes after pruning using Akaike information criterion.

Subgroup composition (node number)	Odds ratio (OR) (95% CI)	Probability of willingness to get vaccinated relative to control group	P	Sample size (final node)
Treatment: COVID-19 risk reduction
High misinformation, curfew, 18 to 44 years (4)	0.85[0.53;1.37]	−15.2%	0.498	280
High misinformation, curfew, 45 to 65+ years (5)	0.59*[0.32;1.09]	−41.0%	0.092	260
High misinformation, no curfew, primary or secondary education (7)	0.67[0.28;1.59]	−33.3%	0.360	161
High misinformation, no curfew, further and higher education (8)	1.28[0.81;2.01]	+27.6%	0.292	928
Low/medium misinformation, curfew, high trust in government, female (12)	1.35[0.92;1.97]	+34.8%	0.121	467
Low/medium misinformation, curfew, high trust in government, male, primary or secondary education (14)	1.59*[0.93;2.75]	+59.4%	0.093	343
Low/medium misinformation, curfew, high trust in government, male, further or education (15)	1.30[0.60;2.79]	+29.6%	0.506	108
Low/medium misinformation, curfew, low/medium trust in government (16)	1.18[0.73;1.90]	+17.7%	0.507	550
Low/medium misinformation, no curfew, high health literacy, high trust in government, 25 to 34 years (20)	0.93[0.48;1.82]	−6.8%	0.836	143
Low/medium misinformation, no curfew, high health literacy, high trust in government,18 to 24, 35 to 65+ years, no employment (22)	1.46[0.60;3.55]	+46.2%	0.402	100
Low/medium misinformation, no curfew, high health literacy, high trust in government,18 to 24, 35 to 65+ years, employment (23)	1.25[0.79;1.96]	+24.6	0.339	306
Low/medium misinformation, no curfew, high health literacy, low/medium trust in government, 18 to 24 years (25)	1.21[0.58;2.53]	+21.1%	0.612	115
Low/medium misinformation, no curfew, high health literacy, low/medium trust in government, 25 to 65+ years (26)	0.80[0.49;1.29]	−20.3%	0.358	487
Low/medium misinformation, no curfew, low health literacy, no employment (28)	0.49**[0.25;0.93]	−51.5%	0.030	151
Low/medium misinformation, no curfew, low health literacy, employment (29)	0.89[0.58;1.37]	−10.8%	0.601	386
Treatment: Hedonistic benefits
High misinformation, curfew, 18 to 54 years (4)	0.82[0.55;1.22]	−18.1%	0.331	415
High misinformation, curfew, 55 to 65+ years (5)	0.55[0.20;1.54]	−44.6%	0.259	119
High misinformation, no curfew, primary or secondary education (7)	0.72[0.31;1.69]	−27.7%	0.452	163
High misinformation, no curfew, further or higher education (8)	1.21[0.77;1.92]	+21.3%	0.410	918
Low misinformation, curfew, high trust in government, female (12)	1.42*[0.97;2.07]	+41.6%	0.071	465
Low misinformation, curfew, high trust in government, male, primary or secondary education (14)	1.80**[1.05;3.07]	+79.5%	0.032	345
Low misinformation, curfew, high trust in government, male, further or higher education (15)	0.55[0.24;1.22]	−45.4%	0.141	111
Low misinformation, curfew, low/medium trust in government, female (17)	1.47[0.66;3.28]	+46.6%	0.352	286
Low misinformation, curfew, low/medium trust in government, male (18)	1.44[0.80;2.58]	+44.1%	0.220	267
Low misinformation, no curfew, high health literacy, high trust in government, 25 to 34 years (22)	0.87[0.45;1.67]	−13.5%	0.665	147
Low misinformation, no curfew, high health literacy, high trust in government, 18 to 24 years, 35 to 65+ years (23)	0.85[0.57;1.27]	−15.3%	0.421	406
Low misinformation, no curfew, high health literacy, low trust in government, 18 to 24 years (25)	1.21[0.58;2.53]	+21.1%	0.612	115
Low misinformation, no curfew, high health literacy, low trust in government, 25 to 65+ years (26)	1.19[0.76;1.88]	+19.3%	0.446	486
Low misinformation, no curfew, low health literacy, 18 to 44 years (28)	1.14[0.74;1.78]	+14.3%	0.552	395
Low misinformation, no curfew, low health literacy, 45 to 65+ years (29)	0.83[0.89;2.21]	−17.0%	0.564	155
Treatment: Vaccination certificate
High misinformation, curfew, 18 to 44 years (4)	1.05[0.65;1.68]	+4.6%	0.851	281
High misinformation, curfew, 45 to 65+ years (5)	1.00[0.57;1.79]	+0.0%	0.973	255
High misinformation, no curfew, primary or secondary education (7)	1.20[0.56;2.61]	+20.4%	0.638	164
High misinformation, no curfew, further or higher education, no employment (9)	0.14*[0.02;1.19]	−86.0%	0.071	178
High misinformation, no curfew, further or higher education, employment (10)	1.33[0.81;2.19]	+33.4%	0.253	750
Low/medium misinformation, curfew, high trust in government, female (14)	1.31[0.90;1.90]	+30.6%	0.166	469
Low/medium misinformation, curfew, high trust in government, male, primary or secondary education (16)	1.95**[1.15;3.29]	+94.8%	0.013	358
Low/medium misinformation, curfew, high trust in government, male, further or higher education (17)	0.76[0.35;1.67]	−23.9%	0.495	109
Low/medium misinformation, curfew, low trust in government, no employment (19)	2.33*[0.95;5.76]	+133.3%	0.066	128
Low/medium misinformation, curfew, low trust in government, employment (20)	0.72[0.40;1.32]	−27.6%	0.294	421
Low/medium misinformation, no curfew, high health literacy, high trust in government, 25 to 34 years (25)	0.45**[0.23;0.84]	−56.1%	0.013	153
Low/medium misinformation, no curfew, high health literacy, high trust in government, 18 to 24 and 35 to 44 years (26)	1.80**[1.05;3.10]	+80.1%	0.034	213
Low/medium misinformation, no curfew, high health literacy, high trust in government, 45 to 65+ years (27)	0.78[0.42;1.47]	−21.8%	0.444	186
Low/medium misinformation, no curfew, high health literacy, low/medium trust in government, 18 to 24 years (29)	1.44[0.68;3.01]	+43.5%	0.339	114
Low/medium misinformation, no curfew, high health literacy, low/medium trust in government, 25 to 65+ years (30)	1.44[0.93;2.25]	+44.2%	0.106	482
Low/medium misinformation, no curfew, low health literacy, 18 to 44 years (32)	0.99[0.64;1.53]	−1.2%	0.958	388
Low/medium misinformation, no curfew, low health literacy, 45 to 65+ years (33)	1.10[0.58;2.09]	+10.1%	0.768	156

*P < 0.10 (significance level set at 10 as the minimum size per group was set to 100).

**P < 0.05.

We further found that lower levels of health literacy in a country were associated with no or reduced treatment effects across messages. In contrast, there was no consistent pattern in terms of how the existence of a curfew affected treatment effects for any of the three messages (see figs. S31 to S33). Last, we found some heterogeneity along participants’ socioeconomic characteristics. Across messages, treatment effects were tentatively lower among older participants (see figs. S31 to S33). Further, we found that employment status—paired with low levels of country-level health literacy—significantly altered the effectiveness of the COVID-19 risk reduction message. Specifically, our results reveal that respondents with no employment who lived in settings with little misinformation exposure but low levels of health literacy were 52% (OR = 0.49; 95% CI, 0.25 to 0.93; P = 0.030) less likely to be willing to get vaccinated, relative to the control group (see Table 4).

DISCUSSION

Vaccines offer the most cost-effective instrument in combating the COVID-19 pandemic. This study set out to determine the prevalence and determinants of COVID-19 vaccine hesitancy in Bulgaria, France, Germany, Italy, Poland, Spain, Sweden, and the United Kingdom. Even across countries that are geographically close and culturally similar, we documented substantial heterogeneity in the prevalence of stated COVID-19 vaccine hesitancy, ranging from more than half of the population being unwilling to get vaccinated in Bulgaria, to one in every 20 persons in Spain. Our findings corroborate the cross-country variance in vaccine hesitancy rates reported by previous studies (, , ) but add new insights on the correlates of vaccine hesitancy and include Bulgaria and Poland, countries that have, to date, been less investigated.

We also found profound differences in participants’ perceptions of different COVID-19 vaccines. While acceptance was generally higher for the mRNA vaccines by BioNTech/Pfizer and Moderna, the AstraZeneca vaccine was marked by notably divergent levels of acceptance, with a third of British participants reporting high levels of trust in this United Kingdom–produced vaccine, compared to less than 5% of German and Italian participants. These insights motivate an important policy lesson: COVID-19 vaccination and booster campaigns will likely be more effective if citizens can actively decide which kind of vaccine they will receive. A previous study in Germany revealed that allowing participants to choose between vaccine types decreased the refusal rate from 42 to 6% ().

Despite considerable cross-country variation in the level of vaccine hesitancy, some key determinants were consistent across countries. In five of the eight countries, women appeared to be more hesitant toward the COVID-19 vaccine than men, which corroborates the findings of previous studies—both with regard not only to COVID-19 vaccine hesitancy (, , , , , , , ) but also to vaccine hesitancy in general and as it extends beyond the pandemic (, , ). Explanations for this pattern remain speculative, but the gender gap may be linked to a lower mortality risk for female patients with COVID-19 (), to greater concerns about unforeseen negative side effects (such as infertility) (), to broader gender differences in risk-taking (), or to more general gender disparities in health care access and patient-provider relationships (). In addition, similar to previous studies, we found that in some countries, vaccine hesitancy declined with higher education and employment (, , ). An important policy implication that emerges from these findings is the need for a more intensified outreach to groups with a higher probability of refusing the COVID-19 vaccine, including women and socially disadvantaged population segments.

Innovatively, our study sheds light on the underlying motives behind participants’ COVID-19 vaccine hesitancy. In line with previous research (, , , ), fear of side effects, fast development and approval, and distrust in the effectiveness and quality of the COVID-19 vaccines were cited as major driving factors of vaccine hesitancy. In light of these obstacles to vaccine acceptance, public health messages that focus on the rigorous and extensive testing of the vaccines currently available as well as on the prevalence, prevention, and treatment of possible side effects could prove most effective in addressing public concerns and fears directed at the COVID-19 vaccines. Relatedly, a recent experimental study found that presenting transparent information about the possible side effects of COVID-19 vaccines, compared with presenting only vague information, both reduced respondents’ vaccine skepticism in the short term and alleviated sympathies for conspiracy beliefs and mistrust in health authorities in the longer term (). In support of this, a previous survey experiment in the United States found that participants’ vaccine acceptance could be increased significantly by providing information on the safety of COVID-19 vaccines (). Building on this finding, future studies should explore the effectiveness of similar messages in other countries. Furthermore, our study found that some participants considered COVID-19 vaccination to be unnecessary. This result might be related to a substantial decline in infection cases in most European countries at the time participants were surveyed, resulting in a lower perceived threat posed by the pandemic. To address this aspect of vaccine hesitancy, public health messaging that emphasizes the salience of the disease threat—including the risk and prevalence of “long-COVID” symptoms—may help increase vaccination acceptance (, ).

Crucially, our randomized controlled survey experiments revealed that none of the tested messages consistently increased COVID-19 vaccine acceptance across all European countries. The message that significantly improved vaccination acceptance in Germany and the United Kingdom, highlighting the individual benefits and leisure privileges of having a vaccination certificate, did not work elsewhere. Even more alarmingly, the message highlighting the COVID-19 risk reduction associated with vaccination, which improved vaccination willingness in Germany, not only did not work in most other countries but even had negative effects in Italy and Spain. Health authorities and governments are urged to learn from other countries’ interventions that have been found to work, but to pretest them in their own country before adopting them on a large scale.

Our study is also the first to apply a machine learning approach to gain a more nuanced understanding of the sociodemographic characteristics and country-level factors behind the heterogeneity in the effectiveness of different messaging strategies to increase COVID-19 vaccine uptake. We find that our messages—particularly those emphasizing the medical and hedonistic benefits of COVID-19 vaccination—were considerably less effective in countries marked by high levels of misinformation and conspiracy beliefs. They may even backfire. Campaigns that rely solely on subtle, information-based nudges will likely have a limited power to change people’s COVID-19 vaccination decisions in these contexts. In these populations, interventions that feature stronger behavioral stimuli—including (i) restrictions for nonvaccinated individuals (as presented in our vaccination certificate message), (ii) provision of monetary or nonmonetary incentives for those who get vaccinated (), and (iii) (ultima ratio) vaccine mandates ()—could prove to be more promising strategies.

In addition, trust in government was identified as an important determinant of treatment effectiveness. A possible interpretation of this finding builds on previous evidence that points to a strong correlation between governmental trust and generalized trust (), which could translate into a higher inclination among certain subgroups to perceive our messages as trustworthy. A more general insight may be that the effectiveness of public health messages depends on the perceived trustworthiness of their sender, which may vary substantially across groups. It may therefore prove worthwhile to involve group-specific opinion leaders in current and future vaccination campaigns.

Our findings, moreover, suggest that the effectiveness of all three messages was hampered by low levels of health literacy in the population. As a consequence, it may be necessary to explain the medical risks and benefits of vaccination in a more accessible way. In the longer run, vaccination campaigns should be integrated into more holistic and extensive health education programs. Last, treatment effectiveness varied along some socioeconomic characteristics. Notably, both the hedonistic benefits and vaccination certificate message had a significantly positive effect on the willingness to get vaccinated among men with lower education. Specific outreach campaigns to groups with this profile are therefore particularly warranted. Considering that vaccination rates generally tend to be lower among less educated subpopulations, policy makers might find the largest scope for improvement here.

While our analysis may offer some important insights about the predictors of heterogeneity, future research should aim for a more nuanced understanding of the factors that contribute to a positive or negative reaction to different messages promoting COVID-19 vaccination. Specifically, hierarchical modeling approaches could be a powerful analytical tool to systematically analyze factors such as socioeconomic characteristics on the individual level and exploit regional variation within countries to examine factors such as COVID-19 subnational infection and vaccination rates or trust in local health authorities. Hierarchical modeling would also allow examining interactions between the individual, regional, and, possibly, national level (). Apart from this, future research should seek to capitalize on individual-level variation in the country-level factors assessed here (institutional trust, health literacy, and conspiracy beliefs) to gain a more fine-grained understanding of their impact on individuals’ reactions and responsiveness to different themes in public health messaging.

A limitation of our study is the reliance on participants’ self-reports with regard to their vaccination intentions, which may deviate, in some cases, from their actual vaccination behavior (). In addition, we cannot claim full representativeness of the sample in view of the quota-based sampling and selecting participants contingent on their vaccination status. Moreover, the dynamic and heterogeneous progress of nationwide vaccination campaigns affected the sample composition and representativeness in each target country differently, particularly among older age groups. Findings from our survey can therefore not be generalized to the general population of each country. Linked to this, the timing of surveys in each country might have interacted with the effects of the different messages. For example, it is possible that the risk message was more effective in Germany than in other countries simply because citizens—at this earlier point in time—were less exposed to additional, and potentially contradictory, vaccine-related information. In addition, the fielding of the German survey coincided with a phase of increasing infection numbers and strict contact and mobility restrictions, which might have caused participants to react more strongly to the treatment that emphasized the role of COVID-19 vaccines in facilitating a return to normality. In contrast, the surveys in the other seven countries could only be launched 2 months later when transitioning from the end of the second (or third) wave to a phase of low case numbers and an easing of lockdown measures, which may have attenuated the perceived need for getting vaccinated (see figs. S5 to S8 and table S7). We urge future research into an analysis of the potential moderating impact of these factors, for example, by using controlled laboratory experiments or by meta-analyzing pooled findings across survey experiments to examine temporal and cross-region variation in effect sizes.

Further, because of budget constraints, we prioritized broad geographical coverage across countries over statistical power to detect very small differences within a single country, and it is thus possible that potential small differences of around two to three percentage points in participants’ willingness to get vaccinated were not detected. Last, guided by our finding that the altruism message had the smallest treatment effect in the German sample, we excluded this message from the survey experiments implemented in Spain, Italy, France, Poland, Bulgaria, Sweden, and the United Kingdom. However, in view of the considerable cross-country heterogeneity in treatment effects, it is possible that the message would have been received differently in these countries and might have increased the willingness to get vaccinated among survey respondents there.

Despite these shortcomings, this study offers fresh insights into our understanding of COVID-19 vaccine hesitancy and public health communication strategies to overcome it in an equitable manner across Europe. Our analysis sheds light on substantial differences in the prevalence and nature of vaccine hesitancy across European countries, as well as in the response to different messaging strategies. Beyond these empirical insights, our findings suggest that future research and public health interventions should capitalize on the growing toolbox of powerful and data-rich methodologies to help shift the focus from investigating average treatment effects to studying heterogeneous individual treatment responses, thus applying a personalized medicine lens to behavioral interventions and public health programming (, ). The European Commission is currently advocating for a coordinated vaccination strategy and calling on EU member states to “share best practices on effective ways to address vaccine hesitancy” (). While sharing information about experience with different communication strategies will surely be beneficial, our findings seem to suggest that governments and health officials, both at the European and country level, should avoid relying on a “one-size-fits-all” approach. Instead, vaccination campaigns and communication strategies should be carefully tailored around each country’s target population and consider its specific concerns and psychological barriers, as well as education and employment status. In view of COVID-19’s global death toll of almost 6 million at the time of writing, emerging new variants such as Omicron, and the pandemic’s devastating economic and social repercussions, promoting population-wide confidence in COVID-19 vaccines has become more important than ever. Carefully designed and target group–specific public health communication strategies are urgently needed, to promote equitable access to vaccines, to prevent future infection waves, and to build resilience for future pandemics.

MATERIALS AND METHODS

Study sample

We conducted an online survey experiment in eight European countries: Bulgaria, France, Germany, Italy, Poland, Spain, Sweden, and the United Kingdom. These countries were selected to cover all major regions in Europe (north, west, east, south, and center) and include the states with the largest population size. The German survey was fielded in April 2021; the remaining surveys were carried out in June 2021. In each country, we recruited respondents 18 years and older from panels maintained by the survey company Respondi. We sampled participants based on quotas that were matched to the official census of each country with regard to (i) gender, (ii) age, (iii) education, and (iv) geographic or political-administrative subdivision. On the first page of the online survey, we informed participants of the study’s purpose and data protection regulations and reminded them of their right to decline or withdraw from participation at any time. Those who elected to proceed received a voucher worth 3 to 5 euros, distributed through the survey company, for participating in the survey. We screened out respondents who had already received one or two COVID-19 vaccine shots because, for our purposes, it was most important to collect information on the intentions and perspectives of individuals who had not yet been vaccinated. We report information on eligibility criteria for COVID-19 vaccination in each of the eight countries to provide further details on the sample composition of the unvaccinated population at the time the surveys were launched (see table S6). Notably, all countries gave prioritized access to people if they were older, bore an increased risk for severe illness from COVID-19, or if they were working in essential services (e.g., health care workers, teachers, police officers). France, Germany, and Sweden assigned additional priority to vulnerable groups such as homeless people, refugees, or people with mental illnesses. While most countries had extended eligibility for COVID-19 vaccination to all adults by the time the survey was launched, Germany and Spain had not yet given access to age groups below 70 and 40 years, respectively.

Survey experiment

As part of the online survey, we implemented a randomized controlled experiment in each target country by randomly assigning participants to a control group or one of the three messages, which according to prior studies should increase their willingness to get vaccinated against COVID-19 (, , ). The messages in the experimental conditions were as follows: (i) COVID-19 risk reduction: information about the efficacy of different COVID-19 vaccines, specifically highlighting the effectiveness of vaccines to prevent COVID-19–related deaths and severe disease progressions among vaccinated individuals, relative to unvaccinated individuals; (ii) vaccination certificate: information about exclusive benefits for the vaccinated, including access to travel and leisure activities contingent on providing proof of vaccination in the form of a COVID-19 vaccination certificate; (iii) hedonistic benefits: information about the prospects of a full restoration of public life and a return to normality, including a wide range of leisure activities (restaurants, theatres, bars, sports, etc.), after population-wide vaccination; (iv) (in Germany only) altruistic benefits: information portraying vaccination as a prosocial, altruistic act, which could help protect groups for whom COVID-19 vaccines had not yet been approved in Spring 2021, such as pregnant women and children. All messages were translated from English into each country’s national language and piloted before launching the survey. Moving beyond previous studies () and building on previous research suggesting greater effectiveness of visual information (), the text-based messages were combined with images (see the Supplementary Materials, figs. S1 to S4). We first carried out the survey experiment with a larger sample size in Germany, which allowed us to test the effectiveness of the four different messages. On the basis of these findings, the three most effective messages were selected for testing in the remaining seven countries.

All participants (including those assigned to the control group) were first provided with general information about the COVID-19 vaccines available in their respective countries at the time of the survey. Subsequently, participants were randomized into one of four (in the case of Germany: five) groups, using stratification to ensure equal distribution across the treatments by participants’ gender, age, and education level: respondents either received no message (control group) or one of the three (in the case of Germany: four) different messages. To nudge participants toward taking sufficient time to read through and process the presented information, the survey design forced respondents in all treatment groups to spend 20 s on the page with the respective message before they could progress to the next page of the survey.

Outcomes

The study’s primary outcome was participants’ intention to get vaccinated against COVID-19, elicited by means of a question asking participants whether they would get vaccinated if they were given the opportunity in the following week. Participants were able to select one of four response options to indicate whether they (i) would definitely get vaccinated, (ii) would only get vaccinated if offered certain vaccine types, (iii) were still undecided, or (iv) would definitely not get vaccinated. Participants who selected the second response option were prompted to indicate the different vaccines with which they were willing to get vaccinated, based on a list of vaccines that were approved and available in their respective country at the time of the survey. These included the mRNA vaccines BioNTech-Pfizer and Moderna and the nonreplicative viral vector vaccines AstraZeneca and Johnson & Johnson/Janssen (the latter was excluded in the United Kingdom, where it was not yet available at the time the survey was launched). Participants who indicated that they were unsure or unwilling to get vaccinated were required to specify reasons in their own words in an open text field. Participants who indicated that they had gained natural immunity through a COVID-19 infection were not defined as vaccine hesitant.

To estimate the prevalence of vaccine hesitancy, we created an indicator variable of COVID-19 vaccine hesitancy, which was coded 1 for respondents who were unsure or unwilling to take the COVID-19 vaccine, and 0 otherwise. To obtain a sufficiently large sample size, we estimated vaccine hesitancy based on data from all experimental groups, including those who received a treatment. Given that the messages were intended to increase vaccine intention, this approach yielded, if anything, a conservative estimate of vaccine hesitancy rates in the different countries. For the respective prevalence estimates of vaccine hesitancy, we considered the COVID-19 vaccination rate at the point in time at which the surveys were fielded in each respective target country (see Table 1). The population-level estimate of COVID-19 vaccine hesitancy was then calculated as the product of the survey-based share of vaccine-hesitant participants and the rate of yet unvaccinated individuals in each respective country. By means of illustration, in the United Kingdom, 17% of the adult population were not yet vaccinated against COVID-19 when the online survey launched, which was multiplied by the share of female respondents who reported hesitancy toward the COVID-19 vaccine—59%, which yielded a population-based estimate of COVID-19 vaccine hesitancy of about 10% (0.17 × 0.59) of women in the United Kingdom.

To evaluate the effectiveness of the experimental treatments, we focused on unconditional vaccine willingness, measured by an indicator variable coded 1 if the respondents reported that they would definitely get vaccinated, and 0 otherwise. Our outcome measure deviates from the specification outlined in our preanalysis plan. We had originally planned to construct an additive variable that discretely increases with the number of vaccine types that a respondent is willing to accept. In the main analysis, we have opted for a dichotomous outcome variable instead to (i) align our outcome with the measures of similar previous studies (, ) and (ii) account for the fact that younger age groups became ineligible for the AstraZeneca vaccine in several European countries by the time that our surveys launched, which thus partly predefined the choice between mRNA- and vector-based vaccines. We report estimates based on the additive outcome specification in the Supplementary Materials (see table S4).

Sociodemographic determinants of COVID-19 vaccine hesitancy

We focus on sociodemographic determinants of vaccine hesitancy: gender (male, female, and nonbinary), age (grouped into 18 to 24, 25 to 34, 35 to 44, 45 to 54, 55 to 64, and 65+ years of age), education (i.e., primary education, secondary education, higher education including A-levels, and university degrees), and employment status.

Data analyses

The association between sociodemographic factors and COVID-19 vaccine hesitancy was analyzed separately for each country using a logistic regression model. For ease of interpretation, we also calculated predicted probabilities of vaccine hesitancy for different sociodemographic profiles. Participants’ free-text statements explaining their reluctance or refusal to accept the COVID-19 vaccine were translated into English (except for the survey data collected in the United Kingdom) and arranged into overarching themes based on thematic analysis. The themes were coded independently by three authors (J.I.S., H.P., and H.S.). A subset was discussed jointly to assess intercoder agreement and consistency in the interpretation of statements.

The power calculations for the randomized controlled experiment assumed a minimum detectable effect size of OR = 1.57, based on a recent comparable survey experiment implemented in the United Kingdom (). With an alpha level of 0.05, a sample size of approximately 260 participants for each study arm would ensure statistical power of 80%. We therefore followed a target sample size of 1040 participants per country (for three treatment arms and one control arm). We substantially increased the sample size for the German experiment to ensure sufficient statistical power to allow for the testing of four distinct messages. We defined participants’ willingness to get vaccinated against COVID-19 as the outcome variable and estimated a logistic regression model including stratification variables (gender, age, and education) and subnational regions as controls to estimate the treatment effects.

Last, we examined heterogeneity in treatment effects by using a nonparametric model-based recursive partitioning approach (, ). Model-based recursive partitioning has several key strengths. First, it is able to detect heterogeneity in treatment effects even when statistical power in the experiment is limited. Second, it allows modeling nonlinear relationships (in our case: a logistic regression). Third, it automatically detects interactions between a range of included covariates and is therefore well suited for identifying the profiles of participants who are most likely to benefit from a given policy program or intervention. Fourth, it has advantages over purely data-driven methods such as regression trees in that it combines a theoretical approach with data-driven theory validation and refinement (, , ). On the basis of a classification tree, the model-based recursive partitioning procedure identifies links between the outcome variable (here: willingness to get vaccinated) and combinations of a range of prespecified covariates (). Specifically, the algorithm is set up to cluster respondents into subgroups with similar values of the outcome variable. Every split of the classification tree indicates that the parameters of the initial theory-based model are too heterogeneous to be explained by only one overarching model (). The subgroups identified by each split are referred to as “nodes.” The partitioning process is repeated until the model cannot detect any further associations between the partitioning variables, i.e., the prespecified covariates and variations in the outcome variable, or until the number of observations within one node falls below a prespecified threshold (here defined as no less than 100 individuals) (). Further technical information about the analysis is provided in the Supplementary Materials.

We examined heterogeneity along individual-level socioeconomic variables, namely, (i) gender, (ii) age, (iii) education, and (iv) employment. In addition, we explored whether heterogeneity was driven by certain country-level factors that we compiled from secondary data sources, including (i) COVID-19–related restrictions in place at the time at which the surveys were fielded (see table S7) (), (ii) trust in government (see fig. S9), (iii) health literacy (see fig. S10) (), and (iv) exposure to misinformation (captured by the share of adults who reported believing in selected conspiracy theories) (see fig. S11). More details on the secondary data sources, operationalization, and cross-country variance of these country-level variables are provided in the Supplementary Materials (see figs. S9 to S11 and notes).

Analyses were carried out in Stata SE 17.1 and R 4.1.0. The survey experiment and the corresponding preanalysis plan were registered on the Open Science Foundation platform. All data and code are accessible at https://osf.io/53zdk/.

Ethics review for human subject research

The study received approvals from the ethics committees of the medical faculty at the Technical University of Munich (TUM; IRB 227/20 S) and the ethics board at the University of Trento (Trento, IRB 2021-027); it also complies with the London School of Economics and Political Science (LSE) research ethics policy (LSE, REC 41495).