Can a COVID-19 vaccine live up to Americans' expectations? A conjoint analysis of how vaccine characteristics influence vaccination intentions.

OBJECTIVE: A vaccine for the novel coronavirus (COVID-19) could prove critical in establishing herd immunity. While past work has documented the prevalence and correlates of vaccine refusal, I assess how a less explored topic -- properties of vaccines themselves (e.g., national origin, efficacy, risk of side effects) -- might influence vaccination intentions. This information can help public health officials preempt differential intentions to vaccinate, and inform health communication campaigns that encourage vaccine uptake. RATIONALE: Previous research suggests that Americans should be more likely to intend to vaccinate if presented with a US-made vaccine that carries a low risk of minor side effects, is highly effective, is administered in just one dose, and has spent significant time in development.
METHODS: I administered a conjoint experiment (N = 5940 trials) in a demographically representative survey (N = 990) of US adults to assess how variation in vaccine properties influence self-reported public vaccination intentions.
RESULTS: I find that respondents prefer vaccines that are US-made, over 90% effective, and carry a less than 1% risk of minor side effects. This is potentially problematic, as some leading vaccine candidates are produced outside the US, and/or may be more likely to produce minor side effects than respondents would otherwise prefer. Worryingly, intended vaccine refusal rates exceed 30% for a vaccine meeting these optimal characteristics. Encouragingly, though, Americans show no clear preference for vaccines administered in one dose, or developed in under a year, and do not appear to draw a distinction between weakened viral vs. mRNA-based vaccines.
CONCLUSION: Americans' preferences for a novel coronavirus vaccine may be at odds with the vaccine that ultimately hits the market, posing both policy and health communication challenges for vaccination uptake.

Introduction

A safe and effective vaccine against the novel coronavirus (COVID-19) could prove critical in establishing herd immunity against it. According to current epidemiological estimates, between 40 and 60% of Americans must become immune to the disease – either through recovering from the disease, or via inoculation from a vaccine – to put the virus’ spread into decline (Kwok et al., 2020; Britton et al., 2020; Bartsch et al., 2020). As of this writing, there are dozens of potential vaccine candidates in development (Craven 2020).

However, a growing line of research suggests that between one-fifth to one-third of American adults plan to refuse a vaccine for COVID-19, once one becomes available (Thigpen and Funk 2020; Lunz-Tujilo and Motta 2020). Previous work has also found that Americans are considerably more likely to refuse vaccination if they distrust scientific authorities (e.g., Yaqub et al., 2014; Motta 2020), and has documented stark demographic differences in vaccine intention (e.g., racial and ethnic minority groups are less likely to intend to vaccinate, citing concerns about vaccine safety, efficacy, and affordability; Callaghan et al., 2021 unpublished).

Less is known, however, about how properties of vaccines themselves might influence vaccination intentions. With hundreds of potential candidates advancing through clinical trials, the vaccine or vaccines made available to the US public could vary along a number of different dimensions (e.g., country of origin, effectiveness, risk of minor side effects). Even in a time of global pandemic -- when we might expect Americans to be less scrupulous about the factors that influence their pursuit of a vaccine that could put the virus’ spread into decline and help bring about the resumption of “life as normal” -- I expect that even minor differences in how vaccines are developed and manufactured could have large increases on vaccination intentions.

I find that Americans have strong preferences for vaccine candidates produced in the US, that are highly (at least 90%) effective at protecting against infection, and that offer a low chance of experiencing minor side effects (1 in 100). This finding is potentially problematic, as leading vaccine candidates may miss some or all of what Americans deem to be optimal benchmarks. Worryingly, intended vaccine refusal rates exceed 30% for even a vaccine that meets all of the above specifications.

Still, there is some room for optimism. The results also suggest that Americans are generally indifferent regarding candidates’ time spent in development (with a slight preference for those taking more than a year to develop), doses (with a slight preference for a 1-dose vaccine), and antigen type. This news is encouraging, as a vaccine could be released for public use by the end of the year, and may take multiple doses to achieve effectiveness.

I conclude by discussing how the results of this research can help inform effective vaccine uptake health communication efforts; both concerning the novel coronavirus, and for future vaccines. I also discuss how these results can aid public health officials in better preempting intentions to vaccinate for different vaccines that may ultimately come to market.

How vaccine characteristics might influence intended uptake

Potential vaccine candidates for the novel coronavirus can vary along a number of different dimensions that may influence Americans’ willingness to get vaccinated. For example, candidates can vary in their country of origin -- with the US (e.g., Moderna), China (e.g., Wuhan Institute of Biological Products), and the UK (e.g., Oxford AstraZeneca) having made considerable progress in producing a viable vaccine candidate (Craven 2020).

Previous public opinion research documents widespread misinformation about the origins of COVID-19, and public suspicion about the possibility that the Chinese government had a role to play in “creating” or disseminating the virus (e.g., Motta et al., 2020; Cassese, Miller and Farhart 2020; Uscinski et al., 2020). Perhaps relatedly, initial survey evidence suggests that Americans are more likely to intend to vaccinate against COVID if a vaccine is produced in the US, compared to China (Morning Consult, 2020). More generally, recent market research studies have found that -- given the choice -- Americans tend to prefer consuming goods produced in the US compared to those produced elsewhere (e.g., McGoldrick et al., 2020). Consequently, I suspect that Americans will be more likely to prefer a vaccine originating from the US to those originating from the UK and (especially) China (H1a).

It is important to note that, shortly before this study was conducted, an untested Russian vaccine (Sputnik V) was approved for use in Russia. The announcement received substantial media attention in the US, much of which was negative (Cohen, 2021). Consequently, as will become apparent shortly (see Table 1 ), I also investigate opinions about a hypothetical vaccine originating from Russia. Consistent with the evidence reviewed above, and in the context of strong public outcry about the vaccine, I hypothesize that Americans will be less likely to prefer a hypothetical Russian vaccine than one originating from the US, UK, or China (H1b).

Table 1

Conjoint experimental design summary.

Attribute	Level 1	Level 2	Level 3	Level 4
Country of Origin	United States	United Kingdom	China	Russia
Effectiveness	50%	70%	90%	–
Risk of Minor Side Effects (e.g., fever, chills)	1 in 2	1 in 10	1 in 100	–
Doses Required	1	2	–	–
Vaccine Type	mRNA	Weakened virus	–	–
Months Spent in Development	9	12	15	–

Note. Summary of Conjoint Experimental Procedure. Note that respondents are shown three possible combinations of the attributes listed above, in two successive trials (six total ratings).

Additionally, vaccine candidates will vary in their potential effectiveness at preventing people from contracting the virus. Although we cannot know candidates’ potential effectiveness until clinical trials conclude and are replicated by the FDA, we do know that the US has set a 50% minimum effectiveness threshold in order for vaccines to be disseminated (FDA, 2020), and that (encoruagingly) early late-stage clinical trial data suggest effectiveness of greater than 90% for leading vaccine candidates (e.g., Grady 2020). Previous research suggests Americans will be more likely to intend to vaccinate if they perceive greater health benefits from doing so (e.g., Nan and Kim 2014). Consequently, I hypothesize that Americans will prefer more-effective vaccine candidates to those that are less effective (H2).

Vaccines may also vary in their risk of producing minor side effects. Early clinical trials suggested, for example, that more than 1 in 2 individuals who received Oxford-Astra-Zeneca's mRNA vaccine experienced minor side effects like swelling at the injection site, chills, and/or fever (Folegatti et al., 2020). Whether or not these findings hold in later stage clinical trials is yet to be determined (e.g., vaccine makers may abandon antigen dosage levels that most strongly elicit minor side effects), but some medical experts have raised the possibility that leading coronavirus vaccine candidates may be “reactogenic” (Branswell 2020); i.e., eliciting perceptible, yet minor, manifestations of an immune response (Herve et al., 2019). Several studies have documented that perceived risks of experiencing side effects due to vaccinating -- even if those side effects are fairly minor, or inconsistent with scientific data on vaccine safety -- can have a powerful influence on decreasing vaccination intentions (e.g., Tucker Edmonds et al., 2011; Jolley and Douglas 2014; Nyhan and Reifler 2015; Callaghan et al., 2019). Consequently, I hypothesize that Americans will be less likely to prefer vaccines with higher risks of minor side effects (H3).

Potential COVID-19 vaccines will also vary in their time to development. Vaccine candidates, which typically take several years to become commercially available, are being developed at a historically rapid pace (Lurie et al., 2020), in part due to the consolidation of Stage 2 and 3 clinical trials via “Operation Warp Speed” (RDA, 2020). Some leading candidates (e.g., AstraZeneca) may be available for public consumption by December 2020 (Aripaka and Nadkar 2020). Still, while the pace of development is remarkable, many Americans are nevertheless concerned about the possibility that development might be “rushed; ” potentially compromising safety and efficacy for the sake of speed (Morning Consult, 2020). Consequently, I hypothesize that Americans may be more likely to prefer vaccines that spend a greater amount of time in development (H4).

Moreover, it is possible that a vaccine for COVID-19 will need to be administered in multiple doses to achieve full effectiveness. For example, early clinical trial data suggest that Moderna's mRNA vaccine candidate must be administered in two doses to stimulate a strong immune response (Jackson et al., 2020). Past research on other multi-dose vaccines suggests that Americans are less likely to pursue vaccination -- and may have more difficulty doing so -- when multiple doses are required to achieve full effectiveness (e.g., Nelson et al., 2009). As a result, I hypothesize that Americans will be less likely to prefer vaccines that require just one dose to achieve its projected effectiveness (H5).

Finally, vaccines will vary in the type of procedure by which they aim to generate an immune response. While some promising candidates are mRNA based (i.e., they contain the viral code necessary to create viral proteins, which the body can then identify and fight off; e.g., Moderna), others hope to simulate an immune response via “inactivated” viruses or viral proteins (e.g., Novavax). Still others feature a weakened or “attenuated” version of the virus (e.g., University of Melbourne, Sinovac; Callaway 2020). These differences have received substantial attention in the popular press – in part due to the idea that an effective mRNA vaccine would be the first of its kind.

Conceivably, Americans might be more reluctant to try vaccines that have never successfully been brought to market in the past (i.e., mRNA), or that feature a version of the virus that they may worry could get them sick (i.e., attenuated vector vaccines). However, as many Americans lack even basic knowledge about viral transmission and immunity (Miller, 1998; Winneg et al., 2018), it is unclear whether or not they might draw meaningful distinctions between these types of vaccines. Consequently, in lieu of a formal hypothesis, I propose the following research question: Does the procedure by which vaccines aim to produce an immune response influence Americans’ preference for some vaccines over others? (RQ1)

Methods

Data

Data for this study come from a demographically representative survey (N = 990) of US adults conducted on August 24–25, 2020. Respondents were recruited via Lucid Theorem's online opt-in sampling service, which uses quota sampling to ensure representativeness on nationally representative benchmarks for respondents' age, gender, race, educational attainment, household income, four-category Census region, and political party (see: Callaghan et al., 2019 for an example of U.S. vaccine hesitancy research using this platform). Data from Lucid have been shown to closely mirror known demographic benchmarks, and replicate well-studied experimental findings (Coppock and McClellan 2019). Please refer to Supplemental Table S1 for a comparison of data from this study to nationally representative demographic benchmarks.

Outcome measures

The primary outcome variable in this study is a measure of whether or not respondents intend to receive each of the vaccine candidates (labeled A-C in the first rating task, and D-F in the second) to which they were randomly assigned (more on this procedure below). I asked respondents, in a matrix grid placed directly below the conjoint experiment to “Please tell us how likely you would be to choose to receive any of the following vaccines.” Respondents could then indicate their intentions on a ten-point numeric scale with verbal labels at 1 “not likely at all” and 10 “very likely.”

Conjoint experimental design & analytical strategy

I test these expectations using a conjoint experimental design (Luce and Tukey 1964; Strezhnev et al., 2013; Hainmueller et al., 2014). Conjoint experiments are useful for testing hypotheses where the number of varied dimensions (“attributes”) and the different values they can take on (“levels”) would create a prohibitively large design. That is, if we were to divide up the sample and randomly assign just one treatment per respondent, the number of possible assignments would approach or even exceed the number of respondents participating in the survey (a “small cell” problem).

Conjoint experiments circumvent this issue by (1) asking respondents to rate multiple treatments simultaneously (and/or in repeated trials), (2) randomly assigning levels within each attribute across treatments and individuals, and (3) estimating the effect of each attribute on an outcome variable of interest; assuming random assignment to all other level combinations, and accounting for correlated errors within respondents (e.g., standard error clustering or multilevel modeling).

Although one limitation of conjoint experiments is that we often cannot estimate the effect of any one possible treatment -- i.e., specific combinations of levels across attributes -- the approach has been shown to produce valid treatment effect estimates at the attribute level (Hainmueller et al., 2014). In this case, we can determine which types of vaccines are the most effective at influencing uptake, even if we cannot say with certainty which particular combination of attributes is the most effective.

In this case, I designed a conjoint experiment that works by first presenting respondents with three different vaccine candidates, and then asks them to assess how likely they would be to receive each one. Vaccine candidates vary along six dimensions (attributes), corresponding to the hypotheses and research questions outlined above (see: column 1 in Table 1).

After finishing this first rating task, I asked respondents to complete the procedure a second time; thus, in total, each respondent provides six ratings of all six attributes. Most of these attributes can take on three possible values (again, informed by the above discussion), except for the number of doses required to achieve effectiveness (which takes on only two values), antigen type (which takes on only two values), and the country of origin (which takes on four values). I assigned levels randomly both across attributes and respondents. Fig. 1 provides an image of the screen respondents saw while completing this survey, including a hypothetical rating task to which they could have been assigned.

Fig. 1

Illustration of the conjoint experiment's rating task. Note. The above image reflects a potential rating task that respondents could have been assigned while taking this survey. This image is a printed version of a survey preview screen, using Qualtrics survey software.

This setup implies that there are 432 different vaccine combinations that respondents could potentially be asked to rate (in conventional experimental parlance, a 4 × 3 × 3 × 3 × 2 × 2 design). Because I obtain six attribute ratings per individual (see above), in a sample of 990 US adults (see: Data), which implies that every level of most attributes will be rated at least 1980 times on average; with the exception that each level of the country attribute will be rated 1485 times on average, whereas the dosage and antigen attributes will be rated 2970 times on average. It further implies that every possible combination of attributes and levels will be rated at least 13 times (on average). As such, I have a sufficiently large sample to detect even small attribute-level effects, and can be reasonably assured that every possible combination will be rated.

I test the theoretical expectations outlined earlier (i.e., H1–H5, and RQ1) by constructing an OLS regression model that regresses respondents’ vaccination intentions on their assignments to each level of each attribute, for each rating task. These data are structured in “long” format, such that each row corresponds to one of six rating tasks completed by each respondent (i.e., the unit of observation is each rating task).

To account for the possibility of non-random variance across participants (e.g., the possibility that some rate all vaccines more favorably or negatively than others) I cluster standard errors at the respondent level; as is recommended in previous research (see: Hainmueller et al., 2014). Additionally, and pursuant with the assumptions outlined in Hainmueller and colleagues’ work (2014), I assume that ratings on the rating tasks are independent of one another (i.e., that presenting respondents with any one particular vaccine candidate does not influence how other candidates are evaluated). All analyses are conducted using Stata 15. Note also that I do not impute missing data, as there were very few missing observations across respondents and experimental trials (missingness levels <0.7% in all cases).

Before moving on, it is important to note this conjoint experiment features a rating based outcome variable, as opposed to a discrete choice outcome. A discrete-choice experiment asks respondents to choose between one of several products they might buy, candidates they might vote for, or, in this case, which vaccine they might choose to receive. Some argue that choice-based tasks are conceptually superior to both rating and ranking tasks; especially in economic contexts where individuals are more-commonly asked to select one product over another (Louviere et al. 2010).

However, in the specific case of studying COVID-19 vaccination intentions, a rating-based task is likely more conceptually appropriate than a discrete choice task. This may be the case for at least two reasons. First, although several vaccine candidates may become available for public consumption in the coming months, high demand and supply chain limitations could imply that Americans have little choice regarding which vaccine is available to them. In other words, COVID-19 vaccine uptake is not a straightforward economic or purchasing decision. Consequently, it is important to investigate not whether Americans seek out one vaccine over another, but to assess change in r their stated intentions to vaccine under different potential offerings (i.e., the vaccine that might be most available to them).

Second, it may be the case that people are willing to receive multiple vaccines with different properties; i.e., that they deem several potential candidates to be acceptable. This is a possibility we might miss if looking only at respondents’ choice of one vaccine over another.

Moreover, in addition to the conceptual appropriateness of a rating-based conjoint task, recent research has found that results from the discrete choice and ratings approaches often closely mirror one another (e.g., see Hainmueller et al., 2014). Consequently, it appears both conceptually and empirically appropriate to employ a rating-based task in this study.

Results

The conjoint experimental results are displayed in Fig. 2 . Each circle corresponds to a parameter estimate from the regression model, with 95% confidence intervals extending out from each one. Because all variables are scored to range from 0 to 1, parameter estimates can be interpreted as the percentage point change in vaccination intention, compared to an omitted reference group (denoted in the left-hand side of the Figure). Recognizing that percentage point changes do not provide a sense of baseline vaccination intention, however, I supplement these analyses by transforming parameter estimates into linear predictions (expressed as likelihood ratings), holding all other attributes constant (in text). Full model output is available in Table 2 (Column 1).

Fig. 2

The effects of hypothetical coronavirus vaccine characteristics on vaccination intentions. Note. OLS regression coefficients presented (circles) with 95% confidence intervals extending from each one. Standard errors are clustered at the respondent level. All independent variables are dichotomous, and the outcome variable is scored to range from 0 to 1. Coefficients are expressed (substantively) as percent change in vaccination intentions, relative to each denoted baseline on the left-hand side of the Figure.

Table 2

Models used to estimate values shown in Fig. 2, along with a summary of robustness checks.

	(1)	(2)	(3)
	OLS	Mixed	OLS
	Respondent Clustered SEs	Respondent Random Effects	Respondent Clustered SEs plus Demographic Controls
Origin: China (vs. US)	−0.21*(0.01)	−0.21*(0.01)	−0.21*(0.01)
Origin: Russia (vs. US)	−0.18*(0.02)	−0.19*(0.01)	−0.19*(0.01)
Origin: UK (vs. US)	−0.06*(0.01)	−0.06*(0.01)	−0.06*(0.01)
Effectiveness: 70% (vs. 50%)	0.06*(0.01)	0.06*(0.01)	0.06*(0.01)
Effectiveness: 90% (vs. 50%)	0.11*(0.01)	0.12*(0.01)	0.11*(0.01)
Side Effect: 1 in 10 (vs. 1 in 100)	−0.02(0.01)	−0.02*(0.01)	−0.01(0.01)
Side Effect: 1 in 2 (vs. 1 in 100)	−0.04*(0.01)	−0.05*(0.01)	−0.04*(0.01)
Type: Weakened Virus (vs. mRNA)	−0.00(0.01)	−0.00(0.01)	−0.01(0.01)
Doses: 2 (vs. 1)	−0.02*(0.01)	−0.02*(0.01)	−0.02*(0.01)
Dev. Time: 12 months (vs. 9)	−0.00(0.01)	0.01(0.01)	−0.00(0.01)
Dev. Time: 15 months (vs. 9)	0.02*(0.01)	0.03*(0.01)	0.02+(0.01)
Household Income	–	–	0.17*(0.03)
Age 25–44	–	–	−0.01(0.02)
Age 44–65	–	–	−0.12*(0.03)
Age 65+	–	–	−0.09*(0.03)
College Educ.	–	–	0.10*(0.02)
Black (Non-Hispanic)	–	–	−0.01(0.03)
Hispanic	–	–	0.01(0.03)
Female	–	–	−0.03*(0.02)
Intercept	0.55*(0.02)	0.55*(0.01)	0.54*(0.03)
Random Effects Components
Variance (Intercept)	–	0.05*(0.01)	–
Variance (Residual)	–	0.06 *(0.01)	–
N	5901	5901	5584
Likelihood Ratio Test	–	LR = −1039χ2(1) = 1911*	–
R2	0.08	–	0.17

*p < 0.05, + p < 0.10; two-tailed.

Note. OLS regression coefficients presented, with standard errors in parentheses. Column 1 presents the parameters used to construct the results presented in Fig. 2 (please refer to the figure for additional details). Column 2 presents a model analogous to that presented in Column 1, swapping the use of clustered standard errors for respondent-level random effects in a mixed linear model using the mixed command in Stata 15. Random effects components are presented at the bottom of the table. Finally, Column 3 presents a model analogous to that presented in Column 1, with the addition of demographic controls.

Identifying an optimal vaccine candidate

Before assessing each set of testable hypotheses, Fig. 2 presents a general picture of which vaccine attributes are most likely to maximize vaccine uptake. Generally speaking, respondents strongly prefer vaccines that are produced in the US, compared to those produced in the UK, China, or Russia. This preference is perhaps problematic, given that promising vaccine candidates may be produced by a UK company (e.g., AstraZeneca). Respondents also, perhaps somewhat self-evidently, prefer highly (90%) effective to less-effective vaccines, and those that carry the low risk (1 in 100) of minor side effects. This one, too, is potentially problematic, given that – as reviewed earlier – leading vaccine candidates suggest both lower levels of efficacy and higher prevalence of minor side effects than the public otherwise deems optimal.

Encouragingly, though, respondents say that they would only be slightly less likely to vaccinate if a vaccine must be administered in multiple doses, and are generally accepting of vaccines produced on rapid timetables (i.e., they only slightly more likely to prefer a vaccine taking more than a year to develop). Respondents are also neither more nor less likely to prefer mRNA-based vaccines to those that feature a weakened virus.

Consequently, the optimal vaccine candidate, per this conjoint analysis, would be one that is made in the US, 90% effective, carries a less than 1 in 100 chance of experiencing minor side effects, is administered in 1 dose, and spent 15 months in development. The predicted likelihood that respondents would choose to receive a vaccine with these properties is just over 68% (95% CI: 65, 71). Unfortunately, this analysis suggests that even a vaccine meeting respondents' expectations could have a refusal rate of over 30%. This result raises the possibility that Americans’ expectations for a COVID-19 vaccine may not match reality. While it is encouraging to see that multi-dose vaccines, and those developed on accelerated timeframes, do not necessarily discourage Americans from vaccinating, resistance to vaccines produced outside the US – and preferences for vaccines that are more-effective and less-risky than leading candidates – could potentially interfere with vaccine uptake if the vaccines that become available for public use in the US are (subjectively) “sub-optimal.”

Attribute-specific analyses

Having identified the most preferred vaccine characteristics from the conjoint setup, I next turn to each of the specific hypotheses outlined earlier. First, Fig. 2 presents strong support for H1a. On average, respondents report that they are more likely than not (58%) to intend to receive a US based vaccine. Vaccination intentions drop to 52% for a vaccine produced in the UK (β = −0.06), and just 37% for a vaccine produced in China (β = −0.21); both of which are statistically significant decreases (p < 0.05 in both cases).

Fig. 2 also offers some support for H1b. Respondents are significantly less likely (β = −0.18, p < 0.05) to intend to receive a Russian vaccine, compared to an American vaccine or UK vaccine. However, as evidenced by the overlapping confidence intervals in Fig. 2, this difference is not statistically discernible, and, in fact, is slightly smaller, than that of a vaccine produced in China.

The results also offer support for H2. On average, respondents are more likely than not (41%) to refuse a vaccine with 50% effectiveness (the minimum to be considered for public use in the US). Compared to a minimally-effective vaccine, Americans are significantly more likely to intend to receive a vaccine with 70% effectiveness (47%; β = 0.06, p < 0.05), and more likely than not to intend to receive a vaccine with 90% effectiveness (52%; β = 0.10, p < 0.05).

Likewise, and perhaps unsurprisingly, respondents are more likely to prefer vaccines that carry lower risks of minor side effects. On average, the likelihood that respondents choose to vaccinate – if a vaccine carries a 1 in 100 risk of minor side effects – is 48%. Encouragingly, vaccine intentions drop off only slightly (47%) for vaccines that carry a 1 in 10 risk of minor side effects; which does not represent a statistically significant decline (β = −0.02, p > 0.10). However, consistent with H3, intentions drop off significantly (45%; β = − 0.04, p < 0.05) for vaccines presenting a 1 in 2 risk of side effects.

Interestingly, the results provide only mixed evidence in support of H4 and H5. Whereas respondents are more likely than not (53%) to intend to vaccinate if presented with a vaccine developed in 9 months, a 12 month timetable offers only a modest increase in intention (54%) which fails to attain statistical significance (β = 0.00, p > 0.10); inconsistent with H4. However, and consistent with H4, respondents indicated significantly higher intentions to vaccinate (57%; β = 0.02, p < 0.05) when presented with candidates that spend more than a year (15 months) in development.

Additionally, I find limited evidence in support of H5. While respondents are significantly more likely to prefer a candidate that requires only a single dose (compared two doses) to achieve effectiveness (β = −0.02, p < 0.05), the size of this effect is substantively modest. The likelihood that Americans vaccinate drops from 56% for single dose vaccines, to 53% for two-dose vaccines.

Finally, with respect to RQ1, I observe no statistically significant differences in vaccination intentions across antigen types. Compared to mRNA-based vaccines, respondents are neither more nor less likely to prefer vaccines with more-conventional weakened virus antigen elements (β = 0.00, p > 0.10).

Robustness checks

The effects presented in Fig. 2 are robust to several alternate estimation strategies. First, while the models presented earlier account for the possibility of non-random variance across participants by clustering standard errors at the respondent level, some may prefer to estimate these adjustments hierarchically. Consequently, I re-estimated the results presented in Fig. 2 via a mixed-effects linear model with respondent-level random effects. The results are summarized in Column 2 in Table 2 (which includes the results presented visually in Fig. 2, for reference, in Column 1).

The results are highly similar across estimation strategies, with just one exception. Whereas a 1 in 10 chance of experiencing minor side effects was negative and failed to attain statistical significance (β = −0.02, p > 0.10) in Fig. 2, an effect of the same size (β = −0.02) did attain conventional levels of two-tailed significance in the random-effects model (p < 0.05). The two sets of results were otherwise substantively and statistically consistent.

Further, in addition to accounting for the possibiltiy of correlated errors at the respondent level, some might also ask whether or not there are systematic demographic differences between respondents concerning their intentions to vaccinate (e.g., see Callaghan et al., 2021 unpublished), and whether or not these differences might influence the effects depicted in Fig. 2. Table 2, Colum 3, presents the results of models which adds control variables denoting respondents’ age (dichotomous indicators of whether respondents are aged 25–44, 44–65, or older than 65), educational attainment (a dichotomous indicator of whether or not respondents completed college), race (indicators denoting whether respondents self-identify as Black [non-Hispanic] or Hispanic), gender (a dichotomous indicator of whether respondents self-identify as female), and annual household income (measured on a 24 point scale ranging from <$14,999 to $250,000 or more; and intervalized to range from 0 to 1).

The results again suggest a strong degree of consonance between the results presented in Fig. 2 and the results that include a series of demographic controls (Table 2, Column 2). Additionally, I find that some evidence of demographic differences in Americans’ intentions to vaccinate. Higher income (β = 0.17) and college-educated (β = 0.10) respondents tend to rate vaccine candidates significantly more favorably (p < 0.05, in both cases). Women (β = −0.03) and individuals older than 44 (β = −0.12 for the 45–65 age range; β = −0.09 for the 65 and older age range) are significantly less likely to do so (again, p < 0.05 in both cases). Collectively, these results suggest that while there are systematic demographic differences in how respondents rate vaccine candidates, these results do not alter the results presented in Fig. 2.

Finally, to further probe the possibility of heterogeneous treatment effects, I estimated four additional models where I interact each vaccine attribute with all demographic factors that exhibited a statistically significant effect on the vaccine rating outcomes in Table 2, Column 3. The models are presented in the Supplementary Materials in Table S2. There, I find little evidence of heterogeneous treatment effects. Of the 66 interactions tested across four models – stratified (i.e., estimated in separate models, to avoid overfitting concerns) by respondents' household income, age, educational attainment, and gender – I find little consistent evidence of moderated treatment effects. Just 10 of the 66 tests produced statistically significant effects (15% of tests conducted); the majority of which reflect differences attributable to respondents’ age.

Specifically, I find no evidence of moderated treatment effects by respondents’ household income levels or gender. However, I do note some evidence of moderated treatment effects by age category. Both individuals aged 44–65 and individuals aged 65 and older are significantly less likely to want to receive a vaccine originating from China (β = −0.20, p < 0.05 and β = −0.36, p < 0.05, respectively), Russia (β = −0.19, p < 0.05 and β = −0.32, p < 0.05, respectively), and the UK (β = −0.09, p < 0.05 and β = −0.11, p < 0.05, respectively), relative to the United States. I also find that individuals aged 65 and older are significantly more likely to prefer vaccines with 90% effectiveness or greater (β = 0.07, p < 0.05), and less likely to prefer vaccines with a 1 in 2 risk of experiencing minor side effects (β = −0.07, p < 0.05), while individuals aged 44–65 are significantly less likely to prefer vaccines that contained a weakened virus as an antigen (β = −0.05, p < 0.05). Finally, I find that college educated individuals are somewhat more likely to prefer a vaccine made in the UK to one made in the US (β = 0.05, p < 0.05).

Discussion

The conjoint experiment reported in this study suggests that several differences in COVID-19 vaccine properties can have a powerful impact on Americans' intentions to vaccinate. Specifically, Americans are significantly more likely to prefer vaccines developed in the US (compared to the UK, China, or Russia), pose a minimal risk of side effects, and that are highly effective at warding off infection. This is potentially problematic, as some leading vaccine candidates may fall short of achieving some or all of these benchmarks. Further, results from the conjoint experiment suggest that, even under these ideal conditions, Americans’ intentions to vaccinate may be suboptimal. However, and perhaps more-encouragingly, Americans are less discerning regarding the type of antigen featured in the vaccine (mRNA vs. a weakened virus), whether the vaccine is administered in one (vs. two) doses, and the amount of time the vaccine spent in development. All conjoint experimental are robust to several different modeling specifications, and are generally uniform across demographic subgroups.

The results observed in this study could have important implications for public health policy, and health communication about the vaccine. Concerning the former, public health officials can preempt the possibility of differential intentions to vaccinate for vaccines with different qualities. This may prove useful in avoiding vaccine shortages and accurately estimating the supply necessary to meet demand; both with respect to the novel coronavirus, and for vaccines that may share properties in common with those studied in the conjoint experiment.

Additionally, understanding how differences in vaccine characteristics influence intentions could help inform effective health communication encouraging vaccine uptake. If Americans prefer some vaccines to others (e.g., those that are American-made), highlighting that information in promotional messages could help increase public enthusiasm for a vaccine. If, on the other hand, Americans find some characteristics undesirable (e.g., those originating from outside the US), health communicators may consider making a special effort to assuage public concerns in that area (e.g., by emphasizing that foreign-made vaccines must meet US health guidelines to be administered in the US, and/or that manufacturing includes the involvement of US companies).

Lessons from this research may also prove useful in helping to inform public health policy and health communication beyond the ongoing pandemic. Although this research focuses specifically on a vaccine against just one illness, many of the vaccine properties studied in this paper apply to other vaccines currently in development (e.g., a novel Lyme disease vaccine candidate developed by a French company, and which may require multiple doses to be effective; see Motta 2020). Some might argue that assessing the effect of vaccine properties on uptake during a pandemic poses a “conservative” test of the effects documented in this paper; i.e., that the public may be willing to overlook differences that would ordinarily disincentivize vaccination for the sake of overcoming the pandemic. Consequently, insights gleaned from this research could be even more impactful when studying other vaccines in development.

Limitations

This study is not without some noteworthy limitations. First, this research studies self-reported vaccine intention ratings. The use of hypothetical self-reported data is, of course, partially due to the timing of this study; i.e., because data were collected prior to the availability of a novel vaccine. Still, it is important to caveat that – while individuals’ vaccination intentions have been shown to be strong predictors of whether or not they actually choose to get vaccinated (e.g., Godin et al., 2010; Gerend and Shepherd 2012) – the degree to which intentions are associated with uptake behavior for this specific vaccine is an open question.

Moreover, it is important to bear in mind that the conjoint analyses presented here feature a ratings-based task. As mentioned earlier, ratings-based tasks are appropriate in application to a COVID-19 vaccine, as people may not have the ability to choose which specific vaccine they ultimately receive – especially in the early phases of vaccine rollout – and may be receptive to receiving several different potential vaccines. Still, a notable limitation of this research is that the data do not allow for an investigation into whether or not people might choose one vaccine over another. When studying new and developing vaccines in the future, scholars should consider investigating whether the results of ratings-based tasks differ from tasks that ask respondents to choose one vaccine over another; relying on decision (vs. rating) based tasks when appropriate.

Additionally, this study was conducted at only a single moment in time, and amid global pandemic. Although recent research suggests that public confidence in vaccine safety (in general) tends to resemble opinion estimates observed prior to the pandemic (e.g., Lunz-Trujilo & Motta, 2020 unpublished), and that intentions to receive a coronavirus vaccine have held relatively steady throughout the pandemic (e.g., see Callaghan et al., 2021 unpublished), it is nevertheless possible that surges or declines in cases could impact public intentions to vaccinate. Vaccine intentions could also fluctuate over time due to factors unrelated to pandemic conditions. Consequently, I urge readers to consider the specific conditions (e.g., disease prevalence) under which this study was conducted when attempting to draw conclusions about it. Constant monitoring of vaccine opinion, either via the use of repeated cross-sectional surveys or (ideally) longitudinal surveys, could help detect shifts in intentions that might impact the insights gleaned from this study, and necessitate additional conjoint experimental research.

Given these limitations, future research may benefit from validating self-reported intention data with self-reported retrospective behavioral data, and by re-administering this study at several points in time. Researchers could, for example, re-contact participants in a study like the one presented in this paper (i.e., longitudinally), or by re-administering the experiment throughout the rollout of a new vaccine. If possible, efforts to then validate self-reported behaviors with actual evidence of vaccine uptake (e.g., providing respondents with cash incentives to provide photographic, redacted evidence of a receipt from a pharmacy; doctor's note; etc.) would further strengthen the validity of insights gleaned from this study.

Additionally, to better understand the relevance of this work for other types of vaccines, scholars ought to consider replicating this study with respect to other (voluntary) vaccines in development. Although I was not able to do so in the present study, researchers might consider conducting qualitative interviews with vaccine specialists to determine which vaccine attributes are worth further study, and/or conduct pre-test interviews with survey respondents to ascertain relevant vaccine preferences. While elements of the conjoint experimental setup would almost certainly need to be amended on a case-by-case basis, my hope is that the utility of the paradigm makes this an ideal approach for future work.

Conclusions

The work offers both encouraging and discouraging findings about Americans' willingness to get vaccinated against COVID-19. On the one hand, Americans' preferences regarding a potential vaccine's country of origin, effectiveness, and risk of minor side effects are at odds with some characteristics of leading vaccine candidates. This could potentially pose a challenge to vaccine uptake, as the results suggest that even a (subjectively) optimal vaccine – that is, one that is highly effective, low-risk, and U.S.-made – could produce vaccine refusal rates exceeding 30%. On the other hand, though, Americans have no clear preferences regarding how long vaccine candidates spend in development, the number of doses that must be administered to effectively ward off infection, or antigen type. This, more encouragingly, may imply some amount of receptivity to vaccines released before the end of the year in 2020, and administered in more than one dose.

Credit author statement

MM was responsible for all components of this manuscript, including: project, conceptualization; data collection, cleaning, and analysis; manuscript write-up; and all revisions.

Funding statement

I have no funding to report for this project.