Are vaccine lotteries worth the money?

This research evaluates the effects of the twelve statewide vaccine lottery schemes that were announced as of June 7, 2021 on state vaccination rates. We construct a dataset that matches information on the timing and location of these lotteries with daily, county-level data from the U.S. Centers for Disease Control (CDC) on the cumulative number of people who have received at least one dose of an emergency-authorized Covid-19 vaccine. We find that 10 of the 12 statewide lotteries studied (i.e., all but Arkansas and California) generated a positive, statistically significant, and economically meaningful impact on vaccine uptake after thirty days. On average, the cost per marginal vaccination across these programs was approximately $55.

Introduction

In spite of widespread availability of COVID-19 vaccines and a complete subsidy to make them free at the point of consumption, as of July 16, 2021, only 48% of eligible Americans have been fully vaccinated (Wagner et al., 2021). While vaccinations protect the recipients from mortality and morbidity, they also bear some risks to that individual, and some of the vaccination’s benefits are enjoyed by others as the population moves toward herd immunity. Accordingly, individual decisions may not alone reach socially efficient levels of vaccination (Robertson et al., 2020).

States have implemented various incentive programs to encourage hesitant Americans to get vaccinated, including small tokens (such as a free shot of alcohol), direct cash transfers (such as $50), and lottery schemes, where individuals who are vaccinated have a chance to win millions of dollars (Tinari and Riva, 2021). The first of these, Ohio’s Vax-A-Million program, was announced on May 12, 2021. According to our systematic review of public announcements and media reports, 18 states have instituted vaccine lotteries as of July 7, 2021.1

Smaller drawing-based incentives have been effective for encouraging a range of health behaviors (Kimmel et al., 2012, Sen et al., 2014, Patel et al., 2018). Nonetheless, behavioral economists and ethicists have prominently suggested that “financial incentives are likely to discourage vaccination” (Volpp et al., 2021) or “stoke new fears and, perversely, increase resistance to vaccination” (Largent and Miller, 2021).

News media has concluded the COVID-19 lottery schemes were not effective (Goldberg and Doherty, 2021). One early published study of the Ohio lottery incentive used an interrupted time series analysis and concluded that the incentive was ineffective (Walkey et al., 2021). Likewise, an unpublished study using synthetic controls and full vaccination as the outcome found no effect for the lottery (Lang et al., 2021). Another unpublished study, using both differences-in-differences and synthetic controls, found that the Ohio lottery “increased first dose Covid-19 vaccinations by between 50,000 and 100,000” at a cost of about $112 per starting dose (Brehm et al., 2021). In this paper, we expand beyond the work on Ohio and look across all current vaccine schemes to evaluate their effects on state vaccination rates. We also estimate the cost per marginal vaccination.

Methodology

To determine whether statewide vaccine lottery schemes are “worth the money”, we construct a dataset that matches information on the timing and location of statewide vaccine lotteries with daily, county-level data from the U.S. Centers for Disease Control (CDC) on the cumulative number of people who have received at least one dose of an emergency-authorized Covid-19 vaccine. Using this data, we estimate a series of econometric models to investigate the extent to which lottery schemes increased vaccine uptake among hesitant populations.

For the purposes of this econometric analysis, we consider the twelve statewide vaccine lottery schemes that were announced as of June 7, 2021.2 This cut-off date is chosen as it is thirty days prior to when our vaccination dataset ends (July 7, 2021).

Econometric model

We postulate that, in a given county and as of a given date , the cumulative number of people who have received at least one dose of an emergency-authorized Covid-19 vaccine (denoted and expressed as a percentage of the eligible population in the county) is a function of the cumulative number of people vaccinated in periods immediately prior and the vaccination hesitancy among the remaining unvaccinated population, which determines the rate at which additional individuals become vaccinated. We allow—but do not impose—that the announcement of a statewide lottery scheme for vaccinated residents may induce some fraction of the remaining unvaccinated population to elect for vaccination.

Thus, to estimate the impacts of a specific statewide vaccine scheme, we specify the following model: where dependent variable is the percentage of the eligible population in county that has received at least one dose of an emergency-authorized Covid-19 vaccine.3 Variable is observed on a given date , where date is the date the vaccine lottery was first announced and is the number of days following the announcement. Explanatory variable is percentage of the eligible people that had received at least one dose of an emergency-authorized Covid-19 vaccine on the day before the lottery was announced. Vector includes a series of county-level sociodemographic variables discussed in further detail in Section 2.2. These factors have been shown in prior research to be strong correlates for vaccine hesitance (Robertson et al., 2021).

Our variable of interest (denoted ) is a binary variable, which distinguishes between our treatment and control groups. The variable takes value one if the observed county lies in the state offering the vaccine lottery. The variable takes value zero if the county lies outside the lottery state. The coefficient on () measures the responsiveness of vaccine uptake days following the announcement of the lottery.

Sampled Counties: For each regression, we include as “treated” counties all those in the state that is subject to the lottery being analyzed. Counties chosen as “controls” in the model are those that lie in states that do not currently have their own lottery. Additionally, we exclude counties that have other substantial cash or in-kind vaccine incentive schemes, such as West Virginia’s $100 savings bond. These non-lottery incentive schemes may have a confounding influence on vaccine uptake.4 The timing and location of such programs are detailed in Appendix Table B.2.

Table B.2

Description of excluded statewide Covid-19 vaccine incentive schemes.

State	Date announced	Individual value	Incentive description
MN	5/27/2021	$25	Variety of prizes
NM	6/13/2021	$100	$100 cash incentive
NC	5/4/2021	$25	Cash card
WV	4/26/2021	$100	Savings bond

Thirty-Day Iteration: For a given lottery, we estimate the model for each day from 1 to 30 to obtain a semi-parametric estimate of the impacts of the lottery over a thirty-day window. We obtain 30 estimates of the impact of a lottery on vaccine uptake—one for each day from the announcement to thirty days after.

Ohio Vax-a-Million Example: To illustrate, consider our approach to analyzing the impacts of the Vax-a-Million lottery, announced in Ohio on May 12, 2021. For this analysis, our sample consists of all counties in Ohio (“treated”) and counties outside of Ohio that do not have a statewide lottery or their own substantial vaccine incentive (“untreated”). We first estimate the model with the dependent variable observed on date May 13, 2021. Explanatory variable is expressed at date May 11, 2021. Estimated coefficient yields an estimate of the impact one day following the announcement of the lottery. We then re-estimate the model with the dependent variable observed on May 14, 2021. We repeat the analysis for the 30 days following the lottery announcement.

We repeat this analysis separately for each of the twelve states that announced statewide vaccine lottery schemes as of June 7, 2021. The timing and location of detailed in Appendix Table B.1.

Table B.1

Description of Major statewide Covid-19 vaccine lottery schemes.

State	Date announced	Program outlay	Description
AR	5/25/2021	$1,000,000	$20 lottery ticket
CA	5/27/2021	$15,000,000	10 winners selected to win $1,500,000
CO	5/25/2021	$5,000,000	$1,000,000 lottery once a week for 5 weeks
DE	5/25/2021	$302,000	Lottery Program
IL	6/18/2021	$7,000,000	Illinois Vaccine Sweepstakes
KY	6/4/2021	$3,000,000	Kentucky Vaccine Sweepstakes
LA	6/17/2021	$1,400,000	weekly lottery, and grand prize
ME	6/16/2021	$896,809	Do not Miss Your Shot
MD	5/20/2021	$2,000,000	$2 million in prize money
MA	6/15/2021	$5,000,000	Five $1 million prices
MI	7/1/2021	$4,500,000	MI Shot to Win
NV	6/17/2021	$5,000,000	Variety of lottery opportunities
NM	6/1/2021	$10,000,000	Vax to the Max Sweepstakes
NY	5/20/2021	$5,078,340	Vax & Scratch
NC	6/10/2021	$4,000,000	$1,000,000 Lottery
OH	5/12/2021	$5,000,000	“Vax-A-Million”
OR	5/21/2021	$1,360,000	“Take Your Shot, Oregon”
WA	6/3/2021	$2,000,000	Shot of a Lifetime”
WV	6/3/2021	$9,176,000	“Do It For Babydog: Save a life, Change your life”

Data and summary statistics

Information on the timing and location of vaccine lotteries was collected from various sources with the help of supervised research assistants (see Appendix Table B.1).5 We categorized all state-level vaccine incentives as of July 7, 2021. Incentives were classified as either lotteries, cash payments, in-kind incentives, or youth-specific programs. Daily, county-level data on the cumulative number of people who have received at least one dose of an emergency-authorized Covid-19 vaccine are from the CDC (2021). Our analysis includes information on 2,887 U.S. counties. Panel (a) of Fig. 1 shows the daily U.S.-average and cross-county dispersion in vaccine uptake from January 1, 2021 to July 7, 2021. The kernel density plot in panel (b) of Fig. 1 shows the cross-county dispersion in vaccination rates as of June 1, 2021.

Fig. 1

Summary statistics for county-level vaccinations. Notes: Panel (a) shows the daily U.S.-average and cross-county dispersion in vaccine uptake from January 1, 2021 to July 7, 2021. The kernel density plot in panel (b) shows the cross-county dispersion in vaccination rates as of June 1, 2021.

Summary statistics for the sociodemographic control variables used to model vaccine hesitancy are reported in Table 1. County-level GOP vote shares in the 2020 Presidential Election are obtained from the MIT Election Data Science Lab (2021). Data on county-level population density, percent of population over 65 years of age, percent of population white (non-Hispanic), percent black (non-Hispanic), percent Hispanic, percent with at least college education, unemployment rate, per capita income, percent with less than high school education, and percent foreign-born are from the Rural Atlas (ERS, 2020).

Table 1

Summary statistics of sociodemographic control variables.

Variable			Description	Mean	Std. Dev.	Min	Max
	Political affiliation
		GOP	% voting GOP in 2020 Pres Election	0.64	0.16	0.05	0.94
	Age
		Age 65+	% of pop 65 years or older	0.16	0.04	0.04	0.43
	Ethnicity
		White	% of pop white, non-hispanic	0.80	0.18	0.03	0.99
		Black	% of pop black, non-hispanic	0.09	0.15	0.00	0.85
		Hispanic	% of pop hispanic	0.06	0.09	0.00	0.83
		ForeignBorn	% of pop foreign born	0.04	0.05	0.00	0.53
	Educational attainment
		CollegePlus	% of pop with bachelors degree or higher	0.22	0.10	0.05	0.79
		EdLessHS	% of pop with less than HS education	0.13	0.06	0.01	0.39
	Economic performance
		UnempRate	% of workforce unemployed	0.04	0.01	0.01	0.18
		Income	U.S. dollars per capita (annual)	27063	6605	10148	72832
	Covid-19 transmission
		Pop Density	People per sq mile	292	1885	0	69468

Results

In this Section, we begin by discussing—in Section 3.1— outcomes for the Ohio Vax-a-Million program. In Section 3.2, we then expand our discussion to the broader set of vaccine lottery programs.

Ohio Vax-a-Million Program

Fig. 2 summarizes the impacts of the Ohio Vax-a-Million Lottery on Statewide Vaccine Uptake. Panel (a) shows the number of vaccinations (percentage of the eligible population) induced by the lottery program over the thirty-day period following the announcement of the lottery. Panel (b) of Fig. 2 plots the actual vaccination rates in Ohio (total number of eligible people received at least one shot) from April 28, 2021 to June 12, 2021, alongside the counterfactual evolution in state vaccinations had the lottery not occurred. The vertical red line in panel (b) corresponds to the announcement of the lottery (May 12, 2021).

Fig. 2

Impacts of Ohio Vax-a-Million lottery on statewide vaccine uptake. Notes: Panel (a) shows the number of vaccinations (expressed as a percentage of the eligible population) induced by the lottery program over the thirty-day period following the announcement of the lottery. Panel (b) plots the actual vaccination rates in Ohio (expressed as the total number of eligible people who have received at least one shot) from April 28, 2021 to June 12, 2021, alongside the counterfactual evolution in state vaccinations had the lottery not occurred. The vertical red line in panel (b) corresponds to the announcement of the lottery.

Referring to panel (a) of Fig. 2, the impacts of the lottery are small over the few days following the announcement and rise over the subsequent thirty-day period. This is as expected. Even if the lottery announcement changes a person’s intentions to receive the vaccine immediately upon announcement, it may take some time for the person to schedule and obtain a vaccination. Additionally, some portion of the population may not hear about the lottery immediately upon announcement.

We estimate that—evaluated thirty days after announcement—the Vax-a-Million program, induced an additional 0.003 (95% C.I.: 0.0013) percent of the eligible population to opt for vaccination. Referring to panel (b) of Fig. 2, this corresponds to an additional 32.09 (95% C.I.: 15.59) thousand vaccinations. Importantly, the estimated impact does not appear to return to zero. This suggests that the program is inducing at least some people who otherwise would not have been vaccinated to opt for vaccination, rather than encouraging those who would have been vaccinated anyway simply to do so more quickly. Appendix Fig. B.1 reports results corresponding to those in panel (b) of Fig. 2 for each of the 12 state lottery programs considered in our analysis. These results are summarized below.

Fig. B.1

Implied statewide vaccination levels. Notes: Panels plot actual state vaccination rates (expressed as the total number of eligible people who have received at least one shot), alongside the counterfactual evolution in state vaccinations had the lottery not occurred. In each panel, the vertical red line corresponds to the announcement of the lottery.

All statewide vaccine lottery programs

Panel (a) of Fig. 3 summarizes the impacts of each of the 12 statewide lottery programs on vaccine uptake, evaluated thirty days following the announcement of the lottery.6 These impacts are expressed as a percentage of the eligible population who have received at least one shot. Referring to panel (a) of the Figure, we estimate a positive, and statistically significant, impact of vaccine lotteries for 10 of the 12 states (i.e., all states but Arkansas and California).7 Across all lottery programs considered, the average impact was a 0.007 percentage point increase in vaccine uptake. The lotteries in Colorado and New York appear to have generated the largest impact on vaccine uptake as a percent of the eligible population (0.013 and 0.012 percentage points, respectively), although these estimates are statistically indistinguishable from those for several other states.

Fig. 3

Thirty-day impact of statewide lottery programs on vaccine uptake. Notes: Panel (a) summarizes the impacts of statewide lottery programs on vaccine uptake, evaluated thirty days following the announcement of the lottery. These impacts are expressed as a percentage of the eligible population who have received at least one shot. Panel (b) reports the cost per marginal vaccination for each program, calculated as the number of induced vaccinations divided by program outlay.

Finally, panel (b) of Fig. 3 reports the cost per marginal vaccination for each program, calculated as the number of induced vaccinations divided by program outlay. On average, the cost per marginal vaccination across these programs was approximately $55. The lottery in New York appears to have been most efficient at increasing vaccinations, with a cost per marginal vaccination of $20.90 (95% C.I.: $3.17). On the other hand, the lottery in West Virginia was least efficient, with a cost per marginal vaccination of $769.60 (95% C.I.: $204.83).

Conclusion

This research evaluates the effects of the twelve statewide vaccine lottery schemes that were announced as of June 7, 2021 on state vaccination rates. We find that 10 of the 12 statewide lotteries studied (i.e., all but Arkansas and California) generated a positive, statistically significant, and economically meaningful impact on vaccine uptake after thirty days. Moreover, lottery programs appear to induce at least some people who otherwise would not have been vaccinated to opt for vaccination, rather than encouraging those who would have been vaccinated anyway simply to do so more quickly. On average, the cost per marginal vaccination across these programs was approximately $55.

Of course, this research is not without limitations. Perhaps most important among these is the implicit assumption in our model that vaccination rates in treatment and control counties followed “parallel trends” prior to the lottery announcement.8 Further, we note the possibility of endogeneity between the policy treatment and the outcome variable. For example, at least in some states, policymakers were induced to opt for lottery schemes because of systematically low vaccination rates relative to their counterparts. While we believe inclusion in our model of vaccination rates just prior to the lottery announcement date and sociodemographic correlates to proxy for vaccine hesitancy substantially reduce any biases associated with this endogeneity, we are unable to eliminate it.

CRediT authorship contribution statement

Christopher Robertson: Conceptualization, Literature search, Original drafting of manuscript, Visualization, Editing of manuscript, Submission. K. Aleks Schaefer: Conceptualization, Data analysis, Visualization, Editing of manuscript, Verified the underlying data. Daniel Scheitrum: Conceptualization, Data analysis, Visualization, Editing of manuscript, Verified the underlying data.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.