Expert Forecasts of COVID-19 Vaccine Development Timelines.

INTRODUCTION

As of July 2020, the COVID-19 pandemic has resulted in close to 12 million infections and more than 500,000 deaths globally.[1] Rapid development of a SARS-CoV-2 vaccine is a public health priority.

Because of the unprecedented nature of research efforts, historic experience with vaccine development may not provide accurate estimates for SARS-CoV-2 vaccine development timelines and outcomes. Expert elicitation and aggregation synthesize diverse insights that are spread across expert communities, and offer an alternative way to estimate when vaccines will be available.[2,3]

METHODS

We identified experts through the contact network of the vaccinologist author BJW and corresponding author searches for review articles on vaccinology in highly cited medical journals. We solicited experts for participation by e-mail on the week of June 21st and a second time on the week of June 28th.

We asked for timeline forecasts for three vaccine development milestones: (1) a field study enrolling at least 5000 participants reporting results, (2) a vaccine being available to those at highest risk from the virus in the USA and/or Canada, and (3) a vaccine being available to the general public in the USA and/or Canada. Experts provided best, soonest, and latest estimates for when each milestone would occur. We also asked for probability estimates of two setbacks: (1) the first vaccine widely deployed in the USA and/or Canada receiving a boxed warning from the FDA to highlight serious or life-threatening adverse reactions and (2) the first large field trial in the USA and/or Canada reporting a null or negative result on an efficacy endpoint. We calculated the median and interquartile ranges of the forecasts to measure both the consensus and range of opinions.

Finally, our survey collected basic demographic information and two Likert scale questions measuring comfort answering questions about the US and Canadian health care system and the extent to which experts think the pace of COVID-19 research will lead to more errors in the vaccine development process.

RESULTS

Our sample consisted of 28 experts with 25 median years of experience working with vaccines (range 8–42); 20 worked in academia, 6 in industry, and 2 in government. Participants were mostly based in Canada (20) and the USA (6); participants worked in academia (20), industry (6), and public health (2). Participants indicated they were very comfortable (46% or 13/28) or somewhat comfortable (46% or 13/28) answering questions about vaccine development in the USA or Canada.

The median and interquartile range of forecasters’ best guesses, soonest occurrence, and latest occurrence for each milestone are indicated in Figure 1. See Figure 2 for histograms of setback probabilities.

Figure 1

Boxplots of the best guess, soonest estimate, and latest estimate for each of the vaccine development milestones. The width of each box represents the interquartile range and the bar represents the median. The median best guess, soonest guess, and latest guess for the first field trial reporting results were respectively March 2021, December 2020, and July 2021 (interquartile ranges were respectively January 2021–April/May 2021, November 2020–January 2021, and June 2021–January 2022). The median best guess, soonest guess, and latest guess for a vaccine being available to at-risk individuals were respectively March/April 2021, February 2020, and December 2021 (interquartile ranges were respectively February 2021–September 2021, January 2021–April/May 2021, and June 2021–May 2022). The median best guess, soonest guess, and latest guess for the vaccine being available to the public were respectively September/October 2021, June 2021, and July 2022 (interquartile ranges were respectively June 2021–January 2022, March 2021–September 2021, and December 2021–October/November 2023).

Figure 2

Histograms of probabilities provided for the two setback questions. The median forecast that the FDA would issue a black box warning for a widely deployed vaccine was 30% (interquartile range 15–40%). For the probability of the first field study in the USA or Canada reporting a null or negative result on an efficacy endpoint, the median probability was 40% (interquartile range 23–51%).

Experts were split on whether the pace of COVID-19 research is increasing the potential for errors; 53% (15/28) agreed, 14% (4/28) were neutral, and 32% (9/28) disagreed.

DISCUSSION

Experts in our survey offered forecasts on vaccine development that were generally less optimistic than the timeline of early 2021 offered by US public officials.[4] Experts seem to believe that a publicly available vaccine next summer is the best case scenario with the possibility that it may take until 2022. In addition, the median probabilities that a vaccine would receive a warning label and that a field study might not report positive outcomes were 30% and 40% respectively.

Our survey has important limitations. First, our sample is predominantly Canadian, though it is worth noting Canada has a productive vaccine development enterprise.[5] Second, our elicitation approach was relatively simple to minimize survey burden for vaccine developers. Nevertheless, our estimates provide a more complete picture of expert belief than single point estimates or media quotes from individuals.