Safeguarding evidence-based decision making in the FDA for COVID-19 vaccines.

Public polls in the summer and fall of 2020 indicated two-thirds of Americans would be reluctant to get a COVID-19 vaccine once available [1], worrying that political pressure could cause the US Food and Drug Administration (FDA) to rush its review. Such a lack of public confidence suggests a weakening of the FDA’s organizational reputation and credibility [2], which could adversely affect COVID-19 vaccine uptake and thereby undermine public health efforts to achieve herd immunity. In light of the ongoing importance of vaccination to COVID-19 and a growing array of diseases, we consider the causes of such public skepticism and propose means to overcome it.

Trust in government can be undermined if officials make premature statements of efficacy or safety that are later contradicted by emerging evidence. For example, despite limited evidence, government officials in March 2020 sought to reassure the public by highlighting the potential effectiveness of hydroxychloroquine, a malaria treatment, against COVID-19, followed by an Emergency Use Authorization (EUA) the following month. The EUA was revoked in June when serious adverse events came to light revealing its benefits did not outweigh its risks. Misleadingly optimistic characterizations of efficacy can similarly undermine trust. When the FDA issued an EUA for convalescent plasma, the FDA commissioner reported the intervention’s efficacy in terms of relative rather than absolute risk reduction, which made the treatment appear more effective than it actually is, leading to broad criticism by the scientific community. Finally, trust can further erode if regulatory authorities and public institutions fail to coordinate research efforts, thereby leaving important questions of comparative effectiveness unanswered despite substantial public investments, a shortcoming that became apparent in the COVID-19 evidence ecosystem after several large multi-arm trials were undertaken [3].

These regulatory missteps can be exacerbated by private sector communications that imply greater certainty than is warranted by regulatory standards. For example, many published reports and media communications failed to adequately explain the difference between an EUA and FDA licensure, leading to unjustified expectations that drugs available under EUAs will be safe and beneficial. The FDA’s grant of an EUA is an extraordinary measure, available only after an official declaration that the United States is encountering a chemical, biological, radiological, or nuclear threat. Products subject to EUAs in most cases have not been, and may never be, FDA-approved, although EUAs can also be used to authorize widespread off-label uses of approved products, as occurred with hydroxychloroquine.

Given this lack of understanding and fear that FDA decision-making was yielding to political pressure, the biopharmaceutical industry took an unprecedented pledge to “stand with science,” [4] and senior FDA officials committed to protect the agency’s science-based decisions from political interference [5]. The US Department of Health and Human Services (HHS), the parent agency to which the FDA reports, then issued an internal memorandum limiting the ability of FDA officials to sign new administrative rules, prompting a joint statement by seven past FDA commissioners that the agency’s political independence was critical to maintaining public trust [6].

This tug-of-war between the FDA and its parent agency reflects the tension between the ideals of public accountability to elected leaders and independent, evidence-based policy. By Congressional design, the FDA is an executive agency under the umbrella of HHS and is headed by a single commissioner who is appointed by the President. Given recent events and waning public trust, Congress may wish to consider whether the FDA should be restructured as an independent agency with certain safeguards to provide greater freedom from political influence [7], such as governance by multiple commissioners that serve staggered terms and who are removable only for cause. However, the benefits of independence must be weighed against the potentially greater difficulty in creating policy alignment among the FDA and other HHS agencies, such as the National Institutes of Health and the Centers for Medicare & Medicaid Services [8], and also the likely reduction in democratic accountability that helps to ensure FDA policies reflect public values, such as the appropriate trade-off between speed and safety [9].

The stakes for public confidence in a health intervention have never been higher given the centrality of widespread immunization to ending the COVID-19 pandemic, and the issue is likely to endure as future health challenges emerge. Until Congress considers whether greater agency independence is appropriate, the FDA should act to restore public confidence by shoring-up scientific standards at all stages of development and approval [10]. We consider a number of measures that could be taken at the pre-licensing, licensing and post-licensing stages to ensure science-based decision making in the FDA for COVID-19 vaccines (see Table 1 ).

Table 1

Pre-licensing, licensing and post-licensing measures to ensure science-based decision making in the FDA for COVID-19 vaccines.

Stage	Measures
Pre-licensing	Expansion of existing laws restricting the advertising of unapproved products; expanded use of established whistleblowing protection to encourage reporting of wrongful political interference
Pre-licensing	Publication of full trial protocols before their initiation; transparency in trial suspensions with disclosure of causation
Pre-licensing	Provision of an information statement disclosing key efficacy and safety outcomes to recipients of vaccines administratered under an EUA; use of stepwise authorization expanding progressively to include a broader population
Licensing/ Post-licensing	Demonstration of actual patient benefit as a requirement for approval (no accelerated approval surrogates); inclusion of reduction in moderate or severe infection cases as a clinical endpoint
Licensing/ Post-licensing	Comparison of vaccines via platform trials with common controls; conduct of “challenge studies” to directly compare vaccines’ effectiveness and differences in benefit-risk balance
Licensing/ Post-licensing	Implementation of FDA’s Benefit Risk Framework to improve transparency in the recommendations of the Vaccines and Related Biological Products Advisory Committee
Licensing/ Post-licensing	Collection of long-term immunity and safety data via post-approval studies and surveillance; stricter implementation of monetary penalties; progressive expansion of licensing conditional on submission of additional evidence

Pre-licensing stage

Before a drug or vaccine is licensed, government officials should refrain from endorsing products with unproven efficacy and safety, which by definition include any product made available under an EUA. Health officials and the media must take greater care to emphasize that products made available under EUAs have not yet been proven either safe or effective, and that an EUA is more akin to the FDA’s expanded access program, which allows seriously-ill patients to request access to experimental therapies while those therapies are still undergoing clinical trials. Existing laws restrict (or authorize the FDA to restrict) advertising of unapproved products, including those made available under expanded access and EUA protocols. Although these laws are primarily directed at manufacturers, analogous internal policies and enforcement mechanisms are needed, possibly through the use of penalties (financial and non-financial), to ensure that government officials do not make premature, misleading, or otherwise unjustified claims about unproven products. To deter undue political influence, FDA staff can utilize established whistleblowing mechanisms and legal protection against retaliation provided by the Whistleblower Protection Act (1989) and the Whistleblower Protection Enhancement Act (2012). The process and evidence requirements for whistleblowing, which are managed by the US Office of Special Counsel, should be fit-for-purpose to FDA staff, ensuring they incentivize the effective reporting of political pressure to sidestep scientific principles [11].

Greater clarity of clinical trial protocols would also help. Currently, trial sponsors are required to submit specific information for certain trials at clinicaltrials.gov [12]. In the case of COVID-19 vaccines, manufacturers eventually disclosed details of the clinical study designs, but only after public pressure mounted. Existing requirements could be expanded to include publication of full trial protocols before their initiation at the patient enrollment stage, with detailed information on statistical requirements, expected number of events at completion, and stopping rules for interim analysis, all of which is usually kept confidential while studies are ongoing. Disclosure requirements could be implemented incrementally, starting with increased reporting requirements for the issuance of an EUA, followed by more comprehensive requirements for eventual approval. Similarly, full transparency of the reasons leading to trial suspensions (along with disclosure that causation has not necessarily been established) could increase public trust.

FDA guidance for COVID-19 vaccine EUAs was updated in October 2020 to recommend a median follow-up duration of at least 2 months and a minimum of 5 severe cases in the placebo group [13]. Yet the threshold for granting an EUA remains generous, requiring only that the “known and potential benefits of a product… outweigh the known and potential risks of the product.“ [14]. Because the public may not understand that this threshold does not assure either safety or efficacy, the administration of a vaccine under an EUA should be accompanied by an information statement (similar to a Vaccine Information Statement). Such a statement should specify, in a simple columnar format, the number of patients in whom the product has been tested and the number of patients in the control group, and provide a quantitative assessment of the principal efficacy and safety outcomes for each of these groups. They should also include a listing of specific rare adverse events observed to occur with past vaccines and their frequency, along with a clear indication that the EUA product has not been licensed by the FDA and that known or unknown rare events may later be identified. Information statements should be written in non-technical terminology and be communicated by appropriately trained health care personnel prior the administration of the EUA products, and should also be made available online for advance viewing. To ensure that use is aligned with the latest evidence, an EUA could also be granted in a stepwise fashion, such as by authorizing administration of a vaccine first to those subpopulations with higher disease or transmission risk (e.g. health care professionals, elderly, those with certain co-morbidities, etc.) and progressively expanding to include a broader population as the vaccine sponsor provides additional evidence.

Licensing and post-licensing stages

Availability of the EUA mechanism, which provides pre-approval access in times of urgent need, should help minimize pressure to rush FDA licensure. However, because current understanding of COVID-19 is incomplete, clinical efficacy may not be adequately established by immunogenicity data, cautioning against accelerated approval [15]. Although the accelerated pathway generally requires validation of surrogate endpoints following approval, the value of such validation is questionable in the current context as confirmatory trials may take several years to complete, an unreasonably long time given a global strategy to rapidly achieve herd immunity. Rather than administer hundreds of millions of doses based on unverified surrogate immunogenicity endpoints, any new vaccine should be licensed only if it has demonstrated actual patient benefit in clinical risk reduction. Given the greater public health relevance of severe cases of COVID-19, reduction of such cases should be included as a co-primary or secondary endpoint.

To facilitate comparison of multiple experimental vaccines, governments should facilitate platform trials with common control groups, which may also conserve resources and speed recruitment. Once effective rescue treatments become available and the first vaccines have entered the market, “challenge studies” in which subjects are intentionally exposed to SARS-CoV-2 could be ethically conducted to directly compare the effectiveness of different vaccines and better understand differences in their benefit-risk balance for particular sub-groups [16].

To improve the transparency of recommendations of the Vaccines and Related Biological Products Advisory Committee (VRBPAC), a more structured approach to risk–benefit analysis should be implemented, such as that described in the FDA’s 2018 Benefit-Risk Framework [17]. This is composed of two key components: i) the Benefit–Risk Dimensions, outlining the critical clinical elements that are considered in the analysis, together with statements on “evidence and uncertainties” and “conclusion and reasons”; and ii) the Benefit–Risk Integrated Assessment, combining all dimensions in an overall analysis and providing an explanation of the regulatory decision. Use of such a framework would systematize the evaluation of a vaccine's key safety and effectiveness dimensions, leading to a concise rationale for or against approval while minimizing reliance on ad hoc decision making.

Long-term immunity and safety data must still be collected through post-approval studies and surveillance. Follow-up data should allow regulators to determine the duration of protection and the risk for vaccine-associated enhanced respiratory disease or other adverse events that may not immediately present. Section 303(f)(4) of the Federal Food, Drug and Cosmetic Act allows the FDA to impose civil monetary penalties if post-marketing studies or clinical trial requirements are violated [18]. However these penalty allowances should be more strictly imposed, possibly at pre-specified intervals if data are not submitted by mutually agreed deadlines, as past experience has shown limited success in the completion of such post-marketing requirements [19], [20]. As an alternative to monetary penalties, the FDA could implement progressive authorization or licensing (e.g. as part of a stepwise EUA, described above), requiring the completion of medium to long-term immunity and safety studies as a regulatory milestone for further authorization or licensing to lower-risk populations. Linking progressive authorization or licensing with follow up studies would create an additional incentive to fulfill study commitments.

Conclusion

Given the unprecedented resources devoted to COVID-19 vaccine development, it would be truly unfortunate if safe and effective vaccines were refused by the public due to distrust of the regulatory process. Vaccine hesitancy, already problematic for many proven and well-established products, could worsen if a new and unproven product is rushed to market. The public should be aware that FDA-regulated products involve an inherent trade-off between the speed of availability and the evidence base on which such products are made available. When evaluating the merits of faster availability, such as under an EUA or accelerated approval, policymakers should take into account not only benefits and risks, but also the potential loss of credibility and public trust that could occur if products are released into the market before sufficient evidence is available.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.