Lessons from an international trial evaluating vaccination strategies for recovered inpatients with COVID-19 (VATICO).

The protection provided by natural versus hybrid immunity from COVID-19 is unclear. We reflect on the challenges from trying to conduct a randomized post-SARS-CoV-2 infection vaccination trial study with rapidly evolving scientific data, vaccination guidelines, varying international policies, difficulties with vaccine availability, vaccine hesitancy, and a constantly evolving virus.

Main text

The development of vaccines for SARS CoV-2 reduced COVID-19 morbidity and mortality and changed the landscape of the pandemic. The duration and extent of the humoral response to both SARS-CoV-2 infection and vaccination vary based on age, immune status, and disease severity. , Moreover, the protection provided by previous SARS-CoV-2 infection is uncertain and varies significantly based on changes in the SARS-CoV-2 spike protein epitopes as well as on level and duration of immune response. ,

Additional uncertainties exist regarding the contribution of natural versus vaccine-acquired immunity to protection from subsequent symptomatic or severe COVID-19. Vaccination of seropositive individuals results in a recall of humoral and T cell immunity that is highly variable and additional mRNA vaccination over two doses results in hybrid immunity that provides some additional protection even among individuals with prior infection.

The Vaccination Strategies for Recovered Inpatients with COVID-19 (VATICO) study was an international, multi-arm, phase 4, open-label trial, leveraging the Therapeutics for Inpatients with COVID-19 (TICO; ACTIV-3) randomized controlled clinical trial part of the Accelerating COVID-19 Therapeutic Interventions and Vaccines (ACTIV) public-private partnership. The objective of the trial was to investigate the optimal number of doses and timing of mRNA vaccination on antibody response after 48 weeks among patients who had recovered from COVID-19-related hospitalization.

In this commentary, we discuss the lessons learned from the 2 trying years working on this post-COVID-19 vaccination trial during the pandemic. We reflect on why a large, funded, international, and experienced study group faced insurmountable obstacles in completing the trial as planned and use this experience to outline some potential solutions. In the supplemental information for this commentary, we provide information and a detailed timeline of the study. Even though we focus on the challenges associated with the evaluation of vaccination protocols for recovered inpatients with COVID-19, this experience provides insights on the difficulties associated with the post-approval study of vaccination protocols, especially when the health care system is under stress and faces a new threat.

Challenges with implementation

VATICO was designed as a 2 x 2 factorial study to measure the level of neutralizing antibody levels specific to an mRNA vaccine 48 weeks after randomization to one of four vaccination strategies to help address two questions: (1) timing (immediate versus 12-week deferral) of the first vaccination and (2) the number of mRNA vaccine doses (1 or 2). Given that VATICO enrolled patients who participated in trials carried out with the TICO master protocol, the study was designed to both compare the vaccination strategies in patients assigned placebo in TICO as well as those assigned to receive antibody treatment. Despite clear scientific need and a relatively simple protocol, the study failed to meet its recruitment goal of 320 participants (80 per group) who were assigned a placebo in TICO and 320 who were assigned treatment. The trial was stopped after 66 participants were enrolled because the required sample size (640 total participants) was not going to be achieved.

Challenge 1: Site participation and staffing

The study opened to enrollment just as the Delta surge was rising at many hospitals, and many hospital-based research teams were overwhelmed with the clinical responsibilities of the pandemic. Therapeutic trials in hospitalized patients require staff and infrastructure for inpatient monitoring, whereas standard vaccine trials are almost exclusively outpatient. The same busy study investigators and staff were asked, in addition to their clinical responsibilities during the pandemic, to enroll and follow hospitalized patients into therapeutic trials and also perform a vaccine-evaluation study in the outpatient setting. These considerable clinical and research activities of the potential site teams had a direct impact on recruiting sites for this study. For example, 73 sites expressed interested in participating in VATICO, with 1151 potentially eligible TICO participants at these sites between June 2021 and February 2022. Of these 73 sites, only 30 opened for enrollment, and only 19 of the 30 open sites enrolled at least a participant in VATICO.

Challenge 2: Rapidly changing vaccination guidelines

Rapidly evolving local and international guidelines may have been difficult for patients to understand and directly impacted the implementation of the study as well. For example, there were changes on the need for “booster” vaccination, and the CDC initially recommended delaying vaccination to 90 days for persons who received antibody therapy (convalescent plasma/monoclonal antibodies), a recommendation was later removed based on expert opinion. This resulted in uncertainty among patients, physicians, and study personnel, both in the US and internationally. Other countries also implemented different vaccination strategies and travel mandates, while employer and airline policies varied in regard to timing and number of doses that defined immunization status. This resulted in additional hesitancy or reluctance for vaccination or enrollment in a clinical trial that limited vaccination options.

Challenge 3: Working within the regulatory framework

Even simple regulatory issues, such as determining the need to submit a new investigational new drug (IND) application, resulted in weeks-long delays. Moreover, after positive initial discussions, the vaccine manufacturer subsequently decided they would not allow use of US government-purchased vaccine in this clinical trial. Overall, there was little commercial incentive, despite the public health importance, of determining the optimal regimen for recently hospitalized patients. Therefore, our study team had to find alternative sources of vaccine for VATICO. Given expanding international vaccine availability at the sites, a decision was made to utilize locally sourced vaccine, either the Pfizer or the Moderna mRNA vaccines, under the scope of their local Emergency Use Authorizations (EUAs) or equivalent regulations. However, local authorities and pharmacies had great difficulty adjusting the schedule of vaccination for study participants.

Challenge 4: Vaccine availability

The significant delays in obtaining vaccine, especially for non-US sites, and the closure of TICO studies to future enrollment of investigational agents resulted in far fewer TICO patients potentially eligible than had been the case when the study was envisioned. Sourcing vaccine from local supplies and the significant vaccine access challenges in non-US countries made study vaccine supply a major limitation in successfully conducting VATICO. Moreover, per the acquisition contracts, US-government-purchased vaccines were not available for research purposes unless approved by the manufacturers. Further complicating the matter was the fact that the original vaccine supply purchased by the US government had a short expiry (6 months from manufacture) and little shelf-life remaining by the time enrollment into the study began. This limited availability of vaccine and the inability of the study to provide it was a particular challenge. As a result, although there was strong interest in VATICO from study sites in non-US sites, there were difficulties and considerable delays obtaining a vaccine supply for the study in those countries. Another practical consideration of a vaccine supply challenge in both the US and non-US sites was that single-dose vaccine packaging could not be provided. The study team had to work to avoid wastage when vaccine was provided in 10-dose vaccine vials with 24-h expiry after vial opening, in a study where patients are enrolled variably by day and time.

Challenge 5: Participant availability

The parent TICO trial recruited patients while they were in the hospital with a focus on treatment and involved minimal outpatient follow-up visits. VATICO, on the other hand, involved recruiting outpatients who had recovered from their hospitalization, and required a series of outpatient evaluations. The eligibility requirement to be recovered from their illness and back to normal living arrangements for at least 2 weeks and the additional visits and blood sampling made it difficult for some patients to participate. These barriers to recruitment influenced the composition of the VATICO study population and should be taken into consideration when designing similar studies in the future. For example, having the ability to recruit and follow patients transferred in post-acute facilities and nursing homes could have allowed for the recruitment of individuals with delayed recovery.

Challenge 6: Vaccination beliefs

As detailed in the > supplemental information and Table S4, to gain insights on the low enrollment, the study team conducted a chart review at high-recruiting participating US sites. The most common reasons for refusal were reluctance to continue participating in a research protocol, fear of being randomized to the deferred arm, or just wanting to get vaccinated at a place known to them other than the study site. In contrast, some did not want to enroll because they did not want to delay receiving vaccine or to give up the opportunity to receive two doses of vaccine. Anecdotally, this opinion was particularly prevalent among patients who were immunocompromised. Even providers were often misinformed on the existing guidelines and provided partially correct, or outdated information to patients when they asked them for advice regarding study participation. For example, providers often discouraged enrollment and vaccination for individuals who had received a monoclonal antibody even after this recommendation (that was based on expert opinion) was removed from the relevant CDC guidance.

Lessons learned for future vaccine studies

Lesson 1: International, randomized, post-marketing (or post-EUA) studies of post-infection vaccination face substantial regulatory challenges

The lessons we learned from trying to conduct the VATICO trial highlight that during a pandemic, post-marketing (or post-EUA) studies designed to inform post-infection vaccination protocols face multiple challenges, including vaccine availability, changing guidelines, evolving understanding of immune response to the vaccine and the infection, varying treatment protocols (such as the variable adoption and availability of monoclonal antibodies for outpatient treatment), emergence of new variants, and vaccine acceptance by the population. Collaboration, agreement on, and planning for resources to conduct such trials should be a feature of planning for pandemic research.

Lesson 2: Vaccine manufacturers have limited incentive to collaborate in post-marketing studies focused on optimizing use of vaccines

Studies of post-infection vaccination protocols can be performed even during an evolving pandemic and are needed to understand the biology and durability of the immune response. Policy changes regarding trials after an EUA is granted are needed to make such trials possible in a global setting. This experience suggests that there is a post-EUA role for regulatory agencies to confirm that implementation studies are supported. Addressing this challenge might require the involvement and collaboration of regulatory authorities such as the FDA, the European Medicines Agency (EMA), and other international counterparts and should create incentives or requirements for the post-approval (or post-EUA) tailoring of vaccination protocols for special populations.

Lesson 3: Without primary data, there is an increasing need for data from randomized trials to inform vaccination recommendations

Recent CDC data focusing on children and adolescents suggest that as of February 2022, approximately 75% of children had serologic evidence of previous infection with SARS-CoV-2 and approximately one-third became newly seropositive since December 2021. However, our understanding of reinfections and immunologic escape as well as the durability and incremental benefit of mRNA vaccines against SARS-CoV-2 following hospitalization for COVID-19 remains incomplete. For example, the protection from hospitalization or death caused by reinfection could be significantly higher regardless of variant. Also, the anti-spike protein response appears to vary depending on the variant of SARS-CoV-2, with Omicron resulting in reduced immunogenicity. , Without primary data, retrospective reports on hybrid immunity have been unable to address the questions regarding the optimal vaccination protocols and timing of potential vaccination among individuals who are recovering from COVID-19. These studies should be pursued even if they are outside the immediate interests of the manufacturer in the setting of an EUA.

Lesson 4: The window to perform post-market or post-EUA evaluation of vaccines during a pandemic is exceedingly narrow

As the perceived standard of care evolves, there is a need for prospective data that address questions on vaccine protocols. Other forms of clinical investigation can provide extremely valuable insights but cannot replace randomized clinical trials. However, the window of opportunity for post-market randomized trials is very narrow, and the lack of such trials results in a vacuum that is then filled with guidelines and policies based on expert opinion and minimal data. Performing randomized trials during pandemic conditions, especially while broad vaccine campaigns are under way, is extremely challenging and requires a pre-existing infrastructure. Before the upheaval of a pandemic, establishing cohorts of individuals willing to participate in post-market or post-EUA research should be considered. Once in a pandemic emergency, use of observational data to mimic clinical trials may be a powerful tool but cannot substitute the completeness of data provided from randomized clinical trials. For example, data from the UK showed that for both Delta and Omicron variants, prior infection gave protection against death both in vaccinated (HR 0 · 47 [0 · 32–0·68]) and unvaccinated (0 · 18 [0 · 06–0·57]) cases and that many individuals potentially do not develop anti-spike antibodies. However, there are no randomized data on the additional benefit from post-infection vaccination or boosters. Investing in an infrastructure that can be rapidly implemented would be of considerable value. Such infrastructure requires close collaboration between pharma and international groups of investigator networks and incentives by regulatory agencies.

Lesson 5: Policies guiding the optimization of vaccination protocols through prospective randomized clinical trials should be reconsidered

The challenges in implementing VATICO reveal that even for a well-resourced group of clinical trialists, numerous barriers exist that make developing evidence-based recommendations for vaccination using gold-standard randomized controlled trials difficult after a vaccine has been given EUA status. In the medium and long term. these studies will provide the necessary context for a more nuanced approach to vaccination. Along with the other changes, our approach to thinking about trial design needs to evolve. Although clinical endpoints provide the most convincing information, they require a substantially larger sample size and viral surveillance of the population. In this regard, surrogate endpoints can be valuable, but they should be interpreted within the context of changing knowledge base and changing virus. For example, we now know that Omicron breakthrough infections induce overall higher neutralization titers against all different variants of concern and BA.2.12.1 and BA.4/BA.5 can substantially escape NAbs induced by both vaccination and infection. , Investigators will need to be prepared to re-evaluate study samples and to select an adaptive design that could allow for re-evaluation of surrogate markers.

The road forward

With more than 500 million individuals now survivors of COVID-19 infection, a data-based vaccination protocol is needed for individuals with previous COVID-19 infection and especially for those who required hospitalization and thus may have a higher risk for severe disease in the setting of another SARS-CoV-2 infection. However, in the initial vaccine trials, assessment of efficacy was restricted to volunteers who were seronegative for anti-spike antibodies at baseline. Evaluating post-infection vaccination protocols would require larger initial trials of vaccine effectiveness that would delay the availability of data on clinical efficacy of vaccines.

After vaccination trials that focus on the initial effectiveness of vaccines, there is limited incentive for manufacturers to sponsor trials that focus on optimizing vaccination protocols in the post-infection setting. As detailed in Table 1 , post-approval and post-EUA trials face several challenges and the lessons learned from trying to conduct the VATICO trial identify a number of areas for improvement for investigators, policy and guideline-setting organizations, vaccine manufacturers, and health providers.

Table 1

Challenges and areas for improvement

Investigators• Site Participation	• Hospital-based research teams are overwhelmed with the clinical responsibilities. • Only a few sites have both an inpatient and outpatient study team. • Most sites cannot enroll groups such as pregnant persons, pediatrics, or individuals at long term care facilities and nursing homes.
Policy and guideline-setting organizations• Rapidly changing vaccination guidelines	• Different countries have different vaccination and treatment guidelines. • Guidelines and definitions changed rapidly over time, including recommendations for timing of doses following infection or Nab therapy, additional vaccine doses, and defining immunization. • Requirements for travel, employer policies, and other regulations varied from some national and international guidelines.
Regulators and vaccine manufacturers• Vaccine availability and regulatory framework that provides incentives	• International vaccine availability was highly variable. • International vaccine distribution was more challenging than distribution of therapeutics. • Manufacturers seemingly lost motivation, especially after the Emergency Use Authorization (EUA) in the US was issued.
Study design• Challenges with serial trials • Surrogate outcomes cannot substitute clinical endpoints	• The requirements of an outpatient study involving additional visits and blood sampling caries additional challenges. • The trial design selected for patients who had recovered from their illness and may influence the composition of the study population. • Clinical endpoints require large number of participants and long follow up. • Surrogate endpoints can be useful but may need to be adjusted during the study at pre-specified points in the trial.
Educating the public and potential study participants• Vaccination beliefs	• Anti-vax sentiment was strong even in this group of individuals who had a COVID-19 infection that required admission. • The negative sentiment was specific to vaccines. These potential participants were already enrolled in a study that evaluated therapeutics and had established a level of trust and a strong line of communication with the trial team. • Some participants declined due to confusion from the different policies and guidelines and did not want to delay receiving vaccine or to give up the opportunity to receive two doses of vaccine.
Communicating with providers and other health care professionals• Outdated opinions about vaccination and treatment protocols	• Providers were often misinformed on the existing guidelines, and provided incorrect, partially correct, or outdated information to patients when they asked them for advice regarding study participation.

We are in the third year of the pandemic, yet significant questions remain on the optimal approach for post-infection vaccination. As the VATICO experience reveals, studies that could answer these questions face a series of regulatory and practical challenges that only increase as vaccines become widely available. As detailed in Table 2 , there is a need for investment in clinical trial infrastructure, coordinating policy and guideline-setting organizations, incentives for vaccine manufacturers, innovative study design, education of the public and health professionals, and the use of adaptive trials and all different forms of clinical investigation, including an international post-market registry for vaccines. Potential solutions need to include incentives to vaccine manufacturers and the investment in infrastructure for the post-market evaluation of vaccines. Coordinating guidelines and policies and the effective communication of these vaccination protocols to the public, health practitioners, pharmacists, and other health care professionals would not only motivate participation in clinical trials but might also help with vaccine acceptance overall.

Table 2

Proposed solutions

Investment in clinical trial infrastructure	Ongoing investment in the clinical infrastructure to support randomized controlled trials is required. Such infrastructure should be based on appropriate regulations and incentives that will foster a close collaboration between pharma and international groups of investigator networks. This investment is necessary in order to create the infrastructure that can address ongoing questions and scale up as needed.
Coordinating policy and guideline-setting organizations	The polyphony of guidelines and vaccination policies not only challenges the conduct of clinical trials, but also confuses the public. There is a need for clear, data-based, uniform guidelines with clear outline of the gaps in knowledge.We are moving to an era where vaccination policies vary based on immune suppression, age, or comorbid conditions. Guidelines and policies can benefit from embracing the need for “personalized” vaccination protocols that could help vaccine update overall.
Incentivize vaccine manufacturers	Increased collaboration and cooperation among national and international regulatory bodies and vaccine manufacturers are needed in order to incentivize or mandate post-market vaccine studies.
Study design using and adaptive approach	Willingness to update and adjust surrogate endpoints, using adaptive or a 2 x 2 factorial design could decrease the need for prolonged follow up and allow for more realistic numbers of trial participants.
Educating the public by communicating limitations of our knowledge	Communication of the vaccination guidelines in a simple but scientifically accurate manner that includes an appraisal of knowledge gaps could help individuals understand the need for further study of vaccines and prepare them to participate in studies and follow future changes in vaccination policies.
Effective communication with providers, pharmacists, and other stakeholders	Changes in guidelines and vaccination policies need to be communicated to care providers in an effective and timely manner. A closer collaboration between agencies such as the CDC and professional organizations could facilitate communication. One consideration could be a centralized website housed under national agencies (such as the CDC or the FDA) with clear schemas and “web chat” capabilities could help clarify guidance.
International post market registry for vaccines	Registries can provide a more practical approach for the collection of data on the safety and efficacy of vaccines among specific populations.

The need for expedited initial availability of vaccines (such as the timely implementation of booster vaccines that include an Omicron BA.4/5 component), combined with the changes in the epidemiology of the disease, result in a clear need for the post-approval (or post-EUA) evaluation of optimal vaccination strategies in clinical trials. We envisage a scalable international clinical trials collaboration between vaccine manufacturers and investigators that can quickly optimize vaccination protocols. Increased collaboration and cooperation among national bodies and industry, along with ongoing investment in the infrastructure to support randomized controlled trials are required to overcome the existing barriers to acquiring the data that will identify the optimal vaccination protocols for different patient populations, including those with post-SARS-CoV-2 infection or hybrid immunity.