Clinical Evidence Generation During a Pandemic: Lessons Learned for Sustaining Progress.

ABSTRACT: Because of significant adaptations forced by the COVID-19 pandemic, resultant changes within health care delivery and clinical research introduced the potential for evaluation of novel evidence generation approaches in oncology. On July 26 and 27, 2021, the National Academies of Science, Engineering, and Medicine, National Cancer Policy Forum hosted a virtual workshop entitled "Cancer Care and Cancer Research in the Context of the COVID-19 Pandemic: A Workshop on Lessons Learned." This workshop examined changes in cancer care and cancer research that occurred in response to the COVID-19 pandemic and considered lessons learned from that experience. The goal was to identify what changes could improve the delivery of high-quality cancer care and the conduct of cancer clinical trials in the postpandemic era, with an emphasis on health equity. How can we sustain the valuable lessons learned that might accelerate progress and enhance clinical evidence generation for patients and clinicians? In this overview, we discuss ways in which the COVID-19 experience has catalyzed research efficiencies as well as fostered a broader array of trial design and research methods that may facilitate improved cancer drug development during the pandemic and beyond.

The COVID-19 pandemic continues to force societal adaptations at the individual, community, national, and global levels. Clinical research has also needed to adapt, and the drug development paradigm is being actively reexamined to accelerate discoveries intended to protect public health and mitigate disease. Rapid clinical evidence generation during the pandemic has been essential, requiring new approaches to achieve what can only be seen as remarkable accomplishments, amid challenging circumstances. Scientists, regulators, clinicians, manufacturers, public health professionals, and patients have united in a common goal to not only develop preventive and therapeutic treatments for COVID-19, but also maintain drug development, including clinical trials, for patients with cancer and other areas of high clinical need. Innovations and modifications implemented during the pandemic create an opportunity to reflect on what can be done to efficiently improve patient-centered drug development postpandemic.

MAXIMIZING OPERATIONAL EFFICIENCY

Various approaches to evidence generation were harnessed during the pandemic, and operational efficiency was pivotal in the race for effective COVID-19 therapeutics and vaccines. Along with rapid progress from sequence identification of the causative virus (severe acute respiratory syndrome coronavirus 2 [SARS-CoV-2]) in January 2020 to clinical study initiation approximately 3 months later, the speed necessary to provide public access to a vaccine was supported by accelerated clinical development programs that culminated in emergency use authorization from the US Food and Drug Administration (FDA) within the same year. Despite principal differences between prophylactic vaccines and oncology therapies, there are potential lessons gained from this pandemic experience and parallels that might aid in clinical cancer research.

During the workshop, the development of the mRNA-1273 SARS-CoV-2 vaccine was discussed as one example of vaccine development founded on a strong scientific understanding of coronavirus immunobiology and scalable mRNA-based therapeutics, combined with open communication, rapid information flow, prompt decision-making, and risk management. Development was further facilitated by frequent dialog with regulators who provided clarity on acceptable endpoints and statistical assumptions, as well as frequent, timely information sharing to support and optimize vaccine development while ensuring safety and efficacy.

Safety and data quality are foremost in any clinical study; however, prophylactic vaccine trials follow particularly stringent safety guidelines, as vaccines are administered to a large number of healthy individuals. Robust safety oversight was essential as clinical vaccine studies were rapidly executed with several layers of monitoring and oversight including the investigator, the medical monitor, the protocol safety review team, and the Data Safety and Monitoring Board. The collective sense of urgency alongside a common goal of developing a safe, effective vaccine helped focus and economize time and resources, including preparing for the next phase of development in parallel with the ongoing phase. In addition, a strong understanding of the SARS-CoV-2 biology and past experience with other investigational mRNA vaccines helped clarify requirements and streamline decisions.

Essential to expeditious vaccine development was rapid enrollment of phase III trials, in part due to the desire of many members of the public to contribute to the development of a safe and effective COVID-19 vaccine. The result of this commitment on behalf of the global community coupled with prioritization of trial equity as an integral element of development resulted in a more diverse and inclusive trial population that better represents the US population at risk of COVID-19 disease. By making a concerted effort to educate trial personnel and stakeholders regarding the importance of diversity and inclusion, the clinical development of COVID-19 vaccines is a leading example of a patient-focused research approach. A diverse trial population can be facilitated when organizers participate in meaningful dialog with community leaders regarding trial protocol design and enrollment demographic metrics. Close partnership with trusted community organizations and experts can provide outreach to potential trial participants. The COVE study[1] focused specifically on such efforts that enabled enrollment of a broad demographic of participants and ensured that the population was highly representative of the US population, including more than 6000 Hispanic participants; more than 3000 Black or African American participants; a high representation of older participants, with 25% of the participants 65 years or older; and a notable representation of individuals whose work environments increased risk of COVID-19 exposure and infection. The trial also included an allowance for individuals at high risk who developed severe COVID-19, as they were permitted to access investigational treatments such as antivirals or monoclonal antibodies. These efforts helped position this pivotal study as representative of the US population at risk of COVID-19 disease.

In summary, the rapid progress of COVID-19 vaccine development across both clinical and regulatory settings showcases a number of opportunities applicable to future oncology drug development. Indeed, the benefits of frequent, open communication and information sharing across stakeholders, including government, academia, and community partners, as well as clarity from regulators, could be applied to the clinical cancer drug development paradigm. Speed and agility of trial initiation together with the collective sense of necessity underpinned by robust science may benefit clinical development of novel cancer therapies. Furthermore, practical and meaningful efforts to enroll a diverse population can lead to greater representativeness and advance health equity in clinical research; this can and should be applied to cancer drug development.

SIMPLIFYING TRIAL DESIGN

Adaptations to sustain clinical research during the pandemic have forced reflection on the need for broader changes in the drug development paradigm at the design phase, including the design and conduct of clinical trials as outlined by a recent report from the National Academies.[2] There are opportunities to construct clinical trials that are more innovative, reliable, and efficient by focusing on purposeful trial design. Most of these changes are not completely new, but clinical drug development is costly, leading to a culture of risk aversion and historical resistance to change.

The pandemic produced a compelling need to implement fundamental approaches that have been a part of the fabric of the clinical trials enterprise for decades, but often have been eschewed in favor of more labor-intensive and complex approaches as the clinical trials enterprise has evolved. The principles of efficient trial design are elucidated in detail in the Quality by Design resources extensively documented by the Clinical Trials Transformation Initiative, a public-private partnership that includes more than 80 stakeholders across the clinical trial enterprise.[3] When quality is defined as “absence of errors that matter,” it becomes clear that many of the design elements of current clinical trials are not essential to the primary purpose of the trial.[4]

The Randomized Evaluation of COVID-19 Therapy (RECOVERY) Trial in the United Kingdom epitomizes this approach. By focusing on questions of broad importance to patients, clinicians, and public health practice, eliminating unnecessary data collection and procedures, and embedding the platform for a series of trials within the clinical workflow, the RECOVERY Trial has reeled off a series of vital high-quality results.[5,6] The key elements that have led to the success of RECOVERY and its continuing contribution to new knowledge about therapeutics span almost every aspect of trial design.

There is ample evidence that protocols have become more complex over time and that a conscious effort to simplify will make participation more feasible for investigators, particularly when trials are conducted in the context of clinical care.[7] When designing trials, an important step is scrutinizing the data collection plan and ensuring that all data have a meaningful purpose that justifies the time, effort, and cost of data collection and cleaning. There is a balance between the value of additional data and the impact of the resource commitments that ultimately can lead to slower enrollment and smaller sample sizes.

Another key opportunity is the use of designs that produce efficiency by using common elements across a series of trials that address similar questions about a disease. These approaches include the use of adaptive methods, common protocols, and common control arms, as well as platform and basket trial designs. These strategies can be used across trials to reduce redundancy and optimize the number of questions that can be answered within a given trial by including various cohorts. Such strategies have been used by the platform trial, Randomized, Embedded, Multi-factorial, Adaptive Platform Trial for Community-Acquired Pneumonia (REMAP-CAP), which has focused in particular on patients with severe COVID-19. Similar to RECOVERY, REMAP-CAP has produced several important findings.[8-14] REMAP-CAP has also integrated with other platform trials under a “multiplatform randomized” design, to speed up knowledge generation.[11,12] These multiplatform designs have spurred innovative solutions around integration of study procedures and oversight across disparate clinical coordination centers. The use of embedded trials and registry-based trials could also add significant efficiency. If the trial question pertains directly to issues in clinical practice, and the needed data are already part of the electronic health record (EHR), embedding the trial in practice can dramatically reduce additional clinician effort required to perform per-protocol assessments. Such strategies have been instrumental for the success of RECOVERY and REMAP-CAP trials. Although less prevalent in cancer clinical trials, the registry-based trial is becoming more of a standard tool in cardiovascular trials.[15] Whether designing a platform trial or a new registry, there must be a belief that the extra work needed to configure the system and develop the required infrastructure leads to benefits in the ability to make trial adaptations, conduct patient centric studies, and add new research objectives more rapidly and efficiently.

Pragmatic trials (PCTs) can incorporate many of the aforementioned elements such as embedding or decentralizing aspects within the design as exemplified by the PRECIS criteria.[16] In addition, a comparison of the efficiency of the vaccine development programs with the disarray across COVID-19 therapeutic clinical trials emphasizes the importance of identifying the most salient questions and specifically focusing collaborative efforts across sectors. Despite enormous energy and volunteerism, a disappointingly high proportion of trials initiated to test COVID-19 therapeutics were not designed to answer a meaningful question.[17] Had this energy been channeled into enrolling participants in more relevant trials and had they been conducted efficiently, the time to derivation of clinically meaningful results would have been much shorter.

INTEGRATING DECENTRALIZED APPROACHES

Patient-focused drug development seeks to take patients into account throughout the development continuum. The core of clinical drug development is the rigorously designed and conducted randomized controlled trial (RCT), and it is incumbent upon the scientific community to identify ways to make clinical trials more efficient and patient-centric. There has long been an interest in reducing the burden of trial participation on patients and their caregivers by “decentralizing” clinical trials, but the use of remote assessments and other decentralized trial methods has been uncommon, particularly in oncology.

The pandemic necessitated rapid modification of clinical trials to mitigate patient exposure to COVID-19 by reducing travel to clinical trial sites. Remote assessments and other decentralized procedures were rapidly deployed in multicenter trials. As the pandemic continued, these procedures became more commonplace, and COVID-19 provided a clear proof-of-concept for the ability to conduct hybrid decentralized clinical trials (DCTs) in oncology. The question is no longer whether we can conduct aspects of cancer trials in a decentralized fashion, but how we can best prioritize the most useful DCT methods moving forward, and prospectively design DCT studies in oncology that maintain patient safety and data integrity.

The COVID-19 response provided valuable insight into the conduct of remote trials. To evaluate methodological challenges, some sponsors have started designing solutions and flexible approaches to DCT conduct that, once implemented, can be continuously monitored and refined. This approach enables a strong learning feedback loop: new data are continuously collected and used to recalibrate signals and measure ongoing activities. Developing a continuous process improvement approach requires an efficient network of collaboration, with optimal communication among research and clinical teams, central coordinators, local providers, and patients. Success requires clear objectives, adherence to quality standards, and effective implementation—all supported by rigorous use of digital health technology.

To advance the appropriate use of DCT methods beyond the pandemic, lessons learned should be extensively applied in areas such as remote monitoring of clinical trial sites, improved access to EHR systems, optimized virtual clinical follow-up visits (telephone or video call) with secure data collection, and enhanced central monitoring capabilities. When designing a DCT, the use of blended aspects of trial activities may be appropriate, where certain activities can be safely done remotely, whereas other activities may require a visit to the research site to ensure safety or study integrity, such as in a hybrid decentralized approach. Difficulty accessing clinical sites for FDA inspections can be mitigated by developing sound approaches to digital site inspections, where appropriate. In addition to remote assessments, trial flexibilities could include allowing trial participants to visit alternative clinical centers that may be conducting the same study. Finally, operating procedures should be developed to enable patient-centric efficiencies including telemedicine, transportation to treatment sites, and direct-to-patient medication shipments. These enhancements to improve access may encourage patients residing farther from clinical sites to participate in trials, potentially increasing the representativeness of trial populations and furthering health equity.

In addition to improvements in clinical trial conduct, important learnings have been gleaned from the COVID-19 response to clinical cancer care. Although not new, telemedicine in oncology had been infrequently deployed for various reasons, and COVID-19 forced accelerated adoption of telemedicine for routine follow-up visits. In addition, health care providers explored alternative treatment administration options that included preference for oral formulations, pharmacy shipping of therapies direct-to-patient or home-based or outsourced care models, and newly approved dosing regimens that can increase the interval between doses of infused medicines.[18] All of these measures were done to protect patients from COVID-19 exposure related to travel to health care facilities and were particularly useful for routine follow-up visits and care of patients in remote or rural areas. These measures are important as new therapies and available treatment modalities continue to increase the population of cancer survivors.

The COVID-19 pandemic has provided a unique opportunity to rethink cancer clinical trial conduct.[19] The response to COVID-19 in clinical trials and health care delivery was a remarkable example of local adaptation and effective, rapid system evolution. There is still much to learn, and a data-driven approach to characterize any effects that DCT remote assessments may have on data variability will be important. The ability to identify whether an assessment was performed remotely or at a clinical site within trial datasets would facilitate this effort. Moving forward, flagging remote assessments at the case-report form level could allow analyses to unlock further information on potential effects of DCT modifications on data variability that can be targeted for process improvement or other mitigation strategies. Whether in the trial or health care setting, continuous quality improvement will be necessary to advance remote assessments and realize the goal of patient-focused cancer care and research.

FOSTERING RIGOROUS USE OF REAL-WORLD DATA

Although it is accepted that traditional RCTs are the gold standard for generation of high-quality clinical evidence of therapeutic effect, real-world data (RWD) may have a role in clinical evidence generation when used appropriately. Real-world data can be defined as data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources. Real-world evidence (RWE) is the clinical evidence about use and potential benefits and risks of a therapeutic intervention derived from RWD.[20] Regardless of whether data are derived from an RCT or RWD, generating high-quality evidence requires careful a priori study design and an appropriate analysis plan. While RWD studies frequently use retrospective observational approaches, it is important to note that design strategies including randomized prospective PCT designs can also incorporate RWD sources.

The pandemic catalyzed unparalleled large-scale, multistakeholder collaborations that rapidly generated RWD from EHRs, administrative health care claims, and registries. Several instrumental efforts focused on RWD generation to understand COVID-19 in oncology including registries such as the COVID-19 and Cancer Consortium, the American Society of Clinical Oncology Survey on COVID-19 in Oncology Registry, and the NCI COVID-19 in Cancer Patients Study, as well as broader methodological efforts within the Reagan-Udall Foundation for FDA Evidence Accelerator, among others.[21-23] These registries were designed to include cancer survivors and patients with active cancer with a positive SARS-CoV-2 test or clinical evidence of COVID-19. Clinical data collection included patient risk factors, symptoms, severity interventions related to COVID-19, and clinical outcomes, as well as cancer type and stage, and anticancer therapies. The RWD collected in the registries produced several valuable insights around early characterization of COVID-19, cancer types and risk factors for poor outcomes, COVID-19 treatment utilization and clinical outcomes, and disparities in medication access among patients with cancer and COVID-19.[22,24,25]

The use of RWD informed and guided the pandemic response in a rapidly evolving environment; however, there is a significant distinction between use of RWD for descriptive purposes and using RWD to generate the evidence necessary to attribute a positive or negative outcome to a therapeutic intervention (causal inference). Demonstrating substantial evidence of treatment effect using reliable, relevant RWD requires fit-for-purpose data and rigorous application of epidemiology principles, study design methods, and prospectively planned statistical analysis to address common challenges with RWD including incomplete capture and missing data.[26,27] Indeed, the difference between RWD and RWE was highlighted by several instances of poor-quality COVID-19 RWD studies that were published and subsequently considered to have spurious associations or disseminated without proper peer review. The pandemic provides the field with a clear lesson that the benefit of RWD availability and faster analytics must be balanced with the importance of careful design and methods to create the high-quality RWE necessary to support evidence-based intervention strategies.[28]

Data quality is a key challenge facing RWD, and it requires establishing greater certainty around the reliability of RWD to generate decisional evidence[29] for regulatory, clinical, and patient decision-making. Whether RWD is prospectively or retrospectively collected, it is imperative to develop a study protocol and statistical analysis plan prior to study initiation, including prespecified study objectives, methods, and statistical approaches. Considerations include defining the relevant patient population, determining clinical covariates (including potential confounding variables), controlling for biases using methodological approaches, defining endpoints, specifying the handling of missing data, and ensuring the sample size is sufficiently large to address the research question(s). All methods for data collection, RCT or RWD, must address missingness and evaluate measurement issues to ensure data reliability and validity. The appropriate use of RWD requires analytical knowledge and understanding of data provenance, quality, and context. Therefore, RWD should only be used when the data are fit-for-purpose to answer the study question, and analytic methods that account for potential biases are properly conducted following Good Pharmacoepidemiology Practice.[30]

In advancing the use of RWD to generate RWE, there is a need to focus on data quality, relevance, reliability, integrity, reproducibility, and rigorous methodology, from study design through the analysis and interpretation of the data. Although RWD sources have limitations compared with highly controlled traditional clinical trials, there are steps that can be taken to optimize the ability to make valid inferences from registry and other RWD sources. An important, often overlooked, consideration is selection of a prospective randomized design; RWD need not be retrospective. Prospective studies that utilize RWD can maintain some essential core trial elements necessary to mitigate bias, including randomization. As previously mentioned, some of the most important findings guiding the evaluation of effective COVID-19 therapies were based on the use of randomization in the setting of routine clinical practice with streamlined data collection through PCTs.

The pandemic has accelerated our understanding of how we might use RWD to further public health response. In order to expand the use of RWD to generate RWE that can inform cancer drug development, collaborative efforts are needed among stakeholders to advance standardization across many areas of RWD including data quality, methods, interoperability, and transparency.[31] The use of prospective designs that are embedded in registry, EHR, or other RWD sources is particularly promising, given the ability to implement randomization while providing potential efficiencies for patients, clinical investigators, and trial sponsors. The lines are blurring between traditional clinical trials, DCTs, and prospective randomized trials using RWD (e.g., PCTs), and each can have an important role depending on the trial objectives and context. Trial designs are not mutually exclusive, and current approaches may incorporate hybrid elements across designs. By making trials less burdensome to patients, we may improve accrual and retention and reach a more diverse population, expanding our ability to identify therapeutic risks and benefits while realizing efficiencies that may lead to lower clinical research costs. Integration of operational efficiencies by moving research into clinical care, innovation in trial design, use of decentralized approaches, and advancing use of RWD can facilitate moving toward a patient-centered learning health care system that provides purposeful, high-quality clinical evidence generation without compromising data integrity, scientific standards, or patient safety.

CREATING SUSTAINABLE CHANGE

Amid unparalleled public health and global humanitarian challenges, the COVID-19 pandemic has been a catalyst for innovation and change in the scientific, patient care, and regulatory communities. The challenges that were faced during the pandemic can be summarized in 2 major categories. The first is the collective hurdles that prevented patients from reaching the point of care and hindered participation in cancer clinical trials including the inability to collect follow-up information on already enrolled patients. Factors contributing to these hurdles include, but are not limited to, local and national public health guidance, the closure of facilities or services, staffing challenges, and prioritization of care in health care institutions. The second category of challenges was the urgent need to develop preventive and therapeutic measures to combat a severe infectious disease under operational conditions that must be designed to accomplish such a task rapidly. Challenges included integrating appropriate trial designs, streamlining modes of operation, overcoming institutional regulations and bureaucracy, and navigating regulatory guidance, among others. In short, existing health care and clinical trial systems had not been designed to meet such challenges.

To meet these challenges, academic and health care institutions, funding and regulatory governmental agencies, and biotechnology and pharmaceutical companies had to collaboratively adapt in a short time, pressure testing a modified system at its fulcrum out of necessity. Fortuitously, this rapid adaptation led to the implementation of an array of innovative solutions with unprecedented speed and agility to support execution of clinical trials and efficient generation of vital new knowledge. The positive impact of such change was obvious, including the development of vaccines from inception to market in less than a year, along with the development of novel and repurposed COVID-19 treatments in record times—all while maintaining progress in drug development in other therapeutic areas, including oncology.[32-34]

Moving forward, we must seize the opportunity to extract as much as we can learn from clinical trial modifications and RWD collected and analyzed during the pandemic (Fig. 1). Identifying changes and modifications to the system that were successful will be necessary to continue adopting them in the appropriate context, thereby sustaining the efficiencies demonstrated during the pandemic. Such adaptations include continued collaboration across the clinical trial enterprise, use of telehealth and remote assessments to facilitate both trial conduct and health care delivery, reduction of unnecessary and excess trial procedures where appropriate, and removal of trial start-up inefficiencies. Expanding high-quality data sources through development of registries and improvements in EHRs to facilitate prospective collection of RWD are just a few areas of opportunity. Improving the efficiency and generalizability of evidence generation through these measures can facilitate scientific research and expand clinical trial access as well as advance repurposing of drugs, while including a broader, more diverse patient population toward our goal of achieving true health equity. The pandemic can become a watershed moment to forge unprecedented change in health care delivery and clinical trials through a dynamic modernization effort supported by adequate infrastructure funding and carefully coordinated global stakeholder collaboration.

FIGURE 1

Lessons learned from evidence generation during a pandemic.