Evidence generation and reproducibility in cell and gene therapy research: A call to action.

Main text

The emergence of new cell and gene-based therapies (CGTs) utilizing innovative technologies has recently intensified. Long-standing efforts in publicly funded biomedical research have resulted in breakthrough therapeutic approaches for patients with devastating and life-threatening diseases. Transformative gene-based therapeutic tools include human genome editing technologies, refined transposon systems, and synthetic immunoreceptors, such as chimeric antigen receptor (CAR) T cell and natural killer cell engineered immunotherapies. Cancer has been a leading disease target, with the treatment of B cell malignancies yielding compelling clinical outcomes, resulting in the regulatory approval of several CAR T cell therapies. Concurrently, intensive research on solid tumor indications is underway. Similarly, rare diseases are prominent targets for gene therapy and gene editing technologies. Founded on these scientific advances, next-generation CGTs are expected to transform into treatment options for a wider spectrum of conditions., Moreover, while these treatments, to-date, target mostly patients with advanced illnesses, future therapies may be introduced at earlier disease stages, even as primary therapeutic options. Here, we highlight some of the obstacles inherent in CGT evidence generation and research reproducibility and recommend concerted actions on how they can be overcome.

Developers, regulators, funders and payers involved in the development and delivery of next-generation CGTs need to rely on robust evidence of their benefits and risks to support decision making and ensure their translation from promising discoveries to effective therapeutics. Inadequate evidence on their comparative efficacy has led several CGTs to be withdrawn from the European Union (EU) market mainly due to inability to satisfy national reimbursement requirements. This is due, in part, to certain unique attributes of CGTs such as heterogeneity in treatment response and toxicities, targeting rare diseases with low patient accrual and lack of suitable comparators in clinical trials, and the need for long-term safety and efficacy follow up studies, among others. Importantly, the mode of action for gene therapies, in many cases, relies on introducing permanent changes to human cells and tissues, which, in turn, increases the risk of unforeseen and delayed adverse events. As a result, regulatory agencies require developers to conduct long-term patient follow-up, amounting to 15 years of observation, with the right infrastructure in place to collect longitudinal patient data, e.g., through patient registries., Additionally, CGTs are rarely readily available as “off the shelf” therapies and must be customized, leading to high development costs; thus, accessibility becomes an issue for patients and health care providers.

It is also acknowledged that pre-clinical testing of CGTs has, in some instances, limited capability for generating informative evidence. In vivo models in highly inbred, specialized mouse strains may not adequately reproduce features of the target patient population. Often, normal donor cells are employed to obtain pre-clinical evidence on the safety and efficacy of genetic engineering strategies, which may not accurately reflect later findings in treated patients. Additionally, some pre-clinical studies, such as toxicokinetics and mode of action, are technically difficult to perform for CGTs. The use of models that incorporate human cells and tissues and exhibit highly differentiated features (e.g., organoids, organs-on-chips) could be beneficial for preclinical validation efforts. For instance, patient-derived organoids that recapitulate clinically relevant features of organ pathophysiology may be useful to test delivery, predict toxicities, or assert the validity of early clinical findings of a gene therapy approach, particularly for rare genetic disorders. Transcriptomic technologies would be beneficial for in-depth investigation and validation of such organoid models. Crucially, ethical concerns associated with deployment of organoids and gene editing approaches require continuing deliberations.,

Ideally, CGT clinical trial design would consider the harmonization of outcome measures and their reporting to facilitate comparisons across studies and the pooling of data needed for statistical meta-analysis. Engaging statisticians in the earliest phases of clinical development is essential in assuring appropriate study design. Some new technologies may require preliminary studies with smaller patient cohorts to demonstrate their feasibility, e.g., phase 0/1 trials, including the observation of unexpected toxicities, identification of patient populations most likely to benefit, and an understanding of key barriers for implementation. This approach is compatible with improving trial enrollment in subsequent larger clinical studies. It is also important to ensure that socioeconomic and racial/ethnic disparities are considered in CGT trial design and patient enrollment. Patient advocates can play an active role in improving patient recruitment and retention in CGTs clinical trials.

Consistent and high-quality evidence on the health benefits of a new therapeutic modality is not only needed to justify regulatory licensing but also health insurance coverage and reimbursement decisions. CGT prices are often elevated on the basis of their high development costs and anticipated curative value as a one-time treatment. Additionally, the clinical benefits of curative therapies are associated with significant uncertainties, complicating their appraisal using traditional economic evaluation methods such as cost-effectiveness analysis, which may require methodological recalibration. Ensuring and monitoring long-term data collection through post-approval studies and surveillance as a regulatory prerequisite can help overcome this limitation. Where feasible, data should be collected and curated to facilitate access and analysis by independent investigators. These efforts could also benefit from initiatives such as the National Patient-Centered Clinical Research Network (https://pcornet.org/), the National Institutes of Health Collaboratory (https://commonfund.nih.gov/hcscollaboratory), or the European Data Analytics and Real World Interrogation Network (DARWIN) initiative to capture real world experiences. While the exploration of new alternative financing mechanisms and innovative insurance schemes for CGTs are certainly welcome, health technology assessments should be flexible enough and adapt to evidence uncertainties associated with the potential curative benefits of CGTs for serious or life threatening illnesses when no alternative therapies exist. Given all these challenges, robust scientific evidence based on reproducible and replicable research is critically needed to inform decision-making throughout the CGT life cycle.

In summary, the development of highly effective CGTs offers hope to millions of patients with severe and previously incurable diseases. However, providing an evidence base for their effective and safe deployment must be a priority. Goals for optimal product development should include: (1) avoiding marketed products being withdrawn by manufacturers or regulators due to lapses in evidence generation; (2) systematic monitoring for potentially new and/or delayed adverse events not identified during clinical research phases (especially in orphan diseases with small pre-authorization studies); and (3) limiting the instances in which post-approval real-world evidence fails to confirm therapeutic benefits. Additionally, the high upfront costs of some CGTs and their reimbursement challenges could potentially jeopardize their continued use and undermine confidence in the broader therapeutic category. In a field with such strong scientific prospects but also high degree of vulnerability due to limited clinical experience and evidence, transparency throughout research and development stages is key. Some of these issues have been raised in the International Society for Stem Cell Research (ISSCR) guidelines governing clinical translation of novel stem cell-based therapeutics intended to protect patients against false hopes or potential harms that can result from unproven stem cell interventions.

We call for increased attention to methodological improvements in pre-clinical and clinical study designs, including robust data collection and evaluation, together with the complete disclosure of protocols and publication of results, be they positive or negative. Experimental replication/validation should be achieved at the pre-clinical stage to maximize the prospects of successful clinical translation. This will require a change in culture of the scientific ecosystem, including the need to both conduct and publish the findings of meaningful attempts to reproduce results (that is, computational reproducibility from the same data) or to replicate findings through new sets of experiments, to either strengthen or challenge current evidence. Assessing both the quality of evidence and experimental replicability requires accurate reporting of the original study’s methods and a sample population that can be accessed or recreated, including access to deidentified patient data to underpin secondary analyses. Multiple stakeholders, particularly research funding agencies and academic institutions, will be required to support these efforts and provide tools and infrastructure, ultimately enabling researchers to achieve these goals. An investment in evidence generation and reproducibility of results will pay off with improved efficiency in the development and application of CGTs and will eventually save resources. The multidisciplinary approaches discussed here could strengthen evidence, reduce uncertainties, and diminish potential biases, thus enhancing clinical, regulatory, and payers decision making. This proposed blueprint for CGTs relies on rigorous research to meet pressing clinical needs while generating the societal support required for delivering these promising therapies across the globe.