Disseminating trial results: We can have both faster and better.

The average delay from clinical trial completion to publication of results is one to two years. , Welsh et al. described the different sources of delay from enrollment to publication, establishing that industry-funded trials publish results an average of six months faster than government-sponsored trials. In industry-funded trials, incentives for rapid reporting and requirements for continuous monitoring of data quality and participant safety typically force creation and maintenance of a complete analytic and reporting “pipeline” from the beginning of trial recruitment. Data are cleaned as they are created. Analytic variables are fully specified in advance and computed along the trial's journey. Primary safety and effectiveness outcomes are regularly reported to blinded monitors, following analytic plans completely specified in advance. With this infrastructure in place, primary results can be submitted for publication within days of trial completion. This continuous attention to data quality and transparency regarding data provenance and analysis help bolster scientific rigor and integrity. In a true learning health system, the delay from completing research to changing practice should shrink from years to weeks. Trials supported by other sponsors are not subject to the same financial incentives or regulatory requirements. Nevertheless, all investigators leading clinical trials, including trials of pharmacologic treatment, non-pharmacologic treatments, and health services interventions, should ask how early creation of an analytic and reporting pipeline could improve the speed and transparency of dissemination. Rapid reporting during the COVID-19 pandemic has created an opportunity to re-evaluate traditional processes and timelines for reporting trial results.

Here, we use our experience with embedded pragmatic trials supported by the National Institutes of Health (NIH) Health Care Systems Research Collaboratory (“Collaboratory”) to explore faster dissemination of results. We examine controllable and uncontrollable delays in the steps from the end of outcome data collection to dissemination (Fig. 1 ). We focus on challenges to rapid reporting in embedded trials using routinely collected health system data.

Fig. 1

Data and dissemination delays in learning health systems.

Real-time access to outcome data

Because outcome data in pragmatic trials are often extracted from health system electronic health record (EHR), claims, administrative and other sources (Fig. 1), investigators cannot control the timing and format of the clinical data collection, nor can they control the consistency of data across multiple sites. However, we found two characteristics in trials with rapid access to data sets: 1) pre-standardized data, such as those previously mapped to a common data model as part of a research network, and 2) experienced IT staff with research training.

Interoperable, research-ready data infrastructures can be built within health systems, as is commonly done with industry-sponsored trials. Health systems measure outcomes and other quality measures, and synchronization, standardization, and communication with researchers could enable opportunities to coordinate initiatives across studies and health systems. When the underlying data curation changes, alerts could flag important changes that would impact research results and (more important) alert health systems to data quality issues.

Access to the variety of data sources that inform vital status or other outcomes through claims also needs reform, as these have a data latency of over a year. There are few incentives to improve the liquidity of these data, even though these results could have a material impact on research implementation. Special access policies for covered research programs could shift the data latency and lag of results to achieve greater impact sooner for embedded research studies.

Real-time data cleaning

Industry-sponsored trials have explicit and reproducible processes for monitoring and improving data quality as well as dedicated data and analytic staff for this purpose. For embedded health system trials, real-time monitoring of data quality and integrity is less common Paradoxically, health systems are using these data every day to deliver care and make management decisions. Shifting the paradigm of embedded pragmatic trials to expect continuous data curation and cleaning would have a profound effect on the evidence generated in, pragmatic trials that support patient care and a learning health system.

Immediate data analysis

Industry-funded trials typically complete primary outcome analyses only days or weeks after first availability of outcome data, compared to several months for trials supported by other sponsors. That rapid completion depends on complete and transparent specification of primary outcomes and analytic methods long before data are available for analysis. Other trial sponsors and trial registration systems should expect that level of advance specification for all clinical trials. Rapid completion of those planned analyses will also require adequate funding to support concentrated effort by study staff during that critical period.

Rapid reporting of initial results

The dramatic increase in use of pre-print servers, such as MedArXiv, during the COVID-19 pandemic illustrates both the rewards and risks of publication prior to traditional peer review. The risk of rapidly disseminating invalid or misleading findings can be reduced by early publication of trial methods, including complete specification of study outcomes and analytic methods. Peer review prior to trial completion would focus on the rigor of trial methods rather than satisfaction with trial results. Any publication of results prior to traditional peer review could be held accountable to all methods specified in advance.

Conclusions

In an era where guidelines and clinical practice are constantly evolving, years-long delays in dissemination reduce the relevance of clinical research. Delays reduce the ability for researchers to apply trial findings to new research questions, impede clinicians from having the most up-to-date information, and perhaps most importantly, are a disservice to patients who could benefit from the information.

Cultural incentives are aligned in industry sponsored trials to favor speed: readiness for generalizing topline results is considered valuable to shareholders, and the culture encourages a system where data are liquid, available, and continuously cleaned and curated, such that topline results can be reported within a timespan of two weeks rather than two years. We argue that all clinical trials should model their processes with industry-sponsored trials. The urgency of the COVID-19 pandemic creates a window of opportunity for identifying and reforming traditional practices that contribute to delay.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.