Real-World Data for Regulatory Decision Making: Challenges and Possible Solutions for Europe.

Real‐world data (RWD) offers the possibility to derive novel insights on the use and performance of medicines in everyday clinical use, complementing rather than competing with evidence from randomized control trials. While Europe is rich in healthcare data, its heterogeneous nature brings operational, technical, and methodological challenges. We present a number of potential solutions to address the full spectrum of regulatory use cases and emphasize the importance of early planning of data collection.

There is increasing interest in the use of real‐world data (RWD) to support regulatory decision making across the product life cycle. Key sources of RWD are electronic health records, claims data, prescription data, and patient registries. Increasingly incorporated into the definition is data from wearables, m‐health apps, and environmental data including data on social status, education, and other lifestyle factors. These latter data offer much promise to deliver a holistic picture of an individual's health status but from a regulatory standpoint present substantial challenges in deriving actionable evidence. From the perspective of the European Medicines Agency (EMA), RWD are defined as “routinely collected data relating to a patient's health status or the delivery of health care from a variety of sources other than traditional clinical trials.” We specifically exclude traditional clinical trials even if single arm but would incorporate data from pragmatic clinical trials if data were collected remotely through an electronic health record or other observational data source and solely under conditions of normal clinical care.1 Real‐world evidence (RWE) is then defined as the information derived from analysis of RWD, and it is the acceptability of this evidence for regulatory decision making in different use cases across the product life that has become the subject of intense debate.

The use of RWD to support regulatory decision making is not new. For decades such data have been used for safety signal evaluation, risk management and for studies to support life cycle benefit‐risk evaluation;2, 3 contexts where opportunities to capture data, especially in a timely fashion, are more limited and where multiple sources of information of varying quality from multiple stakeholders must be balanced to inform decision making. In fact, for pharmacovigilance decisions, it could be argued that it is essential that safety is understood in the context of how care is delivered rather than under the stringent and highly monitored conditions of the clinical trial. To directly support EMA committees, the EMA is routinely using three real‐world databases for in‐house studies and over recent years has commissioned 15 external studies, most of them multidatabase and multinational. It is also well recognized that RWD are an underutilized source for assessing the public health impact of risk minimization measures, including any unintended consequences4 and for informing health technology assessment, pricing, and reimbursement decisions.5 The natural extension to these safety‐orientated questions includes disease characterization and prevalence, understanding current standard of care, and confirming the clinical outcome of short term surrogate markers.

To date, however, there is a lower acceptability of RWD where the outcome of interest is efficacy/effectiveness.6 Great caution is generally exercised where positive regulatory decisions result in patients being exposed to a new medical product, and hence an estimate of efficacy free from the biases of observational data is required.7 The best available standard of evidence to date has been the randomized control trial (RCT). The RCT will, in our view, remain the best available standard and be required in many circumstances, but the rapid pace of change in the scientific and technological landscapes is shifting the regulatory landscape. We are seeing an increasing number of products that face challenges to align with the traditional drug development pathway; often these are advanced therapies or orphan products for conditions with significant unmet need and for which a traditional RCT may be unfeasible or unethical. Table provides recent examples where RWE has been pivotal for European regulatory decisions in terms of supporting the initial regulatory decision or postmarketing obligations. For many of these examples, the need was to enable both safe and early access to promising medicines for patients with limited treatment options or when uncertainties around the medicines remained. Where sufficient efficacy is demonstrated but uncertainties exist around long‐term safety and efficacy (Strimvelis, nusinersen (Spinraza)), postauthorization evidence generation coupled with adequate pharmacovigilance activities needs to be in place to quickly address uncertainties. However, where available evidence of efficacy requires contextualization, there have been examples where RWD provided an external control arm (Zalmoxis), were used to confirm a response rate in a single‐arm trial (axicabtagene ciloleucel (Yescarta), tisagenlecleucel (Kymirah)) or provided data to extend an indication (eculizumab (Soliris)). As personalized medicine becomes a closer reality, it is anticipated that such examples are likely to increase.

Operational, Technical, and Methodological Challenges

From a European perspective, utilizing RWD is faced with operational, technical, and methodological challenges, but possible solutions exist (Table  1). Operational challenges include feasibility, governance, and sustainability issues, which complicate access to and the routine use of multiple national data sources, many of which will have different legal and ethical requirements for sharing data. As a minimum, appropriate consents and data anonymization techniques are required to ensure data privacy obligation requirements are met; while of paramount importance, current operational processes designed to address obligations may prevent efficient and timely delivery of data, which may be particularly problematic in the context of safety decisions where urgent access to data is needed to inform a regulatory decision.

Table 1

OPerational, TechnIcal, and MethodologicAL framework (OPTIMAL) for regulatory use of real‐world evidence (RWE)

Objective	Desired criteria for acceptability of RWE	Challenges with use of RWD to generate acceptable RWE	Possible solutions (EU context)
Appropriate use of valid RWE for regulatory purposes (e.g. safety, efficacy, benefit–risk monitoring)	Evidence is: Derived from data source of demonstrated good quality Valid (internal and external validity) Consistent (across countries/data sources) Adequate (e.g., precision, adequate range of characteristics of population covered, dose and duration of treatment, length of follow‐up)	Operational Feasibility (e.g., data access and cost, availability of relevant data needed, data protection, patients’ consent, availability of hospital data source) Governance (e.g., data‐sharing policy, transparency, policy towards funding source) Sustainability (sustained data collection and analysis)	Operational Early and repeated consideration of the need for RWD during drug development Landscaping of potential data sources Long‐term funding for data infrastructures Published documentation of data source characteristics and policy for collaboration and data sharing Management of access in line with European Union General Data Protection Regulation and national legislation Data anonymization processes where required Data sharing agreements at study inception Use of ENCePP Code of Conduct
		Technical Extent of data collected on clinical outcomes, exposure, and individuals Collection of adequate time elements Data completeness (missing data) Consistent use of appropriate terminologies and data formats Potential for data linkage Consistent, accurate, and timely data collection, recording, and management	Technical Use of common data elements, data formats and terminologies, or mapping to international system Partial or full data mapping to CDM, including routine validation process Quality assurance and control procedures—indicators of data quality Internal or external data audit Benchmarking to external data source EMA qualification procedure for data source
		Methodological Variability in results from multi–data source studies. Understanding the data source environment Adequate data collection on potential confounders (e.g., smoking, indications) and effect modifiers (e.g., drug dose, disease severity) Identifying the potential for selection bias and information bias Management of missing data Sound data analysis and interpretation	Methodological Detailed description of study design and data collected in data sources Documentation of feasibility analyses Registration of study in public database, with study protocols and results Use of best methodological standards in statistics and epidemiology Use of EMA Scientific Advice procedures for study protocols

CDM, common data model; EMA, European Medicines Agency; ENCePP,European Network for Centres of Pharmacoepidemiology and Pharmacovigilance; RWD, real‐world data.

Technological challenges describe those associated with the data, and solutions require addressing differences in terminologies, data formats, quality, and content that exist across multiple European databases. Europe is fortunate in the richness of its healthcare data and in particular its longitudinal nature, which stems from the principle of universal healthcare coverage, which remains at the heart of most European healthcare systems. However, the data are heterogeneous as differences in healthcare systems, national guidelines, and clinical practice have driven different content; a recent analysis revealed that the number of European databases that meet minimum regulatory requirements across a broad range of regulatory use cases and which are readily accessible is disappointingly low and geographically skewed to Western and Northern Europe.8 This poses problems when results from multiple datasets must be pooled in order to deliver evidence representative of the wider European population or when larger numbers are needed to explore rare diseases, events, or outcomes. Resolving differences across data sources requires agreement on common sets of data elements, data formats and terminologies, or mapping of these components to an international system. Obvious advantages of common data quality systems and common data analytics are to facilitate data exchange, data analysis, and interpretation of results arising from multiple datasets. New approaches to harmonization that involve a priori transformation of the data into a common data model (i.e., same data structure, format, and terminology) independent of any particular study have become possible in recent years due to improvements in the computational capacity to store, extract, and analyze large datasets. By enabling the use of common standardized analytics, this facilitates a consistency of approach and minimizes the need for decision making at the level of individual data sources. Within Europe, such approaches have the potential to significantly accelerate studies, but a careful characterization is required to determine whether there is loss of information or validity when EU data are transformed into a common data model and to assess any impact on downstream outputs.

Methodological challenges arise from the fundamental fact that observational data are not collected with research as their principle purpose, may be derived from different care settings, and therefore suffer from variable amounts of missing data and from multiple different biases and confounders.7 However, in all these scenarios, a significant barrier to acceptability remains concerns around the reliability and validity of the evidence generated through RWD, especially when conducted across multiple countries and databases across Europe. Even when the protocol is standardized, significant variability may remain, increasing the heterogeneity of the results.9 Such issues have long been recognized, but compliance with the best methodological standards, a detailed description of study design and data collected, and full transparency on the protocol and study report (with registration in a publicly available database) would do much to build confidence in results and avoid the confusion created by disparate results. The European Network for Centres of Pharmacoepidemiology and Pharmacovigilance (ENCePP) has developed, and updates annually, standards for pharmacoepidemiology research, and there have been multiple publications recently proposing the establishment of reporting requirements.10 Ideally such reporting would be consistent with common parameters and terminology to enable comparability and be publicly available at a single source. All studies imposed by European regulators must be registered in the European Union electronic Registry of Post Authorisation Studies (EU PAS Register), and extending this requirement to all studies would seem one obvious route.

Conclusions

The digitization of health care and, increasingly, lifestyle data bring new opportunities to complement and enhance the data traditionally utilized in regulatory decision making. The hope is that this will improve the timeliness, accuracy, and relevance of decisions across the product life cycle. Defining the exact evidentiary standards of such RWE a priori is challenging as necessary standards will vary depending on the context within which the question is asked. Given the broad range of regulatory use cases, it seems clear that a one‐size‐fits‐all approach will not be sufficient; a hybrid approach to evidence generation will be required, depending on the question being asked and the context in which the derived evidence will be used, and early planning of the strengths and limitations of the possible approaches is required. However, whatever the approach, there is a need to address operational, technical, and methodological challenges in both designing, running, and assessing a study to enhance the quality of evidence generated and the consistency of regulatory decision making. Moreover, as more data sources become available and infrastructures are developed to enable access, there is an urgent need to consider and plan for the data needs for the future. Standardizing and validating data retrospectively is expensive, time consuming, and potentially introduces errors and biases, and hence it is important to consider in advance the scope, depth, and quality of data that will be required to generate reliable evidence suitable for multiple regulatory use cases. This work requires effort from the multiple stakeholders who may potentially wish to utilize these data for decision making. With the combination of technological and scientific advances available today, there has never been a more opportune time to address this.

Funding

No funding was received for this work.

Conflict of interest

The authors declared no competing interests for this work.

Disclaimer

The views expressed in this article are the personal views of the authors and may not be understood or quoted as being made on behalf of or reflecting the position of the European Medicines Agency or one of its committees or working parties.

Table S1. Recent regulatory examples in which RWE has been utilized to support regulatory decisions either at authorization or to support an extension of indication.

Click here for additional data file.