Real world data: an opportunity to supplement existing evidence for the use of long-established medicines in health care decision making.

Evidence from medication use in the real world setting can help to extrapolate and/or augment data obtained in randomized controlled trials and establishes a broad picture of a medication's place in everyday clinical practice. By supplementing and complementing safety and efficacy data obtained in a narrowly defined (and often optimized) patient population in the clinical trial setting, real world evidence (RWE) may provide stakeholders with valuable information about the safety and effectiveness of a medication in large, heterogeneous populations. RWE is emerging as a credible information source; however, there is scope for enhancements to real world data (RWD) sources by understanding their complexities and applying the most appropriate analytical tools in order to extract relevant information. In addition to providing information for clinicians, RWE has the potential to meet the burden of evidence for regulatory considerations and may be used in approval of new indications for medications. Further understanding of RWD collection and analysis is needed if RWE is to achieve its full potential.

Background

New therapeutic products are typically approved following many years of rigorous randomized controlled trial (RCT) evaluation to ensure their efficacy and safety. The foundation of this evidence comes from carefully designed, adequately powered RCTs, which seek to evaluate the efficacy and safety of medications in a specifically enrolled and well-controlled target population. Post approval, medications are used in the real world setting as well as being evaluated in post approval studies. In clinical practice, data on medication use are routinely captured in different formats, including information on efficacy and tolerability in populations that extend beyond those enrolled in clinical trials. Real world data (RWD) refer to data associated with patient health, collected from sources other than RCTs; these data may be used for decision-making purposes.1,2 RWD can be analyzed to produce real world evidence (RWE), that is, evidence from RWD on the usage and/or benefits and risks of a medication or a medical product.1,3 Not only do RWD support clinical interpretation of how products act in more diverse patient populations, but they may also point to additional therapeutic benefits or uses beyond those originally studied in RCTs.4,5 This article critically considers the current and future role of RWD in the regulatory approval process and discusses important key issues surrounding the use of these data.

Evolution of clinical trials and regulatory processes

Clinical trials in various formats have been conducted for millennia, but the first controlled clinical trial is attributed to James Lind in 1747, who found that citrus fruits could prevent and cure scurvy; the first RCT was conducted for streptomycin in 1946. Despite the long history of clinical trials, it was not until 1947, following the atrocities committed during World War II, that the Nuremburg Code was formulated; it provided the first international guidance for ethical human experimentation.6 In 1964, the World Medical Association developed the Declaration of Helsinki, “as a statement of ethical principles for medical research involving human subjects, including research on identifiable human material and data”. It has undergone several revisions over the years and now addresses issues such as posttrial access to interventions, research registration, and the publication and dissemination of results.7

The development and evolution of ethical and regulatory guidelines in the 20th century have allowed and enabled clinical trials to evolve, introducing scientific rigor in establishing efficacy and safety of medications through introduction of controls, randomization, placebos, and double-blind assessments. Consequently, RCTs have become the gold standard in study trial design.8

Types of clinical evidence: RCTs versus RWD

In traditional evidence-based medicine, hierarchies have been established to rate the relative strength of evidence of a particular clinical study methodology. Different levels of hierarchies exist, with RCTs considered to provide higher-quality evidence than observational studies, because of their scientifically rigorous and robust methodology and randomized setting that eliminates both measured and unmeasured confounding. Typically, RCTs measure the short- or long-term efficacy, and/or tolerability, of a medical intervention versus a known comparator (or placebo), using a carefully selected and narrowly defined homogenous patient population. These trials are then conducted under standardized and controlled conditions in order to minimize bias and potential confounders (Table 1).9,10 Meta-analyses and systematic reviews of RCTs are largely thought to provide the next highest-quality evidence (Figure 1),11–15 but they can suffer from limitations that do not exist in individual RCTs, for example, heterogeneity of results, inclusion of poor-quality data, and publication bias.12,15,16 Indeed, while a high-quality RCT holds a “higher” position in the level-of-evidence hierarchy compared with observational studies due to a greater capacity to attribute treatment effect, a poor-quality RCT with biased results will inherently lower the quality of any given meta-analysis through its inclusion. Evidence hierarchies provide useful guidance; however, in the process of clinical decision making, all available evidence should be critically appraised.16

Table 1

Overview of the strengths and weaknesses of randomized controlled trials9,10

Strengths	• Adequately powered
	• Internal validity due to unbiased methodology, for example, narrowly defined study population, randomization, blinding, and inclusion of control groups
	• Scientifically robust
	• Provide substantial information regarding the efficacy and safety of interventional products
	• Prospective design
	• Prespecified, well-defined end points
Weaknesses	• Lack external validity and generalizability to different settings
	• Lack statistical power if sample size is not large enough to answer research question
	• Can be hampered by ethical and practical considerations
	• Do not provide all evidence required for medical decision making or guide patient-centered care
	• Require a lot of investment in terms of finances, resources, and time
	• Restrictive enrollment criteria; involve homogeneous patient populations, which are not reflective of those seen in real-life clinical settings
	• Often conducted over a shorter period of time than is required to fully assess the clinical and economic impact of a medical intervention
	• Volunteer bias
	• “Placebo” response

Figure 1

General hierarchy of study types.11–15

Most institutes now have transitioned to electronic health records (EHRs) resulting in a rise in the prevalence of collecting and reporting clinical data. In addition, the breadth and quality of the RWD being collected are improving incrementally. These data can now allow scrutiny at a granular level and have the potential to inform dosing, the types of patients who are prescribed a particular medication, and the impact of the use of a pharmaceutical intervention on real world patient outcomes. In addition to clinical outcomes, RWD databases also enable assessment of health care utilization and associated costs, thus further improving understanding of the broader impact of the use of a particular medication. RWD may also allow for longer-term follow-up of patients being evaluated, compared with RCTs.

The level of evidence is particularly important when guidelines are developed by international scientific organizations.17,18 Despite the various advantages of RWE, only a few guidelines currently review RWD, and few actually use RWE to guide clinical practice recommendations.1,2,5,19 As the robustness of the information available from RWD increases, scientific organizations and regulatory authorities should consider elevating the status of RWE. An integral part of this process will be to understand and address the barriers that currently challenge the application and utility of RWD.20

RCTs have high internal validity and, for this reason, remain the gold standard for informing both pre- and postmarket regulatory decision making. For example, data from RCTs are used in initial registration of a medication, expansion of a product label to include a novel indication, postauthorization safety surveillance, and development and support of treatment guidelines.21 However, RCTs have limited external validity since the results from a study conducted in a specially selected homogeneous patient subpopulation, and carried out under ideal circumstances, may not necessarily extend well to the general (heterogeneous) patient population with complex care needs, multiple comorbidities, or concomitant medications.4,21 Indeed, in a recent review discussing RWE from a China perspective, Sun et al reflected on RWE versus RCT data, and proposed that “there is a continuum in the study features of traditional RCTs and real world studies, with external validity increasing as more real world features are included in the design”.22 As RCT data are easier to combine into a meta-analysis than observational data, the latter are often considered as supportive evidence; that is, they are not recognized as causal evidence for guidelines or as providing key support in regulatory approval. However, within the past decade there are early indications from international regulators to suggest that RWD are starting to be recognized as a strong repository, although specific guidance needs to be developed regarding how data can be optimally collected and analyzed.2,5,20 Observational studies can address important clinical questions that RCTs cannot answer, such as the expanded efficacy or tolerability of therapies in nontarget populations. Hence, observational studies may extend our knowledge of products to provide a broader view of a medication’s place in clinical practice, rather than in a niche, controlled population. This knowledge will come from a wide range of potential sources of RWD, each with its associated strengths and limitations (Table 2).2,23–25 These sources include EHRs, patient/disease registries, routinely collected administrative and insurance claims data, primary and secondary patient-level data collection (prospective and retrospective), and population health surveys, as well as emerging sources, such as social media and data collected via mobile devices and apps.21

Table 2

Overview of the strengths and limitations of different types of real world data sources2,23–25

Source	Key characteristics	Strengths	Limitations
Supplements to registration RCTs	• Additional data such as patient-reported outcomes, medical resource use, and costs gathered alongside standard, clinically focused registration RCTs• May provide evidence on treatment patterns for common events	• Randomized design	• Restricted patient population• Carefully controlled clinical setting for data collection• Protocol-driven resource use• Lack of statistical power to detect events other than specified key end points• Relatively short time frame
Practical/pragmatic clinical trials	• Simple trials involving prospective, randomized study designs but with larger and more diverse patient populations than conventional RCTs• Often focusing on obtaining policy-relevant outcomes data	• Broad patient population• Randomized design• Sufficient statistical power to establish significant differences inkey outcomes• Resource use less likely to be protocol driven	• Increased cost of data collection due to larger number of patients and clinical settings involved• Potential for reduced data quality (missing data, data entry/coding errors)• Lack of standardization across settings
Patient/disease registries	• Observational, prospective, cohort studies assessing real world safety and effectiveness, quality of care/provider performance, and cost-effectiveness• Often conducted to collect postauthorization marketing safety data (to address specific safety concerns or to satisfy regulatory requirements)	• Larger and more diverse patient groups than RCTs• Reflect real world outcomes, as well as treatment patterns and clinical decision making• Longer time frame than RCTs	• Nonrandomized design• Visit schedules not required/data not collected at fixed intervals• Potential for reduced data quality (missing data, data entry/coding errors)• Lack of standardization across settings• Risk factors/outcomes may change during follow-up• Statistical adjustments may be required to address confounding/imbalance• Causality cannot be confirmed
Administrative data (claims databases)	• Retrospective, longitudinal, and cross-sectional analyses of clinical and economic outcomes at patient level• Claims data are collected primarily for reimbursement, but databases may also contain some clinical diagnosis/procedure information and details on related resource use and costs	• Large size of databases allows for identification of outcomes of patients with rare events• Analyses can be performed at low cost and over a short time frame	• Nonrandomized design• Potential for reduced data quality (missing data, data entry/coding errors)• Limited comprehensive clinical data across health care settings• Lack of distinction between costs and charges
Population health surveys	• Designed to collect descriptions of health status and well-being, health care utilization, treatment patterns, and health care expenditures from patients, providers, or individuals in the general population	• Provide unique contributions about generalizability of treatments and their impacts, and about use of and expenditures for health services• Methodologically rigorous, relying on complex sample survey designs	• Lacking relevant data on specific products• Data subject to issues of subjectivity and recall bias
EHRs/other technologies capturing real-time clinical treatment and outcomes	• Used for medical chart reviews to produce specific information on the real world use of specific tests or medications for particular conditions	• Important sources for RWD from a wide range of clinical settings throughout the world• Expansion of electronic data capture is lowering the cost of the medical chart reviews• May contain detailed, longitudinal information, including patient-level, disease-specific symptoms	• High-end statistical analysis tools required to transform the information for research purposes

Abbreviations: EHR, electronic health record; RCT, randomized controlled trial; RWD, real world data.

Potential sources of RWD

In essence, RWD originate from a variety of sources associated with, or used in, routine clinical settings (Table 2).2,23–25 The majority of sources are available electronically and, combined with recent technological advances that make systematic data collection increasingly easy, integration of a wealth of data from different sources is becoming possible. The significant increase in the use of EHRs within clinical practice, combined with the surge in technology applications that record health information, means that the amount of information available to track the use of medication products in the real world has never been greater.

A number of databases have emerged that collate the growing volume of RWD (eg, Optum®, Premier Healthcare Database, Truven Health MarketScan® [IBM Watson Health], Kantar Health’s National Health and Wellness Survey, and IQVIA™). These global databases store data relating to large numbers of patients, including physician and patient survey data, costs, medications and procedures, laboratory results, and outcomes, alongside demographic and socioeconomic status. This information can be analyzed by health care professionals to generate RWE for research hypotheses that cannot be addressed by RCTs. An institution perfectly placed to contribute a wealth of RWD is the National Health Service (NHS) in the UK. The NHS provides health care from birth to death for the entire UK population, and therefore holds a vast repository of RWD. However, a problem up until now has been that the data are held in many different databases, and therefore have been an underutilized resource. Initiatives designed to collect and link these data are underway, such as the Clinical Practice Research Datalink, which is the world’s largest database of anonymized, longitudinal primary care medical records and is also linked to secondary care datasets.26

There is also a growing trend for hospitals and primary practices to merge under the umbrella of integrated delivery networks and accountable-care organizations. This merging of multiple institutes requires alignment of their processes and treatment algorithms, as well as their ability to track data. This necessitates a common, unified, highly sensitive, and detailed EHR system. These large databases can quickly become resources for analyzing outcomes and impacts of various interventions.27

The increasing volume of RWD needs to be balanced against data integrity and usability. Indeed, IBM classify big data, that is, large and complex datasets, into four categories (four Vs): volume, velocity, variety, and veracity.28 The complexity and robustness of data will vary for each source. This underscores why some health care providers and guidance committees have long-standing concerns about using RWD when making key regulatory decisions. Observational studies of RWD, such as cross-sectional studies or surveys, case–control studies, and cohort studies,15 are often conducted in larger patient populations than would be enrolled in an RCT, and are not subject to the recruitment biases found with RCTs.4 However, this strength in validity is achieved at the expense of the robustness of interpreting the findings. In addition to selecting the most appropriate source of RWD, understanding the nuances of each data source is critical for researchers using RWD to supplement data from RCTs.29 For example, although claims data provide useful information on baseline characteristics and comorbidities of patients taking medications, thereby facilitating a group-wise comparison, they provide only a “snapshot” rather than a longitudinal assessment regarding cause and effect.30 In comparison, EHR data can be more complex and provide an assessment of cause and effect through longitudinal assessment of patient care, but sophisticated analysis tools are required to decipher the large amount of data captured by EHRs.31 Only with the most appropriate methods of analysis, capable of extracting meaningful patterns of information from the best sources of RWD, will acceptance of RWE move forward in the opinion of many key decision makers, to become useful beyond observational studies.

Potential uses of RWD

RWD are increasingly recognized as an important contributor to the evidence originating from traditional RCTs; however, application stops short of having a real influence in decision making.2,32,33 A major driver for the integration of RWE into the clinical decision-making process is understanding that RWD may provide valuable information on both short- and long-term medication safety and effectiveness in patient populations that are not well represented in RCTs. RWD may also be used to provide insights into other aspects of medication use in clinical practice, including how a medication compares with other therapies in terms of surrogate outcomes, for example, biomarkers, clinical uptake, safety, and cost.34 Through this application, RWD plays a critical role in bridging the gaps in information that cannot be met by RCTs alone. In resource-scarce health care environments, RWD may increasingly be used to generate more cost- and time-efficient postmarketing evidence than RCTs.

The use of RWD may also change throughout the lifecycle of a product, particularly postauthorization (Figure 2). For example, RWD requirements are already mandatory in some European Union (EU) countries, with positive evidence being essential to maintain access to medications in countries such as France, Italy, the Netherlands, Belgium, and the Nordic countries.35 Health technology assessment (HTA) agencies in the EU use RWD to provide evidence for relative effectiveness assessment of medications in the settings of pharmacoeconomic analyses, initial reimbursement discussions, and conditional reimbursement schemes. However, a recent study found that policies for the use of RWD in relative effectiveness assessment vary across agencies, which presents challenges when using these data for HTA.36 Therefore, in order to fully exploit the potential of RWD, agencies and pharmaceutical companies need to collaborate in order to provide guidance on data acquisition, management, and use across a range of settings.37

Figure 2

Schematic illustration of the utilization of randomized controlled trial data and real world data through the lifecycle of a medical intervention.

One area where RWE may be particularly useful is for medications that have shown beneficial effects in indications for which they are not approved. In such cases, real world use of the medication may provide adequate effectiveness and safety evidence to meet the requirements of regulatory authorities to grant approval for the expanded indications. Indeed, the US Food and Drug Administration (FDA) has suggested that RCTs using RWD may have a role in supporting regulatory approval of new indications or label expansion for existing medications.38 This may be especially pertinent in situations where, due to the wide availability of data in many different types of patients, a medication has already demonstrated efficacy “off label”, but no, or only small-scale, RCT observations can support its approval.39

Some professional bodies and organizations involved with coverage and reimbursement decisions are seeking to use RWD to help inform health care policies.2,40 Regulatory decision makers, such as the FDA and the European Medicines Agency (EMA), are already accepting RWE in the form of postauthorization drug safety surveillance studies.23,41 The FDA has suggested that they may start to consider RWD alongside clinical RCTs when evaluating new therapies, devices, or indications of existing products, but particularly in the evaluation of medical devices in real world clinical practice.1,42 Other attempts to review evidence systematically have also gained momentum, including those by private and public health plans.2,43 Stakeholders stand to gain from the development of an RWD-inclusive evidence strategy in health care, including physicians, patients, guidelines, committees, regulators, researchers, reimbursement committees and payers. The key to using RWD more effectively to guide decision making may lie in the validity of observations surrounding the risk–benefit ratio.

Role of RWD in determining the risk–benefit profile of a medicine

Establishing a risk–benefit profile is a pivotal part of the regulatory process for assessing the safety of medicines. As a result, regulatory agencies are increasingly viewing safety through the lens of the risk–benefit ratio alongside issues of cost; analysis of RWD can help in assessment of both. For example, both the FDA and EMA have implemented a proactive approach toward safety surveillance and overall risk assessment.44 These regulators have similar data needs and are driven by common objectives with regard to the identification, monitoring, and minimization of risk.

A united approach within the international pharmaceutical community in assessing the risks and benefits of medications will be essential, as any significant differences would inevitably lead to increased pre- and postmarketing approval costs. In today’s global market environment, robust and generalizable data on risk–benefit assessment also facilitate the exchange of information between major regulators. There is a growing appreciation of the benefit of an expanded approach to gathering information and providing evidence on risk–benefit assessment to patients. For example, the EMA has recently advocated an “adaptive pathway” approach in order to improve timely access for patients to new medicines; this includes assessment of risk–benefit.45 This approach builds on regulatory processes already in place within the existing EU legal framework, developed mainly for conditional approval of medicines addressing life-threatening illnesses.45 These patient registries, in combination with other pharmacovigilance tools, allow collection and development of risk-management plans to augment RCT data. Utilizing a broader set of information for assessing the risk–benefit profile, the adaptive pathway seeks a balance between shortening approval times for promising medicines while ensuring there is sufficient information on the safety and benefits of a particular medicine.45 Adoption of this progressive approach by other international bodies would enhance use of RWE from their own health care settings.2,41,46 The challenge in maximizing uptake of RWE will be finding which data sources provide the most appropriate observations to support specific regulatory or guideline changes.

Maximizing the applicability of RWE to inform aspects of health care and decision making

With the current focus on maximizing the utility of RWE, publications have aimed to provide guidance on conducting real world studies in order to maximize their credibility and usefulness within the regulatory approval setting.20,24,47,48 From a regulatory perspective, RWE has the potential to inform regulators on many fronts (Figure 3), but concerns raised often surround accuracy, reproducibility, and validity of the records, as well as reproducibility of analyses, and the handling of missing data.32 For example, claims data are used for different purposes than EHRs and could present biased data by their purpose in supporting reimbursements.2,30 Alternatively, EHRs may supply information on prescribing practices, but not on the filling or dispensing of medications. Therefore, maximizing the applicability of RWE to inform decision makers will depend on well-designed post hoc analyses, using the most appropriate data sources and analytics.48 Bringing together the restrictions around RCT methodology and the breadth and strengths of RWD, it may not be until an RCT can be conducted within an RWD source that we see RWE being routinely used in decision making beyond that of rare diseases, or expansion of existing product labels. Indeed, Hernán and Robins suggest that large observational databases–the so-called big data–can be used to emulate an RCT (the target trial) when the latter is not feasible.49 Moreover, the authors describe a framework for comparative effectiveness research using observational studies, analyzing how the protocol components of a target trial may be considered in the design and analysis of observational studies to maximize emulation while acknowledging the inherent limitations.49

Figure 3

Interconnected platform for maximizing the use of real world evidence.

Despite key advantages and the wealth of RWD already available, limitations in its application are well known. Key stakeholders need to ensure that the terminology surrounding different RWE is clearly defined and used consistently. In particular, the distinction between RWD and RWE, efficacy and effectiveness, and the strengths and weaknesses of each data source should be clearly defined and understood (Table 2).2,23–25 Concerns surrounding informed consent privacy and data integration also need to be addressed, and protocols developed to maintain data integrity. Methodical challenges will need to be worked out to understand what RWD can and, importantly, cannot provide. In order to provide robust and reliable evidence, analytical experts should develop reproducible, standard analytical approaches within the most appropriate data source.5,49 Missing data are a known limitation; however, accuracy of data records will always be a potential confounder when using any database or other RWD source, where the personnel capturing the data may not all be following the same protocols. Finally, regulatory bodies need to have a clear alignment on policies for the use of RWD,36 provide direction on the evidence they require in order to evaluate innovative or existing medicines appropriately, and disseminate information on how data sharing can work to the benefit of all. Considering these challenges, the onus will be on health care providers and payers to become more experienced in assessing the quality of and outcomes from RWE, and associated data sources.

Conclusion

Although RWD are generally accepted as an adjunct to RCTs, more work must be done to clarify which types of RWD and RWE are robust enough to provide information on aspects such as risk–benefit assessment and cost-effectiveness analysis to support guideline and regulatory decisions (Figure 3). While the importance of data from RCTs is well known for establishing the efficacy and safety of new medical interventions, comprehensive assessment of therapeutic strategies requires evaluation of efficacy under optimal conditions (high internal validity) as well as effectiveness in real-life populations and situations (high external validity). RWE therefore has the potential to support, improve, and potentially accelerate the delivery of safe and cost-effective therapeutic interventions, or to support the expansion of approved therapeutics into a new indication.