Real world evidence (RWE) - Are we (RWE) ready?

Real world evidence is important as it complements data from randomised controlled trials (RCTs). Both have limitations in design, interpretation, and extrapolatability. It is imperative one designs real world studies in the right way, else it can be misleading. An RCT is always considered higher in the evidence ladder and when there is discordance between a real world study and an RCT, it is the latter which is always considered pristine because of the way it is conducted, e.g., randomization, prospective, double-blind, etc. A real world study can also be done prospectively, and propensity score matching can be used to construct comparable cohorts but may not be able to account for certain biases or confounding factors the way an RCT can do. Nevertheless, comparative effectiveness research in the real world is being resorted to, especially for efficiency studies or pharmacoeconomic analyses, and with the advent of machine learning, the electronic healthcare database mining can result in algorithms that help doctors identify clinical characteristics that correlate with optimal response of a patient to a drug/regimen, thus helping him/her select the right patient for the right drug.

What is real-world evidence? Obviously, evidence that is generated or exists in the real world. Why is it important? Typically, evidence generated within a randomized controlled clinical trial (RCT) is considered higher than that in the real world. Where hypotheses are generated double blinding and randomization (which ensures every patient has an equal chance of being allocated to treatment A or treatment B) are ways of ensuring that comparable cohorts are created, and bias is minimized to the extent possible. Naturally, a hypothesis can be tested within an RCT. However, there are limitations of an RCT. For example, in an RCT, there are inclusion and exclusion criteria. These eligibility criteria ensure that a homogeneous and representative sample is collected. However, in the real world, can any patient be excluded? Data from an RCT can only be extrapolated to the kind of patients who were eligible for the RCT. Hence, there are limitations of generalizability. Moreover, that is where real-world evidence comes in, to supplement data from RCTs, and hopefully bridge the gap between the controlled environment of an RCT and the harsh realities of the real world.[1]

In an ideal world, both the RCT and real-world evidence coexist and one can even do large simple studies, where the two elements are blended such that the results do mirror what happens in the real world. Such hybrid, efficacy-effectiveness studies can help in advancing a closer correlation to the real world within a clinical development program. However, real-world studies are fraught with their own limitations. Can one randomize in the real world? What about retrospective analyses of databases (electronic medical records) in the real world or comparative effectiveness research? They have their uses viz., in comparing the cost-effectiveness of two regimens in the real world, beyond the rigors of an RCT. They can help in certain policy decisions. However, they are at best studies that can generate hypotheses, and if one wants to confirm the same, one will need to do RCTs. Moreover, what if there is discordance between the results of an observational, real-world study and an RCT? As happened in the US cohort of CVD real, a large retrospective analysis of outcomes in patients on sodium-glucose cotransporter-2 (SGLT-2) inhibitors versus in patients on other glucose-lowering drugs.[2]

Where the all-cause mortality was reduced by canagliflozin by a whopping 62%, which was statistically significant for superiority, but in the RCT, the integrated analysis of the CVOT, the CANVAS Program, the all cause mortality risk reduction with canagliflozin was only 13% and not significant. In fact, in the left truncated dataset, it was only 4%, again not significant. A recent article in Diabetes Care, by Samy Suissa, has dissected out this registry and pointed out reasons why the results seemed exaggerated. For example, he mentioned immortal time bias or time-lag bias, which means that, for patients to have been prescribed SGLT-2 inhibitors, they would first have had to be other glucose-lowering drugs, and then an SGLT-2 inhibitor was added. This meant that some of the benefits attributed to the gliflozin could have been due to the other glucose-lowering drugs prescribed earlier, so was this a fair comparison? Hence, this is also called survivor bias. If the real-world study is not done prospectively, then one more bias creeps in, called channeling bias, as doctors generally tend to put patients on a new drug, only those who did not respond well to the first drug.[3]

Real-world data are data captured in a noninterventional, observational manner, in a natural, uncontrolled setting – outside of traditional clinical trials. Per ISPOR, data used for decision making that are not collected in conventional randomized controlled trials are termed real-world data. The European Working Relative Effectiveness Working Group defines real-world data as a measure in understanding health-care data collected under real-life practice circumstances.[4]

A noninterventional study (NIS) conducted to assess safety, tolerability, and effectiveness of marketed medicines in clinical practice, i.e., in a naturalistic setting where choice of therapy is consistent with approved prescribing information (no study drug to be supplied) and in line with current practice at the study site, and other aspects of patient care, including clinical examinations, laboratory investigations, and the use of instrumentation, other invasive and noninvasive procedures, are in consonance with current practice at the study site, is an example of a real-world study. It tests the effectiveness of the drug, as against an RCT in the premarketing phase which tests the efficacy of the drug. It is easier to conduct than an RCT, cheaper too, but the credibility is considered low, as one is never able to account for all biases and confounders. Examples of NIS are postmarketing surveillance studies, which are large, multicenter studies involving many physicians and subjects, certain safety assessment of marketed medicines studies, and Anwendungsbeobachtung studies.

Safety is another reason why real-world studies are conducted. No matter what we may achieve in terms of characterizing the safety profile of a drug in up to Phase III trials, unless the drug is tested in the real world, where no patient can be excluded, where the drug is not given free of cost, where there is no monitoring for compliance, the full safety profile of the drug can never be fully characterized. This is why it is said in the Oxford Textbook of Clinical Pharmacology that, unless a drug is capable of doing some harm, it is unlikely that it will have much of an effect.

Practice-based medicine is another term that is bandied about as bedside to bench is also an important way to advance science and not only bench to bedside. However, for this to happen, one should have clinician researcher investigators. Doctors who, even in their busy practice, are able to see patterns in their practice, then formulate a research hypothesis, and then proceed to test it. Thus, a doctor can be good at both good clinical practice (GCP) and good clinical research practice (GCRP). One can feed the other and vice versa, and this way medicine advances. Practice uses knowledge, focuses on individuals, has a short action span, the reward is immediate, it respects authority, follows custom, earns revenue, and stimulates research. Research creates knowledge, focuses on groups, has a long action span, the reward is delayed, it questions authority, challenges custom, earns reputation, and enriches practice.

Just as there is a time and place for everything, and everything in its place, just as there is a place for both generics and patent-protected innovator drugs, so also there is a place for both RCT generated and real-world evidence. There are limitations with both RCTs and real-world analyses. Taken together, information from both RCTs and real-world analyses is important to confirm the validity of safety and efficacy data for new agents. What makes real-world data robust? Preferably, a naïve patient population, adjustment for differences in baseline characteristics using a statistical tool called propensity score matching (to create comparable cohorts in the real world, after observing how a doctor decides which drug for which patient), large patient numbers, prospective evaluation, and a long study period and extensive follow-up.[56789]

Typically, doctors in India believe all they need to do is collect data from patients in their practice to whom they have prescribed, for example, a new drug. In other words, a one-arm longitudinal study. They do not realize that unless one has a control group for comparison, the results will not be relevant. Plus, one can have all sorts of safety concerns being reported in an unbalanced manner. Generally, adverse events on the older standard of care are not reported, while safety information on the new drug is always reported, at least within the first 2 years (Weber effect). Sometimes, if the two comparator groups are not well matched, and the real-world study does not account for confounders and biases, one is never sure of the authenticity and veracity of the interpretation of such data.

If our patient is older than, younger than, sicker than, healthier than, ethnically different from, taller, shorter, simply different from the subjects of a study, do the results pertain? The art of clinical decision-making is judgment, an even more difficult concept to grapple with than evidence. RCTs focused on a representative sample and made homogeneous through strict eligibility criteria; drug is given free of cost and laboratory tests are also free of cost, they are monitored for adherence and findings are extrapolated to the population, but with the limitation of generalizability. In the real world of clinical practice, it is always the individual patient, and every patient can be heterogeneous in the way he experiences the disease and responds to a drug. Naturally, it is not about whether one drug is better than another, but about to which drug or regimen (in sequence or combination) does the patient respond best. From an RCT or GCRP to the real world of GCP, one moves from standardized care to standard of care. Guidelines based on evidence from RCTs may not be applicable to every individual patient one sees in practice.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.