Conducting real-world evidence studies in India.

Real-world data (RWD) are data relating to patients' health status and/or the delivery of health care routinely collected from a variety of sources. Real-world evidence (RWE) is the clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of RWD. RWE can provide information on diverse areas, for example, natural history and course of disease, effectiveness studies, outcome research, and safety surveillance. India has unique opportunity of conducting RWE studies in several interesting areas, for example, natural history of communicable and noncommunicable diseases and rare disease and Health Economics and Outcomes Research in public versus private hospitals. However, utility of RWE is challenged by diversity of information, large data sets of uncertain quality, and methodological rigor. For India, there are additional challenges in conducting RWE studies - Indian clinical practice pattern and physicians' interest in RWE studies. It is recommended that RWE studies in India should focus on health management issues of relevance to the country India and should be well planned to generate high-quality data.

INTRODUCTION

Real-world data (RWD) are data relating to patients' health status and/or the delivery of health care routinely collected from a variety of sources.[1] Real-world evidence (RWE) is the clinical evidence regarding the usage and potential benefits or risks of a medical product derived from the analysis of RWD.[1] RWD includes variety of data – electronic health records (EHRs), insurance data, product and disease registries, etc. RWD has potential to complement the knowledge available from conventional randomized clinical trials (RCTs), whose design limitations make it difficult to generalize findings to population at large.[2] RWE can provide information on diverse areas, for example, natural history and course of disease, effectiveness studies, outcome research, and safety surveillance. However, utility of RWE is challenged by diversity of information, large data sets of uncertain quality, and methodological rigor. These issues are relevant to conduct of RWE studies in India.

REAL-WORLD EVIDENCE STUDY VERSUS RANDOMIZED CLINICAL TRIALS

RCTs are designed to provide robust evidence of premarket evaluation of efficacy and safety of an investigational product. RCT design focuses on attaining internal validity, which may sometimes compromise generalizability.[2] In contrast, RWE studies are planned to provide data with relevance to clinical practice. Differences between RWE study and RCT [3] are summarized in Table 1.

Table 1

Differences between randomized clinical trial and real-world evidence studies

	RCT	RWE
Type	Interventional/experimental	Noninterventional/observational
Conduct	Protocol based GCP compliant	Usual care/clinical practice
Outcome	Efficacy and safety	Effectiveness and economic assessments
Population	Narrow extensive selection criteria	Wide unrestricted few exclusions
Comparator	Gold standard/placebo	No comparator/standard care
Randomization and blinding	Yes	No
Relevance	Internal validity	Clinical practice

RCT=Randomized clinical trial, RWE=Real-world evidence, GCP=Good clinical practice

REAL-WORLD EVIDENCE STUDIES: POTENTIAL BENEFITS

RWE studies are valuable across product lifecycle and can provide multiple benefits.[234]

Prior to product development

Natural history of disease – prevalence, incidence, and unmet medical need

Current treatment patterns and standard of care

Understanding current service structures, especially if they may need to change to deliver the new medicine

Generate hypotheses for prospective trials.

At or following market authorization

Insights into early and/or long-term clinical experience

Prospective studies for safety surveillance

Evidence to support health outcomes, reduction in resource use, and patients' acceptability

Detection of untreated/undiagnosed patients (“unmet need”).

After product launch

Comparative effectiveness and health outcomes

Compliance with product label or treatment guidelines

Identification of suboptimal dosing or treatment

Support for expansion of indications

Characterizing appropriate treatment subgroups

Monitoring safety, especially uncommon/rare adverse reactions.

Despite many potential benefits, RWE studies are primarily conducted to meet demands or expectations of reimbursement or insurance agencies and patients and to satisfy drug regulatory agencies.[3] The payers and health-care decision-makers have been increasingly demanding postapproval evidence to support reviews of pricing, reimbursement, licensing for new therapies, and formulation and indication changes.[3] However, according to a recent survey of 30 companies, the objectives of most RWE studies were postapproval safety and commercialization strategies.[5]

REAL-WORLD EVIDENCE STUDIES: DESIGNS

RWE studies employ retrospective and prospective designs.[34]

Retrospective design studies have predictable short-term data collection period, are dependent on the quality of available clinical records, and are usually low-cost studies. However, such studies have limitations of incomplete or inaccurate data. Examples of such studies include:

Clinical/hospital record/chart review

Administrative data/insurance claim data

Electronic medical records (EMRs).

Prospective design studies can include robust patient data set and clinical data not routinely captured, and provide information on long-term data on natural history or disease progression. However, such studies have uncertain timelines, which are dependent on the rate of presentation of eligible patients and are likely to be costly. Examples of such studies include:

Registries

Health surveys

Prospective observational studies

Postauthorization safety study.

REAL-WORLD EVIDENCE STUDIES: CHALLENGES

There are several challenges in interpretation and in utility of RWE studies.

Validity – Open, unblinded, and nonrandomized designs used in RWE studies raise questions about their validity [23]

Quality of data – Most of the sources for RWE studies are data collected to support clinical care and reimbursement. These data are not gathered to support any specific research study. Hence, there are gaps in the accuracy, completeness, and quality of RW data.[236] This is likely to be a big challenge in India, especially in the case of RWE studies based on retrospective hospital databases

Confounding – A small observed effect in an RWE study could be due to confounding factors.[2]

These challenges point to the need for proper planning and conduct of RWE studies.

PLANNING OF REAL-WORLD EVIDENCE STUDY

Several factors should be considered in planning of RW study, for example, design, conduct, monitoring, quality, ethics, reporting, safety, etc.[367]

The design of an RWE study, for example, registry is based on the following considerations:[37]

Research question – Clinical or public health relevance

Resources – Funding, sites, clinicians, and patients

Exposures – Measurable exposures

Outcomes – Objective and measurable, appropriate and valid for the research question, detailed recording

Data sources – Availability and completeness

Study design – Relevant to the question and consideration of confounders

Study population – Types of patients, need for comparison group, and selection process

Sampling – Process of sampling in relation to target population and design

Study size and duration

Internal and external validity – Potential biases and concerns about external validity, i.e., generalizability of the results.

As pragmatic trials are conducted in real-world usual care setting, standardized collection of common minimum data set is essential to ensure high-quality data. This would require utilizing special electronic data capture or modifying EMR to gather demographic and disease-specific data.[3]

For long-term prospective studies or retrospective data collection, propensity scoring is effective at reducing bias.[3] Propensity scoring considers classification of the relationship between treatment assignment and baseline characteristics. Factors, which are different between two treatment groups and are associated with treatment preference, are weighted to estimate the probability of any participant in the cohort being assigned a specific treatment. This estimated propensity score is used to match the participants across two treatment groups.[3] For analysis of outcomes, “matchable” participants are compared, and unmatchable participants are excluded. The issues of bias and nonrandomization require following good research practice. The use of checklists, for example, STrengthening the Reporting of OBservational Studies in Epidemiology for general cohort studies, Good ReseArch for Comparative Effectiveness checklist, or PRagmatic Explanatory Continuum Indicator Summary (PRECIS) tool is recommended to ensure that RWE study is valid.

Several problems reported with product registries, for example, delayed start, low accrual rate, low use of product, and missing data need careful consideration while planning.[8]

PRAGMATIC EXPLANATORY CONTINUUM INDICATOR (PRECIS-2) TOOL

A pragmatic RWE study design is expected to meet the PRECIS-2 tool.[9]

Presic-2 tool includes 9 domains of study design, which are:

Eligibility – Similarity between the participants in the trial and those who would receive this intervention if it was part of usual care in practice

Recruitment – Extent of extra effort required to recruit participants over and above what would be needed in the usual care setting to engage with patients

Setting – Difference between the settings of the trial from the usual care setting

Organization – Difference between the resources, clinician/provider expertise, and the organization of care delivery in the intervention arm of the trial from those available in usual care

Flexibility (delivery) – Differences in the flexibility of delivery of the intervention and the flexibility anticipated in usual care

Flexibility (adherence) – Differences in the flexibility in how participants are monitored and encouraged to adhere to the intervention from the flexibility anticipated in usual care

Follow-up – Differences in the intensity of measurement and follow-up of participants in the trial from the typical follow-up in usual care

Primary outcome – Extent of direct relevance of the trial's primary outcome to participants

Primary analysis – Extent of inclusion of all data in the analysis of the primary outcome.

Each domain is scored using a 5-point Likert scale as follows:

Very explanatory

Rather explanatory

Equally pragmatic and explanatory

Rather pragmatic

Very pragmatic.

The score for each domain is plotted on a PRECIS wheel, in which Score 1 is close to center and Score 5 is away from the center, and the score for each domain is connected to other domains. If the design is RWE or pragmatic, the shape connecting all the domain scores would be away from the center. If the design is explanatory or randomized, the shape would be closer to the center. PRESCIS-2 tool should be used at the stage of planning and design to ascertain whether the design fits the RWE study requirements. If not, the design can be modified to meet RWE objectives.

CONDUCT OF REAL-WORLD EVIDENCE STUDY

Conduct of RWE study, like any clinical trial, requires review by the ethics committee, and requires informed consent of potential participants for access to identifiable medical records for the purposes of research, whether interventional or noninterventional.[5] A well-trained site team, risk-based monitoring strategy, and medical monitoring are vital in ensuring data integrity. Site monitoring should focus on consent procedures, randomization integrity, safety, and complete follow-up.[6]

QUALITY OF REAL-WORLD EVIDENCE STUDY

Quality assurance of data, data collection/registry procedures, and computerized systems are essential to provide confidence that the design, conduct, and analysis of the registry could protect against bias (systematic error) and errors in inference, that is, erroneous conclusions drawn from a study.[7] For RWE studies – especially for registries – focus on external validity, internal validity, and analysis, and reporting is essential.

External validity depends on confirming that (1) registry participants were like the target population, (2) selection bias was minimized, and (3) completeness of information on eligible patients was evaluated and described.[6]

Internal validity requires (1) collection of necessary information for relevant key exposures, risk factors, and mitigating or protective factors; (2) collection of specific exposure data to support the main research questions; and (3) data checks for range and consistency checks. In addition, for safety registry studies, a clear and specific approach to ask about complaints or adverse events is a must.

Analysis and reporting entails (1) detailed description of methods, (2) reporting of results reported for all the main objectives, (3) use of accepted analytic techniques used, (4) description of follow-up time to enable assessment of the impact of the observation period on the conclusions, (5) consideration of impact of missing data and confounding factors, (6) clarity of conclusions drawn and implications of study results, and (7) consistency in comparison of results with other relevant researches.

REAL-WORLD EVIDENCE STUDIES IN INDIA

Although there are potential benefits of generating evidence from RW studies for India,[1011] there are some unique challenges in conducting such studies – (a) Indian clinical practice pattern and (b) physicians' interest in RWE studies.

Indian clinical practice pattern versus real-world evidence data

In clinical practice, diagnosis and treatment are as per physicians' decision. However, the investigations and follow-up of patients are variables based on the patients' affordability and education about his/her medical condition. Due to this, investigations for assessing effectiveness and safety endpoints are not routinely measured. Often, the clinic or hospital documentation about medical history is not in detail, and information about treatment provided is sometimes inadequate.[12] Some of the published investigator-initiated registries have discussed such limitations in data. In a review of 28 observational studies in rheumatoid arthritis (RA), Handa et al. reported that only seven described the epidemiology of RA, described comorbidities, and extra-articular manifestations; nine described the functioning abilities and quality of life among patients; and 10 provided information on treatments.[13]

In a registry of lymphomas, Nimmagadda et al. reported that survival data of patients appeared to be high possibly because a large proportion of patients were lost to follow-up.[14] In this study, 25% of patients received < 6 cycles of standard chemotherapy and 42% received a monoclonal antibody. The authors cite noncompliance, financial factors, and logistic constraints as some of the reasons for inferior outcomes of treatment.

RWE studies of a new product require good documentation of frequency and adequacy of patients' follow-ups, measurable exposures, and outcomes. In a clinical trial – Phase III – the sponsor pays for all expenses and provides investigational and standard therapies. However, in RWE studies – registries, the patients must purchase new expensive drugs and pay for the investigations to measure effectiveness and safety and follow-up visits. This means that conclusions from Indian registries cannot be generalized to all the patients as registries will include the patients who are from higher socioeconomic class and who can afford to pay for the investigations and therapy.

For registries of generic products or biosimilars, there is an additional factor of widespread availability of low-cost generic options, and clinicians' and patients' preference for switching brands to cheaper biosimilar/generic options.

Physician's interest

Most Indian physicians are busy practitioners with little time or interest in clinical research. Some of them gained experience of clinical research while conducting clinical trials for pharmaceutical company products.

RWE study, for a clinician, is the use of available patients' data from his/her practice for research, without any additional effort. Hence, when a clinician participates in a structured RWE study, for example, registry, she/he is concerned about amount of data collection and documentation. Some of the studies – registry – are long term and last for several years. For an Indian physician, RWE studies appear to be postmarketing or Phase IV clinical trials, as the pharma company RWE studies require collection of effectiveness and safety data in all patients treated with a drug. However, the investigator grant is lower compared to Phase III clinical trial. The sponsor's support and oversight for RWE studies are limited compared to support and oversight for a Phase III RCT for new drug development. There is hardly any monitoring and source data verification. Hence, a physician may wonder about quality of data and relevance of results to his practice and patients. Lack of physicians' commitment is one of the main factors in low recruitment in registries.[8] The sponsors could consider employing a variety of tools – newsletters, training meetings, site audit/retraining visits, physicians' satisfaction/opinion surveys, regular data reports on registry growth to participants, presentations at conferences, and publications – to keep the physicians motivated and engaged.[7]

Myths of real-world evidence studies in India

In India, there is no demand from insurance agencies, patients' groups, or regulators to conduct RWE studies. Hence, such studies are mostly conducted by pharmaceutical companies on a newly launched expensive product. These are prospective observational studies in consenting patients who meet selection criteria and are prescribed a new product for adequate duration. The sample size is based on efficacy endpoints. The sponsor makes efforts to ensure the quality of data by regular monitoring. Some of the pharma-sponsored RWE studies collect data on treatment-related efficacy endpoints in cross-sectional survey manner. Such studies suffer from selection bias as the prescribers are in private clinic/hospital setting, whose patients can afford the cost of treatment and investigations. The results of such studies support premarket efficacy results. If one applies PRECIS-2 criteria, such studies will fall in the category of explanatory studies and not real-world pragmatic studies.

Other types of RWE are investigator-initiated observational studies or disease registries, which describe prevalence and disease characteristics, but often have limited details on patients' profile, comorbidities, treatments, natural history of disease, and long-term outcomes.

Real-world evidence: Scope in India

India has unique opportunity of conducting RWE studies in several interesting areas:

Natural history and long-term complications of communicable, for example, drug-resistant tuberculosis and noncommunicable diseases, for example, heart failure

Registry for orphan/rare disease, for example, thalassemia

Health Economics and Outcomes Research in public versus private hospitals

Identification of unmet medical needs relevant to Indian population

Comparative effectiveness of brand versus generic and biologic versus biosimilar

Comparative effectiveness of Indian prescribed dose (usually lower than the Western dose) versus package insert recommended dose.

Obviously, planning and conducting such studies would require participation of diverse stakeholders – government, public and private medical institutions, patients' support groups, nongovernmental organizations, and pharmaceutical industry to support and conduct RWE studies which are relevant to Indian health care.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.