Pragmatic randomised controlled trials in COPD and asthma: how to guide clinical practice.

The use of real-world evidence (RWE) studies, including pragmatic randomised controlled trials (RCTs; randomised RWE studies), to aid the development of treatment guidelines, is gradually becoming a mainstay within clinical practice. RWE is an integral part of patient-driven decision-making and offers important value to add complimentary evidence to traditional RCTs; these provide a more well-rounded view of the benefits to patient-reported outcomes and improve the external validity of a given treatment versus findings from traditional RCTs alone. Discussions in recent scientific workshops explored the importance of pragmatic RCTs in optimising guideline development and patient care in chronic obstructive pulmonary disease (COPD) and asthma. The Salford Lung Study in patients with COPD (NCT01551758) and asthma (NCT01706198) were the world's first prelicence pragmatic RCTs that compared novel investigational treatments with existing COPD and asthma treatments and, more recently (2021), RWE studies have been used by the American Thoracic Society and the US Food and Drug Administration to support the approval of an immunosuppressant drug in patients receiving lung transplants. This highlights the importance of RWE data in supporting clinical guideline development and emphasises the advantages for the use of pragmatic RCTs in guiding clinical practice. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Real-world evidence (RWE) studies can provide a range of important data for patients, physicians and policy-makers that help to optimise disease management. However, various limitations in the design of high-quality RWE studies, data from which are primarily observational and without randomisation, mean that treatment guidelines for disease management tend to be developed from traditional randomised controlled trials (RCTs) rather than RWE studies, despite RCTs carrying their own limitations.1 Nevertheless, recent technological advances in health informatics have facilitated the evolution of RWE studies into prospective, randomised, pragmatic RCTs (also known as clinical effectiveness studies).

Unlike traditional RCTs, which investigate the efficacy of an intervention in a highly selected patient population in an idealised, controlled setting designed to minimise bias, pragmatic RCTs are designed to resemble as closely as possible the real-world setting of routine clinical practice and examine the usefulness of an intervention. Thus, patients recruited to such trials are similar to those seen in routine clinical practice and ideally are recruited during usual care, while the trial is ideally conducted in the usual care setting, has limited need for extra resources and reports data applicable to multiple settings and heterogeneous patient populations.2 3 Pragmatic RCTs present unique opportunities for data collection that could have a transformative impact in guiding clinical practice when combined with results from traditional RCTs. In this paper, we propose how pragmatic RCTs can be used to advise the development of future guidelines for the clinical management of chronic obstructive pulmonary disease (COPD) and asthma.

Current guidelines in COPD and asthma

Treatment guidelines are usually based on RCTs, which are limited in their ability to represent the relevant population in everyday clinical practice.4 5 Furthermore, the tendency of guidelines to have an overall ‘disease focus’ for the stepwise pharmacological treatment of respiratory diseases—rather than a ‘patient focus’ on individual patient needs—is potentially disadvantageous for optimisation of patient care. Guidelines are often developed by specialists, but delivered by generalists, which contributes to the reduction of their effectiveness. Where specific patient-focused recommendations are included, the overall complexity of the messaging within guidelines means that key points can be overlooked by physicians. In addition, some guidelines are updated more regularly than others, which leads to conflicting information being available simultaneously, and makes optimal decision-making difficult. Therefore, a shift from complex, disease-focused guidelines towards more concise treatment guidelines with a patient-focused approach could revolutionise guideline development and facilitate optimisation of clinical practice and patient care. This is becoming increasingly important as healthcare evolves from individual patient and physician interactions into a more collaborative and integrative venture between patients and multiple healthcare professionals.

Traditional RCTs and RWE studies: pros and cons

Traditional RCTs have historically been the cornerstone of evidence-based medicine. They provide invaluable information on the safety and efficacy of therapies and, due to the randomisation approach, typically have high internal validity (ie, reliability and robustness of the study) that helps to minimise effects from bias, and also controls for potential confounding factors.1 However, this type of trial design is not without limitations6: strict inclusion/exclusion criteria may limit the clinical relevance of the results, as it is difficult to extrapolate and generalise the study findings to the wider patient population. Also, while traditional RCTs focus on drug efficacy (ie, ‘what a drug can do in a strictly defined setting’), they do not necessarily focus on drug effectiveness (ie, ‘what a drug does in normal clinical practice/a real-world setting’). Further criticisms of traditional RCTs are that patients may demonstrate higher adherence to a medication regimen when closely observed by the researchers than they would outside the clinical trial setting, and that their design may lead to over-reporting or under-reporting of adverse events and healthcare resource utilisation (HRU).

RWE studies aim to reflect routine clinical practice by focusing on individual patient-level data, rather than population-level or disease-level data independent of treatment performance, as in the RCT setting. RWE studies have expanded inclusion criteria for participation, more diverse populations and a reduced number of study visits, compared with traditional RCTs. This increases their external validity (ie, applicability across a general population) and, subsequently, enables an assessment of clinical outcomes and drug safety in a broad, real-life setting—for instance, the inclusion of patients with comorbidities,1 and those that are taking other medications usually excluded from traditional RCTs. RWE studies have typically not been randomised and are one of two study designs: retrospective real-world data analyses (eg, using information from healthcare databases, electronic medical records (EMRs), insurance claims or prescription records), or non-interventional observational studies (see figure 1 for a summary of RWE study types). The main limitation of such RWE studies, particularly observational studies lacking randomisation, is the potential for confounding effects and bias (eg, indication or attrition bias).1 RWE studies are also limited by the reliability and completeness of the data sources used. Observed data may differ between geographical regions and/or populations due to multiple local factors and may not reliably reflect the profound effects of health inequalities (including variations in health economies, healthcare access, health literacy and beliefs and socioeconomic statuses) on patient outcomes. Importantly, as safety signals for new interventions may not emerge until treatments are available in the clinical practice, RWE can be used to inform drug safety following approval and may also play a part in supporting regulatory approval.7 Moreover, RWE is useful for supplementing the findings of RCTs with regard to insights on cost-effectiveness of new drugs.

Figure 1

Types of RWE studies and sources of real-world data. Hybrid studies include, for example, collection of retrospective claims data and prospective patient survey data. EHR, electronic health records; RCT, randomised controlled trial; RWE, real-world evidence.

A summary of the pros and cons of traditional RCTs and RWE studies is shown in figure 2.

Figure 2

The pros and cons of RCTs and RWE studies. AE, adverse event; HRU, healthcare resource utilisation; RCT, randomised controlled trial; RWE, real-world evidence.

Using RWE in clinical guidelines and decision-making

The American Thoracic Society (ATS)6 and US Food and Drug Administration (FDA)8–11 have reported their support for integrating RWE in the clinical guidance and decision-making. The ATS considers data from observational studies to complement those from RCTs, providing additional high-quality evidence.6 The FDA uses RWE as a primary source of clinical evidence to guide regulatory decisions, and it supports the increasing use of RWE to build on current understanding of clinical outcomes.11 In a landmark decision, the FDA announced the approval in July 2021 of tacrolimus (Prograf, Astellas Pharma US), an immunosuppressant drug, for use in patients receiving lung transplants, based on RWE from an observational study.10 This approval based on observational data demonstrates the value of incorporating RWE in clinical decision-making.

The European Respiratory Society (ERS) has also outlined the benefits of implementing RWE into the development of clinical practice guidelines, complementing data obtained from RCTs by confirming RCT populations, safety data, exploring additional clinical outcomes and treatment effects to be investigated in prospective RCTs.12 The ERS approach aims to use RCT and RWE data to provide the best possible evidence for clinical decision-making. Newer electronic systems will be invaluable in this approach, as will the utilisation of new evidence evaluation systems that are not considered in traditional approaches for guideline development: for instance, The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system,13 14 a stepwise process that can be used to assess the quality of evidence in studies15 (including pragmatic RCTs) and, subsequently, inform their use in guiding healthcare recommendations.

Challenging the status quo with pragmatic RCTs

Despite their limitations, RWE studies nevertheless provide complementary evidence to traditional RCTs that can fill potential data gaps16; for example, HRU data inform decisions concerning reimbursement, guideline development and healthcare service design, and provide additional information on patient-reported outcomes and/or safety of a treatment, but also information on patient populations not included/under-represented in RCTs. RWE studies can be particularly useful in detecting rare events or safety signals, for analysis of data from national disease/specific therapies registries, and are beneficial in cases where randomisation of patients is impractical or unethical. This could include, for example, evaluating treatments for very rare diseases (eg, orphan medicinal products) or interventions for life-threatening conditions.16

Building on the design of RWE studies (including those without randomisation), pragmatic RCTs have now become possible due to various technological advances that enable remote, non-intrusive monitoring of patients (eg, EMRs, smart devices and mobile/web applications). Pragmatic RCTs are undertaken within routine clinical practice with ethics approval, and can generate greater quantities of higher-quality evidence than non-randomised RWE studies; these could be in the context of research or service evaluation (pilot studies), but they also provide an opportunity to assess the effectiveness of an intervention in a real-life setting. This potentially encourages more natural behaviour than would be observed under strict observation in a traditional RCT. One example of a pragmatic RCT is the Salford Lung Study (SLS) in patients with COPD (NCT01551758)17 and asthma (NCT01706198)18: these were the world’s first prelicence pragmatic RCTs to compare the real-world effectiveness of a novel investigational inhaled therapy with existing treatments for COPD and asthma.19 Other examples of recent effectiveness trials in the respiratory field are the INvestigation of TRELEGY Effectiveness: usual PractIce Design (NCT03467425)20 and the TRICOP21 studies, both in COPD.

The findings from studies such as these can be used to help design future studies in respiratory medicine and inform guideline development for COPD and asthma. Importantly, these research methodologies could be applied across a range of chronic diseases. These types of studies are useful in informing guidelines because they reflect a broader patient population (with comorbidities and taking other medications) that is not seen in traditional regulatory RCTs. If findings align with the results of RCTs then prescribers are reassured regarding the medicine’s profile in the real-world setting. It is noteworthy that it is not unexpected for the outcomes of pragmatic RCTs to be less positive than typically reported by traditional RCTs, observations that are rooted in key differences in study design (such as differing trial durations and study endpoints (table 1)); the inclusion in pragmatic RCTs of a more heterogeneous patient population that better represents everyday clinical practice than traditional RCTs; and generally lower treatment adherence that could all contribute to lower treatment effects and less positive outcomes in pragmatic versus traditional RCTs. Pragmatic RCTs also have an important role in generating hypotheses for and contributing to the design of more efficient RCTs, especially with more novel data types, predictive analytics and machine learning to drive analysis. This facilitates the undertaking of larger, more representative studies, where the difference between ‘classic’ RCTs and RWE studies starts to disappear. Importantly, it should be noted that such RWE can also be used to evaluate treatment comparisons.

Table 1

Examples of different outcomes in pragmatic and traditional RCTs

Study	Type	RWE used?	Indication	Total patients randomised, N	Primary outcome	Proportion of patients who dropped out, %
SLS COPD(NCT01551758)17	Pragmatic RCT	Yes	COPD	2802	FF/VI cut the rate of moderate or severe exacerbations by 8.4% vs usual care	7.0
INTREPID(NCT03467425)31	Pragmatic RCT	Yes	COPD	3092	Patients treated with FF/UMEC/VI had a significantly greater proportion of CAT responders at week 24 vs non-ELLIPTA MITT	3.3
SLS asthma(NCT01706198)18	Pragmatic RCT	Yes	Asthma	4233	Patients were more likely to achieve an ACT score >20 or an increase of ≥3 in ACT score at week 24 if they initiated treatment with FF/VI rather than usual care	9.0
TORCH(NCT00268216)32	Traditional RCT	No	COPD	6184	FP/SAL reduced the risk of death by 17.5% vs placebo	34.0–44.0
SUMMIT(NCT01313676)33	Traditional RCT	No	COPD	16 590	Treatment with FF/VI or its componentsFF and VI did not affect the risk of all-cause mortality vs placebo	23.0–29.0
FLAME(NCT01782326)34	Traditional RCT	No	COPD	3362	The annual rate of all COPD exacerbations was 11% lower with IND/GLY than FP/SAL	16.6–19.0

ACT, Asthma Control Test; CAT, COPD Assessment Test; COPD, chronic obstructive pulmonary disease; FF, fluticasone furoate; FLAME, Effect of Indacaterol Glycopyrronium vs Fluticasone Salmeterol on COPD Exacerbations; FP, fluticasone propionate; GLY, glycopyrronium; IND, indacaterol; INTREPID, INvestigation of TRELEGY Effectiveness: usual PractIce Design; MITT, multiple-inhaler triple therapy; RCT, randomised controlled trial; RWE, real-world evidence; SAL, salmeterol; SLS, Salford Lung Study; SUMMIT, Study to Understand Mortality and Morbidity In COPD; TORCH, TOwards a Revolution in COPD Health; UMEC, umeclidinium; VI, vilanterol.

Pragmatic RCTs also have value in exploring complex interventions and digital technologies (eg, connected digital inhaler platforms and smart inhalers, which allow clinicians to evaluate inhaler use and technique and, in some cases, inhalation profiles),22 and also provide important safety data which cannot be collected by traditional RCTs. The Medicines and Healthcare products Regulatory Agency (MHRA) cautions the use of tiotropium (a long-acting muscarinic antagonist medicine) in patients with arrhythmias and heart failure,23 despite cardiovascular comorbidities being common among patients with COPD; such patients are frequently excluded from traditional RCT populations (sometimes referred to as the ‘COPD trial paradox’),1 thus creating an evidence gap in this particular group of patients. Focusing on, for example, subanalyses of larger datasets, could provide insights into unexpected or rarer outcomes in patients with respiratory disorders, such as the effects of beta blockers and statins on reducing mortality in patients with COPD,24 25 thereby helping to alleviate evidence gaps among patients of special interest. Furthermore, analysing safety data both by actual treatment received and by randomised treatment strategy can help to optimise the understanding of safety data from pragmatic RCTs.26 Compared with traditional RCTs, pragmatic RCTs also tend to be less burdensome on participating patients because they have fewer study visits and procedures, which may subsequently lead to fewer study dropouts (table 1).

Reducing the limitations of pragmatic RCTs

As with any activity that involves data collection from individuals, there are unavoidable limitations in the design of pragmatic RCTs that need to be acknowledged. Although the use of digital interventions greatly increases the capacity for data collection in a real-world setting, compared with non-randomised RWE studies or traditional RCTs, this presents several challenges relating to data collection, storage and usage (General Data Protection Regulation procedures must always be followed), and ensuring that appropriate ethics requirements are adhered to across patient groups. In addition, varying designs and degrees of complexity among digital interventions will influence the quality of the data retrieved; new technology may also require expensive hardware and software that are not affordable or accessible by patients in deprived areas or low-income and middle-income countries, and some patients (including the elderly or those with disabilities or learning difficulties) may struggle to correctly use digital technologies. Observed differences or similarities in the apparent effectiveness of drugs may arise from differences in medication adherence, for example, rather than the clinical properties of the drugs themselves, which could potentially lead to incorrect medication recommendations.

Pragmatic RCTs and the future of guiding clinical practice

From a stakeholder perspective in decision-making for clinical practice, regulators prefer traditional RCTs because these provide robust clinical safety and efficacy data. Payers (funders and developers) are interested in clinical safety and efficacy data from traditional RCTs in the early stages of drug development, and the cost-effectiveness/market share data from RWE studies/pragmatic RCTs after drug licensing. Commissioners are interested in traditional RCTs for their relevance to the clinical sphere and RWE studies/pragmatic RCTs for their information on drug effectiveness; patients are also interested in traditional RCTs to determine the cost and burden of a drug, and RWE studies/pragmatic RCTs to determine whether a drug helps improve their quality of life. Different stakeholder needs could explain why, historically, regulatory guidance has been cautious or even sceptical about using RWE generated from non-randomised retrospective or observational studies to guide clinical practice; however, through the evolution of RWE studies and pragmatic RCTs, there is increasing awareness among regulators and payers on the potential value of RWE to provide clinically meaningful information about the safety and effectiveness of medical products. The FDA published a strategic framework in December 2018 to support the use of RWE in regulatory decision-making,11 and the European Medicines Agency has published a draft guideline on the use of registry-based studies to support regulatory decision-making that underwent a public consultation from 24 September 2020 to 31 December 202027 (a final document has not been published at the time of writing). In October 2020, the MHRA published draft guidance on using pragmatic RCTs to support regulatory decisions,28 and a recent policy paper published by the UK Department of Health and Social Care in February 2021 proposed that the MHRA develop and maintain national medicines registries to encourage focus on postlaunch evaluations and RWE, thereby combining primary care databases with epidemiological analyses of patient-level data.29 However, despite this increased recognition, data from pragmatic RCTs are still not recognised within regulatory guidance as having the same value in guiding patient care as evidence from traditional RCTs. The focus continues to be on efficacy and achieving minimal clinically important differences and/or statistically significant results, rather than examining the broader effectiveness of an intervention. For example, although the Global Initiative for Chronic Obstructive Lung Disease Report 2021 discusses findings from the SLS in COPD (NCT01551758), it is hesitant to base treatment recommendations on the study results due to the inherent heterogeneity of treatments reported in the usual care group.30 Regulatory guidance should, therefore, look outside the narrow confines of traditional RCTs to view pragmatic RCTs as a source of valuable additional evidence on the effectiveness of a medicine. In addition, methodologies for rating the robustness of studies (such as the GRADE system)13–15 better recognise the value of pragmatic RCTs and incorporate such trials into the assessment framework for clinical practice recommendations.

Conclusions

RWE offers significant value in adding to the evidence base of therapies/interventions, and should be routinely considered to guide clinical practice as a complementary and supportive addition to the evidence generated from traditional RCTs. RWE studies offer greater patient inclusivity and thus have findings applicable across a broader general population/greater external validity than traditional RCTs.