Suitability of external controls for drug evaluation in Duchenne muscular dystrophy.

OBJECTIVE: To evaluate the suitability of real-world data (RWD) and natural history data (NHD) for use as external controls in drug evaluations for ambulatory Duchenne muscular dystrophy (DMD).
METHODS: The consistency of changes in the 6-minute walk distance (Δ6MWD) was assessed across multiple clinical trial placebo arms and sources of NHD/RWD. Six placebo arms reporting 48-week Δ6MWD were identified via literature review and represented 4 sets of inclusion/exclusion criteria (n = 383 patients in total). Five sources of RWD/NHD were contributed by Universitaire Ziekenhuizen Leuven, DMD Italian Group, The Cooperative International Neuromuscular Research Group, ImagingDMD, and the PRO-DMD-01 study (n = 430 patients, in total). Mean Δ6MWD was compared between each placebo arm and RWD/NHD source after subjecting the latter to the inclusion/exclusion criteria of the trial for baseline age, ambulatory function, and steroid use. Baseline covariate adjustment was investigated in a subset of patients with available data.
RESULTS: Analyses included ∼1,200 patient-years of follow-up. Differences in mean Δ6MWD between trial placebo arms and RWD/NHD cohorts ranged from -19.4 m (i.e., better outcomes in RWD/NHD) to 19.5 m (i.e., worse outcomes in RWD/NHD) and were not statistically significant before or after covariate adjustment.
CONCLUSIONS: We found that Δ6MWD was consistent between placebo arms and RWD/NHD subjected to equivalent inclusion/exclusion criteria. No evidence for systematic bias was detected. These findings are encouraging for the use of RWD/NHD to augment, or possibly replace, placebo controls in DMD trials. Multi-institution collaboration through the Collaborative Trajectory Analysis Project rendered this study feasible.

Enrollment challenges for clinical trials of Duchenne muscular dystrophy (DMD) have intensified the need to make informative use of real-world data (RWD) and natural history data (NHD) in drug evaluations. The enrollment challenges stem from the rarity of DMD itself, the further rarity of certain treatment-amenable genetic subtypes, and the practical and ethical implications of placebo arms for a progressive, life-limiting disease with few treatment options. Compounding these enrollment challenges, variability in primary outcomes such as the 6-minute walk distance (6MWD)[1] makes it difficult to detect, or exclude, clinically important treatment effects with readily achievable sample sizes. In this setting, using RWD/NHD to contextualize, augment, or potentially replace placebo arms could significantly accelerate the development of effective therapies for DMD.

However, use of RWD/NHD controls for patients enrolled in clinical trials raises well-founded concerns. It is reasonable to ask whether differences between these settings might produce a disqualifying bias in patient outcomes; indeed, the potential for such bias has been observed in other neuromuscular diseases[2] and noted by regulators.[3] Particular points of concern in DMD are potential differences in patient characteristics, background therapies,[4,5] and levels of motivation[6,7] and fatigue, the last being especially relevant for performance-based outcomes such as 6MWD.

The suitability of RWD/NHD controls in DMD depends on the magnitude of these potential biases. To assess this bias empirically, we conducted a multi-institution, multi-registry study comparing 48-week changes in 6MWD between DMD clinical trial placebo arms and RWD/NHD, accounting for inclusion criteria and baseline prognostic factors.

Methods

Data sources

Data were obtained from 6 clinical trial placebo arms and 5 RWD or NHD sources. RWD for this study were contributed by Universitaire Ziekenhuizen Leuven. Participating collaborators contributing NHD included the DMD Italian Group, the CINRG Duchenne Natural History Study (DNHS), ImagingDMD, and the PRO-DMD-01 prospective natural history study, from which data were provided to the Collaborative Trajectory Analysis Project (cTAP) by CureDuchenne, a DMD patient foundation.

Clinical trial placebo arms

Clinical trials in DMD were identified via reviews of PubMed, clinicaltrials.gov, and briefing documents posted by the Food and Drug Administration. Placebo arms were included if they reported change from baseline in 6MWD at 48 weeks in patients with DMD. The following trials were identified as eligible, and their placebo arms were included in the present study: tadalafil DMD phase 3 trial (NCT01865084)[8]; ataluren phase 2b (NCT00592553)[9]; ataluren phase 3 (NCT01826487)[10]; drisapersen phase 2 (pooled data across 2 phase 2 trials [NCT01153932 and NCT01462292]), and drisapersen phase 3 (NCT01254019).[11] Published results from the 6 identified clinical trial placebo arms were extracted in duplicate by 2 reviewers working independently.

These 6 clinical trial placebo arms encompassed 4 distinct sets of key inclusion criteria that differed with respect to baseline age and functional status (table 1). The ataluren phase 2b trial did not require enrolled patients to be on steroids, but all other trials required patients to have used steroids for at least 6 months and to have been on a stable corticosteroid regimen for at least 3 months before trial entry. All trials were 48 weeks in duration. Results from the 2 phase 2 trials of drisapersen were combined for analyses due to equivalent inclusion/exclusion criteria and the relatively small number of patients in each trial.

Table 1

Key inclusion/exclusion criteriaa for trials

RWD and NHD sources

RWD/NHD were drawn from the following sources, which collectively represent >50 care centers. Key characteristics of each data source are summarized in table 2.

Table 2

Characteristics of included RWD/NHD sources

Universitaire Ziekenhuizen Leuven (Leuven)

Data were collected from boys with DMD who were monitored in routine clinical practice at the Universitaire Ziekenhuizen pediatric neurology clinic in Leuven, Belgium. Clinic visits occurred approximately every 6 months.

DMD Italian Group

The DMD Italian Group is a registry of 13 tertiary neuromuscular clinical centers in Italy. Data were collected from routine clinical practice and curated at 12-month intervals.

The CINRG DNHS

CINRG is a clinical trial network comprising >20 worldwide clinical study sites.[12] The specific data used in this study were from the CINRG DNHS (NCT00468832),[13,14] a prospective longitudinal study of a cohort of patients with DMD.[13] For ambulatory patients, assessments were conducted every 3 months for the first year of follow-up, every 6 months in the second year of follow-up, and annually thereafter.

ImagingDMD

ImagingDMD (NCT01484678)[15] is a longitudinal, multicenter, observational study of boys with DMD and age-matched controls without DMD. Only data from boys with DMD were included in this study. Study participants were recruited from across the United States and received assessments at 1 of 3 sites (University of Florida, Oregon Health & Science University/Shriners Hospital Portland, and the Children's Hospital of Philadelphia). Data were collected every 12 months and every 3 or 6 months in the first year in a subset of patients.

PRO-DMD-01

The PRO-DMD-01 study (NCT01753804) was a prospective observational study of disease progression in boys with DMD sponsored by BioMarin Pharmaceutical. Data were provided by CureDuchenne, a 501(3)c DMD patient foundation. The study included patients from 16 centers in the United States, Argentina, Belgium, Brazil, France, Germany, Italy, the Netherlands, Sweden, and Turkey. Study assessments occurred every 6 months.

Standard protocol approvals, registrations, and patient consents

RWD/NHD sources were approved by ethics committees from each institution (the University Hospitals Leuven [Leuven], Catholic University, Rome [DMD Italian Group], each participating center for The Cooperative International Neuromuscular Research Group [CINRG] and PRO-DMD-01, the University of Florida, Children's Hospital of Philadelphia, and Oregon Health and Science University [Imaging DMD]). Written informed consent/assent was obtained from each participant or caregiver as appropriate before the study procedures were conducted.

Outcome assessments

This study focused on 48-week change in 6MWD (Δ6MWD), which served as the primary outcome in each of the included clinical trials. All assessments of 6MWD in the placebo arms and RWD/NHD sources were based on modified American Thoracic Society criteria[1] and administered by trained assessors or clinical experts. Training procedures are described for each data source in table 2. Participants who were unable to complete the 6-minute walk test at the time of their assessment were assigned a 6MWD of 0 meters.

Sample selection in RWD/NHD sources

Separate samples of RWD/NHD were drawn for each set of clinical trial inclusion/exclusion criteria. To be included in the analysis, an interval of follow-up from RWD/NHD was required to meet each of the following criteria: (1) at the first clinic visit in the interval (referred to as the baseline visit), the patient met the age, steroid duration, and functional criteria specified in the inclusion/exclusion criteria of the comparator trial as detailed in table 1; (2) the baseline visit and a subsequent follow-up visit were separated by ∼48 weeks (9–13 months, inclusive); and (3) 6MWD was assessed at the baseline and endpoint visits (figure 1). If a patient had multiple intervals meeting these criteria, all nonoverlapping intervals were included in the analyses. The endpoint visit for 1 interval was allowed to serve as the baseline visit for the subsequent interval, but further overlap was disallowed.

Figure 1

Study design diagram

6MWD = 6-minute walk distance.

Statistical analysis

For RWD/NHD sources, observed changes in 6MWD over the 9- to 13-month period were rescaled to reflect 48-week changes in 6MWD. If a patient lost ambulation before his endpoint visit or if a linear rescaling of his change in 6MWD resulted in a projected 6MWD at 48 weeks of <0 meters, the patient was assumed to have lost the ability to complete the 6MWD assessment by week 48, and their Δ6MWD was the negative of their baseline 6MWD.

In each RWD/NHD source, means and standard errors of Δ6MWD were estimated from all eligible patient intervals. When estimating standard errors for statistical comparisons, we used generalized estimating equations with an exchangeable covariance structure to account for use of multiple intervals from individual patients.[16] Pairwise differences in mean Δ6MWD among RWD/NHD cohorts subjected to the same inclusion/exclusion criteria were also assessed. Control of the false discovery rate[17] was used to assess statistical significance across multiple comparisons.

Sensitivity analyses

Two sets of sensitivity analyses were conducted. First, to assess whether differences in distribution of steroid type at baseline influenced the results, Δ6MWD was compared within the subgroup of patients receiving deflazacort, which was the most commonly used steroid (68%–84%) in the RWD/NHD sources with such information available. Second, to assess the sensitivity of the results to the time frame used to approximate 48-week follow-up periods in RWD/NHD sources, the analysis was repeated using observed changes that were 10 to 12 months apart, instead of 9 to 13 months as used in the primary analyses.

Covariate adjustment

Additional analyses comparing Δ6MWD between placebo arms and RWD/NHD after baseline covariate adjustment were conducted using data sources for which patient-level data, with measurement of 6MWD and the prognostic factors listed below, were currently available within cTAP. These included the placebo arms from 3 trials (tadalafil DMD phase 3, ataluren phase 2b, ataluren phase 3) and from 3 RWD/NHD sources (Leuven, ImagingDMD, PRO-DMD-01). Similar to the analyses of RWD/NHD sources described above, these analyses were based on all nonoverlapping intervals of ∼48-week changes. To be included in these analyses, patients were required to have 6MWD ≥75 m at the baseline visit of the interval and 6MWD available at both the baseline and follow-up visits. Multivariable regression models, with generalized estimating equations to account for the use of multiple follow-up intervals from some patients, were used to compare changes in 6MWD between trial placebo and RWD/NHD after adjustment for age and known prognostic factors: steroid duration, baseline 6MWD, steroid type, ability to perform rise from supine, and completion times for timed function tests (rise from supine, 4-stair climb, and 10 meter walk/run),[13] as well as height, weight, and body mass index.[18]

Data availability

All relevant data are reported within the article. Data requests may be directed to the individual institutions and clinical networks that have collected and curated patient data. These organizations (Universitaire Ziekenhuizen, the DMD Italian Group, CINRG, Imaging DMD, and CureDuchenne) will consider data requests according to their own data-sharing policies and governance.

Results

Patient characteristics

The analyses included n = 383 patients on placebo arms, each contributing one ∼48-week follow-up interval, and n = 430 patients contributing a total of 919 ∼48-week follow-up intervals from RWD/NHD sources. The numbers of patients and intervals analyzed varied according to the specific inclusion/exclusion criteria applied to each source (table 3).

Table 3

Patient characteristics in trial placebo arms and RWD/NHD sources subjected to the inclusion/exclusion criteria of the trial

After harmonization of inclusion/exclusion criteria, the mean age was generally older for participants in RWD/NHD sources compared to placebo arms by 1 to 2 years, except for CINRG DNHS, in which boys were closer to 3 years older on average for most comparisons (table 3). In nearly all of the comparisons between trial placebo arms and harmonized RWD/NHD cohorts, baseline 6MWD was within ±30 m; the largest difference was 74 m for placebo. Data on type of steroid used at baseline were not always available for comparison between trials and RWD/NHD sources. However, for comparisons for which steroid type was available, deflazacort was more commonly used in these RWD/NHD sources than in the corresponding trials.

Comparison of 48-week changes in 6MWD

Differences in mean Δ6MWD between trial placebo arms and harmonized RWD/NHD cohorts ranged from −19.4 m (indicating smaller declines in RWD/NHD than in placebo) to 19.5 m (indicating larger declines in RWD/NHD than in placebo) (figure 2). Mean Δ6MWD in RWD/NHD cohorts was numerically smaller in magnitude of decline (indicating better preservation of function) than placebo in 17 comparisons and larger than placebo in 7 comparisons. None of the differences observed between trial placebo arms and RWD/NHD cohorts were statistically significant (table 4).

Figure 2

Forty-eight-week changes in 6MWD in trial placebo arms and harmonized real-world data/natural history data

CI = confidence interval; CINRG DNHS = The Cooperative International Neuromuscular Research Group Duchenne Natural History Study; DMD = Duchenne muscular dystrophy; 6MWD = 6-minute walk distance.

Table 4

Forty-eight–week changes in 6MWD in trial placebo arms vs harmonized RWD/NHD

Among the harmonized RWD/NHD cohorts, means for Δ6MWD were all within 25 m of each other; only 1 of the 46 pairwise comparisons of RWD/NHD cohorts reached a nominal p < 0.05. This was not statistically significant at the 5% level after adjustment for multiple comparisons.

Analyses among deflazacort users at baseline were possible for comparisons of RWD/NHD against placebo arms from the tadalafil DMD phase 3 and ataluren phase 3 trials; these were the only 2 trials that reported outcomes in deflazacort-treated subpopulations. Analyses were done only among deflazacort users because deflazacort was the corticosteroid predominantly used in the RWD/NHD sources analyzed here. As in the primary analyses, mean Δ6MWD was similar between the trial placebo arms and RWD/NHD sources in this subgroup (table 5). Separately, sensitivity analyses based on follow-up visits spanning 10 to 12 months of follow-up in RWD/NHD data sources yielded results similar to those observed in the primary analyses: differences in mean Δ6MWD between trial placebo arms and harmonized RWD/NHD cohorts ranged from −18.3 to 20.4 m, and none reached statistical significance (results not shown).

Table 5

Sensitivity analysis among deflazacort users: 48-week changes in 6MWD in trial placebo arms vs harmonized RWD/NHD

Analyses with adjustment for baseline covariates were based on 822 intervals of follow-up among patients in placebo arms of the tadalafil phase 3, ataluren phase 2b, and ataluren phase 3 trials (239 intervals from 239 patients) and from the Leuven, ImagingDMD, and PRO-DMD-01 RWD/NHD sources (583 intervals from 274 patients). In this sample, the difference in mean Δ6MWD between placebo and RWD/NHD before adjustment was −8.7 m (p = 0.2), numerically indicating greater decline in placebo than in RWD/NHD. After adjustment for known baseline prognostic factors, the difference in mean Δ6MWD was 7.7 m (95% confidence interval −3.8 to 19.3; p = 0.2).

Discussion

The findings of this study are supportive of the use of RWD/NHD to contextualize, augment, or potentially replace placebo arms in DMD clinical trials and thus have important implications for drug evaluation in DMD. In particular, a fundamental motivation for this research was the concern that external controls might not be suitable for DMD because performance-based outcomes such as 6MWD could be biased by differences between clinical trial and RWD/NHD settings.[2,3] We analyzed >1,000 patient-years of follow-up, encompassing data from all available clinical trial placebo arms and from 5 multinational, multi-institutional RWD/NHD sources. We found that changes in 6MWD over 48 weeks were strikingly consistent between these settings when subjected to equivalent inclusion/exclusion criteria and after adjustment for multiple known prognostic factors. There was no evidence that changes in 6MWD were systematically milder in placebo arms compared to RWD/NHD. The 6MWD outcomes were also consistent among the different RWD/NHD sources analyzed. From these findings, we conclude that external controls can be suitable for drug evaluations in DMD.

The notable consistency we observed in 6MWD outcomes occurred despite a number of important differences across the studied data sources. Patients in the placebo arms and their caregivers and providers were blinded to receipt of placebo but were also aware of the patient's enrollment in an interventional trial and the possibility of receiving active therapy, a possibility that was absent in the RWD/NHD. With this awareness, patients in the placebo arms may have experienced different levels of hope for improvement or motivation during functional tests or received different cues, even unintentionally, from family members or from the health care professionals administering the 6MWD tests compared to those in RWD/NHD. In addition, boys enrolled in placebo arms were younger on average than those in the corresponding RWD/NHD cohorts. Because declines in 6MWD are progressive and tend to accelerate with age, this age difference would be expected to bias toward more favorable 6MWD outcomes in the placebo arms. Geography also varied, with different RWD/NHD sources representing centers in North America, Europe, Oceania, and Asia; the multicenter clinical trials also represented diverse geographies. While some of the RWD/NHD data were drawn from centers providing care consistent with published recommendations[19,20] (Leuven, DMD Italian Group, ImagingDMD), others captured data from a more globally diverse collection of study sites (PRO-DMD-01, CINRG) across which standards of care may have been more variable. In the absence of evidence to the contrary, these differences would have supported appropriate concerns about risk of bias in comparing effort-based outcomes such as the 6MWD across these settings and data sources. The findings of the present study mitigate such concerns and indicate that differences between clinical trial and RWD/NHD settings in DMD are unlikely to drive significant bias in external controls for 6MWD outcomes in DMD.

While these results are supportive of the use of external controls in DMD, we emphasize that any future application of external controls would need to be evaluated on a case-by-case basis. The results of the present study are most directly generalizable to comparisons of 48-week change in 6MWD, measured by trained assessors following modified American Thoracic Society criteria, between ambulatory, steroid-treated individuals in randomized placebo-controlled trials and the studied RWD/NHD sources. It is especially important to note that all RWD/NHD sources included in our study used consistent methodologies to assess 6MWD as previously reported.[1] Physical therapists and investigators at these centers are often involved in concurrent clinical trials and are therefore trained with similar manuals and instructions. This highlights the importance of consistency of clinical assessments across care centers in DMD for maximizing the value of patient data for research and the development of therapies. Finally, it should be noted that comparisons between RWD/NHD and single-arm or uncontrolled studies would require an additional layer of caution; patients included in those studies are certain that they are receiving active therapy, whereas patients in the placebo arms included in the present study had only a probability of receiving blinded active therapy.

An important factor that could not be fully explored in this study was the use of steroids at different doses or administration frequencies. While all patients included in the analysis were receiving steroids, consistent with standard-of-care recommendations and clinical trial inclusion criteria, the particular steroid types, dosing regimens, and ages at initiation were not always available and are likely to have varied across time and across care centers. Sensitivity analyses conducted among patients receiving deflazacort, which was the most commonly used steroid in the RWD sources with such information available, and multivariable analyses adjusted for steroid type were consistent with the primary analyses and did not indicate any significant differences in ∆6MWD between placebo and RWD/NHD.

Overall, the results of this study provide a strong foundation for use of RWD/NHD in DMD drug development and confirm the worthiness of additional research to extend these findings to additional clinical outcomes and to evaluate different study designs for using RWD/NHD in DMD. The overall promise of such research is further supported by the passage of the 21st Century Cures Act (2016) in the United States, which has spurred an increased need for understanding and evaluating appropriate uses of RWD in regulatory decision-making[21,22] and an emerging framework for doing so.[23,24] Representatives of the European Medicines Agency and the Organisation for Economic Co-operation and Development have also highlighted the importance of RWD/NHD for drug evaluation.[25-27] Comparisons to external controls have proved important to drug approvals in several rare diseases, including neuronal ceroid lipofuscinoses type 2 (a form of Batten disease)[28] and Pompe disease.[29] Against this background, the use of RWD/NHD in DMD clinical trials is of particular interest because it could make efficient use of existing patient data, facilitate faster trial enrollment, and enable more patients to access active therapies as opposed to placebo.

A number of specific-use cases for RWD/NHD have merit in DMD drug development and are supported by the consistency in 6MWD outcomes observed in the present study. Placebo augmentation, for example, is being studied in DMD[30] as a way to preserve much of the benefit of a randomized placebo arm while limiting the proportion of patients who receive placebo, e.g., to a 1:3 or 1:4 ratio with active therapy. In a placebo augmentation design, the randomized placebo arm data provide an internal, unbiased reference point that can be augmented with external NHD/RWD controls, provided that they exhibit reasonably consistent outcomes, to increase the statistical precision of the measured drug effect. Minimizing sample size requirements for placebo arms will be especially important, for example, in DMD trials targeting extremely rare subpopulations (e.g., ultrarare genotypes such as duplications in exon 2, which occur in only ∼1% of all patients with DMD[31]), or when invasive assessments (e.g., muscle biopsies for monitoring the protein product of a gene therapy) present practical and ethical challenges for a blinded placebo arm. In some cases, no randomized placebo data will be available, and external RWD/NHD will be the only source of potential comparative data. This often occurs in early-phase trials, in evaluations of long-term extension data after crossover of any patients receiving placebo to active therapy, or in phase 4 trials that are initiated after initial market authorization. While randomized placebo controls may not be feasible in these settings, longer-term comparative evidence based on suitable external controls may be required in regulatory evaluations after an accelerated or conditional approval, or for health economic evaluations.

Beyond the present study of 6MWD, similar assessments are needed for the consistency of additional outcomes important to DMD drug development, including the North Star Ambulatory Assessment,[32,33] Performance of Upper Limb,[34] and measures of pulmonary function.[35] The performance of different statistical methods for using RWD/NHD in DMD drug evaluation also warrants evaluation, especially in terms of type I error control (i.e., avoiding false positives) while reducing sample size requirements. This includes different approaches to placebo augmentation,[36-38] as well as approaches for making adjusted comparisons to fully external controls such as individual patient matching, propensity score–based methods,[39] and multivariable regression. Evaluating the consistency of additional outcome measures, as provided for 6MWD in the present study, will establish the basis for the selection and justification of study designs and statistical approaches for incorporating RWD/NHD into DMD drug evaluations.

Our goal of understanding the consistency of 6MWD between RWD/NHD and placebo arm settings could be thoroughly addressed only by collaboratively analyzing a comprehensive collection of data sources. Accessing patient data and conducting consistent analyses across multiple registries and geographies can be challenging; collaborating through cTAP simplified and accelerated this process. The potential impact of this study to provide stronger context for drug evaluation and to potentially reduce the number of patients who need to be enrolled in placebo arms highlights the importance of data collection, data sharing, and collaboration for DMD drug development.