Delayed-start analysis for demonstrating disease modification of solanezumab.

To the editor:

Recently in Alzheimer's and Dementia: Translational Research and Clinical Interventions, Liu-Seifert et al. [1] described the results of the EXPEDITION-EXT study, an open-label extension of the randomized EXPEDITION and EXPEDITION2 trials. Although the original trials assessed the efficacy of solanezumab in patients with mild-to-moderate Alzheimer's disease (AD) [2], the open-label extension was intended for safety-monitoring (https://clinicaltrials.gov/ct2/show/NCT01127633). The authors implemented certain modifications to the delayed-start analysis [3], first proposed by Leber [4] in 1996 for differentiating disease-modifying treatment effects from symptom relief, and applied these to the EXPEDITION-EXT data. From this, they conclude that their proposed modifications of the delayed-start methodology are “appropriate for ascertaining long-term effectiveness and possible disease-modifying effects of AD treatments.” Although the presented methodology is interesting, we believe there are several limitations to the method and analyses, which are insufficiently, if at all, addressed in the article.

First, since its introduction in 1996, the delayed-start method has rarely been applied in practice, in part, because of uncertainty about how to determine the difference between groups in the extension phase. Here, the authors use a noninferiority approach. Even in case of perfect study attrition, this approach suffers from much lower power than the original trial because of comparison of two measured differences with their separate variances. In fact, the power for their model was estimated by the authors to be less than 50% [3]. Moreover, it is arguable whether the chosen 50% noninferiority margin (reflecting a 0.9-point gain on the 90-point ADAS-Cog14 scale) qualifies as the “largest clinically acceptable loss of treatment benefit” strived for. In comparison, acetylcholinesterase inhibitors cause an average 2.7-point improvement on the 70-point ADAS-Cog scale [5], and clinical benefits are generally considered marginal.

Second, the delayed-start design is susceptible to bias for several reasons. Of the original 1322 participants in EXPEDITION and EXPEDITION2 with mild AD, only 975 (73.8%) were enrolled in the extension phase, of whom 581 (43.9%) completed 2-year follow-up [1]. Equal drop-out percentage in both treatment groups does not rule out selection bias, when data are not missing completely at random. Furthermore, even nondifferential misclassification of outcome can introduce bias toward the 0. For the delayed-start design, this means that nondifferential misclassification suffices to have no converging of groups in the extension phase, and, thus, falsely imply noninferiority. Moreover, randomization at baseline does not prevent post-randomization confounding and selection bias [6]. For instance, use of acetylcholinesterase inhibitors during the study period may differ over time between groups, which is not accounted for by adjusting for baseline use.

Third, concerning the analyses, the authors fit mixed models to various time-points during follow-up and conclude treatment differences persisted during “much of the delayed-start period.” Although the treatment differences indeed persisted up till a year, at the end of the 2-year follow-up period, differences were no longer significant and the noninferiority criteria for the primary outcome measures ADAS-Cog14 and ADCS-iADL no longer met. In any case, assessing each time-point separately is equivalent to performing (in this case 9) interim analyses, for which correction of the P value threshold for statistical significance should be made.

Finally, none of the brain imaging markers during the EXPEDITION trials showed any difference in favor of treatment with solanezumab [2]. As these were measured to detect disease modification, discrepancy between established biomarkers of disease progression and analytical methods as the delayed-start design should be met with caution. Before considering the delayed-start design as a valid, stand-alone method for demonstrating disease modification by a treatment, we believe validation on well-established disease-modifying and symptomatic drugs for other conditions than AD is warranted.

Since presentation and subsequent publication of the delayed-start findings of the EXPEDITION trials [3], media have reported this as a breakthrough in treatment for AD [7], which has had its effects on the lay public, policy makers, even stock markets, and other stakeholders. We hope to have clarified here why this euphoria is at best premature.