Effect of reductions in amyloid levels on cognitive change in randomized trials: instrumental variable meta-analysis.

OBJECTIVE: To evaluate trials of drugs that target amyloid to determine whether reductions in amyloid levels are likely to improve cognition.
DESIGN: Instrumental variable meta-analysis.
SETTING: 14 randomized controlled trials of drugs for the prevention or treatment of Alzheimer's disease that targeted an amyloid mechanism, identified from ClinicalTrials.gov. POPULATION: Adults enrolled in randomized controlled trials of amyloid targeting drugs. Inclusion criteria for trials vary, but typically include adults aged 50 years or older with a diagnosis of mild cognitive impairment or Alzheimer's disease, and amyloid positivity at baseline. MAIN OUTCOME MEASURES: Analyses included trials for which information could be obtained on both change in brain amyloid levels measured with amyloid positron emission tomography and change in at least one cognitive test score reported for each randomization arm.
RESULTS: Pooled results from the 14 randomized controlled trials were more precise than estimates from any single trial. The pooled estimate for the effect of reducing amyloid levels by 0.1 standardized uptake value ratio units was an improvement in the mini-mental state examination score of 0.03 (95% confidence interval -0.06 to 0.1) points. This study provides a web application that allows for the re-estimation of the results when new data become available and illustrates the magnitude of the new evidence that would be necessary to achieve a pooled estimate supporting the benefit of reducing amyloid levels.
CONCLUSIONS: Pooled evidence from available trials reporting both reduction in amyloid levels and change in cognition suggests that amyloid reduction strategies do not substantially improve cognition. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Amyloid plaques and oligomers are hypothesized to cause a cascade of pathological events resulting in cognitive decline in Alzheimer’s disease.1 2 3 Motivated by the amyloid cascade hypothesis, a primary aim of many new treatments for the prevention or management of Alzheimer’s disease has been to reduce amyloid β levels in the brain.4 Although the presence of amyloid plaques and oligomers in the brain is highly correlated with the progression of Alzheimer’s disease,5 6 the mechanisms by which amyloid might mediate neuronal pathology are currently not well understood.7 To date, no anti-amyloid treatments have progressed sufficiently to receive approval from the Food and Drug Administration (the regulatory agency for pharmaceuticals in the United States).8 Drugs have targeted various amyloid species—amyloid plaques, amyloid oligomers, and soluble oligomers—and have been performed in populations with mild to moderate Alzheimer’s disease, as well as earlier stages of disease (prodromal Alzheimer’s disease).9 Most trials targeting amyloid failed to produce positive results in either early or late stages of the disease. The negative findings from these trials have prompted skepticism about amyloid’s role in neuronal disease, and many have instead argued that amyloid could be a marker for other disease processes and therefore is not a viable drug target.10 11

No single trial has, however, provided conclusive evidence about the potential impact of reduction in amyloid levels on cognitive decline. Most trials have been powered to evaluate specific treatments and were not designed to accurately estimate the effect of amyloid reduction itself on cognitive outcomes. Individual trials generally have been small, and often data on amyloid burden has been assessed only in a subsample of participants (eg, trials reviewed in12). For any individual trial of a drug targeting amyloid that fails to deliver cognitive benefits, several alternative explanations exist: amyloid reduction might not improve cognition, or the mechanism by which amyloid is targeted might affect whether and the extent to which reductions in brain amyloid levels slow cognitive decline (eg,13). Alternatively, the benefit of amyloid reduction might be too small to detect within an individual trial, a plausible situation because the expense of trials often limits the number of participants.9 This lack of statistical power (ie, the chance an effect of any given size would lead to a statistically significant result in the study) to detect an effect could be overcome by combining results from multiple trials of different drugs that all target amyloid. For example, findings from any one study might have wide confidence intervals that would not allow the ruling out of either substantial benefit or substantial harm. Combining evidence from multiple trials provides more precise effect estimates (narrower confidence intervals). To date, evidence from multiple trials of amyloid targets has yet to be systematically combined, and trial data have not been leveraged to evaluate whether changes in amyloid levels are likely to improve cognition. Using a modification of an intention-to-treat meta-analysis based on instrumental variable analyses,14 the results of multiple studies can be aggregated into one combined estimate of the effect of decreasing amyloid levels on cognitive decline.

We analyzed individually and in aggregate changes in brain amyloid levels and changes in cognition using data from randomized trials of amyloid reducing drug treatments to estimate the effect of reductions in amyloid levels on change in cognition. Using randomization as an instrumental variable for amyloid reduction, we evaluated the plausibility of the hypothesis that reductions in amyloid levels slow cognitive decline. We also aggregated results across multiple studies of drugs and evaluated the plausibility of differential effects of reduction in amyloid levels on cognitive change by drug type (antibody versus small molecule drugs).

Methods

Data sources

On 9 May 2019, we searched the Alzheimer Research Forum (alzforum.org) for “Amyloid-Related” drugs for “Alzheimer’s Disease” and “Mild Cognitive Impairment”, which yielded a comprehensive list. We then searched for trials of these drugs on ClinicalTrials.gov and restricted our searches to trials for “Alzheimer’s Disease” or “Mild Cognitive Impairment” that were “completed,” “terminated,” or “active, not recruiting.” On the basis of information available on ClinicalTrials.gov, we excluded trials that did not have a placebo control and trials that did not quantify measures of brain amyloid using the standardized uptake value ratio obtained from amyloid positron emission tomography and change in a cognitive score within randomization arms. We added information on three trials that became available during the analysis and met the inclusion criteria.

Study selection

We excluded trials for which the evidence indicated no plausible differences in standardized uptake value ratio as a result of treatment (one trial of ACC-001 (vanutide cridificar); Janssen Biotech, Horsham, PA): NCT01227564). When available, we obtained data from ClinicalTrials.gov, peer reviewed publications, or other publicly available materials, such as press releases. When data were unavailable online, we contacted pharmaceutical companies directly using telephone and email contact information posted on ClinicalTrials.gov. For the BAN2401 (lecanemab; Biogen, Cambridge, MA; Eisai, Tokyo, Japan) trial (NCT01767311),15 we obtained data from the study contact at Eisai pharmaceuticals. For the two trials of aducanumab (Biogen, Cambridge, MA), EMERGE (NCT02484547) and ENGAGE (NCT02477800), a press release was available in a portable document format with raw data (standard errors were measured from graphs). This analysis is based on all data available to us on 30 April 2020. We assumed that studies with no data available were missing completely at random. Results for a drug with notable cognitive benefits would almost certainly be publicly reported, so if the missing completely at random assumption was violated, any bias would likely be in a direction favorable to amyloid.

Data extraction and synthesis

Our approach combined instrumental variables analyses of each trial with a meta-analysis of effects across trials. This method allowed us to estimate the effect of amyloid reduction on cognitive change for each trial and to combine results for trials with different durations of follow-up. Analyses use maximum likelihood estimation. Supplementary appendices 1 and 3 provide additional details on our methods, and simulations showing unbiased results with valid confidence intervals.

First, we estimated effects within drug categories to assess evidence that the effects of reducing amyloid levels on cognitive change might differ based on the mechanism of amyloid reduction. A pooled estimate was then derived. To account for differences in populations enrolled in each trial and time in follow-up, we estimated separately for each trial how much change in cognition would be expected if the standardized uptake value ratio had not changed. In pooling results across trials, we assume that the effect on cognition of reducing amyloid levels does not vary with mechanism by which amyloid β is targeted—that is, by drug. Furthermore, our analysis is premised on the assumption that these drugs primarily act through reducing amyloid levels. If a drug effectively both reduces amyloid levels and improves cognition through other mechanisms, our estimates would attribute this to efficacy of amyloid reduction in slowing cognitive decline.

We performed sensitivity analyses restricted to antibody drugs and with and without the unpublished trials of BAN2401 and aducanumab. Supplementary appendix 1 provides additional sensitivity analyses. The analysis was repeated using alternative cognitive outcomes (clinical dementia rating scale-sum of boxes (CDR-SB) and various versions of the Alzheimer’s disease assessment scale-cognitive subscale (ADAS-Cog)), when available. We also conducted sensitivity analyses stratified by the tracer used in amyloid positron emission tomography and applied a scaling transformation to deal with the possibility that estimates from different tracers were not comparable.16 Lastly, we considered the effect of a hypothetical additional large and successful trial on our pooled results. To provide a comparison for the clinical relevance of the effect sizes, we used a similar estimation procedure to estimate the effect of carrier status of the apolipoprotein E (APOE) ε4 allele on annual change in cognition, based on the trials for which stratified cognitive outcomes were available.

Evidence from new trials, or updates of existing trials, are frequently released. The web based interface at https://amyloidintegratingevidence.shinyapps.io/Shiny/ allows for a recalculation of results with either updated data or new data when available.

Patient and public involvement

This instrumental variable meta-analysis was done without patient involvement. Patients were not invited to comment on the study design and were not consulted to develop patient relevant outcomes or interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy. Two family members of people living with Alzheimer’s disease provided valuable feedback on the framing and interpretation of results.

Results

Overall, 196 studies were identified that potentially met the inclusion criteria (fig 1). Two authors reviewed these trials, and 34 trials seemed likely to meet the inclusion criteria based on outcomes listed on ClinicalTrials.gov. Data from 14 (including 40 randomization arms) of these 34 trials were obtained, and data from the other 20 trials were requested from pharmaceutical companies by telephone or email but not obtained. Of the 20 eligible trials not included in this analysis, seven were classed as “completed,” five as “terminated,” and eight as “active, not recruiting.” The 14 trials for which publicly available data were obtained tested the following drugs: bexarotene (Targretin; Ligand Pharmaceuticals, San Diego, CA; ReXceptor, Cleveland, OH), solanezumab (three trials17; LY2062430; Eli Lilly, Indianapolis, IN), LY450139 (Semagacestat, two trials; Eli Lilly, Indianapolis, IN), gantenerumab (Chugai Pharmaceutical, Tokyo, Japan; Hoffmann-La Roche, Basel, Switzerland), bapineuzumab (two trials, three clinical trial numbers12 18 19 20; Janssen Biotech, Philadelphia, PA; Pfizer, New York, NY), Verubecestat (two trials; MK-8931, Merck Sharp & Dohme, Kenilworth, NJ), BAN2401 (lecanemab; Biogen, Cambridge, MA; Eisai, Tokyo, Japan),13 15 21 and aducanumab (Biogen, Cambridge, MA).22 All but two of these trials reported change in mini-mental state examination (MMSE) score as a cognitive outcome; one trial, of Verubecestat (NCT01953601), reported the clinical dementia rating scale-sum of boxes, and the trial of BAN2401 reported the Alzheimer’s disease composite score (ADCOMS). To provide a common scale, we converted changes in these cognitive measures to change in MMSE scores (see appendix 4 for details).23 When pooling trials of solanezumab, only two independent trials were used, because the third trial was an extension of an earlier trial. Appendix 2 summarizes the trials included in the aggregated analysis and where data were obtained.

Fig 1

Flowchart of trial exclusions and inclusions

The drugs found most effective at reducing amyloid did so by 0.2 to 0.3 standardized uptake value ratio units, whereas some drugs reduced amyloid levels by far less. The current study therefore presents estimated effects on cognition associated with a 0.1 unit reduction in standardized uptake value ratio. Figure 2 shows the estimated effect of a 0.1 reduction in standardized uptake value ratio on change in MMSE scores for eight drugs and 14 trials, with 95% confidence intervals. Estimates from these trials were consistent with no effect of change in amyloid levels on MMSE score. When results from the 14 trials were pooled, the estimated effect of a 0.1 unit reduction in amyloid standardized uptake value ratio on the MMSE was 0.03 (95% confidence interval −0.06 to 0.1) points. Pooling by drug type (antibody and small molecule) also yielded results consistent with no effect of amyloid reduction on cognition.

Fig 2

Forest plot of estimated effects (95% confidence intervals) of a 0.1 decrease in standardized uptake value ratio on mini-mental state examination score for each trial and drug (top panel) and pooled across all drugs and by drug type (bottom panel). Trials of BAN2401 (lecanemab) and aducanumab are unpublished and were excluded from the “all published antibody” category. Centre and width of diamonds represent pooled estimates and 95% confidence intervals, respectively. The numbered key shows multiple trials of the same drug (see appendix table S2 for clinical trial numbers)

Individual trial results are also reported using various versions of the Alzheimer’s disease assessment scale-cognitive subscale and individual and pooled results using the clinical dementia rating scale-sum of boxes (see appendix 1). Using these alternative cognitive measures, a single trial (EMERGE) showed a statistically significant effect of reduction in amyloid levels on cognitive change. For the clinical dementia rating scale-sum of boxes it was possible to pool across most trials (n=11), and the finding was not statistically significant. Sensitivity analyses stratified by tracer and applying transformations to tracer values indicated similarly null results (see appendix 1). As amyloid changed little in some trials (see appendix 1), the overall null findings of the current study could be sensitive to the addition of a highly successful trial. We evaluated how the pooled estimate would change with the addition of a new, hypothetical trial (similar to the EMERGE trial with respect to size, effect on amyloid, and standard errors). A hypothetical trial was defined that compared a low dose treatment group that decreased the standardized uptake value ratio by −0.15 and increased the MMSE score by 0.7 relative to the placebo group, and compared a high dose treatment group that decreased the standardized uptake value ratio by −0.30 and increased the MMSE score by 1.4 relative to the placebo group. The pooled estimate of the effect of a 0.1 reduction in standardized uptake value ratio on MMSE score incorporating this trial plus all previous results would be 0.094 (95% confidence interval 0.0038 to 0.18).

To contextualize findings, effect estimates were compared with the effect of carrying an APOE-ε4 allele on annual cognitive decline. The estimated effect on annual change in MMSE score of carrying an APOE-ε4 allele from trials 2a and 2b of bapineuzumab is −0.7 (95% confidence interval −1.09 to −0.31) and from the trial of BAN2401 is −1.33 (−1.99 to −0.67) (see appendix table S1). Given the pooled estimate of the effect of amyloid reduction on the MMSE score in the current study, a reduction of 0.3 standardized uptake value ratio units, such as that achieved by BAN2401 (one of the largest achieved in any trial), would correspond to a difference in change in MMSE score of 0.1 (95% confidence interval −0.17 to 0.37) points, much smaller than the expected effect of carrying an APOE-ε4 allele.

Discussion

In this report we combined all available results from trials on drugs that targeted amyloid into a single estimate of the effect of reducing amyloid levels on cognitive change. In addition to obtaining data from published trials or those posted on ClinicalTrials.gov, we obtained data from recent unpublished trials reporting statistically significant benefit: BAN2401 has been reported to be effective (Alzheimer’s Association International Conference 2018, Chicago, IL), as well as aducanumab in a reanalysis of terminated studies.24 Aggregated results from 40 randomization arms in 14 trials indicate that reducing amyloid levels with drug treatment has, at most, a small effect on cognition as measured with the MMSE. Findings from these trials individually were consistent with no effect of change in amyloid level on cognitive decline, but point estimates were variable and usually had wide confidence intervals. However, even the largest potential benefit consistent with the upper bound of the 95% confidence interval of our pooled estimate was small compared with the annual effect of APOE-ε4 on decline in MMSE score.

Strengths and limitations of this study

Combining results from several studies is more precise than providing results from individual trials. The pooled estimate in our study indicates that reduction of amyloid levels within the time frame of the conducted trials is unlikely to provide substantial cognitive benefits. Among all trials, unpublished reports on aducanumab and BAN2401 indicated statistically significant benefit in at least one cognitive measure in intention-to-treat analyses. As noted in the external review in the FDA filing of aducanumab,25 these associations should be interpreted with caution because of the potential for a chance finding. Such chance findings become more likely as the number of trials targeting the same mechanism increases. It is therefore critical to consider pooled evidence from all available trials to contextualize results for any single trial.

Some caveats exist to this interpretation. First, we assumed that changes in cognition as a result of randomization to drug treatment were fully mediated by reductions in amyloid β. It is plausible, however, that reductions in amyloid β improve cognitive outcomes but that the drugs evaluated negatively affected cognition through other mechanisms. We cannot rule out this possibility, but given the precision of our null effect estimate, the adverse direct effects would need to precisely counterbalance the positive effects of amyloid reduction, which is unlikely.

Second, we focused on the MMSE because it was reported by almost all the identified trials. Though MMSE has low sensitivity to cognitive deterioration in adults with normal cognition, nearly all these studies enrolled cognitively impaired participants who manifested deterioration in MMSE scores, indicating that the MMSE is sensitive to changes in cognition in the trial populations. In our sensitivity analyses of alternative cognitive outcomes (see appendix 1), evaluating only data from aducanumab for the Alzheimer’s disease assessment scale-cognitive subscale and clinical dementia rating scale-sum of boxes cognitive outcomes, reduction in amyloid levels seemed to be effective at slowing cognitive decline. However, we also derived a pooled estimate for clinical dementia rating scale-sum of boxes for those trials reporting this outcome, and the pooled estimate was also null.

Third, after review of publicly available reports and contact with pharmaceutical companies, we were unable to obtain data for 20 trials that met our eligibility criteria. Most companies would presumably complete and post trial results if the drug showed promising findings, so it is unlikely that drugs evaluated in trials with no available results showed evidence of benefit of amyloid reduction. Any bias from omitting data for these studies is likely to be in favor of a beneficial effect of amyloid reduction. Our null results should thus be interpreted as an optimistic estimate for the cognitive benefit of amyloid reduction based on data from randomized controlled trials to date. Data for the trials classed as “active, not recruiting” might become available in the future.

Fourth, another limitation of our study is that we did not have access to full reports, so error in the input data is possible. To address this, we have published an interactive version of our analysis online at https://amyloidintegratingevidence.shinyapps.io/Shiny/. The interface is autopopulated with the values we used in the analysis for each trial, and we provide the option of editing those values or including a hypothetical trial. Thus, input values can be manually modified and new data can be added to recalculate individual trial and pooled estimates under different assumptions.

Finally, we did not account for covariance between measured cognition and measured standardized uptake value ratio that might be induced if other factors influence amyloid levels and independently affect cognition because covariance was not reported. Simulation results indicate that this is unlikely to appreciably affect our estimates (see appendix 3).

Implications

Drugs that reduce brain amyloid levels comprise a large fraction of pharmaceuticals in the development pipeline for the prevention and treatment of Alzheimer’s disease. Evidence for reduction in amyloid levels is routinely provided as evidence of target engagement for new drugs and support for the promise of efficacy in improving cognitive outcomes. Over the past two decades, numerous drugs that target amyloid have been evaluated in phase I, II, and III trials. Although two drugs seem promising in post hoc analyses, the data remain unpublished on ClinicalTrials.gov or in a peer reviewed journal. Null findings in trials of anti-amyloid drugs call into question the viability of amyloid as a target for the prevention and treatment of Alzheimer’s disease. However, each of these trials was evaluated individually, with no research combining the evidence from all trials to evaluate whether amyloid reduction is likely to improve cognitive outcomes.

These findings suggest that reduction in amyloid levels is unlikely to have notable cognitive benefits within the time frame of typical trials. This does not conclusively invalidate the amyloid cascade hypothesis because amyloid reduction might have delayed effects on cognition that only manifest years later. If this is the case, trials on amyloid reduction would need to substantially extend the typical follow-up period to detect any benefit. Amyloid reduction strategies might be effective in other patient populations—for example, people with early stage Alzheimer’s disease. The hypothesis that people with a lower baseline amyloid burden experience greater cognitive benefit from reduction in amyloid levels could be evaluated with additional data from extant trials.

Conclusions

Combined results from 14 randomized controlled trials provide evidence that reduction in amyloid levels alone is unlikely to substantially slow cognitive decline within the follow-up period of most typical trials. The results of pooled estimates suggest that use of anti-amyloid drugs is not a viable strategy for the prevention or treatment of Alzheimer’s disease and that other potential targets may merit more attention.

Most trials of drugs targeting amyloid for the treatment of Alzheimer’s disease have not shown statistically significant cognitive benefit

Individual trials do not, however, provide conclusive evidence about the potential impact of amyloid reduction on cognitive decline

Evidence from multiple trials of amyloid targets has not been systematically combined

Pooled estimates from 14 randomized controlled trials of drugs to reduce amyloid levels suggest no substantial improvement in cognition within the timeframe of typical clinical trials