Interpreting Neurologic Outcomes in a Changing Trial Design Landscape: An Analysis of HeartWare Left Ventricular Assist Device Using a Hybrid Intention to Treat Population.

Randomized controlled trials can provide optimal clinical evidence to assess the benefits of new devices, and it is these data that often shape device usage in real-world practice. However, individual clinical trial results sometimes appear discordant for the same device, and alternative devices are sometimes not employed in similar patient populations. To make sound evidence-based decisions, clinicians routinely rely on cross-trial comparisons from different trials of similar but not identical patient populations to assess competing technology when head-to-head randomized comparisons are unavailable.

Randomized clinical trials provide the best means to compare devices; however, there are no such trials with current left ventricular assist device (LVAD) technology. Comparing results across trials of LVADs becomes problematic owing to differences in trial inclusion and exclusion criteria, the evolution of trial design from discrete bridge to transplant (BTT) and destination therapy (DT) trials to a combined indication, evolving definitions of adverse events, and evolution of surgical and medical management over time. An improved understanding of the morbidity related to LVAD technology has resulted in increasing specificity of definitions, sometimes resulting in overly conservative definitions in the older trials when compared with newer trials.

Another limitation when comparing nonrandomized clinical trials involves potential differences in the methodology in the reporting of events. Evolving clinical trial designs have led to mixed populations with varying success criteria, which limit the use of the traditional incidence, or risk of an event over an entire population, as a reliable expression of adverse event probability. For example, HeartWare Left Ventricular Assist Device System (HVAD) BTT clinical trials reported neurologic event rates,[1-3] while the MOMENTUM 3 clinical trial reported cumulative incidence (frequency) of neurologic events.[4,5] In the MOMENTUM 3 trial, 40 of 189 HeartMate 3 (HM3) patients were transplanted before 2 years, with some as early as the first 30 days.[5] Despite this, all 40 patients were included in the denominator as “at risk” for stroke at 2 years. This leads to an error in actual patients “at risk” and has the potential effect of artificially lowering the incidence of adverse events. Thus, direct cross-trial comparisons of neurologic events are not possible.

Given the lack of randomized trials and differences in the existing mechanical circulatory support trials, we undertook a novel approach to the comparison of LVAD trials via the generation of a “hybrid” intention to treat population, using existing clinical trial datasets. While such an approach is not a substitute for a randomized comparison, this analysis highlights the need for a more rigorous examination of clinical trial designs and data analyses.

To generate a “HYBRID-HVAD” population that would be comparable with the HeartMate 3 cohort of the MOMENTUM 3 trial, a mix of DT:BTT patients was needed.[5] Thus, patients receiving an HVAD system from the DT ENDURANCE Supplemental[7] and the continued access protocol (CAP) cohort of the ADVANCE BTT+CAP trials were combined to form the HYBRID-HVAD population for this analysis. Because the MOMENTUM 3 trial design specifically excluded all patients with fulminant cardiogenic shock,[8] these patients were removed from the HYBRID-HVAD population.

Of the 550 patients in the combined ADVANCE+CAP and ENDURANCE SUPPLEMENTAL cohorts, 14 of 242 patients in the CAP dataset and 12 of 308 patients in the SUPPLEMENTAL dataset were excluded due to presence of cardiogenic shock at baseline (Figure 1). The final combined HYBRID-HVAD population included 228 ADVANCE+CAP and 296 ENDURANCE SUPPLEMENTAL patients for a total of 524 patients. Apart from more HYBRID-HVAD patients having ischemic cardiomyopathy, the HYBRID-HVAD cohort was clinically similar to HM3 patients in the MOMENTUM 3 clinical trial (Table 1). Importantly, the overall BTT:DT ratio was very similar, with 56.5% DT and 43.5% BTT, compared with 58% long-term (DT) and 42% short-term (BTT) patients in MOMENTUM 3.[5] The proportion of patients transplanted within 2 years in the HYBRID-HVAD BTT cohort was 50.4% compared with 50% of BTT HM3 patients in MOMENTUM 3.[5]

Figure 1.

Generation of the hybrid long-term HVAD population.

Table 1.

Baseline Characteristics of the HVAD-HYBRID Population

The HVAD and MOMENTUM 3 trials used differing adverse event definitions. HVAD clinical trials used the neurologic event definitions from Interagency Registry of Mechanically Assisted Circulatory Support (INTERMACS) protocol 3.0[1,2,7] while MOMENTUM 3 applied the neurologic definitions used in INTERMACS protocol 5.[4,5,8] These protocols were significantly different in the adjudication of what qualified as a stroke event. For example, a subdural hematoma after a fall was adjudicated as a stroke in the HVAD trials per the definitions in protocol 3.0,[9] whereas these were appropriately classified as an “other neurologic event” in the MOMENTUM trial using the new INTERMACS adverse event protocols.[4] To allow for a comparison of neurologic outcomes, the incidence of total neurologic events was compared. The “neurologic dysfunction” category included strokes (both ischemic and hemorrhagic events), transient ischemic attacks (TIAs), and other nonstroke events such as confusion, seizure, and encephalopathy; therefore, total neurologic events should be broadly and equally captured between trials. We found no significant difference in the incidence of overall neurologic events occurring at either 6 months or 2 years in the HYBRID-HVAD population (14.1% and 23.3%, respectively) relative to the MOMENTUM 3 HM3 population (13.9% and 21.7%, respectively) (Figure 2A).

Figure 2.

A: Incidence of total neurologic events in the HYBRID HVAD group compared with the MOMENTUM 3 HeartMate 3 (HM3) group.[6,7] There were 413/524 of HVAD patients and 130/151 HM3 patients on support at the time of the 6-month analysis, and 224/524 of HVAD patients and 117/189 HM3 patients on support at the time of the 2-year analysis. B: Comparison of freedom from disabling stroke rates in MOMENTUM 3 and ENDURANCE Supplemental. The freedom from disabling strokes (strokes with a modified Rankin Score (mRS) > 3) in the HM3 and HM2 cohorts of the MOMENTUM Trial compared with the freedom from disabling strokes in the ENDURANCE Supplemental Trial. (HM3 and HM2 curves reprinted with permission from Mehra, et al.[5] Copyright ©2018 Massachusetts Medical Society.)

This analysis might be criticized for a lack of discussion of the most severe of the neurologic events, disabling strokes. However, the ADVANCE BTT+CAP study was performed before understanding the need for analysis of mRS scores and stroke recovery. However, the freedom from disabling strokes (mRS>3) in the ENDURANCE Supplemental trial was presented by Teuteberg et al.[4] at the 2018 ISHLT Congress in Nice, France. Despite the fact that the ENDURANCE Supplemental trial used a definition that pulled in more “other neurologic events” into the stroke category (e.g. an “other neurologic event” patient who died in the MOMENTUM 3 short-term cohort due to a subdural hematoma after a fall[4] was not counted as a disabling stroke, but would have been in ENDURANCE Supplemental), and the excess by more than 10% of ischemic patients and inclusion of cardiogenic shock patients in the pure DT population in the HVAD trial,[4] we find similar results when compared with MOMENTUM 3 (Figure 2B). The MOMENTUM 3 trial reported a 2-year freedom from disabling strokes of 92.5% in the HM3 cohort,[5] compared with 90.7% in the HVAD cohort of ENDURANCE Supplemental.[10]

The goal of these comparisons is to assist the shared decision-making process in the absence of head-to-head clinical trial comparisons. In this analysis, with a focus on neurologic events as an example of difficulties encountered when comparing outcomes across trials of contemporary LVADs. Considerations of the manner in which event rates are determined can also limit comparisons. In the MOMENTUM 3 trial, 40 of 189 HM3 patients were transplanted before 2 years, with some as early as the first 30 days.[5] Despite this, all 40 patients were included in the denominator as “at risk” for stroke at 2 years. This leads to an error in actual patients “at risk” and has the potential effect of artificially lowering the incidence of adverse events and severely limiting the appropriateness of cross-trial comparisons. The HM3 device was also evaluated in a Conformité Européene (CE) Mark trial, which also used the short-term/long-term design of MOMENTUM 3 but with a single HM3 study arm. The results of the long-term cohort of patients supported for 2 years was recently published.[6] In that trial, 24% of patients were reported to have experienced a stroke through 2 years of support. The difference in the stroke rate reported in the CE Mark trial at 24% and the low 10% rate reported in MOMENTUM 3 can be partly attributed to the fact that they also reported a very low transplant rate of 10%, less than half that of the MOMENTUM 3 trial. Therefore, more patients in the denominator in that trial were actually on support and at risk of having a stroke, which lends to it likely being a more reliable report of stroke incidence with the HM3.

There are some inherent limitations to this analysis. The ADVANCE CAP and ENDURANCE Supplemental trials were not performed contemporaneously with the MOMENTUM 3 trial. Also, the sample size of the HM3 population was quite small compared with the HYBRID HVAD population. Also, HM3 outcomes in MOMENTUM 3 appear discordant with those observed in similar trials in Europe for CE Mark;[11] therefore, more experience with the HM3 system could result in more refined outcomes not evident in a smaller population. The MOMENTUM 3 trial includes a large CAP experience which may address the concerns of limited experience in the future.[9] Uncaptured differences in patient selection and patient management arising from knowledge gained between clinical trials may have led to unmeasured differences that could impact the incidence of neurologic events.

In summary, there is an urgent need in the medical community to make evidenced-based and informed decisions when it comes to device selection. However, until head-to-head randomized studies of contemporary devices are available, clinicians must resort to cross-trial comparisons, frequently from significantly discordant trials. In our analysis, neurologic event rates appear to be equivalent between the HVAD, HM3 and HMII, particularly fatal and disabling strokes. We have outlined a careful examination of the trial design, patient populations, and statistical analyses required to minimize biased conclusions. Future studies with uniform event definitions, careful consideration of statistical designs and analyses are required to further elucidate comparative contemporary device outcomes.

ACKNOWLEDGMENT

The authors gratefully acknowledge the thoughtful discussions and insights contributed by Thomas Vassiliades of Medtronic, as well as the statistical programming support of Jie Qin and Wei Xu, also of Medtronic.