Trial and Error: Code, Guideline, or Recommendation? Implementation of Endovascular Thrombectomy Trial Data in Clinical Practice and the Future of Endovascular Trial Design.

The negative endovascular thrombectomy (EVT) trials of 2013 left the neurovascular community both disappointed and in a state of conundrum, as the results seemed to conflict with basic tenets of stroke pathophysiology—as opening acutely occluded vessels should save brain tissue. Even prior to this period, many operators had performed endovascular reperfusion procedures and witnessed encouraging results first hand. This perplexing conclusion was deemed unacceptable, and a second set of trials were re‐designed to examine the same question whether EVT improves outcomes. , , , , The positive results of subsequent trials starting in 2015, revolutionized acute stroke therapy, as well as reinforced the importance of technical advancements and rigorous research methodology. Consequently, the robust data published in the DAWN and DEFUSE 3 trials resulted from meticulous inclusion/exclusion criteria and trial design.

The article published in this issue of the Journal of the American Heart Association (JAHA) by Leischner et al, Study Criteria Applied to Real Life—A Multicenter Analysis of Stroke Patients Undergoing Endovascular Treatment in Clinical Practice, analyzes the application of the positive randomized controlled trial (RCT) inclusion criteria on the multicenter German Stroke Registry—Endovascular Treatment (GSR‐ET) database that included 2611 patients. They report the following major findings: (1) Only a minority of patients in the GSR‐ET fulfilled the RCT criteria, (2) cases that did fulfill criteria had better outcomes compared with those that did not, and (3) even cases that did not fulfill RCT criteria still had favorable outcomes.

The authors concluded that these findings support the implementation of EVT in patients who do not meet strict trial criteria, but, at the same time urge the practitioner to keep in mind that the results reported in the large EVT trials do not accurately represent real world outcomes given the stricter inclusion/exclusion criteria. While the findings reported in this article are an important contribution to the stroke literature, the underlying interplay among scientific hypotheses, trial design, and ethical medical practice in real world situations require careful attention.

The purpose of conducting a RCT is to demonstrate that a given treatment or intervention is safe and effective. When designing a RCT, the investigator first forms a hypothesis based on rigorously established scientific principles of underlying disease pathophysiology and mechanisms of the intervention. The next step is to determine the trial design and methodology that best utilizes these concepts to best determine safety and efficacy. Precise patient selection parameters are critical in guiding trial outcomes. The success of the later thrombectomy RCTs was primarily due to strict patient selection parameters that ensured the treatment effect would be significant if good reperfusion was achieved by choosing the most severe situations, and minimizing confounding factors. Once EVT was shown to be safe and effective in tightly controlled conditions, further trials are then conducted to expand the inclusion criteria. The process of serial trials is an ongoing machine with the DAWN (DWI or CTP Assessment with Clinical Mismatch in the Triage of Wake‐Up and Late Presenting Strokes Undergoing Neurointervention with Trevo) and DEFUSE 3 (Endovascular Therapy Following Imaging Evaluation for Ischemic Stroke) trials extending the time window, and now with large core , and low National Institutes of Health Stroke Scale (NIHSS) trials , potentially further expanding indications. While this process obeys scientifically sound methodology, it often leaves the clinician with guess work as to who would truly benefit, as they rely on their best clinical judgement to mitigate risk versus benefit. This article by Leischner et al clearly reflects that concept as the overwhelming majority of patients were still taken for intervention without meeting trial criteria. The treating physicians clearly felt confident doing so based on available data and their underlying understanding of ischemic stroke pathophysiology. The results from this study retrospectively show that their treatment decisions were “scientifically” correct.

The patient population in GSR‐ET registry deviated from the trial cohorts in a number of critical areas, most notably older patient age, higher initial mRS, unknown time of onset, and a high percentage of cases with distal occlusions (20% middle cerebral artery ‐ 2, 3% anterior cerebral artery, 5% posterior circulation). Yet in spite of these factors which would predispose to a worse or equivocal outcomes, the authors still report 26% excellent outcome (mRS 0–1) and 37% good outcome (mRS 0–2), which are better than the results reported in the MR CLEAN (Multicenter Randomized Clinical Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands) trial, and are comparable to the results reported in some of the positive EVT RCTs. These elements which deviated from trial selection criteria do not have established evidence in large RCTs, but have either been shown to have benefit in registry data, retrospective analysis, or in proof‐of‐concept studies. , , Assuming the imaging profile was favorable, most clinicians would offer EVT to an otherwise healthy 81 year old with unknown onset of expressive aphasia due to left middle cerebral artery ‐2 occlusion, even if they required some assistance with activities of daily living.

In 2018 the American Heart Association (AHA) released guidelines for EVT. In spite of the basis for the recommendations on evidence gleaned from the large RCTs and subsequent sub‐analyses, many experts took issue that the AHA guidelines were overly restrictive and excluded many patients who are likely to benefit from EVT. The lack of a clear consensus over treatment recommendations, even when based solely on established data from rigorous trials, further highlights the complexity of strictly adhering to parameters utilized in large RCTs, as there is often debate about what qualifies as evidence‐based standards. This trend is not unlike what was seen in the utilization of tissue plasminogen activator. In 2016 the AHA/ASA released a scientific statement reviewing the evidence for the cotemporary AHA guidelines for tissue plasminogen activator administration. It was the opinion of the authors that the AHA guidelines, which were heavy based on the NINDS ( National Institute of Neurological Disorders and Stroke) trial criteria, were overly restrictive and excluded patients who would likely benefit from the administration of tissue plasminogen activator. “The writers used systematic literature reviews, references to published clinical and epidemiology studies, morbidity and mortality reports, clinical and public health guidelines, authoritative statements, personal files, and expert opinion to summarize existing evidence and to indicate gaps in current knowledge and, when appropriate, formulated recommendations using standard American Heart Association criteria.” This holistic approach to treatment recommendations that is not exclusively bound to the specific datapoints of large prospective RCTs represents the foundation of clinical medicine and the application of scientific data that most honors the Hippocratic oath.

When reviewing the data from the large RCTs, it is clear that EVT is still a highly underutilized intervention, since many patients who could potentially benefit were not included in the trial population due to strict inclusion criteria. This becomes clear when examining the number needed to treat (NNT) in the large EVT RCTs. In the EXTEND IA (Extending the Time for Thrombolysis in Emergency Neurological Deficits ‐ Intra‐Arterial) trial the NNT to improve one point disability score outcome was 2.8 and 3.2 for functional independence at 90 days. These numbers were similar in the SWIFT (Solitaire With the Intention For Thrombectomy as Primary Endovascular Treatment) Prime trial with a NNT of 2.6 for improved disability outcome and 4 for functional independence. Remarkably, in the late window, the NNT in the DAWN trial was even lower with a NNT of 2 for improvement in disability, and 2.8 for functional independence. For comparison, the NNT in the cardiology literature for primary angioplasty versus intravenous therapy is 50. These striking numbers require some consideration, as they reveal that when following the strict criteria of the EVT trials, many patients who could benefit are excluded. Furthermore, the fact that the DAWN NNT was lower than the NNT found in the earlier trials, even though the patients in DAWN were in the later time window, points to the strict selection criteria used in the trial, as well perhaps to improvements in operator experience and device development. It should be noted that the ideal NNT for EVT is a moving target, and while we often do so, it is unfair to compare to the cardiology literature, since the risk of intervention is not the same. Undoubtedly, the NNT for thrombectomy will increase as the results of future EVT trials continue to expand indications to more severe as well as to mild strokes and more distal occlusions. Until then, the clinician is left to make use of their best judgement when deviating from trial protocols.

When considering risk benefit analysis, operator experience and comfort with EVT is crucial. A skilled/experienced operator might be willing to have a lower threshold to intervene for a non‐debilitating, albeit significant deficit as far as the patient and/or clinician is concerned. In a recent article evaluating operator proficiency in performing EVT it was shown that it takes ≈100 cases before an operator maximizes their skill set. This number of procedures is much higher than the minimum required to be eligible to participate in the large RCTs implying that better outcomes could have been achieved. Conversely, there are many seasoned operators in large centers with experiences well beyond 100 cases for whom the data from large RCTs are not generalizable. The strict standardization of selection criteria makes such situations complicated as the clinician is forced to second guess their best judgment, as even the best operators are not perfect. This becomes even more complicated, with medical legal implications looming in the background. In such situations it is important that the patient/family have a clear understanding of the rationale for care, which is often difficult due to the emergent nature of acute stroke and gaps between the knowledge base between the clinician and patient which take time to bridge.

This article by Leischner et al, Study Criteria Applied to Real Life—A Multicenter Analysis of Stroke Patients Undergoing Endovascular Treatment in Clinical Practice, serves to remind us that medicine is both a science and an art, and it is the responsibility of the clinician to integrate multifaceted sources of information. Ultimately, clinical trials are designed to inform, and should not interfere with common sense and sound judgement.

Disclosures

None.