Retrospective Real-World Studies of Paclitaxel and Mortality: Defining the Many Faces of Bias.

Studies of paclitaxel (PTX) drug-coated balloons (DCBs) have established their effectiveness in reducing interventions in patients with symptomatic peripheral artery disease through 5-year follow-up. The superior effectiveness of DCBs, as compared with percutaneous transluminal angioplasty, led to their widespread clinical use and inclusion in societal patient treatment guidelines. However, the December 2018 publication by Katsanos et al. (1) upended the use of this standard of interventional care in patients with lifestyle limiting claudication. In their 28-trial analysis of DCB and drug-eluting stents (DES) (n = 4,663) at 1 year, 12 trials (n = 2,316) at 2 years, and 3 trials (n = 863) at 4- to 5-year follow-up, they claimed an overall 1.9 (95% confidence interval [CI]: 1.27 to 2.93) increased relative risk and a 6.6% increased absolute risk of all-cause mortality in patients exposed to both a DCB and DES, compared with control subjects. Results of additional data analyses, revealed at a 2-day U.S. Food and Drug Administration (FDA) Medical Device Advisory Panel Meeting in June 2019, did little to provide clarity (2). The FDA’s internal meta-analyses of industry-sponsored, pivotal randomized controlled trials (RCTs) concluded that there was a 1.57 (95% CI: 1.16 to 2.13) increase in relative risk over 5 years (2). Ultimately, the Advisory Panel concluded that, indeed, there was a late mortality signal present, but given the small sample size, missing data, lack of apparent dose-related effect, and lack of potential physiologic mechanism, the finding should be interpreted with caution, given the remaining uncertainty of the risk magnitude and its impact on benefit-risk consideration of device use (3). Ultimately, the FDA recommended that PTX devices be reserved for patients judged to be at “high risk” for restenosis, in which the benefits of device use may outweigh their risk of use and updated device labeling to communicate the mortality risk (4). However, despite these pronouncements, the PTX mortality concern persists as an unresolved controversy, slowing patient enrollment clinical trials of PTX-coated devices in other vascular beds, consuming innumerable regulatory agency and societal financial and human resources, and prompting realignment of strategic investments and device development toward non–PTX-coated devices. In the face of this unresolved issue, rooted in limitations of the underlying data, the medical care of these patients remains impacted.

Recently, VIVA (Vascular InterVentional) Physicians (5) reported their independent mortality assessment of manufacturer-supplied, independent patient-level data from RCTs that evaluated FDA approved paclitaxel-coated balloons and stents used to treat peripheral arterial disease. Their primary analysis reported a 38% relative risk in mortality hazard (hazard ratio: 1.38; 95% CI: 1.06 to 1.8) through 5 years for PTX-coated devices compared with uncoated devices. Notably, when the lost to follow-up rates of 25% and 27% in the control and treatment arms, respectively, were reduced to 9% and 10% through additional efforts to obtain complete follow-up-status, the increased mortality risk dropped to 27% (hazard ratio: 1.27; 95% CI: 1.03 to 1.58). No mechanism of action or association between doses and mortality was identified. The VIVA meta-analysis, based on the most complete available dataset of mortality events from PTX-coated devices in RCTs to date, appeared to corroborate a PTX mortality signal identified by Katsanos et al. (1), and the FDA, although using patient-level data with more complete patient follow-up.

In subsequent public discussions, the FDA Center for Device and Radiologic Health representatives have emphasized the importance of analysis of the “totality of data.” In this regard, 2 large analyses of Medicare peripheral artery disease patients offer additional insights into PTX-coated device use and mortality. Secemsky et al. (6,7) demonstrated no increase in mortality in combined paclitaxel DCB and DES patients compared with non–PTX-coated devices (n = 16,560) at a median of 389 days (interquartile range: 277 to 507 days) or for DES versus Bristol-Myers Squibb patients (n = 51,456) at a median follow-up of 2 (interquartile range: 1.2 to 3.0) years. Although claims-based observational data experience uncontrolled residual confounding and selection bias, the size and completeness of vital status-based analyses are valuable.

In this issue of JACC: Cardiovascular Interventions, it is into this statistical cauldron that Bohme et al. (8) present their single-center, retrospective real-world analysis of DCB and plain old balloon angioplasty (POBA)–associated mortality in 1,579 patients followed for a mean follow-up of 52 months. After propensity score matching, they concluded that DCB use conferred a survival benefit. The investigators hypothesized that this survival benefit may be from increased patient mobility, although they presented no data to support this theory.

The inclusion of this large, retrospective observational dataset must come with full acknowledgment of the potential inherent bias. Observational studies and randomized trials can contribute complementary evidence about the effects of treatments on mortality and nonfatal outcomes. However, owing to the inherent potential for moderate and large biases, the role of observational studies is generally limited, as potential biases can obscure, overestimate, and even reverse the real effect of the treatment under question. As such, their role in the direct assessment of the impact of a particular treatment on a major outcome (i.e., mortality) must be carefully scrutinized (9,10). First, it is essential to acknowledge the possible multiple confounding variables, extraneous influences, which may impact a conclusion. In this regard, Bohme et al. note that their analysis is a single-center study, casting doubt on generalizing their conclusions. More importantly, they fail to advise the reader of the risk of a substantial selection bias introduced by selecting only 800 patients from a database of more than 7,000 patients. The study initiation period began in 2011, a time frame in which both treatments were available. The nonrandom assignment interjects another selection bias: over the 6-year observation period, the ratio of patients who received DCB versus POBA reversed, with DCB use increasing >200%. It may be unlikely that those not selected for analysis and received POBA in 2011 were different than those in 2017. Moreover, the rigor of the propensity score modeling is unstated; reliance on statistical significance for model selection rather than principles of causality and assessment of balance after matching are critical issues (11). Furthermore, the potential for informational bias is a concern when such observational studies use electronic medical records to identify patients with pre-existing conditions or who have undergone previous procedures that can be reported or that have been incorrectly/incompletely reported. As a result, such misclassification may distort the association between treatment exposure and outcome and alter conclusions. Importantly, the extent of potential attrition bias, those patients lost to follow-up, was not fully defined, although the investigators acknowledge that when mortality could not be discerned in their database, either the patient or patient’s physician was called. However, the exact lost to follow-up rates in the 2 treatment arms were not disclosed. Attrition bias is particularly concerning when it is unequal between treatment arms, as patients with missing vital status data may have poorer outcomes. Moreover, the inability of the investigators to define the cause of death in nearly one-half of the cases detracts from any inference of a potential mortality benefit of DCB use.

Close attention to study methodologies and inherent, undisclosed bias is essential to weighing the veracity of a study’s conclusions. Bohme et al. (8) leave many questions unanswered, including the mechanism of observed increase in late-term POBA-related mortality and how the risk-benefit profile of these devices may shift across patient populations. Regardless, critical analysis of any conclusion is part of a larger story that builds a body of knowledge and allows for the further consideration of the effect of DCBs versus POBA on mortality, if any. However, as our medical community turns to address the challenges of the COVID-19 (coronavirus disease 2019) pandemic, this PTX mortality issue will take its rightful “back burner” place to our more pressing concerns.