A Desirability of Outcome Ranking Analysis of a Randomized Clinical Trial Comparing Seven Versus Fourteen Days of Antibiotics for Uncomplicated Gram-Negative Bloodstream Infection.

Background: Although a short course (7 days) of antibiotics has been demonstrated to be noninferior to a conventional course (14 days) in terms of mortality and infectious complications for patients with a Gram-negative bacterial bloodstream infection (GNB), it is unknown whether a shorter treatment duration can provide a better overall clinical outcome.
Methods: We applied a bloodstream infection-specific desirability of outcome ranking (DOOR) analysis to the results of a previously completed, randomized controlled trial comparing short versus conventional course antibiotic therapy for hospitalized patients with uncomplicated GNB. We determined the probability that a randomly selected participant in the short course group would have a more desirable overall outcome than a participant in the conventional duration group. We performed (1) partial credit analyses allowing for calculated and variable weighting of DOOR ranks and (2) subgroup analyses to elucidate which patients may benefit the most from short durations of therapy.
Results: For the 604 patients included in the original study (306 short course, 298 conventional course), the probability of having a more desirable outcome with a short course of antibiotics compared with a conventional course was 51.1% (95% confidence interval, 46.7% to 55.4%), indicating no significant difference. Partial credit analyses indicated that the DOOR results were similar across different patient preferences. Prespecified subgroup analyses using DOOR did not reveal significant differences between short and conventional courses of therapy. Conclusions: Both short and conventional durations of antibiotic therapy provide comparable clinical outcomes when using DOOR to consider benefits and risks of treatment options for GNB.

When selecting an antibiotic treatment duration, clinicians must balance the potential risks of stopping antibiotics with the potential benefits of this decision [1]. For more than a decade, many clinicians have advocated for shorter courses of antibiotic therapy [2-5]. There have been 3 published randomized controlled trials (RCTs) assessing antibiotic duration in Gram-negative bacterial bloodstream infection (GNB) [6-8]. All studies demonstrated noninferiority with a short course (7 days) compared with a conventional course (14 days) of antibiotics with respect to a binary efficacy variable, and one study performed a post hoc desirability of outcome ranking (DOOR) analysis suggesting that a 7-day strategy might be preferred to 14 days [8]. More comprehensive benefit-to-risk analyses can help continue to elucidate effects of treatment duration on patient outcomes and inform the adoption of a treatment strategy for GNB.

Desirability of outcome ranking is an innovative analytic method that provides a global assessment of a patient’s outcome, accounting for both benefits and harms, which is lacking with traditional noninferiority approaches. The DOOR analysis combines the benefits and risks of an intervention or new therapy into a single outcome measure and allows for a superiority assessment [9]. Using a “partial credit” strategy, patients or clinicians can also choose the relative weighting of DOOR ranks, allowing for a more individualized analysis and approach to treatment [10].

The Antibacterial Resistance Leadership Group (ARLG) has used DOOR in studies of patients with multidrug-resistant infections [11-13] and developed a DOOR analysis plan for use in Staphylococcus aureus bloodstream infection trials [14]. In the current study, we performed a post hoc DOOR analysis of Yahav et al’s [7] RCT comparing 7 versus 14 days of antibiotics for uncomplicated GNB. Our primary objective was to determine the probability that a patient assigned to a short course of antibiotics (7 days) would have a more desirable outcome than a patient assigned to a conventional course (14 days) of antibiotics.

METHODS

Study Design and Overview of the Original Study

We performed a retrospective analysis of a multicenter, randomized, open-label, noninferiority trial of hospitalized patients comparing 7 versus 14 days of antibiotics for GNB [7]. Patients enrolled in the original study were afebrile, hemodynamically stable for ≥48 hours before randomization, and had adequate source control. The primary outcome in the original study was a composite of 90-day all-cause mortality, clinical failure (included relapse, suppurative or distant complications), and readmission or extended hospitalization [7].

Development of Desirability of Outcome Ranking Analysis Strategy

We adapted the DOOR analysis strategy that was described and validated for S aureus bloodstream infection [14]. The DOOR analysis strategy was iteratively reviewed and edited by all study authors. With input from the investigators of the original RCT, we then determined which study variables would map to the ARLG DOOR analysis (Figure 1).

Figure 1.

Primary desirability of outcome ranking (DOOR) analysis strategy and definitions of the components used in the analysis. a. Within 90-days postrandomization. b. Increased creatinine level ≥1.5× from baseline or glomerular filtration rate decrease >25% or urine output of <0.5 mL/kg per hour for 6 hours. c. Defined as 3 episodes of diarrhea/day for ≥2 days. ADL, activities of daily living; ARLG, Antibacterial Resistance Leadership Group.

In the DOOR analysis, patients were assigned a mutually exclusive rank 1 through 5. Rank 1 represented the most desirable outcome and included anyone who was alive and did not experience any of the prespecified undesirable events within the follow-up period, whereas rank 5 indicated the least desirable outcome and included all patients who died within 90 days of randomization. Ranks 2 through 4 included patients who were alive but had 1, 2, or 3 undesirable events occur during the follow-up period, respectively (Figure 1). The undesirable events included in the analysis were categorized as follows: (1) treatment failure (including recurrence of infection), (2) infectious complications thought to be direct complications or progression of the original infection (not present at enrollment), and (3) other significant adverse events, not necessarily related to the original infection. Figure 1 displays which variables from the original RCT were mapped to each DOOR event. As a surrogate for health-related quality of life, we used both the functional status at 90-days and the change in functional status (from baseline to 90 days) as a “tiebreaker” in the DOOR analysis for patients with the same rank (eg, if 2 patients both had 1 undesirable event occur, then the patient with the better functional status at 90 days or the most improved functional status at 90 days would be given the better rank). Functional status in the original RCT was assessed at baseline and 90 days postrandomization and was categorized as follows: independent, needs assistance in activities of daily living (ADLs), dependent in ADLs or bedridden.

In a post hoc sensitivity analysis, we removed diarrhea from the other adverse event category because the primary study definition of diarrhea (3 episodes of diarrhea/day for ≥2 days) may not be considered severe. The DOOR probability was not significantly different from the primary analysis and these results were not shown.

Statistical Analysis

We analyzed all randomized participants using an intention-to-treat strategy. The distribution of DOOR ranks was compared between the short and conventional duration strategies, and we determined the probability that a randomly selected participant assigned to a short course of therapy would have a better rank, or more desirable outcome, than a randomly selected participant assigned to a conventional course of therapy. A DOOR probability of 50% indicates no difference between groups. We also explored the difference in duration of actual antibiotics received between the 2 treatment groups and plotted this as function of the DOOR probability.

We performed a partial credit analysis in which DOOR ranks are scored like an academic exam. This method allows individuals to provide a personalized scoring key for how to grade the different DOOR ranks based on individual preferences. Survival without any events (rank 1) is always assigned a score of 100, and death (rank 5) is always assigned a 0, but ranks 2–4 can be given any score between 0 and 100 as long as the original order is maintained [10]. We chose 3 different partial credit scenarios that represented a range of theoretical patient preferences. For each scenario, we subtracted the mean partial credit score for the conventional group from the mean partial credit score for the short group. A difference of zero indicates no difference between groups. To visually display contours of the difference in mean partial credit scores in a 2-dimensional plot, we combined being alive with 2 or 3 events in Figure 4.

Figure 4.

Desirability of outcome ranking (DOOR) partial credit scenarios. The top panel provides the partial credit scoring key. Scenario A represents a patient who only places value on surviving (equivalent to a 90-day mortality outcome). Scenario B represents a patient who places more value on quality of life and considers any adverse event as very undesirable. Scenario C represents a patient who places significant value on survival but also tries to balance this with avoiding complications. For each scenario, the difference in mean partial credit scores is calculated by subtracting the mean score for the conventional group from the mean score for the short group. The bottom panel displays contours of the difference in mean partial credit scores. The partial credit score assigned to being alive with 2 or 3 events is combined on the horizontal axis, and the partial credit score assigned to being alive with one event is on the vertical axis. The red line at zero indicates no difference between the mean partial credit scores. Positive differences (shaded in gray) suggest the short course could provide a more desirable outcome, and negative differences (shaded in yellow) suggest the conventional course could provide a more desirable outcome. However, none of the theoretical partial credit scenarios (A–C) demonstrated a significant difference between treatment groups (P > .1 for all scenarios).

Fianlly, we compared the DOOR distributions by treatment strategy for prespecified subgroups defined by the primary source of bacteremia (urinary tract infection [UTI] versus nonurinary source) and the presence of a ceftazidime-resistant Enterobacterales pathogen, indicative of an extended-spectrum beta-lactamase (ESBL)-producing Enterobacterales in this trial. These subgroups were similar to the ones performed in the original study and were chosen because we thought clinicians may have more concern with using a short course of antibiotics in patients with a nonurinary source of infection or in those with a multidrug-resistant organism. Our hypothesis was that outcomes would be similar in patients randomized to the short course compared with those randomized to the conventional course, even in these potentially more challenging infectious scenarios.

Patient Consent Statement

This study was reviewed by the Duke University Health System Institutional Review Board, which determined that this study was exempt from human subjects research review. Because this study was a secondary analysis of previously collected data, reconsent of participants was not required. As part of the initial trial, participants provided written informed consent, as described in the primary manuscript by Yahav et al [7].

RESULTS

All 604 patients included in the original RCT were analyzed; 306 were assigned to the short course (7 days) and 298 were assigned to the conventional course (14 days). Descriptive characteristics of the patient population have been previously published [7]. Most patients (n = 411, 68%) had a UTI as the primary source of GNB. Most patients (n = 543, 90%) had Enterobacterales as the primary pathogen, 105 (19%) of which were ESBL-producing organisms. Most patients (n = 375, 62%) were independent in their ADLs but 40 (7%) were bedridden at baseline.

Overall, 248 (41%) patients were alive without any undesirable events, 218 (36%) with 1 event, 64 (11%) with 2 events, 6 (1%) with 3 events, and 68 (11%) died. The DOOR distribution was similar between the short and conventional treatment groups with probability of having a more desirable outcome with the short versus conventional course of 51.1% (95% confidence interval [CI], 46.7%–55.4%), indicating no significant difference (Figure 2). In addition, no significant differences were observed between the short and conventional groups for any of the component undesirable events contributing to the DOOR rank (Figure 3 and Supplemental Table 1). The median duration of appropriate covering antibiotics received for the index bacteremia was 7 days in the short course group and 14 days in the conventional course group [7]. However, when looking at the mean difference in antibiotic days received, the difference between the 2 treatment groups was closer to 6 days, and this is relatively consistent across the 95% confidence region of the DOOR probability (Supplemental Figure 1).

Figure 2.

Desirability of outcome ranking (DOOR) distribution by treatment groups. The DOOR probability of having a more desirable outcome when assigned to the short course of antibiotics was 51.1% (95% confidence interval, 46.7%–55.4%).

Figure 3.

Forest plot demonstrating the desirability of outcome ranking (DOOR) probabilities for each individual DOOR component (treatment failure, infectious complications, adverse events, and death) as well as the overall DOOR probability of having a more desirable outcome with a short course of treatment compared with a conventional duration. The individual components are not mutually exclusive, and patients can have more than one of these events. CI, confidence interval.

When the patient’s functional status at 90 days was used as a tiebreaker in the DOOR analysis, the probability of having a more desirable outcome in the short versus conventional group remained similar (51.5%; 95% CI, 46.9%–56.0%). There was also no difference when using the change in functional status (from baseline to 90 days) as a tiebreaker (data not shown).

Using a partial credit analysis, we evaluated the effect of ascribing different partial credit scores to the DOOR ranks based on hypothetical patient preferences, rather than having each rank receive equal weight (Figure 4). Scenario A represents a patient who places value only on avoiding mortality, regardless of the complications that arise on the path to survival (this is equivalent to using 90-day mortality as the primary outcome). Alternatively, a patient represented by scenario B places more value on quality of life and would consider anything other than an uncomplicated survival to be objectionable. In scenario C, the representative patient places significant value on surviving their illness but does place some value on avoiding complications, unlike the patient in scenario A. In all scenarios, we did not observe a significant difference in the mean partial credit scores between the short and conventional duration groups (Figure 4). The DOOR distribution was also similar comparing short versus conventional treatment duration, stratified by the primary source of bacteremia (UTI vs other source) or by the presence of an ESBL-producing pathogen (Supplemental Figures 2 and 3).

DISCUSSION

We applied a DOOR analysis to a previously published RCT assessing duration of antibiotics for GNB [7]. Using DOOR, we found that patients randomized to either treatment duration experienced similar rates of desirable outcomes. These results persisted after including a marker of functional status at 90 days. The DOOR results also remained similar in a partial credit analysis with various weights applied to each DOOR level to adjust for different patient values.

Our results confirm and extend those reported in the original RCT by incorporating both efficacy and safety concerns into the primary outcome [7]. We ultimately observed that the overall desirability of the outcome in patients receiving a short course of antibiotics was comparable to those receiving a conventional course, and the patients assigned to the short course of antibiotics received, on average, 6 fewer days of antibiotics. At both the individual and the population level, this decrease in antibiotic duration can represent a significant advantage. Decreasing overall antibiotic usage can minimize the risk and global burden of antimicrobial resistance, and patients and clinicians should strongly consider this factor when deciding on an antibiotic duration [15, 16].

Our results are similar to the recently published study by Molina et al [8] who also performed a post hoc DOOR analysis of a trial comparing 7 versus 14 days of antibiotics for GNB. In the Molina et al [8] study, they used Response Adjusted for Duration of Antibiotic Risk (RADAR), a specific version of DOOR, which incorporates the total number of antibiotic days received in the DOOR endpoint (ie, if 2 participants had the same clinical outcome, the participant with the fewest antibiotic days would be given the more desirable rank) [9]. We did not use RADAR in our study because we chose to focus more directly on the clinical outcomes most impactful to patients rather than duration of antibiotics received, which closely tracked with assigned study arm in this study. It should be noted, however, that the assigned duration of antibiotics does not always mirror the actual duration patients receive in clinical trials, and therefore we also analyzed the mean difference in antibiotics received between groups as a function of the DOOR probability and the 95% confidence region. In addition, in contrast to the Molina et al [8] study, the DOOR endpoint used in our analysis accounts for the cumulative nature of undesirable events that can occur throughout a patient’s clinical course without assigning priority to the events. We favor this approach over the DOOR endpoint used by Molina et al [8] in which clinical efficacy was prioritized over safety (ie, a patient with a serious adverse event but who met the definition of cure was ranked higher than a patient with clinical failure but no adverse events). This prioritization might not be the same for all patients, clinicians, or antibiotics tested. In addition, our report incorporated a partial credit analysis, which is another unique aspect of this work. We are encouraged that DOOR is being used in more clinical trials. Further work should be done to standardize the DOOR analysis which will help clinicians compare results across clinical studies.

One of the advantages of the DOOR analysis is the ability to help elucidate when the potential risks of one therapy outweigh the benefits. In traditional registrational drug trials, the efficacy and safety analyses are usually performed in separate populations, and this can make it difficult to translate clinical trial results to individual patient care. Subgroup analyses using DOOR can incorporate both the potential benefits and harms of a new therapy or strategy and can help physicians individualize treatment decisions. In this study, we specifically addressed patients who had a UTI as the source of the bloodstream infection or those with an ESBL-producing pathogen, and we did not find that one antibiotic duration strategy was superior to the other. Performing similar, prespecified subgroup analyses in prospective clinical trials using DOOR could be a useful tool to analyze diverse patient populations, including patients who may be older, have poor functional status, or are immunocompromised. These subgroups are traditionally underrepresented in clinical trials, and inclusion should be encouraged to understand treatment effects in diverse populations [17, 18]. Differing partial credit scores using the DOOR analysis allow for individualized assessment of benefit/risk based on patient preferences.

This study is limited by the fact that we retrospectively fit the original study variables into our DOOR analysis strategy. Not all DOOR events were monitored for the same duration of follow-up. Recurrence and infectious complications were evaluated for 90 days, but many adverse events were only followed for 30 days. Other important clinical considerations such as the antibiotic susceptibly profile, requirements for intravenous antibiotics, and cost were not included in the DOOR analysis. In addition, our current study did not include patient reported outcomes or include a comprehensive assessment of health-related quality of life. Understanding how to best incorporate both of these factors into future prospective studies using DOOR will be critical to ensure these studies remain relevant to patients [19].

CONCLUSIONS

In summary, we applied DOOR to a large RCT comparing 7 versus 14 days of antibiotics for patients with GNB and did not observe significant differences between groups using a global measure of clinical success. For most patients with uncomplicated GNB, a short course of antibiotics is equivalent to a conventional course. The DOOR can provide useful information regarding the totality of treatment effects on patient outcomes and provide clinicians with comprehensive analyses of the risks and benefits of an intervention to help guide therapeutic decision making. We believe DOOR should be prospectively included in future RCTs.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Click here for additional data file.

Click here for additional data file.