Noninferior Antibiotics: When Is "Not Bad" "Good Enough"?

Novel treatment options are urgently needed for patients with serious multidrug-resistant infections seen increasingly in routine everyday clinical practice, both in the hospital and nursing home as well as in the clinic and office setting. Unfortunately, the problem is no longer confined to chronically ill, repeatedly hospitalized patients. This essay explores the role of noninferiorly studies in addressing the pressing need for new antimicrobial agents to combat the emerging "superbugs", calling attention to the nuances of interpreting their sometimes less-than-straightforward results. The overriding aim is not to find better antibiotics for routinely treatable infections but to identify safe and efficacious treatment options where none presently exist.

THE PROBLEM: WAITING FOR GODOT

Even routine nosocomial and community-acquired infections are becoming frighteningly difficult to treat (www.who.int/iris/bitstream/10665/112642/1/9789241564748_eng.pdf?ua=1; www.cddep.org/publications/state_worlds_antibiotics_2015_executive_summary). Developing new antibiotics for serious life-threatening infections caused by pathogens widely resistant to the current armamentarium of antibacterial drugs is no easy task [1-3]. Beyond the capricious microbes, pharmaceutical companies face enormous costs, high failure rates, unanticipated toxicities, and restricted hospital use if/when the drug eventually comes to market [4, 5]. To preserve susceptibility and prolong the utility of a new agent, prescribing is likely to be constrained to narrowly circumscribed conditions. Such hurdles scare away investment and thereby compound the problem of a shrinking antibiotic pipeline for the emerging superbugs.

Simple practical solutions are not to be found on the near horizon (National Action Plan for Combating Antibiotic-Resistant Bacteria, www.whitehouse.gov/sites/default/files/docs/national_action_plan_for_combating_antibotic-resistant_bacteria.pdf). Cooperation among academic institutions, private industry, regulatory bodies, hospital administrators, Pharmacy and Therapeutics committees, general practitioners, clinical microbiologists, and infectious diseases experts seems essential to stem the tide of an impending post-antibiotic era created by mounting drug resistance [1, 2, 4, 5]. Despite a dissenting minority opinion [6], increasing available treatment options, while at the same time possibly mitigating drug toxicities, by demonstrating investigational alternatives are adequately efficacious against susceptible organisms offers a reasonable and prudent step forward for the immediate future while antibiotic discovery is hopefully being ramped up.

THE INTERIM SOLUTION: A MODEST PROPOSAL?

The ideal clinical study would pit the new kid on the block against an established comparator. Alas, in the case of extensively resistant pathogens, a suitable standard of care may not be found on the shelf. Amidst the emerging antibiotic crisis, the concept of noninferiority trials has taken root and flourished [7-9]. A popular paradigm is to begin by testing novel antibiotics in patients infected with serious but susceptible bacteria in order to demonstrate that a candidate drug with activity against panresistant pathogens in vitro or in animal models is essentially as safe and efficacious as more conventional treatments for serious but not invincible infections.

When looking for new antibiotics for “untreatable” infections where no good therapy exists, there cannot be a gold (or even bronze) standard to use as the comparator in a randomized head-to-head trial. The ongoing question is how to move forward with developing better agents for clinical use other than by establishing that promising drugs in the pipeline are acceptable alternatives to standard therapies for serious but susceptible infections. For extensively drug-resistant pathogens, the lack of cross-resistance between an investigational agent and the inadequate available drugs is, by necessity, impossible to ethically test upfront in comparative trials, and these must initially be extrapolated from in vitro and animal results (and subsequently confirmed by noncomparative trials, clinical experience, and dogged pharmacovigilance).

Noninferiority trials primarily aim to establish that at worst a clinically acceptable decrement in efficacy between a standard and experimental therapy could exist. Any tradeoff in efficacy might potentially be compensated by decreased toxicity, more convenience, lower cost, and/or a broader (or narrower) spectrum of activity. As critics rightfully assert, a possibly lower response rate with the test drug (even if modest) cannot be casually dismissed as inconsequential in the setting of life-threatening infections. On the other hand, the upper bound of the confidence interval around the point estimate of the between-group difference in efficacy almost always allows for the alternative possibility that the new drug is actually better for the indications under study, creating equipoise. Perhaps criteria for noninferiority should explicitly require that the 95% confidence interval bracketing the point estimate contain or exceed 0 [9]. Anticipated advantages of innovative treatments such as lower toxicity or an extended range of activity arguably would make further development of these drugs worthwhile in the contemporary era of mounting antibiotic resistance because their use is ultimately intended for infections where safe and effective therapies are not currently at hand.

To stall antibiotic development to formally demonstrate frank superiority against pathogens for which adequate choices are already available seems shortsighted when untreatable infectious diseases are presently responsible for substantial mortality and morbidity worldwide. In the short term, the less demanding but admittedly indirect hurdle of noninferiority versus superiority offers the potential to relatively rapidly identify drugs comparable in efficacy, jumpstarting antibiotic development.

Optimistically, some novel antimicrobials might not be fully cross-resistant with traditional classes of antibiotics. Such an auspicious result opens the door for similarly effective drugs overall to find unfilled niches where they favorably compare with (and can be substituted for) older suboptimal regimens as resistance to established treatments inexorably spreads. Given the current unyielding circumstances, the advantages of noninferiority trials with their lower burden of proof over conventional superiority studies are worthy of tactical consideration (Rex JH, et al. The critical role of non-inferiority trials in developing new antibacterial agents, 2015), [10]. Per protocol, superiority can be sequentially tested once noninferiority criteria are satisfied without increasing the probability of a Type 1 error [7–9, 11].

THE DILEMMA: CATCH 22!

What are the downsides of noninferiority comparisons? It is clear that these studies in isolation cannot definitively answer the central question of whether the investigational agent has clinically useful activity against bacteria resistant to the comparator, which must be excluded from the clinical trial on ethical grounds [6, 10]. Interpretation of the results is sometimes less than straightforward [9-14]. What constitutes a fair noninferiority margin [15-18]? For life-threatening illness, caregivers may find the prospect of any decrement in efficacy unconscionable unless clearly outweighed (or at minimum counterbalanced) by improved safety and tolerability [19]. The effect size scaling any loss of efficacy to gains in other parameters is rarely quantified ahead of time, although such composite outcome measures have recently been proposed [9, 20]. Serial application of noninferiority margins could theoretically lead to creeping erosion of the control referent if each standard bearer is progressively (albeit slightly) less active than its predecessor [9]. The size of the beneficial effect over placebo could then shrink toward the null over time as new comparators replace the prior standard [16, 18]. The rationale and justification for noninferiority studies have not always been precisely articulated, and execution and interpretation of these trials have suffered from systemic flaws [21]; these limitations have hopefully been remedied with increasing experience, although such advances remain to be formally documented.

When a suitable comparator is available and sufficient number of patients can be recruited in a reasonable time frame, superiority trials are to be preferred. However, demonstrating superiority over impotent or toxic options affords little progress or solace. The inherent limitations to answers provided by noninferiority studies, when recognized and acknowledged, do not necessarily constitute fatal flaws under the present circumstances.

POINTS TO PONDER: THE PRINCE AND THE PAUPER

Given the voluminous literature reporting such studies, clinicians ought to be cognizant of lurking perils and pitfalls when applying conclusions from noninferiority trials to their real-life practices. Intrinsic complexities and underappreciated nuances can confound simple translation of the results (Figure 1). In addition to 2 straightforward possibilities (Figure 1A and B) [5], 3 hypothetical scenarios with 2-sided 95% confidence intervals serve as illustrative thought experiments to deepen the reader's understanding of this increasingly prevalent trial strategy.

Figure 1.

Possible results of noninferiority trials demonstrated by 2-sided 95% confidence intervals. Five distinct outcomes of noninferiority studies are diagramed using 2-sided 95% confidence intervals. The null hypothesis for the study is that the experimental treatment is inferior to the standard comparator, leaving noninferiority of the new treatment as the alternate hypothesis if the null hypothesis is rejected. Failure to reject the null hypothesis does not necessarily imply that the alternate hypothesis has been excluded. The specific point estimate within limits does not alter the qualitative conclusion. On the other hand, both the upper and lower bounds of the confidence interval can provide critical information depending on the circumstances. Each panel indicates the appropriate interpretation (A–E).

Consider first a noninferiority trial comparing a novel treatment to an established comparator that affords only marginal benefit over placebo. The 95% confidence interval around the point estimate for the between-group difference in efficacy might hopefully fall completely above zero (Figure 1C), supporting an inference of superiority for a potential breakthrough investigational drug over the current disappointing standard of care. Sequential comparisons—first satisfying noninferiority criteria and then testing for superiority—have been proposed as analytic prototypes for certain chemotherapeutic and antibiotic trials [11]. Such a 2-step paradigm in which superiority is only tested after noninferiority has been concluded does not increase the probability of a Type 1 error [22]. Of note, this widely accepted approach coincidently underscores that most superiority trials do not define a superiority margin a priori (akin to the noninferiority margin) to gauge whether the observed effect size actually constitutes a clinically consequential improvement [9, 23].

In contradistinction, imagine a noninferiority trial in which the standard bearer has proven substantially better than placebo. Given the track record of the control, the entire 2-sided 95% confidence interval for the investigational drug or procedure could be anticipated to fall below the prespecified noninferiority threshold in some cases (Figure 1D). This result not only countermands rejecting the null hypothesis of inferiority, but it reasonably supports a presumption that the experimental intervention is in fact meaningfully inferior to the reigning gold standard. Demonstration that a new drug is inferior to the older comparator is equivalent to showing that the established therapy is superior to the investigational agent [9].

Finally, consider a mega trial that, by virtue of the large number of enrolled participants, may yield an impressively precise point estimate accompanied by a narrow confidence interval. Occasionally, the 95% confidence interval will lie wholly above the noninferiority cutoff but totally below zero (Figure 1E); in other words, the tight 95% confidence interval would be fully contained between 0 and the noninferiority threshold [21, 24]. This curious finding appears to violate the excluded middle and support both clinical noninferiority and statistical inferiority at the same time with the same data [9]. In the face of such a paradoxical result, practitioners must be particularly wary about the underlying clinical evidence used to quantify the noninferiority margin.

EPILOGUE: BRIDGE OVER TROUBLED WATERS

Expediting antibiotic development for extensively drug-resistant pathogens remains a pressing unmet need [1, 4, 25]. Moreover, time is of the essence, because more and more resistant microbes propel us toward a post-antibiotic era. Discovery must keep pace with the pathogens. Although a laudable goal in itself, the principal target is not to find superior antibiotics for routinely treatable infections but to identify effective treatment options where none currently exist.

CONCLUSIONS

Establishing noninferiority in serious but treatable infections is only a prelude to finding effective treatments for similar populations suffering from refractory pathogens. Noninferiority trials serve as an initial strategic bridge to more antibiotic choices when acceptably safe and reliably efficacious therapy is lacking [7, 10]. Once we have better treatment choices for resistant infections, the raison d′être underlying many noninferiority studies in the antibiotic space will recede in turn.