Triangulation of pharmacoepidemiology and laboratory science to tackle otic quinolone safety.

BACKGROUND: The scientific method requires studies with high internal and external validity. Though both are necessary, they do not go hand-in-hand: The more controlled a study is to enhance internal validity, the less applicable to real-world clinical care, and vice versa. In the many instances where evidence from clinical trials is not available, scientific inference must rely on more extreme approaches on this spectrum, such as mechanistic (limited generalizability/strong bias control) and real-world evidence (RWE) studies (higher generalizability/lesser bias control).
OBJECTIVES: Illustrate how triangulating mechanistic and RWE studies can enhance scientific inference by delivering the supporting evidence for both.
METHODS: We describe our research on an unexpected and highly unlikely drug safety issue: the risk of tympanic membrane (TM) perforations resulting from otic quinolone therapy. Tightly controlled laboratory studies using cell culture and rodent models were complemented with pharmacoepidemiological studies of real-world data to translate mechanistic findings and corroborate RWE.
RESULTS: We present a cascade of mechanistic and RWE studies investigating fibroblast cytotoxicity, delayed healing of perforated TMs, and spontaneous TM perforations after otic quinolone exposure, all suggesting local tissue toxicity.
CONCLUSION: Triangulation of mechanistic and RWE studies allowed incremental progress toward robust evidence on otic quinolone toxicity.

INTRODUCTION—BETWEEN A ROCK AND A HARD PLACE

The scientific method holds that studies aiming to make inferences about causal effects must demonstrate both internal and external validity. Internal validity is accomplished by design features that reduce the chance for systematic error (bias) and by using adequate sample sizes able to reduce random error. The more controlled a study is, that is, the more it removes pathways that produce the observed outcome other than the tested exposure, the higher its ability to make causal inferences. External validity in turn postulates that the study must be generalizable to the scenario in which its conclusions are applied. Failure to ensure that the experimental conditions reflect the real‐world application is prone to inductive fallacy where study findings are applied in clinical scenarios in which they do not hold true.

Trial phases required in regulatory drug approval are built on this concept as they carry highly controlled preclinical proof of concept studies through clinical trial phases where the clinical scenario is becoming increasingly realistic. Phase III studies are considered the optimal approach to combine principles of internal and external validity, though regulatory requirements may extend beyond a drug's approval into phase IV. Phase III studies employ strong mechanisms to control bias, including randomization, protocol‐based outcomes ascertainment, blinding, and analytical approaches such as intention‐to‐treat analyses. External validity is assumed by defining trial enrolment criteria that would eventually resemble the population with the drug's indications when approved.

However, RCTs are costly and effort‐intensive. Their prospective nature and the time needed for subject accrual make RCTs unfavourable for examining long‐term or rare effects. Their narrow enrolment criteria may create overly homogenous populations that cannot capture drug risk–benefit in real‐world users. About a quarter of newly approved medications receive a black box warning or are withdrawn from the market after approval, largely because of trial design features (e.g., too short follow‐up) or lack of generalizability to the real‐world drug users. Experimental conditions may also not be reproducible in real‐world clinical practice. For example, beneficial effects of spironolactone among patients with severe heart failure described in a landmark trial did not realize in real‐world populations, partially because trial participants differed in key clinical characteristics from the patients treated in clinical practice and because monitoring and management of hyperkalaemia in clinical practice was challenging. Finally, the appetite for funding RCTs beyond direct support of drug approval is limited, leaving a plethora of questions about drug safety and effectiveness unanswered. There are other instances where randomization is not feasible, and evidence from non‐randomized studies is accepted for regulatory decision‐making.

Ways to supplement and enhance RCT evidence are available on both sides of the research spectrum (Figure 1). Population‐based pharmacoepidemiological studies provide real‐world evidence (RWE), with strong and sometimes perfect generalizability (e.g., in countries where real‐world data are available for all citizens as a result of universal healthcare coverage and integrated data systems). Inherent in the data sources, however, is also the risk of bias. Although patients may start and discontinue drug treatment, this process is not introduced via a study protocol, not randomized, and not systematically continued and followed to capture relevant outcomes. Thus, without understanding the processes that resulted in drug initiation, discontinuation or switching of treatment, assessment for certain outcomes (in some patients, but not others) or availability of measurements that allow quantification of baseline risk and assessment of confounding—without understanding the process that generated the data—bias in pharmacoepidemiological design may be inevitable. Absence of information does not necessarily equate to absence of disease and MCAR (missing completely at random) assumptions hold rarely true for real‐world data, and thus, complete case analyses where subjects with incomplete data are discarded, will introduce bias.

FIGURE 1

Features of study types in the evaluation of drug outcomes

There are situations where the process that generated the data may be close to a RCT scenario. Consider, for example, two drugs in the same class, released at the same time, for the same indication, with no evident competitive advantage. One would assume that physicians did not consider patient factors when selecting one or the other, resulting in pseudo‐randomization as illustrated in the cardiac safety evaluation of rosiglitazone versus pioglitazone. Because the safety concern was unknown to the clinical community and the drug effect unintended, the chance for confounding was further reduced. Thus, the internal validity of a pharmacoepidemiological study is highly dependent on the clinical scenario being studied and the data source that is available to assess the full spectrum of pathways that may produce non‐causal associations between exposure and outcome. With appropriate high‐quality design and validated measurements, use of real‐world data sources that are fit‐for‐purpose, and similar population and outcome definitions, results from RWE, and RCTs tend to align, , , and may be acceptable for regulatory decisions.

On the other side of the research spectrum are pharmacological studies that aim to explore a drug's effects on a mechanistic level. The focus on specific pathway(s) related to the drug's effects allows a high level of control but opens the risk of inductive fallacy. This problem is quantifiable based on the likelihood of a new molecular entity to advance to a successful new drug application, which is about 10%. The predominant cause of failure across the clinical trial phases is lack of drug efficacy and not safety, that is, the principal focus of preclinical and early phase clinical studies. Common reasons why preclinical and early phase clinical trials fail include use of inadequate models that do not translate to the targeted clinical scenario or that do not fully capture the net drug effect (Figure 2). For example, only a small fraction of animal models translates into human pathophysiology and pharmacology. Use of a specific surrogate outcome as proxy for a drug safety or efficacy construct, even if used in clinical studies, may miss important pathways of drug action and thus, capture only a portion of the drug effect. ,

FIGURE 2

Common problems in the translation of mechanistic studies to clinical efficacy and safety

The strengths and weaknesses of pharmacoepidemiology and laboratory research are highly complementary: Laboratory models deliver the mechanistic explanation for the observed association in real‐world data, thus strengthening biological plausibility. RWE in turn validates the translatability of a laboratory model to clinical practice, thus reducing the chance for inductive fallacy. Thus, in the many scenarios where RCT data are not available, the intentional alignment of mechanistic and pharmacoepidemiological studies may aid in developing sufficient evidence to inform decision‐making. This approach follows closely the idea of triangulation (i.e., the practice of strengthening causal inferences by integrating results from several different approaches, where each approach has different [and assumed to be largely unrelated] key sources of potential bias). Here, we describe how such a joint (triangulation) approach allowed development of evidence in support of an unexpected (and highly unlikely) drug safety problem: that the known tissue‐toxic effect of systemic quinolones would extend to topical administration and damage of non‐load‐bearing tissue.

QUINOLONE SAFETY CASE STUDY ON TRIANGULATION

Quinolone otic preparations were introduced to the United States in the late 1990s, roughly 10 years after the introduction of oral ciprofloxacin. These preparations were eagerly received by the clinical community, as aminoglycosides constituted the practice standard for otic therapy and these were well known to carry a risk of ototoxicity. , As otic quinolones proved to be both highly efficacious for treating outer and middle ear infections and they showed no evidence of inner ear toxicity, they rapidly outpaced aminoglycosides as the new standard for otic antimicrobial therapy.

Concurrently, RWE was emerging about quinolone toxicity when used systemically. Initial concerns were focused on load‐bearing tissue (e.g., tendon rupture, tendinopathy, and arthropathy). These were followed by reports of aortic dissection and toxicity in tissues that were not load‐bearing. , We found a report on retinal detachment following systemic quinolone exposure to be particularly concerning. Otic quinolones are typically delivered at 3000 μg/ml, over 1000 times greater than the concentrations that are achieved in the retina with systemic administration. , We hypothesized that the high concentrations of quinolones with otic administration would be toxic to the soft tissue in the ear, and this would be manifest by tympanic membrane (TM) perforations.

We set out to test this hypothesis in the laboratory, using cell culture of mouse TM fibroblasts. We found that collagen production and fibroblast viability were markedly reduced after exposure to both 0.01% and 0.3% ciprofloxacin, the latter concentration being most commonly available in commercial ear drop formulations. Though an important step toward proving our hypothesis, fibroblast models had no demonstrated transferability to soft tissue toxicity.

We built on this preliminary evidence with two approaches: an animal model and examination of real‐world data. Following good pharmacoepidemiological practice, RWE studies of drug effects on TM perforations required us to validate the use of claims data for the identification of TM perforations in children that had received tympanostomy tubes.

In our subsequent first RWE study assessing the effects of otic quinolones on the TM, we examined the rate of persistent TM perforations after tympanostomy tube placement among children who were exposed to otic quinolones or otic neomycin preparations. Our findings mirrored those of our cell culture experiments, with a significantly elevated risk of TM perforations after otic quinolone exposure (hazard ratio 2.26, 95% confidence interval 1.34–3.83). Concerns were raised that otic quinolones were used preferentially in children with more severe disease, suggesting presence of confounding. Of note, similar concerns though suggesting channelling of neomycin to the more severe cases had been raised for one of our prior studies, which used the same population to examine the ototoxicity of otic neomycin preparations with non‐intact TMs (i.e., perforations or tympanostomy tubes) and which illustrates the uncertainty in interpreting the processes that resulted in drug initiation. Drug utilization studies that evaluate determinants of treatment selection including shifts in preference over time, as we did for this case, are helpful to understand potential mechanisms of confounding.

Concurrent to our ongoing RWE study, we had expanded our initial cell culture study that assessed only ciprofloxacin with a broader examination of otic fluoroquinolones and found that the quinolone fibroblast toxicity was drug‐specific (i.e., ciprofloxacin was toxic to TM fibroblasts, but ofloxacin was not).

With both in vitro evidence and RWE, we moved to an in vivo study, assessing the effect of otic quinolone exposure on rat TM healing. Our findings were consistent with our in vitro studies: Otic ciprofloxacin delayed TM healing relative to ofloxacin and neomycin preparations, and ofloxacin showed no effect when compared with normal saline.

We tried to replicate the comparative safety assessment in our real‐world population and found significant effects for both quinolones, with hazard ratios of 1.49 (1.05–2.09) for ofloxacin and 1.94 (1.32–2.85) for ciprofloxacin/hydrocortisone when compared to neomycin/hydrocortisone ear drops. Importantly, the point estimates suggested a weaker though statistically significant safety risk for ofloxacin, while our rat model did not support an effect for ofloxacin.

Even though our evidence had demonstrated that quinolones contributed to persistent perforation of an already damaged TM, the effect on healthy tissue was unclear. In order to most effectively link otic quinolones to TM perforations, we therefore decided to evaluate the effect of otic quinolones on otherwise healthy TMs. This was relatively easily done with both laboratory models and RWE. In the laboratory, we expose rat TMs to otic quinolones. In keeping with our prior studies, we found that commercial quinolone ear drops caused new TM perforations in healthy TMs. This effect appeared to be both drug‐specific and potentiated by steroids. Interestingly, we found spontaneous perforations in ears exposed to ofloxacin though the incidence was higher among rat ears exposed to ciprofloxacin.

In our RWE approach, we measured the rate of TM perforations in both children and adults who received otic quinolones or otic neomycin for uncomplicated acute otitis externa, a condition that is usually self‐limited and has never been associated with TM perforations. Again, we found otic quinolones to be associated with a higher rate of newly diagnosed TM perforations than otic neomycin. In this case, effect estimates for different quinolones were quite similar, with adjusted hazard ratios of 2.53 (1.27–5.05) for ofloxacin and 2.24 (1.03–4.85) for ciprofloxacin plus hydrocortisone, though confidence intervals were wide due to the small incidence rate of new TM perforations. In our discussion with peer reviewers and other peers, we found that while no mechanism for a specific bias was suggested, there was significant doubt about the validity of our findings based on the perceived biological plausibility. Our animal study proved critical in delivering the supporting evidence in these discussions. We completed this line of research with the conclusion that otic quinolones have local toxicity, but more work needs to be done to identify the safest preparation, considering the risk–benefit of both individual quinolones and individual steroids.

As so often happens in research, our findings led us in an unanticipated direction—the possibility of systemic toxicity from otic quinolones. In our initial study in rats, we performed myringotomies bilaterally, but quinolones were instilled unilaterally, with the contralateral ear serving as an internal control. We found that myringotomies contralateral to both ciprofloxacin + dexamethasone and neomycin + hydrocortisone treatment had significantly delayed healing. Thus, we attributed the observation to the steroid. Systemic effects of otic steroids have long been recognized. However, ciprofloxacin alone also demonstrated a modest contralateral effect, which suggested that just a small volume of otic quinolone drops might impose a sufficient effect on soft tissues, even if not directly exposed to the local application. This surprising finding prompted a RWE study of systemic toxicity of otic quinolones. This manuscript is currently under review.

All of these studies would be of purely academic interest if they did not find application in clinical experience. Not long after the publication of our study on TM perforations following treatment of AOE with otic quinolones, one of us (PJA) saw a 4‐year‐old girl with no prior ear disease who had been seen in our hospital's emergency department because of a foreign body in one ear canal. She could not tolerate its removal in the emergency department, so she was taken to the operating room to have this done under anaesthesia. The foreign body was easily removed and the TM was carefully inspected and found to be intact and normal by a well‐trained, experienced otolaryngologist. However, since her ear canal had been manipulated, she was prescribed a 1‐week course of otic ciprofloxacin + dexamethasone. On follow‐up with the author, she was found to have a persistent TM perforation in the otic‐quinolone‐treated ear. This anecdotal experience cannot establish a causal link between otic quinolones and TM perforations. However, as the child had no other risk factors to develop a TM perforation, her case did support the possibility that what we observed in both RWE and laboratory studies was real.

Adverse outcomes, such as the development of TM perforations, are both common and largely without serious consequences. While they typically require surgical repair—not an insignificant event—this is usually successful. As a result, their occurrence usually fails to garner enough clinical curiosity to consider a well‐regarded pharmaceutical preparation a possible underlying cause, to generate event reporting with a regulatory agency, or to meet the scientific rigour necessary to merit publication, thereby making widespread clinical validation extraordinarily unlikely.

We have been most fortunate to have the basic laboratory, epidemiological resources, and clinical practice to find and validate the risk of TM perforation with otic quinolone exposure, a rare and seemingly implausible safety issue of a widely used medication. Our experience illustrates the interdependence of mechanistic and RWE studies, neither of which could have delivered actionable evidence on its own. Ideally, such evidence would be developed jointly, supported by funding mechanisms that reward such broad scale translational approaches and presented jointly in a single publication that includes both the mechanistic and RWE, which may require additional flexibility in the publishing process (e.g., a broader spectrum of peer reviewers). It is only through rigorous scientific study, using the full spectrum of research methodologies, that this drug safety concern involving otic quinolones may be appreciated.

CONFLICT OF INTEREST

AGW has received research funding from Merck, Sharpe and Dohme, NIH, AHRQ, PCORI, FDA, and the state of Florida and received honoraria of consultant from Arbor Pharmaceuticals and Genentech Inc, none of which is related to this work. PJA has received research funding from Next Science, unrelated to this work.