Can Disproportionality Analysis of Post-marketing Case Reports be Used for Comparison of Drug Safety Profiles?

Clinical trials usually do not have the power to detect rare adverse drug reactions. Spontaneous adverse reaction reports as for example available in post-marketing safety databases such as the FDA Adverse Event Reporting System (FAERS) are therefore a valuable source of information to detect new safety signals early. To screen such large data-volumes for safety signals, data-mining algorithms based on the concept of disproportionality have been developed. Because disproportionality analysis is based on spontaneous reports submitted for a large number of drugs and adverse event types, one might consider using these data to compare safety profiles across drugs. In fact, recent publications have promoted this practice, claiming to provide guidance on treatment decisions to healthcare decision makers. In this article we investigate the validity of this approach. We argue that disproportionality cannot be used for comparative drug safety analysis beyond basic hypothesis generation because measures of disproportionality are: (1) missing the incidence denominators, (2) subject to severe reporting bias, and (3) not adjusted for confounding. Hypotheses generated by disproportionality analyses must be investigated by more robust methods before they can be allowed to influence clinical decisions.