Jim Slattery1, Yolanda Alvarez, Ana Hidalgo. 1. Pharmacovigilance and Risk Management Sector, European Medicines Agency, 7 Westferry Circus, Canary Wharf, London, E14 4HB, UK. Jim.Slattery@ema.europa.eu
Abstract
BACKGROUND: Identification of potential drug safety problems using statistical screening algorithms in routinely collected databases of adverse drug reactions (ADRs) requires decision rules based on thresholds of the chosen parameters. Choosing higher or lower thresholds changes both the sensitivity of the screening and the number of false alarms produced, and thus has an impact on the effectiveness of the detection process. OBJECTIVE: The aim of this study was to evaluate the impact on the effectiveness of signal detection activities of choosing different warning thresholds for the proportional reporting ratio (PRR) and for the count of reports of any drug-event combination. METHODS: Signal detection methods were tested within the EudraVigilance database of suspected ADRs. Using an established set of known ADRs, the number that could be detected and the changes in time gained for earlier investigation of the signal were calculated over a range of signal detection thresholds. These figures were set against the number of false positive signals produced by the statistical signal detection algorithms. RESULTS: Higher thresholds for the lower confidence bound of the PRR produced fewer false positives but this benefit was offset by important losses of sensitivity in the detection of ADRs. By contrast, increases in the threshold for the count of a specific drug-event combination produced fewer false positives with little loss of either sensitivity or time gained for investigation of adverse events. A threshold of five compared with the current European Medicines Agency threshold of three gave a reduction of 25 % in false positive signals in return for a loss of 12 % in true signals detected early. CONCLUSION: Changes in the standard threshold for the count of drug-event combinations can result in a substantial improvement in efficiency of the signal detection process. Initially this change might be applied only to products with a well-established safety profile.
BACKGROUND: Identification of potential drug safety problems using statistical screening algorithms in routinely collected databases of adverse drug reactions (ADRs) requires decision rules based on thresholds of the chosen parameters. Choosing higher or lower thresholds changes both the sensitivity of the screening and the number of false alarms produced, and thus has an impact on the effectiveness of the detection process. OBJECTIVE: The aim of this study was to evaluate the impact on the effectiveness of signal detection activities of choosing different warning thresholds for the proportional reporting ratio (PRR) and for the count of reports of any drug-event combination. METHODS: Signal detection methods were tested within the EudraVigilance database of suspected ADRs. Using an established set of known ADRs, the number that could be detected and the changes in time gained for earlier investigation of the signal were calculated over a range of signal detection thresholds. These figures were set against the number of false positive signals produced by the statistical signal detection algorithms. RESULTS: Higher thresholds for the lower confidence bound of the PRR produced fewer false positives but this benefit was offset by important losses of sensitivity in the detection of ADRs. By contrast, increases in the threshold for the count of a specific drug-event combination produced fewer false positives with little loss of either sensitivity or time gained for investigation of adverse events. A threshold of five compared with the current European Medicines Agency threshold of three gave a reduction of 25 % in false positive signals in return for a loss of 12 % in true signals detected early. CONCLUSION: Changes in the standard threshold for the count of drug-event combinations can result in a substantial improvement in efficiency of the signal detection process. Initially this change might be applied only to products with a well-established safety profile.
Authors: Conny Berlin; Carles Blanch; David J Lewis; Dionigi D Maladorno; Christiane Michel; Michael Petrin; Severine Sarp; Philippe Close Journal: Pharmacoepidemiol Drug Saf Date: 2011-10-12 Impact factor: 2.890
Authors: Gianmario Candore; Kristina Juhlin; Katrin Manlik; Bharat Thakrar; Naashika Quarcoo; Suzie Seabroke; Antoni Wisniewski; Jim Slattery Journal: Drug Saf Date: 2015-06 Impact factor: 5.606
Authors: Carmen Ferrajolo; Preciosa M Coloma; Katia M C Verhamme; Martijn J Schuemie; Sandra de Bie; Rosa Gini; Ron Herings; Giampiero Mazzaglia; Gino Picelli; Carlo Giaquinto; Lorenza Scotti; Paul Avillach; Lars Pedersen; Francesco Rossi; Annalisa Capuano; Johan van der Lei; Gianluca Trifiró; Miriam C J M Sturkenboom Journal: Drug Saf Date: 2014-02 Impact factor: 5.606
Authors: Alexandra C Pacurariu; Sabine M Straus; Gianluca Trifirò; Martijn J Schuemie; Rosa Gini; Ron Herings; Giampiero Mazzaglia; Gino Picelli; Lorenza Scotti; Lars Pedersen; Peter Arlett; Johan van der Lei; Miriam C Sturkenboom; Preciosa M Coloma Journal: Drug Saf Date: 2015-12 Impact factor: 5.606
Authors: Luis C Pinheiro; Gianmario Candore; Cosimo Zaccaria; Jim Slattery; Peter Arlett Journal: Pharmacoepidemiol Drug Saf Date: 2017-11-16 Impact factor: 2.890