Manfred Hauben1, Jeffrey K Aronson2, Robin E Ferner3,4. 1. Department of Medicine, Division of Clinical Pharmacology, New York University School of Medicine, New York, NY, USA. manfred.hauben@Pfizer.com. 2. Centre for Evidence Based Medicine, Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK. 3. West Midlands Centre for Adverse Drug Reactions, City Hospital, Birmingham, UK. 4. School of Clinical and Experimental Medicine, University of Birmingham, Birmingham, UK.
Abstract
INTRODUCTION: Pharmacovigilance includes analysis of large databases of information on drugs and events using algorithms that detect disproportional frequencies of associations. In order to test such algorithms, attempts have been made to provide canonical reference lists of so-called 'positive controls' and 'negative controls'. Reference sets with even modest levels of misclassification may result in under- or overstatement of the performance of algorithms. AIM: We sought to determine the extent to which 'negative control' drug-event pairs in the Observational Medical Outcomes Partnership (OMOP) database are misclassified METHODS: We searched the medical literature for evidence of associations between drugs and events listed by OMOP as negative controls. RESULTS: The criteria used in OMOP to classify positive and negative controls are asymmetric; drug-event associations published only as case series or case reports are classified as positive controls if they are cited in Drug-Induced Diseases by Tisdale and Miller, but as negative controls if case series or case reports exist but are not cited in Tisdale and Miller. Of 233 drug-event pairs classified in the 2013 version of OMOP as negative controls, 21 failed to meet pre-specified OMOP adjudication criteria; in another 19 cases we found case reports, case series, or observational evidence that the drug and event are associated. Overall, OMOP misclassified, or may have misclassified, 40 (17 %) of all 'negative controls.' CONCLUSIONS: Results from studies of the performance of signal-detection algorithms based on the OMOP gold standard should be viewed with circumspection, because imperfect gold standards may lead to under/overstatement of absolute and relative signal detection algorithm performance. Improvements to OMOP would include omitting misclassified drug-event pairs, assigning more specific event labels, and using more extensive sources of information.
INTRODUCTION: Pharmacovigilance includes analysis of large databases of information on drugs and events using algorithms that detect disproportional frequencies of associations. In order to test such algorithms, attempts have been made to provide canonical reference lists of so-called 'positive controls' and 'negative controls'. Reference sets with even modest levels of misclassification may result in under- or overstatement of the performance of algorithms. AIM: We sought to determine the extent to which 'negative control' drug-event pairs in the Observational Medical Outcomes Partnership (OMOP) database are misclassified METHODS: We searched the medical literature for evidence of associations between drugs and events listed by OMOP as negative controls. RESULTS: The criteria used in OMOP to classify positive and negative controls are asymmetric; drug-event associations published only as case series or case reports are classified as positive controls if they are cited in Drug-Induced Diseases by Tisdale and Miller, but as negative controls if case series or case reports exist but are not cited in Tisdale and Miller. Of 233 drug-event pairs classified in the 2013 version of OMOP as negative controls, 21 failed to meet pre-specified OMOP adjudication criteria; in another 19 cases we found case reports, case series, or observational evidence that the drug and event are associated. Overall, OMOP misclassified, or may have misclassified, 40 (17 %) of all 'negative controls.' CONCLUSIONS: Results from studies of the performance of signal-detection algorithms based on the OMOP gold standard should be viewed with circumspection, because imperfect gold standards may lead to under/overstatement of absolute and relative signal detection algorithm performance. Improvements to OMOP would include omitting misclassified drug-event pairs, assigning more specific event labels, and using more extensive sources of information.
Authors: Matthew A Weir; Michael M Beyea; Tara Gomes; David N Juurlink; Muhammad Mamdani; Peter G Blake; Ron Wald; Amit X Garg Journal: Arch Intern Med Date: 2011-04-11
Authors: Suehyun Lee; Jeong Hoon Lee; Grace Juyun Kim; Jong-Yeup Kim; Hyunah Shin; Inseok Ko; Seon Choe; Ju Han Kim Journal: J Med Internet Res Date: 2022-10-06 Impact factor: 7.076