Taxiarchis Botsis1, Emily Jane Woo, Robert Ball. 1. Office of Biostatistics and Epidemiology, Center for Biologics Evaluation and Research CBER, US FDA, Woodmont Office Complex 1, Rm 306N, 1401 Rockville Pike, Rockville, MD 20852, USA. Taxiarchis.Botsis@fda.hhs.gov
Abstract
BACKGROUND: Automating the classification of adverse event reports is an important step to improve the efficiency of vaccine safety surveillance. Previously we showed it was possible to classify reports using features extracted from the text of the reports. OBJECTIVE: The aim of this study was to use the information encoded in the Medical Dictionary for Regulatory Activities (MedDRA(®)) in the US Vaccine Adverse Event Reporting System (VAERS) to support and evaluate two classification approaches: a multiple information retrieval strategy and a rule-based approach. To evaluate the performance of these approaches, we selected the conditions of anaphylaxis and Guillain-Barré syndrome (GBS). METHODS: We used MedDRA(®) Preferred Terms stored in the VAERS, and two standardized medical terminologies: the Brighton Collaboration (BC) case definitions and Standardized MedDRA(®) Queries (SMQ) to classify two sets of reports for GBS and anaphylaxis. Two approaches were used: (i) the rule-based instruments that are available by the two terminologies (the Automatic Brighton Classification [ABC] tool and the SMQ algorithms); and (ii) the vector space model. RESULTS: We found that the rule-based instruments, particularly the SMQ algorithms, achieved a high degree of specificity; however, there was a cost in terms of sensitivity in all but the narrow GBS SMQ algorithm that outperformed the remaining approaches (sensitivity in the testing set was equal to 99.06 % for this algorithm vs. 93.40 % for the vector space model). In the case of anaphylaxis, the vector space model achieved higher sensitivity compared with the best values of both the ABC tool and the SMQ algorithms in the testing set (86.44 % vs. 64.11 % and 52.54 %, respectively). CONCLUSIONS: Our results showed the superiority of the vector space model over the existing rule-based approaches irrespective of the standardized medical knowledge represented by either the SMQ or the BC case definition. The vector space model might make automation of case definitions for spontaneous report review more efficient than current rule-based approaches, allowing more time for critical assessment and decision making by pharmacovigilance experts.
BACKGROUND: Automating the classification of adverse event reports is an important step to improve the efficiency of vaccine safety surveillance. Previously we showed it was possible to classify reports using features extracted from the text of the reports. OBJECTIVE: The aim of this study was to use the information encoded in the Medical Dictionary for Regulatory Activities (MedDRA(®)) in the US Vaccine Adverse Event Reporting System (VAERS) to support and evaluate two classification approaches: a multiple information retrieval strategy and a rule-based approach. To evaluate the performance of these approaches, we selected the conditions of anaphylaxis and Guillain-Barré syndrome (GBS). METHODS: We used MedDRA(®) Preferred Terms stored in the VAERS, and two standardized medical terminologies: the Brighton Collaboration (BC) case definitions and Standardized MedDRA(®) Queries (SMQ) to classify two sets of reports for GBS and anaphylaxis. Two approaches were used: (i) the rule-based instruments that are available by the two terminologies (the Automatic Brighton Classification [ABC] tool and the SMQ algorithms); and (ii) the vector space model. RESULTS: We found that the rule-based instruments, particularly the SMQ algorithms, achieved a high degree of specificity; however, there was a cost in terms of sensitivity in all but the narrow GBSSMQ algorithm that outperformed the remaining approaches (sensitivity in the testing set was equal to 99.06 % for this algorithm vs. 93.40 % for the vector space model). In the case of anaphylaxis, the vector space model achieved higher sensitivity compared with the best values of both the ABC tool and the SMQ algorithms in the testing set (86.44 % vs. 64.11 % and 52.54 %, respectively). CONCLUSIONS: Our results showed the superiority of the vector space model over the existing rule-based approaches irrespective of the standardized medical knowledge represented by either the SMQ or the BC case definition. The vector space model might make automation of case definitions for spontaneous report review more efficient than current rule-based approaches, allowing more time for critical assessment and decision making by pharmacovigilance experts.
Authors: James J Sejvar; Katrin S Kohl; Jane Gidudu; Anthony Amato; Nandini Bakshi; Roger Baxter; Dale R Burwen; David R Cornblath; Jan Cleerbout; Kathryn M Edwards; Ulrich Heininger; Richard Hughes; Najwa Khuri-Bulos; Rudolf Korinthenberg; Barbara J Law; Ursula Munro; Helena C Maltezou; Patricia Nell; James Oleske; Robert Sparks; Priscilla Velentgas; Patricia Vermeer; Max Wiznitzer Journal: Vaccine Date: 2010-06-18 Impact factor: 3.641
Authors: Jens U Rüggeberg; Michael S Gold; José-Maria Bayas; Michael D Blum; Jan Bonhoeffer; Sheila Friedlander; Glacus de Souza Brito; Ulrich Heininger; Babatunde Imoukhuede; Ali Khamesipour; Michel Erlewyn-Lajeunesse; Susana Martin; Mika Mäkelä; Patricia Nell; Vitali Pool; Nick Simpson Journal: Vaccine Date: 2007-03-12 Impact factor: 3.641
Authors: Taxiarchis Botsis; Michael D Nguyen; Emily Jane Woo; Marianthi Markatou; Robert Ball Journal: J Am Med Inform Assoc Date: 2011-06-27 Impact factor: 4.497
Authors: Taxiarchis Botsis; Thomas Buttolph; Michael D Nguyen; Scott Winiecki; Emily Jane Woo; Robert Ball Journal: J Am Med Inform Assoc Date: 2012-08-25 Impact factor: 4.497
Authors: Jan Bonhoeffer; Katrin Kohl; Robert Chen; Philippe Duclos; Harald Heijbel; Ulrich Heininger; Tom Jefferson; Elisabeth Loupi Journal: Vaccine Date: 2002-12-13 Impact factor: 3.641