Rosa Gini1, Caitlin N Dodd2, Kaatje Bollaerts3, Claudia Bartolini4, Giuseppe Roberto5, Consuelo Huerta-Alvarez6, Elisa Martín-Merino7, Talita Duarte-Salles8, Gino Picelli9, Lara Tramontan10, Giorgia Danieli11, Ana Correa12, Chris McGee13, Benedikt F H Becker14, Charlotte Switzer15, Sonja Gandhi-Banga16, Jorgen Bauwens17, Nicoline A T van der Maas18, Gianfranco Spiteri19, Emmanouela Sdona20, Daniel Weibel21, Miriam Sturkenboom22. 1. Agenzia regionale di sanità della Toscana, Osservatorio di epidemiologia, Florence, Italy. Electronic address: rosa.gini@ars.toscana.it. 2. Erasmus University Medical Center, Post Box 2040, 3000 CA Rotterdam, Netherlands; Julius Global Health, University Medical Center, Utrecht, Heidelberglaan 100, the Netherlands. 3. P95 Epidemiology and Pharmacovigilance, Koning Leopold III laan 1, 3001 Heverlee, Belgium. Electronic address: kaatje.bollaerts@p-95.com. 4. Agenzia regionale di sanità della Toscana, Osservatorio di epidemiologia, Florence, Italy. Electronic address: claudia.bartolini@ars.toscana.it. 5. Agenzia regionale di sanità della Toscana, Osservatorio di epidemiologia, Florence, Italy. Electronic address: giuseppe.roberto@ars.toscana.it. 6. BIFAP Database, Spanish Agency of Medicines and Medical Devices, Madrid, Spain. Electronic address: chuerta@aemps.es. 7. BIFAP Database, Spanish Agency of Medicines and Medical Devices, Madrid, Spain. Electronic address: emartinm@aemps.es. 8. Institut Universitari d'Investigació en Atenció Primària Jordi Gol (IDIAP Jordi Gol), Barcelona, Spain. Electronic address: tduarte@idiapjgol.org. 9. Epidemiological Information for Clinical Research from an Italian Network of Family Paediatricians (PEDIANET), Padova, Italy. Electronic address: g.picelli@virgilio.it. 10. Epidemiological Information for Clinical Research from an Italian Network of Family Paediatricians (PEDIANET), Padova, Italy; Consorzio Arsenal.IT, Veneto Region, Italy. Electronic address: ltramontan@consorzioarsenal.it. 11. Epidemiological Information for Clinical Research from an Italian Network of Family Paediatricians (PEDIANET), Padova, Italy; Consorzio Arsenal.IT, Veneto Region, Italy. 12. University of Surrey, Guildford, Surrey GU2 7XH, UK. Electronic address: s.lusignan@surrey.ac.uk. 13. University of Surrey, Guildford, Surrey GU2 7XH, UK; Royal College of General Practitioners, Research and Surveillance Centre, 30 Euston Square, London NW1 2FB, UK. Electronic address: c.mcgee@surrey.ac.uk. 14. Erasmus University Medical Center, Post Box 2040, 3000 CA Rotterdam, Netherlands. Electronic address: benedikt.becker@posteo.de. 15. Sanofi Pasteur, 1755 Steeles Ave W, North York, ON M2R 3T4, Canada. 16. Sanofi Pasteur, 1755 Steeles Ave W, North York, ON M2R 3T4, Canada. Electronic address: sonja.banga@sanofi.com. 17. University Children's Hospital, Basel, Switzerland; University of Basel, Switzerland; Brighton Collaboration Foundation, Switzerland. Electronic address: j.bauwens@brightoncollaboration.org. 18. University of Basel, Switzerland; Brighton Collaboration Foundation, Switzerland. Electronic address: nicoline.van.der.maas@rivm.nl. 19. European Centre for Disease Prevention and Control, Gustav III's Boulevard 40, 16973 Solna, Sweden. Electronic address: Gianfranco.Spiteri@ecdc.europa.eu. 20. European Centre for Disease Prevention and Control, Gustav III's Boulevard 40, 16973 Solna, Sweden. 21. Erasmus University Medical Center, Post Box 2040, 3000 CA Rotterdam, Netherlands. Electronic address: d.weibel@erasmusmc.nl. 22. Julius Global Health, University Medical Center, Utrecht, Heidelberglaan 100, the Netherlands; P95 Epidemiology and Pharmacovigilance, Koning Leopold III laan 1, 3001 Heverlee, Belgium; VACCINE.GRID Foundation, Spitalstrasse 33, Basel, Switzerland. Electronic address: m.c.j.sturkenboom@umcutrecht.nl.
Abstract
BACKGROUND: The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines using European healthcare databases. Event misclassification can result in biased estimates. Using different algorithms for identifying cases of Bordetella pertussis (BorPer) infection as a test case, we aimed to describe a strategy to quantify event misclassification, when manual chart review is not feasible. METHODS: Four participating databases retrieved data from primary care (PC) setting: BIFAP: (Spain), THIN and RCGP RSC (UK) and PEDIANET (Italy); SIDIAP (Spain) retrieved data from both PC and hospital settings. BorPer algorithms were defined by healthcare setting, data domain (diagnoses, drugs, or laboratory tests) and concept sets (specific or unspecified pertussis). Algorithm- and database-specific BorPer incidence rates (IRs) were estimated in children aged 0-14 years enrolled in 2012 and 2014 and followed up until the end of each calendar year and compared with IRs of confirmed pertussis from the ECDC surveillance system (TESSy). Novel formulas were used to approximate validity indices, based on a small set of assumptions. They were applied to approximately estimate positive predictive value (PPV) and sensitivity in SIDIAP. RESULTS: The number of cases and the estimated BorPer IRs per 100,000 person-years in PC, using data representing 3,173,268 person-years, were 0 (IR = 0.0), 21 (IR = 4.3), 21 (IR = 5.1), 79 (IR = 5.7), and 2 (IR = 2.3) in BIFAP, SIDIAP, THIN, RCGP RSC and PEDIANET respectively. The IRs for combined specific/unspecified pertussis were higher than TESSy, suggesting that some false positives had been included. In SIDIAP the estimated IR was 45.0 when discharge diagnoses were included. The sensitivity and PPV of combined PC specific and unspecific diagnoses for BorPer cases in SIDIAP were approximately 85% and 72%, respectively. CONCLUSION: Retrieving BorPer cases using only specific concepts has low sensitivity in PC databases, while including cases retrieved by unspecified concepts introduces false positives, which were approximately estimated to be 28% in one database. The share of cases that cannot be retrieved from a PC database because they are only seen in hospital was approximately estimated to be 15% in one database. This study demonstrated that quantifying the impact of different event-finding algorithms across databases and benchmarking with disease surveillance data can provide approximate estimates of algorithm validity.
BACKGROUND: The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines using European healthcare databases. Event misclassification can result in biased estimates. Using different algorithms for identifying cases of Bordetella pertussis (BorPer) infection as a test case, we aimed to describe a strategy to quantify event misclassification, when manual chart review is not feasible. METHODS: Four participating databases retrieved data from primary care (PC) setting: BIFAP: (Spain), THIN and RCGP RSC (UK) and PEDIANET (Italy); SIDIAP (Spain) retrieved data from both PC and hospital settings. BorPer algorithms were defined by healthcare setting, data domain (diagnoses, drugs, or laboratory tests) and concept sets (specific or unspecified pertussis). Algorithm- and database-specific BorPer incidence rates (IRs) were estimated in children aged 0-14 years enrolled in 2012 and 2014 and followed up until the end of each calendar year and compared with IRs of confirmed pertussis from the ECDC surveillance system (TESSy). Novel formulas were used to approximate validity indices, based on a small set of assumptions. They were applied to approximately estimate positive predictive value (PPV) and sensitivity in SIDIAP. RESULTS: The number of cases and the estimated BorPer IRs per 100,000 person-years in PC, using data representing 3,173,268 person-years, were 0 (IR = 0.0), 21 (IR = 4.3), 21 (IR = 5.1), 79 (IR = 5.7), and 2 (IR = 2.3) in BIFAP, SIDIAP, THIN, RCGP RSC and PEDIANET respectively. The IRs for combined specific/unspecified pertussis were higher than TESSy, suggesting that some false positives had been included. In SIDIAP the estimated IR was 45.0 when discharge diagnoses were included. The sensitivity and PPV of combined PC specific and unspecific diagnoses for BorPer cases in SIDIAP were approximately 85% and 72%, respectively. CONCLUSION: Retrieving BorPer cases using only specific concepts has low sensitivity in PC databases, while including cases retrieved by unspecified concepts introduces false positives, which were approximately estimated to be 28% in one database. The share of cases that cannot be retrieved from a PC database because they are only seen in hospital was approximately estimated to be 15% in one database. This study demonstrated that quantifying the impact of different event-finding algorithms across databases and benchmarking with disease surveillance data can provide approximate estimates of algorithm validity.
Authors: Andrea Spini; Pietro Rosellini; Cristiana Bellan; Folco Furiesi; Silvano Giorgi; Sandra Donnini; Rosa Gini; Marina Ziche; Francesco Salvo; Giuseppe Roberto Journal: PLoS One Date: 2022-06-08 Impact factor: 3.752
Authors: Kaatje Bollaerts; Alexandros Rekkas; Tom De Smedt; Caitlin Dodd; Nick Andrews; Rosa Gini Journal: PLoS One Date: 2020-04-22 Impact factor: 3.240
Authors: Corinne Willame; Caitlin Dodd; Lieke van der Aa; Gino Picelli; Hanne-Dorthe Emborg; Johnny Kahlert; Rosa Gini; Consuelo Huerta; Elisa Martín-Merino; Chris McGee; Simon de Lusignan; Giuseppe Roberto; Marco Villa; Daniel Weibel; Lina Titievsky; Miriam C J M Sturkenboom Journal: Drug Saf Date: 2021-01-19 Impact factor: 5.606
Authors: Nicolas H Thurin; Romin Pajouheshnia; Giuseppe Roberto; Caitlin Dodd; Giulia Hyeraci; Claudia Bartolini; Olga Paoletti; Hedvig Nordeng; Helle Wallach-Kildemoes; Vera Ehrenstein; Elena Dudukina; Thomas MacDonald; Giorgia De Paoli; Maria Loane; Christine Damase-Michel; Anna-Belle Beau; Cécile Droz-Perroteau; Régis Lassalle; Jorieke Bergman; Karin Swart; Tania Schink; Clara Cavero-Carbonell; Laia Barrachina-Bonet; Ainhoa Gomez-Lumbreras; Maria Giner-Soriano; María Aragón; Amanda J Neville; Aurora Puccini; Anna Pierini; Valentina Ientile; Gianluca Trifirò; Anke Rissmann; Maarit K Leinonen; Visa Martikainen; Sue Jordan; Daniel Thayer; Ieuan Scanlon; Mary E Georgiou; Marianne Cunnington; Morris Swertz; Miriam Sturkenboom; Rosa Gini Journal: Clin Pharmacol Ther Date: 2021-11-26 Impact factor: 6.903