Joseph S Ross1,2,3,4, Jonathan Bates4, Craig S Parzynski4, Joseph G Akar4,5, Jeptha P Curtis4,5, Nihar R Desai4,5, James V Freeman4,5, Ginger M Gamble4, Richard Kuntz6, Shu-Xia Li4, Danica Marinac-Dabic7, Frederick A Masoudi8, Sharon-Lise T Normand9,10, Isuru Ranasinghe11, Richard E Shaw12, Harlan M Krumholz2,3,4,5. 1. Section of General Medicine, Department of Medicine. 2. Robert Wood Johnson Foundation Clinical Scholars Program, Yale School of Medicine. 3. Department of Health Policy and Management, Yale School of Public Health. 4. Center for Outcomes Research and Evaluation, Yale-New Haven Hospital. 5. Section of Cardiovascular Medicine, Department of Medicine, Yale School of Medicine, New Haven, CT. 6. Medtronic Inc, Minneapolis, MN. 7. Division of Epidemiology, Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, MD. 8. Division of Cardiology, Department of Medicine, University of Colorado, Aurora, CO. 9. Department of Health Care Policy, Harvard Medical School. 10. Department of Biostatistics, Harvard TH Chan School of Public Health, Boston, MA, USA. 11. Discipline of Medicine, University of Adelaide, Adelaide, SA, Australia. 12. Department of Clinical Informatics, California Pacific Medical Center, San Francisco, CA, USA.
Abstract
BACKGROUND: Machine learning methods may complement traditional analytic methods for medical device surveillance. METHODS AND RESULTS: Using data from the National Cardiovascular Data Registry for implantable cardioverter-defibrillators (ICDs) linked to Medicare administrative claims for longitudinal follow-up, we applied three statistical approaches to safety-signal detection for commonly used dual-chamber ICDs that used two propensity score (PS) models: one specified by subject-matter experts (PS-SME), and the other one by machine learning-based selection (PS-ML). The first approach used PS-SME and cumulative incidence (time-to-event), the second approach used PS-SME and cumulative risk (Data Extraction and Longitudinal Trend Analysis [DELTA]), and the third approach used PS-ML and cumulative risk (embedded feature selection). Safety-signal surveillance was conducted for eleven dual-chamber ICD models implanted at least 2,000 times over 3 years. Between 2006 and 2010, there were 71,948 Medicare fee-for-service beneficiaries who received dual-chamber ICDs. Cumulative device-specific unadjusted 3-year event rates varied for three surveyed safety signals: death from any cause, 12.8%-20.9%; nonfatal ICD-related adverse events, 19.3%-26.3%; and death from any cause or nonfatal ICD-related adverse event, 27.1%-37.6%. Agreement among safety signals detected/not detected between the time-to-event and DELTA approaches was 90.9% (360 of 396, k=0.068), between the time-to-event and embedded feature-selection approaches was 91.7% (363 of 396, k=-0.028), and between the DELTA and embedded feature selection approaches was 88.1% (349 of 396, k=-0.042). CONCLUSION: Three statistical approaches, including one machine learning method, identified important safety signals, but without exact agreement. Ensemble methods may be needed to detect all safety signals for further evaluation during medical device surveillance.
BACKGROUND: Machine learning methods may complement traditional analytic methods for medical device surveillance. METHODS AND RESULTS: Using data from the National Cardiovascular Data Registry for implantable cardioverter-defibrillators (ICDs) linked to Medicare administrative claims for longitudinal follow-up, we applied three statistical approaches to safety-signal detection for commonly used dual-chamber ICDs that used two propensity score (PS) models: one specified by subject-matter experts (PS-SME), and the other one by machine learning-based selection (PS-ML). The first approach used PS-SME and cumulative incidence (time-to-event), the second approach used PS-SME and cumulative risk (Data Extraction and Longitudinal Trend Analysis [DELTA]), and the third approach used PS-ML and cumulative risk (embedded feature selection). Safety-signal surveillance was conducted for eleven dual-chamber ICD models implanted at least 2,000 times over 3 years. Between 2006 and 2010, there were 71,948 Medicare fee-for-service beneficiaries who received dual-chamber ICDs. Cumulative device-specific unadjusted 3-year event rates varied for three surveyed safety signals: death from any cause, 12.8%-20.9%; nonfatal ICD-related adverse events, 19.3%-26.3%; and death from any cause or nonfatal ICD-related adverse event, 27.1%-37.6%. Agreement among safety signals detected/not detected between the time-to-event and DELTA approaches was 90.9% (360 of 396, k=0.068), between the time-to-event and embedded feature-selection approaches was 91.7% (363 of 396, k=-0.028), and between the DELTA and embedded feature selection approaches was 88.1% (349 of 396, k=-0.042). CONCLUSION: Three statistical approaches, including one machine learning method, identified important safety signals, but without exact agreement. Ensemble methods may be needed to detect all safety signals for further evaluation during medical device surveillance.
Authors: Stephen C Hammill; Mark S Kremers; Lynne Warner Stevenson; Paul A Heidenreich; Christine M Lang; Jeptha P Curtis; Yongfei Wang; Charles I Berul; Alan H Kadish; Sana M Al-Khatib; Ileana L Pina; Mary Norine Walsh; Michael J Mirro; Bruce D Lindsay; Matthew R Reynolds; Kathryn Pontzer; Laura Blum; Frederick Masoudi; John Rumsfeld; Ralph G Brindis Journal: Heart Rhythm Date: 2010-07-18 Impact factor: 6.343
Authors: Clyde W Yancy; Mariell Jessup; Biykem Bozkurt; Javed Butler; Donald E Casey; Mark H Drazner; Gregg C Fonarow; Stephen A Geraci; Tamara Horwich; James L Januzzi; Maryl R Johnson; Edward K Kasper; Wayne C Levy; Frederick A Masoudi; Patrick E McBride; John J V McMurray; Judith E Mitchell; Pamela N Peterson; Barbara Riegel; Flora Sam; Lynne W Stevenson; W H Wilson Tang; Emily J Tsai; Bruce L Wilkoff Journal: Circulation Date: 2013-06-05 Impact factor: 29.690
Authors: Frederic S Resnic; Arjun Majithia; Danica Marinac-Dabic; Susan Robbins; Henry Ssemaganda; Kathleen Hewitt; Angelo Ponirakis; Nilsa Loyo-Berrios; Issam Moussa; Joseph Drozda; Sharon-Lise Normand; Michael E Matheny Journal: N Engl J Med Date: 2017-01-25 Impact factor: 91.245
Authors: Sana M Al-Khatib; F Lee Lucas; James G Jollis; David J Malenka; David E Wennberg Journal: J Am Coll Cardiol Date: 2005-09-23 Impact factor: 24.094
Authors: Frederic S Resnic; Thomas P Gross; Danica Marinac-Dabic; Nilsa Loyo-Berrios; Sharon Donnelly; Sharon-Lise T Normand; Michael E Matheny Journal: JAMA Date: 2010-11-10 Impact factor: 56.272
Authors: Justin A Ezekowitz; Brian H Rowe; Donna M Dryden; Nicola Hooton; Ben Vandermeer; Carol Spooner; Finlay A McAlister Journal: Ann Intern Med Date: 2007-08-21 Impact factor: 25.391
Authors: Isuru Ranasinghe; Craig S Parzynski; James V Freeman; Rachel P Dreyer; Joseph S Ross; Joseph G Akar; Harlan M Krumholz; Jeptha P Curtis Journal: Ann Intern Med Date: 2016-05-03 Impact factor: 25.391
Authors: Hae Reong Kim; MinDong Sung; Ji Ae Park; Kyeongseob Jeong; Ho Heon Kim; Suehyun Lee; Yu Rang Park Journal: Medicine (Baltimore) Date: 2022-06-24 Impact factor: 1.817
Authors: Jonathan Bates; Craig S Parzynski; Sanket S Dhruva; Andreas Coppi; Richard Kuntz; Shu-Xia Li; Danica Marinac-Dabic; Frederick A Masoudi; Richard E Shaw; Frederick Warner; Harlan M Krumholz; Joseph S Ross Journal: Pharmacoepidemiol Drug Saf Date: 2018-06-12 Impact factor: 2.890
Authors: Amarnath R Annapureddy; Shady Henien; Yongfei Wang; Karl E Minges; Joseph S Ross; Erica S Spatz; Nihar R Desai; Pamela N Peterson; Frederick A Masoudi; Jeptha P Curtis Journal: JAMA Date: 2020-11-03 Impact factor: 56.272
Authors: Grace Brown; Samuel Conway; Mahmood Ahmad; Divine Adegbie; Nishil Patel; Vidushi Myneni; Mohammad Alradhawi; Niraj Kumar; Daniel R Obaid; Dominic Pimenta; Jonathan J H Bray Journal: Open Heart Date: 2022-07