Sicong Huang1,2, Jie Huang1, Tianrun Cai1,2, Kumar P Dahal1, Andrew Cagan1,3, Zeling He1, Jacklyn Stratton1, Isaac Gorelik1, Chuan Hong4,5, Tianxi Cai1,4,5, Katherine P Liao1,4. 1. Division of Rheumatology, Inflammation, and Immunity, Brigham and Women's Hospital. 2. Department of Medicine, Harvard Medical School. 3. Research Information Science and Computing, Partners Healthcare. 4. Department of Biomedical Informatics, Harvard Medical School. 5. Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA.
Abstract
OBJECTIVE: The objective of this study was to compare the performance of an RA algorithm developed and trained in 2010 utilizing natural language processing and machine learning, using updated data containing ICD10, new RA treatments, and a new electronic medical records (EMR) system. METHODS: We extracted data from subjects with ≥1 RA International Classification of Diseases (ICD) codes from the EMR of two large academic centres to create a data mart. Gold standard RA cases were identified from reviewing a random 200 subjects from the data mart, and a random 100 subjects who only have RA ICD10 codes. We compared the performance of the following algorithms using the original 2010 data with updated data: (i) a published 2010 RA algorithm; (ii) updated algorithm, incorporating ICD10 RA codes and new DMARDs; and (iii) published algorithm using ICD codes only, ICD RA code ≥3. RESULTS: The gold standard RA cases had mean age 65.5 years, 78.7% female, 74.1% RF or antibodies to cyclic citrullinated peptide (anti-CCP) positive. The positive predictive value (PPV) for ≥3 RA ICD was 54%, compared with 56% in 2010. At a specificity of 95%, the PPV of the 2010 algorithm and the updated version were both 91%, compared with 94% (95% CI: 91, 96%) in 2010. In subjects with ICD10 data only, the PPV for the updated 2010 RA algorithm was 93%. CONCLUSION: The 2010 RA algorithm validated with the updated data with similar performance characteristics as the 2010 data. While the 2010 algorithm continued to perform better than the rule-based approach, the PPV of the latter also remained stable over time.
OBJECTIVE: The objective of this study was to compare the performance of an RA algorithm developed and trained in 2010 utilizing natural language processing and machine learning, using updated data containing ICD10, new RA treatments, and a new electronic medical records (EMR) system. METHODS: We extracted data from subjects with ≥1 RA International Classification of Diseases (ICD) codes from the EMR of two large academic centres to create a data mart. Gold standard RA cases were identified from reviewing a random 200 subjects from the data mart, and a random 100 subjects who only have RA ICD10 codes. We compared the performance of the following algorithms using the original 2010 data with updated data: (i) a published 2010 RA algorithm; (ii) updated algorithm, incorporating ICD10 RA codes and new DMARDs; and (iii) published algorithm using ICD codes only, ICD RA code ≥3. RESULTS: The gold standard RA cases had mean age 65.5 years, 78.7% female, 74.1% RF or antibodies to cyclic citrullinated peptide (anti-CCP) positive. The positive predictive value (PPV) for ≥3 RA ICD was 54%, compared with 56% in 2010. At a specificity of 95%, the PPV of the 2010 algorithm and the updated version were both 91%, compared with 94% (95% CI: 91, 96%) in 2010. In subjects with ICD10 data only, the PPV for the updated 2010 RA algorithm was 93%. CONCLUSION: The 2010 RA algorithm validated with the updated data with similar performance characteristics as the 2010 data. While the 2010 algorithm continued to perform better than the rule-based approach, the PPV of the latter also remained stable over time.
Authors: Bernard Ng; Fawad Aslam; Nancy J Petersen; Hong-Jen Yu; Maria E Suarez-Almazor Journal: Arthritis Care Res (Hoboken) Date: 2012-10 Impact factor: 4.794
Authors: Katherine P Liao; Tianxi Cai; Vivian Gainer; Sergey Goryachev; Qing Zeng-treitler; Soumya Raychaudhuri; Peter Szolovits; Susanne Churchill; Shawn Murphy; Isaac Kohane; Elizabeth W Karlson; Robert M Plenge Journal: Arthritis Care Res (Hoboken) Date: 2010-08 Impact factor: 4.794
Authors: Katherine P Liao; Tianxi Cai; Vivian S Gainer; Andrew Cagan; Shawn N Murphy; Chihchin Liu; Susanne Churchill; Stanley Y Shaw; Isaac Kohane; Daniel H Solomon; Robert M Plenge; Elizabeth W Karlson Journal: Arthritis Care Res (Hoboken) Date: 2013-12 Impact factor: 4.794
Authors: Yichi Zhang; Tianrun Cai; Sheng Yu; Kelly Cho; Chuan Hong; Jiehuan Sun; Jie Huang; Yuk-Lam Ho; Ashwin N Ananthakrishnan; Zongqi Xia; Stanley Y Shaw; Vivian Gainer; Victor Castro; Nicholas Link; Jacqueline Honerlaw; Sicong Huang; David Gagnon; Elizabeth W Karlson; Robert M Plenge; Peter Szolovits; Guergana Savova; Susanne Churchill; Christopher O'Donnell; Shawn N Murphy; J Michael Gaziano; Isaac Kohane; Tianxi Cai; Katherine P Liao Journal: Nat Protoc Date: 2019-11-20 Impact factor: 13.491
Authors: Chaitanya Shivade; Preethi Raghavan; Eric Fosler-Lussier; Peter J Embi; Noemie Elhadad; Stephen B Johnson; Albert M Lai Journal: J Am Med Inform Assoc Date: 2013-11-07 Impact factor: 4.497
Authors: Zhi Yu; Seoyoung C Kim; Kathleen Vanni; Jie Huang; Rishi Desai; Shawn N Murphy; Daniel H Solomon; Katherine P Liao Journal: Arthritis Res Ther Date: 2018-06-01 Impact factor: 5.156
Authors: Chen Lin; Elizabeth W Karlson; Helena Canhao; Timothy A Miller; Dmitriy Dligach; Pei Jun Chen; Raul Natanael Guzman Perez; Yuanyan Shen; Michael E Weinblatt; Nancy A Shadick; Robert M Plenge; Guergana K Savova Journal: PLoS One Date: 2013-08-16 Impact factor: 3.240
Authors: Katherine P Liao; Tianxi Cai; Guergana K Savova; Shawn N Murphy; Elizabeth W Karlson; Ashwin N Ananthakrishnan; Vivian S Gainer; Stanley Y Shaw; Zongqi Xia; Peter Szolovits; Susanne Churchill; Isaac Kohane Journal: BMJ Date: 2015-04-24
Authors: Sicong Huang; Tianrun Cai; Brittany N Weber; Zeling He; Kumar P Dahal; Chuan Hong; Jue Hou; Thany Seyok; Andrew Cagan; Marcelo F DiCarli; Jacob Joseph; Seoyoung C Kim; Daniel H Solomon; Tianxi Cai; Katherine P Liao Journal: Arthritis Care Res (Hoboken) Date: 2021-10-08 Impact factor: 4.794
Authors: Vanessa L Kronzer; Weixing Huang; Alessandra Zaccardelli; Cynthia S Crowson; John M Davis; Robert Vassallo; Tracy J Doyle; Elena Losina; Jeffrey A Sparks Journal: J Rheumatol Date: 2021-10-15 Impact factor: 4.666
Authors: Harrison G Zhang; Boris P Hejblum; Griffin M Weber; Nathan P Palmer; Susanne E Churchill; Peter Szolovits; Shawn N Murphy; Katherine P Liao; Isaac S Kohane; Tianxi Cai Journal: J Am Med Inform Assoc Date: 2021-11-25 Impact factor: 7.942