Lidia M V R Moura1,2, Natalia Festa3, Mary Price4, Margarita Volya4, Nicole M Benson4,5, Sahar Zafar1, Max Weiss4, Deborah Blacker6,7, Sharon-Lise Normand8,9, Joseph P Newhouse8,10,11,12, John Hsu4,8. 1. Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts, USA. 2. Department of Neurology, Harvard Medical School, Boston, Massachusetts, USA. 3. Department of Internal Medicine, Section of Geriatric Medicine, Yale School of Medicine, New Haven, Connecticut, USA. 4. Mongan Institute, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA. 5. Department of Psychiatry, Harvard Medical School, Boston, Massachusetts, USA. 6. Department of Psychiatry, Massachusetts General Hospital/Harvard Medical School, Boston, Massachusetts, USA. 7. Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA. 8. Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts, USA. 9. Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA. 10. Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA. 11. Division of Health Policy Research and Education, Harvard Kennedy School, Cambridge, Massachusetts, USA. 12. Programs on Health Care, Health Economics, Productivity, and Children, National Bureau of Economic Research, Cambridge, Massachusetts, USA.
Abstract
BACKGROUND/ OBJECTIVES: No data exist regarding the validity of International Classification of Disease (ICD)-10 dementia diagnoses against a clinician-adjudicated reference standard within Medicare claims data. We examined the accuracy of claims-based diagnoses with respect to expert clinician adjudication using a novel database with individual-level linkages between electronic health record (EHR) and claims. DESIGN: In this retrospective observational study, two neurologists and two psychiatrists performed a standardized review of patients' medical records from January 2016 to December 2018 and adjudicated dementia status. We measured the accuracy of three claims-based definitions of dementia against the reference standard. SETTING: Mass-General-Brigham Healthcare (MGB), Massachusetts, USA. PARTICIPANTS: From an eligible population of 40,690 fee-for-service (FFS) Medicare beneficiaries, aged 65 years and older, within the MGB Accountable Care Organization (ACO), we generated a random sample of 1002 patients, stratified by the pretest likelihood of dementia using administrative surrogates. INTERVENTION: None. MEASUREMENTS: We evaluated the accuracy (area under receiver operating curve [AUROC]) and calibration (calibration-in-the-large [CITL] and calibration slope) of three ICD-10 claims-based definitions of dementia against clinician-adjudicated standards. We applied inverse probability weighting to reconstruct the eligible population and reported the mean and 95% confidence interval (95% CI) for all performance characteristics, using 10-fold cross-validation (CV). RESULTS: Beneficiaries had an average age of 75.3 years and were predominately female (59%) and non-Hispanic whites (93%). The adjudicated prevalence of dementia in the eligible population was 7%. The best-performing definition demonstrated excellent accuracy (CV-AUC 0.94; 95% CI 0.92-0.96) and was well-calibrated to the reference standard of clinician-adjudicated dementia (CV-CITL <0.001, CV-slope 0.97). CONCLUSION: This study is the first to validate ICD-10 diagnostic codes against a robust and replicable approach to dementia ascertainment, using a real-world clinical reference standard. The best performing definition includes diagnostic codes with strong face validity and outperforms an updated version of a previously validated ICD-9 definition of dementia.
BACKGROUND/ OBJECTIVES: No data exist regarding the validity of International Classification of Disease (ICD)-10 dementia diagnoses against a clinician-adjudicated reference standard within Medicare claims data. We examined the accuracy of claims-based diagnoses with respect to expert clinician adjudication using a novel database with individual-level linkages between electronic health record (EHR) and claims. DESIGN: In this retrospective observational study, two neurologists and two psychiatrists performed a standardized review of patients' medical records from January 2016 to December 2018 and adjudicated dementia status. We measured the accuracy of three claims-based definitions of dementia against the reference standard. SETTING: Mass-General-Brigham Healthcare (MGB), Massachusetts, USA. PARTICIPANTS: From an eligible population of 40,690 fee-for-service (FFS) Medicare beneficiaries, aged 65 years and older, within the MGB Accountable Care Organization (ACO), we generated a random sample of 1002 patients, stratified by the pretest likelihood of dementia using administrative surrogates. INTERVENTION: None. MEASUREMENTS: We evaluated the accuracy (area under receiver operating curve [AUROC]) and calibration (calibration-in-the-large [CITL] and calibration slope) of three ICD-10 claims-based definitions of dementia against clinician-adjudicated standards. We applied inverse probability weighting to reconstruct the eligible population and reported the mean and 95% confidence interval (95% CI) for all performance characteristics, using 10-fold cross-validation (CV). RESULTS: Beneficiaries had an average age of 75.3 years and were predominately female (59%) and non-Hispanic whites (93%). The adjudicated prevalence of dementia in the eligible population was 7%. The best-performing definition demonstrated excellent accuracy (CV-AUC 0.94; 95% CI 0.92-0.96) and was well-calibrated to the reference standard of clinician-adjudicated dementia (CV-CITL <0.001, CV-slope 0.97). CONCLUSION: This study is the first to validate ICD-10 diagnostic codes against a robust and replicable approach to dementia ascertainment, using a real-world clinical reference standard. The best performing definition includes diagnostic codes with strong face validity and outperforms an updated version of a previously validated ICD-9 definition of dementia.
Authors: Marilyn S Albert; Steven T DeKosky; Dennis Dickson; Bruno Dubois; Howard H Feldman; Nick C Fox; Anthony Gamst; David M Holtzman; William J Jagust; Ronald C Petersen; Peter J Snyder; Maria C Carrillo; Bill Thies; Creighton H Phelps Journal: Alzheimers Dement Date: 2011-04-21 Impact factor: 21.566
Authors: Truls Ostbye; Donald H Taylor; Elizabeth C Clipp; Lynn Van Scoyoc; Brenda L Plassman Journal: Health Serv Res Date: 2008-02 Impact factor: 3.402
Authors: Thomas H McCoy; Larry Han; Amelia M Pellegrini; Rudolph E Tanzi; Sabina Berretta; Roy H Perlis Journal: Alzheimers Dement Date: 2020-01-16 Impact factor: 21.566
Authors: Michael D Hurd; Paco Martorell; Adeline Delavande; Kathleen J Mullen; Kenneth M Langa Journal: N Engl J Med Date: 2013-04-04 Impact factor: 91.245
Authors: Francine Grodstein; Chiang-Hua Chang; Ana W Capuano; Melinda C Power; David X Marquez; Lisa L Barnes; David A Bennett; Bryan D James; Julie P W Bynum Journal: J Gerontol A Biol Sci Med Sci Date: 2022-06-01 Impact factor: 6.591
Authors: Jessica E Pritchard; Lauren E Wilson; Samuel M Miller; Melissa A Greiner; Harvey Jay Cohen; Deborah R Kaye; Tian Zhang; Michaela A Dinan Journal: J Am Geriatr Soc Date: 2022-05-02 Impact factor: 7.538
Authors: Natalia Festa; Mary Price; Lidia M V R Moura; Deborah Blacker; Sharon-Lise Normand; Joseph P Newhouse; John Hsu Journal: JAMA Health Forum Date: 2022-04-22
Authors: Liqin Wang; John Laurentiev; Jie Yang; Ying-Chih Lo; Rebecca E Amariglio; Deborah Blacker; Reisa A Sperling; Gad A Marshall; Li Zhou Journal: JAMA Netw Open Date: 2021-11-01
Authors: Ayush Noori; Colin Magdamo; Xiao Liu; Tanish Tyagi; Zhaozhi Li; Akhil Kondepudi; Haitham Alabsi; Emily Rudmann; Douglas Wilcox; Laura Brenner; Gregory K Robbins; Lidia Moura; Sahar Zafar; Nicole M Benson; John Hsu; John R Dickson; Alberto Serrano-Pozo; Bradley T Hyman; Deborah Blacker; M Brandon Westover; Shibani S Mukerji; Sudeshna Das Journal: J Med Internet Res Date: 2022-08-30 Impact factor: 7.076