Pascal Lambert1,2, Marshall Pitz1,3,4,5, Harminder Singh1,4,5, Kathleen Decker6,7,8. 1. CancerCare Manitoba Research Institute, 675 McDermot Avenue, Winnipeg, Manitoba, R3E 0V9, Canada. 2. Department of Epidemiology and Cancer Registry, CancerCare Manitoba, 675 McDermot Avenue, Winnipeg, Manitoba, R3E 0V9, Canada. 3. Department of Medical Oncology, CancerCare Manitoba, 675 McDermot Avenue, Winnipeg, Manitoba, R3E 0V9, Canada. 4. Department of Internal Medicine, University of Manitoba, 820 Sherbrook Street, Winnipeg, Manitoba, R3A 1R9, Canada. 5. Department of Community Health Sciences, University of Manitoba, 750 Bannatyne Avenue, Winnipeg, Manitoba, R3E 0W3, Canada. 6. CancerCare Manitoba Research Institute, 675 McDermot Avenue, Winnipeg, Manitoba, R3E 0V9, Canada. Kdecker@cancercare.mb.ca. 7. Department of Epidemiology and Cancer Registry, CancerCare Manitoba, 675 McDermot Avenue, Winnipeg, Manitoba, R3E 0V9, Canada. Kdecker@cancercare.mb.ca. 8. Department of Community Health Sciences, University of Manitoba, 750 Bannatyne Avenue, Winnipeg, Manitoba, R3E 0W3, Canada. Kdecker@cancercare.mb.ca.
Abstract
BACKGROUND: Algorithms that use administrative health and electronic medical record (EMR) data to determine cancer recurrence have the potential to replace chart reviews. This study evaluated algorithms to determine breast and colorectal cancer recurrence in a Canadian province with a universal health care system. METHODS: Individuals diagnosed with stage I-III breast or colorectal cancer diagnosed from 2004 to 2012 in Manitoba, Canada were included. Pre-specified and conditional inference tree algorithms using administrative health and structured EMR data were developed. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) correct classification, and scaled Brier scores were measured. RESULTS: The weighted pre-specified variable algorithm for the breast cancer validation cohort (N = 1181, 167 recurrences) demonstrated 81.1% sensitivity, 93.2% specificity, 61.4% PPV, 97.4% NPV, 91.8% correct classification, and scaled Brier score of 0.21. The weighted conditional inference tree algorithm demonstrated 68.5% sensitivity, 97.0% specificity, 75.4% PPV, 95.8% NPV, 93.6% correct classification, and scaled Brier score of 0.39. The weighted pre-specified variable algorithm for the colorectal validation cohort (N = 693, 136 recurrences) demonstrated 77.7% sensitivity, 92.8% specificity, 70.7% PPV, 94.9% NPV, 90.1% correct classification, and scaled Brier score of 0.33. The conditional inference tree algorithm demonstrated 62.6% sensitivity, 97.8% specificity, 86.4% PPV, 92.2% NPV, 91.4% correct classification, and scaled Brier score of 0.42. CONCLUSIONS: Algorithms developed in this study using administrative health and structured EMR data to determine breast and colorectal cancer recurrence had moderate sensitivity and PPV, high specificity, NPV, and correct classification, but low accuracy. The accuracy is similar to other algorithms developed to classify recurrence only (i.e., distinguished from second primary) and inferior to algorithms that do not make this distinction. The accuracy of algorithms for determining cancer recurrence only must improve before replacing chart reviews.
BACKGROUND: Algorithms that use administrative health and electronic medical record (EMR) data to determine cancer recurrence have the potential to replace chart reviews. This study evaluated algorithms to determine breast and colorectal cancer recurrence in a Canadian province with a universal health care system. METHODS: Individuals diagnosed with stage I-III breast or colorectal cancer diagnosed from 2004 to 2012 in Manitoba, Canada were included. Pre-specified and conditional inference tree algorithms using administrative health and structured EMR data were developed. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) correct classification, and scaled Brier scores were measured. RESULTS: The weighted pre-specified variable algorithm for the breast cancer validation cohort (N = 1181, 167 recurrences) demonstrated 81.1% sensitivity, 93.2% specificity, 61.4% PPV, 97.4% NPV, 91.8% correct classification, and scaled Brier score of 0.21. The weighted conditional inference tree algorithm demonstrated 68.5% sensitivity, 97.0% specificity, 75.4% PPV, 95.8% NPV, 93.6% correct classification, and scaled Brier score of 0.39. The weighted pre-specified variable algorithm for the colorectal validation cohort (N = 693, 136 recurrences) demonstrated 77.7% sensitivity, 92.8% specificity, 70.7% PPV, 94.9% NPV, 90.1% correct classification, and scaled Brier score of 0.33. The conditional inference tree algorithm demonstrated 62.6% sensitivity, 97.8% specificity, 86.4% PPV, 92.2% NPV, 91.4% correct classification, and scaled Brier score of 0.42. CONCLUSIONS: Algorithms developed in this study using administrative health and structured EMR data to determine breast and colorectal cancer recurrence had moderate sensitivity and PPV, high specificity, NPV, and correct classification, but low accuracy. The accuracy is similar to other algorithms developed to classify recurrence only (i.e., distinguished from second primary) and inferior to algorithms that do not make this distinction. The accuracy of algorithms for determining cancer recurrence only must improve before replacing chart reviews.
Entities:
Keywords:
Algorithms; Breast cancer; Canada; Colorectal cancer; Recurrence; Validation studies
Authors: Michael J Hassett; Hajime Uno; Angel M Cronin; Nikki M Carroll; Mark C Hornbrook; Debra Ritzwoller Journal: Med Care Date: 2017-12 Impact factor: 2.983
Authors: Rebecca L Siegel; Kimberly D Miller; Stacey A Fedewa; Dennis J Ahnen; Reinier G S Meester; Afsaneh Barzi; Ahmedin Jemal Journal: CA Cancer J Clin Date: 2017-03-01 Impact factor: 508.702
Authors: Nils D Arvold; Alphonse G Taghian; Andrzej Niemierko; Rita F Abi Raad; Meera Sreedhara; Paul L Nguyen; Jennifer R Bellon; Julia S Wong; Barbara L Smith; Jay R Harris Journal: J Clin Oncol Date: 2011-09-06 Impact factor: 44.544
Authors: Jessica Chubak; Onchee Yu; Gaia Pocobelli; Lois Lamerato; Joe Webster; Marianne N Prout; Marianne Ulcickas Yood; William E Barlow; Diana S M Buist Journal: J Natl Cancer Inst Date: 2012-04-30 Impact factor: 13.506
Authors: Michael J Hassett; Debra P Ritzwoller; Nathan Taback; Nikki Carroll; Angel M Cronin; Gladys V Ting; Deb Schrag; Joan L Warren; Mark C Hornbrook; Jane C Weeks Journal: Med Care Date: 2014-10 Impact factor: 2.983
Authors: Sarah P Huepenbecker; Hui Zhao; Charlotte C Sun; Shuangshuang Fu; Weiguo He; Sharon H Giordano; Larissa A Meyer Journal: JCO Clin Cancer Inform Date: 2022-03