Yasmin H Karimi1, Douglas W Blayney1, Allison W Kurian1,2, Jeanne Shen3, Rikiya Yamashita3, Daniel Rubin4,5, Imon Banerjee6,7. 1. Department of Medicine, Stanford University School of Medicine, Stanford, CA. 2. Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, CA. 3. Department of Pathology, Stanford University School of Medicine, Stanford, CA. 4. Department of Radiology, Stanford University School of Medicine, Stanford, CA. 5. Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA. 6. Department of Biomedical Informatics, Emory University, Atlanta, GA. 7. Department of Radiology, Emory University, Atlanta, GA.
Abstract
PURPOSE: Large-scale analysis of real-world evidence is often limited to structured data fields that do not contain reliable information on recurrence status and disease sites. In this report, we describe a natural language processing (NLP) framework that uses data from free-text, unstructured reports to classify recurrence status and sites of recurrence for patients with breast and hepatocellular carcinomas (HCC). METHODS: Using two cohorts of breast cancer and HCC cases, we validated the ability of a previously developed NLP model to distinguish between no recurrence, local recurrence, and distant recurrence, based on clinician notes, radiology reports, and pathology reports compared with manual curation. A second NLP model was trained and validated to identify sites of recurrence. We compared the ability of each NLP model to identify the presence, timing, and site of recurrence, when compared against manual chart review and International Classification of Diseases coding. RESULTS: A total of 1,273 patients were included in the development and validation of the two models. The NLP model for recurrence detects distant recurrence with an area under the curve of 0.98 (95% CI, 0.96 to 0.99) and 0.95 (95% CI, 0.88 to 0.98) in breast and HCC cohorts, respectively. The mean accuracy of the NLP model for detecting any site of distant recurrence was 0.9 for breast cancer and 0.83 for HCC. The NLP model for recurrence identified a larger proportion of patients with distant recurrence in a breast cancer database (11.1%) compared with International Classification of Diseases coding (2.31%). CONCLUSION: We developed two NLP models to identify distant cancer recurrence, timing of recurrence, and sites of recurrence based on unstructured electronic health record data. These models can be used to perform large-scale retrospective studies in oncology.
PURPOSE: Large-scale analysis of real-world evidence is often limited to structured data fields that do not contain reliable information on recurrence status and disease sites. In this report, we describe a natural language processing (NLP) framework that uses data from free-text, unstructured reports to classify recurrence status and sites of recurrence for patients with breast and hepatocellular carcinomas (HCC). METHODS: Using two cohorts of breast cancer and HCC cases, we validated the ability of a previously developed NLP model to distinguish between no recurrence, local recurrence, and distant recurrence, based on clinician notes, radiology reports, and pathology reports compared with manual curation. A second NLP model was trained and validated to identify sites of recurrence. We compared the ability of each NLP model to identify the presence, timing, and site of recurrence, when compared against manual chart review and International Classification of Diseases coding. RESULTS: A total of 1,273 patients were included in the development and validation of the two models. The NLP model for recurrence detects distant recurrence with an area under the curve of 0.98 (95% CI, 0.96 to 0.99) and 0.95 (95% CI, 0.88 to 0.98) in breast and HCC cohorts, respectively. The mean accuracy of the NLP model for detecting any site of distant recurrence was 0.9 for breast cancer and 0.83 for HCC. The NLP model for recurrence identified a larger proportion of patients with distant recurrence in a breast cancer database (11.1%) compared with International Classification of Diseases coding (2.31%). CONCLUSION: We developed two NLP models to identify distant cancer recurrence, timing of recurrence, and sites of recurrence based on unstructured electronic health record data. These models can be used to perform large-scale retrospective studies in oncology.
Authors: Reina Haque; Jiaxiao Shi; Joanne E Schottinger; Syed Ajaz Ahmed; Joanie Chung; Chantal Avila; Valerie S Lee; Thomas Craig Cheetham; Laurel A Habel; Suzanne W Fletcher; Marilyn L Kwan Journal: Med Care Date: 2015-04 Impact factor: 2.983
Authors: David S Carrell; Scott Halgrim; Diem-Thy Tran; Diana S M Buist; Jessica Chubak; Wendy W Chapman; Guergana Savova Journal: Am J Epidemiol Date: 2014-01-30 Impact factor: 4.897
Authors: Jessica Chubak; Onchee Yu; Gaia Pocobelli; Lois Lamerato; Joe Webster; Marianne N Prout; Marianne Ulcickas Yood; William E Barlow; Diana S M Buist Journal: J Natl Cancer Inst Date: 2012-04-30 Impact factor: 13.506
Authors: Elizabeth B Lamont; James E Herndon; Jane C Weeks; I Craig Henderson; Craig C Earle; Richard L Schilsky; Nicholas A Christakis Journal: J Natl Cancer Inst Date: 2006-09-20 Impact factor: 13.506