Samir Gupta1, Anas Belouali1, Neil J Shah2, Michael B Atkins3, Subha Madhavan1. 1. Innovation Center for Biomedical Informatics (ICBI), Georgetown University, Washington, DC. 2. Memorial Sloan Kettering Cancer Center, Manhattan, New York, NY. 3. Lombardi Comprehensive Cancer Center, MedStar Georgetown University Hospital, Washington, DC.
Abstract
PURPOSE: Although immune checkpoint inhibitors (ICIs) have substantially improved survival in patients with advanced malignancies, they are associated with a unique spectrum of side effects termed immune-related adverse events (irAEs). To ensure treatment safety, research efforts are needed to comprehensively detect and understand irAEs. Retrospective analysis of data from electronic health records can provide knowledge to characterize these toxicities. However, such information is not captured in a structured format within the electronic health record and requires manual chart review. MATERIALS AND METHODS: In this work, we propose a natural language processing pipeline that can automatically annotate clinical notes and determine whether there is evidence that a patient developed an irAE. Seven hundred eighty-one cases were manually reviewed by clinicians and annotated for irAEs at the patient level. A dictionary of irAEs keywords was used to perform text reduction on clinical notes belonging to each patient; only sentences with relevant expressions were kept. Word embeddings were then used to generate vector representations over the reduced text, which served as input for the machine learning classifiers. The output of the models was presence or absence of any irAEs. Additional models were built to classify skin-related toxicities, endocrine toxicities, and colitis. RESULTS: The model for any irAE achieved an average F1-score = 0.75 and area under the receiver operating characteristic curve = 0.85. This outperformed a basic keyword filtering approach. Although the classifier of any irAEs achieved good accuracy, individual irAE classification still has room for improvement. CONCLUSION: We demonstrate that patient-level annotations combined with a machine learning approach using keywords filtering and word embeddings can achieve promising accuracy in classifying irAEs in clinical notes. This model may facilitate annotation and analysis of large irAEs data sets.
PURPOSE: Although immune checkpoint inhibitors (ICIs) have substantially improved survival in patients with advanced malignancies, they are associated with a unique spectrum of side effects termed immune-related adverse events (irAEs). To ensure treatment safety, research efforts are needed to comprehensively detect and understand irAEs. Retrospective analysis of data from electronic health records can provide knowledge to characterize these toxicities. However, such information is not captured in a structured format within the electronic health record and requires manual chart review. MATERIALS AND METHODS: In this work, we propose a natural language processing pipeline that can automatically annotate clinical notes and determine whether there is evidence that a patient developed an irAE. Seven hundred eighty-one cases were manually reviewed by clinicians and annotated for irAEs at the patient level. A dictionary of irAEs keywords was used to perform text reduction on clinical notes belonging to each patient; only sentences with relevant expressions were kept. Word embeddings were then used to generate vector representations over the reduced text, which served as input for the machine learning classifiers. The output of the models was presence or absence of any irAEs. Additional models were built to classify skin-related toxicities, endocrine toxicities, and colitis. RESULTS: The model for any irAE achieved an average F1-score = 0.75 and area under the receiver operating characteristic curve = 0.85. This outperformed a basic keyword filtering approach. Although the classifier of any irAEs achieved good accuracy, individual irAE classification still has room for improvement. CONCLUSION: We demonstrate that patient-level annotations combined with a machine learning approach using keywords filtering and word embeddings can achieve promising accuracy in classifying irAEs in clinical notes. This model may facilitate annotation and analysis of large irAEs data sets.
Authors: Guergana K Savova; James J Masanz; Philip V Ogren; Jiaping Zheng; Sunghwan Sohn; Karin C Kipper-Schuler; Christopher G Chute Journal: J Am Med Inform Assoc Date: 2010 Sep-Oct Impact factor: 4.497
Authors: Allison Betof Warner; Jessica S Palmer; Alexander N Shoushtari; Debra A Goldman; Katherine S Panageas; Sara A Hayes; Raazi Bajwa; Parisa Momtaz; Margaret K Callahan; Jedd D Wolchok; Michael A Postow; Paul B Chapman Journal: J Clin Oncol Date: 2020-02-13 Impact factor: 44.544
Authors: Julie R Brahmer; Christina Lacchetti; Bryan J Schneider; Michael B Atkins; Kelly J Brassil; Jeffrey M Caterino; Ian Chau; Marc S Ernstoff; Jennifer M Gardner; Pamela Ginex; Sigrun Hallmeyer; Jennifer Holter Chakrabarty; Natasha B Leighl; Jennifer S Mammen; David F McDermott; Aung Naing; Loretta J Nastoupil; Tanyanika Phillips; Laura D Porter; Igor Puzanov; Cristina A Reichner; Bianca D Santomasso; Carole Seigel; Alexander Spira; Maria E Suarez-Almazor; Yinghong Wang; Jeffrey S Weber; Jedd D Wolchok; John A Thompson Journal: J Clin Oncol Date: 2018-02-14 Impact factor: 44.544
Authors: Ioana Danciu; James D Cowan; Melissa Basford; Xiaoming Wang; Alexander Saip; Susan Osgood; Jana Shirey-Rice; Jacqueline Kirby; Paul A Harris Journal: J Biomed Inform Date: 2014-02-14 Impact factor: 6.317