Charlotta Lindvall1, Chih-Ying Deng2, Edward Moseley2, Nicole Agaronnik3, Areej El-Jawahri4, Michael K Paasche-Orlow5, Joshua R Lakin6, Angelo Volandes7, The Acp-Peace Investigators James A Tulsky6. 1. Department of Psychosocial Oncology and Palliative Care, Dana-Farber Cancer Institute (C.L., CY.D.,E.M., N.A., JR.L., JA.T.), Boston, Massachusetts; Department of Medicine, Brigham and Women's Hospital (C.L., JR.L., JA.T.), Boston, Massachusetts; Harvard Medical School, Boston (C.L., N.A., A.EJ., JR.L., A.V., JA.T.), Massachusetts. Electronic address: charlotta_lindvall@dfci.harvard.edu. 2. Department of Psychosocial Oncology and Palliative Care, Dana-Farber Cancer Institute (C.L., CY.D.,E.M., N.A., JR.L., JA.T.), Boston, Massachusetts. 3. Department of Psychosocial Oncology and Palliative Care, Dana-Farber Cancer Institute (C.L., CY.D.,E.M., N.A., JR.L., JA.T.), Boston, Massachusetts; Harvard Medical School, Boston (C.L., N.A., A.EJ., JR.L., A.V., JA.T.), Massachusetts. 4. Harvard Medical School, Boston (C.L., N.A., A.EJ., JR.L., A.V., JA.T.), Massachusetts; Department of Medicine, Massachusetts General Hospital (A.EJ., A.V.), Boston, Massachusetts. 5. Department of Medicine, Boston University School of Medicine, Boston Medical Center (MK.PO.), Boston, Massachusetts; ACP Decisions (MK.PO., A.V.), Boston, Massachusetts. 6. Department of Psychosocial Oncology and Palliative Care, Dana-Farber Cancer Institute (C.L., CY.D.,E.M., N.A., JR.L., JA.T.), Boston, Massachusetts; Department of Medicine, Brigham and Women's Hospital (C.L., JR.L., JA.T.), Boston, Massachusetts; Harvard Medical School, Boston (C.L., N.A., A.EJ., JR.L., A.V., JA.T.), Massachusetts. 7. Harvard Medical School, Boston (C.L., N.A., A.EJ., JR.L., A.V., JA.T.), Massachusetts; Department of Medicine, Massachusetts General Hospital (A.EJ., A.V.), Boston, Massachusetts; ACP Decisions (MK.PO., A.V.), Boston, Massachusetts.
Abstract
CONTEXT: Large multisite clinical trials studying decision-making when facing serious illness require an efficient method for abstraction of advance care planning (ACP) documentation from clinical text documents. However, the current gold standard method of manual chart review is time-consuming and unreliable. OBJECTIVES: To evaluate the ability to use natural language processing (NLP) to identify ACP documention in clinical notes from patients participating in a multisite trial. METHODS: Patients with advanced cancer followed in three disease-focused oncology clinics at Duke Health, Mayo Clinic, and Northwell Health were identified using administrative data. All outpatient and inpatient notes from patients meeting inclusion criteria were extracted from electronic health records (EHRs) between March 2018 and March 2019. NLP text identification software with semi-automated chart review was applied to identify documentation of four ACP domains: (1) conversations about goals of care, (2) limitation of life-sustaining treatment, (3) involvement of palliative care, and (4) discussion of hospice. The performance of NLP was compared to gold standard manual chart review. RESULTS: 435 unique patients with 79,797 notes were included in the study. In our validation data set, NLP achieved F1 scores ranging from 0.84 to 0.97 across domains compared to gold standard manual chart review. NLP identified ACP documentation in a fraction of the time required by manual chart review of EHRs (1-5 minutes per patient for NLP, vs. 30-120 minutes for manual abstraction). CONCLUSION: NLP is more efficient and as accurate as manual chart review for identifying ACP documentation in studies with large patient cohorts.
CONTEXT: Large multisite clinical trials studying decision-making when facing serious illness require an efficient method for abstraction of advance care planning (ACP) documentation from clinical text documents. However, the current gold standard method of manual chart review is time-consuming and unreliable. OBJECTIVES: To evaluate the ability to use natural language processing (NLP) to identify ACP documention in clinical notes from patients participating in a multisite trial. METHODS: Patients with advanced cancer followed in three disease-focused oncology clinics at Duke Health, Mayo Clinic, and Northwell Health were identified using administrative data. All outpatient and inpatient notes from patients meeting inclusion criteria were extracted from electronic health records (EHRs) between March 2018 and March 2019. NLP text identification software with semi-automated chart review was applied to identify documentation of four ACP domains: (1) conversations about goals of care, (2) limitation of life-sustaining treatment, (3) involvement of palliative care, and (4) discussion of hospice. The performance of NLP was compared to gold standard manual chart review. RESULTS: 435 unique patients with 79,797 notes were included in the study. In our validation data set, NLP achieved F1 scores ranging from 0.84 to 0.97 across domains compared to gold standard manual chart review. NLP identified ACP documentation in a fraction of the time required by manual chart review of EHRs (1-5 minutes per patient for NLP, vs. 30-120 minutes for manual abstraction). CONCLUSION: NLP is more efficient and as accurate as manual chart review for identifying ACP documentation in studies with large patient cohorts.
Authors: Mary H Stanfill; Margaret Williams; Susan H Fenton; Robert A Jenders; William R Hersh Journal: J Am Med Inform Assoc Date: 2010 Nov-Dec Impact factor: 4.497
Authors: Thomas M Atkinson; Yuelin Li; Charles W Coffey; Laura Sit; Mary Shaw; Dawn Lavene; Antonia V Bennett; Mike Fruscione; Lauren Rogak; Jennifer Hay; Mithat Gönen; Deborah Schrag; Ethan Basch Journal: Qual Life Res Date: 2011-10-08 Impact factor: 4.147
Authors: Jennifer Freytag; Richard L Street; Deborah E Barnes; Ying Shi; Aiesha M Volow; Janet K Shim; Stewart C Alexander; Rebecca L Sudore Journal: J Am Geriatr Soc Date: 2020-03-10 Impact factor: 5.562
Authors: Kuldeep N Yadav; Nicole B Gabler; Elizabeth Cooney; Saida Kent; Jennifer Kim; Nicole Herbst; Adjoa Mante; Scott D Halpern; Katherine R Courtright Journal: Health Aff (Millwood) Date: 2017-07-01 Impact factor: 6.301
Authors: Corita R Grudzen; Abraham A Brody; Frank R Chung; Allison M Cuthel; Devin Mann; Jordan A McQuilkin; Ada L Rubin; Jordan Swartz; Audrey Tan; Keith S Goldfeld Journal: BMJ Open Date: 2019-07-27 Impact factor: 2.692
Authors: Robert Y Lee; Lyndia C Brumback; William B Lober; James Sibley; Elizabeth L Nielsen; Patsy D Treece; Erin K Kross; Elizabeth T Loggers; James A Fausto; Charlotta Lindvall; Ruth A Engelberg; J Randall Curtis Journal: J Pain Symptom Manage Date: 2020-08-25 Impact factor: 3.612
Authors: Nienke Fleuren; Marja F I A Depla; Daisy J A Janssen; Martijn Huisman; Cees M P M Hertogh Journal: BMC Palliat Care Date: 2020-03-06 Impact factor: 3.234
Authors: Anahita Davoudi; Hegler Tissot; Abigail Doucette; Peter E Gabriel; Ravi Parikh; Danielle L Mowery; Stephen P Miranda Journal: AMIA Annu Symp Proc Date: 2022-05-23
Authors: Charlotta Lindvall; Chih-Ying Deng; Nicole D Agaronnik; Anne Kwok; Soujanya Samineni; Renato Umeton; Warren Mackie-Jenkins; Kenneth L Kehl; James A Tulsky; Andrea C Enzinger Journal: JCO Clin Cancer Inform Date: 2022-06
Authors: Angelo E Volandes; Sophia N Zupanc; Michael K Paasche-Orlow; Joshua R Lakin; Yuchiao Chang; Edith A Burns; Nancy A LaVine; Maria T Carney; Diana Martins-Welch; Kaitlin Emmert; Jennifer E Itty; Edward T Moseley; Aretha D Davis; Areej El-Jawahri; Daniel A Gundersen; Gemmae M Fix; Andrea M Yacoub; Pamela Schwartz; Shira Gabry-Kalikow; Cynthia Garde; Jonathan Fischer; Lori Henault; Leah Burgess; Julie Goldman; Anne Kwok; Nimisha Singh; Armando L Alvarez Suarez; Valeria Gromova; Sonia Jacome; James A Tulsky; Charlotta Lindvall Journal: JAMA Netw Open Date: 2022-02-01
Authors: Nwamaka D Eneanya; Joshua R Lakin; Michael K Paasche-Orlow; Charlotta Lindvall; Edward T Moseley; Lori Henault; Amresh D Hanchate; Ernest I Mandel; Susan P Y Wong; Sophia N Zupanc; Aretha Delight Davis; Areej El-Jawahri; Lisa M Quintiliani; Yuchiao Chang; Sushrut S Waikar; Amar D Bansal; Jane O Schell; Andrew L Lundquist; Manjula Kurella Tamura; Margaret K Yu; Mark L Unruh; Christos Argyropoulos; Michael J Germain; Angelo Volandes Journal: BMJ Open Date: 2022-04-08 Impact factor: 2.692
Authors: Joshua R Lakin; Sophia N Zupanc; Charlotta Lindvall; Edward T Moseley; Sophiya Das; Kate Sciacca; Howard J Cabral; Edith A Burns; Maria T Carney; Jennifer Itty; Santiago Lopez; Kaitlin Emmert; Narda J Martin; Sherene Lambert; Jennifer Polo; Shreya Sanghani; Julianne N Dugas; Michele Gomez; Michael R Winter; Na Wang; Shira Gabry-Kalikow; Alexandra Dobie; Meredith Amshoff; Traci Cucinotta; Milton Joel; Lisa B Caruso; Ana Maria Ramirez; Kathleen Salerno; Qausarat Ogunneye; Lori Henault; Aretha Delight Davis; Angelo Volandes; Michael K Paasche-Orlow Journal: BMJ Open Date: 2022-07-25 Impact factor: 3.006