Katherine C Lee1, Brooks V Udelsman2, Jocelyn Streid3, David C Chang2, Ali Salim4, David H Livingston5, Charlotta Lindvall6, Zara Cooper4. 1. The Center for Surgery and Public Health, Brigham and Women's Hospital, Boston, Massachusetts, USA; Department of Surgery, University of California, San Diego, La Jolla, California, USA. Electronic address: kclee@bwh.harvard.edu. 2. Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts, USA; Codman Center for Clinical Effectiveness, Massachusetts General Hospital, Boston, Massachusetts, USA. 3. Harvard Medical School, Boston, Massachusetts, USA. 4. Department of Surgery, University of California, San Diego, La Jolla, California, USA; Division of Trauma, Burns, and Surgical Critical Care, Brigham and Women's Hospital, Boston, Massachusetts, USA. 5. Division of Trauma and Surgical Critical Care, Rutgers New Jersey Medical School, Newark, New Jersey, USA. 6. Department of Psychosocial Oncology and Palliative Care, Dana Farber Cancer Institute, Boston, Massachusetts, USA.
Abstract
CONTEXT: The Trauma Quality Improvement Program Best Practice Guidelines recommend palliative care (PC) concurrent with restorative treatment for patients with life-threatening injuries. Measuring PC delivery is challenging: administrative data are nonspecific, and manual review is time intensive. OBJECTIVES: To identify PC delivery to patients with life-threatening trauma and compare the performance of natural language processing (NLP), a form of computer-assisted data abstraction, to administrative coding and gold standard manual review. METHODS: Patients 18 years and older admitted with life-threatening trauma were identified from two Level I trauma centers (July 2016-June 2017). Four PC process measures were examined during the trauma admission: code status clarification, goals-of-care discussion, PC consult, and hospice assessment. The performance of NLP and administrative coding were compared with manual review. Multivariable regression was used to determine patient and admission factors associated with PC delivery. RESULTS: There were 76,791 notes associated with 2093 admissions. NLP identified PC delivery in 33% of admissions compared with 8% using administrative coding. Using NLP, code status clarification was most commonly documented (27%), followed by goals-of-care discussion (18%), PC consult (4%), and hospice assessment (4%). Compared with manual review, NLP performed more than 50 times faster and had a sensitivity of 93%, a specificity of 96%, and an accuracy of 95%. Administrative coding had a sensitivity of 21%, a specificity of 92%, and an accuracy of 68%. Factors associated with PC delivery included older age, increased comorbidities, and longer intensive care unit stay. CONCLUSION: NLP performs with similar accuracy with manual review but with improved efficiency. NLP has the potential to accurately identify PC delivery and benchmark performance of best practice guidelines.
CONTEXT: The Trauma Quality Improvement Program Best Practice Guidelines recommend palliative care (PC) concurrent with restorative treatment for patients with life-threatening injuries. Measuring PC delivery is challenging: administrative data are nonspecific, and manual review is time intensive. OBJECTIVES: To identify PC delivery to patients with life-threatening trauma and compare the performance of natural language processing (NLP), a form of computer-assisted data abstraction, to administrative coding and gold standard manual review. METHODS:Patients 18 years and older admitted with life-threatening trauma were identified from two Level I trauma centers (July 2016-June 2017). Four PC process measures were examined during the trauma admission: code status clarification, goals-of-care discussion, PC consult, and hospice assessment. The performance of NLP and administrative coding were compared with manual review. Multivariable regression was used to determine patient and admission factors associated with PC delivery. RESULTS: There were 76,791 notes associated with 2093 admissions. NLP identified PC delivery in 33% of admissions compared with 8% using administrative coding. Using NLP, code status clarification was most commonly documented (27%), followed by goals-of-care discussion (18%), PC consult (4%), and hospice assessment (4%). Compared with manual review, NLP performed more than 50 times faster and had a sensitivity of 93%, a specificity of 96%, and an accuracy of 95%. Administrative coding had a sensitivity of 21%, a specificity of 92%, and an accuracy of 68%. Factors associated with PC delivery included older age, increased comorbidities, and longer intensive care unit stay. CONCLUSION: NLP performs with similar accuracy with manual review but with improved efficiency. NLP has the potential to accurately identify PC delivery and benchmark performance of best practice guidelines.
Authors: Anahita Davoudi; Hegler Tissot; Abigail Doucette; Peter E Gabriel; Ravi Parikh; Danielle L Mowery; Stephen P Miranda Journal: AMIA Annu Symp Proc Date: 2022-05-23
Authors: Angelo E Volandes; Sophia N Zupanc; Michael K Paasche-Orlow; Joshua R Lakin; Yuchiao Chang; Edith A Burns; Nancy A LaVine; Maria T Carney; Diana Martins-Welch; Kaitlin Emmert; Jennifer E Itty; Edward T Moseley; Aretha D Davis; Areej El-Jawahri; Daniel A Gundersen; Gemmae M Fix; Andrea M Yacoub; Pamela Schwartz; Shira Gabry-Kalikow; Cynthia Garde; Jonathan Fischer; Lori Henault; Leah Burgess; Julie Goldman; Anne Kwok; Nimisha Singh; Armando L Alvarez Suarez; Valeria Gromova; Sonia Jacome; James A Tulsky; Charlotta Lindvall Journal: JAMA Netw Open Date: 2022-02-01
Authors: Joshua R Lakin; Sophia N Zupanc; Charlotta Lindvall; Edward T Moseley; Sophiya Das; Kate Sciacca; Howard J Cabral; Edith A Burns; Maria T Carney; Jennifer Itty; Santiago Lopez; Kaitlin Emmert; Narda J Martin; Sherene Lambert; Jennifer Polo; Shreya Sanghani; Julianne N Dugas; Michele Gomez; Michael R Winter; Na Wang; Shira Gabry-Kalikow; Alexandra Dobie; Meredith Amshoff; Traci Cucinotta; Milton Joel; Lisa B Caruso; Ana Maria Ramirez; Kathleen Salerno; Qausarat Ogunneye; Lori Henault; Aretha Delight Davis; Angelo Volandes; Michael K Paasche-Orlow Journal: BMJ Open Date: 2022-07-25 Impact factor: 3.006