Michael E Detsky1, Michael O Harhay2, Dominique F Bayard3, Aaron M Delman4, Anna E Buehler5, Saida A Kent6, Isabella V Ciuffetelli7, Elizabeth Cooney8, Nicole B Gabler7, Sarah J Ratcliffe9, Mark E Mikkelsen10, Scott D Halpern11. 1. Palliative and Advanced Illness Research (PAIR) Center, University of Pennsylvania Perelman School of Medicine, Philadelphia2Sinai Health System, Toronto, Ontario, Canada3Depatment of Medicine, University of Toronto, Toronto, Ontario, Canada4Interdepartmental Division of Critical Care, University of Toronto, Toronto, Ontario, Canada. 2. Palliative and Advanced Illness Research (PAIR) Center, University of Pennsylvania Perelman School of Medicine, Philadelphia5Department of Biostatistics and Epidemiology, University of Pennsylvania Perelman School of Medicine, Philadelphia6Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia. 3. Pulmonary and Critical Care of Atlanta, Atlanta, Georgia. 4. Wayne State University School of Medicine, Detroit, Michigan. 5. University of California, San Diego School of Medicine. 6. University of Kentucky College of Medicine, Lexington. 7. Palliative and Advanced Illness Research (PAIR) Center, University of Pennsylvania Perelman School of Medicine, Philadelphia. 8. Palliative and Advanced Illness Research (PAIR) Center, University of Pennsylvania Perelman School of Medicine, Philadelphia6Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia. 9. Department of Biostatistics and Epidemiology, University of Pennsylvania Perelman School of Medicine, Philadelphia. 10. Department of Biostatistics and Epidemiology, University of Pennsylvania Perelman School of Medicine, Philadelphia11Division of Pulmonary, Allergy, and Critical Care Medicine, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia. 11. Palliative and Advanced Illness Research (PAIR) Center, University of Pennsylvania Perelman School of Medicine, Philadelphia5Department of Biostatistics and Epidemiology, University of Pennsylvania Perelman School of Medicine, Philadelphia6Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia11Division of Pulmonary, Allergy, and Critical Care Medicine, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia12Department of Medical Ethics and Health Policy, University of Pennsylvania, Philadelphia.
Abstract
IMPORTANCE: Predictions of long-term survival and functional outcomes influence decision making for critically ill patients, yet little is known regarding their accuracy. OBJECTIVE: To determine the discriminative accuracy of intensive care unit (ICU) physicians and nurses in predicting 6-month patient mortality and morbidity, including ambulation, toileting, and cognition. DESIGN, SETTING, AND PARTICIPANTS: Prospective cohort study conducted in 5 ICUs in 3 hospitals in Philadelphia, Pennsylvania, and enrolling patients who spent at least 3 days in the ICU from October 2013 until May 2014 and required mechanical ventilation, vasopressors, or both. These patients' attending physicians and bedside nurses were also enrolled. Follow-up was completed in December 2014. MAIN OUTCOMES AND MEASURES: ICU physicians' and nurses' binary predictions of in-hospital mortality and 6-month outcomes, including mortality, return to original residence, ability to toilet independently, ability to ambulate up 10 stairs independently, and ability to remember most things, think clearly, and solve day-to-day problems (ie, normal cognition). For each outcome, physicians and nurses provided a dichotomous prediction and rated their confidence in that prediction on a 5-point Likert scale. Outcomes were assessed via interviews with surviving patients or their surrogates at 6 months. Discriminative accuracy was measured using positive and negative likelihood ratios (LRs), C statistics, and other operating characteristics. RESULTS: Among 340 patients approached, 303 (89%) consented (median age, 62 years [interquartile range, 53-71]; 57% men; 32% African American); 6-month follow-up was completed for 299 (99%), of whom 169 (57%) were alive. Predictions were made by 47 physicians and 128 nurses. Physicians most accurately predicted 6-month mortality (positive LR, 5.91 [95% CI, 3.74-9.32]; negative LR, 0.41 [95% CI, 0.33-0.52]; C statistic, 0.76 [95% CI, 0.72-0.81]) and least accurately predicted cognition (positive LR, 2.36 [95% CI, 1.36-4.12]; negative LR, 0.75 [95% CI, 0.61-0.92]; C statistic, 0.61 [95% CI, 0.54-0.68]). Nurses most accurately predicted in-hospital mortality (positive LR, 4.71 [95% CI, 2.94-7.56]; negative LR, 0.61 [95% CI, 0.49-0.75]; C statistic, 0.68 [95% CI, 0.62-0.74]) and least accurately predicted cognition (positive LR, 1.50 [95% CI, 0.86-2.60]; negative LR, 0.88 [95% CI, 0.73-1.06]; C statistic, 0.55 [95% CI, 0.48-0.62]). Discriminative accuracy was higher when physicians and nurses were confident about their predictions (eg, for physicians' confident predictions of 6-month mortality: positive LR, 33.00 [95% CI, 8.34-130.63]; negative LR, 0.18 [95% CI, 0.09-0.35]; C statistic, 0.90 [95% CI, 0.84-0.96]). Compared with a predictive model including objective clinical variables, a model that also included physician and nurse predictions had significantly higher discriminative accuracy for in-hospital mortality, 6-month mortality, and return to original residence (P < .01 for all). CONCLUSIONS AND RELEVANCE: ICU physicians' and nurses' discriminative accuracy in predicting 6-month outcomes of critically ill patients varied depending on the outcome being predicted and confidence of the predictors. Further research is needed to better understand how clinicians derive prognostic estimates of long-term outcomes.
IMPORTANCE: Predictions of long-term survival and functional outcomes influence decision making for critically ill patients, yet little is known regarding their accuracy. OBJECTIVE: To determine the discriminative accuracy of intensive care unit (ICU) physicians and nurses in predicting 6-month patient mortality and morbidity, including ambulation, toileting, and cognition. DESIGN, SETTING, AND PARTICIPANTS: Prospective cohort study conducted in 5 ICUs in 3 hospitals in Philadelphia, Pennsylvania, and enrolling patients who spent at least 3 days in the ICU from October 2013 until May 2014 and required mechanical ventilation, vasopressors, or both. These patients' attending physicians and bedside nurses were also enrolled. Follow-up was completed in December 2014. MAIN OUTCOMES AND MEASURES: ICU physicians' and nurses' binary predictions of in-hospital mortality and 6-month outcomes, including mortality, return to original residence, ability to toilet independently, ability to ambulate up 10 stairs independently, and ability to remember most things, think clearly, and solve day-to-day problems (ie, normal cognition). For each outcome, physicians and nurses provided a dichotomous prediction and rated their confidence in that prediction on a 5-point Likert scale. Outcomes were assessed via interviews with surviving patients or their surrogates at 6 months. Discriminative accuracy was measured using positive and negative likelihood ratios (LRs), C statistics, and other operating characteristics. RESULTS: Among 340 patients approached, 303 (89%) consented (median age, 62 years [interquartile range, 53-71]; 57% men; 32% African American); 6-month follow-up was completed for 299 (99%), of whom 169 (57%) were alive. Predictions were made by 47 physicians and 128 nurses. Physicians most accurately predicted 6-month mortality (positive LR, 5.91 [95% CI, 3.74-9.32]; negative LR, 0.41 [95% CI, 0.33-0.52]; C statistic, 0.76 [95% CI, 0.72-0.81]) and least accurately predicted cognition (positive LR, 2.36 [95% CI, 1.36-4.12]; negative LR, 0.75 [95% CI, 0.61-0.92]; C statistic, 0.61 [95% CI, 0.54-0.68]). Nurses most accurately predicted in-hospital mortality (positive LR, 4.71 [95% CI, 2.94-7.56]; negative LR, 0.61 [95% CI, 0.49-0.75]; C statistic, 0.68 [95% CI, 0.62-0.74]) and least accurately predicted cognition (positive LR, 1.50 [95% CI, 0.86-2.60]; negative LR, 0.88 [95% CI, 0.73-1.06]; C statistic, 0.55 [95% CI, 0.48-0.62]). Discriminative accuracy was higher when physicians and nurses were confident about their predictions (eg, for physicians' confident predictions of 6-month mortality: positive LR, 33.00 [95% CI, 8.34-130.63]; negative LR, 0.18 [95% CI, 0.09-0.35]; C statistic, 0.90 [95% CI, 0.84-0.96]). Compared with a predictive model including objective clinical variables, a model that also included physician and nurse predictions had significantly higher discriminative accuracy for in-hospital mortality, 6-month mortality, and return to original residence (P < .01 for all). CONCLUSIONS AND RELEVANCE: ICU physicians' and nurses' discriminative accuracy in predicting 6-month outcomes of critically ill patients varied depending on the outcome being predicted and confidence of the predictors. Further research is needed to better understand how clinicians derive prognostic estimates of long-term outcomes.
Authors: Mark Unroe; Jeremy M Kahn; Shannon S Carson; Joseph A Govert; Tereza Martinu; Shailaja J Sathy; Alison S Clay; Jessica Chia; Alice Gray; James A Tulsky; Christopher E Cox Journal: Ann Intern Med Date: 2010-08-03 Impact factor: 25.391
Authors: Christopher E Cox; Tereza Martinu; Shailaja J Sathy; Alison S Clay; Jessica Chia; Alice L Gray; Maren K Olsen; Joseph A Govert; Shannon S Carson; James A Tulsky Journal: Crit Care Med Date: 2009-11 Impact factor: 7.598
Authors: Margaret S Herridge; Leslie M Chu; Andrea Matte; George Tomlinson; Linda Chan; Claire Thomas; Jan O Friedrich; Sangeeta Mehta; Francois Lamontagne; Melanie Levasseur; Niall D Ferguson; Neill K J Adhikari; Jill C Rudkowski; Hilary Meggison; Yoanna Skrobik; John Flannery; Mark Bayley; Jane Batt; Claudia Dos Santos; Susan E Abbey; Adrienne Tan; Vincent Lo; Sunita Mathur; Matteo Parotto; Denise Morris; Linda Flockhart; Eddy Fan; Christie M Lee; M Elizabeth Wilcox; Najib Ayas; Karen Choong; Robert Fowler; Damon C Scales; Tasnim Sinuff; Brian H Cuthbertson; Louise Rose; Priscila Robles; Stacey Burns; Marcelo Cypel; Lianne Singer; Cecelia Chaparro; Chung-Wai Chow; Shaf Keshavjee; Laurent Brochard; Paul Hebert; Arthur S Slutsky; John C Marshall; Deborah Cook; Jill I Cameron Journal: Am J Respir Crit Care Med Date: 2016-10-01 Impact factor: 21.405
Authors: Deborah Cook; Graeme Rocker; John Marshall; Peter Sjokvist; Peter Dodek; Lauren Griffith; Andreas Freitag; Joseph Varon; Christine Bradley; Mitchell Levy; Simon Finfer; Cindy Hamielec; Joseph McMullin; Bruce Weaver; Stephen Walter; Gordon Guyatt Journal: N Engl J Med Date: 2003-09-18 Impact factor: 91.245
Authors: Gary E Weissman; Rebecca A Hubbard; Lyle H Ungar; Michael O Harhay; Casey S Greene; Blanca E Himes; Scott D Halpern Journal: Crit Care Med Date: 2018-07 Impact factor: 7.598
Authors: Ariel Jaitovich; Malik M H S Khan; Ria Itty; Hau C Chieng; Camille L Dumas; Pallavi Nadendla; John P Fantauzzi; Recai M Yucel; Paul J Feustel; Marc A Judson Journal: Chest Date: 2018-10-28 Impact factor: 9.410
Authors: Michael E Detsky; Michael O Harhay; Dominique F Bayard; Aaron M Delman; Anna E Buehler; Saida A Kent; Isabella V Ciuffetelli; Elizabeth Cooney; Nicole B Gabler; Sarah J Ratcliffe; Mark E Mikkelsen; Scott D Halpern Journal: Ann Am Thorac Soc Date: 2017-10
Authors: Gary E Weissman; Lyle H Ungar; Michael O Harhay; Katherine R Courtright; Scott D Halpern Journal: J Biomed Inform Date: 2018-12-14 Impact factor: 6.317