Erin F Carlton1,2, Jeylan Close3,4, Kelli Paice3, Alyssa Dews5, Stephen M Gorga1, Julie Sturza3, Ryan P Barbaro1,2, Timothy T Cornell6, Hallie C Prescott7,8. 1. Department of Pediatrics, Division of Critical Care Medicine, University of Michigan, Ann Arbor, MI. 2. Susan B. Meister Child Health Evaluation and Research Center, Department of Pediatrics, University of Michigan, Ann Arbor, MI. 3. Department of Pediatrics, University of Michigan, Ann Arbor, MI. 4. Department of Child and Adolescent Psychiatry and Behavioral Sciences, Children's Hospital of Philadelphia, Philadelphia, PA. 5. College of Literature, Science and the Arts, University of Michigan, Ann Arbor, MI. 6. Department of Pediatrics, Division of Critical Care Medicine, Stanford University, Palo Alto, CA. 7. Department of Internal Medicine, Division of Pulmonary and Critical Care, University of Michigan, Ann Arbor, MI. 8. VA Center for Clinical Management Research, HSR&D Center of Innovation, Ann Arbor, MI.
Abstract
OBJECTIVES: To determine clinician accuracy in the identification and prediction of multiple organ dysfunction syndrome. DESIGN: Prospective cohort study. SETTING: University of Michigan's C.S. Mott Children's Hospital PICU. PATIENTS: Patients admitted to the PICU with an anticipated PICU length of stay greater than 48 hours. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: For each patient, the clinical team (attending, fellow, resident/nurse practitioner) was surveyed regarding existing and anticipated organ dysfunction. The primary outcomes were clinicians' accuracy at identifying multiple organ dysfunction syndrome and predicting new or progressive multiple organ dysfunction syndrome, compared to the objective assessment of multiple organ dysfunction syndrome using Proulx criteria. We also measured sensitivity, specificity, negative and positive predictive values, and negative and positive likelihood ratios of clinician assessments. We tested for differences in accuracy by clinician type using chi-square tests. Clinicians rated their confidence in prediction on a 5-point Likert scale. There were 476 eligible PICU admissions, for whom 1,218 surveys were completed. Multiple organ dysfunction syndrome was present in 89 patients (18.7%) at enrollment, and new or progressive multiple organ dysfunction syndrome occurred in 39 (8.2%). Clinicians correctly identified multiple organ dysfunction syndrome with 79.9% accuracy and predicted additional organ dysfunction with 82.6% accuracy. However, the positive and negative likelihood ratios for new or progressive multiple organ dysfunction syndrome prediction were 3.0 and 0.7, respectively, indicating a weak relationship between the clinician prediction and development of new or progressive multiple organ dysfunction syndrome. The positive predictive value of new or progressive multiple organ dysfunction syndrome prediction was just 22.1%. We found no differences in accuracy by clinician type for either identification of multiple organ dysfunction syndrome (80.2% vs 78.2% vs 81.0%; p = 0.57) or prediction of new or progressive multiple organ dysfunction syndrome (84.8% vs 82.8% vs 80.3%; p = 0.26) for attendings, fellows, and residents/nurse practitioners, respectively. There was a weak correlation between the confidence and accuracy of prediction (pairwise correlation coefficient, 0.26; p < 0.001). CONCLUSIONS: PICU clinicians correctly identified multiple organ dysfunction syndrome and predicted new or progressive multiple organ dysfunction syndrome with 80% accuracy. However, only 8% of patients developed new or progressive multiple organ dysfunction syndrome, so accuracy was largely due to true negative predictions. The positive predictive value for new or progressive multiple organ dysfunction syndrome prediction was just 22%. Accuracy did not differ by clinician type, but was correlated with self-rated confidence and was higher for negative predictions.
OBJECTIVES: To determine clinician accuracy in the identification and prediction of multiple organ dysfunction syndrome. DESIGN: Prospective cohort study. SETTING: University of Michigan's C.S. Mott Children's Hospital PICU. PATIENTS: Patients admitted to the PICU with an anticipated PICU length of stay greater than 48 hours. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: For each patient, the clinical team (attending, fellow, resident/nurse practitioner) was surveyed regarding existing and anticipated organ dysfunction. The primary outcomes were clinicians' accuracy at identifying multiple organ dysfunction syndrome and predicting new or progressive multiple organ dysfunction syndrome, compared to the objective assessment of multiple organ dysfunction syndrome using Proulx criteria. We also measured sensitivity, specificity, negative and positive predictive values, and negative and positive likelihood ratios of clinician assessments. We tested for differences in accuracy by clinician type using chi-square tests. Clinicians rated their confidence in prediction on a 5-point Likert scale. There were 476 eligible PICU admissions, for whom 1,218 surveys were completed. Multiple organ dysfunction syndrome was present in 89 patients (18.7%) at enrollment, and new or progressive multiple organ dysfunction syndrome occurred in 39 (8.2%). Clinicians correctly identified multiple organ dysfunction syndrome with 79.9% accuracy and predicted additional organ dysfunction with 82.6% accuracy. However, the positive and negative likelihood ratios for new or progressive multiple organ dysfunction syndrome prediction were 3.0 and 0.7, respectively, indicating a weak relationship between the clinician prediction and development of new or progressive multiple organ dysfunction syndrome. The positive predictive value of new or progressive multiple organ dysfunction syndrome prediction was just 22.1%. We found no differences in accuracy by clinician type for either identification of multiple organ dysfunction syndrome (80.2% vs 78.2% vs 81.0%; p = 0.57) or prediction of new or progressive multiple organ dysfunction syndrome (84.8% vs 82.8% vs 80.3%; p = 0.26) for attendings, fellows, and residents/nurse practitioners, respectively. There was a weak correlation between the confidence and accuracy of prediction (pairwise correlation coefficient, 0.26; p < 0.001). CONCLUSIONS: PICU clinicians correctly identified multiple organ dysfunction syndrome and predicted new or progressive multiple organ dysfunction syndrome with 80% accuracy. However, only 8% of patients developed new or progressive multiple organ dysfunction syndrome, so accuracy was largely due to true negative predictions. The positive predictive value for new or progressive multiple organ dysfunction syndrome prediction was just 22%. Accuracy did not differ by clinician type, but was correlated with self-rated confidence and was higher for negative predictions.
Authors: Francis Leclerc; Stéphane Leteurtre; Alain Duhamel; Bruno Grandbastien; François Proulx; Alain Martinot; France Gauvin; Philippe Hubert; Jacques Lacroix Journal: Am J Respir Crit Care Med Date: 2004-10-29 Impact factor: 21.405
Authors: Vineet Chopra; Sushant Govindan; Latoya Kuhn; David Ratz; Randy F Sweis; Natalie Melin; Rachel Thompson; Aaron Tolan; James Barron; Sanjay Saint Journal: Ann Intern Med Date: 2014-10-21 Impact factor: 25.391
Authors: Katri V Typpo; Nancy J Petersen; D Michael Hallman; Barry P Markovitz; M Michele Mariscalco Journal: Pediatr Crit Care Med Date: 2009-09 Impact factor: 3.624
Authors: Philip C Spinella; Marisa Tucci; Dean A Fergusson; Jacques Lacroix; Paul C Hébert; Stéphane Leteurtre; Kenneth B Schechtman; Allan Doctor; Robert A Berg; Tina Bockelmann; J Jaime Caro; Fabrizio Chiusolo; Lucy Clayton; Jill M Cholette; Gonzalo Garcia Guerra; Cassandra D Josephson; Kusum Menon; Jennifer A Muszynski; Marianne E Nellis; Amrita Sarpal; Stephanie Schafer; Marie E Steiner; Alexis F Turgeon Journal: JAMA Date: 2019-12-10 Impact factor: 56.272
Authors: Michael E Detsky; Michael O Harhay; Dominique F Bayard; Aaron M Delman; Anna E Buehler; Saida A Kent; Isabella V Ciuffetelli; Elizabeth Cooney; Nicole B Gabler; Sarah J Ratcliffe; Mark E Mikkelsen; Scott D Halpern Journal: JAMA Date: 2017-06-06 Impact factor: 56.272
Authors: Nawar Bestati; Stéphane Leteurtre; Alain Duhamel; François Proulx; Bruno Grandbastien; Jacques Lacroix; Francis Leclerc Journal: Crit Care Date: 2010-11-09 Impact factor: 9.097