Elaine C Khoong1,2, Sarah S Nouri3, Delphine S Tuot2,4,5, Shantanu Nundy6,7, Valy Fontil1,2, Urmimala Sarkar1,2. 1. Division of General Internal Medicine at Zuckerberg San Francisco General Hospital, Department of Medicine, UCSF, San Francisco, CA, USA. 2. Center for Vulnerable Populations at Zuckerberg San Francisco General Hospital, UCSF, San Francisco, CA,USA. 3. Division of General Internal Medicine, Department of Medicine, UCSF, San Francisco, CA, USA. 4. Division of Nephrology, Department of Medicine, UCSF, San Francisco, CA, USA. 5. Center for Innovation in Access and Quality at Zuckerberg San Francisco General Hospital, UCSF, San Francisco, CA, USA. 6. George Washington University Milken Institute School of Public Health, Washington, DC, USA. 7. Accolade, Inc, Plymouth Meeting, PA.
Abstract
BACKGROUND: Studies report higher diagnostic accuracy using the collective intelligence (CI) of multiple clinicians compared with individual clinicians. However, the diagnostic process is iterative, and unexplored is the value of CI in improving clinical recommendations leading to a final diagnosis. METHODS: To compare the appropriateness of diagnostic recommendations advised by individual physicians versus the CI of physicians, we entered actual consultation requests sent by primary care physicians to specialists onto a web-based CI platform capable of collecting diagnostic recommendations (next steps for care) from multiple physicians. We solicited responses to 35 cases (12 endocrinology, 13 gynecology, 10 neurology) from ≥3 physicians of any specialty through the CI platform, which aggregated responses into a CI output. The primary outcome was the appropriateness of individual physician recommendations versus the CI output recommendations, using recommendations agreed upon by 2 specialists in the same specialty as a gold standard. The secondary outcome was the recommendations' potential for harm. RESULTS: A total of 177 physicians responded. Cases had a median of 7 respondents (interquartile range: 5-10). Diagnostic recommendations in the CI output achieved higher levels of appropriateness (69%) than recommendations from individual physicians (45%; χ2 = 5.95, P = 0.015). Of the CI recommendations, 54% were potentially harmful, as compared with 41% of individuals' recommendations (χ2 = 2.49, P = 0.11). LIMITATIONS: Cases were from a single institution. CI was solicited using a single algorithm/platform. CONCLUSIONS: When seeking specialist guidance, diagnostic recommendations from the CI of multiple physicians are more appropriate than recommendations from most individual physicians, measured against specialist recommendations. Although CI provides useful recommendations, some have potential for harm. Future research should explore how to use CI to improve diagnosis while limiting harm from inappropriate tests/therapies.
BACKGROUND: Studies report higher diagnostic accuracy using the collective intelligence (CI) of multiple clinicians compared with individual clinicians. However, the diagnostic process is iterative, and unexplored is the value of CI in improving clinical recommendations leading to a final diagnosis. METHODS: To compare the appropriateness of diagnostic recommendations advised by individual physicians versus the CI of physicians, we entered actual consultation requests sent by primary care physicians to specialists onto a web-based CI platform capable of collecting diagnostic recommendations (next steps for care) from multiple physicians. We solicited responses to 35 cases (12 endocrinology, 13 gynecology, 10 neurology) from ≥3 physicians of any specialty through the CI platform, which aggregated responses into a CI output. The primary outcome was the appropriateness of individual physician recommendations versus the CI output recommendations, using recommendations agreed upon by 2 specialists in the same specialty as a gold standard. The secondary outcome was the recommendations' potential for harm. RESULTS: A total of 177 physicians responded. Cases had a median of 7 respondents (interquartile range: 5-10). Diagnostic recommendations in the CI output achieved higher levels of appropriateness (69%) than recommendations from individual physicians (45%; χ2 = 5.95, P = 0.015). Of the CI recommendations, 54% were potentially harmful, as compared with 41% of individuals' recommendations (χ2 = 2.49, P = 0.11). LIMITATIONS: Cases were from a single institution. CI was solicited using a single algorithm/platform. CONCLUSIONS: When seeking specialist guidance, diagnostic recommendations from the CI of multiple physicians are more appropriate than recommendations from most individual physicians, measured against specialist recommendations. Although CI provides useful recommendations, some have potential for harm. Future research should explore how to use CI to improve diagnosis while limiting harm from inappropriate tests/therapies.
Entities:
Keywords:
collective intelligence; diagnosis; diagnostic errors; health information technology
Authors: Tejal K Gandhi; Allen Kachalia; Eric J Thomas; Ann Louise Puopolo; Catherine Yoon; Troyen A Brennan; David M Studdert Journal: Ann Intern Med Date: 2006-10-03 Impact factor: 25.391
Authors: Olga Kostopoulou; Andrea Rosen; Thomas Round; Ellen Wright; Abdel Douiri; Brendan Delaney Journal: Br J Gen Pract Date: 2015-01 Impact factor: 5.386
Authors: Reed T Sutton; David Pincock; Daniel C Baumgart; Daniel C Sadowski; Richard N Fedorak; Karen I Kroeker Journal: NPJ Digit Med Date: 2020-02-06