Juliane E Kämmer1,2, Wolf E Hautz3, Stefan M Herzog1, Olga Kunina-Habenicht4, Ralf H J M Kurvers1. 1. Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany (JEK, SMH, RHJMK). 2. AG Progress Test Medizin, Charité Medical School, Berlin, Germany (JEK). 3. Department of Emergency Medicine, Inselspital, University Hospital Bern, Bern, Switzerland (WEH). 4. German Institute for International Educational Research, Centre for International Student Assessment, Frankfurt am Main, Germany (OK-H).
Abstract
BACKGROUND: Evidence suggests that pooling multiple independent diagnoses can improve diagnostic accuracy in well-defined tasks. We investigated whether this is also the case for diagnostics in emergency medicine, an ill-defined task environment where diagnostic errors are rife. METHODS: A computer simulation study was conducted based on empirical data from 2 published experimental studies. In the computer experiments, 285 medical students independently diagnosed 6 simulated patients arriving at the emergency room with dyspnea. Participants' diagnoses (n = 1,710), confidence ratings, and expertise levels were entered into a computer simulation. Virtual groups of different sizes were randomly created, and 3 collective intelligence rules (follow-the-plurality rule, follow-the-most-confident rule, and follow-the-most-senior rule) were applied to combine the independent decisions into a final diagnosis. For different group sizes, the performance levels (i.e., percentage of correct diagnoses) of the 3 collective intelligence rules were compared with each other and against the average individual accuracy. RESULTS: For all collective intelligence rules, combining independent decisions substantially increased performance relative to average individual performance. For groups of 4 or fewer, the follow-the-most-confident rule outperformed the other rules; for larger groups, the follow-the-plurality rule performed best. For example, combining 5 independent decisions using the follow-the-plurality rule increased diagnostic accuracy by 22 percentage points. These results were robust across case difficulty and expertise level. Limitations of the study include the use of simulated patients diagnosed by medical students. Whether results generalize to clinical practice is currently unknown. CONCLUSION: Combining independent decisions may substantially improve the quality of diagnoses in emergency medicine and may thus enhance patient safety.
BACKGROUND: Evidence suggests that pooling multiple independent diagnoses can improve diagnostic accuracy in well-defined tasks. We investigated whether this is also the case for diagnostics in emergency medicine, an ill-defined task environment where diagnostic errors are rife. METHODS: A computer simulation study was conducted based on empirical data from 2 published experimental studies. In the computer experiments, 285 medical students independently diagnosed 6 simulated patients arriving at the emergency room with dyspnea. Participants' diagnoses (n = 1,710), confidence ratings, and expertise levels were entered into a computer simulation. Virtual groups of different sizes were randomly created, and 3 collective intelligence rules (follow-the-plurality rule, follow-the-most-confident rule, and follow-the-most-senior rule) were applied to combine the independent decisions into a final diagnosis. For different group sizes, the performance levels (i.e., percentage of correct diagnoses) of the 3 collective intelligence rules were compared with each other and against the average individual accuracy. RESULTS: For all collective intelligence rules, combining independent decisions substantially increased performance relative to average individual performance. For groups of 4 or fewer, the follow-the-most-confident rule outperformed the other rules; for larger groups, the follow-the-plurality rule performed best. For example, combining 5 independent decisions using the follow-the-plurality rule increased diagnostic accuracy by 22 percentage points. These results were robust across case difficulty and expertise level. Limitations of the study include the use of simulated patients diagnosed by medical students. Whether results generalize to clinical practice is currently unknown. CONCLUSION: Combining independent decisions may substantially improve the quality of diagnoses in emergency medicine and may thus enhance patient safety.
Entities:
Keywords:
collective intelligence; emergency medicine; follow-the-plurality rule; medical diagnostics; simulation; wisdom of crowds
Authors: Elaine C Khoong; Sarah S Nouri; Delphine S Tuot; Shantanu Nundy; Valy Fontil; Urmimala Sarkar Journal: Med Decis Making Date: 2021-08-11 Impact factor: 2.583
Authors: Cédric M Panje; Markus Glatzer; Joscha von Rappard; Christian Rothermundt; Thomas Hundsberger; Valentin Zumstein; Ludwig Plasswilm; Paul Martin Putora Journal: BMC Med Res Methodol Date: 2017-08-16 Impact factor: 4.615
Authors: Dorothea Eisenmann; Fabian Stroben; Jan D Gerken; Aristomenis K Exadaktylos; Mareen Machner; Wolf E Hautz Journal: West J Emerg Med Date: 2017-12-14
Authors: Ralf H J M Kurvers; Annemarie de Zoete; Shelby L Bachman; Paul R Algra; Raymond Ostelo Journal: PLoS One Date: 2018-04-03 Impact factor: 3.240
Authors: Andrea Tacchella; Silvia Romano; Michela Ferraldeschi; Marco Salvetti; Andrea Zaccaria; Andrea Crisanti; Francesca Grassi Journal: F1000Res Date: 2017-12-22