David L Schriger1, Joshua W Elder2, Richelle J Cooper3. 1. Department of Emergency Medicine, University of California, Los Angeles, CA. Electronic address: schriger@ucla.edu. 2. Robert Wood Johnson Clinical Scholars Program, Yale University School of Medicine, New Haven, CT. 3. Department of Emergency Medicine, University of California, Los Angeles, CA.
Abstract
STUDY OBJECTIVE: We determine how often studies that evaluate the performance of an aid for decisionmaking, be it a simple laboratory or imaging test or a complex multielement decision instrument, compare the aid's performance to independent, unaided physician judgment. METHODS: This was a cross-sectional survey of all Original Research and Brief Research Report articles in Annals of Emergency Medicine from 1998 to 2015. We included all articles that evaluated the performance of an aid for decisionmaking in assisting a physician with a decision about testing, treatment, diagnosis, or disposition. Two authors independently characterized the intent and purpose of each aid for decisionmaking, determined whether each study had a comparison to unaided physician judgment within the article or in a separate article, and recorded the result of that comparison. RESULTS: One hundred seventy-one (8.3%) of 2,060 research articles studied the performance characteristics of an aid for decisionmaking, 48 of which were formal clinical decision instruments. Forty of the 171 studies retrospectively analyzed existing databases and therefore could not assess physician judgment. Investigators compared the aid for decisionmaking to physician judgment in 11% (15/131) of the prospective studies, including 15% (6/41) of studies that evaluated a formal clinical decision instrument. For 9 articles that had no comparison to physician judgment, we found 6 unique external publications that compared that aid to physician clinical judgment. The decision aid was superior to clinical judgment in 2 of the 21 studies that contained a comparison. CONCLUSION: Physician judgment is infrequently assessed when the performance of an aid for decisionmaking is evaluated, and, when reported, the decision aid seldom outperformed physician judgment.
STUDY OBJECTIVE: We determine how often studies that evaluate the performance of an aid for decisionmaking, be it a simple laboratory or imaging test or a complex multielement decision instrument, compare the aid's performance to independent, unaided physician judgment. METHODS: This was a cross-sectional survey of all Original Research and Brief Research Report articles in Annals of Emergency Medicine from 1998 to 2015. We included all articles that evaluated the performance of an aid for decisionmaking in assisting a physician with a decision about testing, treatment, diagnosis, or disposition. Two authors independently characterized the intent and purpose of each aid for decisionmaking, determined whether each study had a comparison to unaided physician judgment within the article or in a separate article, and recorded the result of that comparison. RESULTS: One hundred seventy-one (8.3%) of 2,060 research articles studied the performance characteristics of an aid for decisionmaking, 48 of which were formal clinical decision instruments. Forty of the 171 studies retrospectively analyzed existing databases and therefore could not assess physician judgment. Investigators compared the aid for decisionmaking to physician judgment in 11% (15/131) of the prospective studies, including 15% (6/41) of studies that evaluated a formal clinical decision instrument. For 9 articles that had no comparison to physician judgment, we found 6 unique external publications that compared that aid to physician clinical judgment. The decision aid was superior to clinical judgment in 2 of the 21 studies that contained a comparison. CONCLUSION: Physician judgment is infrequently assessed when the performance of an aid for decisionmaking is evaluated, and, when reported, the decision aid seldom outperformed physician judgment.
Authors: Julie C Leonard; Lorin R Browne; Fahd A Ahmad; Hamilton Schwartz; Michael Wallendorf; Jeffrey R Leonard; E Brooke Lerner; Nathan Kuppermann Journal: Pediatrics Date: 2019-07 Impact factor: 7.124
Authors: Angel M R Schols; Robert T A Willemsen; Tobias N Bonten; Martijn H Rutten; Patricia M Stassen; Bas L J H Kietselaer; Geert-Jan Dinant; Jochen W L Cals Journal: Ann Fam Med Date: 2019-07 Impact factor: 5.166
Authors: Laura E Simon; Mamata V Kene; E Margaret Warton; Adina S Rauchwerger; David R Vinson; Mary E Reed; Uli K Chettipally; Dustin G Mark; Dana R Sax; D Ian McLachlan; Dale M Cotton; James S Lin; Gabriela Vazquez-Benitez; Anupam B Kharbanda; Elyse O Kharbanda; Dustin W Ballard Journal: Acad Emerg Med Date: 2020-04-02 Impact factor: 3.451
Authors: Yuval Barak-Corren; Isha Agarwal; Kenneth A Michelson; Todd W Lyons; Mark I Neuman; Susan C Lipsett; Amir A Kimia; Matthew A Eisenberg; Andrew J Capraro; Jason A Levy; Joel D Hudgins; Ben Y Reis; Andrew M Fine Journal: J Am Med Inform Assoc Date: 2021-07-30 Impact factor: 4.497
Authors: George Glass; Thomas R Hartka; Jessica Keim-Malpass; Kyle B Enfield; Matthew T Clark Journal: J Clin Monit Comput Date: 2020-03-19 Impact factor: 2.502