Chaan S Ng1, Christopher R Palmer. 1. Department of Radiology, University of Texas M.D. Anderson Cancer Center, Houston, TX 77030-4009, USA. cng@mdanderson.org
Abstract
BACKGROUND: Changes in diagnostic confidence are used as a measure of a test's efficacy. There are several methods for analyzing such data, but it is unclear which are most robust. PURPOSE: To compare analytical methods for assessing diagnostic confidence, applied to data from a prospective study of computed tomography (CT) for acute abdominal pain. MATERIAL AND METHODS: Changes in diagnostic confidence in an illustrative case study were evaluated using five methods: "Basic," "Retained diagnosis," "Omary," "Tsushima," and "Score-based." The case study was a randomized controlled trial of patients admitted to hospital for acute abdominal pain for whom immediate CT was not indicated, comparing "early" CT undertaken within 24 h of admission versus routine standard practice. Admitting surgeons recorded their diagnoses and confidences (5-point scale, 10-90%) both on admission and after 24 h. One- and two-sample t tests, and intention-to-treat and as-treated analyzes, were performed using all five analytic methods. RESULTS: In the case study, 118 patients were randomized to early CT (n=55) or standard practice (n=63). Mean (SD) diagnostic confidence increased between the two time points by 20.7 (25.2)% and 13.0 (24.9)%, respectively. Early CT showed increases in diagnostic confidence by all one-sample analyzes (all P<0.055). Compared with standard practice, early CT showed significant increases in diagnostic confidence on an a) as-treated basis when using Basic, Omary and Score-based analyzes (P<0.045), but not by the other two analytic methods, and b) intention-to-treat, only by Omary analysis. CONCLUSION: The method of analysis used to evaluate diagnostic confidence can influence conclusions about a test's efficacy. Methods incorporating the soundest analytical principles are recommended.
RCT Entities:
BACKGROUND: Changes in diagnostic confidence are used as a measure of a test's efficacy. There are several methods for analyzing such data, but it is unclear which are most robust. PURPOSE: To compare analytical methods for assessing diagnostic confidence, applied to data from a prospective study of computed tomography (CT) for acute abdominal pain. MATERIAL AND METHODS: Changes in diagnostic confidence in an illustrative case study were evaluated using five methods: "Basic," "Retained diagnosis," "Omary," "Tsushima," and "Score-based." The case study was a randomized controlled trial of patients admitted to hospital for acute abdominal pain for whom immediate CT was not indicated, comparing "early" CT undertaken within 24 h of admission versus routine standard practice. Admitting surgeons recorded their diagnoses and confidences (5-point scale, 10-90%) both on admission and after 24 h. One- and two-sample t tests, and intention-to-treat and as-treated analyzes, were performed using all five analytic methods. RESULTS: In the case study, 118 patients were randomized to early CT (n=55) or standard practice (n=63). Mean (SD) diagnostic confidence increased between the two time points by 20.7 (25.2)% and 13.0 (24.9)%, respectively. Early CT showed increases in diagnostic confidence by all one-sample analyzes (all P<0.055). Compared with standard practice, early CT showed significant increases in diagnostic confidence on an a) as-treated basis when using Basic, Omary and Score-based analyzes (P<0.045), but not by the other two analytic methods, and b) intention-to-treat, only by Omary analysis. CONCLUSION: The method of analysis used to evaluate diagnostic confidence can influence conclusions about a test's efficacy. Methods incorporating the soundest analytical principles are recommended.
Authors: Elaine C Khoong; Valy Fontil; Natalie A Rivadeneira; Mekhala Hoskote; Shantanu Nundy; Courtney R Lyles; Urmimala Sarkar Journal: J Am Med Inform Assoc Date: 2021-03-01 Impact factor: 4.497
Authors: Rossano Girometti; Francesco Fabris; Andrea Sgarro; Gloria Zanella; Serena Pullini; Lorenzo Cereser; Giuseppe Como; Chiara Zuiani; Massimo Bazzocchi Journal: Comput Math Methods Med Date: 2014-01-27 Impact factor: 2.238