RATIONALE: Management of pulmonary nodules depends critically on the probability of malignancy. Models to estimate probability have been developed and validated, but most clinicians rely on judgment. OBJECTIVES: The aim of this study was to compare the accuracy of clinical judgment with that of two prediction models. METHODS: Physician participants reviewed up to five clinical vignettes, selected at random from a larger pool of 35 vignettes, all based on actual patients with lung nodules of known final diagnosis. Vignettes included clinical information and a representative slice from computed tomography. Clinicians estimated the probability of malignancy for each vignette. To examine agreement with models, we calculated intraclass correlation coefficients (ICC) and kappa statistics. To examine accuracy, we compared areas under the receiver operator characteristic curve (AUC). MEASUREMENTS AND MAIN RESULTS: Thirty-six participants completed 179 vignettes, 47% of which described patients with malignant nodules. Agreement between participants and models was fair for the Mayo Clinic model (ICC, 0.37; 95% confidence interval [CI], 0.23-0.50) and moderate for the Veterans Affairs model (ICC, 0.46; 95% CI, 0.34-0.57). There was no difference in accuracy between participants (AUC, 0.70; 95% CI, 0.62-0.77) and the Mayo Clinic model (AUC, 0.71; 95% CI, 0.62-0.80; P = 0.90) or the Veterans Affairs model (AUC, 0.72; 95% CI, 0.64-0.80; P = 0.54). CONCLUSIONS: In this vignette-based study, clinical judgment and models appeared to have similar accuracy for lung nodule characterization, but agreement between judgment and the models was modest, suggesting that qualitative and quantitative approaches may provide complementary information.
RATIONALE: Management of pulmonary nodules depends critically on the probability of malignancy. Models to estimate probability have been developed and validated, but most clinicians rely on judgment. OBJECTIVES: The aim of this study was to compare the accuracy of clinical judgment with that of two prediction models. METHODS: Physician participants reviewed up to five clinical vignettes, selected at random from a larger pool of 35 vignettes, all based on actual patients with lung nodules of known final diagnosis. Vignettes included clinical information and a representative slice from computed tomography. Clinicians estimated the probability of malignancy for each vignette. To examine agreement with models, we calculated intraclass correlation coefficients (ICC) and kappa statistics. To examine accuracy, we compared areas under the receiver operator characteristic curve (AUC). MEASUREMENTS AND MAIN RESULTS: Thirty-six participants completed 179 vignettes, 47% of which described patients with malignant nodules. Agreement between participants and models was fair for the Mayo Clinic model (ICC, 0.37; 95% confidence interval [CI], 0.23-0.50) and moderate for the Veterans Affairs model (ICC, 0.46; 95% CI, 0.34-0.57). There was no difference in accuracy between participants (AUC, 0.70; 95% CI, 0.62-0.77) and the Mayo Clinic model (AUC, 0.71; 95% CI, 0.62-0.80; P = 0.90) or the Veterans Affairs model (AUC, 0.72; 95% CI, 0.64-0.80; P = 0.54). CONCLUSIONS: In this vignette-based study, clinical judgment and models appeared to have similar accuracy for lung nodule characterization, but agreement between judgment and the models was modest, suggesting that qualitative and quantitative approaches may provide complementary information.
Authors: S J Swensen; M D Silverstein; E S Edell; V F Trastek; G L Aughenbaugh; D M Ilstrup; C D Schleck Journal: Mayo Clin Proc Date: 1999-04 Impact factor: 7.616
Authors: Gerarda J Herder; Harm van Tinteren; Richard P Golding; Piet J Kostense; Emile F Comans; Egbert F Smit; Otto S Hoekstra Journal: Chest Date: 2005-10 Impact factor: 9.410
Authors: Christopher G Slatore; Nanda Horeweg; James R Jett; David E Midthun; Charles A Powell; Renda Soylemez Wiener; Juan P Wisnivesky; Michael K Gould Journal: Am J Respir Crit Care Med Date: 2015-08-15 Impact factor: 21.405
Authors: Nichole T Tanner; Alexander Porter; Michael K Gould; Xiao-Jun Li; Anil Vachani; Gerard A Silvestri Journal: Chest Date: 2017-01-20 Impact factor: 9.410
Authors: Roger Y Kim; Jason L Oke; Lyndsey C Pickup; Reginald F Munden; Travis L Dotson; Christina R Bellinger; Avi Cohen; Michael J Simoff; Pierre P Massion; Claire Filippini; Fergus V Gleeson; Anil Vachani Journal: Radiology Date: 2022-05-24 Impact factor: 29.146
Authors: Cesar D Lopez; Jessica Ding; Joel R Peterson; Rifat Ahmed; John T Heffernan; Mario H Lobao; Charles M Jobin; William N Levine Journal: J Shoulder Elb Arthroplast Date: 2022-04-13