OBJECTIVES: To propose a visual display-the probability threshold plot (PTP)-that transparently communicates a predictive models' measures of discriminative accuracy along the range of model-based predicted probabilities (Pt). MATERIALS AND METHODS: We illustrate the PTP by replicating a previously-published and validated machine learning-based model to predict antihyperglycemic medication cessation within 1-2 years following metabolic surgery. The visual characteristics of the PTPs for each model were compared to receiver operating characteristic (ROC) curves. RESULTS: A total of 18 887 patients were included for analysis. Whereas during testing each predictive model had nearly identical ROC curves and corresponding area under the curve values (0.672 and 0.673), the visual characteristics of the PTPs revealed substantive between-model differences in sensitivity, specificity, PPV, and NPV across the range of Pt. DISCUSSION AND CONCLUSIONS: The PTP provides improved visual display of a predictive model's discriminative accuracy, which can enhance the practical application of predictive models for medical decision making.
OBJECTIVES: To propose a visual display-the probability threshold plot (PTP)-that transparently communicates a predictive models' measures of discriminative accuracy along the range of model-based predicted probabilities (Pt). MATERIALS AND METHODS: We illustrate the PTP by replicating a previously-published and validated machine learning-based model to predict antihyperglycemic medication cessation within 1-2 years following metabolic surgery. The visual characteristics of the PTPs for each model were compared to receiver operating characteristic (ROC) curves. RESULTS: A total of 18 887 patients were included for analysis. Whereas during testing each predictive model had nearly identical ROC curves and corresponding area under the curve values (0.672 and 0.673), the visual characteristics of the PTPs revealed substantive between-model differences in sensitivity, specificity, PPV, and NPV across the range of Pt. DISCUSSION AND CONCLUSIONS: The PTP provides improved visual display of a predictive model's discriminative accuracy, which can enhance the practical application of predictive models for medical decision making.
Authors: David S Salkever; Stephen Johnston; Mustafa C Karakus; Nicholas S Ialongo; Eric P Slade; Elizabeth A Stuart Journal: Adm Policy Ment Health Date: 2008-03-15
Authors: Stephen S Johnston; John M Morton; Iftekhar Kalsekar; Eric M Ammann; Chia-Wen Hsiao; Jenna Reps Journal: Value Health Date: 2019-05 Impact factor: 5.725
Authors: Laure Wynants; Maarten van Smeden; David J McLernon; Dirk Timmerman; Ewout W Steyerberg; Ben Van Calster Journal: BMC Med Date: 2019-10-25 Impact factor: 8.775
Authors: David S Carrell; Ladia Albertson-Junkans; Arvind Ramaprasan; Grant Scull; Matt Mackwood; Eric Johnson; David J Cronkite; Andrew Baer; Kris Hansen; Carla A Green; Brian L Hazlehurst; Shannon L Janoff; Paul M Coplan; Angela DeVeaugh-Geiss; Carlos G Grijalva; Caihua Liang; Cheryl L Enger; Jane Lange; Susan M Shortreed; Michael Von Korff Journal: J Drug Assess Date: 2020-04-28