PURPOSE: When clinicians contemplate the use of a new predictive technology in their practice, such as a nomogram, there is always a question of whether the new test is beneficial to their own clinical population. Unfortunately, traditional validation methods require a large number of subjects for validation testing and delay the decision-making process. We present an efficient and easy-to-use method based on the concept of sequential data analysis. PATIENTS AND METHODS: We illustrate with an example determining the validity of a technology for predicting Gleason score upgrading from biopsy to postprostatectomy (the Chun nomogram) in a clinical population different from the one used to initially validate the technology. Clinical data required by the Chun nomogram were available from 201 patients from the Cooperative Prostate Cancer Tissue Resource. RESULTS: Of 124 patients predicted by the Chun nomogram to have an upgrading event, 47 actually did. The positive predictive value (PPV) of the model was therefore 38% and significantly (P < .05) less than the value of 80% which we considered to be the smallest clinically useful PPV in this situation. Had the sequential methods introduced in this article been employed prospectively in this cohort, the same conclusion would have been reached using data from only the first 15 patients. CONCLUSION: In-clinic validation of predictive technologies will help the clinician adopt truly beneficial technologies and avoid the adoption of technologies which provide no significant benefit to their local patient population. For this task, sequential methods offer clear advantages.
PURPOSE: When clinicians contemplate the use of a new predictive technology in their practice, such as a nomogram, there is always a question of whether the new test is beneficial to their own clinical population. Unfortunately, traditional validation methods require a large number of subjects for validation testing and delay the decision-making process. We present an efficient and easy-to-use method based on the concept of sequential data analysis. PATIENTS AND METHODS: We illustrate with an example determining the validity of a technology for predicting Gleason score upgrading from biopsy to postprostatectomy (the Chun nomogram) in a clinical population different from the one used to initially validate the technology. Clinical data required by the Chun nomogram were available from 201 patients from the Cooperative Prostate Cancer Tissue Resource. RESULTS: Of 124 patients predicted by the Chun nomogram to have an upgrading event, 47 actually did. The positive predictive value (PPV) of the model was therefore 38% and significantly (P < .05) less than the value of 80% which we considered to be the smallest clinically useful PPV in this situation. Had the sequential methods introduced in this article been employed prospectively in this cohort, the same conclusion would have been reached using data from only the first 15 patients. CONCLUSION: In-clinic validation of predictive technologies will help the clinician adopt truly beneficial technologies and avoid the adoption of technologies which provide no significant benefit to their local patient population. For this task, sequential methods offer clear advantages.
Authors: Sally Stabler; Tatsuki Koyama; Zhiguo Zhao; Magaly Martinez-Ferrer; Robert H Allen; Zigmund Luka; Lioudmila V Loukachevitch; Peter E Clark; Conrad Wagner; Neil A Bhowmick Journal: PLoS One Date: 2011-08-10 Impact factor: 3.240