David Sheyn1, Mingxuan Ju, Sixiao Zhang, Caleb Anyaeche, Adonis Hijaz, Jeffrey Mangel, Sangeeta Mahajan, Britt Conroy, Sherif El-Nashar, Soumya Ray. 1. Division of Female Pelvic Medicine and Reconstructive Surgery, Department of Urology, University Hospitals Cleveland Medical Center, the Case School of Engineering, Department of Electrical Engineering and Computer Science, Case Western Reserve University, the Division of Female Pelvic Medicine and Reconstructive Surgery, Department of Obstetrics and Gynecology, University Hospitals Cleveland Medical Center, and the Division of Female Pelvic Medicine and Reconstructive Surgery, MetroHealth Medical Center, Case Western Reserve University School of Medicine, Cleveland, Ohio.
Abstract
OBJECTIVE: To develop and externally validate a prediction model for anticholinergic response in patients with overactive bladder (OAB). METHODS: A machine learning model to predict the likelihood of anticholinergic treatment failure was constructed using a retrospective data set (n=559) of female patients with OAB who were treated with anticholinergic medications between January 2010 and December 2017. Treatment failure was defined as less than 50% improvement in frequency, urgency, incontinence episodes, and nocturia, and the patient's subjective impression of symptomatic relief. Patients were stratified by age (younger than 40 years, 40-60 years, and older than 60 years), and number of previously failed medications. K-fold stratified cross-validation was performed on each stratum using machine learning algorithms. Of these, the random forest model was the most accurate. This model was refined using internal cross validation within each stratum. The area under the curve (AUC) was calculated for each stratum and used to identify the optimal operating points for prediction of treatment failure. The random forest model was then externally validated using a prospectively collected data set (n=82) of women treated with anticholinergic medications at a different clinical site between January 2018 and December 2018. RESULTS: The global accuracy of the final model was 80.3% (95% CI 79.1-81.3), and the AUC was 0.77 (95% CI 0.74-0.79). Using the external validation data set, the model's sensitivity and specificity was 80.4% (95% CI 66.5-89.7%) and 77.4% (95% CI 58.6-89.7%), respectively. The model performed best in women aged younger than 40 years (AUC 0.84, 95% CI 0.81-0.84) and worst in women aged older than 60 years who had previously failed medication (AUC 0.71, 95% CI 0.67-0.75). CONCLUSION: Our externally validated machine learning prediction model can predict anticholinergic treatment failure during the standard 3-month treatment trial period with greater than 80% accuracy. The model can be accessed at https://oabweb.herokuapp.com/app/pre/.
OBJECTIVE: To develop and externally validate a prediction model for anticholinergic response in patients with overactive bladder (OAB). METHODS: A machine learning model to predict the likelihood of anticholinergic treatment failure was constructed using a retrospective data set (n=559) of female patients with OAB who were treated with anticholinergic medications between January 2010 and December 2017. Treatment failure was defined as less than 50% improvement in frequency, urgency, incontinence episodes, and nocturia, and the patient's subjective impression of symptomatic relief. Patients were stratified by age (younger than 40 years, 40-60 years, and older than 60 years), and number of previously failed medications. K-fold stratified cross-validation was performed on each stratum using machine learning algorithms. Of these, the random forest model was the most accurate. This model was refined using internal cross validation within each stratum. The area under the curve (AUC) was calculated for each stratum and used to identify the optimal operating points for prediction of treatment failure. The random forest model was then externally validated using a prospectively collected data set (n=82) of women treated with anticholinergic medications at a different clinical site between January 2018 and December 2018. RESULTS: The global accuracy of the final model was 80.3% (95% CI 79.1-81.3), and the AUC was 0.77 (95% CI 0.74-0.79). Using the external validation data set, the model's sensitivity and specificity was 80.4% (95% CI 66.5-89.7%) and 77.4% (95% CI 58.6-89.7%), respectively. The model performed best in women aged younger than 40 years (AUC 0.84, 95% CI 0.81-0.84) and worst in women aged older than 60 years who had previously failed medication (AUC 0.71, 95% CI 0.67-0.75). CONCLUSION: Our externally validated machine learning prediction model can predict anticholinergic treatment failure during the standard 3-month treatment trial period with greater than 80% accuracy. The model can be accessed at https://oabweb.herokuapp.com/app/pre/.
Authors: Andrew B Chen; Taseen Haque; Sidney Roberts; Sirisha Rambhatla; Giovanni Cacciamani; Prokar Dasgupta; Andrew J Hung Journal: Urol Clin North Am Date: 2021-10-23 Impact factor: 2.766