Siyuan Zhang1, Bernard Puang Huh Lau2, Yau Hong Ng2, Xinyu Wang2, Weiliang Chua3. 1. Yong Loo Lin School of Medicine, National University of Singapore, 10 Medical Drive, Singapore, 117597, Singapore. 2. Department of Orthopaedic Surgery, National University Hospital, Level 11, NUHS Tower Block. 1E Kent Ridge Road, Singapore, 119228, Singapore. 3. Department of Orthopaedic Surgery, National University Hospital, Level 11, NUHS Tower Block. 1E Kent Ridge Road, Singapore, 119228, Singapore. wei_liang_chua@nuhs.edu.sg.
Abstract
PURPOSE: Patient-reported outcome measures (PROMs) are important measures of success after total knee arthroplasty (TKA) and being able to predict their improvements could enhance preoperative decision-making. Our study aims to compare the predictive performance of machine learning (ML) algorithms and preoperative PROM thresholds in predicting minimal clinically important difference (MCID) attainment at 2 years after TKA. METHODS: Prospectively collected data of 2840 primary TKA performed between 2008 and 2018 was extracted from our joint replacement registry and split into a training set (80%) and test set (20%). Using the training set, ML algorithms were developed using patient demographics, comorbidities and preoperative PROMs, whereas the optimal preoperative threshold was determined using ROC analysis. Both methods were used to predict MCID attainment for the SF-36 PCS, MCS and WOMAC at 2 years postoperatively, with predictive performance evaluated on the independent test set. RESULTS: ML algorithms and preoperative PROM models performed similarly in predicting MCID for the SF-36 PCS (AUC: 0.77 vs 0.74), MCS (AUC: 0.95 vs 0.95) and WOMAC (AUC: 0.89 vs 0.88). For each outcome, the most important predictor of MCID attainment was the patient's preoperative PROM score. ROC analysis also identified optimal preoperative threshold values of 33.6, 54.1 and 72.7 for the SF-36 PCS, MCS and WOMAC, respectively. CONCLUSION: ML algorithms did not perform significantly better than preoperative PROM thresholds in predicting MCID attainment after TKA. Future research should routinely compare the predictive ability of ML algorithms with existing methods and determine the type of clinical problems which may benefit the most from it. LEVEL OF EVIDENCE: II.
PURPOSE: Patient-reported outcome measures (PROMs) are important measures of success after total knee arthroplasty (TKA) and being able to predict their improvements could enhance preoperative decision-making. Our study aims to compare the predictive performance of machine learning (ML) algorithms and preoperative PROM thresholds in predicting minimal clinically important difference (MCID) attainment at 2 years after TKA. METHODS: Prospectively collected data of 2840 primary TKA performed between 2008 and 2018 was extracted from our joint replacement registry and split into a training set (80%) and test set (20%). Using the training set, ML algorithms were developed using patient demographics, comorbidities and preoperative PROMs, whereas the optimal preoperative threshold was determined using ROC analysis. Both methods were used to predict MCID attainment for the SF-36 PCS, MCS and WOMAC at 2 years postoperatively, with predictive performance evaluated on the independent test set. RESULTS: ML algorithms and preoperative PROM models performed similarly in predicting MCID for the SF-36 PCS (AUC: 0.77 vs 0.74), MCS (AUC: 0.95 vs 0.95) and WOMAC (AUC: 0.89 vs 0.88). For each outcome, the most important predictor of MCID attainment was the patient's preoperative PROM score. ROC analysis also identified optimal preoperative threshold values of 33.6, 54.1 and 72.7 for the SF-36 PCS, MCS and WOMAC, respectively. CONCLUSION: ML algorithms did not perform significantly better than preoperative PROM thresholds in predicting MCID attainment after TKA. Future research should routinely compare the predictive ability of ML algorithms with existing methods and determine the type of clinical problems which may benefit the most from it. LEVEL OF EVIDENCE: II.