Can Machine-learning Algorithms Predict Early Revision TKA in the Danish Knee Arthroplasty Registry?

BACKGROUND: Revision TKA is a serious adverse event with substantial consequences for the patient. As the demand for TKA rises, reducing the risk of revision TKA is becoming increasingly important. Predictive tools based on machine-learning algorithms could reform clinical practice. Few attempts have been made to combine machine-learning algorithms with data from nationwide arthroplasty registries and, to the authors' knowledge, none have tried to predict the likelihood of early revision TKA. QUESTION/PURPOSES: We used the Danish Knee Arthroplasty Registry to build models to predict the likelihood of revision TKA within 2 years of primary TKA and asked: (1) Which preoperative factors were the most important features behind these models' predictions of revision? (2) Can a clinically meaningful model be built on the preoperative factors included in the Danish Knee Arthroplasty Registry?
METHODS: The Danish Knee Arthroplasty Registry collects patients' characteristics and surgical information from all arthroplasties conducted in Denmark and thus provides a large nationwide cohort of patients undergoing TKA. As training dataset, we retrieved all preoperative variables of 25,104 primary TKAs from 2012 to 2015. The same variables were retrieved from 6170 TKAs conducted in 2016, which were used as a hold-out year for temporal external validation. If a patient received bilateral TKA, only the first knee to receive surgery was included. All patients were followed for 2 years, with removal, exchange, or addition of an implant defined as TKA revision. We created four different predictive models to find the best performing model, including a regression-based model using logistic regression with least shrinkage and selection operator (LASSO), two classification tree models (random forest and gradient boosting model) and a supervised neural network. For comparison, we created a noninformative model predicting that all observations were unrevised. The four machine learning models were trained using 10-fold cross-validation on the training dataset after adjusting for the low percentage of revisions by over-sampling revised observations and undersampling unrevised observations. In the validation dataset, the models' performance was evaluated and compared by density plot, calibration plot, accuracy, Brier score, receiver operator characteristic (ROC) curve and area under the curve (AUC). The density plot depicts the distribution of probabilities and the calibration plot graphically depicts whether the predicted probability resembled the observed probability. The accuracy indicates how often the models' predictions were correct and the Brier score is the mean distance from the predicted probability to the observed outcome. The ROC curve is a graphical output of the models' sensitivity and specificity from which the AUC is calculated. The AUC can be interpreted as the likelihood that a model correctly classified an observation and thus, a priori, an AUC of 0.7 was chosen as threshold for a clinically meaningful model.
RESULTS: Based the model training, age, postfracture osteoarthritis and weight were deemed as important preoperative factors within the machine learning models. During validation, the models' performance was not different from the noninformative models, and with AUCs ranging from 0.57 to 0.60, no models reached the predetermined AUC threshold for a clinical useful discriminative capacity.
CONCLUSION: Although several well-known presurgical risk factors for revision were coupled with four different machine learning methods, we could not develop a clinically useful model capable of predicting early TKA revisions in the Danish Knee Arthroplasty Registry based on preoperative data. CLINICAL RELEVANCE: The inability to predict early TKA revision highlights that predicting revision based on preoperative information alone is difficult. Future models might benefit from including medical comorbidities and an anonymous surgeon identifier variable or may attempt to build a postoperative predictive model including intra- and postoperative factors as these may have a stronger association with early TKA revisions.