Developing Machine Learning Algorithms to Support Patient-centered, Value-based Carpal Tunnel Decompression Surgery.

Sir:

We read with great interest the article entitled “Developing Machine Learning Algorithms to Support Patient-centered, Value-based Carpal Tunnel Decompression Surgery.”[1] The authors describe the application of machine learning (ML) models in the prediction of functional and symptomatic improvement following carpal tunnel decompression (CTD) surgery. We commend the authors on the originality of this article, as this is one of the first ever reported applications of this technology in hand surgery.[2] However, the methodology of creation of the machine learning algorithm, and the clinical applicability presented in the discussion is questionable.

From a technological standpoint, it is unclear why some of the predictors were chosen to train the described ML algorithms, leading to “black-box” medical decisions. To-date, there is no evidence that factors such as anemia, stomach ulcers, lung disease, or backpain affect the postoperative outcome of CTD. However, these are utilized as predictors in the described ML models and are given significant weight to predict CTD surgery outcomes. This contributes to what is known as “black-box” medical decision-making, meaning there is no justification or explanation as to why a set of seemingly unrelated factors influence the algorithms decision-making. In healthcare, there is very low acceptance to utilizing algorithms with “black-box” decisions to treat patients, due to a lack of trust in the absence of utilization evidence-based practices.[3,4] To overcome barriers of introduction of this technology to clinical practice, it is pivotal to make ML interpretable, by choosing to incorporate key predictors that relate to the outcome in hand and allowing the algorithm to provide justification as to why the outcome is chosen.

During the creation of the novel ML algorithms proposed in this article, the authors describe the use of training set (80%) and test set (20%). This fundamentally lacks the utilization of a validation set.[2] When creating an ML model, the authors must split the total dataset into three subsets: training, validation, and testing sets. After training, algorithms need to be validated on a set of different data than what it was trained on, called a validation set, aiming to select the best model architecture and perform tuning of the hyperparameters. Following this process, the final model can then be finally examined on the test set. The authors have not utilized a validation set and as such the proposed ML models are likely overfitted to the test set and report a misleading higher area under the curve performance.[5]

Besides these methodological limitations, the proposed utility of these novel models is contentious. The authors suggest that the utilization of the described ML algorithm can reduce costs and risks associated with unbeneficial surgery. With an area under the curve of 0.759, this statement is inaccurate, as such models would refuse surgery to approximately one in four patients that would have benefited from CTD. Although this is a promising technology, it is still in its infancy and its potential at present has been overstated.

DISCLOSURE

The authors have no financial interest to declare in relation to the content of this article.