A Novel Application of Unsupervised Machine Learning and Supervised Machine Learning-Derived Radiomics in Anterior Cruciate Ligament Rupture.

PURPOSE: We aim to present an unsupervised machine learning application in anterior cruciate ligament (ACL) rupture and evaluate whether supervised machine learning-derived radiomics features enable prediction of ACL rupture accurately. PATIENTS AND METHODS: Sixty-eight patients were reviewed. Their demographic features were recorded, radiomics features were extracted, and the input dataset was defined as a collection of demographic features and radiomics features. The input dataset was automatically classified by the unsupervised machine learning algorithm. Then, we used a supervised machine learning algorithm to construct a radiomics model. The t-test and least absolute shrinkage and selection operator (LASSO) method were used for feature selection, random forest and support vector machine (SVM) were used as machine learning classifiers. For each model, the sensitivity, specificity, accuracy, and the area under the curve (AUC) of receiver operating characteristic (ROC) curves were calculated to evaluate model performance.
RESULTS: In total, 5 demographic features were recorded and 106 radiomics features were extracted. By applying the unsupervised machine learning algorithm, patients were divided into 5 groups. Group 5 had the highest incidence of ACL rupture and left knee involvement. There were significant differences in left knee involvement among the groups. Forty-three radiomics features were extracted using t-test and 7 radiomics features were extracted using LASSO method. We found that the combination of LASSO selection method and random forest classifier has the highest sensitivity, specificity, accuracy, and AUC. The 7 radiomics features extracted by LASSO method were potential predictors for ACL rupture.
CONCLUSION: We validated the clinical application of unsupervised machine learning involving ACL rupture. Moreover, we found 7 radiomics features which were potential predictors for ACL rupture. The study indicated that radiomics could be a valuable method in the prediction of ACL rupture.

Introduction

Anterior cruciate ligament (ACL) rupture is a common and devastating knee injury.1,2 Clinically, MRI has been widely used for the assessment of ACL rupture.3,4 The typical MRI findings of ACL rupture include abnormal orientation and abnormal signal intensity of ACL.5–7 In a study conducted by Zhao et al., 78 participants were investigated; the diagnostic sensitivity, specificity, and accuracy of MRI for ACL injury were 95.45%, 91.67%, and 94.87%.5

Although MRI has high sensitivity and specificity, the diagnosis rate at the early stage of ACL rupture remains poor.8–10 The potential causes include less experienced radiologists and insufficient awareness of physicians, especially for emergency physicians.11,12 Delayed diagnosis of ACL rupture usually results in higher risk of secondary knee injury and higher economic burden.8,13–15 In a study conducted by Church and Keating, 183 participants were investigated; a higher incidence of meniscal tears was observed in patients undergoing ACL reconstruction more than 12 months from injury.16 Therefore, early diagnosis of ACL rupture is crucial.

Recently, remarkable achievements have been made in knee MRI analysis using machine learning techniques.17,18 Bien et al. applied a deep learning model to analyze a dataset including 1370 knee MRI exams; they found that their deep learning model can generate accurate clinical pathology classifications and improve clinical experts’ performance.17 Usually, machine learning methods include an unsupervised algorithm, a supervised algorithm, and a semi-supervised algorithm. For a supervised machine learning algorithm, the algorithm learns on a labeled dataset and predicts the unlabeled dataset. In contrast, an unsupervised machine learning algorithm does not require labels and tries to work on its own to discover information.19,20

Radiomics focuses on extracting huge amounts of information from medical images.21,22 The hypothesis is that massive “hidden” information that is hardly perceived by radiologists and physicians could be quantitatively mined using a radiomics approach from medical images.23 Indeed, numerous studies have confirmed that radiomics can be used in the diagnosis and prognosis of diseases, with the advantages of non-invasiveness, low cost, and high efficiency.24–26 The key challenge of radiomics is the analysis of this “hidden” information. Coincidentally, many machine learning algorithms are essentially dimensionality reduction for big data. Several studies have indicated that the novel combination of machine learning and radiomics contributes to maximize physician performance and reduce errors, paving the path for modern precision and personalized medicine.18,27–29

In this study, we aim to present unsupervised machine learning application in anterior cruciate ligament (ACL) rupture and evaluate whether supervised machine learning-derived radiomics features enable accurate prediction of ACL rupture.

Patients and Methods

Participants

Sixty-eight patients undergoing knee surgery between October 2020 and December 2020 were reviewed. The inclusion criteria were: (1) first knee surgery; (2) surgery for knee disease; (3) the disease was definitively confirmed by arthroscopy. The exclusion criteria were: (1) simultaneous bilateral knee surgery; (2) revision surgery; (3) follow-up data missing. The patients’ demographic features were recorded. The study was approved by the institutional review board, and written informed consent was obtained.

MR Imaging Segmentation and Radiomics Feature Extraction

T1-weighted imaging (T1WI) of the knee was obtained in all patients. A musculoskeletal radiologist with 10 years of experience delineated regions of interest (ROIs) on T1WI using 3D Slicer software 4.11 (). The superior border of ROIs was at the level of the upper edge of the patella. The inferior border of ROIs was at the level of tibial insertion of the patella tendon (Figure 1). Then, all ROIs were manually checked by a musculoskeletal radiologist with 15 years of experience who were blind to the medical records of the patients. All disagreements were resolved through consensus.

Figure 1

Image acquisition and segmentation.

The radiomics features were extracted using an open-source Pyradiomics package (). All radiomics features were normalized to a value of 0 to 1.

Machine Learning

First, we used an unsupervised machine learning algorithm which has been validated and extensively tested in other diseases to convert the input dataset into a compact representation space.30–32 The input dataset was defined as a collection of demographic features and radiomics features. In the representation space, the subjects were positioned according to their similarity, while blinded to the patient’s outcome status. Once positioned in the compact representation space, subjects were clustered to identify phenotypically distinct categories of patients undergoing knee surgery. Clustering was performed using the K-means clustering algorithm and the best value of k was determined automatically.

Second, we used a supervised machine learning algorithm to construct a radiomics model based on extracted radiomics features. The t-test and least absolute shrinkage and selection operator (LASSO) method were adopted to select potential radiomics features contributing to the prediction of ACL rupture. Briefly, the LASSO technique constructs a model by constructing a penalty function, the LASSO model penalty function enables an efficient shrinking of coefficients to zero. The complexity of the LASSO model is controlled by the penalty parameter lambda, the maximum lambda value was selected with minimum mean-squared error (MSE) values. Then, random forest and support vector machine (SVM) were used as machine learning classifiers. For each model, the sensitivity, specificity, accuracy, and the area under the curve (AUC) of receiver operating characteristic (ROC) curves were calculated to evaluate model performance.

Statistical Analysis

The machine learning algorithms and the statistical analysis were implemented in Python (Python Software Foundation). Multiple comparisons among groups were analyzed by chi-square test for categorical variables, and ANOVA for continuous variables. A two-sided P value < 0.05 was considered significant.

Results

Characteristics of the Participants

There were 30 males and 38 females in the study. Five demographic features were recorded including gender, age, involved knee, trauma-history, and diagnosis. The demographic characteristics of the patients are listed in Table 1.

Table 1

The Demographic Characteristics of the Patients

Demographic Characteristics
Gender, male/female	30/38
Mean age, years	46.1 ± 15.3
Knee involvement, left/right	31/37
Trauma-history, yes/no	34/34
Diagnosis, ACL rupture/non- ACL rupture	26/42

In total, 106 radiomics features were extracted, including first-order features (n = 18), shape and size features (n = 13), and textural features (n = 75). The textural features included the gray-level co-occurrence matrix (GLCM), gray-level dependence matrix (GLDM), gray-level run-length matrix (GLRLM), gray-level size zone matrix (GLSZM), and neighbouring gray tone difference matrix (NGTDM).

In total, the input dataset included 111 features (5 clinical features and 106 radiomics features).

Results of the Unsupervised Machine Learning Algorithm

By applying the unsupervised machine learning algorithm, patients were divided into 5 groups. The scatter plot of principal component is shown in Figure 2. Group 5 has the highest incidence of ACL rupture and left knee involved. There were significant differences in left knee involvement among the groups. The demographic characteristics among groups are listed in Table 2.

Figure 2

The patients were divided into 5 groups, group 1 is represented as blue dots, group 2 is represented as green dots, group 3 is represented as black dots, group 4 is represented as red dots, and group 5 is represented as yellow dots.

Table 2

The Demographic Characteristics Among Groups

	Group 1	Group 2	Group 3	Group 4	Group 5	P value
Gender, male/female	3/3	12/8	7/11	5/13	3/3	0.3519
Mean age, years	45.8 ±21.6	41.9 ± 15.7	45.7 ± 16.4	51.7 ± 11.2	44. 7 ± 15.5	0.4148
Knee involvement, left/right	3/3	1/19	6/12	15/3	6/0	0.0226
Trauma-history, yes/no	3/3	12/8	8/10	7/11	4/2	0.6842
Diagnosis, ACL rupture/non- ACL rupture	3/3	9/11	6/12	4/14	4/2	0.2956

Results of the Supervised Machine Learning Algorithm

Based on the t-test, 43 radiomics features were extracted and considered as potential predictors of ACL rupture, meanwhile, the sensitivity, specificity, accuracy, and AUC corresponding to random forest and SVM classifiers are shown in Table 3.

Table 3

The Sensitivity, Specificity, Accuracy, and AUC of the Four Prediction Models

Feature Selection Method	Number of Radiomics Features	Random Forest				SVM
		Sensitivity	Sensitivity	Specificity	AUC	Sensitivity	Specificity	Accuracy	AUC
t-test	43	0.67	0.92	0.62	0.85	0.33	0.75	0.54	0.90
LASSO	9	0.80	0.94	0.90	0.92	0.57	0.79	0.71	0.74

Abbreviations: LASSO, Least absolute shrinkage and selection operator; SVM, support vector machine; AUC, Area under the curve of receiver operating characteristic curves.

Based on the LASSO method, 7 features were extracted and considered as potential predictors of ACL rupture. The MSE values and the corresponding lambda values are shown in Figure 3. The values of the coefficients and the corresponding lambda values are shown in Figure 4. Meanwhile, the sensitivity, specificity, accuracy, and AUC corresponding to random forest and SVM classifiers are listed in Table 3.

Figure 3

Different mean-squared error (MSE) values within the range of lambda, maximum lambda was selected with minimum MSE value.

Figure 4

The values of the coefficients and the corresponding lambda values, each curve represents each feature in the model.

We found that the combination of LASSO selection method and random forest classifier has the highest sensitivity, specificity, accuracy, and AUC. The AUC of the four prediction models are shown in Figure 5. The 7 radiomics features extracted by LASSO selection method are listed in Table 4, including 1 shape feature, 1 GLCM feature, 1 GLDM feature, 2 GLSZM features, and 2 NGTDM features. The radiomics features’ weights are shown in Figure 6.

Figure 5

The AUC of the four prediction models. (A) t-test and random forest, (B) t-test and SVM, (C) LASSO and random forest, (D) LASSO and SVM.

Table 4

The Radiomics Features Extracted by LASSO Selection Method

Radiomics Features	Feature Class
Flatness	Shape
Maximum Probability	GLCM
Large Dependence High Gray Level Emphasis	GLDM
Large Area Low Gray Level Emphasis	GLSZM
Size Zone Non-Uniformity
Busyness	NGTDM
Complexity

Abbreviations: GLCM, gray-level co-occurrence matrix; GLDM, gray-level dependence matrix; GLSZM, gray-level size zone matrix; NGTDM, neighbouring gray tone difference matrix.

Figure 6

The radiomics features weight.

Discussion

Anterior cruciate ligament (ACL) rupture is a common and devastating knee injury.1,2 In this study, we validated the clinical application of unsupervised machine learning involving ACL rupture. Moreover, we found 7 radiomics features which were potential predictors of ACL rupture, by applying supervised machine learning. The prediction model (the combination of LASSO selection method and random forest classifier) has high sensitivity, specificity, accuracy, and AUC.

By applying the unsupervised machine learning algorithm, we found that the highest incidence rate of left knee involvement followed by the highest incidence of ACL rupture, which indicated that left knee could be a risk factor for ACL rupture. In a study conducted by Westin et al., 339 alpine ski students were investigated, the majority of ACL injuries significantly occurred in the left knee.33 Our study showed that the unsupervised machine learning algorithm could be applied effectively in the research of ACL injury.

By applying supervised machine learning-derived radiomics, we found 7 radiomics features were potential predictors for ACL rupture, which indicated that radiomics could be a valuable method in the prediction of ACL rupture. So far, many deep learning algorithms have been proposed for identifying ACL rupture,17,18,34 as far as we know, this is the first study to investigate machine learning-derived radiomics features for ACL rupture. Although the detection of ACL rupture is not a diagnostic challenge for specialized musculoskeletal radiologists and sports physicians, our study may be helpful for non-trained radiologists and non-sport physicians; especially, our study may provide a “second-reader opinion” in a rural area without access to subspecialty radiology or subspecialty radiology interpretation is not readily available.

The limitations of our study include the relatively small numbers of patients and it being a retrospective single-center study. Due to the relatively small numbers of patients, we may not be able to screen out the most valuable and stable radiomics features, and the developed model may not be the most effective. A further larger, multicenter study was needed to confirm our findings.

Conclusion

We validated the clinical application of unsupervised machine learning involving ACL rupture. Moreover, we found 7 radiomics features which were potential predictors for ACL rupture. It indicated that radiomics could be a valuable method in the prediction of ACL rupture.