Could Machine Learning Better Predict Postoperative C5 Palsy of Cervical Ossification of the Posterior Longitudinal Ligament?

STUDY
DESIGN: This was a retrospective cohort study.
OBJECTIVE: The objective of this study was to investigate whether machine learning (ML) can perform better than a conventional logistic regression in predicting postoperative C5 palsy of cervical ossification of the posterior longitudinal ligament (OPLL) patients. SUMMARY OF BACKGROUND DATA: C5 palsy is one of the most common postoperative complications after surgical treatment of OPLL, with an incidence rate of 1.4%-18.4%. ML has recently been used to predict the outcomes of neurosurgery. To our knowledge there has not been a study to predict postoperative C5 palsy of cervical OPLL patient with ML.
METHODS: Four sampling methods were used for data balancing. Six ML algorithms and conventional logistic regression were used for model development. A total of 35 ML prediction model and 5 conventional logistic prediction models were generated. The performances of each model were compared with the area under the curve (AUC). Patients who underwent surgery for cervical OPLL at our institute from January 1998 to January 2012 were reviewed. Twenty-five variables of each patient were used to make a prediction model.
RESULTS: In total, 901 patients were included [651 male and 250 female, median age: 55 (49-63), mean±SD: 55.9±9.802]. Twenty-six (2.8%) patients developed postoperative C5 palsy. Age (P=0.043), surgical method (P=0.0112), involvement of OPLL at C1-3 (P=0.0359), and postoperative shoulder pain (P≤0.001) were significantly associated with C5 palsy. Among all ML models, a model using an adaptive reinforcement learning algorithm and downsampling showed the largest AUC (0.88; 95% confidence interval: 0.79-0.96), better than that of logistic regression (0.69; 95% confidence interval: 0.43-0.94).
CONCLUSIONS: The ML algorithm seems to be superior to logistic regression for predicting postoperative C5 palsy of OPLL patient after surgery with respect to AUC. Age, surgical method, and involvement of OPLL at C1-C3 were significantly associated with C5 palsy. This study demonstrates that shoulder pain immediately after surgery is closely associated with postoperative C5 palsy of OPLL patient.

Ossification of the posterior longitudinal ligament (OPLL) is a disease caused by fibrosis, calcification, and OPLL of the spine.1–3 It was once considered a unique disorder in Asians, but it is now recognized as a disease occurring in people of various ethnicities.4–7 While the exact mechanism is unknown, its progression leads to a narrowing of the spinal canal and compression of the spinal cord, which eventually requires surgical decompression. Surgical options include anterior corpectomy with fusion, posterior laminectomy with fusion, and extensive laminoplasty. However, these are considered technically demanding and associated with serious complications. C5 palsy is one of the most common postoperative complications after surgical treatment of cervical OPLL.4,8–10

C5 palsy is a weakness of the deltoid or biceps brachii by at least 1 grade in the manual muscle test without deterioration of lower extremity function.11,12 It occurs soon after surgery, which significantly reduces the quality of life.13 Its incidence is not rare, ranging from 1.4% to 18.4%.9 However, the exact mechanism has not been established, and a few hypotheses have been proposed.14 Thus, to date, the treatment of C5 palsy relies on uncertain hypotheses and experiences, not mechanism-based methods. Therefore, the ability to predict C5 palsy in advance will allow for better decision-making in the type of surgery or earlier treatments for prevention.

Machine learning (ML) has recently been used to predict the outcomes of neurosurgery.15–23 Because clinical data is often large, manual analysis requires a lot of human resources. ML can be used to discover hidden patterns by self-learning with a given algorithm to analyze large-scale clinical data. The use of ML in medicine is increasing owing to its ability to process large data and transform the analysis into clinical insights, which ultimately leads to better outcomes, lower costs, and higher patient satisfaction.17,24 Furthermore, the use of ML has been increasing in the field of spinal surgery.21–23,25–32 However, there have been no ML studies on postoperative C5 palsy after surgical treatment of cervical OPLL patients.

This study aims to investigate whether ML can perform better than conventional logistic regression in predicting postoperative C5 palsy after surgical treatment of cervical OPLL patients.

METHODS

Patient Demographics

A retrospective analysis was performed in patients who underwent surgery for cervical OPLL at our institute from January 1998 to January 2012. The Human Research Protection Center of our university waived off the need for institutional review board approval. The patients were enrolled according to the following criteria: (1) age 18 years and above, (2) symptomatic cervical OPLL, (3) imaging evidence of cervical OPLL, and (4) follow-up for at least 12 months. We excluded nonsurgical cases and patients with spinal cord injury, neoplastic disease, active infection, congenital disorder, or inflammatory disease. Patients who underwent occipitocervical or cervicothoracic fusion were also excluded.

Baseline Data

A total of 25 variables were examined. The variables were divided into demographic, clinical, radiologic, operation, and postoperative variables. The demographic variables included age, sex, and previous spine surgery. The clinical variables included posterior neck pain, radicular arm pain, tingling sensation, numbness, weakness, myelopathy, and Japanese Orthopedic Association (JOA) scores. The radiologic variables included number of involved levels, involved segments (C1–C3, C4–C5, and C6–C7), and cervical OPLL types (continuous, segmental, mixed, and circumscribed). The operative variables included surgical methods (open-door laminoplasty, French-door laminoplasty, posterior fusion, anterior fusion, and combined fusion) and postoperative shoulder pain that was newly developed within 24 hours after surgery. The primary outcome measure was postoperative C5 palsy, which was defined as a weakness of the deltoid or biceps brachii by at least 1 grade in the manual muscle test without deterioration of lower extremity function. Patients who did not develop C5 palsy were defined as a control group.

Data Processing

To balance the small size of the C5 palsy group with that of the control group, the following 4 resampling methods were used: upsampling, downsampling, synthetic minority oversampling technique, and random oversampling examples (ROSE). A total of 5 datasets including the original dataset, were created for the development of each model.

Model Development

Two types of models were developed. Model 1 included all variables. In model 2, postoperative shoulder pain was excluded. In other words, model 1 implies perioperative prediction, which forecast C5 palsy after observing whether the patient develop postoperative shoulder pain. In contrast, model 2 implies preoperative prediction, which presume C5 palsy before the surgery. Therefore, the importance of postoperative shoulder pain in predicting C5 palsy could be derived from comparison between models 1 and 2.

To reduce overfitting, models were developed using 5-fold cross-validation, which randomly allocated each dataset as either a training set (80%) or validation set (20%).17 The results were averaged, and the SDs were calculated.

We used 6 ML algorithms—decision tree, random forest, artificial neural network (ANN), gradient boosting machine (GBM), adaptive reinforcement learning (ADA), and support vector machine using radial kernel (SVM). We also performed logistic regression to compare it with the ML algorithms. Each algorithm was applied to prepared 5 datasets (original dataset+4 sampled dataset). As a result, 30 ML models and 5 logistic regression models were developed for each model 1 and 2. At last, 35 types of model 1 and 35 types of model 2 were prepared for performance analysis.

Statistical Analysis

The performances of the ML algorithms and logistic regression were compared using a receiver operating characteristic curve, the area under the curve (AUC), sensitivity, specificity, predictive values, and 95% confidence interval (CI). The performance of the best model was displayed in a confusion matrix. A P-value <0.05 was considered statistically significant. The χ2 test and Fisher exact test were used to identify significant differences between categorical variables. The Wilcoxon rank-sum test was used to compare the median of continuous variables between groups. RStudio (R Foundation for Statistical Computing, Vienna, Austria) was used for data processing, model creation, and statistical analysis.

RESULTS

A total of 901 OPLL patients were treated with surgery. Among them, 26 (2.8%) patients developed postoperative C5 palsy. The patients’ baseline characteristics are shown in Table 1. Age, surgical method, involvement of OPLL at C1–C3, surgical methods, and presence of postoperative shoulder pain were significantly associated with C5 palsy. Patients in the C5 palsy group were on an average 5 years older than those in the control group (59.58±9.03 vs. 55.75±9.81 y, P=0.043). Involvement of OPLL at C1–C3 was observed about 1.5 times more in the C5 palsy group than in the control group (52.23% vs. 73.08%, P=0.0359). C5 palsy was significantly associated with surgical methods (P=0.0112). Among surgical methods, C5 palsy were more frequent in the posterior approach. In posterior approach, regardless of specific method, incidence of postoperative C5 palsy were about 2 times higher than anterior approach [open-door laminoplasty, right: 15.38% (palsy group) vs. 8.57% (control group); open-door laminoplasty, left: 15.38% vs. 9.37%; French-door laminoplasty: 3.85% vs. 1.71%; laminectomy with fusion: 7.69% vs. 4.11%; anterior cervical discectomy and fusion or anterior cervical corpectomy and fusion: 57.69% vs. 54.63%]. No patient with a combined approach developed C5 palsy. Postoperative shoulder pain was observed about 3 times more in the C5 palsy group than in the control group (53.9% vs. 18.6%, P<0.001).

TABLE 1

Demographics of Patients in the C5 Palsy and Control Groups

	n (%)
Characteristic	Nonpalsy (N=875)	Palsy (N=26)	P
Age [median (range)]	55 (49–62)	60 (53–66)	0.043
Sex
Male	630	21
Female	245	5	0.325
Duration [median (range)]	6 (3–24)	13 (11–15)	0.082
Previous cervical disease	36 (4.11)	0	0.6199
Previous lumbar disease	49 (5.6)	1 (3.85)	1
Posterior neck pain	469 (53.91)	14 (53.85)	0.995
Right arm pain	453 (52.07)	13 (50)	0.8352
Left arm pain	465 (53.51)	14 (53.85)	0.973
Tingling sensation	618 (71.12)	20 (76.92)	0.519
Numbness	447 (51.5)	11 (42.31)	0.3556
Weakness	473 (54.43)	17 (65.38)	0.2688
Myelopathy	581 (67.09)	20 (76.92)	0.292
Preoperative JOA score [median (range)]
Total	12 (11–15)	13 (11–15)	0.680
I	2 (2–4)	2 (2–4)	0.320
II	3 (2–4)	3 (3–4)	0.686
III_A	1 (1–2)	2 (1–2)	0.185
III_B	1 (1–2)	2 (1–2)	0.185
III_C	2 (1–2)	2 (1–2)	0.226
IV	3 (2–3)	3 (2–3)	0.605
No. involved level	4 (3–5)	4 (3–5)	0.183
OPLL type
Continuous	75 (8.68)	1 (3.85)	0.4852
Segmental	374 (43.39)	9 (34.62)
Mixed	268 (31.02)	9 (34.62)
Circumscribed	147 (17.01)	7 (26.92)
OPLL involvement at C1, C2, C3	457 (52.23)	19 (73.08)	0.0359
OPLL involvement at C4, C5	848 (96.91)	25 (96.15)	0.5651
OPLL involvement at C6, C7	754 (86.17)	21 (80.77)	0.3937
Operation method			0.0112
Laminoplasty—right open door	75 (8.57)	4 (15.38)
Laminoplasty—left open door	82 (9.37)	4 (15.38))
Laminoplasty—French door	15 (1.71)	1 (3.85)
LMSF	36 (4.11)	2 (7.69)
ACDF or ACCF	478 (54.63)	15 (57.69)
Anterior and posterior combined	189 (21.6)	0 (0.00)
Postoperative shoulder pain	163 (18.63)	14 (53.85)	<0.0001

Significant P <0.05 values are in bold.

ACCF indicates anterior cervical corpectomy and fusion; ACDF, anterior cervical discectomy and fusion; JOA, Japanese Orthopedic Association; LMSF, lateral mass screw fixation and fusion; OPLL, ossification of the posterior longitudinal ligament.

Performance of Prediction Models

The overall performance of model 1, based on the AUC, ranged from 0.88 (95% CI: 0.79–0.96) for a combination of ADA algorithm and downsampling to 0.38 (95% CI: 0.34–0.41) for a combination of ROSE sampling and ADA or GBM algorithms (Table 2). Among the ML models, the ADA algorithm using downsampling showed the largest AUC (0.88; 95% CI: 0.79–0.96), which was larger than that of the best logistic regression model (0.69; 95% CI: 0.43–0.94) (Fig. 1). The overall performance of model 2, based on the AUC, ranged from 0.81 (95% CI: 0.68–0.94) for a combination of downsampling and ADA algorithm to 0.38 (95% CI, 0.34–0.41) for the combination of ROSE sampling and GBM algorithm or upsampling and random forest algorithm. Among the ML models, the ADA algorithm using downsampling showed the largest AUC (0.81; 95% CI, 0.68–0.94), which was larger than that of the best logistic regression model (0.55; 95% CI: 0.33–0.78).

TABLE 2

Comparison of Area Under the Curve Values Between Machine Learning and Logistic Regression Models

Sampling Method	Logistic Regression	Decision Tree	Random Forest	ANN	SVM	Gradient Boosting Machine	Adaptive Reinforcement Learning
Model 1
Original	0.69 (0.43–0.94)	0.50	0.71 (0.47–0.94)	0.66 (0.36–0.95)	0.61 (0.30–0.91)	0.77 (0.63–0.91)	0.71 (0.51–0.92)
Down	0.58 (0.32–0.84)	0.76 (0.64–0.89)	0.71 (0.49–0.93)	0.75 (0.62–0.89)	0.47 (0.30–0.65)	—	0.88 (0.79–0.96)
Up	0.62 (0.38–0.87)	0.53 (0.51–0.55)	0.51 (0.18–0.84)	0.46 (0.19–0.73)	0.64 (0.34–0.93)	0.63 (0.41–0.86)	0.71 (0.51–0.91)
SMOTE	0.67 (0.43–0.91)	0.64 (0.30–0.99)	0.80 (0.69–0.91)	0.64 (0.33–0.95)	0.64 (0.43–0.85)	0.74 (0.54–0.95)	0.81 (0.70–0.93)
ROSE	0.59 (0.37–0.82)	0.40 (0.37–0.43)	0.40 (0.37–0.43)	0.66 (0.43–0.89)	0.72 (0.43–1.000)	0.38 (0.34–0.41)	0.38 (0.34–0.41)
Model 2
Original	0.40 (0.20–0.59)	0.50	0.60 (0.57–0.63)	0.57 (0.54–0.60)	0.48 (0.17–0.78)	0.65 (0.42–0.88)	0.61 (0.38–0.85)
Down	0.55 (0.33–0.78)	0.76 (0.64–0.89)	0.66 (0.34–0.99)	0.61 (0.37–0.85)	0.50 (0.34–0.66)	—	0.81 (0.68–0.94)
Up	0.47 (0.27–0.66)	0.54 (0.52–0.56)	0.38 (0.14–0.61)	0.47 (0.23–0.71)	0.57 (0.29–0.86)	0.59 (0.38–0.80)	0.60 (0.40–0.79)
SMOTE	0.51 (0.33–0.69)	0.54 (0.30–0.78)	0.60 (0.41–0.80)	0.59 (0.39–0.80)	0.52 (0.34–0.70)	0.70 (0.51–0.88)	0.62 (0.47–0.76)
ROSE	0.53 (0.35–0.71)	0.400 (0.37–0.43)	0.40 (0.37–0.43)	0.52 (0.52–0.79)	0.57 (0.25–0.90)	0.38 (0.34–0.41)	0.465 (0.45–0.48)

ANN indicates artificial neural network; Down, downsampling; Original, original data (no sampling); ROSE, random oversampling examples; SMOTE, synthetic minority oversampling technique; SVM, support vector machine; Up, upsampling.

FIGURE 1

Comparison of area under curve of the conventional logistic regression model with the best machine learning models that were generated for models 1 and 2.

Model Validation

In the best ML model 1 (ADA algorithm with downsampling), of the 5 cases of C5 palsy in the test set, 1 was incorrectly predicted. The accuracy and error rate were 94.3% and 5.7%, respectively. The sensitivity, specificity, positive predictive value, and negative predictive value were 80%, 83.53%, 12.5%, and 99.3%, respectively. The effect of exclusion of postoperative shoulder pain was studied using the best ML model 2 (ADA algorithm with downsampling). Of the 5 cases of C5 palsy in the test set, it incorrectly predicted 3 patients. The accuracy and error rate were 89.1% and 10.9%, respectively. The sensitivity, specificity, positive predictive value, and negative predictive value were 40%, 90.6%, 11.1%, and 98.1%, respectively. The exclusion of postoperative shoulder pain resulted in a deterioration of performance, except specificity (Table 3).

TABLE 3

Validation of Best Machine Learning Model

	Actual C5 Palsy (+)	Actual C5 Palsy (−)
Predicted C5 palsy (+)	4	28
Predicted C5 palsy (−)	1	142

Significance of Variable

The average weighted sum of improvement in error by each variable during model training was calculated. The variable showing the largest improvement of in error were given 100% importance, and the importance of other variables were represented as relative values.33 In best ML model 1 (ADA algorithm with downsampling), the significant variables included postoperative shoulder pain (100.0%), surgical methods (95.0%), JOA score (72.7%), myelopathy (68.5%), and weakness (66.9%). In best ML model 2 (ADA algorithm with downsampling), the significant variables included surgical methods (100.0%), JOA score (76.5%), myelopathy (72.1%), weakness (70.5%), and age (62.6%) (Fig. 2).

FIGURE 2

Importance of variables in best models of model 1 (A) and model 2 (B). JOA indicates Japanese Orthopedic Association; OPLL, ossification of the posterior longitudinal ligament.

DISCUSSION

In this study, we showed that ML algorithms seems to be superior to logistic regression for predicting postoperative C5 palsy in terms of AUC. Our result was consistent with previous studies which argued the superiority of ML in spine surgery. Merali et al28 reported that a ML model using the random forest method was able to predict the surgical outcome of degenerative cervical myelopathy with an AUC of 0.7. Arvind et al25 reported that a ML model using the ANN method was able to predict postoperative complications of anterior cervical discectomy and fusion with an AUC of 0.518–0.979. Kim et al27 reported that a ML model using the ANN method was able to predict postoperative complications of posterior lumbar fusion with an AUC of 0.567–0.710. Based on these studies, ML algorithms are promising tools for predicting surgical outcome and complications.

This study revealed that age was significantly associated with C5 palsy; patients in the C5 palsy group were older than those in the control group. Nassr et al34 investigated risk factors of C5 palsy after various cervical spine surgery and found that age was significantly associated with C5 palsy in anterior cervical corpectomy. Nori et al35 reported that among patients who received cervical laminectomy or laminoplasty, patients in the C5 palsy group were significantly older than those in the nonpalsy group (70.4±7.2 vs. 62.7±10.8 y). In both anterior and posterior approach, older age was a risk factor of postoperative C5 palsy, and it is similar with the results of our study.

Our study showed that involvement of OPLL at C1–C3 was significantly associated with C5 palsy, which was surprising because we hypothesized that involvement of OPLL at C4–C5 would be associated with C5 palsy. However, a previous study reported a similar finding; Minoda et al36 reported that only anterior cord compression at C3 was significantly associated with postoperative C5 palsy after French-door laminoplasty. Although the exact mechanism of C5 palsy is still unknown, several hypotheses have been proposed. The suggested mechanisms include direct injury to the nerve root, traction injury by extradural tethering, spinal cord dysfunction, and reperfusion injury after spinal cord ischemia.37–40 The association between cervical levels and C5 palsy requires further anatomic and physiological studies. According to our result, when dealing with high proximal level (C1–C3) OPLL, spine surgeon should have in mind that the risk of postoperative C5 palsy is higher than other cervical level.

In this study, postoperative shoulder pain was significantly associated with C5 palsy. In contrast to Nassr et al34 who included increased postoperative pain in C5 dermatome in their definition of C5 palsy, postoperative shoulder pain preceded deltoid weakness in OPLL patients who were treated with surgery in our study. Hashimoto et al41 report a similar finding that among 17 patients who developed C5 palsy after anterior cervical discectomy and fusion, 16 patient presented posterior neck and shoulder pain before muscle weakness. According to our result, postoperative shoulder pain seems to represent the beginning stages of a developing C5 palsy. This result can be applied clinically. We recommend preventively administering a steroid to patients who have new shoulder pain immediately after surgery even if C5 palsy is not yet evident because postoperative shoulder pain predict the development of a full-blown palsy. Dombrowski et al42 reported that perioperative dexamethasone administration at the initial incision and every 8 hours for 24 hours postoperatively helped to significantly reduce cases of C5 palsy compared with the group without perioperative dexamethasone administration (3.6% vs. 9.5%). They suggested perioperative inflammation as potential pathogenesis of C5 palsy in addition to direct nerve injury.34 Combination of their result that steroid was effective in preventing postoperative C5 palsy and our result that postoperative shoulder pain is a signal forecasting possible C5 palsy produce a practice that might be helpful in a clinical setting. Therefore, we recommend preventively administering steroids to patients who have shoulder pain immediately after surgery, even if C5 palsy is not yet evident.

To summarize, spine surgeon facing the operation of OPLL patients could weigh up the risk of C5 palsy according to our study. OPLL Patients with older age, high proximal index level (C1–C3), and who is a candidate for posterior approach should be warned before surgery that they may develop C5 palsy after surgery. Surgeons may use known preventive measures including C4/C5 foraminotomy43–45 or intraoperative intravenous steroid.42 In addition, if shoulder pain develops newly after surgery, we may look carefully whether the patient does not develop C5 palsy.

Our study has some limitations. First, only age, surgical method, involvement of OPLL at C1–C3, and postoperative shoulder pain were significantly associated with C5 palsy. However, we included other variables that were not significantly associated with C5 palsy in developing the prediction model. Second, C5 palsy group was so smaller compared with the control group (n=26, 2.8%). Although we used sampling methods, data imbalance is not completely solved. Therefore, there is a high likelihood for overfitting in model development due to inclusion of >20 variables, along with so fewer number of C5 palsy group. The multicenter study could recruit more C5 palsy patient and improve the inherent error from small sample size. Third, there was no independent testing set for model validation. Although, we conducted cross-validation in model development, the performance of these models in a nontraining cohort is unknown. Fourth, we used only AUC as criteria for model comparison. However, calibration and decision curve analysis is other critical metric for the prediction model. Our study is not free from criticism.

CONCLUSIONS

The ML algorithm seems to be superior to logistic regression for predicting postoperative C5 palsy of OPLL patient after surgery with respect to AUC. Age, surgical methods, and involvement of OPLL at C1–C3 were significantly associated with C5 palsy. This study demonstrates that shoulder pain immediately after surgery is closely associated with postoperative C5 palsy of OPLL patient.