Comparison Between the Japanese Orthopaedic Association (JOA) Score and Patient-Reported JOA (PRO-JOA) Score to Evaluate Surgical Outcomes of Degenerative Cervical Myelopathy.

STUDY
DESIGN: A retrospective cohort study.
OBJECTIVE: To investigate whether the Japanese Orthopaedic Association (JOA) score can be used for patients with degenerative cervical myelopathy as a patient-reported outcome (PRO) through the JOA written questionnaire.
METHODS: A total of 75 patients who underwent posterior decompression surgery for degenerative cervical myelopathy were reviewed. Patients responded to questionnaires including PRO-JOA, EuroQOL-5D, Neck Disability Index, and Short Form-12 preoperatively and at >12 months postoperatively. Spearman's rho and Bland-Altman analyses were used to investigate the correlations.
RESULTS: Preoperative JOA and PRO-JOA scores were 10.8 and 10.6, respectively, with Spearman's rho of 0.74. Similarly, postoperative JOA and PRO-JOA scores were 13.3 and 12.9, respectively, with Spearman's rho of 0.68. However, the recovery rates for JOA and PRO-JOA scores were 42% and 27%, respectively, with Spearman's rho of 0.45. Compared with other PROs, JOA and PRO-JOA scores were moderately correlated. The minimum clinically important difference was 2.5 for JOA score, 3.0 for PRO-JOA score, 42% for JOA recovery rate, and 33% for PRO-JOA recovery rate. Bland-Altman analyses revealed that limits of agreement were -4.3 to 4.7, -3.4 to 4.3, and -75% to 106% for the preoperative score, postoperative score, and recovery rate, respectively.
CONCLUSION: PRO-JOA score can also be used as a disease-specific scoring measure instead of JOA score. However, although both measures demonstrate a similar trend as a group analysis, PRO-JOA and JOA scores should be regarded as different outcomes.

Introduction

Degenerative cervical myelopathy can cause various symptoms, such as numbness and hand clumsiness, walking disability, and urinary problems. Because these symptoms are considered to deteriorate gradually, surgical intervention is required when neurological problems are moderate or severe. To evaluate postoperative neurological changes, several quantitative measurements have been developed. Among them, the Japanese Orthopaedic Association (JOA) score, originally developed by JOA in 1975, is a widely used disease-specific outcome tool that can provide a quantitative measure for patients with cervical compression myelopathy.[1,2] JOA score consists of 6 domains: motor function in upper extremities, motor function in lower extremities, sensory function in upper extremities, sensory function in the trunk, sensory function in lower extremities, and bladder function, with a minimum total score of 0 and maximum of 17 (Table 1). Because the JOA score was originally designed for Japanese patients using chopsticks, its modified version, modified JOA (mJOA) score considers the use of knives and forks instead of chopsticks because they are more popular in Western countries. Both JOA and mJOA scores are widely used as primary outcomes for cervical myelopathy, and their minimum clinically important difference (MCID) have been reported,[4,5] and the inter- and intraobserver reliability of the JOA score is established.

Table 1.

Japanese Orthopaedic Association (JOA) Score.

1. Upper extremity motor function
0	Impossible to eat with chopsticks or spoon
1	Possible to eat with spoon but not with chopsticks
2	Possible to eat with chopsticks, but to a limited degree
3	Possible to eat with chopsticks, awkward
4	No disability
2. Lower extremity motor function
0	Cannot walk
1	Needs cane or aid on flat ground
2	Needs cane or aid only on stairs
3	Can walk without cane or aid but slowly
4	No disability
3. Sensory function
A. Upper extremity
0	Apparent sensory loss
1	Minimal sensory loss
2	Normal
B. Trunk (same as A)
C. Lower extremity (same as A)
4. Bladder function
0	Complete retention
1	Severe disturbance
2	Mild disturbance
3	Normal

One possible problem of the JOA score is that it is based on the physicians’ preference and can be biased, as opposed to patient-reported outcomes (PROs), such as EuroQOL (EQ-5D), Neck Disability Index (NDI), and Short Form–12 (SF-12). Furthermore, physicians should still personally examine the patients to provide an accurate JOA score. Therefore, whether JOA score can function as a PRO remains to be elucidated, wherein patients can respond to the JOA written questionnaire, similar to other PROs. This study aimed to compare the JOA score provided by patients (PRO-JOA score) with the JOA score provided by physicians.

Methods

The study protocols were approved by the institutional review board of our institution. This was a retrospective cohort study of 88 patients undergoing posterior decompression surgery for degenerative cervical myelopathy between 2009 and 2013 at a single academic institution. Patients with rheumatoid arthritis, tumors, or history of trauma were not included. Fifteen patients with incomplete data were excluded. Therefore, 75 patients were eligible for the analysis. All patients were followed for >12 months.

Patients were instructed to fill in the questionnaires both pre- and postoperatively, including PRO-JOA score, NDI, EuroQOL-5D (EQ-5D), SF-12 Health Survey score (physical and mental component summary scores [PCS and MCS, respectively]), and treatment satisfaction. As for the treatment satisfaction, a 7-point Likert-type scale was used, namely, the patients were asked to answer whether they were satisfied with the treatment, with possible answers of “very dissatisfied,” “dissatisfied,” “somewhat dissatisfied,” “neither,” “somewhat satisfied,” “satisfied,” and “very satisfied.” The JOA score was detected by spine surgeons at the outpatient 1 to 2 weeks before the PROs including PRO-JOA score were taken. The recovery rate was calculated according to the following formula (Hirabayashi method) : Recovery rate (%) = (postoperative JOA − preoperative JOA) / (17 [full score] − preoperative JOA) × 100.

As for the cutoff values for the MCID between JOA and PRO-JOA scores, the anchor-based method was used with the satisfaction as an anchor. The cutoff values were set at points on the receiver operating characteristic (ROC) curve, in which (1 − sensitivity) + (1 − specificity) was the smallest according to the least squares method. Sensitivity was defined as the proportion of patients with score changes of greater than the MCID among those who met the golden standard criterion. The external criterion was defined as patients who were satisfied with the treatment, including “somewhat satisfied,” “satisfied,” and “very satisfied.”

Statistical Analysis

All analyses were conducted using the IBM SPSS Statistics software program, version 25 (IBM Corp). To analyze the difference in scores pre- and postoperatively, a paired t test or Wilcoxon signed rank test was used. The scores and outcomes between the JOA and PRO-JOA scores were compared using Spearman’s rho and Bland-Altman analyses to investigate their correlations. The internal consistency of the JOA and PRO-JOA scores were assessed by Cronbach’s alpha. For all statistical tests, P values of <.05 were considered significant.

Results

The mean age of patients was 65 years (range, 31-86 years), and the sample comprises 55 males and 20 females. A total of 48 patients were diagnosed with cervical spondylotic myelopathy, whereas 27 patients as ossification of the longitudinal ligament. The mean follow-up period was 20 months (range, 12-28 months).

The mean preoperative JOA and PRO-JOA scores (10.8 and 10.6, respectively) were not significantly different (P = .45), with a correlation coefficient (Spearman’s rho) of 0.74. Cronbach’s alpha for the JOA and PRO-JOA scores were 0.56 and 0.70, respectively. Likewise, the postoperative JOA and PRO-JOA scores (13.3 and 12.9, respectively) were not significantly different (P = .06), with a correlation coefficient of 0.68. However, the recovery rates for JOA and PRO-JOA scores (42% and 27%, respectively) were significantly different (P < .01), with a correlation coefficient of 0.45 (Table 2).

Table 2.

Correlation Coefficient Between JOA Score and PRO-JOA Score.

	Mean JOA score	Mean PRO-JOA score	Spearman’s rho
Preoperative
Total	10.8	10.6	0.74
MU	2.4	2.6	0.54
ML	1.9	2.1	0.73
SU	1.0	0.9	0.30
ST	1.3	1.2	0.47
SL	1.9	1.7	0.34
BL	2.3	2.1	0.40
Postoperative
Total	13.3	12.9	0.68
MU	3.3	3.4	0.47
ML	2.7	2.7	0.69
SU	1.3	1.3	0.41
ST	1.5	1.3	0.37
SL	2.0	1.9	0.15
BL	2.6	2.4	0.42
Recovery rate (%)	42.4	27.1	0.45

Abbreviations: JOA, Japanese Orthopaedic Association; PRO, patient-reported outcome; MU, motor function in the upper extremities; ML, motor function in the lower extremities; SU, sensory function in the upper extremities; ST, sensory function in the trunk; SL, sensory function in the lower extremities; BL, bladder function.

When comparing JOA and PRO-JOA scores with other PROs, both scores showed moderate correlations (Tables 3 and 4). A total of 53 (70%) patients replied “satisfied” to the treatment.

Table 3.

Correlations Coefficient (Spearman’s rho) Between Preoperative Total Scores Among JOA Score, PRO-JOA Score, NDI, EQ-5D, and SF-12 (PCS and MCS).

	JOA	PRO-JOA	NDI	EQ5D	SF-12 PCS	SF-12 MCS
JOA	1	0.68	−0.47	0.51	0.42	0.02
PRO-JOA		1	−0.60	0.63	0.52	0.29
NDI			1	−0.88	−0.71	−0.22
EQ5D				1	0.55	0.31
SF-12 PCS					1	−0.04
SF-12 MCS						1

Abbreviations: JOA, Japanese Orthopaedic Association; PRO, patient-reported outcome; NDI, Neck Disability Index, EQ-5D, EuroQOL-5D; SF-12, Short Form-12; PCS, physical component summary score; MCS, mental component summary score.

Table 4.

Correlations Coefficient (Spearman’s rho) Between Postoperative Total Scores Among JOA Score, PRO-JOA Score, NDI, EQ-5D, and SF-12 (PCS and MCS).

	JOA	PRO-JOA	NDI	EQ5D	PCS	MCS
JOA	1	0.68	−0.54	0.58	0.45	0.11
PRO-JOA		1	−0.45	0.66	0.46	0.22
NDI			1	−0.69	−0.72	−0.29
EQ5D				1	0.65	0.27
PCS					1	0.25
MCS						1

Abbreviations: JOA, Japanese Orthopaedic Association; PRO, patient-reported outcome; NDI, Neck Disability Index, EQ-5D, EuroQOL-5D; SF-12, Short Form-12; PCS, physical component summary score; MCS, mental component summary score.

Next, ROC curves were created using the patient satisfaction question results to determine MCID. The area under the curve (AUC) was 0.67 and MCID was 2.5 for the JOA score (Figure 1), whereas AUC was 0.68 and MCID was 3.0 for the PRO-JOA score (Figure 2). Similarly, the AUC and MCID for the JOA recovery rate were 0.68 and 42%, respectively (Figure 3), whereas those for PRO-JOA recovery rate were 0.70 and 33%, respectively (Figure 4).

Figure 1.

Receiver operating characteristic curve of the Japanese Orthopaedic Association (JOA) score to determine the minimum clinically important difference.

Figure 2.

Receiver operating characteristic curve of the patient-reported outcome–Japanese Orthopaedic Association (PRO-JOA) score to determine the minimum clinically important difference.

Figure 3.

Receiver operating characteristic curve of the Japanese Orthopaedic Association (JOA) score recovery rate to determine the minimum clinically important difference.

Figure 4.

Receiver operating characteristic curve of the patient-reported outcome–Japanese Orthopaedic Association (PRO-JOA) score recovery rate to determine the minimum clinically important difference.

Finally, Bland-Altman analyses revealed that limits of agreement were −4.3 to 4.7 for the preoperative score (Figure 5), −3.4 to 4.3 for the postoperative score (Figure 6), and −75% to 106% for the recovery rate (Figure 7).

Figure 5.

A Bland-Altman plot comparing the preoperative JOA (PreJOA) and PRO-JOA (PrePRO-JOA) scores. The bias is shown as a solid line, and the upper and lower limits of agreement are shown as broken lines. JOA, Japanese Orthopaedic Association; PRO, patient-reported outcome.

Figure 6.

A Bland-Altman plot comparing the postoperative JOA (PostJOA) and PRO-JOA (PostPRO-JOA) scores. The bias is shown as a solid line, and the upper and lower limits of agreement are shown as broken lines. JOA, Japanese Orthopaedic Association; PRO, patient-reported outcome.

Figure 7.

A Bland-Altman plot comparing the JOA score recovery rate and PRO-JOA score recovery rate (PRO-JOARR) scores. The bias is shown as a solid line, and the upper and lower limits of agreement are shown as broken lines. JOA, Japanese Orthopaedic Association; PRO, patient-reported outcome.

Discussion

This study identified that PRO-JOA score can also be used as a disease-specific scoring measure, for example, to determine the neurological status of patients who reside distantly from the hospital and are unable to visit their physicians postoperatively. However, although both measures demonstrate a similar trend as a group analysis, our results identified that PRO-JOA and JOA scores should be regarded as different outcomes.

The JOA score is a simple scoring measure for cervical myelopathy, consisting of 6 domains with a total score of 17 points, and its recovery rate has been widely used. The validity of the JOA score has been reported by comparing with other scales for cervical myelopathy, such as European myelopathy scale, Cooper myelopathy scale, and Short Form–36 (SF-36) PCS.[8,9] Similarly, the modified JOA (mJOA) score has been compared with the Nuric grade, NDI, and SF-36.[10,11] Moreover, JOA score has been reported to be strongly correlated with mJOA score, although they were not considered to be interchangeable according to the Bland-Altman analysis. The MCID of the JOA score and its recovery rate has been reported to be 2.5 and 52.8%, respectively, whereas that of mJOA was approximately 2. Because both JOA and mJOA scores are not PROs, namely, they are scored by physicians and can be biased by investigators and may not represent patient satisfaction as much as other PROs. However, owing to their feasibility, JOA and mJOA scores are widely used in Asian and Western countries, respectively.

Because the JOA score, as well as mJOA score, is detected by physicians, patients are required to visit the hospital and see the doctors face-to-face for them to provide the scores. Conversely, PROs utilize questionnaires so that patients need not visit the hospital or see doctors directly, which may lead to the objectiveness of the scores compared. In this study, we showed that the PRO-JOA score can also be used as a disease-specific scoring measure, because the JOA score was moderately to strongly correlated with PRO-JOA scores, which are also both further moderately correlated with other PROs. Compared with the JOA score, the PRO-JOA score showed slightly lower postoperative values and significantly lower recovery rates. We speculate that the results might have represented doctors’ bias that they tend to expect better results postoperatively. Nevertheless, the AUC of both JOA and PRO-JOA scores was approximately 0.70, indicating that both scores represented patients’ satisfaction to some extent.

Conversely, data with nearly identical statistical properties is often different when graphed. Indeed, Bland-Altman analysis indicated large limits of agreement by PRO-JOA and JOA scores, although they showed strong correlations. Therefore, they should be regarded as different outcome measures. Remarkably, as both measures demonstrate a similar trend as a group analysis, the PRO-JOA score may be used as one of PROs instead of JOA score. PRO-JOA score can be used, for example, to determine the neurological status of patients who reside distantly from the hospital and cannot visit their physicians postoperatively. Moreover, a detailed interpretation of patients whose JOA and PRO-JOA scores show discrepancy may provide valuable information. Taken together, the PRO-JOA score can be used to measure outcomes in patients with cervical myelopathy.

In this study, the MCID was calculated for JOA and PRO-JOA scores. The MCID of the JOA score was 2.5, which was compatible with those of reports. Conversely, the recovery rate of the JOA score has been more commonly used as a parameter of postoperative improvement of cervical myelopathy than MCID. Strictly speaking, the cutoff value of the recovery rate calculated using satisfaction as an anchor should be known as the minimum clinically important recovery rate, but we followed a past report that called it MCID of the JOA recovery rate. Nevertheless, because the recovery rate of 50% has been used as a treatment goal, MCID of 42% in the JOA score recovery rate was considered reasonable. On the contrary, the MCID for the PRO-JOA recovery rate was calculated to be 33%. Therefore, we should keep in mind that the recovery rate of the PRO-JOA score tends to be lower than that of the JOA score. Once again, the recovery rates of JOA and PRO-JOA scores should be considered distinct with each other.

Several limitations have been detected in this study. First, reliability and validation analyses were not sufficient. For example, a multitrait multimethod analysis would have strengthened the results of this study. Moreover, we did not perform the test-retest reproducibility analysis. Therefore, the PRO-JOA score validation might be insufficient, although we considered that JOA score is an established outcome and well validated before. Second, MCIDs can vary according to the patient population or surgical procedures, although we selected an anchor-based analysis that has been commonly used to calculate MCID. Third, imaging parameters were not taken into account. Fourth, the existence of depression was not investigated and might have affected the results of this study, although a previous report showed that the involvement of depression was not significant in patients with degenerative cervical myelopathy. Despite these problems, we believe that the PRO-JOA score can be used as one of the disease-specific PROs for degenerative cervical myelopathy. Further investigations will elucidate these problems.

In conclusion, PRO-JOA score can be used as a disease-specific scoring measure instead of JOA score, although PRO-JOA and JOA scores should not be used interchangeably.