RELIABILITY OF A BRAZILIAN PORTUGUESE TRANSLATED AND CROSS-CULTURALLY ADAPTED VERSION OF THE MJOA SCALE.

OBJECTIVE: To assess the intra- and inter-observer reliability of a Brazilian Portuguese translated and cross-culturally adapted version of the mJOA questionnaire.
METHODS: The reliability of the Brazilian Portuguese version of the mJOA scale was assessed through the evaluation of a sample of patients with cervical myelopathy by two independent experienced spine surgeon examiners. Inter-observer reliability was defined by the Intraclass Correlation Coefficient (ICC) between the evaluations of the two examiners, and intra-observer reliability was assessed by the ICC between the two evaluations of one examiner.
RESULTS: Fifty-five patients were included in the study (mean age 58.7 years). The ICC for inter-observer reliability of the Brazilian Portuguese version of the mJOA was 0.967, and the ICC for intra-observer reliability was 0.869, both classified as "almost perfect" (> 0.81).
CONCLUSION: The Brazilian Portuguese translated and cross-culturally adapted version of the mJOA questionnaire appears to be valid and reliable. Level of evidence I, Diagnostic Studies, Investigating a Diagnostic Test.

INTRODUCTION

Cervical myelopathy is a common source of disability associated with spinal disease, especially in the elderly population. Spondylotic spinal cord compression is the primary cause of cervical myelopathy – and may present with different degrees of neurological compromise, leading to a range of levels of disability. Due to such variation in the degree of neurological involvement and severity of incapacity, grading scales have been developed to objectively evaluate the neurological compromise and the severity of the disease, as well as the outcomes of surgical treatment.

The scale proposed by the Japanese Orthopaedic Association (JOA) to assess the severity of cervical myelopathy has been translated into English and cross-culturally adapted to a Western population, including replacing references to chopsticks to spoons. As a means of broadening the global adoption of spine outcomes tools originally developed in English, the modified Japanese Orthopaedic Association (mJOA) questionnaire was translated and adapted into Dutch. Recently, the same protocol was applied to produce a version of the mJOA translated and cross-culturally adapted to Brazilian Portuguese. Such translations are important since they enable application of this objective clinical instrument to broader populations of patients.

The aim of the present study was to assess the intra- and inter-observer reliability of the Brazilian Portuguese version of the mJOA questionnaire.

MATERIALS AND METHODS

Design, participants, and ethics

This is a cross-sectional study. Prior to study initiation, the protocol was approved by the Institutional Review Board (CAAE: 52578015.1.0000.5463). Written consent was obtained from all study subjects prior to enrollment. In order to analyze the validity of the Brazilian Portuguese translated and cross-culturally adapted version of the mJOA, the intra- and inter-observer reliabilities were assessed by applying this version to a series of patients with cervical spondylotic myelopathy (CSM). Patients presenting for clinical evaluation in the authors’ spine clinics in Brazil between August of 2016 and October of 2016 were considered eligible for study participation if their clinical complaint or abnormality was suggestive of cervical myelopathy. The exclusion criteria were any factors that could compromise effective communication, the presence of any other known neurologic or psychiatric condition that could affect the clinical presentation, and those who declined study participation. Based on previously reported study with similar design, it was estimated that evaluation of 50 patients would be sufficient for the present study.

Variables and measurements

Initially, each patient was independently evaluated and scored based on the translated version of the mJOA by two experienced spine surgeons (RRP and CFPSH). In a second appointment, each patient was re-evaluated and scored by one of the spine surgeons (RRP). The total overall score of the mJOA and the individual scores for each of the four questions were assessed. The intra-observer reliability was determined by comparing the scores obtained in the two evaluations by examiner RRP, and the inter-observer reliability was calculated by comparing the scores of the two examiners (RRP and CFPSH) at the initial evaluation.

Statistical Analysis

The intra- and inter-observer reliability of the Brazilian version of the mJOA were quantified using the Intraclass Correlation Coefficient (ICC), with a confidence interval (CI) of 95%. ICC values of 0.00 to 0.20 were considered slight agreement, 0.21 to 0.40 fair agreement, 0.41 to 0.60 moderate agreement, 0.61 to 0.80 substantial agreement, and 0.81 to 1.00 almost perfect agreement. To assess for possible trends on the examiner evaluations, the investigators also evaluated the Bland-Altman plot of total score values. The statistical analysis was performed with IBM SPSS Statistics software program, version 20 (SPSS, Inc., Somers, NY, USA).

RESULTS

55 patients met criteria and were included in the present study, including 22 women and 33 men. All patients screened and found to be eligible for the study consented to participate and were included in the analysis, with no missing data. The mean patient age was 58.7 years (standard deviation: ±9.3 years; median: 58 years and range: 31-76 years). Table 1 provides a summary of the values obtained for the total mJOA score and each question of the mJOA, including the two evaluations of investigator 1 and the evaluation of investigator 2.

Table 1

Summary of the values obtained for the total mJOA score and each question of the mJOA, including the two evaluations of investigator 1 and the evaluation of investigator 2.

mJOA values	Investigator 1		Investigator 2
	1st Evaluation	2nd Evaluation
Total
Mean (SD)	14,1 (3,1)	14,4 (2,8)	14,1 (3,1)
Median (min.; max.)	15 (3; 18)	15 (6; 18)	14 (2; 18)
Motor Dysfunction Score of Upper Extremities
Mean (SD)	4,1 (1,1)	4,3 (0,9)	4,1 (1)
Median (min.; max.)	4 (1; 5)	5 (2; 5)	4 (1; 5)
Motor Dysfunction Score of Lower Extremities
Mean (SD)	5 (1,6)	5,2 (1,6)	5,1 (1,6)
Median (min.; max.)	5 (0; 7)	6 (1; 7)	5 (0; 7)
Sensation
Mean (SD)	2,2 (0,8)	2,2 (0,7)	2,2 (0,8)
Median (min.; max.)	2 (0; 3)	2 (0; 3)	2 (0; 3)
Sphincter Dysfunction Score
Mean (SD)	2,7 (0,7)	2,7 (0,7)	2,7 (0,7)
Median (min.; max.)	3 (0; 3)	3 (0; 3)	3 (0; 3)

Inter-observer Reliability

For the Brazilian version of the mJOA, both the reliability for the total score and for each question were classified as “almost perfect” (> 0.81) (Figure 1). The ICC obtained for the mJOA total score was 0.967 (95% CI: 0.944-0.981), 0.943 (95% CI: 0.904-0.966) for “Motor dysfunction score of the upper extremities”, 0.943 (95% CI: 0.903-0.966) for “Motor dysfunction score of the lower extremities”, 0.868 (95% CI: 0.784-0.921) for “Sensory dysfunction score of the upper extremities” and 0.961 (95% CI: 0.933-0.977) for “Sphincter dysfunction score”. The Bland-Altman plot for the total score did not suggest any trends between the evaluations of the two examiners (Figure 1).

Figure 1

Bland-Altman plot for the total score between the evaluations of the two examiners (inter-observer reliability).

Intra-observer reliability

The intra-observer reliability for the mJOA total score was also classified as “almost perfect,” with an ICC of 0.869 (95% CI: 0.784-0.921) (Table 2). Each question of the mJOA had at least “substantial” (> 0.70) intra-observer reliability. The ICC was 0.786 (95% CI: 0.657-0.870) for “Motor dysfunction score of the upper extremities”, 0.897 (95% CI: 0.829-0.939) for “Motor dysfunction score of the lower extremities”, 0.726 (95% CI: 0.572-0.830) for “Sensory dysfunction score of the upper extremities” and 0.775 (95% CI: 0.643-0.863) for “Sphincter dysfunction score”. The Bland-Altman plot for the total score did not suggest any trends between the different evaluations by the same examiner (Figure 2).

Figure 2

Bland-Altman plot for the total score between the different evaluations by the same examiner (intra-observer reliability).

DISCUSSION

The value of translating and cross-culturally adapting a clinical assessment scale into different languages is to encourage broader application and to help standardize and facilitate the exchange of information within the clinical and scientific communities. To help stimulate the global adoption of the mJOA cervical myelopathy score assessment tool, it was translated and adapted to Dutch. Following a similar systematic, standardized approach as was used to generate the Dutch version, the mJOA was recently translated and cross-culturally adapted to Brazilian Portuguese. However, after translating a clinical assessment tool to a different language, it is important to assess the reliability of the new version. , , The present study provides the reliability assessment of the translated and cross-culturally adapted to Brazilian Portuguese mJOA questionnaire, demonstrating strong intra- and inter-observer reliability. This translated version was tested in a sample of patients with a clinical complaint or abnormality suggestive of cervical myelopathy and two experienced spine surgeons scored these patients. For the Brazilian mJOA overall score, the reliability obtained in the present study could be considered as “almost perfect” and is at least as favorable as that reported for the Dutch version. The high degree of reliability suggests that the Brazilian version is a consistent measurement tool for severity in cervical myelopathy. One major advantage of the mJOA is that motor function of the legs, motor function of the arms, sensation of the hands, and micturition are scored separately through four different questions. In addition to the total score (sum of the scores for each of the four questions), the present study also assessed the reliability for each question. Although the questions on the mJOA could have the potential for subjective interpretation, there was strong reliability for the overall score and for each question. That the mJOA is scored by physicians based on patient evaluation, instead of being a patient-reported measure, may help to explain the strong reliability found in the study. Despite being classified as “substantial”, the intra-observer reliabilities for both the total score and for each question were somewhat lower than for the inter-observer reliabilities. It is possible that during the time interval between the two evaluations for the intra-observer reliability assessment, the patients’ complaints and findings may have progressed or changed. It is also possible that the examiners’ subjective interpretation of the patients’ complaints and findings may have differed between the two evaluations. This influence of the natural symptom fluctuation associated with the time memory effect has been previously discussed in the literature. The major limitation of the present study was that it only assesses the reliability of the translated and cross-culturally adapted to Brazilian Portuguese version of the mJOA score. There remains a lack of validation for the mJOA questionnaire in general, regardless of language, with regard to its effectiveness as a health-related quality of life instrument, despite its general recognition and acceptance as a cervical myelopathy severity tool.

CONCLUSION

In line with the need for international standardization of spine outcomes instruments, the present study demonstrated that the translated and cross-culturally adapted to Brazilian Portuguese version of the mJOA questionnaire is reliable as a cervical myelopathy severity tool.