Japanese Orthopaedic Association Back Pain Evaluation Questionnaire. Part 3. Validity study and establishment of the measurement scale : Subcommittee on Low Back Pain and Cervical Myelopathy Evaluation of the Clinical Outcome Committee of the Japanese Orthopaedic Association, Japan.

BACKGROUND: The Japanese Orthopaedic Association decided to revise the JOA score for low back pain and to develop a new outcome measure. In February 2002, the first survey was performed with a preliminary questionnaire consisting of 60 evaluation items. Based on findings of that survey, 25 items were selected for a draft of the JOA Back Pain Evaluation Questionnaire (JOABPEQ). The second survey was performed to confirm the reliability of the draft questionnaire. This article further evaluates the validity of this questionnaire and establishes a measurement scale.
METHODS: The subjects of this study consisted of 355 patients with low back disorders of any type (201 men, 154 women; mean age 50.7 years). Each patient was asked to fill in a self-administered questionnaire. Superficial validity was checked in terms of the completion rate for filling out the entire questionnaire. Factor analysis was then performed to evaluate the validity of the questionnaire and establish a measurement scale.
RESULTS: As a result of the factor analysis, 25 items were categorized into five factors. The factors were named based on the commonality of the items: social function, mental health, lumbar function, walking ability, and low back pain. To establish a measurement scale for each factor, we determined the coefficient for each item so the difference between the maximum factor scores and minimum factor scores was approximately 100. We adjusted the formula so the maximum for each factor score was 100 and the minimum was 0.
CONCLUSIONS: We confirmed the validity of the JOA Back Pain Evaluation Questionnaire and established a measurement scale.

Introduction

The evaluation criteria were based on physiological, biological, and anatomical outcome measure results of the Japanese Orthopaedic Association (JOA) score for low back pain.1 The criteria include laboratory values, physiological findings, and imaging findings. These findings are significant for doctors but have little meaning for patients. From a patient’s perspective, the presence of a symptom or its degree and functional condition must have real meaning. This means that outcome measures need to be translatable from an objective evaluation to a subjective one, or from the doctor’s perspective to the patient’s perspective. The JOA decided to revise the JOA score for low back pain and develop a new scientific, patient-oriented outcome measure.

The first committee meeting was held in June 2000, and the first survey was initiated in February 2002 using a preliminary questionnaire consisting of 60 items. It was a self-administered, disease-specific measure developed with reference to the Japanese editions of SF-362,3 and the Roland-Morris Disability Questionnaire (RDQ)4,5 to assess health-related quality of life. Based on findings of the survey, 25 items were selected for a draft of the JOA Back Pain Evaluation Questionnaire (JOABPEQ) (see Appendix 1).

The second survey was started in January 2004 to evaluate the reliability of the 25 items selected for the draft JOABPEQ. We successfully confirmed the reliability, and these details have been described in previous reports of Part 16 and Part 2.7 Part 3 of this study involves further development of the new JOA questionnaire, evaluation of the validity of the draft JOABPEQ, and establishment of a measurement scale.

Materials and methods

Recruitment of patients

A total of 369 of the 829 Japanese board-certified spine surgeons were randomly selected and asked to recruit at least three patients each to participate in evaluating the JOABPEQ during February 2004. The inclusion criterion was any type of lumbar spine disorder. Exclusion criteria were patients who had:

Other musculoskeletal diseases requiring medical treatment

Psychiatric disease, potentially leading to inappropriate answers

Postoperative condition

Participation in previous surveys related to this study

Testing the questionnaire

Each patient was asked to fill in the self-administered questionnaire. The attending surgeon filled out information on the diagnosis, presence or absence of concomitant diseases, and a judgment regarding the severity of symptoms using a three-step rating scale (mild, moderate, severe). The severity of the symptoms was determined subjectively by the attending surgeon, who was asked not to select a similar patient based only on the severity. This study was approved by the Ethics Committee of the Japanese Society for Spine Surgery and Related Research, and informed consent was obtained from each patient.

Factor analysis was used to check the statistical validity of the questionnaire and establish the measurement scale. All statistics were calculated using SPSS software (version 12; SPSS, Chicago, IL, USA).

Results

Patient characteristics

Of the 452 patients selected for participation in this survey, 1 patient who was judged inappropriate by the attending doctor and 60 patients with other musculoskeletal diseases requiring medical treatment were excluded. The responses from 36 patients who answered incompletely were also excluded, leaving 355 patients available for analysis: 201 men and 154 women, with a mean ± SD age of 50.7 ± 18.0 (Table 1). The diagnosis was lumbar disc herniation in 167, lumbar spinal canal stenosis in 103, and spondylolisthesis in 37.

Table 1

Distribution of age and severity of symptoms (n = 355)

	Severity of symptoms
Age (years)	Mild	Moderate	Severe	Total
Males
10–19	3	4		7
20–29	8	4	9	21
30–39	12	14	6	32
40–49	12	14	8	34
50–59	12	9	11	32
60–69	10	12	12	34
70–79	11	16	8	35
80+	4	2		6
Total	72	75	54	201
Females
10–19	0	1	3	4
20–29	9	5	5	19
30–39	7	13	10	30
40–49	5	14	7	26
50–59	4	9	5	18
60–69	7	12	7	26
70–79	11	10	4	25
80+		3	3	6
Total	43	67	44	154
Total no.	115	142	98	355

According to the judgment of the attending doctor, there were 115 mild, 142 moderate, and 98 severe cases. Table 2 summarizes the severity of low back pain evaluated by the current JOA scoring system and shows that the characteristics of the recruited patients were not specific. There was no marked difference in the distribution of the severity levels between the 451 patients who were initially recruited and the 355 who were finally analyzed.

Table 2

Distribution of the severity evaluated by the current JOA scoring system and finger-floor distance (n = 355)

Parameter	No.
Straight-leg raising (SLR) test
Normal	183
30°–70°	130
<30°	42
Motor function
Normal	182
Slight weakness (MMT good)	126
Severe weakness (MMT less than good)	47
Sensory function
Normal	127
Slight disturbance	162
Severe disturbance	66
Bladder function
Normal	315
Mild dysuria	36
Severe dysuria	4
Finger-to-floor distance (cm)
to −15	1
−14 to 5-5	12
−4 to 4	69
5 to 14	73
15 to 24	69
25 to 34	43
35 to 44	25
45 to 54	30
55 to 64	6
65 to 74	4
Not measurable	14
Total number	355

JOA, Japanese Orthopaedic Association; MMT, manual muscle testing

Superficial validity

Superficial validity was checked in terms of the completion rate for filling out the questionnaire. Regarding the distribution of responses for each item, it was judged that none of the questions was too difficult to answer because the highest rate of nonrespose was 1.8%. With regard to deflection of an answer, the highest rate (78.3%) was concentrated on “yes” responses to question 1–14, although this was judged not to be inappropriate. Therefore, the distribution was not skewed, which would indicate “floor and ceiling” effects (Table 3).

Table 3

Distribution of answers for each item in the questionnaire (n = 451)

	Choices for answer
Item	1	2	3	4	5	No answer
Q1-1	336 (74.5%)	114 (25.3%)				1 (0.2%)
Q1-2	152 (33.7%)	297 (65.9%)				2 (0.4%)
Q1-3	302 (67.0%)	146 (32.4%)				3 (0.7%)
Q1-4	157 (34.8%)	291 (64.5%)				3 (0.7%)
Q1-5	242 (53.7%)	209 (46.3%)				0
Q1-6	167 (37.0%)	281 (62.3%)				3 (0.7%)
Q1-7	215 (47.7%)	236 (52.3%)				0
Q1-8	240 (53.2%)	208 (46.1%)				3 (0.7%)
Q1-9	272 (60.3%)	177 (39.2%)				2 (0.4%)
Q1-10	288 (63.9%)	160 (35.5%)				3 (0.7%)
Q1-11	158 (35.0%)	292 (64.7%)				1 (0.2%)
Q1-12	156 (34.6%)	286 (63.4%)				9 (2.0%)
Q1-13	116 (25.7%)	333 (73.8%)				2 (0.4%)
Q1-14	353 (78.3%)	90 (20.0%)				8 (1.8%)
Q2-1	4 (0.9%)	27 (6.0%)	155 (34.4%)	185 (41.0%)	79 (17.5%)	1 (0.2%)
Q2-2	103 (22.8%)	233 (51.7%)	113 (25.1%)			2 (0.4%)
Q2-3	126 (27.9%)	253 (56.1%)	67 (14.9%)			5 (1.1%)
Q2-4	181 (40.1%)	175 (38.8%)	95 (21.1%)			0
Q2-5	62 (13.7%)	111 (24.6%)	206 (45.7%)	48 (10.6%)	23 (5.1%)	1 (0.2%)
Q2-6	113 (25.1%)	124 (27.5%)	138 (30.6%)	50 (11.1%)	23 (5.1%)	3 (0.7%)
Q2-7	53 (11.8%)	66 (14.6%)	225 (49.9%)	72 (16.0%)	35 (7.8%)	0
Q2-8	52 (11.5%)	76 (16.9%)	224 (49.7%)	75 (16.6%)	23 (5.1%)	1 (0.2%)
Q2-9	11 (2.4%)	57 (12.6%)	190 (42.1%)	132 (29.3%)	60 (13.3%)	1 (0.6%)
Q2-10	64 (14.2%)	125 (27.7%)	114 (25.3%)	102 (22.6%)	45 (10.0%)	1 (0.2%)
Q2-11	48 (10.6%)	149 (33.0%)	141 (31.3%)	89 (19.7%)	23 (5.1%)	1 (0.2%)

Factor analysis

First, we tried to extract some observed variables from 25 items by the Maximum Likelihood Method. It was found that the eigenvalue was >1.0 for five items, and the accumulative contribution ratio until the fifth factor was 53.1% (Table 4).

Table 4

Results of factor analysis: eigenvalue of each item

Factor	Eigenvalue	Cumulative contribution rate (%)
1	7.600	30.4
2	1.795	37.6
3	1.556	43.8
4	1.217	48.7
5	1.095	53.1
6	0.996	57.0
7	0.942	60.8
8	0.893	64.4
9	0.783	67.5
10	0.756	70.5
11	0.728	73.4
12	0.680	76.2
13	0.656	78.8
14	0.643	81.4
15	0.617	83.8
16	0.584	86.2
17	0.505	88.2
18	0.482	90.1
19	0.433	91.9
20	0.427	93.6
21	0.387	95.1
22	0.361	96.6
23	0.320	97.8
24	0.302	99.0
25	0.239	100.0

Bold typeface indicates eigenvalues over 1.0

Next, we performed orthogonal rotation by the direct oblimin method. The results are shown in Table 5. Each item was categorized into five factors: Four items (Q2- 6, Q2-5, Q1-2, Q2-4) related to factor 1; seven items (Q2-8, Q2-7, Q2-11, Q1-13, Q2-9, Q2-10, Q2-1) related to factor 2; six items (Q1-9, Q1-6, Q2-3, Q1-8, Q1-5, Q1-4) related to factor 3; five items (Q1-10, Q2-4, Q1- 12, Q1-14, Q2-2) to factor 4; and the last four items to factor 5. Although factor loading was <0.30 in three items (Q1-4 to factor 3, Q2-2 to factor 4, Q1-11 to factor 5), we adopted all of them for the reason that the question itself was important for the factor or the number of questions in each factor needed to be more than four.

Table 5

Results of factor analysis: factor loading of each item

	Factors
Item	1	2	3	4	5
Item	1	2	3	4	5
Q2-6	0.81	0.04	0.04	−0.04	0.14
Q2-5	0.71	0.01	0.08	0.14	0.06
Ql-2	0.33	0.16	0.21	0.34	−0.21
Q2-8	0.07	0.68	0.08	−0.08	0.10
Q2-7	0.15	0.62	−0.07	0.12	0.15
Q2-11	−0.03	0.62	−0.12	−0.02	0.06
Ql-13	−0.04	0.35	0.08	−0.01	0.14
Q2-9	−0.23	−0.52	−0.11	0.05	0.05
Q2-10	0.06	−0.55	−0.06	−0.10	0.15
Q2-1	0.03	−0.55	−0.14	−0.11	−0.02
Ql-9	0.02	−0.07	0.69	−0.07	0.10
Ql-6	−0.01	0.12	0.56	0.08	−0.10
Q2-3	0.23	0.05	0.56	−0.04	0.07
Ql-8	−0.03	−0.09	0.38	0.15	0.31
Ql-5	0.00	0.09	0.32	0.03	−0.02
Ql-4	0.10	0.11	0.28	0.13	0.05
Ql-10	0.14	0.04	−0.04	0.62	0.03
Q2-4	0.39	0.05	−0.08	0.61	−0.13
Ql-12	−0.05	0.01	0.01	0.46	0.06
Ql-14	−0.07	0.06	0.20	0.34	0.13
Q2-2	0.30	−0.03	0.19	0.26	0.05
Ql-1	0.03	0.11	0.00	0.04	0.46
Ql-3	0.18	0.13	−0.05	0.13	0.43
Ql-7	0.07	0.06	0.20	0.01	0.41
Ql-11	0.10	0.04	0.16	0.25	0.28

Bold typeface indicates absolute value of the factor loading of more than 0.26

Factor names were determined based on the commonality of the items that showed a large value on factor loading: factor 1, social function (four items); factor 2, mental health (seven items); factor 3, lumbar function (six items); factor 4, walking ability (five items); and factor 5, low back pain (four items).

Measurement scale

To establish a measurement scale for each factor, we determined the size of the coefficient for each item so the difference between the maximum factor scores and minimum factor scores was approximately 100 (Table 6). When a coefficient became a negative numerical value, we changed the coefficient to a positive numerical value by reversing the order of the answer choice. We adjusted the formula so the maximum for each factor score was 100 and the minimum was 0 (see Appendix 2).

Table 6

Coefficient for each item of the formula for measurement scale

Item	1 Social function	2 Mental health	3 Lumbar function	4 Walking ability	5 Low back pain
Ql-1					20
Ql-2	2
Ql-3					20
Ql-4			10
Ql-5			10
Ql-6			20
Ql-7					20
Ql-8			10
Ql-9			30
Ql-10				30
Ql-11					10
Ql-12				20
Ql-13		3
Ql-14				10
Q2-1		−4
Q2-2				10
Q2-3			20
Q2-4	4			30
Q2-5	6
Q2-6	10
Q2-7		6/
Q2-8		6
Q2-9		−3
Q2-10		−3
Q2-11		3

Discussion

It is considered ideal for the outcome measure to evaluate patients from various perspectives, such as dysfunction, disability, handicap, and psychological problem. The outcome measure should be patient-oriented, and its reliability and validity should be confirmed by statistical analysis. However, the current JOA score does not include subjective evaluations and does not meet such requirements. We developed a new questionnaire, JOABPEQ, specifically to evaluate low back pain. It is patient-oriented and mainly based on recognizing problems with activities of daily living. We categorized 25 questions into five factors; each factor is then scored up to 100 points using the measurement scale. The factors are then evaluated separately. The point is to be aware that it is meaningless and inadequate to total the five factors’ scores; rather, they should be treated by nonparametric analysis. The reliability of the questionnaire including 25 items for the JOABPEQ was confirmed in Part 2 of this project. The validity of the questionnaire was evaluated using factor analysis, and the measurement scale was established in Part 3 of this study. Further studies must be performed to confirm the responsiveness of the calculations of the severity score.

Conclusions

We confirmed the validity of the JOA Back Pain Evaluation Questionnaire (JOABPEQ) and established a measurement scale.