Cross-Cultural Adaptation and Psychometric Validation of the Dutch Version of the Core Outcome Measures Index for the Neck in Patients Undergoing Surgery for Degenerative Disease of the Cervical Spine.

OBJECTIVE: To perform the psychometric validation of the Dutch version of the Core Outcome Measures Index (COMI) for the neck.
METHODS: A total of 178 patients, who had an indication for surgery due to degenerative cervical spinal disease, were enrolled in the study. They filled in a baseline booklet containing the Dutch version of the COMI-neck, Likert-scales for neck and arm/shoulder pain, the Neck Disability Index (NDI), the EuroQol-5 dimensions (EQ-5D) and the 12-item Short Form health survey (SF-12). Aside from analyzing construct validity using the Spearman correlation test, test-retest reliability, and responsiveness at 3 months were assessed using the intraclass correlation coefficient (ICC) and the receiver-operating characteristic (ROC) curve, respectively.
RESULTS: The COMI-neck showed good acceptability with missing data ranging from 0% to 4.5% and some floor/ceiling effects for 3 of the domains at baseline. The COMI-summary score showed good to very good correlation with the EQ5D (ρ = -0.43), the physical component summary of the SF-12 (ρ = -0.47) and the NDI (ρ = 0.73). Individual domains showed correlations of -0.28 to 0.85 with the reference questionnaires. Test-retest reliability analysis showed an ICC of 0.91 with a minimal detectable change of 1.7. Responsiveness analysis of the COMI-neck showed an area under 0.79 under the ROC-curve. The standardized response mean for a good outcome was 1.24 and for a poor outcome 0.37.
CONCLUSION: The current study shows that the Dutch version of the COMI-neck is a valid, reliable and responsive Patient-Reported Outcome Measure, among patients undergoing surgery for degenerative cervical spinal disorders.

INTRODUCTION

Surgery for degenerative cervical spinal disorders is frequently performed by spine surgeons [1]. Similar to the variation of the surgical techniques applied by surgeon, outcomes of these procedures may also vary per surgeon or center. To monitor the quality of the care given and measure the burden of disease, outcomes of patients and procedures are increasingly being collected by clinicians [2]. Aside from surgical outcomes such as complications and intraoperative blood loss, clinical outcomes of patients, are deemed to be of more importance in tracking outcomes [3].

Patient-Reported Outcome Measures (PROMs) are the outcomes of choice when outcome domains as function, quality of life or pain, are assessed in patients undergoing cervical spine surgery [4]. As assessing these domains through different questionnaires can be time-consuming for both patients and clinicians, there is a necessity for a brief PROM to assess these patient outcomes. With the intention to address this issue, the core outcomes measures index (COMI) was developed as a brief PROM that measures outcome on 5 domains [4,5]. Separate versions of the COMI exist to assess outcomes for the lower back and neck. Previous studies have extensively studied the psychometric properties of the COMI-back and COMI-neck in different languages and showed that the COMI is a valid and reliable PROM to measure outcomes in patients with spinal disorders [6-9]. A validation of the COMI-neck in the Dutch language, however, had yet to be conducted. Therefore, the purpose of the current study was to perform a psychometric validation of the COMI-neck in Dutch language among patients with degenerative cervical spinal disorders.

MATERIALS AND METHODS

1. COMI-Neck and Translation Process

The COMI-neck is a questionnaire containing 7 questions, testing 5 outcome domains [8]. These domains are pain, function, symptom-specific well-being, quality of life, and disability. Aside from these domains, a summary score can be calculated in which a “0” indicates the “best score” and “10” the “worst score.”

In a previous study, the COMI-back was cross-culturally adapted and validated in Dutch [10,11]. For the current study, 2 researchers (1 laymen and 1 researcher in spine surgery) made adaptations to the Dutch version of the COMI-back, independently. These adaptations include changing “back” to “neck,” “leg pain/buttock pain” to “arm/shoulder pain” and “back problem” to “neck problem.” The remaining wording and structure of the COMI-back was retained. Both adapted versions were discussed by both researchers and a neurosurgeon and lead to the final version of the COMI-neck. This study was approved by the Institutional Review Board of Erasmus University Medical Center (approval number: MEC-2016-261) before the involvement of patients.

2. Patients

To validate the Dutch version of the COMI-neck, patients were included at 1 secondary and 1 tertiary referral centers for spine surgery in the Netherlands from January 2016 to January 2018. Patients were informed about the study after their consultation with the neurosurgeon and were subsequently approached by mail to be enrolled. To be included in this study, patients needed to (1) have age above 18 years or above; (2) have an indication for surgery due to cervical radiculopathy or myelopathy caused by degenerative disease; and (3) have adequate knowledge of Dutch language to complete the forms. Patients were excluded if they had (1) psychiatric disorders; (2) would undergo surgery due to infectious or oncological causes and (3) had general exclusion criteria for surgery (e.g., pregnancy, high American Society of Anesthesiologists physical status classification).

3. Study Procedures

After providing written informed consent, patients received a booklet containing the COMI-neck and other, prior validated, questionnaires. Five days after completing the baseline booklet and before undergoing surgery, a random selection of patients was approached to fill in the test-retest questionnaire. Finally, 3 months after surgery, another subset of patients were approached to fill in the responsiveness questionnaires. Besides the COMI-neck, the baseline booklet contained questions regarding demographics, Likert-scales on neck and arm pain, the Neck Disability Index (NDI), the EuroQol-5 dimensions (EQ-5D) and the 12-item Short Form health survey (SF-12). The retest and responsiveness questionnaires also contained a global treatment outcome (GTO) questionnaire in addition to the COMI-neck.

4. Outcomes

1) Demographics

Questions were included regarding baseline demographics. Data was collected on sex, marital status, age, employment, level of education, and intoxications.

2) Likert-scale pain

Two 5-point Likert-scales were added which measured neck and arm/shoulder pain. The Likert-scales for neck pain and arm/shoulder pain were tested for correlation with the corresponding COMI-domains on neck and arm/shoulder pain.

3) Neck Disability Index

The NDI is a PROM that measures functional disability due to neck disorders [12]. The NDI is expressed in a score ranging from “0” indicating “no disability” to “100” indicating “maximal disability.” The NDI was tested for correlation with the COMI-domains function, symptom-specific well-being, disability and the summary score.

4) EuroQol-5 dimensions

The EQ5D is a five-question PROM that measures health-related quality of life [13]. In addition to these 5 questions, a 0 to 100 visual analogue scale (VAS) is included in which patients can indicate their health status (EQ-5D-VAS). The domains symptom-specific well-being and quality of life were correlated to both EQ-5D-scores. Furthermore, the COMI-summary score was tested for correlation with the EQ-5D.

5) 12-item Short Form health survey

The SF-12 is a 12-question PROM focused on the General Health status [14]. The SF-12 measures outcomes on 8 domains, including General Health, Bodily Pain, and Physical Functioning among others. The scores on the different domains of the SF-12 can be transformed into physical component summary (PCS) and mental component summary (MCS) scores. The domain General Health was tested for correlation with the COMI-domain quality of life, while the domain Bodily Pain was tested for correlation with the domains neck and arm/shoulder pain. The PCS was tested for correlation with the domains function, disability, and the summary score.

6) Global Treatment Outcome

The GTO was a 5-point Likert-scale in which patients could indicate recovery. The 2 items indicating “full recovery” and “almost full recovery” were dichotomized as “recovered.” The other 3 options were categorized as “not recovered.”

5. Statistical Analysis

Descriptive statistics were used to demonstrate demographics. Missing data were handled as per instructions of the respective PROMs, with no missing data allowed for the COMI-neck. To validate the Dutch version of the COMI-neck, it was tested for (1) acceptability, (2) floor/ceiling effects, (3) construct validity, (4) test-retest stability, and (5) responsiveness. Acceptability was measured by depicting the percentages of missing data. Floor/ceiling effects were indicated if > 15% of the answers were the worst (floor) or best (ceiling) scores on the COMI-summary score or any of the COMI-domains. A floor/ceiling effect > 75% was considered to be detrimental. Construct validity was calculated using the Spearman ρ correlation test. A ρ lower than 0.21 indicated no correlation, a ρ between 0.21–0.40 indicated poor correlation, a ρ between 0.41–0.61 indicated a good correlation, a ρ between 0.61–0.81 indicated a very good correlation and a ρ≥ 0.81 indicated an excellent correlation [15].

Test-retest stability was measured by calculating the intraclass correlation coefficient (ICC) of the COMI score at baseline and approximately 5 days later. An ICC> 0.7 in a sample of at least 50 patients, indicated acceptable reproducibility [11]. In addition to the ICC, the standard error of measurement agreement (SEM) and minimum detectable change (MDC) were calculated. Responsiveness was tested by using a t-test comparing baseline and 3-month scores on the COMI-neck, calculating the standardized response mean (SRM) and by constructing a receiver-operating characteristic (ROC) curve anchored on the GTO. An area under the curve (AUC) of at least 0.70 on the ROC-curve was considered acceptable [11].

All analyses were conducted using IBM SPSS Statistics ver. 25.0 (IBM Co., Armonk, NY, USA) with a p-value of < 0.05 indicating statistical significance.

RESULTS

The Dutch version of the COMI-neck is shown in the Supplementary Material. During the study’s enrollment period, 178 patients filled in a baseline booklet. Of these patients 134 returned the COMI-neck approximately 5 days after inclusion and 86 returned the questionnaire 3-months after surgery (Fig. 1).

Fig. 1.

Flowchart of study procedures.

1. Demographics

Table 1 gives an overview of the baseline demographics of the enrolled patients. Patients had a mean age of 57.2±13.4 years, a mean body mass index of 26.8±5.2, and 52.8% was male. In general, majority of the patients were in a relationship (77.1%), had no (paid) job (55.6%) and followed middle education (50.2%). Furthermore, most used pain medication (59.6%), while cigarettes/tobacco, antidepressants, and muscle relaxants were used by 33.7%, 10.1%, and 8.4%, respectively. Baseline scores on the PROMS are also shown in Table 1. Of the 178 included patients, 175 underwent surgery because 3 patients with cervical radiculopathy recovered spontaneously while awaiting surgery. Of the remaining patients, 110 (62.9%) underwent posterior decompression due to cervical myelopathy, while 65 (37.1%) underwent anterior cervical discectomy with or without fusion due to cervical radiculopathy.

Table 1.

Baseline demographics of the included patients

Characteristic	Value
Age (yr)	57.2 ± 13.4
Sex	n = 178
Female	84 (47.2)
Male	94 (52.8)
Body mass index (kg/m2)	26.8 ± 5.2
Marital status	n = 175
Married/in a relationship	135 (77.1)
Single	40 (22.9)
Job status	n = 176
Paid job	78 (44.4)
No (paid) job	98 (55.6)
Educational level	n = 175
Lower education	82 (46.9)
Middle education	88 (50.2)
Higher education	5 (2.9)
Use of the following	n = 199
Cigarettes/tobacco	60 (33.7)
Antidepressants	18 (10.1)
Muscle relaxants	15 (8.4)
Pain medication	106 (59.6)
Patient-Reported Outcome Measure
Neck Disability Index	41.2 ± 18.1
Physical component summary	31.8 ± 7.1
Mental component summary	38.5 ± 9.9
EQ-5D	0.736 ± 0.087
COMI-summary score	6.6 ± 2.1
Procedure undergone	n = 178
Anterior cervical discectomy with or without fusion	65 (36.5)
Posterior decompression	110 (61.7)
None (due to spontaneous recovery)	3 (1.7)

Values are presented as mean±standard deviation or number (%).

EQ-5D, EuroQol-5 dimension; COMI, Core Outcome Measures Index.

2. Acceptability and Floor/Ceiling Effects

Table 2 gives an overview of the missing data at baseline and the floor/ceiling effects at baseline and 3-month follow-up. Missing data ranged from 0% to 4.5%, with 4.5% missing data for the COMI-summary score. At baseline, the COMI-domains symptom-specific well-being (70.1%), social disability (45.7%), and work disability (37.6%) had floor effects while the domain work disability showed ceiling effects (29.5%). At 3-month follow-up, floor effects were visible in the domains symptom-specific well-being (26.7%), social (24.7%) and work (27.3%) disability. Ceiling effects at 3-month follow-up were visible at all domains, except for quality of life, ranging from 17.8% to 42%. The COMI-summary score had no floor or ceiling effects at baseline or 3-month follow-up.

Table 2.

Floor and ceiling effects at baseline and at 3 months of follow-up

COMI-domain	Range	Missing at baseline	Baseline		3-Month follow-up
			Floor effects	Ceiling effects	Floor effects	Ceiling effects
Neck pain	0–10	3 (1.7)	3 (1.7)	21 (12.0)	0 (0)	24 (27.3)
Arm/shoulder pain	0–10	4 (2.2)	4 (2.3)	13 (7.5)	0 (0)	27 (31.4)
Function	0–10	0 (0)	25 (14.0)	12 (6.7)	7 (7.8)	16 (17.8)
Symptom-specific well-being	0–10	1(0.6)	124 (70.1)	5 (2.8)	24 (26.7)	18 (20.0)
Quality of life	0–10	1 (0.6)	16 (9.0)	4 (2.3)	1 (1.1)	8 (8.9)
Social disability	0–10	3 (1.7)	80 (45.7)	21 (12.0)	22 (24.7)	24 (27.0)
Work disability	0–10	5 (2.8)	65 (37.6)	51 (29.5)	24 (27.3)	37 (42.0)
Summary score	0–10	8 (4.5)	1 (0.6)	0 (0)	0 (0)	4 (4.7)

Values are presented as number (%).

COMI, Core Outcome Measures Index.

3. Construct Validity

The results of the correlation tests to assess construct validity are shown in Table 3. Excellent correlation was shown for the COMI-domain neck pain and the Likert-scale for neck pain (ρ=0.85), while the item arm/shoulder pain showed a very good correlation with the Likert-scale for arm/shoulder pain (ρ=0.80). The COMI-domain function showed good correlations with the NDI (ρ=0.58) and PCS (ρ=-0.41), while the domain symptom-specific well-being showed a good correlation with the NDI (ρ=0.41), but a poor correlation with the EQ-5D (ρ=-0.28). Furthermore, the domain quality of life showed a poor correlation with the EQ-5D (ρ=-0.40) and a good correlation with the NDI (ρ=0.51). The domains social and work disability both showed a poor correlation with the PCS (ρ=-0.39 and ρ=-0.33, respectively) and a good correlation with the NDI (ρ=0.48 and ρ=0.53, respectively). The overall, COMI-summary score showed a very good correlation with the NDI (ρ=0.73) and a good correlation with the EQ-5D (ρ=-0.43) and PCS (ρ=-0.47).

Table 3.

Construct validity

COMI-domain	Reference questionnaire	Spearman ρ
Neck pain	Likert neck pain	0.85[*]
Arm/shoulder pain	Likert arm/shoulder pain	0.80[*]
Function	NDI	0.58[*]
	PCS	-0.41[*]
Symptom-specific well-being	EQ5D	-0.28[*]
	NDI	0.41[*]
Quality of life	EQ5D	-0.40[*]
	NDI	0.51[*]
Social disability	NDI	0.48[*]
	PCS	-0.39[*]
Work disability	NDI	0.53[*]
	PCS	-0.33[*]
Summary score	EQ5D	-0.43[*]
	NDI	0.73[*]
	PCS	-0.47[*]

COMI, Core Outcome Measures Index; NDI, Neck Disability Index; PCS, physical component summary; EQ-5D, EuroQol-5 dimension.

p<0.05, statistical significance.

4. Test-Retest Reliability

The results of the test-retest reliability are shown in Table 4. All COMI-domains, except for symptom-specific well-being and quality of life, showed acceptable reproducibility with ICCs ranging from 0.78 to 0.90. The domains symptom-specific wellbeing and quality of life had an ICC of 0.66 and 0.65, respectively. The COMI-summary score had an ICC of 0.91 (95% confidence interval [CI], 0.86–0.94). Furthermore, the summary score had a SEM of 0.63 which leads to MDC 95% of 1.7.

Table 4.

Test-retest reliability

COMI-domain	Mean ± SD		ICC (95% CI)	SEM	MDC 95%
	1st measurement	2nd measurement
Neck pain	4.6 ± 2.9	4.9 ± 2.7	0.89 (0.83–0.93)	0.96	2.7
Arm/shoulder pain	5.6 ± 2.7	6.1 ± 6.5	0.90 (0.84–0.94)	0.85	2.4
Function	6.2 ± 2.8	6.0 ± 2.8	0.78 (0.67–0.86)	1.31	3.6
Symptom-specific well-being	8.6 ± 2.5	8.4 ± 2.5	0.66 (0.50–0.77)	1.46	4.0
Quality of life	6.0 ± 2.2	5.6 ± 2.0	0.65 (0.49–0.76)	1.30	3.6
Social disability	7.0 ± 3.5	6.5 ± 3.7	0.84 (0.76–0.90)	1.40	3.9
Work disability	5.5 ± 4.3	5.8 ± 4.1	0.88 (0.82–0.93)	1.49	4.1
Summary score	6.6 ± 2.1	6.6 ± 2.5	0.91 (0.86–0.94)	0.63	1.7

COMI, Core Outcome Measures Index; SD, standard deviation; ICC, intraclass correlation coefficient; CI, confidence interval; SEM, standard errors of measurement agreement; MDC, minimum detectable change.

5. Responsiveness

Three months after surgery, 58.3% of the patients were fully recovered according to the dichotomized GTO. Mean COMI-summary score at baseline (6.6±2.1) decreased to 4.2±2.7 at 3-month follow-up, leading to a mean difference of 2.29 with a standard deviation of 2.8 and thus a SRM of 0.81. The SRM for a good outcome was 1.24, while the SRM for a poor outcome was 0.37. The ROC-curve (shown in Fig. 2) had an AUC of 0.79 (95% CI, 0.69–0.89). The optimal cutoff point at the ROC-curve yielded a sensitivity of 100% and a specificity of 42%.

Fig. 2.

Receiver-operating characteristic (ROC) curve.

DISCUSSION

The current study is the first to construct a Dutch version of the COMI-neck and assess its’ psychometric properties. The Dutch version of the COMI-neck showed good acceptability with missing data for the individual domains ranging from 0% to 2.8%. As no missing data is accepted from the individual domains to produce the summary score, 4.5% was missing for the overall score. Three domains showed floor/ceiling effects at baseline, while 6 showed floor/ceiling effects at 3-month follow-up. However, no domains had detrimental floor/ceiling effects and the COMI-summary score had negligible floor/ceiling effect. Individual domains showed poor to excellent correlations with the reference questionnaires. In contrast, the COMI-summary score showed good to very good correlation with the EQ-5D, the PCS of the SF-12 and the NDI. Test-retest reliability analysis showed acceptable reliability for the COMI-summary score and all but 2 domains (symptom-specific well-being and quality of life). Responsiveness analysis of the COMI-neck showed an acceptable ability to discriminate between a good and a poor outcome.

Previously, translations and validations of the German, Italian, Polish, Turkish and Japanese versions of the COM-neck have been published (Table 5) [6,8,9,16,17]. These validations were conducted in both surgical and conservatively treated patients. All of these studies assessed floor/ceiling effects and construct validity. Test-retest reliability was previously researched in 4 versions while responsiveness was assessed in 3 previous validations.

Table 5.

Comparison with psychometric properties in other validated versions of the COMI-neck

Study	Language	Studied population	Floor/ceiling effects summary score	Construct validity summary score	Test-retest reliability	MDC 95% summary score	Responsiveness
Fankhauser et al. [8], 2012	German	Patients with degenerative disease of the cervical spine with an indication for surgery (n=89)	1.1% and 0 at baseline, 0 and 13.3% at follow-up	-0.60 (EQ-5D)	NR	NR	AUC
				0.7 (NASS-cervical disability)			(ROC)=0.95
							(0.88–0.99)
Monticone et al. [9], 2014	Italian	Patients with chronic neck pain undergoing rehabilitation (n=103)	0 at baseline and 0 at follow-up	NR	0.87 (0.81–0.91)	NR	AUC
							(ROC)=0.73
							(0.62–0.85)
Miekisiak et al. [17], 2014	Polish	Patients with neck pain, with or without radiating arm/shoulder pain, with or without an indication for surgery (n=123)	0.8% and 0.8% at baseline	0.65 (NDI)	0.88 (0.82–0.92)	1.97	NR
Karabicak et al. [16], 2020	Turkish	Patients with chronic neck pain from outpatient physiotherapy and rehabilitation clinics (n=106)	0 at baseline	0.70 (NDI)	0.96 (0.94–0.97)	NR	NR
				0.65 (NPDS)
				0.62 (VAS/NRS)
Oshima et al. [6], 2021	Japanese	Patients with an indication for surgery due to cervical spinal disorders (n=177) and volunteers with chronic neck pain (n=59)	1.2% and 0.6% at baseline, 0.1% and 5.3% at follow-up	-0.62 (EQ-5D)	0.97 (0.96–0.98)	0.77	AUC
				0.78 (NDI)			(ROC)=0.78
							MCID 2.1
Gadjradj et al. [10], 2021	Dutch	Patients with degenerative disease of the cervical spine with an indication for surgery (n=178)	0.6% and 0 at baseline, 0 and 4.7% at follow-up	-0.43 (EQ-5D)	0.91 (0.86–0.94)	1.7	AUC
				0.73 (NDI)			(ROC)=0.79
				-0.47 (PCS)			(0.69–0.89)

COMI, Core Outcome Measures Index; MDC, minimum detectable change; EQ-5D, EuroQol-5 dimension; NR, not reported; MCID, minimal clinically important difference; NPDS, Neck Pain and Disability Scale; AUC, area under the curve; ROC, receiver-operating characteristic; NASS, North American Spine Society; VAS, visual analogue scale; NRS, Numeric Rating Scale; NDI, Neck Disability Index; PCS, physical component summary.

Overall, floor/ceiling effects of the COMI-summary score were negligible at baseline and follow-up moments. Floor/ceiling effects of the domain symptom-specific well-being, is known to be present in the COMI-neck, but also at the COMI-back [7,8,10,17,18]. The findings of the present study are in line with this. Furthermore, construct validity and test-retest reliability of the present study was also comparable to those reported in previous validations. Responsiveness analyses showed that the current study had an AUC of 0.79 and a SRM of 1.4 for a good outcome at 3-month follow-up, which is comparable to the Japanese validation which was conducted among a similar study population but at 12-month follow-up [6]. The discriminative ability of these studies, however, was somewhat lower than in the German validation which assessed responsiveness at 3-month follow-up. The difference in study population might partially explain this difference as Fankhauser et al. [8] studied patients who were candidates for disc replacement surgery and had a mean age of 46 years, while the study population of the current and the study of Oshima et al. [6] was 11 to 18 years older and underwent surgery for mixed indications such as radiculopathy and/or myelopathy. Nevertheless, all these studies showed more than acceptable (AUC > 0.70) discriminatory abilities of the COMI-neck.

Some study limitations have to be acknowledged. Firstly, this version of the COMI-neck was formulated based on the Dutch version of the COMI-back, rather than the English version of the COMI-neck for pragmatic reasons [8,10]. This was decided because it would be less-time consuming and would lead to a version the COMI-neck which would be more conform the Dutch version of the COMI-back. Furthermore, the modifications from the Dutch version of the COMI-back would only be minor and previous studies have shown that this method would also lead to valid translations [9,17]. Another limitation might be that this version of the COMI-neck was only validated in a surgical population and might have other psychometric populations in conservatively treated patients. However, based on the literature and other studies validating the COMI-neck in conservatively treated patients (Table 5), the psychometric properties seem to be comparable. Another limitation might be that no selection based on GTO was applied, when test-retest reliability was tested. In the validation of the Dutch version of the COMI-back this ‘no change’- filter was applied when the retest questionnaire was distributed approximately 2 weeks after completing the baseline booklet. By shortening the retest period to 5 days, the filter expected to be unnecessary. Post hoc analyses confirmed this, as ICCs did not change significantly when this filter was applied. Finally, the GTO-question was based on a 5-point Likert-scale, whereas a 7-point Likert-scale may have been more appropriate. However, since the COMI-neck as used in the Spine Tango also contains a 5-point Likert-scale to assess GTO, the inclusion of a 5-point scale seemed more appropriate.

The current study shows that the Dutch version of the COMI-neck is a valid, reliable and responsive PROM. As the COMI-neck is a brief questionnaire and has appropriate psychometric properties, it can be recommended as a PROM among Dutch patients undergoing surgery for the degenerative cervical spine.