Minimal clinically important difference of improvement on the Arm Function in Multiple Sclerosis Questionnaire (AMSQ).

BACKGROUND: The Arm Function in Multiple Sclerosis Questionnaire (AMSQ) has been developed to assess upper extremity function of patients with multiple sclerosis (MS). A minimal clinically important difference (MCID) value has not been determined yet.
OBJECTIVE: The objective of this study is to determine an MCID for AMSQ.
METHODS: We used the sensitivity- and specificity-based approach with dichotomized global perceived effect as an anchor.
RESULTS: The receiver operating characteristic (ROC) curve yielded an optimal threshold value of 14.5 (sensitivity 0.68 and specificity 0.79). The area under the ROC curve value was 0.77.
CONCLUSION: We identified an MCID of 15 points for the AMSQ (range 31-186).

Introduction

Traditionally, the focus of clinical assessment in multiple sclerosis (MS) has been on ambulation. However, other domains are being increasingly assessed in conjunction with ambulation. This includes the assessment of upper extremity function (UEF). Various measures are available of which the 9-hole peg test (9HPT) is considered as the gold standard for manual dexterity.[1] Nevertheless, a performance-based measure, such as the 9HPT, does not provide any insight into the patient perspective of UEF. For this purpose, patient-reported outcome measures (PROMs) are valuable tools. To date, only one PROM is available that has been specifically developed to assess UEF in MS patients: the Arm Function in Multiple Sclerosis Questionnaire (AMSQ).[2] The AMSQ is a unidimensional 31-item questionnaire with good psychometric properties.[3]

However, a minimal clinically important difference (MCID) has not been reported yet. An MCID defines the smallest amount of change on a scale that is important or meaningful to a patient.[4] Determining an MCID of a PROM is important because a given change on the score generally does not have an obvious clinical importance to the clinician. The objective of this study is to determine an MCID of improvement on the AMSQ.

Methods

Data were derived from patients who have been treated with fampridine. Fampridine increases axonal conduction velocity by selectively blocking potassium channels, which may lead to improvement of various motor functions, including UEF.[5] Effects generally occur within 2 weeks of treatment. Therefore, patients treated with fampridine are good subjects to assess change in AMSQ and determine an MCID.

Patients

Patients were recruited in the VU Medical Center in Amsterdam from an outpatient clinic that was specifically organized to assess eligibility for and efficacy of treatment with fampridine. All patients provided written informed consent prior to inclusion, and the study was approved by the local ethics committee. Patients were considered eligible if they complied with the official treatment label of fampridine.[6] Demographical data and MS characteristics, including an Expanded Disability Status Scale (EDSS), were collected for this study.

AMSQ

Patients were asked to complete the AMSQ before treatment and during a follow-up visit after a minimum of 2 weeks of treatment. The AMSQ consists of 31 items concerning activity limitations due to hand and arm functioning. A patient assigns a number to each item on a 6-point Likert-type scale ranging from “not at all” to “no longer able to.” The sum score ranges from 31 to 186, with a higher score indicating more impairment. Change in the sum score was calculated by subtracting the AMSQ score of the follow-up visit from the baseline value. Consequently, a positive change score indicates an improvement of UEF, and conversely a negative change score indicates worsening of UEF. Questionnaires with more than two missing items were excluded from analysis. If one or two items were missing, the average of the other items was calculated and used as substitutes.

Determining the MCID

We used the sensitivity and specificity anchor-based method to determine an MCID. In short, with an anchor-based approach, the change in PROM score is being compared with change in another measure that is understandable and is considered as an anchor or external criterion.[7] As anchor, we used a global perceived effect (GPE) score that specifically addressed change in UEF and consisted of a 5-point Likert-type scale, including “much deteriorated” (1), “deteriorated” (2), “unchanged” (3), “improved” (4), and “much improved” (5). The GPE was determined by the treating physician on the follow-up visit by asking the patient how much the UEF was changed since the baseline visit. Because we used the sensitivity and specificity approach, the GPE scores were dichotomized into “improved” or “unchanged.” Since we wanted to address improvement of UEF, we excluded scores 1, 2, and 5 to minimize the impact of these scores on the MCID value.[8] With this method the GPE is considered the gold standard and the AMSQ as a diagnostic test for which the sensitivity and specificity is discriminated between “improved” and “unchanged.”[9] A receiver operating characteristic (ROC) curve was used to determine the MCID, that is, the AMSQ score that produces the greatest combined sensitivity and specificity (determined with the highest Youden’s index). In addition, the area under the ROC curve (AUROC) was determined. This value represents the probability that scores will correctly discriminate between “improved” and “unchanged” UEF.[7] A value of 0.7 to 0.8 was considered acceptable and 0.8 to 0.9 excellent.[10] The correlation between AMSQ change and GPE was determined using Spearman’s rank-order correlation statistics.

To investigate the statistical properties of the underlying distribution of change scores of the AMSQ, we calculated the standard error of measurement (SEM). The SEM was calculated by multiplying the baseline standard deviation by the square root of one minus its reliability coefficient (i.e. the intraclass correlation coefficient). Statistical analyses were performed in IBM SPSS Statistics for Macintosh, Version 24.

Results

Data from 223 patients were analyzed. The mean age was 51.3 years (standard deviation 10.5), with 57.4% females. Most patients had a progressive disease type (56.5%). The median (interquartile range) for disease duration was 11.4 years (4.4–16.6) and for EDSS was 6.0 (4.0–6.5). The correlation coefficient between AMSQ and GPE was 0.37 (p < 0.001). The AMSQ thresholds from the ROC curve with corresponding sensitivity and specificity are displayed in Table 1. A threshold value of 14.5 yielded the highest sensitivity (0.68) and specificity (0.79). The ROC curve is shown in Figure 1. The AUROC value was 0.77. The SEM was 13.0.

Table 1.

Threshold values from receiver operating characteristic curve with corresponding sensitivity, specificity, and Youden’s index.

Threshold value	Sensitivity	1—Specificity	Youden’s index
0.5	0.868	0.581	0.287
1.5	0.868	0.531	0.337
2.5	0.842	0.494	0.348
3.5	0.789	0.444	0.346
4.5	0.789	0.419	0.371
5.5	0.763	0.388	0.376
6.5	0.763	0.356	0.407
7.5	0.737	0.325	0.412
8.5	0.737	0.300	0.437
9.5	0.737	0.281	0.456
10.5	0.711	0.281	0.429
11.5	0.711	0.263	0.448
12.5	0.711	0.250	0.461
13.5	0.684	0.225	0.459
14.5	0.684	0.213	0.472
15.5	0.632	0.206	0.425
16.5	0.579	0.206	0.373
17.5	0.579	0.188	0.391
18.5	0.579	0.175	0.404
19.5	0.553	0.156	0.396
20.5	0.553	0.144	0.409
21.5	0.474	0.144	0.330
22.5	0.447	0.144	0.304
23.5	0.447	0.138	0.310
24.5	0.421	0.131	0.290
25.5	0.342	0.119	0.223
26.5	0.342	0.106	0.236
27.5	0.342	0.094	0.248
29.5	0.316	0.081	0.235
32.0	0.289	0.081	0.208
34.0	0.289	0.075	0.214
35.5	0.289	0.069	0.221
37.0	0.289	0.063	0.227
38.5	0.289	0.056	0.233
39.5	0.289	0.050	0.239
40.5	0.263	0.050	0.213
43.0	0.237	0.044	0.193
48.0	0.211	0.044	0.167
51.5	0.211	0.038	0.173
54.5	0.184	0.031	0.153
60.0	0.158	0.031	0.127
63.5	0.158	0.025	0.133
66.0	0.132	0.025	0.107
69.0	0.132	0.019	0.113
71.5	0.105	0.019	0.087
75.0	0.053	0.019	0.034
79.5	0.053	0.013	0.040
82.5	0.026	0.013	0.014
84.5	0.026	0.006	0.020

Bold highlights the threshold value that yielded the highest sensitivity and specificity.

Figure 1.

Receiver operating characteristic (ROC) curve.

Discussion

We found an MCID value of 14.5 with a sensitivity of 0.68, a specificity of 0.79, and an acceptable AUROC value. Since the AMSQ has no decimals, we rounded the threshold to 15 points.

There is no consensus about the preferred threshold value that determines sensitivity and specificity. Mostly, the threshold is chosen that jointly maximizes sensitivity and specificity in order to have the lowest overall misclassification.[11] We used this rationale to determine the threshold.

There is a certain degree of uncertainty in our findings. This is reflected in a weak, albeit sufficient,[8] correlation between AMSQ and GPE, a moderate sensitivity and specificity and rather similar threshold values around the MCID value. Therefore, our findings will have to be confirmed in future studies.

This is the first study to determine an MCID value for the AMSQ. The strength of our study lies in the large sample size and the normal distribution of AMSQ results. However, our study also has some limitations. First, we used only one method to determine an MCID, while there are others available.[4,7] Second, we used a subjective anchor that is prone to recall bias in which case a patient may have recalled answers given at baseline that have subsequently influenced completion of the questionnaire at follow-up. Therefore, an MCID should also be assessed with an objective anchor for UEF, such as the 9HPT. Finally, a placebo effect might have influenced our findings. This accounts particularly for the GPE, since a patient may have experienced improvement of UEF, while no improvement has been noticed on the ability to perform a task of UEF (as assessed with AMSQ). Inclusion of other measures that assess capacity of UEF objectively, such as the 9HPT, allows more certainty and magnitude of true improvement of UEF. Furthermore, additional objective measures contribute to a more detailed description of UEF of a group of patients.

In conclusion, our MCID estimate for AMSQ is 15 points based on a sensitivity and specificity anchor-based method. Future studies should investigate reproducibility of this finding with similar and other methods, in a cohort with extensive assessment of different domains of UEF.