Mapping of Multiple Sclerosis Walking Scale (MSWS-12) to five-dimension EuroQol (EQ-5D) health outcomes: an independent validation in a randomized control cohort.

BACKGROUND: Mapping of patient-reported outcomes to the five-dimension EuroQol (EQ-5D) health index is increasingly being used for understanding the relationship of outcomes to health states and for predicting utilities that have application in economic evaluations. The 12-item Multiple Sclerosis Walking Scale (MSWS-12) is a patient-reported outcome that assesses the impact of walking impairment in people with MS. An equation for mapping the MSWS-12 to the EQ-5D was previously developed and validated using a North American Research Committee on MS (NARCOMS) registry cohort.
MATERIALS AND METHODS: This analysis retested the validity of the equation mapping the MSWS-12 to the three-level EQ-5D (EQ-5D-3L) by using an independent cohort of patients with MS enrolled in a randomized controlled trial. Mapping was evaluated at two separate time points (baseline and week 4) during the clinical trial. The mapping equation's performance was subsequently assessed with mean absolute error (MAE) and root-mean-square error (RMSE) by comparing equation-based estimates to values elicited in the trial using the actual EQ-5D-3L questionnaire.
RESULTS: The mapping equation predicted EQ-5D-3L values in this external cohort with reasonable precision at both time points (MAE 0.116 and RMSE 0.155 at baseline; MAE 0.105 and RMSE 0.138 at week 4), and was similar to that reported in the original NARCOMS cohort (MAE 0.109 and RMSE 0.145). Also as observed in the original NARCOMS cohort, the mapping equation performed best in patients with EQ-5D-3L values between 0.50 and 0.75, and poorly in patients with values <0.50.
CONCLUSION: The mapping equation performed similarly in this external cohort as in the original derivation cohort, including a poorer performance in MS patients with more severe health-state severity.

RCT Entities:   Population Interventions Outcomes

Introduction

Multiple sclerosis (MS) is a progressive neurologic disease that is characterized by axonal demyelination in the central nervous system, resulting in action-potential-conduction deficits that are believed to contribute to the clinical symptoms experienced by people with MS.1,2 While these symptoms include visual deficits, weakness, paralysis, numbness, and other sensory disturbances, walking impairment is the most visible symptom, and is considered a hallmark of MS. MS has a profound impact on patient function and quality of life (QoL), with walking impairment and its effects on mobility in particular substantially contributing to the patient burden.3–5

The 12-item MS Walking Scale (MSWS-12) is a patient-reported measure that was developed to facilitate assessment of the impact of MS on walking impairment from the patient’s perspective. While the MSWS-12 is increasingly being used in clinical trials, observational studies, and clinical practice, it lacks the ability to measure health utilities that are generally used in economic evaluations. A commonly used technique for adapting such outcomes for economic evaluations is regression-based mapping, which maps the outcome measure of interest to a health-utility index,6 such as the five-dimension, three-level EuroQol (EQ-5D-3L).6 The EQ-5D-3L is a widely used generic patient-reported measure that enables both evaluation of health-related QoL (HRQoL) and generation of health-utility scores for use in economic analyses.7

An equation to map the MSWS-12 onto the EQ-5D-3L health-utility index in North American patients with MS has been developed.8 This equation, which was derived and validated in >3,500 MS patients from the North American Research Committee on MS (NARCOMS) registry, has been shown to predict EQ-5D-3L scores using individual MSWS-12 item scores with reasonable precision, especially in patients with moderate disease severity. Unfortunately, due to a lack of availability of another independent North American or US data set for MS patients that contains both the MSWS-12 and the EQ-5D-3L, the equation has never been further validated.

A randomized controlled trial (DER-401) was conducted to evaluate the safety and efficacy of 5 mg and 10 mg doses of dalfampridine extended-release tablets versus placebo in patients with MS.9 While the primary objective of this trial was to evaluate change in walking speed as measured by the Timed 25-Foot [7.62 m] Walk, both the MSWS-12 and the EQ-5D-3L health-status questionnaire were administered. Therefore, DER-401 provided an opportunity to independently validate the MSWS-12 to the EQ-5D-3L mapping equation in an external cohort of MS patients.

Materials and methods

Data for the additional validation of the original mapping equation were taken from DER-401 (clinicaltrials.gov registration: NCT01328379), a multicenter (65 centers in the US), randomized, parallel-group trial designed to assess the safety and efficacy of two doses of dalfampridine extended release and placebo in patients with MS.9 Eligibility was determined during an initial screening visit; qualifying participants returned 1 week later (visit 1, baseline) and were randomized in a 1:1:1 ratio to twice-daily treatment with dalfampridine 5 mg, 10 mg, or placebo. At 4 weeks after initiating treatment, study participants returned for their final safety and efficacy assessments (visit 3). As part of DER-401, the MSWS-12 and the EQ-5D-3L were both assessed at baseline and week 4.

The EQ-5D-3L is a validated, generic, preference-based, health-status measure consisting of five descriptive questions encompassing five domains of HRQoL (mobility, self-care, usual activities, pain/discomfort, anxiety/depression).10 Each question is answered based on three response options (1= “no problems”, 2= “moderate problems”, 3= “severe problems”), with the 243 potential patterns of responses enabling classification of a participant into a distinct health state associated with a specific index score. For the US general population, the possible EQ-5D-3L index scores range from −0.11 (ie, 3-3-3-3-3) to 1.0 (ie, 1-1-1-1-1) on a scale where 0.0= death and 1.0= perfect health.11 The EQ-5D-3L was scored using the US scoring algorithm in both the original NARCOMS validation study and DER-401.

The MSWS-12 is a validated, patient-reported outcome measure assessing the extent to which MS impacts an individual’s walking ability.12 The MSWS-12 comprises 12 questions concerning different aspects of walking function and quality that are rated on a scale ranging between 1 (not at all) and 5 (extremely) (see Table S1). Individual item scores are summed to achieve a total score that ranges from 12 to 60. This total score is then transformed to achieve a range of 0–100, with higher scores reflecting a greater impact of MS on walking.

Participants in the DER-401 study who completed both the EQ-5D-3L and MSWS-12 outcome measures at baseline and week 4 were included in this analysis. MSWS-12 data at both time points of DER-401 were applied to the previously derived mapping equation to elicit predicted EQ-5D-3L values. This mapping equation was derived using over 3,500 participants (mean EQ-5D-3L and MSWS-12 scores 0.74±0.18 and 50.8±33.5, respectively) in the NARCOMS registry using ordinary least squares regression.8 The equation EQ-5D-3L = 0.002× (item 1) −0.009× (item 2) −0.01× (item 3) −0.029× (item 4) −0.019× (item 5) −0.0000881× (item 6) −0.008× (item 7) −0.002× (item 8) +0.013× (item 9) −0.011× (item 10) +0.001× (item 11) −0.008× (item 12) +0.983 uses all of the individual MSWS-12 item scores to estimate EQ-5D-3L index scores with a mean absolute error (MAE) of 0.109±0.096. These predicted EQ-5D-3L values were then compared to observed EQ-5D-3L values from DER-401 using the actual EQ-5D-3L health-status index questionnaire.

The precision of our original mapping equation was assessed in the DER-401 cohorts by calculating the MAE (the average of the absolute difference between observed EQ-5D-3L and predicted EQ-5D-3L values) and root-mean-square error (RMSE; the positive square root of the average squared prediction error), which attaches greater weight to larger errors. In addition, the percentage of EQ-5D-3L estimates within various absolute errors of the actual value was calculated, along with goodness-of-fit statistics when the sample was stratified by different levels of EQ-5D-3L health-state severity (EQ-5D-3L score <0.50, 0.50–0.75, and ≥0.75). Finally, scatterplots of predicted versus observed EQ-5D-3L values at baseline and week 4 were constructed.

Descriptive statistics are reported as means ± standard deviations for continuous data and percentages for categorical data. All analyses and reporting are consistent with the National Institute for Health and Clinical Excellence Decision Support Unit’s technical support document for using mapping methods to estimate health-state utility values.13

Results

As shown in Table 1, there were some differences in the NARCOMS and DER-401 populations, including a higher proportion of females in the NARCOMS population, which was also older and reported a likely clinically relevant lesser impact of MS on walking ability. In the DER-401 population, total MSWS-12 score and scores for all individual items were consistently lower at week 4 than at baseline, reflecting an improvement in patient-reported walking ability, and the EQ-5D-3L score was higher, suggesting an improvement in the health index.

Table 1

Characteristics of the NARCOMS and DER-401 cohorts

Characteristic	NARCOMS cohort8 (n=3,505)	DER-401 study9
		Baseline (n=429)	Week 4 (n=398)
Age, mean (SD)	56.7 (10.0)	52.6 (9.5)	52.6 (9.5)
Female,%	80.7	69.9	70.4
MSWS-12 score, mean (SD)
Total score	50.8 (33.5)	62.4 (24.2)	53.0 (25.8)
Item 1	2.67 (1.39)	3.12 (1.03)	2.88 (1.07)
Item 2	3.86 (1.57)	4.33 (1.24)	4.18 (1.28)
Item 3	2.91 (1.44)	3.42 (1.12)	3.20 (1.17)
Item 4	2.68 (1.38)	3.07 (1.09)	2.81 (1.12)
Item 5	2.86 (1.35)	3.36 (1.08)	3.01 (1.10)
Item 6	3.23 (1.49)	3.65 (1.12)	3.23 (1.18)
Item 7	3.04 (1.46)	3.39 (1.17)	2.96 (1.18)
Item 8	2.63 (1.61)	3.28 (1.39)	2.87 (1.39)
Item 9	2.81 (1.72)	3.38 (1.51)	3.03 (1.46)
Item 10	3.23 (1.50)	3.52 (1.19)	2.95 (1.26)
Item 11	3.26 (1.49)	3.66 (1.11)	3.10 (1.19)
Item 12	3.29 (1.49)	3.78 (1.20)	3.21 (1.25)
EQ-5D-3L, mean (SD)	0.74 (0.18)	0.70 (0.17)	0.73 (0.15)

Abbreviations: NARCOMS, North American Research Committee on Multiple Sclerosis; DER, dalfampridine extended release; EQ-5D-3L, EuroQol 5-dimension 3-level version; MSWS-12, 12-item Multiple Sclerosis Walking Scale; SD, standard deviation.

Statistics depicting the mapping equation’s precision at both the baseline (n=429) and week 4 time points in DER-401 (n=398) are provided in Tables 2 and 3. The mapping equation was most accurate in the DER-401 cohort at week 4, as indicated by the slightly lower values for both the MAE and RMSE (Table 2), although precision was not markedly different than that seen at baseline. High accuracy was consistently observed at both baseline and week 4 for EQ-5D-3L scores 0.50–<0.75, as indicated by the lowest MAE and RMSE values (Table 2); the lowest accuracy was observed for EQ-5D-3L scores <0.50 at both time points.

Table 2

Precision of the NARCOMS ordinary least squares mapping equation applied to data from study DER-401a

Model	MAE (SD)	RMSE
NARCOMS cohort8
Overall EQ-5D-3L (n=3,505)	0.109 (0.096)	0.145
EQ-5D-3L <0.50 (n=438; 12.5%)	0.297 (0.094)	0.312
EQ-5D-3L 0.50–<0.75 (n=867; 24.7%)	0.063 (0.054)	0.083
EQ-5D-3L ≥0.75 (n=2,200; 62.8%)	0.089 (0.060)	0.107
DER-401 at baseline9
Overall EQ-5D-3L (n=429)	0.116 (0.103)	0.155
EQ-5D-3L <0.50 (n=59; 13.8%)	0.313 (0.098)	0.328
EQ-5D-3L 0.50–<0.75 (n=147; 34.3%)	0.063 (0.048)	0.080
EQ-5D-3L ≥0.75 (n=223; 51.9%)	0.099 (0.062)	0.116
DER-401 at week 49
Overall EQ-5D-3L (n=398)	0.105 (0.090)	0.138
EQ-5D-3L <0.50 (n=40; 10.0%)	0.295 (0.080)	0.305
EQ-5D-3L 0.50–<0.75 (n=136; 34.2%)	0.081 (0.065)	0.104
EQ-5D-3L ≥0.75 (n=222; 55.8%)	0.085 (0.058)	0.104

Notes:

Mapping model is EQ-5D-3L =+0.002× (item 1), −0.009× (item 2), −0.01× (item 3), −0.029× (item 4), −0.019× (item 5), −0.0000881× (item 6), −0.008× (item 7), −0.002× (item 8), +0.013× (item 9), −0.011× (item 10), +0.001× (item 11), −0.008× (item 12), and +0.983 (constant).

Abbreviations: NARCOMS, North American Research Committee on Multiple Sclerosis; DER, dalfampridine extended release; EQ-5D-3L, EuroQol 5-dimension 3-level version; MAE, mean absolute error; RMSE, root-mean-square error; SD, standard deviation.

Table 3

Percentage of predicted EQ-5D-3L values within ranges of absolute error of observed value in DER-401 relative to data from the original validation study

Absolute error	EQ-5D-3L estimates
	NARCOMS cohort8	DER-401 study9
		Baseline	Week 4
≤0.01	6.5%	4.3%	8.3%
0.01< error ≤0.03	12.5%	13.1%	9.1%
0.03< error ≤0.05	11.0%	13.4%	13.9%
0.05< error ≤0.07	11.5%	11.1%	12.3%
0.07< error ≤0.10	19.4%	14.4%	15.1%
0.10< error ≤0.12	7.4%	4.0%	8.6%
>0.12	31.8%	39.8%	32.8%

Abbreviations: NARCOMS, North American Research Committee on Multiple Sclerosis; DER, dalfampridine extended release; EQ-5D-3L, EuroQol 5-dimension 3-level version.

More than half of the EQ-5D-3L estimates in DER-401 were within 10% of the observed value at both the baseline (56.3%) and week 4 (58.7%) time points (Table 3). These estimates of precision were generally similar to those observed in the initial NARCOMS cohort, which showed that 49.4% of the estimates were within 10% of the observed value (Table 3).

Figures 1 and 2 depict scatterplots of observed versus predicted EQ-5D-3L values for the DER-401 cohort at baseline and week 4. These plots show that at EQ-5D-3L scores <0.50, the mapping equation overestimated the observed health-state utility. Conversely, the mapping equation underestimated health-state utility in cases of perfect health (EQ-5D-3L score of 1.0).

Figure 1

Scatterplot of observed versus predicted EQ-5D-3L scores for the DER-401 cohort at baseline.

Note: Data points lying closest to the diagonal line (representing the line of perfect fit) illustrate more precise estimates of observed EQ-5D-3L scores.

Abbreviations: DER, dalfampridine extended release; EQ-5D-3L, EuroQol 5-dimension 3-level version.

Figure 2

Scatterplot of observed versus predicted EQ-5D-3L scores for the DER-401 cohort at week 4.

Note: Data points lying closest to the diagonal line (representing the line of perfect fit) illustrate more precise estimates of observed EQ-5D-3L scores.

Abbreviations: DER, dalfampridine extended release; EQ-5D-3L, EuroQol 5-dimension 3-level version.

Discussion

Results of this analysis suggest that the previously derived and validated mapping equation using each individual item of the MSWS-12 predicted EQ-5D-3L values in the DER-401 cohort with similar precision as in the original NARCOMS cohort.8 Notably, the differences between MAE and RMSE values were small, indicating only low variance for the individual errors in the sample. Moreover, visual inspection of the MAE and RMSE values combined with the subgroup analysis that stratified patients by observed EQ-5D-3L values suggested that the mapping equation performed best among patients with moderate EQ-5D-3L health-state values (ie, EQ-5D-3L scores 0.50–0.75) and performed less optimally in patients with more severe health states (ie, EQ-5D-3L scores <0.50). The overestimation for more severe EQ-5D-3L health states is also consistent with other studies.14

In addition to the validation study in the NARCOMS population,8 two other studies have been published that mapped MSWS-12 to EQ-5D-3L.15,16 Hawton et al15 developed a mapping equation from a sample of MS patients in the UK and using the UK scoring algorithm for the EQ-5D. The UK and NARCOMS scoring equations performed similarly in their distinct populations, with MAEs approximating 0.10. However, while this degree of error in EQ-5D-3L prediction may not be optimal, it is not unexpected, as the MSWS-12 measures self-reported walking limitations due to MS while the EQ-5D-3L is a generic measure of health status. The other study used data from a clinical trial to map the MSWS-12 to the EQ-5D-3L using both the NARCOMS-based equation and the one derived from the UK to evaluate the relationship between change in MSWS-12 and health utilities.16 The results of that study, which showed that improvements in MSWS-12 with treatment translated into improvements in health-utility scores regardless of which mapping equation was used, suggest the use of mapping for potentially understanding the relationship between specific disease-related outcomes and overall health status.

While mapping equations are useful for attempting to predict how changes in a measure for a specific outcome may impact generic HRQoL measures, it is likely that targeted, disease-specific measures, such as the MSWS-12, capture only a portion of information contained in broader measures of HRQoL or general health. In this instance, the equation was less precise at predicting health-state values at extreme ends of the range. It is important to note that mobility is a single domain of the multidimensional EQ-5D-3L, while the MSWS-12 is used to assess walking. Therefore, the use of mapping equations, such as the one described here, to estimate EQ-5D-3L health-state index values may be less than ideal, and should only be used when directly measured EQ-5D-3L values are not available. Despite these caveats associated with this technique, the National Institute for Health and Clinical Excellence’s Decision Support Unit endorses the use of mapping equations in these instances, and provides guidance to aid in the development and validation of mapping equations.13 These recommendations support the use of an external sample to validate mapping equations. At the time of the original mapping study,8 no external population was available. Instead, the NARCOMS cohort was randomly split into derivation and validation cohorts, with the final equation refined using the entire cohort. With the emergence of the DER-401 study that collected both MSWS-12 and EQ-5D-3L data in a North American population, it was prudent to validate MSWS-12 to EQ-5D-3L mapping further using the derived equation.

In summary, independent evaluation in an external population is an important step in confirming the validity of mapping equations to estimate health-state index values. The NARCOMS-based mapping equation performed in a consistent manner in this external randomized trial cohort to the original cohort, including a poorer performance in MS patients with more severe health-state severity. For this reason, additional efforts to improve the existing equations would be helpful. This may be achieved by recalibrating the equation in a study population consisting of a larger proportion of patients with severe disability. Of note, revisions to the MSWS-12 questionnaire are ongoing,17 and it is possible that as improvements to the MSWS-12 are made, more accurate mapping equations may arise, although they too will need to undergo external validation. The availability of accurate and reliable mapping can enhance the evaluation of health status in these individuals when HRQoL assessments have not directly been made, and the results of such mapping have the additional potential for use in economic analyses.

Table S1

The 12-item Multiple Sclerosis Walking Scale (MSWS-12)

• These questions ask about limitations to your walking due to MS during the past 2 weeks.

• For each statement, please circle the one number that best describes your degree of limitation.

• Please answer all questions even if some seem rather similar to others, or seem irrelevant to you.

• If you cannot walk at all, please tick this box.□

In the past two weeks, how much has your MS …	Not at all	A little	Moderately	Quite a bit	Extremely
1. Limited your ability to walk?	1	2	3	4	5
2. Limited your ability to run?	1	2	3	4	5
3. Limited your ability to climb up and down stairs?	1	2	3	4	5
4. Made standing when doing things more difficult?	1	2	3	4	5
5. Limited your balance when standing or walking?	1	2	3	4	5
6. Limited how far you are able to walk?	1	2	3	4	5
7. Increased the effort needed for you to walk?	1	2	3	4	5
8. Made it necessary for you to use support when walking indoors (eg, holding on to furniture, using a stick, etc.)?	1	2	3	4	5
9. Made it necessary for you to use support when walking outdoors (eg, using a stick, a frame, etc.)?	1	2	3	4	5
10. Slowed down your walking?	1	2	3	4	5
11. Affected how smoothly you walk?	1	2	3	4	5
12. Made you concentrate on your walking?	1	2	3	4	5

Please check that you have circled ONE number for EACH question

Note: Reprinted with permission from Hobart JC, Riazi A, Lamping DL, Fitzpatrick R, Thompson AJ. Measuring the impact of MS on walking ability: the 12-item MS Walking Scale (MSWS-12). Neurology. 2003;60(1):31–36.1 Promotional and commercial use of the material in print, digital or mobile device format is prohibited without the permission from the publisher Wolters Kluwer Health. Please contact healthpermissions@wolterskluwer.com for further information.