Validation and Reliability of the Japanese Version of the Modified Parkinson Activity Scale (M-PAS).

OBJECTIVE: The Modified Parkinson Activity Scale (M-PAS) is used to identify the most important activity limitations in patients with Parkinson's disease. We developed a Japanese version of the M-PAS and evaluated its reliability and validity.
METHODS: Twenty-five patients with Parkinson's disease (median age 71 years old, range 58-83) were enrolled, and two raters used the Japanese version of M-PAS to assess the subjects. The inter-rater reliability was evaluated using Cohen's weighted kappa coefficient for the total score and three domain scores; systematic error was investigated using Bland-Altman analysis. Concurrent validity of the Japanese M-PAS was measured using Spearman's rank correlation coefficients.
RESULTS: Cohen's kappa coefficients for the total score and the three domain scores were in the range 0.81-0.98, and 95% confidence intervals included zero for each item, suggesting excellent agreement and no systematic errors. The scores of the Japanese version of M-PAS were significantly correlated with the scores of the Movement Disorder Society-Unified Parkinson's Disease Rating Scale Part II (Spearman's rho=-0.56, P <0.01) and Part III (Spearman's rho=-0.32, P <0.01). The percentage of patients with the highest and the lowest scores in the Japanese version of M-PAS suggested no ceiling or floor effects.
CONCLUSION: The Japanese version of M-PAS showed excellent inter-rater reliability and good concurrent validity without ceiling or floor effects. 2021 The Japanese Association of Rehabilitation Medicine.

INTRODUCTION

Parkinson’s disease (PD) is the second most common neurodegenerative disorder[1]) and is characterized by four cardinal signs: resting tremor, bradykinesia, rigidity, and postural instability. In Japan, the total number of patients with PD is estimated to be 200,000.[2]) Aging is the largest risk factor for the development and progression of PD,[2],[3]) and because of Japan’s super-aging society, the number of people diagnosed with PD is growing.

The Movement Disorder Society–Unified Parkinson’s Disease Rating Scale (MDS-UPDRS) is currently the most widely used assessment tool in physical therapy for PD worldwide.[4]) Although the MDS-UPDRS is a well-established method for the comprehensive assessment of PD,[5]) earlier studies have suggested that the MDS-UPDRS may have some shortcomings. For instance, it can be subjective,[6]) because most items (e.g., all the questions in Parts I, II, and IV) rely on the participant’s memory (e.g., sub-item 2.9: “Over the past week, do you usually have trouble turning over in bed?”) and are scored by the patient’s responses (e.g., 0-points “Not at all,” 1-point “I have a bit of trouble turning, but I do not need any help,” 2-points “I have a lot of trouble turning and need occasional help from someone else,” 3-points “To turn over I often need help from someone else,” and 4-points “I am unable to turn over without help from someone else”). Another study by Regnault et al. indicated that the MDS-UPDRS can be insensitive[7]) because most items in Part III reflect levels of severity of motor symptoms that do not manifest in the early stage of PD, resulting in floor effects in most items. Additionally, Nieuwboer et al. suggested that the UPDRS does not include items in core areas of rehabilitation[8]) (e.g., rigidity, bradykinesia, and tremor do not seem to be the main target of rehabilitation intervention and are more relevant to medical treatments). Moreover, a relatively large number of items are needed to obtain the scores,[8]) for example, the motor examination consists of 33 elements. For these reasons, the MDS-UPDRS may not be able to accurately reflect the effects of interventions.[8])

Recently, the Modified Parkinson’s Activity Scale (M-PAS) was developed as an objective evaluation tool for activity limitations within the core areas of motor rehabilitation. The M-PAS aims to identify limitations in activities of daily living for which rehabilitation can be provided and to assess changes following intervention.[9],[10])

The M-PAS was introduced as the only rating scale recommended in the physical therapy guideline for PD[11]) and in occupational therapy guidelines for patients with PD.[12]) The M-PAS consists of 16 items divided into 3 domains that describe core activities related to functional mobility for patients with PD: Chair Transfer (2 items), Gait Akinesia (6 items), and Bed Mobility (8 items). Each item is scored on a 5-point scale (0–4), with higher scores indicating greater independence.

There are several advantages associated with administering the M-PAS to assess motor mobility-related activity limitations of the core areas of motor rehabilitation in PD. The M-PAS possesses good concurrent validity and inter-rater agreement, lacks a ceiling effect, delivers consistent results for experts and non-experts in PD,[10]) and is valid for “on” and “off” phases.[8]) Although the M-PAS is available in English[8],[10]) and Portuguese,[13]) a Japanese translation is not currently available.

Therefore, the primary purpose of this study was to develop a Japanese version of the M-PAS and to confirm its validity and reliability. The secondary purpose was to examine systematic errors in the Japanese version of M-PAS using Bland-Altman analysis and to establish the minimal detectable changes.

MATERIALS AND METHODS

Ethical Considerations

In accordance with the Declaration of Helsinki, we obtained written informed consent from all the participants before the assessment. This study was approved by the Ethics Committee of Hyogo Prefectural Rehabilitation Hospital at Nishi-Harima (approval number: 1809). The study was conducted after pre-registration in the UMIN Clinical Trials Registry (UMIN000036262). We obtained written informed consent from each participant after explaining the purpose, expected benefits, and potential harm of this research.

Participants

Twenty-five outpatients with PD (mean age 70.5 ± 6.5 years old, range 58–83; 16 women and 9 men) were enrolled consecutively at the Hyogo Prefectural Rehabilitation Hospital at Nishi-Harima through advertisements and consultation referrals. The criteria for inclusion were: (1) patients with a diagnosis of idiopathic PD based on the MDS clinical diagnostic criteria for PD,[14]) (2) Hoehn and Yahr stage 2 to 4 during the on phase, (3) having problems in at least one of the three domain scores in the Japanese M-PAS. All subjects were receiving dopamine treatment and were tested during an optimal on phase.

The raters were a nurse with 2 years of clinical experience (rater A) and a nurse with 8 years of clinical experience (rater B) who regularly treated patients with PD. According to the validation study of the original Dutch version of the M-PAS,[10]) the performance of participants was scored in real-time while the activity was being carried out. The raters were positioned some distance apart to minimize any interaction that might bias the assessments.

The severity of PD was measured using the Hoehn and Yahr scale, and MDS-UPDRS was evaluated by a neurologist (K.M.) who has received extensive training in the administration of the MDS-UPDRS. The impact of dyskinesia on the examination was determined by the final question in MDS-UPDRS Part III. The percentages of participants with minimum or maximum total scores on the Japanese M-PAS were calculated to define any ceiling or floor effects.

After obtaining permission from the developer of the original version (Prof. Alice Nieuwboer, KU Leuven, Belgium),[8]) the Japanese version of the M-PAS was established by the authors in accordance with a forward–backward translation procedure based on the guidelines of Beaton and colleagues.[15])

The Japanese Translated Version of the M-PAS

The participants underwent evaluation in three domains: (1) Chair Transfer, (2) Gait Akinesia, and (3) Bed Mobility.

(1) Chair Transfer: The patient is seated on a chair with hands resting on their lap (starting position) and is asked to rise from the chair. The patient may lean forward with hands on the arms of the chair or on the knees. When a standing position is achieved, the patient will have to maintain it for one second.

(2) Gait Akinesia: The patient is seated on a chair with hands resting on their lap (starting position). The patient is asked to rise from the chair and walk straight ahead for 3 m at their own pace, turn around at a particular turning point, and then return to the chair and sit down.

Four dual tasks (carrying a plastic cup with water as a motor dual-task and counting backward as a cognitive dual-task) were added to this domain to provoke freezing of gait.

(3) Bed Mobility: The patient stands in front of the bed on the preferred side (starting position). The patient is asked to lie down on his/her back under the bed cover in a comfortable position, just as if they were at home. The patient is then asked to roll over with or without a cover and is finally asked to get out of bed with or without a cover.

All items are scored on an ordinal scale, ranging from 4 (highest=best performance) to 0 (lowest=impossible or dependent on help).

Reliability and Agreement

The inter-rater reliability of a questionnaire quantifies the extent to which the raters produce the same scores during repeated measurements, provided the participant’s condition remains stable. Bland-Altman analysis was conducted to assess systematic errors in the translated version of the M-PAS. This was based on the mean difference between the two raters and the upper and lower limit of agreement (LOA). If the 95% confidence interval of LOA does not contain zero, it is determined that systematic errors exist.[16],[17]) The smallest change in the assessment score that cannot be discounted as measurement error is defined as the minimum detectable change (MDC). The MDC and standard error of measurement (SEM) provide crucial information on the absolute reliability of an assessment tool by indicating the range in which the theoretical “true” score lies.[18])

Concurrent Validity

Concurrent validity was examined by determining the correlation coefficient between the total score of the translated version of the M-PAS and the MDS-UPDRS Part II and Part III using Spearman’s rank correlation coefficient. In the original variation study, validity was assessed by calculating the correlation coefficient between the scores of the version of the M-PAS and the MDS-UPDRS Part III (motor examination), whereas the relationship with the MDS-UPDRS Part II (motor experiences of daily living) was not assessed.[10]) However, we believe that it is important to confirm the correlation between the translated version of M-PAS and MDS-UPDRS Part II as well as Part III because the M-PAS is a battery that assesses the three basic activities performed in daily life. Consequently, we hypothesized that the translated M-PAS would correlate significantly with MDS-UPDRS Part II and Part III.

Statistical Analyses

The inter-rater reliability for each domain was measured using Cohen’s weighted kappa coefficient,[19]) Cronbach’s alpha coefficient (α), and intraclass correlation coefficient (ICC) with its associated 95% confidence interval. Cohen’s weighted kappa coefficient was interpreted as follows: in the range 0.61–0.80 as substantial, and >0.80 as almost perfect agreement. Additionally, ICC in the range 0.75–0.90 was considered as good and >0.90 as excellent,[20]) while Cronbach’s α in the range 0.76–0.95 was considered to be fairly high.[21])

The MDC was calculated using the formula (MDC=[SEM] × 1.96 × √2), while SEM was calculated as a variable for the difference between the scores of the two raters. Concurrent validity was considered based on Spearman’s rho characterized as weak (0.10–0.39), moderate (0.40–0.69), strong (0.70–0.89), and perfect (>0.90).[22]) To identify ceiling or floor effects, the percentage of the population with the highest and the lowest scores were calculated following the commonly used method.[23],[24],[25],[26]) A threshold of 15% for patients achieving the highest scores and lowest scores defined ceiling and floor effects, respectively. All statistical analyses were performed using the R software (R Foundation for Statistical Computing; R package version 4.0.2).[27])

RESULTS

The study period was from August 1, 2019, to December 31, 2019.

Patient Demographics

The results of patient assessments are presented in Table 1. The Hoehn and Yahr scale was stage 2 (2 patients, 8%), stage 3 (18 patients, 72%), or stage 4 (5 patients, 20%). The median score in MDS-UPDRS Part II was 14 points (range 1–25), and Part III was 31 points (range 11–46). The median duration of Parkinson’s disease was 12 years (range 5–34).

Table 1.

 Results of patient assessments

		Mean ± SD
Hoehn and Yahr	Stage (II/III/IV)a	2/18/5
MDS-UPDRS	Part II (0–52)	13.8 ± 6.6
MDS-UPDRS	Part III (0–132)	31.2 ± 10.1
Japanese M-PAS	Total score (0–64)	51.4 ± 16.1
	Chair Transfer (0–8)	5.8 ± 2.3
	Gait Akinesia (0–24)	18.7 ± 8.9
	Bed Mobility (0–32)	27.7 ± 6.3

The data are means and standard deviations (SDs).

aNumber of patients with each stage of PD.

Three patients had difficulty in completing the Gait Akinesia domain independently, and their data for sub-item 5 (“Start akinesia with a motor dual-task”) and sub-item 6 (“Turning 180° with a motor dual-task”) were excluded from the analyses.

The median total score for the Japanese version of M-PAS was 58 points (range 6–64); the median subdomain scores were 6 points for Chair Transfer (range 0–8), 24 points for Gait Akinesia (range 0–24), and 31 points for Bed Mobility (range 6–32). The percentage of participants with the highest score for the Japanese version of the M-PAS was 10%, and the percentage with the lowest score was 0%. No patients showed dyskinesias during the examination.

Inter-rater Agreement and Reliability

The results for inter-rater agreement and reliability are summarized in Table 2. Cohen’s kappa coefficient showed excellent agreements (≥0.81) between the two raters for the total score and all three domain scores of the Japanese version of M-PAS, suggesting excellent reliability. Moreover, the results for ICC (≥0.97) and Cronbach’s alpha (≥0.82) further supported the inter-rater agreement.

Table 2.

 Summary of the inter-rater reliability and agreement in the Japanese version of the M-PAS between raters

Japanese M-PAS	(Range)	Cohen’s κ	ICC (95% CI)	Cronbach’s α	Mean differences (95% CI)	SEM	MDC
Total score	(0–64)	0.97	0.99 (0.99–0.99)	0.99	0.24 (−1.60 to 2.08)	1.0	2.9
Chair Transfer	(0–8)	0.81	0.97 (0.89–0.98)	0.82	0.32 (−0.27 to 0.91)	0.3	0.9
Gait Akinesia	(0–24)	0.98	0.98 (0.95–0.99)	0.87	–0.14 (−0.80 to 0.52)	0.4	1.0
Bed Mobility	(0–32)	0.94	0.98 (0.95–0.99)	0.89	0.04 (−1.63 to 1.71)	1.0	2.6

Cohen's κ, Cohen's weighted kappa coefficient which indicates the level of agreement between raters; ICC, intraclass correlation coefficient; Mean differences, the mean difference in scores between the raters; SEM: standard error of measurement; MDC: minimum detectable change.

The SEM values were 1.0 for the M-PAS total score, 0.3 for Chair Transfer, 0.4 for Gait Akinesia, and 1.0 for Bed Mobility. Consequently, the calculated MDC values were 2.9 for the total score, 0.9 for Chair Transfer, 1.0 for Gait Akinesia, and 2.6 for Bed Mobility.

The mean difference between the two raters for the total score of the Japanese version of M-PAS was 0.24 points (95% CI=–1.60 to 2.08); for the subscores, the differences were as follows: Chair Transfer, 0.32 points (95% CI=–0.27 to 0.91); Gait Akinesia, −0.14 points (95% CI=–0.80 to 0.52); and Bed Mobility 0.04 points (95% CI=–1.63 to 1.71). The fact that the confidence intervals contain zero means that there were no systematic errors between the two raters of the Japanese version of the M-PAS.

Figure 1 shows the relationship between the total scores of the Japanese version of the M-PAS and the MDS-UPDRS Part II (A) and Part III (B) scores represented as scatterplots. There were statistically significant negative correlations between the total score of the Japanese version of the M-PAS and the MDS-UPDRS Part II score (Spearman’s rho=–0.56, P <0.01) and the MDS-UPDRS Part III score (Spearman’s rho=–0.32, P <0.01).

Fig. 1.

 Correlation in the total scores between the Japanese version of the M-PAS and the MDS-UPDRS. Scatterplots show the relationships between the scores of the Japanese version of the M-PAS and the scores of MDS-UPDRS Part II (A) and MDS-UPDRS Part III (B).

DISCUSSION

The M-PAS is an objective tool for assessing the most important activity limitations in PD patients receiving rehabilitation.[11]) The major benefit of using the M-PAS is that it includes items in the core areas of rehabilitation. This allows the rater to measure changes following therapeutic intervention. Furthermore, the M-PAS has good inter-rater agreement for raters without PD-specific expertise.

The Japanese version of M-PAS that we developed showed excellent reliability (Cohen’s kappa coefficient ≥0.81); moreover, no systematic errors or floor or ceiling effects were observed, which is consistent with the original version of M-PAS.[10]) As expected, the Japanese M-PAS showed a weak correlation with the MDS-UPDRS Part III score and a moderate correlation with the MDS-UPDRS Part II score. This likely indicates that the M-PAS reflects activities of daily living rather than motor symptoms.

Most of the patients in the current study were assessed as Hoehn and Yahr stages 3–4, and the median UPDRS Part III total score was 31. These patients exhibited a relatively high median M-PAS total score, even though no ceiling effect was evident in this study. A previous study suggested that M-PAS scores may be higher when patients are assessed in a clinical setting because most motor problems occur when the patient is at home.[10]) This may explain the distribution of scores in the current study, in which patient activity was evaluated in a hospital. Moreover, according to the Dutch Occupational Therapy Guidelines, the performance of patients with PD is influenced by the environment and context.[12]) Consequently, future research should also be conducted in patients’ homes.

Some participants were relatively independent with high M-PAS scores even though they had severe motor symptoms as assessed using MDS-UPDRS. This can be explained by the distinct nature of the two scales. The M-PAS scale is specific to daily physical activities, whereas the MDS-UPDRS scale focuses more on the symptoms of PD. In general, patients with PD can perform daily activities independently even if they have severe motor symptoms. For instance, turning in bed independently with rigidity is known as “en bloc turning,”[28]) and walking independently with small amplitude limbs movements known as “shuffling gait.”[29])

Overall, the relationship between each item of the M-PAS and MDS-UPDRS remains unclear; further analyses with more patients using multivariate regression are required.

The current study established that changes of more than 2.9, 0.9, 1.0, and 2.6 points, respectively, in the total score and the Chair Transfer, Gait Akinesia, and Bed Mobility scores of the M-PAS can be accepted as true changes. Although the two raters had different levels of clinical experience (rater A had 2 years and rater B had 8 years of clinical experience), a previous study reported no differences between experts and non-experts in the assessment of PD using M-PAS.[10]) This could be the reason why the Japanese version of the M-PAS was not greatly affected by differences in the experience of the raters.

This study has some potential limitations. First, although the sample size of the current study was larger than that of the original M-PAS validation study,[10]) more participants should be included to draw more definitive conclusions. Second, the raters in this study were nurses. Other healthcare professionals should be included as raters in future studies. Third, although no ceiling or floor effects in the total score were evident in this study, the relatively small sample size limits the wider applicability of these outcomes. Finally, this validation study mainly focused on domain and total scores. To enable a more in-depth interpretation of the M-PAS, follow-up studies should include analysis of each sub-item of the M-PAS and a wider distribution of participants (in-patients and patients in the community and those with advanced-stage PD). Future studies should be carried out to address these limitations.

In conclusion, the current study demonstrated that the Japanese version of the M-PAS was a valid, specific, quantitative assessment tool for the basic activities of patients with PD, including Chair Transfer, Gait Akinesia, and Bed Mobility. Further studies with larger sample sizes and a wider target group of PD patients should be conducted to establish the wider applicability of the Japanese version of the M-PAS.