The Persian Version of a Participation Scale: Is It Valid and Reliable Enough for Use among Iranian Patients with Multiple Sclerosis?

BACKGROUND AND
PURPOSE: The aim of this study was to determine the face and criterion validity, stability reliability, and internal consistency of the Persian version of the Impact on Participation and Autonomy (IPA-p) scale among Iranian people with multiple sclerosis (MS).
METHODS: Trained experts interviewed 364 MS patients and their relatives to assess the criterion validity, stability reliability, and internal consistency of the IPA-p scale. Ten specialists from different disciplines were also recruited to assess its face validity. A consent form was completed by the patients and their relatives. Internal consistency reliability was measured using Cronbach's alpha and stability reliability was assessed using interclass correlation coefficients (ICCs). The test-retest method was used to detect the reliability of the questioner. The study subjects completed the IPA-p scale on two occasions separated by an interval of 30-45 days. Study checklists were also used to assess the face validity, stability reliability, and internal consistency of the IPA-p scale.
RESULTS: About 50% of the respondents reported their perceived overall participation to be "good" or "very good" and 60% of the specialists rated the ability of the IPA-p scale to measure what it was designed for as "excellent." Spearman correlation coefficients were >0.8 for all but one IPA-p domain. Cronbach's alpha between the mean IPA-p scale scores achieved on two separate occasions ranged from 0.858 to 0.913. The highest and lowest internal consistencies belonged to the "social relationships" and "education and learning" domains, respectively. The test-retest ICCs for the nine domains were between 0.789 and 0.919, and all were significant at p<0.001.
CONCLUSIONS: The IPA-p questionnaire can be considered a valid and reliable instrument for assessing self-reported participation among Iranian MS patients.

INTRODUCTION

The concepts of quality of life (QOL), disability, impairment, and handicap are used increasingly in the medical and health sciences due to a perceived need in these fields. Health systems prioritize diseases according to the severity of the associated disability in both individuals and the community.

Multiple sclerosis (MS) is a major neurological disorder that affects many people worldwide, both in developed and developing countries.1 MS encompasses a variety of conditions, all of which include some level of disability. As a result of the increasing life expectation among MS patients, they live with the health and social consequences of their condition for a long time.2 Unsurprisingly, disability due to the disease affects the QOL of patients in all aspects of their life.3

A common method for estimating the magnitude of the issues they face is using self-reported data collected via questionnaires.4 There are many general questionnaires that have been used to assess the QOL of patients with MS. There are also some dedicated questionnaires, such as the 54-item Multiple Sclerosis Quality of Life instrument, which were designed specifically to assess the QOL of people with MS.5 Nevertheless, QOL questionnaires have been criticized for the emphasis they place on measuring the subjective satisfaction of individuals.6 In particular, the presence of cognitive impairment-which is reported to have an important impact on the reporting of QOL7-among 40-60% of MS patients8,9 makes QOL questionnaires potentially problematic as health outcome measures in this population. Given these difficulties, a more accurate assessment of the outcome of disease and related interventions has been achieved using handicap questionnaires, such as the London Handicap Scale.10,11

The World Health Organization model of disablement defines handicap as "a disadvantage for an individual that limits or prevents the fulfillment of a role that is normal for that individual".12 More recently, the term "participation" has been used in measures aimed at assessing the abilities of individuals in a range of activities for the purpose of improving the level of health and QOL among communities.13

The Impact on Participation and Autonomy (IPA) scale is a questionnaire that was designed to measure the level of participation among people with neurological disorders.11 This questionnaire was originally developed in Dutch and has been translated for use among English-language speakers with neurological diseases. Its reliability and validity have been reported previously.14 The IPA scale was validated during its development; it has been used in numerous studies and is considered to be valid and relevant instrument.

There is currently no questionnaire designed to measure the level of participation specifically among Persian-language-speaking populations with neurological disorders. The present study was designed to test the face and criterion validity, stability reliability, and internal consistency of the Persian version of the IPA (IPA-p) scale among a group of Iranian people with MS.

METHODS

All people with MS registered in the two Special Disorders Clinics of Rafsanjan and Kerman, Southeast Iran, were invited to take part in this study, and 450 patients were sent information about the objectives of the study. MRI images revealed MS plaques in the CNS of all of the patients, and the disease had been diagnosed by a neurologists.

Of the 450 patients who were invited to take part, 364 agreed to help assess the reliability (stability and internal consistency) and criterion validity of the new IPA-p scale. In addition, ten experts from different disciplines were recruited to assess the face validity of the scale.

The original version of the IPA scale was translated for use among English-language speakers with neurological disorders. The English version (the IPA-e scale) was found to be acceptable for use among English language people with MS,15 although suggestions for extra items in the scale were made.

Eight domains of participation are measured by the IPA scale, and there are two additional questions: one is a general question asking respondents about their overall perception of their participation [perceived overall participation (POP)] as the ninth domain, and the other asks respondents to report the three most important problems that they experience as a result of their health condition. The eight domains examined as predictors of POP were "mobility," "self care," "family role," "looking after money," "leisure," "social relationships," "paid work," and "education and learning."

Forward and backward translation was performed by two bilingual (English and Persian) experts to generate the IPA-p scale. The translated version was then discussed by an expert panel, including eight university colleagues with related disciplines, to ratify the new IPA-p scale. The validity (face and criterion) and reliability (stability and internal consistency) of the IPA-p scale was measured using the following steps.

Face and criterion validity

Ten specialists (experts) agreed to participate in the assessment of the face validity of the IPA-p scale, comprising three epidemiologists (one professor, one associate professor, and one assistant professor), three psychologists (one assistant professor and two associate professors), and four neurologists (one professor, two associate professors, and one assistant professor). These specialists were visited at their offices at three Iranian national universities. They received full details about the IPA scale and the objectives of the study. A copy of IPA-p scale was presented to each of the ten specialists, along with a structured checklist. They were asked to review the scale and to complete the checklist items within a few days. The checklists were collected when they were completed. There were several items on the checklist that were designed to measure the specialists' opinion about the extent to which the IPA-p scale items as a whole and within each section, were able to assess what they were supposed to. The main objective of asking these items was to identify first whether "people understand the concept of the questions," and second whether "people understand the questions in the same way as the investigators do."

Since there is currently no instrument that can be used as a gold standard against which the results obtained with the IPA-p scale can be compared, criterion validity was assessed by collecting objective data about the nine domains of the respondents' (i.e., MS patients) life from their friends or relatives with whom they were living (mother, father, brother, sister, spouse, children, and friends). Trained colleagues were asked to conduct an in-depth interview with the patients' relatives in face-to-face sessions using a separate checklist. There were only nine general items enquiring about the performance of MS patients, one for each domain on the IPA-p scale. One mean score for each domain of the IPA-p scale was calculated from the data collected. These means were compared with the scores obtained from the nine items on the checklist, which were considered to present gold-standard results.

Stability reliability and internal consistency

For the purpose of assessing stability reliability and internal consistency of the IPA-p scale, MS patients were invited to participate in face-to-face interview sessions. The respondents received all of the details about the research and its objectives while they were waiting to visit their neurologist in the clinic. A checklist was used to record some demographic details. The respondents completed the IPA-p scale (and/or their caregivers/relatives if needed) after providing written consent to participate. The interviewer also recorded the time taken for individual respondents to complete the scale. The respondents were asked to leave any items that could not be understood by either themselves or the people accompanying them.

The respondents were also invited to complete the IPA-p scale a second time (the retest) 30-45 days later at the same clinic and under the same conditions as on the first occasion. The test-retest method was used to establish the reliability of the questionnaire. Internal consistency reliability for each section was measured using Cronbach's alpha16 and stability reliability was assessed using interclass correlation coefficients (ICCs) between scores achieved on first and second attempts at the questionnaire. ICCs for the rating scale and the kappa statistic were used for analyses.17 Except where stated otherwise, the data are presented as mean±SD values.

All of the questions (items) in the IPA-p scale were analyzed separately and by a single researcher; however, due to the large number of questions, only the results based on the nine domains are reported here.

RESULTS

Of the 450 MS patients who were invited to participate in this research, 364 agreed to complete the IPA-p scale a second time. The time taken for the respondents to complete the IPA-p scale was 16.8±2.7 minutes (range, 12.2-23.2 minutes). The demographic characteristics of these patients are presented in Table 1. The respondents were 34.1±8.2 years old, and their age did not differ significantly between the men (35.4±7.5 years) and women (33.9±8.3 years). About 15% of the respondents reported their POP as "low" or "very low," and about 50% of reported their POP as "good" or "very good."

Table 1

Demographic characteristics and perceived overall participation (POP) status of respondents

Characteristic	n	%
Age (years)
<30	104	28.6
30-49	240	65.9
>49	20	5.5
Gender
Female	232	91.2
Male	32	8.8
Education status
Illiterate	24	6.6
<Diploma	144	39.5
Diploma	112	30.8
>Diploma	84	23.1
Marital status
Single	60	16.5
Married	304	83.5
Social status
Low	64	17.6
Moderate	192	52.7
High	108	29.7
POP
Very low	16	4.4
Low	36	9.9
Moderate	124	34.1
Good	144	39.5
Very good	44	12.1
Disease duration (years)
<5	180	49.5
5-9	172	47.2
>9	12	3.3

The opinions of the specialists regarding the nine domains of the IPA-p scale are given in Table 2. The majority confirmed that the items in the nine domains of the IPA-p scale were indeed measuring what were they were supposed to. Only one of the specialists (10%) believed that IPA-p scale as a whole was not "good" or "excellent" for assessing the impact of MS on patient participation and autonomy. The ability of this Persian version of the IPA scale to measure what it was designed for was rated as "excellent" by 60% of the specialists (Table 2).

Table 2

Specialists' judgments about the adequacy of the items in the Persian version of the Impact on Participation and Autonomy (IPA-p) scale for assessing different domains

Domain	Judgment
	Unacceptable		Poor		Acceptable		Good		Excellent
	n	%	n	%	n	%	n	%	n	%
Mobility	0	0	0	0	0	0	2	20	8	80
Self care	0	0	0	0	0	0	3	30	7	70
Household tasks	0	0	0	0	1	10	3	30	6	60
Looking after money	0	0	0	0	2	20	2	20	6	60
Leisure	0	0	0	0	2	20	3	30	5	50
Social relationships	0	0	0	0	1	10	2	20	7	70
Paid work	0	0	0	0	2	20	4	40	4	40
Education and learning	0	0	1	10	1	10	6	60	2	20
Living life as a whole	0	0	0	0	1	10	3	30	6	60

Since there was no participation scale that could be used as a gold standard, criterion validity was evaluated by comparing the IPA-p data with objective data collected from relatives living with the MS patients. Spearman correlation coefficients between the nine domains of the IPA-p scale and the nine objectively addressed items are presented in Table 3. The results show that based on objective data, the IPA-p items are strongly valid. Correlation coefficients between the objective data and the mean scores of the IPA-p domains were all higher than 0.8, except for "social relationships," for which the correlation coefficient was 0.688; however, this correlation was also statistically significant.

Table 3

Correlations between the respondents' scores on the IPA-p scale and scores obtained objectively for the nine domains

Domain	Correlation
	Pearson correlation coefficient	Number of respondents	p
Mobility	0.829	364	<0.001
Self care	0.851	364	<0.001
Household tasks	0.872	364	<0.001
Looking after money	0.979	364	<0.001
Leisure	0.971	364	<0.001
Social relationships	0.688	364	<0.001
Paid work	0.88	72	<0.001
Education and learning	0.959	112	<0.001
Living life as a whole	0.958	360	<0.001

IPA-p: Persian version of the Impact on Participation and Autonomy.

Internal consistency reliability

Cronbach's alpha coefficients between the mean scores of the nine domains for the two completions of the IPA-p scale ranged from 0.858 to 0.913, indicating high internal consistency for all domains of the IPA-p scale (Table 4). The highest and lowest internal consistencies belonged to the "social relationships" and "education and learning" domains.

Table 4

Internal consistency and stability of the different domains of the Persian version of Impact of Participation and Autonomy scale

Domains	Internal consistency	Stability (test-retest reliability)		ICC
	Cronbach's α	Test score	Retest score
Mobility	0.956	2.41±0.73 (0.8-3.6), n=364	2.38±0.74 (0.8-3.6), n=364	0.916
Self care	0.957	3.27±0.43 (2-4), n=364	3.27±0.42 (2-3.67), n=364	0.918
Household tasks	0.924	2.68±0.62 (1.14-3.71), n=364	2.63±0.63 (1.14-3.71), n=364	0.859
Looking after money	0.875	1.45±0.92 (0.0-3.0), n=364	1.34±0.92 (0.0-3.0), n=364	0.852
Leisure	0.957	1.17±0.78 (0.0-2.5), n=364	0.99±0.81 (0.0-2.5), n=364	0.919
Social relationships	0.882	2.96±0.47 (1.88-3.75), n=364	2.95±0.48 (1.75-3.75), n=364	0.789
Paid work	0.886	3.28±0.42 (1.5-3.0), n=72	2.35±0.45 (1.5-3.0), n=72	0.872
Education and learning	0.858	2.00±0.65 (0.5-3.0), n=112	2.02±0.65 (0.5-3.0), n=112	0.819
Living life as a whole	0.960	1.90±0.65 (0.5-3.0), n=360	2.00±0.77 (0.33-3.33), n=364	0.923

Data are mean±SD (range) values; the number of respondents for whom the data were collected is also provided.

ICC: interclass correlation coefficient.

Test-retest reliability

Of the 450 MS patients who were invited to participate in this study, the retest questionnaire was completed by 364 at the follow-up session. However, some of the respondents did not answer all of the items for the 2 domains of "paid work" and "education and learning," with only 72 and 112 respondents, respectively, answering all items for these 2 domains. The mean scores of the respondents for the nine domains for the first and second attempts at completing the IPA-p scale are presented in Table 4. The test-retest ICCs for the nine domains ranged between 0.789 and 0.919, and all were significant at p<0.001, indicating a considerably high consistency of responses to individual questions.18 All items also had a kappa value of >0.8.

DISCUSSION

The findings of this study suggest that completing the IPA-p scale is not too time-consuming for people with MS (16.8±2.7 minutes). This is an important concern for research scientists and clinicians using instruments such as the IPA in research and in medicine. It is also important when postal methods are used for collecting data to encourage a high response rate, because fatigue is one of the most prevalent symptoms of MS.19

Based on the opinions of the Persian specialists in this study, the IPA-p scale demonstrated very good face validity, with only "education and learning" rating as "poor" with respect to the ability to measure what it was meant to (Table 2). This should be taken into account by researchers who plan to develop new Persian participation scales using the domains of the IPA scale as their core. However, the IPA-p scale overall demonstrated has good face validity in this study. For six domains, the highest proportion of the specialists who believed the items were either not "good" or not "very good" was 20%; the remaining 80% rated items in different domains as "good" and "very good."

Since there was no gold standard scale for measuring the criterion validity of the IPA-p scale, objective data (collected from relatives) were used. Based on these objective data, the criterion validity of the IPA-p scale was also very high. The respondents (Persian-language-speaking MS patients) answered the IPA-p items accurately by comparison with the collected objective data. However, the lower correlation coefficient between scores obtained from the subjective assessment of the respondents regarding their performance in the "social relationships" domain and the scores reported based on objective assessment of this domain could be important. For instance, in the "social relationships" domain, not only is the role of MS patients important, but so also is the reaction of other people in the community, which could have a significant effect on the quality of this relationship. However, for other domains such as "mobility" or "self care", only the role of the MS patients themselves will affect the situation. This shows that people with MS are able to cope with their own situation, but when they are not able to control the behavior and attitudes of others in the community the situation may become undesirable for them. When designing a questionnaire for assessing the QOL or participation, this point should not be ignored.

As Kersten et al.20 reported when developing the IPA-e scale, the present findings confirm that the IPA-p scale is sufficiently valid for use among Persian-language-speaking MS patients.

Based on the test-retest findings and both the ICC and kappa statistic results, the reliability coefficient was also relatively high for almost all of the items, with ICC and kappa values of >0.8.21 On the other hand, Cronbach's alpha coefficients ranged from 0.858 to 0.913, indicating high internal consistency for all domains of the IPA-p scale. However, less than one-third of the respondents in the present study answered items on employment, and this could be a weakness of the scale. Yorkston and colleagues concluded that there is a significant association between the employment and participation scores among MS patients.22

Memory is an unavoidable problem in retest situations, since subjects may remember how they answered the questions previously and attempt to reproduce those answers during the retest. In part to minimize over- or underestimation of reliability (due to the influence of memory or actual change in work conditions), the research team chose to impose a 4- to 6-week interval between the first (test) and second (retest) attempts at the IPA-p.

Notwithstanding these limitations, the present findings show that despite the cultural differences between the two nations of the Netherlands and Iran, items on the IPA-p scale can reliably measure the participation of Iranian people with MS from their own perspective. This means that even if a new Persian participation scale was to be designed for use among Persian-language speakers with MS, the IPA-p items could provide a suitable core for the new scales.

The level of disease severity among the respondents was not considered in the present study, but this could be an important factor. However, we believe that the severity of disease and the resulting disability are important when measuring QOL, which is a subjectively measured concept. Measuring participation is different due to its objective nature and it being less sensitive to the severity of disease compared with QOL. However, it is strongly recommended that the effect of disease severity on the level of participation among MS patients be considered in future investigations.

The findings of this study provide evidence for a high level of face and criterion validity, stability reliability, and internal consistency of the IPA-p scale among a group of Iranian people with MS. However, it seems that despite the suitability of this scale, some improvements are necessary to develop a more complete scale, such as adding items about the respondents' ability to complete their regular tasks (i.e., activities of daily living).

In conclusion, the results of this study show that the IPA-p scale has a good conceptual structure and provides reliable information on the respondents' participation. This questionnaire can be considered a valid and reliable instrument for assessing self-reported participation among Iranian people with MS. This study is the first to have considered participation and measured it as an outcome metric among a group of Iranian MS patients. More investigations are needed to develop Persian participation scales for using among Persian-language-speaking populations and to assess the possibility of adding some practical items, such as those suggested by Kierkegaard et al.,23 who concluded that the proposed cutoff values in the 2×5-m walk test and the Nine-hole Peg Test could be used as indicators of functioning and to identify the respondent's risky activity limitations and participation restrictions.

Clinical message

The IPA-p scale appears to be a valid and reliable measure for use among Persian-language-speaking MS patients. However, some improvements could increase the utility of this scale.