Construct validity, test-retest reliability, and the ability to detect change of the Canadian Occupational Performance Measure in a spinal cord injury population.

Study design: Cross-sectional study.
Objectives: To examine the construct validity and the ability to detect change, of the Italian version of the Canadian Occupational Performance Measure (COPM) in a spinal cord injury (SCI) population. Settings: Rehabilitation service of the Paraplegic Center of Ostia, Italy.
Methods: Thirty-nine spinal cord injury participants were recruited. The clinimetric properties of the measure were assessed following international guidelines. Cronbach's alpha and the intraclass correlation coefficient were assessed for internal consistency and test-retest reliability, respectively. Construct validity was evaluated, by calculating correlation between COPM and the Spinal Cord Independence Measure (SCIM) through Pearson's correlation coefficient and Spearman's Rho. The ability to detect change was evaluated on the overall sample.
Results: The COPM was shown to be reliable in a spinal cord injury sample with positive and statistically significant results for Cronbach's alpha (0.89) and ICC (0.99 for the performance subtest and 0.98 for the satisfaction subtest). Correlation coefficients did not show a correlation between the COPM total score and the SCIM. The COPM scores improved significantly during in-patient rehabilitation, moreover the mean change between the start of treatment and the end of the therapy as evaluated with the Wilcoxon signed-rank test was -4.25 points for the performance score and -2.96 points for the satisfaction score. Conclusions: This study showed that the COPM is a reliable tool for assessing SCI clients' perceived performance of daily activities and their satisfaction with their performance. © International Spinal Cord Society 2019.

Introduction

Spinal cord injury (SCI) is an event that results not only in disorders to normal sensory, motor, or autonomic function but also affects an individual’s physical, psychological, and social well-being [1]. The primary area of focus in rehabilitation includes preventing secondary complications following injury, other goals comprise improving the individual’s independence in the activities of daily living, helping the individual accept the new lifestyle, facilitating community reintegration, and improving quality of life. In the present article, we refer to quality of life (QOL) as health-related quality of life (HR-QOL) [2, 3], which is a subjective and multi-dimensional concept that includes domains related to occupational functioning (physical, mental, social dimensions, spiritual dimensions etc.) [4]. Physical functioning (one of the dimensions of HR-QOL) [5, 6] is defined as the ability to perform different physical activities, ranging from simple ones to more complex ones that require higher rates of endurance, strength, or mobility [7].

Functioning and HR-QOL correspond to the concept of occupational performance, which is defined as the ability to choose, organize, and satisfactorily perform meaningful activities and the tasks of everyday life, which are specific to the individual. Occupation refers to everything that people do, including self-care, leisure, and productivity [8]. The importance that people attribute to specific occupational activities may differ. Researchers have observed that, in various clinical populations and settings, when the clients’ preferences are integrated into evaluations and treatments, benefits in terms of motivation, participation, and functional recovery occur [9-11]. To determine and prioritize client-specific occupational problems and to assess improvement in these problems, Law et al., in 2005, developed the Canadian Occupational Performance Measure (COPM) [12]. The COPM is an individualized, client-centered instrument designed to measure an individual’s self-perception of physical functioning and satisfaction in the activities identified as important by the client. The therapist starts the COPM semi-structured interview by asking the client to identify important daily activities that he or she wants to do, needs to do, or is expected to do but is unable to perform. Areas of everyday living explored during the interview include self-care, productivity, and leisure time. The COPM can be utilized for many different population categories, regardless of diagnosis, as long as the respondents are able to reflect on and report the activities they do every day. Recently, it has been used as an outcome measure on several studies on SCI rehabilitation [13-17]. It is currently used in over 40 countries and has been translated into more than 35 languages. Furthermore, there exists a consensus on the practicability of the COPM for different populations [18-27], as an instrument for establishing target outcomes and planning multidisciplinary interventions by identifying problems in daily activities [28-33]. The evidence from past analyses offers useful and significant insights into its clinical applications; even so, these results cannot be generalized or directly translated to resolve the difficulties encountered when administering formal goal setting actions with SCI individuals [27, 34]. The COPM is a supplemental instrument as designated by the National Institute of Neurological Disorders and Stroke in an overview of all Spinal Cord Injury (SCI) recommendations; and has been used quite extensively in SCI [18]. Gustafsson et al. in 2013 analyzed the clinical utility of the COPM for an SCI population, through a focus group composed of six occupational therapists; however, the clinimetric properties of this tool have never been studied for SCI populations. Usually, for construct validity, all the dimensions of the COPM have been compared with traditional, non-individualized questionnaires constructed to measure the functioning of the specific population in which the construct should theoretically relate with that of COPM [19-28]. Despite difficulties in studying validity in individualized tools such as the COPM [35], no other method yet exists to determine its validity besides testing its correlation with more traditional measures of functioning and quality of life. Clinically, the score that is the most significant in this tool is the change score, which evaluates changes between the client’s satisfaction scores or performance scores in the activities identified by him at the initial assessment and reevaluation of the same activities [19-31]. This means that responsiveness to change over time is a crucial and needed factor of the validation process, although it has often remained unstudied.

The main purpose of the study is to analyze the construct validity, test-retest reliability, and the ability to detect change of the Italian version of the COPM [24] in a sample of SCI clients by comparing its results with the results of a diagnosis-specific measure of functioning: the Italian version of the Spinal Cord Independence Measure Version III (SCIM III) [36, 37]. The SCIM III is a fixed-item questionnaire constructed to measure physical functioning.

Methods

Participants and procedures

Participants were recruited from an Italian rehabilitation service for SCI patients in a spinal unit. The individuals from the chronic SCI population were enrolled in a rehabilitation program after having from pressure injuries. The Occupational Therapy program was part of usual care for these patients. The intervention of occupational therapist (OT) started 30 days after the surgery for pressure injuries with short-sitting in the edge of the bed to monitor skin and sitting tolerance [38]. OTs, like their colleagues in other disciplines, adhered to pressure relief programs and were involved in procurement of appropriate equipment and in activities and specific therapy interventions included in the skin management. The goals of the OT included: procurement of new equipment to address seating needs (pressure mapping, cushion and backrest evaluation, and trials); patient education regarding skin inspection and transfers safety; and the prevention of a new pressure injury. The OT has the responsibility to ensure that treatments match needs are safe and evidence-based. The COPM is often used to determine the priority of each individual when prescribing new equipment and to detect any eventual problem among the activities of daily living to work on. All the participants were also attending physiotherapist’s and nursing therapies.

To qualify for participation in the study, interested participants had to be adults (>18 years old) with chronic SCI (>1 year since injury). All eligible individuals were informed of the study procedures and scope; those who were interested in taking part in the research gave their explicit consent before being included [39]. The participants included in the study were scheduled for three testing sessions (T1,T2,T3). All participants were tested twice in the first week (T1,T2—within 6 days) for test-retest reliability and once in the last week of the rehabilitation program (T3—within 12 weeks)—by the same occupational therapist—to assess the ability to detect change. The problems were identified in the first administration and in the following testing sections the participants were asked to rate again their performance and satisfaction in the same activities. The therapist who administered the Italian translated and culturally adapted version of the COPM [24] was not the same as the one who treated the individual; this choice was made to limit the “hello-goodbye effect,” [40] a phenomenon where clients undervalue their skills at the first administration to get help and after the treatment overvalue their skills. This often happens when individuals are interviewed after treatment by their own therapists [41]. The program, composed of six hours of occupational therapy each week, aimed to improve participants’ daily living independence through activities, training, and aids for everyday life based on the personal activities identified by clients as occupational performance problems during the administration of the COPM.

Instruments

The COPM and the SCIM III were administered together, the first was administered by interview, whereas the SCIM III was self-reported. The COPM [23] is an individualized, client-centered outcome measure. In a semi-structured interview, the individuals, identify upto five occupational activities considered by them as a problem for themselves. This step is the basis for identifying treatment goals. The individuals are then asked to use a 10-point scale in order to assess their current performance and satisfaction with each of the selected issues. The therapist evaluates COPM performance and satisfaction score dividing the sum of the assigned score by the number of problems identified [42]. After the treatment, the individuals are asked again to self-rate their level of performance and satisfaction for the same problems. These scores are used by the therapist to calculate the change scores.

The SCIM III-self report is a tool that assesses the independence of individuals with SCIs. It is comprised of 19 items of daily living activities that are categorized into three subscales that evaluate self-care, respiration and sphincter management, and mobility. The total SCIM III score ranges from 0 to 100, with higher scores reflecting better performance or greater independence.

Reliability, validity, and the ability to detect change

The clinimetric properties of the COPM were evaluated in a SCI sample in accordance to the “consensus-based standards for the selection of health status measurement instruments” (COSMIN) checklist [43]. All the statistical analyses were carried out with the Statistical Package for the Social Sciences (SPSS) version 18.0. Following the COSMIN checklist, the clinimetric properties of the COPM were assessed through Cronbach’s alpha for internal consistency and the intraclass correlation coefficient (ICC) for test-retest reliability. Internal consistency and test-retest reliability were calculated on the performance average score and satisfaction average score and a minimum value of 0.70 was considered minimum acceptable for both coefficients. For construct validity, the correlation between the COPM and the SCIM III was evaluated using Pearson’s correlation coefficient and Spearman’s Rho [44, 45]. The strength of association was interpreted according to Portney and Watkins: 0.0–0.25 little or no relationship; 0.25–0.50 fair association/relationship; 0.50–0.75 moderate to good association; above 0.75 good to excellent relationship [46, 47]. Values greater than 0.5 or less than −0.5 were considered acceptable, to believe that there is support for COPM construct validity, where a positive value meant positive correlation and a negative value meant negative correlation between tools. In analyzing the correlation between the two tools, we expected to find weak correlations between the COPM and the SCIM III because of the structure of the questionnaires, which clearly differed from each other. The COPM was administered a second time after the rehabilitation program, and the first scores were subtracted from the seconds to obtain the change scores following the instruction of the Canadian developers [42]. The participants knew the purpose of the study; however, they were blind or naïve about their scores at admission. Regarding interpretability, that is the degree to which one can assign qualitative meaning to an instrument’s quantitative scores or change in scores, as stated in the COPM manual, a change of two or more points on the performance, satisfaction is considered clinically significant [42]. Moreover, one study has found the cutoff to be 1.37 points and 1.90 points for occupational performance and satisfaction with occupational performance, respectively [44]. However, the Dutch authors used a transition measure, and the change score was the average COPM change score for patients who after their treatment sequence was completed stated that they had improved. The ability to detect change was measured by evaluating changes on the COPM from the start of rehabilitation program (T0) to the end of the treatment period of 12 weeks (T2). To assess the ability to detect change between T0 and T2 for the COPM, a Wilcoxon signed-rank test was used. The significance level has been set for p-value less than or equal to 0.05 [44].

Results

Population and procedures

Thirty nine adults with chronic SCI were recruited from the rehabilitation service of the paraplegic center in Ostia, Rome. All of the 39 eligible participants agreed to participate and were included in the study. The demographic characteristics of the participants are reported in Table 1. The population was evaluated with the COPM and the SCIM III. Table 2 represents frequencies and percentages of the occupation, resulted from the COPM administration, summarized in area of interest. Area 1 represent the first occupational problem mentioned, area 2 the second, and so on. ‘/’ represents “I don’t have a second/third/fourth/fifth problem”. The problems resulted from the administration were: Self-care (e.g., bathroom management), functional mobility with wheelchair (e.g., outdoor mobility), socialization, leisure (e.g., outdoor activities), household management, productivity (e.g., sitting tolerance at work). These problems have been addressed by the OT through analysis and training on the activities of daily living; assessment of bathroom equipment for example proposing new technologies, such as the Transanal irrigation (TAI) a safe bowel management therapy; and procurement of new equipment to address seating needs.

Table 1

Demographic characteristics for the 39 participants in the reliability study of the COPM

Age (mean ± SD)	53 ± 15
Gender men (%)	71
Years from Injury (mean ± SD)	15 ± 14
Neurological Level of injury	Number (%)
C4	1 (2.5)
C5	8 (20.5)
C6	5 (13)
C7	2 (5)
C8	1 (2.5)
D1	1 (2.5)
D3	1 (2.5)
D4	1 (2.5)
D5	3 (8)
D6	4 (10)
D7	2 (5)
D8	2 (5)
D9	1 (2.5)
D10	1 (2.5)
D12	3 (8)
L1	2 (5)
L3	1 (2.5)

Table 2

Frequencies and percentages of the meaningful occupation resulted from the COPM synthetized in area of interest

	Frequency	Percentage
Area 1
Self-care	19	49
Functional mobility with wheelchair	14	36
Leisure	4	10
Household management	2	5
Area 2
Functional mobility with wheelchair	14	36
Self-care	11	28
/	8	20
Household management	4	10
Productivity	1	3
Leisure	1	3
Area 3
/	17	44
Leisure	8	20
Functional mobility with wheelchair	6	15
Household management	5	13
Self-care	3	8
Area 4
/	28	71
Self-care	3	8
Functional mobility with wheelchair	3	8
Leisure	2	5
Household management	1	3
Productivity	1	3
Socialization	1	3
Area 5
/	35	90
Household management	2	5
Functional mobility with wheelchair	1	3
Socialization	1	3

Note: Area 1 represents the first occupational problem mentioned, area 2 the second, and so on. ‘/’ represents “I don’t have a second/ third/ fourth/ fifth problem”

The COPM had a high degree of internal consistency with a Cronbach’s alpha value of 0.93 for performance score and 0.89 for satisfaction score (T0). In addition, the tool had a high ICC value, which indicates great test-retest reliability, after repeated measurements with a value of 0.99 for the performance and 0.98 for the satisfaction. The reliability values are reported in Table 3. Furthermore, the COPM’s construct validity with the SCIM III did not demonstrate a statistically significant correlation (p < 0.05), neither with the Pearson’s correlation coefficient nor with the Spearman’s Rho coefficient, as shown in Table 4. However, Pearson’s correlation coefficient performed between the first occupational problem mentioned by participants and SCIM III showed moderate to excellent relationship as shown in Table 5. For the ability to detect change of the measure, the null hypothesis that no clinical change would occur between T0 and T2 was rejected, as shown in Table 6. In fact, the mean change between T0 and T2 as evaluated with a Wilcoxon signed-rank test was −4.25 for the performance score and −2.96 for the satisfaction score, respectively. Therefore, the improvement on the COPM scores during the treatment period were not only clinically significant ( > −2.0 points), as suggested by the Canadian developers of the tool [43], but also statistically significant (p < 0.01).

Table 3

Test-retest analysis: Range of ICC parameters of each item the Italian version of the COPM

	Mean test (SD)	Mean (SD) retest	ICC	CI [95%]	p
COPM performance	4.5 (2.2)	4.6 (2.2)	0.99	0.992–0.996	<0.01
COPM satisfaction	4.6 (2.5)	4.7 (2.4)	0.89	0.98–0.94	<0.01

Table 4

Construct validity analysis: Spearman’s Rho correlation and Pearson’s correlation coefficient between COPM and the Italian version of the SCIM III

Spearman’s Rho	SCIM	Mean performance T0	Mean satisfaction T0
SCIM	1	0.30	0.17
Mean performance T0	0.30	1	0.80a
Mean satisfaction T0	0.17	0.80a	1

aCorrelation is significant at the 0.01 level (2-tailed)

Table 5

Pearson’s correlation coefficient between COPM (first occupational problem mentioned by participants) and the SCIM III

Area 1-the first occupational problem mentioned for COPM	Self-care subscale	Respiration and sphincter management subscale	Mobility subscale	SCIM III Total
Self care-mean performance. T0	−0.63a	−0.74a	−0.60a	−0.73a
Self care-mean satisfaction. T0	−0.571b	−0.66a	−0.51b	−0.66a
Household management -mean performance. T0	0.99a	0.99a	0.99a	0.99a
Household management- mean satisfaction. T0	0.97a	0.97a	0.91a	0.96a
Functional mobility with wheelchair -mean performance. T0	0.54b	0.49b	0.60a	0.56b
Functional mobility with wheelchair -mean satisfaction. T0	0.63a	0.51b	0.64a	0.62a
Leisure-mean performance. T0	0.77a	0.84a	0.73a	0.93a
Leisure-mean satisfaction. T0	0.80a	0.85a	0.77a	0.91a

aCorrelation is significant at the 0.01 level (2-tailed)

bCorrelation is significant at the 0.05 level (2-tailed)

Table 6

Ability to detect change: responsiveness of the COPM Wilcoxon signed-rank test on 39 participants

	T0 mean±SD	T1 mean±SD	Mean difference±SD [CI 95%]	p
COPM performance	4.55 ± 2.21	6.20 ± 2.12	−4.25a	0.001b
COPM satisfaction	4.62 ± 2.49	6.54 ± 2.27	−2.96a	0.003b

aBased on negative ranks

bCorrelation is significant at the 00.01 level

Discussion

Consistent with the structure of the COPM [43], the change score is the most relevant. It measures change between the scores at an initial evaluation and at reevaluation; this means that although not commonly studied, the assessment of the responsiveness to change over time is a crucial and needed factor for the validation process. In the last few years, the COPM has been used as an outcome measure in several studies of spinal cord injury rehabilitation [13-17]. However, its clinimetric properties, particularly the ability to detect change, have never been analyzed for this population prior to this study. Therefore, this research focused on the ability to detect change of the measure and the construct validity of the COPM in a sample of persons with SCI. The 39 respondents were evaluated at T0 with the COPM and the SCIM III, and at T2, after a 3-month rehabilitation program based on the daily activities that were reported during the administration of the COPM. During the administration of the COPM, the first activities that the participants wanted or needed to work with to improve their satisfaction were related mainly to personal care (39%) and functional mobility (20%). The rehabilitation program was specific to each individual and based on the activity he or she identified. For example, for personal care, treatments were oriented to training on assistive technologies identified by the therapist, and for functional mobility, treatments were oriented to wheelchair configuration and postural alignment.

The clinimetric properties of the COPM in a sample of SCI clients demonstrated positive values for internal consistency, test-retest reliability, and the ability to detect change (Table 3-5), proving that the COPM is a suitable and reliable tool for assessing the achievement of results in terms of satisfaction and the perception of improvement in performance. The construct validity results showed that the SCIM III was not linked to the COPM total score, which is expressed by the mean scores for performance and satisfaction of all occupational problems identified by clients. In discovering this correlation, the results of this study coincide with previous ones [9, 29, 33, 40, 48–51]. However, not all participants expressed the same number of problems, performing the Pearson’s correlation coefficient between the first occupational problem mentioned as by participants and SCIM III subscale resulted in moderate to excellent relationship. The study’s results suggest that one explanation for the lack of correlation between the COPM and other existing measures is the strong value of client-specific items in rehabilitation intervention planning and monitoring at an individual level [33].

Limits of the study

This study has some limitations. The sample size was too small to allow comparisons among the problem category distribution of those interviewed. For example, correlations with the neurological level of lesions, years since injury, and so on. Furthermore, future research should first determine the most common priorities of the studied population and then select the comparison tool for construct validity or the ability to detect change testing of the COPM [33].

Conclusions

Based on the results of this study, it can be concluded that the COPM is a suitable tool for SCI populations when used to determine treatment goals and measure the perception of improvement in everyday-life independence in SCI clients. This work represents a starting point for the in-depth study of SCI patients’ perception of their disability and represents an important step for clinical and research practice, especially for qualitative studies.