Performance of the measures of processes of care for adults and service providers in rehabilitation settings.

INTRODUCTION: In recent years, client-centered care has been embraced as a new philosophy of care by many organizations around the world. Clinicians and researchers have identified the need for valid and reliable outcome measures that are easy to use to evaluate success of implementation of new concepts.
OBJECTIVE: The current study was developed to complete adaptation and field testing of the companion patient-reported measures of processes of care for adults (MPOC-A) and the service provider self-reflection measure of processes of care for service providers working with adult clients (MPOC-SP(A)).
DESIGN: A validation study. SETTINGS: In-patient rehabilitation facilities. MAIN OUTCOME MEASURES: MPOC-A and measure of processes of care for service providers working with adult clients (MPOC-SP(A)).
RESULTS: Three hundred and eighty-four health care providers, 61 patients, and 16 family members completed the questionnaires. Good to excellent internal consistency (0.71-0.88 for health care professionals, 0.82-0.90 for patients, and 0.87-0.94 for family members), as well as moderate to good correlations between domains (0.40-0.78 for health care professionals and 0.52-0.84 for clients) supported internal reliability of the tools. Exploratory factor analysis of the MPOC-SP(A) responses supported the multidimensionality of the questionnaire.
CONCLUSION: MPOC-A and MPOC-SP(A) are valid and reliable tools to assess patient and service-provider accounts, respectively, of the extent to which they experience, or are able to provide, client-centered service. Research should now be undertaken to explore in more detail the relationships between client experience and provider reports of their own behavior.

Introduction

In the past decade, researchers and clinicians around the world have begun systematically to ask their clients about the important aspects of health care that make the processes and experiences easier and more helpful to them. Not surprisingly, patients and their families want to be sufficiently informed about the illness and their progress to be able to participate in what is happening to them; to be treated respectfully and with dignity; to be confident that they receive comprehensive and well-coordinated care; and to feel good about asking questions.1–3 Based on what people have reported, many frameworks have been developed of what we now call client- or person-centered care.1–4 In recent years, this new philosophy has been embraced by many organizations around the world. The literature has identified the need for valid, reliable, and easy-to-use outcome measures to evaluate success of implementation of new concepts.4–6 Moreover, measures are needed that will allow clients to evaluate the quality of care, and enable clinicians to be able to reflect on their practices and beliefs and identify areas that require attention. In the past, satisfaction with care measures were often used to assess the quality of care. However, these tools tend to have high ceiling effects and do not provide a comprehensive picture of clients’ and clinicians’ experiences.7,8

In 2010, we reported the validation of the adapted measure of processes of care (MPOC) from the perspectives of adult clients undergoing elective joint replacement.7 At the time of that study, the measure was the only available generic tool that allowed evaluation of the client- and family-centeredness of care in adults. In the last decade, several new measures have been developed and implemented in research studies, ranging from disease-specific tools to questionnaires applied in specific health care settings. Examples of these new tools for clients include the Person-centred Climate Questionnaire, available in Swedish- and English-patient versions;4,5 Client-centred Care Questionnaire developed for use in home care;8 Client-centred Rehabilitation Questionnaire;9 and Patient-centredness Questionnaire-Infertility.10 Similar tools were developed to assess health care professionals’ (HCPs’) experiences: the Person-centred Care Assessment Tool developed by Sjögren et al and Rokstad et al and validated in several languages;11,12 Patient–Practitioner Orientation Scale;13 and the Nursing Context Index.14 The strengths and limitations of some of the tools mentioned above were well described in the comparative review of published tools completed by Edvardsson and Innes in 2010.15

Despite many new tools available today, we believe that the MPOC family of measures provides several advantages.16 First, unlike other tools, the multidimensionality of client-centeredness is preserved by presenting the results of each domain separately, with no composite score. Second, the perspectives of all the participants in the health care encounter can be assessed simultaneously with client and service provider versions that were developed based on the same theoretical framework. Third, unlike most tools that were developed for research purposes, the original MPOC and the MPOC for service providers (MPOC-SP) have been translated to many languages and used to assess the quality of care in child programs in many countries.16 The adult versions adapted in this research program, although still under development, have already been used in several countries for program evaluation (Carmen Hall [personal communication, October, 2012], Julie Evans [personal communication, March, 2012]).

CanChild Centre for Childhood Disability Research has developed a family of self-report outcome measures that assess family-centered experiences of parents of children with disabilities (MPOC) and of service providers (MPOC-SP). The psychometric properties of these tools are well documented in a variety of pediatric settings and countries around the world and they have been translated into several languages (ie, French, Danish, Japanese, Arabic, Hebrew, and Portuguese).17–19 A manual for use and interpretation of the MPOC is available by writing to CanChild (http://www.canchild.ca).

In the past decade, adaptation of the MPOC for adults was undertaken as part of an occupational therapy student project under the supervision of one of the developers of the original version. Students undertook an extensive literature review to help identify important characteristics of client-centered care from adult patients’ and caregivers’ perspectives. MPOC-56 and MPOC-20 were used as a base for the adapted version. The original domains were retained, and the wording of the questions was minimally changed to reflect a different target population. The MPOC of adults (MPOC-A) consisted of 34 of the original 56 questions that were found to be applicable based on the literature review. In 2008, the MPOC-A was field-tested with patients and families after elective hip-or knee-replacement surgery.7 Following publication of the results of that pilot study, inquiries about administration, scoring, and interpretation of the tool were received from across Canada (Quebec, Alberta) and abroad (the Netherlands, USA, Italy, Brazil, Australia). In addition, researchers and clinicians were asking if the companion measure, similar to the MPOC-SP, was available for use with service providers caring for the adult population.

The objective of the current study was to complete adaptation and field testing of the companion MPOC for service providers working with adult clients (MPOC-SP(A)) and the MPOC-A. These clinical measures capture “the extent to which” patients, their families, and health professionals experience partnerships.19,20 The main question was: are MPOC-A and MPOC-SP(A) valid and reliable measures of client-centeredness in adult rehabilitation settings. To assess the measurement properties and performance of these tools, the following hypotheses were tested.

For MPOC-SP(A):

Internal consistency: Cronbach’s α ≥0.7 was expected for each scale of the measure if the items are measuring the same domain.

Exploratory factor analysis would support the four-factor structure of MPOC-SP(A).

MPOC-SP(A) would be able to detect differences across programs.

For MPOC-A:

Internal consistency: Cronbach’s α ≥0.7 was expected for each scale of the measure if the items are measuring the same domain.

Construct validity: Pearson product-moment correlations of 0.5–0.7 were expected between MPOC-A scale scores and a standardized general measure of client satisfaction with treatment.

Interrater reliability: intraclass correlation coefficients (ICCs) between 0.5 and 0.7 were expected between patients’ and families’ scores on MPOC-A.7

Methods

The data used in this study have been collected in three projects (over the past 3 years). Our main goal was to look at performance of the measures, and not to assess any specific program performance (we were seeking responses from people who had common experiences of care). Hence, we were not concerned with generalizability of the results and no information on the response rate was collected at this time.

First, all HCPs and clients (patients and family members collectively) of four inpatient stroke rehabilitation units across South-Central Ontario, Canada, were invited to respond to MPOC-SP(A) and MPOC-A, respectively. HCPs were included if they had been practicing in the facility for at least 3 months. Clients were included if they were expected to remain within the program for at least 2 weeks, and had been receiving treatment for at least 2 weeks (so as to have some perspective on service quality). Exclusion criteria included lack of English fluency (as all questionnaires are currently available in English only), and severe cognitive impairment as assessed by the Montreal Cognitive Assessment (<19) or the mini-mental state examination (<21) (routine outcome measures used by stroke centers).

Second, data from the use of MPOC-SP(A) were collected during Speech-Language Demonstration Project Person, Child and Family-Centred Care Committee, Alberta Health Services. All speech language pathologists practicing in the province of Alberta were invited to respond to a survey that included MPOC-SP(A). The responses of speech language pathologists working with adult clients were identified and included in the current study.

The third study was conducted by the Department of Veterans Affairs in the USA. Clinical staff in the polytrauma network sites and polytrauma support clinical teams were invited to participate. The results pertaining to MPOC-SP(A) were included in the present study.

The rationale for combining the studies was grounded in the belief that since all three settings share a similar philosophy of care, similar trends in responses can be expected. However, we would also expect certain dissimilarities due to different settings and professionals involved, allowing testing of construct validity by exploring correlations between MPOC-SP(A) scores reported by different programs.

The Grand River Hospital Research Committee, Hamilton Health Sciences/McMaster University Faculty of Health Sciences Research Ethics Board, Brant Community Health Care System Ethics Committee, Hotel Dieu Shaver Health and Rehabilitation Centre Research Ethics Board, and Conjoint Health Research Ethics Board, Alberta approved the protocols of the studies.

Instruments

The MPOC-A is a self-administered 34-item questionnaire with positively worded short statements and a seven-point Likert-type response scale. MPOC-A demonstrated strong psychometric properties in the previous study.7 The stem question for all items is: “To what extent do health care providers who work with you …”. The response scale ranges from “to a very great extent” to “not at all” (from 7 to 1, respectively), and a “not applicable” box is also provided (rated as 0). Five main domains have been identified for MPOC-A based on extensive literature review and previous work in pediatric settings.3,8,19,21 The domains are labeled as: Enabling and Partnership; Providing General Information; Providing Specific Information; Coordinated and Comprehensive Care; and Respectful and Supportive Care. Each domain score, ranging from 0.0 to 7.0, is computed by averaging the ratings for the items of that domain. The “not applicable” responses are treated as missing data and any questionnaires that have more than 50% missing and “not applicable” answers combined are discarded. In order to be able to calculate the mean of the scale, at least two-thirds of valid responses have to be present for that domain’s items.

The MPOC-SP is a 27-item self-administered questionnaire. The items represent four domains, and the stem question and domain score calculation procedures are identical to MPOC-A. MPOC-SP has demonstrated good validity and reliability in previous testing in pediatric settings.18 However, no proper validation of MPOC-SP in adult health care has been undertaken.

The Client Satisfaction Questionnaire (CSQ) is an eight-item global satisfaction measure with reported psychometric properties (coefficient α of 0.92, Pearson correlation of 0.56, P<0.01 for estimates of client satisfaction by health provider).21 CSQ scores have been shown to be moderately correlated with MPOC-A scores.7

One of the goals of this study was to assess the factor structure of the MPOC-A and MPOC-SP questionnaires. The general rule of thumb for this procedure is to have a minimum of five, and preferably ten, subjects per variable.22 MPOC-A has 34 items and MPOC-SP has 27 items, so samples of at least 200 patients and 140 HCPs, respectively, were planned.

Statistical analysis was conducted with the Statistical Package for the Social Sciences (SPSS Version 20; IBM Corporation, Armonk, NY, USA). The range and the skewedness of responses were explored, including differences according to demographic characteristics, such as age and sex. The internal consistency was estimated using Cronbach’s α coefficient. The factor analysis was performed where sample size permitted. Pearson product-moment correlation coefficients between the scales of the current measure and Global Satisfaction Scale were calculated to assess construct validity. The interrater reliability of MPOC-A was assessed with the ICC.

Results

MPOC-SP(A)

A total of 384 valid responses were available for the analysis (Ontario: 54 HCPs, Alberta: 45, and USA: 285). The Alberta study sample consisted of speech language pathologists employed in different health care settings across the province. The Ontario and US samples consisted of different health professionals employed in inpatient rehabilitation (nurses, physiotherapist, occupational therapists, social workers, doctors, etc). The data of the three samples were examined for differences in summary statistics and distribution. In addition, all presented analyses were completed on each sample as well as on pulled data. Since no differences were identified, the results for the combined sample were presented.

The full range of available responses was used in the majority of questions (excepting questions 3, 6, 13, and 22 in which the lowest score was not used). The skewedness and kurtosis values were all between −1 and 1 allowing use of parametric statistics for further analysis. The Internal Consistency varied from 0.67 to 0.88. Due to the small number of items in the Communicating Specific Information domain, the split-half Spearman–Brown prediction formula was applied and the coefficient was 0.71. A summary of domain means, range of scores, and internal consistency is presented in Table 1. Pearson product-moment correlation coefficients between domains varied from 0.40 to 0.78.

Table 1

Descriptive statistics and internal consistency as assessed by Cronbach’s coefficient α of measure of processes of care for service providers working with adult clients domains

Domain (number of items) [N]	Mean (standard deviation)	Minimum–maximum	Internal consistency (Cronbach’s α)
Showing Interpersonal Sensitivity (10) [344]	5.1 (0.92)	2.4–7	0.88
Providing General Information (5) [329]	4.1 (1.36)	1–7	0.88
Communicating Specific Information (3) [368]	4.8 (1.17)	1.5–7	0.67*
Treating People Respectfully (9) [352]	5.6 (0.78)	3.1–7	0.87

Note:

Spearman–Brown Coefficient 0.71.

Principal component analysis (PCA) (used as the default extraction method in the SPSS Factor Analysis routines) with varimax rotation was performed to assess the multidimensional quality of the questionnaire. Both the Bartlett’s test of sphericity and Kaiser–Meyer–Olkin measure of sampling adequacy (=0.936) demonstrated that the data were appropriate for factor analysis. Four factors with eigenvalues ≥1 were extracted, and these accounted for 58% of variance. Four items, three in the original Showing Personal Sensitivity and one in the original Treating People Respectfully domains, had higher loadings on other domains. Specifically, questions 11 and 21 had higher loadings (0.576 and 0.718, respectively) on the Treating People Respectfully domain, while question 12 loaded more heavily on the Providing General Information domain. Question 13 from the original Treating People Respectfully domain had higher loading on the Communicating Specific Information domain (0.542). The factor analysis was rerun excluding the four problematic items (N=23 items). Four factors with eigenvalues ≥1 were extracted, and these accounted for 60% of variance. The final loadings are presented in Table 2.

Table 2

Loadings of factor analysis by measure of processes of care for service providers working with adult clients domains

Item#	Domains with corresponding items: In the past year, to what extent did you …	Factor loading: 27 items	Factor loading: four items removed
	A: Showing Interpersonal Sensitivity
1	Suggest treatment/management activities that fit with each patient’s and family’s needs and lifestyle?	0.641	0.644
2	Offer patients and families positive feedback or encouragement (eg, in carrying out a home program)?	0.664	0.666
3	Take the time to establish rapport with patients and families?	0.699	0.699
4	Discuss expectations for each patient with other service providers, to ensure consistency of thought and action?	0.639	0.659
5	Tell patients and families about options for services or treatments for their condition (eg, equipment, therapy)?	0.652	0.668
8	Discuss/explore each patient’s and family’s feelings about having a condition (eg, their worries about their health or function)?	0.504	0.510
9	Anticipate patients’ and families’ concerns by offering information even before they ask?	0.501	0.492
11	Let patients and families choose when to receive information and the type of information they wanted?	0.228*	Removed
12	Help each family to secure a stable relationship with at least one service provider who works with the patient over a long period of time?	0.371*	Removed
21	Help patients and families to feel competent in managing their own care?	0.355*	Removed
	B: Providing General Information
23	Promote family-to-family “connections” for social, informational, or shared experiences?	0.772	0.777
24	Provide support to help families cope with the impact of the chronic condition (eg, informing patients and families of assistance programs, or counseling how to work with other service providers)?	0.751	0.759
25	Provide advice on how to get information or to contact other patients (eg, through a community’s resource library, support groups, or the Internet)?	0.857	0.846
26	Provide opportunities for the entire family to obtain information?	0.701	0.715
27	Have general information available about different concerns (eg, financial costs or assistance, respite care)?	0.776	0.778
	C: Communicating Specific Information
14	Tell patients about the results from tests and/or assessments?	0.810	0.786
15	Provide patients with written information about their condition, progress, or treatment?	0.707	0.720
16	Tell patients and families details about their services, such as the types, reasons for, and durations of treatment/management?	0.670	0.703
	D: Treating People Respectfully
6	Accept patients and their family in a nonjudgmental way?	0.600	0.635
7	Trust patients as the “experts” on themselves?	0.591	0.591
10	Make sure patients and families had a chance to say what was important to them?	0.577	0.556
13	Answer patients’ and families’ questions completely?	0.384*	Removed
17	Treat each patient and their family as an individual rather than as a “typical” patient?	0.711	0.723
18	Treat patients as equals rather than just as a patient?	0.720	0.737
19	Make sure patients and families had opportunities to explain their treatment goals and needs (eg, for services or equipment)?	0.548	0.488
20	Help patients and families feel like a partner in their own care?	0.622	0.585
22	Treat patients and their families as people rather than as “cases” (eg, by not referring by diagnosis)?	0.684	0.700

Note:

Items with larger loadings on other domains.

To test the hypothesis that domain responses will differ slightly between different programs, a one-way analysis of variance (ANOVA) was performed. Leven’s test supported homogeneity of variance across the samples. The results of the one-way ANOVA confirmed differences on three of four domains, with domain 1 (Showing Interpersonal Sensitivity) just above the significance level (P=0.053). The Tukey’s post hoc test showed that there were small but significant differences across results from Ontario, Alberta, and USA (Table 3).

Table 3

Results of one-way analysis of variance and Tukey’s post hoc test demonstrating site differences (measure of processes of care for service providers working with adult clients)

Domain	Location1	Location2	Mean difference (Location 1,2)	Standard error	Significance	95% confidence interval
						Lower bound	Upper bound
Showing Interpersonal	Ontario	USA	0.18	0.14	0.392	−0.15	0.51
Sensitivity	Alberta	Ontario	0.15	0.19	0.71	−0.30	0.59
		USA	0.33	0.15	0.069	−0.02	0.67
Providing General	Ontario	USA	0.60*	0.20	0.009	0.12	1.07
Information	Alberta	Ontario	−0.25	0.27	0.625	−0.89	0.39
		USA	0.35	0.22	0.252	−0.17	0.86
Communicating	Ontario	USA	−0.11	0.16	0.753	−0.49	0.26
Specific Information	Alberta	Ontario	0.93*	0.24	0.001	0.37	1.5
		USA	0.82*	0.19	0.000	0.38	1.3
Treating People	Ontario	USA	0.11	0.11	0.626	−0.16	0.37
Respectfully	Alberta	Ontario	0.26	0.16	0.216	−0.11	0.63
		USA	0.37*	0.12	0.009	0.08	0.66

Note:

The mean difference is significant at the 0.05 level.

MPOC-A

A total of 61 valid MPOC-A questionnaires were completed by patients, and 16 by family members. More than half of patient responders were women (59%), following a stroke episode (74%), and they had experiences of multiple HCPs. The majority of family responders were spouses (mostly wives) with a similar distribution of ages. The demographic characteristics of the sample are presented in Table 4.

Table 4

Demographic characteristics of the measures of processes of care for adults responders

Age, mean (min–max) in years	63.7 (19–85)
Sex, n (%)
Male	25 (41)
Female	36 (59)
Main condition, n (%)
Stroke	45 (74)
Other	15 (24.6)
Length of stay (days) (min–max)	39.9 (14–140)
Health care professionals encountered, mean (min–max)	5 (3–7)
Family member, n (%)
Spouse	11 (70), 2 husbands
Child	2 (12.5)
Parent	1 (6)
Family member age, mean (min–max) in years	60.3 (45–69)

Abbreviations: max, maximum; min, minimum.

The responses were analyzed to detect missing data patterns. Only one question (question 33 “… provide advice on how to contact other people with the same condition?”) had more than 20% missing responses (not applicable and missing combined). Analysis of missing data was performed by each domain, and in cases where missing items did not exceed two-thirds of the questions, estimated scores for missing values (but not for not applicable) were calculated by averaging the scores of the domain for that respondent.

The skewedness and kurtosis values were all between −2 and 2, allowing use of parametric statistics for further analysis. Domain means varied from 4.03 to 5.78 for patients and from 4.2 to 5.67 for family members, demonstrating a similar trend for both subsamples. No statistically significant differences were detected in responses by age (one-way ANOVA) or sex (independent t-test). Internal consistency varied between 0.82 and 0.90 for patients, and 0.87 and 0.95 for family. The summary of domain means, range of scores, and internal consistency are presented in Table 5.

Table 5

Descriptive statistics and internal consistency as assessed by Cronbach’s coefficient α of measures of processes of care for adults domains

Domain name (number of items)	Patient responses (n =61)			Family responses (n =16)
	Mean (SD)	Min–max	Cronbach’s α	Mean (SD)	Min–max	Cronbach’s α
Enabling and Partnership (9)	5.51 (1.06)	2–7	0.82	5.1 (1.1)	2.67–6.56	0.92
Providing General Information (5)	4.72 (1.74)	1–7	0.85	4.25 (1.78)	1.8–7	0.87
Providing Specific Information (5)	4.03 (1.75)	1–7	0.85	4.2 (1.77)	1.4–6.8	0.91
Coordinated and Comprehensive Care (9)	5.78 (0.99)	3.5–7	0.89	5.58 (0.85)	4.11–6.89	0.94
Respectful and Supportive Care (6)	5.6 (0.93)	3.8–7	0.90	5.67 (0.98)	3.83–7	0.93
Global Satisfaction	27.87 (3.9)	16–32	0.89	27.13 (4)	22–32	0.89

Abbreviations: Max, maximum; Min, minimum; SD, standard deviation.

Pearson correlation coefficients between the domains and global satisfaction measure varied from 0.32 to 0.67 for patients, and from 0.58 to 0.78 for family members (Table 6). Pearson correlation coefficients between the domains varied from 0.52 to 0.84. Sufficient data were only available for 13 pairs (patients and their family members) for interrater reliability analysis. The ICC coefficients varied from 0.41 to 0.81 with wide 95% confidence intervals (Table 6).

Table 6

Pearson product-moment correlation coefficients of measures of processes of care for adults domain scores with client satisfaction scores

Domain	Pearson coefficients (95% CI)		Interrater reliability (ICC) between patient and family memberN=13 (95% CI)
	Patient global satisfaction	Family global satisfaction
Enabling and Partnership	0.51 (0.29–0.68)	0.65 (0.15–0.88)	0.50 (−0.33 to 0.80)
Providing General Information	0.52 (0.29–0.69)	0.62 (0.16–0.86)	0.81 (0.50–0.94)
Providing Specific Information	0.32 (0.05–0.54)	0.58 (0.07–0.85)	0.74 (0.40–0.90)
Coordinated and Comprehensive Care	0.67 (0.49–0.79)	0.78 (0.44–0.93)	0.41 (−0.10 to 0.80)
Respectful and Supportive Care	0.65 (0.47–0.78)	0.66 (0.18–0.89)	0.56 (−0.30 to 0.87)

Abbreviations: CI, confidence interval; ICC, intraclass correlation coefficient.

Due to a limited sample size, we were unable to complete the planned factor analysis and test–retest analysis.

Discussion

The purpose of this study was to initiate formal validation of the adapted measure of client-centeredness of care for HCPs working with adult clients (MPOC-SP(A)), and to continue to explore the properties of the companion measure for patients and family members (MPOC-A). The tools were well accepted by all participants and a wide range of responses was provided. Similar to previous studies, the results of the Providing General Information and Providing Specific Information domains were lower in both the HCP and client samples, identifying the well-recognized problem of providing timely and appropriate information.3,8,23–27 Some clinicians, however, commented that the limitations of their scope of practice (ie, therapist or nurse versus physician) often prevented them from communicating specifics related to the diagnosis or prognosis of the client. In addition, privacy and confidentiality laws require explicit patient consent for the information to be shared with the family. Hence, the not applicable option was more often used in these two domains.

Good to excellent internal consistency (0.71–0.88 for HCPs, 0.82–0.90 for patients, and 0.87–0.94 for family members), as well as moderate to good correlations between domains (0.40–0.78 for HCPs, and 0.52–0.84 for clients) supported internal reliability of the tools. Further, exploratory factor analysis of the MPOC-SP(A) responses supported the multidimensionality of the questionnaire with four extracted factors, corresponding to MPOC-SP(A) domains, accounting for 58% of variance, and with loadings varying from 0.501 to 0.857. However, we would recommend considering a shorter version of the MPOC-SP(A) containing 23 questions, excluding the problematic items (questions 11, 12, 13, and 21). The properties of the shorter version will have to be examined in future studies.

As hypothesized, the MPOC-SP(A) was able to detect subtle differences among program participants in the study. Unfortunately, due to incomplete data, we were unable to test whether the responses also differed among different HCPs. These differences could be expected since some disciplines (eg, occupational therapist, social worker) have had client-centered care principles imbedded in their philosophy of care for some time and are introduced to these ideas during their studies.27 Future studies should try to explore this idea more fully. This in turn could inform educators of future clinicians about specific areas that should be addressed during their schooling.

Similar to previous studies reporting on development and validation of MPOC,7,19 we used the CSQ, a well-known global satisfaction tool, as a criterion measure to assess construct validity of the MPOC-A. Moderate correlations between the MPOC-A domains and CSQ scores, with the exception of Providing Specific Information, supported the idea that although they are measuring related constructs, MPOC-A provides more specific and detailed information about quality of services than global satisfaction alone.

Interestingly, no differences by age or sex of clients were observed in the present studies. Previous studies reported that women and younger clients tended to report lower satisfaction scores with the services.28–30 The relatively small sample in the present study might have prevented the detection of statistically significant difference.

The main goal of our study was to assess performance of these outcome measures, not to explore quality of care of each individual unit. Future studies focusing on quality assurance can explore correlations between clients’ and HCPs’ reports collected with the help of MPOC-A and MPOC-SP(A). Such a cross-measurement analysis was beyond the scope of the present study.

It is important to discuss the challenges, and consequently the limitations, that we have experienced during this study in the hopes that this can assist others planning future studies. Following major illness, patients and families represent a vulnerable population that is often not particularly interested in participating in research studies. In addition, following stroke, there are often issues of cognitive involvement that might prevent patients from being able to evaluate the quality of care. Although we had foreseen some of these difficulties, it was necessary to adjust our recruitment and data collection strategies along the way. Initially, eligible clients were identified by a charge nurse and asked for consent to participate in the study. After their discharge, a package with questionnaires was mailed to their homes with a self-addressed, prepaid envelope included for return. However, there were very few packages returned, and the decision was made to hire a research assistant (RA) to help with data collection. Toward the end of their stay on the unit, the clients were still approached first by the nurse for consent, followed by a visit from the RA, who helped them to complete the questionnaires. However, this mostly limited the number of family members who agreed to complete the surveys to the ones who were present at the time of the RA visit. Although the packages for the families were left with the patients, they were seldom returned. There was a similar outcome with the repeated packages that were sent to clients’ homes 2–4 weeks following the completion of the first survey, with only a small number of responses returned. This prevented us from being able to complete the test–retest analysis that was originally planned.

In addition, the charge nurses who were involved in recruitment commented that many patients were prevented from participating due to poor scores on cognitive assessment. In such cases, it is possible that the opinion of the closely involved family member can be solicited as a proxy report for that patient. Although limited by the small sample, the results of this study demonstrated moderate to good agreement between patients and their family members (ICC of 0.41–0.81). These results were also observed in our previous work.7 Similar results were demonstrated by Lovat et al.23 In their study, they directly adapted the MPOC-56 (the original pediatric version of MPOC containing 56 questions) to evaluate the perceptions held by caregivers of stroke survivors of the services provided by allied HCPs. They reported strong face validity and internal consistency (Cronbach’s α varying from 0.915 to 0.986).

Conclusion

In conclusion, in order to plan and deliver health services, it is essential to understand the processes of care that are important from both clients’ and HCPs’ perspectives, especially in the context of chronic conditions. Exploring long-term relationships among patients, families, and HCPs with the help of MPOC-A and MPOC-SP(A) will not only assist with further development of the measures, but will also improve our understanding of the specific needs of clients who live with chronic illness and disability, as well as those of their families. Both the MPOC-SP(A) and MPOC-A demonstrated promising psychometric properties in the adult rehabilitation settings. There were several limitations to the study, most prominent being limited sample size in the clients’ section that prevented completion of some of the planned analyses. Future larger studies should focus on exploring the validity and reliability of the measures in different health care settings.