Palliative Care Integration Project (PCIP) quality improvement strategy evaluation.

This study evaluated the effectiveness of implementation of common assessment tools, collaborative care plans, and symptom management guidelines for cancer patients as a strategy to improve the quality, coordination, and integration of palliative care service across organizations and health care sectors. A pre-post design to measure the impact on symptom management, caregiver burden and satisfaction with care delivery, and service utilization was used. Two cohorts of eligible patients and caregivers completed Edmonton Symptom Assessment Scales, Caregiver Reaction Assessment and FAMCARE Scales and chart audits were conducted. Administrative data from each participating site were examined for utilization trends. Audits of 53 charts preimplementation and 63 postimplementation showed an increase in documentation of pain from 24.5% to 74.6% (P<0.001) of charts. Administrative data showed a decrease in the percentage of patients with at least one emergency room visit from 94.3% to 84.8% (P<0.001), in the percentage of patients with at least one admission to the acute care hospital (P<0.001), and deaths in acute care 43.1%-35.7% (P=0.133). There was minimal change in the intensity of symptoms (P=0.591), and no change in the burden on the caregiver (P=0.086) or caregiver satisfaction with care (P=0.942). This study showed that implementation of common assessment tools, collaborative care plans, and symptom management guidelines across health sectors can result in some increased documentation of symptoms and efficiencies in care. Future projects should consider imbedding a continuous quality improvement methodology and longer timelines into their projects to improve outcomes.

Introduction

Cancer patients in the palliative phase of their illness receive care from almost every sector of the health care system. The provision of quality palliative care requires the commitment and cooperation of a multiplicity of health care service providers, community organizations, professionals, and volunteers. Unfortunately, numerous studies across Canada have demonstrated that there is limited integration of community-based physicians, nurses, continuing and long-term care facilities, acute care (AC) hospitals, cancer centers, and palliative care specialists.1, 2 This lack of integration can result in late referrals, unnecessary delays for treatment, or transfer to an appropriate setting of care and unnecessary suffering and cost.1, 2, 3

Similar deficits and gaps to those identified in other areas of Canada were identified in studies conducted in the southeastern Ontario region from 1998 to 2000.1, 2, 4, 5, 6 The surveys and focus groups revealed that there was minimal use of assessment tools, lack of evidence-based practice, discomfort of physicians in managing palliative-related symptoms, inconsistent symptom management practices, discontinuity of care at transition points, lack of coordination and communication between health care providers/services, underutilization of available resources, and unmet patient and family needs.4, 5, 6, 7 In response to these issues, a quality improvement project entitled the Kingston, Frontenac, Lennox & Addington Palliative Care Integration Project (PCIP) was initiated in 2001.

The overall objective of the PCIP was to ensure efficient, high quality palliative care delivery to cancer patients through the collaboration, and development of service integration across the continuum of care. The integration was led by a project structure that included a Steering Committee of senior administrators, clinicians, and decision makers and supported by subcommittees and teams addressing Development, Education, and Implementation. Primary activities for the project teams included: identification of common assessment tools; development of five evidence and consensus-based collaborative care plans (CCPs) designed to address functional status and symptom intensity; creation of six evidence and consensus-based symptom management guidelines; and education, implementation and use of common assessment tools, CCPs, and symptom management guidelines in every sector of the health care system.

As part of the implementation strategy between 2002 and 2004, approximately 600 nurses and allied health professionals, and over 200 family physicians, including residents, were educated about this project. The sessions included outlining the project's objectives, processes and the administration of the assessment tools, CCPs, and symptom management guidelines. Education occurred in a variety of formats, including workshops with “train-the-trainer” sessions, community rounds, formal presentations, self-directed web-based learning, continuing professional development sessions, newsletters, and other print media.

The evaluation study addressed the following questions: Would the implementation and use of standardized assessment tools, CCPs, and symptom management guidelines improve and facilitate continuity of care for palliative cancer patients in and across all points of care? Would the use of common assessment tools, evidence-based CCPs, and symptom management guidelines by regulated and unregulated health care providers decrease variability of palliative care practice, improve efficiencies in patient care, improve access to palliative care services, and improve responsiveness of the system to meet the needs of palliative patients and their families?

To answer these questions, we conducted process and outcome evaluations. The key findings of the outcome evaluation study are presented here.

Methods

The PCIP outcome evaluation study applied a pre-post design to measure the impact that specific clinical practice changes had on symptom management in cancer patients, caregiver burden and satisfaction with care delivery. Two cohorts of patients, caregivers, and health professionals (one before implementation and one after implementation) were examined. Administrative data from each participating site were also examined for utilization trends.

The five participating care sites included:

Palliative Medicine Program—ambulatory and acute care consultative service.

Outpatient Cancer Centre.

Acute Care Hospitals and their Emergency Departments.

Community and residential care organizations.

Complex Continuing Care Unit.

Approval to conduct this study was received from each of the primary participating sites and the Queen's University Research Ethics Board. Consents were obtained from all eligible patients and primary caregivers. All data were kept strictly confidential and secured.

The following data collection and analysis methods were used.

Chart Audits

Chart audits were conducted on all medical charts of eligible palliative oncology patients who were admitted to one of the five participating sites in November 2001, 2002, or 2003 to determine if there was a decrease in variability of palliative care practice. The chart reviews examined the frequency of documentation of symptoms and their intensity, Do Not Resuscitate status, patient's desired place of death, and the palliative performance status as determined by the standardized functional assessment tool, the Palliative Performance Scale (PPS), for the first 48-hour period following the patient's admission to a hospital setting and for the first two visits following admission to a community-based service.

Administrative Databases

Administrative databases from each site were analyzed to determine if the PCIP altered service utilization patterns of cancer patients. “Palliative” cancer patients were identified using lists generated from administrative data of Kingston, Frontenac, Lennox & Addington Community Care Access Centre, the Queen's Palliative Care Medicine Program, and from the St. Mary's of the Lake Complex Continuing Care Palliative Care Unit. Additional data on AC emergency department (ED) visits and AC stays were obtained from the Kingston General Hospital Information Analysis and Distribution department. The databases provided data related to patients’ predisposing characteristics, comorbidities, and utilization and referral patterns. Outcomes identified included number of ED visits, AC hospital admissions, AC hospital days, home and long-term care facility and palliative care unit deaths, referrals to specialized palliative care teams (in community, clinics, and hospitals), and referrals for palliative care inpatient unit admission.

Patient/Caregiver Questionnaires

Consenting patients/caregivers completed questionnaires to determine if the PCIP increased the responsiveness of the system to meet the needs of palliative patients and their families. The questionnaires measured the intensity of symptoms most commonly experienced by palliative patients, family satisfaction with palliative care service delivery and caregiver burden.

All adult (>18 years of age) cancer patients living in the Kingston, Frontenac, Lennox and Addington region who were identified as “palliative” and receiving care at the Cancer Centre, AC or complex continuing care or community home care, and/or from the Palliative Care Medicine Program were eligible. Patients were ineligible if they were determined to be cognitively impaired by the primary health professional, unable to speak and read English, unable to complete questionnaires, or if death was imminent. Patients from long-term care facilities were excluded from the study because too few had a cancer diagnosis. Adult caregivers were eligible if they considered themselves to be the primary caregiver and the patient concurred with this. Caregivers were excluded if they did not read and speak English.

All eligible patients who were receiving services from one of the participating sites on any day of November 2002 and November 2003 were invited to participate. Patients and caregivers who expressed an interest in participating were informed about the details of the study and then contacted by the study coordinator for an interview.

The Edmonton Symptom Assessment Scale (ESAS) is a well-validated instrument which uses a Likert scale (0–10) to assess the intensity of the following symptoms: pain, tiredness, nausea, depression, anxiety, drowsiness, appetite, well-being, and shortness of breath. Zero represents no symptoms while 10 represents the worst possible symptom intensity. The ESAS was administered to patients in November 2002 and November 2003.

The FAMCARE Scale is a 20-item Likert-type scale that measures the degree of patient and caregiver satisfaction with palliative care. According to the Q-sort cluster analysis, questions are grouped into four factors: information giving (five questions), availability of care (four questions), physical patient care (seven questions), and psychosocial care (four questions). An increase in scale value represents an increase in satisfaction. The FAMCARE scale was administered to caregivers in both November 2002 and November 2003.

The Caregiver Reaction Assessment (CRA), a measure of caregiver burden administered to caregivers, comprises 24 questions (divided into five categories: Impact on Finance, Impact on Health, Caregiver Esteem, Lack of Family Support and Impact on Schedule) scored on a five-point Likert scale from strongly disagree (1) to strongly agree (5). Low scale values reflect a decrease in caregiver burden. The CRA was administered to caregivers in both November 2002 and November 2003.

A detailed timeline for the key evaluation measurements and data sources is outlined in Fig. 1. Preimplementation patient/caregiver questionnaire data were collected in the fall of 2002. Chart audits were completed for the November 2001 and 2002 periods. Postimplementation questionnaires and chart audit data were collected for the month of November 2003.

Fig. 1

Evaluation study timeline.

Data Analyses for Outcomes Evaluation

The analyses for the entire patient–caregiver-related data began by examining the sources for data quality and accuracy. Distributions of the continuous variables from patient and caregiver interviews were also examined for normality using visual examination of data (e.g., scatter plots). Outliers, extreme, and influential values were noted.

Univariate analysis of all the data provided description demographic statistics, for example, frequency distribution, percentages, mean, median. The results from the ESAS, FAMCARE, and CRA were analyzed for mean, standard deviation, and median statistics. Bivariate analyses using cross-tabulations were also conducted. Levels of significance between normally distributed data were determined using parametric independent samples t-test. Unmatched data that were not normally distributed were tested for significant differences using the Mann–Whitney nonparametric test. Both the Fisher's Exact Test and an Exact Cochran-Armitage Trend Test were also conducted on categorical variables. Although these analyses test against the null hypothesis (assume that the proportions are the same across all three years of sampling), the Fisher's Exact Test is based on the assumption that at least one year differs from the other two years. The Exact Cochran-Armitage Trend Test assumes there is a continuous increasing or decreasing trend across all three years. We report only P-values from the Fisher's Exact Test due to similar results from both tests and its familiarity.

Results

For all audit time periods, the community setting represented the location for the majority of the charts audited (56.6%, 62.5%, and 80.9% for 2001, 2002, and 2003, respectively). In 2001 and 2002, female patients (59.6%, 58.7%, respectively) represented the majority of this population. In 2003, the gender of this population was evenly distributed between males and females. The average age (mean ± SD years) for this population in 2001, 2002, and 2003 was 66.5 ± 13.5 (median 71), 68.1 ± 11.1 (median 68), and 69.2 ± 10.5 (median 72) years, respectively. For 2001 (23.9%) and 2002 (27.0%), lung cancer was the primary cancer diagnosis, with breast cancer representing the primary cancer diagnosis for 2003 (24.2%).

In 2001, 62.3% (33/53) of patient charts had no documentation of symptoms. In 2002, 56.3% (28/64), and in 2003, 53.9% (29/63) of charts had no documentation of symptoms (P  = 0.103). Pain symptoms were documented in 87.5% (56/64) and 74.6% (47/63) of patients’ charts in 2002 and 2003, respectively, in contrast to only 24.5% (13/53) of patients’ charts in 2001 (P  < 0.001).

Do Not Resuscitate status was recorded in 24.5% (13/53) of patient charts in 2001, 23.4% (15/64) in 2002, and 17.5% (11/63) of patient charts in 2003 (P  = 0.618).

Over the three time points there was a slight increase in the recording of patient's desired place of death: 2001, 2/53 (3.8%); 2002, 5/64 (7.8%); and 2003, 5/63 (7.9%) (P  = 0.701).

A modest number of documented recordings of the PPS were seen over time. Before 2003, there was no documentation of PPS. In contrast, 9.5% (six of 63 patient charts) of the 2003 chart audits had PPS documentation (P  = 0.002).

The entire PCIP sample included 1,347 cancer patients followed by Community Care, Palliative Care Medicine, or the Complex Continuing Care Palliative Care Unit during the calendar years of 2001, 2002, or 2003. Eleven patients were excluded due to missing health card number (n  = 1) or health care coverage other than Ontario Ministry of Health—either an out-of-province provider (n  = 6) or Correctional Services of Canada (n  = 4). This resulted in a total sample size of n  = 1,336 (99.2%) of the original number of patients. Some of the patients (n  = 4) were alive for more than one year.

Table 1 summarizes the ED and AC Hospital admissions during the three-year study period. Although there was a slight increase in the total visits to the EDs for 2003 and 2002 compared to 2001, a smaller percentage of patients had at least one visit (P  < 0.001). The percentage of AC hospital admissions decreased over the three years (P  < 0.001) There were no differences in the percentage of persons discharged home vs. admitted or transferred to another institution.

Table 1

ED and AC Hospital Visits

	Calendar Year
	2001	2002	2003
Total number of patients	513	570	579
Number of patients with at least one ED visit (%)	484 (94.3)	501 (87.9)	491 (84.8)
Number of patients with at least one AC visit (%)	476 (92.8)	497 (87.2)	498 (86.0)

Table 2 summarizes what could be determined regarding location of death. Estimates of home deaths were not included because it was thought that the data was too unreliable at this time.

Table 2

Location of Death

	Calendar Year
	2001	2002	2003
Total number of patients	513	570	579
Total number of deaths (%)	288 (56.1)	347 (60.9)	328 (56.6)
Location of death (% of total deaths)
AC hospitals	124 (43.1)	144 (41.5)	117 (35.7)
Complex continuing care (palliative care unit)	63 (21.9)	74 (21.3)	73 (22.3)
EDs	6 (2.1)	4 (1.2)	7 (2.1)
Home or other facility/service	95 (33.0)	125 (36.0)	131 (39.9)

Table 3 outlines incidence rates based on person-years for encounters to the following health services: ED, AC, Complex Continuing Care Palliative Care Unit, and Community Care. For a given time period, the total number of person-years is the sum of the time followed over the sample of patients. By using rates based on this value, we weight patients by how long they were followed and in doing so adjust for the known variation in utilization due to death. Because all of the encounters considered have duration, person-year rates of cumulative lengths of stay over the patient and year are also provided. For example, the incidence rate of ED encounters in 2001—2.56 encounters (i.e., visits) per person-year—indicates that if a “typical” patient from the 2001 sample were to be followed for the entire year, he or she would have presented to the ED roughly 2.56 times. The cumulative time spent in the ED for such a patient would be approximately 0.53 days.

Table 3

Total Number of Patient Encounters

	Calendar Year
	2001	2002	2003
Total number of patients followed	513	570	579
Total number of deaths (%)	288 (56.1)	347 (60.9)	328 (56.6)
Person-years	383.1	398.8	409.0
Cumulative LOS/person-year
ED	0.53	0.72	0.60
AC hospital	21.6	22.0	18.3
Continuing complex care (palliative care unit)	4.91	4.74	4.26
Community care	118.9	126.5	131.8

LOS = Length of stay; Cumulative LOS = Total LOS in days over all patients and encounters within the year under consideration.

The Palliative Care Medicine administrative data showed that the total number of consults for 2001, 2002, and 2003 was 729, 945, and 872, respectively. Complex Continuing Care Palliative Care Unit consults for 2002 and 2003 were 140 and 106, respectively. Noteworthy is the decrease in consults experienced throughout the 2003 months defined by Severe Acute Respiratory Syndrome (SARS). This decrease in consults was similar with other local clinical services.

Patient and Caregiver Demographics. Patient demographics for the two cohorts did not significantly differ between the two time periods ( Table 4). For both periods of time, the level of education for this patient population was evenly distributed between elementary school, high school, and college and higher.

Table 4

Demographics

Patient	2002		2003
Gender	n	(%)	n	(%)
Female	31	(58.5)	24	(49.0)
Male	22	(41.5)	25	(51.0)
Total	53	(100)	49	(100)
Age	n	Median	n	Median
Total	53	65	49	68
Marital status	n	(%)	n	(%)
Married	41	(77.4)	32	(65.3)
Divorced/separated	4	(7.5)	6	(12.2)
Single	1	(1.9)	1	(2.0)
Widowed	7	(13.2)	9	(18.4)
Response not known	0	(0.0)	1	(2.0)
Total	53	(100)	49	(100)
Caregiver	2002		2003
Gender	n	(%)	n	(%)
Female	28	(70.0)	23	(65.7)
Male	12	(30.0)	12	(34.3)
Total	40	(100)	35	(100)
Age	n	Median	N	Median
Total	40	62	35	58
Marital status	n	(%)	N	(%)
Married	36	(90.0)	28	(80.0)
Divorced/separated	2	(5.0)	1	(2.9)
Single	0	(0.0)	4	(11.4)
Widowed	2	(5.0)	0	(0.0)
Response not known	0	(0.0)	2	(5.7)
Total	40	(100)	35	(100)

In 2002 and 2003, 53 of 120 eligible patients (44.2%) and 49 of 186 eligible patients (26.3%), respectively, consented to participate. The majority of the questionnaires were completed by patients in the community (50.9% in 2002, n  = 27; 61.2% in 2003, n  = 30) and Cancer Centre setting (47.2% in 2002, n  = 25; 32.7% in 2003, n  = 16).

Like the patient population, the caregiver demographics did not significantly differ between the two time periods (Table 4). Most caregivers were the spouse/partner of the patient (2002: 76.9%; 2003: 80.0%) For both periods of time, high school was the highest level of education achieved for caregivers (2002: 44.7%; 2003: 48.4%), with a smaller percentage educated at the college or higher level (2002: 23.7%; 2003: 25.8%).

ESAS. The majority of patients (n  = 34; 65.4% in 2002; n  = 40; 81.6% in 2003) completed the ESAS on their own. All mean symptom and total scores, except for depression, were less in the 2003 patient sample than in 2002 ( Fig. 2) but these differences were not statistically significant (P  = 0.121–0.914).

Fig. 2

Patient ESAS responses.

FAMCARE—Caregiver Responses. There were no significant differences (P  = 0.942) in overall satisfaction with care (i.e., total FAMCARE score) between 50 and 49 patients who completed this questionnaire in 2002 and 2003, respectively ( Fig. 3). There were also no significant differences (P  = 0.515) in overall satisfaction as reported by 40 caregivers in 2002 and 36 caregivers in 2003 (data not shown).

Fig. 3

Caregiver FAMCARE responses.

CRA. In Fig. 4, the data show that there were no significant differences (P  = 0.086) in perceived burden of care across all five factors for the 39 and 35 caregivers of palliative patients who completed this questionnaire in 2002 and 2003, respectively.

Fig. 4

Caregiver Reaction Assessment (CRA).

Discussion

This is the first study in Canada to prospectively evaluate the impact of a regional quality improvement strategy, the PCIP, to improve the care of cancer patients in the palliative phase of their illness. The PCIP was designed to decrease the variability of palliative care practice, improve access to palliative care services, improve efficiencies in patient care, and improve responsiveness of the system to meet needs of palliative patients and their families.

Analysis of the administrative data showed that across the three-year study period (2001, 2002, and 2003) there was evidence of improvement in efficiencies in patient care. There was a consistent decrease in the percentage of patients with at least one visit to the ED (P  < 0.001), at least one admission (P  < 0.001), AC hospital deaths (P  = 0.12), and the cumulative length of stay per person-year in AC and in the palliative care unit. The home or non-AC facility deaths also consistently increased (33%, 36%, and 39.9%, for 2001, 2002, and 2003, respectively) across the three years of the project. Community Care had a consistent increase in the number of days that people received services (118.9, 126.5, and 131.8 days, for 2001, 2002, and 2003, respectively).

There was some evidence of a decrease in the variability of palliative care practice with a slight increase in the timely documentation of symptoms, particularly pain (P  < 0.001) and PPS (P  = 0.002). It was anticipated that a common documentation form would be in place prior to implementation of PCIP and that this would form the basis of the chart audit. Unfortunately, this did not happen because SARS interrupted the form's development and implementation. It is possible that the improvement we saw was underestimated because the chart audit was limited to a specific window of time following admission to a service. The auditors noted, in a number of cases, that documentation occurred but in a different time frame or portion of the record.

The study showed minimal but not statistically significant improvements in patient symptom intensity scores and no changes in family satisfaction with care and caregiver burden. It is possible that such slight improvements, although clinically relevant to practice, were a consequence of implementation of the assessment tools and CCPs processes taking much longer than anticipated. This occurred, in part, because of unforeseen events, such as SARS, loss of many nurses trained in the project, and competing workload demands. These challenges meant that the projected time for uptake and application of the various project, components was significantly shortened and subsequently had an impact on the evaluation results.

One of the limitations of this study is its pre-/postimplementation design. It is impossible to control for the many factors that change over a one-year period in the health care system. In 2002, one of the nursing agencies that provided home palliative care was closed. In 2003, SARS greatly affected the health care system and no doubt had some influence on ED visits and admissions to the AC hospital. It is interesting, however, to note that hospital length of stay for all cancer patients in this whole region increased during the same time period.

One of the important lessons from this evaluation study was that it takes longer to implement projects than anticipated because unexpected situations, such as SARS or the demise of a key nursing organization, arise. It is, therefore, important to build longer timelines into the evaluation phase when trying to demonstrate system-level changes. Our experience would also suggest that standardized documentation should be in place prior to initiating such a project because it will facilitate both uptake and evaluation of key outcomes. Future projects should consider embedding a continuous quality improvement methodology in their processes so that there is simultaneous implementation, evaluation, and refinement of processes throughout the project duration.