The use of a validated pre-discharge questionnaire to improve the quality of patient experience of orthopaedic care.

Patient experience is central to the delivery of excellent healthcare. As such it is enshrined within the 2015 NHS outcomes framework, a set of indicators against which quality in healthcare is measured. A variety of tools are available to quantify patient experience across clinical settings. When combined with a framework for continued data collection and suitable mechanisms for analysis, feedback, and intervention, these tools allow improvements in patient care and clinical services to be realised. In response to an increasing number of patient complaints and friends and family scores below the trust average within our orthopaedic department we instituted an improvement programme in March 2015. The programme was based around the Picker Patient Experience 15 questionnaire and aimed to improve friends and family test scores, reduce complaints and improve patient experience scores over an 18-month period. An improvement model including baseline measurement and 2 improvement cycles over an 18-month period was used. Initial benchmarks for practice were created by referencing national data allowing problem areas of care to be identified and interventions to address these developed. This process identified areas for improvement including improving staff awareness and engagement with patient experience, improving staff and patient communication and ensuring patients were aware and involved in plans for their own care. Actions to address these issues resulted in a 38% decrease in patient complaints, a >10% increase in patient experience, and improvements in patient satisfaction and friends and family scores.

Problem

Patient experience is one of the key domains within the NHS outcomes framework1 alongside patient safety and clinical effectiveness. During their inpatient stay patients and their relatives undergo a range of feelings and emotions, many of which may go unrecognised by healthcare teams. Failure to address concerns, communicate effectively, and to provide advice, reassurance and information at the appropriate time exacerbate feelings of anxiety and distress. This may result in reduced satisfaction and poor experience with the healthcare service that may in turn influence treatment outcomes and lead to complaints against service providers.

Information on patient experience is collected in many healthcare setting using the friends and family test. Data on the number and type of complaints is collected through formalised complaints processes within individual institutions. In August 2014 the orthopaedic department at South Tees NHS Trust audited these data to determine levels of performance relative to other departments within the hospital. This audit demonstrated that the orthopaedic wards friends and family scores were below the trust average (76% versus a trust average of 83%), that two of our three wards were in the bottom five of all of the wards audited (August 2014 data), and that orthopaedic complaints against the department had increased in the first 6 months of 2014 (Total 40 complaints in Q1&Q2 2014).

In response to this, the Head of Nursing organised a professional forum for senior staff to discuss this issue and themes emerging from the complaints. This forum aimed to highlight the problem, engage staff in developing a solution and identify individual and team actions throughout the patients care episode that would help reduce complaints and improve the patient's experience of their care.

Following the meeting the following SMART aim was developed:

The orthopaedic patient experience team (consisting of the Head of Nursing, Discharge Sister, Orthopaedic Consultant and Patient experience lead) aimed to deliver an improvement in patient experience through the use of a validated patient experience questionnaire. The project aimed to improve friends and family test scores from 76% to >90%, reduce 6 monthly complaints from 40 to <30 and improve patient experience scores by 10% from baseline measurement over an 18-month period. To achieve this aim an expanded patient experience measurement (over and above the friends and family test) using a validated questionnaire would be required to provide greater detail and granularity about the current issues experienced by patients. It was felt this would facilitate the identification of issues, development of action plans and the targeted interventions on individual orthopaedic wards allowing the targeted improvements to be achieved. To facilitate delivery the project would be delivered by the orthopaedic discharge sister at the point of discharge. As the discharge sister is independent from the direct care team and comes in to contact with patients at the end of their inpatient stay this arrangement would ensure that regular, impartial sampling of the target population was achieved.

South Tees Hospitals NHS Foundation Trust offers a full range of general healthcare services and specialist services for patients in the North East of England. It is a Major Trauma Centre and a tertiary referral centre for orthopaedic sub-specialties including arthroplasty, limb reconstruction, paediatric orthopaedics, spinal injuries and trauma. There are 4 Orthopaedic Wards (Two elective, one trauma, one hip fracture) and a specialist spinal injuries unit on two hospital sites (James Cook University Hospital and Friarage Hospital) comprising approximately 140 inpatient beds.

Background

During the last decade services within the National Health Service (NHS) have evolved in an attempt to deliver high quality, patient centred care for all. Recent government directives2,3 are increasing emphasis on the delivery of a ‘good patient experience’ as a core element of this process (Figure 1). Patient experience is one of the five domains of the NHS outcomes framework1. Despite this focus, patient experience attracts fewer attempts at improvement as it is subjective and difficult to measure in comparison to the traditional objective clinical measures used to record safety and effectiveness.

Figure 1

The NHS outcomes framework

The NHS Patient Experience Framework includes an evidence-based definition of patient experience and considers how this concept should be measured.3 It is a broader entity than patient satisfaction and “reflects occurrences and events that happen independently and collectively across the continuum of care”.4 Measurement is therefore more complex and needs to include the assessment and evaluation of a variety of different aspects of the healthcare episode.

Within many organisations the strategies used to sample patient experience are based around simple questions such as the friends and family test, asking patients about their willingness to return to the hospital for further treatment and the quality of specific services (cleanliness, food quality, time keeping etc.). While these measures are useful for benchmarking within and between organisations they lack the detail required to allow meaningful targeted improvement of services. The Picker Institute Europe has developed a variety of tools and mechanisms for the evaluation and subsequent analysis of patient experience data. However, these tools are not routinely use in many UK healthcare organisations.

Baseline measurement

There were two key elements to consider when instigating this project: 1) the content and design of the patient experience questionnaire 2) the timing and mode of questionnaire delivery.

An initial scoping review of the literature revealed that, within orthopaedics, there is no tool specifically designed to measure patient experience and that this outcome is generally under measured. The Health Foundation document “Measuring Patient Experience”5 outlines the methods and tools available to assess the quality of clinical care, patient feedback and patient experience. After reviewing a number of tools we opted to conduct patient surveys using the validated Picker Patient Experience-15 Questionnaire (PPE-15)6 based on its simple design, availability of national data to benchmark against and its target population of hospital inpatients.

The (PPE-15) is a 15-question survey, assessing 8 key aspect of care: Information and Education; Co-ordination of Care; Physical Comfort; Emotional Support; Respect for Patient Preferences; Involvement of Family and Friends; Continuity and Transition; and Overall Impression (Table 1)6. Questions are asked with a range of possible responses that are subsequently turned in to a binary outcome reflecting a problem or no problem with the questioned domain. Based on these binary problem/no problem responses the percentage of patients with or without problems can be calculated for each question. The collection of the PPE-15 was supplemented by basic demographic data, an overall satisfaction rating (Visual Analogue Scale 1-10), friends and family score, return to hospital question and a free text element to record other experiences and issues whether positive or negative. These measures were chosen to allow us to meet our specific aims relating to the targeted improvements in friends and family test score and patients experience. We anticipated that the domains of care recorded within the PPE-15 would allow for a greater understanding of the specific issues experienced by patients thus allowing targeted improvements to be made.

Table 1

The 15 domain PPE-15 questionnaire

	PPE question
1	When you had important questions to ask a doctor, did you get answers that you could understand?
2	When you had important questions to ask a nurse, did you get answers that you could understand?
3	Sometimes in hospital, one doctor or nurse will say one thing and another will say something quite different. Did this happen to you?
4	If you had any anxieties or fears about your condition or treatment, did a doctor discuss them with you?
5	Did doctors talk in front of you as if you weren't there?
6	Did you want to be more involved in decisions made about your care and treatment?
7	Overall, did you feel you were treated with respect and dignity while you were in hospital?
8	If you had any anxieties or fears about your condition or treatment, did a nurse discuss them with you?
9	Did you find someone in the hospital staff to talk to about your concerns?
10	Do you think the hospital staff did everything they could to control your pain?
11	If your family or someone else close to you wanted to talk to a doctor, did they have enough opportunity to do so?
12	Did the doctors or nurses give your family or someone close to you all the information they needed to help you recover?
13	Did a member of staff explain the purpose of the medicines you were to take home in a way you could understand?
14	Did a member of staff tell you about medication side effects to watch for when you went home?
15	Did someone tell you about danger signals regarding your illness or treatment to watch for after you went home?

Using the agreed outcome measures the survey was then created within an electronic survey platform and made this available on tablets devices within the ward environment for completion by patients prior to discharge assisted by a member of the discharge team under the supervision of the discharge sister.

Once commenced data collection was performed on each ward 1-2 times per week. These data were analysed and fed back through the orthopaedic patient experience team to staff members on a monthly basis. The intention was to have an initial 2-month period of baseline measurement, followed by a 6-month period of continual analysis and feedback during which time interventions to improve patient experience could be instituted and evaluated. Improvements from baseline in the friends and family test score and the PPE-15 domains during this 6-month period were taken to represent an improvement in patient experience. Further improvement cycles would then be instituted as necessary.

Baseline measurement

The questionnaire was introduced to 3 orthopaedic inpatient wards in March 2015 and baseline measurement commenced over a period of 2 months (March-April 2016). During this period questionnaire responses were collected from 179 patients and no changes were made to standard care. This therefore allowed us to make an assessment of current patient experience and identify deficiencies in care that could be addressed during subsequent improvement cycles. Data from this baseline period was reported at a departmental level and was compared against national data taken from the PPE-15′s original paper6 to benchmark practice against national standards.

During the baseline period the scores in the majority of the 15 PPE-15 domains were above the defined national benchmark (Table 2). However, some areas of poor performance were identified; namely PPE-15 questions 3 (patients being given conflicting information), 4 (doctors failing to address patients fears) and 6 (patients desire to be more involved in their care). During the initial 2 month period the overall satisfaction rating was 8.4 out of 10, the friends and family test score was 99% and the mean number of problems across the PPE-15 domains was 16.3%.

Table 2

Data from 2 month baseline data collection period, data reported as the percentage of patients not reporting a problem for each PPE-15 domain. Key *= Target (Based on national data from source paper), Red=below benchmark, Green=100% patients reporting no problems

	Target %*	Mar-15	Apr-15
n		61	118
PPE-15 question
1	71.9	83.6	83.1
2	75.9	82.0	83.1
3	76.7	60.7	75.9
4	84.9	81.7	86.2
5	65.9	78.7	92.2
6	67.4	76.3	58.3
7	69.4	78.7	96.6
8	70.3	86.9	91.3
9	40.7	90.2	91.5
10	79.9	91.0	92.0
11	67.2	90.0	88.9
12	61.7	90.0	88.1
13	76.8	86.0	89.8
14	64.2	87.8	83.2
15	40.1	69.2	76.5
Rating (Mean)	≥8.5	8.5	8.4
Friends and family (%Yes)	≥95	98.3	100.0
Return to dept (%Yes)	≥95	98.3	99.2

The baseline measurement highlighted 3 specific areas of poor performance that required action in the subsequent improvement cycles:

Patients were being given conflicting information by nurses and doctors

Doctors were failing to address patient's fears

Patients had a desire to be more involved in their care

Design

Based on the baseline analysis the following actions were implemented:

– The results were drafted as a newsletter and circulated to all staff members to highlight areas of concern and good practice. This was done to increase awareness of the problems across the entire orthopaedic department.

– Ward managers and junior doctor's representatives were consulted to create action plans designed to improve communication between nurses and doctors on the wards (In response to concern 1). This involved a number of interventions which included:

Making the senior ‘nurse in charge’ available for the junior doctors at specific times of the day to ensure direct communication between the senior nursing and medical teams.

Doctors were instructed not to see patients on the ward without first communicating with the ‘nurse in charge’.

The junior doctor's induction was revised to highlight the importance of communication between doctors and nurses with an emphasis on good communication practices and the importance of handover.

– The workforce and working patterns on each ward were reconfigured with the deployment of ward based nurse practitioners to provide continuity of care and facilitate improved communication between the medical and nursing teams (In response to concerns 1 and 2)

– A daily ‘patient awareness’ audit was instigated to assess the level of patients knowledge of what was happening to their care during each day of their stay. Each morning patients were asked ‘Do you know what is happening with your care today?’. When patients stated they did not know what was happening the issue was escalated to the senior ward nurse or team doctor to review the patient and explain the current clinical care plan (In response to concerns 2 and 3)

Strategy

Improvement cycle 1

The first improvement cycle of the project started after implementation of the actions targeting areas of underperformance listed above. The action plans were instituted over a 2 month period. We hypothesised that these actions would lead to improvements in team and patient communication and patient involvement and awareness of their care which would, in turn, result in an increase in patient experience in the underperforming domains (Qus 3,4,6). We also expected that a general awareness of the project through the newsletter and the ‘patient awareness’ audit would lead to general improvements across all domains.

Data was collected over a 6 month period (July to December 2016). During this cycle data collection was expanded to include an additional ward in another hospital within the trust and analysis and reporting was again performed at a departmental level. Responses were collected from 411 patients.

Analysis of data from improvement cycle 1 demonstrated an improvement in scores across all PPE-15 domains including the targeted domains (Qus 3,4, 6) when compared to baseline data. The mean number of problems across the PPE-15 domains for each month is presented in figure 2. This showed that the introduction of the initial action plans led to an improvement compared to baseline which was sustained throughout improvement cycle 1, albeit without further significant improvement during this period. A similar pattern was seen in the targeted ‘problem’ domains (Qus 3,4,6) during cycle 1 (Figure 3). No domains fell below the national benchmark during these 6 months.

Figure 2

Mean number of problems across the PPE-15 domains for each month of the program (pooled data across all 15 PPE-15 domains)

Figure 3

Mean percentage of patients reporting no problems with their care for each of the targeted PPE-15 domains (Q3,4,6).

Data from improvement cycle 1 demonstrated:

Improvement in overall patient satisfaction from 8.4 to 9.2 out of 10 (a 10% improvement)

Friends and family test score remained at 99%

The mean number of problems across the PPE-15 domains decreased from 16.3% to 5.3% (a 67% reduction)

The mean number of patient experiencing problems in each of the targeted domains decreased from 31.7% to 12.6% for Qu3 (a 60.1% reduction), from 16.1% to 3.3% for Qu4 (a 79.6% reduction) and from 32.7% to 17.1% for Qu6 (a 47.7% reduction).

Based on the improvement cycle 1 analysis the following additional actions were implemented. The actions instigated before cycle 1 continued to be employed:

Requests were made from ward teams for individualised ward feedback. In addition to the routine monthly departmental feedback additional feedback to individual wards was delivered in a quarterly basis. Ward managers reviewed their own data and were required to develop ward action plans for each PPE-15 domains and highlight their 3 priority areas for the upcoming quarter. Therefore if specific problems were highlighted within individual clinical areas the ward managers had a mandate to address these directly.

It was felt that detail about the nature of the problems patients were experiencing would provide useful additional information to help address their problems. The patient experience questionnaire was therefore redesigned to collect an additional layer of data for each question for which the respondent entered a ‘problem’ response. This question asked for further detail on the nature of the problem the patient had experienced and apologised for any inconvenience or distress it may have caused.

An analysis of complaints received by the department during improvement cycle 1 (Q3/Q4 2015) was undertaken to understand whether the improvement in patient experience seen during improvement cycle 1 was associated with a reduction in complaints. The audit was performed as a reduction in complaints was one of the drivers for, and key aims of the program. Patient complaints for (Q3/Q4 2015) were compared to an equivalent period during the previous year before the initiation of the program (Q3/Q4 2014).

Improvement cycle 2

The second improvement cycle started after implementation of the actions listed above. The action plans were again instituted over a 2 month period. We hypothesised that these actions would lead to further improvements in patient experience and satisfaction by allowing targeted interventions at an individual ward level. We felt engagement with the program would be increased by giving ward managers access and responsibility for their own ward data.

Data was collected over a 6 month period (March to August 2016). As per the action plan the analysis and reporting of data was done at both a departmental and ward level. Responses were collected from 380 patients

Analysis of data from improvement cycle 2 demonstrated negligible further improvement in scores across all PPE-15 domains.

Data from improvement cycle 2 demonstrated:

Patient satisfaction was 8.9 out of 10 (compared to 9.2 in cycle 1)

Friends and family test score approached 100%

The mean number of patients experiencing a problem in one of the PPE-15 domains decreased further from 5.3% to 4.1%

The mean number of patients experiencing problems in each of the targeted domains decreased further from 12.6% to 10.1% for Qu3, from 3.3% to 1.9% for Qu4 and from 17.1% to 10.8% for Qu6.

Finally the audit of complaints demonstrated that during an equivalent period complaints had reduced by 38% from 40 in Q3/Q4 2014 to 25 in Q3/Q4 2015 (a 38% reduction).

Results

The patient experience improvement program instituted with our department identified areas of poor patient experience allowing targeted action plans to be developed and instituted. These action plans focussed on improving staff awareness and engagement with patient experience, improving staff and patient communication and ensuring patients were aware of plans for their own care. These actions helped us to achieve our aims namely:

Improved patient satisfaction and overall patient experience by >10% for our orthopaedic patients.

Delivery of a sustained high level of patient experience with all domains demonstrating results better than published national standards

Achieve a friends and family test score of >90%. Friends and family test score recorded in improvement cycle 2 approached 100%.

Reduced departmental complaints from 40 to 28 during a comparable period, a 38% reduction.

Lessons and limitations

There are a number of limitations with the program we have used. Firstly questionnaires require the patient to have capacity to answer. In cases where the patient lacks capacity questions can be discussed with carers / relatives although their views may be different front that of the patient. Secondly due to the volume of discharges it is not possible to undertake questionnaires with every patient. Rates of sampling were in the region of 10-15% of all discharges in any given week. In an attempt to limit selection bias the discharge team randomly selected patients to complete the questionnaires each day, a process that is independent from the nursing team on the ward. The nursing staff were therefore unable to direct the discharge team to patients likely to have had a good experience and away from potential problem patients. Thirdly the questionnaires are collected while the patient was still in hospital. While this may positively bias the results, as patients are less likely to complain about their care whilst still in hospital, it allows any problems presented to be actioned directly. It is our experience that patients appreciate seeing their concerns recognised and immediate action being taken. We believe this may be one of the reasons for the significant drop in formal written complaints. Finally, between each improvement cycle a number of changes were made to patient care. It is therefore difficult to be certain which of these were responsible for the improvements observed. Each of these changes targeted a specific aspect of underperformance; namely communication, awareness of patient experience and patient involvement in care. The changes employed were therefore complimentary in improving patient experience and would have been unlikely to have the same effect if used in isolation. The results observed are unlikely to be due to chance. Other than the actions listed no other changes were made to standard care during that time and the actions implemented led to a significant, sustained and consistent improvement in patient experience over a period of 18 months. There were minor monthly variations in patient experience but overall the trend was for a consistent improvement with over time.

The model used for this improvement cycle is generalizable and applicable to all inpatient care. Patient experience is one of the core pillars of the NHS outcomes framework and as such should be considered and evaluated for all patients. The metrics we have used (PPE-15, patient satisfaction, complaints, friends and family test) are not specific to orthopaedic nor are the action plans and measures we have used to improve performance. We therefore believe that a similar process could be used in any clinical area and in any hospital to facilitate the measurement and improvement of patient experience.

This project was labour and time intensive with over 1000 patients surveyed during the 18 month period. It was not sustainable to continue this level of sampling in the longer term. However, the success of this project has led our trust to support the use of the PPE-15 and other patient experience metrics more widely within the trust. Moving forward sampling of each clinical area will co-ordinated centrally by the trusts central patient experience team as part of their ‘1000 voices’ project. Although sampling will occur less frequently the data collected as part of this project will allow the orthopaedic team to continue to monitor performance in our clinical areas, share our experience more widely others clinical teams and sustain the high levels of patient experience achieved during the initial quality improvement program.

Conclusion

Patient experience and departmental complaints performance can be improved by the use of a targeted quality improvement program such as the one outlined in this report. Using this model we have achieved sustained levels of patient experience greater than those reported for national UK practice in the literature.6 Direct interaction with the patient during their care episode helps to get them involved and offers the opportunity for them to raise concerns which can be addressed promptly helping to reduce complaints and improve satisfaction. The model we have described is generic and suitable for adoption in a variety of inpatient and outpatient settings across a range of specialties. Due to its success the model has now been adopted in all clinical areas within our trust, with central co-ordination to facilitate trust wide sampling of patient experience data.