Patient experiences: a systematic review of quality improvement interventions in a hospital setting.

Purpose: In the era of value-based healthcare, one strives for the most optimal outcomes and experiences from the perspective of the patient. So, patient experiences have become a key quality indicator for healthcare. While these are supposed to drive quality improvement (QI), their use and effectiveness for this purpose has been questioned. The aim of this systematic review was to provide insight into QI interventions used in a hospital setting and their effects on improving patient experiences, and possible barriers and promoters for QI work.
Methods: Prisma guidelines were used to design this review. International academic literature was searched in Embase, Medline OvidSP, Web of Science, Cochrane Central, PubMed Publisher, Scopus, PsycInfo, and Google Scholar. In total, 3,289 studies were retrieved and independently screened by the first two authors for eligibility and methodological quality. Data was extracted on the study purpose, setting, design, targeted patient experience domains, QI strategies, results of QI, barriers, and promotors for QI.
Results: Twenty-one pre-post intervention studies were included for review. The methodological quality of the included studies was assessed using a Critical Appraisal Skills Program (CASP) Tool. QI strategies used were staff education, patient education, audit and feedback, clinician reminders, organizational change, and policy change. Twenty studies reported improvement in patient experience, 14 studies of the 21 included studies reported statistical significance. Most studies (n=17) reported data-related barriers (eg, questionnaire quality), professional, and/or organizational barriers (eg, skepticism among staff), and 14 studies mentioned specific promoters (eg, engaging staff and patients) for QI. Conclusions: Several patient experience domains are targeted for QI using diverse strategies and methodological approaches. Most studies reported at least one improvement and also barriers and promoters that may influence QI work. Future research should address these barriers and promoters in order to enhance methodological quality and improve patient experiences.

Introduction

In the era of value-based healthcare we strive for the most optimal outcomes and experiences from the perspective of the patient. Therefore, patient experience has become a key quality indicator for healthcare and is positively associated with patient safety and clinical effectiveness.1 Measuring and analyzing experiences is seen to support improvement in healthcare quality governance, public accountability, and patient choice.2–5 Through the years, a variety of patient experience measures have been developed and used in healthcare, among which are questionnaires, focus groups, and interviews. While such tools are supposed to drive quality improvement (QI), their use and effectiveness for this purpose has been questioned.6,7 The lack of QI may be linked to methodological barriers (eg, using a survey with poor psychometric properties, infrequent data-collection, ineffective monitoring), hampering the assessment of effectiveness. Also the lack of local ownership for QI, limited training and education of staff for QI, as well as the absence of an organizational culture for change has a negative effect on the improvement of patient experiences.8,9 Moreover, patient experiences cover diverse domains, which all require appropriate measurement and different quality improvement initiatives.10

Previous systematic reviews examining one or more aspect of QI initiatives confirms the aforementioned barriers, and all conclude that the optimal approach for using experience data effectively is lacking.11–13 The aim of this systematic review, compared to other reviews, was to broaden our scope to national as well as local patient experience measures in a hospital setting and gain more insight into the effectiveness of diverse QI initiatives and their influencing factors. The following research questions were addressed:

Which QI strategies are being used to improve patient experiences?

What is the effectiveness of QI interventions to improve patient experiences?

What are the barriers and promoters of QI interventions aimed at improving patient experiences?

Methods

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were used to design this review.14

Scope of the review

Patient experiences were defined as; “the sum of all interactions, shaped by the organization’s culture, that influence patient perceptions, across the continuum of care“.15 We limited our scope to patient experiences related to Picker’s eight domains of Person Centered Care; 1) Accessibility, 2) Effective treatment and trusted professionals, 3) Continuity of care and transitions, 4) Involvement in decisions and respect for preferences, needs, and values 5) Comprehensible information and support for self-care, 6) Involvement of and support for family and friends, 7) Emotional support, empathy, and respect, and 8) Attention for physical and environmental needs.16 Studies that were limited to evaluating patient satisfaction, rather than patient experience, were beyond the scope of this review. Patients generally tend to overrate their satisfaction, for example due to gratitude bias.17 Therefore, the validity and usefulness of satisfaction data is questionable.18

Information sources and search parameters

The following databases were searched on September 29, 2017: Embase, Medline OvidSP, Web of Science, Cochrane Central, PubMed Publisher, Scopus, PsycInfo, and Google Scholar.

Search terms were derived from previous studies11,19 and our research questions. The thesaurus in Embase which formed the basis for the search strategies for the other electronic databases is shown in Figure 1.

Figure 1

Flowchart literature search.

Eligibility criteria

Included studies met the following criteria: 1) QI interventions that targets patient experiences; 2) patients’ experiences are examined pre- and post-intervention; 3) hospital setting; 4) written in English; and 5) published after 2006. Non-intervention studies and editorials, conference papers, reviews, books, interviews, or columns were excluded, as well as studies that could not be retrieved in full-text.

Data extraction

Two authors (CB and HB) independently screened titles and abstracts for inclusion. Eligible studies were evaluated in full-text by both authors. A third author (LdJV) was consulted when agreement was not reached. For all eligible studies, details about study design, patient experience, topic, measurements, sample size, interventions, and outcomes were extracted.

Data synthesis and analysis

Due to the variation of the used methodology, interventions, topics, heterogeneity of data, and method of reporting outcomes, we performed a narrative synthesis of all relevant themes within and across the studies.

Risk of bias

The methodological quality of the included studies was assessed independently by the same researchers using the Critical Appraisal Skills Program (CASP) Qualitative Checklist.20 The checklist was adapted using two questions in order to assess and compare all eligible studies with diverse methodology. The question “Is a qualitative methodology appropriate?” was adapted into “Is a qualitative/quantitative methodology appropriate?” For quantitative studies, the question “Was the data analysis sufficiently rigorous?” was judged by considering size of the confidence intervals and by examining whether the following variables were considered: confounding factors, blinding of providers, and response rate. Studies that obtained negative ratings for at least five out of ten items (ie, “no”, “can’t tell”, or “unclear”) were excluded from this review.

Results

In Figure 1, a flow diagram of the search process is presented. After removal of duplicates, a total of 3,289 records were identified. Of these, 3,139 studies were excluded based on title and abstract. Of the remaining 150 full-text articles, 21 studies were in agreement with the inclusion criteria and were included for review.

Characteristics of included studies

The search resulted in 15 pre–post intervention studies, two qualitative studies,21,22 three RCT’s,23–25 and a longitudinal study.26 One study was performed in Tanzania,24 and the other studies in either Europe, the US, or Canada. The majority of studies (n=15) included patients from a specific department (eg, neurosurgery). One study focused on the transition of hospital to primary care in a radical prostatectomy pathway.27 In 12 studies, patient experiences were assessed using an existing survey (eg, Hospital Consumer Assessment of Healthcare Providers and Systems, HCAHPS), and seven studies used a self-developed survey. The remaining two studies used informal interviews21 or a combination of methods.22 The study characteristics are summarized in Table 1.

Table 1

Study characteristics

Author/Year	Setting	Design and size	PE assessment method(s) and PE topic(s) to be improved
Ahrens and Wirges39(2013)	Neuro-medical surgery, US	Pre–post designn=60 pre vs 61 post	Survey (H-CAHPS)Medication side-effects
Bellamkonda et al32(2016)	Emergency department, US	Pre–post designn=193 pre vs 45 post	Survey (Point-of-service cards)Provider compassion
Bookout et al28(2016)	Cardiac telemetry, US	Pre-post designn=N/R	Survey (H-CAHPS)Pain management
Davies et al37(2007)	N/A, UK	Pre–post designn=N/R	Survey (Modified CAHPS)Overall patient experiences
Indovina et al23(2015)	General internal medicine, US	RCTn=35 pre vs 30 post	Survey (H-CAHPS)Provider specific experiences
Jayasinha33(2016)	Pediatrics, US	Pre–post designn=94 pre vs N/R post	Survey (self-developed)Cycle time
Jiang et al38(2016)	Otolaryngology surgery, US	Pre–post designn=17 pre vs 10 post	Survey (S-CAHPS)Enough time, involvement and respect
Kamiya et al24(2017)	N/A, TZ	RCTn=1,101 pre vs 1,070 post	Survey (self-developed)Communication, confidence and trust
Kane et al30(2015)	Emergency department, US	Pre–post designn=N/R	Survey (Press Ganey survey)Crowding
Khan et al34(2014)	Neurosurgery, UK	Pre–post designn=150 pre vs 150 post	Survey (self-developed)Communication
Maqbool et al35(2016)	Orthopedics, plastics, CA	Pre–post designn=42 pre vs 20–25 post	Survey (self-developed)Stress levels related to waiting
Nieboer et al26(2014)	N/A, NL	Longitudinal studyn=140 pre vs 177 post	Survey (Mind the GAP scale)Transitional care delivery
Norgaard et al40(2012)	Orthopedics, DK	Pre–post designn=1,279 pre vs 1,854 post	Survey (ISRF)Communication
Norton et al31 (2014)	N/A, UK	Pre–post designn=749 pre vs 783 post	Survey (self-developed), interviewsSleep disturbance
Pratt et al, 201121(2011)	Pediatric intensive care, UK	Qualitative studyn=4 families pre vs 8 parents post	Informal interviewsAdmission to healthcare
Reeves et al25(2013)	N/A, UK	RCTn=987 pre vs 648 post	Survey (NHS Adult inpatient questionnaire)Nursing care
Roberts41(2013)	Physiotherapy, UK	Pre–post designn=100 pre vs 349 post	Survey (CSP’s patient feedback questionnaire)Overall patient experience
Ugarte22 (2015)	N/A, UK	Qualitative studyn=76 pre vs 106 post	Narrative stories, survey (FFT), interviewsWaiting time
Van Houdt et al27(2013)	Radical prostatectomy pathway, BE	Pre–post designn=46 pre vs 46 post	Survey (self-developed)Coordination between caregivers
Waldhausen et al29(2009)	Surgery, US	Pre–post designn=N/R	Survey (Picker Questionnaire)Waiting and value added time
Wilson et al42(2017)	Medical oncology, surgery, US	Pre–post designn=N/R pre vs 27 postInterviews n=30 pre vs 30 post	Survey (H-CAHPS), interviewsHospital environment noise at night

Abbreviations: BE, Belgium; CA, Canada; CSP, the chartered society of physiotherapy; DK, Denmark; FFT, family and friends test; H-CAHPS, hospital consumer assessment of healthcare providers and systems; ISRF, interpersonal skills rating form; NHS, national health service; N/A, not applicable; NL, the Netherlands; PE, patient experiences; S-CAHPS, consumer assessment of healthcare providers and systems surgical care survey; TZ, Tanzania; UK, United Kingdom; US, United States.

Methodological quality

For all the differences of methodological design and quality, none of the 21 studies obtained more than five negative ratings, thus were all included (Table 2). All studies clearly described the aims of their research, used appropriate methodology and research design, and collected data in a way that addressed the research question. However, in six studies it could not be determined whether the recruitment strategy was appropriate to the aims of the research.21,22,28–31 Two qualitative studies21,22 did not report on the relationship between researcher and participants and, for 14 out of 19 quantitative studies, patients remained anonymous during the entire study. Six studies did not report whether they had taken ethical issues into consideration. The rigor of data-analyses was rated insufficient in 14 studies mostly because they didn’t report statistical significance of pre–post changes in patient experience scores, or multiple comparisons were made without correcting for multiple testing. The latter increases the chance of false positives. Seven studies did not clearly describe their findings in relation to other studies or current practice.21,25,28,32–35 Lastly, three studies were rated “unclear,” because the authors did not consider the findings in relation to current practice or policy or they did not identify new areas for research.21,32,33

Table 2

CASP quality assessment of included papers

First author	1	2	3	4	5	6	7	8	9	10
Ahrens39	Yes	Yes	Yes	Yes	Yes	Can’t tell	Yes	No	Yes	Valuable
Bellamkonda32	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	No	Unclear
Bookout28	Yes	Yes	Yes	Can’t tell	Yes	Can’t tell	Can’t tell	No	No	Valuable
Davies37	Yes	Yes	Yes	Yes	Yes	Can’t tell	Yes	No	Yes	Valuable
Indovina23	Yes	Yes	Yes	Yes	Yes	Can’t tell	Yes	No	Yes	Valuable
Jayasinha33	Yes	Yes	Yes	Yes	Yes	Can’t tell	Can’t tell	No	No	Unclear
Jiang38	Yes	Yes	Yes	Yes	Yes	Can’t tell	Yes	Yes	Yes	Valuable
Kamiya24	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Valuable
Kane30	Yes	Yes	Yes	Can’t tell	Yes	Can’t tell	Can’t tell	No	Yes	Valuable
Khan34	Yes	Yes	Yes	Yes	Yes	Yes	Can’t tell	No	No	Valuable
Maqbool35	Yes	Yes	Yes	Yes	Yes	Can’t tell	Yes	No	No	Valuable
Nieboer26	Yes	Yes	Yes	Yes	Yes	Can’t tell	Yes	No	Yes	Valuable
Norgaard40	Yes	Yes	Yes	Yes	Yes	Can’t tell	Yes	No	Yes	Valuable
Norton31	Yes	Yes	Yes	Can’t tell	Yes	Can’t tell	Yes	Yes	Yes	Valuable
Pratt21	Yes	Yes	Yes	Can’t tell	Yes	Can’t tell	Can’t tell	Yes	No	Unclear
Reeves25	Yes	Yes	Yes	Yes	Yes	Can’t tell	Yes	Yes	No	Valuable
Roberts41	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Valuable
Ugarte22	Yes	Yes	Yes	Can’t tell	Yes	Can’t tell	Yes	Can’t tell	Yes	Valuable
Van Houdt27	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Valuable
Waldhausen29	Yes	Yes	Yes	Can’t tell	Yes	Can’t tell	Can’t tell	Yes	Yes	Valuable
Wilson42	Yes	Yes	Yes	Yes	Yes	Can’t tell	Yes	No	Yes	Valuable

Notes: 1) Was there a clear statement of the aims of the research? 2) Is a qualitative/quantitative methodology appropriate? 3) Was the research design appropriate to address the aims of the search? 4) Was the recruitment strategy appropriate to the aims of the research? 5) Was the data collected in a way that addressed the research issue? 6) Has the relationship between researcher and participants been adequately considered? 7) Have ethical issues been taken into consideration? 8) Was the data analysis sufficiently rigorous? 9) Is there a clear statement of findings? 10) How valuable is the research?

QI interventions

Various QI strategies were applied (Table 3). These can be categorized into staff education, patient education, audit and feedback, clinician reminders, organizational change, promotion of self-management, and policy change.36 The most common strategies are organizational change21,22,24,26–35,37,38 and staff education.23–25,29–32,34,37,39–42 These strategies all relate to changing ward procedures and staff behavior. Most studies applied multiple QI strategies,21,23–26,29–32,34,37–39,41,42 while other studies used only one of the aforementioned QI strategies.22,27,28,33,35,40 Eleven studies reported to use a specific change management approach or tool. These include Lean or Lean Six Sigma,24,29,30,32,33,38 Plan-Do-Study-Act,22,34,35 Kotter’s Model of Change,42 and a 30-step-scenario.27 One study used The CAHPS improvement guide.37

Table 3

Interventions and results

First author	PE topic: Outcome measure(s)	Main QI method and intervention (theoretical model, tool)	Number of statistically significant pre/post comparisons; significant results in words
Ahrens39	Medication side-effects: understanding the description of medication	Patient education: information brochure/website; Staff education: communication skills; Clinician reminders: repeated communications through work- and e-mail	N/R
Bellamkonda32	Provider compassion: receiving compassionate care	Organization change: survey cards and sending a follow-up letter; Staff education: communication skills of shared decision-making; Patient education: giving information by staff(Lean, Kano)	1/1; Improvement in perceived concern and sensitivity
Bookout28	Pain management: experienced pain management; overall patient experience	Organization change: implementation of a patient and family advisory council and comfort carts	N/R
Davies37	Overall patient experiences: kept informed of a clinic wait; taken to exam room within 15 minutes; schedule appointment when needed; treated with courtesy and respect by staff; received follow-up of test results; doctor is informed and up-to-date; rating of doctor’s knowledge of medical history; patients see their personal doctor; understandable explanation by doctor; understanding/satisfaction with FU-plan	Organization change: redesigning processes and better information; Staff education: communication skills; Audit and feedback: patient and staff evaluations and focus groups(The CAHPS improvement guide)	1/15; More patients were taken to the exam room within 15 minutes
Indovina23	Provider specific experiences: overall hospital rating; courtesy/respect; clear communication; listening	Staff education: communication skills; Audit and feedback: real time patient feedback	1/4; The overall hospital rating was higher in the intervention group than in the control group
Jayasinha33	Cycle-time: overall cycle time; friendliness of staff, nurses, and front desk	Organization change: relocation of staff and revise unnecessary processes(Lean Six Sigma)	N/R
Jiang38	Overall patient experience; did the provider spend enough time with you; did the provider encourage you to ask questions; did the provider show respect for what you had to say	Organization change: new procedure of scheduling post-operative appointments; Clinician reminders: a list of standardized questions in each clinic room?(Lean, A3)	4/6; 2 weeks post-intervention: improvement of provider spend time with the patient; encourage to ask questions; show respect for what the patient had to say 2 months post-intervention: provider showed more respect for what the patient had to say
Kamiya24	Communication: enough time to discuss; reason for treatment; listen; get answers; confidence and trust; reason of test; how to take medication; purpose of medication; side-effect	Organization change: redesigning workspaces and reorganize processes and procedures; Staff education: training of the 5S(Lean, 5S)	1/10; Improvement in understandable explanation by health worker about test results
Kane30	Crowding: likelihood to recommend (percentile); waiting time to see doctor (percentile); informed about delays (percentile)	Staff education: rapid process improvement workshop; Organization change: 5S workshop and value stream mapping; Audit and feedback: data sharing with visibility walls(Lean, 5S)	N/R
Khan34	Communication: mean experience with surgeons/junior doctors/nurses/hospital service	Organization change: optimizing staff collaboration Staff education: communication skills(Plan Do Study Act)	2/4; Improved experience with junior doctors and nurses
Maqbool35	Waiting experience: Stressful waiting process	Organization change: floor signage and navigation guide(Plan Do Study Act)	1/1; Reduction in patient stress levels
Nieboer26	Transitional care delivery: staff knows how to talk and listen to teenagers; treats as an individual and understands needs; staff understands realities of being a teenager; providers work well together; interested in me as a person, not just the illness; make own decisions about healthcare options; opportunities to be seen in the clinic alone; provides info to other involved professionals; decide who is in consultation/examination room; helps prepare for move to adult services; helps plan for future; providers arrange joint appointments hours; helps improve independence with action plan; does not waste my time at the clinic; staff to talk about sensitive or difficult issues; staff member coordinating my transitional care	Patient education: group education; Promotion of self- management: the completion of an individual transition plan; Organization change: optimizing caregiver consultation; Policy Change; joint policies to align procedures and treatment; Clinician reminders: formats and instruments for intervention(Breakthrough Series improvement and implementation strategy)	2/16; Improved provision of opportunities for adolescents to visit the clinic alone and to decide who should be present during consultations
Norgaard40	Communication: doctor/nurse/nursing assistant prepared for interview; understandable language doctor/nurse/nursing assistant; opportunity explain problem to doctor/nurse/nursing assistant; doctor/nurse/nursing assistant explain examination/treatment; doctor/nurse/nursing assistant explain future plans; satisfied with information from doctor/nurse/nursing assistant; coherent information from doctors/nurse/nursing assistant; overall information received coherent; experience kindness and obligingness; doctors/nurse/nursing assistant enough time; involved in care and treatment	Staff education: communication skills	15/19; Improved preparation for interview of nurses/nursing assistants; understandable language of doctor/nurse/nursing assistant; opportunity to explain problems to nurse/nursing assistant; explanation of examination/treatment and future plans by nurse/nursing assistant; satisfaction with information from doctor/nurse/nursing assistant; coherent information from doctors/nurse/nursing assistant; coherent received overall information; experience of kindness and obligingness; time with doctors/nurse/nursing assistant/time and involvement in care and treatment
Norton31	Sleep disturbance: patient rating of sleep	Audit and feedback: ward-specific patient feedback Staff education: ward-specific action plan; Organization change: window blinds instalment and reduce noise; Clinician reminders: text notifications by posters and telephone	1/1; Improved ratings of sleep
Pratt21	Admission to healthcare: experiences of parents about the use of a structured checklist to ensure a successful admission	Clinician reminders: new admission checklist; Organization change: a pre- admission key-member of staff	N/A
Reeves25	Nursing care: basic feedback or control vs feedback plus	Audit and feedback: ward-specific patient feedback; Staff education: ward-specific action plan	1/1; Nursing care is improved more for Feedback Plus than Basic Feedback or Control
Roberts41	Overall patient experience of physiotherapy care: 37 items of patient experiences from initial contact to discharge	Audit and feedback: ward-specific patient feedback; Staff education: ward-specific action plan; Clinician reminders: item on the agenda of a departmental meeting	8/37; Improved choices of appointment times; addressment by the name of choice; change to say what was on the mind; listening to the patient; choice of options for treatment; information of possible achievements; satisfaction with care; involvement in deciding about treatment plan
Ugarte22	Waiting time: overall patient experience of waiting time; time spend in the clinic	Organization change: new appointment scheduling profile(Plan Do Study Act)	N/R, N/A
Van Houdt27	Coordination between caregivers: specialist/GP familiar with recent medical history; GP aware of results of surgery; GP aware of recommended treatment; GP had information to make treatment decision; received info you wanted about condition/treatment; contradictory info in hospital; contradictory info from caregivers at home; contradictory info between caregivers; you knew who to ask if anxious or worried; you knew who to contact if you experienced problems; received a clear/understandable response to questions; you knew what the next step in your care would be; home care staff worked well together; home care staff made good agreements; caregivers aware of special conditions/needs	Organization change: Implementation of a care pathway(30-step-scenario)	0/16
Waldhausen29	Waiting and value added time: overall patient experience of value-added time with provider	Staff education: rapid process improvement workshop; Organization change: standardization of exam rooms and revise unnecessary processes(Lean, 5S)	1/1; Improved overall patient experience
Wilson42	Hospital environment noise at night	Staff education: purposeful rounding to inform patients and the use of a flashlight; Clinician reminders: visual aids for staff (Kotter’s model of change)	N/R

Abbreviations: N/R, not reported; N/A, not applicable.

QI outcomes

With the exception of one study,27 all studies reported at least one improved patient experience score following intervention. A dichotomy can be approximately found; six studies focused on improving the interaction of staff with patients (eg, communication, compassion, respect),23,24,32,34,38,40 and 10 studies focused on improving processes (eg, waiting time, noise disturbance, pain management).21,22,27–31,35,39,42 Five studies had objectives in both areas.25,26,33,37,41 Fourteen studies examined whether statistically significant change had occurred following intervention. In these 14 studies, 106 pre–post comparisons were made, of which 38 pre–post improvements were labeled statistically significant by the researchers. Six of these studies were targeted on staff–patient interaction,23,24,32,34,38,40 and four studies on improving processes.27,29,31,35 Within the studies focusing on improving interactions, 55% of the pre–post comparisons significantly improved, while this was 16% within studies of improving processes and 17% within studies who wanted to improve on both levels. Noteworthy is the fact that studies that in advance targeted on the improvement of one outcome measure, such as improving waiting experience,35 compassionate care,32 ratings or sleep,31 nursing care,25 or overall patient experience,29 were most successful.

Barriers and promoters

Eighteen studies mentioned specific barriers for QI (Figure 2).22–27,31–35,37–42 These can be categorized into data-related, professional, and organizational barriers.8 Commonly reported data-related barriers were the risk of bias due to a small sample size23,32,37,38,42 or a low response rate,25,26,40 and confounding by simultaneously applied interventions22,23,26,32,39,41,42 or a lack of blinded providers.27,34,38,41 Furthermore, four studies mentioned that their QI intervention may have been too short to induce significant change.24,26,35,37 Skepticism amongst staff about the necessity or usefulness of the proposed change was the most frequently reported professional barrier.25,26,33,35,37,39 Also, staff changes, especially at management level, were held responsible for not achieving objectives,24,27,34,40 along with the lack of time required for a successful implementation.25,27,34,37,39,40 The organizational barriers mentioned were mostly related to a lack of engaged management for QI24,26,27,37 or no culture of change.33

Figure 2

QI initiative.

Fourteen studies mentioned specific promoters for QI (Figure 2).22–26,31,34,35,37–42 Several studies indicate that a QI intervention only succeeds if the organization supports system change and approaches this through engaged leadership.22,25,30,37–41 Staff must be involved in data collection and be given help and insight into the interpretation of departmental patient experience scores.25,34,37 It is important to support staff by means of coaching, provision of information, education, and multi-disciplinary collaboration.23,25,26,34,35,37,39 Another way that may facilitate QI is to involve patients in designing QI interventions.23,25,35,37 Finally, frequent or continuous assessment of patient experiences has been mentioned as an important element to maintain a culture of change in healthcare.31,34,37,38,42

Discussion

The aim of this systematic review was to broaden our scope to national as well as local patient experience measures and gain more insight into the effectiveness of diverse QI initiatives and their influencing factors in a hospital setting. Although all studies reported positive results, they showed large variability in their methodology of QI initiatives which hamper the comparison of results. However, similarities were found in experienced barriers and the proposed promoters for QI.

QI strategies used to improve patient experiences

Most studies applied a combination of QI strategies. Organization change was one of the most frequently used QI strategies, probably because it encompasses a wide range of topics; from physical changes to the hospital surrounding, to changes in staff. Another frequently used QI strategy is staff education. About half of the included studies educated staff as part of their QI intervention. The other half reported resistance among staff,25,26,35 discussed staff changes as a barrier for QI success,27,32,38 or mentioned not having a culture that supports QI.33 Besides involving staff, it may also be valuable to involve patients in QI efforts. Five studies involved patients in designing QI interventions by patient focus groups or participation in a patient and facility advisory council, and may well offer an additional strategy for QI.21,28,31,32,37 To reach its full potential, it is, however, important that staff members recognize and value patient involvement.23,25,35,37,43,44

Effectiveness of QI interventions to improve patient experiences

It is noteworthy that studies which targeted improving interactions of staff with the patient seem more successful than studies which targeted improving processes. Furthermore, studies which targeted the improvement of one outcome measure in advance were all successful.29,31,32,35,45 Within the studies with multiple outcome measures,23,24,26,27,34,37,38,40,41 it often remained unclear whether they actually intended to improve all outcomes, this could be an explanation for the lack of significant change. Other explanations can be found in the mentioned data-related, professional, and organizational barriers (Figure 2). Obviously, the type of study design is also an important determinant of the results and their interpretation. Three of the studies were Randomized Controlled Trials (RCTs).23–25 These studies were successful in improving patient–provider communication. An obvious advantage of an RCT is the possibility to assign differences in pre–post scores to the effects of the QI intervention. However, in clinical practice an RCT is not always feasible for practical and methodological reasons (eg, ethical issues and costs). The 11 studies reporting the use of a specific change management approach or tool (eg, Lean or Lean Six Sigma, Plan-Do-Study-Act) had no better results in terms of methodology or significance.

Seven studies reported improved patient experiences but did not examine whether this improvement was statistically significant,21,22,28,30,33,39,42 for example because this was beyond the scope of their research question. Data had served as a communication tool to establish the need for change33 or to provide insight into the development or operation of a QI strategy.22

Barriers and promoters for QI

Almost all studies reported on specific barriers or promoters for QI, and a relationship is assumed with (a lack of) significant results. For instance, four of the studies did not adequately report on the number of patients included, or included a small sample size.28,30,39,42 The risk of a small sample size is that changes in score results reflect random fluctuations rather than actual improvement. Regarding professional and organizational barriers, the findings are in line with previous studies among healthcare professionals and managers8,9,46 and frequently reported barriers for QI in other healthcare settings such as mental healthcare.47,48 This highlights the importance of designing and implementing strategies to involve and educate staff.9,12,49 Physician engagement may, for instance, be enhanced by developing clear and efficient communication channels with physicians by building trust, understanding, and identifying or developing physician leaders.50

Promoters of QI interventions were focused around engagement of patients, staff, management, and culture. This is in line with previous systematic reviews on the use of patient experiences for QI11,12 and qualitative studies on promoters and barriers for improving patient experiences in healthcare.8,51 A barrier that was not identified in the current review was changing the employees’ mind-set from “provider-focused” to “patient-focused,” which is an important aspect of patient-centered care.8,51

Strengths and limitations

A strength of this review is that outcomes, barriers, and promoters for QI were derived from the studies included as a valuable source for further QI work. Also, the findings of previous reviews11–13 were extended by this, looking beyond national patient experience surveys and gaining insight into the effectiveness of QI. In clinical practice, it is usually the case that departments obtain national as well as local patient feedback using a variety of measures (eg, surveys, focus groups). The inclusion of a wide variety of patient experience measures can also be considered a limitation of the current review. The many differences between studies (eg, study design, type of patient experience measures) hamper the interpretation of results. The studies that did meet inclusion criteria were evaluated for their methodological quality using the CASP Qualitative Checklist. As its name already implies, this checklist was developed for qualitative studies and was, therefore, less appropriate for quantitative studies.

Implication for future policy and research

Knowledge on barriers and promoters provides a valuable source of information that can be used to guide future QI initiatives. Addressing data-related, professional, and organizational barriers may positively influence the effectiveness of QI interventions that target patient experiences. Ideally, healthcare organizations or hospital departments develop structured plans on how to use patient feedback for QI and methods to engage clinicians in this process. In current practice, such plans are often lacking.19,52 Also, it is encouraged to include a follow-up assessment to examine changes in patient experience following QI intervention. This is important, as a change is an improvement only when the patient experiences it as such. Large-scale RCT’s are needed to determine whether improvements are actually the direct result of a QI intervention and also to compare the effectiveness of different QI strategies. Another potentially valuable direction for future research is to examine the extent to which patients could and should be involved in designing QI interventions. Just as experiences may differ between patients and staff, this could also be the case with their perceptions on future healthcare.

Conclusion

Despite the heterogeneity of methodology and methodological quality of studies reviewed, many lessons can be learned. A wide range of patient experience domains were targeted for QI, but outcome measures focused on improving communication and interaction were more successful than outcome measures focused on changing processes. Alongside this, studies with a small number of outcome measures were most effective, organizational change, and staff education were the most frequently used QI strategies in those cases. While most studies report positive outcomes, they also report on significant barriers and promoters that can influence QI work, not least a sound design of research. Furthermore, engagement of patients and all stakeholders at both departmental and management level is commonly recommended for successful QI. Future research should address barriers and promoters in order to enhance methodological quality and study outcomes.