Impact of a blended curriculum on nursing handover quality: a quality improvement project.

CONTEXT AND
OBJECTIVE: The negative consequences of inadequate nursing handovers on patient safety are widely acknowledged, both within the literature as in practice. Evidence regarding strategies to improve nursing handover is, however, lacking. This study investigates the effect of a tailored, blended curriculum on nurses' perception of handover quality.
METHODS: We used a pre-test/post-test design within four units of a Belgian general hospital. Our educational intervention consisted of an e-learning module on professional communication and a face-to-face session on the use of a structured method for handovers. All nurses completed this blended curriculum (n=87). We used the Handover Evaluation Scale (HES) to evaluate nurses' perception of handover quality before and after the intervention. The HES was answered by 87.4% of the nurses (n=76 of 87) before and 50.6% (n=44 of 87) after the intervention. Confirmatory factor analysis was used to assess the validity of the HES.
RESULTS: The original factor structure did not fit with our data. We identified a new HES structure with acceptable or good fit indices. The overall internal consistency of our HES structure was considered adequate. Perception of nurses on Relevance of information showed a significant improvement (M=53.19±4.33 vs M=61.03±6.01; p=0.04). Nurses also felt that the timely provision of patient information improved significantly (M=4.50±0.34 vs M=5.16±0.40; p=0.01).
CONCLUSION: The applied intervention resulted in an improved awareness on the importance of Relevance of information during handovers. After our intervention, the nurses' perception of the HES item 'Patient information is provided in a timely manner' also improved significantly. We are aware that the educational intervention is only the first step to achieve the long-term implementation of a culture of professional communication based on mutual support. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Patient handover is a formal moment in the patient care continuum. It involves the transfer of essential information, responsibility, and accountability between caregivers or healthcare teams. Handovers can take place between shifts, units, disciplines and care settings.1 2 Inadequate handover is a serious concern, with tangible consequences for institutions, caregivers and patients alike. That is why the WHO listed ‘communication during patient care handovers’ as one of its five patient safety priorities.3 Despite handover being recognised as a vulnerable aspect of patient care, little is known on how to improve its quality.2 4–6 A systematic review of the literature on nursing handover concluded that inadequate handover’s negative consequences are well known. However, little research to identify best practices has been conducted.7 This presents us with a critical gap in research and practice; handover is recognised as a determining aspect of care. At the same time, little is known in the literature about improving and safeguarding its quality.8

The rationale for a change

Handover requires effective communication between sender and receiver, which is hampered by the complexity of the underlying system and varying performance among caregivers. This impacts handover effectiveness and efficiency.9 10 Hence, improvement programmes should not solely focus on the content and structure of handover; a more comprehensive perspective is needed.

The SBAR acronym (ie, Situation, Background, Assessment and Recommendation) has been introduced to optimise effective communication within hospital settings.11 12 It is one of the most widely used tool to structure patient handover and is promoted by many organisations and institutes (eg, the WHO and the Belgian Federal Public Health Service). The use of SBAR has been related to a significant reduction in unexpected mortality among hospitalised patients and an improvement of interprofessional communication.13 14

Until 2018, caregivers of the regional hospital of Liege (CHR Citadelle Liege) did not use any form of structured communication during handovers. In 2018, based on accreditation standards15 and a TeamSTEPPS pilot quality project by the Regional Platform for Continuous Improvement of Quality of Care and Patient Safety, CHR Citadelle Liege decided that SBAR would become the unified communication tool within the institution, especially during handovers.

Aims and objectives

In this study, we aimed to improve the quality of nursing handover. We assessed nurses’ perception of handover quality before and after completing an educational intervention on professional communication.

Methods

This paper reports on an improvement project aimed to improve nursing handover quality. We used the Standards for Reporting Implementation Studies Statement and Standards for Quality Improvement Reporting Excellence V.2.0 as a guideline in the preparation of this report.16 17

Design and settings

CHR Citadelle Liege is a non-teaching hospital. It is one of the biggest tertiary referral hospitals of the French-speaking part in the Euregio Maas-Rhine, with a campus of 897 accredited beds, and employs about 2000 caregivers (nurses and physicians). CHR Citadelle Liege is a partner of the Euregio Maas-Rhine SafePAT Project.18

Our study was the first step of the SBAR implementation project within CHR Citadelle Liege. We included pilot wards where head nurses imposed the use of SBAR after completion of a blended curriculum. We used a quantitative survey design using a non-equivalent pre-test/post-test sampling strategy. From May to November 2018, we compared the perceived quality of nursing intershift handover, for patients admitted into one geriatric ward, two rehabilitation wards, and a maternal intensive care unit (MICU), before and after our educational intervention (see online supplemental appendix 1).

Intervention

One educator, two head nurses and two advanced nurse practitioners were involved in the SBAR Implementation Team (SBAR IT). The project began with designing and developing the blended curriculum. The curriculum was designed to facilitate reflective learning based on nurses’ habits concerning the intershift handover process. The educational programme consisted of two parts: an interactive e-learning module and a brief face-to-face training session to complete the blended curriculum.

The design and development of the training programme followed an iterative process guided by design science methodology. After identifying the problem to be solved (ie, lack of knowledge and skills regarding SBAR structured handover) and describing the requirements, several rounds of design, testing and adjustment followed. The SBAR IT used a changes flow chart during this ‘Plan–Do–Study–Act’ process to improve the programme’s consistency. Defining the learning objectives required two successive revisions. In particular, the vocabulary had to be simplified. The graphics and user interface of the e-learning also had to be improved. An independent nursing staff of CHR Citadelle Liege tested the e-learning during the iterative process. The difficulty level of the e-learning appeared to be motivating because it requires the learner to be concentrated throughout all duration of the self-learning. No changes were requested by the nurses who tested the face-to-face training. Finally, the SBAR IT reached a consolidated version of the blended curriculum.

Based on the study of Bergs et al,19 the implementation strategy was informed by concepts of Normalisation Process Theory20 and Appreciative Inquiry. The intervention was based on PARHIS21 22 and TeamSTEPPS23 concepts, using a design-based research model. The final version of the blended curriculum was reached after an iterative process conducted from January to April 2018.

Interactive e-learning

The blended curriculum first foresaw a 45-minute interactive e-learning session. At the end of the e-learning, participants were invited to complete a multiple-choice questionnaire to assess their knowledge. Ten questions were randomly generated by the system from a database of 30 questions. A score of 80% was required for learners to pass the course. Each participant had two attempts.

Face-to-face training

The second part of the curriculum entailed face-to-face training for SBAR end-users. It included a 90-minute class with a short theoretical reminder (10 min) followed by practical exercises using clinical simulations. Each simulation was followed by a debriefing conducted on the Center for Medical Simulation model.24

Methods of evaluation

The Handover Evaluation Scale (HES) was used to measure handover quality. It comprises 14 items divided into the following dimensions: relevance and comprehensiveness of information (Quality of information), opportunity to clarify and discuss information (Interaction and support), and timeliness and efficiency of the process (Efficiency).25 26 The participants were asked to answer each item using a 7-point Likert scale (with 1 ‘strongly disagree’ to 7 ‘strongly agree’). Data were coded from 1 to 7, and negatively worded items were reverse scored so that, for all items, higher scores reflected more positive perceptions.

The items were translated to French, several experts involved in the Euregio SafePAT project validated this translation.

Data collection

The sampling method was non-probabilistic. The questionnaire was completed within departments providing different levels of care: MICU for acute care, geriatrics as general wards and two rehabilitation wards. In April 2018, all 87 nurses of the involved wards received face-to-face information on the survey’s objectives and ethical conditions. Nurses were requested to consider each item in terms of what generally happens during intershift handover on their unit. All the 87 nurses completed the blended curriculum before the post-test. Data were collected during two measurement periods of 2 weeks: before the intervention (May 2018) and 3 months after the intervention (November 2018). Data collection sheets were kept in closed envelopes in the wards, and nurses could anonymously fill them out during their shifts voluntarily.

Analysis methods

Demographic characteristics were analysed using proportions for discrete variables, average scores for ordered categorical variables, and means (M) with a 95% CI and lower and upper limits (LL–UL) for continuous variables. Analysis of differences in HES items and factor scoring before and after the intervention was performed using the Mann-Whitney U test.

We assessed the psychometric properties of 14-item HES with our date, based on two a priori conceptualised models. The first one is constructed by O’Connell et al,26 and the second one is constructed by Bergs et al.19 Fit indices of these two models were considered inadequate for the collected data. Modifications to the questionnaire structure were made by mixing the two models and using modification indices of these models. To evaluate the construct validity of the new HES structure, a first-order confirmatory factor analysis (CFA) was performed. The CFA assesses fit between observed data and theoretically grounded models that specifies the hypothesised causal relations between latent factors and their observed indicator variables.27 Our model fit was evaluated using three indicators: the Comparative Fit Index (excellent ≥0.96, acceptable 0.90–0.95, inadequate <0.90), Tucker-Lewis Index (excellent ≥0.95, acceptable 0.90–0.94, inadequate <0.90), and Root Mean Square Error of Approximation (excellent ≤0.06, good ≤0.08, mediocre 0.08–0.10, inadequate >0.10). The analyses were performed using R: an environment for statistical computing and graphics V.3.5.1 (https://www.r-project.org/).

Results

Respondents’ characteristics

The final sample comprised 120 respondents—76 of 87 respondents before and 44 of 87 after the intervention. There was no significant difference in respondent’s characteristics between the two measurement periods. In general, age ranged from 22 to 57 years. The sample included more women than men (94.17% n=113 vs 5.83% n=7). Two-thirds of respondents worked as registered nurses at a general ward (66.67%). The other respondents worked as midwives at the MICU (33.33%). The median experience was 9.5 years (4.2–18.0 years) (Q25–Q75). Details of the respondent’s characteristics, per observation period, are provided in table 1.

Table 1

Respondent’s characteristics (n=120)

Characteristics	Pre-intervention (n=76)	Post-intervention (n=44)
Age,median (Q25–Q75)	37.5 (30.0–46.2)	37.0 (31.0–38.5)
Gender
Female	73 (95.89%)	38 (97.43%)
Male	3 (4.11%)	1 (2.56%)
Experience unit, median (Q25–Q75)	10.0 (5.0–18.0)	8.0 (4.0–14.0)
Unit, n (%)
MICU	26 (34.21)	14 (31.81)
Rehabilitation 1	19 (25)	15 (34.09)
Rehabilitation 2	21 (27.63)	7 (15.91)
Geriatrics	10 (13.15)	8 (18.18)

MICU, maternal intensive care unit.

Validation of the HES

The new questionnaire structure includes four factors. The factor Relevance of information is made up of two items and is the same factor as in the model construct by Bergs et al (2017). The factor Efficiency is made up of two of the three items selected by O’Connell et al (2013): ‘I find handover takes too much time’ and ‘Patient information is provided in a timely fashion’. The factor Interaction and support contained three items and was modified by deleting the item ‘I am educated to different aspects of nursing care’. Finally, the factor Quality of information was modified by removing the item ‘I feel that important information is not always given to me’ and by deleting two items ‘I am able to clarify information that has been provided’ and ‘I am able to keep my mind focused on the information being given to me’. An overview of the two old questionnaire structures and the new one is provided in table 2.

Table 2

Handover Evaluation Scale items’ corresponding factors and summary of model fit indices

Items	Original model (O’Connell et al25)	Model 2(Bergs et al19)	New model
The information that I receive is up to date	Quality of information	Quality of information	Quality of information
I am provided with sufficient information about the patient	Quality of information	Quality of information	Quality of information
I am able to clarify information that has been provided	Quality of information	Quality of information	/
The way in which information is provided to me is easy to follow	Quality of information	Quality of information	Quality of information
I feel that important information is not always given to me	Quality of information	Relevance of information	Relevance of information
I am able to keep my mind focused on the information being given to me	Quality of information	Quality of information	/
I have the opportunity to debrief with other colleagues when I have had a difficult shift	Interaction and support	Interaction and support	Interaction and support
I have the opportunity to discuss workload issues	Interaction and support	Interaction and support	Interaction and support
I have the opportunity to discuss difficult clinical situation I have experienced	Interaction and support	Interaction and support	Interaction and support
I am educated about different aspects of nursing care	Interaction and support	Interaction and support	/
I have the opportunity to ask questions about things I do not understand	Interaction and support	Interaction and support	Interaction and support
I find handover takes too much time	Efficiency	/	Efficiency
I am often given information during handover that is not relevant to patient care	Efficiency	Relevance of information	Relevance of information
Patient information is provided in a timely fashion	Efficiency	Quality of information	Efficiency
Model fit indices
Comparative Fit Index	0.857(inadequate)	0.869(inadequate)	0.940 (acceptable 0.90–0.95)
Tucker-Lewis Index	0.82(inadequate)	0.835(inadequate)	0.914 (acceptable 0.90–0.94)
Root Mean Square Error of Approximation	0.099(poor)	0.095(poor)	0.068(good ≤0.08)

We identified a new questionnaire structure with acceptable or good fit indices (see table 2). Furthermore, the overall internal consistency of the new HES structure was considered adequate (see figure 1). The resulting four factors were used to evaluate the intervention’s effect on the perception of nursing handover.

Figure 1

Final model of the factorial structure of perceptions of handover using the revised version of the Handover Evaluation Scale (HES), with standardised parameter estimates. eff, Efficiency; int, Interaction and support; qlt, Quality of information; rlv, Relevance of information.

Baseline measurement

The four aggregated factor scores were used for further description and analysis. Figure 2 provides boxplots comparing pre-intervention and post-intervention scoring for the included units.

Figure 2

Boxplots of scoring factors comparing pre-intervention and post-intervention scoring for wards as a whole.

Mean scores (M), 95% CI and lower and upper limits (LL–UL), for the HES factors at baseline, follow-up and change from baseline to follow-up are summarised in table 3.

Table 3

Summary statistics for the HES factors at baseline, follow-up and changes from baseline to follow-up across all units

HES factor	All units
	Baseline	Follow-up	Change score
Quality of information	69.39±3.62 (65.77–73.01)	73.26±4.36 (68.89–77.62)	3.87
Interaction and support	70.86±4.41 (66.45–75.26)	72.75±5.43 (67.32–78.18)	1.89
Relevance of information	53.19±4.33 (48.86–57.51)	61.03±6.01 (55.02–67.04)	7.84*
Efficiency	58.92±4.09 (54.82–63.02)	63.8±4.60 (79.20–68.40)	4.88

*Significant difference at p<0.05.

HES, Handover Evaluation Scale.

Statistics for the HES factors at baseline, follow-up and change from baseline to follow-up (mean, 95% CI and lower and upper limits (LL–UL)) in the four units are detailed in online supplemental appendix 2. At baseline, factor scoring for Quality of information was the lowest in MICU (M=63.92±6.32) with a 95% probability for the value to fall between these limits (67.59–70.25), and the best in the rehabilitation wards (M=71.16±6.82 and 75.51±6.97) with 95% CI limits (respectively, 64.34–77.98 and 68.54–82.47). Pre-intervention factor scoring for Interaction and support was perceived the lowest in MICU (M=61.81±8.48) and the best in the two rehabilitation wards (M=81.95±6.06 and M=73.47±8.94). Pre-intervention factor scoring for Relevance of information was the best in one of the two rehabilitation wards (M=59.52±8.31). Last, pre-intervention factor scoring for Efficiency was the best in a rehabilitation ward (M=66.32±7.77) and the lowest in the other one (M=53.01±7.55).

Statistics for every HES item at baseline, follow-up and change from baseline to follow-up across all units are detailed in online supplemental appendix 3. The mean scores are based on the original 7-point Likert scale, while the percentages represent the agreement (score from 5 to 7).

Effect of the intervention on the perception of nursing handover quality

In general, mean changes in HES factor scores ranged from −4.08 to +15.99 (see online supplemental appendix 2). Concerning the perception of units as a whole, no significant difference in factor scoring for Quality of information (M=69.39±3.62 vs M=73.26±4.36; U=1386, p=0.14), Interaction and support (M=70.86±4.41 vs M=72.75±5.43; U=1577, p=0.76), or Efficiency (M=58.92±4.09 vs M=63.8±4.60; U=1370, p=0.098) was found in this study. However, perception of units in factor scoring for Relevance of information showed a significant improvement (M=53.19±4.33 vs M=61.03±6.01; U=1274, p=0.04). This factor involves the items ‘I feel that important information is not always given to me’ (reverse scored) and ‘I am often given information during handover that is not relevant to patient care’ (reverse scored). We noticed also a significant improvement of the factor scoring Efficiency in MICU: M=56.59±7.87 vs M=70.91±8.78 (p<0.05). Mann-Whitney U tests on the individual HES items revealed a significant positive difference in responses following the intervention for the item ‘Patient information is provided in a timely fashion’: M=4.50±0.34 vs M=5.16±0.40 (four units, p=0.01) and M=4.27±0.6 vs M=5.21±0.76 (MICU, p=0.04). See online supplemental appendix 3 for a detailed overview of items’ scoring and change.

Discussion

This study’s objective was to compare the perceived quality of intershift nursing handover before and after a tailored educational intervention. Post-intervention changes in HES factors show both large improvements and declines (from −4.08 to +15.99) (see online supplemental appendix 2). Nevertheless, our results are encouraging as they are to be put in a context where handovers represent a daily routine activity: perception for Relevance of information showed a significant improvement in all included units (M=53.19±4.33 vs M=61.03±6.01; U=1274, p=0.04). This significant improvement was obtained for the factor that shows the weakest baseline measurement. Nurses also stated saving time on handovers after the intervention (M=4.5±0.34 vs M=5.16±0.40, p=0.01). These findings indicate that a structured handover requires less time compared with the pre-intervention methods of handover. We assume that the applied intervention facilitated a focus on transmitting important information at the expense of ‘background noise’ towards the handover process.

The pre-intervention and post-intervention ranges are remarkably large for the four HES factors (from −4.08 to +15.99). The noticed disparity of the results shows that stakeholders had an individual, rather than a collective perception on handovers’ quality. But situational awareness and collective consciousness are the pillars of teamwork, as described in the TeamSTEPPS principles and training. This dimension of non-technical skills needed for patient safety therefore seems to require improvement. This could be achieved using an observational tool validated for measuring the intrinsic quality of SBAR-based handover. This tool would be used to build a common referential to carry out an objective and reflexive improvement process.

Contrary to what Bergs et al19 showed, there is no improvement in the perception of Interaction and support in our sample. In our study, we already started with a positive perception on interaction and support (baseline measurement for Interaction and support factor=70.86±4.41). Moreover, the study by Bergs et al took place in the context of interunit handover, and our study is about intershift handovers within the same unit. Finally, the intervention focused on the pragmatic and operational using of SBAR tool associated with an organisational change and appreciative inquiry approach. Our blended curriculum offers a reflection on the sustainability of new communication habits. However, the participants seem rather have retained the interest in a ‘turnkey ready-to-use communication tool’. However, new behaviours of mutual support within teams may emerge over time. These new behaviours could create teams’ demand for the creation of communication tools such as SBAR. But our study seems to suggest the possibility of a favourable evolution of behaviours based on the deliberate but mandatory use of SBAR. A positive experience of daily use of the tool may prove more effective and sustainable than a reflexive invitation to change behaviours.

In addition to our study, since November 2018, more than 1500 caregivers of CHR Citadelle Liege have completed the blended curriculum. An observational assessment tool measuring the quality of SBAR-based handover is needed and currently being validated within the institution (Content of Handover and Environment Evaluation Form). It will allow the continuous collection of data and the development of quality indicators. More, an internal survey at CHR Citadelle Liege has shown that SBAR is not used outside the pilot units. Beyond the interest of dedicated field coaches, the head nurses’ role appears crucial in the sustainability of the implementation plan. A study is currently underway to identify the head nurses’ needs and level of acceptance in the implementation of SBAR-based handovers at CHR Citadelle Liege.

There is currently no universal and replicable strategy that guarantees significant handover quality improvement.28 The intrinsic quality, the expected contents of handovers must be designed explicitly within each ward, unit, medical specialty, chain and care institution. Implementing the SBAR tool should indeed continue with a reflection of the teams on what can and should be improved. Training alone is not enough to implement change. To change a professional culture, and communication is part of that culture, it is probably necessary to implement a culture of change. Changing communication habits will undoubtedly require the voluntary involvement of stakeholders in a participatory construction process. It, therefore, seems that head nurses, as direct managers, will need to exercise their project management skills. This transition will take time and will only succeed if it continues to be supported by an institutional willingness to promote quality of care and patient safety.

Our study has several limitations. The interpersonal and the social desirability response bias were not considered. The sample was limited, and the results, therefore, cannot be generalised. This pre-test/post-test design focused on the formative stage of a more extended and more complex implementation plan. A delay of 3 months between pre-test and post-test was a compromise. We had to fit our post-test into the practitioner skill curve without delaying the mass training process within the institution. Our study design allowed us to isolate the subjective impact of the blended curriculum on a limited number of SBAR users. We have discussed the need to measure the actual impact of the curriculum on the intrinsic quality of handovers in a large population. Nevertheless, it would be possible to replicate the study on a larger scale. The HES tool only evaluates participants’ perceptions. Negatively worded items may be a source of method bias.29 However, the modifications made allowed the items to aggregate into meaningful factors. Complementary studies are currently underway at CHR Citadelle Liege to validate an assessment tool for handover quality and to facilitate the change of communication habits mediated by the head nurses.

Conclusion

We have shown that a blended curriculum, aimed at contributing SBAR-based handover implementation, can improve nurses’ perceptions of the relevance of information during intershift handovers. After our intervention, the nurses’ perception of the HES item ‘Patient information is provided in a timely manner’ also improved significantly. It appears that the educational intervention is only the first step in the long-term implementation of a culture of professional communication based on mutual support.