Managing deteriorating patients: registered nurses' performance in a simulated setting.

AIM: To examine, in a simulated environment, rural nurses' ability to assess and manage patient deterioration using measures of knowledge, situation awareness and skill performance.
BACKGROUND: Nurses' ability to manage deterioration and 'failure to rescue' are of significant concern with questions over knowledge and clinical skills. Simulated emergencies may help to identify and develop core skills.
METHODS: An exploratory quantitative performance review. Thirty five nurses from a single ward completed a knowledge questionnaire and two video recorded simulated scenarios in a rural hospital setting. Patient actors simulated deteriorating patients with an Acute Myocardial Infarction (AMI) and Chronic Obstructive Pulmonary Disease (COPD) as the primary diagnosis. How aware individuals were of the situation (levels of situation awareness) were measured at the end of each scenario.
RESULTS: KNOWLEDGE OF DETERIORATION MANAGEMENT VARIED CONSIDERABLY (RANGE: 27%-91%) with a mean score of 67%. Average situation awareness scores and skill scores across the two scenarios (AMI and COPD) were low (50%) with many important observations and actions missed. Participants did identify that 'patients' were deteriorating but as each patient deteriorated staff performance declined with a reduction in all observational records and actions. In many cases, performance decrements appeared to be related to high anxiety levels. Participants tended to focus on single signs and symptoms and failed to use a systematic approach to patient assessment.
CONCLUSION: Knowledge and skills were generally low in this rural hospital sample with notable performance decrements as patients acutely declined. Educational models that incorporate high fidelity simulation and feedback techniques are likely to have a significant positive impact on performance.

INTRODUCTION

In recent years the acuity of patients managed on general wards has increased whilst access to critical care beds has decreased [1] with subsequent increases in demands on ward staff. Recent evidence from the ‘failure to rescue’ literature indicates a high level of disturbed physiological variables (e.g. heart/respiratory rate and blood pressure) in the general ward population [2] and poorer patient outcomes due to mismanaged patient deterioration [1, 3-7]. Moreover, there is evidence that up to 80% of in-hospital cardiac arrests are predictable [4]. Missed indicators of deterioration have also been noted in patients following discharge from hospital [8] and in smaller rural hospitals [6]; whilst further evidence has indicated that nurses are not always clear about when to call for assistance [8, 9], do not seek advice and fail to appreciate clinical urgency [10]. Small regional hospitals receive high acuity patients less frequently than their metropolitan counterparts with the risk of skill decline and skill mix concerns, whilst an aging population increases the likelihood of co-morbidities. These issues and the remoteness of many rural hospitals emphasize the need for good clinical assessment skills to identify deterioration earlier. Medical Emergency Teams (MET) may have improved the treatment of patient deterioration [11] but observations are often taken but not acted upon. The frequency of MET team calls has increased substantially [12], with the view that nurses are losing what limited ability they had to identify and manage patient deterioration.

It is known that the management of deteriorating patients can be influenced by education and past experience [13-15]. In health care contemporary education approaches emphasise the need for active learning [16] and increased use of simulated environments [17-19] to reduce medical errors [20] in settings which have high equipment, environmental and psychological fidelity (‘believability’) [21]. Our experience, and recent work suggests that patient actors are able to mirror the true contextual environment and have a significant impact on learning [22-26]. Major benefits of using simulation in conjunction with clinical placements include the opportunity to rehearse skills in a safe environment without compromising patient safety [18] and for the controlled, objective measurement of clinical performance. Unique to this and our previous studies [27-31] we have measured levels of ‘situation awareness’ [32, 33] with the objective of understanding health professional focus of intention and ultimately enhancing patient safety. Originally developed from aircraft industry work on cockpit crew resource management, and later developed in anaesthesia, Situation Awareness (the perception of environmental elements) is measured on three levels (perception, understanding and prediction); and can be mapped and measured in a simulated environment using previously validated software known as Situation Awareness Global Assessment Technique (SAGAT) [34].

Best practice evidence-based models of education that are known to impact on clinical practice and the management of patient deterioration are unusual [35]. Educational models such as FIRST2ACT (Feedback Incorporating Review and Simulation Techniques to Act on Clinical Trends) [30] are likely to be a good framework for the development of education through incorporation of didactic teaching, formative assessment (to drive learning), high fidelity simulation and structured feedback techniques. Assessing patient deterioration management performance will enable us to understand performance decrement issues and build a program of education that has an impact on clinical outcomes. With these issues in mind our aim was to: Examine registered nurses’ ability to manage patient deterioration in a simulated environment with measures of knowledge, situation awareness and skill performance.

The following research questions were addressed:

How do registered nurses manage patients who are deteriorating, or at risk of deterioration?

How is knowledge applied in a simulated learning environment? For example, what is the relationship between pathophysiology knowledge, and recognition and management of physiological changes?

How situationally aware are nurses, and is this related to their knowledge and skill performance?

MATERIALS AND METHODOLOGY

Participants

This exploratory quantitative performance review was set in an Australian rural hospital in the state of Victoria. All registered nurses (Division 1) from the mixed medical/surgical ward (n=41) were invited to participate in the study. Enrolled nurses and locum nurses were excluded. Thirty-five nurses participated (85%) attending a 1.5 hour individual session in a training facility designed to match the ward setting. Each participant completed a demographic survey, a ‘pre-test’ multiple choice questionnaire, two 8 minute simulation exercises and video-based, facilitated reflective review, followed by expert feedback. There was limited data from which to base a sample size calculation however based on the hypothesis that there will be a change in individual performance between scenarios guidance suggests that 30-60 participants are sufficient for each situation awareness query [34].

Ethics

Ethics approval was obtained from the Monash University Standing Committee on Ethical Research in Humans. Participants were required to give informed written consent and no incentives such as payment were offered for participation. In recruiting participants, the study was summarised openly and honestly by the research team. Additional applicable training was offered to participants and non-participants on completion of the study. The supportive role of the research team and the debriefing and feedback components of the study were emphasised.

Instruments

A 10-item demographic questionnaire identified age, sex, nursing grade/qualifications, clinical speciality, recent training and relevant work experience. An 11 item multiple choice questionnaire with four response options identified participants’ knowledge of patient deterioration with items relating to observational parameters (n=8), equipment use (n=2) and terminology (n=1). This previously validated question set had good face and content validity and was developed from peer reviewed questions [36] and piloted in a previous study with student nurses [27].

Scenarios and Skill Performance Assessment (Objective Structured Clinical Examination - OSCE)

In a training facility designed to mirror the ward setting participants complete two 8-minute simulation exercises that were video recorded (to enable participant feedback) and incorporated patients with an acute myocardial infarction (AMI) and chronic obstructive pulmonary disease (COPD). AMI and COPD were selected as they were the most common presenting condition to the ward. Three trained and fully briefed professional patient actresses (standardised patients) were employed to simulate clinical scenarios. To mimic reality, the conduct of the simulations met three conditions: (i). relatively little information was provided initially, (ii) the clinicians were allowed to investigate freely and (iii) participants were given clinical information over time in accord with process-based information- giving [37]. This approach was adopted to improve the ecological validity of the simulation [17] enabling the participant to experience clinical thinking in a dynamic manner.

In both scenarios subtle deterioration cues were present in the first 4 minutes, prior to more obvious and significant signs of deterioration during the final 4 minutes. Two factors were taken into account in order to achieve applicable learning outcomes. These were: the level of relevant information provided and the predictability of perceived relationships between variables (e.g. tachycardia and low blood pressure indicating reduced cardiac output [13]. These dimensions were incorporated into the AMI scenario - high level of relevant information and low level of uncertainty (i.e. the ‘easier scenario’); which was followed by the COPD scenario - low level of relevant information and high level of uncertainty (i.e. the more difficult scenario).

On arrival, participants were given a brief description of the patient’s presenting condition prior to each scenario. One of the researchers played the role of a newly qualified doctor, remaining in the ward to ‘prescribe’ medication/s on request, or to support participants with unfamiliar equipment. A standardised check list (OSCE) was used to rate the number of correct observations/actions for each scenario. Primary skill ratings were made by the researcher playing the doctor’s role and by a second non-participant observer at the end of each scenario (to ensure inter-rater reliability).

Situation Awareness

At the conclusion of each scenario the participant was taken from the simulation room and asked 12 ‘yes/no’ questions to assess their overall awareness. The questions were developed by a panel of experts using Goal Directed Task Analysis to identify goals, decisions and situation awareness requirements [34]. The ‘scene’ was altered between each scenario, for example suction was available in the first scenario but not in the second. Immediate ‘gut reaction’ responses were encouraged from participants with questions for each scenario covering three domains:

Physiological perception [3 items] e.g. ‘what is the heart rate at the moment?’, and ‘what is the respiratory rate’; and global perception [3 items] e.g. ‘Is suction available?’, and ‘what is attached to the foot of the bed?’

Comprehension [2 items] e.g. ‘what is wrong with the patient?’

(iii) Projection [4 items] e.g. ‘what medications may be required?’, and ‘what investigations may be required?’

Data Analysis

Survey data was entered into a database of PASW statistics. The normality of the multiple choice test, skill performance and situation awareness score data was confirmed, enabling parametric inferential analyses. The relationship between key demographic variables and the multiple choice questionnaire, OSCE and situation awareness scores were investigated using applicable statistics e.g. Pearson product-moment correlation coefficients for scale variables; Spearman’s rank-order correlation for non-normally distributed variables (e.g. age). Repeated-measures t-tests were used for differences between related variables (e.g. skill performance over two scenarios) and independent group t-tests for dichotomous demographic variables and mean assessment scores. All tests for statistical significance were two tailed, with p ≥ 0.05 regarded as significant.

RESULTS

Demographic Characteristics

Thirty-five nurses participated from a staff population of 41 (85.4% sample). One nurse withdrew after the first scenario due to high levels of anxiety [in the results below we therefore list outcomes from the first scenario (n=35) and the second (n=34)]. The majority were female (33; 94.3%) with an average age of 41 years (range: 22-60, SD=10.6) and employed for an average of 27 hours per week (range 2-40).

All nurses were Division 1 registered nurses with Bachelor level qualifications. Three-quarters (74.3%) had been qualified for more than 3 years (range 0-33 years; mean 13.57: SD=10.27). Fifteen nurses (42.9%) had postgraduate certificate or diploma level qualifications; including five in midwifery, one in cardiac care and two in critical or intensive care. One nurse had a Masters in Nursing.

Participants had worked in a variety of clinical settings during their career including (in decreasing frequency) surgical, medical, aged care, emergency, theatre, midwifery, rehabilitation, mental health, community and intensive care. Half had worked in one of these posts for more than 5 years (range: 0-33) with an average duration of 8.6 years (SD=7.75).

Fifty one percent of participants (18) had completed a basic or advanced life support course within the last two years. In response to a question about the number of seriously ill patients they had managed in the last year - 7 participants had no experience, 16 had managed 1-3; five had managed 4-6; and three had managed more than 12 patients (median= 2; mean 4.26 SD 8.9; range 0-48).

Knowledge: Multiple Choice Questionnaire

The average total score for the multiple choice questionnaire was 66.5% with a wide range of results (n=35, range 27-91%, SD=12.9). Most participants responded correctly to the first three questions relating to circulation, with a range of results for the remaining questions (Table ).

Skills Performance (OSCE)

Skills performance for the two scenarios was calculated from the standardised rating forms. Participants in the cardiac scenario scored an average of 52.1% (n=35, range 36-72%, SD=9.3), and 48.6% for the respiratory scenario (n=34, range 26.1-73.9%, SD=11.7). Performance was consistent between the scenarios i.e. there was no significant difference between scores in the first and second scenarios (t =0.662, p =0.513 (CI: -3.23 (1.56) 6.35). Total scores for both scenarios (48 items) averaged 50.4% ranging from 26.1-73.9%, (n=34, SD=10.50) raising significant concerns over performance as half of the applicable actions were not performed.

Cardiac Scenario

In the first four minutes of the cardiac scenario the average number of correct observations/actions was 45.5% (n=35, range=0.0-91.4%). Of the eight subtle cues, the most frequently performed were recording/requesting the blood pressure (BP) and obtaining a rhythm strip (91.4%) (Table ). The least frequent observations were respiratory rate (11.2%) and temperature (34.3%) with no nurse recording the capillary refill time (CRT).

Performance in the second half of the scenario was wide ranging (mean 58.7%; range 0.0- 94.3%). Nitrates were frequently requested and oxygen administered (94.3%) but again the respiratory rate was rarely recorded (5.9%) and the CRT was never performed. There was a decline in all of the actions that should have been taken twice with a statistically significant decrease in the proportion of participants who checked pain score and oxygen saturation, despite the more obvious cues in the second half of the scenario.

Respiratory Scenario

In the first four minutes of the respiratory scenario the average number of correct observations/actions was 52.5% (n=35, range=0.0-97.1%) (Table ). Of the twelve subtle cues, the most frequently performed were recording/request-ing observations (97.1%) and administering oxygen (94.1%). No nurse recording the capillary refill time (CRT) or the peak flow rate.

Performance in the second half of the scenario was wide ranging (mean 44.6%; range 2.9- 91.2%). Patients were positioned correctly (91.2%), but antibiotics (15.2%), non invasive ventilation (11.8%) and CRT (2.9%) were rarely requested/performed. There was a decline in the majority of actions that should have been taken twice with a statistically significant decrease in the proportion of participants who recorded the BP, heart, and respiratory rates despite the more obvious cues in the second half of the scenario.

Situational Awareness

Ratings of situation awareness (perception, comprehension, projection) were performed at the end of each scenario with an average score of 12/24 (50.0%) (n=33, range 6-20).

Cardiac Scenario - Situation Awareness

Participants scored on average 37.5% for ‘perception’ of the environment with poor recall of important physiological parameters such as heart rate or respiratory rate. Overall ‘comprehension’ of the situation was moderate (mean 55.7%) but with good understanding of the diagnosis (74.3%). Anticipation (‘projection’) of potential decline and required treatment was high (82.1%).

Respiratory Scenario - Situation Awareness

Again participants had low scores (average 37.5%) for ‘perception’ of the environment with poor recall of important physiological parameters such as heart rate and BP. Overall ‘comprehension’ of the situation was moderate (mean 60.3%) but with good anticipation (‘projection’) of the future situation (72.1%).

Relationships Between Key Variables

Knowledge (multiple choice questionnaire) was significantly higher for participants (n=18) who had completed ALS or BLS training or an emergency paediatrics course in the past two years (r= 0.692; p=0.004). However this correlation did not translate into higher performance in the scenarios or situation awareness scores.

DISCUSSION

In this research we aimed to identify how registered nurses from a rural Australian hospital manage deteriorating patients, how knowledge is applied, and to measure levels of situation awareness. The 35 participants were typical of the rural nursing population in that the majority were experienced female nurses who worked part time, but with infrequent experience of managing the acutely ill patient. Knowledge scores were acceptable averaging 67% and were found to be significantly higher for those who had recently completed life support training, however they did not perform any better in the scenarios (OSCEs). Of note is that registered nurse participants’ knowledge scores were found to be significantly lower than those of ‘point of completion’ third year student nurses who were assessed in a parallel study using the same multiple choice questionnaire [27] (t = -2.84; df 82; p=0.006, CI: -13.04 (-7.67) -2.30) [27]. This indicates a failure to acquire and maintain knowledge at a contemporary level for registered practice.

Skill performance was notably lower than knowledge averaging 50% (range 26-74%). In other words half the performance requirements were not completed, especially the respiratory rate and the CRT. In addition there was a significant decline in performance as the patient deteriorated despite the more obvious deterioration cues. From our observations this appeared to be related to increasing levels of participant anxiety - leading to performance decline as the patients’ condition worsened. Situation awareness outcomes were consistent across both scenarios with participants demonstrating a poor perception of the situation, including low recall of important physiological parameters (this was often because they had not been recorded in the first instance leading to a lack of base line data and difficulty in judging the speed and extent of deterioration). Moderate levels of understanding of the situation (comprehension) were apparent and anticipation (projection) of future events was generally good.

This study was set in a single hospital in rural Victoria with a sample of Caucasian nurses and therefore a lack of cultural and international diversity. The study sample was small although for situation awareness measures a sample of 30-60 participants are considered sufficient for each situation awareness query in a repeated measures design [34]. For a correlation design, analysis of our previous data sets suggests that in examining the relationship between skills and knowledge, a sample of 100 would be sufficient to set a 5% significance level (avoiding Type I error) and be powered sufficiently to avoid Type II error. The response rate was high (85%), however we do not know how non-participants perform. It may be that they were more confident in their ability and therefore did not feel the need to attend the training, or perhaps - more likely, were less confident and competent and therefore fearful of attending.

From the literature it is quite clear that resuscitation practices are inconsistent and that patient deterioration management can be improved [1-3, 27-29]. Our findings support this view [27-31]. In our previous work with student nurses and midwives [27, 29] the core issue was a failure to apply knowledge in stressful emergency situations. This knowledge-practice gap is apparent in different scenarios with varying levels of fidelity (medium to high) and in samples with quite different levels of experience. In other words performance cannot be predicted based on years of general experience and educational level. The infrequency of acute events in such settings ensures that clinical skills are not developed emphasising the need for applicable training, for example ‘foresight’ training [38] and FIRST2ACT [30] (see below). Staged and repetitive training in this area is likely to up-skill and maintain skills whilst bridging the knowledge-practice divide.

Workloads, inexperience and a lack of skill have been noted as important limitations to nurses’ ability to detect deteriorating patients [39], with a need to better appreciate clinical urgency [10]. Rather than utilizing more objective assessments, such as changes in physiological parameters, nurses have been found to use pattern recognition and/or intuition to identify changes in clinical course [39]. Such strategies, especially where the nurse is inexperienced, may not be effective and can lead to errors in judgment. For example, in our work with student nurses [28] trends of deterioration were missed and systematic assessment was lacking. Undergraduate and professional development education should therefore aim to enhance the ability to link information, pathophysiology and patient assessment, and identifying trends.

Although educational trends are changing undergraduate and professional development courses remain largely dependent on didactic educational methods that fail to incorporate practice in realistic settings. International trends away from hospital based training to the higher education sector has reduced clinical practice time, and there is increasing competition for existing clinical placements [40, 41]. Alternative methods, therefore, have to be considered to enhance clinical skills. Clinical skills scenarios - repetitive high stakes and high fidelity - ideally in the clinical setting, are a good solution to this problem [42]. Development of situation awareness is also core and should be considered in light of aircraft industry, military experience, anaesthetics [43] and resuscitation teams [44]. In military aircraft circles it is argued that working hard leads to target fixation: a ‘red mist’ obscuring lateral thinking and a loss of situation awareness [45]. In other words there is an overemphasis on a target with ignorance of the threats around you. In medical terms, this relates to fixation on one sign or symptom with a drowning in one’s “ability reservoir” [45: p 127]. Or put another way an overriding focus on a single issue, with all one’s reserves focussed in that particular area, leading to failures in holistic assessment.

Finally, from our work in this area the research team have been able to develop an educational model that is likely to enhance practice - FIRST2ACT (Feedback Incorporating Review and Simulation Techniques 2 Act on Clinical Trends) [30]. In line with the research approach described in this paper a number of educational stages are advocated including formative assessment, simulation, self review and expert feedback. High fidelity simulation is the key, noting that high psychological fidelity (believability) is essential without necessarily the need for high technology.

CONCLUSION

The international literature supports the notion that deteriorating patients could be managed more effectively. Medical Emergency Teams have made a difference to outcomes but there is evidence to suggest that those in the front line of patient care lack the skills to identify deterioration trends and to act appropriately. The solution to this is an integrated educational skills model that incorporates assessment, high fidelity simulation and reflective feedback techniques in addition to supervised clinical practice.

Table 1.

Group Totals for Clinical Knowledge Responses: Multiple Choice Questionnaire (n=35)

Item Question Focus	Grouped Number of Correct Answers (N/%)
Hypovolaemic shock	33(94.3)
Hypoxia	35(100)
Capillary refill	31(88.6)
Pulse palpation	7 (20.0)
Heart rate	14(40.0)
Pulse oximeters	33(94.3)
Respiration	24(68.6)
Cannulation	24(68.6)
Cardiac output	22(62.9)
Neurological response	19(54.3)
Non-rebreath masks	13 (37.1)

Table 2.

Cardiac Scenario - Proportion of Participants who Performed Correctly in Early and Late Phases of Each Scenario (n=35)

Observation or Action	Subtle Cues (1-4 Mins)		Obvious Cues (4-7.5 Mins)		Paired Sample- Repeated Measurest-Test
	Parameter	% Correct	Parameter	% Correct	t	df	Significance (2-Tailed); 95% CI†
Observation
Pain assessment	5/10	73.5	9/10	38.2	3.55	32	0.001 (CI: 0.14-0.55)
Nitrates	N/A	-		94.3
BP	150/95	91.4	170/95	74.3	1.79	34	NS: 0.08 (CI: -0.24-0.37)
HR	110	57.1	140	42.9	1.22	34	NS: 0.23 (CI: -0.09-0.38)
RR	20	11.4	32	5.9	0.81	33	NS: 0.42 (CI: -0.09-0.21)
CRT	2 secs	0.0	2 secs	0.0	0.00	34	NS: 1.00
Oxygen saturation	95%	65.7	89%	29.4	3.42	34	0.002 (CI: 0.16-0.61)
Temperature	36.8	34.3	N/A	-
Early Action
Obtain immediate history		80.0
Obtain/review rhythm strip/12 lead		91.4
Investigate current medication usage		37.1
Identify other symptoms		8.6
Consider non-cardiac causes of chest pain		17.1
Give Aspirin (sublingual)		23.5
Late Action
Call for urgent assistance				91.4
Position appropriately				67.6
Administer oxygen				94.3
Ensure /IV cannulation				82.9
Morphine				82.9
Sub-total cues		45.5%		58.7%
Overall: combined cues				52.1%

NS: non-significant change between early and late performance.

Table 3.

Respiratory Scenario- Proportion of Participants who Performed Correctly in Early and Late Phases of Each Scenario (n= 34)

Observation or Action	Subtle Cues (1-4 Mins)		Obvious Cues (4-7.5 Mins)		Paired Sample-Repeated Measures t-Test
	Parameter	% Correct	Parameter	% Correct	t	df	Significance (2-Tailed); 95% CI
Observation
BP	135/95	97.1	170/95	52.9	4.59	33	0.000 (CI: 0.25-0.64)
HR	100	72.7	140	41.2	3.22	32	0.003 (CI: 0.12-0.54)
RR	28	50.0	32	14.7	4.24	34	0.000 (CI: 0.18-0.52)
CRT	2 secs	0.0	2 secs	2.9	-1.00	32	NS: 0.33(CI:-0.09-0.03)
Oxygen saturation	90%	82.4	89%	70.6	1.28	33	NS: 0.21 (CI: -0.07-0.31)
Temperature	38.8	47.1	N/A	-
Peak Flow (320L/Min)	400 L/Min	0.0	N/A	-
Early Action
Obtain immediate history		76.5
Investigate current medication usage		21.2
Identify symptoms/negative features		21.2
Auscultate chest		67.6
Administer oxygen		94.1
Late Action
Call for urgent assistance				63.6
Position appropriately				91.2
Request IV cannulation				61.8
Request nebuliser				58.8
Consider antibiotics				15.2
Consider non-invasive vent				11.8
Sub-total cues		52.5%		44.6%
Overall: combined cues				48.6%

NS: non-significant change between early and late performance.