Filming for auditing of real-life emergency teams: a systematic review.

Introduction

Delivering high-quality emergency care is the ambition for every emergency team. To succeed requires not only that the individual provider is well trained; it also commands a rapid and coordinated team effort.1 2 However, performance often falls short of expectations.3–5 Therefore, strategies like simulation training, audits, feedback and debriefings have been studied.6–10 Furthermore, filming of emergency teams was introduced back in 1969.11 Filming makes it possible to review and analyse the performance in detail.12–14 Despite the widespread availability and acceptability of video as a method for auditing and quality improvement in healthcare today,15 16 it is still not used by the majority of emergency teams.17

This review describes current evidence for video review to audit emergency teams’management of real-life patients. Video review is defined in this manuscript as any assessment, evaluation or audit where video is used

The key questions in this systematic review are

Where has video review been used; populations and settings?

How has video review been used; technical solutions, legal and ethical issues?

What is the evidence that video review improves patient care?

Methods

This systematic review used the protocol for systematic reviews (Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols).18 The full study protocol in registered with PROSPERO.

Eligibility criteria

The eligibility criteria were based on the PICOS (Population, Intervention, Control, Outcome and Study design) guideline.19

Population was resuscitation teams, code teams, emergency teams, trauma teams, rapid response teams in hospitals.

Intervention was video review.

Control/comparison was non-exposed teams.

Outcome was any assessment of the team’s performance and/or patient outcome.

Study designs eligible for inclusion include randomised controlled trials (RCTs) and non-randomised studies (non-RCTs, interrupted time series, controlled before-and-after studies, cohort studies) and cross-sectional studies. Studies of single case reports and unpublished studies (eg, conference abstract and non-English papers) were excluded. We excluded studies of teams performing any planned activities (elective procedures or operations) and teams performing procedures in out-of-hospital settings or simulated environments.

Literature search

The full search strategy for MEDLINE is visualised in PROSPERO. The search was conducted on 15 March 2018, and this search strategy was adapted to other databases. The databases used were: (1) Ovid MEDLINE (1946 to present). (2) Embase (1974 to present; Ovid). (3) PsycINFO (1806 to present; Ovid). (4) The Cochrane Central Register of Controlled Trials (CENTRAL). (5) Cochrane Database of Systematic Reviews (current issue; part of the Cochrane Library). The literature review was supplemented with studies found by reviewing the references of the included studies.

Study selection and data extraction

LB and our Liberian KRS conducted the literature search. LB checked for duplicates and conducted an initial screening, drawing up a preliminary list of records (n=157). LB and our colleague KJ independently assessed and extracted the data and conducted risk of bias assessment.

Data synthesis and risk of bias

The included studies were characterised by setting, population, outcomes, technical solution, ethical solution and risk of bias (low, unclear or high according to four criteria).20

Selection bias was categorised as low risk if inclusion of participants was clearly described and representative of the population.

Performance bias was categorised as low risk if the majority of healthcare providers were included in the study and as high risk if only a small part of the teams agreed to be filmed.

Measurement bias 1 was categorised as low if a validated tool, checklist or protocol was used.

Measurement bias 2 was categorised as low if several raters independently assessed the video and their agreement was acceptable (eg, measured by kappa or intra class correlation >0.75).

Patient and public involvement

This systematic review was conducted with no patient and public involvement.

Results

Study selection

The literature search identified 7077 papers, the full texts of 157 of which were assessed by two reviewers. A further eight studies were identified by checking the references lists. Among these 165 studies, 50 were eligible according to the inclusion criteria (online supplementary table S1),4 5 8 13 21–66 and five of these evaluated the impact of video review in regards to improvement in patient care (PICOS)8 21 23 41 62 (figure 1).

Figure 1

PRISMA flow diagram. PICOS, Population, Intervention, Control, Outcome and Study design; PRISMA, Preferred Reporting Items for Systematic Review and Meta-Analysis

Study designs

There were 6 cohort or case–control studies,8 21–23 27 42 27 cross-sectional studies24 25 28 30 33–40 43 44 46 47 49 51–54 56–59 65 66 and 17 case reports or series4 5 13 26 27 29 31 32 45 48 50 55 60–64 (figure 2). There were no trials or protocols of RCTs.

Figure 2

The included studies in the systematic review. CRP, Cardiopulmonary resuscitatio; ROSC, return of spontaneous circulation.

Population and setting

We identified five categories of team: cardiac arrest team (n=7),41–47 neonatal resuscitation team (n=18),48–65 trauma team (n=15),4 5 8 13 21–31 paediatric trauma team (n=9)32–40 and obstetric emergency team (n=1).66 The 50 studies were conducted at 30 different hospitals situated in the USA (n=12), Australia (n=1), Europe (n=11) and Asia (n=5) (figure 2).

Technical solution: how to capture the team’s management

The technical solution was described in 43 of the studies. The procedure of filming was continuous video recording (n=6),4 28 31 43 56 58 manual activation where you have to ‘press the button’ (n=20)5 8 13 23 24 26 27 29 32 33 44–47 50 53–55 57 64 or recording automatically activated either by motion triggers (n=3)48 59 60 or Bluetooth in the team leader’s telephone which activated the camera when the leader entered the room (n=1).66

In 20 studies, the cameras were placed either as a bird’s eye view or at a high angle. In bird’s eye view, the camera is placed directly above the patient; for example, the camera could be mounted on the radiant heater above the neonatal resuscitation table.48 50 53 57–60 62 63 In high angle view, the cameras were mounted in the ceiling in the corner presenting an oblique view.32 33 41 45–47 66 Cameras with low angle view were used in studies prioritising patient anonymity.8 13 26 27 An example of a low angle view could be a camera mounted on the resuscitation bed either at the foot of the bed or behind the patient’s head.

Only a few studies described recording sound. Where described, microphones were either integrated into the camera or as separate microphones placed in the ceiling.4 31 57 None of the included studies reported staff wearing microphones, which has been described in other video reviews, for example, elective operations.67

To inform the video review of the patients’ condition, data such as vital signs and echocardiography, the monitor was recorded in five studies,31 42 49 51 65 and in one of those, several views had been recorded simultaneously, viz three camera views of the team’s management and one of the patient’s monitor.

Informed consent

Consent was collected in 14 studies. Consent was collected either from all who participated in the video (n=4)26 40 53 66 or in part from either staff or patients (n=10).48–51 54–58 65 In the majority of the studies, informed consent from the patient could not be obtained before video recording due to the patient’s situation, for example, cardiac arrest or severe trauma. In these studies, the hospital approved the project ethically and legally and waived informed consent on the grounds that video recording was considered a quality assurance measure.

Outcomes

To audit the care provided in the video review, four categories of outcomes were identified: teams’ clinical performance (n=34), teams’ technical performance (n=13), teams’ non-technical performance (n=17) and patient outcome (n=14) (figure 2). Teams’ clinical performance consisted of the overall management of the emergency and was assessed in terms of adherence to protocol or guideline.4 5 8 21–28 30 32–45 47 54 55 57 58 62 63 66 Technical performance was assessed for procedures like intubation or chest compression depth.13 46 48 49 51–53 56 59–61 64 65 Non-technical performance comprised decision-making, situation awareness, communication, leadership, teamwork or vigilance and was assessed in terms of use of checklist or descriptive analysis.24 25 27 28 30 31 35–37 39 40 43 47 53 57 58 63 Patient outcomes comprised return of spontaneous circulation (ROSC), length of hospital stay or survival and were assessed based on either the video or medical charts (n=14).4 21 22 26 28 41–43 48 49 51 65

Evidence that video review improves patient care

Five studies evaluated the effect of video review on the provided care8 21 23 41 62; trauma teams (n=3),8 21 23 cardiac arrest teams (n=1)41 and neonatal resuscitation teams (n=1)62 (table 1).

Table 1

Studies evaluating the effect of video review on provided care (PICOS)

Study	Comparator group	Video review	Teamoutcome	Patient outcome	Results	Risk of bias
						A	B	C	D
Hoyt(1988)8	n=60(verbal feedback)	n=180	ATLS*		Team: Improved in timely delivery of care (ATLS) for trauma patients ISS†>20 (44/15%, p≤0.001)	?	+	+	–
Townsend(1993)21	n=361(video no feedback)	n=522	ATLS*	Survival	Team: Improved significantly in timely delivery of care (mean 97.5/88.6 min, p<0.01)Patient: Mortality reduced (10.8% to 10.4%), p<0.01, By TRISS‡ No difference between groups by ISS†	?	?	+	?
Scherer(2002)23	n=27(verbal feedback)	n=24	ATLS*		Team: Improved significantly in 8/10 of theATLS checklist.	–	?	+	–
Carbine(2000)62	n=25(first of consecutive)	n=25(last of100 consecutive teams)	Neonatal Resuscitation Protocol*		Neonatal resuscitation: No overall improvement in resuscitation score. One action ‘deep suctioning’ improved significantly (p=0.002)	?	?	+	+
Jiang(2010)41	n=15(first of consecutive)	n=15(last of 45 consecutive teams)	ALS*	ROSC survival	Team: In 3/8 resuscitation tasks, the team improved significantly (p<0.001)Survival: No improvement (survival to discharge n=2 in control group and n=1 in video group)	+	+	+	+

(A) Selection bias was categorised as low risk if inclusion of participants was clearly described and representative of the population.

(B) Performance bias was categorised as low risk if the majority of healthcare providers were included in the study and as high risk if only a small part of the team agreed to be filmed.

(C) Measurement bias 1 was categorised as low if a validated tool, checklist or protocol was used.

(D) Measurement bias 2 was categorised as low if several raters independently assessed the video and their agreement was acceptable (eg, measured by kappa or intra class correlation>0.75).

Information about quality items (risk of bias A-D) results in: + Adequate; ? Unclear; - Inadequate.

*Adherence to a protocol or clinical guideline, for example, ATLS (Advanced Trauma Life Support), ALS (Advanced Life Support for cardiac arrest).

†ISS: (Injury Severity Score) is an anatomical scoring system that provides an overall score for patients with multiple injuries.

‡TRISS: The probability of survival (Ps) of a patient from the ISS and RTS using a formula 68.

PICOS, Population, Intervention, Control, Outcome and Study design; ROSC, return of spontaneous circulation.

Intervention by video review

The teams’ management of real-life patients was filmed, and the teams received feedback at an educational conference where they reviewed their performance.

Controls

The control groups in the five studies were not blinded as both the intervention group and the control group were aware of the filming. Three studies evaluated the effect ‘before and after’21 41 62 and two studies compared verbal feedback with no video to a video-feedback conference.8 23

Quality of studies

Four studies had high or unclear risk of bias8 21 23 62 and one study of cardiac arrest teams41 had low risk of bias (table 1).

Meta-analysis

Meta-analysis was waived due to the heterogeneity of the studies and the lack of consistently applied reference standards.

Level of evidence for video review

All five studies were in favour of video review as an educational intervention to improve patient care. Four of the five studies found that video review significantly improved teams’ clinical performance, which was assessed either in terms of improved guideline adherence or less time spent on providing patient care8 21 23 41 (table 1). Two studies evaluated whether the improved clinical performance improved survival rates.21 41 The first study by Townsend et al21 found a significant reduction of mortality from 10.8% to 10.4%, p<0.01, and controlled for the confounder of the Trauma and injury severity score (TRISS) score (probability of survival based on Injury Severity Score68). However, the second study,41 evaluating whether improved clinical performance resulted in better patient outcome for 45 cardiac arrest cases, reported no improvement in survival or ROSC, as only two patients survived until discharge in the control group and one patient survived until discharge in the video group. Two studies evaluated the educational impact of video review where teams reviewed their own performance compared with verbal feedback.8 23 Both found that video review outperformed verbal feedback alone in improving clinical performance (figure 1).

Recommendation

Educational intervention by reviewing video of actual emergencies may improve teams’ clinical performance. The Oxford strength of this recommendation is B.69

Discussion

This systematic review of 50 observational studies provides insight into the use of video review of resuscitation teams, trauma teams, emergency paediatric teams and emergency obstetric teams. This technical solution is affordable and relatively easy to instal; however, legal and ethical issues may be challenging. Investigating the impact of video review, with regards to improved patient care, all five studies were in favour of the use of video review as an educational intervention. Four of the five studies found that video review significantly improved teams’ clinical performance, and one study found improved survival of trauma patients.

After systematically reviewing the included studies, the question remains whether we may rely on video review to improve team performance in the future. In an attempt to answer this question, we discuss the strengths, weaknesses, opportunities and threats (SWOT analysis) of video review below.

Strength

Easy to install cameras

Cameras and microphones are easy to instal and this solution is affordable compared with other medicotechnical equipment.

Data collection

Once data are collected, the data can be reviewed repeatedly.5 58 70

Improve teams’ performance

As the need for high-value, cost-conscious medical education is more prevalent than ever, video may be a learning tool to expedite mastery of necessary techniques.71 Video can capture teams’ behaviour, timing of medicine, flow of an algorithm and time taken to master techniques.23 72 73 Video review also allows teams to evaluate local algorithms for patient care. Furthermore, one can conduct research by linking these processes to patient outcomes.8 21 23 41 62

Video in feedback and debriefing

The present literature of video review to improve patient care favours video review over other modalities.8 21–23 41 62

Quality assessment

Video review can be used for quality assessment and benchmarking of performance between departments or countries.

Documentation

Video provides more detailed and accurate information than paper-based records,53 56 and videos can be saved as a supplement to patient records.

Weaknesses

Blinding

When we use video, we are limited by the ‘eye’ of the camera. Thus, we can easily be blind to missing information, for example, information that is obvious to the team but not to the video reviewer.74

Forgetting to turn on the cameras

This is a well-known problem.5 8 21 Therefore, continuous video recording is recommended; however, if this is not feasible, we recommend considering how cameras can be activated automatically by either motion triggers48 59 60 or Bluetooth.66

Hindsight bias

Knowing the outcome affects how we perceive the team’s performance.75 Therefore, teams reviewing their own performance are biassed to overestimating both the importance of error in cases of poor outcomes and to overestimate the importance of their management when the outcome is good.

How to use the video as an educational tool

We have limited information regarding how to use the video as an educational tool, for example, is the video review facilitated by the team itself or a trained facilitator and used for structured feedback or debriefing? Debriefing in simulation-based education suggests that the effectiveness of facilitator-led debriefings seems to the same whether performed with video or without use of videos.76 77 However, in simulation, the facilitator observes the team’s management, whereas in real-life emergencies the facilitator is not there; in those cases, video may serve as an eye witness.

Difficult to benchmark performance

Standardisation of appropriate outcomes is needed before we can benchmark outcomes between studies, and this would also open the possibility for meta-analysis.78 Guideline adherence is the most frequently used outcome; however, this should be standardised by use of systematically developed checklists for evaluation of teams’ performance.66 79 Time can be a relevant outcome; however, outcomes should always be selected based on relevance for the clinicians and the patients and not because they are easy to assess.78

Time-consuming and costly

To date, the majority of selected outcomes are assessed manually by observers reviewing the video using a checklist or protocol, and this is tedious, time-consuming and costly.74 80

Opportunities

Work environment

Research into teams’ working environment may identify factors that complement teams’ management but also factors that prevent teams from becoming effective. Such research may be the first step towards innovative solutions to improve environments, for example, the arrangement of the room, how to diagnose or to deliver the right dose of medicine. In the future, such development may improve the quality of care as it simply becomes easier to deliver the right care.81–83

Automatic assessment

Manual analysis of video is time-consuming and expensive as several raters are usually needed. Therefore, development of innovative solutions is needed to reduce the cost and increase the objectivity and the quality of video analyses. Innovative solutions could combine analysis of the team and other objective measures, for example, technical skills like chest compression depth measured by the monitor device.29 55 65

Faster education

By reviewing not only our own performance but also that of our colleagues, video review could have the potential to reduce the time to mastery.8 41 63

Threats

Legal issues

Some projects may be stopped before they ever get started because it can be difficult to get legal permission for recording where informed consent cannot be collected, for example, in cardiac arrest cases.

Patient compliance

Overall, patients are reportedly positive regarding the use of video to improve care. In an Australian survey, 96% of parents agreed to the use of video in neonatal resuscitation to improve patient safety.84 In a Danish study, patients and their relatives gave consent in 94% of the cases of major postpartum haemorrhage.85

Staff compliance

Studies find that staff in general are positive and find that video review improves their knowledge and the provided care.8 However, in a survey on trauma teams, 30% of the staff found video to provoke moderate anxiety; still, 90% of the staff agreed on the educational value of using video.86 Although the benefits of video seem to outweigh the potential liability risk, there will always be a concern for how videos can be used in malpractice trial, even if this risk is minimal17 87 and knowing that video is more likely to provide evidence of good care.88–90

Video versus audio recording

The gold standard is video and audio recording. However, if this is not possible, audio recording is an alternative; and audio has been used in analysis of telephone conversations.91

Ethical issues

Video recording of patients without informed consent raises the ethical question whether the public benefit of video recording outweighs the patient’s right to privacy.88 There is no unambiguous answer to this, and several studies video record without consent.24 39 89 Reviewing the timeline of the included studies, the number of studies using video review has dropped dramatically since 2003.26 In 2003, a new act, The Health Insurance Portability and Accountability Act (HIPAA), was passed in the USA, and this affected the use of video in trauma centres there as informed consent was now needed before filming. Hence, a survey from 200590 found that the use of video review in trauma centres dropped from 58% to 18% after the HIPAA became law, and the most cited reason for discontinuation of video review was the need for informed consent, legal concerns and concerns about patient privacy. A survey from 201017 reported that the use of video had not changed since 2005 as only 20% of 108 trauma centres used video review in the USA, although 100% of all trauma centres agreed that it can improve the trauma resuscitation process.

Building on findings from this review, future research may address two main aspects:

The teams’ conditions, for example, development of new innovative solutions for the emergency room, monitors providing information and guiding the team, reduction of noise, improved lighting of the room, premixed syringes with correct dose medicine, new tool as surgical instruments, checklists or other cognitive adds.

Video as an educational tool; as the educational strategies of real-life video are lacking, future research should therefore focus on describing all characteristics involved in how to use video in debriefing to maximise educational efficiency.

Conclusion

The need for high-value, cost-conscious medical education is more prevalent than ever, with shrinking time for training and increasing complexity of the care delivered by the emergency teams. Filming emergencies has educational value and is an important priority for clinical research seeking to identify factors that complement teams’ management and factors that prevent teams from becoming effective. However, the ethical and legal concerns remain unresolved. If we can solve these ethical and legal concerns, video review can provide us with the opportunity to analyse, understand and improve our performance and the quality of patient care.