Augmented reality glasses improve adherence to evidence-based intubation practice.

Background: The risk of failing or delaying endotracheal intubation in critically ill patients has commonly been associated with inadequate procedure preparation. Clinicians and trainees in simulation courses for tracheal intubation are encouraged to recall the steps of how to intubate in order to mitigate the risk of a failed intubation. The purpose of this study was to assess the effectiveness of using optical head mounted display augmented reality (AR) glasses as an assistance tool to perform intubation simulation procedure.
Methods: A total of 32 subjects with a mean age of 30±7.8, AR (n1=15) vs non-augmented reality(non-AR) (n2=17). The majority were males (n=22, 68.7%). Subjects were randomly assigned into two groups: the AR group and the non-AR group. Both groups reviewed a video on how to intubate following the New England Journal of Medicine (NEJM) intubation guidelines. The AR group had to intubate using the AR glasses head mount display compared to the non-AR where they performed regular intubation.
Results: The AR group took longer median (min, max) time (seconds) to ventilate than the non-AR group (280 (130,740) vs 205 (100,390); η 2 =1.0, p=0.005, respectively). Similarly, there was a higher percent adherence of NEJM intubation checklist (100% in the AR group vs 82.4% in the non-AR group; η2=1.8, p<0.001).
Conclusion: The AR glasses showed promise in assisting different health care professionals on endotracheal intubation simulation. Participants in the AR group took a longer time to ventilate but scored 100% in the developed checklist that followed the NEJM protocol. This finding shows that the AR technology can be used in a simulation setting and requires further study before clinical use.

RCT Entities:   Population Interventions Outcomes

Introduction

The incidence of failed or delayed endotracheal intubation ranges from 1 in 1,000–2,000 cases in the elective setting,1,2 to 1 in 100 in the emergency department (ED).3,4 Acute respiratory failure and airway compromise are life-threatening moments in the critically ill patient. It often necessitates meticulous planning to reduce the rate of complications and maximize successful first-time intubation. While the curriculum for airway intubation differs among training programs, most undergo training through lecture-based presentations, videos, and hands-on simulations. At the same time, trainees and current practitioners are expected to recall steps of intubation to mitigate errors in endotracheal intubations; thus, using a checklist is recommended.5 A recent study by Miller6 supported the notion that simulation-based training was the more preferred training method for endotracheal intubations performed by respiratory therapists.6 Joffe, Liew, Olivar, Dagal, Grabinsky, Hallman, Treggiari7 reported in a national survey of airway management in the United States that most internal medicine and critical care programs have a formal airway management program utilizing simulation-based learning as a method of learning. In their report, almost 70% of the programs also used simulation-based training.7 McGaghie, Issenberg, Barsuk, Wayne8 reported how using simulation in medical education can improve technical skills in addition to interpersonal skills of the learner.8 As a result, simulation-based training is increasing among health care institutions as an effective and safe educational tool for health care providers.9–12 In fact, health care simulation is now considered to be an essential part of translational education research in medical education.13,14

The evolution of technology has enabled health care providers to access data efficiently and swiftly. In the realm of medical technological devices, the utilization of a medical device that can generate and enhance computer imagery is extremely beneficial to the proceduralist, trainee, and the educator. The purpose of this study was to evaluate the possibility and effectiveness of using optical head mounted display augmented reality (AR) glasses during endotracheal intubation simulation procedure. The AR glasses in this study were used to give real-time feedback to the learner while performing the procedure.

Methods

Ethical statement

This study was approved by the Institutional Review Board at Loma Linda University, Loma Linda, CA USA

Participants

A total of 32 subjects participated in this study. Participants were recruited from a health care medical center institution in Southern California. Participants represent a total of three professions (Medicine, Respiratory Care, and Sleep Technology). Subject enrolment was open to interested health care providers to participate as long as they have a minimum of Basic Life Support certification. Sleep technologists were allowed to join as they hold BLS certification and have taken an anatomy course of the airway. All participants provided their written informed consent to participate.

Design

Participants were randomized into two groups: AR group and non- augmented reality group (non-AR). The AR glasses used were AiRScouter WD-200B glasses developed by Brothers (Figure 1). The random allocation sequence was generated using an excel sheet. Both groups answered a questionnaire at baseline. The questionnaire covered demographic characteristics and questions about participant’s perception of the use of AR in intubation. The AR perception questions were ranked using a 4-point Likert scale (1= Strongly Disagree, 2= Disagree, 3= Agree, 4= Strongly Agree). All participants watched an educational video provided by the New England Medical Journal of Medicine (NEJM) on endotracheal intubation.15 AR subjects were given an instructional session on how to use the AR glasses. Participants were then divided into their perspective groups before performing the actual intubation on a manikin. In addition, equipment commonly available during intubations was provided during the simulation session for the participants to use (Figures 2 and 3).

Figure 1

AiRScouter WD-200B by Brothers (http://www.brother-usa.com/airscouter/). Not seen in the photo is the strap that connects to either of the headbands in the back. Label on this figure was added by the authors.17

Figure 2

Simulated view that the participant sees with AR glasses. The image area is approximated and can be adjusted by the viewer as needed. Participants are able to follow video image instructions to properly complete the procedure.

Figure 3

Picture showing how the AR glasses were used in the intubation procedure in terms of their position.

In the AR group, participants were trained on the use of the AR glasses until they were comfortable and familiar with the equipment (Figure 1). After training, research members reset the glasses to the manufacturer’s settings and then the participants were asked to start the procedure when they were ready.

The procedure would start once the participant places the AR glasses on his/her head (with the headband) and followed instructions that were projected on the eyepiece. The learner had a Bluetooth foot pedal that was connected to the glasses where they could control the real-time feedback PowerPoint slides. The presentation displayed on the AR glasses included intubation instructions as well as video graphics demonstrating proper intubation techniques. The PowerPoint slides contained a total of 14 slides all text instructions with three short instructional videos (Figure 2). The provided instructions were an adapted checklist that followed the NEJM intubation protocol.15 The research team developed a 17-point intubation checklist from the NEJM intubation video (Table 1). An observer from the research team timed the procedure from initiation to end while observing if participants adhered to the presented checklist in the eyepiece. Time to endotracheal intubation and ventilation were recorded as well. After the procedure was completed, participants were asked to fill out a validated post-AR experience survey that was developed by Santos et al, on the use of AR glasses.16 Figure 4 shows a diagram of the study design.

Table 1

17-point intubation checklist developed by the research team from the NEJM video

17-point intubation checklist
Pre-Intubation procedure
Pulse Oximeter Available Blood Pressure Available Heart Monitor Available Removes Dentures Has Bag Valve Mask Suction Available CO2 Monitor Available Checks Laryngoscope light Checks ETT Balloon Loads Stylet Bags appropriately
Post-Intubation procedure
Removes stylet Inflates balloon Checks Breath sounds Checks for chest rise Checks CO2 Calorimeter Secures Tube

Abbreviations: CO2, carbon dioxide; ETT, endotracheal tube.

Figure 4

Study design.Abbreviations: NEJM, New England Journal of Medicine; AR, augmented reality.

In the non-AR group, participants were asked to start the intubation procedure after completing the questionnaire and watching the NEJM intubation video. Participants were required to audibly state every step they could recall from the NEJM video while performing the procedure. A research member recorded the procedure time from beginning to end and assessed the participants’ adherence to the checklist that was developed and adapted from the NEJM intubation protocol (Table 1). Time to endotracheal intubation and ventilation were recorded.

Data analyses

A sample size of 32 subjects was estimated using a large effect size of 0.9, power of 0.80, and level of significance of 0.05. Data were summarized using mean ± SD for quantitative variables and frequency (%) for categorical variables. The normality of the quantitative outcomes was assessed using the Kolmogorov–Smirnov Test. The distribution of characteristics of participants by study group was examined using Pearson’s chi-square test. An independent t-test was used to compare the mean age between the two study groups. Mann–Whitney Test was used to compare median time to ventilation and adherence between the two study groups. The level of significance was set at p≤0.05. Thematic analysis was used to analyze participants’ comments regarding the use of AR in the future for the AR group.

Results

A total of 32 subjects participated in this study with a mean age of 30±7.8 (Respiratory, n=19 (59.4%); Medical Doctor, n=11 (34.4%); Sleep Technology, n=2 (6.3%). The majority of the participants were males (n=22, 68.8%). There was no significant difference in baseline characteristics between the two groups (p>0.05, Table 2). The distribution of time to place the AR device on head and time for ventilation, and adherence were not approximately normal. There was a significant difference in median (min, max) time (seconds) to ventilate between AR and non-AR (280 (130,740) vs 205 (100,390); η2=1.0, p=0.005; Figure 5), respectively. Similarly, there was a significant difference in percent adherence to the intubation checklist between the two groups (100% in the AR group vs 82.4% in the non-AR group; η2=1.8, p<0.001). The most common intubation misses in the non-AR group in pre-intubation preparation were the removal of dentures, suction available, pulse oximeter, and blood pressure monitor. In the post-intubation checklist, the most common misses were securing tube, and forgetting to check the CO2 calorimeter.

Table 2

Frequency (%) of baseline characteristics of participants by study group (N=32)

	AR (n1=15)	Non-AR (n2=17)
Experience intubating humans	7 (46.7)	9 (52.9)
Experience intubating manikins	15 (100)	14 (82.4)
ProfessionRTMDSleep Tech	10 (66.7)4 (26.7)1 (6.7)	9 (52.9)7 (41.2)1 (5.9)
Male	9 (60)	13 (76.5)
Age (mean ± SD)	32.4±10.6	27.5±3.1
BLS	15 (100)	16 (94)
ACLS	10 (66.7)	12 (70.6)
PALS	7 (46.6)	3 (17.6)
NRP	6 (40)	3 (17.6)

Abbreviations: AR, augmented reality; RT, respiratory therapist; MD, medical doctor; Sleep Tech, sleep technologist; BLS, basic life support; ACLS, adult cardiac life support; PALS, pediatric advanced cardiac life support; NRP, neonatal resuscitation provider.

Figure 5

Box and Whisker plot of time to ventilation by use of augmented reality glasses.

For baseline questionnaire on the subject’s perception of AR glasses, there was no significant difference in the perception of AR between the two groups (p>0.05), refer to Table 3.

Table 3

Median (min, max) of the baseline perceptiona of AR by study group

	AR (n1=15)	Non-AR (n2=17)
AR glasses can decrease the number of required intubation attempts	3 (2.4)	3 (2.3)
AR glasses can help in medical education	3 (2.4)	3 (1.4)
AR glasses can help in real world clinical situations	3 (2.3)	3 (2.4)
AR glasses does not need that much training	2 (1.3)	2.5(1,3)
AR glasses does not have any side effect such as headache on the user	3 (1.4)	2 (1.4)

Notes: a1= Strongly Disagree, 2= Disagree, 3= Agree, 4= Strongly Agree.

Abbreviation: AR, augmented reality.

The majority of AR participants reported that they disagreed that the device caused fatigue, was bulky, heavy, and required additional work. Participants disagreed that the information that was displayed on the screen was difficult, confusing, displayed flickering, required additional body muscle work, and interfered with procedure skills. In addition, the majority of the AR participants agreed that the amount of information that was displayed was appropriate, fast, clear, easily understandable, and consistent. The application was comfortable for arm and hand, easy to control, and simple to operate (Table 4).

Table 4

Frequency (%) of post-AR perception from the AR group participants, N=13*

	Strongly disagree	Disagree	Agree	Strongly agree
Caused fatigue	3 (23.1)	9 (69.2)	1 (7.7)	0 (0.0)
Too bulkya	2 (16.7)	9 (75.0)	1 (8.3)	0 (0.0)
Too heavy	4 (30.8)	9 (69.2)	0 (0.0)	0 (0.0)
Head, hand, and foot interaction were difficulta	4 (33.3)	5 (41.7)	3 (25)	0 (0.0)
Requires lots of work	4 (30.8)	8 (61.5)	1 (7.7)	0 (0.0)
Requires mental effort	3 (23.1)	9 (69.2)	1 (7.7)	0 (0.0)
Amount of information is appropriate in the screen	0 (0.0)	2 (15.4)	7 (53.8)	4 (30.8)
Displayed information was difficult to read	3 (23.1)	8 (61.5)	1 (7.7)	1 (7.7)
Displayed Information was fast enough	2 (15.4)	1 (7.7)	8 (61.5)	2 (15.4)
Information was confusing	4 (30.8)	9 (69.2)	0 (0.0)	0 (0.0)
Words and symbols on screen were easy	0 (0.0)	1 (7.7)	8 (61.5)	4 (30.8)
Display was flickering too much	7 (53.8)	6 (46.2)	0 (0.0)	0 (0.0)
Information displayed was consistent		1 (7.7)	9 (69.2)	3 (23.1)
Interaction with AR requires lots of body muscle work	6 (46.2)	6 (46.2)	1 (7.7)	0 (0.0)
Application was comfortable for arms and hands	0 (0.0)	1 (7.7)	8 (61.5)	4 (30.8)
Device physically interfered with procedure skill	1 (7.7)	8 (61.5)	4 (30.8)	0 (0.0)
Foot became tired after use	6 (46.2)	7 (53.8)	0 (0.0)	0 (0.0)
Application was easy to control	0 (0.0)	4 (30.8)	6 (46.2)	3 (23.1)
Device was interfering with ability to perform	1 (7.7)	9 (69.2)	3 (23.1)	0 (0.0)
Operation of application is simple	0 (0.0)	3 (23.1)	5 (38.5)	5 (38.5)

Notes: *Two participants did not respond to the post-questionnaire. aOne participant did not respond to this question.

Participants’ comments showed an overarching theme, an effective education tool rather than actual use in the clinical setting. Following is a list of quotes that the participants provided: “The foot pedal was extremely slow to respond and often required several strong taps to change the slide on the PowerPoint. I felt that AR slowed me down as I already knew the next step that needed to be taken in the intubation procedure,” “feel that in certain procedures (like intubation) that are routinely done by some, using the AR is slowing us down, but I still consider it’s a great teaching tool”, “I believe the use of AR is helpful in the new coming learners. I as a person who has already learned how to intubate without the AR and has practiced intubating, it was hard not to get ahead of myself and stop to read the directions. I believe that this tool is and might be helpful for certain people but isn’t a complete replacement.”

Discussion

The objective of this study was to evaluate the effectiveness of using AR glasses as an assistance tool across different health care professions. Our findings indicate that using AR would have a significant positive impact in the simulation setting. Participants in the AR group had better adherence to the checklist compared to the non-AR group. To our knowledge, this is the first study to utilize AR glasses as a real-time feedback teaching and assistance tool during an intubation simulated procedure. The use of AR within the medical field has been evaluated and reported in other areas such as surgery and other bedside procedures.18–21 Most of these reported evidence conclude that there is still a need to further evaluate the technology, and more studies are needed to show significant effectiveness in clinical practice. However, when comparing these findings to medical education, several studies showed real promise and effectiveness for student learner. Ma, Fallavollita, Seelbach, Von Der Heide, Euler, Waschke, Navab22 studied AR as a tool for anatomy education where they developed a “magic mirror” prototype for the learner. Their findings showed future promise on the use of AR technology in terms of anatomy education for medical students.22 Moro, Stromberga, Raikos, Stirling23 on the other hand evaluated the same domain, teaching anatomy for health science students, and evaluated the effectiveness of AR technology vs other technologies such as virtual reality (VR) and Tablet-based education. Positive results were also reported regarding the effectiveness of AR in teaching anatomy to health science learners. In addition, AR use did not show any significant side-effects such as headache and dizziness when compared to VR.23 Jeon, Choi, Kim24 evaluated the use of AR in ultrasound-guided vascular access in a randomized study. They found a significant decrease in time and number of redirections of vascular access to this technology. Wilson, Doswell, Fashola, Debeatham, Darko, Walker, Danner, Matthews, Weaver25 studied the use of AR as clinical support to treat tension pneumothoraces. Their study was similar to ours where they had two groups (Control vs AR) but measured different outcomes (ie, treating an emergency situation, tension pneumothorax in the battlefield using human cadavers). They found that participants in the AR group had a better odds ratio when it comes to their competency in performing needle decompression for tension pneumothorax. These findings are consistent with our study as our AR group resulted in the better adaption of the checklist and less errors (100% vs 82%).

Overall, based on our results, there is promise for the use of AR technology in medical education for health care professionals. The AR group was able to achieve 100% adherence to the NEJM intubation checklist.15 However, time to ventilation for the AR group was significantly longer as this was primarily due to the fact that the AR group had to read and follow real-time instructions projected on the glasses. These findings translate the effectiveness of AR use in an educational setting instead of real-life procedures. Therefore, future studies should evaluate the actual implementation of such technology within the curriculum of health care programs to understand it better. Other examples where AR can be tested and used within several health care professions are: 1) in the field of Respiratory Care (eg, arterial blood gas draws, intubation); 2) in the field of sleep studies (eg, electroencephalography placements). Kamphuis, Barsom, Schijven, Christoph26 shared great potential research questions future researchers should consider when implementing such a tool. An example of a question related to curriculum implementation: “What factors influence the implementation of an AR training system in a curriculum and how does that affect learning?”26

The findings of this study mimic the finding of a previously published study by the authors on the effectiveness of AR as an educational method in the simulation.27 The results of these two studies show how the use of advanced technology such as AR can enhance learning in advanced procedures such as intubation and central line placements.

Our study had some limitations. The factor of user experience and profession was not taken into consideration in group randomization. In the AR group, participants had less human intubation experience when compared to the non-AR group (46.7% vs 60%). This might explain why the AR group took a longer time in the procedure to ventilate. Another possibility of this delay is the amount of information being displayed in the glasses. Thus, a larger prospective study with balanced intubation experience in addition to evaluating the amount of information being displayed is warranted to provide stronger evidence on the use of AR as an endotracheal intubation tool.

Conclusion

Based on our findings, AR showed promise in assisting different health care professionals in endotracheal intubation simulation. Participants in the AR group had a long time to ventilate but also scored 100% in the developed checklist that followed NEJM endotracheal intubation protocol.