Impact of aerosol box on intubation during COVID-19: a simulation study of normal and difficult airways.

PURPOSE: Patients with coronavirus disease (COVID-19) are at risk of requiring mechanical ventilation, and concerns of protecting healthcare workers during aerosol-generating medical procedures has led to the design of the aerosol box.
METHODS: We conducted a randomized crossover mannequin-based simulation study to compare airway management with and without the aerosol box. Thirty-five anesthesiology participants and three critical care participants with more than 50 intubations with videolaryngoscopes were recruited. There were four airway simulations with and without the aerosol box (normal, pharyngeal swelling, cervical spine rigidity, and tongue edema). Each participant intubated the mannequin in eight consecutive simulations. The primary outcome of the study was time to intubation. Secondary outcomes included intubation attempts, optimization maneuvers, and personal protective equipment breaches.
RESULTS: Mean (standard deviation [SD]) time to intubation overall with the box was 30.9 (23.0) sec, while the time to intubation without the box was 25.1 (12.2) sec (mean difference, 5.8; 95% confidence interval [CI], -2.9 to 14.5). For the normal airway scenario, the mean (SD) time to intubation was 18.6 (3.5) sec for no box and 20.4 (3.3) sec for box (mean difference, 1.8; 95% CI, 0.2 to 3.4). During difficult airway scenarios only, the time to intubation was 34.4 (25.6) sec with the aerosol box and 27.3 (13.2) sec without the aerosol box (mean difference, 7.1; 95% CI, -2.5 to 16.7). There were more intubation attempts, personal protective equipment breaches, and optimization maneuvers during use of the aerosol box.
CONCLUSIONS: In this mannequin-based simulation study, the use of the aerosol box increased the time to intubation in some contexts but not others. Further studies in a clinical setting should be conducted to make appropriate modifications to the aerosol box to fully elicit its efficacy and safety prior to implementation in airway guidelines for managing patients with COVID-19.

The coronavirus disease (COVID-19) pandemic was first officially declared a pandemic by the World Health Organization on 11 March 2020.1 According to an article from Meng et al., as of March 2020, of the 80,000 patients in Wuhan, 3.2% required invasive ventilation.2 The risks posed to healthcare workers contracting respiratory viral illnesses during aerosol-generating medical procedures (AGMP) has been well documented in the literature.3

Worldwide shortages of personal protective equipment have been clearly established in the literature and media4: “the lack of proper masks, gowns, and eye gear is imperiling the ability of medical workers to fight coronavirus and putting their own lives at risk”.5 As such, healthcare workers have investigated alternative ways to protect themselves while caring for patients with COVID-19.

Initially designed by Dr. Lai Hsien Yung, the aerosol box has gained popularity on social media for its resourcefulness.6 The aerosol box is designed to be placed over a patient’s head during AGMPs such as intubation, with two holes for the proceduralist’s arms to navigate airway management.6

Although the aerosol box has been trialed informally in various healthcare institutions, the efficacy and safety of these boxes have not been well shown. Furthermore, the opinions of healthcare experts have been quite polarizing, with calls for robust testing of novel devices before widespread institutional implementation.7 As such, we designed a randomized crossover mannequin-based study to explore the impact of the aerosol box on intubation in both normal and difficult airways. We hypothesized that the aerosol box has a negative impact on intubation management.

Methods

The research protocol received ethics approval from the Health Research Ethics Board at the University of Alberta (REB#Pro00100356) as of April 2020, and written informed consent was obtained from all participants.

Study design

The study design was a randomized crossover mannequin-based study utilizing several simulated scenarios with and without an aerosol box.

Theoretically, the design of the study was a non-inferiority trial.8 The null hypothesis of a non-inferiority study is that there is a difference in time to intubation with and without the box. If the null hypothesis is rejected, it would indicate non-inferiority. Nevertheless, at the time of study implementation, there was no available empirical evaluation of the use of the aerosol box. Without prior evidence, it was not possible to calculate a delta value to set a margin for non-inferiority. As it was not appropriate to make an ad hoc determination of a potential value for delta, statistically the present study is treated as a superiority trial with no assumptions about directionality.9

Study protocol

The study was conducted at our local simulation centre on the University of Alberta campus. Our aerosol box is 45.7 cm tall, 61.0 cm long, and 35.6 cm deep. The holes used by the operator are 20.0 cm in diameter, with the silicone rubber reducing the size to 10.0 cm. There are also two side holes that are 10.0 cm in diameter, which allows cables to pass into the box. The box is open-ended towards the mannequin’s feet. There are slits on the side to allow cables to pass into the box, but no specific holes for the airway assistant to maneuver in. The mannequin used for our study is the SimMan 3G mannequin (Laerdal Medical Canada Ltd, Toronto, ON, Canada), which has pre-set functions to alter the difficulty of the airway, along with simulated capnography monitoring (Fig. 1).

Fig. 1

Image of our simulation setup, including the design of our aerosol box.

Each participant was allowed three practice intubations on the mannequin without the aerosol box to familiarize with the mannequin’s normal airway anatomy. The participants could adjust the height of the stretcher and position the mannequin. The participants donned airborne and droplet personal protective equipment (PPE) in adherence to local institutional guidelines, which included a face shield, N95 masks, sterile gown, and gloves. Expired N95 masks were used to conserve PPE. All intubations were conducted using a GlideScope® (Verathon, Inc., Bothell, WA, USA) with a hyper-angulated disposable size 3 blade, and a rigid GlideRite® stylet. Each participant intubated four different airway scenarios with and without the aerosol box; these scenarios were normal airway, pharyngeal obstruction, cervical spine rigidity, and tongue edema. The participant’s order of scenarios was randomized using the online software, Research Randomizer,10 and the participants were blinded to the order to minimize learner bias (Fig. 2).

Fig. 2

Our study design is a randomized crossover trial, where each participant was asked to intubate the mannequin in eight different simulations consecutively. The order of the simulations was randomized.

An anesthesiology resident, who acted as an airway assistant, was blinded to the order of scenarios. The tasks of the airway assistant were standardized, which included removal of the stylet, inflation of the tracheal tube cuff, and attachment of the self-inflating bag to ventilate the mannequin. The airway assistant also provided optimization maneuvers if requested; these maneuvers included application of the backwards and upwards pressure (BURP) maneuver, re-positioning or removal of the aerosol box, and providing a bougie or a pillow for re-positioning.

Each participant was video recorded and reviewed to confirm time to intubation, number of optimization maneuvers, number of intubation attempts, and breaches of PPE. All analysis was carried out in R software, version 3.6.1.11 Analysis was conducted using the rstatix,12 psych,13 and WRS214 packages.

Objectives

Our primary outcome was the impact of the aerosol box on intubation time, defined as the time the video laryngoscope blade passed the teeth until confirmation of tracheal intubation via positive end-tidal CO2 (ETCO2) waveform using the mannequin’s simulated capnography monitor upon initiation of manual ventilation.

Secondary outcomes included intubation attempts per scenario, number of optimization maneuvers required, failed intubation attempts, and PPE breaches. We defined intubation attempts as the number of times the video laryngoscope and/or tracheal tube was withdrawn and re-inserted past the teeth. Optimization maneuvers included re-positioning of the mannequin, requesting for BURP, use of a bougie stylet, changing the position of the aerosol box, or removing the aerosol box. We also investigated the number of PPE breaches during each attempt, which was defined as exposure of the skin due to movement of participant’s gloves, and adjustment or removal of the aerosol box during the procedure. A survey was provided to the participants after the simulation.

Study population

Sample size was calculated for a two-way (box vs no box) repeated measures design with four levels (airway scenarios) using G-power version 3.1.9.4.15 As there was no prior evidence for the aerosol box, conservative estimates were set for power (0.95) and correlation between repeated measures (0.5). An alpha level of 0.05 was chosen. An effect size (0.25) was chosen based on lower estimates of effect sizes for adverse events during tracheal intubation using video and direct laryngoscopes.16 The necessary sample size was determined to be 36.

We recruited 38 participants for the study. Inclusion criteria included being either a resident physician or staff physician in anesthesiology, critical care, or emergency medicine, with at least 50 intubations using a videolaryngoscope. Participants were recruited through departmental emails, personal connections, and snowball sampling. We asked participants about their number of GlideScope® intubations prior to their participation.

Statistical analysis

Primary outcome assessment

Trials with and without the aerosol box across the four airway scenarios were analyzed using repeated measures analysis of variance (RM-ANOVA). The data were checked for outliers, influential cases, and missing data. The data were also examined for assumptions of normality and sphericity to ensure appropriateness for RM-ANOVA. Data points were considered for removal if they were extreme cases as identified by the boxplot method, > 3 times the third interquartile range, and had a Mahalanobis distance > 12.17

Secondary outcomes assessment

The effect of experience was investigated through two methods. First, correlation between years of experience and time for intubation under each condition was examined. Second, group differences between residents and staff physicians were explored. Because of sample size limitations for the subgroups, unequal cell sizes, and non-normality, residents and staff physicians were compared using the non-parametric Mann-Whitney test, and within group comparisons were made using the Wilcoxon signed rank test. Frequencies and descriptive statistics were examined for optimization maneuvers, number of intubation attempts, PPE breaches, and results from the post-simulation survey. The survey included questions on the difficulty of box use, likelihood of box use, and demographics.

Data preprocessing

Video data were missing for three participants because of unexpected technical problems; therefore, only the time to intubation recorded during the trials was used for these participants. Two raters scored all videos independently. Interrater reliability was calculated for each variable. The raters scored participants nearly identically. The intraclass correlation coefficient (ICC) for time was 0.99. The ICC for number of optimization maneuvers and intubation attempts was one, and the kappa for PPE breaches was one.

Results

Demographics

One participant’s data were removed as the mannequin malfunctioned during their trials, leaving a total of 37 participants used for the analysis. Two participants did not complete the demographics survey. The final sample consisted of 34 participants from anesthesiology and three participants from critical care. There were 18 residents and 19 staff physicians. All the residents were from anesthesiology. The residents had a mean (standard deviation [SD]) of 3.5 (1.3) years of experience, while staff physicians had 12.2 (12.0) years of experience in practice. Our participants had an mean (SD) age of 36.8 (8.8) yr. A total of 296 intubations were performed.

Based on violations of the assumptions of normality and sphericity, as well as the presence of outliers, robust methods for ANOVA were used. For comparative purposes, traditional methods were also used.16,18 Results from the robust RM-ANOVA indicated there was significant difference for use of the aerosol box: P = 0.03, = 0.03, 95% confidence interval (CI), 0.00 to 0.23. Note that for (generalized eta squared), the effect sizes are 0.02 = small, 0.13 = medium, and 0.26 = large.19

The mean (SD) time to intubation with the box was 30.9 (23.0) sec, while the time to intubation without the box was 25.1 (12.2) sec (mean difference, 5.8; 95% confidence interval [CI], −2.9 to 14.5). The significant difference between trials with and without the aerosol box indicated non-inferiority is not supported. In a normal airway scenario, the time to intubation was 20.4 (3.3) sec, while time to intubation without the box was 18.6 (3.5) sec (mean difference, 1.8 sec; Cohen’s d = 0.53; 95% CI, 0.25 to 0.85). In the difficult airway only scenarios, the time to intubation was 34.4 (25.6) sec with the box and 27.3 (13.2) sec without the box (mean difference, 7.1 sec; Cohen’s d = 0.34; 95% CI, −0.06 to 0.75). Cohen’s d represents a standardized mean difference. An effect size of 0.2 is small, 0.5 is medium and, 0.8 is large (Table 1).

Table 1

Primary outcome data for simulated trials with and without the aerosol box across four different airway conditions

		No box	Box	Difference in means (95% CI)	Effect size* (95% CI)
Normal	Mean (SD)	18.6 (3.5)	20.4 (3.3)	1.8 (0.2 to 3.4)	0.53 (0.25 to 0.85)
	Median (IQR [range])	18 (16.4-20.0 [14.0-32.5])	20 (18.5-22.0 [16-29])
C-Spine	Mean (SD)	21.9 (3.6)	28.6 (9.8)	6.7 (3.2 to 10.2)	0.88 (0.36 to 1.4)
	Median (IQR [range])	21 (19.4-24.3 [15-30.5])	27.5 (21.0-32.5 [16-55])
Tongue edema	Mean (SD)	24.4 (8.1)	25.3 (7.8)	0.9 (−2.8 to 4.6)	0.11 (−0.3 to 0.52)
	Median (IQR [range])	22.8 (20-25.1 [15.5-53.5])	25.0 (20-26.5 [18.5-59])
Pharyngeal	Mean (SD)	35.7 (18.6)	49.1 (38.8)	13.4 (−0.9 to 27.7)	0.43 (0.01 to 0.89)
	Median (IQR [range])	29.3 (23.3-39.5 [16.5-93.5])	31.0 (23.5-51.5 [19-144])
Overall	Mean (SD)	25.1 (12.2)	30.9 (23.0)	5.8 (−2.9 to 14.5)	0.03† (0.00 to 0.23)
	Median (IQR [range])	21.5 (19-25.6 [14-93.5])	24.3 (20-30 [16-144])

*Cohens d, 0.2 = small, 0.5 = medium, 0.8 = large (Cohen, 1988)

†Generalized eta squared, 0.02 = small, 0.13 = medium, 0.26 = large

Values reported are all in seconds.

CI = confidence interval; IQR = interquartile range; SD = standard deviation.

There was a significant difference in intubation time based on airway scenarios as well (P < 0.001; = 0.23; 95% CI, 0.10 to 0.38). Robust pairwise comparisons using an M-estimator and a bootstrapped sample indicated a significant difference between all airway scenarios except for the cervical spine rigidity and tongue edema conditions. The normal airway scenario required the shortest time to intubation, while the pharyngeal obstruction scenario required the longest time to intubation.

There was no interaction when the box was present across the scenarios (P = 0.14; = 0.02; 95% CI, 0.00 to 0.11). The time to intubation for the airway scenarios relative to the other scenarios does not change based on the presence of the box; this suggests that difficulty uniformly increases with or without the box across each scenario.

The traditional RM-ANOVA, applied with the Greenhouse-Geiser correction (ɛ = 0.38), did not show any substantive differences from the robust methods (Table 2).

Table 2

Pairwise comparison conducted for differences between each airway scenario to understand the main effect of the airway scenario

	Robust method	Traditional method
	P value	P value¥	Mean difference (s)	Effect size* (95% CI)
Normal and C-spine	0.001	<0.001	6.4	0.89 (0.38 to 1.4)
Normal and pharyngeal	<0.001	<0.001	23.6	1.05 (0.50 to 1.6)
Normal and tongue	0.001	<0.001	5.1	0.96 (0.40 to 1.4)
Pharyngeal and C-spine	<0.001	<0.001	17.2	0.75 (0.29 to 1.29)
Pharyngeal and tongue	<0.001	<0.001	18.5	0.77 (0.27 to 1.26)
C-spine and tongue	0.54	0.80	1.3	0.03 (−0.42 to.49)

¥Bonferroni correction

*Cohens d, 0.2 = small, 0.5 = medium, 0.8 = large (Cohen, 1988).

Comparisons using both robust ANOVA methods and traditional methods are shown. There were no substantial differences in results between the two methods.

ANOVA = analysis of variance; CI = confidence interval.

When using the aerosol box, there were six failed intubation attempts on the pharyngeal obstruction scenario, and two failed intubation attempts on the tongue edema scenario. Comparatively, there were no failed attempts when not using the box for these two scenarios. Lastly, there were no failed intubation attempts for the normal and cervical spine rigidity scenarios. Time to intubation with and without the aerosol box was correlated at r = 0.3, P < 0.001.

Secondary analysis

No significant correlation was found between years of experience and any of the intubation scenarios. No difference in time to intubation was found between residents and staff physicians (P = 0.33; Cohen’s d = 0.02; 95% CI, −0.22 to 0.26). Sub-analysis showed that residents and staff physicians performed similarly with the box (P = 0.27; Cohen’s d = 0.06; 95% CI, −0.27 to 0.40), and without the box (P = 0.78; Cohen’s d = 0.06; 95% CI, −0.40 to 0.30). Residents performed slower with the box (P < 0.001; Cohen’s d = 0.4; 95% CI, 0.02 to 0.72), as did staff physicians, P = 0.004; Cohen’s d = 0.3; 95% CI, −0.02 to 0.60) (Table 3).

Table 3

Comparison of descriptive statistics between resident physicians and staff physicians

Comparison	Group	Mean (SD)	Median (IQR [range])
Overall time	Residents	29 (20)	24 (20-29 [14-144])
	Staff	28 (18)	22 (19-29 [14-131])
Box	Residents	32 (25)	25 (21-30 [17-144])
	Staff	31 (22)	24 (20-31 [16-131])
No Box	Residents	25 (12)	22 (14-94 [19-25])
	Staff	26 (13)	21 (19-26 [14-8])

Comparison of descriptive statistics for overall time, and aerosol box vs no aerosol box across airway conditions. Data units are in seconds.

IQR = interquartile range.

With the aerosol box, 18 trials required multiple intubation attempts. Without the aerosol box, ten trials required multiple intubation attempts. Under the normal airway scenario, all participants completed intubation on the first attempt. Under the pharyngeal obstruction scenario without the aerosol box, seven participants required a total of 15 attempts. With the aerosol box, eight participants required a total of 32 attempts. In the cervical spine rigidity scenario without the box, one participant required three attempts. In the same scenario with the box, seven participants required 15 total attempts. Lastly, in the tongue edema condition without the aerosol box, two participants made five attempts; with the box, three participants made 13 attempts.

In terms of optimization maneuvers, participants used more maneuvers in the difficult airway scenarios and when using the aerosol box. Nevertheless, the overall use was dominated by a few individuals (Table 4).

Table 4

Optimization maneuvers in the four different airway scenarios

Scenario	Box		No box		Overall
	Unique maneuvers	Total maneuvers	Unique maneuvers	Total maneuvers	Unique maneuvers	Total maneuvers
Normal	-	-	-	-		-
C-spine	4	6	2	2	6	8
Tongue	3	12	2	3	5	15
Pharyngeal	10	21	7	9	17	30
Overall	17	39	11	14

Unique maneuvers are defined as the number of participants who conducted optimization maneuvers. For example, in C-spine with the aerosol box, four participants conducted optimization maneuvers in their simulations, with the total number of maneuvers being 6.

Personal protective equipment breaches only occurred in scenarios with the aerosol box. Wrist exposure was seen seven times, lifting and adjusting the box was seen three times, and complete removal of the box was seen once, giving a total of 11 breaches.

Finally, in terms of survey results, 25 (71%) participants indicated they would not use the box in predicted difficult airway scenarios, while 10 (29%) participants would use it. For predicted normal airways, 12 (34%) participants indicated they would not use the box, while 23 (66%) indicated they would. Participants rated the difficulty of using the box on a 1-10 Likert scale with ten being the most difficult. The mean (SD) was 3.7 (1.6). There was no significant difference in rating of difficulty between residents and staff (P = 0.59; Cohen’s d = 0.2; 95% CI, −0.48 to 0.85). A Chi square test indicated staff were less likely to use the box in the predicted normal airway scenario than residents were (P = 0.02; Cramer’s V = 0.4; 95% CI, 0.09 to 0.6). There was no significant difference in likelihood of use between residents and staff for a predicted difficult airway (P = 0.75; Cramer’s V = 0.07; 95% CI, −0.22 to 0.21). Cramer’s V is a measure of strength of association; a value of 0 indicates no association while a value of 1 indicates perfect association.

Discussion

The results of our study show that the aerosol box increased the time to intubation by six seconds, and when only comparing difficult airway scenarios, the use of the aerosol box increased time to intubation by seven seconds. Although a time difference of six seconds may be small, patients with COVID-19 requiring mechanical ventilation already have drastically impaired lung function and are prone to rapid desaturation and hypoxemia.20 Furthermore, despite a time difference of only seven seconds when using the aerosol box in a difficult airway, what was more concerning was the increase in optimization maneuvers, intubation attempts, and failed intubations. These factors may further contribute to delay in achieving intubation. We recognize that outcomes were more prominent in a few individuals; however, heterogeneity in clinical skill level will always be present in real clinical situations. Also, it is important to note that airway management involves not just intubation, but also patient preoxygenation, supraglottic airway use, oropharyngeal suctioning, extubation, and surgical airway management if needed. Using the box may elevate AGMP risks associated with these other components.

Difficult airway scenarios presented a greater challenge, and increased time to intubation regardless of the presence of the box; however, there was also increased variance in times across conditions with the aerosol box present. Based on the greater variability in time to intubation, there may be less certainty about intubation success for a patient with a difficult airway, and subsequently a higher risk of harm compared with patients with a normal airway. These concerns were also echoed in several correspondence letters by Gould et al.,21 which mentioned that patients with COVID-19 have a higher risk of laryngeal edema based on anecdotal experience.

Based on the secondary analysis, the rate of PPE breaches was relatively low compared with the results from a similar study conducted by Begley et al.22 Nevertheless, we found that all the PPE breaches occurred when using the aerosol box, which was also found in the aforementioned study.

From our survey results, most participants did not find the box difficult to use, with an average of 3.7 on the Likert scale. Nevertheless, many participants commented that the rigid cut outs of the box decreased arm maneuverability, which made it difficult to optimize patient position while intubating. Participants also reported that the height of the aerosol box impaired removal of the intubating stylet and use of a bougie. Participants did not find the aerosol box particularly challenging to use, but most indicated they would not use it in a predicted difficult airway scenario.

Lastly, it does not appear that years of experience or status as a resident or staff physician influenced the time for intubation whether the aerosol box was present or not. This may be because once physicians reach a certain level of technical proficiency, greater experience does not make a difference in the scenarios examined in the study. This should be qualified by the limited analytic ability of the study design since experience was not a main variable of interest.

Our findings expand on those of Begley et al.,22 who recently published a similar study using an aerosol box. First, the main strength of our study was assessing the aerosol box across four different airway scenarios, which better simulates real clinical situations. Second, our study had a sample size of 37 individuals performing 296 intubations. This allowed us to make statistical conclusions for both primary and secondary outcomes. Our study also allows for an initial determination of the specific effect of the box on technical aspects of intubation, absent of other non-technical factors of airway management.

We identified three limitations of our study. First, the repeated measures design allowed for a possible learning effect to occur. Despite randomization, some participants mentioned that it only took them one or two trials with the box to get used to it. Future investigations may use a fully independent factorial design, although this would require a substantially larger sample size. Second, our original study design also intended to recruit emergency and critical care physicians. Although we recruited three critical care physicians, it was not large enough to make any conclusions about differences between specialties. Furthermore, we were unable to recruit any emergency physicians, therefore limiting the generalizability of our data. Lastly, our study was conducted with our box design and may not apply to future iterations of the box. Modifications in the maneuverability of the box, as well as airway assistant access, could impact the ease of use.

Advancements in the aerosol box design are already underway. Several published studies, including that by Begley et al., have used different designs of the box.22 Others have proposed using plastic drapes suspended over a patient as a protective method.23 With future designs that address current limitations, the aerosol box may play a bigger role in the COVID-19 pandemic.