Procedural training models among emergency medicine residency programs.

OBJECTIVE: Optimal training methods remain controversial for rarely performed emergency procedures. Previous research has failed to demonstrate the superiority or inferiority of live anesthetized animal models (LAA) as compared to other modalities. Most of the data on LAA use comes from military contexts; less information is available for civilian emergency medicine (EM) training. We sought to characterize the prevalence of LAA use among civilian EM residency programs and reasons for its use or discontinuation.
METHODS: Survey study of program directors of EM residency programs accredited by the Accreditation Council for Graduate Medical Education. A 16-item questionnaire was electronically delivered to program directors, including program region, current and historical use of LAA, and attitudes regarding the optimal procedural training modalities.
RESULTS: Of 179 survey recipients, 83 completed the survey (46.4%). Twelve programs (14.3%) currently use LAA, and 17 programs (20.5%) report previous LAA use. Reasons for discontinuing LAA use included ethical concerns, financial and logistical limitations, political pressures, and feeling that there were superior or equivalent alternative models available. Programs that currently use LAA were more likely to rank LAA as being the most preferable training modality while programs that do not currently use LAA were more likely to rank human cadavers as the most preferable modality.
CONCLUSION: Despite a lack of data showing educational outcomes-driven differences between LAA and alternative training models, LAA use is declining among civilian EM residencies. Despite this, disagreement exists among programs that do and do not use LAA regarding the most optimal procedural training.

INTRODUCTION

Training in the management of traumatic injuries is a continuously evolving process. Live anesthetized animal models (LAA), sometimes referred to as ‘live tissue training,’ have been used historically but have been replaced in both military and civilian settings by increasingly sophisticated high fidelity synthetic mannequins [1,2]. There is debate over which method results in superior training, with proponents of each side citing historical precedent, ethical concerns, and expert opinion to support their views [3,4]. A systematic review of the available literature failed to find high-quality evidence in favor of either LAA or synthetic mannequins as training models [5].

Despite the lack of outcomes-based data favoring one teaching modality over the other, the use of LAA in procedural training has been gradually phased out and replaced by other simulation modalities. The United States military was directed to transition away from LAA use in 2013 by congressional mandate [6], which mirrors worldwide trends in military medical training [1].

Owing in part to the changing nature of the topic, little information is available on the prevalence of LAA in civilian education. Most of the information that does exist on LAA procedural training focuses on the use of LAA in the military or in surgical training. There is little data specific to civilian emergency medicine (EM) training. Additionally, there is a paucity of information regarding the optimal training modalities for specific procedures relevant to the practice of EM [2]. Our goal was to quantify the number of EM residency programs utilizing LAA in procedural training. We sought to assess educators’ perspectives on optimal training modalities for specific procedures. Finally, we sought to elicit educators’ reasons for using or not using LAA as a training model.

METHODS

Study design

This was a survey study of program directors (PDs) of EM residency programs accredited by the Accreditation Council for Graduate Medical Education (ACGME). The survey was conducted over 2 months between December 1, 2018 and January 31, 2019. Our survey tool was developed using an iterative process in keeping with published best practices in survey design [7,8]. We conducted a literature review to identify relevant variables in the use of various models for procedural training. After developing survey items in keeping with the terminology and data present in the relevant literature, we assessed for content validity of the survey items using local content expert review. Content experts included academic EM faculty with experience in survey design methodology and core EM educational leaders who had experience in procedural teaching. Experts reviewed the wording of each item for clarity, content, and utility, and their comments were integrated into the survey. After assessment for content validity the survey was administered to a group of assistant program directors to assess for response process validity using immediate retrospective probing [7]. This group was chosen based on their similarity to the intended study population (namely PDs of EM residency programs), their experience with procedural teaching, and their familiarity with their own programs’ educational practices. Their impressions of each item were recorded and integrated into the final version of the survey [9]. After finalizing survey items, the survey was electronically delivered to the study population. This study was determined to be exempt from review by our institutional review board (2018D-000744).

Study setting and population

The study population included PDs of ACGME-accredited EM residency programs. Our study population includes both three and four year residency programs at training hospitals across the United States. This population was chosen based on their familiarity with the training modalities utilized by their own programs, as well as their familiarity with various means of procedural training. In an effort to minimize curricular heterogeneity in our study population, only ACGME-accredited programs were included, and exclusively osteopathic programs were excluded from our study population.

Study protocol

Study data were collected and managed using Research Electronic Data Capture (REDCap; Vanderbilt University, Nashville, TN, USA) [10]. All survey responses were anonymous. Recipients were sent individualized links to the survey that automatically de-identified their answers and generated a unique survey response identifier.

Measurements

We sought to describe the total number of EM programs using LAA for procedural training and the frequency with which they use these models. We also sought to characterize reasons for discontinuing the use of LAA for procedural training and PDs’ opinions on optimal training models for specific low-frequency procedures.

Data analysis

Response rates were calculated using the calculator tool provided by the American Association for Public Opinion Research [11]. We used the Fisher exact test to evaluate the difference between programs that do and do not utilize LAA and variables such as region of residency, length of residency program training, and educational background. We used the Fisher exact test since our sample size was small and more than 20% of cells had expected frequencies lower than 5 [12]. The ranking for each training modality in each procedure was reported as 4 being most favorable and 1 least favorable. We used the Mann-Whitney test to evaluate the mean procedure-specific ranking of various training modalities, while comparing respondent sites that do and do not utilize LAA. All statistical analyses were performed using R ver. 3.6.1 (R Foundation for Statistical Computing, Vienna, Austria). The figures were produced using the package ggplot2 [13].

RESULTS

Out of 179 programs surveyed, 83 participated from various regions of the United States (46.4%). The seven regions, as defined according to those used by the Society for Academic Emergency Medicine, include New England, Mid-Atlantic, New York State, Southeastern, Great Plans, Midwest and Western United States (Table 1). Of the responding programs, 61 are 3-year programs (73.5%) and 22 are 4-year programs (26.5%). In the United States, approximately three quarters of EM residencies are three-year training programs and the remaining quarter are four-year programs. Within these 83 participating programs, 51 programs’ trainees (61.4%) include both doctors of medicine (MD) and doctors of osteopathic medicine (DO), while 32 programs (38.6%) include exclusively MD trainees (Table 1). Our contact list did not include any programs with only DO trainees.

Table 1.

Characteristics of respondent programs that do and do not utilize live anesthetized animal models for procedural training

Question	Utilize live animal models (n = 12)	Do not utilize live animal models (n = 71)	P-value
Region[a)]			0.60
New England	3 (25.0)	7 (9.9)
Mid-Atlantic	2 (16.7)	10 (14.1)
New York State	0 (0.0)	8 (11.3)
Southeastern	2 (16.7)	15 (21.1)
Great Plains	2 (16.7)	4 (5.6)
Midwest	1 (8.3)	9 (12.7)
Western	2 (16.7)	16 (22.5)
Length of residency program training (yr)			0.72
3	8 (66.7)	52 (73.2)
4	4 (33.3)	19 (26.8)
Educational background			0.19
MD	7 (58.3)	25 (35.2)
DO	0 (0.0)	0 (0.0)
Both	5 (41.7)	46 (64.8)

Values are presented as number (%).

MD, doctors of medicine; DO, doctors of osteopathic medicine.

Regions were defined according to those used by the Society for Academic Emergency Medicine.

Seventy-one programs do not use LAA (85.5%) and 12 programs do currently use LAA for procedural training (14.3%). Of the programs that currently use LAA, 75% hold training sessions 1 to 5 times per year. Two programs (16.7%) hold sessions 6 to 11 times per year, and one program (8.3%) holds monthly LAA training sessions. Within these programs, residents of all postgraduate year levels participate in LAA procedural training.

Respondents reported previous LAA use and reasons for discontinuation. Forty-two programs (50.6%) report they have never used LAA. Of programs that do not currently use LAA, seventeen (20.5%) reported that they have used LAA for procedural training in the past, 11 (13.3%) did not know if they had ever used LAA, and 1 program (1.2%) did not answer this question.

Of the 17 programs that have discontinued LAA use, ten (58.8%) cited ethical reasons. Eight programs (47.1%) discontinued LAA training because they felt there was a better or equivalent model available, seven (41.2%) cited financial limitations, and six (35.3%) cited lack of access to LAA resources. Two programs (11.8%) checked “other” as a reason and specifically reported pressure from activist groups.

Respondents reported training modalities they currently use for thoracotomy, pericardiocentesis, thoracostomy, and cricothyrotomy. Many programs reported using more than one modality for each procedure. Human cadaver was the most frequently used modality for thoracotomy and pericardiocentesis, followed by commercial simulation model and homemade model. Commercial simulation model and human cadaver were utilized at similar rates for thoracostomy and cricothyrotomy training. Homemade models were used less commonly for thoracostomy and cricothyrotomy. LAA was least commonly used for all procedures investigated, however all 12 programs utilizing LAA did so for thoracotomy and thoracostomy training, and 11 of these programs used LAA for pericardiocentesis and cricothyrotomy training (Table 2). All programs using “other” training modalities cited ex vivo animal parts such as pig tracheas for cricothyrotomy training and pork ribs for thoracostomy training.

Table 2.

Respondents’ utilization of various training modalities for four procedures specific to emergency medicine

	LAA	Human cadaver	Commercial simulation model	Homemade model	Other
Thoracotomy	12 (14.5)	56 (67.5)	21 (25.3)	20 (24.1)	3 (3.6)
Pericardiocentesis	11 (13.2)	51 (61.4)	36 (43.3)	33 (39.8)	3 (3.6)
Thoracostomy	12 (14.5)	53 (63.9)	55 (66.2)	23 (27.7)	5 (6.0)
Cricothyrotomy	11 (13.2)	53 (63.9)	50 (60.2)	40 (48.2)	21 (25.3)

Values are presented as number (%).

LAA, live anesthetized animal model.

Finally, respondents ranked training models in order of preference for each of these four procedures, under the assumption there are no financial or ethical barriers and that the LAA would survive unharmed (Fig. 1). For thoracotomy, programs that currently use LAA were significantly more likely to rank LAA as most preferable modality. Although these programs ranked LAA as being most preferable for the remaining three procedures as well, this failed to reach statistical significance. Programs that do not currently use LAA were significantly more likely to rank human cadavers as being the most preferable modality for every procedure except pericardiocentesis. These programs also ranked human cadaver as preferable to LAA for pericardiocentesis training, but this failed to reach statistical significance. There was significant disagreement between LAA-utilizers and non-utilizers regarding the most preferable procedural teaching modality. All programs ranked commercial simulation models and homemade models as the third- and fourth-most preferable training modalities, respectively.

Fig. 1.

Average procedure-specific ranking of various training modalities, assuming no financial or ethical barriers, according to respondent sites that do and do not utilize live anesthetized animal models. (A) Thoracotomy, (B)pericardiocentersis, (C) thoracostomy, and (D) cricothyrotomy. 4=most favorable; 1=least favorable. *P≤0.05.

DISCUSSION

Emergency medicine (EM) residencies are tasked with training residents to perform life saving procedures [14,15]. Many of these critical procedures are performed infrequently in the clinical setting, yet EM physicians must be prepared to perform them as needed to prevent patient morbidity and mortality [16]. There is no consensus on the best way to train residents, especially on rare procedures such as thoracotomy, pericardiocentesis, thoracostomy, and cricothyrotomy. Our study demonstrates the wide variety of training practices for these critical procedures. While our data are specific to ACGME-accredited EM residency programs, these procedures, and hence training in performing them, are relevant to EM physicians of all backgrounds and locations. The wide variety of training practices among our respondents mirrors the inconclusive data behind the optimal means of training EM physicians to perform these procedures.

Two systematic reviews comparing LAA against other models failed to find evidence for either modality’s superiority for procedural training [2,5]. However, these studies’ generalizability to civilian medical training is limited by the fact that the majority of their data came from military investigations. Neither study reviewed EM-specific investigations, further limiting their generalizability to EM civilian training. da Luz et al. [2] reviewed ten prospective observational studies and two randomized controlled trials. Eight (66.7%) of these studies were performed in the military setting. The ten prospective studies evaluated LAA training based on pre- and post-training questionnaires, but none of these studies compared LAA to other training modalities. The post-training evaluations demonstrated increased knowledge and self-perceived confidence with respect to procedural performance, but none reviewed clinical outcomes. The randomized controlled trials had too few participants to draw statistically significant conclusions. Goolsby et al. [5] also highlighted the paucity of quality studies on this topic, finding 12 studies that met their inclusion criteria, two of which were only presented as abstracts and three that were already cited in the da Luz review. The Goolsby review focused on prehospital trauma providers and ten (83.3%) of these studies were performed in a military setting. The review rated all studies as poor to moderate with limited clinical impact, citing small sample size and lack of controls, blinding, and validity as their primary limitations. Neither review looked at trends in LAA use, reasons for LAA incorporation or discontinuation, or comparative views of other training modalities.

While we did not attempt to compare training modalities for EM-specific procedures, our study describes some of the reasons behind programs’ use or discontinuation of LAA. We found a marked decrease in LAA use among United States civilian EM residencies, with 14.4% of responding programs utilize LAA in procedural training compared to historical reports as high as 87% [17]. The most commonly cited reason for this was ethical concerns. Resource limitations were also commonly cited.

The second most common reason for discontinuing LAA for procedural training was the assertion that better or equivalent models were available. In contrast, programs that do utilize LAA for procedural training assert that this modality is preferable for training on all of the procedures investigated, compared to commercial simulation models, homemade models, and human cadavers. Additionally, respondents that do use LAA for procedural training were significantly more likely to rank LAA as being preferable to commercial simulation models, whereas respondents not using LAA for procedural training ranked these two modalities similarly. We found this to be the case for all four procedures investigated. These findings suggest that programs that do and do not utilize LAA disagree on the optimal modality for procedural training, despite the lack of outcomes based evidence for or against its use.

Our study has several important limitations. Our study had a response rate of 46.4%, representing all respondents who submitted survey answers for analysis. The remainder of our study sample did not submit survey responses. Although there is no universally agreed-upon response rate that indicates a survey’s results are representative, respondent bias could impact our results. Our response rate mirrors historical difficulties in obtaining survey data from physicians [18]. Our study population did not include residency programs with only DO trainees, which limits the degree to which our results are representative of all types of EM training nationally. A majority of our respondents have a 3-year residency program, which may reflect the prevalence of 3-year programs over 4-year training programs across the country. We do not have specific demographic information about our non-respondents.

Given the nature of this topic, there may be reporting biases. Strong feelings for or against LAA usage may affect our survey responses independently of actual LAA use. Additionally, survey responses and response rate may be affected by fear of reporting LAA use given the possibility of public or activist scrutiny. This latter possibility is mitigated by the anonymous nature of our study, however our results may underestimate the degree to which LAA is used for procedural training.

In conclusion, responding PDs of EM residencies across the Unites States report using multiple modalities for procedural training. While no definitive outcomes based data exist to support the use of one procedural training modality over another, our survey shows that a minority of respondents uses LAA at this time. Those that use LAA do so for teaching multiple EM-specific procedures, and tend to regard LAA as being preferable to other training modalities. Respondents from programs not utilizing LAA regard this teaching modality as inferior to human cadavers, but of similar quality to commercial simulation models when ethical or financial concerns are removed. More outcomes-based research may be needed to assess the utility of LAA compared to other procedural training modalities.