Procedural frequency: Results from 18 academic, community and freestanding emergency departments.

BACKGROUND: Emergency physicians must maintain procedural skills, but clinical opportunities may be insufficient. We sought to determine how often practicing emergency physicians in academic, community and freestanding emergency departments (EDs) perform 4 procedures: central venous catheterization (CVC), tube thoracostomy, tracheal intubation, and lumbar puncture (LP).
METHODS: This was a retrospective study evaluating emergency physician procedural performance over a 12-month period. We collected data from the electronic records of 18 EDs in one healthcare system. The study EDs included higher and lower volume, academic, community and freestanding, and trauma and non-trauma centers. The main outcome measures were median number of procedures performed. We examined differences in procedural performance by physician years in practice, facility type, and trauma status.
RESULTS: Over 12 months, 182 emergency physicians performed 1582 of 2805 procedures (56%) and supervised (resident, nurse practitioner or physician assistant) an additional 1223 of the procedures they did not perform (43%). Median (interquartile range) physician performance for each procedure was CVC 0 [0, 2], tube thoracostomy 0 [0, 0], tracheal intubation 3 [0.25, 8], and LP 0 [0, 2]. The percentage of emergency physicians who did not perform at least one of each procedure during the 1-year time frame ranged from 25.3% (tracheal intubation) to 76.4% (tube thoracostomy). Physicians who work at high-volume EDs (>50,000 visits per year) performed nearly twice as many tracheal intubations, CVCs, and LPs than those at low-volume EDs or freestanding EDs when normalized per 1000 visits. Years out of training were inversely related to total number of procedures performed. Emergency physicians at trauma centers performed almost 3 times as many tracheal intubations and almost 4 times as many CVCs compared to non-trauma centers.
CONCLUSION: In a large healthcare system, regardless of ED type, emergency physicians infrequently performed the 4 procedures studied. Physicians in high-volume EDs, trauma centers, and recent graduates performed more procedures. Our study adds to a growing body of research that suggests clinical frequency alone may be insufficient for all emergency physicians to maintain competency.

INTRODUCTION

Background

Emergency physicians must maintain their procedural skills. The literature on the frequency of emergency physician performance of procedures is limited, but several recent studies suggest procedural performance may be too infrequent to maintain skills, especially in academic centers. Data from a national emergency medicine group of 135 emergency departments in 19 states found that the median number of tracheal intubations performed by 2108 emergency physicians was 10 per year, with 25% performing 4 or less and 5% performing zero intubations.

Emergency physicians are responsible for both the care of critically ill patients and, in many EDs, the training of resident physicians and the supervision of nurse practitioners and physicians assistants. Both trainees and other clinicians may require teaching and supervision of procedural performance. Emergency physicians likely perform fewer procedures in academic settings, and the impact of procedural supervision on skill maintenance is unclear.

Importance

The aforementioned study on emergency physician performance of tracheal intubation is limited by not differentiating between procedural supervision and performance and did not include procedures other than tracheal intubation. In the editorial response to this study's publication, the central question was how often do practicing emergency physicians need to perform a procedure to maintain skills. There is a general assumption that clinical practice is sufficient to maintain procedural proficiency, but the available literature is limited. Moreover, we were unable to find studies examining the impact of ED type on procedural frequency. Physicians who work exclusively at lower‐volume EDs may have lower acuity and perform critical procedures less frequently. Emergency physicians have a professional obligation to maintain procedural skills, and competent procedural performance may contribute to emergency physician well‐being and job satisfaction. An accurate understanding of the frequency of emergency physician procedural performance is also essential to addressing any performance gaps. Thus, there is a critical need for studies evaluating a broader range of emergency physician performance of procedures.

Goals of the investigation

Our primary objective was to evaluate the frequency of emergency physicians' performance of 4 procedures in a large healthcare system: central venous catheterization (CVC), tube thoracostomy, tracheal intubation, and lumbar puncture (LP). We specifically sought to examine differences in the frequency of each procedure based on ED facility type and physician years in practice.

METHODS

Study design and setting

This was a retrospective cohort study of emergency physician procedural performance across 18 EDs of a large healthcare system. The 4 procedures evaluated were CVC, tube thoracostomy, tracheal intubation, and LP. We studied procedural performance over the 2018 calendar year. Our institutional review board approved the protocol before study commencement.

There were approximately 84 emergency medicine residents across all sites in addition to rotating residents from other specialties. All EDs are located in northeast Ohio, with the exception of one ED in southeast Florida, and include academic, community, and freestanding settings.

Selection of participants

Emergency physicians were eligible for the study if they practiced solely in an ED within our healthcare system during the study period. Emergency physicians in our healthcare system, whether part time or full time, are not allowed to work clinically outside the system. Emergency physicians were excluded if they were not employed for the entire calendar year. Additionally, we excluded emergency medicine‐critical care physicians because they did not practice exclusively in the ED.

Data collection

Trained investigators collected study data from 2 sources: the electronic medical record (EMR; EPIC, Verona, Wisconsin, USA) and a separate, proprietary database maintained by the healthcare system. We first used the internal database to identify procedural performance based on billing records. We then used the corresponding encounter information to extract ED procedural records from the EMR. From these data, we recorded the performing and the billing clinician. If the performing clinician in the EMR and the billing clinician in the billing record matched, no further review was undertaken. If the performing clinician was different from the billing clinician, then the procedure was classified as supervised and a manual review was performed to adjudicate who actually performed the procedure and ensure proper attribution.

ED classifications included hospital‐based versus freestanding, academic versus non‐academic, trauma (American College of Surgeons Level 1 or 2) versus non‐trauma, and low volume versus high‐volume. We considered a site academic if any resident physician rotated through the ED. We defined low volume as <50,000 annual visits (actual range: from 25,833 to 43,116) and high volume as >50,000 annual visits (actual range: 50,625–90,802).

We determined emergency physician years‐in‐practice by years since residency graduation date, calculated as the number of years from the end of emergency medicine residency to the 2018 calendar year. For internal medicine (IM), family practice (FP), and pediatrics, we used the year of graduation from residency as years in practice. For pediatric emergency physicians, we used years in practice since completing fellowship.

Outcomes

The primary outcome was the frequency of emergency physician performance of 1 of the 4 procedures over 12 months, measured as median (interquartile range). We also report the frequency of procedural supervision.

Statistical analysis

We began by tabulating all data and generating standard descriptive statistics. We present medians with interquartile range for continuous variables. Descriptive statistics were reported for frequency of procedure per 1000 patient encounters with 95% confidence interval (CI). Categorical variables are summarized with frequency (%). Analyses were performed using SAS (version 9.4, The SAS Institute, Cary, North Carolina, USA).

The Bottom Line

Competent performance of critical procedures is foundational to the practice of emergency medicine. A growing body of literature suggests some procedures, like tracheal intubation, may be performed uncommonly by practicing emergency physicians, calling into question whether clinical experience alone is sufficient to maintain skills. This multicenter study adds to this literature, providing further evidence that tracheal intubation is performed uncommonly by emergency physicians and that other critical procedures, like central line placement, are equally infrequent for individual physicians.

RESULTS

Two‐hundred eleven emergency physicians billed for at least 1 of the 4 study procedures in 2018. Of those, 182 were employed exclusively in our system for 1 full year and were included in the analysis: 171 emergency medicine‐trained, 7 pediatric emergency medicine, 2 pediatrics, and 2 non‐emergency medicine (1 IM, 1 FP). Table 1 presents data for the 18 EDs in the healthcare system‐ 6 (33.3%) are high volume, 7 (38.9%) are academic, and 3 (16.7%) are trauma centers.

TABLE 1

Characteristics of the 18 study emergency departments (EDs)

Number	Emergency physicians	Institution	Teaching	Trauma	2018 volume
1	36	Hospital ED	Teaching	Non‐trauma	65,014
2	29	Hospital ED	Teaching	Trauma	54,396
3	20	Hospital ED	Non‐teaching	Non‐trauma	26,948
4	19	Hospital ED	Teaching	Trauma	63,854
5	15	Freestanding ED	Teaching	Non‐trauma	19,092
6	14	Hospital ED	Teaching	Trauma	50,625
7	13	Hospital ED	Non‐teaching	Non‐trauma	42,443
8	13	Hospital ED	Non‐teaching	Non‐trauma	25,833
9	12	Hospital ED	Non‐teaching	Non‐trauma	18,100
10	12	Hospital ED	Non‐teaching	Non‐trauma	37,185
11	12	Freestanding ED	Non‐teaching	Non‐trauma	22,037
12	11	Hospital ED	Non‐teaching	Non‐trauma	41,459
13	11	Hospital ED	Non‐teaching	Non‐trauma	43,116
14	9	Hospital ED	Teaching	Non‐trauma	36,166
15	8	Freestanding ED	Non‐teaching	Non‐trauma	18,300
16	7	Freestanding ED	Teaching	Non‐trauma	18,320
17	7	Freestanding ED	Non‐teaching	Non‐trauma	16,286
18	7	Hospital ED	Non‐teaching	Non‐trauma	37,594

For 2018, 2805 procedures of interest were performed. Emergency physicians performed 1582 (53%) and supervised 1223 (41%), and there were no missing data for procedural performance. Median procedures performed per emergency physician were 0 [0, 2] CVC, 0 [0, 0] tube thoracostomy, 3 [0.25, 8] tracheal intubation, and 0 [0, 2] LP. Median procedures supervised by an emergency physician were 0 [0, 2] CVC, 0 [0, 0] tube thoracostomy, 1 [0, 5.75] tracheal intubation, and 0 [0,1] LP. The number of emergency physicians (182 physicians) who did not perform a single procedure was: 92 CVC (50.6%), 139 tube thoracostomy (76.4%), 46 tracheal intubations (25.3%), and 92 LP (50.6%). Emergency physicians performed a smaller proportion of CVCs (46.4%) compared to tracheal intubations (60.1%) and LPs (59.3%).

Physicians who work at high‐volume EDs performed almost twice as many CVCs, tracheal intubation, and LPs than those at low‐volume EDs or freestanding EDs when normalized per 1000 visits. Trauma centers had more CVCs and tracheal intubations, nearly similar rates of tube thoracostomy and similar rates of LP than non‐trauma centers did (Table 2). Tube thoracostomy were performed with similar frequency at high‐volume EDs and low‐volume EDs. Table 3 compares procedure frequency by site.

TABLE 2

Procedures per 1000 emergency department (ED) encounters by select characteristics

	Procedure grouper	Procedures per 1000 encounters	Lower 95% CI	Upper 95% CI
Higher‐volume HBED
	CVC	1.67	−0.19	3.53
	Tube thoracostomy	0.23	−0.46	0.91
	Tracheal intubation	3.83	1.01	6.64
	Lumbar puncture	0.92	−0.46	2.31
	Total	6.65	2.95	10.35
Lower‐volume HBED
	CVC	0.85	−1.03	2.73
	Tube thoracostomy	0.24	−0.75	1.22
	Tracheal intubation	2.00	−0.88	4.87
	Lumbar puncture	0.44	−0.91	1.79
	Total	3.53	−0.29	7.34
FSED
	CVC	0.34	−2.18	2.87
	Tube thoracostomy	0.11	−1.31	1.53
	Tracheal intubation	1.13	−3.44	5.71
	Lumbar puncture	0.32	−2.10	2.74
	Total	1.90	−4.03	7.83
Academic
	CVC	1.74	−0.09	3.58
	Tube thoracostomy	0.25	−0.45	0.95
	Tracheal intubation	3.81	1.10	6.53
	Lumbar puncture	0.65	−0.47	1.77
	Total	6.45	2.93	9.98
Non‐academic
	CVC	0.56	−0.90	2.02
	Tube thoracostomy	0.17	−0.64	0.99
	Tracheal intubation	1.58	−0.87	4.04
	Lumbar puncture	0.61	−0.91	2.13
	Total	2.92	−0.41	6.25
Trauma
	CVC	2.36	−0.19	4.90
	Tube thoracostomy	0.27	−0.59	1.12
	Tracheal intubation	5.03	1.32	8.75
	Lumbar puncture	0.61	−0.69	1.91
	Total	8.27	3.51	13.02
Non‐trauma
	CVC	0.68	−0.60	1.97
	Tube thoracostomy	0.19	−0.49	0.87
	Tracheal intubation	1.79	−0.28	3.87
	Lumbar puncture	0.63	−0.60	1.86
	Total	3.30	0.49	6.11

Abbreviation: CVC, central venous catheterization; FSED, freestanding emergency department; HBED, hospital‐based emergency department.

TABLE 3

Procedures by site data

Factor	Total n = 2805)	CVC n = 679)	Tube thoracostomy n = 134)	Tracheal intubation n = 1606)	Lumbar puncture n = 386)
ED category, number (%)
FSED	194(6.9)	36(5.3)	9(6.7)	119(7.4)	30(7.8)
Lower‐volume	895(31.9)	221(32.5)	64(47.8)	497(30.9)	113(29.3)
Higher‐volume	1716(61.2)	422(62.2)	61(45.5)	990(61.6)	243(63.0)
Teaching site, number (%)
Non‐teaching	961(34.3)	187(27.5)	62(46.3)	512(31.9)	200(51.8)
Teaching	1844(65.7)	492(72.5)	72(53.7)	1,094(68.1)	186(48.2)
ED Trauma status, number (%)
Non‐trauma	1528(54.5)	320(47.1)	93(69.4)	823(51.2)	292(75.6)
Trauma	1277(45.5)	359(52.9)	41(30.6)	783(48.8)	94(24.4)

Abbreviations: CVC, central venous catheterization; FSED, freestanding emergency department.

Years out of training were inversely related to number of procedures performed (Table 4). Overall 33% of procedures were performed by physicians with <5 years of experience (n = 41), (Table 4). For all 4 procedures combined, physicians with <5 years’ experience performed almost 3 times as many procedures as those with 20 years or more experience.

TABLE 4

Procedure performance by provider years of experience

Factor
Years of experience, no. (%)	Total N = 2805)	CVC N = 679)	Tube thoracostomy N = 134)	Tracheal intubation N = 1606)	Lumbar puncture N = 386)
——————————
<5 (n = 41; 22.5%)	925(33.0)	284(41.8)	43(32.1)	499(31.1)	99(25.6)
> = 5; <10 (n = 43; 23.6%)	747(26.6)	194(28.6)	44(32.8)	391(24.3)	118(30.6)
> = 10; <15 (n = 37; 20.3%)	513(18.3)	103(15.2)	24(17.9)	309(19.2)	77(19.9)
> = 15; <20 (n = 25; 13.7%)	274(9.8)	44(6.5)	8(6.0)	182(11.3)	40(10.4)
> = 20 (n = 36; 19.8%)	346(12.3)	54(8.0)	15(11.2)	225(14.0)	52(13.5)

Abbreviation: CVC, central venous catherization.

LIMITATIONS

This study has several limitations. First, this is a retrospective study of EMR and billing data and therefore is dependent on critical procedures being documented. It is possible some critical procedures were not documented and therefore not captured. Furthermore, this analysis was conducted using a full year emergency physician as the unit of study. Consequently, it does not fully capture the total number of procedures performed at a site that may have had a locum or other physician performing a procedure. Second, we treated all emergency physicians equally regardless of annual clinical hours worked as procedure frequency is important regardless of employment status. As the unit of analysis was the emergency physician, we did not separate residents from APPs and thus are unable to determine the impact of APPs as opposed to residents. Finally, our population represents data primarily from one region of the country within a single healthcare system; thus, these data may not be generalizable to all areas of the United States or all practice settings.

DISCUSSION

In this study, the median frequency of procedural performance for emergency physicians was 3 for tracheal intubation and 0 for the remaining procedures. Even when combining both emergency physician‐performed and emergency physician‐supervised procedures, tracheal intubation, the most common procedure, had a median frequency of 4. For 3 of the 4 procedures, we found that 50% of emergency physicians performed 0 of each procedure.

Although there is likely a minimum number of procedures needed to maintain competency in procedural skills, whether that best occurs via performance, simulation, or supervision of trainees or a combination of these is unknown. The Accreditation Council for Graduate Medical Education requires a specific number of procedures for new learners to be competent. A recent study demonstrated that performing at least 3 or supervising at least 5 tracheal intubations annually predicted proficient performance on simulation‐based skills assessments. This study suggests that supervision plays a role in skill maintenance. In a recent survey of pediatric emergency physicians, >90% of respondents felt it was important to maintain competency for 5 of the 6 critical procedures including the 4 procedures evaluated in our study. However, less than half of respondents felt clinical care alone provided opportunity to maintain skills. Up to 40% of respondents indicated they had no mandatory training for the procedural skills studied. Frequent deliberate practice has been associated with superior skill retention and maintenance. , , An analysis of emergency services in Norway showed patients had better outcomes after critical procedures were performed if the physician was both specifically trained in the procedure and performed it at least once yearly. There is evidence that competency can be maintained through deliberate practice and mastery learning with procedural simulation training. , ,

Mentored procedures in which another clinician supervises the procedure with real‐time feedback is another method to guide emergency physicians through infrequently performed procedures and also allow for the exchange of knowledge between clinicians. In one study, a simulation‐based curriculum was developed for teaching faculty at an academic program. Physicians self‐rated their confidence levels in performing the procedure before and after the simulation‐based procedural skills lab and perceived the lab as most helpful for rarely encountered clinical procedures. Another study created a theoretical representation of the synthesis between simulation and clinical practice on procedural skill development and maintenance. A 6‐step pedagogical framework for procedural skills training was developed and entailed Learn, See, Practice, Prove, Do, and Maintain. The methods that will best help emergency physicians maintain procedural proficiency in CVC, tube thoracostomy, tracheal intubation and LP requires additional research across various practice settings.

All of our EDs had nurse practitioners, medical assistants or residents present and all high‐volume EDs had some form of residents present. Although it is possible physicians who work at high‐volume EDs are merely supervising more procedures because of the presence of trainees, the incidence per 1000 encounters indicates that there is more exposure to critical procedures at these sites. Despite this greater experience, it is likely below necessary levels to maintain performance. Overall when looking at facility type, we found that physicians in low‐volume EDs and freestanding EDs had fewer critical procedure opportunities than those in high‐volume EDs. This information may be useful when targeting certain physicians for additional education or continuing medical education. Whether a physician works at 1 site only or a combination of high‐volume, low‐volume, or freestanding EDs may play a role in procedural frequency.

When we examined years in practice, we found that fewer years in practice meant more procedures performed, which fits with relevant literature. Although a systematic review found the majority of studies evaluated showed decreasing performance of procedures with increasing years in practice for all clinical outcomes assessed, a few studies showed that performance increased as experience increased, peaked, and then began to decline. The exact nature of the relationship between experience and performance or quality is not clear for the procedures we studied. The relationship is likely non‐linear and complex, being related to a variety of factors and not just years, especially because actual performance is so low for individuals. We found no data on whether a minimum number of procedures are required to maintain skills based on years since training. Less experience may lead to worse performance over time. More years in practice is not the same as experience per se, as recent graduates may have more current experience and knowledge that leads to more procedures and better performance.

In conclusion, our study provides insight into procedural skill frequency for emergency physicians across a large healthcare system by examining overall frequency, years of practice, and facility type. Additional research is needed to examine how the frequency of CVC, tube thoracostomy, tracheal intubation, and LP procedures affect skill performance as well as whether simulation versus procedural skills labs help to maintain competency, the impact of practice environment, and whether supervising is comparable to performing a procedure. Such work would, ideally, inform skills‐maintenance programs, and accreditation.

AUTHOR CONTRIBUTION

Baruch S. Fertel and Erin L. Simon conceived and designed the study. Erin L. Simon, Baruch S. Fertel, and McKinsey R. Muir contributed to data collection. Isaac Briskin provided statistical advice on study design, analyzed the data. Erin L. Simon, Andrew Suchan, and Baruch S. Fertel drafted the article, and all authors contributed substantially to its revision. Erin L. Simon takes responsibility for the paper as a whole.

CONFLICTS OF INTEREST

None.