Potential impact of a bedside procedure service on training procedurally competent hospitalists in a community-based residency program.

BACKGROUND: The Society of Hospital Medicine has delineated procedures as one of the core competencies for hospitalists. Little is known about whether exposure to a medical procedure service (MPS) impacts the procedural certification rate in internal medicine trainees in a community hospital training program.
OBJECTIVE: To determine whether or not exposure to an MPS would impact both the number of procedures performed and the rate of resultant certifications in a community hospital internal medicine training program.
DESIGN: Retrospective review.
METHODS: Five cohorts of resident physicians and their procedure data were analyzed comparing months where residents were unexposed to the intervention (pre-MPS) to months where residents were exposed to the intervention (post-MPS). We calculated the average number of procedures performed per month for pre- versus post-MPS periods. For procedural certification, we compared two proportions: the number of certifications over the number of 6-month pre-MPS periods and the number of certifications over the number of 6-month post-MPS periods. SETTING/
SUBJECTS: The study was conducted at a community-based academic medical center. Subjects included all internal medicine residents.
RESULTS: We found a statistically significant difference between the groups, with pre-MPS groups performing 4.3 procedures per month compared with post-MPS groups performing 6.7 procedures per month (p=0.0010). For certification rates, we found statistically significant differences in several categories - overall, paracentesis, femoral central lines, and jugular central lines.
CONCLUSIONS: This study demonstrated that resident exposure to an MPS statistically significantly increased the total number of procedures performed. This study also showed that overall certification rates were statistically significantly different between the pre- and post-MPS groups for several procedures.

Hospital medicine has become part of the mainstream delivery of health care in the United States (1) with surveys from both the Society of Hospital Medicine (SHM) and the American Hospital Association (AHA) estimating that there were approximately 30,000 hospitalists in the United States in 2010 (2). Additionally, a 2010 study by Ratelle et al. found that hospital medicine was the chosen career path for nearly 10% of all graduating internal medicine residents (3). In fact, in our residency program, over 60% of our graduates become hospitalists.

In 2006, the SHM developed a document of core competencies that delineated the core knowledge, skills, and attitudes necessary for effective inpatient practice. The competencies were divided into three sections: Clinical Conditions, Procedures, and Healthcare Systems. Specifically, the Procedures section delineated those procedures that a hospitalist was likely to perform or supervise in caring for a hospitalized patient (4). This list includes both invasive and non-invasive procedures.

Despite the delineation of these procedural core competencies for hospitalists, it has been shown that many hospitalists do not perform these designated procedures (5). The reasons for this are not clearly defined but some may be traced back to internal medicine training. Prior to 2007, many of the procedures listed as hospitalist core competencies were required by the American Board of Internal Medicine (ABIM) in order to complete internal medicine training. Despite these requirements, there were indications that residents were not comfortable performing bedside procedures (6) and the training model was inadequate (7–9). In July 2007, the ABIM repealed the numeric procedural requirements for many of these procedures, instead requiring residents only to be able to ‘recognize indications, contraindications and manage complications’ of selected bedside procedures (10). In addition, duty hour restrictions, availability of other specialties such as interventional radiology (11), and lack of motivation may all play a part in the overall decline of procedural certification in residents. Research has demonstrated the positive impact of a specialized medical procedure service (MPS) in the training of procedural skills to internal medicine residents. Having a procedure service has been shown to improve the procedural comfort (12), competence (7, 13), and overall opportunities for procedural training (14). Little is known about whether or not exposure to an MPS impacts the procedural certification rate in internal medicine trainees. In addition, most of these studies were performed in academic hospital settings. We sought to determine whether or not exposure to an MPS would impact both the number of procedures performed by residents and the rate of resultant certifications in a community-based teaching hospital setting.

Methods

The hospital

This retrospective review was performed at Mount Sinai Hospital Medical Center (MSH), a not-for-profit community hospital located on the west side of Chicago, providing primary, secondary, and tertiary care to its surrounding neighborhoods. The hospital has been a major teaching hospital for many years and has residency programs in all the major specialties. It serves as a Level I Trauma Center and a Level III Perinatology and Neonatal Care Center. The medical service census averages between 90 and 110 patients. There is a Medical Intensive Care Unit, a step-down (intermediate care) unit, and a telemetry unit.

The procedure service

The MPS was established at MSH in January of 2014. The MPS was comprised of three pulmonary/critical care faculty along with one procedural hospitalist. These faculty were available to supervise residents performing procedures in a variety of settings, including the medical wards and intensive care units. The procedure service was available during normal business hours plus 1 week of nights per month when the procedural hospitalist was available during his night shift. A rotating call schedule was made using these four staff, and the schedule was posted and available to the residents. If supervision for a bedside procedure was needed, the residents would page the procedure service attending on call. The attending would then supervise the procedure. A portable ultrasound device was available to the MPS at all times.

Target population

The study group included all the internal medicine residents at MSH from July 2010 to August 2015. The residency program consists of 39 total residents per year, 12 categorical residents in the postgraduate (PG) years one, two, and three (PGY1, PGY2, PGY3) with three additional preliminary residents in each new PGY1 year. All the procedures for the preliminary residents were left out of the analysis because they did not proceed beyond the PGY1 year. Procedural data are entered by the residents into myevaluations.com, an online medical education training management service provider which allows residency programs to design, assign, and review their own evaluations, procedures, and patient logs. Any procedure completed by the residents is logged into the system. In order for the procedure to be counted as successful, it must be electronically signed off by the supervising attending or resident.

Data collection

Procedure logs and certification data were collected from the myevalations.com website. The study spanned the period July 2010 – August 2015 and included five cohorts of PG residents (n=63). Six types of procedures were included in the analysis: paracentesis, thoracentesis, lumbar puncture, jugular vein, femoral vein, and subclavian vein.

Intervention description

The MPS intervention (intervention) was implemented in January 2014. Any resident requesting procedural help was encouraged to call the procedure service for supervision. Any procedure performed prior to the MPS being launched was labeled ‘pre-MPS’ and any procedure performed after was labeled ‘post-MPS’.

Procedure data

We first calculated the number of procedures performed per month, comparing pre-MPS months to post-MPS months across all three PG years. Between July 2010 and August 2015, there were a total of 148 1-month periods available for analysis: 90 pre-MPS 1-month periods and 58 post-MPS 1-month periods. We calculated the average number of procedures performed per month for pre- versus post-MPS periods. Because the average monthly procedure data were non-normally distributed, we used the Wilcoxon rank-sum test to determine whether differences between the number of procedures per month in the pre- and post-MPS periods were statistically significant. These data were analyzed using SAS 9.3 and a P value ≤0.05 was considered statistically significant.

Certification data

The certification data were analyzed overall (all six certification types included) and by specific certification type. For each resident, each PG year was divided into two 6-month periods: January – June and July – December, with an indicator for whether the period was pre-MPS or post-MPS, and whether they achieved a certification or not. Certification was defined as being the primary operator on a minimum of five successful attempts of a procedure. Once a certification was achieved in a particular 6-month period for a resident, they contributed no additional time periods to the analysis; specific certifications happen only once and they were thus no longer at risk for the outcome of interest. We compared two proportions: the number of certifications over the number of 6-month pre-MPS periods and the number of certifications over the number of 6-month post-MPS periods. We used the two-sample test of proportions to test the hypothesis that there was no statistically significant difference in the proportion of certifications between the pre- and post-MPS periods. We compared proportions for overall certifications, overall certifications by PG year, and each of the six specific certifications by PG year. These data were analyzed using Stata/SE 14.1 and a P value ≤0.05 was considered statistically significant.

Between July 2010 and August 2015, there were a total of 1,752 6-month periods when residents had the opportunity to obtain a new certification: 1,155 pre-MPS 6-month periods and 597 post-MPS 6-month periods (Table 1). Within PG years 1, 2, and 3, there were 540, 388, and 227 pre-MPS 6-month periods, respectively, and 216, 197, and 184 post-MPS 6-month periods, respectively. For paracentesis certification opportunities, within PG years 1, 2, and 3, there were 90, 63, and 36 unexposed, and 36, 35, and 36 exposed six-month periods. For femoral vein certification opportunities, within PG years 1, 2, and 3, there were 90, 66, and 40 unexposed, and 36, 32, and 28 exposed six-month periods. For jugular vein certification opportunities, within PG years 1, 2, and 3, there were 90, 63, and 30 unexposed, and 36, 23, and 17 exposed six-month periods. For subclavian vein certification opportunities, in PG years 1, 2, and 3, there were 90, 65, and 41 unexposed, and 36, 35, and 34 exposed six-month periods. For lumbar puncture certification opportunities, in PG years 1, 2, and 3, there were 90, 65, and 38 unexposed, and 36, 36, and 33 exposed six-month periods. For thoracentesis certification opportunities, in PG years 1, 2, and 3, there were 90, 66, and 42 unexposed, and 36, 36, and 36 exposed six-month periods.

Table 1

Number of 6-month periods with opportunity for residents to obtain a new certification by PGY and certification type

	Pre-MPS (N)	Post-MPS (N)
Overall
All PG years	1,155	597
PGY1	540	216
PGY2	388	197
PGY3	227	184
Paracentesis
PGY1	90	36
PGY2	63	35
PGY3	36	36
Femoral
PGY1	90	36
PGY2	66	32
PGY3	40	28
Jugular
PGY1	90	36
PGY2	63	23
PGY3	30	17
Thoracentesis
PGY1	90	36
PGY2	66	36
PGY3	42	36
Subclavian
PGY1	90	36
PGY2	65	35
PGY3	41	34
Lumbar puncture
PGY1	90	36
PGY2	65	36
PGY3	38	33

Results

Comparison of number of procedures

In order to determine whether exposure to an attending staffed MPS would increase the number of procedures performed by residents, we calculated the average number of procedures performed per month among pre- versus post-MPS groups. We found a statistically significant difference between the groups, with the pre-MPS group performing 4.3 procedures per month compared to the post-MPS group performing 6.7 procedures per month (p=0.0010) (Table 2).

Table 2

Number of procedures performed per month in the pre-MPS and post-MPS periods

Pre-MPS	Post-MPS	p
4.29	6.72	0.0010

Comparison of certification rates

In order to determine whether exposure to an attending staffed MPS would increase the number of certifications obtained by residents, we calculated the 6-month certification rates for the pre- and post-MPS groups. We found differences in several categories – overall (all procedures not broken down by type), paracentesis, femoral central lines, and jugular central lines (Table 3).

Table 3

Six-month certification rates in pre-MPS and post-MPS groups

	Pre-MPS	Post-MPS	p
Overall
All PG years	0.036	0.069	0.0025
PGY1	0	0.037	0.0000
PGY2	0.052	0.096	0.0396
PGY3	0.097	0.076	0.4576
Paracentesis
PGY1	0	0.083	0.0056
PGY2	0.048	0.057	0.8373
PGY3	0.056	0	0.1515
Femoral
PGY1	0	0.028	0.1124
PGY2	0.045	0.188	0.0224
PGY3	0.175	0.143	0.7231
Jugular
PGY1	0	0.111	0.0013
PGY2	0.159	0.391	0.0214
PGY3	0.3	0.529	0.1200
Thoracentesis
PGY1	0	0
PGY2	0	0
PGY3	0	0
Subclavian
PGY1	0	0
PGY2	0.015	0.029	0.6532
PGY3	0.049	0.029	0.6700
Lumbar puncture
PGY1	0	0
PGY2	0.046	0.028	0.6502
PGY3	0.053	0	0.1813

Specifically, looking across all PG years and all procedures, we found a significant difference in the 6-month certification rates among post-MPS versus pre-MPS groups. The 6-month certification rate was 1.9 times higher for the post- versus the pre-MPS group (0.069 compared to 0.036 certifications per 6-month period, respectively; p=0.0025). Breaking these numbers down, we found differences in the rate of certification for both the PGY1 and PGY2 years. In PGY1, the 6-month certification rate for pre-MPS PGY1 residents was 0, compared to 0.037 among post-MPS PGY1 residents (p=0.0000), resulting in an additional 0.22 certifications per resident in a fully exposed residency program compared to a program with no MPS. The certification rate for post-MPS PGY2 residents was 1.8 times as high as that for pre-MPS PGY2 residents (0.096 compared to 0.052 certifications per 6-month period, respectively, p =0.0369). The overall certification rates for the pre- versus post-MPS PGY3 residents did not differ significantly.

Other procedures for which the exposure to the procedure service made a difference were paracentesis, femoral central lines, and jugular central lines. For paracentesis, the 6-month certification rate for pre-MPS PGY1 residents was 0 compared to 0.083 among post-MPS PGY1 residents (p=0.0056), resulting in an additional 0.5 certifications per resident in a fully exposed residency program compared to a program with no MPS.

We also observed a difference in the 6-month femoral lines certification rate for the PGY2 class. The certification rate for post-MPS PGY2 residents was four times as high as that for pre-MPS PGY2 residents (0.188 compared to 0.045 certifications per 6-month period, respectively; p=0.0224).

The largest difference in certification rates was seen for jugular central line certifications. The certification rate for pre-MPS PGY1 residents was 0, compared to 0.111 certifications per 6-month period among post-MPS PGY1 residents (p=0.0013), resulting in an additional 0.16 certifications per resident in a fully exposed residency program compared to a program with no MPS. The certification rate for post-MPS PGY2 residents was 2.5 times as high as that for pre-MPS PGY2 residents (0.391 compared to 0.159 certifications per 6-month period, respectively; p=0.0214).

Discussion

This study was a retrospective review of the potential impact of an MPS on the number of procedures performed and the certification rates of residents in a community-based internal medicine residency program. To our knowledge, this is the first study to demonstrate that exposure to an MPS affects both the number of procedures performed by residents and the rate of resultant certifications in a community-based residency program. Per the SHM, procedural competence is one facet of preparing internal medicine residents to become competent hospitalists (4). Although there are variations in what may be considered the definition of procedural certification, the ABIM states that in order to gain procedural certification, residents must be the ‘primary operator or assist another primary operator’ in the specific procedure and ‘should be an active participant for each procedure five or more times’ (15). We felt that it was important to look at certification because hospital medical staff appointments (credentialing) require a delineation of clinical privileges. Procedural privileges are determined by procedures in which physicians are certified. Because of the large number of our residents who choose the hospitalist path, we felt it was important to determine whether we could better prepare them for this career choice.

This study demonstrated that resident exposure to an MPS statistically significantly increased the total number of procedures performed. A previous study indicated similar findings (16). These findings suggest that having a readily accessible supervising attending physician may encourage internal medicine residents to perform more procedures because help is readily available and more attempts at procedures can be made with direct supervision. Additionally, performing more procedures has been shown to increase trainees’ comfort and confidence (6, 17) while also providing added experience. Finally, having a supervised procedure service may help prevent procedural complications (12), thereby improving patient safety and potentially decreasing hospital days.

This study also showed that overall certification rates were statistically significantly different between the pre- and post-MPS groups in both PGY1 and PGY2 classes. We also found significant differences in the certification rates for specific procedures – paracentesis, femoral central lines, and jugular central lines. Being certified earlier in training suggests more opportunity to both perform additional procedures and also potentially gain experience in becoming a supervisor for these procedures. Moreover, gaining certification during residency allows for additional clinical privileges when the residents acquire postgraduate employment.

Our study had several limitations. First, our study relied on self-reported entry of procedures into the myevaulations.com system. Although the procedures had to be signed off by supervisors in order to be counted, we cannot rule out reporting bias or error. Second, the exposure time was only 18 months. Unfortunately, our MPS was discontinued due to faculty attrition. There is always a ‘run-in’ period with any new service. The MPS was likely underutilized for the first few months it was in place. Longer exposure time to MPS may have had greater impact on the number of procedures and certifications obtained. In addition, a longer exposure may have led to increased numbers of certifications in more varied procedures. The procedures in which we found statistical significance are the ones that are performed most frequently in our program. With more exposure, residents may find more opportunity and confidence to perform the other procedures as well. Third, the study was conducted at a single institution. Because of this, the overall number of residents was small and thus the number of procedures was limited. In addition, because of the small numbers, there is a possibility that a few very motivated residents could account for some of the differences we found.

Conclusion

In summary, our study showed that having an attending-driven MPS may have potential benefit to internal medicine residents training to become competent hospitalists. Future studies could address some of our limitations by looking at larger programs and more prolonged exposures. In addition, similar studies could serve as potential justification for the development of a specialized hospitalist training track with a dedicated MPS to help those who are interested in gaining experience in and becoming certified in those procedures defined by the SHM as core procedures.