Do PGY-1 residents in Emergency Medicine have enough experiences in resuscitations and other clinical procedures to meet the requirements of a Competence by Design curriculum?

BACKGROUND: With the transition to a Competence by Design (CBD) curriculum, Fellow of the Royal College of Physicians in Emergency Medicine (FRCP-EM) training has created guidelines on experiences residents should have before progressing. We sought to quantify adult medical resuscitations and clinical procedures completed by PGY1 FRCP-EM residents to compare them to CBD requirements with the aim to identify areas of limited exposure requiring curriculum revisions prior to nation-wide CBD implementation.
METHODS: Twenty-two PGY1 residents from four FRCP-EM programs recorded their activities from July 2017 to June 2018 in an online log that tracked resuscitations and procedures along with role assumed, supervision, and level of comfort.
RESULTS: In total 515 resuscitations were logged with the median number per resident 15 (range 0 to 98). The most frequent resuscitation was altered mental status and the least was unstable dysrhythmia. 557 total procedures were logged with the median number 75 (range 8 to 273). The most frequent procedure done was simple laceration repair and the least frequent was intraosseous access.
CONCLUSIONS: Unstable dysrhythmias and cardiorespiratory arrest along with intraosseous access and arthrocentesis are low event clinical exposures. In the era of CBD, the misalignment of entrustrable professional activity (EPA) targets and curriculum delivery should be monitored/reviewed to ensure expectations are realistic and that sufficient exposures are available.

Introduction

Emergency medicine (EM) training aims to develop effective patient centred physicians who can meet the needs of the people who come to the ER for medical assistance. One of the core areas of expertise is the performance of appropriate procedures for an array of medical emergencies and, when required, resuscitation. There are few data on the number of patient experiences that require resuscitations or key procedures. In the era of Competence by Design (CBD), with a focus on quality and safe patient experiences, this information is needed to properly direct EM training in Canada.

In July 2018, Fellow of the Royal College of Physicians of Canada in Emergency Medicine (FRCP-EM) training transitioned to a CBD model.[1] In this model, trainees progress through four stages: Transition to Discipline (TD), Foundations of Discipline (F), Core of Discipline (C), and Transition to Practice (TP).[2] Learning and assessment in each of these stages is guided by several Entrustable Professional Activities (EPAs), or essential skills that trainees should be able to perform upon completion of residency[3] Competence Committees (CC) are tasked with using the EPA data, amongst other data, to make progression and promotion decisions.[3]

During TD and F, equivalent to the first year of EM residency training, residents work towards acquiring the requisite seven EPAs. Out of those seven, TD-1 and F-1 focus on recognition and management of critically ill patients presenting in cardiorespiratory arrest, unstable dysrhythmia, shock, respiratory distress, or with altered neurologic status.[4] F-4 requires experiences with seven procedures: simple wound repair, abscess drainage, immobilization of extremity injuries, joint arthrocentesis, anterior nasal packing, slit lamp exam/intraocular pressure measurement, and intraosseous access (I.O.).[4]

Reaching a point at which residents can be entrusted to perform these skills without oversight requires sufficient access to clinical events. The Canadian Speciality Committee for FRCP-EM set a minimum number for TD-1 resuscitations and F-1 resuscitations at 10 and 15 respectively. The target total number of F-4 procedures was set at 25.[4] These targets, though based on expertise, are arbitrary.

Predicting a bottleneck to access of resuscitations and procedures, the Specialty Committee placed a statement allowing some resuscitations and I.O. procedures to be obtained in the simulation setting. In a survey study, EM residents expressed that their experience with simulation improved their ability to care for patients in the clinical setting. The same survey demonstrated that all Canadian EM programs have access to a simulation curriculum, with the grand majority using simulation as a training tool and not for assessments.[5] The role of simulation in assessments remains controversial.[6]

In our review of the literature attempting to understand EM resident access to clinical events, a limited number of studies were identified that tracked resuscitations and procedures.[7],[8],[9],[10],[11] A Canadian retrospective survey study by Petrosoniak et al. evaluated the number of resuscitations against a minimum acceptable frequency determined by program directors. They found that 24%, 20%, 5%, and 1% of residents met the consensus standard for adult medical resuscitations, adult trauma resuscitations, pediatric medical resuscitations and pediatric trauma resuscitations respectively.[10]

Dire et al. followed a single cohort of residents during a 3-year American EM residency and found that the most common procedure documented was lumbar puncture although this study did not track minor procedures. In this same study the mean number of arthrocentesis done per resident was four, suggesting limited access.[7] Hayden et al. evaluated paper logbooks from 65 American EM residency programs for 22 index procedures and found that infrequently done procedures included arthrocentesis, and I.O. placement.[7] Among Canadian residents, Petrosoniak et al. found that only 52% of residents felt they received adequate procedural skills training.[10]

In summary, during residency, patient experiences requiring resuscitations and clinical procedures can be limited, which can be a barrier to EPA acquisition (and reaching competence). With this prospective observational study, we sought to quantify adult medical resuscitations and procedures done by PGY1 FRCP-EM residents prior to nation-wide CBD implementation with the aim to identify if there is a bottleneck that may hinder EPA completion in TD and F and to inform our program’s future EPA collection processes to ensure residents reach a level of competence to perform quality and safe procedures and resuscitations and achieve positive patient experiences.

Methods

The University of Toronto Research Ethics Board granted ethical approval under protocol number 37414. We used data collected from an online recording tool (Qualtrics) to track resuscitations and procedures completed by 22 Canadian PGY1 FRCP-EM residents enrolled in four residency programs (Table 1) from July 2017 to June 2018 (prior to nation-wide CBD implementation). These were self-reported by residents prospectively using a unique password protected account to record their clinical encounters. We contacted program directors from all 14 Canadian FRCP-EM programs to enrol their residents with four choosing to participate, two declining and all others not responding. Residents gave consent for their inclusion and had the option to decline participation.

Table 1

Number of blocks of each speciality in residency year 1 by participating university at time of the study. The number of residents enrolled from each program is in parentheses.

	University of Calgary	University of Manitoba	Queen's University	University of Toronto
	(4)	(4)	(4)	(10)
Anesthesia	1	1	2	0
Adult Emergency Medicine	4	4	8	4
Elective/Vacation	0	1	0	1
EMS	0	0	1	0
General Surgery	1	1	0	0
Medicine	2	2	0	1
Neurology	0	0	0	1
Obstetrics & Gynecology	1	1	0	1
Orthopedics	0	1	0	1
Pediatrics (All rotations)	1	1	2	1
Plastic Surgery	0	0	0	1
Psychiatry	1	0	0	1
Trauma	1	1	0	0
Ultrasound	1	0	0	1

We specifically tracked procedures and resuscitation laid out in TD-1, F-1, and F-4 (available to programs pre-CBD implementation).[5] A pilot period involving residents from one program during the first month of data collection was used to streamline data entry. Two reminders were sent: one midway to participants and a second email was sent to participating program directors. The log was developed using Qualtrics (Provo, USA) and was accessible by smartphone and computer.

Our analysis used descriptive statistics including mean, median, and 95% confidence interval to identify resuscitations and procedures that were encountered infrequently. Data from all enrolled residents was included for analysis which was done using Microsoft Excel (Redmond, USA).

Results

The 22 residents enrolled included 18 men and four women A total of 515 resuscitations and 557 procedures were logged. Rotations completed by residents are not standardized across training programs and varied by institution. These are detailed in Table 1. The most frequent resuscitation was altered mental status and the least was unstable dysrhythmia. In total 557 procedures were logged with the median number 75 (range 8 to 273). The most frequent procedure done was simple laceration repair and the least frequent was intraosseous access. Table 2 lists the frequency with which residents completed all TD-1, F-1, and F-4 procedures and resuscitations.

Table 2

Frequency of experiences in TD-1/F-1 resuscitations and F-4 procedures by PGY1 residents. Data is sorted in order of low to high frequency

	Total n	Mean (95% CI)	Median	Min	Max	Residents not logging experiences (%)
Resuscitations
Unstable dysrhythmia	32	1.5 (0.7-2.2)	1	0	5	36
Cardiorespiratory arrest	65	3.0 (1.9-4.0)	3	0	10	9
Shock	68	3.1 (1.4-4.8)	1	0	11	41
Respiratory distress	121	5.5 (3.2-7.8)	4	0	16	14
Altered mental status	136	6.2 (2.6-8.8)	2	0	21	14
Procedures
Intraosseous access	9	0.4 (0.1-0.7)	0	0	2	73
Anterior nasal packing	13	0.6 (0.1-1.1)	0	0	4	73
Arthrocentesis	17	0.8 (0.3-1.3)	0	0	4	55
Incision and drainage	58	2.6 (1.7-3.6)	3	0	8	18
Fracture immobilization	88	4.0 (1.9-6.1)	2	0	16	27
Slit lamp exam	97	4.4 (1.5-7.3)	2	0	21	41
Simple laceration repair	275	12.5 (9.4-15.6)	10.5	1	27	0

Discussion

Our prospective review of the resuscitations and procedures completed by 22 Canadian FRCP-EM residents found that many PGY1 residents would not have met the targets laid out in the EPAs. Our self-reported data is supported by a recent study comparing EPA collection across all Canadian EM programs post-CBD implementation. In this study, looking at all TD and F EPAs, despite CBD targeted curriculums, TD-1, F-1 and F-4 targets were frequently not met.[12]

For example, 64% of residents in our study had fewer than the 25 combined resuscitations required in TD-1 and F-1. In comparison, in the Thoma et al. study TD-1 and F-1 EPA completion averages were 8 out of 10 and 12 out of 15 respectively.[12] Overall, the mean number of resuscitations completed by residents in our cohort appears similar to those documented in previous studies of EM residents. Dire et al. found that the mean number of resuscitations directed or assisted for a 3-year US residency was 67.[7] The mean number of resuscitations participated in among residents in our study was 23.4. If extrapolated to a 3-year residency, this number appears consistent with their findings.

For procedures, 14% of residents completed less than the 25 procedures required in F-4. Thoma et al. found that the Canadian average for F-4 EPA completion was 19 out of 25.[12] Specific procedures that were difficult to complete were I.O., arthrocentesis, and anterior nasal packings with a median of zero for each. Hayden et al. found that during a 3-4 year American EM residency, the mean number of arthrocentesis was 11 (95% CI 8,14) and mean number of I.O. insertions 2 (95% CI 1,3).[8] Their data is in line with our finding that these are low frequency procedures.

Our study, and a recent study by Thoma et al., underscore the importance of monitoring EPA collection against arbitrary targets set by the Specialty Committee.[12] We did not look at barriers of access, though we could hypothesize the following as possible contributors: low encounters in the clinical setting, lack of junior resident involvement by more senior colleagues, high learner ratios competing for the same cases, and resident lack of confidence in seeking these cases. Regardless, with insufficient numbers of resuscitations and procedures to support the current EPAs in TD-1 and F-1, residency program curricula should be reviewed and modified as needed to ensure adequate clinical experiences are available.

In response to the finding of our study, we have made a number of program modifications. We considered the following two questions: Is it reasonable to expect procedural competence to only be measured in one stage of residency? Will achievement of competence in one stage ensure adequate competence is attained and maintained? In answer to these questions, we have allowed our residents to obtain stage-specific procedural EPAs in all stages, including allowing of F-4 procedural EPAs outside of the F stage alone.

Another modification was the inclusion of an TD-1 centered simulation OSCE at the end of TD to supplement, not replace, EPA data. This enables our CC to have another layer of data in making promotion decisions for a stage that is only 3 months long. Simulation training has been shown to be beneficial in resident training improving competence and self-reported comfort levels in resuscitations and this may be a good option for clinical situations that are infrequently encountered.[12] However, simulation is in itself a resource intensive educational activity, and it remains unclear to what extent it can replace clinical setting experiences. Using simulation as an assessment tool is a departure from the formative nature of simulation programs across the country and warrants further discussion.[12]

Lastly, our Competence Committee has created a procedure log whereby we track the procedures completed by our residents during different stages and it continues to regularly monitor EPA collection to inform iterative curriculum modifications and changes.

Limitations

Despite our efforts at prospective collection, we cannot be certain how many cases were under-reported and if our results represent a true lack of experiences. This issue is also described in Dire et al. where they estimated 75% logging compliance in their study.[7] A similar publication by Langdorf et al. that used logbooks to track resident procedures estimated only 60% of procedures were recorded.[9] It is also possible that under-reporting of a given category of resuscitation has occurred because some cases have been logged into another category because of overlap in clinical presentation (e.g. I.O. as part of a resuscitation).

We did not set out a priori to look at variation in experiences amongst sites, or to correlate these numbers with types of rotations. Further, the ratio of male to female trainees was not equal in our study and the difference in clinical experiences by sex was not specifically investigated.

Finally, our data collection occurred in the year prior to the switch to CBD in most Canadian FRCP-EM programs except at Queen’s where CBD had already been implemented. As a result, most residents did not target their experiences to meet the requirement of specific EPAs. With added attention paid to EPAs by current CBD cohorts it is conceivable that residents would be more active in seeking out low-event procedures and resuscitations to meet minimums, however this was not substantiated in the post-CBD implementation study by Thoma et al.[12]

Conclusion

Despite the cited limitations, this study provides a rare view into the clinical landscape available to EM trainees. We showed that certain resuscitation and procedural EPAs are low frequency clinical experiences that can affect EPA collection and limit meeting the requirements of the CBD curriculum with focus on quality and safe patient experiences. In the era of CBD, we suggest iterative monitoring and adaptations at the Speciality Committee and local levels to address the misalignment of EPA target requirements and curriculum delivery. This is consistent with the broad evolutionary principles laid out at the outset of CBD implementation by the Specialty Committee.[14]