A Resident-Authored, Case-Based Electrocardiogram Email Curriculum for Internal Medicine Residents.

Introduction: The interpretation of electrocardiograms (ECGs) is a critical competency for internal medicine trainees, yet time and resources to foster proficiency are limited.
Methods: This resident-authored ECG email curriculum for first-year residents involved 129 first-year internal medicine residents at three major academic university hospitals. Residents either received the resident-authored ECG email curriculum (intervention group) or continued standard training (control group). The curriculum involved 10 multiple-choice ECG cases emailed biweekly over the 6-month study period. All participants were asked to complete a pre- and postintervention test to assess ECG interpretation competency and attitudes. The primary outcome was improvement in ECG test performance.
Results: Among the 129 first-year residents participating, 21 of the 65 (32%) randomized to the intervention group and 13 of the 64 (20%) randomized to the control group completed both the pre- and posttests for analysis. While all participants' ECG test scores improved over the study period (p < .001), improvement did not differ between groups (p = .860). We found that the effect of the intervention on ECG test performance varied significantly by the number of cardiology rotations an intern experienced (p = .031), benefiting naïve learners the most. All intervention group participants who completed the posttest reported they would recommend it to a colleague. Discussion: While it did not improve resident performance on an ECG posttest, this resident-authored ECG email curriculum offers a scalable way to provide trainees additional practice with ECG interpretation, with particular benefit to trainees who have not yet rotated on cardiology.

RCT Entities:   Population Interventions Outcomes

Educational Objectives

By the end of this activity, learners will be able to:

Demonstrate a systematic approach to interpreting electrocardiograms.

Analyze and differentiate between several electrocardiogram findings and diagnoses.

Identify the appropriate next step in management of several electrocardiogram diagnoses.

Introduction

In an era where the mastery of increasingly complex clinical competencies must be balanced with the limitations of resident work hour restrictions and finite faculty resources, it is difficult for residency programs to prioritize structured learning on every needed topic. One important example is electrocardiograms (ECGs).

Prior studies have demonstrated low proficiency in interpreting ECGs and self-perceived confidence among internal medicine residents.[1] Given that internal medicine residents are often first responders within the hospital, it is critical that all residency programs ensure competence with ECG interpretation. Others have called for more ECG training among residents but without clear guidance on how to incorporate additional training into saturated resident schedules and curricula.[2]

This widely recognized need for additional training in ECG interpretation is not unique to internal medicine training. A search of MedEdPORTAL identified six other curricula that have been designed and implemented to help address this need, targeting different stages of training and different specialties.[3-8] Roberts[7] and Zachow, Coromilas, and Sadley[6] offer curricula designed for preclinical medical students that both cleverly aim to integrate the learning of cardiac physiology and anatomy with ECG interpretation. Meanwhile, Weinberg, Ottolini, Sestokas, and Greene[5] and Gopwani, Patel, Greene, and Chapman[3] describe online, computer-based curricula, similar to the curriculum we describe below, but designed for pediatric residents and pediatric emergency medicine fellows, respectively. Kutzin, Milligan, and Justiniano[8] describe a more resource-intensive, hands-on curriculum using simulation to teach the diagnosis and management of arrhythmias to hospital staff. Finally, Goldberger[4] describes an excellent web-based resource for ECG self-assessment and learning with a large number of cases of varying difficulty, potentially useful to anyone wanting to improve their competency with ECGs. Our curriculum adds to this body of work by offering the first resident-authored ECG email curriculum designed for graduate internal medicine trainees with an emphasis on scalability.

We developed a resident-authored, self-directed, and interactive ECG email curriculum and evaluated the effectiveness of this curriculum among first-year residents in internal medicine across three major academic programs.

Methods

Curriculum

The curriculum consisted of 10 targeted ECG cases, each including a brief clinical vignette, representative ECG, and an associated multiple-choice question regarding diagnosis and management. We chose cases with guidance from the American College of Physicians’ Medical Knowledge Self-Assessment Program and based on clinical relevance to the internal medicine trainee (Appendices A–J). The resident authors curated and deidentified the ECGs used in the cases from actual patients at the University of Pittsburgh Medical Center. The resident authors researched and wrote the cases and case solutions. The faculty author, who was board-certified in cardiology, evaluated the cases and solutions for content and accuracy. We estimate that resident time spent was 6 hours total per case, including writing and revision; faculty time was less than 1 hour per case.

Study Design and Participants

We conducted a randomized controlled trial over 6 months from June to December of 2016 at three major academic internal medicine residency programs. All 129 categorical internal medicine and medicine-pediatric first-year residents (56 from the University of Pittsburgh Medical Center, 43 from Indiana University, and 30 from the University of Rochester Medical Center) were eligible to participate. We excluded preliminary and transitional first-year residents due to the difficulty in obtaining the necessary permissions from additional program directors for inclusion. We randomized participants using a random number generator to either receive the ECG email curriculum (intervention group) or continue standard training (control group). Participation was voluntary, and participants were given the opportunity to opt out. The study was deemed exempt from full review by the institutional review board (IRB) at each participating institution.

The intervention group received one case via email every 2 weeks (first page of Appendices A–J) from a dedicated Gmail account (beatblasts@gmail.com). We encouraged participants to voluntarily reply to the emails with their answer. Participants who responded received an automatic reply email with the case solution, which included bulleted key learning points, detailed ECG interpretation (“Solution Email” section of Appendices A–J), and a link to a detailed explanation of the case (“Link to more information” section of Appendices A–J). If clicked, the link to a more detailed explanation led to a cloud-based document hosted by the dedicated Gmail account. Regardless of reply, we sent the case solution to all participants in the intervention group after 2 weeks, prior to the next case. We explicitly asked participants in the intervention group not to share the cases with participants in the control group to minimize cross-contamination. Upon completion of the 6-month study period, we shared the curriculum with all 129 first-year residents to ensure equipoise.

Measurements

We asked participants to complete an anonymous online test through a third-party survey tool (Google Forms) prior to the start and at the end of the 6-month study period (Appendix K). The test was modeled after previously published ECG assessments.[1] On the test, we asked participants to examine 10 ECGs and identify the primary diagnosis (e.g., atrial fibrillation) with any other associated findings (e.g., left atrial abnormality). For each ECG, we asked participants to answer a multiple-choice question regarding further management based upon their ECG interpretations. We scored the tests by awarding up to 2 points for the correct interpretation and up to 1 point for the correct multiple-choice answer, for a total possible score of 30 points, as outlined in the facilitator's guide (Appendix L). Two board-certified cardiologists independently interpreted the ECGs with 100% concordance to establish the correct diagnoses. Finally, we included survey questions with both the pretest and posttest to assess participant learning preferences, barriers to further ECG learning, satisfaction with the curriculum, and confidence in routine ECG interpretation and management using a 5-point Likert scale (Appendix M). The pre- and posttests were identical to ensure that they were of equal difficulty. Since these tests were separated by the first 6 months of residency, we viewed the potential impact of pretest recall on posttest performance as being low. The tests were uniquely coded for each participant to allow for direct comparison. We incentivized participants to complete the pre- and posttests with five ∃50 gift cards awarded by lottery at each site.

Our primary outcome was improvement in ECG test performance between the intervention and control groups. Our secondary outcomes included overall improvement in ECG test performance, self-directed learning preferences, major barriers to further ECG learning, satisfaction with the curriculum, and the effect of self-reported resident interest in cardiology, self-reported resident confidence in ECG interpretation, and self-reported number of cardiology rotations on improvement in ECG test performance.

Statistical Analyses

We used descriptive statistics to characterize study participants and constructed linear regression models to perform our analysis. Only participants with both pre- and posttest data were included in the analysis of the primary outcome.

For our primary analysis, we constructed simple linear regression models with improvement on the ECG competency test as the outcome and intervention group versus control group as the predictor. For our secondary analyses, we constructed multivariable regression models by augmenting the simple linear regression model to include number of rotations, group-by-rotations interaction, and baseline interest level as predictors. We deemed additive and/or multiplicative effects of predictors on improvement scores significant based on two-sided t tests of their respective regression coefficients. The assumptions of linear regression were checked after fitting the models. Residuals were approximately normally distributed and homoscedastic, no deviations from linearity were observed, and observations were assumed to be independent. We conducted statistical analyses using Stata SE version 14.2 (StataCorp) with a 5% significance level and no adjustments for multiplicity.

Results

Participation

Among the 129 first-year residents included, 65 were randomized to the intervention group, and 64 were randomized to the control group. A total of 83 of 129 (67%) participants completed the pretest. In the intervention group, 25 participants (38%) completed the posttest, with 21 (32%) of those participants having previously completed the pretest for comparison. In the control group, 16 participants (25%) completed the posttest, with 13 (20%) of those participants having previously completed the pretest for comparison. There were no significant differences in baseline characteristics between those in the intervention group and the control group who completed both assessments (Table 1).

Table 1.

Comparison of Baseline Characteristics and Number of Cardiology Rotations Among Participants Who Completed Both the Pre- and Posttests in Each Group

In the intervention group, a mean of 26 (40%) participants actively responded to each case; participants reported reading an average of 7.81 (SD = 2.60) of the 10 cases and solutions. Participants reported utilizing the link to more information on the case for an average of 4.16 of the 10 cases, but with marked variation (SD = 3.70).

Pre- and Posttests

Overall, all participants’ ECG test scores significantly improved over the study period (p < .001). The improvement among the intervention group was not significantly greater than the control group (p = .860; Figure 1).

Figure 1.

Performance on pretest and posttest in intervention and control groups. Overall test scores significantly improved (p < .001), but there was no significant difference in improvement between the two groups (p = .860). The 13 participants in the control group who completed both the pre- and posttests scored a mean score of 14.2 and 17.4, respectively, out of a total possible score of 30 points, with a mean improvement of 3.2 points. The 21 participants in the intervention group who completed both the pre- and posttests scored a mean score of 14.1 and 17.1, respectively, with a mean improvement of 3.0 points. Error bars indicate 95% confidence intervals.

We found that the effect of the intervention on ECG test performance varied significantly by the number of cardiology rotations an intern experienced (p = .031; Figure 2), benefiting those who had not yet rotated on cardiology the most. No difference was seen in the mean reported ECG confidence levels among those in the intervention group compared to the control group (p = .174). Baseline interest level in cardiology did not individually correlate with a greater degree of improvement in ECG test score (p = .393). Among those in the intervention group, self-reported participation in the curriculum was not associated with improved ECG test scores, regardless of self-reported number of cases read (p = .393) or number of cases responded to (p = .999).

Figure 2.

Variation in improvement in the intervention group (marked by xs and solid trend line) and control group (marked by open circles and dotted trend line) on the posttest by number of cardiology rotations. Among participants who completed both the pre- and posttests, nine residents did not rotate on cardiology, 18 residents rotated on cardiology once, and seven residents rotated on cardiology twice during the study period. The curriculum led to the greatest improvement for those who did not rotate on cardiology during the study period (p = .031). Error bars indicate 95% confidence intervals.

Learner Barriers, Preferences, and Satisfaction

Learning barriers to and preferences for ECG learning were similar across all three institutions, favoring interactive, online, and self-directed resources (Table 2). On the posttest, 25 of 25 (100%) participants in the intervention group who completed the postsurvey reported they would recommend it to another trainee and reported learning from the curriculum. Specifically, 25 of 25 (100%) residents reported the curriculum was instructive in ECG interpretation, and 24 of 25 (96%) reported it was instructive in diagnostic and management next steps, while 10 of 25 (40%) reported it provided a helpful review of the current evidence surrounding ECGs.

Table 2.

Learning Preferences and Barriers to ECG Learning by Site (Participants Were Able to Choose More Than One Answer to Each Question)

Discussion

To our knowledge, this is the first multi-institutional randomized trial of a resident-authored email curriculum designed for graduate internal medicine trainees. We found that there was no significant difference in improvement in ECG test performance regardless of whether a participant was randomized to receive our curriculum (p = .860). However, all of the participants demonstrated significant improvement regardless of whether they were in the intervention group. While self-reported interest in cardiology and confidence in ECG competence did not correlate with ECG test performance, the number of cardiology rotations did (Figure 2). In the intervention group, improvement was highest for participants who had not yet experienced a cardiology rotation. While Figure 2 appears to show worsening performance in the intervention group for those who experienced multiple cardiology rotations, the decrease in estimated mean improvement was not statistically significant. Rather, a more accurate interpretation is that while observed improvement was greatest for cardiology-naïve participants, any apparent difference in improvement diminished once they had rotated on at least one cardiology rotation. In the control group, improvement increased as the number of cardiology rotations during the study period increased.

Our study demonstrates the feasibility and scalability of a curriculum that addresses a vital clinical competency with minimal expense. Authored by residents, this curriculum leverages developing resident aspirations in cardiology and medical education to help address an underemphasized and important area in resident training while minimizing necessary faculty time. Furthermore, our curriculum is consistent with our participants’ reported learning preferences, which favored interactive, online, and self-directed resources (Table 2). While the posttest must be interpreted within the limitations of our response rate (38%), all participants in the intervention group who completed the posttest reported they had learned from the curriculum and would recommend it to a colleague.

Several limitations may help to explain why we did not see greater improvement in posttest scores as a result of this curriculum. Participant response rate on the posttest was low despite incentives. This could be attributed to the time commitment required: approximately 45–60 minutes to complete each test. Response rates may have also been affected by concerns about the anonymity of answers, despite our assurances to participants. Our results may have been confounded by participants in the intervention group sharing cases with participants in the control group, although this was explicitly discouraged. We do not know whether learning from the curriculum or parts of the curriculum were shared between those in the intervention and control groups. Furthermore, our observation that the effect of the curriculum varied significantly with the number of cardiology rotations that a learner experienced may suggest that the effect size of our curriculum on ECG interpretation was not able to be detected over the impact of rotating on cardiology.

This observed benefit for cardiology-naïve residents suggests that our curriculum would be helpful early in training, priming residents prior to rotating on cardiology or medical students prior to starting residency. This type of curricular model offers a promising approach to enhancing the training of other underemphasized clinical competencies, particularly those that residents may not otherwise gain focused instruction on during their day-to-day training (e.g., chest radiographs or blood smears). Additional studies are needed to continue to explore the applicability of this curricular model, both in this setting and with other topics and learners.

We learned many lessons in the development and evaluation of this curriculum. Online, self-directed resources are popular among our trainees, particularly when the content is targeted and appropriately distributed in time. Our larger, lottery-based incentives were not very effective at motivating participation, particularly in a 1-hour challenging ECG competency test. In retrospect, it might have been more effective to incentivize participation with smaller, guaranteed incentives to all participants. We suspect participation in the pre- and posttests would have been further increased if we had secured protected time for residents to participate. With competing demands for time and curriculum, the sponsorship and support of program leadership are essential. This is particularly true when attempting to evaluate a curriculum at multiple specialties (internal medicine and medicine-pediatrics) and multiple institutions. It would have been impossible to do this without the support of the program directors and one resident champion at each site. In addition, while we obtained IRB approval from each clinical site, the coordination of this was challenging. We have since learned that educational studies can often be safely done at multiple institutions with the approval of one IRB, although this may be different depending on the institutions involved and the curricular intervention.

There is an ongoing imperative in medical education to condense large amounts of necessary clinical knowledge and competencies into a finite amount of time with a limited amount of resources. ECGs are one important example. This resident-authored ECG email curriculum offers a learner-centered, inexpensive, and scalable way to provide trainees additional practice with ECG interpretation and management.

Case 1 Right Ventricular Infarct.docx

Case 2 Supraventricular Tachycardia.docx

Case 3 Pericarditis.docx

Case 4 Left Ventricular Hypertrophy.docx

Case 5 Infective Endocarditis.docx

Case 6 Hyperkalemia.docx

Case 7 Limb Lead Reversal.docx

Case 8 Wenckebach.docx

Case 9 Ventricular Tachycardia.docx

Case 10 Posterior Myocardial Infarction.docx

ECG Competency Test.docx

ECG Competency Test Facilitator Guide.docx

Pre- and Postcurriculum Surveys.pdf

All appendices are peer reviewed as integral parts of the Original Publication.