Introduction of an academic medical center's point-of-care ultrasound curriculum to internal medicine residents at a community-based teaching hospital.

BACKGROUND: Despite its proven utility, integration of point-of-care ultrasound (POCUS) into internal medicine (IM) residency training has been inconsistent. Due to their unique constraints, community-based teaching hospitals may face particular challenges in providing POCUS training to IM residents.
OBJECTIVES: To evaluate short-term educational outcomes of an academic center's POCUS curriculum following its adaptation and delivery to IM residents at a community-based teaching hospital.
METHODS: A needs assessment (NA) regarding POCUS training was distributed to PGY-2 and PGY-3 IM residents at a community-based teaching hospital in 2017. Based on the NA results, a POCUS curriculum from an academic center was modified and a revised course was offered to the same residents. Participants completed cognitive assessments before and after three of the four didactic sessions. Observed placement of an ultrasound-guided peripheral IV before and after the training program comprised the skills assessment.
RESULTS: 17 of 28 (61%) residents completed the NA; eleven participated in the course. Of 33 possible quiz pairs, 15 (45%) were completed. Average quiz scores rose after the first and third sessions. Skills assessment scores increased after course completion.
CONCLUSION: Adaptation of POCUS curricula from academic centers may be a feasible instructional strategy for community-based IM residency programs.

Introduction

Point-of-care ultrasound (POCUS) is an established diagnostic modality in the training of some medical specialties, such as emergency medicine [1]. However, its integration into the training of internal medicine (IM) physicians is less uniform despite its demonstrated utility in clinical decision-making and common procedures [2]. A 2012 survey of American IM residency program staff showed that about 25% of responding programs had existing integrated POCUS curricula [3]. Calls for integration of POCUS into IM training preceded [4] both this survey and the release of milestones for IM residency training by the Accreditation Council for Graduate Medical Education (ACGME); however, no POCUS-related milestones were issued. In essence, there is room for significant growth for POCUS training within IM.

Notably, descriptions of full POCUS curricula for IM residents generally emanate from academic medical centers [5-7] and a roadmap for development of these programs cited four curricula also developed at academic centers [8]. In contrast, reports of POCUS training courses at community-based teaching hospitals tend to focus on a single POCUS application such as renal sonography [9], musculoskeletal ultrasound [10], and cardiovascular ultrasound [11]. As articulated by Sabath and Singh in their call to add educational objectives specific to POCUS to the ACGME’s milestones for IM training [12], community hospitals face distinct challenges in adding POCUS curricula to IM residency education. Smaller numbers of housestaff and fewer faculty members can present logistical challenges to introduction of a POCUS training program, as local faculty expertise may not be available to deliver content material and supervise resident learning. Costs related to equipment and lost faculty productivity may present additional obstacles, which can be relatively more impactful at smaller hospitals. Additionally, the smaller pool of residents may limit programs’ abilities to schedule POCUS courses.

In recognition of these specific challenges, this pilot study sought to evaluate short-term educational outcomes of a POCUS course offered to IM residents at a community-based teaching hospital; the curriculum was tailored to this setting from its original form at an academic medical center. Outcomes of interest included short-term changes in knowledge, skills and attitudes among the program participants.

Materials & methods

A needs assessment (NA) questionnaire was developed using the Qualtrics platform (Provo, UT). The questionnaire asked respondents about their general attitudes towards the clinical utility of POCUS, their familiarity with POCUS and their interest in undergoing POCUS training. Respondents were also asked to assign educational value to specific POCUS applications. Questions were multiple-choice in structure with unipolar Likert scales as answer choices. Prior to any teaching activity, the NA was distributed to all second-year and third-year IM residents at Mount Auburn Hospital, a community-based teaching affiliate of Harvard Medical School. Participants were allowed three weeks to submit their responses, and received a weekly email reminder about the survey.

Details regarding the source curriculum, which was developed at Boston University School of Medicine, were reported at the annual meeting of the American College of Chest Physicians in 2015 [13]. Cognitive assessments and a skills assessment were included to evaluate the efficacy of the curriculum.

After the modified curriculum was finalized, all second-year and third-year IM residents were invited by email to participate in the course. As comprehension of introductory material was critical for subsequent sessions, eligibility to participate was limited to those able to attend all four sessions of the course. The course was offered twice to allow participation by a greater number of residents. Informed consent was obtained from all course participants. Lectures and simulation sessions were led by the pulmonologist who developed the source curriculum and a hospitalist on staff at the study site. Three ultrasound platforms were used during the course. Practicum sessions consisted of supervised use of POCUS techniques by course participants on other participants, as well as simulated procedures using mannequins.

Before and after the first, third and fourth course sessions, participants received cognitive assessments via the Qualtrics platform that evaluated their knowledge regarding the content delivered in each lecture. Similarly, for the skills assessment, participants volunteered to attempt ultrasound-guided placement of a standard peripheral intravenous catheter (US-PIV) into a vascular access simulator before and after participating in the course. Their technique was scored utilizing a procedural performance checklist specific for this activity [14].

All surveys and forms were approved by the Mount Auburn Hospital Institutional Review Board. Cognitive assessments were paired; the last four digits of each participant’s telephone number were used as unique identifiers to allow for repeated-measures analysis. Only complete pairs of cognitive assessments were included in the analysis.

All statistical analyses were performed using SPSS (IBM Corporation, Armonk, NY). The Wilcoxon Signed Rank test was used to compare continuous data in the repeated-measures design, and McNemar’s test was used to compare categorical variables using a repeated-measures design; a p value ≤0.05 was considered to be significant.

Results

Needs assessment

The NA was completed by 17 of the 28 second-year and third-year residents for an overall response rate of 61%. A combined majority of respondents rated themselves as slightly familiar (8/17; 47%) or moderately familiar (7/17; 41%) with POCUS. Nearly all respondents rated themselves as not at all comfortable (7/17; 41%) or somewhat comfortable (9/17; 53%) with actual use of POCUS techniques. Interest in POCUS training was reported by 94% of participants. With respect to perceived importance of POCUS, a majority of participants identified POCUS training as important (8/17; 47%) or extremely important (8/17; 47%). Respondents’ rankings regarding the educational value of specific POCUS applications is represented in Figure 1.

Figure 1.

Ranked values of selected POCUS applications among NA respondents.

Curriculum modification & course participants

Based on the responses supplied in the NA as well as the limited time available for instruction, the source curriculum was adjusted. Typically delivered over a combined 36 hours of lecture, simulation and bedside time, the source curriculum was modified to consist of 8 hours of lecture and simulation, to be delivered over four sessions. Topics reviewed during the sessions included knobology, pleural & pulmonary ultrasound, vascular access, inferior vena cava (IVC) visualization and the extended focused assessment with sonography for trauma (eFAST). A schematic of the modified curriculum is shown in Table 1. Transthoracic echocardiography, hepatobiliary ultrasound and renal ultrasound were excluded from the modified curriculum.

Table 1.

POCUS curriculum schema.

Day 1		Day 2		Day 3		Day 4
Duration	Activity	Duration	Activity	Duration	Activity	Duration	Activity
1.5 hours0.5 hours	Lecture:Ultrasound physics & ‘knobology’Practicum:Probe orientation & image optimization	1 hour1 hour	Lecture:Pleural & pulmonary ultrasoundPracticum:Supervised scanning of course participants	1 hour1 hour	Lecture:Vascular access techniquesPracticum:Simulated US-PIV & US-CVC on models	1 hour1 hour	Lecture:eFAST & IVC visualizationPracticum:Supervised scanning of course participants

The POCUS curriculum consisted of four two-hour sessions. Each session consisted of a didactic lecture and a practicum.

Eleven residents participated in the POCUS course; the first iteration consisted of five participants, while the second iteration consisted of six participants. The course participants included five second-year residents and six third-year residents. Four residents were graduates of American medical schools while seven had undergone undergraduate medical education abroad.

Cognitive assessments

Only one participant responded to all six cognitive assessments. The remainder of the participants responded to some, but not all, of them. Of the 33 possible assessment pairs, 15 pairs were retrieved. Of the 66 disseminated cognitive assessments, 45 were returned. The average scores of the five cognitive assessment pairs before and after the first session were 49% and 65%. The average scores of the seven cognitive assessment pairs before and after the third session were 33% and 68%. The average scores of the three cognitive assessment pairs before and after the fourth session were 39% and 28%. Wilcoxon Signed Rank analysis of the paired cognitive assessments showed improvement in the average score for the first and third course sessions with p < 0.05, as shown in Figure 2(a).

Figure 2.

Average scores of (a) cognitive assessments & (b) skills assessment among participants before and after POCUS curriculum.

Skills assessment

Five residents participated in the skills assessment. Participants in the skills assessment returned within 30 to 64 days after course completion to repeat the exercise. The average number of checklist items achieved before and after the course were 4.2 and 9, respectively. Wilcoxon Signed Rank analysis of the scored checklists showed improvement in the average checklist score with p < 0.05, as shown in Figure 2(b).

Course evaluation

All eleven residents completed a course evaluation after its conclusion. Increases in participants’ familiarity with POCUS as well as comfort with POCUS were reported with p < 0.01 for both comparisons, as shown in Figure 3.

Figure 3.

Participants’ ratings of (a) familiarity & (b) comfort before and after POCUS curriculum.

Discussion

A majority of residents responded to the NA. Participants generally felt that POCUS training was important and reported some familiarity with the clinical role of POCUS. However, they rated lower levels of confidence in their own ability to use POCUS at the bedside. High levels of interest in POCUS training were observed, consistent with the report by Elhassan et al. [15]. Vascular access applications such as US-PIV and US-CVC were ranked highest, suggesting that IM residents value procedural POCUS applications more than diagnostic POCUS applications. As suggested by Keddis et al. [16]. IM residents may value procedural POCUS applications due to a recognition that use of POCUS is rapidly becoming standard of care, and also improves procedural safety.

Among the smaller group of residents who underwent the POCUS curriculum, improvements in knowledge were demonstrated by serial cognitive assessments. Scores on these assessments improved after the first session and the third session, suggesting that the course was effective in teaching these topics. The improvement in scores noted after the first session and third session was not observed after the fourth session. A few reasons may have contributed to this finding. The topics covered during the fourth session consisted of POCUS applications ranked lower in importance on the NA. Also, the raw number of cognitive assessments received after the fourth session was lower than the number of cognitive assessments completed by the residents after the preceding sessions. Finally, the presentation and delivery of the curriculum during the fourth session may have been relatively less effective. Meanwhile, the improvement in skills assessment scores suggests that, among the course participants who also took part in the skills assessment, true skills acquisition was achieved by the modified curriculum.

Attitudinal changes were observed among the course participants; both familiarity with POCUS as well as comfort with its use increased. Similar attitudinal changes were reported by Schnobrich et al. in their study of a POCUS training course for first-year IM residents at an academic medical center [6].

Although the course discussed in this study was brief and included relatively few participants, it did emphasize that interest in POCUS among IM residents at community-based teaching hospitals is high, and that positive changes in knowledge and skills are achievable. Other authors have noted that a brief POCUS training program can demonstrate lasting benefit for learners [17]. Ideally, future iterations of this program would be longitudinally integrated into the residency curriculum, not only developing residents’ familiarity and comfort with POCUS, but also ensuring their exposure to its clinical benefits. Of note, some additional features of this POCUS curriculum may bear relevance in real world settings; the use of relatively inexpensive portable ultrasound technology and the presence of only two instructors may reflect the realities of community-based teaching hospitals nationwide.

Several limitations to this study are readily apparent. The curriculum was only offered at one center, and the number of participants was small. The voluntary nature of the program and the restriction of course participation only to those who could attend all sessions further contributed to the potential for selection bias. To mitigate the effect of the small study size, nonparametric statistical testing was used. Another salient limitation was the quantity of missing data, as only 45% of possible cognitive assessment pairs were received. In addition, while the skills assessment tool was a published, consensus-generated instrument, the cognitive assessments had been applied only in the context of this program and the source curriculum. Finally, only short-term changes in knowledge and skills were evaluated, and first-year IM residents were not included in the curriculum.

The results of this pilot study show promising short-term improvements in knowledge, skills and attitudes among IM residents in this adapted POCUS course at a community-based teaching hospital. Overall, larger studies investigating long-term outcomes are needed to better understand whether POCUS curricula from academic medical centers can be successfully tailored for use by smaller community-based residency programs.