A Distance-Learning Approach to Point-of-Care Ultrasound Training (ADAPT): A Multi-Institutional Educational Response During the COVID-19 Pandemic.

PROBLEM: The COVID-19 pandemic significantly disrupted point-of-care ultrasound (POCUS) education. Medical schools and residency programs placed restrictions on bedside teaching and clinical scanning as part of risk mitigation. In response, POCUS faculty from 15 institutions nationwide collaborated on an alternative model of ultrasound education, A Distance-learning Approach to POCUS Training (ADAPT). APPROACH: ADAPT was repeated monthly from April 1 through June 30, 2020. It accommodated 70 learners, who included 1- to 4-week rotators and asynchronous learners. The curriculum included assigned prework and learning objectives covering 20 core POCUS topics. A rotating group of 30 faculty and fellows delivered daily virtual teaching sessions that included gamification to increase learner engagement and hands-on instruction through teleguidance. After participation, faculty and learners completed anonymous surveys. OUTCOMES: Educators reported a significant decrease in preparatory time (6.2 vs 3.1 hours per week, P < .001) dedicated to ultrasound education after implementing ADAPT. The majority of 29 learners who completed surveys felt "somewhat confident" or "very confident" in their ability to acquire (n = 25, 86.2%) and interpret (n = 27, 93.1%) ultrasound images after the intervention; the majority of 22 educators completing surveys rated the program "somewhat effective" or "very effective" at contributing to learner's ability to acquire (n = 13, 59.1%) and interpret (n = 20, 90.9%) images. Most learners (n = 28, 96.6%) and all educators (n = 22, 100%) were "satisfied" or "very satisfied" with ADAPT as a whole, and the large majority of educators were "very likely" (n = 18, 81.8%) to recommend continued use of this program. NEXT STEPS: A virtual curriculum that pools the efforts of multiple institutions nationwide was implemented rapidly and effectively while satisfying educational expectations of both learners and faculty. This collaborative framework can be replicated and may be generalizable to other educational objectives.

Problem

The COVID-19 pandemic disrupted traditional models of medical education. [1] Social distancing mandates and clinical staffing concerns led to a rapid transition from bedside education to virtual platforms or outright cancelation of training experiences. [1,2] Point-of-care ultrasound (POCUS) is integral to the practice of multiple specialties and a required skill for emergency medicine (EM) residents. [3] Core POCUS competencies include medical knowledge, psychomotor skills for image acquisition, image interpretation, and integration of findings into clinical decisions. [3] Medical knowledge can be acquired through asynchronous methods such as modules developed by national societies and peer-reviewed blogs or podcasts. [4,5] However, it is unknown whether the remaining competencies can be achieved without in-person expert guidance. [3] During a time when hands-on instruction is limited, educators are challenged to identify novel methods in providing POCUS education.

To address these demands, a coalition of POCUS experts from 15 academic institutions nationwide collaborated on creating and executing an interactive virtual curriculum, A Distance-learning Approach to POCUS Training (ADAPT). This distributed the educational workload that would have otherwise been placed upon each individual program. Remote teaching of didactic and hands-on ultrasound skills has been previously described. [2,6-9] ADAPT is unique in its scope as there is no comprehensive, multicenter, remote POCUS training curriculum that addresses both the cognitive and hands-on components of POCUS education. We describe the development of ADAPT, the perceptions of its participants, and lessons learned during implementation. This curriculum can inform others as they adapt to the new educational environment created by the COVID-19 pandemic.

Approach

Design and implementation

In March 2020, we sent an invitation via targeted emails and via the American College of Emergency Physicians Emergency Ultrasound Section listserv proposing a collaboration in response to the challenges posed by COVID-19. Fifteen EM residencies expressed interest: Carolinas Medical Center, Dartmouth-Hitchcock Medical Center, Emory University School of Medicine, Harvard Medical School, Johns Hopkins University School of Medicine, Mayo Clinic Alix School of Medicine, New York-Presbyterian Brooklyn Methodist Hospital, Resurrection Medical Center, St. Barnabas Hospital Health System, Stanford University School of Medicine, University of Cincinnati College of Medicine, University of Florida at Jacksonville College of Medicine, University of North Carolina School of Medicine, University of Texas Southwestern Medical Center, and Yale University School of Medicine.

Thirty ultrasound-trained faculty and fellows finalized curriculum development and teaching format. The curriculum mirrored flipped classroom design and took advantage of available free open access medical education resources. Educators developed lesson plans and learning objectives for each topic. A shared document of the curriculum was made available to all participants and included daily logistics, prework, and learning objectives (see Supplemental Digital Appendix 1, at http://links.lww.com/ACADMED/B181). ADAPT ran from April 1 to June 30, 2020, and targeted first-year EM residents on their 1- to 4-week POCUS rotations. Similar to traditional POCUS rotations, ADAPT included a blended audience of fellows, off-service residents, and medical students. Before each virtual session, learners were assigned prework consisting of peer-reviewed educational blogs, videos, and journal articles. Students received live, daily 2-hour teaching sessions delivered via Zoom. In addition, Yale University invited learners to participate virtually in their weekly image review quality assurance (QA) sessions.

Educators used various teaching methods including gamification and teleguidance to increase engagement. Gamification included online quiz platforms such as Kahoot!, Jeopardy-structured teaching, peel-and-reveal games, and competitive challenges. Interested programs were loaned Butterfly iQ (Butterfly Network Inc., Guilford, Connecticut) handheld ultrasound devices (HUDs) that connected to a smartphone or tablet. Learners practiced scanning on themselves while a faculty observed and critiqued their image acquisition technique via teleguidance (Figure 1). Studies were saved to the Butterfly Cloud, enabling shared image QA across institutions.

Figure 1

Comparison of ADAPT curriculum image review and feedback with traditional POCUS training. Designed to replace EM POCUS rotations during the COVID-19 pandemic, ADAPT is distinguished by preparatory work, virtual sessions, supplemental learning for learners, and teleguidance. During traditional image review, learner ultrasound images are reviewed asynchronously by faculty and comments are provided post hoc, creating limited opportunity for teaching and learning real-time image optimization. With teleguidance, learners scan themselves or others while an educator observes and provides live feedback on techniques and interpretation, allowing robust and personalized learning. Abbreviations: EM, emergency medicine; POCUS, point-of-care ultrasound; ADAPT, A Distance-learning Approach to POCUS Training.

Curricular adjustments were made based on real-time participant feedback including adjusting schedules to accommodate time zones, refining learning objectives, curating relevant prework, and integrating engaging strategies. Topic generation, communication of expectations, and updates were driven by a handful of core faculty who maintained curricular structure.

Survey and data analysis

We developed 2 surveys with the assistance of a department statistician who reviewed for clarity, relevance, and topic coverage. The surveys were pretested on educational leadership including content review and cognitive interviewing. Afterward, they were piloted using convenience sampling of ultrasound section faculty and residents at a single institution. The final surveys for learners and faculty included 20 and 18 questions, respectively (Supplemental Digital Appendices 2 and 3, at http://links.lww.com/ACADMED/B182 and http://links.lww.com/ACADMED/B183). Both surveys included questions about participant demographic characteristics (e.g., region of the country, year in training or practice), engagement, confidence in ability to acquire and interpret images, and satisfaction with ADAPT. The faculty survey also asked the amount of time spent in weekly preparation before and after curricular implementation. Surveys were sent by email on the last day of each month. We gave respondents 1 month to complete the survey and sent reminders biweekly. No incentives were offered for completion.

This prospective, educational intervention research was reviewed and granted exempt status by the Stanford University Institutional Review Board. We collected deidentified data using the web-based database Research Electronic Data Capture (REDCap). Survey data were reported using descriptive statistics. Chi-squared tests were used to evaluate for differences in learner comfort in acquiring and interpreting images among those who did and did not perform clinical scanning. A Wilcoxon signed-rank test was used to compare educator hours dedicated with ultrasound teaching before and after the intervention. We performed all quantitative analyses using Stata statistical software, version 16.1/IC (StataCorp TM, College Station, Texas).

Four investigators (ultrasound fellowship-trained emergency physicians: K.N., E.L.L., Y.D., and N.A.) used principles of thematic analysis to review open-ended survey responses. They independently read all survey comments to familiarize themselves with the data and searched for themes; then, as a group reviewed common patterns and reached a consensus to define the final themes. [10]

Outcomes

ADAPT provided 3 iterations of the curriculum over April, May, and June 2020. Twenty-nine of the 70 participating learners (41%) and 22 of the 30 participating educators (73%) completed the survey. Learners and faculty spanned all geographic regions of the United States. Most learners were EM interns (n = 45, 60%). The remainder were 9 senior EM residents, 5 ultrasound fellows, 4 pediatric EM fellows, 3 medical students, 3 nonteaching EM faculty, and 1 sonographer (Supplemental Digital Appendix 4, at http://links.lww.com/ACADMED/B184). Learners participated a median of 6.5 days (range, 1–20 days), compared with once weekly didactic instruction in prepandemic times. Most learners (n = 24, 82.8%) reported extra clinical responsibilities that interfered with attendance. Each faculty participated a median of 3.0 days (range, 1–12 days). The mean time that educators spent weekly on preparatory work before ADAPT was 6.2 hours; after implementation, it was 3.1 hours (P < .001).

Of the 29 learner respondents, 25 were EM interns, 1 was a senior resident, 2 were medical students, and 1 was a pediatric EM fellow. Most learners reported prior ultrasound training (n = 20, 68.9%), which could include an ultrasound course during preinternship orientation or residency conference education. Nine learners reported no prior experience (31%); 6 were EM interns, and the remaining 3 were medical students and a pediatric EM fellow. Most learners (n = 28, 96.6%) and all educators (n = 22, 100%) were “satisfied” or “very satisfied” with ADAPT. Additionally, educators were “very likely” (n = 18, 81.8%) to recommend it is continued. After completing ADAPT, the majority of learners were “somewhat confident” in their ability to acquire (n = 18, 62.1%) and interpret (n = 19, 65.5%) ultrasound images. The majority of educators felt that ADAPT was “somewhat effective” or “very effective” (n = 13, 61.9%) in assisting learners’ abilities to acquire ultrasound images; 8 (38.1%) thought the program “a little effective” for this purpose. Educators believed that the program was “somewhat effective” (n = 13, 61.9%) or “very effective” (n = 7, 33.3%) in contributing to their learners’ ability to interpret ultrasound images (Table 1).

Table 1

Learner and Educator Survey Responses Upon Completion of ADAPT, a Virtual Platform Ultrasound Training Program in Pediatrics and Emergency Medicine Residencies at 15 U.S. Medical Institutions, April–June 2020

Of the programs with access to HUDs, 29 learners cumulatively performed and saved 998 exams to the Butterfly Cloud. This is an underestimate of total exams as it excludes exams performed by rotators while working clinically, those conducted using HUDs, and unsaved ultrasound exams. Fifteen learners (51.7%) reported limited access to clinical scanning during their ultrasound rotation. There was no significant difference in learner confidence regarding image acquisition (P = .60) or interpretation (P = .96) among those who did or did not have access to clinical scanning.

Nineteen of 29 learners (65.5%) and 19 of 22 educators (86%) surveyed responded to open-ended questions. Analysis of their responses yielded multiple recurring themes related to the strength of ADAPT: access to a diverse group, depth of didactic offerings, access to HUDs, and sharing the workload. Themes related to areas of improvement included prioritizing learner engagement and optimizing schedule coordination (Table 2). One faculty educator commented that it “salvaged our POCUS rotation during the pandemic.” Residency leadership praised ADAPT as “the most organized” and “the most robust virtual learning of all the educational subdivisions.”

Table 2

Thematic Analysis of Open-Ended Learner and Educator Survey Responses Evaluating ADAPT, a Virtual Platform Ultrasound Training Program in Pediatrics and Emergency Medicine Residencies at 15 U.S. Medical Institutions, April–June 2020

Next Steps

ADAPT was well received and effective in teaching both didactic and hands-on components of POCUS. While a virtual curriculum may not completely replace in-person training, this was the only alternative for many programs facing the cancelation of their rotations. Sharing design and execution of this curriculum halved the administrative and educational time devoted by participating faculty. Residents had opportunities to interact with each other and engage with nationally recognized POCUS experts. Learners were exposed to pathology and clinical management styles transcending regional practices.

A crucial element to the success of ADAPT was utilization of a shared web-based cloud platform for QA. This enabled screen sharing interesting images during didactic sessions and highlighting scan techniques. Residents could import scans into their residency portfolios. This is essential to meet requirements of the Accreditation Council for Graduate Medical Education. Teleguidance added real-time observation of learners’ hands-on performance, ensured understanding of troubleshooting maneuvers, and offered individualized attention.

We attempted to standardize the educational experience by ensuring all learners received similar core content. However, because programs integrated ADAPT in variable ways, there was some heterogeneity of its application. Other logistical challenges included variable length of rotations and start dates, time zones, competing clinical responsibilities, a limited number of scan models, and a potential absence of pathology.

We addressed nonresponse bias by sending reminder emails to all participants. Although there was a high educator response rate, the learner response rate was moderate. Further analysis of factors that contributed to nonresponse was not possible due to respondent anonymity. Survey responses may be subject to overconfidence bias, a known phenomenon among trainees. Anchoring bias may result from respondents comparing this curriculum with traditional POCUS training.

Learners and faculty did not perceive a significant deficit in training during this transition. We acknowledge that learner and faculty satisfaction does not equal competency. The ADAPT program assessed skill acquisition through the number of exams performed, direct observation of image performance during teleguidance sessions, evaluation of image interpretation during QA, and discussion of clinical integration. The goal of an intern POCUS curriculum is to lay a foundation upon which to build psychomotor skills, interpretive abilities, and clinical acumen. The achievement and assessment of competency, whether the curriculum is virtual or not, is a longitudinal process that spans the entire residency.

Future work will focus on assessing competency longitudinally using the ADAPT framework, standardizing the curriculum to address logistic concerns, mentoring faculty to deliver engaging virtual didactics, and incorporating consistent teleguidance into sessions for real-time feedback. Future research exploring cost and time analyses of the virtual curriculum may further demonstrate the value of sharing resources across multiple institutions.

A multi-institutional collaboration such as ADAPT is both effective and feasible. We addressed components of POCUS competency by combining virtual didactics, a web-based image archival solution, and a teleguidance platform. The collaboration satisfied educational needs while increasing national engagement. Beyond the pandemic, many participating faculty plan to continue developing this initiative and growing this academic network. This endeavor’s success provides a usable framework for further collaborations during pandemic events and may be used as part of a regular teaching strategy.

Acknowledgments:

The authors would like to acknowledge the faculty at Carolinas Medical Center, Dartmouth-Hitchcock Medical Center, Emory University School of Medicine, Harvard Medical School, Johns Hopkins University School of Medicine, Mayo Clinic Alix School of Medicine, Resurrection Medical Center, Stanford University School of Medicine, University of Cincinnati College of Medicine, University of Florida at Jacksonville, University of North Carolina School of Medicine, University of Texas Southwestern Medical Center, and Yale University School of Medicine for their contributions to the collaborative curriculum. The authors also would like to thank Stefanie Sebok-Syer, Douglas Ander, and Tim Moran for their review and guidance during survey design and manuscript review. They thank the Butterfly Network, Inc. for its loan of 20 handheld devices that made teleguidance possible.