Procedure Training Workshop for Internal Medicine Residents that Emphasizes Procedural Ultrasound: Logistics and Teaching Materials.

Introduction: Early formal instruction in procedural skills may increase the frequency with which residents perform procedures in the clinical setting. This workshop trained internal medicine residents in ultrasound skills and manual skills required to perform procedures common on medicine wards and required for board eligibility.
Methods: Since 2016, our internal medicine residency program has executed three annual half-day workshops for interns during orientation, before clinical duties began. Prior to the workshop, we directed interns to relevant educational resources in the form of online modules and videos. At the workshop, trainees rotated in small groups through facilitated stations to learn basics of procedural ultrasound and to practice manual tasks performed during paracentesis, thoracentesis, lumbar puncture, and peripheral intravenous catheter placement. We administered questionnaires before and immediately after the workshop and used Wilcoxon signed rank tests to compare self-assessed independence and confidence.
Results: Two hundred four interns with little prior procedural training participated in the workshop. Most participants (85%) indicated that orientation was the best timing for this training experience when compared to later options. Confidence and independence increased for ultrasound-marked thoracentesis, paracentesis, and peripheral intravenous catheters and for lumbar puncture without ultrasound. Discussion: This internal medicine intern orientation workshop on procedures and procedural ultrasound was well received and increased participants' confidence and sense of independence. This publication contains materials needed to reproduce the training experience.

Educational Objectives

By the end of this activity, learners will be able to:

Describe key anatomic landmarks used in planning and performing thoracentesis, paracentesis, lumbar puncture, and peripheral intravenous (PIV) catheter placement.

Describe appropriate patient positioning for thoracentesis, paracentesis, lumbar puncture, and PIV catheter placement.

Perform thoracentesis, paracentesis, lumbar puncture, and PIV catheter placement on a simulated task trainer guided by a checklist.

Demonstrate appropriate use of ultrasound guidance for paracentesis, thoracentesis, and PIV catheter placement.

Introduction

Internal medicine residents require procedure training for board eligibility and, in some cases, to prepare for future practice. The American Board of Internal Medicine (ABIM) requires internal medicine residency graduates to competently perform a few procedures, such as peripheral intravenous catheter placement, for board eligibility.[1] In addition, ABIM mandates that all graduates understand indications, patient preparation methods, and risks of several clinical procedures they will commonly recommend, if not perform. ABIM recommends using simulated training followed by active participation under observation to achieve both knowledge and performance competence.

Simulation-based training is an effective method for teaching complex procedural skills.[2] Simulation-based procedural training leads to high learner satisfaction and increases learner-perceived confidence and competence.[3-5] Furthermore, simulation training increases knowledge, improves skill, and reduces complication rates.[3-6] Use of ultrasound (US) to guide procedures reduces harm and cost and has become a clinical standard; therefore, procedural training should integrate US.[7]

Before 2015, our residency program provided simulated procedure training throughout the first quarter of intern year. At an education-focused retreat in 2015, our residents requested earlier and more frequent opportunities to practice US and procedures they commonly performed on patients. In response, we implemented this time-limited introductory workshop during intern orientation. Herein, we describe our workshop objectives, logistics, and outcomes. In the appendices, we provide materials other programs can use to reproduce the workshop. Some of these materials could also be used for teaching outside the context of a workshop or for self-study.

Other currently available curricula for clinical procedures allow learners to access didactic materials for self-/peer instruction, are tailored to different types of learners, or instruct in procedures not typically performed by internists.[8-13] Our curriculum is unique because it facilitates hands-on simulated training for procedures commonly performed by internists. The curriculum integrates procedural US education and promotes active learning by incorporating locally developed training videos in a flipped classroom format.

Methods

This 4-hour annual workshop trained interns in basic US skills, US-marked paracentesis, US-marked thoracentesis, US-guided peripheral intravenous catheter placement, and lumbar puncture. Each workshop took place during intern orientation, before interns started clinical rotations. We designed the workshop assuming no or minimal prior experience performing procedures. A few weeks ahead of orientation, residency program staff emailed participants a list of required preparatory work to complete prior to the workshop. Before attending the workshop, interns viewed several online videos covering US basics, paracentesis, thoracentesis, lumbar puncture, and peripheral intravenous catheter placement. In the later years of the workshop, we augmented publicly available copyrighted resources overviewing manual procedure skills[14,15] with locally developed videos teaching procedural US (Appendice A-D).

Residency program leaders selected in advance a prespecified number of voluntary workshop instructors. All chief residents of internal medicine served as instructors according to their job description. We invited additional instructors from a select group of fellows and faculty from pulmonary medicine, critical care medicine, and general internal medicine based on our knowledge of their procedural competence and interest in teaching. The number of instructors ensured specific instructor-to-learner ratios (Appendix E). To invite questions and promote consistent teaching, organizers emailed instructors in advance and provided the station objectives, checklists for performing manual procedures, and US teaching guides (Appendix F). We asked instructors to reply with a prioritized list of the stations that they felt most comfortable teaching and questions about teaching strategies. Immediately prior to the workshop, instructors received live refresher training on procedural technique that combined brief didactics with simulated practice.

Our institution's simulation center hosted the workshops because of its large laboratory spaces and close proximity to the required equipment. In order to accommodate approximately 67 learners each year, half the learners attended a morning workshop and the other half a same-day afternoon workshop. Each workshop began in a large group. After interns had completed a brief preworkshop questionnaire in hard copy (Appendix G), the organizer introduced the workshop schedule and locations of the learning stations. Interns then rotated through the learning stations in predetermined small groups of four to five interns. Each small group was assigned an order in which to rotate through the five learning stations. Each intern's rotation schedule was printed on the back of their name tag. The station that comprehensively covered chest and abdominal US took 50 minutes; all other learning stations were 25 minutes long (Appendix H). We located thoracentesis and paracentesis learning stations in close physical proximity to ease transition from one station to the next. We similarly clustered lumbar puncture and intravenous catheter stations. To make the rotation schedule work, we designated one 50-minute block as flextime. The content of this time changed annually based on educational priorities. For example, flextime could be used for instruction on prerounding, bedside rounding, informed consent, death verification exams, or communication skills.

Table.

Resident Confidence and Independence in Performing Procedures

Interns spent the majority of time with their hands on the equipment, practicing manual tasks with verbal guidance from instructors. Instructors asked permission before using touch to guide trainees’ physical position. Learning objectives, locally developed procedure-specific checklists, and US teaching guides were posted at each learning station (Appendix F). Learning stations had sufficient equipment for each trainee to practice hands-on skills for the entire time allotment, except for the US station at which the group of trainees took turns examining a volunteer human model (often a medical student interested in learning US). Each procedure station included procedure kits used in our hospitals and a container for safe disposal of sharp equipment such as needles. We equipped learning stations with teaching-dedicated recycled kits when new kits were unavailable. Instructors reorganized the training kits as interns rotated to the next assigned learning station. Located at each station was a task trainer specific to that station's procedure. Most of these task trainers were prefabricated and belonged to our institution's simulation center. Since a paracentesis task trainer was not available, we created a paracentesis model that would accommodate two learners per model using a plastic tray from a previously used procedure kit, a closed bag of saline like those used in clinical care, and pigmented silicone pseudoskin (purchased from SmoothOn, Inc., Macungie, Pennsylvania; see Appendix I; photo credit: Dr. Amanda Shepherd). The US stations included laptop computers preloaded with instructional slides covering thoracic and abdominal anatomy and pathology as identified by US (Appendix J).

Immediately after the workshop, residency program staff emailed all participants a questionnaire we developed to assess the workshop's impact and gather feedback about the workshop and individual instructors (Appendix G). We consulted with a survey methods expert to design the self-assessment scales. Confidence was assessed on a 5-point scale (1 = Not at all confident, 5 = Extremely confident). Independence was assessed on a 5-point scale (1 = I mostly observe, 2 = I can do some of the steps on my own, 3 = I can do many of the steps on my own, 4 = I can do all of the steps on my own, 5 = I can model for others). We compared pre- and postworkshop numeric values for the group with complete data using Wilcoxon signed rank tests for matched pairs. This approach assumes equal differences between each pair of consecutive points on the assessment scale. We considered a two-sided p value less than .05 as statistically significant. Analyses were performed using Stata 12.0 software (StataCorp, College Station, Texas). The postworkshop questionnaire required respondents to write free text describing one thing they liked best about the workshop and one thing they would change. We informally reviewed these responses to identify key feedback and themes.

Our institution reviewed our study plan and concluded it did not meet the federal definition of human subjects research. Therefore, this study did not require approval. We have no conflicts of interest to disclose.

Results

Two hundred four interns participated in this workshop over 3 years, including interns from the primary care, categorical, and preliminary tracks. Response rates to the preworkshop questionnaire were 91% in 2016 (61 out of 67), 91% in 2017 (63 out of 69), and 97% in 2018 (66 out of 68). Response rates for the postworkshop questionnaire were 45% in 2016, 51% in 2017, and 40% in 2018. Eighty-two interns (40%) completed both the pre- and postworkshop questionnaires.

Interns reported performing a median (interquartile range) of 0 (0-0) thoracenteses, 0 (0-1) paracenteses, 0.75 (0-1) lumbar punctures, and 3 (1-5) peripheral intravenous catheter placements prior to this learning activity, excluding simulated practice. Results for self-assessed confidence and independence are included in the Table. Confidence and independence increased for all procedures immediately after the workshop. Using a 5-level Likert scale of agreement, 97% of respondents agreed or strongly agreed that simulation was an effective educational tool for procedural training. Eighty-five percent responded that the best time to teach these procedural skills using simulation was orientation, 5% selected early in intern year but after orientation, and 10% selected the start of intensive care unit rotation. No respondent selected the option “late in intern year” or “never.”

When interns were asked what they liked best about the workshop, the responses most commonly appreciated the hands-on, active learning format. Learners also reported that the high teacher-to-learner ratio with real-time feedback and the low-pressure environment augmented their learning. Illustrative quotes from free-text responses on the postworkshop questionnaire follow, arranged by theme:

Learners liked active learning:

“I liked that we actually looked through the kits and practiced some with all of the components and procedures (draping, etc.).”

“Lots of hands-on practice, instructors talked us through while we actively did it which allowed for more time to practice.”

“Being walked through all the different aspects of each of the procedures step by step and then getting a chance to do it under guided simulation.”

Learners liked the high teacher-to-learner ratio and enthusiastic teachers:

“I liked that we were divided into small groups and had more one-on-one attention with the experts.”

“It was clear those who were teaching enjoyed what they did.”

Learners benefited from a low-pressure environment:

“[Instructors] fielded all manner of questions in an educationally safe space.”

“I really appreciated the opportunity to try these simulated procedures (which I had no experience with) in a more comfortable learning environment prior to hitting the wards.”

Learners’ suggestions to improve the workshop highlighted the idea that trainees acquired skills at individual paces. Many residents expressed a desire for more practice time during the workshop, especially with US, but a few respondents requested less practice time. Several respondents recognized a need for ongoing practice beyond this single experience. In other comments, residents mentioned limitations of simulation-based training and requested more coverage of knowledge competence, for example, by expanding training on the indications for a specific procedure. Illustrative quotes from free-text responses on the postworkshop questionnaire follow, arranged by theme:

Learners acquired skills at variable rates:

“I would have liked if there was an hour of ‘independent study’ at the end where you could return to any of the workstations if you felt like you wanted more time to practice.”

“Either allow more time for each module or ensure there are enough models/dummies for each student to have sufficient hands on experience.”

“Give people the opportunity to opt out of procedures they are already comfortable with.”

A related response to the postworkshop survey question “List one thing you liked about the workshop” was “The stations were just the right length. The whole thing felt appropriately fast paced.”

Learners wanted more focus on knowledge competence:

“Give more information about the most common pitfalls for each procedure.”

“Have a minute at each station about issues that frequently need troubleshooting.”

“Maybe give context for when and how often you will use a given procedure.”

Simulation had limitations:

“Provide more practice finding peripheral veins on different people by ultrasound.”

“If there were a way to simulate actually finding a fluid pocket for paracentesis or thoracentesis on ultrasound that would be a useful addition.”

“Would be helpful to still practice all the steps to mimic real life as closely as possible.”

Learners desired ongoing education:

“It'd be much better to be exposed to procedural simulation several times throughout intern year, as the needs of each learner are different at different points in the year.”

“It would be nice to have some refreshers with these simulations before or during rotations where we are likely to use these often, such as the ICU or wards.”

Discussion

For this group of internal medicine interns with little prior procedural experience, this orientation workshop increased participants’ self-reported confidence and independence in performing common medical procedures. The materials provided in this publication can facilitate reproduction of this successful workshop in other settings.

The workshop participants’ suggestions for improvement highlight two important points to consider when planning procedural training time lines. First, learners achieve procedural competence at varying paces. Mastery-based learning is one successful approach to individualizing training.[6,16-18] When applied to procedure education, experts evaluate learner competence at multiple time points throughout a simulation-based training session. Instead of completing training based on time, a trainee completes the training when he/she demonstrates sufficient skill to meet a minimum standard. The mastery-based approach may require greater investment of time and a higher level of teacher expertise than our workshop. Our large residency program chose a time-based format to balance between competing interests. The current workshop leverages the orientation period free of clinical service obligations but allows distribution of time and teachers across other orientation activities. Future teachers could adapt many of our curricular materials to fit a mastery-based schedule. Additionally, the materials included in this publication could be delivered in fragments across separate time points to accommodate other institutions’ time and resource constraints. Second, procedural skills likely decay following simulation-based training, and ongoing training may reduce losses.[19,20] Longitudinal instruction may most effectively facilitate competence and can be combined with simulation-based mastery learning.[16]

Our program complements this introductory workshop with additional formal procedure training throughout intern year. Interns attend workshops on the first day of clinical rotations that have historically high procedure volumes to reinforce and advance skills. We teach paracentesis on general medical wards and thoracentesis in the medical intensive care unit. Data gathered from interns at these subsequent workshops seem to indicate they prefer training delivered closer to the point of care. Whether residents maintain gains in knowledge and skill from this workshop at these later times is an area of active investigation. Research is needed to determine which approaches to training—early training, training closer to the point of care, or a combination of these—are most effective at achieving durable knowledge and procedural competence.

Limitations

The design of our workshop has limitations. Our workshop may not be generalizable across institutions or types of trainees as the procedures we taught were somewhat specific to internal medicine and possibly interventional radiology. Moreover, our workshop required access to task trainers and expert instructors. Due to our institution's financial support for the simulation center, our program did not pay for equipment and staff. To minimize cost, educators might ask hospitals to donate expired procedure kits and create lower-fidelity simulation models from low-cost materials as seen in our paracentesis model (Appendix I). Paper cups or pork ribs can serve as the chest wall, and saline bags as a pocket of fluid. The cost of synthetic skin substitutes could be shared across departments likely to use them in similarly small quantities. Laptop computers and US machines could be borrowed from vendors, local educational programs, or clinical wards. Although validated procedural checklists exist in the literature, we developed the checklists for learning stations (Appendix F) based on our expertise to include local priorities and practices. Intrarater reliability and interrater reliability are unknown.

Our examination of workshop outcomes also has limitations. Interns coincidentally attended a separate mandatory session in US-guided central venous catheter placement the day prior to this workshop, which may have influenced self-reported skills. The 82 respondents with complete pre- and postworkshop data had lower baseline levels of confidence in performing US-marked paracentesis and lumbar puncture and postworkshop levels of confidence similar to the whole cohort. This response pattern may have biased the result of Wilcoxon tests for these comparisons away from the null. We originally created our questionnaires for programmatic purposes rather than research and did not systematically validate them. To improve their validity in future use, we have changed numeric rating scales to descriptive verbal scales and reframed the examination of confidence, which could have variable interpretations, to an assessment of preparedness.[21] Self-assessment is easily collected but is a limited outcome measure. We did not measure the impact of this workshop on patient safety or likelihood of engaging in opportunities to practice procedures that arise as part of patient care. A systematically observed test of manual skill would be a more compelling measure of this workshop's impact but would be resource intensive, as previously discussed. The ideal learner- and patient-centered outcomes of frequency and percentage of potential procedures completed safely by each trainee are difficult to measure. These outcomes are often distanced in time from this time-limited education, calling into question any observed association between them.

Challenges

We encountered some challenges in implementing this curriculum through which we learned many lessons about logistics and optimal teaching techniques. Interns who attended early workshops reported that some instructors gave relatively long introductions resulting in less hands-on practice or lacked expertise in some manual tasks. In response, we hosted train-the-trainer open houses in the simulation center in advance of the workshop. Due to low attendance, we later replaced open houses with an orientation for all instructors immediately prior to the workshop (included in the Appendix H schedule). During this session, we provided tips to maximize teaching effectiveness and used didactics and hands-on practice to review key manual components of the procedures. We also added an expert teacher to circulate during the workshop and provide instructors with real-time coaching. In response to last-minute cancellations by volunteer instructors, we scheduled a surplus instructor capable of teaching any station. If all instructors came, the surplus instructor circulated as a teaching assistant and provided other teachers breaks when needed. On one occasion, a volunteer US model did not come to the workshop, and the surplus instructor substituted.

Future Directions

We plan to continue this workshop in the future with some changes. The workshop did not include training in US-guided lumbar puncture due to variable instructor experience and local practice. As local expertise increases, we plan to incorporate it in future orientation workshops and the neurology rotation. To increase data capture, we will administer postsurveys in hard copy at the workshop. If supported by our ongoing data analysis and trainees, we may refocus this orientation activity entirely on US skills for procedures and diagnostics and shift the manual procedure training to relevant clinical rotations.

Point-of-Care Ultrasound Basics.mp4

Bedside Ultrasound for Paracentesis.mp4

Ultrasound-Guided Peripheral Vascular Access.mp4

Thoracic Ultrasound for Bedside Thoracentesis.mp4

Procedure Workshop Logistical Planning.docx

Station Learning Objectives and Checklists.doc

Pre- and Postworkshop Questionnaires.docx

Learning Station Rotation Schedule.docx

Paracentesis Model Setup.jpg

Thoracic and Abdominal Ultrasound.pptx

All appendices are peer reviewed as integral parts of the Original Publication.