Simulation-Based Mastery Learning Course for Tube Thoracostomy.

Introduction: Tube thoracostomy is a relatively infrequent, high-risk procedure that is a required competency for emergency medicine residents. Simulation-based mastery learning is the gold standard for procedure training and has been used to successfully train residents in high-risk procedures.
Methods: We developed a simulation-based mastery learning course for tube thoracostomy for PGY 2 emergency medicine residents. The course included (1) precourse work, (2) baseline assessment using a modified version of the TUBE-iCOMPT checklist, (3) anatomy/radiology review, (4) deliberate practice to master individual aspects of the procedure, and (5) final assessment. If a minimum passing score was not achieved, additional coaching and deliberate practice occurred until the learner was able to achieve a minimum passing score.
Results: After piloting the course with a cohort of seven PGY 2 emergency medicine residents, we successfully trained 24 additional PGY 2 residents in the subsequent two classes. Combining all three cohorts (N = 31), there was a statistically significant increase in learners' modified TUBE-iCOMPT scores (pretest M = 61.2, SD = 10.0; posttest M = 75.5, SD = 2.9; p < .001). Learners' confidence in their ability to correctly place a chest tube increased, rated on a 10-point Likert scale (1 = not very confident, 10 = very confident; precourse M = 5.6, SD = 1.8; postcourse M = 8.3, SD = 1.1; p < .001). Discussion: This simulation-based course was well received by learners. Our assessment demonstrated that learners improved directly observed procedural skills in simulation and confidence in tube thoracostomy placement.

Educational Objectives

By the end of this activity, learners will be able to:

Demonstrate appropriate setup for placement of a chest tube, including sterile technique and patient positioning.

Demonstrate placement of a chest tube using sterile technique on a partial task trainer.

Discuss the diagnosis of pneumothorax on two different point-of-care diagnostic radiographic tests (portable chest radiograph and point-of-care ultrasound).

Discuss the diagnosis of hemothorax on two different point-of-care diagnostic radiographic tests (portable chest radiograph and point-of-care ultrasound).

Introduction

Tube thoracostomy is an ACGME-required procedural competency for residents in emergency medicine (EM).[1] The procedure is performed on patients who have been diagnosed with pneumothorax and/or hemothorax. It is a high-risk and relatively infrequent bedside procedure that is time sensitive, with a large number of possible complications.[2] As this is an increasingly rare procedure, it is often difficult for EM residents to complete the 10 tube thoracostomy procedures required by the ACGME based on clinical experience alone.[3] Thus, challenges arise in providing adequate training for all learners. For rare, high-risk procedures, simulation is an ideal modality to supplement clinical training. Simulation allows for deliberate practice of key procedural skills without concern for causing harm to patients. Simulation-based mastery learning is the gold standard for procedure training and has been used to successfully train residents in other high-risk procedures, such as central venous access, thoracentesis, and lumbar puncture.[4-6] Simulation-based mastery learning has been shown to have a significant impact on clinical outcomes for patients individually and at an organizational level.[7]

Mastery learning is an educational strategy whose end goal is for all learners to achieve the same level of mastery of the skill; however, the time required to achieve mastery may vary from learner to learner. Instead of achieving only the level of competence allowed during a set period of training time, mastery learning involves repetitive deliberate practice until all learners have achieved a predetermined level of competence. McGaghie describes a mastery learning bundle of seven features: (1) baseline or diagnostic testing, (2) clear learning objectives, (3) educational activities, (4) minimum passing standards, (5) formative testing with actionable feedback, (6) evidence-based advancement, and (7) continued practice and assessment until the minimum passing standard is reached.[8]

To our knowledge, there is no published mastery learning course for tube thoracostomy available for EM residents in the medical education literature. In MedEdPORTAL, the only chest tube–related publication is Wade and colleagues’ curriculum for teaching chest tubes to first-year medical students to enhance learning in a gross anatomy lab.[9] Additionally, there is a mastery learning course teaching sterile technique for bedside procedures, but it does not focus on the procedure itself.[10] At our institution, current training in tube thoracostomy for EM residents is limited to one Adult Trauma Life Support (ATLS) training course in addition to ad hoc bedside teaching and simulation sessions. These sessions often do not allow enough time for going into great detail about the specific steps of the procedure or for individualized coaching and feedback on portions of the procedure that are difficult for a learner. We created this simulation-based mastery learning course on tube thoracostomy to provide residents with a more structured, intentional training on this high-risk, relatively low-occurrence procedure in preparation for when they perform the procedure on patients in the clinical environment.

Methods

This course was implemented as part of the regular educational curriculum for PGY 2 EM residents at our institution, and anonymous data were collected in a mixed-methods fashion. Pre- and postcourse quantitative data were obtained from the procedure checklists, and pre- and postcourse quantitative and qualitative data were collected from online surveys.

Participants

PGY 2 EM residents were chosen as the target audience for this tube thoracostomy simulation course. The rationale for this was that all PGY 2s had received formal ATLS training and were familiar with basic trauma resuscitation skills. In addition, they were the designated examining and procedural residents during trauma resuscitation at our institution. Since our PGY 2 residents performed the majority of chest tubes in trauma and medical resuscitation at our institution, this decision additionally provided an opportunity to more readily obtain feedback on how the course affected their immediate practice. Given the variable roles of learners at different institutions, this course could also be administered to novice EM or surgery residents.

Modified TUBE-iCOMPT Assessment Tool

The Modified TUBE-iCOMPT checklist used in this course (Appendix A) was adapted by the authors based on Salamonsen and colleagues’ TUBE-iCOMPT assessment (Appendix B), an instrument with validity evidence to assess physician skills at chest tube insertion.[11,12] We adapted this assessment to meet the needs of our residents in coordination with our institution's EM and trauma surgery faculty. One trauma surgery faculty member and two EM faculty members reviewed the original assessment tool and agreed on the following changes based on how the procedure was taught at our institution: The scoring sections for removing trocar (4 points) and inserting wound closure suture mid-wound (1 point) were removed, as those procedural techniques were not commonly used at our institution. In the original validity evidence study on the TUBE-iCOMPT tool, a minimum passing score of 62 out of 84 points was established using two separate methods: contrasting groups’ standard setting and the modified Angoff method using experts from five different institutions.[12] Given that 5 of the 84 total points in the assessment were removed, the minimum passing score was lowered by 5 points to 57 on the modified rubric. In adapting this course to other institutions, we recommend reviewing the checklist to ensure consistency with an institution's standard practice.

The original TUBE-iCOMPT tool and our modified tool utilized three distinct sections of the overall procedure: (1) patient positioning and local anesthetic, (2) blunt dissection skills, and (3) suturing, drain connection, and dressing. Within each section of the assessment tool, there was a checklist portion where points were given on a yes/no basis whether or not the learner completed that step, as well as a global rating scale assessed on 10-point Likert scale (0 = unsatisfactory, 9 = excellent). After the pilot study, the original scoring sheet was modified to a more user-friendly grid format to improve the ease of scoring (Appendix A). The scores from each of the three sections were then totaled at the bottom of the scoring sheet to produce a total score out of 79 points.

Course Outline

This tube thoracostomy course consisted of required prework followed by an approximately 2-hour, intensive, hands-on session in the simulation center. For learners, the only prerequisite was completing ATLS training. The course was taught and assessed by EM attendings and senior residents. The complete outline for the course can be found in Appendix C.

Prior to the class, all learners completed approximately 30 minutes of precourse work. The precourse work consisted of reviewing the Modified TUBE-iCOMPT checklist (Appendix A) as well as the instructional video on chest tube insertion developed by Dev and colleagues.[13] Prior to attending the course, each learner completed a precourse survey (Appendix D). The survey included questions regarding previous experience with the procedure and subjective confidence in chest tube placement.

This course was intended to be completed in small groups of two to four learners, with at least one instructor for every two learners to maximize individual feedback. Materials needed for the course included the following:

Chest tube trainer manikin with consumable chest drain pad (e.g., Chest Drain & Needle Decompression Trainer, Limbs & Things)

Three to four packages of suture (e.g., 0-1.0 silk)

Scalpel

10-cc syringe for simulated lidocaine

21- to 23-gauge needle

Sterile gloves of various sizes

Face mask with eye protection

Sterile drape or towels

Multiple sizes of curved Kelly clamps, mosquito clamps, needle driver

Xeroform

4 × 4 gauze

Silk tape or equivalent

Chest tube drainage system

Sterile gown

Sterile bouffant cap

Learners were given the checklist in advance of the session to allow them time to prepare. The course began with learners completing a directly observed baseline skills pretest using the modified TUBE-iCOMPT checklist, as outlined by step 1 of the mastery learning bundle described by McGaghie.[8] This was done without prompting or teaching from instructors to allow it to be a true baseline assessment. Following the pretest, the instructors went over the results with each learner and gave direct feedback in order to review specific areas for improvement and to tailor the deliberate practice session to the needs of each individual learner. Time allotted for this section was approximately 20 minutes.

The next part of the course was the anatomy/radiology segment using the PowerPoint presentation (Appendix E). The focus here was on addressing indications for tube thoracostomy placement and diagnosis of pneumothorax/hemothorax on chest radiography and ultrasound. Learners were shown a number of chest radiographs and asked to interpret them. The series included a normal chest X-ray and several X-rays with pneumothorax or hemothorax depicted. Images of normal and abnormal lung sliding on ultrasound were also shown and interpreted by the learners. Complications of the procedure were also discussed. Time allotted for this portion was 10–15 minutes and required a minimum of one instructor for the entire group.

After the anatomy/radiology portion of the course, learners divided into small groups for deliberate practice with their instructors to review the entire procedure in a controlled, stepwise fashion. This worked best with one instructor for every one to two learners. Each learner had their own manikin as well as a complete set of chest tube insertion instruments kits. Individualized teaching focused on the portions of the procedure that each learner had struggled with during the pretest and allowed for ample time to ask questions, try different techniques, and ensure understanding of the complete procedure. Particular attention was paid to each of the three aspects of the procedure: positioning/local anesthetic/sterile technique, blunt dissection skills/tube placement, and suturing/securing. Time allotted for this portion was 45-60 minutes.

At the conclusion of the course, all learners completed a posttest using the same modified TUBE-iCOMPT assessment tool as in the pretest (Appendix A) and were assessed for the achievement of the minimum passing standard. Those who did not achieve the minimum passing standard underwent further coaching and deliberate practice until a minimum passing score was achieved. We recorded the results of the pretest and the posttest, as well as the number of attempts to pass the posttest. Time allotted for the posttest was approximately 20 minutes per attempt. If learners required further deliberate practice prior to reattempting the posttest, an additional 20-30 minutes were allotted.

We conducted a postcourse survey (Appendix F) to assess for change in confidence with the procedure as well as to obtain feedback on how to improve the course to meet the needs of the learners. Total time allotted for completion of the course was 120-150 minutes. This allowed adequate time for deliberate practice. However, if less advanced learners are being taught, additional time for deliberate practice may be required.

Results

We successfully piloted this simulation-based mastery learning course on tube thoracostomy with our first cohort of seven PGY 2 EM residents. The course was completed by two subsequent classes of PGY 2 EM residents, which comprised 24 additional residents, for a total of 31 residents.

All learners met the minimum passing standard on the posttest in their first attempt. Results of a paired-sample t test of learners’ pretest and posttest assessments showed a statistically significant increase in participants’ scores (pretest M = 61.2, SD = 10.0; posttest M = 75.5, SD = 2.9; t(30) = 9.2; p < .001).

The response rate to the anonymous postcourse survey was 81% (25 out of 31). Ninety-six percent of respondents stated that they were extremely likely to recommend the course to other learners. Results of a paired-sample t test demonstrated a statistically significant increase in learners’ confidence in their ability to correctly place a chest tube using blunt dissection (precourse M = 5.6, SD = 1.8; postcourse M = 8.3, SD = 1.1; t(24) = 7.7; p < .001), as rated on a 10-point Likert scale (1 = not very confident, 10 = very confident).

We also obtained feedback from our postcourse survey. In response to the questions “What worked well? What was your favorite part about the course?”, we received the following responses:

“Practicing multiple times was key to improving. It was great that we were able to make mistakes on our first try and then be coached through the errors that were made.… This session should definitely be run for all junior EM residents!”

“Repetition and the immediate feedback in a 1:1 setting. Also really liked the pretest part to see what I actually knew and remembered from prior training sessions/prior chest tubes.”

“Having us walk through it on our own really helped to draw attention to areas of uncertainty and missed steps. Overall a great learning experience!”

“I liked having to gown up and do it sterile like in real life.”

“I like the repetition with specific one on one feedback.”

“Focused practice is really rewarding and useful.”

“I like how it was done to simulate a real event with getting sterile and being graded on comfort and efficiency.”

Additionally, the feedback we received from the postcourse survey allowed us to further tailor the course by building on prior experiences. In response to the question “How can this course be improved in the future?”, we received the following responses:

“Would also be nice to do on cadavers.”

“Add other chest tube placement such as pigtails.”

“Having a mannequin that is ‘wetter.’ One aspect I struggle with is the tactile gripping of my tools when there is blood/Vaseline/ultrasound gel on the field.”

Discussion

Chest tube thoracostomy is a high-risk procedure required by the ACGME's procedural competency for EM residents.[1] Current procedural training in this competency is limited. We created a simulation-based mastery learning curriculum to teach this procedure to residents to improve their procedural skills and confidence. We found that learners improved significantly from pretest to posttest after completing our curriculum and also showed a significant increase in confidence in chest tube placement. The expectation is that learners, through completion of this course and passing the posttest, will at least achieve the competence level of the Dreyfus and Dreyfus model of skill acquisition[14] and be ready to perform the procedure on patients in the emergency department, with supervision.

The TUBE-iCOMPT assessment form utilizes a combination of an objective checklist portion and a subjective global rating scale portion that adds up to a total final score. Combining the benefits of a checklist, such as its specific and objective nature, and a global rating scale provides a broader assessment of procedural competency.[15] Anecdotally, supervising attendings and senior residents noticed that many learners knew most or all of the steps after completing the precourse work, but the course was especially helpful for improving both learners’ fluidity between steps and their overall ease with placing a chest tube.

Limitations

The subjectivity of the global rating scale scoring added potential variance in scoring between instructors. Although an assessment instrument with previous validity evidence was used, we did not perform extensive rater training or measure interrater reliability at our institution, which would have increased the internal structure validity of our tool.

Another limitation to our curriculum was the lack of rigorous retesting of the modified assessment tool to create a new minimum passing score. In the validity evidence study of the original TUBE-iCOMPT tool, a contrasting groups’ standard setting and the modified Angoff method were used to determine the minimum passing standard.[12] When we removed 5 points from the total to more accurately reflect the procedure at our institution, the minimum passing standard was also lowered by 5 points. More rigorous testing via methods similar to those described with the original tool would have added additional validity to our use of the modified assessment tool within our curriculum.

A combination of these two limitations potentially led to the mean pretest score being higher than the minimum passing score. Despite this, there was a significant increase in scores on the posttest, indicating a significant amount of procedural skill gained from taking the mastery learning course. For residents who met the minimum passing score on the pretest, we still found that they improved during the course, indicating a benefit to the course regardless of beginning knowledge. The large standard deviation on the pretest (M = 61.2, SD = 10.0) also indicates that while the mean score was higher than the minimum passing score, the variation was large and there were many learners who performed below the passing score on the pretest.

The fact that the mean pretest score was slightly higher than the minimum passing score also suggests that this course could likely be taught earlier in residency with less experienced residents, perhaps toward the beginning of the PGY 1 year.

Future Steps

This course has become a required part of the procedure curriculum at our institution for all incoming PGY 2 EM residents to prepare them to be the procedure resident during trauma resuscitations. As next steps, we plan to perform delayed postintervention testing to assess for knowledge and skill retention. At the suggestion of our residents, a version of this course including Seldinger technique/pigtail chest tube insertion is planned, again using a modified version of the TUBE-iCOMPT assessment tool.[11] Additionally, preliminary data have been collected looking at chest tube complication rates at our institution before and after implementation of this curriculum in an attempt to assess outcomes at a higher Kirkpatrick level.[16] The preliminary data suggest a trend toward lower complication rates after implementation of the course.

This simulation-based mastery learning chest tube course is the first of its kind that we know of in the medical education literature. The course was able to increase both directly observed procedural skills in simulation and self-reported learner confidence in performing tube thoracostomy.

Modified TUBE-iCOMPT Rubric.docx

TUBE-iCOMPT.docx

Simulation Chest Tube Course Outline.docx

Simulation Chest Tube Precourse Survey.docx

Chest Tube Radiology Presentation.pptx

Simulation Chest Tube Postcourse Survey.docx

All appendices are peer reviewed as integral parts of the Original Publication.