Ian P Mayne1, Ryan Brydges2, Joel Moktar1, M Lucas Murnaghan3. 1. Division of Orthopaedic Surgery, University of Toronto, Toronto, Ontario, Canada. 2. The Wilson Centre, Toronto General Hospital, Toronto, Ontario, Canada. 3. Division of Orthopaedic Surgery, University of Toronto, Toronto, Ontario, Canada The Wilson Centre, Toronto General Hospital, Toronto, Ontario, Canada Women's College Hospital, Toronto, Ontario, Canada The Hospital for Sick Children, Toronto, Ontario, Canada lucas.murnahgan@sickkids.ca.
Abstract
BACKGROUND: Simulation-based learning is increasingly prevalent in the curricula of many surgical training programs. Newly developed simulators must undergo rigorous validity testing before they are used to assess and evaluate surgical trainees. We describe the development of a model that simulates a distal radial fracture requiring closed reduction and cast application and demonstrate its validity. METHODS: We developed a model for simulated treatment of a distal radial fracture with use of a modified Sawbones forearm. Ten junior and ten senior orthopaedic residents were videotaped performing a closed reduction and applying a cast on the model. After each procedure, standard anteroposterior and lateral radiographs of the forearm model were obtained. Two blinded orthopaedic surgeons then rated each resident using a task-specific checklist (Objective Structured Assessment of Technical Skills [OSATS]) and a global rating scale (GRS) as well as radiographic measurements of palmar tilt and three-point index. RESULTS: Compared with the junior residents, senior residents had significantly higher OSATS (p < 0.001) and GRS scores (p < 0.001). The groups did not differ significantly with respect to radiographic palmar tilt (p = 0.86) and three-point index (p = 0.43). All residents were able to restore anatomical alignment, with a mean palmar tilt of 9.1°. In addition, the mean three-point index of all residents was acceptable (0.76). There was a strong correlation between OSATS and GRS scores (r > 0.87; p < 0.01). The inter-rater reliability was high (≥ 0.79) for the OSATS, GRS, and radiographic measurements. CONCLUSIONS: We developed an educational model that simulates a distal radial fracture requiring closed reduction and cast application. We demonstrated construct validity, as the GRS and OSATS tools were able to differentiate senior from junior residents. We were unable to differentiate trainees using radiographic assessment, as all residents restored anatomical alignment and had comparable three-point index scores.
BACKGROUND: Simulation-based learning is increasingly prevalent in the curricula of many surgical training programs. Newly developed simulators must undergo rigorous validity testing before they are used to assess and evaluate surgical trainees. We describe the development of a model that simulates a distal radial fracture requiring closed reduction and cast application and demonstrate its validity. METHODS: We developed a model for simulated treatment of a distal radial fracture with use of a modified Sawbones forearm. Ten junior and ten senior orthopaedic residents were videotaped performing a closed reduction and applying a cast on the model. After each procedure, standard anteroposterior and lateral radiographs of the forearm model were obtained. Two blinded orthopaedic surgeons then rated each resident using a task-specific checklist (Objective Structured Assessment of Technical Skills [OSATS]) and a global rating scale (GRS) as well as radiographic measurements of palmar tilt and three-point index. RESULTS: Compared with the junior residents, senior residents had significantly higher OSATS (p < 0.001) and GRS scores (p < 0.001). The groups did not differ significantly with respect to radiographic palmar tilt (p = 0.86) and three-point index (p = 0.43). All residents were able to restore anatomical alignment, with a mean palmar tilt of 9.1°. In addition, the mean three-point index of all residents was acceptable (0.76). There was a strong correlation between OSATS and GRS scores (r > 0.87; p < 0.01). The inter-rater reliability was high (≥ 0.79) for the OSATS, GRS, and radiographic measurements. CONCLUSIONS: We developed an educational model that simulates a distal radial fracture requiring closed reduction and cast application. We demonstrated construct validity, as the GRS and OSATS tools were able to differentiate senior from junior residents. We were unable to differentiate trainees using radiographic assessment, as all residents restored anatomical alignment and had comparable three-point index scores.