The Effectiveness of Bioskills Training for Simulated Lumbar Pedicle Screw Placement.

STUDY
DESIGN: Prospective randomized study.
OBJECTIVES: To define the impact of an inexpensive, user-friendly, and reproducible lumbar pedicle screw instrumentation bioskills training module and evaluation protocol.
METHODS: Participants were randomized to control (n = 9) or intervention (n = 10) groups controlling for level of experience (medical students, junior resident, or senior resident). The intervention group underwent a 20-minute bioskills training module while the control group spent the same time with self-directed study. Pre- and posttest performance was self-reported (Physician Performance Diagnostic Inventory Scale [PPDIS]). Objective outcome scores were obtained from a blinded fellowship-trained attending orthopedic spine surgeon using Objective Structured Assessment of Technical Skills (OSATS) and Objective Pedicle Instrumentation Score metrics. In addition, identification of pedicle breach and breach anatomic location was measured pre- and posttest in lumbar spine models.
RESULTS: The intervention group showed a 30.8% improvement in PPDIS scores, compared with 13.4% for the control group (P = .01). The intervention group demonstrated statistically significant 66% decrease in breaches (P = .001) compared with 28% decrease in the control group (P = .06). Breach identification demonstrated no change in accuracy of the control group (incorrect identification from 32.2% pre- to posttest 35%; P = .71), whereas the intervention group's improvement was statistically significant (42% pre- to posttest 36.5%; P = .0047).
CONCLUSIONS: We conclude that a concise lumbar pedicle screw instrumentation bioskills training session can be a useful educational tool to augment clinical education.

Introduction

While orthopedic and neurosurgical programs aim to provide a comprehensive curriculum to physicians-in-training to ensure competence in surgical skills and decision making, external pressures including work-hour restrictions, declining budgets, and medicolegal ramifications of surgical complications can limit these experiences. The performance of complex skills and tasks, including surgical skills, demonstrate a “learning curve” and require frequent repetition to ensure accuracy and consistency of the task.[1] Simulated training sessions are commonly utilized for skill acquisition and early development of complex technical tasks. For example, Gonzalvo et al concluded that approximately 40 to 80 pedicle screws were needed before a spine fellow was able to achieve reproducible accuracy consistent with attending staff skill.[1]

Bioskills training modules can be practical and effective forms of education within the time and budgetary constraints of many surgical training programs. While several studies utilizing cadaveric training modules have demonstrated improvements in pedicle screw placement for trainees,[2,3] the time and cost required can be prohibitive for residency programs. With budgetary and time constraints in mind, the purpose of our study was to define the effectiveness of an inexpensive, user-friendly, readily deployable, and reproducible lumbar pedicle screw training module and evaluation protocol that can be readily implemented into residency training programs in order to augment the clinical education of orthopedic and neurosurgical physicians-in-training. We hypothesize that the implementation of a concise training session would lead to improvement in simulated surgical skills performance compared with self-directed study.

Methods

A total of 19 participants were enrolled from an orthopedic training program at a single tertiary academic medical center. The study design met the criteria for an institutional review board exemption at our institution and participants gave informed consent. Participants were stratified into 3 groups based on level of experience (medical students, PGY 1-3, and PGY 4&5) to facilitate block randomization that would control for participant level of experience. Blinded assessors were unaware of control (n = 9) or intervention (n = 10) group assignment.

All participants performed a pretest consisting of pedicle screw placement in the right-sided pedicles of L1 through L5 of a Sawbones normal anatomy lumbar spine model in a prefabricated spine holder, simulating a standard posterior open view. Standard instrumentation and surgical tools were provided and participants were allowed to select the screw size and length. The dorsal exposure was limited to the posterior elements, and foam tape was placed over both the midline elements and transverse processes, obscuring direct visualization of the pedicle or vertebral body while allowing visualization of the pedicle screw start point (Figure 1). Participants were given 30 minutes to complete the task (Figure 2). Additionally, participants completed a pedicle screw breach detection station. A lumbar spine Sawbones model with previously prepared pedicle screw tracts with either breaches at random locations or no breach was used to test participants’ ability to properly identify the presence or absence of a breach with a ball-tip probe. The breaches were made using a 2 mm drill bit at the appropriate start point for pedicle screws but headed in an aberrant trajectory in order to simulate a Lenke probe breach. The size or depth of breaches was not standardized for created breaches. Responses were subsequently scored as “correct” or “incorrect,” with “correct” scores given for answers either correctly identifying no breach or correctly identifying the location of the breach (Figure 3).

Figure 1.

Representative lumbar Sawbones model for pedicle instrumentation bioskills module. Posterior spinal approach simulated with foam tape covering spinal canal and anterior structures.

Figure 2.

Pedicle instrumentation bioskills simulation laboratory arrangement.

Figure 3.

Lumbar Sawbones model for breach identification with prefabricated pedicle screw tracts with anatomical breaches.

Following the pretest, the control group participants were instructed to read standard texts of their choosing for 20 minutes. The intervention group underwent a 20-minute training module consisting of an interactive PowerPoint presentation created by the authors demonstrating the performance of pedicle screw placement and technique demonstration using sawbones models (see Supplemental Material for PowerPoint; available in the online version of the article). The posttest evaluation for both groups was then performed on the left sided pedicles of L1 through L5 of the same model. Following the posttest, the pedicle breach detection test and written quiz were administered to all participants. Total completion time for the study was approximately 75 minutes. The cost of the Sawbones lumbar spine model was approximately US$72 per participant.

Participants’ performances of lumbar pedicle screw placement were evaluated using a modified Objective Structured Assessment of Technical skills (OSATS) scale, which has been previously utilized to assess proficiency with surgical tools, technical skills, and performance of the procedure.[4,5] The participants completed a Physician Performance Diagnostic Inventory Scale (PPDIS) following the pre- and posttests, a survey given to assess the participants’ self-reported knowledge and skill. Third, the pedicle screw models were directly inspected for accuracy of screw placement and each level was categorized as “breach,” defined as direct visualization of the pedicle screw or identification of breach with probing after screw removal, versus “no breach.” Following data collection, the subsection scores of OSATS and PPDIS for each participant’s pre- and posttest were recorded as composite scores (eg, a score of “2” for each of the 5 OSATS subsections would be reported as a score of “10”; see Supplemental Material for OSATS and PPDIS grading forms; available in the online version of the article). In order to determine the size of effect resulting from the intervention or control group, the mean change in participants’ composite scores from pre- to posttesting for OSATS and PPDIS metrics were analyzed. A 2-sample t test was used to compare the mean change from pre- to posttest scores for OSATS, PPDIS, and written quiz measures within the control and intervention groups. The breach scores and breach identification station results for the control and intervention groups were treated as categorical variables and analyzed using a Bowker’s test modification of the χ2 test.

Results

The intervention group reported a significant improvement of 30.8% in PPDIS scores (mean +5.3, SD 2.98), compared with 13.4% for the control group (mean +2.0, SD 2.60) (P = .01). While the intervention group demonstrated a 37.1% improvement of OSATS scores (mean +4.2, SD 4.05) compared with the control group’s improvement of 13.6% (mean +1.89, SD 2.47), this was not statistically significant (P = .41; Table 1).

Table 1.

Statistics for OSATS and PPDIS Outcome Metrics.

	Group	N	Mean Pretest Composite Score	Mean Posttest Composite Score	Mean Change in Score	SD	P Value
OSATS	Control	9	12.111	14	1.889	2.472
	Intervention	10	11.3	15.5	4.2	4.05	P = .4101
PPDIS	Control	9	15.222	17.222	2	2.598
	Intervention	10	17.3	22.6	5.3	2.983	P = .0104

Abbreviations: OSATS, Objective Structured Assessment of Technical Skills; PPDIS, Physician Performance Diagnostic Inventory Scale.

The intervention group demonstrated a significant reduction in total breaches from pre- to posttesting (Bowker’s test; P = .001), while the control group demonstrated no significant differences (Bowker’s test; P = .06). The intervention group demonstrated 66% decrease in breaches (22% pretest vs 7% posttest) compared with a 28% decrease in the control group (26% pretest vs 17% posttest). Additionally, groups showed a no significant difference in performance on a 6-question written quiz on the technique of pedicle screw placement (P = .07; Table 2).

Table 2.

Pedicle Screw Breaches χ2 Tables.

		Pretest
Control		No Breach	Breach	Total
Posttest	No breach	14	13	27	Bowker’s test; P = .0593
	Breach	5	13	18
	Total	19	26	45

Similarly, the intervention group displayed a significant difference with correctly identifying anatomic breach location with ball-tip probe in separate prefabricated lumbar sawbones. Incorrect responses for the breach identification station increased from pre- to posttesting for the control group (32.2% to 35%, Bowker’s test P = .71), while the intervention group showed a reduction of incorrect responses (42% to 36.5%, Bowker’s test P = .0047; Table 3). Subgroup analysis of breach site identification suggests that the improvement seen in the intervention group is from improved identification of “no breach” (pretest 86.7%, posttest 96.7%) and “medial breach” (pretest 78%, posttest 92%). Identification of anterior, inferior, and multiple breaches demonstrated the lowest accuracy for both groups, averaging 10% to 30% with minimal changes from pre- to posttesting. The majority of the incorrect answers for breach identification testing were misclassified as “no breach,” accounting for 75.4% and 65.7% of the overall incorrect answers from the control and intervention groups, respectively.

Table 3.

Breach Identification Station χ2 Tables.

		Pretest
Control		Incorrect	Correct	Total
Posttest	Incorrect	48	15	63	Bowker’s test; P = .7055
	Correct	13	104	117
	Total	61	119	180

Discussion

Spine surgery requires a highly technical set of surgical skills with a known increased propensity for technical error and complications during the learning process.[6-15] Simulation training demonstrates value for trainees through protected development of basic skills, with the goal of providing a foundation to prepare for high-risk real-world experiences.[1,16,17] The results of our study propose that a concise, inexpensive, well-designed training module can improve participants’ technical skills and proficiency with simulated performance of lumbar pedicle screw placement. Participants enrolled in the intervention group reported significant improvements in subjective understanding and self-reported improvements in the technical skill of pedicle screw placement (PPDIS). Furthermore, the intervention group demonstrated a significant reduction in breach rate with placement of pedicle screws as well as correct identification of pedicle breaches when compared with the control groups. While greater improvements in technical skills (OSATS) were seen for the intervention group compared with the control group, this did not reach statistical significance.

Historically, spine bioskills modules have had difficulty demonstrating significant improvement for its participants.[18,19] Using 10 subjects including residents and medical students, Sundar et al demonstrated a reduction in suboptimal screw placement after a sawbones and cadaver training module.[20-23] Similarly, Harrop et al noted no statistically significant differences in pre- and postdidactic scores and in the OSATS using a Sawbones model for a posterior laminectomy with 8 subjects.[24] However, recent studies of orthopedic simulation programs have been able to demonstrate significantly improved cervical lateral mass and thoracic screw placement.[7,24] Gottschalk and colleagues reported that a 3D simulation-training group of 15 residents exhibited significantly improved trajectory of cervical lateral mass screw placement.[25]

Several limitations were encountered in the development and implementation of this study. Participants may have demonstrated “testmanship” during screw placement by selecting undersized screws to reduce breach rates. Additionally, the transferability of subjective and performance skills obtained during a Sawbones training lab to clinical performance has yet to be proven and was not specifically addressed in our study. Furthermore, we were not able to assess the durability of the intervention effect as posttesting occurred immediately after the intervention. While the OSATS metric has been extensively utilized and validated for the evaluation of resident’s performance of technical skills,[10-15] its use as a tool to evaluate the efficacy of simulation laboratories has only been recently described.[6,8,9,26] Anderson and colleagues suggested that any study that utilized the OSATS as a stand-alone assessment measure ignored critical objective assessments regarding the end product of the tested technical procedure.[27] For this reason, we supplemented the OSATS evaluation with direct evaluation of pedicle screw placement. Additionally, although the PPDIS can experience response bias from the Hawthorne effect (where participants modify responses or activity due to the awareness of being monitored), the outcome metric has been used in similar medical education studies and provides insight on participants’ perceived efficacy and value of the bioskills simulation training.[28,29] Furthermore, these assessment tools seem to be best suited for parametric distributions of technical skills and may exhibit floor/ceiling effects while assessing participants with very advanced or very minimal proficiency for the technical skills assessed. Additionally, the use of novel evaluation metrics precluded our ability to perform power analysis and sample size determination. Last, although our sample size (n = 19) represented the majority of our training program, this limited subgroup statistical analyses.

Despite the aforementioned limitations, we believe Sawbones simulation training can contribute positively to surgical education. The improvement in trainees’ subjective and objective skills assessments with a brief Sawbones training session suggests acquisition of essential fundamental surgical skills can be efficiently and effectively conducted in the simulation laboratory. Furthermore, trainees reported both a high level of satisfaction with simulation training and subjectively improved procedural skills. We believe the value of Sawbones simulation for pedicle screws is to improve proficiency with normal pedicle anatomy, surgical instruments, and pedicle screw technique in a controlled and low-risk environment. Although no clinical correlation of learned surgical skills was conducted, we believe simulation training is safe and effective for teaching core surgical skills, laying the foundation for trainees to learn complex anatomy and pathology with subsequent operating room experiences.

Conclusion

A standardized, cost-effective pedicle screw bioskills training module with Sawbones simulation can be a useful surgical resident and student educational tool, leading to significantly improved scores in residents’ subjective self-assessment of performance (PPDIS), improvement in breach identification, and decrease in pedicle screw placement breach rate, while trending toward improvement in OSATS. We believe that the observed improvement in participant skills has educational relevance and that similar simulation training modules can be powerful tools to augment the trainee’s surgical education.

Click here for additional data file.

Supplemental Material, GSJ743505_suppl_mat for The Effectiveness of Bioskills Training for Simulated Lumbar Pedicle Screw Placement by Barrett S. Boody, Sohaib Z. Hashmi, Brett D. Rosenthal, Joseph P. Maslak, Michael H. McCarthy, Alpesh A. Patel, Jason W. Savage, and Wellington K. Hsu in Global Spine Journal