The Effect of Triangulation Simulator Training on Arthroscopy Skills: A Prospective Randomized Controlled Trial.

PURPOSE: To prospectively evaluate the transferability of skills acquired on a low-cost, at-home, nonanatomic triangulation simulation system to cadaveric models.
METHODS: We randomized 28 medical students into either a simulator-training group (n = 14) or group with no training (control, n = 14). All subjects were pretested using a standardized checklist of arthroscopic skills on cadaveric knees and shoulders. Training-group subjects practiced on the triangulation simulator for 90 minutes per week for 4 consecutive weeks. Control subjects received no training. All subjects completed a post-test checklist of arthroscopic skills on cadaveric knees and shoulders, as well as 4 training tasks on the simulator. A blinded orthopaedic surgeon evaluated the arthroscopic videos using the Arthroscopic Surgical Skill Evaluation Tool (ASSET) score.
RESULTS: Training-group knee and shoulder ASSET scores increased from 12.2 ± 1.85 to 14.6 ± 2.76 (P = .02) and from 14.6 ± 3.5 to 17.9 ± 4.5 (P = .29), respectively. In the control group, knee and shoulder ASSET scores increased from 14.3 ± 3.12 to 14.25 ± 4.67 (P = .99) and from 14.2 ± 2.7 to 17.07 ± 6.7 (P = .58), respectively. There were no significant differences in the mean post-test ASSET scores between the training group and control group for either knee or shoulder arthroscopy. The post-test ASSET safety subscore during knee arthroscopy was significantly higher in the training group (P = .03). The training group was able to complete significantly more simulator tasks compared with controls (P = .003) at post-testing. A significant positive correlation was found between knee arthroscopy performance and the number of tasks completed during simulation post-testing (P = .043). There was no significant correlation between shoulder arthroscopy performance and simulation performance (P = .532).
CONCLUSIONS: Basic triangulation skills may be acquired by training on a low-cost, at-home, nonanatomic triangulation simulation system, although the degree of transferability and universal joint applicability, as well as the existence of an early ceiling effect in skill development, could not be shown. LEVEL OF EVIDENCE: Level II, randomized controlled trial.

Arthroscopic procedures have been a staple in orthopaedic surgery for decades. For the most part, surgeons require years of exposure and diligence to develop a competent arthroscopic skill set. With the growing emphasis on surgical efficiency, patient safety, and outcomes-based health care, the quality of orthopaedic arthroscopy training has become a concern. Although the exact number varies based on the individual trainee, a substantial number of arthroscopic cases are required to obtain competence. A difficult learning curve is associated with arthroscopy, with a reported number of approximately 55 arthroscopies needed to obtain competency at diagnostic tasks and close to approximately 250 cases needed to be competent at more complex tasks., In addition, resident work-hour restrictions have further limited resident exposure and repetition, making the development of new effective and efficient residency training tools essential.

In particular, there is heightened interest in implementing arthroscopic simulation into orthopaedic residency training., Currently, a plethora of arthroscopic trainers, which range from low-fidelity, nonanatomic simulation models, such as triangulation box systems, to high-fidelity simulation models, such as anatomic virtual-reality systems, are available. Simulation offers trainees a protected, controlled environment in which they can practice and acquire skills safely outside of the operating room.5, 6, 7, 8, 9 In addition, simulators may provide residents valuable flexibility in the training schedule, and stepwise basic skill proficiency may be monitored.

Despite the proposed advantages of simulation training, there remains no consensus gold-standard simulation model or protocol. Authors have shown that some anatomic knee and shoulder simulators can be valuable arthroscopy training tools and may aid in the development of skills transferable to cadaveric or patient models.10, 11, 12, 13 However, several anatomic virtual-reality or laboratory-based trainers may be impractical or cost prohibitive, leading to barriers to implementation., In addition, most simulators focus on the acquisition of basic skill sets, and very few, if any, have true procedural training capabilities. Although the evidence examining the impact of nonanatomic simulators on the development of clinically relevant arthroscopic triangulation and motor skills is unclear,,16, 17, 18 these low-cost, sometimes portable trainers may solve important accessibility, flexibility, and financial barriers facing some orthopaedic trainees and residency programs.

The ArthroBox (Arthrex, Naples, FL) is a previously validated multi-portal triangulation training system., Previous investigations have shown that training on the system significantly improved triangulation skills and task completion on the system., The purpose of this study was to prospectively evaluate the transferability of skills acquired on a low-cost, at-home, nonanatomic triangulation simulation system to cadaveric models. We hypothesized that training would result in transferable improvements in arthroscopic performance on a cadaveric specimen as assessed by the Arthroscopic Surgical Skill Evaluation Tool (ASSET) score.

Methods

After we obtained institutional review board approval, recruitment of medical student subjects from 3 different allopathic medical schools was commenced via an informational pamphlet by email. Study enrollment occurred between March and May 2018. The inclusion criteria included students of all ages and at all levels of medical training with the ability to use a simulator. The exclusion criteria included previous arthroscopy training, previous arthroscopic simulator training, or previous surgical simulator use of any kind or receipt of any formal arthroscopy education. Participation was voluntary and not related to any student rotation, specialty interest, or evaluation. In total, 37 students responded to the recruitment email. We excluded 6 subjects because of the inability to attend the pretesting, 2 because of prior arthroscopy simulator use, and 1 because of prior formal arthroscopy exposure. Demographic information was collected prior to testing, including age, sex, handedness, and video game use. Demographic data were collected via surveys with questions on video game and musical instrument use scored on a 5-point Likert scale (Appendix).

The study design consisted of a single-blinded, prospective, randomized controlled trial with a parallel-group design. Subjects were randomized into either a simulator-training group (n = 14) or group with no training (control, n = 14). The sample size was limited by the number of volunteers and is comparable with sample sizes in previously published trials; no power analysis was conducted.,, Prior to pretesting, both groups were supplied standardized self-guided arthroscopy orientation material, including videos of a standard diagnostic knee arthroscopy and shoulder arthroscopy and a standardized lecture on the basics of arthroscopy. On the pretesting day, all subjects underwent an informational session including review of diagnostic knee arthroscopy and shoulder arthroscopy as well as a 5-minute overview of relevant anatomy. The information was delivered in a standardized manner to all subjects. Prior to cadaveric testing, subjects were allowed 1 minute to familiarize themselves with the arthroscopic equipment. Subjects were allowed a total of 5 minutes to complete preset standardized arthroscopic checklists for both knee and shoulder cadaveric specimens (Appendix). All cadaveric diagnostic pretests were monitored by a study author (M.L.R.). Subjects were provided a standardized, distraction-free environment with no feedback or simulator access during testing. No third-party intervention was offered during cadaveric testing (no aid was offered to subjects during testing).

Simulator Training

The ArthroBox Arthroscopic Triangulation Training System (Arthrex) was used for training (Fig 1). Each training-group subject was provided an identical simulator and a standardized training protocol consisting of 4 tasks. The training group practiced on the simulator for three 30-minute sessions (90 minutes per week) for 4 weeks. The control group received no simulator orientation or training access throughout the training period.

Fig 1

(A) ArthroBox triangulation system. (B) Example of triangulation training exercise.

Cadaveric Post-Test

Participants in both groups returned between 28 and 30 days later. Prior to the post-test, all subjects were again shown instructional diagnostic arthroscopy videos and provided with an anatomic landmark review. The post-test cadaveric diagnostic knee and shoulder arthroscopies were completed with the identical protocol and conditions used during the pretest. The order in which cadaveric knee and shoulder post-tests were completed was randomized. In addition, all subjects completed 4 skill-based tasks on the simulator with 1 minute to complete each task. The tasks included moving rubber bands from peg to peg, creating a box, creating a cross, and pushing a ring to the opposite side of a helix (Fig 2). A prior randomized controlled trial by Frank et al. using the same system showed that the group without training did not display an increased task completion rate. In light of this finding, the decision was made not to have control subjects complete the 4 aforementioned tasks at pretest to limit the exposure of the control group to the training system as a confounding factor. Pretest and post-test arthroscopic videos were evaluated by a blinded orthopaedic surgeon (G.C.). The primary outcome was the ASSET score.,

Fig 2

Triangulation training tasks, including moving rubber bands from peg to peg (A), creating a box (B), creating a cross (C), and pushing a ring to the opposite side of a helix (D).

Arthroscopic Surgical Skill Evaluation Tool

The ASSET allows the assessment of global arthroscopic skill. The tool is a video-based assessment that allows trained orthopaedic surgeons to use intraoperative video as a means to evaluate 8 domains of arthroscopic skill. The ASSET has previously been shown to be a useful, reliable, and valid method for evaluating the performance of diagnostic arthroscopy in both the operating room and surgical simulation laboratory.,

Statistical Analysis

Statistical analysis was performed using descriptive statistics, χ2 testing, independent-samples t tests, paired-samples t tests, multivariate linear regression, and Spearman correlation. The multivariate linear regression model included age, sex, year in medical school, and number of post-test tasks completed on the simulator. Independent-samples t tests were used to analyze the means between groups. Paired-samples t tests were used to analyze the means within groups between time points. Spearman correlation was used to analyze the correlation between year of medical school, developmental video game history, current video game use, and number of post-test task completions and the post-test ASSET score. All reported P values are 2-tailed with an α level of .05 signifying significance (SPSS Statistics, version 25.0; IBM, Armonk, NY).

Results

Demographic Characteristics

A total of 28 subjects completed the study, with 14 participants in each group. All subjects completed all portions of the trial. The average ages of the control and training groups were 25.42 ± 1.70 years and 25.21 ± 2.69 years, respectively (P = .80). The average number of years of medical school training was 1.79 ± 1.05 for the control group and 1.71 ± 1.07 for the training group (P = .86). No significant differences in sex, handedness, current video game use, or video game use during development were found between the groups (Table 1).

Table 1

Baseline Patient Demographic Factors

Demographic Factors	Training Subjects	Control Subjects	P Value
n	14	14
Age, yr	25.2 (2.6)	25.4 (1.7)	.80
Sex, n	12 M and 2 F	11 M and 3 F	.86
Handedness, n	2 L and 12 R	2 L and 12 R	.99
Years of medical school training	1.71 (1.06)	1.79 (1.05)	.86
Current video game use	1.64 (0.50)	1.71 (1.27)	.84
Video game use during development	3.57 (1.01)	3.21 (1.05)	.37

NOTE. Data are presented as mean (standard deviation).

F, female; L, left; M, male; R, right.

Cadaveric Knee Arthroscopy ASSET Scores

Knee arthroscopy pretest and post-test ASSET scores are displayed in Figure 3 for the training group and Figure 4 for the control group. In the training group, mean knee ASSET scores significantly increased from 12.2 ± 1.85 at pretest to 14.6 ± 2.76 at post-test (P = .01), whereas in the control group, the mean ASSET scores were not statistically different from pretest (14.3 ± 3.12) to post-test (14.25 ± 4.67, P = .99). No significant difference was found between the training and control groups’ post-test mean knee ASSET scores (P = .79). At final testing, the training group displayed nonsignificant improvement in the mean change in total ASSET score (2.45 ± 3.22 in training group vs 0.17 ± 5.9 in control group, P = .20). On ASSET subscore analysis, the training group had a significantly higher mean post-test ASSET safety subscore (2.79 ± 0.70) than the control group (2.00 ± 1.04, P = .04). In addition, a significant improvement in mean change in ASSET safety subscore was observed in the training group (0.79 ± 0.90) compared with the control group (–0.42 ± 1.08, P = .003). No other significant differences in the mean change in ASSET subscores were observed (Table 2). No significant correlations were found between year in medical school (Spearman ρ = 0.110, P = .592), current video game use (Spearman ρ = –0.256, P = .207), or developmental video game history (Spearman ρ = 0.157, P = .443) and post-test knee ASSET score. Multivariate linear regression was run to predict the post-test knee ASSET score from age, sex, year in medical school, and number of post-test tasks completed on the simulator. After regression, only the number of post-test tasks completed on the simulator contributed significantly to predicting the post-test knee ASSET composite score (P = .027).

Fig 3

Cadaveric knee Arthroscopic Surgical Skill Evaluation Tool (ASSET) scores in training group. Asterisks signify significant differences between pretest and post-test ASSET scores. (FOP, flow of procedure; FOV, field of view; QOP, quality of procedure.)

Fig 4

Cadaveric knee Arthroscopic Surgical Skill Evaluation Tool (ASSET) scores in control group. (FOP, flow of procedure; FOV, field of view; QOP, quality of procedure.)

Table 2

Mean Difference in Cadaveric Knee ASSET Scores Between Pretest and Post-Test

ASSET Score Category	Difference in Knee ASSET Score		P Value
	Training Group	Control Group
Composite	2.43 (3.23)	0 (6.05)	.20
Subscore
Safety	0.79 (0.80)	–0.42 (1.08)	<.01∗
Field of view	0.31 (0.94)	0.08 (0.90)	.55
Camera dexterity	0.64 (0.84)	0.08 (0.90)	.11
Instrument dexterity	1.66 (0.83)	–0.33 (0.71)	.16
Bi-manual dexterity	0.29 (1.06)	0 (1.23)	.51
Flow of procedure	0.36 (0.63)	0 (0.95)	.27

NOTE. Data are presented as mean (standard deviation).

ASSET, Arthroscopic Surgical Skill Evaluation Tool.

Statistically significant (P < .05).

Cadaveric Shoulder Arthroscopy ASSET Scores

Shoulder arthroscopy pretest and post-test ASSET scores are displayed in Figure 5 for the training group and Figure 6 for the control group. In the training group, mean shoulder ASSET scores improved from 14.6 ± 3.5 at pretest to 17.9 ± 4.5 at post-test (P = .29). The control group’s mean ASSET scores were not statistically different from pretest (14.2 ± 2.7) to post-test (17.07 ± 6.7, P = .58). No significant difference was found between the training and control groups’ mean shoulder post-test ASSET scores (P = .07). At final testing, the training group did not display a larger improvement in mean change in total ASSET score (1.25 ± 4.77 in training group vs 1.27 ± 5.52 in control group, P = .99). In addition, no significant differences in the mean changes in ASSET subscores were observed (Table 3). No significant correlations were found between year in medical school (Spearman ρ = –0.148, P = .463), developmental video game history (Spearman ρ = –0.173, P = .390), or current video game use (Spearman ρ = 0.133, P = .509) and post-testing shoulder ASSET score. Multivariate linear regression was run to predict the post-test shoulder ASSET composite score from age, sex, year in medical school, and number of post-test tasks completed on the simulator. None of the variables contributed significantly to predicting the post-test shoulder ASSET composite score.

Fig 5

Cadaveric shoulder Arthroscopic Surgical Skill Evaluation Tool (ASSET) scores in training group. (FOP, flow of procedure; FOV, field of view; QOP, quality of procedure.)

Fig 6

Cadaveric shoulder Arthroscopic Surgical Skill Evaluation Tool (ASSET) scores in control group. (FOP, flow of procedure; FOV, field of view; QOP, quality of procedure.)

Table 3

Mean Difference in Cadaveric Shoulder ASSET Scores Between Pretest and Post-Test

ASSET Score Category	Difference in Shoulder ASSET Score		P Value
	Training Group	Control Group
Composite	1.27 (5.51)	1.25 (4.77)	.99
Subscore
Safety	–0.82 (2.32)	0.00 (1.20)	.29
Field of view	–0.72 (2.72)	0.33 (0.78)	.21
Camera dexterity	–1.00 (3.10)	0.00 (0.60)	.28
Instrument dexterity	–1.27 (3.80)	0.25 (0.86)	.19
Bi-manual dexterity	–1.63 (4.27)	0.08 (0.79)	.18
Flow of procedure	0.27 (1.01)	0.17 (0.71)	.27

NOTE. Data are presented as mean (standard deviation).

ASSET, Arthroscopic Surgical Skill Evaluation Tool.

Simulator Post-Testing Analysis

On simulator post-testing after training, the training group was able to complete significantly more tasks on average (2.1 ± 1.0 of 4 tasks) than the control group (0.8 ± 0.7 of 4 tasks, P = .003). A significant positive correlation was found between knee arthroscopy performance and the number of tasks completed during simulation post-testing (Spearman ρ = 0.417, P = .043). No significant correlation was found between shoulder arthroscopy performance and the number of tasks completed (Spearman ρ = –0.134, P = .532).

Discussion

The principal findings of this study suggest that although an increase in the mean ASSET score from baseline in the training group compared with the control group was observed, the difference was not statistically significant. In addition, this study suggests the following: Triangulation simulator training did not improve post-test knee or shoulder ASSET scores compared with the control post-test scores, triangulation simulator training significantly improved post-test knee arthroscopy ASSET scores compared with pretest scores, the training group was able to complete significantly more tasks on the triangulation simulator at final testing than the control group, there was a positive correlation between the number of simulator tasks completed at final testing and knee ASSET scores, and triangulation simulator training did not impact shoulder arthroscopy skill-set development.

Currently, limited evidence exists on the transfer validity of nonanatomic simulation training, such as the ArthroBox system, to anatomic simulation models for the development of basic triangulation skills. Two studies have evaluated the transferability of laparoscopic simulation training to arthroscopy., In a randomized controlled trial by Akhtar et al., trainees who underwent laparoscopic simulation training showed improvements in completion time and economy of movement on arthroscopic tasks from baseline. In addition, Safir et al. reported that a laparoscopic training protocol improved performance on specific arthroscopic tasks. Furthermore, in a randomized controlled trial of 43 novice trainees, Cychosz et al. investigated the transferability of nonanatomic arthroscopic training to an anatomic knee model. They reported a significantly greater composite performance in the training group, as well as an improved camera path length.

Despite nonsignificant differences in post-test knee or shoulder ASSET scores between the training and control groups, our investigation showed that nonanatomic triangulation training did improve final knee ASSET scores, knee ASSET camera dexterity subscores, and knee ASSET safety subscores compared with pretesting. In addition, triangulation training improved performance on the simulation trainer, and performance on the simulator trainer was positively correlated with improved performance on the final cadaveric knee model. It is important to note that previous investigations have also shown that training on the ArthroBox system significantly improved triangulation, particularly task completion on the system., However, the specific aforementioned correlation found in this study may be due in part to more manually skilled subjects performing superiorly and may not be a direct result of training. The current findings, in conjunction with previous investigations, suggest that skills acquired by nonanatomic simulation may aid in the development of some early basic arthroscopic triangulation skills in knee arthroscopy, yet the clinical transferability of the learned skills to clinical arthroscopy could not be shown.

It is important to mention that several studies have reported that nonanatomic simulation training has limited or no benefit in improving arthroscopic skill. Ferguson et al. investigated the transferability of knee simulation training to shoulder arthroscopy and vice versa. They found no evidence of skill transferability from one joint to the other and observed diminishing marginal benefit with continued additional training. Ström et al. echoed the previous finding by reporting that training protocols with several different non-knee simulator models did not improve performance or skill level on an anatomic knee model. Both of the aforementioned investigations challenge the current existence of a generalizable simulator model and assert that simulation training may experience a ceiling effect, providing only marginal or diminishing returns in development.

These valid assertions highlight some limitations in our study. First, the control group’s pretest knee ASSET scores were higher than the training group’s scores, yet both groups displayed similar post-test values. This finding is concerning for a proposed ceiling effect and may be responsible for the isolated improvement observed in the training group. A ceiling effect on the overall benefit of triangulation training may also be responsible for the lack of a significant difference between the post-test control and training groups because once the maximal benefit is achieved, ASSET score improvement becomes stagnant. Unfortunately, our investigation, as currently designed, cannot evaluate for a proposed ceiling effect with training on the current system. Moreover, this study suggests that triangulation training improved final knee scores, but no improvement was identified on shoulder ASSET composite scores or ASSET subscores. These findings emphasize the notion that arthroscopic skills acquired on a simulator may not be universally transferrable to all joints. In addition, it is unclear whether the benefits shown in the training group were achieved through triangulation training or from an additional exposure to the cadaveric model. The investigated triangulation system was designed to practice basic skills with a video-optic arthroscopy system. Although these skills are imperative to successful arthroscopy, they also require a strong foundation of anatomic and pathologic knowledge. The effects from pretest to post-test time points are likely partially a result of better understanding the anatomy after repeated exposure to cadaveric testing and only in part a result of improved basic triangulation skills. Finally, although some of our investigation’s findings are statistically significant, the clinical value of the reported improvements, as well as how to interpret marginal improvements in ASSET scores, remains unclear.

Recently, there has been an increasing emphasis on clinically significant findings as opposed to statistically significant findings. Many outcome measurements, such as the ASSET, require further investigation to determine clinically significant changes if and when they are used. To date, the ASSET has been validated to assess competency as a pass-fail examination, with competency being defined as a score of 3 or greater on all 8 domains. Besides the knee safety subscore, the training subjects displayed no difference in ASSET composite or subscore values compared with controls. Thus, it is difficult to elucidate whether the observed improvements in ASSET scores are clinically relevant. In summation, owing to the study design and limitations in technology, caution must be used when recommending triangulation training for the development of universally clinically significant skills.

When the utility of an arthroscopic simulator is being evaluated, it is imperative to consider the goals of the target trainee in conjunction with the ability of the simulator to help the trainee obtain those goals. Proper attention to basic foundational skills, such as probing and triangulation, is often neglected. Although the clinical value of triangulation training solely via the ArthroBox remains unclear, the system has been shown to be able to discriminate between participants of different levels of triangulation skill, and use of the system improves basic triangulation skills performed on the system., These basic skills may be beneficial to novices currently setting the foundation for video-optic arthroscope use, allowing improved familiarity, comfort, and focus on advanced training. Further studies of basic triangulation systems, such as the ArthroBox, in tandem with anatomic knowledge, as well as more advanced training, may be warranted.

Although arthroscopic simulation is increasingly being used and supported to augment traditional orthopaedic training, several key limitations still exist in the field. Despite the growing sophistication of high-fidelity models, technology remains a limitation to meaningful simulation training. Most models, including the currently investigated system, are designed only to acquire basic skill sets. Few validated simulators, if any, have true procedural training capabilities, and the lack of mandated proficiency testing on available models may be stifling technological progress. In addition, funding for orthopaedic training remains a challenge.,, In a systematic review examining surgical simulation training, fewer than 2% of included studies reported any cost analysis. Once more is known about the cost-effectiveness and the return on investment of simulation training, a stronger economic stimulus may spark private-sector development of more advanced technology. Regarding our study, the low-cost nature of the ArthroBox system may prove valuable, and future investigations on the cost-effectiveness of the system when augmenting triangulation training are warranted.

Limitations

This investigation has several limitations. First, although our sample size is comparable to much of the current body of literature, the sample size and the power of the study were limited by the number of volunteers. Moreover, our study was conducted using a single triangulation simulator model, and thus, assumptions cannot be made on the effectiveness of similar models. The study population (medical students) may not fully represent the target population of novice orthopaedic trainees. This limitation is exemplified by a lower mean pretest ASSET score compared with other studies that classified orthopaedic surgery residents as “beginner”-level trainees, who intuitively would have higher mean pretest ASSET scores than our population. In addition, the number of female participants was limited by the number of volunteers. Finally, the study population’s limited or variable baseline anatomic knowledge may have limited the ability to accurately show improvement, with the students who had more advanced knowledge potentially benefitting more from cadaveric model exposure.

Conclusions

Basic triangulation skills may be acquired by training on a low-cost, at-home, nonanatomic triangulation simulation system, although the degree of transferability and universal joint applicability, as well as the existence of an early ceiling effect in skill development, could not be shown.