A Prospective Assessment of Knee Arthroscopy Skills Between Medical Students and Residents-Simulator Exercises for Partial Meniscectomy and Analysis of Learning Curves.

BACKGROUND: The Covid-19 pandemic has created the largest disruption of education in history. In a response to this, we aimed to evaluate the knee arthroscopy learning curve among medical students and orthopaedic residents.
METHODS: An arthroscopy simulator was used to compare the learning curves of two groups. Medical students with any prior knowledge of arthroscopy (n=24) were compared to a residents group (n=16). Analyzed parameters were "time to complete a task," assessment of the movement of tools and values scoring damage to the surrounding tissues.
RESULTS: After several repetitions, both groups improved their skills in terms of time and movement. Residents were on average faster, had less camera movement, and touched the cartilage tissue less often than did students. Students showed a steeper improvement curve than residents for certain parameters, as they started from a different experience level.
CONCLUSION: The participants were able to reduce the time to complete a task. There was also a decrease in possible damage to the virtual surrounding tissues. In general, the residents had better mean values, but the students had the steeper learning curve. Particularly less experienced surgeons can especially train their hand-eye coordination skills required for arthroscopy surgery. Training simulators are an important training tool that supplements cadaveric training and participation in arthroscopic operations and should be included in training.

Introduction

Arthroscopy requires different skills than open surgery due to limited visibility, reduced motion freedom, and non-intuitive hand–eye coordination.[1,2] Orthopedic resident surgeons are expected to acquire their early arthroscopy skills under the supervision of an orthopedic consultant.[1,3] Arthroscopic procedures require additional training as complex trajectory may be necessary to perform arthroscopic inspection of a narrow joint space with complex anatomical structures. While in several fields like aerospace, military, or critical care, training simulators are well established training methods although health care lags behind in terms of stakeholder engagement, terms of implantation of simulation outcomes. In surgical training, the Halstead’s method “see one, do one, teach one” has traditionally been preferred. Although for a number of reasons including increased public awareness for medical errors, patient safety, heightened patient expectations, strict regulation of residents’ duty-hours, surgeons’ liability, and an increasing emphasis on the efficient use of operating room time, different studies state that the Halstead training method is not appropriate anymore.[6-8] Computer-based simulation can provide objective quantitative data for the measurement of performance and skills assessment.

For this reason, arthroscopic simulators were developed that also document the paradigm shift in orthopedic education. In a study of McCarthy et al, novices showed significant improvements in task completion time, shorter arthroscope path lengths, shorter probe path lengths, and fewer arthroscope tip contacts when using arthroscopic training simulator with haptic feedback, facilitating the process of acquiring and improving manual skills during arthroscopic surgery.[11,12]

According to the current literature, there is increased use of skills training modalities, and increasing numbers of simulators have been developed as a result.[13-17]

Purpose

The aim of this study was to compare the learning curves of medical students and orthopedic residents during simulated knee arthroscopy procedures using an arthroscopy training simulator.

Methods

A prospective comparative study was conducted. All participants gave their oral and written informed consent for participation in the study. The local ethics committee waived the need for ethics approval.

A total of 24 medical students, without any prior knowledge of arthroscopy (never performed arthroscopy), and 16 orthopedic residents were recruited. The latter had already taken arthroscopy courses (cadaver training course or training with arthroscopic device on a phantom) and performed a small number (less than five) of arthroscopy surgeries.

All participants were familiarized with the equipment and received always the same standardized instructions concerning the arthroscopy simulator. They were taught how to manage the 30° arthroscope and the tools. The knee arthroscopic training simulator GMV/insightArthroVR (GMV, Madrid, Spain) was used (Figure 1). It consisted of an anatomic knee model and two instrument phantoms mounted on a cart, a monitor, and training and simulation software (InsightArthroVR Version 2.9). Knee movements from 0° (total extension) to 90° of flexion as well as varus and valgus movements could be simulated. Both anterolateral and anteromedial portals were represented, enabling various arthroscopy knee exercises. Instruments were simulated with two force feedback robotic arms, with instrument phantoms providing haptic feedback. A 30° angled camera lens enables an arthroscopic display just like in real arthroscopic surgery. The probe was physically represented by the Phantom Omni Stylus (GMV, Madrid, Spain).

Figure 1.

Virtual reality arthroscopy trainer, GMV/insightArthroVR, by courtesy of GMV, Madrid, Spain.

The simulator offers a broad spectrum of tasks from diagnostic arthroscopy to complex surgical procedures. Our study included training modules for diagnostic arthroscopy using the camera only, diagnostic arthroscopy using camera and probe in finding a sequence of spheres, partial meniscectomy at several locations in the menisci. At the outset of each training exercise, the difficulty level must be determined (choosing from initial, intermediate, and expert), resulting in different sizes of the spheres to be palpated. In the current study, the difficulty of all exercises was set at intermediate.

The data were collected over a period of two years in various training sessions. A total of 33 different exercises were carried out by the members of the two groups under investigation.

The following parameters were assessed:

Time [s] to complete the task. The interval started when one of the instruments was inserted through a portal, and ended when the exercise was finished.

The distance covered by the tip of the camera (CDC), the instrument (CDP), and the grasper (CDG) inside the knee phantom. CDP and CDG were held with the right hand only, while CDC was mostly held with the left hand except during diagnostic exercises.

Cartilage damage [mm] was the maximum depth applied to specific damageable tissues according to a contact computation algorithm. The colliding tissues, in which the cartilage damage was considered, were the tibial plateau cartilage, femoral condyles, and articular patella cartilage for the knee. The algorithm computes the distance between the collision point and the maximum penetration. The penetration distance is translated to an opposite-direction force applied by the associated Phantom. This force is what is considered cartilage damage. The parameter was recorded for the camera (PDC) and the probe instrument (PDP) as well as for an open grasper (OG)

Amount of resected meniscus (%): On completion of a meniscectomy the percentage of resected meniscus was determined. The percentage of the remaining meniscus was calculated as the volume of the meniscus left as a proportion of the total volume (Figure 2).

Figure 2.

Example of a meniscectomy tear repair exercise. Trainees had to remove the indicated red area (% amount of resected tear) in order to remove the tear and not exceed the red border using a virtual grasper (% amount of resected meniscus).

Amount of resected tear (%): Shows the percentage of meniscus tear resected. 100% means resection of the whole meniscus volume depicted by experts (view picture). This includes resection of both the tear itself and the part of healthy meniscus that needs to be surgically removed in order to have a smooth meniscus edge (Figure 2).

Statistics

A learning curve is defined as a mathematical description of someone’s progress in gaining experience or a new skill by repeating it for a period of time. In health care, there is no widely accepted standard way to define or measure a learning curve. The most common variable used to measure a learning curve is the operating time. Furthermore, the duration of the surgical learning curve to reach the plateau phase depends on the investigated item and if the surgeons are novice, experienced, senior, younger, or older. In the present study, the term learning curve is used to describe the progress in improving several simulation parameters gained over time by the individual participants.

Learning curves were analyzed using the mean time to complete the task, CDC, CDP, and CDG and by using a logarithmic fitting line to assess the slope of the learning curves. Logarithmic regression was calculated for the learning curves to determine the steepness of the learning curves. Slope, R2, and 95% confidence interval were reported and graphs are shown with log-log line. Logarithmic regression provided the best fit to the learning curves and has been used in several other studies. The higher the slope of the regression curves, the steeper the learning.

Mean and standard deviation were reported for the initial and final exercise performed. Differences between start and end point were analyzed with a T-test for dependent samples for the parameters mean time to complete the task, CDC, CDP, CDG, resected meniscus, and remaining mensiscus. Differences between the two groups regarding their learning curves were analyzed using the two-tailed T-test for independent samples with correction for multiple comparisons using the Holm-Sidak procedure considering the first and last exercises of the learning curve. In all analyses (Version 8.0, GraphPad Software, Inc, La Jolla, US-CA), a P-value of .05 was considered statistically significant in all cases.

Results

Each exercise in the pool of 33 training exercises was repeated on average 39 (median 37, range 1–170) times.

Residents improved their arthroscopic skills by reducing the time to complete the task from 7 min to 2 min on average (P<0.0001), while students reduced the time from 6 min to 3 min (Figure 3) with 38 repetitions. However, the students group showed no statistical significant improvement in time (P=0.171). Residents had a steeper learning curve (learned faster) in terms of “time to complete the task” (slope=−0.37, R2=0.72, 95% CI −0.43 to −0.30) by comparison to students (slope=−0.23, R2=0.45, 95% CI −0.31 to −0.15).

Figure 3.

Learning curves for the mean time including logarithmic regression curves of residents and students performing 38 repetitions of several exercises.

Residents improved their arthroscopic skills by reducing camera movement (CDC) from 2.4 m to .3 m on average over 38 simulations (P<0.001). Residents reduced movement with the probe from 1.1 m to 0.2 m over 17 simulations (P=0.001) while no improvement was observed on the movement with the grasper (CDG) over ten simulations (P=0.776). Students reduced their CDC movements from 2.1 m to .5 m on average over an observation period of 38 simulations (P=0.011) and the CDP from 1.6 m to 0.4 m over 17 simulations (P=0.008). No improvement was observed for CDG over ten simulations (P=0.619). Residents had a steeper learning curve in terms of CDC (slope=−0.50, R2=0.76, 95% CI −0.58 to −.42) in comparison to students (slope=−0.38, R2=0.47, 95% CI −0.49 to −0.26) (Figure 4A). When considering the CDP, the slope of the residents’ learning curve (slope=−0.38, R2=0.59, 95% CI −0.54 to −0.22) was similar to that of students (slope=−0.39, R2=0.63, 95% CI −0.54 to −0.24) (Figure 4B). In CDG, a steeper learning curve was found for students (slope=−0.22, R2=0.41, 95% CI −0.42 to −0.01) than for residents (slope=−0.39, R2=0.66, 95% CI −0.60 to −0.17) (Figure 4C).

Figure 4.

Learning curves for the distance covered with the camera (CDC) (A), with the probe (CDP) (B), and with the grasper (CDG) (C) including the logarithmic regression curves of residents and students in several exercises.

No statistically significant difference was observed between residents and students at the start or the end of the learning curve for the parameters time to complete the task, CDC, and CDG (P>0.05) (Table 1). Residents showed a statistically significant lower CDP at the end of the learning curve (P=0.032) than did students, while no statistically significant difference was found at the start of the learning curve.

Table 1.

Mean (SD) of Time to Complete the Task, CDC, CDP, and CDG were Assessed for the First and Last Exercise and Compared Between Groups.

		Start (SD)	End (SD)	P-value
Time to complete the task (min)	Residents	7 (4)	2 (1)	<0.001
	Students	6 (5)	3 (3)	0.171
	P-value	0.671	0.172
CDC (m)	Residents	2.4 (1.4)	0.3 (0.4)	<0.001
	Students	2.1 (1.3)	0.5 (0.4)	0.011
	P-value	0.484	0.515
CDP (m)	Residents	1.1 (0.7)	0.2 (0.1)	0.001
	Students	1.6 (0.9)	0.4 (0.1)	0.008
	P-value	0.109	0.032
CDG (m)	Residents	1.0 (1.0)	0.9 (0.7)	0.776
	Students	1.5 (2.4)	0.8 (0.1)	0.619
	P-value	0.498	0.795

When considering the cartilage damage values for the camera (PDC), probe (PDP), and grasper (RG), no statistically significant difference was found between residents and students (P>0.05). In 45% of their cases, residents did not touch the bone while moving the camera, while students were able to not touch the bone with the camera in 32% of their cases. The maximum penetration depth of the instruments was around 40 mm for all three tools attached to the simulator (Table 2).

Table 2.

The Mean (SD) of the Maximum Penetration Depth Is Reported for Residents and Students for the Various Instruments (Camera PDC, Probe PDP, and Grasper PDG) Used During the Exercises. The Percentage of Cartilage Contact Is Reported for Residents and Students Based on All Exercises.

		Contact (%)	No Contact (%)	Max Penetration Depth (mm)
PDC	Residents	55	45	37 (9)
	Students	68	32	39 (16)
	P-value			0.714
PDP	Residents	100	0	44 (18)
	Students	100	0	44 (20)
	P-value			0.999
PDG	Residents	100	0	48 (15)
	Students	100	0	47 (20)
	P-value			0.889

Results of the different exercises for removing a meniscus tear are shown in Figure 4. No statistical significant improvement could be observed for the residents meniscus between first and last exercise for the remaining meniscus (P=0.202) as well as for the resected meniscus (0.744) (Table 3). The students group showed also no statistically significant improvement between first and last exercise for the remaining meniscus (P=0.744) and resected meniscus (P=.164) (Table 3). While students showed a higher variation over time in deciding how much tear has to be removed, residents showed a more constant curve (Figure 5A and 5B). However, no statistically significant differences were found between residents and student for the first iteration or after 21 exercises (Table 3). Residents were more conservative in resecting meniscus than were students (Table 3).

Table 3.

The Mean (SD) of the Percentage of Remaining Meniscus as well as the Percentage of Resected Meniscus Was Assessed for the First and Last Exercise and Compared Between Groups.

		Start (SD)	End (SD)	P-value
Remaining meniscus (%)	Residents	70 (18)	82 (9)	0.202
	Students	80 (12)	76 (8)	0.540
	P-value	0.073	0.362
Resected meniscus (%)	Residents	75 (25)	63 (21)	0.744
	Students	80 (20)	81 (7)	0.164
	P-value	0.193	0.912

Figure 5.

Mean percentage of remaining meniscus (A) and resected mensicus (B) divided by groups.

Discussion

Residents showed a statistically significant difference from the first to last exercise, while students did not show a statistically significant improvement in terms of time to complete the task. The learning curve (Figure 1) shows a steeper slope for residents than students in terms of time to compete the task. However, when comparing both groups between each other at the first and last exercise, no statistically significant difference could be found. Therefore, the improvement in terms of time was more visible for the residents group probably due to a better familiarity with the anatomy of the knee joint.

Both groups showed a statistically significant improvement in the hand–eye coordination, which was assessed by the parameters CDC and CDP (Table 3). The learning curve of residents was steeper than the learning curve of students in CDC (Figure 4A) resulting in a statistically significant better outcome for residents at the end point of the curve (Table 3). In CDP, the slope of the learning curve of both groups was similar, and no statistically significant difference could be observed between both groups at start and end points (Figure 4B and Table 3).

When changing the probe (CDP) with the grasper (CDG), surprisingly both groups showed no statistically significant improvement in the learning curves (Table 3, Figure 4C), and there was no statistically significant difference between both groups at start or end point of the learning curve (Table 3). The usage of the grasper was not required in exercises where anatomical structures had to be localized and palpated although it was required in all menisectomy exercises.

When comparing students and residents concerning their performance of several arthroscopy exercises, it was seen that there were no statistically significant differences at the beginning or the end of the learning curves except for movement of the probe, where residents required fewer movements than did students in performing the specific tasks. Students reached the same level of dexterity in almost all cases, with the only exception being CDP at the end of the learning curve.

The training caused a reduction in time needed to finish an exercise, as well as a decrease in possible damage to the virtual surrounding joint tissues (i.e., cartilage).

Less experienced surgeons may therefore benefit more from training on the knee arthroscopy simulator. Nevertheless, even more experienced users were able to enhance their skills as the residents’ knee arthroscopy performance was seen to improve in all investigated exercises.

The high and steep learning curve seen in both groups in our study is well in line with the results previously published by Rahm et al and Dammerer et al In their study, they assume that this rapid improvement of skills is the result of the increased possibilities the training session offers to also simulate difficult arthroscopically guided surgical interventions (i.e., meniscal repair and anterior cruciate ligament reconstruction).[7,21,23,24] The orthopedic resident surgeons in the current study already started from a level of improved knee arthroscopy skills in comparison to the medical students. This might explain why the residents’ learning curve did not turn out to be as steep and high as that of the medical students. Nevertheless, the residents consistently achieved higher scores during the course of knee arthroscopy training, similar to the study by Camp et al and Dammerer et al.[12,22] So far, traditional arthroscopic training during residency lacks a standardized, objective evaluation system, which creates difficulties in comparing different training methods. Martin et al showed in a study the progress of trainees by performing the same exercises before and after cadaveric training on an arthroscopy simulator.

In the meniscectomy exercises, residents improved their skills by increasing the remaining meniscus by 10%, while students showed no improvement. However, residents seemed to remove less necessary meniscus than required by comparison to students although the amount did not reach statistical significance. It seems that residents were more conservative in removing the meniscus as compared to students, who had no prior knowledge of meniscectomy, although this could not be proven statistically (Table 3, Figure 5A and 5B). The goal of the menisectomy exercises was to remove as much as possible the damaged meniscus as shown in Figure 2. Like in real surgical scenarios, the surgeon has to decide case by case how much meniscus tissues they want to remove, by not harming the healthy remaining tear. The goal of the exercise shown as grew surface in Figure 2 with the exact boundaries of the damaged meniscus cannot be displayed in the exercise itself. This might explain why the learning effect is less prominent. Each participant had to find out by attempts if he removed too much or too less of damaged meniscus. While the participants were gaining experience in taking decisions, the assessed parameters were not showing an improvement in both groups as it was unclear like in real surgeries what would be the best compromise in removing damaged tear by maintaining as much as possible healthy meniscus tissue. It has to be also said that grasping event haptics of the simulator should be improved as the amount of resected tissue was very difficult to control by the users.

Both groups, however, showed a large number of cartilage contacts, when using the different tools. Especially when using the pointing or grasping device in each exercise, contact with the cartilage occurred in all exercises (Table 2). When using the camera, residents touched the cartilage tissue in 55% of their cases, while students did so in 68% of their cases (Table 2). It was seen that the number of repetitions was not large enough to reduce possible cartilage damage in most of the cases.

We acknowledge the following limitations of our study. It is not possible to predict any clinical outcome of surgeries in accordance with the use of a training arthroscopy simulator as the training environment differs considerably from a common operation room setting. However, patient outcome will benefit from the skills acquired in the hands-on training. Measurements made with arthroscopy training simulators consider only a few parameters that may not be sufficient to demonstrate the overall learning experience and the complexity of certain arthroscopic surgeries.

When removing the meniscus with an open grasper, poor tactile feedback was noticed, which might influence the quantity of meniscus resected. In addition, the information on how much meniscus should be resected was not visualized in the simulation, which made it impossible to score 100%. However, also in real cases the surgeon has to rely on their personal experience when deciding the resection borders during meniscectomy. Both groups showed a high incidence of touching cartilage structures during the training sessions. Training modules performed on current training simulators should focus on giving appropriate feedback to the trainees concerning damage to the cartilage surface. The effects of training on the arthroscopy simulator may directly lead to a potentially positive impact of arthroscopic interventions in clinical practice. This will result in a benefit for the patient, such as shorter anesthesia duration, reduced risk of infection, as well as a lesser danger for the incorporation of irrigation fluid used in the context of arthroscopy.[27-29]

In conclusion, our results demonstrate the usefulness of an arthroscopy training simulator as an important tool for the improvement of surgical and arthroscopic skills in orthopedic resident surgeons and in medical students. Our study shows a fast (steep) learning curve for orthopedic residents and medical students undergoing a standardized training program on a validated virtual reality-based arthroscopy knee training simulator. Consequently, it can be expected that simulator training sessions will and should become an even more important training tool as a supplement to cadaveric training.