Effect of Teaching Session on Resident Ability to Identify Anatomic Landmarks and Anterior Cruciate Ligament Footprint: A Study Using 3-Dimensional Modeling.

BACKGROUND: Femoral tunnel positioning in anterior cruciate ligament reconstruction (ACLR) is an intricate procedure that requires highly specific surgical skills.
PURPOSE: To report the ability of residents to identify femoral landmarks and the native ACL footprint before and after a structured formal teaching session as a reflection of overall surgical skill training for orthopaedic surgery residents in Canada. STUDY
DESIGN: Controlled laboratory study.
METHODS: A total of 13 senior orthopaedic residents were asked to identify a femoral landmark and an ACL footprint on ten 3-dimensional (3D)-printed knee models before and after a teaching session during the fall of 2018. The 3D models were made based on actual patients with different anatomic morphologic features. ImageJ software was used to quantify the measurements, which were then analyzed through use of descriptive statistics.
RESULTS: Before and after the teaching session, residents attempted to identify a specific anatomic location (bifurcate and intercondylar ridge intersection) with a mean error per participant ranging from 5.00 to 10.95 mm and 4.79 to 12.13 mm in magnitude, respectively. Furthermore, before and after the teaching session, residents identified the specific position to perform the surgical procedure (ACL femoral footprint), with a mean error per participant ranging from 4.58 to 8.80 mm and 3.87 to 11.07 mm in magnitude, respectively. The teaching session resulted in no significant improvement in identification of either the intersection of the bifurcate and intercondylar ridges (P = .9343 in the proximal-distal axis and P = .8133 in the anteroposterior axis) or the center of the femoral footprint (P = .7761 in the proximal-distal axis and P = .9742 in the anteroposterior axis).
CONCLUSION: Although a formal teaching session was combined with a hands-on session that entailed real surgical instrumentation and fresh cadaveric specimens, the intervention seemed to have no direct impact on senior residents' performance or their ability to demonstrate the material taught. This puts into question the format and efficacy of present teaching methods. Also, it is possible that the 3D spatial perception required to perform these skills is not something that can be taught effectively through a teaching session or at all. Further investigation is required regarding the effectiveness and application of surgical skill laboratories and simulations on the competencies of orthopaedic residents.

Surgical education has evolved with the aim of increasing patient safety and broadening the skill competencies of graduating residents.[39] Due to restrictions in residents’ working hours, Canadian orthopaedics programs are transitioning toward competency-based training to optimize practical abilities. A crucial component of an orthopaedic surgeon’s residency training is understanding the intricate complexities of the musculoskeletal system. This is particularly imperative with regard to the anterior cruciate ligament (ACL) and its anatomic footprint, given that ACL injuries are among the most common knee conditions treated surgically.[17] The rate of ACL reconstruction is estimated to be roughly 200,000 per year in the United States.[10,41] Studies have shown that among patients receiving surgical treatment, roughly 10% to 15% will experience either a rerupture or clinical failure requiring revision surgery.[13] This failure occurs for many reasons, including technical mistakes, chronic or acute trauma, biological causes, and infection.[34]

Within the category of technical errors, nonanatomic tunnel placement in either the femur, the tibia, or both represents 70% to 80% of errors.[17,25] More specifically, femoral tunnel malposition was the most common cause of technical failure (80%) reported by the Multicenter ACL Revision Study (MARS) group.[34] This malpositioning is believed to increase graft stress, resulting in changes to graft length and tension and ultimately causing failure.[12,24,29,32] For these reasons, research has been conducted to characterize morphologic features and location of the ACL femoral tunnel.[6]

Surgeons use radiography, computer-assisted systems, drill guides, general rules of thumb, and anatomic reference points to aid with proper graft positioning.[4,16,18,20,36] However, intraoperative positioning still represents a challenge that is ultimately determined by the surgeon’s working knowledge of the musculoskeletal system. The intercondylar and bifurcate ridges have been defined as specific intra-articular femoral landmarks that can be used intraoperatively to assess ideal graft position.[9,11,30] Therefore, practical training for residents is an essential component of their program that will prepare them to make decisions during difficult intraoperative situations.

The objective of this study was to determine the effect of a structured formal teaching session on residents’ ability to identify anatomic femoral landmarks, specifically the intersection of the intercondylar and bifurcate ridges, and positioning of the ACL footprint, using 3-dimensional (3D) models from actual patient scans. The primary goal of this study was to determine the magnitude and direction of error of the residents’ identification of the intercondylar and bifurcate ridge intersection and femoral tunnel position, relative to the radiographically defined true intersection and ACL footprint. The secondary goal of this study was to compare the magnitude and direction of this error before and after a teaching session. We hypothesized that the teaching session would improve the ability of orthopaedic surgery residents to identify the point of intersection of the intercondylar and bifurcate ridges as well as the placement of the ACL femoral graft.

Methods

Study Protocol

Institutional review board approval was obtained prior to the onset of this study. Drawing on history and physical examination, our research coordinator recruited patients who were suspected to have an acute ACL tear. Patients with previous knee conditions including previous surgery, previous ligamentous injury, inflammatory arthropathy, or osteoarthritis and patients with a suspected multiligamentous knee injury were excluded. The study ultimately included 6 women and 14 men, with a mean ± SD age of 33.7 ± 11.33 years (median, 32 years). As part of the normal preoperative workup, a conventional 2-dimensional (2D) magnetic resonance imaging (MRI) protocol was performed to confirm the diagnosis of an acute ACL tear. Afterward, a 3D MRI scan was performed on the injured knee to acquire high quality images of a skeletally mature knee before undergoing ACL reconstruction.

The imaging protocol for this study was a previously validated isotropic 3D MRI protocol.[19,21,27] Both 2D and 3D MRI scans were performed by use of the same 1.5-T TwinSpeed Excite high-definition MRI scanner (GE Medical Systems). In the scanner, the knees were positioned in near full extension, and an 8-channel high-definition surface coil was applied. The 3D MRI entailed an oblique-coronal proton density sequence along the plane of the ACL with slice gaps of 0.6 mm.[19,21]

3D Model Generation

The 3D MRI DICOM was imported in 3D slicer software.[7,14,28,37,38] This software enables segmentation of the DICOM to create a 3D model that mimics the native anatomic features. Models were then printed with a QidiTech1 dual extruder 3D printer. Technical information on the printer parameters used can be found in Table 1. All 20 models were created from actual patient scans, thus reproducing the variability in features that one can expect from patient to patient in a clinical setting. The apex of the deep cartilage (ADC) is a landmark that can be easily identified arthroscopically. Anatomically, the ADC represents the proximal and anterior corner of the articular cartilage margin located on the medial side of the lateral femoral condyle. Using this previously validated reference point, a senior musculoskeletal radiologist (M.B.) identified the ADC on the 3D MRIs of all the patients.[20] That same point was then marked on the 3D models using multiplanar DICOM views overlaid on the 3D model to serve as a reference point for the measurements.

TABLE 1

Technical Specification of the 3-Dimensional Printer Used

Filament type	PLA
Filament diameter	1.75 mm
Extruder temperature	200°C
Build plate temperature	60°C
Nozzle diameter	0.40 mm
Resolution (primary layer height)	0.1 mm
Top solid layer	8
Outer perimeter shells	3
Infill	15%, rectilinear pattern

PLA, polylactic acid.

Ridge and Femoral Footprint Analysis

Senior orthopaedic residents were asked to use a wooden pin to identify the intersection between the intercondylar ridge and the bifurcate ridge on 10 different 3D models in 1 of 2 stations set up for this purpose (Figure 1). Afterward, the residents were asked to identify the center of their preferred femoral tunnel location on the same set of models. Measurements of those positions were made with respect to the ADC, identified on the models with the help of a standard picture. ImageJ software[40] was used to perform the measurements. The same task was performed by the resident before and after a teaching session described below. The measurements were then performed by 2 separate observers (C.L. and E.H.) to corroborate the measurement method. Each observer was blinded to the measurements of the other observer. Because the 3D models were generated from MRI scans, reference measurements were performed on the MRI itself by the same senior musculoskeletal radiologist. The center of the native femoral footprint was identified following a previously validated method.[21] The intersection of the native intercondylar and bifurcate ridges was also determined with MRI, which was used as a reference measurement.

Figure 1.

Resident identifying the intersection of the ridges and location of the center of the femoral footprint. Each model was positioned in the same orientation such that it represented a knee at 90° of flexion. The ruler was used as a gauge for measurements, which were subsequently analyzed through ImageJ software via a digital caliper.

Teaching Session

The didactic teaching session totaled a duration of 4 hours and was led by sports fellowship–trained orthopaedic surgeons from 5 Canadian academic teaching centers. The session covered a variety of topics and included a 30-minute morning lecture discussing anatomic ACL reconstruction principles, landmarks, and techniques. The landmarks discussed included the bifurcate and intercondylar ridges as well as the ADC. The didactic session was then followed in the afternoon by a 3-hour practical training session with fresh cadaveric knees in a fully equipped surgical simulation center (Shriners Hospital, Montreal, QC, Canada). A total of 8 arthroscopic stations with a complete arthroscopic ACL instrumentation set were accessible for training. Each resident was given the same opportunity at each station while being supervised by a sports fellowship–trained staff surgeon, who provided guidance and the possibility of hands-on teaching to ensure uniform experience and education across the residents.

Statistical Analysis

Descriptive statistics and the Student t test were used to describe and compare the positions identified by the surgeons on the 3D model relative to the native structures. A Kruskal-Wallis 1-way analysis of variance was then used to compare the error distances between the 13 residents based on the results of a Shapiro-Wilks test. All statistical analysis was conducted with the MatLab software suite (MatLab R2018a; The MathWorks). A P value less than .05 was deemed statistically significant. The intraclass correlation coefficient was used to assess the agreement between both observers as defined by McGraw and Wong.[35]

Results

A total of 13 orthopaedic surgery residents were recruited for this study from 3 different Canadian orthopaedic residency programs (7 fourth-year residents and 6 fifth-year residents). More than 250 ACL reconstructions are performed per year in each of the teaching programs, with a dedicated fellowship-training-led sport rotation in each. Throughout the residency, each resident receives exposure to a dedicated sport rotation that lasts between 4 and 6 months (2-3 months as a junior R1-2-3 and 2-3 months as a senior R4-5) and an additional 3 to 4 months of sports exposure with a fellowship-trained sports surgeon in a community setting. All residents go through graduated learning in the operating room based on competencies achieved such that they progress from assistant role to primary supervised surgeon for select parts of the case, and potentially to primary surgeon for the whole case.

Regarding the residents' identification of the intersection of the intercondylar and bifurcate ridges, the error distribution of the points is shown in Figure 2. Before the teaching session, no resident was able to consistently identify the junction of the ridges (P < .05) (Table 2), as the mean error per participant when compared with the reference measurements ranged from 5.00 to 10.95 mm in magnitude. After the teaching session, the residents were still unable to identify the junction of the ridges (P < .05), with a mean error ranging from 4.79 to 12.13 mm in magnitude. The point actually identified by the residents was proximal and posterior relative to the native ridge in 82% of the cases. We noted that 11 of the 13 residents had a similar magnitude of error regarding the intersection of the ridges compared with before the teaching session (P > .05) (Table 2). Only 1 resident (resident 2) performed significantly better after teaching, and 1 did significantly worse (resident 6) after teaching. Overall, the positions identified by the residents were not significantly different in the proximal-distal axis (P = .9343) or the anteroposterior axis (P = .8133). When we compared the error distance of the 13 residents, we noted a significant difference between them according to Kruskal-Wallis 1-way analysis (P = 4.085e-05) (Figure 3).

Figure 2.

Error distribution of identified intersection of the intercondylar and bifurcate ridges by residents on 3-dimensional models. Each color represents a different participant, with the (0,0) point representing the intersection of the ridges on the magnetic resonance imagings of the native knees. The filled circles represent data before the teaching session, and the diamonds represent data after the teaching session.

TABLE 2

Intersection of the Intercondylar Ridge and Bifurcate Identified by Residents on 3D Femoral Model vs Native Anterior Cruciate Ligament

	Magnitude of Error Before Teaching Session	Magnitude of Error After Teaching Session	P Value Between Pre- and Postsession Findings
Resident 1	10.95 ± 2.48b	12.13 ± 3.20b	.3690
Resident 2	9.83 ± 3.18b	6.71 ± 2.64b	.0277
Resident 3	6.55 ± 2.04b	8.93 ± 3.14b	.0596
Resident 4	8.30 ± 3.51b	7.38 ± 2.92b	.2012
Resident 5	5.14 ± 2.69b	5.76 ± 2.14b	.5794
Resident 6	8.05 ± 3.21b	11.40 ± 3.44b	.0367
Resident 7	5.00 ± 2.21b	4.79 ± 2.06b	.8329
Resident 8	6.73 ± 2.49b	6.68 ± 2.34b	.9615
Resident 9	6.53 ± 2.45b	6.68 ± 2.95b	.9039
Resident 10	8.38 ± 3.18b	9.15 ± 3.41b	.6055
Resident 11	7.70 ± 3.33b	7.31 ± 2.46b	.7710
Resident 12	7.96 ± 3.51b	7.53 ± 2.90b	.7658
Resident 13	7.99 ± 3.02b	6.10 ± 2.97b	.1740

aValues are expressed in millimeters as mean ± SD.

< .05 between identification on 3-dimensional (3D) femoral model and native anterior cruciate ligament.

Figure 3.

Boxplot of error distances (native vs reconstructed footprint position) by each resident for the ridges. The central mark is the median, whereas the edges of the box are the 25th and 75th percentiles. Differences between surgeons and their respective techniques resulted in different error distances.

Regarding the residents’ identification of the optimal location for femoral anatomic ACL reconstruction, the overall error distribution is shown in Figure 4. Again, no resident was able to appropriately identify the center of the femoral footprint on the anatomic 3D models in either the proximal-distal or anteroposterior axis before the teaching session (P < .05 compared with reference measurement), with a mean error ranging from 4.58 to 8.80 mm in magnitude (Table 3). After the teaching session, the residents were again unable to identify the femoral footprint (P < .05), with a mean error ranging from 3.87 to 11.07 mm in magnitude. The footprint actually identified by the residents was proximal and posterior relative to the native ACL in 76.9% of the cases. Interestingly, 4 of the 7 residents who had a greater postteaching mean error on the ridges also had a greater postteaching error for the location of the ACL. Further, 11 of the 13 residents had a similar magnitude of error regarding the center of the femoral footprint compared with before the teaching session (P > .05) (Table 3). The same residents performed significantly better (resident 2) and significantly worse (resident 6) after the teaching session. Overall, the position identified by the residents was not significantly different in the proximal-distal axis (P = .7761) or the anteroposterior axis (P = .9742). As shown in Figure 5, when we compared the error distance among the 13 residents we saw a significant difference according to Kruskal-Wallis 1-way analysis (P < .05).

Figure 4.

Error distribution of identified tunnel position by residents on 3D models. Each color represents a different participant, with the (0,0) point representing the center of the femoral footprint on the magnetic resonance images of the native knees. The filled circles represent data before the teaching session, and the diamonds represent data after the teaching session.

TABLE 3

Residents’ Anterior Cruciate Ligament (ACL) Tunnel Placement vs Native ACL Footprint on 3D Femoral Model

	Magnitude of Error Before Teaching Session	Magnitude of Error After Teaching Session	P Value Between Pre- and Postsession Findings
Resident 1	7.91 ± 1.61b,c	9.67 ± 2.26b,c	.0606
Resident 2	8.80 ± 2.88b	4.98 ± 2.26b	.0040
Resident 3	5.03 ± 1.51b,c	6.34 ± 2.86b,c	.2198
Resident 4	7.89 ± 3.78b	6.62 ± 1.37b	.3311
Resident 5	4.70 ± 1.97b	4.57 ± 1.60b	.8698
Resident 6	8.22 ± 1.63b	11.07 ± 2.29b	.0049
Resident 7	4.58 ± 1.69b	3.87 ± 1.66b	.3555
Resident 8	7.18 ± 1.62b	6.51 ± 1.58b	.3611
Resident 9	7.06 ± 1.72b	5.77 ± 2.69b	.2186
Resident 10	6.62 ± 3.03b	7.35 ± 2.96b	.5892
Resident 11	6.78 ± 2.05b,c	7.94 ± 2.20b	.2366
Resident 12	6.65 ± 2.69b	7.28 ± 2.21b,c	.5755
Resident 13	7.78 ± 3.02b	5.70 ± 1.53b	.0678

Values are expressed in millimeters as mean ± SD.

< .05 between identification on 3-dimensional (3D) femoral model and native ACL.

> .05 in anteroposterior axis only.

Figure 5.

Boxplot of error distances by each resident for the femoral tunnel.

Regarding the accuracy of the measurement method, for the 2 independent observers who performed the measurement, we noted an intraclass coefficient of 0.9545 in the distal-proximal axis and 0.9898 in the anteroposterior axis, indicating excellent interrater agreement.

Discussion

Medical education has evolved over the years to optimize the training of residents to prepare them for various challenges throughout their careers.[8] A crucial aspect of their training is the ability to extrapolate what is learned to actual patients. This is especially important for procedures where individual musculoskeletal anatomy is relevant, such as ACL reconstruction. Although people have the same major landmarks, minor variability exists that has the potential to affect biomechanics and lead to improper placement of surgical hardware and grafts.[5] This is reflected in this study by the variability of the locations of the ridge junction and native ACL footprint among the different 3D models. Hence, understanding these variabilities and how to adjust for them intraoperatively is important. This is particularly relevant with regard to femoral tunnel placement within the native ACL footprint, which has varied locations and morphologic characteristics among the population.[23,31] Osseous landmarks have been described to aid with this process, yet determining the exact location for femoral tunnel placement remains difficult.[9] Participating residents in this study were unable to identify the intersection of the bifurcate and intercondylar ridges (P < .05 compared with reference measurements). Inaccurate intraoperative identification of the native footprint remains a problem that hinders precise ACL reconstruction.[3,41]

Despite the evolution of medical education, the participating residents in this study were unable to determine specific osseous landmarks accurately. Before the training session, the residents marked the location of the junction of the intercondylar and bifurcate ridges, as well as that of the native ACL footprint, with a mean error ranging from 5.00 to 10.95 mm and 4.58 to 8.80 mm in magnitude, respectively. After the teaching session, residents inaccurately identified the aforementioned landmarks with a greater error range: from 4.79 to 12.13 mm and 3.87 to 11.07 mm in magnitude, respectively. This result implies that the teaching session did not have a significant, immediate effect on the ability of the residents to identify those landmarks.

Interestingly, when analyzing the location of the points identified by the residents, we found that the majority of misses (82% for the ridges and 76.9% for the ACL footprint) were located in the proximal-posterior quadrant relative to the native ACL. There was no mention of this position or quadrant during the teaching session. Interestingly, misses in this quadrant were similar to those that we demonstrated in a separate study performed on actual patients with senior fellowship-trained surgeons (unpublished data). Therefore, there seems to be some preexisting bias in the senior residents that created this systematic error pattern. This suggests that the error was not purely random but instead converged in an area where the residents believed it was safer to place the graft (posterior and proximal). This finding also may show that the trend by residents to miss in this direction reflects something they have routinely seen beforehand in their surgical exposure. Thus, further research is needed to investigate the root cause of repeated systematic error by both residents and staff in identifying and reproducing the locations of key ACL specific landmarks.

A national survey conducted by the American Academy of Orthopedic Surgeons (AAOS) established that 58% of program directors and 83% of residents believed that the surgical skills of residents were not being objectively measured.[26] Furthermore, 80% of program directors and 86% of residents responded that surgical skill simulations should be a mandatory component of resident training, and the majority of both groups supported the standardization of the surgical skill curriculum across all programs.[26] Additionally, a moderate relationship (r = 0.68; P < .0001) has been established between Orthopedic In-Training Examination (OITE) results and the American Board of Orthopedic Surgery Examination (ABOS parts I and II) results.[2] It was shown that residents who failed the ABOS I and II had lower mean OITE year-in-training percentile rank scores,[2] thus showing that in-training evaluation of residents can indicate potential success or failure at the time of graduation. Therefore, further measurement of practical skills at intervals throughout training can allow residents to track their progress and identify specific skills they find difficult, creating the opportunity to develop these skills while still in training, such that graduating surgeons have a broader range of surgical skill competencies.

However, this possibility depends on whether 3D spatial perception is an inherent skill or one that can be taught. If it is inherent, then identification of individuals with or without this natural ability early in their training through objective, practical evaluations can help steer their careers in the proper direction. Conversely, if this skill can be taught, then it must be determined whether competency-based teaching will result in more competent surgeons relative to the more traditional methods, requiring “10,000 hours” of training.[15] Thus far, certain studies have begun to evaluate the effectiveness of competency-based training, and it seems potentially to be more effective than traditional methods.[1,8] Another interesting option would be to obtain postoperative 3D imaging to evaluate the performance of the procedure and potentially improve based upon that feedback, similar to postoperative radiography in fracture fixation. However, given the large cost associated with the single teaching session implemented in this study, perhaps arthroscopy in itself is an art that can be developed only through the extensive training of a fellowship. Fellowship training not only teaches skills, it has also been demonstrated to increase knowledge and competency for decision making relative to residency training.[42]

Cost is a common concern and limiting factor, according to program directors who participated in the AAOS survey.[26] Assessment of skills could be done for a relatively low cost of US$350 using the Fundamentals of Orthopedic Surgery (FORS) assessment tool. Lopez et al[33] showed that their psychomotor training and assessment tool resulted in significantly improved skills in students who participated in 30-minute sessions for a mere 4 weeks. This tool provides focused training that can be measured objectively, including specific skills such as 3D drill accuracy and drill-by-feel, which are relevant for training to place a femoral tunnel intraoperatively. Hence, this assessment tool has a potential role in resident surgical skill education and, more specifically, sports medicine as a subspecialty.

The present study indicates that current training methods are deficient with regard to instruction of musculoskeletal anatomy and specific surgical skills. However, a few limitations exist regarding the data of this study. First, only 13 residents participated in the study, all of whom were enrolled in training programs within Canada. This brings into question the generalizability of the study results to all orthopaedic surgery residents from various training programs. Second, the teaching session was only 7 hours, of which only 3 hours covered surgical skill training. This perhaps limited the first-hand exposure to different anatomic presentations of the intercondylar and bifurcate ridges and the native ACL footprint in various cadaveric knees, thereby restricting the residents’ ability to extrapolate their learning to the 3D models on which they were tested. Third, only senior residents participated in this study due to limitations in the number of participants for the cadaver laboratory. Having completed most of their residency, these senior residents might have maximized their abilities. Thus, these findings might not be generalizable to more junior residents as they have greater room for improvement, and teaching might have a greater impact on their accuracy. Fourth, the process for evaluating residents’ ability to properly identify the intersection of femoral ridges and native ACL graft placement had possible drawbacks. The 3D printed models consisted of only the bony landmarks, with no connective tissue or ACL remnants to help with intra-articular orientation. Furthermore, the models were attached to an apparatus that did not indicate the flexion angle of the knee or allow knee movement to observe the joint from various aspects, both of which have been shown to affect graft positioning.[22]

Conclusion

It is well recognized that orthopaedic surgery residents undergo intensive training; however, the residents in this study were still unable to correctly identify the junction of the intercondylar and bifurcate ridges or the native ACL footprint on 3D models. Our results suggest that a single didactic and practical teaching session does not significantly improve the practical performance of senior residents at that time. Further investigation into the effectiveness and application of surgical skill laboratories and simulations on orthopaedic residents’ competencies can help establish the future directions of surgical education for orthopaedic trainees. With the evolution toward competency-based residency programs, we need to understand the efficacy of our teaching tools.