Fundamentals of laparoscopic surgery: a surgical skills assessment tool in gynecology.

OBJECTIVE: To describe our experience with the Fundamentals of Laparoscopic Surgery (FLS) program as a teaching and assessment tool for basic laparoscopic competency among gynecology residents.
METHODS: A prospective observational study was conducted at a single academic institution. Before the FLS program was introduced, baseline FLS testing was offered to residents and gynecology division directors. Test scores were analyzed by training level and self-reported surgical experience. After implementing a minimally invasive gynecologic surgical curriculum, third-year residents were retested.
RESULTS: The pass rates for baseline FLS skills testing were 0% for first-year residents, 50% for second-year residents, and 75% for third- and fourth-year residents. The pass rates for baseline cognitive testing were 60% for first- and second-year residents, 67% for third-year residents, and 40% for fourth-year residents. When comparing junior and senior residents, there was a significant difference in pass rates for the skills test (P=.007) but not the cognitive test (P=.068). Self-reported surgical experience strongly correlated with skills scores (r-value=0.97, P=.0048), but not cognitive scores (r-value=0.20, P=.6265). After implementing a curriculum, 100% of the third-year residents passed the skills test, and 92% passed the cognitive examination.
CONCLUSIONS: The FLS skills test may be a valuable assessment tool for gynecology residents. The cognitive test may need further adaptation for applicability to gynecologists.

INTRODUCTION

The traditional adage “see one, do one, teach one” has become antiquated in today's surgical training environment. Although seeing, doing, and teaching surgical skills remain critical components of the learning process, simulation training has added a new dimension to surgical education. Simulation provides trainees with a safe, nonthreatening learning environment where errors have no significant consequences. In an era when resident duty hour restrictions limit learning time in the operating room and patient safety is paramount, simulation allows trainees to learn new techniques, instruments, and procedures before operating on patients. This in turn minimizes using valuable operating room time to teach basic surgical techniques to novices.

Some experts have noted that it is particularly challenging to use a traditional apprentice approach to teach laparoscopic surgery to trainees. With its rapidly evolving technologies and perhaps less intuitive techniques, laparoscopy has been a particular focus for simulation training. A variety of simulation tools are available for laparoscopic training, ranging from simple box trainers to interactive virtual reality simulators.[1-3] Many of these models can be used both for skills acquisition and assessment. Data support that simulator performance predicts intraoperative laparoscopic skill.[4] Trainees in general surgery who were exposed to simulated skills training have been noted to have improved operating performances during their intern year on laparoscopic surgeries as assessed by faculty.[5] General surgical residency training programs have already embraced simulation for both training and assessment of surgical skills, and endorsed the development of a comprehensive technical skills curriculum for all levels of general surgery training. However, gynecology training programs have lagged behind in integrating formal simulation training and assessment programs into their residency curricula.[6,7]

The Fundamentals of Laparoscopic Surgery (FLS) program was developed by the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) and launched in 2004 as a tool to assess fundamental knowledge and surgical skills necessary for basic laparoscopic surgery. The FLS program has been shown to be a reliable and valid system for gauging laparoscopic skills and knowledge among general surgeons.[8-10] Data support that FLS training improves laparoscopic performance in the operating room.[11] Numerous general surgery residency programs have adopted the FLS program into their curriculum, and since July 1, 2009, the American Board of Surgery requires successful completion of the FLS program to qualify for board certification.

The FLS program was designed to be universally applicable to all surgical specialists including general surgeons, urologists, and gynecologists.[12] The objective of our observational study was to confirm that the FLS test could be used as an assessment tool for basic laparoscopic competence for Obstetrics and Gynecology (Ob/Gyn) residents, and to study the impact of a formal laparoscopic curriculum on resident performance of the FLS test. The specific goals of our study were to (1) assess for construct validity of the FLS test for gynecologists; (2) analyze the FLS test performance of third-year Ob/Gyn residents before and after the minimally invasive gynecologic surgical (MIGS) curriculum.

METHODS

The institutional review board (IRB) at Beth Israel Deaconess Medical Center (BIDMC) reviewed this study and determined the project did not constitute human subjects research since FLS testing was being administered for resident education. Therefore, IRB approval for the study was not required.

In 2006, all Ob/Gyn residents at all levels of training (n=20) and 4 division directors of gynecologic surgical specialties were invited to participate in baseline FLS testing. The FLS examination consists of 2 parts: a computer-based cognitive examination with 75 multiple-choice questions, and a manual skills test with 5 tasks (peg transfer, 2-handed precision cutting, Endoloop ligation, laparoscopic suturing with extracorporeal knot tying, and suturing with intracorporeal knot tying). A score of 356 or higher qualified as a passing grade for the skills and cognitive components of the FLS examination. A passing grade for both components of the FLS test is required for FLS certification.

Baseline FLS testing was administered before the FLS program was introduced and before the minimally invasive gynecologic surgical curriculum was formalized. None of the participants had received formal laparoscopic teaching and were not familiar with the FLS program at the start of the study. After baseline FLS testing, residents were formally introduced to the FLS program. A 2-part CD-ROM from SAGES that contains educational materials on fundamental concepts of endoscopic surgery along with visual demonstrations of the 5 tasks required for the test was made available for review. Residents were given unlimited access to an on-site simulation center at BIDMC to independently practice the 5 technical tasks on an FLS box trainer and study the CD-ROM-based educational modules before being retested.

The FLS examinations were conducted at the Carl J. Shapiro Simulation and Skills Center, which is a regional FLS testing center located at BIDMC in Boston, Massachusetts. A certified FLS proctor administered the examinations. Each technical task was assessed for both accuracy and speed according to the FLS testing guidelines. Participants who successfully passed both the cognitive and skills components of the examination received FLS certification.

Prior to FLS testing, examinees were asked to complete a pretest questionnaire to report their level of training, number of years performing laparoscopic surgery, and level of endoscopic surgical experience. Surgical experience was measured by the reported number of cases performed as the primary surgeon or assistant for the following procedures:

Diagnostic laparoscopy

Laparoscopic tubal ligation

Laparoscopic ovarian cystectomy

Laparoscopic oophorectomy and/or salpingectomy

Laparoscopic surgery for management of ectopic pregnancy

Laparoscopic hysterectomy

Other advanced laparoscopic procedures.

The pass rates of the cognitive and skills test were analyzed with the Fisher's exact test to assess the construct validity of each component of the FLS examination when applied to resident gynecologists. The pass rates were categorized by junior (first-year and second-year) and senior (third- and fourth-year) level of Ob/Gyn resident training to calculate the P-values listed in . The Spearman rank correlation test was used to analyze whether one's self-reported level of surgical experience correlated with FLS test scores.

After obtaining baseline FLS testing at BIDMC in the academic year of 2006–2007, the FLS program was incorporated into our Ob/Gyn curriculum. A formal 2-year minimally invasive surgical curriculum was established. Prior to this intervention, all surgical training was done in the traditional apprentice model of clinical teaching in the operating room. By the academic year 2008–2009, the Ob/Gyn department at BIDMC made FLS certification a requirement for all Ob/Gyn residents before they completed their 4-year residency training program.

The minimally invasive simulation surgical curriculum for our residents includes the following components:

Didactic lectures (eg, energy sources, patient positioning)

3-hour Simulation Workshop twice a year (laparoscopy workshop, hysteroscopy workshop)

Monthly 1-hour simulation laboratory sessions

24-hour access to on-site simulation laboratory

FLS examination

The FLS program is introduced to our Ob/Gyn interns during their orientation. All residents have 24-hour access to an on-site simulation laboratory that includes FLS stations, other dry laboratory stations, and the FLS computer-based educational modules (accessible by 2-part CD-ROM or web-based). Residents are encouraged to practice in the simulation laboratory, especially before application of a new instrument, technique, or procedure. A surgical video library of commonly performed laparoscopic procedures is also available to our residents. The residents take the FLS examination in their third year of training (PGY-3 testing).

The graduating class of 2008 retook the FLS examination during their fourth year of training, since their baseline testing was taken during their third year of residency. All other residents retook their FLS examination during their third year of training. We analyzed the pass rates of the FLS test scores before (baseline testing) and after (PGY-3 testing) initiation of the formal MIGS surgical curriculum for comparison.

RESULTS

Twenty Ob/Gyn residents were offered the FLS examination, and 19 participated. Eighteen residents completed the cognitive examination, 16 completed the skills examination, and 15 fulfilled both portions for complete testing. All 4 gynecologic division directors (2 gynecologic oncologists, 1 reproductive infertility specialist, and 1 minimally invasive gynecologic surgeon) at BIDMC participated and completed the FLS test.

All 4 gynecologic directors successfully passed both components of the FLS examination and obtained FLS certification. Among the residents who participated, the pass rates for the skills test were 0% for first-year residents (PGY-1), 50% for second-year residents (PGY-2), and 75% for third-year (PGY-3) and fourth-year (PGY-4) residents. The pass rates for the cognitive test were 60% for PGY-1s and PGY-2s, 67% for PGY-3s, and 40% for PGY-4s (. Given the small number of residents, the pass rates for the junior residents (PGY-1 and PGY-2) and senior residents (PGY-3 and PGY-4) were pooled for a 2-group comparison. Application of the Fisher's exact test demonstrated a statistically significant difference in pass rates for the FLS skills test (P=.007), but not for the cognitive test (P=.068) when comparing junior (PGY-1 and 2) and senior (PGY-3 and 4) residents.

All 4 gynecologic directors and 16 of the 19 residents completed the pretest questionnaire for self-reported level of surgical experience. Analysis with the Spearman rank correlation test demonstrated that greater surgical experience strongly correlated with higher manual FLS skills test scores (R-value=0.97, P=.0048), whereas the FLS cognitive test scores had a weak correlation (R-value = 0.20, P=.6265). The strongest correlations were noted among the laparoscopic procedures that require more advanced skills. Performing greater numbers of laparoscopic hysterectomies, ovarian cystectomies, and salpingo-oophorectomies was significantly correlated with higher skills test scores. In contrast, no statistically significant correlation was found between cognitive test scores and reported surgical experience (.

When the Ob/Gyn residents took the FLS examination in their third (or early fourth) year of training, there was a 100% (13/13) pass rate for the skills component, and a 92% (12/13) pass rate for the cognitive component ().

DISCUSSION

Surgical simulation teaching has become an important training component for many residency programs across all surgical disciplines. After SAGES launched the FLS program in 2004, the American College of Surgeons (ACS) joined SAGES in 2005 for a joint educational effort to establish standards for fundamental skills and knowledge necessary to care for patients undergoing laparoscopic surgery. The growing number of minimally invasive procedures and the need to teach and assess these procedures in a simulated setting have been recognized by general surgery residency training programs. As a result, simulation teaching, specifically with the FLS program, has become popular among training programs. Passing the FLS examination is now a requirement for board certification in general surgery. Multiple studies support the reliability and validity of the FLS program as a tool for teaching and evaluating the fundamentals of laparoscopic surgery. However, the vast majority of the validation studies were performed among general surgeons, with a few including urologists[12,13] and one study including gynecologists.[14] When developing the FLS program, SAGES intended for the examination to be universal, ie, nonspecific to a procedure or surgical discipline, so it could be universally applied to all surgical specialties.

We tested our Ob/Gyn residents with the FLS examination in the academic year 2006–2007 to assess its applicability to gynecologists. The skills portion of FLS testing was able to distinguish competency and reflect surgical training and experience between our junior versus senior gynecology residents, whereas the cognitive test scores did not. We observed a positive correlation for the skills test, with greater pass rates noted with increasing level of resident training and greater reported surgical experience. These findings support the construct validity of the FLS skills test to assess competency among gynecology residents. No such correlation was noted for the cognitive examination (. The construct validity observed from the baseline FLS testing of our small cohort of Ob/Gyn residents suggests that the skills portion of the FLS test may be a valuable assessment tool for gynecology residents. Our findings support the conclusions made by Zheng et al[15] regarding the validity of the FLS skills test, but not cognitive test, to discriminate between advanced versus novice surgeons.

We were also interested in exploring which questions were incorrectly answered on the cognitive examination to analyze why the fourth-year residents scored so low. We inquired with SAGES, but we were unable to obtain this information. SAGES provides the overall test scores for the skills and cognitive examinations but does not provide a breakdown of the scores, which we believe would have been helpful for the study and helpful feedback for the learner.

Over the past 3 academic years (from 2007–2010), all 13 residents who took the FLS examination during their third year of training passed the skills component, and 12 passed the cognitive component, resulting in a 100% pass rate for the skills test, 92% pass rate for the cognitive test, and a 92% overall pass rate for FLS certification (. After initiating our minimally invasive gynecologic surgical curriculum and engaging in the FLS program, we found that residents improved their FLS test scores dramatically.

Our study supports the findings from Scott et al[16] that suggest laparoscopic skills proficiency can be obtained by practice. The pass rate for baseline FLS testing was quite low, because none of the residents were given the opportunity to practice before their baseline testing. After the FLS program and minimally invasive gynecologic curriculum were formally introduced, the residents had the opportunity to practice their FLS tasks. With practice on the FLS training box and formal laparoscopic teaching, the residents demonstrated markedly improved FLS skills performance reflected by improved FLS test scores.

Although our results support construct validity for FLS skills testing among gynecologists, our small sample size limits our ability to make definitive conclusions. The most common complaint among our test takers was that the FLS multiple-choice questions focused primarily on general surgical practice. They felt the cognitive examination favored general surgeons for this reason. This may explain the lack of construct validity for the cognitive portion of the FLS examination among gynecology residents. Unfortunately, we were unable to access which questions were incorrectly answered to further evaluate this complaint. Based on our experience, the cognitive test is not as universally applicable as originally intended by SAGES. Consequently, the skills test may be more readily accepted by gynecologists than the cognitive examination.

Nonetheless, we agree with SAGES that the basic underlying principles necessary for safe laparoscopic surgery are common to all disciplines. Some common concepts include an understanding of energy sources, patient positioning, potential complications, and anatomy and physiology associated with laparoscopic surgery. We acknowledge that safe surgical practices require not only good technical skills, but also a sound understanding of fundamental principles for safe and effective application of endoscopic techniques; therefore a cognitive component is critical for teaching and assessing one's endoscopic skills. We incorporated a formal 2-year minimally invasive gynecologic surgical curriculum into our residency program, which includes both didactic and skills training components with use of the SAGES FLS program (see Methods above). A didactic supplement for the current FLS program may facilitate its use among gynecologists.

Our observational study has several limitations. As mentioned previously, our sample size is small, and further studies are needed to confirm our findings regarding the validity of the FLS examination as a measure of laparoscopic competence among gynecologists. Another limitation of the study is the undetermined significance of FLS test scores and intraoperative surgical skills, performance, judgment, and outcomes. The efficacy of the FLS program and minimally invasive gynecologic surgical curriculum are unknown, and present an opportunity for future study.

Even with all the limitations, we feel a simulation program focused on teaching fundamental laparoscopic principles and techniques is a critical component for teaching safe endoscopic practices in our Ob/Gyn residency training program. The growing demand for minimally invasive surgery from our patients, the constant evolution of endoscopic technology and techniques, the limitations of the 80-hour resident work week, and the diverse range of women's healthcare practices (both obstetrics and gynecology) that must be mastered in a 4-year training period are some of the challenges of training Ob/Gyn residents today. Simulation teaching is an essential component of our residency curriculum at BIDMC. Given the strengths of the FLS program, we have made FLS certification a graduation requirement for our Ob/Gyn residents. The FLS program has gained further popularity among attending general surgery, urology, and gynecology staff at the Harvard Medical School teaching hospitals since our malpractice insurance carrier now provides incentives for surgeons to participate in the FLS program.[17]

CONCLUSION

The FLS manual skills component may be a valuable training and assessment tool for gynecologists. Level of training and surgical experience positively correlate with pass rates for the FLS skills but not the cognitive examination. Modifications of the FLS cognitive examination and/or a supplemental curriculum specific to gynecologic surgery may be necessary to make the cognitive portion of the FLS examination more universally applicable to gynecologists.

Table 1.

Pass Rates for Cognitive and Skills Components of Fundamentals of Laparoscopic Surgery Examination by Level of Training

Participants	Cognitive Test #Pass/#Tested (%) P=.068	Manual Skills Test #Pass/#Tested (%) P=.007
Residents (N=19)	10/18 (56)	8/16 (50)
PGY-I	3/5 (60)	0/4 (0)
PGY-II	3/5 (60)	2/4 (50)
PGY-III	2/3 (67)	3/4 (75)
PGY-IV	2/5 (40)	3/4 (75)
Attendings (N=4)	4/4 (100)	4/4 (100)
Total	14/22 (63.6)	12/20 (60)

Table 2.

Correlation Between Fundamentals of Laparoscopic Surgery Test Scores and Surgical Experience

Procedure	Cognitive Score [r value (P value)]a	Skills Score [r value (P value)]a
Diagnostic laparoscopy	−0.04 (0.90)	0.72 (0.0177)
Laparoscopic tubal ligation	0.02 (0.9475)	0.66 (0.0539)
Laparoscopic ovarian cystectomy	0.17 (0.6020)	0.91 (0.0007)
Laparoscopic oophorectomy	0.15 (0.6387)	0.94 (0.0002)
Laparoscopic surgery for ectopic pregnancy	0.42 (0.1734)	0.86 (0.0028)
Laparoscopic hysterectomy	−0.01 (0.9835)	0.91 (0.0015)
Advanced procedures	−0.04 (0.9103)	0.94 (0.0054)
Overall (all procedures)	0.20 (0.6265)	0.97 (0.0048)

Note: r value≥0.7=strong association; 0.3≤r value≤moderate association; r value<0.3=weak association.

Table 3.

Scores for Fundamentals of Laparoscopic Surgery Test Taken 3rd Year After Formal Surgical Curriculum Initiated

Academic Year	No. of Test Takers	(%) Passed Cognitive	(%) Passed Manual Skills	Total (%) Passed
Baseline Testing (PGY 3)
2006–2007	n=4	2/3 (67)	3/4 (75)	2/4 (25)
Follow-up Testing (PGY 3)
2009–2010	n=5	4/5 (80)	5/5 (100)	4/5 (80%)
2008–2009	n=5	5/5 (100)	5/5 (100)	5/5 (100)
2007–2008	n=3	3/3 (100)	3/3 (100)	5/5 (100)
Overall	n=13	12/13 (92)	13/13 (100)	12/13 (92)