Veronica Lerner1, Christopher DeStephano2, Amanda Ulrich2, Esther S Han2, Edgar LeClaire2, Chi Chiung Grace Chen2. 1. Department of Obstetrics & Gynecology and Women's Health, Albert Einstein College of Medicine/Montefiore Medical Center, Bronx, New York (Dr. Lerner); Department of Obstetrics and Gynecology, Mayo Clinic, Jacksonville, Florida (Dr. DeStephano); Department of Obstetrics and Gynecology, University of Connecticut Health Center, Farmington, Connecticut (Dr. Ulrich); Department of Obstetrics & Gynecology, Columbia University Medical Center, New York, New York (Dr. Han); Department of Obstetrics and Gynecology, University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma (Dr. LeClaire); Department of Gynecology and Obstetrics, Johns Hopkins University, Baltimore, Maryland (Dr. Chen). Electronic address: lernervt02@gmail.com. 2. Department of Obstetrics & Gynecology and Women's Health, Albert Einstein College of Medicine/Montefiore Medical Center, Bronx, New York (Dr. Lerner); Department of Obstetrics and Gynecology, Mayo Clinic, Jacksonville, Florida (Dr. DeStephano); Department of Obstetrics and Gynecology, University of Connecticut Health Center, Farmington, Connecticut (Dr. Ulrich); Department of Obstetrics & Gynecology, Columbia University Medical Center, New York, New York (Dr. Han); Department of Obstetrics and Gynecology, University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma (Dr. LeClaire); Department of Gynecology and Obstetrics, Johns Hopkins University, Baltimore, Maryland (Dr. Chen).
Abstract
OBJECTIVE: The Fundamentals in Laparoscopic Surgery (FLS) examination is designed to test laparoscopic surgery skills. Our aim for this systematic review was to examine validity evidence supporting or refuting the FLS examination specifically as a high-stakes summative assessment tool in gynecology. DATA SOURCES: The data sources were PubMed, MEDLINE, Embase, and Scopus. METHODS OF STUDY SELECTION: The study eligibility criterion was the subject of the FLS examination as an assessment tool in gynecology. We developed a data extraction tool and assigned articles for screening and extraction to all authors, who then abstracted data independently. Conflicts that arose during the extraction process were resolved by consensus. We organized validity evidence for the cognitive and manual skills portions on the basis of the categories of current validation standards. TABULATION, INTEGRATION, AND RESULTS: From 1971 citations identified, 9 studies were included, involving 319 participants. For the cognitive portion of the test, the results were mixed in 5 studies in relationships with the other variables category. For the manual portion of the test, most of the studies focused on the relationships with other variables evidence with mixed findings. The concerning findings in the manual skills portion included the lack of transferability of skills to the operating room, limited mixed evidence for improvement in operating room performance, and worse performance by obstetrics and gynecology surgeons compared with other specialties. We did not find supportive content-based, response process, or consequential evidence in either the cognitive or manual skills portion of the test. CONCLUSION: Validity evidence for the FLS examination was either mixed, as it pertained to relationships with other variables, or lacking in other important evidence categories. Further evidence is required to justify the use of the FLS examination scores as a high-stakes summative assessment.
OBJECTIVE: The Fundamentals in Laparoscopic Surgery (FLS) examination is designed to test laparoscopic surgery skills. Our aim for this systematic review was to examine validity evidence supporting or refuting the FLS examination specifically as a high-stakes summative assessment tool in gynecology. DATA SOURCES: The data sources were PubMed, MEDLINE, Embase, and Scopus. METHODS OF STUDY SELECTION: The study eligibility criterion was the subject of the FLS examination as an assessment tool in gynecology. We developed a data extraction tool and assigned articles for screening and extraction to all authors, who then abstracted data independently. Conflicts that arose during the extraction process were resolved by consensus. We organized validity evidence for the cognitive and manual skills portions on the basis of the categories of current validation standards. TABULATION, INTEGRATION, AND RESULTS: From 1971 citations identified, 9 studies were included, involving 319 participants. For the cognitive portion of the test, the results were mixed in 5 studies in relationships with the other variables category. For the manual portion of the test, most of the studies focused on the relationships with other variables evidence with mixed findings. The concerning findings in the manual skills portion included the lack of transferability of skills to the operating room, limited mixed evidence for improvement in operating room performance, and worse performance by obstetrics and gynecology surgeons compared with other specialties. We did not find supportive content-based, response process, or consequential evidence in either the cognitive or manual skills portion of the test. CONCLUSION: Validity evidence for the FLS examination was either mixed, as it pertained to relationships with other variables, or lacking in other important evidence categories. Further evidence is required to justify the use of the FLS examination scores as a high-stakes summative assessment.