James R Korndorffer1, Steven J Kasten, Steven M Downing. 1. Department of Surgery, Tulane University School of Medicine, SL-22, 1430 Tulane Avenue, New Orleans, LA 70112-2699, USA. korndorffer@tulane.edu
Abstract
BACKGROUND: Assessment methods and theory continue to evolve in the general education literature. Nowhere is this more evident than in the framework of validity methods and concepts. The consensus standards of the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education have changed from "types of validity" (criterion, construct, and content) and "valid instruments," last used in 1974, to a concept of identifying evidence for the validity of results and the use of those results. The purpose of this study was to evaluate the surgical education literature for the adoption of the current consensus standards. METHODS: As a representative sample of the surgical educational literature, the validation effort in laparoscopic simulator education was chosen. A MEDLINE search using the terms validity.tw and laparoscop$.tw between 1996 and 2008 (September week 1) yielded 192 citations. All titles and abstracts were reviewed, resulting in 47 studies appropriate for in-depth analysis. RESULTS: Validation studies have evaluated 21 different simulators. Twenty-three percent of the studies adhere, in part, to the new consensus standards for validity. One hundred percent use the old framework of types of validity including 75% using construct validity, 38% using face validity, and 11% using content. CONCLUSION: The widespread use of the currently (after 1999) accepted framework for validity is lacking in the surgical education literature. Surgical educators must remain current and begin to investigate our assessments within the contemporary framework of validity to avoid improper judgments of performance.
BACKGROUND: Assessment methods and theory continue to evolve in the general education literature. Nowhere is this more evident than in the framework of validity methods and concepts. The consensus standards of the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education have changed from "types of validity" (criterion, construct, and content) and "valid instruments," last used in 1974, to a concept of identifying evidence for the validity of results and the use of those results. The purpose of this study was to evaluate the surgical education literature for the adoption of the current consensus standards. METHODS: As a representative sample of the surgical educational literature, the validation effort in laparoscopic simulator education was chosen. A MEDLINE search using the terms validity.tw and laparoscop$.tw between 1996 and 2008 (September week 1) yielded 192 citations. All titles and abstracts were reviewed, resulting in 47 studies appropriate for in-depth analysis. RESULTS: Validation studies have evaluated 21 different simulators. Twenty-three percent of the studies adhere, in part, to the new consensus standards for validity. One hundred percent use the old framework of types of validity including 75% using construct validity, 38% using face validity, and 11% using content. CONCLUSION: The widespread use of the currently (after 1999) accepted framework for validity is lacking in the surgical education literature. Surgical educators must remain current and begin to investigate our assessments within the contemporary framework of validity to avoid improper judgments of performance.
Authors: Karl-Friedrich Kowalewski; Jonathan D Hendrie; Mona W Schmidt; Carly R Garrow; Thomas Bruckner; Tanja Proctor; Sai Paul; Davud Adigüzel; Sebastian Bodenstedt; Andreas Erben; Hannes Kenngott; Young Erben; Stefanie Speidel; Beat P Müller-Stich; Felix Nickel Journal: Surg Endosc Date: 2016-09-07 Impact factor: 4.584
Authors: Domenico Veneziano; Achilles Ploumidis; Silvia Proietti; Theodoros Tokas; Guido Kamphuis; Giovanni Tripepi; Ben Van Cleynenbreugel; Ali Gozen; Alberto Breda; Joan Palou; Kemal Sarica; Evangelos Liatsikos; Kamran Ahmed; Bhaskar K Somani Journal: World J Urol Date: 2019-03-27 Impact factor: 4.226
Authors: Pieter J van Empel; Lennart B van Rijssen; Joris P Commandeur; Mathilde G E Verdam; Judith A Huirne; Fedde Scheele; H Jaap Bonjer; W Jeroen Meijerink Journal: Surg Endosc Date: 2012-02-21 Impact factor: 4.584