Nanna Jo Borgersen1,2,3, Therese M H Naur1,3, Stine M D Sørensen1,3, Flemming Bjerrum1,4, Lars Konge1,3, Yousif Subhi2,3, Ann Sofia S Thomsen1,5. 1. Copenhagen Academy for Medical Education and Simulation, the Capital Region of Denmark, Copenhagen, Denmark. 2. Department of Ophthalmology, Zealand University Hospital, Roskilde, Denmark. 3. Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark. 4. Department of Surgery, Herlev Gentofte Hospital, Herlev, Denmark. 5. Department of Ophthalmology, Rigshospitalet-Glostrup, Copenhagen, Denmark.
Abstract
OBJECTIVE: To identify current trends in the use of validity frameworks in surgical simulation, to provide an overview of the evidence behind the assessment of technical skills in all surgical specialties, and to present recommendations and guidelines for future validity studies. SUMMARY OF BACKGROUND DATA: Validity evidence for assessment tools used in the evaluation of surgical performance is of paramount importance to ensure valid and reliable assessment of skills. METHODS: We systematically reviewed the literature by searching 5 databases (PubMed, EMBASE, Web of Science, PsycINFO, and the Cochrane Library) for studies published from January 1, 2008, to July 10, 2017. We included original studies evaluating simulation-based assessments of health professionals in surgical specialties and extracted data on surgical specialty, simulator modality, participant characteristics, and the validity framework used. Data were synthesized qualitatively. RESULTS: We identified 498 studies with a total of 18,312 participants. Publications involving validity assessments in surgical simulation more than doubled from 2008 to 2010 (∼30 studies/year) to 2014 to 2016 (∼70 to 90 studies/year). Only 6.6% of the studies used the recommended contemporary validity framework (Messick). The majority of studies used outdated frameworks such as face validity. Significant differences were identified across surgical specialties. The evaluated assessment tools were mostly inanimate or virtual reality simulation models. CONCLUSION: An increasing number of studies have gathered validity evidence for simulation-based assessments in surgical specialties, but the use of outdated frameworks remains common. To address the current practice, this paper presents guidelines on how to use the contemporary validity framework when designing validity studies.
OBJECTIVE: To identify current trends in the use of validity frameworks in surgical simulation, to provide an overview of the evidence behind the assessment of technical skills in all surgical specialties, and to present recommendations and guidelines for future validity studies. SUMMARY OF BACKGROUND DATA: Validity evidence for assessment tools used in the evaluation of surgical performance is of paramount importance to ensure valid and reliable assessment of skills. METHODS: We systematically reviewed the literature by searching 5 databases (PubMed, EMBASE, Web of Science, PsycINFO, and the Cochrane Library) for studies published from January 1, 2008, to July 10, 2017. We included original studies evaluating simulation-based assessments of health professionals in surgical specialties and extracted data on surgical specialty, simulator modality, participant characteristics, and the validity framework used. Data were synthesized qualitatively. RESULTS: We identified 498 studies with a total of 18,312 participants. Publications involving validity assessments in surgical simulation more than doubled from 2008 to 2010 (∼30 studies/year) to 2014 to 2016 (∼70 to 90 studies/year). Only 6.6% of the studies used the recommended contemporary validity framework (Messick). The majority of studies used outdated frameworks such as face validity. Significant differences were identified across surgical specialties. The evaluated assessment tools were mostly inanimate or virtual reality simulation models. CONCLUSION: An increasing number of studies have gathered validity evidence for simulation-based assessments in surgical specialties, but the use of outdated frameworks remains common. To address the current practice, this paper presents guidelines on how to use the contemporary validity framework when designing validity studies.
Authors: Mona M Savran; Anders Bo Nielsen; Bente Baekholm Poulsen; Poul Bak Thorsen; Lars Konge Journal: Surg Endosc Date: 2018-10-17 Impact factor: 4.584
Authors: Peter Hertz; Katrine Jensen; Saleh N Abudaff; Michael Strøm; Yousif Subhi; Hani Lababidi; Lars Konge Journal: BMJ Open Respir Res Date: 2018-12-10