BACKGROUND: Formal assessment of clinical competencies is necessary to ensure that all residents are acquiring important skills and, in the United States, will soon become a requirement for residency programme accreditation by the Accreditation Council for Graduate Medical Education (ACGME). The Eye Surgical Skills Assessment Test (ESSAT), a laboratory-based surgical skills obstacle course, was developed in response to the need for improved tools for the assessment of surgical skills during residency. The ESSAT has previously been shown to have face and content validity, and in this study we sought to determine its inter-rater reliability and, to some extent, its construct validity. METHODS: Twenty-seven content experts (residency programme directors and faculty members involved with resident surgical training) watched videos of a junior resident and senior resident completing the three ESSAT stations (skin suturing, muscle recession, and phacoemulsification: wound construction & suturing technique) and completed assessment forms, both task-specific checklists and a global rating scale of performance. RESULTS: The ESSAT showed strong inter-rater reliability for determining whether a resident "passed" a threshold of competency at each station for both the checklists and global rating scale. In addition, for each station, the senior resident was consistently rated above a "passing" threshold using either assessment form, whereas the junior resident was more often rated below (94% vs 30% passing on completed forms). CONCLUSION: These results, along with the findings of our face and content validity analysis, support the reliability and validity of the ESSAT, and indicate that it could be a useful tool for improving the assessment of surgical skill during residency. The ESSAT is a tool that all residency programmes could implement as a part of their ophthalmic surgical curriculum and competency assessment, and may be useful to set a threshold of competence that all residents would need to achieve prior to entering the operating room.
BACKGROUND: Formal assessment of clinical competencies is necessary to ensure that all residents are acquiring important skills and, in the United States, will soon become a requirement for residency programme accreditation by the Accreditation Council for Graduate Medical Education (ACGME). The Eye Surgical Skills Assessment Test (ESSAT), a laboratory-based surgical skills obstacle course, was developed in response to the need for improved tools for the assessment of surgical skills during residency. The ESSAT has previously been shown to have face and content validity, and in this study we sought to determine its inter-rater reliability and, to some extent, its construct validity. METHODS: Twenty-seven content experts (residency programme directors and faculty members involved with resident surgical training) watched videos of a junior resident and senior resident completing the three ESSAT stations (skin suturing, muscle recession, and phacoemulsification: wound construction & suturing technique) and completed assessment forms, both task-specific checklists and a global rating scale of performance. RESULTS: The ESSAT showed strong inter-rater reliability for determining whether a resident "passed" a threshold of competency at each station for both the checklists and global rating scale. In addition, for each station, the senior resident was consistently rated above a "passing" threshold using either assessment form, whereas the junior resident was more often rated below (94% vs 30% passing on completed forms). CONCLUSION: These results, along with the findings of our face and content validity analysis, support the reliability and validity of the ESSAT, and indicate that it could be a useful tool for improving the assessment of surgical skill during residency. The ESSAT is a tool that all residency programmes could implement as a part of their ophthalmic surgical curriculum and competency assessment, and may be useful to set a threshold of competence that all residents would need to achieve prior to entering the operating room.
Authors: Roger Webster; Joseph Sassani; Rod Shenk; Matt Harris; Jesse Gerber; Aaron Benson; John Blumenstock; Chad Billman; Randy Haluck Journal: Stud Health Technol Inform Date: 2005
Authors: Eileen L Mayro; Laura T Pizzi; Lisa A Hark; Ann P Murchison; Douglas Wisner; Anish Koka; Benjamin E Leiby; Nooreen Dabbish; Adedoyin Okulate; Alexa Dessy; Caitlin Green; Robert Bailey Journal: Am Health Drug Benefits Date: 2018-12
Authors: Grace L Paley; Rebecca Grove; Tejas C Sekhar; Jack Pruett; Michael V Stock; Tony N Pira; Steven M Shields; Evan L Waxman; Bradley S Wilson; Mae O Gordon; Susan M Culican Journal: J Surg Educ Date: 2021-02-25 Impact factor: 2.891
Authors: M Emre Altunrende; Mustafa Kemal Hamamcioglu; Tufan Hıcdonmez; Mehmet Osman Akcakaya; Barıs Bırgılı; Sebahattin Cobanoglu Journal: J Neurosci Rural Pract Date: 2014-04