Joseph Schwartz1, Adrian Costescu2, Marco A Mascarella1, Meredith E Young3,4, Murad Husein5, Sumit Agrawal5, Kathryn Roth5, Philip C Doyle5, Lily H P Nguyen1,3. 1. Department of Otolaryngology-Head and Neck Surgery, McGill University, Montreal, Quebec, Canada. 2. Faculty of Medicine, McGill University, Montreal, Quebec, Canada. 3. Center for Medical Education, McGill University, Montreal, Quebec, Canada. 4. Department of Medicine, McGill University, Montreal, Quebec, Canada. 5. Department of Otolaryngology-Head and Neck Surgery, Western University, London, Ontario, Canada.
Abstract
OBJECTIVES/HYPOTHESIS: Despite the transition to competency-based education in surgery, few standardized assessment tools exist in otolaryngology training. In particular, myringotomy and tympanostomy tube insertion (M+T) is a common surgical procedure with few validated assessment tools available. Our objectives were to develop an objective structured assessment of operative skills in M+T and to provide validity evidence for the developed assessment tool within otolaryngology training. STUDY DESIGN: Prospective study involving the evaluation of an assessment tool. METHODS: Through consultation with a panel of experts in otolaryngology and medical education we developed a Task-Specific Checklist and Global Rating Scale for M+T. Postgraduate year 2 junior residents, postgraduate year 3 senior residents, and attending otolaryngologists were video recorded performing M+T at a tertiary care pediatric hospital. The videos were subsequently reviewed and independently evaluated by three blinded raters from an unaffiliated academic institution. RESULTS: The average score of junior residents, senior residents, and attending otolaryngologists using the Task-Specific Checklist was 21.7/30 (±7.1), 26.3/30 (±3.5), and 27.3/30 (±6.2), respectively (P = .04). For the Global Rating Scale, the scores for junior residents, senior residents, and attending surgeons were 27.7/50 (±11.2), 34.5/50 (±9.5), and 45.1/50 (±4.6), respectively (P < .001). The inter-rater and intrarater reliability were both above 0.88. CONCLUSIONS: The Task-Specific Checklist and Global Rating Scale for M+T appear reliable, with validity evidence supporting their use in otolaryngology training. LEVEL OF EVIDENCE: NA Laryngoscope, 126:2140-2146, 2016.
OBJECTIVES/HYPOTHESIS: Despite the transition to competency-based education in surgery, few standardized assessment tools exist in otolaryngology training. In particular, myringotomy and tympanostomy tube insertion (M+T) is a common surgical procedure with few validated assessment tools available. Our objectives were to develop an objective structured assessment of operative skills in M+T and to provide validity evidence for the developed assessment tool within otolaryngology training. STUDY DESIGN: Prospective study involving the evaluation of an assessment tool. METHODS: Through consultation with a panel of experts in otolaryngology and medical education we developed a Task-Specific Checklist and Global Rating Scale for M+T. Postgraduate year 2 junior residents, postgraduate year 3 senior residents, and attending otolaryngologists were video recorded performing M+T at a tertiary care pediatric hospital. The videos were subsequently reviewed and independently evaluated by three blinded raters from an unaffiliated academic institution. RESULTS: The average score of junior residents, senior residents, and attending otolaryngologists using the Task-Specific Checklist was 21.7/30 (±7.1), 26.3/30 (±3.5), and 27.3/30 (±6.2), respectively (P = .04). For the Global Rating Scale, the scores for junior residents, senior residents, and attending surgeons were 27.7/50 (±11.2), 34.5/50 (±9.5), and 45.1/50 (±4.6), respectively (P < .001). The inter-rater and intrarater reliability were both above 0.88. CONCLUSIONS: The Task-Specific Checklist and Global Rating Scale for M+T appear reliable, with validity evidence supporting their use in otolaryngology training. LEVEL OF EVIDENCE: NA Laryngoscope, 126:2140-2146, 2016.