BACKGROUND: Delivering quality lectures is a critical skill for residents seeking careers in academia yet no validated tools for assessing resident lecture skills exist. OBJECTIVES: The authors sought to develop and validate a lecture assessment tool. METHODS: Using a nominal group technique, the authors derived a behaviorally anchored assessment tool. Baseline characteristics of resident lecturers including prior lecturing experience and perceived comfort with lecturing were collected. Faculty and senior residents used the tool to assess lecturer performance at weekly conference. A postintervention survey assessed the usability of the form and the quantity and quality of the feedback. Analysis of variance was used to identify relationships in performance within individual domains to baseline data. Generalizability coefficients and scatterplots with jitter were used to assess inter-rater reliability. RESULTS: Of 64 residents assessed, most (68.8%) had previous lecturing experience and 6.3% had experience as a regional/national speaker. There was a significant difference in performance within the domains of Content Expertise (p < 0.001), Presentation Design/Structure (p = 0.014), and Lecture Presence (p = 0.001) for first-year versus fourth-year residents. Residents who had higher perceived comfort with lecturing performed better in the domains of Content Expertise (p = 0.035), Presentation Design/Structure (p = 0.037), and Lecture Presence (p < 0.001). We found fair agreement between raters in all domains except Goals and Objectives. Both lecturers and evaluators perceived the feedback delivered as specific and of adequate quantity and quality. Evaluators described the form as highly useable. CONCLUSIONS: The derived behaviorally anchored assessment tool is a sufficiently valid instrument for the assessment of resident-delivered lectures.
BACKGROUND: Delivering quality lectures is a critical skill for residents seeking careers in academia yet no validated tools for assessing resident lecture skills exist. OBJECTIVES: The authors sought to develop and validate a lecture assessment tool. METHODS: Using a nominal group technique, the authors derived a behaviorally anchored assessment tool. Baseline characteristics of resident lecturers including prior lecturing experience and perceived comfort with lecturing were collected. Faculty and senior residents used the tool to assess lecturer performance at weekly conference. A postintervention survey assessed the usability of the form and the quantity and quality of the feedback. Analysis of variance was used to identify relationships in performance within individual domains to baseline data. Generalizability coefficients and scatterplots with jitter were used to assess inter-rater reliability. RESULTS: Of 64 residents assessed, most (68.8%) had previous lecturing experience and 6.3% had experience as a regional/national speaker. There was a significant difference in performance within the domains of Content Expertise (p < 0.001), Presentation Design/Structure (p = 0.014), and Lecture Presence (p = 0.001) for first-year versus fourth-year residents. Residents who had higher perceived comfort with lecturing performed better in the domains of Content Expertise (p = 0.035), Presentation Design/Structure (p = 0.037), and Lecture Presence (p < 0.001). We found fair agreement between raters in all domains except Goals and Objectives. Both lecturers and evaluators perceived the feedback delivered as specific and of adequate quantity and quality. Evaluators described the form as highly useable. CONCLUSIONS: The derived behaviorally anchored assessment tool is a sufficiently valid instrument for the assessment of resident-delivered lectures.
Authors: Lori R Newman; Dara D Brodsky; David H Roberts; Stephen R Pelletier; Anna Johansson; Charles M Vollmer; K Meredith Atkins; Richard M Schwartzstein Journal: Acad Med Date: 2012-03 Impact factor: 6.893
Authors: Paul A Harris; Robert Taylor; Robert Thielke; Jonathon Payne; Nathaniel Gonzalez; Jose G Conde Journal: J Biomed Inform Date: 2008-09-30 Impact factor: 6.317
Authors: Michael S Beeson; Wallace A Carter; Theodore A Christopher; Jonathan W Heidt; James H Jones; Lynne E Meyer; Susan B Promes; Kevin G Rodgers; Philip H Shayne; Mary Jo Wagner; Susan R Swing Journal: J Grad Med Educ Date: 2013-03
Authors: Julianna Jung; Douglas Franzen; Luan Lawson; David Manthey; Matthew Tews; Nicole Dubosh; Jonathan Fisher; Marianne Haughey; Joseph B House; Arleigh Trainor; David A Wald; Katherine Hiller Journal: West J Emerg Med Date: 2017-12-22