Ariel S Frey-Vogel1, Kristina Dzara2, Kimberly A Gifford3, Erica Y Chung4. 1. Department of Pediatrics, MassGeneral Hospital for Children (AS Frey-Vogel and K Dzara), Boston, Mass; Harvard Medical School (AS Frey-Vogel and K Dzara), Boston, Mass. Electronic address: afrey@mgh.harvard.edu. 2. Department of Pediatrics, MassGeneral Hospital for Children (AS Frey-Vogel and K Dzara), Boston, Mass; Harvard Medical School (AS Frey-Vogel and K Dzara), Boston, Mass. 3. Department of Pediatrics, Dartmouth-Hitchcock Medical Center (KA Gifford), Lebanon, NH; Geisel School of Medicine at Dartmouth (KA Gifford), Hanover, NH. 4. Department of Pediatrics, Hasbro Children's Hospital (EY Chung), Providence, RI; The Warren Alpert Medical School of Brown University (EY Chung), Providence, RI.
Abstract
OBJECTIVE: The Accreditation Council on Graduate Medical Education requires residents to teach and many residency programs assess resident teaching competency. While much formal resident-led teaching is for large groups, no corresponding published assessment instrument with validity evidence exists. We developed an instrument for faculty to assess pediatric resident-led large group teaching and gathered preliminary validity evidence. METHODS: Literature review and our experience leading resident-as-teacher curricula informed initial instrument content. Resident focus groups from 3 northeastern pediatric residency programs provided stakeholder input. A modified Delphi panel of international experts provided iterative feedback. Three investigators piloted the instrument in 2018; each assessed 8 video recordings of resident-led teaching. We calculated Cronbach's alpha for internal consistency and intraclass correlation (ICC) for inter-rater reliability. RESULTS: The instrument has 6 elements: learning climate, goals/objectives, content, promotion of understanding/retention, session management, and closure. Each element contains behavioral subelements. Cronbach's alpha was .844. ICC was excellent for 6 subelements, good for 1, fair for 1, and poor for 3. CONCLUSIONS: We developed an instrument for faculty assessment of resident-led large group teaching. Pilot data showed assessed behaviors had good internal consistency, but inconsistent interrater reliability. With further development, this instrument has potential to assess resident teaching competency.
OBJECTIVE: The Accreditation Council on Graduate Medical Education requires residents to teach and many residency programs assess resident teaching competency. While much formal resident-led teaching is for large groups, no corresponding published assessment instrument with validity evidence exists. We developed an instrument for faculty to assess pediatric resident-led large group teaching and gathered preliminary validity evidence. METHODS: Literature review and our experience leading resident-as-teacher curricula informed initial instrument content. Resident focus groups from 3 northeastern pediatric residency programs provided stakeholder input. A modified Delphi panel of international experts provided iterative feedback. Three investigators piloted the instrument in 2018; each assessed 8 video recordings of resident-led teaching. We calculated Cronbach's alpha for internal consistency and intraclass correlation (ICC) for inter-rater reliability. RESULTS: The instrument has 6 elements: learning climate, goals/objectives, content, promotion of understanding/retention, session management, and closure. Each element contains behavioral subelements. Cronbach's alpha was .844. ICC was excellent for 6 subelements, good for 1, fair for 1, and poor for 3. CONCLUSIONS: We developed an instrument for faculty assessment of resident-led large group teaching. Pilot data showed assessed behaviors had good internal consistency, but inconsistent interrater reliability. With further development, this instrument has potential to assess resident teaching competency.