Annabel K Frank1,2, Patricia O'Sullivan1, Lynnea M Mills1, Virginie Muller-Juge1, Karen E Hauer3. 1. Department of Medicine, University of California, San Francisco, San Francisco, CA, USA. 2. Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA. 3. Department of Medicine, University of California, San Francisco, San Francisco, CA, USA. karen.hauer@ucsf.edu.
Abstract
BACKGROUND: Faculty and students debate the fairness and accuracy of medical student clerkship grades. Group decision-making is a potential strategy to improve grading. OBJECTIVE: To explore how one school's grading committee members integrate assessment data to inform grade decisions and to identify the committees' benefits and challenges. DESIGN: This qualitative study used semi-structured interviews with grading committee chairs and members conducted between November 2017 and March 2018. PARTICIPANTS: Participants included the eight core clerkship directors, who chaired their grading committees. We randomly selected other committee members to invite, for a maximum of three interviews per clerkship. APPROACH: Interviews were recorded, transcribed, and analyzed using inductive content analysis. KEY RESULTS: We interviewed 17 committee members. Within and across specialties, committee members had distinct approaches to prioritizing and synthesizing assessment data. Participants expressed concerns about the quality of assessments, necessitating careful scrutiny of language, assessor identity, and other contextual factors. Committee members were concerned about how unconscious bias might impact assessors, but they felt minimally impacted at the committee level. When committee members knew students personally, they felt tension about how to use the information appropriately. Participants described high agreement within their committees; debate was more common when site directors reviewed students' files from other sites prior to meeting. Participants reported multiple committee benefits including faculty development and fulfillment, as well as improved grading consistency, fairness, and transparency. Groupthink and a passive approach to bias emerged as the two main threats to optimal group decision-making. CONCLUSIONS: Grading committee members view their practices as advantageous over individual grading, but they feel limited in their ability to address grading fairness and accuracy. Recommendations and support may help committees broaden their scope to address these aspirations.
BACKGROUND: Faculty and students debate the fairness and accuracy of medical student clerkship grades. Group decision-making is a potential strategy to improve grading. OBJECTIVE: To explore how one school's grading committee members integrate assessment data to inform grade decisions and to identify the committees' benefits and challenges. DESIGN: This qualitative study used semi-structured interviews with grading committee chairs and members conducted between November 2017 and March 2018. PARTICIPANTS: Participants included the eight core clerkship directors, who chaired their grading committees. We randomly selected other committee members to invite, for a maximum of three interviews per clerkship. APPROACH: Interviews were recorded, transcribed, and analyzed using inductive content analysis. KEY RESULTS: We interviewed 17 committee members. Within and across specialties, committee members had distinct approaches to prioritizing and synthesizing assessment data. Participants expressed concerns about the quality of assessments, necessitating careful scrutiny of language, assessor identity, and other contextual factors. Committee members were concerned about how unconscious bias might impact assessors, but they felt minimally impacted at the committee level. When committee members knew students personally, they felt tension about how to use the information appropriately. Participants described high agreement within their committees; debate was more common when site directors reviewed students' files from other sites prior to meeting. Participants reported multiple committee benefits including faculty development and fulfillment, as well as improved grading consistency, fairness, and transparency. Groupthink and a passive approach to bias emerged as the two main threats to optimal group decision-making. CONCLUSIONS: Grading committee members view their practices as advantageous over individual grading, but they feel limited in their ability to address grading fairness and accuracy. Recommendations and support may help committees broaden their scope to address these aspirations.
Entities:
Keywords:
clerkship grading; clinical competence; evaluation; grading committees; group decision-making; medical education-qualitative methods; medical education-undergraduate
Authors: Michael J Battistone; Caroline Milne; Merle A Sande; Louis N Pangaro; Paul A Hemmer; T Samuel Shomaker Journal: Teach Learn Med Date: 2002 Impact factor: 2.414
Authors: Katherine B Lee; Sanjeev N Vaishnavi; Steven K M Lau; Dorothy A Andriole; Donna B Jeffe Journal: J Natl Med Assoc Date: 2007-10 Impact factor: 1.798