S Beth Bierer1, Elaine F Dannefer. 1. Lerner College of Medicine of Case Western Reserve University, Cleveland Clinic, 9500 Euclid Avenue, Cleveland, OH 44195, USA. biererb@ccf.org
Abstract
PURPOSE: Measurement experts use four criteria to examine the fairness of tests: (1) equitable treatment for examinees, (2) equal outcomes for subgroups, (3) absence of bias, and (4) equal opportunity to learn. These criteria apply to portfolios just as they do to other assessments. This report examines the fairness of portfolio-based promotion decisions for medical students at the Cleveland Clinic Lerner College of Medicine. METHOD: Participants were 182 first-year medical students (97 men, 85 women) from six class cohorts (2004-2009). Chi-square statistics with Yates continuity correction were used to compare overall promotion decisions to students' gender, self-reports of language fluency, and MCAT Writing Sample score. The Cramér V statistic served as an effect size index. Post hoc power analyses identified the minimum sample size to obtain acceptable power. RESULTS: Approximately 85% of students were promoted to Year 2 of the program. Gender, U.S. citizenship, language fluency, and MCAT Writing Sample score were not significantly related to overall promotion decisions. Effect sizes were small (≤0.15) for all contingency tables, suggesting weak associations between overall promotion decisions and students' group characteristics. CONCLUSIONS: Examining fairness, although challenging, is essential to maintain professional standards and avoid potential liability. Preliminary evidence in this study suggests that students' background characteristics and verbal abilities were not strongly related to portfolio-based promotion decisions. Schools should monitor processes that may affect fairness. This study reports on just one aspect of fairness. More research is needed to evaluate other dimensions of fairness.
PURPOSE: Measurement experts use four criteria to examine the fairness of tests: (1) equitable treatment for examinees, (2) equal outcomes for subgroups, (3) absence of bias, and (4) equal opportunity to learn. These criteria apply to portfolios just as they do to other assessments. This report examines the fairness of portfolio-based promotion decisions for medical students at the Cleveland Clinic Lerner College of Medicine. METHOD:Participants were 182 first-year medical students (97 men, 85 women) from six class cohorts (2004-2009). Chi-square statistics with Yates continuity correction were used to compare overall promotion decisions to students' gender, self-reports of language fluency, and MCAT Writing Sample score. The Cramér V statistic served as an effect size index. Post hoc power analyses identified the minimum sample size to obtain acceptable power. RESULTS: Approximately 85% of students were promoted to Year 2 of the program. Gender, U.S. citizenship, language fluency, and MCAT Writing Sample score were not significantly related to overall promotion decisions. Effect sizes were small (≤0.15) for all contingency tables, suggesting weak associations between overall promotion decisions and students' group characteristics. CONCLUSIONS: Examining fairness, although challenging, is essential to maintain professional standards and avoid potential liability. Preliminary evidence in this study suggests that students' background characteristics and verbal abilities were not strongly related to portfolio-based promotion decisions. Schools should monitor processes that may affect fairness. This study reports on just one aspect of fairness. More research is needed to evaluate other dimensions of fairness.