INTRODUCTION: While the extant literature has explored the impact of stations on multiple mini- interview (MMI) scores, the influence of station scenarios has been largely overlooked. METHOD: A subset of MMI scores was purposively sampled from admissions data at one US medical school. Generalizability (G) theory was used to estimate variance components attributable to applicants and two facets of generalization - scenarios, the content of the station, and items, the attributes assessed. RESULTS: G study suggests that the greatest amount of variance is attributable to the main effect of the scenario (s) facet and the interaction between applicant and scenario (ps), which account for 77% of the total variance. The item facet (i) accounts for only 0.6% of total variance; likewise, the scenario-item interaction (si) accounts for only 1.4% of the total variance. DISCUSSION: While the researchers expected to find a large variance component associated with the scenario-item interaction, this analysis does not support this assumption. The researchers interpret the small scenario-item interaction as a result of variance attributable to the item facet being subsumed by the variance attributable to the content of the scenarios. CONCLUSIONS: The results of this study reinforce the need to examine psychometric properties of the MMI.
INTRODUCTION: While the extant literature has explored the impact of stations on multiple mini- interview (MMI) scores, the influence of station scenarios has been largely overlooked. METHOD: A subset of MMI scores was purposively sampled from admissions data at one US medical school. Generalizability (G) theory was used to estimate variance components attributable to applicants and two facets of generalization - scenarios, the content of the station, and items, the attributes assessed. RESULTS: G study suggests that the greatest amount of variance is attributable to the main effect of the scenario (s) facet and the interaction between applicant and scenario (ps), which account for 77% of the total variance. The item facet (i) accounts for only 0.6% of total variance; likewise, the scenario-item interaction (si) accounts for only 1.4% of the total variance. DISCUSSION: While the researchers expected to find a large variance component associated with the scenario-item interaction, this analysis does not support this assumption. The researchers interpret the small scenario-item interaction as a result of variance attributable to the item facet being subsumed by the variance attributable to the content of the scenarios. CONCLUSIONS: The results of this study reinforce the need to examine psychometric properties of the MMI.
Authors: Ann Blair Kennedy; Cindy Nessim Youssef Riyad; Laura H Gunn; April Gant Brown; Kandyce Brooke Dunlap; Melissa Elizabeth Knutsen; Alicia Anne Dahl Journal: Med Sci Educ Date: 2020-07-02
Authors: Ann Blair Kennedy; Cindy Nessim Youssef Riyad; Ryan Ellis; Perry R Fleming; Mallorie Gainey; Kara Templeton; Anna Nourse; Virginia Hardaway; April Brown; Pam Evans; Nabil Natafgi Journal: J Particip Med Date: 2022-08-30