Cassandra Barber1, Robert Hammond, Lorne Gula, Gary Tithecott, Saad Chahine. 1. C. Barber is assessment, data, and reporting analyst for undergraduate medical education, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada. R. Hammond is associate dean of admissions, professor, and program director, Neuropathology Residency Program, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada. L. Gula is professor, Departments of Medicine and Epidemiology and Biostatistics, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada. G. Tithecott is associate dean of undergraduate medical education and section head for general academic paediatrics, Department of Paediatrics, London Health Sciences Center, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada. S. Chahine is scientist, Center for Education Research and Innovation, and assistant professor, Department of Medicine, Faculty of Education, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada; ORCID: http://orcid.org/0000-0003-0488-773X.
Abstract
PURPOSE: To determine which admissions variables and curricular outcomes are predictive of being at risk of failing the Medical Council of Canada Qualifying Examination Part 1 (MCCQE1), how quickly student risk of failure can be predicted, and to what extent predictive modeling is possible and accurate in estimating future student risk. METHOD: Data from five graduating cohorts (2011-2015), Schulich School of Medicine & Dentistry, Western University, were collected and analyzed using hierarchical generalized linear models (HGLMs). Area under the receiver operating characteristic curve (AUC) was used to evaluate the accuracy of predictive models and determine whether they could be used to predict future risk, using the 2016 graduating cohort. Four predictive models were developed to predict student risk of failure at admissions, year 1, year 2, and pre-MCCQE1. RESULTS: The HGLM analyses identified gender, MCAT verbal reasoning score, two preclerkship course mean grades, and the year 4 summative objective structured clinical examination score as significant predictors of student risk. The predictive accuracy of the models varied. The pre-MCCQE1 model was the most accurate at predicting a student's risk of failing (AUC 0.66-0.93), while the admissions model was not predictive (AUC 0.25-0.47). CONCLUSIONS: Key variables predictive of students at risk were found. The predictive models developed suggest, while it is not possible to identify student risk at admission, we can begin to identify and monitor students within the first year. Using such models, programs may be able to identify and monitor students at risk quantitatively and develop tailored intervention strategies.
PURPOSE: To determine which admissions variables and curricular outcomes are predictive of being at risk of failing the Medical Council of Canada Qualifying Examination Part 1 (MCCQE1), how quickly student risk of failure can be predicted, and to what extent predictive modeling is possible and accurate in estimating future student risk. METHOD: Data from five graduating cohorts (2011-2015), Schulich School of Medicine & Dentistry, Western University, were collected and analyzed using hierarchical generalized linear models (HGLMs). Area under the receiver operating characteristic curve (AUC) was used to evaluate the accuracy of predictive models and determine whether they could be used to predict future risk, using the 2016 graduating cohort. Four predictive models were developed to predict student risk of failure at admissions, year 1, year 2, and pre-MCCQE1. RESULTS: The HGLM analyses identified gender, MCAT verbal reasoning score, two preclerkship course mean grades, and the year 4 summative objective structured clinical examination score as significant predictors of student risk. The predictive accuracy of the models varied. The pre-MCCQE1 model was the most accurate at predicting a student's risk of failing (AUC 0.66-0.93), while the admissions model was not predictive (AUC 0.25-0.47). CONCLUSIONS: Key variables predictive of students at risk were found. The predictive models developed suggest, while it is not possible to identify student risk at admission, we can begin to identify and monitor students within the first year. Using such models, programs may be able to identify and monitor students at risk quantitatively and develop tailored intervention strategies.
Authors: Michelle Seer; Charlotte Kampsen; Tim Becker; Sebastian Hobert; Sven Anders; Tobias Raupach Journal: PLoS One Date: 2022-05-11 Impact factor: 3.240