P Lambe1, E Kay1, D Bristow1. 1. Schools of Medicine and Dentistry, Plymouth University, Plymouth, UK.
Abstract
AIM: In 2013 the UKCAT included a non-cognitive situational judgement test in addition to the four cognitive subtests commonly used by UK dental schools to select students. However, little is known about the situational judgement test's psychometric properties and relationship to other selection tools. This study's aim was explore these issues to inform decisions about the inclusion of the UKCAT SJT in the dental student selection process. METHOD: The sample comprised a cohort of applicants to a Bachelor of Dental Surgery programme, at a UK dental school, which does not use achievement in the UKCAT SJT in its selection process. The impact on applicant outcomes of two theoretical uses of the UKCAT SJT was examined. First, SJT Band was used as the criterion for offers of a place instead of the school's admission interview, and second, weighted UKCAT scores, including a weighted SJT Band score, were ranked to make interview invitation decisions. Associations between SJT Band, UKCAT cognitive scores, interview score and performance in first year assessments were examined. RESULTS: If SJT Band 1 & 2 were used as the criterion for an offer of a place, some applicants rejected by this school's interview, including 'red flagged' applicants, would have received an offer of a place. Using a weighted UKCAT/SJT system for invitation for interview decisions increased the mean total UKCAT cognitive score of those invited for interview but included applicants rejected by this school's structured interview, including 'red flagged' applicants. Neither usage disadvantaged under-represented groups. SJT Band correlated with UKCAT score (n = 228, rs = -0.38, P < 0.01) with interview score (n = 186, rs = -0.17, P < 0.05) but not with first year study assessments. CONCLUSION: This study has shown that the UKCAT SJT does not add value to the existing methods of this dental school to make valid, reliable and fair student selection decisions.
AIM: In 2013 the UKCAT included a non-cognitive situational judgement test in addition to the four cognitive subtests commonly used by UK dental schools to select students. However, little is known about the situational judgement test's psychometric properties and relationship to other selection tools. This study's aim was explore these issues to inform decisions about the inclusion of the UKCAT SJT in the dental student selection process. METHOD: The sample comprised a cohort of applicants to a Bachelor of Dental Surgery programme, at a UK dental school, which does not use achievement in the UKCAT SJT in its selection process. The impact on applicant outcomes of two theoretical uses of the UKCAT SJT was examined. First, SJT Band was used as the criterion for offers of a place instead of the school's admission interview, and second, weighted UKCAT scores, including a weighted SJT Band score, were ranked to make interview invitation decisions. Associations between SJT Band, UKCAT cognitive scores, interview score and performance in first year assessments were examined. RESULTS: If SJT Band 1 & 2 were used as the criterion for an offer of a place, some applicants rejected by this school's interview, including 'red flagged' applicants, would have received an offer of a place. Using a weighted UKCAT/SJT system for invitation for interview decisions increased the mean total UKCAT cognitive score of those invited for interview but included applicants rejected by this school's structured interview, including 'red flagged' applicants. Neither usage disadvantaged under-represented groups. SJT Band correlated with UKCAT score (n = 228, rs = -0.38, P < 0.01) with interview score (n = 186, rs = -0.17, P < 0.05) but not with first year study assessments. CONCLUSION: This study has shown that the UKCAT SJT does not add value to the existing methods of this dental school to make valid, reliable and fair student selection decisions.