Vanessa C Burch1, Janet L Seggie. 1. Department of Medicine, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa. vanessa.burch@uct.ac.za
Abstract
CONTEXT: Portfolio-based learning is a popular educational tool usually examined by document review which is sometimes accompanied by an oral examination. This labour-intensive assessment method prohibits its use in the resource-constrained settings typical of developing countries. OBJECTIVES: We aimed to determine the feasibility and internal consistency of a portfolio-based structured interview and its impact on student learning behaviour. METHODS: Year 4 medical students (n = 181) recorded 25 patient encounters during a 14-week medical clerkship. Portfolios were examined in a 30-minute, single-examiner interview in which four randomly selected cases were discussed. Six standard questions were used to guide examiners in determining the ability of candidates to interpret and synthesise clinical data gathered during patient encounters. Examiners were trained to score responses using a global rating scale. Pearson's correlation co-efficient, Cronbach's alpha coefficient and the standard error of measurement (SEM) of the assessment tool were determined. The number of students completing more than the required number of portfolio entries was also recorded. RESULTS: The mean (+/- standard deviation [SD], 95% confidence interval [CI]) interview score was 67.5% (SD +/- 10.5, 95% CI 66.0-69.1). The correlation coefficients for the interview compared with other component examinations of the assessment process were: multiple-choice question (MCQ) examination 0.42; clinical case-based examination 0.37; in-course global rating 0.08, and overall final score 0.54. Cronbach's alpha coefficient was 0.88 and the SEM was 3.6. Of 181 students, 45.3% completed more than 25 portfolio entries. CONCLUSIONS: Portfolio assessment using a 30-minute structured interview is a feasible, internally consistent assessment method that requires less examination time per candidate relative to methods described in published work and which may encourage desirable student learning behaviour.
CONTEXT: Portfolio-based learning is a popular educational tool usually examined by document review which is sometimes accompanied by an oral examination. This labour-intensive assessment method prohibits its use in the resource-constrained settings typical of developing countries. OBJECTIVES: We aimed to determine the feasibility and internal consistency of a portfolio-based structured interview and its impact on student learning behaviour. METHODS: Year 4 medical students (n = 181) recorded 25 patient encounters during a 14-week medical clerkship. Portfolios were examined in a 30-minute, single-examiner interview in which four randomly selected cases were discussed. Six standard questions were used to guide examiners in determining the ability of candidates to interpret and synthesise clinical data gathered during patient encounters. Examiners were trained to score responses using a global rating scale. Pearson's correlation co-efficient, Cronbach's alpha coefficient and the standard error of measurement (SEM) of the assessment tool were determined. The number of students completing more than the required number of portfolio entries was also recorded. RESULTS: The mean (+/- standard deviation [SD], 95% confidence interval [CI]) interview score was 67.5% (SD +/- 10.5, 95% CI 66.0-69.1). The correlation coefficients for the interview compared with other component examinations of the assessment process were: multiple-choice question (MCQ) examination 0.42; clinical case-based examination 0.37; in-course global rating 0.08, and overall final score 0.54. Cronbach's alpha coefficient was 0.88 and the SEM was 3.6. Of 181 students, 45.3% completed more than 25 portfolio entries. CONCLUSIONS: Portfolio assessment using a 30-minute structured interview is a feasible, internally consistent assessment method that requires less examination time per candidate relative to methods described in published work and which may encourage desirable student learning behaviour.