Towards outcome-based programme evaluation: using student comparative self-assessments to determine teaching effectiveness.

BACKGROUND: Programme evaluation of medical education should be multi-dimensional. While structural and organisational aspects of teaching are frequently assessed, programme evaluation tools are rarely matched to specific learning objectives. AIMS: This study used one medical school's catalogue of specific learning objectives to implement and critically appraise a novel programme evaluation tool based on comparative student self-assessments.
METHOD: Medical students enrolled in the clinical phase of the undergraduate curriculum in Göttingen were invited to self-rate their knowledge, skills and attitudes before and after each course. A newly developed formula controlling for student performance levels when entering a course was used to compute a percentage gain in knowledge, skills and attitudes. Data derived from a prospective, longitudinal intervention study on the development of electrocardiogram interpretation skills including 636 students from four consecutive cohorts were used to provide validity evidence of the new approach.
RESULTS: The novel tool appeared superior to plain mean differences and effect sizes in detecting outstanding teaching as well as shortcomings of the curriculum. In addition, it adequately reflected objectively measured performance levels and was responsive to curriculum change.
CONCLUSIONS: Comparative student self-assessment is a valid tool to appraise undergraduate medical curricula at the level of specific learning objectives.