Mike Tweed1, Tim Wilkinson. 1. School of Medicine & Health Sciences, University of Otago Wellington, Wellington, New Zealand. mike.tweed@otago.ac.nz
Abstract
BACKGROUND: Marking of multiple choice type examinations often just takes account of the correct responses. This may encourage guessing of incorrect and potentially unsafe responses. Ideally responses should contain a high proportion of correct, unsafe and the use of 'don't know' response rather than incorrect. AIM: This study explored the effect of instructions on responses. METHODS:Fourth- and fifth-year students sitting for an optional multiple choice examination were randomized to receive one of four instruction options: number-correct marking (the control group); alert to unsafe; mark deduction for unsafe; or correction for guessing. RESULTS: A total of 210 students sat the test. For the fourth-year cohort, compared with the control group, being alerted to unsafe and mark deduction for unsafe or incorrect responses were associated with graduated increases in the use of 'don't know' and reductions in incorrect responses. For the fifth-year cohort, there were no differences in responses between options. The fifth-year cohort, gave more correct, and fewer incorrect, unsafe and 'don't know' responses. CONCLUSIONS: Both the year group and instructions had an effect. Being alerted to potentially unsafe responses, even when there is no mark deduction penalty, had an effect, although mark deduction still had the greatest effect. Assessment instructions may give subliminal messages that have important consequences.
RCT Entities:
BACKGROUND: Marking of multiple choice type examinations often just takes account of the correct responses. This may encourage guessing of incorrect and potentially unsafe responses. Ideally responses should contain a high proportion of correct, unsafe and the use of 'don't know' response rather than incorrect. AIM: This study explored the effect of instructions on responses. METHODS: Fourth- and fifth-year students sitting for an optional multiple choice examination were randomized to receive one of four instruction options: number-correct marking (the control group); alert to unsafe; mark deduction for unsafe; or correction for guessing. RESULTS: A total of 210 students sat the test. For the fourth-year cohort, compared with the control group, being alerted to unsafe and mark deduction for unsafe or incorrect responses were associated with graduated increases in the use of 'don't know' and reductions in incorrect responses. For the fifth-year cohort, there were no differences in responses between options. The fifth-year cohort, gave more correct, and fewer incorrect, unsafe and 'don't know' responses. CONCLUSIONS: Both the year group and instructions had an effect. Being alerted to potentially unsafe responses, even when there is no mark deduction penalty, had an effect, although mark deduction still had the greatest effect. Assessment instructions may give subliminal messages that have important consequences.