OBJECTIVES: In multiple-choice tests using a 'don't-know' option the number of correct minus incorrect answers was used as the test score (formula scoring) in order to reduce the measurement error resulting from random guessing. In the literature diverging results are reported when comparing formula scoring and number-right scoring, the scoring method without the don't-know option. DESIGN: To investigate which method was most appropriate, both scoring methods were used in true-false tests (block tests) taken at the end of a second- and third-year educational module (block). The students were asked to answer each item initially by choosing from the response options true, false or don't know, and secondly to replace all don't-know answers by a true-false answer. SETTING: Maastricht University, The Netherlands. SUBJECTS: Medical students. RESULTS: The correct scores for the don't-know answered items were found to be 4.5% and 5.9%, respectively, higher than expected with pure random guesswork. This represents a source of bias with formula scoring, because students who were less willing to guess obtained lower scores. The average difference in the correct minus incorrect score for the two scoring methods (2.5%, P < 0.001, and 3.4%, P < 0. 001, respectively) indicates the size of the bias (compare: the standard deviation of the score equals 11%). Test reliability was higher with formula scoring (0.72 vs. 0.66 and 0.74 vs. 0.66), but the difference decreased when the test was restricted to items which were close to the core content of the block (0.81 vs. 0.77, resp. 0. 75 vs. 0.70). CONCLUSIONS: In deciding what scoring method to use, less bias (number-right scoring) has to be weighed against higher reliability (formula scoring). Apart from these psychometric reasons educational factors must be considered.
OBJECTIVES: In multiple-choice tests using a 'don't-know' option the number of correct minus incorrect answers was used as the test score (formula scoring) in order to reduce the measurement error resulting from random guessing. In the literature diverging results are reported when comparing formula scoring and number-right scoring, the scoring method without the don't-know option. DESIGN: To investigate which method was most appropriate, both scoring methods were used in true-false tests (block tests) taken at the end of a second- and third-year educational module (block). The students were asked to answer each item initially by choosing from the response options true, false or don't know, and secondly to replace all don't-know answers by a true-false answer. SETTING: Maastricht University, The Netherlands. SUBJECTS: Medical students. RESULTS: The correct scores for the don't-know answered items were found to be 4.5% and 5.9%, respectively, higher than expected with pure random guesswork. This represents a source of bias with formula scoring, because students who were less willing to guess obtained lower scores. The average difference in the correct minus incorrect score for the two scoring methods (2.5%, P < 0.001, and 3.4%, P < 0. 001, respectively) indicates the size of the bias (compare: the standard deviation of the score equals 11%). Test reliability was higher with formula scoring (0.72 vs. 0.66 and 0.74 vs. 0.66), but the difference decreased when the test was restricted to items which were close to the core content of the block (0.81 vs. 0.77, resp. 0. 75 vs. 0.70). CONCLUSIONS: In deciding what scoring method to use, less bias (number-right scoring) has to be weighed against higher reliability (formula scoring). Apart from these psychometric reasons educational factors must be considered.
Authors: James S Rosneck; Joel Hughes; John Gunstad; Richard Josephson; Donald A Noe; Donna Waechter Journal: J Cardiovasc Nurs Date: 2014 May-Jun Impact factor: 2.083
Authors: Dario Cecilio-Fernandes; Harro Medema; Carlos Fernando Collares; Lambert Schuwirth; Janke Cohen-Schotanus; René A Tio Journal: BMC Med Educ Date: 2017-11-09 Impact factor: 2.463
Authors: Nicole M Robertson; Trishul Siddharthan; Suzanne L Pollard; Patricia Alupo; Oscar Flores-Flores; Natalie A Rykiel; Elisa D Romani; Ivonne Ascencio-Días; Bruce Kirenga; William Checkley; John R Hurst; Shumonta Quaderi Journal: Ann Am Thorac Soc Date: 2021-08
Authors: C J Ravesloot; M F Van der Schaaf; A M M Muijtjens; C Haaring; C L J J Kruitwagen; F J A Beek; J Bakker; J P J Van Schaik; Th J Ten Cate Journal: Adv Health Sci Educ Theory Pract Date: 2015-04-26 Impact factor: 3.853