David J Caldwell1, Adam N Pate. 1. University of Louisiana at Monroe College of Pharmacy, Monroe, LA, USA. dcaldwell@ulm.edu
Abstract
OBJECTIVE: To determine the effect of 3 variations in test item format on item statistics and student performance. METHODS: Fifteen pairs of directly comparable test questions were written to adhere to (standard scale) or deviate from (nonstandard scale) 3 specific item-writing guidelines. Differences in item difficulty and discrimination were measured between the 2 scales as a whole and for each guideline individually. Student performance was also compared between the 2 scales. RESULTS: The nonstandard scale was 12.7 points more difficult than the standard scale (p=0.03). The guideline to avoid "none of the above" was the only 1 of the 3 guidelines to demonstrate significance. Students scored 53.6% and 41.3% (p<0.001) of total points on the standard and nonstandard scales, respectively. CONCLUSIONS: Nonstandard test items were more difficult for students to answer correctly than the standard test items, provided no enhanced ability to discriminate between higher- and lower-performing students, and resulted in poorer student performance. Item-writing guidelines should be considered during test construction.
OBJECTIVE: To determine the effect of 3 variations in test item format on item statistics and student performance. METHODS: Fifteen pairs of directly comparable test questions were written to adhere to (standard scale) or deviate from (nonstandard scale) 3 specific item-writing guidelines. Differences in item difficulty and discrimination were measured between the 2 scales as a whole and for each guideline individually. Student performance was also compared between the 2 scales. RESULTS: The nonstandard scale was 12.7 points more difficult than the standard scale (p=0.03). The guideline to avoid "none of the above" was the only 1 of the 3 guidelines to demonstrate significance. Students scored 53.6% and 41.3% (p<0.001) of total points on the standard and nonstandard scales, respectively. CONCLUSIONS: Nonstandard test items were more difficult for students to answer correctly than the standard test items, provided no enhanced ability to discriminate between higher- and lower-performing students, and resulted in poorer student performance. Item-writing guidelines should be considered during test construction.
Keywords:
assessment; examination; multiple-choice questions; test construction; test item-writing guidelines
Authors: Mary Elizabeth Ray; Kimberly K Daugherty; Lisa Lebovitz; Michael J Rudolph; Veronica P Shuford; Margarita V DiVall Journal: Am J Pharm Educ Date: 2018-12 Impact factor: 2.047
Authors: Hamza Mohammad Abdulghani; Mohammad Irshad; Shafiul Haque; Tauseef Ahmad; Kamran Sattar; Mahmoud Salah Khalil Journal: PLoS One Date: 2017-10-10 Impact factor: 3.240