J de Laffolie1,2, D Visser3, M Hirschburger4, S Turial5. 1. Abt Allgemeine Pädiatrie und Neonatologie, Universitätsklinikum Giessen und Marburg, Standort Gießen, Feulgenstr 14, 35392, Gießen, Deutschland. Jan.delaffolie@paediat.med.uni-giessen.de. 2. Klinik für Kinderchirurgie und angeborene Fehlbildungen, Horst-Schmidt-Klinik Wiesbaden, Wiesbaden, Deutschland. Jan.delaffolie@paediat.med.uni-giessen.de. 3. Abt Allgemeine Pädiatrie und Neonatologie, Universitätsklinikum Giessen und Marburg, Standort Gießen, Feulgenstr 14, 35392, Gießen, Deutschland. 4. Abteilung für Chirurgie, Städtisches Krankenhaus Worms, Worms, Deutschland. 5. Klinik für Kinderchirurgie und angeborene Fehlbildungen, Horst-Schmidt-Klinik Wiesbaden, Wiesbaden, Deutschland.
Abstract
AIM: Multiple choice questions (MCQs) are the most common written test item in medical examinations. Writing MCQs is difficult and cues can decrease test validity. The aim of this article is to describe the prevalence and pattern of cues in surgical MCQs in the central German medical examination questions from 2000-2011. METHOD: All surgical questions were entered into a database. The questions were reviewed for cues and pseudocues independently by three students of different academic background and one clinical physician. RESULTS: Initially, 1014 questions were included, 22 questions were not rated uniformly by the reviewers and 3 questions were excluded because no consensus could be reached. Overall 15.2 % of the questions analyzed contained some type of cue or pseudocue. Of the total questions 0.2 % contained type A cues (i.e. disruption of grammatical flow), 6.5 % contained type B cues (i.e. heterogeneous length or differentiation), 4.0 % contained type C cues (two or more answers described closely related topics, serving to focus attention), 0.6 % contained type D convergence cues (i.e. answers with the most items in common with distractors are correct), 0.7 % contained type E verbal association cues (i.e. connection in the use of words leading to the correct answer) and 1.0 % contained type F cues (i.e. answers with absolute terms). Pseudocues were found in 3.7 % of the questions. DISCUSSION: The proportion of questions that contained cues or pseudocues should lead to further efforts to avoid such factors compromising test reliability by specific attention in the process of question design and review. CONCLUSION: Cues are still an important consideration in designing MCQs and are present in considerable numbers in medical state examination questions. Pseudocues should be explicitly avoided so as not to compromise validity and reliability.
AIM: Multiple choice questions (MCQs) are the most common written test item in medical examinations. Writing MCQs is difficult and cues can decrease test validity. The aim of this article is to describe the prevalence and pattern of cues in surgical MCQs in the central German medical examination questions from 2000-2011. METHOD: All surgical questions were entered into a database. The questions were reviewed for cues and pseudocues independently by three students of different academic background and one clinical physician. RESULTS: Initially, 1014 questions were included, 22 questions were not rated uniformly by the reviewers and 3 questions were excluded because no consensus could be reached. Overall 15.2 % of the questions analyzed contained some type of cue or pseudocue. Of the total questions 0.2 % contained type A cues (i.e. disruption of grammatical flow), 6.5 % contained type B cues (i.e. heterogeneous length or differentiation), 4.0 % contained type C cues (two or more answers described closely related topics, serving to focus attention), 0.6 % contained type D convergence cues (i.e. answers with the most items in common with distractors are correct), 0.7 % contained type E verbal association cues (i.e. connection in the use of words leading to the correct answer) and 1.0 % contained type F cues (i.e. answers with absolute terms). Pseudocues were found in 3.7 % of the questions. DISCUSSION: The proportion of questions that contained cues or pseudocues should lead to further efforts to avoid such factors compromising test reliability by specific attention in the process of question design and review. CONCLUSION: Cues are still an important consideration in designing MCQs and are present in considerable numbers in medical state examination questions. Pseudocues should be explicitly avoided so as not to compromise validity and reliability.
Keywords:
Assessment; Cues; Medical Education; Multiple Choice Questions; Surgical Education
Authors: Ralph F Jozefowicz; Bruce M Koeppen; Susan Case; Robert Galbraith; David Swanson; Robert H Glew Journal: Acad Med Date: 2002-02 Impact factor: 6.893
Authors: C Alex McMahan; R Neal Pinckard; Thomas J Prihoda; William D Hendricson; Anne Cale Jones Journal: J Dent Educ Date: 2013-12 Impact factor: 2.264