BACKGROUND: Clinical skills examinations (CSEs) are used frequently in medical education. Standard setting for CSEs may employ different methods with or without prior performance data. METHOD: An expert panel provided item-based (Angoff) and group-based (Hofstee) judgments about two central venous catheter insertion performance checklists on three occasions. Judges did not receive baseline performance data on the first occasion but did on occasions two and three. Judges' ratings were used to calculate a minimum passing standard (MPS) for the CSE. Interrater reliabilities and test-retest reliability (stability) were calculated. Passing standards are compared using performance data from a pilot study. RESULTS: Both methods produced reliable and stable data. Baseline data influenced the judges' decisions. Use of the Angoff method alone yielded lenient MPSs, whereas the Hofstee method alone yielded stringent MPSs. CONCLUSIONS: Standard setting is a critical component of CSEs. Baseline data influence judges' decisions. Averaging Angoff and Hofstee outcomes produced the optimal MPS.
BACKGROUND: Clinical skills examinations (CSEs) are used frequently in medical education. Standard setting for CSEs may employ different methods with or without prior performance data. METHOD: An expert panel provided item-based (Angoff) and group-based (Hofstee) judgments about two central venous catheter insertion performance checklists on three occasions. Judges did not receive baseline performance data on the first occasion but did on occasions two and three. Judges' ratings were used to calculate a minimum passing standard (MPS) for the CSE. Interrater reliabilities and test-retest reliability (stability) were calculated. Passing standards are compared using performance data from a pilot study. RESULTS: Both methods produced reliable and stable data. Baseline data influenced the judges' decisions. Use of the Angoff method alone yielded lenient MPSs, whereas the Hofstee method alone yielded stringent MPSs. CONCLUSIONS: Standard setting is a critical component of CSEs. Baseline data influence judges' decisions. Averaging Angoff and Hofstee outcomes produced the optimal MPS.
Authors: Jeffrey H Barsuk; Elaine R Cohen; Timothy Caprio; William C McGaghie; Tanya Simuni; Diane B Wayne Journal: Neurology Date: 2012-06-06 Impact factor: 9.910
Authors: Lindsay C Johnston; Ruijun Chen; Travis M Whitfill; Christie J Bruno; Orly L Levit; Marc A Auerbach Journal: BMJ Simul Technol Enhanc Learn Date: 2015-05-20
Authors: Peter Hertz; Katrine Jensen; Saleh N Abudaff; Michael Strøm; Yousif Subhi; Hani Lababidi; Lars Konge Journal: BMJ Open Respir Res Date: 2018-12-10
Authors: David H Salzman; Kristin L Rising; Kenzie A Cameron; Rhea E Powell; Dimitri Papanagnou; Amanda Doty; Katherine Piserchia; Lori Latimer; William C McGaghie; Danielle M McCarthy Journal: J Grad Med Educ Date: 2020-02