H Glenn Anderson1, Arthur A Nelson. 1. Texas Tech University Health Sciences Center, School of Pharmacy, USA. glenn.anderson@marshall.edu
Abstract
OBJECTIVE: To compare the reliability and credibility of Angoff-based, absolute criteria derived by faculty, alumni, and a combination of alumni and faculty judge panels. METHODS: Independently, faculty, alumni, and mixed faculty-alumni judge panels developed pass/fail criteria for an 86-item test. Generalizability and decision studies were performed. Root mean square errors (RMSE) and 95% confidence intervals were calculated for reliability and credibility assessment. School graduate performance upon the North American Licensure Examination (NAPLEX) was the comparator for credibility assessment. RESULTS: RMSEs were 1.06%, 1.42%, and 2.32% for the alumni, faculty, and mixed judge panels respectively. The school's NAPLEX pass rate was 97.5%. This rate triangulated well with the faculty judge panel (pass rate = 93.9%, CI95% = 87.1%-98.2%), but did not with either mixed judge or alumni judge panels. CONCLUSIONS: Faculty-derived criteria offer superior pass/fail decision defensibility relative to both alumni derived and mixed faculty-alumni derived criteria.
OBJECTIVE: To compare the reliability and credibility of Angoff-based, absolute criteria derived by faculty, alumni, and a combination of alumni and faculty judge panels. METHODS: Independently, faculty, alumni, and mixed faculty-alumni judge panels developed pass/fail criteria for an 86-item test. Generalizability and decision studies were performed. Root mean square errors (RMSE) and 95% confidence intervals were calculated for reliability and credibility assessment. School graduate performance upon the North American Licensure Examination (NAPLEX) was the comparator for credibility assessment. RESULTS: RMSEs were 1.06%, 1.42%, and 2.32% for the alumni, faculty, and mixed judge panels respectively. The school's NAPLEX pass rate was 97.5%. This rate triangulated well with the faculty judge panel (pass rate = 93.9%, CI95% = 87.1%-98.2%), but did not with either mixed judge or alumni judge panels. CONCLUSIONS: Faculty-derived criteria offer superior pass/fail decision defensibility relative to both alumni derived and mixed faculty-alumni derived criteria.
Authors: B H Verhoeven; A F van der Steeg; A J Scherpbier; A M Muijtjens; G M Verwijnen; C P van der Vleuten Journal: Med Educ Date: 1999-11 Impact factor: 6.251
Authors: H Glenn Anderson; Lisa Frazier; Stephanie L Anderson; Robert Stanton; Chris Gillette; Kim Broedel-Zaugg; Kevin Yingling Journal: Am J Pharm Educ Date: 2017-05 Impact factor: 2.047
Authors: Chris Gillette; Robert B Stanton; Nicole Rockich-Winston; Michael Rudolph; H Glenn Anderson Journal: Am J Pharm Educ Date: 2017-12 Impact factor: 2.047
Authors: Chris Gillette; Michael Rudolph; Nicole Rockich-Winston; Robert Stanton; H Glenn Anderson Journal: Am J Pharm Educ Date: 2017-08 Impact factor: 2.047