OBJECTIVE: To assess effectiveness of a training program in reducing inter-grader variability in grading communication skills during an objective structured clinical exam (OSCE). METHODS: Global communication (GC) skills are assessed by standardized participants (SP) and faculty during each OSCE using a 6 item rubric. Despite criteria delineated in the GC rubric, inter-grader variability was observed. During 2008-2009 academic year, a training program was implemented before each OSCE to achieve more consistent interpretation and grading in GC skills. GC grades between SP and faculty for 2nd and 3rd level student OSCEs during 2008-2009 (post-training) were compared to 2007-2008 (pre-training). Data was analyzed using repeated measures ANOVA. RESULTS: 274 and 281 students participated in OSCEs during 2007-2008 and 2008-2009 academic years, respectively. Training significantly (P<.001) decreased grader variability between SPs and faculty. There was a greater mean difference between faculty vs. SP before training (faculty 14.68, SP 15.87) compared to after training (faculty 13.51, SP 13.78). Mean GC scores for both faculty and SPs also decreased significantly after training. CONCLUSION AND PRACTICE IMPLICATIONS: A training program may be necessary to reduce inter-rater variability in assessment of OSCE communication skills if it is to be truly helpful to student pharmacists learning to counsel patients.
OBJECTIVE: To assess effectiveness of a training program in reducing inter-grader variability in grading communication skills during an objective structured clinical exam (OSCE). METHODS: Global communication (GC) skills are assessed by standardized participants (SP) and faculty during each OSCE using a 6 item rubric. Despite criteria delineated in the GC rubric, inter-grader variability was observed. During 2008-2009 academic year, a training program was implemented before each OSCE to achieve more consistent interpretation and grading in GC skills. GC grades between SP and faculty for 2nd and 3rd level student OSCEs during 2008-2009 (post-training) were compared to 2007-2008 (pre-training). Data was analyzed using repeated measures ANOVA. RESULTS: 274 and 281 students participated in OSCEs during 2007-2008 and 2008-2009 academic years, respectively. Training significantly (P<.001) decreased grader variability between SPs and faculty. There was a greater mean difference between faculty vs. SP before training (faculty 14.68, SP 15.87) compared to after training (faculty 13.51, SP 13.78). Mean GC scores for both faculty and SPs also decreased significantly after training. CONCLUSION AND PRACTICE IMPLICATIONS: A training program may be necessary to reduce inter-rater variability in assessment of OSCE communication skills if it is to be truly helpful to student pharmacists learning to counsel patients.
Authors: Kimberly Baker-Genaw; Maria S Kokas; Syed F Ahsan; Deborah Darnley-Fisch; Sean Drake; Nikhil Goyal; Kedar Inamdar; Vasilios Moutzouros; Deepak Prabhakar; Laurie Rolland; Roopina Sangha; Maria Shreve; Ann Woodward Journal: J Grad Med Educ Date: 2016-07
Authors: Safeera Y Hussainy; Matthew F Crum; Paul J White; Ian Larson; Daniel T Malone; David T Manallack; Joseph A Nicolazzo; Jennifer McDowell; Angelina S Lim; Carl M Kirkpatrick Journal: Am J Pharm Educ Date: 2016-11-25 Impact factor: 2.047
Authors: Susanne G Barnett; Sheila M Allen; Karen Ms Bastianelli; Jennifer S Chen; Colleen A Clark Dula; Marlowe Djuric Kachlic; Kristen L Goliak; Laura E Knockel; David E Matthews; Lucio R Volino; Michael R Lasarev; Jeffrey C Reist Journal: Am J Pharm Educ Date: 2021-07-22 Impact factor: 2.047