OBJECTIVE: To evaluate how the utility (reliability, validity, acceptability, feasibility, cost and educational impact) of a communication-OSCE was influenced by whether or not station-specific (StSp) checklists were used together with a generic instrument and whether or not narrative feedback was provided to students. METHODS: At ten stations, faculty members rated standardized patient-student interactions using the common ground (CG) instrument (at all stations) and StSp-checklists. Both raters and patients provided written feedback. The impact of changing the design on the various utility parameters was assessed: reliability by means of a generalizability study, cost using the Reznick model and the other utility parameters by means of a survey. RESULTS: Use of the generic instrument (CG) proved more reliable (G coefficient=0.67) than using the StSp-checklists (G=0.47) or both (G=0.65) while there was a high correlation between both scale scores (Pearsons'r=0.86). The cost was 6.5% higher when StSp-checklists were used and 5% higher when narrative feedback was provided. CONCLUSION: The utility of a communication OSCE can be enhanced by omitting StSp-checklists and by providing narrative feedback to students. PRACTICE IMPLICATIONS: The same generic assessment scale can be used in all stations of a communication OSCE. Providing feedback to students is promising but it increases the costs.
OBJECTIVE: To evaluate how the utility (reliability, validity, acceptability, feasibility, cost and educational impact) of a communication-OSCE was influenced by whether or not station-specific (StSp) checklists were used together with a generic instrument and whether or not narrative feedback was provided to students. METHODS: At ten stations, faculty members rated standardized patient-student interactions using the common ground (CG) instrument (at all stations) and StSp-checklists. Both raters and patients provided written feedback. The impact of changing the design on the various utility parameters was assessed: reliability by means of a generalizability study, cost using the Reznick model and the other utility parameters by means of a survey. RESULTS: Use of the generic instrument (CG) proved more reliable (G coefficient=0.67) than using the StSp-checklists (G=0.47) or both (G=0.65) while there was a high correlation between both scale scores (Pearsons'r=0.86). The cost was 6.5% higher when StSp-checklists were used and 5% higher when narrative feedback was provided. CONCLUSION: The utility of a communication OSCE can be enhanced by omitting StSp-checklists and by providing narrative feedback to students. PRACTICE IMPLICATIONS: The same generic assessment scale can be used in all stations of a communication OSCE. Providing feedback to students is promising but it increases the costs.
Authors: Musa Cömert; Jördis Maria Zill; Eva Christalle; Jörg Dirmaier; Martin Härter; Isabelle Scholl Journal: PLoS One Date: 2016-03-31 Impact factor: 3.240