Assessment of first-year veterinary students' communication skills using an objective structured clinical examination: the importance of context.

Communication skills are considered to be a core clinical skill in veterinary medicine and essential for practice success, including outcomes of care for patients and clients. While veterinary schools include communication skills training in their programs, there is minimal knowledge on how best to assess communication competence throughout the undergraduate program. The purpose of this study was to further our understanding of the reliability, utility, and suitability of a communication skills Objective Structured Clinical Examination (OSCE). Specifically we wanted to (1) identify the greatest source of variability (student, rater, station, and track) within a first-year, four station OSCE using exam scores and scores from videotape review by two trained raters, and (2) determine the effect of different stations on students' communication skills performance. Reliability of the scores from both the exam data and the two expert raters was 0.50 and 0.46 respectively, with the greatest amount of variance attributable to student by station. The percentage of variance due to raters in the exam data was 16.35%, whereas the percentage of variance for the two expert raters was 0%. These results have three important implications. First, the results reinforce the need for communication educators to emphasize that use of communication skills is moderated by the context of the clinical interaction. Second, by increasing rater training the amount of error in the scores due to raters can be reduced and inter-rater reliability increases. Third, the communication assessment method (in this case the OSCE checklist) should be built purposefully, taking into consideration the context of the case.