Developing an OSCE station to assess the ability of medical students to share information and decisions with patients: issues relating to interrater reliability and the use of simulated patients.

CONTEXT: Patient partnership is being promoted as an aid to compliance with treatment and to improve outcomes for patients. An integral part of this partnership is information-sharing between doctor and patient and negotiation of management decisions, together with an ability by the doctor to analyse critically treatment options. These skills are being taught to some extent in communication skills for medical students and assessments are being devised to test them.
OBJECTIVES: To develop an OSCE station to assess medical students' skills in the application of evidence and the negotiation of treatment options with a simulated patient. To evaluate the station by tests of reliability and comparison of the marking of observing examiners and simulated patients.
DESIGN: An OSCE station was devised using a case scenario in which there were varying opinions as to the treatment options. The marking schedule was designed to assess the students' ability to assess evidence and discuss this with patients.
SETTING: University of Leeds.
SUBJECTS: Undergraduate medical students.
RESULTS: A total of 194 students undertook the station. Inter-rater reliability amongst examiners was poor for individual student marks but there was better agreement as to whether a student passed or failed. There was poor agreement between the observing examiners and the simulated patients for individual marks. The examiners thought this was a useful attempt to assess students' ability to give information but agreed that further work was needed on the marking schedule.
CONCLUSIONS: More work is needed to refine the station to increase its reliability. However, the station is useful in highlighting to students the importance of sharing information and decision-making on treatment options with patients.