Winny Setyonugroho1, Kieran M Kennedy2, Thomas J B Kropmans3. 1. Faculty of Medicine and Health Sciences of the Universitas Muhammadiyah Yogyakarta, Bantul, Indonesia. Electronic address: wsetyonugroho@umy.ac.id. 2. School of Medicine, College or Medicine, Nursing & Health Sciences, National University of Ireland Galway, Galway, Ireland. Electronic address: kieran.kennedy@nuigalway.ie. 3. School of Medicine, College or Medicine, Nursing & Health Sciences, National University of Ireland Galway, Galway, Ireland. Electronic address: thomas.kropmans@nuigalway.ie.
Abstract
OBJECTIVES: To explore inter-rater agreement between reviewers comparing reliability and validity of checklist forms that claim to assess the communication skills of undergraduate medical students in Objective Structured Clinical Examinations (OSCEs). METHODS: Papers explaining rubrics of OSCE checklist forms were identified from Pubmed, Embase, PsycINFO, and the ProQuest Education Databases up to 2013. Included were those studies that report empirical validity or reliability values for the communication skills assessment checklists used. Excluded were those papers that did not report reliability or validity. RESULTS: Papers focusing on generic communication skills, history taking, physician-patient communication, interviewing, negotiating treatment, information giving, empathy and 18 other domains (ICC -0.12-1) were identified. Regarding the validity and reliability of the communication skills checklists, agreement between reviewers was 0.45. CONCLUSIONS: Heterogeneity in the rubrics used in the assessment of communication skills and a lack of agreement between reviewers makes comparison of student competences within and across institutions difficult. PRACTICE IMPLICATIONS: Consideration should be afforded to the adoption of a standardized measurement instrument to assess communication skills in undergraduate medical education. Future research will focus upon evaluating the potential impact of adoption of a standardized measurement instrument.
OBJECTIVES: To explore inter-rater agreement between reviewers comparing reliability and validity of checklist forms that claim to assess the communication skills of undergraduate medical students in Objective Structured Clinical Examinations (OSCEs). METHODS: Papers explaining rubrics of OSCE checklist forms were identified from Pubmed, Embase, PsycINFO, and the ProQuest Education Databases up to 2013. Included were those studies that report empirical validity or reliability values for the communication skills assessment checklists used. Excluded were those papers that did not report reliability or validity. RESULTS: Papers focusing on generic communication skills, history taking, physician-patient communication, interviewing, negotiating treatment, information giving, empathy and 18 other domains (ICC -0.12-1) were identified. Regarding the validity and reliability of the communication skills checklists, agreement between reviewers was 0.45. CONCLUSIONS: Heterogeneity in the rubrics used in the assessment of communication skills and a lack of agreement between reviewers makes comparison of student competences within and across institutions difficult. PRACTICE IMPLICATIONS: Consideration should be afforded to the adoption of a standardized measurement instrument to assess communication skills in undergraduate medical education. Future research will focus upon evaluating the potential impact of adoption of a standardized measurement instrument.
Authors: Joachim Graf; Robert Smolka; Elisabeth Simoes; Stephan Zipfel; Florian Junne; Friederike Holderried; Annette Wosnik; Anne M Doherty; Karina Menzel; Anne Herrmann-Werner Journal: BMC Med Educ Date: 2017-05-02 Impact factor: 2.463
Authors: Zafra Cooper; Helen Doll; Suzanne Bailey-Straebler; Kristin Bohn; Dian de Vries; Rebecca Murphy; Marianne E O'Connor; Christopher G Fairburn Journal: JMIR Ment Health Date: 2017-10-31