Jocelyn Lockyer1. 1. Departmentof Community Health Sciences, Faculty of Medicine, University of Calgary, Canada T2N 4Z6. lockyer@ucalgary.ca
Abstract
INTRODUCTION: High-quality instruments are required to assess and provide feedback to practicing physicians. Multisource feedback (MSF) uses questionnaires from colleagues, coworkers, and patients to provide data. It enables feedback in areas of increasing interest to the medical profession: communication, collaboration, professionalism, and interpersonal skills. The purpose of the study was to apply the 7 assessment criteria as a framework to examine the quality of MSF instruments used to assess practicing physicians. METHODS: The criteria for assessment (validity, reproducibility, equivalence, feasibility, educational effect, catalytic effect, and acceptability) were examined for 3 sets of instruments, drawing on published data. RESULTS: Three MSF instruments with a sufficient body of research for inclusion-the Canadian Physician Achievement Review instruments and the United Kingdom's GMC and CFEP360 instruments-were examined. There was evidence that MSF has been assessed against all criteria except educational effects, although variably for some of the instruments. The greatest emphasis was on validity, reproducibility, and feasibility for all of the instruments. Assessments of the catalytic effect were not available for 1 of the 2 UK instruments and minimally examined for the other. Data about acceptability are implicit in the UK instruments from their endorsement by the Royal College of General Practice and explicitly examined in the Canadian instruments. DISCUSSION: The 7 criteria provided a useful framework to assess the quality of MSF instruments and enable an approach to analyzing gaps in instrument assessment. These criteria are likely to be helpful in assessing other instruments used in medical education.
INTRODUCTION: High-quality instruments are required to assess and provide feedback to practicing physicians. Multisource feedback (MSF) uses questionnaires from colleagues, coworkers, and patients to provide data. It enables feedback in areas of increasing interest to the medical profession: communication, collaboration, professionalism, and interpersonal skills. The purpose of the study was to apply the 7 assessment criteria as a framework to examine the quality of MSF instruments used to assess practicing physicians. METHODS: The criteria for assessment (validity, reproducibility, equivalence, feasibility, educational effect, catalytic effect, and acceptability) were examined for 3 sets of instruments, drawing on published data. RESULTS: Three MSF instruments with a sufficient body of research for inclusion-the Canadian Physician Achievement Review instruments and the United Kingdom's GMC and CFEP360 instruments-were examined. There was evidence that MSF has been assessed against all criteria except educational effects, although variably for some of the instruments. The greatest emphasis was on validity, reproducibility, and feasibility for all of the instruments. Assessments of the catalytic effect were not available for 1 of the 2 UK instruments and minimally examined for the other. Data about acceptability are implicit in the UK instruments from their endorsement by the Royal College of General Practice and explicitly examined in the Canadian instruments. DISCUSSION: The 7 criteria provided a useful framework to assess the quality of MSF instruments and enable an approach to analyzing gaps in instrument assessment. These criteria are likely to be helpful in assessing other instruments used in medical education.
Authors: Jenni Burt; Gary Abel; Marc N Elliott; Natasha Elmore; Jennifer Newbould; Antoinette Davey; Nadia Llanwarne; Inocencio Maramba; Charlotte Paddison; John Campbell; Martin Roland Journal: Ann Fam Med Date: 2018-07 Impact factor: 5.166
Authors: Lonneke Alofs; Jorike Huiskes; Maas Jan Heineman; Caroline Buis; Manon Horsman; Lars van der Plank; Olle Ten Cate Journal: Perspect Med Educ Date: 2015-04