CONTEXT: The effectiveness of multi-source feedback (MSF) tools, which are increasingly important in medical careers, will be influenced by their users' attitudes. This study compared perceptions of two tools for giving MSF to UK junior doctors, of which one provides mainly textual feedback and one provides mainly numerical feedback. We then compared the perceptions of three groups, including: trainees; raters giving feedback, and supervisors delivering feedback. METHODS: Postal questionnaires about the usability, usefulness and validity of a feedback system were distributed to trainees, raters and supervisors across the north of England. RESULTS: Questionnaire responses were analysed to compare opinions of the two tools and among the different user groups. Overall there were few differences. Attitudes towards MSF in principle were positive and the tools were felt to be usable, but there was little agreement that they could effectively identify doctors in difficulty or provide developmental feedback. The text-oriented tool was rated as more useful for giving feedback on communication and attitude, and as more useful for identifying a doctor in difficulty. Raters were more positive than other users about the usefulness of numerical feedback, but, overall, text was felt to be more useful. Some trainees expressed concern that feedback was based on insufficient knowledge of their work. This was not supported by raters' responses, although many did use indirect information. Trainees selected raters mainly for the perceived value of their feedback, but also based on personal relationships and the simple pragmatics of getting a tool completed. DISCUSSION: Despite positive attitudes to MSF, the perceived effectiveness of the tools was low. There are small but significant preferences for textual feedback, although raters may prefer numerical scales. Concerns about validity imply that greater awareness of contextual and psychological influences on feedback generation is necessary to allow the formative benefits of MSF to be optimised and to negate the risk of misuse in high-stakes contexts.
CONTEXT: The effectiveness of multi-source feedback (MSF) tools, which are increasingly important in medical careers, will be influenced by their users' attitudes. This study compared perceptions of two tools for giving MSF to UK junior doctors, of which one provides mainly textual feedback and one provides mainly numerical feedback. We then compared the perceptions of three groups, including: trainees; raters giving feedback, and supervisors delivering feedback. METHODS: Postal questionnaires about the usability, usefulness and validity of a feedback system were distributed to trainees, raters and supervisors across the north of England. RESULTS: Questionnaire responses were analysed to compare opinions of the two tools and among the different user groups. Overall there were few differences. Attitudes towards MSF in principle were positive and the tools were felt to be usable, but there was little agreement that they could effectively identify doctors in difficulty or provide developmental feedback. The text-oriented tool was rated as more useful for giving feedback on communication and attitude, and as more useful for identifying a doctor in difficulty. Raters were more positive than other users about the usefulness of numerical feedback, but, overall, text was felt to be more useful. Some trainees expressed concern that feedback was based on insufficient knowledge of their work. This was not supported by raters' responses, although many did use indirect information. Trainees selected raters mainly for the perceived value of their feedback, but also based on personal relationships and the simple pragmatics of getting a tool completed. DISCUSSION: Despite positive attitudes to MSF, the perceived effectiveness of the tools was low. There are small but significant preferences for textual feedback, although raters may prefer numerical scales. Concerns about validity imply that greater awareness of contextual and psychological influences on feedback generation is necessary to allow the formative benefits of MSF to be optimised and to negate the risk of misuse in high-stakes contexts.
Authors: Habiba Saedon; Shizalia Salleh; Arun Balakrishnan; Christopher H E Imray; Mahmud Saedon Journal: BMC Med Educ Date: 2012-05-02 Impact factor: 2.463