CONTEXT: Colleague surveys are important sources of information on a doctor's professional performance in UK revalidation plans. Colleague surveys are analysed by deriving quantitative measures from rating scales. As free-text comments are also recorded, we explored the utility of a mixed-methods approach to their analysis. METHODS: A volunteer sample of practising UK doctors (from acute, primary and other care settings) undertook a General Medical Council (GMC) colleague survey. Up to 20 colleagues per doctor completed an online Colleague Questionnaire (CQ), which included 18 performance evaluation items and an optional comment box. The polarity of each comment was noted and a qualitative content analysis undertaken. Emerging themes were mapped onto existing items to identify areas not previously captured. We then quantitatively analysed the associations between the polarity of comments (positive/adverse) and their related item scale scores. RESULTS: A total of 1636 of 4269 (38.3%) colleagues recorded free-text comments (median = 14 per doctor) and most were unequivocally positive; only 127 of 1636 (7.8%) recorded negative statements and these were clustered on a subset comprising 80 of 302 (26.5%) doctors. Doctors' overall mean CQ performance scores were significantly correlated with the numbers of colleagues recording positive (r = 0.35; P < 0.0001) and adverse (r = - 0.40; P = 0.0003) comments. In total, 1224 of 1636 (74.8%) comments included statements that mapped on CQ items, and statistically significant associations (P < 0.05) were observed for 14 of 15 items. Five global themes (innovativeness, interpersonal skills, popularity, professionalism, respect) were identified in 904 of 1636 (73.9%) comments. CONCLUSIONS: There is an inevitable trade-off between the capturing of indicators of problematic performance (i.e. adverse statements which contradict a positive scale rating) and the ease with which such statements can be identified. Our data suggest there is little benefit in routinely analysing narrative comments for the purposes of revalidation.
CONTEXT: Colleague surveys are important sources of information on a doctor's professional performance in UK revalidation plans. Colleague surveys are analysed by deriving quantitative measures from rating scales. As free-text comments are also recorded, we explored the utility of a mixed-methods approach to their analysis. METHODS: A volunteer sample of practising UK doctors (from acute, primary and other care settings) undertook a General Medical Council (GMC) colleague survey. Up to 20 colleagues per doctor completed an online Colleague Questionnaire (CQ), which included 18 performance evaluation items and an optional comment box. The polarity of each comment was noted and a qualitative content analysis undertaken. Emerging themes were mapped onto existing items to identify areas not previously captured. We then quantitatively analysed the associations between the polarity of comments (positive/adverse) and their related item scale scores. RESULTS: A total of 1636 of 4269 (38.3%) colleagues recorded free-text comments (median = 14 per doctor) and most were unequivocally positive; only 127 of 1636 (7.8%) recorded negative statements and these were clustered on a subset comprising 80 of 302 (26.5%) doctors. Doctors' overall mean CQ performance scores were significantly correlated with the numbers of colleagues recording positive (r = 0.35; P < 0.0001) and adverse (r = - 0.40; P = 0.0003) comments. In total, 1224 of 1636 (74.8%) comments included statements that mapped on CQ items, and statistically significant associations (P < 0.05) were observed for 14 of 15 items. Five global themes (innovativeness, interpersonal skills, popularity, professionalism, respect) were identified in 904 of 1636 (73.9%) comments. CONCLUSIONS: There is an inevitable trade-off between the capturing of indicators of problematic performance (i.e. adverse statements which contradict a positive scale rating) and the ease with which such statements can be identified. Our data suggest there is little benefit in routinely analysing narrative comments for the purposes of revalidation.
Authors: Michael E Kalu; Sharon Switzer-Mclntrye; Martine Quesnel; Catherine Donnelly; Kathleen E Norman Journal: Physiother Can Date: 2021 Impact factor: 1.037
Authors: Inocencio Daniel Maramba; Antoinette Davey; Marc N Elliott; Martin Roberts; Martin Roland; Finlay Brown; Jenni Burt; Olga Boiko; John Campbell Journal: JMIR Med Inform Date: 2015-05-06
Authors: Ara Tekian; Yoon Soo Park; Sarette Tilton; Patrick F Prunty; Eric Abasolo; Fred Zar; David A Cook Journal: Acad Med Date: 2019-12 Impact factor: 6.893
Authors: Emma Teasdale; Ingrid Muller; Amirah Abdullah Sani; Kim S Thomas; Beth Stuart; Miriam Santer Journal: BMJ Open Date: 2018-01-11 Impact factor: 2.692
Authors: Anna Victoria Bowyer; Ilora Finlay; Jessica Baillie; Anthony Byrne; Jacqui McCarthy; Catherine Sampson; Veronica Snow; Annmarie Nelson Journal: BMJ Support Palliat Care Date: 2016-02-17 Impact factor: 3.568