OBJECTIVE: In recent decades, there has been increased interest in tools for assessing and improving the communication skills of general practice trainees. Recently, experts in the field rated the older Maas Global (MG) and the newer Common Ground (CG) instruments among the better communication skills assessment tools. This report seeks to establish their cross-validity. METHODS: Eighty trainees were observed by 2 raters for each instrument in 2 standardised patient stations from the final year objective structured clinical examination for Belgian trainee general practitioners. Each instrument was assigned 6 raters. RESULTS: Trainees showed the lowest mean scores for evaluating the consultation (MG7), summarising (MG11), addressing emotions (MG9) and addressing feelings (CG5). Inter-rater kappa statistics revealed fair-to-moderate agreement for the MG and slight-to-fair agreement for the CG. Cronbach's alpha was 0.78 for the MG and 0.89 for the CG. A generalisability study was only feasible for the MG: it was more helpful to increase the number of cases than the number of raters. Agreement between the instruments was examined using kappa statistics, Bland-Altman plots and multi-level analysis. Ranking the trainees for each instrument revealed similar results for the least competent trainees. Variances between and within trainees differed between instruments, whereas case specificity was comparable. Multi-level analysis also revealed a rater-item interaction effect. CONCLUSIONS: The 2 instruments have convergent validity, but the drawbacks of the CG, which has fewer items to be scored, include lower inter-rater reliability and score variance within trainees.
OBJECTIVE: In recent decades, there has been increased interest in tools for assessing and improving the communication skills of general practice trainees. Recently, experts in the field rated the older Maas Global (MG) and the newer Common Ground (CG) instruments among the better communication skills assessment tools. This report seeks to establish their cross-validity. METHODS: Eighty trainees were observed by 2 raters for each instrument in 2 standardised patient stations from the final year objective structured clinical examination for Belgian trainee general practitioners. Each instrument was assigned 6 raters. RESULTS: Trainees showed the lowest mean scores for evaluating the consultation (MG7), summarising (MG11), addressing emotions (MG9) and addressing feelings (CG5). Inter-rater kappa statistics revealed fair-to-moderate agreement for the MG and slight-to-fair agreement for the CG. Cronbach's alpha was 0.78 for the MG and 0.89 for the CG. A generalisability study was only feasible for the MG: it was more helpful to increase the number of cases than the number of raters. Agreement between the instruments was examined using kappa statistics, Bland-Altman plots and multi-level analysis. Ranking the trainees for each instrument revealed similar results for the least competent trainees. Variances between and within trainees differed between instruments, whereas case specificity was comparable. Multi-level analysis also revealed a rater-item interaction effect. CONCLUSIONS: The 2 instruments have convergent validity, but the drawbacks of the CG, which has fewer items to be scored, include lower inter-rater reliability and score variance within trainees.
Authors: Jan Matthys; Glyn Elwyn; Marc Van Nuland; Georges Van Maele; An De Sutter; Marc De Meyere; Myriam Deveugele Journal: Br J Gen Pract Date: 2009-01 Impact factor: 5.386
Authors: Brett Vaughan; Vivienne Sullivan; Cameron Gosling; Patrick McLaughlin; Gary Fryer; Margaret Wolff; Roger Gabb Journal: BMC Med Educ Date: 2012-09-29 Impact factor: 2.463
Authors: R van der Gulden; S Heeneman; A W M Kramer; R F J M Laan; N D Scherpbier-de Haan; B P A Thoonen Journal: BMC Med Educ Date: 2020-06-26 Impact factor: 2.463