David A Cook1, Thomas J Beckman, Jayawant N Mandrekar, V Shane Pankratz. 1. Division of General Internal Medicine and Office of Education Research, Mayo Clinic College of Medicine, Baldwin 4-A, 200 First Street SW, Rochester, MN 55905, USA. cook.david33@mayo.edu
Abstract
UNLABELLED: The mini-CEX is widely used to rate directly observed resident-patient encounters. Although several studies have explored the reliability of mini-CEX scores, the dimensionality of mini-CEX scores is incompletely understood. OBJECTIVE: explore the dimensionality of mini-CEX scores through factor analysis and generalizability analysis. DESIGN: factor analytic and generalizability study using retrospective data. PARTICIPANTS: eighty five physician preceptors and 264 internal medicine residents (postgraduate years 1-3). METHODS: preceptors used the six-item mini-CEX to rate directly observed resident-patient encounters in internal medicine resident continuity clinics. We analyzed mini-CEX scores accrued over 4 years using repeated measures analysis of variance to generate a correlation matrix adjusted for multiple observations on individual residents, and then performed factor analysis on this adjusted correlation matrix. We also performed generalizability analyses. RESULTS: eighty-five preceptors rated 264 residents in 1,414 resident-patient encounters. Common factor analysis of these scores after adjustment for repeated measures revealed a single-factor solution. Cronbach's alpha for this single factor (i.e. all six mini-CEX items) was ≥ 0.86. Sensitivity analyses using principal components and other method variations revealed a similar factor structure. Generalizability studies revealed a reproducibility coefficient of 0.23 (0.70 for 10 raters or encounters). CONCLUSIONS: the mini-CEX appears to measure a single global dimension of clinical competence. If educators desire to measure discrete clinical skills, alternative assessment methods may be required. Our approach to factor analysis overcomes the limitation of repeated observations on subjects without discarding data, and may be useful to other researchers attempting factor analysis of datasets in which individuals contribute multiple observations.
UNLABELLED: The mini-CEX is widely used to rate directly observed resident-patient encounters. Although several studies have explored the reliability of mini-CEX scores, the dimensionality of mini-CEX scores is incompletely understood. OBJECTIVE: explore the dimensionality of mini-CEX scores through factor analysis and generalizability analysis. DESIGN: factor analytic and generalizability study using retrospective data. PARTICIPANTS: eighty five physician preceptors and 264 internal medicine residents (postgraduate years 1-3). METHODS: preceptors used the six-item mini-CEX to rate directly observed resident-patient encounters in internal medicine resident continuity clinics. We analyzed mini-CEX scores accrued over 4 years using repeated measures analysis of variance to generate a correlation matrix adjusted for multiple observations on individual residents, and then performed factor analysis on this adjusted correlation matrix. We also performed generalizability analyses. RESULTS: eighty-five preceptors rated 264 residents in 1,414 resident-patient encounters. Common factor analysis of these scores after adjustment for repeated measures revealed a single-factor solution. Cronbach's alpha for this single factor (i.e. all six mini-CEX items) was ≥ 0.86. Sensitivity analyses using principal components and other method variations revealed a similar factor structure. Generalizability studies revealed a reproducibility coefficient of 0.23 (0.70 for 10 raters or encounters). CONCLUSIONS: the mini-CEX appears to measure a single global dimension of clinical competence. If educators desire to measure discrete clinical skills, alternative assessment methods may be required. Our approach to factor analysis overcomes the limitation of repeated observations on subjects without discarding data, and may be useful to other researchers attempting factor analysis of datasets in which individuals contribute multiple observations.
Authors: Alberto Alves de Lima; Diego Conde; Juan Costabel; Juan Corso; Cees Van der Vleuten Journal: Adv Health Sci Educ Theory Pract Date: 2011-12-23 Impact factor: 3.853
Authors: M J B Govaerts; M W J Van de Wiel; L W T Schuwirth; C P M Van der Vleuten; A M M Muijtjens Journal: Adv Health Sci Educ Theory Pract Date: 2012-05-17 Impact factor: 3.853
Authors: Laury P J W M de Jonge; Angelique A Timmerman; Marjan J B Govaerts; Jean W M Muris; Arno M M Muijtjens; Anneke W M Kramer; Cees P M van der Vleuten Journal: Adv Health Sci Educ Theory Pract Date: 2017-02-02 Impact factor: 3.853