OBJECTIVE: To determine which aspects of outpatient attending physician performance (e.g., clinical ability, teaching ability, interpersonal conduct) were measurable and separable by resident report. DESIGN: Self-administered evaluation form. SETTING: University internal medicine resident continuity clinic. PARTICIPANTS: All residents with their continuity clinic at the university hospital evaluated the two attendings who staffed their clinic for the academic years of 1990-1991, 1991-1992, and 1992-1993 (average of 85 total residents per year). The overall response rate was 74%. ANALYSIS: Exploratory analyses were conducted on a preliminary evaluation form in the first two years of the study (236 evaluations of 20 different clinic attendings) and confirmatory analyses using factor analysis and generalizability analysis were performed on the third year's data (142 evaluations of 15 different clinic attendings). Analysis of variance was used to evaluate factors associated with evaluation scores. RESULTS: Analyses demonstrated that the residents did not distinguish between the attendings' clinical and teaching abilities, resulting in a single four-item scale that was named the Clinical/Teaching Excellence Scale, measured on a five-point scale from poor to outstanding (Cronbach's alpha = 0.92). A large amount of the variance for this scale score was associated with attending identity (adjusted R2 = 46%). However, two alternative approaches to evaluating the performance of the attending (preference for him or her to the "average" attending and perceived impact of the attending on residents' clinical skills) did not provide useful information independent of the Clinical/Teaching Excellence Scale. The ratings of three separate conduct scales [availability in clinic (Availability Scale), treating residents and patients with respect (Respect Scale), and time efficiency in staffing cases (Slow Staffing Scale)] were separable from each other and from the rating of clinical/teaching excellence. For the Clinical/Teaching Excellent Scale, as few as four evaluations produced good interrater reliability and eight evaluations produced excellent reliability (reliability coefficients were 0.70 and 0.84, respectively). CONCLUSIONS: Although this evaluation instrument for measuring clinic attending performance must be considered preliminary, this study suggests that relatively few attending evaluations are required to reliably profile an individual attending's performance, that attending identity is associated with a large amount of the scale score variation, and that special issues of attending performance more relevant to the outpatient setting than the inpatient setting (availability in clinic and sensitivity to time efficiency) should be considered when evaluating clinic attending performance.
OBJECTIVE: To determine which aspects of outpatient attending physician performance (e.g., clinical ability, teaching ability, interpersonal conduct) were measurable and separable by resident report. DESIGN: Self-administered evaluation form. SETTING: University internal medicine resident continuity clinic. PARTICIPANTS: All residents with their continuity clinic at the university hospital evaluated the two attendings who staffed their clinic for the academic years of 1990-1991, 1991-1992, and 1992-1993 (average of 85 total residents per year). The overall response rate was 74%. ANALYSIS: Exploratory analyses were conducted on a preliminary evaluation form in the first two years of the study (236 evaluations of 20 different clinic attendings) and confirmatory analyses using factor analysis and generalizability analysis were performed on the third year's data (142 evaluations of 15 different clinic attendings). Analysis of variance was used to evaluate factors associated with evaluation scores. RESULTS: Analyses demonstrated that the residents did not distinguish between the attendings' clinical and teaching abilities, resulting in a single four-item scale that was named the Clinical/Teaching Excellence Scale, measured on a five-point scale from poor to outstanding (Cronbach's alpha = 0.92). A large amount of the variance for this scale score was associated with attending identity (adjusted R2 = 46%). However, two alternative approaches to evaluating the performance of the attending (preference for him or her to the "average" attending and perceived impact of the attending on residents' clinical skills) did not provide useful information independent of the Clinical/Teaching Excellence Scale. The ratings of three separate conduct scales [availability in clinic (Availability Scale), treating residents and patients with respect (Respect Scale), and time efficiency in staffing cases (Slow Staffing Scale)] were separable from each other and from the rating of clinical/teaching excellence. For the Clinical/Teaching Excellent Scale, as few as four evaluations produced good interrater reliability and eight evaluations produced excellent reliability (reliability coefficients were 0.70 and 0.84, respectively). CONCLUSIONS: Although this evaluation instrument for measuring clinic attending performance must be considered preliminary, this study suggests that relatively few attending evaluations are required to reliably profile an individual attending's performance, that attending identity is associated with a large amount of the scale score variation, and that special issues of attending performance more relevant to the outpatient setting than the inpatient setting (availability in clinic and sensitivity to time efficiency) should be considered when evaluating clinic attending performance.
Authors: Thomas J Beckman; Amit K Ghosh; David A Cook; Patricia J Erwin; Jayawant N Mandrekar Journal: J Gen Intern Med Date: 2004-09 Impact factor: 5.128