M H Malloy1, L Perkowski, M Callaway, A Speer. 1. Department of Pediatrics, University of Texas Medical Branch, Galveston 77555-0526, USA. mmalloy@utmb.edu
Abstract
BACKGROUND: We designed 2 pediatric objective structured clinical examination stations, 1 anemia case associated with lead exposure and 1 failure-to-gain-weight case associated with extended breast-feeding, to evaluate third-year medical students who had studied in pediatric community preceptors' offices as part of a 12-week multidisciplinary ambulatory clerkship rotation. OBJECTIVE: To examine the relationship between preceptor expectations and student performance on these 2 objective structured clinical examination stations. METHODS: To elicit community preceptors' expectations of student performance, we constructed a 46-item survey replicating checklists filled out by simulated patients evaluating student performance on the objective structured clinical examination stations. The percentage agreement among preceptors for each checklist item as well as the percentage agreement between preceptor responses and student responses on each checklist item were calculated. A summary score of preceptor responses across all checklist items and a summary score for student responses across all checklist items on each station were calculated. The correlation coefficients between preceptor and student summary scores were then examined. RESULTS: Fifty-nine preceptor surveys were mailed and 38 were returned (64% response rate). Data were usable from 37 surveys. Eighty-nine percent (33 of 37)of the preceptors agreed that a third-year clerkship student should have the knowledge to care for the patient with anemia and 92% (34 of 37)of the preceptors agreed similarly for the growth-delay case. Agreement among preceptors on individual checklist items varied widely for both cases. Fifty-seven students studied at the anemia station and 34 students studied at the growth-delay station. The mean+/-SD agreement across the 26 items on the anemia case between preceptor responses and student responses was 62%+/-23% and, for the 21 items on the growth-delay case, 60%+/-17%. The mean+/-SD preceptor summary score for the anemia case was 17.4+/-3.8 (maximum, 26) and 16.0+/-3.6 (maximum, 21) for the growth-delay case. The mean student score on the anemia case was 15.5+/-3.7 (maximum, 26) and, for the growth-delay case, 10.0+/-4.5 (maximum, 21). The Pearson correlation coefficient between the preceptor and student scores on the anemia case was 0.19 (P=.15), and for the growth-delay case,-0.41 (P=.06). CONCLUSIONS: These data suggest community preceptors agree on topic areas in which students should be clinically competent. There was, however, considerable variation in agreement among preceptors about what preceptors believe students should be able to do and how the students actually perform. The overall percentage agreement between preceptor expectations and student performance appears to be no better than chance.
BACKGROUND: We designed 2 pediatric objective structured clinical examination stations, 1 anemia case associated with lead exposure and 1 failure-to-gain-weight case associated with extended breast-feeding, to evaluate third-year medical students who had studied in pediatric community preceptors' offices as part of a 12-week multidisciplinary ambulatory clerkship rotation. OBJECTIVE: To examine the relationship between preceptor expectations and student performance on these 2 objective structured clinical examination stations. METHODS: To elicit community preceptors' expectations of student performance, we constructed a 46-item survey replicating checklists filled out by simulated patients evaluating student performance on the objective structured clinical examination stations. The percentage agreement among preceptors for each checklist item as well as the percentage agreement between preceptor responses and student responses on each checklist item were calculated. A summary score of preceptor responses across all checklist items and a summary score for student responses across all checklist items on each station were calculated. The correlation coefficients between preceptor and student summary scores were then examined. RESULTS: Fifty-nine preceptor surveys were mailed and 38 were returned (64% response rate). Data were usable from 37 surveys. Eighty-nine percent (33 of 37)of the preceptors agreed that a third-year clerkship student should have the knowledge to care for the patient with anemia and 92% (34 of 37)of the preceptors agreed similarly for the growth-delay case. Agreement among preceptors on individual checklist items varied widely for both cases. Fifty-seven students studied at the anemia station and 34 students studied at the growth-delay station. The mean+/-SD agreement across the 26 items on the anemia case between preceptor responses and student responses was 62%+/-23% and, for the 21 items on the growth-delay case, 60%+/-17%. The mean+/-SD preceptor summary score for the anemia case was 17.4+/-3.8 (maximum, 26) and 16.0+/-3.6 (maximum, 21) for the growth-delay case. The mean student score on the anemia case was 15.5+/-3.7 (maximum, 26) and, for the growth-delay case, 10.0+/-4.5 (maximum, 21). The Pearson correlation coefficient between the preceptor and student scores on the anemia case was 0.19 (P=.15), and for the growth-delay case,-0.41 (P=.06). CONCLUSIONS: These data suggest community preceptors agree on topic areas in which students should be clinically competent. There was, however, considerable variation in agreement among preceptors about what preceptors believe students should be able to do and how the students actually perform. The overall percentage agreement between preceptor expectations and student performance appears to be no better than chance.