Shortcomings in the evaluation of students' clinical skills and behaviors in medical school.

The authors review the methods by which U.S. medical schools have evaluated student achievement during the twentieth century, especially for the assessment of noncognitive abilities, including clinical skills and behaviors. With particular reference to the current decade, information collected by the Liaison Committee on Medical Education (LCME) is used to examine the congruence of assessment methods with the rising tide of understanding--and accreditation requirements--that knowledge, competence, and behavioral objectives require different methods of assessment to measure the extent of students' learning in each domain. Amongst 97 medical schools having accreditation surveys between July 1993 and June 1998, only 186 of 751 basic science courses tested students' noncognitive achievements in things such as the preparation for and participation in small-group conferences, the quality of case-based discussion, library research and literature reviews, and research projects, despite staking out scholarship, habits of life-long learning, and reasoned thinking as educational objectives. In the clerkships of these schools, structured and observed assessments of clinical skills--with standardized patients and/or OSCEs--contributed 7.4-23.1% to a student's grade (depending on the clerkship discipline), while the predominant contribution (50-70% across the clerkships) was made by resident and faculty ratings that were based largely on recollections of case presentations and discussions having little relationship to interpersonal skills, rapport with patients, and logical and sequenced history taking and physical examination. On a more optimistic note, the results show that the number of schools using standardized patients in one or more clerkships increased between 1993 and 1998 from 34.1% to 50.4% of the 125 schools in the United States, and the number of schools using standardized patients in comprehensive fourth-year examinations increased from 19.1% to 48% of the total. Despite such progress, this study shows that too many medical schools still fail to employ evaluation methods that specifically assess students' achievement of the skills and behaviors they need to learn to practice medicine. The findings of this article explain why accreditors are paying closer attention to how well schools provide measured assurances that students learn what the faculties set out to teach.