Managing the new mandate in resident education: a blueprint for translating a national mandate into local compliance.

OBJECTIVE: The Accreditation Council for Graduate Medical Education (ACGME) has mandated that all residency programs implement an assessment process of 6 core competencies. Assessment of surgical competence is also included in the mandate. We describe our local efforts to meet this new mandate.
DESIGN: Systematic literature review.
METHODS: A systematic MEDLINE search (1996-2003) of the literature on residency assessment tools was performed. All relevant titles were reviewed by a content expert, abstracts were selected, and all appropriate full articles were reviewed. The Department of Ophthalmology at the University of Iowa formalized the competency review process by forming an ad hoc departmental task force for "Meeting the Competencies" composed of clinicians, technical staff, education specialists, the program director, the director of residency curriculum, the medical student director, and residents.
RESULTS: The task force reviewed the available literature, reviewed potential best practices, and reached consensus on an implementation plan. The following specific criteria for the assessment process were proposed: (1) there should be multiple assessments by multiple observers using multiple tools at multiple time points, (2) the tools should be reliable, reproducible, and valid; (3) the tools must be practical (i.e., feasible, convenient, low time commitment, easy to use, and inexpensive to implement and maintain); (4) the tools must produce qualitative and quantitative data, with direct linkage to improvement in educational outcomes in the future; (5) the assessment process must be linked to explicit and public learning objectives; and (6) the grading scale should be open and clearly defined, and the process should be judged as fair and accurate by both faculty and residents. The Meeting the Competencies task force reviewed all of the available tools from the literature and recommended a pilot implementation matrix matching specific tools to individual competencies. The 6 pilot tools include (1) written and oral examinations, (2) a 360 degrees global evaluation form (using multiple observers from different perspectives, including nurses, technicians, fellow residents, and patients, to provide a wider assessment), (3) a resident portfolio, (4) direct observation of operative performance and clinical examination, (5) a phone encounter tool, and (6) a journal club tool.
CONCLUSION: We propose a potential blueprint for meeting the challenge of assessing the new ACGME competencies in ophthalmology and translating the national mandate into local compliance.