Further experience in development of an objective structured clinical examination for physical medicine and rehabilitation residents.

Since the last report, two additional objective structured clinical examinations (PGY-2, PGY-3, and PGY-4, as well as incoming PGY-2) have been administered. As a result, our curriculum has been modified to strengthen physical examination skills, including specific workshops. Interrater reliability of evaluators has been tested for the first time to verify reliability, and refinements have been made in the standardized checklist grading system. The interrater grading of history-taking had good reliability (0.73-0.96), as did neurological and spine physical examination (0.84-0.88). The interrater grading reliability of small and large joint examination was more problematic (0.46-0.62) because of examiners' inability to have full visibility, evaluator's fatigue, and confusing evaluation scoring descriptions. We now use a two-point grading scale (correct or incorrect) for history but continue a three-point scale (correct, partially correct, or incorrect) for physical examination. The examination schedule is being modified to add more encounters, give time for trainee feedback, and further refinement of grading expectations for a more efficient and reliable scoring system.