Performance on PMPs and performance in practice: are they related?

A study was conducted to test the criterion validity of a set of patient management problems (PMPs) through a direct behavior-by-behavior comparison of performance on the PMPs and performance in the practice setting. The PMP cases were replicated in the practice setting through the use of role-playing actors. At the time of the encounter the subjects in the study believed these actors to be real patients. Between PMP performance and practice performance there were major inconsistencies, characterized by substantially greater errors of omission and commission in the practice setting.