A test ordering system with automated reminders for primary care based on practice guidelines.

In this article we describe a real-time automated reminder system that has been developed to change Family Physicians' (FP) test ordering behavior. The system focuses on the appropriateness of test requests. We aim at using the system as a substitute for written feedback by human experts. The reminder system consists of a knowledge base, an order entry system and modules to provide passive and active support in the form of reminders to FPs. The system generates critical comments about the rationality of the test requests at the moment the FP orders a test that is not in line with national or regional guidelines. For the first validation of the knowledge base we compared the comments of a human expert to the comments of the reminder system on three random samples of test requests. The overall agreement in the subsequent validation rounds was 46, 60 and 69%. The corrections made in the knowledge base after each validation round resulted in a reminder system with 149 reminders concerning various medical problems. Due to the corrections in the knowledge base the reminder system reacts better over the subsequent validation rounds.