A process to maintain the quality of a computerized knowledge base.

As part of a project to develop knowledge-based reminders for the outpatient setting, we developed a process to help maintain the quality of the knowledge base. The knowledge engineering process involved many parties, including several domain experts, a knowledge engineer, and a programmer and a process was necessary to assure that information transfer among individuals did not become confused. An MS Access database was created to store, among other data, textual versions of the rules as they evolved over time. In a 9-month period 36 rules were entered into the database. Of those, 17 are still active in their original form. The remaining 19 underwent various types of modifications; these changes were tracked in the database. Processes and tools to maintain knowledge bases are necessary if the benefits of clinical decision support systems are to be realized and investments in knowledge engineering are to be protected.