The DELPHI method as a consensus and knowledge acquisition tool for the evaluation of the DIABETES system for insulin administration.

DIABETES is a decision-support system in the field of insulin administration. System performance evaluation is particularly difficult because of the absence of a uniform decision-making model followed by the specialists. The DELPHI method has been selected since it is appropriate for those domains where there is divergence among experts' opinions. The DELPHI approach helps a number of diabetologists arrive at a consensus and thus it facilitates performance evaluation and further knowledge acquisition. Insulin administration regimes, for 100 diabetic subjects, were proposed by DIABETES and five diabetologists (round 1). These suggestions were compiled and forwarded back to the specialists who proposed a second management approach (round 2). In each case, the experts were asked to justify their decision and comment on the suggestions of their colleagues and DIABETES. A novel scoring system for quantification of agreement was adopted. The DELPHI procedure significantly increased the agreement among the diabetologists from 67% to 84% (X2, p = 0.0001). The agreement between experts' and DIABETES recommendations was to a level of 54%. A total of 3500 comments were acquired by the experts.