AIMS: To prove the safety and effectiveness of "Professor Belmonte", a knowledge-based system for bone marrow reporting, a formal evaluation of the reports generated by the system was performed. METHODS: Three haematologists (a consultant, a senior registrar, and a junior registrar), none of whom were involved in the development of the software, compared the unedited reports generated by Professor Belmonte with the original bone marrow reports in 785 unselected cases. Each haematologist independently graded the quality of Belmonte's reports using one of four categories: (a) better than the original report (more informative, containing useful information missing in the original report); (b) equivalent to the original report; (c) satisfactory, but missing information that should have been included; and (d) unsatisfactory. RESULTS: The consultant graded 64 reports as more informative than the original, 687 as equivalent to the original, 32 as satisfactory, and two as unsatisfactory. The senior registrar considered 29 reports to be better than the original, 739 to be equivalent to the original, 15 to be satisfactory, and two to be unsatisfactory. The junior registrar found that 88 reports were better than the original, 681 were equivalent to the original, 14 were satisfactory, and two were unsatisfactory. Each judge found two different reports to be unsatisfactory according to their criteria. All 785 reports generated by the computer system received at least two scores of satisfactory or better. CONCLUSIONS: In this representative study, Professor Belmonte generated bone marrow reports that proved to be as accurate as the original reports in a large university hospital. The haematology knowledge contained within the system, the reasoning process, and the function of the software are safe and effective for assisting haematologists in generating high quality bone marrow reports.
AIMS: To prove the safety and effectiveness of "Professor Belmonte", a knowledge-based system for bone marrow reporting, a formal evaluation of the reports generated by the system was performed. METHODS: Three haematologists (a consultant, a senior registrar, and a junior registrar), none of whom were involved in the development of the software, compared the unedited reports generated by Professor Belmonte with the original bone marrow reports in 785 unselected cases. Each haematologist independently graded the quality of Belmonte's reports using one of four categories: (a) better than the original report (more informative, containing useful information missing in the original report); (b) equivalent to the original report; (c) satisfactory, but missing information that should have been included; and (d) unsatisfactory. RESULTS: The consultant graded 64 reports as more informative than the original, 687 as equivalent to the original, 32 as satisfactory, and two as unsatisfactory. The senior registrar considered 29 reports to be better than the original, 739 to be equivalent to the original, 15 to be satisfactory, and two to be unsatisfactory. The junior registrar found that 88 reports were better than the original, 681 were equivalent to the original, 14 were satisfactory, and two were unsatisfactory. Each judge found two different reports to be unsatisfactory according to their criteria. All 785 reports generated by the computer system received at least two scores of satisfactory or better. CONCLUSIONS: In this representative study, Professor Belmonte generated bone marrow reports that proved to be as accurate as the original reports in a large university hospital. The haematology knowledge contained within the system, the reasoning process, and the function of the software are safe and effective for assisting haematologists in generating high quality bone marrow reports.