W P Oosterhuis1, H J Ulenkate, H M Goldschmidt. 1. Department of Clinical Chemistry and Hematology, St. Elisabeth Hospital, Hilvarenbeekseweg 60, 5022 GC Tilburg, The Netherlands. oosthuis@knmg.nl
Abstract
BACKGROUND: Manual validation of laboratory test results is time-consuming, creating a demand for expert systems to automate this process. We have started to set up the program "LabRespond", which covers five validation levels: administrative, technical, sample, patient, and clinical validation. We present the evaluation of a prototype of an automated patient validation system based on statistical methods, in contrast to the commercially available program "VALAB", a rule-based automated validation system. METHODS: In the present study, 163 willfully altered, erroneous test results out of 5421 were submitted for validation to LabRespond, VALAB, and to a group of clinical chemists (n = 9) who validated these test results manually. The test results rejected by three or more clinical chemists (n = 281) served as a secondary reference standard. RESULTS: The error recovery rates of clinical chemists ranged from 23.9% to 71.2%. The recovery rates of LabRespond and VALAB were 77.9% and 71.8%, respectively (difference not significant). The false-positive rates were 82.7% for LabRespond, 83.6% for VALAB, and 27.8-86.7% for clinical chemists. Using the consensus of three or more clinical chemists as the secondary reference standard, we found error recovery rates of 64.8% for LabRespond and 72.2% for VALAB (P = 0.06). Compared with VALAB, LabRespond detected more (P = 0.003) erroneous test results of the type that were changed from abnormal to normal. CONCLUSIONS: The statistical plausibility check used by LabRespond offers a promising automated validation method with a higher error recovery rate than the clinical chemists participating in this study, and a performance comparable to VALAB.
BACKGROUND: Manual validation of laboratory test results is time-consuming, creating a demand for expert systems to automate this process. We have started to set up the program "LabRespond", which covers five validation levels: administrative, technical, sample, patient, and clinical validation. We present the evaluation of a prototype of an automated patient validation system based on statistical methods, in contrast to the commercially available program "VALAB", a rule-based automated validation system. METHODS: In the present study, 163 willfully altered, erroneous test results out of 5421 were submitted for validation to LabRespond, VALAB, and to a group of clinical chemists (n = 9) who validated these test results manually. The test results rejected by three or more clinical chemists (n = 281) served as a secondary reference standard. RESULTS: The error recovery rates of clinical chemists ranged from 23.9% to 71.2%. The recovery rates of LabRespond and VALAB were 77.9% and 71.8%, respectively (difference not significant). The false-positive rates were 82.7% for LabRespond, 83.6% for VALAB, and 27.8-86.7% for clinical chemists. Using the consensus of three or more clinical chemists as the secondary reference standard, we found error recovery rates of 64.8% for LabRespond and 72.2% for VALAB (P = 0.06). Compared with VALAB, LabRespond detected more (P = 0.003) erroneous test results of the type that were changed from abnormal to normal. CONCLUSIONS: The statistical plausibility check used by LabRespond offers a promising automated validation method with a higher error recovery rate than the clinical chemists participating in this study, and a performance comparable to VALAB.
Authors: Cornelia Mrazek; Giuseppe Lippi; Martin H Keppel; Thomas K Felder; Hannes Oberkofler; Elisabeth Haschke-Becher; Janne Cadamuro Journal: Biochem Med (Zagreb) Date: 2020-06-15 Impact factor: 2.313