Mark A Zaydman1, Jonathan R Brestoff2, Ronald Jackups2. 1. Department of Pathology and Immunology, Washington University School of Medicine, St Louis, MO, USA. Electronic address: zaydmanm@wustl.edu. 2. Department of Pathology and Immunology, Washington University School of Medicine, St Louis, MO, USA.
Abstract
OBJECTIVE: Clinicians order laboratory tests in an effort to reduce diagnostic or therapeutic uncertainty. Information theory provides the opportunity to quantify the degree to which a test result is expected to reduce diagnostic uncertainty. We sought to apply information theory toward the evaluation and optimization of a diagnostic test threshold and to determine if the results would differ from those of conventional methodologies. We used a heparin/PF4 immunoassay (PF4 ELISA) as a case study. MATERIALS AND METHODS: The laboratory database was queried for PF4 ELISA and serotonin release assay (SRA) results during the study period, with the latter serving as the gold standard for the disease heparin-induced thrombocytopenia (HIT). The optimized diagnostic threshold of the PF4 ELISA test was compared using conventional versus information theoretic approaches under idealized (pretest probability = 50%) and realistic (pretest probability = 2.4%) testing conditions. RESULTS: Under ideal testing conditions, both analyses yielded a similar optimized optical density (OD) threshold of OD > 0.79. Under realistic testing conditions, information theory suggested a higher threshold, OD > 1.5 versus OD > 0.6. Increasing the diagnostic threshold improved the global information value, the value of a positive test and the noise content with only a minute change in the negative test value. DISCUSSION: Our information theoretic approach suggested that the current FDA approved cutoff (OD > 0.4) is overly permissive leading to loss of test value and injection of noise into an already complex diagnostic dilemma. Because our approach is purely statistical and takes as input data that are readily accessible in the clinical laboratory it offers a scalable and data-driven strategy for optimizing test value that may be widely applicable in the domain of laboratory medicine. CONCLUSION: Information theory provides more meaningful measures of test value than the widely used accuracy-based metrics.
OBJECTIVE: Clinicians order laboratory tests in an effort to reduce diagnostic or therapeutic uncertainty. Information theory provides the opportunity to quantify the degree to which a test result is expected to reduce diagnostic uncertainty. We sought to apply information theory toward the evaluation and optimization of a diagnostic test threshold and to determine if the results would differ from those of conventional methodologies. We used a heparin/PF4 immunoassay (PF4 ELISA) as a case study. MATERIALS AND METHODS: The laboratory database was queried for PF4 ELISA and serotonin release assay (SRA) results during the study period, with the latter serving as the gold standard for the disease heparin-induced thrombocytopenia (HIT). The optimized diagnostic threshold of the PF4 ELISA test was compared using conventional versus information theoretic approaches under idealized (pretest probability = 50%) and realistic (pretest probability = 2.4%) testing conditions. RESULTS: Under ideal testing conditions, both analyses yielded a similar optimized optical density (OD) threshold of OD > 0.79. Under realistic testing conditions, information theory suggested a higher threshold, OD > 1.5 versus OD > 0.6. Increasing the diagnostic threshold improved the global information value, the value of a positive test and the noise content with only a minute change in the negative test value. DISCUSSION: Our information theoretic approach suggested that the current FDA approved cutoff (OD > 0.4) is overly permissive leading to loss of test value and injection of noise into an already complex diagnostic dilemma. Because our approach is purely statistical and takes as input data that are readily accessible in the clinical laboratory it offers a scalable and data-driven strategy for optimizing test value that may be widely applicable in the domain of laboratory medicine. CONCLUSION: Information theory provides more meaningful measures of test value than the widely used accuracy-based metrics.