Critique of the Guide to the expression of uncertainty in measurement method of estimating and reporting uncertainty in diagnostic assays.

BACKGROUND: The Guide to the Expression of Uncertainty in Measurement (GUM) provides instructions for constructing uncertainty intervals for a measurement. This method is usually reserved for reference materials, but GUM has been recently proposed as a way to express uncertainty for commercial diagnostic assays.
METHODS: Using the official GUM standard and published applications of GUM to commercial diagnostic assays, I undertook an analysis to evaluate whether applying GUM to commercial diagnostic assays is warranted.
RESULTS: Certain important assays, such as troponin I, would not be candidates for GUM because troponin I is not a well-defined physical quantity. Unlike definitive methods, in which efforts are taken to detect and eliminate all systematic error sources, commercial assays often trade off features such as ease of use and cost with accuracy and allow systematic errors to be present as long as the overall accuracy meets the medical need goal. Laboratories are hindered in preparing GUM models because the knowledge required to specify some systematic errors is often available only to manufacturers. Some non-GUM methods to estimate uncertainty rely on observed data, which include both known and unknown sources of error. The occurrence of large, unknown errors for assays in routine use (e.g., outliers) is not unusual because diagnostic assays must be chemically specific in the presence of thousands of potentially interfering substances. There is no provision in GUM to deal with unexplained outliers, which may lead to uncertainty intervals that are not wide enough. Some clinicians assume that diagnostic assay results have little uncertainty. This situation may be made worse by including an uncertainty interval, which implies certification.
CONCLUSIONS: Evaluations for accuracy (total analytical error) based on describing the distribution of result differences between commercial assays and reference methods indicate that some assays have a few results with large differences (e.g., outliers). This leads to a wide accuracy interval (total analytical error limits). It is unlikely that GUM would be able to predict these wide intervals, especially because there is little or no provision for outlier treatment in GUM. Presenting too narrow GUM uncertainty intervals to clinicians would be misleading. The modeling used by practitioners of the GUM method is potentially useful in improving quality, but commercial diagnostic assays are not ready for GUM uncertainty statements.