Barry Nix1, Dave Wright, Amy Baker. 1. Medical School, Heath Campus, Cardiff University, CF14 4XN, UK. nixb@cardiff.ac.uk
Abstract
UNLABELLED: First and second trimester screening protocols for Down syndrome rely on marker values being referred to smoothed median values to produce adjusted multiple of the median (MoM) values to standardise for factors such as assay, gestation, maternal weight, smoking status, and so on. Changes in assay components, such as reagent lot, and inappropriate use of published regression equations for smoothed medians have resulted in biases in reported MoM values that in many applications remain uncorrected. This paper investigates the impact of these biases on patient-specific risk estimates and screening performance, and concludes that a 10% bias for an individual marker can result in an increase of between 1 and 2% in the false positive rate of the programme. A simple formula is also derived that enables the impact of these biases to be determined without the need for simulation, thus making it easier to design effective statistical quality control procedures to monitor the output of screening software algorithms. OBJECTIVE: To determine the impact of bias in MoM values on detection rates, false positive rates and patient-specific risks for Down syndrome. METHODS: We show that bias in MoM values affects risk through a multiplicative factor, and present an approximation to estimate this factor. We then show how bias in MoM values changes the effective risk threshold in the screening test, and hence the test's performance characteristics are determined by reference to a different point on the ROC curve for that test. Our approximation is based on the assumption of equal variance covariance structure for the unaffected and T21 log MoM values. We demonstrate, using computer simulation and supportive theoretical results, that the approximation is reliable in situations encountered in practice. Applications of the approximation are also discussed in respect of establishing effective quality control rules for median MoMs. RESULTS: Substantial changes in patient risk estimates and overall screening performance can result from the sort of biases in marker MoM values encountered in routine practice. In particular, biases of 10% in individual median marker MoM values can produce a four-fold range of risks when using the triple test. A 10% bias in a single marker will change the false positive rates by up to 2%. The effects on the false positive rate are approximately additive and, in cases where all markers are biased towards Down syndrome, biases in all three markers for the triple test can more than double the false positive rate. CONCLUSIONS: Biases in marker MoM values can occur in many ways, inappropriate median values, kit lot change, drift in assay performance and operator effects. We present methods which allow the impact of these changes to be assessed in relation to patient-specific risks and the overall screening performance. This, in turn, will enable appropriate quality control procedures to be established to control the magnitude of reported marker MoM biases, or equivalently, the magnitude of biases associated with the calculation of patient-specific risks. 2007 John Wiley & Sons, Ltd
UNLABELLED: First and second trimester screening protocols for Down syndrome rely on marker values being referred to smoothed median values to produce adjusted multiple of the median (MoM) values to standardise for factors such as assay, gestation, maternal weight, smoking status, and so on. Changes in assay components, such as reagent lot, and inappropriate use of published regression equations for smoothed medians have resulted in biases in reported MoM values that in many applications remain uncorrected. This paper investigates the impact of these biases on patient-specific risk estimates and screening performance, and concludes that a 10% bias for an individual marker can result in an increase of between 1 and 2% in the false positive rate of the programme. A simple formula is also derived that enables the impact of these biases to be determined without the need for simulation, thus making it easier to design effective statistical quality control procedures to monitor the output of screening software algorithms. OBJECTIVE: To determine the impact of bias in MoM values on detection rates, false positive rates and patient-specific risks for Down syndrome. METHODS: We show that bias in MoM values affects risk through a multiplicative factor, and present an approximation to estimate this factor. We then show how bias in MoM values changes the effective risk threshold in the screening test, and hence the test's performance characteristics are determined by reference to a different point on the ROC curve for that test. Our approximation is based on the assumption of equal variance covariance structure for the unaffected and T21 log MoM values. We demonstrate, using computer simulation and supportive theoretical results, that the approximation is reliable in situations encountered in practice. Applications of the approximation are also discussed in respect of establishing effective quality control rules for median MoMs. RESULTS: Substantial changes in patient risk estimates and overall screening performance can result from the sort of biases in marker MoM values encountered in routine practice. In particular, biases of 10% in individual median marker MoM values can produce a four-fold range of risks when using the triple test. A 10% bias in a single marker will change the false positive rates by up to 2%. The effects on the false positive rate are approximately additive and, in cases where all markers are biased towards Down syndrome, biases in all three markers for the triple test can more than double the false positive rate. CONCLUSIONS: Biases in marker MoM values can occur in many ways, inappropriate median values, kit lot change, drift in assay performance and operator effects. We present methods which allow the impact of these changes to be assessed in relation to patient-specific risks and the overall screening performance. This, in turn, will enable appropriate quality control procedures to be established to control the magnitude of reported marker MoM biases, or equivalently, the magnitude of biases associated with the calculation of patient-specific risks. 2007 John Wiley & Sons, Ltd