Uncertainty in measurement: A review of the procedures for determining uncertainty in measurement and its use in deriving the biological variation of the estimated glomerular filtration rate.

Procedures for assessing the uncertainty in measurement and estimates of biological variation are currently available for many measurands capable of direct analytical measurement. However, not all measurands or quantity values determined in a medical laboratory are provided by direct analytical measurement. Estimated glomerular filtration rate (eGFR) is such a quantity value. In this situation, the result is calculated from other measurements through a functional relationship in which the output value (the calculated quantity value) is derived from one or more input quantities by applying a defined mathematical equation. The aims of this review are: to summarise the principal methods for assessing uncertainty in measurement in complicated non-linear expressions; and to describe an approach for estimating the uncertainty in measurement and biological variation of the Chronic Kidney Disease Epidemiology Collaboration equations for eGFR. In practice, either the direct application of the propagation of uncertainty in measurement equation or a Monte Carlo simulation procedure using a readily available spreadsheet may be used to evaluate uncertainty in measurement or the propagation of biological variation. If the only recognised "uncertainty" is the biological variation in the measured serum creatinine, the equation for the propagation of uncertainties in measurement for the eGFR simplifies to an expression in which the coefficient of variation of the eGFR (or the biological variation of the eGFR) is directly proportional to the coefficient of variation of the measured serum creatinine (or the biological variation of the serum creatinine).

Introduction

Glomerular filtration rate (GFR) is generally accepted as the best measure of kidney function. It is an essential measurement for the detection and monitoring of impaired renal function, for the safe use of nephrotoxic pharmaceutical agents and for the appropriate use of drugs which are cleared by the kidneys. Variability in the GFR is associated with age, gender, body size and ethnicity [1], [2]; in addition to analytical uncertainty in measurement (measurement uncertainty, MU) and a person's own intra-individual biological variation. Even though knowledge of a patient's GFR is important from many clinical perspectives, direct measurement of GFR with an “ideal” marker such as inulin, iothalamate, diethylene triamine pentaacetic acid or iohexol is difficult, expensive and impractical for routine practice [1]. Consequently, empirical equations which use serum creatinine concentration (the substance concentration of creatinine in serum –for brevity referred to in this paper as serum creatinine or SCr) as a surrogate filtration marker have been employed for several decades to estimate GFR. In addition to serum creatinine, GFR estimating equations usually include an adjustment for some, or all, of the other variables which influence GFR results; specifically, age, gender, body size and ethnicity [1], [2], [3], [4]. More recent versions of GFR estimating equations may also include serum cystatin-C concentration or serum creatinine plus serum cystatin-C as input variables [5], [6], [7], [8], [9].

Many empirical GFR estimating equations have been proposed, both for adults and for children [10], [11], [12]. However, all such equations are derived using regression statistics which model the relationship between one of the ideal GFR markers (measured in a specified group of individuals) and the chosen surrogate marker (serum creatinine and/or serum cystatin-C). This procedure produces a functional relationship or estimating equation which is essentially an average of the chosen sample population with an uncertainty that reflects the overall dispersion of the individual patient data. The scatter graphs provided in articles which describe the derivation or validation of GFR estimating equations clearly show the wide dispersion of individual data points and considerable individual variation around the fitted regression curve [2], [3], [4]. Results for estimated GFR (eGFR) will thus be an average based on the original statistical sample from which the equation was derived and cannot accurately determine the true GFR for an individual patient. In consequence, any relationship derived in this manner will give the same eGFR for all patients of the same age, gender and ethnicity for a given serum creatinine value irrespective of other considerations [13].

Most laboratory workers will be familiar with the concepts of uncertainty, particularly as they relate to imprecision and bias (systematic error). Imprecision in clinical laboratory methods is usually evaluated using internal quality control (IQC) procedures, with standard uncertainty obtained from the standard deviation of replicate control results [14]. The routine evaluation of bias is usually obtained from proficiency testing surveys when direct traceability studies are impractical or unavailable. The procedures for handling bias and its effects on laboratory results are subjects of on-going debate [15], [16], [17], [18], [19].

However, not all measurands or quantity values determined in a medical laboratory are provided by direct analytical measurement. In some situations, a quantity value is calculated from other measurements through a functional relationship, where the output value (the calculated quantity) is derived from one or more input quantities by applying a defined mathematical equation. The many examples of quantity values which are calculated using this approach include [20, 21 22, 23]: the calculation of serum anion gap and serum osmolal gap; the fractional excretion of substances in urine; creatinine clearance and estimated creatinine clearance by the Cockcroft and Gault procedure [24]; eGFR calculation by the Modification of Diet in Renal Disease Study (MDRD) [2], [25], the Swedish Lund-Malmo eGFR equations [4], the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) eGFR equations [3]; drug clearance and volume of distribution calculations; and estimates of free hormone concentration such as the calculation of serum free testosterone concentration [21].

The use of empirically derived functional relationships is now widespread in medical testing laboratories. These relationships may vary from relatively simple forms such as the anion and osmolal gap calculations, to complex forms such as the various eGFR and free testosterone equations. However, the empirical constants which are determined as part of these more complex regression-based equations are often considered to be free of inherent uncertainty, but equations derived by best-fit numerical analysis of patient data will always incorporate a degree of variability as shown by the scattering of raw data points in a “y” versus “x” scatter graph. Numerical constants derived in this manner must also have associated standard uncertainties. Consequently, the numerical constants as shown in the eGFR equations are not true constants but contribute an uncertainty which potentially increases the estimated interval in which the true GFR may be found [21]. Unless the standard uncertainties associated with such “constants” are also provided and included in any calculations, the full impact on the output of the functional relationship which is now applied to new input variables cannot be accurately assessed. This assessment is important as it potentially limits the interpretation of a given result when compared to a population reference interval or a fixed clinical decision value, or when assessing the significance of a change in the serial results of a patient. In addition, these interpretative situations may also require a knowledge and understanding of biological variation, including the biological variation of quantity values such as the eGFR.

The assessment of biological variation for measurands of clinical importance has been considered important for many years. Estimates of biological variation, both within-subject (CVI) and between-subject coefficient of variation (CVG) are currently available for many measurands capable of direct analytical measurement [26]. The importance of determining biological variation is evidenced by its central role in the development of quality specifications (quality indicators) for laboratory test procedures [27], [28], [29], [30]. When using either the total error approach to medical laboratory quality specifications [31], [32], or uncertainty in measurement procedures based on the Guide to the Expression of Uncertainty in Measurement (usually referred to as the GUM) [33], the analytical tolerance with regard to both random result variation (imprecision) and systematic error (bias) can be judged with respect to biological variation [27], [32], [34].

The aims of this review are: to summarise the principal methods for assessing MU in complex functional relationships; and to describe an approach for determining the uncertainty in measurement and its use in deriving the biological variation of the CKD-EPI eGFR. Even though the full GUM uncertainty propagation procedure and the derivation of the eGFR equations are complicated mathematical processes, the underlying statistical issues should not be ignored or over simplified. Accordingly, more specific aims of this review are to:

Provide a practical example of how GUM uncertainty principles may be applied to a complicated non-linear expression such as the eGFR and to provide a validated working procedure which can be implemented by anyone with a basic understanding of Excel (Microsoft Corporation, Redmond, WA, USA),

Outline the mathematical processes behind the GUM approach but in a way that can be bypassed for those not wishing to undertake their own full derivation as it relates specifically to eGFR,

Show how the GUM uncertainty procedures can also be applied to the propagation of biological variation when all other (analytical) uncertainties are set to zero, with direct application to the calculation and understanding of reference change values (RCV) [27], [35], [36].

eGFR terminology - measurand or quantity value?

The International Vocabulary of Metrology – Basic and General Concepts and Associated Terms (VIM; [37]) is the reference document that provides definitions for metrological terms and is normative for many international standards. In addition, the Eurachem guide Terminology in Analytical Measurement – Introduction to VIM 3 [38] provides additional explanation of selected VIM concepts and definitions which relate to chemical, biological and clinical measurements.

The definitions provided by VIM show a clear distinction between quantity value and measurand, with direct relevance to eGFR and GFR. Equations for estimating GFR are derived by applying statistical curve fitting procedures to an experimental set of paired data points to provide an equation which can predict the average value of the output variable when applied to new data. Various terms which are considered to predict GFR are compared to the “true” GFR, which is measured by a reference procedure. As such, GFR is the true measurand while eGFR is correctly described as an estimate of GFR. In this situation, eGFR should not be called a measurand but a quantity value. That is, eGFR is the estimated quantity value of the measurand GFR.

The eGFR equations

Since the introduction by Cockcroft and Gault in 1976 of their equation for estimating creatinine clearance [24], numerous attempts have been made to provide equations for the estimation of GFR in both adults and children. The first description of a six-variable MDRD equation was published by Levey et al. in 1999 [25]; followed shortly thereafter by an abbreviated four-variable model [39]. The next stage in the development of GFR estimating equations focussed on standardising the analytical measurement of serum creatinine [40].

However, even after re-standardisation and a slight change to the original MDRD equation, lower results were obtained at higher GFR levels when eGFR was compared to the corresponding measured GFR [3], [8], [41]. To overcome this limitation, Levey et al. developed a new set of equations using more complicated statistical procedures: the CKD-EPI equations for eGFR [3]. In addition to the studies by Levey et al., other groups have also developed their own GFR estimating equations using similar methodologies, the results of which may be more applicable to the regional populations on which they are based [4], [42], [43].

Even though there are many studies which compare eGFR with measured GFR or attempt to “clinically validate” a specific eGFR equation, there are few studies which address the important issue of uncertainty in measurement and the mathematical concordance of a given estimate with the true measurand value. Equation compliance in clinical validation studies is generally assessed by the number (percentage) of subjects who are correctly categorised when compared to the chosen reference procedure. Clinical validation studies are certainly appropriate and eminently applicable to eGFR, as the highest level for defining analytical performance is a model “based on the effect of analytical performance on clinical outcomes” [29], [44]. However, the clinical criteria by which eGFR is judged have been developed for GFR, not for eGFR. It is only the degree of agreement between eGFR and the corresponding GFR that allows the same clinical criteria to be employed. This type of “accuracy” assessment is clearly not the same as applying MU procedures to the analytical variables and empirical “constants” within the equation. The analytical agreement between eGFR and the corresponding GFR can be assessed using the procedures established for defining uncertainty in measurement.

GUM propagation of uncertainty procedures

The way in which errors (uncertainties) are propagated from measured values to a calculated quantity through a functional relationship is fully described in the GUM and other publications [20], [22], [33], [45], [46]. In summary, when the individual input quantities to the functional relationship (x1, x2, x3, …, x) are subject to random uncertainties, the variability in each term will often be considered to follow a normal (Gaussian) or other specified distribution, with a spread or dispersion characterised by standard deviations (s1, s2, s3, …, s). These standard deviations are the standard uncertainties associated with each of the x's (that is, u(x1), u(x2), u(x3), …, u(x)). Given this variability in the measured input quantities (the x's), the GUM describes the mathematical procedures required to determine the distribution of values attributable to the output quantity y (that is, the standard uncertainty u(y)) given the known dispersion for each of the measured x values and the functional relationship y = f (x1, x2, x3, …, x).

Monte Carlo simulation procedure for evaluating uncertainty in measurement

Monte Carlo simulation (MCS) provides a practical and robust alternative to the GUM modeling approach and is a general tool for evaluating uncertainty in measurement [21], [47], [48]. Modern personal computers running widely available spreadsheet software can be used for MCS in most medical laboratory applications without resort to the more complex GUM statistical procedures. For this reason, in addition to a potentially more accurate representation of the output distribution (see further comments below), we suggest that MCS may be preferred to the more complicated GUM modeling, particularly when complicated non-linear equations such as eGFR are being evaluated.

The suggested MCS procedure for evaluating measurement uncertainty in the non-linear CKD-EPI eGFR is described in Appendix 1. It is similar to the spreadsheet previously described but modified to accommodate the CKD-EPI equations [21], [47], [48]. Even though it has been shown that simulation of a normal distribution using the Excel pseudo-random number generator provides a distribution which is displaced in relation to a theoretical normal distribution (a slight positive bias), this has “no practical implications for applied metrology” or laboratory medicine [49]. For the eGFR, this slight departure from normality does not alter the result obtained. More importantly, the non-linear features of the CKD-EPI eGFR equation(s), in addition to some non-normal input uncertainty data, make the output distribution unpredictable. These issues make the MCS procedure potentially more reliable than the GUM modeling approach while providing a practical alternative, particularly for non-linear models [21], [47].

In addition to reducing the requirement for mathematical skills, an important feature of MCS is an output which may be used to graph the distribution of data and to directly determine a coverage interval for the output quantity, even when the output probability distribution function (PDF) has significant asymmetry. In contrast, as the GUM modeling approach does not explicitly determine a PDF for the output, this approach can sometimes limit the ability to clearly define an output PDF. As the MCS procedure can always provide a consistent PDF, a coverage interval corresponding to a stipulated coverage probability can always be derived even for non-linear models. For medical laboratory applications with a normally distributed output, this is typically 95% for a coverage factor of 1.96 or 95.4% for a coverage factor of 2.0. For asymmetric distributions, the coverage intervals (whether 95% or any other value) may vary in length. In such cases, the shortest 95% coverage interval is generally quoted since this provides a narrowest uncertainty range for the measurand or quantity value.

A good general procedure for calculating a coverage interval from an MCS output determined using Excel is to copy the output values into another column of the spreadsheet and sort from smallest to largest. The lowest 2.5% and highest 2.5% of values (based on row number) are then excluded to give a 95% coverage interval (or other percentage as required). If the distribution is symmetrical and approximately normal, this will equate to the usual 95% coverage interval of± 1.96 standard deviations.

Possible differences between GUM and MCS

Supplement 1 to the GUM, Propagation of distributions using a Monte Carlo Method [47], describes the propagation of distributions using MCS, limitations to the GUM approach and the procedure for validating the GUM using MCS. The conditions which apply for a valid application of the GUM uncertainty framework for linear models are described in section 5.7 of the Supplement. In other situations, validation using MCS is recommended and fully described in Section 8 (validation of results). Section 8.1.1 states “The GUM uncertainty framework can be expected to work well in many circumstances. However, it is not always straightforward to determine whether all the conditions for its application hold. Indeed, the degree of difficulty of doing so would typically be considerably greater than that required to apply [MCS]. Therefore, since these circumstances cannot readily be tested, any cases of doubt should be validated. Since the domain of validity for [MCS] is broader than that for the GUM uncertainty framework, it is recommended that both the GUM uncertainty framework and [MCS] be applied and the results compared.”

The objective in comparing GUM and MCS is to determine whether the coverage intervals obtained by both procedures agree within a specified numeric tolerance. Section 8.1.3 of the Supplement describes the procedure for evaluating this numeric tolerance, with the reference procedure for this comparison being the more robust MCS. Even though the GUM approach does not explicitly determine an output distribution, a normal distribution is often assumed when calculating an expanded uncertainty. In contrast MCS involves the propagation of distributions and thus automatically takes into account any non-linearities in the functional relationship. As such, the MCS procedure always provides an appropriate output distribution which may then be modeled to provide the required coverage interval, which, by its very nature, may be non-symmetrical. Consequently, MCS generally provides an improved estimate for non-linear models and GUM and MCS do not always give the same results. As outlined in GUM Supplement 1, where GUM and MCS differ, MCS provides the more appropriate coverage interval.

Validation of GUM using MCS

The procedure for validating the GUM approach using MCS is accomplished by comparing the two high end-points and the two low end-points, for a designated coverage interval obtained by using both methods. The absolute differences between the two high end-points and the two low end-points, d(high) and d(low), are then compared to what is regarded as a meaningful numeric tolerance (described as δ) associated with the standard uncertainty u(y), or u(eGFR) in the specific case of eGFR. The high and low end-points using MCS are the end-points of the shortest coverage interval for the designated coverage probability, defined in GUM Supplement 1 as a “coverage interval for a quantity with the shortest length among all coverage intervals for that quantity having the same coverage probability” [47]. Further information and a suggested procedure using Excel is provided in Appendix 1.

The numeric tolerance of a standard deviation or standard uncertainty can be obtained from the number of significant decimal digits considered meaningful in its numeric value. When the value is expressed in scientific notation (that is, m×10 n, where both m and n are integers), δ = 0.5 × 10 n [47]. The end-point differences for the designated coverage probability are then compared to δ. If both d(high) and d(low) are no larger than δ, the comparison is favourable, and the GUM procedure has been validated in this situation. If d(high) and/or d(low) are greater than δ, MCS is the preferred approach. The choice of both numeric tolerance for the standard uncertainty and the coverage probability are both subjective decisions and will influence this comparison. The procedure for GUM validation using MCS and the possible differences between GUM and MCS are summarised in Table 1.

Table 1

Guide to the expression of Uncertainty in Measurement (GUM) validation: comparison of GUM and Monte Carlo Simulation (MCS) procedures.

	GUM procedure	MCS procedure
eGFR (mL/min/1.73 m2)	eGFR calculated according to the designated CKD-EPI equation, (eGFR without further qualification)	Mean of all MCS output values (MCS eGFR)
SD(eGFR) = u(eGFR)	Standard uncertainty calculated using the full GUM propagation procedure	Standard deviation of all MCS output values
Output distribution	Does not explicitly determine an output distribution. Usually assumed to be Gaussian	No assumptions, actual distribution available for assessment
95% CI determined as	± 1.96 u(eGFR)	Shortest 95% interval from actual distribution
High end-point for 95% CI	eGFR + 1.96 u(eGFR)	From actual MCS output distribution, high end-point for shortest 95% interval
Low end-point for 95% CI	eGFR − 1.96 u(eGFR)	From actual MCS output distribution, low end-point for shortest 95% interval
High end-point difference, d(high)	d(high) = | (GUM high end-point for 95% CI) − (MCS high end-point from shortest 95% CI) |
Low end-point difference, d(low)	d(low) = | (GUM low end-point for 95% CI) − (MCS low end-point from shortest 95% CI) |
δ, numerical significance	0.5 for whole numbers	If d(high) and d(low) ≤ δ, GUM = MCS
	0.05 for numbers with one decimal place	If d(high) or d(low)> δ, MCS better than GUM

SD = standard deviation. CI = coverage interval for the designated probability.

The mathematical form of the CKD-EPI eGFR equations

The four CKD-EPI equations which are used to illustrate the procedure for calculating uncertainty in measurement and the propagation of biological variation (as described below) are shown in Table 2. These are the commonly used equations originally described by Levey et al. [3] and subsequently reviewed by others [13], [50], [51], [52]. As suggested previously, when evaluating uncertainty in measurement for relatively complicated non-linear expressions such as the eGFR, a MCS procedure using an Excel spreadsheet is recommended to bypass the mathematical complications of the GUM approach and to provide a clearly defined output distribution. For the specific case of biological variation being propagated within the eGFR equations, the direct GUM procedure provides a relatively simple approach. Even though the initial description of the GUM equation may be somewhat daunting for clinical biochemists, it can be reduced to an equation which is easy to apply when the uncertainty in the measured serum creatinine or the propagation of biological variation from the serum creatinine requires consideration. This simplified approach is described in more detail below.

Table 2

CKD-EPI equations for estimating GFR in white subjects [3].

Female with SCr ≤ 62 μmol/L	eGFR = 144(SCr × 0.0113 / 0.7)−0.329 × (0.993)age in years
Female with SCr> 62 μmol/L	eGFR = 144(SCr × 0.0113 / 0.7)−1.209 × (0.993)age in years
Male with SCr ≤ 80 μmol/L	eGFR = 141(SCr × 0.0113 / 0.9)−0.411 × (0.993)age in years
Male with SCr> 80 μmol/L	eGFR = 141(SCr × 0.0113 / 0.9)−1.209 × (0.993)age in years

SCr: serum creatinine concentration, μmol/L.

eGFR units: mL/min/1.73 m2.

There are four CKD-EPI equations for white persons, described in Table 2. All these equations have the same general form:where:

G = eGFR as derived by one of the CKD-EPI equation,

A = empirical constant (144 for a woman or 141 for a man),

B = serum creatinine in μmol/L (SCr),

K = factor for conversion of serum creatinine units (0.0113) from μmol/L back to the original mg/dL,

C = interior spline knot, the point where the two polynomial functions intersect (0.7 for a woman or 0.9 for a man),

D = empirical constant (− 0.329 for a woman with SCr ≤ 62 μmol/L, − 0.411 for a man with SCr ≤ 80 μmol/L, and − 1.209 for a woman with SCr>62 μmol/L or for a man with SCr>80 μmol/L),

J = empirical constant (0.993)

T = age in years

As described by Levey et al. [3], the CKD-EPI equations for estimating GFR include the estimation of log GFR, which in turn “includes log serum creatinine (modeled as a 2-slope linear spline with sex-specific knots at 62 μmol/L (0.7 mg/dL) in women and 80 μmol/L (0.9 mg/dL) in men), sex, race, and age on the natural scale”. For women, the critical value or knot connecting two splines is thus 62 μmol/L, with D = −0.329 when serum creatinine is equal to or below the knot and D = − 1.209 when above the knot. For men, the critical value is 80 μmol/L, with D = −0.411 when serum creatinine is equal to or below the knot and D = − 1.209 when above the knot.

At the knot, the first-order GUM propagation equation is unsuitable because of the discontinuous change in the first-order derivative with respect to the serum creatinine value. The GUM equation remains unsuitable if the serum creatinine value is too close to the knot, since the uncertainty in this value may create a variation which includes or exceeds the knot value. The same consideration applies to the use of MCS, although the programming for MCS to account for this possible “leakage” to the other side of the knot is much easier than with the GUM procedure. To avoid these complications but with no loss of relevance for serum creatinine values in typical clinical settings, the serum creatinine values may be set sufficiently distant from the knot to reduce the effect of such leakage. For the example value of standard uncertainty in the serum creatinine of 2.0 μmol/L as given in Table 3, values for serum creatinine have been used that are at least 5.0 μmol/L (that is 2.5 × u(SCr)) distant from the knot. This reduces leakage to less than 1%. In this circumstance, it may thus be considered that all GUM and MCS calculations for a given gender and serum creatinine value are based on a consistent value of constant D.

Table 3

Results for eGFR and its uncertainty in measurement calculated by direct application of the GUM procedure and by MCS.

	A	B (= SCr) μmol/L	K	C	D	J	T (= age) year	eGFR	MCS eGFR
(a) eGFR equation for white females with a serum creatinine ≤ 62 μmol/L.
Numeric value of term in equation	144	55	0.01131181	0.700	−0.329	0.993	40.00	113.03	113.07
Standard deviation, standard uncertainty (u)	0.141112	2.0	0.00000013	0.00000	0.05051	0.00020	0.000791	GUMu(eGFR) 1.77	MCSu(eGFR) 1.78
Standard uncertainty as CV (= u × 100 / value)	0.10%	3.64%	0.0012%	0.00%	15.35%	0.02%	0.0020%	1.57%	1.57%
Expanded uncertainty as CV, 95% coverage (= u × 100 × 1.96 / value)	0.19%	7.13%	0.0023%	0.00%	30.09%	0.04%	0.0039%	3.07%	3.09%
95% coverage interval high end-point, (GUM + 1.96 u(eGFR): MCS shortest 95%)								116.50	116.67
95% coverage interval low end-point, (GUM − 1.96 u(eGFR): MCS shortest 95%)								109.56	109.70
High end-point difference, | GUM − MCS shortest | = d(high)								0.17
Low end-point difference, | GUM − MCS shortest | = d(low)								0.14
(b) eGFR equation for white females with a serum creatinine> 62 μmol/L.
Numeric value of term in equation	144	100	0.01131181	0.700	−1.209	0.993	65.00	51.06	51.09
Standard deviation, standard uncertainty (u)	0.141112	2.0	0.00000013	0.00000	0.00561	0.00020	0.000791	GUMu(eGFR) 1.41	MCSu(eGFR) 1.41
Standard uncertainty as CV (= u × 100 / value)	0.10%	2.00%	0.0012%	0.00%	0.46%	0.02%	0.0012%	2.76%	2.76%
Expanded uncertainty as CV, 95% coverage (= u × 100 × 1.96 / value)	0.19%	3.92%	0.0023%	0.00%	0.91%	0.04%	0.0024%	5.41%	5.41%
95% coverage interval high end-point, (GUM + 1.96 u(eGFR): MCS shortest 95%)								53.83	53.87
95% coverage interval low end-point, (GUM − 1.96 u(eGFR): MCS shortest 95%)								48.29	48.34
High end-point difference, | GUM − MCS shortest 95% | = d(high)								0.04
Low end-point difference, | GUM − MCS shortest 95% | = d(low)								0.05
(c) eGFR equation for white males with a serum creatinine ≤ 80 μmol/L.
Numeric value of term in equation	141	70	0.01131181	0.900	−0.411	0.993	30.00	120.38	120.42
Standard deviation, standard uncertainty (u)	0.141	2.0	0.00000013	0.00000	0.04949	0.00020	0.000791	GUMu(eGFR) 1.77	MCSu(eGFR) 1.78
Standard uncertainty as CV (= u × 100 / value)	0.10%	2.86%	0.0012%	0.00%	12.04%	0.020%	0.0026%	1.47%	1.48%
Expanded uncertainty as CV, 95% coverage (= u × 100 × 1.96 / value)	0.20%	5.60%	0.0023%	0.00%	23.60%	0.040%	0.0052%	2.88%	2.90%
95% coverage interval high end-point, (GUM + 1.96 u(eGFR): MCS shortest 95%)								123.85	123.97
95% coverage interval low end-point, (GUM − 1.96 u(eGFR): MCS shortest 95%)								116.91	117.02
High end-point difference, | GUM − MCS shortest 95% | = d(high)								0.12
Low end-point difference, | GUM − MCS shortest 95% | = d(low)								0.11
(d) eGFR equation for white males with a serum creatinine> 80 μmol/L.
Numeric value of term in equation	141	125	0.01131181	0.900	−1.209	0.993	55.00	55.49	55.52
Standard deviation, standard uncertainty (u)	0.141	2.0	0.00000013	0.00000	0.00561	0.00020	0.000791	GUMu(eGFR) 1.25	MCSu(eGFR) 1.25
Standard uncertainty as CV (= u × 100 / value)	0.10%	1.60%	0.0012%	0.00%	0.46%	0.020%	0.0014%	2.25%	2.25%
Expanded uncertainty as CV, 95% coverage (= u × 100 × 1.96 / value)	0.20%	3.14%	0.0023%	0.00%	0.91%	0.040%	0.0028%	4.42%	4.41%
95% coverage interval high end-point, (GUM + 1.96 u(eGFR): MCS shortest 95%)								57.94	57.96
95% coverage interval low end-point, (GUM − 1.96 u(eGFR): MCS shortest 95%)								53.04	53.08
High end-point difference, | GUM − MCS shortest 95% | = d(high)								0.02
Low end-point difference, | GUM − MCS shortest 95% | = d(low)								0.04

A, B, K, C, D, J, T: generic terms in the eGFR equations as described in the main text and in Supplementary Appendix 2, .

eGFR (without further qualification): calculated according to the specified CKD-EPI equation, units mL/min/1.73 m2.

GUM u(eGFR): calculated using the full GUM propagation procedure (Eq. 3 of Supplementary Appendix 2).

MCS eGFR and MCS u(eGFR): evaluated using MCS with 1000,000 trials per simulation as described in Supplementary Appendix 1.

95% coverage interval high end-point: the upper value for the 95% confidence interval, the 97.5 percentile value.

95% coverage interval low end-point: the lower value for the 95% confidence interval, the 2.5 percentile value.

MCS shortest 95%: “coverage interval for a quantity with the shortest length among all coverage intervals for that quantity having the same coverage probability” [47].

High end-point difference: The absolute difference between the upper value for the 95% confidence interval obtained by GUM calculation and the upper value for the shortest 95% confidence interval obtained by MCS.

Low end-point difference: The absolute difference between the lower value for the 95% confidence interval obtained by GUM calculation and the lower value for the shortest 95% confidence interval obtained by MCS.

Using the GUM propagation of uncertainty in measurement equation with the restrictions outlined above, measurement uncertainty in the eGFR may be calculated given the uncertainties associated with all input quantities, including estimated uncertainties in the empirical constants.

Uncertainty in measurement and eGFR – an overview of previous publications

The application of MU procedures as outlined in the GUM have not been widely or consistently applied to any of the eGFR equations. To our knowledge, Cavalier et al. were probably the first in attempting to apply MU methodology to an MDRD eGFR equation [53]. Unfortunately, the procedure they used was incorrectly applied as pointed out by Parry [54], [55], who also provided a procedure which essentially followed the GUM approach but for only one variable: the serum creatinine. This correct but simplified one-variable approach assumes that uncertainty is only associated with the laboratory measurement of serum creatinine. It is essentially the same as the “simplified” approach as described in Appendix 2 and used for the propagation of biological variation as discussed below.

In a more detailed discussion of the GUM propagation approach and comparison with Monte Carlo simulation (MCS) using Excel, Farrance and Frenkel have described how both GUM and MCS procedures could be applied to all terms in the modified (standardised serum creatinine) MDRD equation [21]. Comparisons between the GUM and MCS procedures were made, assuming all terms in the MDRD equation had associated uncertainties.

Badrick and Turner used a simplified one variable GUM procedure to describe the propagation of both MU and biological variation from serum creatinine to the eGFR result [56]. That procedure again assumes that uncertainty is only associated with the laboratory measurement of serum creatinine. However, the propagation of biological variation from serum creatinine to the eGFR and the calculation of RCV are discussed in detail. The significance of RCV in eGFR result interpretation is also discussed in relation to the critical decision points which define the various stages of chronic kidney disease.

Kallner discusses MU in general terms and notes that algorithms for estimating GFR may increase the uncertainty of the final result when related to the MU of serum creatinine [13]. In particular, when evaluating the clinical significance of any difference between eGFR results obtained from different algorithms, the MU of serum creatinine should also be considered. Also discussed are the observations of Stevens et al. [2], that a zone of width± 30% is required around the measured GFR to include 82% of the eGFR results. That is; “an eGFR of 60 units would be associated with an uncertainty interval from 42 units to 78 units”. Kallner's article provides a discussion on MU and its possible relationship to eGFR, while questioning the use of global interpretative set-points.

In a description of “The eGFR-C study” [8], Lamb et al. describe several issues which influence eGFR interpretation and the problem with “GFR-estimating equations to identify the progression of kidney disease against background change in GFR” … “given the biological and measurement variation of both reference and estimated GFR”. Estimates for the within-subject biological variation of eGFR are provided, with a further discussion of the relevance of RCV in eGFR interpretation.

In a technical report provided by the National Measurement Institute, the body responsible for maintaining Australia's units and standards of measurement, Frenkel describes the mathematics which underlies the GUM approach for determining MU [22]. The report is largely directed towards measurements in clinical biochemistry and a discussion of the propagation of uncertainties in empirical equations. In addition to a detailed description of the GUM procedure and the calculation of “sensitivity coefficients”, actual laboratory-based calculations are provided as examples. One such example is the calculation of uncertainty in the CKD-EPI eGFR, with uncertainty estimates associated with all terms in the equation and the utilization of the full GUM approach. Some additional aspects of this approach have recently been summarised [23].

Calculation of MU is now a routine procedure for many clinical laboratories. Measurement procedures for most measurands include internal quality control (IQC) for the assessment of imprecision and external quality assessment (EQA) or proficiency testing for the evaluation of systematic error (bias). For substances determined by direct analytical measurement this is essentially a top-down MU approach. As suggested earlier, however, not all measurands or quantity values are obtained by direct analytical measurement. In some situations, a quantity value is calculated from other measurements through a functional relationship in which the output value (the calculated quantity value) is derived from one or more input quantities. For measurands or quantity values determined in this manner, the evaluation of analytical MU is often overlooked.

Uncertainty in measurement of the CKD-EPI eGFR – terms with low uncertainty

The CKD-EPI equations include a serum creatinine measurement which has been standardised to the Roche enzymatic method (Roche–Hitachi P-Module instrument with Roche Creatininase Plus assay, Hoffman-La Roche, Basel, Switzerland) [3], which was cross-calibrated to a National Institute of Standards and Technology (NIST; U.S. Department of Commerce, Gaithersberg, MD, USA) creatinine standard using isotope dilution mass spectrometry [40]. In addition to the serum creatinine term with its analytical uncertainty in measurement obtained from IQC, the various eGFR equations also contain patient age as a variable and numeric “constants” as both multipliers and power terms. Presented in this manner, it may be assumed that these numeric “constants” are free of uncertainty even though the original reports clearly indicate that this assumption is not correct [57], [58].

Like many empirical equations which have useful applications in medicine, the eGFR equations have been derived by stepwise multiple regression procedures to determine a set of variables that jointly predict GFR. With the exceptions of patient age and the factor 0.0113, the regression coefficients have been derived from data with a wide inherent variability. Thus, to provide a more complete uncertainty estimate of the output variable (the eGFR), the actual uncertainty associated with each of these numeric terms is also required. The original article by Levey et al. does indeed provide confidence intervals for some of these terms but information about the initial multiplier term (the slope term) appears to be missing [57], [58]. The available confidence intervals are given in Table 4.

Table 4

Confidence intervals for CKD-EPI eGFR as provided by Levey et al. [57], [58], with corresponding standard uncertainty.

Empirical constant		95% Confidence interval		Range	Standard uncertainty, u
SCr exponent below knot, F	−0.329	−0.230	−0.428	0.198	0.05051
SCr exponent below knot, M	−0.411	−0.314	−0.508	0.194	0.04949
SCr exponent above knot, F&M	−1.209	−1.198	−1.220	0.022	0.00561
African American multiplier	1.159	1.144	1.170	0.026	0.00663
Female multiplier	1.018	1.007	1.029	0.022	0.00561
Age factor	0.993	0.9925	0.9933	0.00080	0.00020
Slope multiplier (M) 141	141
Slope multiplier 144 = (slope multiplier 141) × (female multiplier, 1.018) = 144 with u(144) = [u(141)2 + u(1.018)2]½

Abbreviations: SCr, serum creatinine; F, female; M, male.

As the factor 0.0113 in the serum creatinine term provides a conversion of units from μmol/L back to the original mg/dL, its value is known with sufficient precision to be treated as a true numeric constant as atomic mass (weight) values are known with much greater accuracy than required for the eGFR calculation. Using IUPAC atomic mass values with their associated uncertainties, a creatinine conversion factor of 0.01131181 with a standard uncertainty of 0.00000013 is obtained [59]. In accord with IUPAC recommendations, a rectangular probability distribution should be used in the calculation of uncertainty with respect to molecular mass. A rectangular or uniform probability distribution is used as a model in situations where the probability of obtaining any value between two stated limits is equal to the probability of obtaining any other value between these limits. The rectangular probability distribution has boundaries or limits which are usually specified as± a from a central value, where a is the half-width of the distribution. In this situation, 100% of the values must fall between – a and + a. The standard deviation (standard uncertainty) of a rectangular distribution is given by a/√3.

Patient age may also present a potential uncertainty depending on how this variable has been programmed within the equation. If age is given as an integer number of years, the uncertainty of this term would be± 0.5 years and again would follow a rectangular probability distribution. If a complete birth date in years, months and days is provided, this will allow a more accurate age which could then justifiably be considered to have zero uncertainty. If age is provided in this manner, it can be entered as an ordinal date, with the months and days being converted to a decimal fraction of 365. If this type of entry is used, the uncertainty in the age is± 0.5 days or 0.5 / 365 = 0.0014 years and this also should be regarded as a rectangular probability distribution. That is, if age is entered with an uncertainty of± 0.5 days, this is effectively the half-width of a rectangular distribution (with any time within the± 0.5 day period having an equal probability of being "correct"), implying a standard uncertainty of 0.0014 / √3 = 0.000791 years.

In addition to the creatinine conversion factor and age, the two other terms which could justifiably be considered to have zero uncertainty are the spline knot values. Even if the knot points were allocated with a degree of subjectivity, it is likely they were optimised to provide a minimum for any residuals associated with the curve fitting procedure. Thus, once allocated, they are essentially “fixed” for the given equation and act much more like true constants. As indicated previously, it is also suggested that a safe distance (2.5 × u(SCr)) be left between the “working area” and the knot points. This distance also helps to mitigate any uncertainty which may be associated with the actual knots.

Uncertainty in measurement of the CKD-EPI eGFR – GUM modeling and MCS approaches compared

The differential equation required for calculating the standard uncertainty of the eGFR using the GUM modeling approach is fully described in Appendix 2. It is derived from the general equation for the propagation of uncertainty [20], [22], [23], [33], reconfigured to represent the CKD-EPI eGFR equations. That is;where u(G) is the standard uncertainty of the eGFR, u(A) the standard uncertainty of the empirical constant (144 for female or 141 for male), u(B) the standard uncertainty of the serum creatinine value, and so on for the other terms in the eGFR equation. By replacing the algebraic terms and partial derivatives with specific numeric values as described in Appendix 2, the standard uncertainty associated with the eGFR (that is, u(eGFR)) can be calculated. The above equation (Eq. 3 in Appendix 2) assumes no correlation between the individual terms in the eGFR equation, but otherwise represents the full GUM modeling approach in order to demonstrate the complete procedure which provides for standard uncertainties to be allocated to all terms (including any uncertainty associated with the creatinine conversion factor, age and the knot values). However, if a specific term is considered to have insignificant or zero uncertainty, replacing the relevant uncertainty value with zero effectively removes that term from the equation and its contribution to any uncertainty in the eGFR.

Table 3(a-d) provide representative examples of the uncertainty components associated with the four eGFR equations described in Table 2 and the known uncertainties for the empirical constants as given in Table 4. Using different sets of input values for each of the four equations, GUM modeling and MCS are compared by producing eGFR results with their respective coverage intervals which relate to different stages of kidney disease. As uncertainty information does not appear to be available for the primary “slope constant”, a small standard uncertainty value (0.1% of the nominal value) has been included in order to demonstrate the propagation of uncertainty procedure; both by the full GUM calculation (Appendix 2) and by MCS (Appendix 1). The term referred to as the primary “slope constant” is the 141 multiplier term in the male CKD-EPI eGFR equation. The corresponding term in the female equations, the 144 multiplier term, is actually derived from the male value multiplied by the female factor of 1.018 (that is; 141 × 1.018 = 144). Even though the CKD-EPI equations are often presented as four separate equations (as in Table 2), they can also be presented as one equation as described by Levey et al. [3]. The one equation format provides various multiplier options which modify the basic white male version for females and African Americans. It is the uncertainty in the base 141 multiplier which appears to be unavailable even though the female factor (the 1.018) is provided with a confidence interval. As such, the standard uncertainty of the product which results from the multiplication of two terms is given directly be the square root of the sum of the variances rule (that is; u(144) = [u(141)2 + u(1.018)2]½).

The results presented in Table 3 show the eGFR values obtained by MCS give identical results to those obtained by direct calculation for the specified input data (which includes a serum creatinine analytical uncertainty obtained from IQC). In contrast to the data in the “MCS column”, which will vary slightly each time a simulation is run, the data in the eGFR and GUM u(eGFR) column give unique values as they represent the same data set processed through a defined equation. The u(eGFR) values determined by GUM and MCS are also the same provided they are considered as whole numbers. Statistically, this is shown by the fact that the values derived for the high and low end-point differences, d(high) and d(low), are both less than the specified numeric tolerance (δ). However, if u(eGFR) is considered with a tolerance to one decimal place, the GUM and MCS give results which are statistically different, with MCS providing the preferred coverage interval.

As the results provided in Table 3 include uncertainty estimates for all terms in the eGFR equation and were obtained using the full GUM propagation procedure, they are different to the u(eGFR) value calculated by the “simplified” equation. The “simplified” equation as derived in Appendix 2 applies when uncertainty or biological variation associated with the serum creatinine term is being considered.

eGFR biological variation

The MU of the eGFR may be ascertained using either the full GUM propagation of uncertainties procedure or a MCS based on the full GUM procedure. However, eGFR biological variation may be derived from the known biological variation for serum creatinine using a “simplified” version of the GUM equation as derived in Appendix 2. In effect, when all input analytical uncertainty values are replaced with biological variation data (either CVI or CVG), the output “uncertainty” will be an estimate of the biological variation of the calculated quantity. In the specific case of the CKD-EPI equations, the propagation of biological variation from serum creatinine requires the following:

zero uncertainty for the empirical constants,

zero uncertainty for the spline knot values as outlined previously,

zero uncertainty for the creatinine conversion factor (0.0113); however, this value is known with sufficient precision to be treated as a true numeric constant as previously described,

zero uncertainty for patient age; which in practice is reasonably true if recorded in years, months and days (as a decimal of a year) as previously described,

zero analytical uncertainty for serum creatinine, but “replaced” with serum creatinine biological variation.

When considering the propagation of biological variation (as distinct from the propagation of uncertainty in measurement), it seems intuitively reasonable to specify zero measurement uncertainty for all input variables, as the requirement for deriving biological variation is to do so in a manner which removes the analytical uncertainty component [27], [31], [60]. The data provided in the biological variation database compiled by Ricos et al. and hosted on the Westgard internet site is reputed to represent “pure” biological variation with analytical variation removed [26]. As such, the estimated biological variation of the eGFR when calculated as described below also represents “pure” biological variation which will require the addition of analytical uncertainty (by the sum of variances procedure) for calculation of a RCV or other procedure [27], [35], [36], [61]. Even though concerns have been raised as to the quality of the information contained within the Ricos et al. database [62], [63], [64], we consider these concerns outside the scope of this review. It is certainly acknowledged that correct biological variation data is required and that the source information requires scrutiny to ensure that it meets the agreed criteria for its proper assessment [31], [65], [66]. However, the principal consideration for this review is that by replacing serum creatinine analytical measurement uncertainty with biological variation data (whatever value from whatever source), the input biological variation associated with a given serum creatinine propagates to provide the biological variation of the eGFR. That is, the mathematics of the propagation process is separate from the quality of the input data.

The stated CVI for serum creatinine within the Ricos et al. database is 5.95%, without differentiation for age or gender [26]. This is the value which can be used to “replace” the analytical standard uncertainty of serum creatinine obtained from internal quality control (IQC) when estimating the propagation of biological variation. This coefficient of variation (CV) is used directly in the “simplified” uncertainty equation where the biological variation of the eGFR is directly proportional to the biological variation of the serum creatinine as derived in Appendix 2. That is:

CV (eGFR biological variation) = |D| × CV (SCr biological variation)

where CV can be either CVI or CVG and |D| is the absolute value of the exponent to the serum creatinine term in the CKD-EPI equation.

This approach was considered previously as a method for estimating analytical measurement uncertainty in a MDRD eGFR by Parry and by Badrick and Turner [55], [56]. It is essentially the same as the method for deriving an approximate equation for estimating the uncertainty in measurement of the International Normalised Ratio (INR) as described by Taberner et al. [67], and by Bennett and Critchfield [68]. As discussed by Badrick and Turner, it can also be used for estimating the biological variation of the eGFR when the uncertainty associated with serum creatinine is “replaced” with the biological variation for serum creatinine [55].

Using this “simplified” equation for calculating the CVI of eGFR, it is interesting to observe that, for eGFR equations with an exponent to the serum creatinine term of less than 1.0, there is a decrease in the CVI(eGFR) relative to that for serum creatinine. That is, taking CVI(SCr) = 5.95% [26]: for women with a serum creatinine ≤ 62 μmol/L, the CVI(eGFR) is 2.0%; for men with a serum creatinine ≤ 80 μmol/L, the CVI(eGFR) is 2.5%. As serum creatinine increases (as in the two equations which relate to higher serum creatinine values), the CVI(eGFR) increases to 7.2% for both men and women.

Summary and general discussion

In any discussion regarding uncertainty in measurement, the reporting interval (that is, the smallest unit of measurement chosen for reporting the result) should also be considered [69]. The number of decimal places used for reporting u(eGFR) is critical for determining the validity of the GUM approach as shown in Table 3. If u(eGFR) is rounded to a whole number, then the numeric tolerance δ = 0.5 and the GUM procedure has been validated for that set of data. However, if u(eGFR) is quoted with one digit after the decimal point, then δ = 0.05 and the GUM and MCS give results which are statistically different. But is this a practical difference and where should any numeric rounding occur?

As a general computational principle, particularly when doing multiple calculations such as MCS, it is prudent to use as many significant figures as available within the calculations and then do any appropriate rounding at the final reporting step. This helps to overcome any potential compounding of rounding errors which may occur during the calculation. The final step in the calculation starts with the u(eGFR). When expressed to the appropriate number of decimal places (this number determining whether GUM has been validated) the final result is:

The expanded uncertainty U(eGFR), where U(eGFR) = 1.96 × u(eGFR) for a 95% coverage interval, in cases where the GUM has been validated.

The shortest 95% MCS-provided coverage interval in cases where the GUM has not been validated.

From a practical perspective, however, either may be used - Table 3 shows the comparison between GUM and MCS for the stated input data (or for similar input data and uncertainties). This remains true even if MCS reveals some non-normal features in the distribution of eGFR results. This practical approach can also be judged by comparing the coverage interval end-points as shown in Table 3. Rounded to whole numbers, all end-points for the respective input data give equivalent results.

Biological variation data are usually provided in the form of a percentage CV [27], [70]. This means that the actual biological variation will depend on the absolute value of the specific measurand or quantity value. In addition, biological variation for measurands derived from healthy persons may not always be directly applicable to persons who are unwell [60], [70]. This certainly seems to be the case for serum creatinine, where a greater biological variation has been observed in patients with impaired renal function [60], [71]. For example: Reinhard et al. found that a group of healthy subjects showed an average within-subject biological variation for serum creatinine of 4.7% (with a mean serum creatinine of 77 μmol/L and an analytical CV of 1.6%), when compared to a group of patients with renal impairment who showed an average within-subject biological variation of 8.9% (with a mean serum creatinine of 224 μmol/L and an analytical CV of 1.4%) [71].

As a direct consequence of the form of the CKD-EPI equations, the two equations for “low” serum creatinine values give eGFR biological variations less than the biological variation for serum creatinine. For the two equations associated with “high” serum creatinine values, the eGFR biological variation will always be greater than that for serum creatinine. For both these “high” serum creatinine equations the eGFR CVI is 7.2% for both men and women. This is similar to the CVI value of 6.7% described by Toffaletti and McDonnell for the MDRD equation, but significantly less than the within-subject biological variation for creatinine clearance as described by Toffaletti and McDonnell (18.7%) or by Gowans and Fraser (13.6%) [72], [73].

In addition to the lower percentage eGFR biological variation relative to that for serum creatinine which occurs when the exponent to the serum creatinine term is less than 1.0, the form of the CKD-EPI eGFR equations also shows a high dependence on the age term at all serum creatinine values. The age term changes from a multiplier of 0.87 at age 20, to a multiplier of 0.57 at age 80 years.

When the uncertainty in measurement for serum creatinine is combined with the given uncertainty in the empirical constants and a 0.1% uncertainty in the base “slope constant”, the combined standard analytical uncertainty (as CV) for eGFR is around 1.5–2.7%, thus giving an expanded analytical uncertainty (for a coverage interval of 95%) of approximately 3–5% (rounded to whole numbers).

When determining the RCV or critical difference between two serial results, the biological and analytical variations are both important considerations [27], [35], [36], [61]. A similar situation also applies when determining the significance of a difference between a patient result and a clinical decision value as would occur for the decision points which define the various stages of chronic kidney disease [1], [74]. For example, using the data presented in Table 3b for a white female with a serum creatinine of 100 μmol/L, an eGFR of 51 mL/min/1.73 m2, an eGFR biological variation of 7.2% and an eGFR analytical variation of 2.8%, the RCV between two consecutive eGFR results from the same laboratory is 21% (or 51 ± 11 mL/min/1.73 m2) and between the actual result and a staging decision point is 15% (or 51 ± 8 mL/min/1.73 m2).

Conclusion

The aims of this review were to summarise the principal methods for assessing uncertainty in measurement in complicated non-linear expressions and to describe an approach for estimating the uncertainty in measurement and biological variation of the CKD-EPI eGFR and similar empirical equations. As the eGFR equations are now routinely used for the assessment of GFR, determining both their analytical uncertainty and biological variation may contribute to the overall utility of this assessment. In addition to a greater understanding of the effects of MU on the determination of eGFR, the procedures described should also provide a firm statistical base for a better understanding of the clinical application and further evaluation of the eGFR.