Limited Reliability of the Spot Urine Protein/Creatinine Ratio in the Longitudinal Evaluation of Patients With Lupus Nephritis.

INTRODUCTION: Cross-sectional studies document that the spot protein/creatinine ratio (PCR) is often an inaccurate estimate of proteinuria magnitude compared with the 24-hour PCR, which is the gold standard. However, the extent to which the inaccuracy of the spot PCR varies over time and between individuals has not previously been reported. We address these crucial questions using a unique database, an National Institutes of Health trial in which lupus nephritis (LN) patients (N = 103) provided spot PCR testing each month and 24-hour PCR testing every 3 months for up to 15 months after induction therapy.
METHODS: A gold standard proteinuria trend line was constructed for each patient by joining the points that represented the serial 24-hour PCR values of the patient. The spot PCR values of the patient were then plotted in relationship to the 24-hour PCR trend line. Using our previous work, which estimated the 95% confidence intervals for the 24-hour PCR at specific levels, we determined in each patient whether the spot PCR values were "reliable," "problematic," or "unreliable." The sequential spot PCR of the patients deviated widely and often from the 24-hour PCR trend line, to the extent that, if the spot PCR results were used in real time for clinical decision-making, it was likely management errors would occur.
RESULTS: Spot PCRs were reliable in 41%, problematic in 24%, and unreliable in 35% of patients. Those with unreliable spot PCRs could not be predicted and were more likely to respond poorly to treatment.
CONCLUSION: The spot PCR should not be used for management of LN, and perhaps, other glomerulopathies.

The protein/creatinine ratio (PCR) of a spot (single void) collection is the method for monitoring proteinuria currently recommended by Kidney Disease Improving Global Outcomes and by the American College of Rheumatology.1, 2, 3, 4 The basis for this recommendation is that spot collections are convenient for the patient and the physician, and produce a PCR value that is significantly correlated with 24-hour proteinuria (24-hour P). However, as previously discussed,5, 6 the high correlation coefficients between the spot PCR and 24-hour P (e.g., typically >0.90) is a mathematical consequence of comparing the spot PCR and 24-hour P over a wide range of values (e.g., 24-hour P range: 0– 10 g/d). Over more restricted ranges (e.g., subnephrotic range proteinuria, ≤3.5 g/d, the most common range of abnormal proteinuria), the correlation coefficient between the spot PCR and 24-hour P is weak, and concordance is poor.5, 6, 7, 8, 9, 10, 11

The reason for the poor correlation and concordance between the spot PCR and 24-hour P (or 24-hour PCR) is the remarkable variability of the spot PCR. This was first demonstrated by Koopman in patients with nephrotic range proteinuria, who were placed at complete bedrest for 3 days, maintained on a constant diet, and received no medications. Despite these measures intended to stabilize proteinuria, the PCR of sequential 3-hour urine collections over 3 days was highly variable. For example, the spot PCR ranged from approximately 3.0 to 9.0. Because of this inherent hour-to-hour variability of the PCR, short urine collections (e.g., spot collections) revealed this PCR variability, whereas long collections (e.g., intended 24-hour urine collections) concealed this PCR variability because the PCR of a long collection is the integrated mean of the PCRs of the entire collection.5, 8

As we and others have shown, it is not necessary to obtain a complete 24-hour collection to reliably determine the 24-hour PCR. Rather, it is sufficient that the intended 24-hour urine collection represents a substantial fraction of a complete 24-hour collection. For example, 50% of a complete collection, based on its creatinine content, provides a PCR that is a reliable estimate of the PCR of a complete 24-hour urine collection in that patient.6, 12 For our previously published comparisons, the definition of a “complete” 24-hour urine collection is the creatinine content of the collection within ±10% of that calculated from the Cockcroft-Gault equation and solved to estimate 24-hour creatinine excretion.6, 13

The use of spot PCRs in research is generally not problematic because much of the spot PCR variability is random. So, in comparing spot PCR values among cohorts, the random variability within each cohort is offset by averaging. However, in the management of individual patients, the remarkable variability of spot PCRs could lead to serious errors in management. The present work is the first to rigorously examine this question. We studied patients with lupus nephritis (LN) who received induction therapy in the ACCESS (Abatacept and Cyclophosphamide Combination Efficacy and Safety Study) trial. By protocol, each patient provided a 24-hour urine collection every 3 months and a spot urine collection monthly, for up to 15 months of follow-up. A proteinuria trend line was constructed for each patient by joining the 24-hour PCR points. This line was deemed the most reliable measure of the proteinuria trend. The rationale was that 24-hour PCR (or 24-hour P) was the most reliable estimate of the proteinuria magnitude.5, 6, 9, 10, 11, 12 Therefore, the line created by joining the individual 24-hour PCR values is the most reliable estimate of the proteinuria trend. The individual spot PCR values of the patients were compared with this trend line. ACCESS was well suited for this analysis because spot and 24-hour urine collections were made at prespecified intervals in a large group of patients whose proteinuria changed over a wide range of values during 15 months of follow-up.

Methods

ACCESS was a phase II multicenter, randomized, double-blind controlled trial of abatacept versus placebo with a standard care low-dose i.v. cyclophosphamide in patients with class III or IV LN. By protocol, each patient provided a single freshly voided spot urine collection at each of the monthly study visits. At the 3-, 6-, 9-, and 12-month visits, only a 24-hour urine collection was provided.

Determining the Ability of the Spot PCR to Reliably Identify Proteinuria Trends During ACCESS Follow-up

For this analysis, the sequential spot PCR values for each patient were displayed graphically over time and in relation to the proteinuria trend line of the patient (sequential 24-hour PCR values). Supplementary Figure S1 illustrates this for each patient according to whether the spot PCRs were determined to be reliable, problematic (generally reliable but with notable exceptions), or unreliable in identifying the proteinuria trends of the patients as defined by the 24-hour PCR trend line. These adjudications were a 2-step process carried out concurrently by co-authors GS, DB, and LH. Step 1 involved the inspection of the spot PCR/24-hour PCR graph of each patient to determine whether it was empirically evident (obvious) that the spot PCR/24-hour PCR met the criteria for reliable spot PCR. The spot PCR values closely followed the 24-hour PCR trend line and would inform clinical decision-making, as well as the 24-hour PCR values. The cases that did not have a reliable spot PCR were evaluated in step 2.

Step 2 objectively measured the frequency of when the spot PCR values of the patient were outside the 95% confidence intervals (CIs) of the corresponding 24-hour PCR values, which were extrapolated from the 24-hour PCR trend line of the patient. The extrapolation was performed as follows. For each spot PCR value that deviated widely from the 24-hour PCR trend line of the patient, a perpendicular extrapolation was made from that spot PCR value to the 24-hour PCR trend line of the patient. To assess whether the spot PCR value was outside of the 95% CIs of the extrapolated 24-hour PCR, we used our previous work, which calculated the 95% CIs for specific levels of 24-hour PCR in LN patients who were followed long-term and had stable 24-hour PCR at prespecified levels. To determine whether the spot PCRs were outside of the 95% CIs of the extrapolated 24-hour PCRs, we selected the 24-hour PCR range that included the spot PCR of interest from our previously published work. If the spot PCR of interest was outside the 95% CIs of the 24-hour PCR range to which it was matched, the spot PCR was deemed an outlier that contributed to the determination of whether the spot PCR of the patient was problematic or unreliable. For the determination of spot PCR reliability and/or unreliability, we did not construct 95% CIs bands for spot PCRs of individual patients because there were too few spot PCR measurements per patient.

Depending on the number and pattern of spot PCRs that deviated widely from the 24-hour PCR trend line of the patient, we defined 2 different degrees of spot PCR unreliability as follows.

Spot PCR is generally reliable but with notable exceptions (problematic). In these cases, 1 or 2 spot PCR values showed large deviations (outside of the 95% CIs) from the 24-hour PCR trend line. Because these deviations were large, it was likely that they would be problematic for the managing physician because these spurious spot PCR values unfolded in real time.

The spot PCR is unreliable. In these cases, spot PCR was deemed obviously unreliable either because of ≥3 large, usually sequential (>1.96 SD, the 95% CI) deviations of the spot PCR from the 24-hour PCR trend line or a single large spike in the spot PCR from the 24-hour PCR trend line that was consistent with a high-threshold criteria (HTC) (using the most stringent of the HTC criteria) proteinuric flare.

To provide further detail regarding the rationale for the unreliable spot PCR classifications, in Supplementary Figure S1, each of the unreliable spot PCR/24-hour PCR graphs (N = 36) is annotated, and the spot PCRs that determined the unreliable classification are cited.

Thresholds for Proteinuria LN Flare

In previous work, we discussed investigators who used different thresholds to define a LN proteinuric flare. We categorized these as low-threshold criteria, intermediate-threshold criteria, or HTC. Examples of HTC for LN proteinuric flares would be a preflare 24-hour PCR ≥2.0 that increased by ≥100%. It is these HTC that we used to determine whether a given spot PCR represented a large deviation from the 24-hour PCR trend line of the patient.

Statistical Analysis

Normally distributed variables are shown as the mean value ± SD. Other variables are shown as the median and interquartile range. The statistical tests used in these comparisons are shown in relationship to the data.

Results

ACCESS demographics and baseline clinical characteristics have previously been reported. The ACCESS cohort is representative of US LN cohorts because most are female and many are of African ancestry.

Ability of the Spot PCR to Correctly Identify the Proteinuria Trend

For this analysis, 103 patients were eligible based on having at least 3 24-hour PCR measures during the interval that the monthly spot PCRs were measured. Each sequential spot PCR values of the patients were displayed in relationship to the proteinuria trend line of that patient (the line formed by joining the sequential 24-hour PCR values of the patient). The outcome was that spot PCRs were found to be reliable in 41% (42 of 103), problematic in 24% (25 of 103), and unreliable in 35% (36 of 103) of patients.

Figure 1 shows the spot PCR/24-hour PCR displays for 5 representative reliable patients matched to 5 representative unreliable patients based on the 24-hour PCR at baseline. The unreliable patients were also selected because their values illustrated that unreliable spot PCRs could greatly and consistently overestimate or underestimate the 24-hour PCR trend line, and included those in whom the spot PCR trend was opposite to that of the 24-hour PCR trend line.

Figure 1

Representative ACCESS patients in whom the spot protein/creatinine (P/C) ratio (PCR) was deemed to be reliable or unreliable based on the degree to which the spot PCR values of the patient follows the proteinuria trend line of the patient (the line joining the patient’s 24 PCR value). See Supplementary Figure S1 for display of all 103 ACCESS patients according to whether the spot PCRs were deemed reliable, problematic, or unreliable.

Supplementary Figure S1 shows the spot PCR/24-hour PCR display for each patient. The displays are arranged in cohorts (reliable, problematic, and unreliable). The unreliable spot PCRs/24-hour PCRs displays are annotated, and the spot PCRs responsible for the unreliable classification are cited.

Association of Demographic and Baseline Clinical Measures With the Reliable, Problematic, or Unreliable Classifications

To assess whether baseline measures could predict whether the spot PCR would be reliable, problematic, or unreliable, we performed univariate assessment of key demographic and baseline clinical measures. As shown in Table 1, this exploratory analysis did not identify predictors of reliability and/or unreliability.

Table 1

Relationship between baseline measures and their association with whether the spot protein/creatinine ratio were deemed to be reliable, problematic, or unreliable

Demographic	Reliable	Problematic	Unreliable	P value
Sex (F/M)	39/3	23/2	32/4	NDa
Race (B/W/O)	22/20/0	6/16/3	12/17/7	NDa
Age (yr)b	29.5 (25.0−37.0)	28.0 (24.5−26.5)	32 (25.0−45.0)	0.400c
Weight (kg)b	72.0 (64.1−88.5)	62.1 (51.4−81.0)	70.6 (63.1−79.4)	0.089c
Serum albumin (g/dl)b	2.9 (2.6−3.3)	2.7 (2.2−3.2)	2.7 (2.1−3.4)	0.181d
Serum creatinine(mg/dl)b	0.94 (0.70−1.20)	0.78 (0.60−1.38)	1.00 (0.80−1.24)	0.158c
24-hour PCRb	2.3 (1.3−3.3)	3.5 (2.1−5.9)	2.3 (1.6−5.6)	0.046c
Abatacept/placebo	18/24	15/10	19/17	0.376a
ACE inhibitor and/or ARB	22	—	21	0.652a

ACE, angiotensin-converting enzyme; ARB, angiotensin receptor blocker; B/W/O, black, white, other race; ND, analysis not performed due to insufficient sample size; PCR, protein/creatinine ratio.

By Fisher exact test.

Median (intraquartiles).

By Kruskal-Wallis test.

By 1-way analysis of variance.

The ACCESS final outcomes of complete remission (CR), partial remission (PR), or treatment failure (TF) were associated with whether the spot PCRs of the patients during ACCESS were classified as reliable, problematic, or unreliable. This analysis involved the 98 ACCESS patients who had ≥3 24-hour PCR measures and accompanying spot PCR measures (this determined whether their spot PCRs were reliable, problematic, or unreliable). Baseline and final 24-hour PCR measurements were assessed (this determined whether the final outcomes of the patient were CR, PR, or TF). As shown in Figure 2, according to the χ2 test, the cohort with unreliable spot PCR was more likely to have experienced TF and less likely to have experienced CR, compared with the cohort whose spot PCRs were reliable (P = 0.032), or compared with the combined cohorts of reliable and problematic PCRs (P = 0.024).

Figure 2

Outcome of induction therapy defined as complete remission (CR), partial remission (PR), or treatment failure (TF) according to whether the spot protein/creatinine ratio (PCR) of the patient during induction therapy were reliable, problematic, or unreliable. As shown, those with unreliable spot PCRs were significantly less likely to have achieved CR and significantly more likely to have TF compared with those whose spot PCRs were reliable (P = 0.032), or compared with the combined cohort of reliable + generally reliable (P = 0.024).

Discussion

It is well established that spot PCR testing often produces a highly variable estimate of 24-hour P or 24-hour PCR.6, 8, 9, 10, 11 This is most clearly shown in calibration plots from cross-sectional studies in which a gold standard estimate of 24-hour P (24-hour PCR) is displayed in relationship to the ratio: spot PCR/24-hour PCR.6, 8, 9 In these studies, spot PCRs were determined in morning collections provided at the time of the clinic visit of the patient, which was a morning visit.6, 8, 9, 12 Based on these cross-sectional studies, it could be anticipated that if the spot PCR is used to guide clinical management, serious errors could occur because of the marked variability of the spot PCR estimate of the 24-hour PCR. However, what could not be determined from previous work was whether spot PCR unreliability was a pervasive problem or confined to certain individuals. The present work was the first to use the spot PCR to test this. This test was in the context of the 103 ACCESS patients who provided concurrent spot and 24-hour urine collections up to 15 months of follow-up in this prospective, placebo-controlled, double-blinded randomized trial of abatacept in patients with World Health Organization class III or IV LN.

We found that when spot PCRs were examined in individual patients and over time, 3 distinct patterns emerged: spot PCRs that were either reliable, problematic (generally reliable but with notable exceptions), or unreliable. Most of the LN patients were in the problematic and unreliable categories.

A unique strength of the present work was that the comparison of the spot PCR to the 24-hour PCR was longitudinal and involved multiple measurements per patient. Previous similar studies involved only a few PCR measurements per patient.16, 17, 18 Therefore, in the present work, it was possible to determine how the variability of spot PCRs compared with that of 24-hour PCRs over time, over a wide range of spot PCR and 24-hour PCR values, and from patient to patient.

A limitation of the present work was that, according to the ACCESS protocol, the comparison of the spot PCR to the 24-hour PCR trend line was not the result of spot and 24 hour urine collections made on the same day. However, we suggest that this concern did not invalidate our conclusions. Three independent studies in which spot PCR and 24-hour PCR were compared “head to head” already showed that the spot PCR is a highly unreliable estimate of the 24-hour PCR.8, 9 The present work simply confirmed the unreliability, and, most importantly, showed for the first time that being classified as unreliable is a characteristic of a subset of patients.

In performing our analysis of spot PCR variability, we did not use correlational analyses such as Bland-Altman or a calibration plot. This would have obscured the fact that unreliable spot PCRs belong only to a subset of the patients.

Our case-by-case analysis of the metrics of unreliable spot PCRs showed that the degree of unreliability varied widely not only from patient to patient, but also within patients. For example, it was common for the patients who were classified as unreliable patients to manifest unreliable spot PCRs for ≥3 months. This means that, if a dubious spot PCR was encountered, the strategy of simply repeating a spot PCR in the hope that it would “regress to the mean” would not be a consistent winning strategy. Also, we showed that the amount by which the spot PCRs either overestimated or underestimated 24-hour PCR was large and could easily confound management. This was simulated in the following scenarios.

Scenario 1 describes a patient in whom serial spot PCRs showed dramatic improvement in proteinuria, and the improvement was consistent for ≥3 consecutive months (see Figure 1, Unreliable cases 1, 3, 5). On this basis, the managing clinician decided to more rapidly taper therapy and change follow-up from monthly to bimonthly. This could result in LN relapse, which would not be discovered until >5 months later. This mismanagement likely would have been avoided if the patient had been monitored with 24-hour PCR.

Scenario 2 describes a patient in whom serial spot PCRs showed dramatic worsening, which continued for 3 or 4 months (see Figure 1, cases 2, 4). On this basis, the managing clinician decided to intensify therapy. This would needlessly expose the patients to the risks of excessive immunosuppressive and glucocorticoid therapy.

With regard to the mechanism of spot PCR variability, Koopman showed that the changes in the spot PCRs are related to changes in the protein excretion rate, not the creatinine excretion rate. Also, Koopman’s patients did not receive medications, were on a constant diet, and at complete bed rest. Therefore, unreliable spot PCRs are not explained by steroid or other therapy, diet, or exercise. Also, ACCESS was a multicenter trial with a standard protocol for specimen collection and handling, and a central laboratory for testing. Finally, spot PCR results are regarded as independent of whether the specimen is concentrated or diluted, because any change in urine volume will affect the concentration of urine protein and creatinine proportionately. Therefore, these factors did not explain spot PCR variability.

The key finding of this work is that unreliable and/or problematic spot PCRs are a characteristic of a large subset of patients. The combined problematic and unreliable cohorts made up the majority of the patients in the ACCESS trial.

We suggest that the present work made a strong case that LN patients should be followed with 24-hour PCR testing, not spot PCR testing. The key arguments include:

There is no way to predict whether spot PCR results will be unreliable in a given patient.

It is a flawed strategy to simply repeat the spot PCR if spot PCR testing provides a dubious result (see Supplementary Figure S1).

The informed LN patient would not wish to be exposed to the risks described in Scenarios 1 and 2, simply to avoid the inconvenience of an intended 24-hour urine collection.

Repeat spot testing is not simple. The logistics (loss of work, and need for child care, travel, parking, and so on) are complex, and often result in long delays.

It is a common misconception that urine collections need to be refrigerated15, 19, 20, 21 and that spot PCR testing is less expensive than 24-hour PCR testing. Neither is true. However, some laboratories do charge for handling the 24-hour urine container.

Intended 24-hour urine collections provide valuable information on nutrient management, particularly sodium, potassium, and protein intake, which cannot be reliably obtained by spot urine testing.19, 20, 21

An alternative to intended 24-hour urine collections is a first morning void that represents a complete overnight collection. This underestimates PCR by approximately 20%. Of greater concern is that an overnight collection is a relatively short collection, so it is more susceptible to variability. For example, variations in the diet of the patient (the evening meal is high in salt and/or protein), variations in depth of sleep, or the occurrence of nocturia might substantially affect the PCR of an overnight collection. Also, contamination of urine with semen, which has a high albumin concentration, is a potential source of variability in urine protein in a first morning void collection.

In summary, clinical decisions in LN patients should not rely on spot PCR testing. The preferred method is an intended 24-hour urine collection that is at least 50% complete based on its creatinine content. Whether these recommendations apply to other glomerulopathies remains to be determined.

Disclosure

All the authors declared no competing interests.