Sensitivity and specificity of procalcitonin in predicting bacterial infections in patients with renal impairment.

BACKGROUND: It is unclear whether procalcitonin is an accurate predictor of bacterial infections in patients with renal impairment, although it is used as a biomarker for early diagnosis of sepsis. We determined the sensitivity, specificity, positive and negative predictive values, accuracy and best predictive value of procalcitonin for predicting bacterial infection in adult patients with severe renal impairment.
METHODS: Retrospective study at a single-center community teaching hospital involving 473 patients, ages 18-65, with Modification of Diet in Renal Disease eGFR ≤30 ml/min per 1.73 m(2), admitted between January 2009 and June 2012, with 660 independent hospital visits. A positive or negative culture (blood or identifiable focus of infection) was paired to the highest procalcitonin result performed 48 hours before or after collecting the culture.
RESULTS: The sensitivity and specificity to predict bacterial infection, using a procalcitonin level threshold of 0.5 ng/mL, was 0.80 and 0.35 respectively. When isolating for presence of bacteremia, the sensitivity and specificity were 0.89 and 0.35 respectively. An equation adjusting for optimum thresholds of procalcitonin levels for predicting bacterial infection at different levels of eGFR had a sensitivity and specificity of 0.55 and 0.80 respectively.
CONCLUSIONS: Procalcitonin is not a reliably sensitive or specific predictor of bacterial infection in patients with renal impairment when using a single threshold. Perhaps two thresholds should be employed, where below the lower threshold (i.e. 0.5 ng/mL) bacterial infection is unlikely with a sensitivity of 0.80, and above the higher threshold (i.e. 3.2 ng/mL) bacterial infection is very likely with a specificity of 0.75.

The measurement of serum concentrations of procalcitonin is a comparatively new and important laboratory tool for predicting the presence of a bacterial infection in patients presenting with sepsis. It has been proposed that the use of procalcitonin can reduce inappropriate antibiotic exposure by ruling out bacterial infection more reliably and accurately than previously used measures including C-reactive protein or white blood cell count [1-4]. Conflicting reports suggest that antibiotics might be used more frequently as a consequence of discovering elevated serum procalcitonin levels in patients, with other data demonstrating no increase in survival, prolonged admission, harm related to antibiotic side effects, and increasing antibiotic resistance [5]. Procalcitonin levels are increased in inflammatory conditions in the absence of bacterial infection, such as acute pancreatitis [6]. These findings are relevant to patients with renal impairment, which is regarded as a proinflammatory state [7]. Therefore renal impairment may affect serum procalcitonin levels, irrespective of the presence or absence of bacterial infection [8-10].

Several studies have evaluated the specificity and sensitivity of procalcitonin in the general population, without regard for estimated glomerular filtration rate (eGFR) [11-14]. One study by Herget-Rosenthal et al [15] demonstrated an increase in procalcitonin levels in correlation to chronic kidney disease stage, peritoneal dialysis, cardiovascular disease, and oliguria. A recent meta-analysis by Lu et al [16], which reviews 7 diagnostic studies and evaluates 803 patients, demonstrates a hierarchical summary receiver operating characteristic-bivariate pooled sensitivity estimate of 73%; however, the applicability of these results remain in question secondary to the wide diversity of the populations evaluated in the various studies. Schuetz et al [1] reported that the mean procalcitonin level in uninfected patients undergoing hemodialysis was 0.49 ± 0.36 ng/mL, compared with 0.12 ± 0.06 ng/mL in uninfected patients with mild to advanced chronic renal failure. One theory to explain this finding is that the elevated procalcitonin level may be related to the crossing of endotoxins or endotoxin fragments across high- and low-flux dialysis membranes, leading to activation of monocytes and production of proinflammatory cytokines, which in turn may stimulate the release of procalcitonin [17-19]. This same study claimed a 97% sensitivity and specificity of procalcitonin levels to predict bacterial infection in 35 patients with chronic renal failure, end-stage renal disease, renal transplants patients, and those with varying comorbidities, using a cutoff of 1.5 ng/mL; however, there are many logistical and statistical questions regarding the generalized application of this result [1].

We investigated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), best predictive value, and accuracy of procalcitonin in detecting bacterial infections in patients with renal impairment, specifically in those with modification in diet in renal disease (MDRD) eGFR ≤30 mL/min per 1.73 m2 during their hospital stay [20]. We performed separate subgroup analysis on patients with bacteremia vs those without any infection and attempted to correlate procalcitonin levels to eGFR and in an attempt to improve the sensitivity and specificity of procalcitonin in patients with renal insufficiency.

MATERIALS AND METHODS

Study Population

A single-center retrospective study was performed between January 2009 and June 2012, and 473 patients who were admitted to a community teaching hospital were included in the study, with 660 independent hospital visits. Multiple admissions from a single patient were treated independently. Entry criteria included males and females between 18 and 65 years old admitted to the hospital with MDRD eGFR ≤ 30 mL/min per 1.73 m2 with a documented procalcitonin level and bacterial culture.

Study Design

Data were collected from hospital electronic medical records. The investigators of this study performed the data management and statistical analyses.

Diagnosis of an infection was defined as the presence of an organism on culture that was confirmed to be clinically relevant and not a contaminant by chart review. A retrospective review of clinical notes pertaining to that admission was performed by a physician who was blinded to procalcitonin values, and various factors were considered, including the number of colonies of the organism, culture source, the presence of clinical manifestations of infection at the culture site, clinical documentation from attending physicians, and whether the organism is generally considered a contaminant when isolated from the culture site. Clinically significant organisms were most commonly isolated from blood cultures, catheter tip cultures, respiratory cultures, urine cultures, bodily fluid cultures most commonly gastrointestinal, and wound cultures.

Intervention and Measurement of Procalcitonin

Patients were divided into 2 groups. Group A includes those with bacterial infections including pneumonia, cystitis, pyelonephritis, bacteremia, spontaneous bacterial pneumonia, and skin and soft tissue infections. Group B includes those without a bacterial infection using definitions as described above. Because the half-life of procalcitonin is 26–33 hours in patients with GFR ≤ 30 mL/min [21], the highest procalcitonin result 48 hours before and 48 hours after a culture was used for analysis. If no procalcitonin level was available in that time frame, the nearest procalcitonin level was used. Using this criterion, as seen in Figure 1, 50 of a total of 660 visits were missing a procalcitonin level in the 96-hour window.

Fig. 1.

Categorization of Group A and Group B.

Because the literature reports various thresholds for procalcitonin to suggest the presence of bacterial infections, we calculated the sensitivity, specificity, PPV, NPV, and accuracy of procalcitonin levels at several thresholds of 0.5, 1, 1.5, and 2 ng/mL [22]. A subgroup analysis was done of patients with bacteremia vs those without any infection. In addition, we included a separate analysis correlating procalcitonin levels to eGFR, and we created an equation to attempt to adjust the best predictive values of procalcitonin for changing eGFR.

Statistical Analysis

Data are presented as a number observed with the percentage of the group in parenthesis for discrete variables, or as a median with the interquartile range in brackets for continuous variables. The interquartile range serves as a measure of dispersion in nonnormal data with skewed distribution. Comparisons between 2 groups were performed using Wilcoxon rank-sum test or Fisher's exact method. Diagnostic accuracy was summarized by presenting the sensitivity, specificity, PPV, NPV, and accuracy. The area under the curve of the receiver operator characteristics curve was used along with other metrics to summarize the effectiveness of laboratory values in predicting infections. A best-fit line was used to summarize the relationship between procalcitonin levels and eGFR using a least-squared method. All statistical tests were 2-tailed, and a P value of < .05 was considered statistically significant. Statistical analysis was performed in the R Statistical Computing Environment, version 2.15 (R Core Team 2012).

Role of the Funding Source

This study received no funding, and the design and conduct were not influenced by external sources. The study was conducted with the permission of the Institutional Review Board and a waiver of informed consent was granted for this retrospective study, in accordance with the principles of the Declaration of Helsinki and the World Medical Association.

RESULTS

Baseline Patient Data

Six hundred and sixty adult hospital visits from patients with an eGFR ≤ 30 mL/min per 1.73 m2 were included in the analysis, with median age of 57 years old, and 54% were male. Clinically significant bacterial infection was present in 362 patients (55%) including local infections and bacteremia, and 149 patients in our study were bacteremic (22%). Baseline characteristics are presented in Table 1. Group A and Group B had statistically significant differences in procalcitonin levels (median 1.1 [4.6] ng/mL vs median 3.8 [22.8] ng/mL, P < .001).

Table 1.

Baseline Patient Characteristics*

	No Bacterial Infection (n = 298)	Bacterial Infection (n = 362)	P Value
Age	57 [14]	56 [15]	.138
Number Males	162 (54.4%)	193 (53.3%)	.788
Receiving Hemodialysis	109 (35.4%)	150 (40.5%)	.494
Received Antibiotics	193 (62.7%)	242 (65.1%)	.518
Procalcitonin	1.1 [4.6]	3.8 [22.8]	<.001
CRP	42.3 [131.8]	72.1 [191.8]	<.001
Creatinine	4.7 [5.4]	4.4 [4.7]	.161
Sepsis as admission diagnosis	48 (16.1%)	42 (11.6%)	.111
Comorbidities			.218
Gastrointestinal	81 (26.3%)	105 (28.4%)	.545
ESLD	27 (8.8%)	32 (8.6%)	.957
Nephrology	182 (59.1%)	233 (63%)	.302
Pulmonary	108 (35.1%)	122 (33%)	.567
Neurological	53 (17.2%)	72 (19.5%)	.451
Cardiac	182 (59.1%)	230 (62.2%)	.415
Endocrine	155 (50.3%)	182 (49.2%)	.768
Hematology	81 (26.3%)	109 (29.5%)	.361
Malignancy	32 (10.4%)	43 (11.6%)	.610
Autoimmune	19 (6.2%)	31 (8.4%)	.270

Abbreviations: CRP, C-reactive protein; ESLD, end-stage liver disease; IQR, interquartile range.

* Data are presented as median [IQR] and number (%).

Determining the Diagnostic Value of Procalcitonin in Predicting Bacterial Infections in Patients With Renal Impairment

Table 2 demonstrates the sensitivity, specificity, PPV, and NPV of procalcitonin levels for predicting bacterial infections at cutoff values of 0.5, 1, 1.5, and 2 ng/mL. A threshold of 0.5 ng/mL produced a fair sensitivity of 0.80; however, all thresholds were deemed unsuitable for clinical application because of the otherwise low sensitivity, specificity, PPV, NPV, and accuracy.

Table 2.

Diagnostic Value of Procalcitonin in Predicting Bacterial Infections

Procalciton in Level (ng/mL)	Sensitivity	Specificity	Positive Predictive Value	Negative Predictive Value	Accuracy
0.5	0.80	0.35	0.60	0.60	0.60
1	0.74	0.47	0.63	0.60	0.62
1.5	0.67	0.59	0.67	0.61	0.65
2	0.60	0.65	0.68	0.58	0.63

We attempted to identify the best predictive value for procalcitonin using mathematical methods as shown in Table 3. A value of 3.2 ng/mL for procalcitonin was the best threshold for predicting infection in patients with renal insufficiency because the specificity was 0.75 and PPV was 0.72; however, the ability for procalcitonin to rule out infection at this level was poor with a sensitivity of only 0.53 and an NPV of 0.57.

Table 3.

Best Predictive Value of Procalcitonin

	Thresholda (ng/mL)	AUCb	Sensitivity	Specificity	Positive Predictive Value	Negative Predictive Value	Accuracy
Procalcitonin	3.2	0.67 (0.63–0.71)	0.53	0.75	0.72	0.57	0.63

Abbreviation: AUC, area under the curve.

a Threshold developed using Youden method.

b AUC is area under curve (95% confidence interval).

Diagnostic Value of Procalcitonin in Predicting Bacteremia in Patients With Renal Impairment

Performance of procalcitonin for predicting bacterial infection in patients with MDRD eGFR ≤ 30 mL/min per 1.73 m2 improved when the analysis was limited to the presence of bacteremia vs the absence of any infection, generating a NPV at a threshold of 0.5 ng/mL of 0.87, but overall performance was still suboptimal. Table 4 demonstrates the sensitivity, specificity, PPV, and NPV of procalcitonin calculated using thresholds of 0.5, 1, 1.5, and 2 ng/mL, and for the best predictive value of 3.2 ng/mL. Of note, 14 patients with bacteremia had a procalcitonin level <0.5 ng/mL within 48 hours of a positive blood culture (P < .001). Removing those patients who received hemodialysis for either acute or end-stage renal disease from our analysis did not significantly impact the sensitivity, specificity, PPV, or NPV (see Supplementary Material, Appendix S1).

Table 4.

Summary of Laboratory Values to Predict Bacteremia

	Thresholda (ng/dL)	AUC	Sensitivity	Specificity	Positive Predictive Value	Negative Predictive Value	Accuracy
Procalcitoninb	0.5	0.76 (0.71–0.81)	0.89	0.35	0.41	0.87	0.53
	1		0.82	0.47	0.43	0.84	0.58
	1.5		0.79	0.59	0.49	0.85	0.65
	2		0.73	0.65	0.51	0.83	0.68
Procalcitonin	3.2	0.76 (0.71–0.81)	0.70	0.75	0.58	0.83	0.73

Abbreviations: AUC, area under the curve (95% confidence interval).

a Threshold developed using Youden method to quantify best predictive values.

b Commonly used procalcitonin threshold within clinical practice.

Calculating the Correlation Between Procalcitonin to Estimated Glomerular Filtration Rate

The relationship between procalcitonin levels and bacterial infections was further explored using multivariate methods. Adjusting procalcitonin for varying levels of eGFR on a discrete or continuous scale was not advantageous to predicting infections. As seen in Figure 2, we found that there was a weak relationship (correlation = 0.06) between procalcitonin levels and eGFR in both the patients with infection and those without (see Supplementary Material, Appendix S2).

Fig. 2.

Procalcitonin and estimated glomerular filtration rate (eGFR) in patients with renal insufficiency with or without infections: lines are fit using a least square method. The relationship between eGFR and procalcitonin for noninfected patients is log(procalcitonin) = 0.4–0.02*eGFR and for infected patients is log(procalcitonin) = 1.33 + 0.0004*eGFR.

An Equation to Accurately Predict Infection using Procalcitonin at Varying Levels of Estimated Glomerular Filtration Rate

We attempted to create an equation to adjust the best predictive value of procalcitonin at varying levels of eGFR. The predictive properties for infection using this equation are a sensitivity of 0.55, specificity 0.80, PPV 0.76, and NPV of 0.59 (see Supplementary Material, Appendix S3). Thus, we were unable to establish a discrete relationship between procalitonin concentration and serum eGFR.

Secondary Outcomes

One of the original suggested uses for procalcitonin was to rule out bacterial infection, to minimize the use of inappropriate antibiotics [1]. However, our study indicated that 193 patients without infection received antibiotics, equating to 63% of the study population without an infection, and 64.2% of Group B had a procalcitonin level >0.5 ng/mL, as seen in Table 5. Whether an elevated procalcitonin level was the driving factor for administering antibiotics is undetermined, but its use in this population of patients for determining whether antibiotics should be prescribed or continued is not supported by our data.

Table 5.

Categorization of Group A and B by Levels of Procalcitonin

Procalcitonin Levels (ng/mL)	Group Aa	Group Bb	P Value
Procalcitonin ≤ 0.5	71 (19.7%)	107 (35.8%)	<.001
0.5 > Procalcitonin ≤ 1.0	25 (6.9%)	33 (11%)
1.0 > Procalcitonin ≤ 1.5	26 (7.2%)	35 (11.7%)
1.5 > Procalcitonin ≤ 2.0	20 (5.5%)	20 (6.7%)
Procalcitonin > 2	219 (60.7%)	104 (34.8%)

a Culture-positive group.

b Culture-negative group.

It is also important to note that a procalcitonin level of >100 ng/mL was identified in 2 patients who were deemed by complete chart review not to have a bacterial infection.

DISCUSSION

This study does not support the use of procalcitonin as a reliable predictor of bacterial infection in patients with an MDRD eGFR ≤ 30 mL/min per 1.73 m2, and procalcitonin levels cannot be used to accurately predict the presence or absence of bacterial infection in this clinical setting. It has been hypothesized that impaired renal function represents an inflammatory state, which alone may be sufficient to elevate inflammatory markers to significant levels [22].

In the context of existing studies, this study is one of few that have attempted to calculate the sensitivity and specificity of the use of procalcitonin levels in renal failure.

Using a procalcitonin threshold of 0.5 ng/mL, the sensitivity and specificity of predicting bacterial infection in patients with MDRD eGFR ≤ 30 mL/min per 1.73 m2 was 0.80 and 0.35, respectively. A threshold of 3.2 ng/mL generated the best predictive value of procalcitonin for supporting the presence of bacterial infections, resulting in a sensitivity and specificity of only 0.53 and 0.75, respectively. Adjusting for the presence of bacteremia vs absence of any bacterial infection, overall performance at the procalcitonin using threshold of 0.5 ng/mL improved somewhat with a sensitivity of 0.89 and a NPV of 0.87, indicating that procalcitonin may be of better use in this infection state; nevertheless, the specificity of 0.35 and PPV of 0.41 limit its utility in clinical practice. This analysis may demonstrate some utility for ruling out bacteremia at a threshold of 0.5 ng/mL.

We attempted to derive an equation to adjust the optimum threshold of procalcitonin for predicting bacterial infection at different levels of eGFR; however, the sensitivity was still low at 0.55, and the specificity was 0.80, likely secondary to the weak correlation found between eGFR and procalcitonin levels in our study of 0.06.

Limitations of our study include the retrospective analysis. The timing between culture and serum procalcitonin was less exact than may have been ideal due to the retrospective nature of the study. Ideally, in a prospective study, these would have been simultaneous. Another limitation is the use of the MDRD equation, which is well suited for chronic kidney disease, but it may overestimate eGFR in patients with acute kidney injury, thereby affecting our results and the correlation of eGFR to procalcitonin levels. Ideally, we would be able to retrospectively correlate procalcitonin levels to eGFR in the separate states of acute kidney injury, chronic kidney disease, and end-stage renal disease without overlap. It should be noted, however, that in our analysis, separating out patients on hemodialysis did not significantly impact the sensitivity, specificity, PPV, or NPV. Finally, it is also possible that our design, which relied heavily on culture data, could lead to misclassification of some cases as nonbacterial infections that were actually culture-negative bacterial infections especially if there had been concomitant antibiotic use. Nevertheless, we performed a thorough chart review to maximize accurate interpretation of the data and confirmation of diagnoses to minimize the potential bias.

Based on our cohort, we conclude that procalcitonin is neither a sensitive nor specific diagnostic test for bacterial infection in patients with MDRD eGFR ≤ 30 mL/min per 1.73 m2, although the procalcitonin may be helpful in ruling out bacteremia at a threshold of 0.5 ng/mL with an NPV of 0.87. We suggest and anticipate further prospective trials to define those factors that influence the serum concentrations of this surrogate marker. Separate studies for chronic and acute kidney injury should be done, because changing kidney function may alter the kinetics of procalcitonin levels. Investigators may also wish to do a prospective study in which infection will be documented prospectively and clinically, and two cutoffs of procalcitonin in renal insufficiency can be evaluated, where any value below 0.05 ng/mL indicates bacterial infection is unlikely with a sensitivity of 0.80, and any value above 3.2 ng/mL indicates bacterial infection is likely with a specificity of 0.75. Between these 2 values, it could be suggested that the clinician should repeat and trend the procalcitonin levels. Some clinicians have used this method of interpretation of procalcitonin levels; however, use of this test in this way is not universally applied. Most importantly, correlation of all laboratory tests with the clinical situation is vital in formulating a diagnosis and therapeutic plan.

CONCLUSIONS

In summary, we have demonstrated that procalcitonin is not a reliably sensitive or specific diagnostic test for bacterial infection in patients with renal impairment when using a single threshold, although at a threshold of 0.5 ng/mL, it does have a reasonable specificity for predicting bacterial infections and a reasonable NPV for predicting bacteremia. Furthermore, procalcitonin should not be the sole deciding factor to help identify those patients with bacterial infection, especially when deciding upon use of antibiotics in these patients.

Supplementary Material

Supplementary material is available online at Open Forum Infectious Diseases (http://OpenForumInfectiousDiseases.oxfordjournals.org/).