Urinary Biomarkers may Complement the Cleveland Score for Prediction of Adverse Kidney Events After Cardiac Surgery: A Pilot Study.

BACKGROUND: The ability of urinary biomarkers to complement established clinical risk prediction models for postoperative adverse kidney events is unclear. We assessed the effect of urinary biomarkers linked to suspected pathogenesis of cardiac surgery-induced acute kidney injury (AKI) on the performance of the Cleveland Score, a risk assessment model for postoperative adverse kidney events.
METHODS: This pilot study included 100 patients who underwent open-heart surgery. We determined improvements to the Cleveland Score when adding urinary biomarkers measured using clinical laboratory platforms (neutrophil gelatinase-associated lipocalin [NGAL], interleukin-6) and those in the preclinical stage (hepcidin-25, midkine, alpha-1 microglobulin), all sampled immediately post-surgery. The primary endpoint was major adverse kidney events (MAKE), and the secondary endpoint was AKI. We performed ROC curve analysis, assessed baseline model performance (odds ratios [OR], 95% CI), and carried out statistical reclassification analyses to assess model improvement.
RESULTS: NGAL (OR [95% CI] per 20 concentration-units wherever applicable): (1.07 [1.01-1.14]), Interleukin-6 (1.51 [1.01-2.26]), midkine (1.01 [1.00-1.02]), 1-hepcidin-25 (1.08 [1.00-1.17]), and NGAL/hepcidin-ratio (2.91 [1.30-6.49]) were independent predictors of MAKE and AKI (1.38 [1.03-1.85], 1.08 [1.01-1.15], 1.01 [1.00-1.02], 1.09 [1.01-1.18], and 3.45 [1.54-7.72]). Category-free net reclassification improvement identified interleukin-6 as a model-improving biomarker for MAKE and NGAL for AKI. However, only NGAL/hepcidin-25 improved model performance for event- and event-free patients for MAKE and AKI.
CONCLUSIONS: NGAL and interleukin-6 measured immediately post cardiac surgery may complement the Cleveland Score. The combination of biomarkers with hepcidin-25 may further improve diagnostic discrimination. © The Korean Society for Laboratory Medicine.

INTRODUCTION

Acute kidney injury (AKI) is a common and serious complication of cardiac surgery with incremental, stage-dependent worsening of prognosis [1]. The most recent clinical practice guidelines for AKI highlight the importance of earliest possible detection of AKI and adjustment of treatment accordingly [2]. In patients undergoing cardiac surgery, treating physicians may consult clinical variable-based risk assessment models to predict postoperative adverse kidney events, especially the widely adopted Cleveland Score [34]. However, additional pathophysiological information derived from kidney biomarker analysis and clinical information are needed, as risk assessment based solely on preoperative information may not be sufficiently accurate. Additionally, an established clinical risk model combining preoperative and intraoperative information on kidney risk/response is not yet available.

Recently, kidney injury biomarkers have been included in AKI definition and risk assessment to complement established renal functional criteria [5]. However, a candidate biomarker should be able to improve a reference model in order to be of diagnostic or prognostic benefit [6]. Oxido-inflammatory stress and iron metabolism are involved in the pathogenesis of cardiac surgery-associated AKI [17]. Given that, in a single patient, several pathomechanisms may be simultaneously active to cause AKI, it may be reasonable to analyze biomarkers in the urine linked to oxido-inflammatory stress and iron metabolism, such as interleukin-6, neutrophil gelatinase-associated lipocalin (NGAL), hepcidin-25, and alpha-1 microglobulin.

Interleukin-6 is a proinflammatory acute phase response cytokine that was recently found to be elevated in the urine of patients with acute tubular injury [89]. Both NGAL [10] and hepcidin-25 are regulators of tubular iron metabolism [711]. Alpha-1 microglobulin is another member of the lipocalin superfamily involved in heme degradation, and, when found increased in urine may indicate proximal tubular injury [12]. Midkine cannot be filtered through the glomerular basement membranes, pointing to generation through tubular injury induced by ischemia and hypoxia when found in urine [1314]. Postoperatively, increased urinary concentrations of hepcidin-25 were found in non-AKI patients suggesting renal-protective capability [1516].

Finally, data regarding prediction of MAKE or AKI in conjunction with the Cleveland Score is unavailable. Therefore, we aimed to identify the urinary kidney injury biomarkers or biomarker combination with the best possible additive predictive ability for adverse kidney-related events post-cardiac surgery. We hypothesized that the predictive ability of above-mentioned urinary biomarkers could improve the predictive performance of the Cleveland Score.

METHODS

Patients and setting

This exploratory ancillary study of the BIC-Multicenter Study used a cohort of 100 patients who underwent elective open-heart surgery with the use of cardio-pulmonary bypass (CPB), enrolled as a control group at the German Heart Center, Berlin, Germany (NCT00672334) from January 2009 through June 2010 (Fig. 1). Full study details have been described previously [17]. This study was approved by the Institutional Review Board of Charité University Medicine Ethics Committee, Berlin, Germany (approval no.: ZS EK 11 654/07), and written informed consent was obtained from each patient. We excluded patients undergoing emergency operations (time between hospital admission to operation <24 hours) or off-pump surgery, patients presenting with advanced chronic kidney disease (serum creatinine >300 µmol/L) or kidney transplant, patients <18 years, patients on immunosuppression medication, and those enrolled in a conflicting research study. Decisions regarding all diagnostic and therapeutic interventions were performed by the intensive care physicians, independent of this investigation.

Fig. 1

Patient flow diagram.

Study endpoints

The primary endpoint was the development of MAKE, including alternatively occurring events of RIFLE-AKI stages Injury or Failure (Risk, Injury, Failure, End-stage renal disease classification [18]), persistent AKI >48 hours, acute renal replacement therapy (RRT) initiation, and in-hospital mortality. The secondary endpoint was AKI, defined and classified by severity according to the RIFLE criteria based on increases in postoperative serum creatinine concentration compared with the preoperative baseline concentration, as well as urine output criteria [18]. We chose the RIFLE criteria because they tend to have a higher discriminative value in predicting hospital mortality in cardiac surgery patients than the Acute Kidney Injury Network or Kidney Disease Improving Global Outcome criteria [1920].

Biomarker sampling and measurement

We obtained urine at 6 hours (referred to as ‘intensive care unit [ICU] admission’) after commencement of CPB. Sampling was performed as previously described [17]. NGAL concentration (ng/mL) was measured using an ARCHITECT Analyzer (Abbott Diagnostics, Abbott Park, IL, USA). Interleukin-6 (pg/mL) and alpha-1 microglobulin (ng/mL) concentrations were determined using the Cobas e/c411 Immunoassay Analyzer Platform (Roche Diagnostics, Mannheim, Germany). ELISA kits were used to measure midkine (pg/mL; PeproTech, Hamburg, Germany) and hepcidin-25 (ng/mL, C-ELISA, Intrinsic Lifesciences, LLC, La Jolla, CA, USA), according to the manufacturer's instructions [21].

Serum creatinine was measured using the enzymatic method standardized by isotope dilution mass spectroscopy (Cobas 8000 modular analyzer, Roche Diagnostics). Laboratory investigators were blinded to the sample sources and clinical outcome.

Statistical analysis

We assessed the discriminative ability and performance of candidate biomarkers to predict study endpoints in a stepwise approach: First, the area under the ROC curve (AUC) based on biomarker concentrations and biomarker/hepcidin-25-ratios measured at ICU admission in conjunction with clinical study endpoints was calculated separately for each marker or ratio (for hepcidin-25, a marker for absence of MAKE and AKI, with reversed orientation indicated as 1-hepcidin-25).

The following variables from the Cleveland kidney risk assessment model were included into the reference logistic regression model [3]: gender, congestive heart failure defined as NYHA (New York Hearth Association classification) class 3 or 4 or left ventricular ejection fraction (LVEF) <35%, chronic kidney disease defined as preoperative creatinine >120 µmol/L, insulin-dependent diabetes mellitus, chronic obstructive pulmonary disease, history of previous cardiac surgery, and type of surgery (defined as coronary artery bypass graft [CABG], valve procedure or concomitant procedure). We did not include the variables “emergency surgery” and “use of intra-aortic balloon pump (IABP)” from the original model as all patients underwent elective open-heart surgery and no patient required IABP [3]. The predictive performance of the risk assessment models is reported as AUC for the derived multivariate scores with a 95% confidence interval (CI).

Thereafter, we sequentially included each biomarker separately into the reference model to exclude interaction. The model odds ratio (Exp [B]) was calculated to assess the ability of a biomarker to independently predict the study endpoints. The goodness of fit of each logistic regression was assessed using the Hosmer-Lemeshow test.

Reclassification statistics offer additional information not available from the AUC for the quantification of incremental improvements in multivariate model performance following the addition of a candidate biomarker to a reference model [6]. Therefore, improvements in the performance of the reference model following the addition of a urinary biomarker were evaluated by net reclassification improvement (NRI) (reported as category-free NRI [cfNRI] to overcome the shortcomings of NRI), and the integrated discrimination improvement (IDI) illustrated as risk assessment plots [22].

To better quantify how accurately the reference and reclassification model would perform with independent data, we adopted a leave-one-out cross-validation. Logarithmic transformations were applied when necessary. SPSS, version 25.0 (IBM Corp., Armonk, NY, USA) and SAS, version 9.4 (SAS Institute Inc., Cary, NC, USA) were used for statistical analysis. P<0.05 (two-sided) was considered statistically significant.

RESULTS

Patient characteristics

Patient baseline characteristics are shown in Table 1. Patients with postoperative MAKE or AKI were older and more likely to have left ventricular dysfunction than patients without MAKE or AKI. Patients with MAKE more frequently had lower baseline estimated glomerular filtration rate (eGFR) than those without MAKE. Type of surgery and gender were similar in patients with and without MAKE or AKI. AKI was associated with increased risk of acute RRT initiation (P=0.005) and in-hospital mortality (P=0.001).

Table 1

Perioperative patient characteristics

Variable	No MAKE N=91	MAKE N=9	P*	No AKI N=91	AKI N=9	P†
Age (yr)	67 (56–73)	74 (70–77)	0.013	67 (56–73)	74 (70–74)	0.015
Sex, female, N (%)	30 (33.3)	3 (33.3)	1.000	31 (34.1)	2 (22.2)	0.377
Insulin-dependent diabetes mellitus, N (%)	4 (4.4)	0 (0)	1.000	4 (4.4)	0 (0)	1.000
Arterial hypertension, N (%)	66 (72.5)	7 (77.8)	0.735	65 (71.4)	8 (88.9)	0.439
Preoperative creatinine >120 μmol/L, N (%)	8 (8.8)	4 (44.4)	0.011	10 (11.0)	2 (22.2)	0.294
Preoperative serum creatinine (μmol/L)	86.6 (77.8–101.7)	105 (77–159)	0.185	87 (78–105)	88.0 (77.0–127.5)	0.558
Preoperative eGFR mL/min (CKD-EPI)	72.8 (58.8–86.9)	53.6 (38.5–85.1)	0.057	71.9 (57.2–86.9)	57.1 (48.3–85.1)	0.226
Left ventricular dysfunction, N (%)‡	15 (16.5)	4 (44.4)	0.064	15 (16.5)	4 (44.4)	0.064
Chronic obstructive pulmonary disease, N (%)	2 (22.2)	9 (9.9)	0.257	8 (8.8)	3 (33.3)	0.058
Procedures
CABG surgery, N (%)	17 (18.7)	2 (22.2)	0.679	17 (18.7)	2 (22.2)	0.679
Valvular surgery, N (%)	46 (50.5)	3 (33.3)	0.488	45 (49.5)	4 (44.4)	1.000
CABG and valvular surgery, N (%)	21 (23.1)	4 (44.4)	0.222	22 (24.2)	3 (33.3)	0.687
Redo cardiac surgery, N (%)	27 (29.7)	1 (11.1)	0.438	26 (28.6)	2 (22.2)	1.000
Biomarker Concentrations 6 hours after commencement of CPB
NGAL (ng/mL)	7.50 (0.70–25.40)	69.10 (28.95–361.25)	0.010	7.50 (0.20–25.40)	108.30 (28.95–361.25)	0.002
Interleukin-6 (pg/mL)	6.21 (3.54–15.45)	45.59 (20.71–189.99)	0.001	6.21 (3.53–15.59)	45.59 (15.03–189.99)	0.001
Midkine (pg/mL)	126.50 (96.13–234.63)	569.50 (132.50–4,044.50)	0.011	124.75 (94.63–232.75)	569.50 (208.25–4,044.50)	0.001
Hepcidin-25 (ng/mL)	792.70 (254.70–1,565.80)	80.30 (44.40–445.20)	0.001	792.70 (254.70–1,565.80)	89.00 (56.30–501.65)	0.004
Alpha-1 microglobulin (ng/mL)	18.40 (13.20–31.30)	11.40 (6.89–29.30)	0.132	18.10 (12.10–30.10)	15.90 (8.53–53.05)	0.665
Adverse Outcome
Postoperative renal replacement therapy, N (%)	0 (0)	5 (55.6)	< 0.001	2 (2.2)	3 (33.3)	0.005
In-hospital mortality, N (%)	0 (0)	3 (33.3)	0.001	0 (0)	3 (33.3)	0.001
MAKE, N (%)	-	-	-	2 (2.2)	7 (77.8)	< 0.001
AKI, N (%)	2 (2.2)	7 (77.8)	< 0.001	-	-	-

Numbers denote median (25th–75th percentile) or N (%) where appropriate.

*P for MAKE vs. No-MAKE; †P refers to comparison between AKI and No AKI patients; ‡Congestive heart failure defined as NYHA class 3 or 4 or LVEF <35%.

Abbreviations: AKI, acute kidney injury; CABG, coronary artery bypass graft; CKD-EPI, Chronic Kidney Disease Epidemiology Collaboration equation for estimation of glomerular filtration rate; CPB, cardio-pulmonary bypass; eGFR, estimated glomerular filtration rate; MAKE, major adverse kidney events; NGAL, urine neutrophil gelatinase associated lipocalin; RIFLE, risk injury failure end-stage renal disease classification [18]; NYHA, New York Hearth Association classification; LVEF, left ventricular ejection fraction.

Discriminative performance of urinary biomarkers

For MAKE, interleukin-6 and 1-hepcidin-25 had an AUC of 0.83 (0.68–0.98) and 0.83 (0.72–0.94), respectively. For AKI, NGAL had an AUC of 0.81 (0.67–0.94), interleukin-6 0.82 (0.69–0.96), and midkine 0.83 (0.69–0.96). Alpha-1 microglobulin was not predictive for either MAKE or AKI (AUC <0.5). The AUC findings for all biomarkers were higher when the biomarker combinations were expressed as 1/hepcidin-25 ratios. NGAL/hepcidin-25 ratio was the best performing biomarker for AKI (AUC 0.89 [0.82–0.97]), while interleukin-6/hepcidin-25 demonstrated the highest discriminative performance for MAKE, with an AUC of 0.91 (0.79–1.00). The univariate AUC values are shown in Fig. 2.

Fig. 2

Ranking of assessed urinary biomarker performance according to the univariate AUC (with 95% confidence interval bars) at ICU admission for predicting (A) MAKE and (B) AKI.

Abbreviations: NGAL, neutrophil gelatinase-associated lipocalin; ICU, intensive care unit; AUC, area under the ROC curve; MAKE, major adverse kidney events; AKI, acute kidney injury.

Urinary biomarkers as independent predictors of MAKE or AKI

We found that NGAL, interleukin-6, midkine, hepcidin-25, and NGAL/hepcidin-25 were independent predictors of MAKE, while NGAL, interleukin-6, midkine, their hepcidin-25 ratios and hepcidin-25 were independent predictors of AKI using multivariate logistic regression based on the Cleveland Score (Table 2 and Supplemental Data Table S1). Alpha-1 microglobulin and alpha-1 microglobulin/hepcidin-25 were not independently associated with MAKE or AKI.

Table 2

Category-free net reclassification improvement (cfNRI) and integrated discrimination improvement (IDI) of selected urinary biomarkers measured at ICU admission

	MAKE as dependent endpoint				AKI as dependent endpoint
	Estimate (95% CI)	P	Estimate (95% CI)	P	Estimate (95% CI)	P	Estimate (95% CI)	P
	NGAL		NGAL/Hepcidin-25		NGAL		NGAL/Hepcidin-25
Exp (B)	1.003 (1.000–1.006)	0.032	2.905 (1.301–6.489)	0.009	1.004 (1.001–1.007)	0.016	3.453 (1.544–7.723)	0.003
cfNRIevents	0.333 (-0.283–0.949)	0.289	0.556 (0.012–1.099)	0.045	0.556 (0.012–1.099)	0.045	0.556 (0.012–1.099)	0.045
cfNRInonevents	0.802 (0.680–0.925)	< 0.001	0.758 (0.624–0.892)	< 0.001	0.670 (0.518–0.823)	< 0.001	0.582 (0.415–0.749)	< 0.001
cfNRI	1.136 (0.508–1.764)	< 0.001	1.314 (0.754–1.873)	< 0.001	1.226 (0.662–1.790)	< 0.001	1.138 (0.570–1.706)	< 0.001
IDIevents	0.082 (-0.132–0.295)	0.454	0.073 (-0.144–0.291)	0.509	0.162 (0.012–0.336)	0.068	0.176 (-0.001–0.352)	0.051
IDInonevents	-0.006 (-0.027–0.014)	0.550	-0.006 (-0.028–0.017)	0.625	0.015 (0.045–0.016)	0.342	-0.016 (-0.047–0.016)	0.333
IDI	0.075 (-0.139–0.290)	0.491	0.068 (-0.151–0.286)	0.543	0.147 (0.029–0.324)	0.102	0.160 (-0.019–0.339)	0.081
AUCdifference	0.051 (-0.044–0.147)	0.291	0.049 (-0.078–0.175)	0.449	0.085 (-0.049–0.220)	0.212	0.079 (-0.088–0.246)	0.351
	Interleukin-6		Interleukin-6/Hepcidin-25		Interleukin-6		Interleukin-6/Hepcidin-25
Exp (B)	1.021 (1.000–1.042)	0.047	5.533 (0.994–30.794)	0.051	1.016 (1.001–1.031)	0.032	6.092 (1.093–33.962)	0.039
cfNRIevents	0.556 (0.012–1.099)	0.045	0.556 (0.012–1.099)	0.045	0.333 (-0.283–0.949)	0.289	0.333 (-0.283–0.949)	0.289
cfNRInonevents	0.733 (0.593–0.874)	< 0.001	0.556 (0.384–0.727)	< 0.001	0.533 (0.359–0.708)	< 0.001	0.422 (0.235–0.610)	< 0.001
cfNRI	1.289 (0.728–1.850)	< 0.001	1.111 (0.541–1.681)	< 0.001	0.867 (0.226–1.507)	0.008	0.756 (0.112–1.399)	0.021
IDIevents	0.122 (-0.121–0.364)	0.325	0.097 (-0.144–0.339)	0.429	0.138 (-0.067–0.343)	0.186	0.173 (-0.028–0.373)	0.091
IDInonevents	-0.009 (-0.038–0.021)	0.567	-0.011 (-0.035–0.013)	0.382	-0.009 (-0.033–0.015)	0.463	-0.013 (-0.039–0.013)	0.331
IDI	0.113 (-0.131–0.357)	0.363	0.087 (-0.156–0.329)	0.484	0.129 (-0.077–0.335)	0.219	0.160 (-0.043–0.362)	0.122
AUCdifference	0.089 (-0.006–0.183)	0.065	0.064 (-0.068–0.197)	0.342	0.085 (-0.007–0.177)	0.069	0.114 (0.007–0.220)	0.036
	Midkine		Midkine/Hepcidin-25		Midkine		Midkine/Hepcidin-25
Exp (B)	1.001 (1.000–1.001)	0.031	1.115 (0.990–1.256)	0.072	1.001 (1.000–1.001)	0.020	1.163 (1.001–1.350)	0.048
cfNRIevents	0.111 (-0.538–0.760)	0.737	0.333 (-0.283–0.949)	0.289	0.333 (-0.283–0.949)	0.289	0.111 (-0.538–0.760)	0.737
cfNRInonevents	0.756 (0.620–0.891)	< 0.001	0.711 (0.566–0.856)	< 0.001	0.733 (0.593–0.874)	< 0.001	0.578 (0.409–0.746)	< 0.001
cfNRI	0.867 (0.203–1.530)	0.010	1.044 (0.412–1.677)	0.001	1.067 (0.435–1.698)	0.001	0.689 (0.018–1.360)	0.044
IDIevents	0.073 (-0.042–0.188)	0.213	0.013 (-0.174–0.200)	0.890	0.071 (-0.089–0.232)	0.383	0.119 (-0.069–0.306)	0.215
IDInonevents	-0.007 (-0.035–0.021)	0.603	-0.006 (-0.026–0.015)	0.587	-0.004 (-0.029–0.021)	0.752	-0.009 (-0.033–0.015)	0.468
IDI	0.066 (-0.053–0.184)	0.277	0.008 (-0.181–0.196)	0.938	0.068 (-0.095–0.230)	0.415	0.110 (-0.079–0.299)	0.255
AUCdifference	0.065 (-0.025–0.156)	0.157	0.056 (-0.061–0.173)	0.352	0.058 (-0.045–0.161)	0.269	0.081 (-0.023–0.186)	0.128
	Alpha-1 Microglobulin		Alpha-1 Microglobulin/Hepcidin-25		Alpha-1 Microglobulin		Alpha-1 Microglobulin/Hepcidin-25
Exp (B)	0.993 (0.972–1.014)	0.493	1.676 (0.443–6.335)	0.447	0.979 (0.933–1.027)	0.383	0.996 (0.212–4.677)	0.996
cfNRIevents	0.333 (-0.283–0.949)	0.289	-0.111 (-0.760–0.538)	0.737	-1.000 (-1.000– -1.000)	0.317	0.556 (0.012–1.099)	0.045
cfNRInonevents	-0.209 (-0.410–-0.008)	0.042	0.055 (-0.150–0.260)	0.600	0.253 (0.054–0.452)	0.013	0.187 (-0.015–0.389)	0.070
cfNRI	0.125 (-0.523–0.773)	0.706	-0.056 (-0.737–0.625)	0.872	-0.747 (-0.956– -0.549)	< 0.001	0.742 (0.163–1.322)	0.012
IDIevents	-0.008 (-0.053–0.036)	0.710	-0.025 (-0.071–0.021)	0.286	-0.030 (-0.049– -0.010)	0.003	0.009 (-0.049–0.067)	0.766
IDInonevents	0.000 (-0.007–0.007)	0.962	0.004 (-0.004–0.013)	0.284	0.003 (-0.001–0.006)	0.135	0.003 (-0.015–0.020)	0.779
IDI	-0.009 (-0.053–0.036)	0.708	-0.021 (-0.068–0.026)	0.387	-0.027 (-0.047– -0.007)	0.007	0.011 (-0.049–0.072)	0.715
AUCdifference	-0.016 (-0.061–0.029)	0.488	0.001 (-0.016–0.018)	0.889	-0.031 (-0.049– -0.012)	0.001	0.016 (-0.043–0.074)	0.595

Goodness of fit for reference models: MAKE: Hosmer-Lemeshow P=0.864, -2 Log likelihood 51.33, Nagelkerke R2 0.195, AUC 0.84; AKI: Hosmer-Lemeshow P=0.968, -2 Log likelihood 45.28, Nagelkerke R2 0.311, AUC 0.79.

Abbreviations: AKI, acute kidney injury; MAKE, major adverse kidney events; AUC, area under the curve; CI, confidence interval; cfNRI, category-free net reclassification improvement; Exp(B), Logit coefficient expressed as odds ratio for one performance unit; NGAL, neutrophil gelatinase-associated lipocalin; IDI, integrated discrimination improvement.

Estimation of improvement of the Cleveland Score with added urinary biomarker data

The cross-validated baseline performance characterized by AUC for the Cleveland reference models was 0.84 (0.73–0.96) for MAKE and 0.79 (0.66–0.91) for AKI. The goodness of fit indicated good calibration of the reference kidney risk assessment model for both endpoints (MAKE: Hosmer-Lemeshow, P= 0.864; -2 Log likelihood, 51.33; AKI: Hosmer-Lemeshow, P= 0.968; -2 Log likelihood, 45.28).

The AUC performance of the new model compared with the reference model with 95% CI (orange) is illustrated in Fig. 3. Generally, the addition of urinary biomarkers NGAL, interleukin-6, and midkine, as well as corresponding 1/hepcidin-25 ratios improved the reference model for MAKE and AKI, while alpha-1 microglobulin did not improve model performance (Table 2).

Fig. 3

Ranking of kidney risk prediction model performance to predict (A) MAKE and (B) AKI according to the area under the ROC curve (AUC with 95% confidence interval [CI] bars) with added urinary kidney injury biomarker at ICU admission (new model) and without (reference model [3], 95% CI highlighted orange).

Abbreviations: MAKE, major adverse kidney events; AKI, acute kidney injury; NGAL, neutrophil gelatinase-associated lipocalin; AUC, area under the ROC curve.

Risk assessment plots illustrating the performance of the reference kidney risk model and the new model are shown in Fig. 4 and Supplemental Data Fig. S1 and S2. Interleukin-6 improved MAKE prediction for events and non-events; however, for AKI this was only true for non-events. In contrast, NGAL improved cfNRI-based AKI prediction of the Cleveland Score for events and non-events; however, for MAKE this was only true for non-events. Only the addition of the NGAL/hepcidin-25 ratio improved the reference model for both MAKE and AKI events and non-events. Midkine and the midkine/hepcidin-25 ratio improved the cfNRI-based prediction of the reference model only for non-events of MAKE and AKI. Finally, IDI, which takes into account the magnitude of changes in predicted risk, showed no improvement for all biomarkers for both endpoints.

Fig. 4

Risk assessment plots showing the changes in model performance. Compared with AUC graphs, risk assessment plots illustrate information for events and non-events separately, representing the preferences and drawbacks of the reclassified risk models (●nonevents, ■events, solid lines) calculated by the addition of urinary biomarker concentrations (NGAL, interleukin-6, NGAL/hepcidin-25, interleukin-6/hepcidin-25) to the reference model (○nonevents, □events, dashed lines). □■ represent model sensitivity (Y-axis) versus the calculated risk (X-axis) for those with the event. ○● represent 1-specificity (Y-axis) versus the calculated risk (X-axis) for those without an event (endpoints MAKE, AKI).

Abbreviations: AUC, area under the ROC curve; MAKE, major adverse kidney events; AKI, acute kidney injury; NGAL, neutrophil gelatinase-associated lipocalin; IL-6, interleukin-6.

DISCUSSION

This pilot study used established reclassification metrics to assess the incremental value of various urinary biomarkers added to the Cleveland Score to predict MAKE and AKI after cardiac surgery [3]. We found that urinary NGAL, interleukin-6, and their 1/hepcidin-25-ratios are independent predictors that improve cfNRI-based MAKE- and AKI-prediction when added to the Cleveland Score. Notably, biomarker-related improvement was particularly dependent on correctly reclassifying patients without the event of interest.

Several statistical proposals have been made to assess the additional benefit of predictive biomarkers. Comparative metrics (AUC, cfNRI, IDI) are differently sensitive to detecting small improvements. Specifically, we found that high cfNRInonevent values indicate the ability of a biomarker to correctly decrease risk estimates for non-events. Thus, they are useful for ruling out adverse kidney events [23]. In contrast, IDI and cfNRIevent are presumably confounded by low event numbers for MAKE and AKI [24].

As no single kidney biomarker will meet every requirement regarding the underlying pathophysiological mechanisms of AKI, we assessed the predictive performance of several urinary biomarkers representing different direct or indirect components of tubular stress or damage [1325] or potential renal protective mechanisms [1516]. We found increased postoperative urinary hepcidin-25 concentrations in non-AKI patients, whereas the concentrations remained low in patients with subsequent AKI, in line with previous studies [151626]. We hypothesize that the improved predictive ability of NGAL and interleukin-6 expressed as 1/hepcidin-25 ratio vs NGAL and interleukin-6 alone may be explained by the role of hepcidin-25 in the metabolism of labile-iron compounds in the presence and absence of AKI [15262728]. During cardiac surgery, unbound labile iron-related injury maintained by oxido-inflammatory stress may be of specific pathophysiological importance [7]. The involvement and contribution of interleukin-6 to inflammatory response-pathways have also been implicated in the pathogenesis of ischemic AKI [29]. We believe that interleukin-6, a pro-inflammatory cytokine detected in urine, in particular, may serve as an indicator of tubular oxido-inflammation in cardiac surgery associated kidney injury. We were also able to extend previous findings on midkine as a potentially valuable predictor of MAKE and AKI [14]. Finally, the weak discriminatory ability of alpha-1 microglobulin for MAKE and AKI after cardiac surgery may be related to other predominant pathogenic tubular stress factors being effective in cardiac surgery compared with critically ill patients for whom alpha-1 microglobulin was previously found to be of value [12].

Perioperative risk assessment provides an opportunity for early diagnosis of adverse kidney events and early implementation of interventional strategies [3031]. Our finding regarding improved biomarker risk assessment after cardiac surgery carries the potential of timely implementation of AKI care bundles, triggering diagnostic and therapeutic modifications [32], which are associated with favorable outcomes [30]. Importantly, given the pronounced improvement in non-event prediction, our findings may facilitate withholding unnecessary diagnostic or therapeutic measures. Considering the performance and immediate availability of the assessed biomarkers in clinical practice, NGAL and interleukin-6, both measurable using clinical laboratory platforms, appear to be the most promising candidates for implementation in kidney risk assessment at the bedside. Finally, our findings on alpha-1 microglobulin are in line with previous findings and do not support further investigation of this biomarker in the cardiac surgery setting [33].

Our study has several strengths and limitations. We investigated typical patients at risk of adverse outcome in a relatively homogenous and well-defined patient cohort after cardiac surgery. Well-calibrated statistical and clinical risk assessment models enabled us to identify specific predictive patterns such as biomarkers improving prediction of non-events. In patient cohorts with only a few MAKE or AKI events, large sample sizes are necessary to derive reliable performance estimates, particularly for the estimation of classification errors. Thus, the limited sample size of this study is clearly a limitation. However, the apparent improvement in the accuracy of prediction models with small sample sizes may be potentially confounded by model overfitting. Hence, we utilized cross-validated estimates. As expected, model accuracy (AUC) was decreased, and the cross-validated measures were more conservative in detecting model improvement [34]. Weaker performance in reclassifying “events” may also reflect the limitations of the markers themselves or may be, at least partly, explained by “imperfect” clinical endpoints, which commonly depend on changes in serum creatinine, a poor reference standard [35]. Finally, the present study does not preclude the conclusion that no other kidney biomarkers may improve AKI risk prediction, as previously shown [36].

Early recognition of AKI or subclinical AKI using NGAL or interleukin-6 analysis at the bedside may help guide early risk-stratified interventions, such as implementation of the Kidney Disease Improving Global Outcomes (KDIGO) care bundles, and may also prevent unnecessary care for event-free patients [2430]. Future studies should clarify whether such clinical kidney risk assessment and interventions guided by readily available biomarkers can improve outcomes.

In summary, biomarkers from multiple, biologically linked pathways related to iron metabolism and/or inflammation are associated with the risk of adverse kidney related outcomes [3738]. In this pilot study, we found urinary NGAL and interleukin-6 to be the most promising candidates for further clinical implementation research. The combination of biomarkers with hepcidin-25 may further improve diagnostic discrimination. Further research into the reno-protective abilities of hepcidin-25 should be conducted.