Evaluation of 32 urine biomarkers to predict the progression of acute kidney injury after cardiac surgery.

Biomarkers for acute kidney injury (AKI) have been used to predict the progression of AKI, but a systematic comparison of the prognostic ability of each biomarker alone or in combination has not been performed. In order to assess this, we measured the concentration of 32 candidate biomarkers in the urine of 95 patients with AKIN stage 1 after cardiac surgery. Urine markers were divided into eight groups based on the putative pathophysiological mechanism they reflect. We then compared the ability of the markers alone or in combination to predict the primary outcome of worsening AKI or death (23 patients) and the secondary outcome of AKIN stage 3 or death (13 patients). IL-18 was the best predictor of both outcomes (AUC of 0.74 and 0.89). L-FABP (AUC of 0.67 and 0.85), NGAL (AUC of 0.72 and 0.83), and KIM-1 (AUC of 0.73 and 0.81) were also good predictors. Correlation between most of the markers was generally related to their predictive ability, but KIM-1 had a relatively weak correlation with other markers. The combination of IL-18 and KIM-1 had a very good predictive value with an AUC of 0.93 to predict AKIN 3 or death. Thus, a combination of IL-18 and KIM-1 would result in improved identification of high-risk patients for enrollment in clinical trials.

Introduction

Acute kidney injury (AKI) is increasing in frequency (1) and is associated with a high incidence of adverse outcomes (2). Identification of biomarkers that diagnose or predict the magnitude of AKI after cardiac surgery has been a goal of investigators for over a decade. The most well studied biomarkers are those that reflect an inflammatory process in AKI such as IL-18 (3) and biomarkers that have increased tubular cell synthesis following renal injury such as neutrophil gelatinase associated lipocalin (NGAL) (4) and kidney injury molecule 1 (KIM-1) (5). Recently, there has been an increased interest in the use of combinations of biomarkers to predict the development of AKI (6). Combinations could account for differing time courses of biomarker release (7) or they could reflect different pathophysiological mechanisms. In a recent study, the AUC values to predict AKI after cardiac surgery were 0.65 for KIM-1, 0.61 for NAG and 0.67 for NGAL. The combination of the three markers had an AUC of 0.78 to predict development of AKI (8). Biomarkers could also be added to clinical variables. Addition of L-fatty acid binding protein and N-acetyl-β-D-glucosaminidase to a clinical model improved the ability to predict the development of AKI after cardiac surgery from an AUC value of 0.79 to 0.86 (9).

Recently, the identification of biomarkers that predict the outcomes of patients with established AKI rather than the development has been highlighted. Predictive biomarkers could be used to select patients at higher risk of adverse outcomes. Identification of patients with existing AKI that will develop worsening kidney disease would enable more timely interventions. The recent KDIGO clinical practice guidelines for AKI suggest that consideration of ICU admission, renal replacement therapy and adjustments in drug dosing be made for patients with more severe AKI (10). Biomarkers have been used to predict worsening AKI among patients with AKI but these studies have attempted to predict any change in AKI (defined as worsening of AKIN stage) rather than development of severe AKI defined as stage 3 or death (11, 12). The single study that has attempted to predict the development of severe AKI at the time of AKI diagnosis demonstrated that urine NGAL had an AUC value of 0.78 although only 9 patients progressed to severe AKI (13). Interventions could be made earlier if patients at high risk of worsening AKI could be identified. Predictive biomarkers could also be used to guide enrollment in clinical trials, allowing for selection of patients most likely to benefit from intervention. Individual biomarkers have not been robust predictors of worsening AKI. While studies have used combinations of biomarkers to predict the development of AKI (6, 9, 14), fewer have used biomarker combinations to predict the worsening of AKI (12). We measured 32 candidate biomarkers in patients with stage 1 AKI after cardiac surgery to determine the ability of the biomarkers alone or in combination to predict worsening AKI.

Results

Urine samples and clinical data were collected from 95 subjects who had AKIN stage 1 AKI at the time of urine collection after cardiac surgery. Seventy-three of these patients achieved a maximum AKIN stage of 1, of whom one died; 12 had a maximum stage of 2 with 2 deaths; and 10 had a maximum stage of 3 with 6 deaths. Twenty-three patients met the combined endpoint of AKI progression (reaching AKIN stage 2 or 3), or death within 30 days of the urine sample collection. There was no difference between the outcome groups based on gender, race, comorbidities, type of surgery, baseline serum creatinine, creatinine at collection and time to collection from the time of surgery (table 1). We measured the concentration of 32 urine analytes in order to determine the ability of each biomarker to predict the combined outcome of AKI progression or death. Seven of the biomarkers had at least 19 (≥20%) samples for which the biomarker was ‘out of range low’ (OOR<), a situation in which the fluorescent signal falls below the lower asymptote of the fitted dose-response curve, thereby precluding concentration estimation. Primary analysis was done using the area under the curve (AUC) of the receiver operating characteristic (ROC) curve using a leave-one-out cross-validation approach. Initial analysis (supplemental tables 1 and 2) showed that adjustment for urine creatinine improved the ability to predict the outcome for most of the biomarkers so adjusted values are reported. In contrast to most of the markers, NGAL had slightly higher predictive values for both endpoints without adjustment. To provide an initial framework for characterization of the biomarkers, we divided the biomarkers into mechanistic groups. Table 2 shows the predictive characteristics for each of the 32 urine analytes broken down by biomarker functional category. We ranked biomarkers by their mean squared error (MSE) – lower MSE indicates better fit – and evaluated predictive performance using AUC. The highest AUC value of 0.74 was seen for IL-18 (Figure 1) and renin. KIM-1 had an AUC of 0.73 and VEGF, IL-6 and NGAL had AUC values of 0.72. For comparison, percent change in serum creatinine at the time of collection and Cleveland Clinic scores had AUC values of 0.76 and 0.64, respectively.

Table 1

Univariate associations of patient characteristics with progression status.

	Progression AKIN stage 2/3 or death
	No (n = 72)	Yes (n = 23)
Variable	Measure*	Measure*	P‡
Female gender	22 (31)	7 (30)	>0.99
African-American†	17 (24)	4 (17)	0.58
Age (years)	65.5 (31–86)	68 (41–88)	0.07
Weight (kg)†	87.3 (52.7–152.3)	88.2 (51–159)	0.89
Preoperative use of intra-aortic balloon pump	11 (15)	4 (17)	0.75
Left ventricular ejection fraction <35%	19 (26)	4 (17)	0.58
Previous cardiac surgery	14 (19)	10 (43)	0.03
Insulin-requiring diabetes mellitus	24 (33)	8 (35)	>0.99
Chronic obstructive pulmonary disease	8 (11)	3 (13)	0.72
Cardiac surgery type
Coronary artery bypass grafting (CABG)	36 (50)	7 (30)	0.15
Valve	21 (29)	6 (26)	>0.99
CABG & valve	10 (14)	7 (30)	0.11
Other	5 (7)	3 (13)	0.40
Emergency surgery	17 (24)	7 (30)	0.58
Bypass	59 (82)	20 (87)	0.75
Congestive heart failure	25 (35)	9 (39)	0.80
Bypass time (minutes)†	151 (55–383)	168.5 (49–396)	0.61
Cleveland Clinic Score	4 (0–10)	5 (0–10)	0.05
Baseline serum creatinine (mg/dl)	1.1 (0.7–2.7)	1.2 (0.7–2.4)	0.51
Collection creatinine (mg/dl)	1.6 (1–3.1)	2 (1.1–4.2)	0.06
Percent change in creatinine	35.5 (14–89)	62 (22–94)	0.0002
Time to collection (days)	0.89 (0.1–3)	1.79 (0.5–2.8)	0.09
Renal replacement therapy	0 (0)	8 (35)	<0.0001

Table 2

Predictive characteristics of molecular and clinical biomarkers for progression defined as AKIN stage 2/3 or death (includes adjustment for urinary creatinine).

					Leave one out cross-validation results
MOLECULAR MARKERS					MSE	% misclassified
Function	Name	OOR<	AUC	95% CI		Progressors	Non-progressors
Inflammation	IL-18	0	0.74	(0.60, 0.85)	0.152	39.1	27.8
	IL-6	4	0.72	(0.57, 0.84)	0.164	34.8	30.6
	VEGF	7	0.72	(0.58, 0.83)	0.168	34.8	29.2
	MCP-1	0	0.68	(0.54, 0.79)	0.172	43.5	43.1
	IL-1ra	0	0.67	(0.52, 0.79)	0.175	47.8	30.6
	IL-8	10	0.71	(0.59, 0.80)	0.176	26.1	38.9
	GRO alpha	18	0.69	(0.54, 0.81)	0.179	39.1	37.5
	LIF	41	0.62	(0.44, 0.78)	0.185	43.5	37.5
	IL-10	0	0.60	(0.46, 0.73)	0.185	39.1	47.2
	Eotaxin	19	0.62	(0.45, 0.76)	0.187	52.2	33.3
	VCAM-1	0	0.57	(0.41, 0.72)	0.189	56.5	38.9
	RANTES	63	0.45	(0.40, 0.50)	0.191	69.6	30.6
	TNF-alpha	74	0.53	(0.35, 0.70)	0.192	78.3	16.7
	MIP-1 alpha	30	0.48	(0.31, 0.65)	0.192	56.5	51.4
Hemodynamic regulation	Renin	0	0.74	(0.61, 0.85)	0.162	43.5	36.1
Injury Response (UP)	NGAL	3	0.72	(0.59, 0.82)	0.163	21.7	45.8
	KIM-1	0	0.73	(0.60, 0.83)	0.164	26.1	36.1
	L-FABP	0	0.67	(0.51, 0.80)	0.168	47.8	31.9
	HGF	0	0.66	(0.51, 0.79)	0.173	43.5	37.5
	Netrin-1	0	0.66	(0.50, 0.79)	0.174	47.8	29.2
	Clusterin	0	0.63	(0.47, 0.76)	0.180	39.1	38.9
	Fetuin-A	2	0.64	(0.48, 0.77)	0.181	56.5	18.1
Tubular Dysfunction	Cystatin-C	0	0.68	(0.53, 0.79)	0.169	52.2	31.9
	Albumin	0	0.67	(0.52, 0.79)	0.180	39.1	48.6
	Total Protein	8	0.66	(0.50, 0.78)	0.180	47.8	34.7
	Beta-2-microglobulin	35	0.59	(0.39, 0.76)	0.181	52.2	41.7
	RBP	0	0.60	(0.45, 0.72)	0.186	56.5	44.4
	alpha-1 antitrypsin	0	0.59	(0.44, 0.72)	0.186	60.9	27.8
Reactive Oxygen Species	8-Isoprostane	0	0.50	(0.34, 0.65)	0.195	87	50
Injury Response (DOWN)	TFF-3	0	0.62	(0.47, 0.75)	0.185	52.2	44.4
Injured Cell Enzymes	NAG	4	0.69	(0.55, 0.80)	0.169	47.8	44.4
Apoptosis	TRAIL	59	0.57	(0.37, 0.75)	0.188	56.5	33.3
CLINICAL MARKERS
Percent change in creatinine			0.76	(0.61, 0.86)	0.154	39.1	11.1
Cleveland Clinic Score			0.63	(0.49, 0.76)	0.184	82.6	5.6

IL-18, Interleukin 18; IL-6, Interleukin 6; VEGF, Vascular endothelial growth factor; MCP-1, Monocyte chemotactic protein-1; IL-1ra, Interleukin 1 receptor antagonist; IL-8, Interleukin 8; GRO alpha, Growth related oncogene alpha; LIF, Leukemia inhibitory factor; IL-10, Interleukin 10; VCAM-1, Vascular cell adhesion molecule-1; RANTES, Regulated on activation, normal T cell expressed and secreted; TNF-alpha, Tumor necrosis factor alpha; MIP-1 alpha, Macrophage inflammatory protein-1 alpha; NGAL, Neutrophil gelatinase associated lipocalin; KIM-1, Kidney injury molecule-1; L-FABP, Liver type fatty acid binding protein, HGF, Hepatocyte growth factor; RBP, Retinol binding protein; TFF-3, Trefoil factor 3; NAG, N-acetyl-beta-D-glucosaminidase; TRAIL, TNF-related apoptosis-inducing ligand.

Figure 1

Prediction of outcomes by IL-18 among patients with stage 1 AKI at collection. 1A. The area under the ROC curve for IL-18 to predict AKIN stage 2/3 AKI or death was 0.74. The area under the ROC curve for IL-18 to predict AKIN stage 3 AKI or death was 0.89. 1B. Box and whisker plot for patients who progressed to a maximum AKIN stage of 1, 2 or 3 and death. All patients had AKIN stage 1 at the time of urine collection. Boxes show the median value and 25th and 75th percentiles. Whiskers represent the range of values.

We next compared the ability of the biomarkers to predict the outcome of development of severe AKI (defined as AKIN stage 3), or death within 30 days. Baseline demographic and clinical characteristics were similar between groups (supplemental table 3). Change in serum creatinine and Cleveland Clinic score showed only marginal improvements in prediction but the predictive ability of many of the biomarkers was markedly improved (table 3). IL-18 was the best predictor with an AUC of 0.89 and smallest MSE. Figure 1 shows the ROC curves for prediction of both outcomes and boxplots for the values of creatinine-adjusted IL-18 for each of the eventual outcomes. These data demonstrate that the currently available biomarkers are better predictors of severe AKI than they are for the outcome of any degree of worsening in AKI and that IL-18 is an excellent predictor of severe AKI or death.

Table 3

Predictive characteristics of molecular and clinical biomarkers for progression defined as AKIN stage 3 or death (includes adjustment for urinary creatinine).

					Leave one out cross-validation results
MOLECULAR MARKERS					MSE	% misclassified
Function	Name	OOR<	AU	95% CI		Progressors	Non- progressors
Inflammation	IL-18	0	0.89	(0.75, 0.95)	0.078	30.8	23.2
	IL-6	4	0.87	(0.75, 0.93)	0.089	15.4	26.8
	VEGF	7	0.8	(0.64, 0.90)	0.096	30.8	31.7
	VCAM-1	0	0.83	(0.67, 0.92)	0.097	38.5	26.8
	MCP-1	0	0.81	(0.66, 0.90)	0.098	38.5	19.5
	IL-1ra	0	0.77	(0.58, 0.89)	0.104	38.5	13.4
	GRO alpha	18	0.77	(0.62, 0.88)	0.113	46.2	13.4
	IL-8	10	0.75	(0.60, 0.86)	0.113	38.5	42.7
	IL-10	0	0.67	(0.47, 0.82)	0.115	53.8	30.5
	LIF	41	0.67	(0.44, 0.85)	0.117	46.2	39
	Eotaxin	19	0.6	(0.37, 0.79)	0.123	53.8	32.9
	MIP-1 alpha	30	0.55	(0.39, 0.70)	0.123	53.8	48.8
	RANTES	63	0.55	(0.49, 0.61)	0.123	100	15.9
	TNF-alpha	74	0.53	(0.49, 0.57)	0.123	17.1	28.4
Hemodynamic regulation	Renin	0	0.73	(0.52, 0.87)	0.108	46.2	12.2
Injury Response (UP)	L-FABP	0	0.85	(0.65, 0.95)	0.079	23.1	20.7
	NGAL	3	0.83	(0.67, 0.93)	0.093	38.5	26.8
	Clusterin	0	0.85	(0.71, 0.93)	0.096	38.5	13.4
	KIM-1	0	0.81	(0.68, 0.90)	0.099	30.8	29.3
	Fetuin-A	2	0.79	(0.60, 0.91)	0.102	38.5	28
	HGF	0	0.82	(0.67, 0.91)	0.103	38.5	26.8
	Netrin-1	0	0.68	(0.47, 0.83)	0.118	53.8	30.5
Tubular Dysfunction	Cystatin-C	0	0.84	(0.69, 0.93)	0.091	30.8	30.5
	Beta-2-microglobulin	35	0.76	(0.51, 0.90)	0.101	46.2	40.2
	alpha-1 antitrypsin	0	0.76	(0.57, 0.88)	0.107	38.5	25.6
	Total Protein	8	0.7	(0.48, 0.86)	0.108	46.2	24.4
	Albumin	0	0.79	(0.62, 0.89)	0.109	46.2	34.1
	RBP	0	0.75	(0.57, 0.87)	0.112	38.5	23.2
Reactive Oxygen Species	8-Isoprostane	0	0.59	(0.38, 0.77)	0.122	53.8	34.1
Injury Response (DOWN)	TFF-3	0	0.75	(0.56, 0.88)	0.116	38.5	14.6
Injured Cell Enzymes	NAG	4	0.81	(0.64, 0.91)	0.095	38.5	9.8
Apoptosis	TRAIL	59	0.62	(0.35, 0.83)	0.120	53.8	22.0
CLINICAL MARKERS
Percent change in creatinine			0.79	(0.60, 0.91)	0.101	38.5	9.8
Cleveland Clinic Score			0.68	(0.53, 0.80)	0.120	53.8	22.0

To determine the relationship of individual biomarkers with each other we performed two analyses. First we performed an unsupervised cluster analysis to determine which biomarkers were similar to each other (Figure 2). We found many similarities to our a priori grouping of biomarkers but also interesting differences. Many of the proteins that we had proposed were filtered plasma proteins that are not reabsorbed in the tubule because of tubular dysfunction were grouped together (albumin, alpha-1 antitrypsin, cystatin c, beta-2 microglobulin and retinol binding protein). Similarly, many of the proteins we described as inflammatory proteins were also grouped together. However, some of the proteins that we thought would be similar to each other were clustered differently. NGAL and KIM-1, which were placed in the injury response (up) group, were geographically distant from each other in the dendogram. We also compared correlation coefficients for each of the biomarkers within the groups and with the best marker in each of the other groups (supplemental tables 4–11). Overall, the correlation within each group was stronger for biomarkers which had better predictive ability. A notable exception was KIM-1 which had a poor correlation with other markers in its group as well as with markers in other groups (supplemental table 6). KIM-1 was a strong predictor of both outcomes (AUC=0.73 and 0.81) but had a correlation coefficient of 0.20 with L-FABP and of 0.24 with IL-18, suggesting that combination of KIM-1 with one of these other markers may be beneficial.

Figure 2

Cluster dendrogram of biomarker concentrations. The creatinine adjusted biomarker concentrations were analyzed by unsupervised clustering to determine how similar values were to each other. The unsupervised clustering resulted in groupings similar to functional groupings for some of the biomarkers. For instance many of the proteins in the group associated with tubular dysfunction were clustered in the same group. A number of differences from the clustering predicted were also found. For instance NGAL and KIM-1 were clustered in different groups. IL-18, Interleukin 18; IL-6, Interleukin 6; VEGF, Vascular endothelial growth factor; MCP-1, Monocyte chemotactic protein-1; IL-1ra, Interleukin 1 receptor antagonist; IL-8, Interleukin 8; GRO alpha, Growth related oncogene alpha; LIF, Leukemia inhibitory factor; IL-10, Interleukin 10; VCAM-1, Vascular cell adhesion molecule-1; RANTES, Regulated on activation, normal T cell expressed and secreted; TNF-alpha, Tumor necrosis factor alpha; MIP-1 alpha, Macrophage inflammatory protein-1alpha; NGAL, Neutrophil gelatinase associated lipocalin; KIM-1, Kidney injury molecule-1; L-FABP, Liver type fatty acid binding protein, HGF, Hepatocyte growth factor; RBP, Retinol binding protein; TFF-3, Trefoil factor 3; NAG, N-acetyl-beta-D-glucosaminidase; TRAIL, TNF-related apoptosis-inducing ligand.

We determined the ability of combinations of biomarkers to predict the two outcomes, ranking groups of biomarkers according to MSE. The combination of IL-18 and percent change in serum creatinine (Table 4) had the lowest MSE to predict AKIN 2/3 or death (AUC = 0.80), while the combination of cystatin c and percent change in serum creatinine had the lowest MSE to predict AKIN 3 or death (AUC = 0.88). As suggested by our correlation analysis, the combination of IL-18 and KIM-1 was also a very good predictor of AKIN 3 or death, with an AUC of 0.93, a positive predictive value of 63%, and sensitivity of 77%. KIM-1 combined with L-FABP also was a strong predictor of AKIN 3 or death (AUC = 0.89) as suggested by our correlation analysis. The combination of IL-18 and percent change in serum creatinine had an excellent AUC (0.93) but a slightly lower MSE (0.074). The positive predictive value and sensitivity of this combination were 65% and 85% respectively. Supplemental tables 12 and 13 show the characteristics of each combination of biomarkers to predict the two outcomes.

Table 4

Biomarker test operating and performance characteristics for combinations.

	AUC	95% CI	MSE	Probability Threshold	T+ N (%)	PPV (D+|T+) N (%)	Sens (T+|D+) N (%)
AKIN 2/3 or DEATH
IL18 + Percent change in creatinine	0.80	(0.67, 0.89)	0.134	0.30	23 (24)	14 (61)	14 (61)
IL8 + Percent change in creatinine	0.81	(0.68, 0.89)	0.138	0.30	24 (25)	14 (58)	14 (61)
NGAL + Percent change in creatinine	0.82	(0.70, 0.90)	0.139	0.28	27 (28)	15 (56)	15 (65)
AKIN 3 or DEATH
Cystatin-C + Percent change in creatinine	0.88	(0.70, 0.96)	0.067	0.13	26 (27)	10 (38)	10 (77)
KIM-1 + IL18	0.93	(0.80, 0.98)	0.069	0.31	16 (17)	10 (63)	10 (77)
NGAL + Percent change in creatinine	0.89	(0.72, 0.96)	0.071	0.13	24 (25)	10 (42)	10 (77)
IL-18 + Percent change in creatinine	0.93	(0.79, 0.98)	0.074	0.17	17 (18)	11 (65)	11 (85)

T+ = Test positive; PPV = positive predictive value; D+|T+ = disease positive given test positive; Sens = Sensitivity; T+|D+ = Test positive given disease positive.

Discussion

We measured the ability of 32 AKI biomarkers to predict worsening of renal function in patients with AKIN stage 1 AKI after cardiac surgery. IL-18 had the highest AUC value for the prediction of both outcomes we evaluated. Most of the biomarkers were better predictors of severe AKI (AKIN 3 or death) than they were of any degree of progression (AKIN 2/3 or death). The values for prediction of AKI were similar to those seen in the literature although this is the first larger scale side-by-side comparison of the ability of these markers to predict an outcome of worsening of AKI. Koyner and colleagues compared the ability of several biomarkers to predict the progression to AKIN stage 3 AKI in patients with an increase in serum creatinine after cardiac surgery (13). They showed that π-GST adjusted for urine creatinine (AUC = 0.86) had the best performance followed by NGAL (AUC = 0.78), Cystatin-C (AUC = 0.77), HGF (AUC = 0.68), KIM-1 (AUC = 0.65) and α-GST (AUC = 0.54). They did not test IL-18 which was the best performer in our study. Because of the small numbers of patients included in the prognostic analysis in that paper (n=46), the confidence intervals were large. The current study has refined the predictive ability and compared a larger number of biomarkers although the total number of outcomes is still small.

In a larger study from the TRIBE-AKI consortium, the ability of urine NGAL, albumin to creatinine ratio, IL-18 and plasma NGAL to predict the progression to a higher AKIN stage for patients with stage 1 or stage 2 AKI at the time of sample collection was determined (12). They found the following AUC values: urine NGAL-0.58; albumin to creatinine ratio-0.67; IL-18-0.63; and plasma NGAL-0.74. Adjustment for clinical factors improved the AUC values to 0.79, 0.78, 0.77 and 0.80 respectively. A second study from TRIBE-AKI showed that combining urine concentrations of IL-18 with a clinical model improved the prediction for the development of AKI from 0.69 in the clinical model to 0.76 for the clinical model plus urinary IL-18 (15). Hall and colleagues looked at the ability of biomarkers to predict worsening AKI defined as an increased stage of AKI at the time of diagnosis with AKI in 284 patients (11). The majority of these patients were thought to have prerenal azotemia. They found that urine NGAL had the best AUC (0.71) followed by KIM-1 (0.64) and IL-18 (0.63). Adjustment for clinical factors improved the AUC values to 0.75, 0.68 and 0.66 respectively. These studies show that combinations of biomarkers with clinical factors may confer better diagnostic ability although the inclusion of a complex clinical algorithm may be cumbersome.

We combined biomarkers and clinical factors to determine if combinations improved the prediction. The combination of IL-18 and percent change in serum creatinine had the lowest MSE to predict AKIN 2/3 or death and had an AUC of 0.80. As we saw with single biomarkers, prediction of AKIN 3 or death was stronger with combinations of biomarkers than it was for AKIN 2/3 or death. The combination of IL-18 and percent change in creatinine was a very strong predictor of the severe outcome with an AUC of 0.93, sensitivity of 85% and PPV of 65%. A very accurate combination of two biomarkers to predict AKIN 3 or death was KIM-1 and IL-18 which had an AUC of 0.93 and the lowest MSE. This combination produced a positive predictive value of 63% and a sensitivity of 77%. The very strong predictive value of this combination is consistent with the excellent prediction of the individual biomarkers but a relatively weak correlation between the concentrations of the biomarkers. Biomarker combinations have been used previously to aid in the diagnosis of AKI (6, 9, 14) and they have been used in combination with clinical score to predict outcome (11–13) but biomarkers have not been used in combinations with each other to predict outcomes. While the combinations had AUC values which were nominally higher than the individual biomarkers, the predictive performance of the various marker combinations were similar as indicated by the respective AUC confidence intervals. An approach where a small number of biomarkers can be combined to produce a kidney injury risk index would facilitate more rapid clinical decision making to direct therapeutic intervention. While the predictive power of the combinations was very strong in our study, a limitation is the relatively small number of outcomes.

The use of existing biomarkers alone has not yet reached the predictive threshold to be useful in the management of patients with AKI. A shorter term need is the ability to enrich enrollment for studies of AKI therapies with a population of subjects who will go on to have adverse outcomes. We have compared the number of patients who would be identified using the biomarkers and biomarker combinations that we examined in this study. If we assume 500 events (AKIN 3 or death) are needed in a clinical trial, based on the 14% prevalence rate of progression observed in our data, 3571 unscreened patients would need to be enrolled (500/3571 = 0.14). Eighty-six percent of these patients could not benefit from the study drug because they would not reach the endpoint regardless of the treatment. The inclusion of these patients increases the cost of the study and dilutes the possibility of seeing an effect. In contrast, if we restrict trial enrollment to patients identified as high risk for AKIN 3 or death based on a combination of IL-18 and KIM-1, 794 patients would need to be enrolled in the study to have 500 patients with events based on the test’s PPV = 63% (500/794 = 0.63). Among those enrolled, only 294 (37%) would receive the therapy without a chance of benefit. Thus, the use of biomarkers to enhance enrollment could decrease the cost of clinical trials while increasing the possibility that they can show a benefit.

Methods

Study Design

Random urine samples were collected and stored as we have previously described (16) by the investigators of the Southern Acute Kidney Injury Network (SAKInet) at 4 centers (MUSC, Duke University, George Washington University and University of Tennessee, Chattanooga) between July 2008 and July 2010. Prior to collection, informed consent was obtained in accordance with the Institutional Review Board approved protocol at each institution. Identification of subjects required the availability of a study coordinator so not all eligible subjects were enrolled. Patients at these institutions who had a cardiac surgery procedure were assessed for an increase in serum creatinine of at least 50% or 0.3 mg/dl. Changes in serum creatinine were assessed by measurements from the clinical lab at the enrolling hospital and entered on the case report form. Urine samples were collected as soon as the increase in serum creatinine was recognized by the study team. In most cases this was the same day that the patient met AKI criteria. Inclusion criteria were consent by the patient or surrogate, cardiac surgery and development of AKI within 3 days of surgery. Exclusion criteria were a baseline serum creatinine greater than 3 mg/dl or heart transplant or AKI of greater than AKIN stage 1 (by creatinine criteria) at the time of collection. Urine samples were collected from 117 patients with AKI of any stage at the time of collection. Urine samples from the 95 patients in the collection at the time of analysis who had AKIN stage 1 AKI and met the inclusion and exclusion criteria were used for the current study. The primary outcome measure was AKI progression, defined as the combined endpoint of worsening of AKI (progressing from AKIN stage 1 to a higher AKIN stage) within 10 days of surgery, or death due to any cause within 30 days. The secondary outcome measure was progression to AKIN stage 3 AKI within 10 days or death within 30 days. Sixty-nine of the 95 patients had a maximum creatinine value within 3 days of sample collection. Six had a maximum creatinine value greater than 7 days postoperatively. Clinical and demographic characteristics, bypass times and outcomes of these patients were collected at each site on a case report form which was sent to the coordinating center where data were entered into a REDcap database. Urine biomarker measurements were made at MUSC as described below. The ability of each biomarker or biomarker combination to predict the endpoints was assessed. Samples from the 2 ml vial without additional freeze-thaw cycles were used.

Biomarker Measurement

Urine cytokine and protein concentrations were measured using commercially available multiplex assays and ELISAs per the manufacturer’s instructions. Measurements were made on a Bioplex 100 Suspension Array System and analyzed using Bioplex manager Software (BioRad) or a SpectraMax Plate reader and analyzed using Soft Max Pro software (Molecular Devices). All assays were performed according to instructions in the manufacturers package insert with minor modifications we have previously described (17). The performance of biomarkers was compared between groups of biomarkers that were chosen based on the presumed pathophysiological mechanism that is reflected by the urinary biomarker.

Statistical methods

All analyses were performed using R version 2.15.1 (18), including R libraries ROCR (19) and pROC (20). Demographic and clinical baseline characteristics were summarized numerically for progressors and non-progressors using frequencies and percents for categorical variables, and median and range for continuous variables. Comparisons of patient characteristics between progressors and non-progressors were performed using Fisher’s exact test or Wilcoxon rank sum test, as appropriate. Distributions of urinary markers were assessed graphically using boxplots. We examined the pairwise associations between markers based on a correlation analysis using Pearson’s correlation coefficient. We identified groups of similar markers by conducting a Euclidean distance-based unsupervised clustering analysis, and constructed a dendrogram to display identified clusters (21). The predictive performance of molecular and clinical markers was assessed using simple logistic regression (SLR) models (22), using a natural logarithm transformation of urine analyte concentrations to achieve approximate linearity on the logit scale. SLR results were used to estimate AUC and corresponding 95% confidence intervals (CIs) (23). In the absence of an independent test data set, we used leave-one-out cross validation (21) to assess model generalizability using three measures of fit: the proportion of progressors misclassified; the proportion of non-progressors misclassified; and mean squared error (MSE) (24). The latter is an average measure of the squared difference between observations’ event status (0 or 1) and the predicted probability of being an event based on the SLR model. Models with highly discriminating markers yield high probabilities for events and low probabilities for non-events, resulting in small differences and therefore small MSE values. For each leave-one-out cross validated model we further estimated AUC to assess performance. We identified the subset of markers significantly associated with progression based on a univariate false discovery rate of 0.05, and investigated the predictive performance of marker combinations from this subset using multivariable LR and leave-one-out cross validation as described. Due to the small number of events, we limited investigation to two variable models.

Finally, we note that for a number of markers, concentrations were flagged as ‘out of range low’ (OOR<), a situation in which the fluorescent signal falls below the lower asymptote of the fitted 5-parameter logistic dose-response calibration curve. In such cases, it is impossible to back-fit to estimate marker concentration; for analysis purposes, we imputed zero for these out-of-range concentrations. We conducted a small simulation study (results not shown) to evaluate the effect of this imputation on the estimation of AUC and corresponding standard error. Based on 500 simulated data sets of 23 progressors and 72 non-progressors, increasing the proportion of out of range low observations replaced with imputed values of zero resulted in biased estimation of AUC; however, the bias was always in the direction of the null. AUC’s estimated standard error was too large whenever data from non-progressors were more likely to be OOR<, a situation we observed for all markers we investigated. We therefore report both AUC and the corresponding 95% CI for all markers since the effect of imputing zero for OOR< values at worst underestimates AUC with interval estimates that are wider than they actually are (i.e. inference is conservative). For transformations in LR models with out-of-range marker concentrations, we added 0.001 to concentrations to avoid taking the logarithm of zero.