Development and Validation of a Risk Score for Prediction of Acute Kidney Injury in Patients With Acute Decompensated Heart Failure: A Prospective Cohort Study in China.

BACKGROUND: Although several risk factors for acute kidney injury (AKI) have been identified, early detection of AKI in acute decompensated heart failure patients remains a challenge. The aim of this study was to develop and validate a risk score for early prediction of AKI in acute decompensated heart failure patients. METHODS AND
RESULTS: A total of 676 consecutive acute decompensated heart failure participants were prospectively enrolled from 6 regional central hospitals. Data from 507 participants were analyzed. Participants from 4 of the 6 hospitals (n=321) were used to develop a risk score and conduct internal validation. External validation of the developed risk score was conducted in participants from the other 2 hospitals (n=186). Sequential logistic regression was used to develop and validate the risk score. The c statistic and calibration plot were used to assess the discrimination and calibration of the proposed risk score. The overall occurrence of AKI was 33.1% (168/507). The risk score, ranging from 0 to 55, demonstrated good discriminative power with an optimism-corrected c statistic of 0.859. Similar results were obtained from external validation with c statistic of 0.847 (95% CI 0.819-0.927). The risk score had good calibration with no apparent over- or under-prediction observed from calibration plots.
CONCLUSIONS: The novel risk score is a simple and accurate tool that can help clinicians assess the risk of AKI in acute decompensated heart failure patients, which in turn helps them plan and initiate the most appropriate disease management for patients in time.

Introduction

Acute decompensated heart failure (ADHF) is one of the leading causes of hospitalization. More than 30% of patients hospitalized for ADHF experience acute kidney injury (AKI),1, 2, 3, 4, 5, 6, 7, 8 which is independently associated with increased risk of morbidity and mortality.9, 10 The clinical importance of the coexistence of acute cardiac and renal dysfunction, known as acute cardiorenal syndrome (CRS), and its management have received great attention recently.11, 12, 13

The poor prognosis of acute CRS makes the identification of patients at high risk especially important. However, early diagnosis of acute CRS has remained elusive due to restrictions on using serum creatinine or urine output for diagnosis of AKI.14 The current diagnostic paradigm for AKI relies largely on biomarkers of renal function (serum creatinine and urine output) that have been in clinical use for over 50 years but are known to be insensitive and slow to change after kidney injury, often leading to a late and inaccurate diagnosis of AKI with resultant adverse outcomes.9, 10, 14 Therefore, the current paradigm of AKI diagnosis needs to be reassessed in light of scientific advances in the understanding of the pathobiology of AKI facilitated in part by recent biomarker discovery programs.15

Although it has been the goal of several studies to develop kidney injury biomarkers (such as neutrophil gelatinase‐associated lipocalin [NGAL] and urinary angiotensionogen [uAGT])13, 16, 17, 18, 19, 20, 21, 22 for early detection of AKI before occurrence of renal dysfunction, these studies have yielded varied performance. This suggests that a single biomarker (or single type of biomarkers) is not sufficient for the evaluation of clinical settings such as CRS.15, 17 Therefore, reformulating the predictive approach of AKI, ie, a multimarker approach, is more likely to be of greater use.23 The major purpose of our study is to incorporate both clinical risk factors and the novel kidney injury biomarkers for such prediction that we can predict CRS early and improve the risk reclassification in this disorder. Precisely, we will develop and validate a risk score that can identify patients with either a very high or a very low probability of developing AKI using only baseline characteristics and effective urine biomarkers before a detectable change in serum creatinine in ADHF patients.

Methods

Study Population and Database

A total of 676 consecutive ADHF participants, aged 18 to 80 years, were prospectively enrolled between September 2011 and August 2014 from 6 regional central hospitals located in 5 cities in China. All participants had given informed consent for this specific study. Among these participants, 169 were excluded according to exclusion criteria (Figure S1). The remaining 507 participants formed our study population, of which 321 participants from 4 of the 6 hospitals served as the development cohort to develop the risk score. The remaining 186 participants from the other 2 hospitals served as independent validation cohort. ADHF was diagnosed based on the European Society of Cardiology criteria.24 Estimated glomerular filtration rate (eGRF) was determined by the Chronic Kidney Disease Epidemiology Collaboration (CKD‐EPI) equation.25 All participants had consecutive measurements of serum creatinine and urine biomarkers every day during the first 7 days of hospitalization. The values on the first day were used to develop the risk score for early prediction of AKI, and serum creatinine values on other days were used to assess the development of AKI.

This research was approved by the Institutional Review Board of the Guangdong Provincial Institute of Nephrology and the ethics committees of the participating hospitals before recruiting patients.

Outcome Definition

The primary outcome was the development of AKI. It was defined as an increase in serum creatinine by 26.5 μmol/L (0.3 mg/dL) within 48 hours of admission, or a 50% increase in serum creatinine from preadmission level (mean of at least 3 measurements over a 6‐month period before admission) within 7 days of admission according to the KDIGO Clinical Practice Guidelines for Acute Kidney Injury.26 We did not use urine output criteria (ie, <0.5 mL/kg per hour for >6 hours) for AKI diagnosis because of lack of practicality in measurement when an indwelling urinary catheter was not present.

Sample Size and Missing Data

Sample size determination for observational study is difficult, especially in multiple regression model settings. We had used the rule of thumb recommended by Peduzzi et al27 and Harrell et al,28 namely, events per variable (EPV) being 10 or greater under this circumstance. We considered about 5 to 8 significant clinical factors and 2 to 3 biomarkers in developing a model. This would have required us to recruit a minimum of 110 (11×10) participants who had events to predict the development of AKI.

There were 2 cases with incomplete measurements. We performed simple imputation for these 2 cases. One case with missing serum albumin was imputed by mean, and the other with missing N‐terminal pro‐B‐type natriuretic peptide (NT‐proBNP) by median because of skewness. All analyses were performed based on imputed complete cases.

Statistical Analysis

We took 2 steps to develop the risk model based on the development cohort. The first step was to select meaningful clinical risk factors (nonurine markers) by logistic regression with backward elimination. Clinical candidate predictors significant at P<0.1 in univariate logistic regression models were considered for backward elimination. In this step, a clinical model with clinical factors was developed. Then, urine biomarkers (urine albumin‐to‐creatinine ratio [UACR], urinary NGAL [uNGAL], and uAGT) were evaluated sequentially based on this clinical model. Category‐free (continuous) net reclassification index (NRI), integrated discrimination improvement (IDI),29 and added value of c statistics were used to quantify the additional contribution of these urine biomarkers to risk reclassification.

To develop the risk score, the scoring method similar to that of Sullivan et al,30 was employed based on the developed risk model. The predictive accuracy of the risk score was assessed by both discrimination measured by c statistic31 and calibration evaluated by Hosmer‐Lemeshow chi‐squared statistic32 and calibration plot, a plot of observed proportions versus predicted probabilities. Bootstrapping technique was used to adjust for overfitting and overoptimistic model performance. An optimism‐corrected c statistic using 1000 bootstrap samples created with replacement was reported. Furthermore, external validation of the risk score was conducted to assess the stability of the model.

Predefined interactions (sex×preexisting chronic kidney disease [pre‐CKD], pre‐CKD×uNGAL, and pre‐CKD×uAGT) between predictors and potential nonlinear association of included biomarkers were also examined at the 5% significance level. Sensitivity analysis was conducted to assess the influence of the 2 incomplete measures by leaving the corresponding 2 cases out. All analyses were done with SAS (SAS Institute Inc, Cary, NC, version 9.4). A 2‐sided P<0.05 was considered statistically significant.

The reporting of the present study closely follows the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement.33 Additional study methods are described in Data S1.

Results

The flow of participants is presented as Figure S1. Among the 507 participants, 168 (33.1%) developed AKI during the first 7 days of hospitalization, of whom 108 (64.3%) developed AKI during the first 48 hours of hospitalization. A total of 113 (35.2%) participants developed AKI during hospitalization in the development cohort, and 55 (29.6%) in the validation cohort. Sixty‐one (12.0%) patients died during hospitalization. Overall, the mean age of the participants was 65.2 years, and 63.1% were male. For the clinical conditions, participants with hypertension, diabetes mellitus and pre‐CKD accounted for 49.9%, 25.4%, and 26.2%, respectively. Table 1 shows the characteristics of eligible participants for both the development and validation cohort recorded on admission. The baseline characteristics of the validation cohort were generally similar to those of the development cohort, although the validation cohort was drawn from an independent group of participants. Two cases with incomplete measurements were imputed as described in the Sample Size and Missing Data section. Baseline characteristics outlining those who developed versus those who did not develop AKI were presented in Table S1.

Table 1

Characteristics for Included Patients in Development and Validation Cohorts

	Development Cohort (n=321)	Validation Cohort (n=186)	All (n=507)
Demographics
Mean (SD) age, y	64.9 (15.5)	65.7 (14.0)	65.2 (15.0)
Male	216 (67.3)	104 (55.9)	320 (63.1)
Preexisting clinical conditions
Hypertension	169 (52.6)	84 (45.2)	253 (49.9)
Diabetes mellitus	86 (26.8)	43 (23.1)	129 (25.4)
Pre‐CKDa	90 (28.0)	43 (23.1)	133 (26.2)
Prior hospitalization for HF	165 (51.4)	96 (51.6)	261 (51.5)
Primary causes of heart failure
Ischemic heart disease	165 (51.4)	106 (57.0)	271 (53.5)
Hypertension	44 (13.7)	21 (11.3)	65 (12.8)
Rheumatic heart disease	46 (14.3)	24 (12.9)	70 (13.8)
Cardiomyopathy	42 (13.1)	20 (10.8)	62 (12.2)
Other	24 (7.5)	15 (8.1)	39 (7.7)
Characteristics on admission
LVEF <45%	155 (48.3)	79 (42.5)	234 (46.2)
NYHA, class IV	151 (47.0)	92 (49.5)	243 (47.9)
Median (IQR) NT‐proBNP, pg/mL	5500 (2292‐9000)	4235 (2271‐8540)	5099 (2284‐9000)
Mean (SD) SBP, mm Hg	125.2 (24.1)	130.6 (24.4)	127.2 (24.3)
Mean (SD) DBP, mm Hg	73.7 (14.7)	78.1 (16.7)	75.3 (15.6)
Mean (SD) serum creatinine, μmol/L	110.4 (52.9)	105.2 (42.1)	108.5 (49.3)
Mean (SD) serum albumin, g/L	32.2 (5.9)	35.8 (5.9)	33.5 (6.1)
Mean (SD) hemoglobin, g/L	123.4 (26.0)	125.1 (22.9)	124.0 (24.9)
Treatment before admission
Use ACEI/ARB preadmission	107 (33.3)	37 (19.9)	144 (28.4)
Mean (SD) drug index of ACEI/ARBc	4.2 (1.2)	4.5 (1.0)	4.3 (1.2)
Use spironolactone	109 (34.0)	54 (29.0)	163 (32.1)
Mean (SD) drug index of spironolactone	1.1 (0.3)	1.1 (0.3)	1.1 (0.3)
Mean (SD) drug score of RAAS blocker	5.1 (1.3)	5.8 (1.1)	5.3 (1.2)
Use diuretic preadmission	124 (38.6)	59 (31.7)	183 (36.1)
Use of high‐dose diuretic	8 (2.5)	2 (1.1)	10 (2.0)
Biomarker measurement
Median (IQR) UACR, mg/g Cr	100.6 (27.9‐314.8)	65.66 (16.98‐191.65)	84.2 (23.2‐245.2)
Median (IQR) uNGAL, μg/g Cr	53.4 (22.2‐174.9)	32.55 (15.17‐89.31)	42.5 (20.1‐141.2)
Median (IQR) uAGT, μg/g Cr	39.31 (10.7‐147.1)	34.85 (14.09‐104.07)	37.4 (10.7‐147.1)
Prognosis
AKI, n (%)	113 (35.2)	55 (29.6)	168 (33.1)

Values are numbers (percentages) unless stated otherwise. AKI indicates acute kidney injury; pre‐CKD, preexisting chronic kidney disease; DBP, diastolic blood pressure; HF, heart failure; IQR, interquartile range; LVEF, left ventricular ejection fraction; NYHA, New York Heart Association; SBP, systolic blood pressure; UACR, urine albumin‐to‐creatinine ratio; uAGT, urinary angiotensionogen; uNGAL, urinary neutrophil gelatinase‐associated lipocalin.

Defined as preadmission eGFR <60 mL/min per 1.73 m2. Preadmission eGFR is the mean of at least 3 measurements over a 6‐month period before admission.

Risk Score Development

The clinical model, including age, sex, pre‐CKD, serum albumin (ALB), and NT‐proBNP (Table 2), had a c statistic of 0.767 (95% CI 0.714‐0.820). The first urine biomarker added to the clinical model was uAGT, with a category‐free NRI of 0.809 (P<0.001) and an IDI of 0.169 (P<0.001). The second urine biomarker added to the clinical model with uAGT was uNGAL, with a category‐free NRI of 0.293 (P=0.012) and an IDI of 0.023 (P=0.011). However, UACR did not improve model prediction significantly, nor did it make an additional contribution to risk reclassification (Table S2). Thus, the risk model was obtained by adding uAGT and uNGAL rather than UACR to the clinical model. Neither nonlinearity nor interactions were found significant. The risk model has excellent discriminative power with a c statistic of 0.874 (95% CI 0.835‐0.913), significantly larger than that of the clinical model (DeLong test, P<0.001).

Table 2

Univariate and Multivariate Logistic Regression Analysis of Candidate Risk Factors for AKI (Development Cohort, n=321)

Variable	Univariate Analysis		Clinical Model		Risk Modela
	OR (95% CI)	P Value	OR (95% CI)	P Value	Coefficient	OR (95% CI)	P Value
Age	1.03 (1.02‐1.05)	<0.001	1.03 (1.01‐1.05)	0.002	0.0141	1.01 (0.99‐1.04)	0.22
Sex, male	0.91 (0.55‐1.51)	0.72	0.83 (0.48‐1.45)	0.516	0.1679	1.18 (0.62‐2.26)	0.61
Hypertension	1.82 (1.09‐3.04)	0.02
Diabetes mellitus	2.43 (1.45‐4.09)	0.001
pre‐CKD	5.47 (3.18‐9.39)	<0.001	4.15 (2.36‐7.28)	<0.001	1.2200	3.39 (1.73‐6.65)	<0.001
Prior hospitalization for HF	1.28 (0.79‐2.06)	0.31
LVEF <45%	1.16 (0.72‐1.87)	0.53
NYHA, class IV	2.24 (1.39‐3.63)	0.001
ln‐NT‐proBNPb	1.58 (1.23‐2.04)	<0.001	1.36 (1.04‐1.77)	0.023	0.2320	1.26 (0.93‐1.72)	0.14
SBP	1.01 (1.00‐1.02)	0.06
DBP	1.01 (1.00‐1.03)	0.12
Serum creatinineb	1.01 (1.01‐1.02)	<0.001
Serum albumin	0.92 (0.88‐0.96)	0.001	0.94 (0.90‐0.98)	0.007	−0.0117	0.99 (0.93‐1.05)	0.69
Hemoglobin	0.99 (0.98‐0.99)	0.003
Use ACEI/ARB preadmission	1.16 (0.71‐1.91)	0.55
Use diuretic preadmission	1.22 (0.75‐1.99)	0.42
ln‐UACRb	1.63 (1.38‐1.93)	<0.001
ln‐uNGALb	2.01 (1.65‐2.44)	<0.001			0.3439	1.41 (1.12‐1.78)	<0.001
ln‐uAGTb	1.99 (1.66‐2.37)	<0.001			0.5532	1.74 (1.43‐2.11)	<0.001

ACEI indicates angiotensin‐converting enzyme inhibitor; AKI, acute kidney injury; ARB, angiotensin receptor blocker; CI, confidence interval; DBP, diastolic blood pressure; HF, heart failure; LVEF, left ventricular ejection fraction; NYHA, New York Heart Association; pre‐CKD, preexisting chronic kidney disease; SBP, systolic blood pressure; UACR, urine albumin to creatinine ratio; uAGT, urinary angiotensionogen; uNGAL, urinary neutrophil gelatinase‐associated lipocalin.

Intercept=−7.325.

Natural logarithm transformed values of those variables were used in developing risk model.

Scores based on the risk model for all predictors are presented in Table 3. The estimated predicted probability of developing AKI during the first 7 days after admission for an individual ADHF patient according to the proposed risk score is expressed aswhere −4.027 and 0.115 are the intercept and slope coefficients of the regression, respectively. The total risk score ranges from a minimum value of 0 (lowest risk) to a maximum value of 55 (highest risk), with corresponding estimated predicted probabilities of developing AKI ranging from 1.8% to 90.9% (Table S3). The optimism‐corrected c statistic of the risk score in the development cohort, by the bootstrapping technique, was 0.859.

Table 3

Risk Score for Single Risk Factors Associated With Developing AKI in ADHF Patients (Development Cohort)a

Risk Factor	Score	c‐Statistic for Single Variable
Age, y		0.638 (0.574‐0.701)
≤55	0
56 to 65	2
66 to 75	3
76 and older	5
Sex		0.514 (0.460‐0.568)
Male	1
Female	0
Pre‐CKD		0.680 (0.627‐0.732)
Yes	9
No	0
Serum albumin, mmol/L		0.640 (0.576‐0.704)
≤35	1
>35	0
NT‐proBNP, pg/mLb		0.635 (0.572‐0.698)
<5578	0
≥5578	5
uNGAL, μg/g Crb		0.762 (0.707‐0.816)
<25	0
25 to 49.99	5
50 to 99.99	6
≥100	11
uAGT, μg/g Crb		0.814 (0.767‐0.861)
<25	0
25 to 49.99	13
50 to 99.99	16
≥100	23

ADHF indicates acute decompensated heart failure; AKI, acute kidney injury; Pre‐CKD, preexisting chronic kidney disease; UACR, urine albumin‐to‐creatinine ratio; uAGT, urinary angiotensionogen; uNGAL, urinary neutrophil gelatinase‐associated lipocalin.

The highest observed total risk score of 55 coincides with the highest theoretical risk score.

Categories and scores of those variables are present according to their untransformed value for clinical use.

To illustrate the application of the risk score, consider a 60‐year‐old woman with ADHF, preexisting CKD, ALB of 40 g/L, NT‐proBNP of 3200 pg/mL, uNGAL of 65 μg/g Cr, and uAGT of 35 μg/g Cr. The total risk score she gets is 0+2+9+0+0+6+13=30 from Table 3, and the estimated predicted probability that she will develop AKI is 36.0% according to Table S3. An easy‐to‐use online tool (http://www.echobelt.org/risk/) as well as an Excel tool was developed whereby an individual's underlying risk of AKI can be estimated by entering the individual's own characteristics.

Validation of Risk Score

In an independent validation cohort, the c statistics of the developed risk score was 0.847 (95% CI 0.795‐0.910). The receiver operating characteristics curves (ROCs) of the clinical model, the risk model, the risk score, and models with only uNGAL or uAGT are shown in Figure 1. The calibration plots for both cohorts show the close agreement between predicted and observed risk of AKI, with no apparent over‐ or under‐prediction (Figure 2). AKI rates from both development and validation cohorts are also concordant with each other.

Figure 1

Receiver operator characteristic curves showing area under the curve for AKI in ADHF patients. Receiver operator characteristic curves showing area under the curve for clinical model alone, 0.765; uNGAL, 0.762; uAGT, 0.814; risk model, 0.874; risk score in development cohort, 0.859; risk score in validation cohort, 0.847.

Figure 2

Calibration plot of observed vs predicted fracture risk for developing AKI during the first 7 days of hospitalization. Hosmer‐Lemeshow chi‐squared statistic is shown for the risk score in both development and validation cohorts. The points and circles indicate the observed frequencies by decile of predicted probability.

Clinical Implications of the Risk Score

We divided the risk scores into 4 groups, that is, low risk (0‐24 points), moderate risk (25‐34 points), high risk (35‐44 points), and very high risk (45‐55 points) (Figure 3), according to the quartiles of estimated risk, to enhance the clinical utility of the risk score. In the development and validation cohorts AKI was present in, respectively, 9.3% and 9.5% participants of the low‐risk group, 31.1% and 40.0% of the moderate‐risk group, 63.9% and 66.7% of the high‐risk group, and 83.7% and 78.6% of the very high‐risk group. As Figure 3 shows, the risk of developing AKI is highly and positively associated with the risk scores (Pearson contingency coefficient=0.517, P for trend <0.001). We also performed subgroup analysis according to LVEF. The c statistics of the risk score for participants with LVEF <45% and ≥45% are 0.849 (95% CI 0.786‐0.912) and 0.870 (95% CI 0.811‐0.928), respectively. The discrimination power of 2 subgroups was similar, suggesting that the risk score was applicable for both subgroups of participants.

Figure 3

Risk levels according to the risk score in development and validation cohorts. Risks were categorized into low risk (0‐24 points), moderate risk (25‐34 points), high risk (35‐44 points), and very high risk (45‐55 points). Higher points means higher risk of developing AKI in patients with ADHF; P for trend <0.001.

The generated risk score was also correlated with the presence and stage of AKI. The score was significantly higher in those patients who developed AKI than those who did not. Higher risk score was observed in patients with higher stage of AKI (Spearman rank correlation coefficient, r=0.59, P<0.001, Figure S2). In addition, the discriminatory ability of the score for predicting death was also good, with a c statistic of 0.815 (95% CI 0.762‐0.867) (Figure S3).

Furthermore, we also developed a clinical score based on the clinical model with prognostic factors presented in Table 2. The clinical score had an acceptable c statistic of 0.729 (95% CI 0.672‐0.786). Validation of the clinical score was performed on the development data set with bootstrap internal validation and the validation data set as external validation, with c statistic of 0.730 (95% CI 0.673‐0.786) and 0.742 (95% CI 0.666‐0.818), respectively. Risk score for each variable is given in Table S4. The ROCs and calibration plots, as well as risk categories, are presented in Figures S4 through S6.

Sensitivity Analysis

First, to assess the influence of the 2 incomplete measures, we derived a risk model on a complete data set by leaving the corresponding 2 cases out. The discrimination power (c statistic 0.873, 95% CI 0.834‐0.912) was almost the same as that from imputed complete cases. Further, applying the risk score to all participants yielded a c statistic of 0.855 (95% CI 0.820‐0.890), which was similar to that from the development cohort, suggesting high stability of the risk score. We also conducted sensitivity analysis to assess the single biomarker discriminative power by adding a single biomarker to the clinical model. The results are presented in Table S5. Briefly, the final risk model had the lowest AIC and highest c statistic in the development data set. Furthermore, because of the correlation between pre‐CKD and eGFR, eGFR was not selected during the backward selection. Thus, we kept the final risk model with pre‐CKD.

Discussion

Main Finding

We have developed and validated a clinical scoring system to identify patients at a very high as well as a very low risk of developing AKI before a detectable change in serum creatinine in ADHF patients. The risk score was derived from a risk model including 5 clinical factors and 2 urine biomarkers. The clinical risk factors in the model were similar to those in previous reports.34, 35 Two selected biomarkers, uAGT and uNGAL, were highly predictive. The proposed risk score shows high discriminant power (optimism‐corrected c statistic of 0.859) and is strongly supported by external validation (c statistics of 0.847). To our knowledge, this is the first clinical scoring system derived and validated for early prediction of AKI in ADHF patients incorporating clinical risk factors and novel kidney injury biomarkers.

Need for Early Predicting AKI in ADHF

Coexistence of ADHF and AKI, namely acute CRS, is a serious clinical condition associated with significantly increased morbidity and mortality in patients with heart failure.10, 36 A major barrier to improving clinical outcomes in ADHF patients is the lack of ability to identify patients at high‐risk of CRS early enough to initiate interventions.37 The clinical benefits of an early diagnosis of AKI have not been fully realized due to the restriction of using serum creatinine for diagnosis. This restriction often leads to diagnostic delays and potential misclassification of actual injury status and provides little information regarding underlying cause. Actually, all successful therapeutic approaches in animal models used in patients have yielded disappointing results.38 Major reasons are the scarcity of early biomarkers for AKI and consequent unacceptable delay in initiating treatment regimens.15

AKI is a major contributor to poor patient outcomes. The International Society of Nephrology (ISN)'s 0by25 (zero preventable deaths by 2025) initiative aims to eliminate preventable deaths from AKI by 2025 by calling for global strategies that permit timely diagnosis and treatment of potentially reversible AKI.39 As Mehta et al addressed in the Future Perspective Section, “the effect of AKI on morbidity and mortality will be shaped by advances in methods to detect AKI earlier in the disease course.”39 Because early detection of AKI is helpful in identifying early kidney injury and preventing the progression of the kidney damage, our work may provide the first step forward to avoid harm to the kidney and devise early preventive strategies in CRS.

Clinical and Policy Implications

The present study derived and validated a potential clinical prediction tool rather than a decision rule. It is to aid the attending physician who will make the clinical decision. The factors incorporated in the proposed risk score are readily available data recorded on admission or from routine medical examination. Both uAGT and uNGAL are accepted and utilized biomarkers of kidney injury. To date, clinical studies that use the biomarkers to predict acute CRS yield only modest performance in general.40, 41 Individual prediction of AKI remains a challenge in the setting of ADHF. Therefore, stratification of risk level according to the score incorporating both clinical factors and biomarkers is clinically relevant, particularly in critically ill patients with multiorgan dysfunctions.

Comparison With NGAL and Other Risk Score

NGAL is a novel biomarker widely utilized to identify AKI patients and detect patients with likely subclinical AKI who have an increased risk of adverse outcomes.42 A systematic review and meta‐analysis43 demonstrated that the overall c statistic of NGAL to predict AKI is 0.815. This meta‐analysis synthesized data from different settings and varied study populations, such as contrast‐induced nephropathy, cardiac surgery–associated acute kidney injury. The c statistics of NGAL reported from individual study were less than 0.74, except for 1 study with the c statistic of 0.93,13 the study population of which was a subgroup of children with an extremely high cutoff point (170 ng/L) and relative small sample size of 119. In contrast, our study has included 507 ADHF adult patients, and the c statistic of NGAL alone as a predictor is only 0.761, much less than that of our risk model (c statistic of 0.874) and risk score (c statistic of 0.859).

Thakar et al44 reported a risk score using clinical parameters from patients receiving open‐heart surgery and yielded an excellent c statistic of 0.81 (95% CI 0.78‐0.83). Our score was developed in a cohort of ADHF patients whose clinical characteristics and risk exposure were different from those of the patients with heart surgery. The c statistic of the proposed score for predicting acute CRS reached 0.859, demonstrating excellent performance of our risk score in ADHF patients.

Sensitivity analysis showed that the 2 incomplete measures had little influence on the risk model. Furthermore, we have kept age, sex, NT‐proBNP, and serum albumin in the final risk model, although they were not statistically significant. Age and sex were given factors for controlling potential patients' heterogeneity. NT‐proBNP and serum albumin were also found significantly associated with the development of AKI in univariate analysis and by other researchers.45, 46 We also developed a clinical score that reached acceptable performance. Clinicians could choose the one that fits their clinical practice best.

Strengths and Limitations of This Study

Our study has several strengths. First, our risk score was derived for early detection of patients at high risk of developing AKI incorporating, for the first time, both baseline characteristics and novel biomarkers recorded on the first day of admission before a clinically significant change in serum creatinine. Second, the risk score possesses high discriminative power as well as high stability and reproducibility, as shown by concordance between internal and external validation. Third, to develop a prediction tool for early detection of patients who would develop AKI during hospitalization, researchers should exclude those who already have AKI on admission. In our study we excluded those having AKI on admission based on preadmission measurements of serum creatinine, but we did not use this information in developing the risk score; thus, the use of the developed predicting tool is not limited by the need for preadmission serum creatinine. Last, our article closely follows the TRIPOD statement.

The study also has limitations. First, AKI diagnosed by an increase in serum creatinine may introduce the dilemma of using a surrogate outcome to assess the performance of novel models.47 Evidence of AKI on renal biopsy would be the gold standard but was not feasible in our large cohort. As is the case in most AKI studies, we did not use urine output criteria (ie, <0.5 mL/kg per hour for >6 hours) for AKI diagnosis because urine output as the diagnostic criterion for AKI has been questioned recently for its limited sensitivity when diuretics are administered, reduced specificity in the presence of dehydration, and lack of practicality when an indwelling urinary catheter is not present (as in our setting).15 Second, our findings remain to be confirmed by other independent studies. The question of whether the model predicts AKI in the special setting of ADHF versus AKI from other causes remains to be addressed. Third, although the study participants were recruited from 6 hospitals located in 5 cities in China, generalizability of the risk score to other populations still needs to be validated. In addition, the bootstrap procedure to estimate the overoptimism was conducted based on the developed risk score, which did not include the model selection step. The estimate of the overoptimism could be substantially biased under this circumstance.

Conclusions

We developed and validated a novel risk score by incorporating both major clinical risk factors and effective urine biomarkers to predict AKI in ADHF patients early. Individual risk prediction, as well as risk stratification based on the risk score, may assist clinicians to assess the risk of AKI in ADHF patients, which, in turn, would help them to plan and initiate the most appropriate disease management for patients in time.

Sources of Funding

This study was supported by the National Natural Science Foundation of China (no. 81273191/81673270) granted to Prof Chen and the National Natural Science Foundation of China (no. 81430016) to Dr Hou.

Disclosures

None.

Data S1. Supplemental Methods

Table S1. Baseline Characteristics Outlining Those Who Developed Versus Not Developed AKI [Values are numbers (percentages) unless stated otherwise]

Table S2. C‐Statistic, NRI, and IDI for Different Models

Table S3. Predicted Risk of AKI in Patients With ADHF Based on the Risk Score Model

Table S4. Risk Score for Single Risk Factors Associated With Developing AKI for Clinical Score Model in ADHF Patients

Table S5. Sensitivity Analysis of Model Performance of Models With Different Included Variables

Figure S1. Flow of enrollment of the study participants.

Figure S2. Boxplot showing the risk score across different AKI stage of the patients. Significant difference in average risk score of different AKI stage was found, with higher risk score in patients with higher stage of AKI (P value by 1‐way ANOVA analysis, <0.001).

Figure S3. Receiver operator characteristic curves showing the discriminatory power of the risk score for predicting death during hospitalization.

Figure S4. Receiver operator characteristic curves showing area under the curve of clinical model and clinical score for AKI in ADHF patients.

Figure S5. Calibration plot as well as the Hosmer‐Lemeshow chi‐squared statistic for the risk score in both development and validation cohorts. The points and circles indicate the observed frequencies by decile of predicted probability.

Figure S6. Risk levels according to the risk score in development and validation cohorts. Risks were categorized into low risk (0‐4 points), moderate risk (5‐9 points), high risk (10‐14 points), and very high risk (15‐18 points). Higher points mean higher risk of developing AKI in patients with ADHF, P for trend <0.001.

Click here for additional data file.