Comparison of urine neutrophil gelatinase-associated lipocalin and interleukin-18 in prediction of acute kidney injury in adults.

BACKGROUND: Neutrophil gelatinase-associated lipocalin (NGAL) and interleukin-18 (IL-18) were considered as the most promising biomarkers in prediction of acute kidney injury (AKI), but the priority of them remains unclear.
METHODS: Databases of PubMed, Elsevier, Cochrane library, and Web of science were searched until August 23, 2017 for studies investigated the diagnostic value of urine NGAL (uNGAL) and urine IL-18 (uIL-18) for AKI in adults. Statistical analysis and investigation of heterogeneity source were using RevMan5.3, MetaDiSc1.40, and Stata14.0.
RESULTS: A total of 7 studies were included involving 2315 patients from 7 countries in this article, of whom 443 (19.1%) developed AKI. The present meta-analysis demonstrated that uNGAL was more valuable compare with uIL-18 with effect size of 1.09 (95% CI 1.03-1.15, P = .004) in specificity, but not in sensitivity with effect size of 1.12 (95% CI 0.98-1.29, P = .104). Subgroup analysis presented that research design may be a foundation affecting the diagnostic accuracy of uNGAL and uIL-18 for AKI. No substantial publication bias was found.
CONCLUSIONS: uNGAL is more specific for prediction of AKI in adults as compared with uIL-18.

Introduction

Acute kidney injury (AKI) is a common and serious condition recognized in nearly all fields of medical practice, and over 2 million people are afflicted by AKI worldwide.[ The RIFLE (Risk, Injury, Failure, Loss, End-Stage Kidney Disease), the modified version AKIN (Acute Kidney Injury Network) criteria, and the KDIGO (Kidney Disease: Improving Global Outcomes), based on serum creatinine (Scr) and urine output, were step forward in diagnosing AKI.[ A major concern today as the most reliable biomarker for early diagnosis of AKI is to continue to improve.[ Clinically, applicable AKI biomarkers should satisfy some certain conditions including rapidly measured by standardized clinical assay platforms, non-invasive, which may decrease patients’ burden like using easily accessible samples, sensitive to facilitate early detection, and specific to differentiate intrinsic AKI from chronic kidney disease and other diseases.[

A number of promising serum and urine biomarkers have recently been characterized to be more sensitive, practical, and accurate in clinical settings, such as neutrophil gelatinase-associated lipocalin (NGAL), cystatin C, kidney injury molecule 1 (KIM-1), and interleukin-18 (IL-18), some of which show brilliant abilities in prediction of AKI.[ NGAL is a new member of the lipid carrier protein superfamily, which is highly expressed in damaged renal tubules and can be quickly detected in urine.[ In experimental and clinical studies, NGAL has been investigated extensively and would appear to be one of the most frequently investigated early biomarkers of AKI.[ Moreover, a meta-analysis of data from 19 studies including >2500 patients, serum, and urine NGAL levels were found to be diagnostic for AKI.[ Interleukin 18 (IL-18) is a pro-inflammatory molecule synthesized by phagocytes from the proximal end of the renal tubule,[ having been proposed as a promising biomarker for the early detection of AKI in recent years.[

Up to date, there is no comparison between these promising biomarkers in the prediction of AKI. Therefore, for the first time, we reported that uNGAL was prior to uIL-18 in specificity.

Methods

The search strategy protocol and summarizing the results was presented based on the Cochrane methods for Systematic Reviews of Diagnostic Test Accuracy[ and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.[

Search strategy

We performed a comprehensive literature search of PubMed, Embase, Web of Science, and Cochrane library until August 23rd, 2017 with the medical subject heading and text words for (“acute kidney injury” OR “acute renal failure” OR “AKI”) AND (“neutrophil gelatinase-associated lipocalin” OR “NGAL”) AND (“Interleukin-18” OR “IL-18”) without language restriction. The reference lists of reviewed full-text articles were checked for fear of losing additional relevant studies. The most recent or complete studies were selected for analysis when the same or similar patient data were included. We used the EndNoteX8 bibliography manager to check the titles and abstracts of all citations and then retrieved and referenced full-text articles. The searches were performed independently by 2 investigators (J.G. and X.Z.). Authors were contacted, where necessary, to provide additional information.

Selection criteria

The inclusion criteria used for this meta-analysis were as follows: studies investigating the diagnostic accuracy of both uNGAL and uIL-18 to predict AKI; human studies with participants ≥18 years of age; studies with mandatory data from which true-positive (TP), false-positive (FP), false-negative (FN), and true-negative (TN) could be found or calculated.

Exclusion criteria

The exclusion criteria used for this meta-analysis were as follows: studies that were systemic review or meta-analysis, animal research, letters, editorials, case reports or case series; studies not available in English; duplicate articles describing the same study.

Data extraction and quality assessment

Methodological quality assessment of studies was performed according to Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) checklist, a quality assessment tool specifically developed for systematic reviews of diagnostic accuracy studies to assess bias in the study,[ including 14 items (each of which is scored as yes, no, or unclear). High-quality articles were the ones with all the 7 QUADAS-2 items were found to have the low risk of bias. The QUADAS-2 sheet was created by RevMan5.3 according to the following bias domains: selection, performance/detection, attrition, and reporting bias.

Data were extracted independently according to the selection and exclusion criteria above. Repetitive articles were removed using EndNoteX8 (Thomson Reuters Company) software and the results were pooled. Two authors (JG and XZ) independently determined study eligibility by reviewing each of the citations and retrieving the literature by titles or abstracts. Subsequently, the full texts and disagreements between reviewers on any item were resolved by face-to-face discussion. Results of the search were recorded in a checklist according to the guidelines designed by PRISMA statement.[ From each study, the following information was received: first author, country of origin, study design, sample size, population setting, assessment assay, and patient characteristics (age and sex), as well as the definition of AKI. In addition, specificity, sensitivity, area under summary receiver operating characteristics curve (AUROC) with 95% confidence interval (CI) and the optimal cut-off thresholds, which were optional. We calculated the TP, TN, FP, and FN if lacked. If only the data could not be extractable from the article, we contacted the corresponding authors by email and asked whether they were willing to share the information. If no reply was received, the study was excluded from the meta-analysis and included in the descriptive analysis only.

Statistical analysis

The Spearman correlation coefficient calculated by MetaDiSc1.40 was used to explore the threshold effect between the pooled sensitivity and 1-specificity. The result indicates nonexistent when the value is negative. The positive result or P < .05 indicated the existence of a threshold effect[ (differences in sensitivity and specificity occurring because of different cut-offs used in different studies to define a positive test result). Effect size presented the diagnostic ratio of uNAGL versus uIL-18 and likelihood ratio was calculated based on the pooled sensitivity and specificity. Based on the bivariate mixed-effects regression model that was developed by Van Houwelingen et al,[ command media was used to calculate effect size of sensitivity, specificity, and area under summary receiver operating characteristics curve. Heterogeneity caused by non-threshold effects was calculated by the X2-based Q test and the inconsistency index I2. Included studies were homogeneous when the statistical significance is set at P > .10. The I2 index measured indicates the degree of heterogeneity between multiple studies. I2 values <25%, of 25% to 50%, and >50% indicated modest, moderate, and substantial heterogeneity, respectively.[ Besides, hierarchical summary receiver operating characteristic curves (HSROC) was also constructed in this analysis by using the metandi command, which can explain TPR (true positive rate) and FPR (false positive rate) intra- and inter- study variation more accurately.[ Remarkable heterogeneity was explored further by subgroup analysis restricted by different study design, description of “gold standard,” sample size, location, clinical setting, and measurement assay. In addition, Deeks funnel plot asymmetry test was used for evaluation of publication bias.[ All analysis above expects Spearman correlation coefficient was conducted with Stata (version 14.0).

Results

Search results and study characteristics

The search initially yielded 1079 articles from various databases, of which 143 were excluded by EndnoteX8 because of duplication. After screening through the titles and abstracts, we excluded 845 studies as not relevant to our interests. Having reviewed the full text of the remaining studies, a total of 7[ studies that included 2315 patients met the inclusion criteria and were included in the meta-analysis (Fig. 1).

Figure 1

Flow chart of search results.

Characteristics of individual studies were listed in Table 1. There were 6 prospective cohort studies and 1 case-control studies included in this meta-analysis. Although the eligible studies were published in English, they represented an international experience conducted in a large range of countries including Taiwan, Australia, America, Germany, Spain, Japan, and China, with the publication years ranging from 2008 to 2014. Four studies focused on patients undergoing cardiopulmonary bypass surgery, and the other included living-donor liver transplantation, ICU patients, and emergency department patients. The sample sizes varied between 31 and 1234 patients. All studies stored samples at −80 °C. Measurements methods most commonly used commercial enzyme-linked immunosorbent assay (ELISA) and 1 study used a standardized clinical laboratory platform (ARCHITECT [Abbott Diagnostics])[ for urine NGAL and IL-18.[ Mean age ranged from 38 to 68 years in these studies. Variable definitions of AKI were adopted in the original studies. Five studies[ defined AKI using AKIN and/or RIFLE criteria. One using Scr level, which diagnosed by the attending physicians or nephrologists, and not fully according to the AKIN criteria in which AKI basically defined as an increase in Scr level of 50% within continuous 96 hours regardless the blood levels of tacrolimus was higher and/or lower than the target range.[ The definition used for AKI in the last study[ included, is not clear.

Table 1

Characteristics of included studies for NGAL and IL-18 to predict AKI in adults.

Quality assessment

Supplement 1 showed Spearman correlation coefficient of these 7 articles was –0.536 and 0.071 for uNGAL and uIL-18, respectively (P = .215 and P = .867), suggesting there was no significant threshold effect in different studies. The risk of bias for patient selection, index test, reference standard, and flow and timing, as well as concerns about the applicability related to the first 3 domains were shown in Fig. 2. Several studies did not clearly delineate the selection criteria. Incomplete operationalization of the chosen reference standard for AKI (RIFLE, AKIN, or other) was evident in some of the studies. Other categories mostly showed a low risk of bias. Deeks test showed P value of .71 and .19 for uNGAL and uIL-18, respectively, which indicated the absence of publication bias (Fig. 3).

Figure 2

Quality assessment of the 7 eligible studies using QUADAS-2. QUADAS-2 = Quality Assessment of Diagnostic Accuracy Studies-2.

Figure 3

Deek Funnel plot for potential publication bias analysis. Each solid rectangle represents an eligible study. uIL-18 = urine interleukin-18, uNGAL = urine neutrophil gelatinase-associated lipocalin.

Data extraction and calculation

The data from 7 eligible studies were extracted and were presented in Table 2, including the measurement timing for the diagnosis of AKI, TP/FP/FN/TN values, various optimal cut-off values for different sample types of NGAL, sensitivities, and specificities as well as AUROC (95% CI). For the studies which didn’t provide TP/FP/FN/TN values, we calculated these indexes from provided sensitivity, specificity, and sample size values.

Table 2

Diagnostic value of NGAL and IL-18 to predict AKI in individual studies.

Diagnostic performance

The forest plots and pooled estimates of sensitivity and specificity were presented in Fig. 4. The pooled effect size of uNGAL measurements to uIL-18 in prediction of AKI was 1.12 (95% CI 0.98–1.29, P = .104) in sensitivity and 1.09 (95% CI 1.03–1.15, P = .004) in specificity. AUCROC of uNGAL is 0.87 (95% CI 0.84–0.90) and 0.71 (0.67–0.75) for uIL-18, as showed that there has a difference of AUCROC between uNGAL and uIL-18 and 95%CI of AUCROC between them was non-overlapping (Supplement 2). The pooled positive likelihood ratio of uNGAL was 4.87, indicating that the possibility of an uNGAL test leading to a correct diagnosis for positive results was 4.87 times higher than that of making a wrong diagnosis for positive results. The pooled negative likelihood ratio indicated in contrast, and similarly with uIL-18 as showed in Supplement 3. The AUROC indicated good, but not brilliant diagnostic accuracy.

Figure 4

Forest plot of effect size of uNGAL to uIL-18 in prediction of acute kidney injury. The subgroups SEN demonstrate the effect size of uNGAL versus uIL-18 in sensitivity, and the subgroups SPE demonstrate the effect size of uNGAL versus uIL-18 in specificity. CAG = coronary angiography, CI = confidence interval, CS = cardiac surgery, SEN = sensitivity, SPE = specificity, uIL-18 = urine interleukin-18, uNGAL = urine neutrophil gelatinase-associated lipocalin.

All the results except the sensitivity above revealed a more diagnostic accuracy of uNGAL to uIL-18 in screening out AKI. Distribution of accurate estimator points in the plots did not show a “shoulder arm” pattern (Supplement 2), suggesting no presence of the threshold effect, which was corresponding with the result of Spearman correlation coefficient.

In addition, subgroup analysis showed that consistency of non-prospective studies had significantly been decreased (Supplement 4), indicating the parameter prodesign may be a factor of heterogeneity in the included studies. The results of uNGAL in hierarchical summary receiver operating characteristic model presented β –0.51 (95% CI –143–0.4, Z = –1.09, P = .274) that reflect the SROC was symmetric, and the effect index representing discriminant ability λ 2.96 (95% CI 1.86–4.05), prompting a moderate diagnostic value of uNGAL for AKI. Similarly, β was 0.51 (95% CI –0.43–1.44, Z = 1.06, P = .289) for uIL-18 also reflected symmetric SROC and λ 1.41 (95% CI 0.53–2.29) prompted a moderate diagnostic value (Fig. 5, Supplement 5). Obviously, uNGAL was more significantly valuable than uIL-18 that had been verified in this model.

Figure 5

Hierarchical summary receiver operating characteristic (HSROC) plot of urine neutrophil gelatinase-associated lipocalin (NGAL) and interleukin-18 to predict acute kidney injury. These data are based on the combined sensitivity and specificity weighted for the sample size of each data set reflected by the size of the circles, showing the average sensitivity and specificity estimate of the study results (solid square).

Discussion

More recently, numerous clinical studies have focused on the diagnostic accuracy of biomarkers in predicting AKI since accumulating evidence demonstrating the weak ability of “gold standard” Scr level.[ Among these promising biomarkers approved lately, NGAL and IL-18 were understanding as a quick, accurate, and precise marker of genomic, transcriptomic, and proteomics techniques,[ which implies that they could become a practical clinical test for diagnosis of AKI. IL-18 is synthesized as an inactive 23-kDa precursor mainly in proximal tubular epithelial cells and getting into tubular fluid after activated. Our choice for urine NGAL but not plasma or serum was in order to avoid the heterogeneity between blood and urine samples. Therefore, we only included the initial researches with needed data containing both uNGAL and uIL-18. Substantial studies existed only considering single uNGAL or uIL-18 were excluded.

Overall, the present meta-analysis has shown the superiority of uNGAL levels to uIL-18 for predicting the progression of AKI, which was demonstrated by the above values including the effect size of specificity, positive likelihood ratio, negative likelihood ratio, AUROC, and even the Lambda of HSROC.

One limitation of this analysis was the heterogeneity with these studies based on different institutions across the world. Meanwhile, varying time of biomarkers measurement, different definitions and various settings of AKI were used in the individual studies. For a predictive biomarker, it is important to determine the cutoff value between the health and disease group. However, we could not calculate the ideal cutoff value of uNGAL or uIL-18 for lack of raw data. We wish to finish it in our continued study. Threshold effects induced by variable cutoff values of the included studies may lead to the heterogeneity. But our analysis did not present the threshold effects. The cause of non-threshold effects maybe the design of studies (non-prospective ones reduce diagnostic effect both of uNGAL and uIL-18) and the study conducted by Liangos et al[ which raised integral value of uNGAL. Included studies in this meta-analysis were constructed with a wide range of clinical settings. Our result is applicable with great most patients to a certain extent. However, AKI is also a defect associated with other comorbidities such as diabetes, arteriosclerosis, and higher immunological experience, which interfered with the sensitivity and specificity of the diagnostic test. Finally, the statistical approach used in this study is not capable of considering the different AKI severity stages on the analysis of the accuracy of the biomarkers.

The main strength of our study is that it appears to be the first analysis comparing the diagnostic performance of uNGAL and uIL-18 directly by using this statistical approach, which make it authentic and reliable.

Conclusions

The present meta-analysis demonstrated that uNGAL was more accurate specifically than uIL-18 in early diagnosis of AKI. Due to restrictive studies be analyzed in this study, more future randomized controlled trials across a broad spectrum of clinical settings with larger sample sizes are needed to validate this issue.

Author contributions

All authors have contributed significantly, Jiadi Gan was mainly responsible for the statistical analysis, and Xiaodong Zhou prepared the manuscript. And that all authors are in agreement with the content of the manuscript.

Funding acquisition: Xiaodong Zhou.

Investigation: Jiadi Gan.

Methodology: Jiadi Gan.

Resources: Jiadi Gan, Xiaodong Zhou.

Software: Jiadi Gan.

Writing – original draft: Jiadi Gan.

Writing – review & editing: Jiadi Gan.

Jiadi Gan orcid: 0000-0002-1256-8950