The value of APACHE II in predicting mortality after paraquat poisoning in Chinese and Korean population: A systematic review and meta-analysis.

BACKGROUND: The Acute Physiology and Chronic Health Evaluation II (APACHE II) score is used to determine disease severity and predict outcomes in critically ill patients. However, the prognostic significance of APACHE after acute paraquat (PQ) poisoning remains unclear. The meta-analysis was aimed to study the value of APACHE II in predicting mortality in PQ-exposed Chinese and Korean patients.
METHODS: Databases that included PubMed, Embase, Cochrane Library, and the Chinese National Knowledge Infrastructure were searched through August 2016. Studies using APACHE II to predict mortality in PQ-poisoned patients were selected. The odds ratio and weighted mean difference (WMD) were used to pool binary and continuous data. Additionally, we aggregated sensitivity, specificity, and other measures of accuracy. Statistical analyses were made using the Stata V.13.0 software.
RESULTS: This study included 29 studies, and 25 studies evaluated APACHE II scores on admission. Pooled data showed that survivors had significantly lower total scores than nonsurvivors (WMD = -7.29, and I = 98.2%, both P <.05). The pooled sensitivity of an APACHE II score ≥5 for predicting mortality was 75% and the pooled specificity was 86%. The positive likelihood ratio (PLR) was 5.3 and the negative likelihood ratio (NLR) was 0.29. The pooled sensitivity of an APACHE II score ≥10 for predicting mortality was 88% and the pooled specificity was 84%. The pooled PLR and NLR was 5.5 and 0.15, respectively.
CONCLUSION: This study showed PQ-poisoned nonsurvivors had significantly higher APACHE II score than did survivors. APACHE II scores satisfactorily predicted mortality.

Introduction

Paraquat (PQ), known by its formal chemical name as 1,1′-dimethyl-4-4′-bipyridinium dichloride is one of the most frequently used herbicides worldwide.[ PQ functions to disrupt photosynthesis in weeds. As a labor saving and inexpensive herbicide, PQ is popular among farmers. However, PQ is forbidden or its use restricted to certified individuals only in the USA and many European countries. Presently, use of PQ is predominantly in developing Asian countries.[ PQ poisoning may occur when patients ingest the pesticide intentionally or accidentally ingest it in an attempt to commit suicide—a major public health problem in Asians.[

The initial treatment for PQ poisoning involves preventing absorption and reducing the blood concentration of PQ by hemoperfusion or hemodialysis. However, the efficacy of these treatments is extremely limited and the mortality remains extremely high.[ It is important to predict the risk of death in order to spare the hopeless or minimally poisoned patients from needless aggressive therapy. The severity index of PQ poisoning (SIPP) is recognized as the most potentially valuable prognostic indicator for PQ-poisoned patients. It is calculated by multiplying the time since PQ ingestion (hour) by the concentration in the plasma (mg/L).[ However, many hospitals do not have access to assaying the plasma PQ level, which limits the accurate evaluation of poisoning severity.

The Acute Physiology and Chronic Health Evaluation (APACHE) II system is widely used in the intensive care unit (ICU) and has been in use for the past 30 years or more.[ The APACHE II system includes a 12-point acute physiology score, age point, and chronic health evaluation, which are readily available in most emergency departments. Further, calculation of the score is robust and uncomplicated. Useful scoring systems could facilitate the emergency triage system and guide treatment choice. Several studies have used the APACHE II scoring system in PQ-poisoned patients to assess prognosis.[ However, as a scoring system that is widely applied for general critically ill patients, some authors proposed that the APACHE II system might underestimate mortality in poisoned patients.[ Moreover, previous studies showed a discordant predictive value of this scoring system. In this study, we conducted a meta-analysis to evaluate the usefulness of APACHE II in predicting mortality in Chinese and Korean patients presenting with PQ poisoning.

Methods

Search strategy

The electronic search was completed through PubMed, Embase, the Cochrane Library, and the Chinese National Knowledge Infrastructure (CNKI) (from inception to August 2016). The search terms used were: APACHE II, PQ, and mortality. The language was restricted to English and Chinese. Studies were included if they met the following criteria: comparing the APACHE II scores between survivors and nonsurvivors in PQ-poisoned patients, or presenting predictive accuracy of APACHE II for mortality; including more than 20 patients; and the study was conducted in China or Korea.

The ethical approval and informed consent were not necessary, because data of the meta-analysis was extracted from published literatures.

Data extraction

Two authors independently assessed the study eligibility and quality and extracted the relevant data. Any disagreement was resolved by consensus or by discussions with the corresponding author. The collected information included author, year, region, study design, sample size, gender, age, mortality, cutoff values, APACHE II scores, and study period. The sensitivity and specificity were directly extracted or indirectly calculated from the primary data. To evaluate the study quality, the Newcastle–Ottawa Scale (NOS) was used.[ This scale included 3 aspects: selection of the study group; comparability of study groups; determination of the outcome of interest. A star rating of 0–9 was allocated to each study based on these parameters. A study with a score ≥7 was deemed high quality.

Statistical analysis

The software program Stata V.13.0 (Stata Corporation, College Station, TX) was used to perform the meta-analysis. For continuous measures of the APACHE II scores, we used weighted mean differences (WMD) with 95% confidence intervals (CIs) as effect estimates. Conversion of a median value to a mean value was conducted by the previously proposed method.[ Random-effects models were used for all analyses. We performed the Begg and Egger's tests, for which an alpha value of P <.05 was regarded as statistically significant. In addition, the funnel plot was generated to visually examine any apparent publication bias.[ To analyze the predictive value of the APACHE II score for mortality, we pooled the sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR), with the corresponding 95% confidence interval (CIs). The summary receiver operating characteristic (SROC) curves were constructed. The bivariate generalized linear mixed model was employed to process data.[ The area under the curve (AUC) was used to assess the overall predictive accuracy. When including studies ≥10, publication bias was examined by Deek's funnel plot.[ The heterogeneity was assessed by Cochran's Q test and I2 test. An I2 less than 25% was considered low heterogeneity, 25% to 75% was considered intermediate or medium heterogeneity, and ≥75% high heterogeneity.[ Sensitivity analysis was performed by excluding selected studies one by one to assess data stability. Subgroup analyses and meta-regression analyses were conducted according to the following variables: region (China vs Korea), mortality (<50% vs ≥50%), sample size (<100 vs ≥100), and male percentage (<50% vs ≥50%). An alpha value of P <.05 was regarded as statistically significant.

Results

Literature search and study characteristics

A total of 273 studies were obtained from an original retrieval of 156 records from PubMed, 16 studies from Embase, 16 records from the Cochrane Central Register of Controlled Trials, and 85 records from CNKI. After excluding irrelevant studies and those with insufficient data, 29 studies were ultimately pooled into a meta-analysis (see Fig. 1). Ten studies were published in English[ and 19 studies were published in Chinese,[ as shown in Table 1. Twenty-seven studies were conducted in Chinese and 2 studies in Korean. With the exception of 1 prospective study, most case series were retrospectively reviewed. The sample size ranged from 20 to 272. The mortality rate ranged from 42.2% to 88.3%. In quality assessment by the NOS, most studies achieved high quality. The item satisfied least was the description of confounding factor adjustment (please see Table 2).

Figure 1

The flow diagram of study selection process.

Table 1

Characteristics of included studies.

Table 2

NOS for quality assessment of included studies.

Comparison of APACHE II scores

Twenty-five studies evaluated APACHE II scores on admission. The pooled data showed that survivors had significantly lower total scores as compared nonsurvivors (WMD = –7.29, 95% CI –8.96 to –5.63, P <.05; I2 = 98.2%, P <.05; Fig. 2). Sensitivity analysis was performed by excluding studies sequentially. No single study significantly altered the overall outcome. Meta-regression analysis showed that sample size (P = .80), male percentage (P = .42), and mortality (P = .36) did not account for the source of heterogeneity.

Figure 2

The forest plot showing the weighed mean difference in APACHE II scores for the survivors as compared with the nonsurvivors. APACHE = Acute Physiology and Chronic Health Evaluation.

Subgroup analysis was conducted based on the following variables: sample size (≥100 vs <100), mortality (≥50% vs <50), male percentage (≥50% vs <50), and follow-up duration of death (<1 month vs ≥1 month). Stratified analyses showed that the results remained significant for all subgroups (Table 3). The funnel plot appeared to be symmetrical (Fig. 3). No publication bias was revealed by Egger's test (P = .94) or Begg's test (P = .44).

Table 3

Subgroup analysis for studies reporting the APACHE II scores.

Figure 3

The funnel plot for examining publication bias of studies on APACHE II score comparisons. APACHE = Acute Physiology and Chronic Health Evaluation.

Predictive value of APACHE II scores

The cutoff value of APACHE II scores was 5 in 7 of the selected studies.[ The pooled sensitivity of an APACHE II score ≥5 for predicting mortality was 75% (95% CI 66%–82%), and the pooled specificity was 86% (95% CI 68%–94%; Fig. 4). The PLR was 5.3 (95% CI 2.2–12.6), and the pooled NLR was 0.29 (95% CI 0.21–0.40). The DOR was 18 (95% CI 7–50). The pooled, weighted AUC was 0.84 (0.80–0.87). The bivariate SROC graph with the 95% confidence region and the 95% prediction region is also shown (Fig. 5). The cutoff value of the APACHE II score was ≥10 in 7 of the studies.[ The pooled sensitivity of the APACHE II score ≥10 for predicting mortality was 88% (95% CI 71%–95%), and the pooled specificity was 84% (95% CI 67%–93%; Fig. 6). The pooled PLR was 5.5 (95% CI 2.6–11.8), and the pooled NLR was 0.15 (95% CI 0.06–0.36). The DOR was 38 (13–106). The pooled, weighted AUC was 0.85 (0.81–0.87). The bivariate SROC graph with the 95% confidence region and the 95% prediction region is also shown (Fig. 7).

Figure 4

The pooled sensitivity and specificity of APACHE II score ≥5 for the prediction of mortality. APACHE = Acute Physiology and Chronic Health Evaluation.

Figure 5

The SROC of APACHE II score ≥5 for the prediction of mortality. APACHE = Acute Physiology and Chronic Health Evaluation, SROC = summary receiver operating characteristic.

Figure 6

The pooled sensitivity and specificity of APACHE II score ≥10 for the prediction of mortality. APACHE = Acute Physiology and Chronic Health Evaluation.

Figure 7

The SROC of APACHE II score ≥10 for the prediction of mortality. APACHE = Acute Physiology and Chronic Health Evaluation, SROC = summary receiver operating characteristic.

Discussion

Our meta-analysis showed that in patients with PQ poisoning, the APACHE II score was significantly lower in survivors as compared with nonsurvivors (WMD = –7.29, P <.05). The pooled results remained significant for all subgroups. No publication bias was revealed. For an APACHE II score with a low cutoff value (≥5), the sensitivity for predicting mortality was 75% and the specificity was 86%. For an APACHE II score at a high cutoff value (≥10), the sensitivity was higher (88%), without substantial compromise in the specificity (84%). The DOR of high cutoff value was approximately twice that of the DOR at a low cutoff value (38 vs 18). It appeared that on using APACHE II scores at cutoff values ≥10, higher predictive accuracies were achieved. Our data supported the contention that the APACHE II score was a simple, robust, reproducible and practical tool for determining the sensitivity of PQ poisoning severity.

There were several reasons for the association between the APACHE II score and the mortality in PQ poisoning. First, the APACHE II score was positively correlated with the PQ concentration in plasma, followed by the estimation of the relative ingested dose of PQ.[ Second, PQ poisoning could cause multiple organ failure and mortality, eventually. Moreover, the lung was vulnerable to PQ poisoning. PQ could induce inflammation by stimulating the secretion of reactive oxygen species and signal transduction pathways. Additionally, PQ could lead to mitochondrial damage.[ The APACHE II score included 12 routine physiological measurements that provided a general assessment of the diseased state, which in turn strengthened the evaluation of the observed mortality caused by multiple organ failure.[ Especially, anoxia and acute respiratory distress syndrome were the 2 frequent clinical manifestations of PQ poisoning, which were also the primary causes of death.[ Correspondingly, the APACHE II score was significantly correlated with the fraction of inspired oxygen and the alveolar–arterial oxygen gradient (P[A–a]O2).[ Third, several variables inherent to APACHE II were associated with high mortality, including age,[ respiratory rate,[ hydrogen ion concentration levels,[ the arterial partial pressure of carbon dioxide,[ hypokalemia,[ increased creatinine levels,[ and elevated differential white blood cell counts.[

No meta-analysis has been performed to explore the association between any prognostic factor and adverse outcomes in PQ-poisoned patients. SIPP was most popular for the assessment of PQ poisoning severity. However, Min et al[ showed that this scoring system was not an ideal model to predict mortality. In China, PQ poisoning mostly occurs in rural areas. Furthermore, the tests of PQ concentration are not always available in many rural hospitals because of limited medical resource. Sequential organ failure assessment (SOFA) score and acute kidney injury network (AKIN) score were also commonly used for critically ill patients. Weng et al[ showed that the sensitivity and specificity of the SOFA score for predicting in-hospital mortality was 77.2% and 69.8%, respectively. The sensitivity and specificity of the AKIN score were both lower than 70%.

Lee et al[ showed that the sensitivity and specificity of SOFA for predicting in-hospital mortality was 58.5% and 86.1%, respectively. Notably, our study showed that the APACHE II score had higher sensitivity without compromising specificity. The results were separated by different cutoff values of APACHE II score, and we observed that the higher cutoff value of APACHE II score was associated with higher sensitivity and lower specificity. In addition, in Chinese rural hospitals, the APACHE II score was easy to calculate even for less experienced ICU physicians with limited medical resources at their disposal.

Several limitations of this meta-analysis should also be acknowledged. Most included studies were retrospectively designed, and these might have caused selection and recall bias. The sample size was not large in most included studies, which was likely an inherent limitation for most toxicological research studies. The treatment of PQ intoxication has evolved over years, and this might have affected the observed outcomes.[

Self-reporting of the time and relative amount of PQ ingested might be underreported and inaccurate.[ Although good predictive value was shown, the accuracy of the APACHE II score for predicting mortality was not very high. In addition to the variables noted for the APACHE II scoring system, other prognostic factors like arterial lactate levels[ and pancreatic enzyme levels[ also exhibited predictive value for mortality. Future studies should better modify the traditional APACHE II scores to achieve higher predictive accuracy for PQ-related death.

The present meta-analysis demonstrated that in patients with PQ intoxication, nonsurvivors had significantly higher APACHE II scores than did survivors. The APACHE II score was a useful tool to select patients at high risk of death.