Can procalcitonin tests aid in identifying bacterial infections associated with influenza pneumonia? A systematic review and meta-analysis.

OBJECTIVE: To summarize evidence for the diagnostic accuracy of procalcitonin (PCT) tests for identifying secondary bacterial infections in patients with influenza.
METHODS: Major databases, including MEDLINE, EMBASE, and the Cochrane Library, were searched for studies published between January 1966 and May 2009 that evaluated PCT as a marker for diagnosing bacterial infections in patients with influenza infections and that provided sufficient data to construct two-by-two tables.
RESULTS: Six studies were selected that included 137 cases with bacterial coinfection and 381 cases without coinfection. The area under a summary ROC curve was 0·68 (95% CI: 0·64-0·72). The overall sensitivity and specificity estimates for PCT tests were 0·84 (95% CI: 0·75-0·90) and 0·64 (95% CI: 0·58-0·69), respectively. These studies reported heterogeneous sensitivity estimates ranging from 0·74 to 1·0. The positive likelihood ratio for PCT (LR+ = 2·31; 95% CI: 1·93-2·78) was not sufficiently high for its use as a rule-in diagnostic tool, while its negative likelihood ratio was reasonably low for its use as a rule-out diagnostic tool (LR- = 0·26; 95% CI: 0·17-0·40).
CONCLUSIONS: Procalcitonin tests have a high sensitivity, particularly for ICU patients, but a low specificity for identifying secondary bacterial infections among patients with influenza. Because of its suboptimal positive likelihood ratio and good negative likelihood ratio, it can be used as a suitable rule-out test but cannot be used as a standalone rule-in test.

Introduction

Coinfections with bacterial pathogens in patients with influenza pneumonia have been well documented for the 1918–19 influenza pandemic and have been shown to be a major cause of influenza‐related deaths. , , , Post‐mortem autopsies uniformly showed severe pathological changes compatible with bacterial pneumonia in patients who had died from influenza in 1918. In the spring of 2009, a novel influenza A virus (H1N1) of swine origin was identified in the USA and Mexico, which rapidly led to a worldwide pandemic. During this pandemic, the incidence of bacterial coinfections was estimated to be 20–30%. , However, owing to the poor sensitivities of culture methods to identify respiratory bacterial etiologies, this figure may well have been an underestimate.

The early identification of secondary bacterial infections among patients with influenza infections may enable the early administration of antibiotics and, possibly, improve patient outcomes. However, clinical variables, such as patient characteristics, chest radiographic findings, or routine laboratory results, are unreliable for distinguishing between viral and bacterial lower respiratory tract infection, which may coexist influenza.

Procalcitonin (PCT) is a precursor protein of calcitonin whose production is stimulated by endotoxin and pro‐inflammatory cytokines, such as interleukin‐one beta and tumor necrosis factor‐alpha. Unlike C‐reactive protein (CRP), PCT production is inhibited by interferon‐gamma, a cytokine that is produced during viral infections. , Because of this characteristic, PCT has been shown to differentiate between bacterial and viral infections in many pediatric studies. Recently, several studies have examined its ability to differentiate influenza pneumonia from mixed bacterial and viral infections. However, the relatively small sample sizes in these studies have limited its generalizability. , , , , ,

The aim of our study was to systemically review and qualitatively summarize the current evidence for the diagnostic role of PCT in discriminating between viral and mixed pneumonia among patients with influenza infections.

Methods

We followed standard guidelines and methods for systematic reviews and meta‐analyses of diagnostic tests. ,

Data sources and searches

We conducted electronic searches, without language restrictions, of the MEDLINE, EMBASE, and Cochrane Library databases from their inceptions to November 2011. PubMed was searched by combining two separate queries comprised of medical subject headings (MeSH terms) and text word (tw) keywords for the diagnostic tests and target outcomes. Keywords used for searching included H1N1, influenza, swine flu, avian flu, and procalcitonin. We adapted this search strategy for the EMBASE and Cochrane Library databases. We also sought additional references from the bibliographies of the selected articles and other recent review articles.

Inclusion and exclusion criteria

Two reviewers independently identified articles that were eligible for in‐depth examinations using the following inclusion criteria: (i) evaluation of procalcitonin alone or compared with other laboratory markers, such as CRP, to diagnose bacterial pneumonia in patients with H1N1 influenza infection, and (ii) sufficient data to construct a 2 × 2 contingency table. We used studies of human subjects aged 18 years or older. We excluded case reports, case series, review articles, editorials, and clinical guidelines. Two of the authors independently assessed all titles and abstracts to determine that the inclusion criteria were satisfied. Full‐text articles were retrieved if any of the reviewers considered the abstract to be suitable. The two authors then independently assessed the full texts of the retrieved studies for their suitability for inclusion. Any discrepancies between reviewers regarding an article’s inclusion were resolved by a consensus meeting of three authors. Study selection is summarized in Figure 1.

Figure 1

 Flow chart used for study identification and inclusion.

Quality assessment

Quality of the selected studies was assessed using Quality Assessment of Diagnostic Accuracy Studies (QUADAS) criteria. , ,

Data synthesis and analysis

We calculated the sensitivity and specificity for each included data set. For studies that reported multiple pairs of sensitivity or specificity, we used the pair of sensitivity and specificity that maximizes the Youden index. The Youden index is calculated by Sensitivity + Specificity−1, which enables the selection of an optimal threshold value for the marker. Given the negative correlation between sensitivity and specificity in different studies using different PCT cutoff values, we estimated the pooled sensitivity and specificity for PCT using a bivariate model. The bivariate approach assumed a bivariate distribution for logit‐transformed sensitivity and specificity. In addition to accounting for study size, the bivariate model estimated and adjusted for the negative correlation between the sensitivity and specificity of an index test that may have arisen from different thresholds used in different studies. A summary receiver operating characteristic (ROC) curve was constructed to summarize the true‐ and false‐positive rates from different diagnostic studies. To deal with zero observations in 2 × 2 contingency tables, ½ was added to each cell, which reduced the performance for small studies. Overall sensitivity, specificity, and their 95% confidence intervals were calculated based on the binominal distributions of the true positives and true negatives.

To formally quantify the extent of between‐study variation (i.e., heterogeneity), we calculated an inconsistency index, I 2, which represents the proportion of heterogeneity not explained by random variation. , Statistically significant heterogeneity was considered for I 2 > 50%. ,

We also conducted a diagnostic odds ratio meta‐analysis. The diagnostic OR is defined as the ratio of the odds of the test being positive for a patient with the disease in relation to the odds of the test being positive for a patient without the disease. It can be used to estimate the risk of secondary bacterial pneumonia in an influenza patient tested positive for PCT. Unadjusted data were used exclusively in all meta‐analyses. Summary diagnostic odds ratios were estimated by random (DerSimonian‐Laird) or fixed (Peto) effect models, depending on whether I 2 was >50%. We used linear regression for log odds ratios for the inverse root of effective sample sizes as a test for funnel plot asymmetry in our diagnostic meta‐analyses. A non‐zero slope was suggestive of significant small study bias (P < 0·10). We defined a priori the following clinical and design characteristics of a study as potential relevant covariates: cutoff value, the definition of bacterial sepsis outcome, age range, and underlying disease of the study patients. Statistical analyses used stata 11·0 (Stata Corp, College Station, TX, USA), notably the midas and metandi commands. All statistical tests were two sided, and statistical significance was defined as P < 0·05.

Results

Identification of studies and study quality

Our initial search yielded 47 citations. We retrieved 16 studies for full‐text reading, of which 6 met our inclusion criteria (Figure 1). , , , , , The six included studies were conducted in Korea and France between 2010 and 2011 and comprised 416 patients (median = 77; range: 25–103). Bacterial pneumonia was diagnosed in 137 patients (overall prevalence = 32·9%; range: 18·5–46·6%). Song et al. studied both PCT and CRP, while the others only studied PCT. , , , , All studies had used an appropriate prospective cross‐sectional study design.

In general, a PCT test had a high sensitivity (range: 74·1–100%), but a low specificity (range: 52·5–70%) for identifying bacterial coinfections among patients with H1N1 influenza. Only one study examined a diagnostic role for CRP, which had a sensitivity of 81·8% and a specificity of 59·3%. Figure 2 provides an overall impression of the methodological quality of these studies. None of the included studies explained subject withdrawals or reported uninterpretable results, one study did not provide adequate descriptions of an index test, and two studies were not blinded for index tests for verification of outcomes and may have been corrupted by incorporation bias. ,

Figure 2

 Assessment of study quality using Quality Assessment of Diagnostic Accuracy Studies criteria for the included studies.

Study and population characteristics

Table 1 lists the other study and population characteristics of the six included study patient populations. The outcome definitions used in the included studies could be classified into two broad categories: microbiologically documented infection (MDI) only and both microbiologically and clinically documented infection (MDI and CDI). Three studies were carried out in intensive care units (ICUs), , , two in hospital wards, , and one in an emergency department. The sensitivities and specificities of different markers are also shown in Table 1. Many of these studies reported multiple sensitivity and specificity pairs using different cutoff values or different measurement time points. When there were multiple sensitivity or specificity values reported in a study, we reported those sensitivity and specificity values that maximized a Youden index.

Table 1

Summary of the included studies

Author, year, country	Age range	Prevalence of case patients (Number of participants)	Biomarkers tested	Cutoff (PCT, ng/ml CRP, mg/l)	Outcomes	Setting	PCT sensitivity, Specificity	CRP sensitivity, Specificity
Guervilly C 2010 Italy 14	Adult	0·21 (38)	PCT	PCT ≥ 0·5	Microbiologically documented infection	ICU	100% 52·5%	N.A
Ingram PR 2010 Australia 16	Adult	0·36 (25)	PCT	PCT ≥ 0·8	Clinically and microbiologically documented infection	ICU	100% 62·0%	N.A
Ahn S 2011 Korea 12	Adult	0·31 (96)	PCT	PCT ≥ 0·3	Microbiologically documented infection	ED	76·2% 60·6%	N.A
Cuquemelle E 2011 France 13	Adult	0·47 (103)	PCT	PCT ≥ 0·8	Microbiologically documented infection	ICU	91·0% 68·0%	N.A
Song JY 2011 Korea 17	Adult	0·19 (81)	PCT, CRP	PCT ≥ 0·35 CRP ≥ 86·5	Clinically and microbiologically documented infection	Inpatient	81·0% 66·7%	81·8% 59·3%
Han SS 2011 Korea 15	Adult	0·37 (73)	PCT	PCT ≥ 0·019	Clinically and microbiologically documented infection	Outpatient	74·1% 70·0%	N.A

PCT, procalcitonin; CRP, C‐reactive protein; ICU, intensive care unit; ED, emergency department; NA, not available.

Diagnostic accuracy indices

Results of our meta‐analysis showed that PCT was a more sensitive rather than a specific test for diagnosing secondary bacterial pneumonia among influenza virus‐infected patients. Figure 3 shows a plot of our summary ROC curve. PCT had an area under the ROC curve of 0·68 (95% CI: 0·64–0·72). The pooled sensitivity and specificity estimates were 0·84 (95% CI: 0·75–0·90) and 0·64 (95% CI: 0·58–0·69), respectively (Figure 4).

Figure 3

 Summary receiver operating characteristic (ROC) curve. This figure shows the summary ROC (solid line), the bivariate summary estimate (solid square), and the corresponding 95% confidence ellipse (inner dashed line) and 95% prediction ellipse (outer dotted line). Symbol size for each study is proportional to the study size.

Figure 4

 Forest plots. Forest plots for (A) sensitivity and (B) specificity for studies using procalcitonin to detect superimposed bacterial infection among patients with influenza virus infection.

As the positive likelihood ratio (LR+) usually needs to be >5 to be recommended as a good rule‐in test, the low LR+ of the PCT test (2·31; 95% CI: 1·93–2·78) did not provide sufficient power to recommend its use as a rule‐in test. However, its low negative likelihood ratio (LR− = 0·26; 95% CI: 0·17–0·40) suggests that the PCT test is a reasonably reliable rule‐out test. We did not observe any substantial degree of heterogeneity for PCT (I 2=0%; 95% CI; 0·0–74·62).

Subgroup analyses

We conducted subgroup analyses by restricting studies to those with similar study settings and outcome definitions. The pooled sensitivity estimates improved appreciably after considering ICU populations only (0·91; 95% CI: 0·82–0·97) or considering studies using MDI as the sole outcome definition (0·89; 95% CI: 0·77–0·92). Although the pooled specificities were slightly decreased in these two subgroups, the global measure of accuracy (i.e., AUROC or diagnostic OR) improved appreciably. The positive likelihood ratios for PCT tests for these two subgroups were still not sufficient to rule‐in bacterial pneumonia among patients with influenza pneumonia, but the negative likelihood ratios, particularly in the ICU populations, were sufficiently low to rule out the possibility of bacterial coinfection among these patients.

No significant evidence of possible publication bias was noted by Egger’s test for asymmetry of a funnel plot (Table 2). An exploratory meta‐regression analysis did not find that any pre‐specified covariate significantly changed the effect estimates.

Table 2

Subgroup analysis of the included studies

Variables	Number of studies	Sensitivity (95% CI)	Specificity (95% CI)	Likelihood ratio+	Likelihood ratio‐	AUROC (95% CI)	Diagnostic OR (95% CI)	I 2 (95% CI)	Publication bias (Egger’s test P)
Procalcitonin
Overall analysis 12 , 13 , 14 , 15 , 16 , 17	6	0·84 (0·75–0·90)	0·64 (0·58–0·69)	2·31 (1·93–2·78)	0·26 (0·17–0·40)	0·68 (0·64–0·72)	8·55 (5·04–14·5)	0·0 (0·0–74·62)	0·518
ICU 13 , 14 , 16	3	0·91 (0·82–0·97)	0·62 (0·52–0·72)	2·53 (1·90–3·30)	0·14 (0·06–0·31)	0·73 (0·50–0·99)	17·9 (6·98–45·8)	0·0 (0·0–84·69)	0·311
MDI 12 , 13 , 14	3	0·89 (0·77–0·92)	0·61 (0·53–0·67)	2·30 (1·82–2·93)	0·23 (0·14–0·39)	0·79 (0·49–0·99)	9·93 (3·49–28·3)	46·1 (0·0–84·1)	0·518

ICU, intensive care unit; MDI, microbiologically documented infection.

Discussion

Influenza virus causes a high disease burden worldwide. It is estimated that 50–80 million people would die if a pandemic flu would occur now that had a similar magnitude and severity to that of the 1918 pandemic flu. Clinical and pathological evidence suggests that a secondary bacterial infection is the major cause of death among patients with influenza virus infections. ,

Given the limited stockpiles of respiratory antibiotics available during flu pandemics and the possibility of rapidly emerging drug resistance, prophylactic antibiotics are not recommended for all influenza‐infected patients with lower respiratory tract symptoms. A reliable screening test that can rapidly differentiate viral from bacterial pneumonia is of paramount importance for managing a large number of patients. Unfortunately, clinical characteristics, chest radiographic findings, and routine laboratory results are unreliable for distinguishing viral from bacterial pneumonia. ,

Several randomized clinical trials have recently shown that it may be possible to safely withhold antibiotics based on PCT test results. , The usefulness of a PCT test among patients with influenza virus infection, however, has just begun to be investigated. Owing to the small sample sizes in the reported studies, results are difficult to interpret. Therefore, a meta‐analysis might be useful for clinicians to understand the actual utility of PCT in this setting and at this stage.

However, it is worth noting that diagnostic test performance can be influenced by disease prevalence and other population characteristics. Therefore, clinicians are advised to use the results from the subgroup that most closely match their own clinical circumstances. For example, a superior performance of the PCT test was shown for ICU populations compared with emergency department or outpatient populations.

In our analysis, a PCT test exhibited a suboptimal rule‐in value for confirming bacterial coinfection. In a population with a 30% prevalence (pre‐test probability) of mixed influenza and bacterial pneumonia, the positive predictive value (post‐test probability) for PCT test would only be 50%. That is, approximately one in two patients with positive PCT results could be expected to have either clinically or microbiologically confirmed secondary bacterial pneumonia. In contrast, the diagnostic value for PCT to rule‐out bacterial coinfection among H1N1 influenza virus‐infected patients was reasonably high. In the same population with a 30% prevalence of mixed influenza and bacterial pneumonia, the negative predictive value for a PCT test could be as high as 90%. Thus, only one in 10 patients with negative PCT results might turn out to have either clinically or microbiologically confirmed secondary bacterial pneumonia.

It may be argued that the rule‐out role for a PCT test in a real clinical setting is still limited given the huge cost of missing a patient with bacterial pneumonia. However, a perfect biomarker with a near zero false‐negative rate cannot be expected in the foreseeable future. A practical way to make the best use of a PCT test may be either to do serial tests to increase its sensitivity (and, hence, lower the false‐negative rate) or to develop an algorithm that incorporates known risk factors, such as older age or an immuno‐compromised state, and clinical variables to make treatment decisions for patients with gray‐zone PCT test results (e.g., 0·25–0·5 ng/ml). A strategy such as this has been shown by randomized controlled trials to successfully reduce the use of empirical antibiotics without impairing the outcomes among patients with community‐acquired pneumonia.

The goal of our systematic review was to critically evaluate the diagnostic accuracy of PCT tests for the diagnosis of mixed bacterial and influenza pneumonia among influenza‐infected patients with lower respiratory tract symptoms. However, there are inherent limitations with regard to sensitivity, specificity, and likelihood ratios as measures of test performance. They do not address the possible added diagnostic value to current practice based on clinical information and routine imaging and laboratory tests. They also do not address whether incorporating a PCT test in the diagnostic workup of patients with influenza will change physicians’ antibiotics prescribing behaviors and ultimately improve patient outcomes. These questions can best be answered with randomized control trials. Prior to this, the currently available data suggest that PCT tests may add to conventional diagnostic workups for identifying bacterial coinfections in patients with influenza pneumonia.

Our meta‐analysis had several limitations. First, current standard criteria for defining a secondary bacterial infection among patients with influenza pneumonia tend to under‐diagnose the disease; the lack of a gold standard disease verification tool may thus affect the evaluation of the index test (PCT). Second, it is likely that unpublished data and ongoing studies were missed. Although we did not observe any signs of significant publication bias, it is also possible that studies that found poor PCT performance were less likely to be published.

In conclusion, the data suggest that PCT tests have a high sensitivity, particularly for ICU patients, but a low specificity for identifying secondary bacterial infections among patients with influenza. Because of its suboptimal positive likelihood ratio and good negative likelihood ratio, it can be used as a suitable rule‐out test but cannot be used as a standalone rule‐in test. These data should help inform evidence‐based practice guidelines for the management of influenza infection.

Conflict of interest

None declared.