A new scoring system for early diagnosis of ventilator-associated pneumonia: LUPPIS.

INTRODUCTION: The Clinical Pulmonary Infection Score (CPIS) based on chest X-ray has been developed to facilitate clinical diagnosis of ventilator-associated pneumonia (VAP); however, this scoring system has a low diagnostic performance. We developed the Lung Ultrasound and Pentraxin-3 Pulmonary Infection Score (LUPPIS) for early diagnosis of VAP and evaluated the performance of this new scoring system.
MATERIAL AND METHODS: In a prospective study of 78 patients with suspected VAP, we assessed the detection accuracy of LUPPIS for pneumonia in adult patients. We also evaluated the diagnostic performance of pentraxin-3 (PTX-3) findings of infection. On the day of the study, lung ultrasound was performed, PTX-3 levels were determined, and an endotracheal aspirate was obtained for Gram staining and culture.
RESULTS: No significant differences were found between groups with respect to age, mechanical ventilation time, APACHE II score, or SOFA score (p > 0.05). Procalcitonin and PTX-3 levels were significantly higher in the VAP (+) group (p < 0.001 and p < 0.001, respectively). The threshold for LUPPIS in differentiating VAP (+) patients from VAP (-) patients was > 7. In predicting VAP, LUPPIS > 7 (sensitivity of 84%, specificity of 87.7%) was superior to CPIS > 6 (sensitivity of 40.1%, specificity of 84.5%).
CONCLUSIONS: LUPPIS appears to provide better results in the prediction of VAP compared to CPIS, and the importance of lung ultrasound and PTX-3 is emphasized, which is a distinctive property of LUPPIS. Copyright:

Introduction

Ventilator-associated pneumonia (VAP) is hospital-acquired pneumonia developing at least 48 h after intubation [1]. VAP is a common and severe problem in intensive care units (ICU) that affects 10% of critically ill patients receiving mechanical ventilation support, and there has been no decline in this rate in the last decade [2]. VAP prolongs mechanical ventilation (MV) time and the length of hospital stay, and it is associated with increased mortality (up to 70%) [3-5]. VAP accounts for 50% of antibiotic consumption in the intensive care unit [6, 7].

Pentraxins are a family of a proteins involved in the acute phase inflammatory response. In response to inflammatory stimuli, C-reactive protein (CRP), which is a short pentraxin, and interleukin-6 are produced in the liver [8]. Pentraxin-3 (PTX-3), the prototype of long pentraxins, primarily acts as a receptor in the activation of the immune system similar to short pentraxins [9]. However, this molecule differs from CRP in structure and gene organization, cellular source, stimuli resulting in its release, and ligand recognition pathways [10]. PTX-3 is elevated in many infections, and there is a correlation between its elevation and the severity of disease [11-13]. Diagnostic and prognostic performances of PTX-3 levels have been evaluated in the diagnosis of VAP; PTX-3 levels in bronchoalveolar lavage fluid were found to have diagnostic value, and plasma PTX-3 levels were found to have prognostic value [14].

Early diagnosis of VAP remains challenging for intensivists due to lack of a gold standard diagnostic method [1]. The diagnosis is primarily based on clinical findings, and the Clinical Pulmonary Infection Score (CPIS) has been developed to facilitate clinical diagnosis; however, this scoring system has a low diagnostic performance [15]. Lung ultrasound is becoming a widespread practice in evaluating lung pathologies in the ICU setting. It was suggested that a lung ultrasound score (LUS) could be reliably used in the diagnosis and follow-up of patients with VAP [16].

The aim of this study was to evaluate the performance of a new clinical scoring system for early diagnosis of VAP in critically in patients, which includes clinical infection signs, LUS score, and PTX-3 levels.

Material and methods

Patient selection and study design

This single-center, observational, prospective study (Ethics Committee approval number: 70737436-050.06.04) included 78 patients, who received therapy as an inpatient in the Intensive Care Unit between January 2015 and April 2016 and who were suspected of having VAP.

The suspicion of VAP was evaluated using classical clinical criteria: MV time ≥ 48 h, new or progressive infiltrations on chest X-ray and presence of two or more clinical criteria: fever (≥ 38.5°C) or hypothermia (< 36.5°C), leukocytosis (white blood cell (WBC) count > 104/ml) or leukopenia (WBC count < 2.103/ml), and purulent tracheal secretions, PaO2/FiO2 < 300. Exclusion criteria were ongoing pneumonia, exacerbations of chronic obstructive pulmonary disease (COPD), those with non-pulmonary infection at the time of suspicion of VAP and contraindication to fiberoptic bronchoscopy.

Patients were included in the study at the time VAP was suspected. At inclusion we calculated CPIS, performed fiberoptic bronchoalveolar lavage (BAL), and endotracheal aspirate (EA) was obtained for Gram staining and culture. PTX-3, procalcitonin (PCT), and CRP serum levels, leukocyte count, body temperature, APACHE II score, SOFA score, and MV time were recorded. Temperature measurement was performed by the tympanic route. Lung ultrasound (LUS), BAL, and EA were performed in the first 8 h upon suspicion that VAP had emerged.

Venous blood samples from the patients were collected to measure serum PTX-3 levels. Thirty minutes after drawing blood samples, tubes were centrifuged for 10 min at 1500 rpm. Samples were aliquoted and stored at –80°C. Serum PTX-3 levels were measured using a commercial ELISA kit. PTX-3 concentrations in the samples were determined by comparing the optical density of each individual sample with the standard curve. The intra-assay coefficient of variation for the assay was 4–6%.

Clinical samples for microbiological culture comprising BAL and cultures were processed using standard microbiological methods. Identification of isolates was performed with the VITEK (bioMérieux, Durham, NC) and API automated systems (bioMérieux, Marcy l’Etoile, France). VAP diagnosis was confirmed according to either tracheal culture positivity or, in tracheal culture-negative patients, the presence of all clinical criteria with an initiated or modified antibiotic regimen within 48 h. The patients were retrospectively grouped as VAP (+) or VAP (–).

Laboratory parameters

Pentraxin-3, PCT and CRP levels were analyzed using the enzyme-linked immunosorbent assay (ELISA) test method (Boster Biological Technology Co. Ltd ELISA).

LUS score

Each hemithorax was assessed using a 1–5 MHz convex probe and by dividing the hemithorax into six areas: after dividing the hemithorax into anterior, lateral and posterior sections based on anterior and posterior axillary lines, each section was divided into superior and inferior halves (Figure 1) [17]. LUS score was calculated after ultrasound examination of the lungs [16]:

Figure 1

A – Lung ultrasonography. B – Appearance of subpleural consolidation on lung ultrasound. Air bronchogram with punctate echogenicity in the ground of hypoechoic subpleural consolidation

Pl – pleural line, Con – consolidation, B li – B lines.

≥ 2 areas with subpleural consolidation, 1 point;

≥ 1 area with dynamic arborescent/linear air bronchogram, 2 points.

Score definition

The Lung Ultrasound and Pentraxin-3 Pulmonary Infection Score (LUPPIS) recently developed by the authors included the following changes:

Chest radiograph was replaced by LUS score;

Leukocyte count was replaced by PTX-3 concentration;

Culture of tracheal aspirate significance was considered positive if the count was 104 colony-forming units/ml;

Tracheal secretion was considered positive only if purulent. Definition of tracheal purulence was made by visual assessment by physicians.

The present study used CPIS as the control method, which is widely used in predicting VAP (Table I).

Table I

Proposed LUPPIS compared with original CPIS

Parameter	Points
	0	1	2
CPIS:
Temperature [°C]	≥ 36 and < 38.4	≥ 38.5 and < 38.9	< 36 or ≥ 39
Blood leukocytes [WBC/mm3]	≥ 4,000 and ≤ 11,000	< 4,000 or > 11,000	< 4,000 or > 11,000 and band forms ≥ 500
Oxygenation: PaO2/FiO2	> 240 or ARDS		≤ 240 and no evidence of ARDS
Tracheal secretions	Absent	Nonpurulent	Purulent
Pulmonary radiography	No infiltrate	Diffuse (or patchy) infiltrate	Localized infiltrate
Culture of tracheal aspirate	Pathogenic bacteria cultured in rare or small quantity or no growth	Pathogenic bacteria cultured in moderate or large quantity	Same pathogenic bacteria seen on Gram stain
LUPPIS:
Temperature [°C]	≥ 36 and < 38.4	≥ 38.5 and < 38.9	< 36 or ≥ 39
Pentraxin-3 [ng/ml]	< 2		≥ 2
Oxygenation: PaO2/FiO2	> 240 or ARDS		≤ 240 and no evidence of ARDS
Tracheal secretions	Absent	Nonpurulent	Purulent
LUS Score		2 areas with subpleural consolidation	≥ 1 areas with dynamic arborescent/linear air bronchogram
Culture of tracheal aspirate	Negative		Positive

ARDS – acute respiratory distress syndrome, CPIS – Clinically Pulmonary Infection Score, LUPPIS – Lung Ultrasound and Pentraxin-3 Pulmonary Infection Score, LUS – lung ultrasound.

Statistical analysis

For this pilot study, we could not establish the number of patients needed or perform a power analysis. Consecutive patients were included as recommended [18]. R v.215.3 (R Core Team, 2013) software was used in statistical analyses. Statistical uncertainty was indicated by reporting 95% CIs. The data were expressed as mean, standard deviation, median, interquartile range, frequency, percentage, minimum and maximum. The Mann-Whitney U test was used to check the difference of non-normally distributed variables between the two groups. Pearson’s χ2 test and Fisher’s exact test were used for comparison of nominal data. ROC curves were built, sensitivity and specificity of variables were calculated for various values, and the value of the highest Youden index was taken as a cut-off point. The areas under the ROC curves were compared using the DeLong method. The level of statistical significance was set at p < 0.05.

Results

The age of 78 patients included in the study ranged from 18 to 85 years, and the mean age was 58.14 ±16.72 years. Of these patients, 32 were grouped as VAP (+), and 46 were grouped as VAP (–). Of 32 patients diagnosed with ventilator-associated pneumonia, microbiologically confirmed VAP was detected in 26 patients. Microbiological analysis of the respiratory tract pathogens showed that 70% were Gram-negative organisms, 26.7 % were Gram-positive organisms, and 3.3% were Candida spp. In the Gram-negative bacilli group, was the most common species Acinetobacter baumannii (37.8%), followed by Klebsiella pneumoniae, Pseudomonas aeruginosa, Escherichia coli, and Proteus mirabilis. In the Gram-positive cocci group, Staphylococcus aureus was the most prevalent. There were no significant differences between groups in age, MV time, leukocyte and CRP values, APACHE II score, or SOFA score (p > 0.05). PCT and PTX-3 levels were significantly higher in the VAP (+) group (p < 0.001 and p < 0.001, respectively) (Table II).

Table II

Clinical characteristics of the patients

Parameter	VAP (–) (n = 46)	VAP (+) (n = 32)	P-value
Age [years]	65 (49, 71)	58.5 (44, 66.5)	0.109
MV [days]	7 (5, 10)	8 (5, 12.5)	0.426
CRP [ mg/l]	210 (117)	186 (194)	0.11
Leukocytes [WBC/mm3]	12.5 (9.8, 15.9)	14.1 (11.1, 18.9)	0.326
APACHE II	17 (14, 20)	17 (15, 22)	0.255
SOFA	6 (5, 7)	6.6 (6, 7.5)	0.157
Procalcitonin [ng/ml]	0.3 (0.13, 0.46)	1.95 (0.66, 2.61)	< 0.001
PTX-3 [ng/ml]	1.94 (1.63, 1.98)	6.21 (2.82, 9.34)	< 0.001

MV – mechanical ventilation, APACHE II – Acute Physiology and Chronic Health Evaluation, SOFA – Sequential Organ Failure Score, PTX-3 – pentraxin 3, CRP – C-reactive protein.

The area under the curve was 1.000 in ROC curve analysis performed for PTX-3 (AUC (95% CI) = 1.000 (1.000, 1.000), p < 0.001). ROC curve analysis for PTX-3 levels revealed an AUC of 0.832 (AUC (95% CI) = 0.832 (0.723, 0.941), p < 0.001) (Figure 2). The sensitivity for PTX-3 was 93.75%, specificity was 100%, positive predictive value (PPV) was 100%, negative predictive value (NPV) was 95.8%, and the cut-off level was ≥ 2 (Table III). Accordingly, in the recently developed LUPPIS, the score was 0 points for patients with PTX-3 < 2, and 2 points for patients with PTX-3 ≥ 2.

Figure 2

Comparison of areas under PTX3, PCT, CPIS and LUPPIS ROC curves

Table III

Assessment of cut-off values for pentraxin-3

PTX-3	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
> 0.97	100 (89.1, 100.0)	15.22 (6.3, 28.9)	45.1 (33.2, 57.3)	100 (59.0, 100.0)
> 1.54	100 (89.1, 100.0)	21.74 (10.9, 36.4)	47.1 (34.8, 59.6)	100 (69.2, 100.0)
> 1.906	100 (89.1, 100.0)	43.48 (28.9, 58.9)	55.2 (41.5, 68.3)	100 (83.2, 100.0)
> 1.98	100 (89.1, 100.0)	80.43 (66.1, 90.6)	78 (62.4, 89.4)	100 (90.5, 100.0)
> 2	93.75 (79.2, 99.2)	100 (92.3, 100.0)	100 (88.4, 100.0)	95.8 (85.7, 99.5)
> 2.17	87.5 (71.0, 96.5)	100 (92.3, 100.0)	100 (87.7, 100.0)	92 (80.8, 97.8)
> 6.18	50 (31.9, 68.1)	100 (92.3, 100.0)	100 (79.4, 100.0)	74.2 (61.5, 84.5)
> 9.03	28.12 (13.7, 46.7)	100 (92.3, 100.0)	100 (66.4, 100.0)	66.7 (54.3, 77.6)
> 13.9	0 (0.0, 10.9)	100 (92.3, 100.0)	0	59 (47.3, 70.0)

PPV – positive predictive value, NPV – negative predictive value, PTX-3 – pentraxin 3.

The cut-off value for LUPPIS in differentiating patients with and without ventilator-associated pneumonia was > 7 (sensitivity of 87.5%, specificity of 91.3%, PPV of 87.5%, and NPV of 91.3% (Table IV). For CPIS > 6, sensitivity was 43.8%, specificity was 82.6%, PPV was 63.6%, and NPV was 67.9% (Table V).

Table IV

Assessment of cut-off values for LUPPIS

LUPPIS	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
> 2	100 (89.1, 100.0)	28.3 (16.0, 43.5)	49.2 (36.6, 61.9)	100 (75.3, 100.0)
> 3	100 (89.1, 100.0)	52.2 (36.9, 67.1)	59.3 (45.0, 72.4)	100 (85.8, 100.0)
> 4	100 (89.1, 100.0)	60.9 (45.4, 74.9)	64 (49.2, 77.1)	100 (87.7, 100.0)
> 5	96.9 (83.8, 99.9)	71.7 (56.5, 84.0)	70.5 (54.8, 83.2)	97.1 (84.7, 99.9)
> 6	96.9 (83.8, 99.9)	78.3 (63.6, 89.1)	75.6 (59.7, 87.6)	97.3 (85.8, 99.9)
> 7	87.5 (71.0, 96.5)	91.3 (79.2, 97.6)	87.5 (71.0, 96.5)	91.3 (79.2, 97.6)
> 8	34.4 (18.6, 53.2)	100 (92.3, 100.0)	100 (71.5, 100.0)	68.7 (56.2, 79.4)
> 9	25 (11.5, 43.4)	100 (92.3, 100.0)	100 (63.1, 100.0)	65.7 (53.4, 76.7)
> 10	12.5 (3.5, 29.0)	100 (92.3, 100.0)	100 (39.8, 100.0)	62.2 (50.1, 73.2)
> 11	0 (0.0, 10.9)	100 (92.3, 100.0)	0	59 (47.3, 70.0)

PPV – positive predictive value, NPV – negative predictive value, LUPPIS – Lung Ultrasound and Pentraxin-3 Pulmonary Infection Score.

Table V

Assessment of cut-off values for CPIS

CPIS	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
> 3	100 (89.1, 100.0)	21.7 (10.9, 36.4)	47.1 (34.8, 59.6)	100 (69.2, 100.0)
> 4	100 (89.1, 100.0)	47.8 (32.9, 63.1)	57.1 (43.2, 70.3)	100 (84.6, 100.0)
> 5	78.1 (60.0, 90.7)	69.6 (54.2, 82.3)	64.1 (47.2, 78.8)	82.1 (66.5, 92.5)
> 6	43.8 (26.4, 62.3)	82.6 (68.6, 92.2)	63.6 (40.7, 82.8)	67.9 (54.0, 79.7)
> 7	37.5 (21.1, 56.3)	100 (92.3, 100.0)	100 (73.5, 100.0)	69.7 (57.1, 80.4)
> 8	21.9 (9.3, 40.0)	100 (92.3, 100.0)	100 (59.0, 100.0)	64.8 (52.5, 75.8)
> 9	18.8 (7.2, 36.4)	100 (92.3, 100.0)	100 (54.1, 100.0)	63.9 (51.7, 74.9)
> 10	6.3 (0.8, 20.8)	100 (92.3, 100.0)	100 (15.8, 100.0)	60.5 (48.6, 71.6)
> 11	0 (0.0, 10.9)	100 (92.3, 100.0)	0	59 (47.3, 70.0)

PPV – positive predictive value, NPV – negative predictive value, CPIS – Clinically Pulmonary Infection Score.

Although LUPPIS > 7 and CPIS > 6 yield comparable specificity, sensitivity of LUPPIS > 7 was higher compared with CPIS > 6. The risk of having VAP was 73.50-fold higher in patients with LUPPIS > 7 compared to patients with LUPPIS ≤ 7 (OR (95% CI) = 73.50 (16.97, 318.42), p < 0.001). The OR for CPIS > 6 was 3.69 (OR (95% CI) = 3.69 (1.31, 10.39), p < 0.001).

The AUC was 0.822 in ROC curve analysis for CPIS (AUC (95% CI) = 0.822 (0.719, 0.899), p < 0.001) (Figure 2). The AUC was 0.952 in ROC curve analysis for LUPPIS (AUC (95% CI) = 0.952 (0.879, 0.988), p < 0.001) (Figure 2). The difference between the areas under the ROC curves was found to be 0.130 using the DeLong method, and this difference was statistically significant (p = 0.001) (Table VI).

Table VI

Comparison of areas under ROC curves for CPIS and LUPPIS

Parameter	Area	Standard error	95% confidence interval		P-value
			Lower	Upper
CPIS	0.822	0.046	0.719	0.899	< 0.001
LUPPIS	0.952	0.022	0.879	0.988	< 0.001
Difference	0.130	0.040	0.051	0.209	0.001

CPIS – Clinically Pulmonary Infection Score, LUPPIS – Lung Ultrasound and Pentraxin-3 Pulmonary Infection Score.

Discussion

General opinions

In the present study, the utility of the new scoring system (LUPPIS) which was developed by the authors and based on five parameters – lung ultrasound, PTX-3 values, oxygenation, body temperature, and tracheal aspirate culture – was evaluated, and LUPPIS was shown to be a valuable method for the diagnosis of VAP.

Many non-infectious processes may cause fever and pulmonary infiltration in patients receiving mechanical ventilatory support, and for this reason, symptoms of VAP are nonspecific [15]. The clinical approach suggests initiation of broad-spectrum antibiotics to all patients suspected of having VAP [1]. However, this approach results in an increased rate of multi-drug resistant bacterial strains [19, 20]. Treatment of microbiologically diagnosed patients only may cause a delay in the initiation of antibiotics and increase mortality [21, 22]. Initiation of appropriate antibiotics without wasting time while avoiding these two extreme approaches is possible with early diagnostic methods for VAP.

In the earliest study, which included 28 patients, CPIS > 6 showed a sensitivity of 93% and a specificity of 100% in predicting VAP [23]. However, later studies that compared CPIS with pathological diagnosis [15] and diagnosis based on bronchoalveolar liquid culture [24] have reported low diagnostic performance for CPIS in predicting VAP. Modified CPIS incorporating Gram staining has shown improved diagnostic performance and sensitivity, but specificity remained at a suboptimal level [24]. Although CPIS has little diagnostic value for VAP, it is currently the most commonly used scoring system.

Pentraxin-3

Soluble triggering receptor expressed on myeloid cells type 1 (sTREM-1), PCT, and CRP are distinguished from other markers playing a supporting role in the diagnosis and management of ventilator-associated pneumonia; however, these markers have variable sensitivities and specificities in predicting VAP [25].

PTX-3, an acute phase inflammatory protein, is locally produced in the infection area, epithelium, endothelial cells, and leukocytes, and plasma levels of PTX-3 correlate with the severity of disease [26, 27]. PTX-3 levels increase within 6–8 h in endotoxic shock and sepsis and reach peak concentrations [28-30]. There is a lack of sufficient information regarding its half-life; however, exogenously administered PTX-3 was shown to have a half-life of 1 h [31]. In an experimental pneumonia model, PTX-3 was shown to differentiate various causes of infections (i.e. bacteria, virus, fungus) [32-34]. Also, plasma PTX-3 levels were also shown to be elevated in VAP and community-acquired pneumonia [11, 35]. In the study by Mauri et al., PTX-3 levels in bronchoalveolar lavage fluid of intubated patients in the ICU were shown to be an early marker of pneumonia with high negative predictive value [14]. Despite the limited number of clinical studies available, PTX-3 is a promising biomarker that could be used in early diagnosis of community-acquired and healthcare-related pneumonia [36]. We, therefore, selected PTX-3 in LUPPIS instead of leukocyte count, which is used in CPIS.

LUS

The diagnosis and follow-up of VAP are currently based on chest X-ray; however, bedside chest X-ray offers poor quality and limited reliability [37-40]. Of patients receiving mechanical ventilatory support in the ICU, 38% have an abnormal appearance in their chest X-rays [41]. Pulmonary infiltrations due to non-infectious causes often complicate detection of infiltrations due to VAP in critically ill patients [16]. The likelihood of an opacity observed on a chest X-ray being due to pneumonia ranges from 27 to 35% [42, 43]. Also, the limited diagnostic performance of bedside chest X-ray [40] complicates detection of VAP. The LUS, the use of which is recommended in critically ill patients [44], provides reliable information about the condition [45], aeration [46], perfusion [47], and morphology [48] of the lungs. The use of lung ultrasonography in the ICU is becoming more prevalent [49] than the chest X-ray with the advantages of being a non-invasive, easily reproducible method with the availability of bedside application and a short learning period [50].

LUPPIS

VAP is multifocal process with different histopathological patterns, so single consolidation in LUS is unlikely to show sufficient sensitivity and specificity for pneumonia [16]. In addition, Mongodi et al. reported that high sensitivity but poor specificity for small subpleural consolidation and high specificity but poor sensitivity for consolidations with dynamic air bronchograms [16]. Regarding the use of LUS in conjunction with laboratory tests, a retrospective study combining consolidation on lung ultrasound with procalcitonin levels showed it to be superior to CPIS in predicting VAP [51]. Mongodi et al. also reported high specificity for one or more areas with small subpleural consolidations detected on LUS when associated with a positive culture of tracheal aspirate [16]. In our study, we found that simple changes in some parameters (e.g., chest radiography and leukocyte count were replaced with LUS score and PTX-3 level, respectively) significantly increased both the sensitivity and the specificity of LUPPIS.

Limitations

Our study has several limitations, due to which it should be considered a pilot, exploratory analysis of a new type of score. LUS is operator-dependent, and the operator must receive training. Obese patients with subcutaneous emphysema or large thoracic wound dressings may pose difficulties in lung ultrasound, and this may affect diagnostic performance of the scoring system. The present study was conducted only on 78 patients; the results of this study must be evaluated in a larger population. The studies that showed the CPIS score not to be accurate recruited 700 patients, so we cannot exclude that the present study may be underpowered.

In conclusion, the present study attempted to introduce a fast, cheap, and reliable approach to the diagnosis of VAP using the bedside applicability advantages of lung ultrasound. For this purpose, we made some changes in the parameters of commonly used CPIS and developed a new scoring system incorporating the LUS score and PTX-3, showing good diagnostic performance in VAP.

Conflict of interest

The authors declare no conflict of interest.