Validation of a prediction rule for prognosis of severe community-acquired pneumonia.

In a previous study, we developed a prognostic prediction rule, based on nine prognostic variables, capable to estimate and to adjust the mortality rate of patients admitted in intensive care unit for severe community-acquired pneumonia. A prospective multicenter study was undertaken to evaluate the performance of this rule. Five hundred eleven patients, over a 7-year period, were studied. The ICU mortality rate was 29.0%. In the 3 initial risk classes, we observed significantly increasing mortality rates (8.2% in class I, 22.8% in class II and 65.0% in class III) (p<0.001). Within each initial risk class, the adjustment risk score identified subclasses exhibiting significantly different mortality rates: 3.9% and 33.3% in class I; 3.1%, 12.9% and 63.3% in class II; and 55.8% and 82.5% in class III. Compared with mortality rates predicted by our previous study, only a few significant differences were observed. Our results demonstrate the performance and reproductibility of this prognostic prediction rule.

INTRODUCTION

Community-acquired pneumonia (CAP) is the most frequent community-acquired infection requiring intensive care unit (ICU) admission [1]. Approximately 10% of hospitalized patients with CAP require admission to the ICU [2]. Despite advances in supportive measures and updated guidelines for antimicrobial treatment, mortality associated with severe CAP remains high, around 20-30% [3-6].

As demonstrated by several multivariate analyses, prognosis of severe CAP depends on the baseline characteristics of patients, on the initial severity of pneumonia and on the evolution during ICU stay [7-9]. Taking into account these results, we developed in a previous work a prognostic prediction rule based on two successive steps: an initial score, based on clinical and radiological data collected at presentation, estimated ICU outcome early after ICU admission. A second score, taking into account the complications occurring during ICU stay, adjusted the final ICU prognosis [3].

Our goal was to evaluate, in a prospective multicenter study, the performance of this prognostic prediction rule for patients admitted into ICU for severe CAP.

PATIENTS AND METHODS

Selection of Cases

From January 1996 to December 2003, we prospectively enrolled, in the validation population, patients admitted to five ICUs with a diagnosis of severe CAP. The study sites and the years of data collection were Tourcoing General Hospital (16-bed ICU affiliated with the Faculté de Médecine de Lille and associated with an infectious diseases unit) from 1996 to 2003, Saint Philibert Hospital (12-bed ICU affiliated with the Université Catholique de Lille) in 1997 and 2002, Arras General Hospital (10-bed ICU) from 1996 to 2002, Lens General Hospital (15-bed ICU) in 1996 and Valenciennes General Hospital (19-bed ICU) from 1997 to 2002. The ethics committee of the Lille University Medical School was informed about the study and raised no objections. Patient consent was not required for this observational study.

Enrolment criteria for patient inclusion and exclusion were those used in our previous study, based on a development population including 472 patients [3]. CAP was defined by the following criteria observed at initial presentation or occurring within 48h following hospitalization [10]: presence of a new radiographic pulmonary infiltrate and acute onset of at least one "major" (cough, sputum production, fever) or two "minor" (dyspnea, pleuretic chest pain, altered mental status, pulmonary consolidation on physical examination, total leukocyte count greater than 12.000/mm3) clinical or biological findings suggestive of pneumonia. Patients coming from nursing home or hospitalized within 30 days prior to developing pneumonia or hospitalized > 48h in general medical wards before ICU admission or with a known positive HIV test or with a diagnosis of AIDS were excluded. Finally, patients with Do-Not-Resuscitate or Do-Not-Intubate orders were not included.

Data Collection and Determination of the Prognostic Prediction Rule

Fig. () details the prognostic prediction rule with an initial classification of patients, on ICU admission, in 3 initial risk classes according to the initial risk score, and the stratification, in each initial risk class, in different subclasses according to the adjustment risk score.

Within 24h of ICU admission, all patients underwent clinical, biological and radiologic evaluation aiming to determine main characteristics of patients. These characteristics were used to determine an initial mortality risk score [3]. This score is based on six independent predictors of mortality. Predictors and their point scoring system are age ≥ 40 years (+1 point), anticipated death within 5 years (+1 point), nonaspiration pneumonia (+1 point), chest radiograph involvement > 1 lobe (+1 point), acute respiratory failure requiring invasive mechanical ventilation (+1 point) and septic shock (+3 points). In adding the points, we obtain the initial risk score. According to its value, each patient is included in one of the three initial risk classes: Class I = score between 0 and 2; Class II = score between 3 and 5; Class III = score between 6 and the maximal value 8.

During ICU stay, all complications occurring beyond the 24 hours following the admission were recorded. An adjustment risk score was determined, for each patient, according to the occurrence or non occurrence of complications [3]. This score is based on 3 independent predictors of mortality. The predictors and their point scoring system are hospital-acquired lower respiratory tract (HA-LRT) superinfections (+ 1 point), non-specific complications (+ 2 points) and sepsis-related complications (+ 4 points). This score varies between 0 point (no complication) and 7 points (occurrence of the 3 different complications). This score allows to precise the final prognosis in each initial risk class in distinguishing in each of them 2 or 3 different subclasses with adjusted risk, as follows: in class I, two subclasses are distinguished according to an adjustment risk score ≤ 2 or > 2; in class II, 3 subclasses are distinguished according to a score = 0, > 0 and ≤ 2 or > 2; in class III, two subclasses are distinguished according to an adjustment risk score ≤ 2 or > 2.

Patients were followed until discharge from the ICU or death in ICU.

In each ICU and for each enrolled patient, a physician collected data using a standardized data collection sheet, when the patient was discharged from the ICU or when the patient died. All sheets were centralized by two physicians of our group (OL and DM) who calculated the score. All recorded data were analyzed by a biostatistician (PD).

Definitions

The underlying clinical conditions were classified according to the criteria proposed by McCabe and Jackson [11] in 3 categories: non fatal, ultimately fatal and rapidly fatal. Patients included in the category ultimately or rapidly fatal were considered as having an anticipated death within 5 years. Septic shock was defined by usual criteria [12, 13]; hypotension is defined by a systolic blood pressure < 90 mm Hg or its decrease by at least 40 mm Hg from baseline a mean arterial pressure or a mean arterial pressure < 40 mmHg, in the absence of other causes for hypotension, persisting despite adequate volume replacement. The need for invasive mechanical ventilation within 12h following ICU admission defined initial acute respiratory failure. Diagnosis of aspiration was retained in patients with either witnessed aspiration or risk factors for aspiration (altered mental status, abnormal gag reflex of swallowing mechanism, intestinal obstruction) and suggestive chest X-ray infiltrates [14].

Among complications occurring during the patient's stay in the ICU, we distinguished 3 groups: sepsis-related complications included events clearly related to CAP that were not present within the 24 hours following ICU admission. These events could be a septic shock, an acute respiratory distress syndrome (ARDS), a multiple organ failure (MOF) or a diffusion of pulmonary infection (i.e., empyema). MOF, ARDS and HA-LRT were defined according to usual criteria [15-17]. HA-LRT superinfections included nosocomial bronchitis and pneumonia developing more than 48h after ICU admission. Nonspecific complications included ICU-related complications (i.e., upper gastrointestinal bleeding, catheter-related infection, deep venous thrombosis and pulmonary embolism) and complications attributed solely to underlying medical conditions.

Statistical Analysis

To evaluate the performance of this prognostic prediction rule in the validation population, we determined its ability to stratify patients into classes and subclasses with statistically significant different observed mortality rates. For this purpose, we compared the mortality rates observed across the three initial risk classes and, within each initial risk class, across the different adjusted risk subclasses.

To evaluate the reproducibility of the rule, we compared, across all risk classes and subclasses, the mortality rates observed in the validation population to those predicted (observed mortality rates in the development population upon elaboration of the rule).

All bivariate comparisons were performed by the χ2 test or the Fisher’s exact test when χ2 test was not appropriate. In all instances, p levels ≤ 0.05 were considered significant.

RESULTS

The validation population consisted of 511 patients. There were 364 men (71.2%) and 147 women (28.8%). The mean age was 61.0 ± 17 years. Main patients’ characteristics focusing on predictors of ICU mortality are reported in Table . During the ICU stay, 148 patients (29.0%) died.

In this population, 122, 272 and 117 patients were included in the initial risk classes I, II and III, respectively (Table ). Respective mortality rates were 8.2%, 22.8% and 65.0% and were significantly different (p<0.001). Within each initial risk class, the adjustment risk score identified subclasses exhibiting significantly different mortality rates: 3.9% vs 33.3% in class I (p<0.001), 3.1% vs 12.9% vs 63.3% in class II (p<0.001) and 55.8% vs 82.5% in class III (p=0.004) (Table ).

The comparison risk class by risk class and subclass by subclass of observed and predicted mortality rates is reported in Table . Only two significant differences were found. Within the initial risk class II, the mortality rates observed in two of the three subclasses were significantly lower in the validation population than predicted by rates observed in the development population.

DISCUSSION

Our results demonstrate the performance and reproductibility of our prognostic prediction rule in the validation population. On ICU admission, an initial risk score classified patients into three classes of increasing mortality. During the ICU stay, an adjustment risk score identified, in each initial risk class, subclasses associated with significantly different mortality rates. Moreover, the reproducibility of the rule appeared satisfying since across the 3 classes and the 7 subclasses, there were only two significant differences between mortality rates observed in the development population and in the validation population.

Numerous rules were built in order to assess the prognosis of CAP. In 1997, Fine and colleagues described a prediction rule based on 20 clinical variables [2]. Called Pneumonia Severity Index, this rule categorizes patients into 5 risk classes with an increasing risk of 30-day mortality. The major strength of this rule is to help physicians identify low risk patients at admission in the emergency room: patients in risk classes I-III can be considered potential candidates for outpatient treatment. Similarly, Lim and colleagues described a prognostic model based on 4 “core” clinical adverse prognostic factors [18]. This score, called CURB score (confusion, urea nitrogen, respiratory rate, blood pressure), identifies patients who do not require hospitalisation (no adverse prognostic factor). A modified version of this score adds age ≥ 65 years as a fifth variable [19]. This CURB-65 score stratifying patients into 6 classes of increasing risk of mortality is taken into account by the 2004 update of the British Thoracic Society pneumonia guidelines stating that patients with a CURB-65 score <2 may be suitable for outpatient treatment [20, 21]. The major interest of these different prediction rules is the identification of patients who do not require hospitalisation for the treatment of their pneumonia. Their implication for the medical care of patients exhibiting severe CAP requiring ICU admission appears questionable. A recent study performed by Espana and colleagues evaluated the performances of these prognostic rules for identifying patients who will develop severe CAP [22]. The authors demonstrated that these rules were less accurate than their prediction rule based on eight independent predictive factors. The factors and their assigned points were the followings: arterial pH < 7.30 (13 points), systolic blood pressure < 90 mm Hg (11 points), respiratory rate > 30 breaths/min (9 points), PaO2 < 54 mm Hg or PaO2/FiO2 < 250 mm Hg (6 points), altered mental status (5 points), blood urea nitrogen > 30 mg/dL (5 points), age ≥ 80 yr (5 points), and multilobar/bilateral lung involvement (5 points). The percentages of patients developing severe pneumonia were fewer than 3% if the score was less than 10, 10% with a score of 10 to 19, 40 % with a score of 20 to 29 and more than 50 % with a score equal or higher than 30.

The goal of our prognostic prediction rule is quite different from the goal of these rules. As demonstrated, the CURB scores and the Pneumonia Severity Index identify as early as the admission in emergency room both low-risk patients who are candidates for outpatient care and high-risk patients exhibiting the highest 30-day mortality [23]. These scores are useful in helping physicians determine rational decisions about hospitalization of these patients. For example, it is obvious that an urgent hospitalization is needed for high-risk patients and that ICU admission must be considered for them. Similarly, the rule developed by Espana and colleagues appears able to quantify the risk for development of severe CAP [22]. However, these rules do not specifically evaluate the prognosis of patients admitted into ICU for severe CAP. Aujesky and colleagues enrolled 3181 patients admitted to an emergency department [23]. The Pneumonia Severity Index identified 200 patients included in the risk class V. Their mortality rate was 24%. The CURB score identified 7 patients in class 4 with a mortality rate of 43%. Finally, the CURB-65 score identified the same number of patients in class 5, with a similar mortality rate. The number of patients assessed as exhibiting a severe CAP was quite low and varied largely according to the rule since 200 patients were identified by the Pneumonia Severity Index and only 7 by the CURB scores. Moreover, the estimated mortality rate appeared imprecise, varying from 24% to 42% according to the rule. The goal of our rule is not the identification of patients exhibiting severe CAP, but the precise evaluation of prognosis for patients admitted into an ICU for severe CAP. Our prospective population included 511 patients admitted into ICU for severe CAP. The overall mortality rate was 29.0%. According to an initial risk score representing the first step of our prediction rule, we identified 3 groups of patients exhibiting significantly different mortality rates varying from 8.2% to 65%. Secondly, in each of these 3 groups, an adjustment risk score corrected this estimation and determined the final prognosis.

Recent studies suggest that some drugs administered as an adjunct to usual antimicrobial and supportive therapeutics could provide significant reduction in mortality for patients suffering from severe CAP. Opal and colleagues demonstrated that drotrecogin alfa (activated) reduced the mortality rate associated with Streptococcus pneumoniae infection [24]. Likewise, analysis of subgroups from the PROWESS study showed that administration of this drug was associated with a decrease in 28-day mortality rates due to lung infection [25]. Despite these findings, numerous physicians are reluctant to use recombinant human activated protein C for all patients admitted in ICU for severe CAP. This might be explained by data emphasizing that drotrecogin alfa (activated) could be associated with both the absence of a beneficial treatment effect and an increased incidence of serious bleeding complications in patients with severe sepsis who are at low risk of death [26]. Our prognostic prediction rule is able to identify, as early as ICU admission, patients with either a low or, conversely, a high risk of death. Thus, it could be used as an aid to clinicians to accurately differentiate, at the beside, patients who could benefit (class III) from patients who might be harmed (class I) by drotrecogin alfa (activated) administration.

The apparent inability of our rule to immediately determine the outcome for all patients upon ICU admission could appear as a limitation. Indeed, for 50% of patients, the initial evaluation predicted an intermediate or an indeterminate outcome with a mortality rate similar to the overall mortality rate. The final outcome depended mainly on the occurrence or non-occurrence of complications during the ICU stay. These points, already discussed in our original publication, are mainly due to our developing a prediction rule taking into account data from previous studies establishing that the prognosis of severe CAP depended on both baseline characteristics, initial severity of infection and evolution during the ICU stay [7-9].

In summary, the results of this prospective multicenter study demonstrate the performance and reproductibility of this prognostic prediction rule. Six predictors of mortality accurately predict the outcome, as early as ICU admission, for 50% of patients. In the remaining patients, complications occurring during the ICU stay are essential to accurately reassess final outcome. As an aid to clinicians in stratifying the prognosis of patients exhibiting severe CAP, this rule could be used for future management strategies and therapeutic research.

Table 1

Patients Characteristics in the Validation Population*

Characteristics	Validation Population n=511 Patients
Age ≥ 40 yr	450 (88.1%)
Anticipated death within 5 yr	244 (47.7%)
Non aspiration pneumonia	408 (79.8%)
Chest radiograph involvement > 1 lobe	224 (43.8%)
ARF requiring immediate invasive MV	312 (61.1%)
Septic shock	132 (25.8%)
HA-LRT superinfections	78 (15.3%)
Non-specific complications	152 (29.8%)
Sepsis-related complications	119 (23.2%)

ARF = acute respiratory failure; MV = mechanical ventilation; HA-LRT = hospital-acquired lower respiratory tract superinfections.

Data are presented as No. (%).

Table 2

Stratification of Patients into Risk Classes and Subclasses in the Validation Population*

	Initial Risk Class
	I		II			III
Patients	122		272			117
Deaths	10 (8.2%)		62 (22.8%)			76 (65.0%)
	Points of Adjustment Risk Score
	≤ 2	> 2	0	1 or 2	> 2	≤2	> 2
Patients	104	18	131	62	79	77	40
Deaths	4 (3.9%)	6 (33.3%)	4 (3.1%)	8 (12.9%)	50 (63.3%)	43 (55.8%)	33 (82.5%)

Data are presented as No. (%).

Table 3

Comparison of Mortality Rates in the Development and the Validation Populations*

	Mortality		p Value
	Development Population	Validation Population
Overall initial risk class	108/472 (22.9%)	148/511 (29.0%)	0.03
I	6/153 (3.9%)	10/122 (8.2%)	0.13
II	63/253 (24.9%)	62/272 (22.8%)	0.57
III	39/66 (59.1%)	76/117 (65.0%)	0.43
Adjustment in each initial risk class
Class IAdjustment risk score < or =2 pointsAdjustment risk score > 2 points	2/145 (1.4%)4/8 (50%)	4/104 (3.9%)6/18 (33.3%)	0.240.66
Class IIAdjustment risk score = 0 pointAdjustment risk score = 1 or 2 pointsAdjustment risk score > 2 points	3/149 (2.0%)24/62 (38.7%)36/42 (85.7%)	4/131 (3.1%)8/62 (12.9%)50/79 (63.3%)	0.710.0010.01
Class IIIAdjustment risk score < or = 2 pointsAdjustment risk score > 2 points	25/51 (49%)14/15 (93.3%)	43/77 (55.8%)33/40 (82.5%)	0.450.42

Data are presented as No. (%).