Modified IDSA/ATS Minor Criteria for Severe Community-Acquired Pneumonia Best Predicted Mortality.

It is not clear whether the IDSA/ATS minor criteria for severe community-acquired pneumonia (CAP) could be simplified or even be modified to orchestrate improvements in predicting mortality.A retrospective cohort study of 1230 CAP patients was performed to simplify and to modify the scoring system by excluding 4 noncontributory or infrequent variables (leukopenia, hypothermia, hypotension, and thrombocytopenia) and by excluding these variables and then adding age ≥65 years, respectively. The simplification and modification were tested against a prospective 2-center validation cohort of 1409 adults with CAP.The increasing numbers of IDSA/ATS, simplified, and modified minor criteria present in the retrospective cohort were positively associated with the mortality, showing significant increased odds ratios for mortality of 2.711, 4.095, and 3.755, respectively. The validation cohort confirmed a similar pattern. The sensitivity, specificity, positive predictive value, and Youden index of modified minor criteria for mortality prediction were the best pattern in the retrospective cohort. High values of corresponding indices were confirmed in the validation cohort. The highest accuracy of the modified version for predicting mortality in the retrospective cohort was illustrated by the highest area under the receiver operating characteristic curve of 0.925 (descending order: modified, simplified, and IDSA/ATS minor criteria). The validation cohort confirmed a similar paradigm.The IDSA/ATS minor criteria could be simplified to 5 variables and then be modified to orchestrate improvements in predicting mortality in CAP patients. The modified version best predicted mortality. These were more suitable for clinic and emergency department.

INTRODUCTION

Despite substantial advances in therapeutic options, the mortality due to community-acquired pneumonia (CAP) remains unacceptably high.[1,2] The assessment of severity is crucial in the management of CAP. The Infectious Disease Society of America (IDSA) and the American Thoracic Society (ATS) 9 minor criteria for sever CAP included variables in the CURB-65 (confusion, urea >7 mmol/L, respiratory rate ≥30 min−1, low blood pressure, and age ≥65 years) score except age, arterial oxygen pressure/fraction inspired oxygen (PaO2/FiO2) ≤250 mmHg, multilobar infiltrates, leucopenia, thrombocytopenia, and hypothermia.[1]

Liapikou et al[3] could not demonstrate an association between hypotension, thrombocytopenia and multilobar involvement, and mortality. Our study suggests that leukopenia, hypothermia, and hypotension were not associated with mortality.[4] However, Phua et al[5] and Chalmers et al[6] revealed that each minor criterion was predictive of mortality. How to deal with the discrepancies? Salih et al[7] recently reported that the criteria could be simplified by removing 3 infrequent variables (leukopenia, thrombocytopenia, and hypothermia), but could not improve the prediction of mortality and intensive care unit (ICU) admission. A recent clinical study showed that the IDSA/ATS minor criteria can improve patient management.[8] Improvement in patient management first depends on the accuracy of ICU admission prediction. Therefore, this is the correct application of the minor criteria. Our previous study showed that CURB-65 score predicted mortality in CAP better than the IDSA/ATS minor criteria in a low-mortality-rate setting.[9] Could the minor criteria scoring system further be simplified to orchestrate an improvement in predicting mortality? Could it even be modified to orchestrate a further improvement? Hence, further studies are warranted.

Two cohort studies were conducted to first derive and then to validate a simplified IDSA/ATS minor criteria and a modified version.

MATERIALS AND METHODS

Design and Setting

We performed a retrospective cohort study of 1245 adult patients admitted for the treatment of CAP to the Department of Respiratory Medicine in a Chinese affiliated tertiary hospital of a medical college from 2005 to 2009. A prospective 2 center cohort study of 1430 adults with CAP between 2010 and 2013 was conducted at the former department and the Department of Respiratory Medicine in an affiliated tertiary hospital of another medical college in China. The latter hospital had more beds than the former.

Simplified and modified minor criteria were derived in the retrospective sample by excluding 4 noncontributory or infrequent variables (leukopenia, hypothermia, hypotension, and thrombocytopenia),[3,4,7] and by excluding these variables and then adding age ≥65 years,[10] respectively (Table 1). The simplification and modification were tested against the prospective 2 center validation cohort.

TABLE 1

The Components of the 3 Minor Criteria Scoring Systems

Criteria for Enrollment

CAP was defined as an acute infection of the pulmonary parenchyma associated with an acute infiltrate on the chest radiograph with 2 or more symptoms including fever (>38 °C), rigors, sweats, new cough or change in color of respiratory secretions, chest discomfort, or dyspnoea.[5] Patients younger than 18 years, admitted to hospital during the 28 days before the study, having severe immunosuppression, active tuberculosis, or end-stage diseases, or with a written “do not resuscitate” order were excluded.

Clinical Management

Patients with CAP were admitted and then attended by respiratory physicians using the ATS guidelines[1,11] and the Surviving Sepsis Campaign guidelines.[12,13] The initial antibiotic regimens were consistent with the guidelines on the management of CAP, in addition to subsequently cultured pathogens. Therefore, all patients were regarded as receiving adequate antibiotics and were discharged home when they reached clinical stability and became afebrile.

Approval of Study Design

The studies were approved by our Institutional Review Boards (Review Board of Guangdong Medical College and Review Board of Peking University). Ethical approval from the regulation committee (Ethical Committee of Shenzhen) was granted for the study protocol. Written informed consent (except that from the patients with confusion) was obtained prior to enrollment.

Outcome

The main outcome measure was hospital mortality.

Data Collection

A total of 1245 patients were enrolled consecutively, and 15 cases were excluded from the retrospective cohort due to exclusion criteria. A total of 21 cases were excluded from 1430 consecutive patients in the validation cohort. Clinical and diagnostic data and radiological features were collected. Laboratory variables were measured by the hospital clinical laboratories. All the patients had chest radiographys and computed tomographic scans. The frontal and lateral chest radiographic findings and computed tomographic scan images were classified independently by 2 senior radiologists (Liang and Zhao). The statistician was blinded to the study.

Statistical Analysis

All statistical analyses were performed with Statistical Package for the Social Science for Windows version 16.0 (SPSS, Chicago, IL) and MedCalc version 14.8.1.0 (Mariakerke, Belgium). Categorical variables and continuous variables were reported as the percentages and the mean ± standard deviation (SD), respectively. Chi-square test and univariate logistic regression were employed. The receiver operating characteristic curves were created and the areas under the curves (area under the receiver operating characteristic curves [AUCs]) were calculated to illustrate and compare the accuracy of the indices. The sensitivities, specificities, positive predictive values (PPVs), negative predictive values, and Youden indices were also calculated. A P value of <0.05 was considered statistically significant.

RESULTS

Baseline Characteristics of Study Cohorts

The baseline characteristics of the patients were shown in Table 2. With relatively important differences in the presentation, the prospective validation cohort included more severely ill patients.

TABLE 2

Baseline Characteristics of Study Cohorts (Mean ± SD)

Associations With Hospital Mortality

The hospital mortalities were 1.3% and 3.8% in the retrospective and prospective cohorts, respectively. The mortality rates in the retrospective cohort increased directly with the increasing numbers of IDSA/ATS, simplified, and modified minor criteria present (x2, P 108.434, <0.001; 124.020, <0.001; 145.953, <0.001, respectively, Table 3). The validation cohort confirmed a similar pattern (x2, P 137.877, <0.001; 343.799, <0.001; 427.802, <0.001, respectively, Table 3). The numbers of IDSA/ATS, simplified, and modified minor criteria present in the retrospective cohort had significant increased odds ratios (OR) for mortality of 2.711, 4.095, and 3.755 (95% confidence interval [CI], P 1.912–3.844, <0.001; 2.716–6.175, <0.001; 2.574–5.478, <0.001; respectively). The 3 scoring systems performed well when applied to the validation cohort (OR, 95% CI, P 2.686, 2.134–3.382, <0.001; 6.664, 4.510–9.845, <0.001; 6.661, 4.452–9.965, <0.001, respectively). Associations between selected clinical features and hospital mortality on univariate analysis were shown in Table 4. The remaining 5 variables and age ≥65 years were all significantly associated with death.

TABLE 3

Relationship Between Number of Adverse Features and Risk of Hospital Mortality

TABLE 4

Association Between Selected Clinical Features and Hospital Mortality on Univariate Analysis

Comparisons of the Scoring Systems

The sensitivities, specificities, and predictive values of the different rules for predicting hospital mortality were given in Table 5. The sensitivity, specificity, PPV, and Youden index of modified minor criteria for predicting mortality were the best pattern in the retrospective cohort. High values of corresponding indices were confirmed in the validation cohort.

TABLE 5

Test Characteristics of Rules With Different Prediction Scores for Hospital Mortality in the Retrospective and Prospective Sets of Patients Hospitalized With CAP

The receiver operating characteristic curves for the 3 scoring systems and CURB-65 score in the 2 study populations illustrated the differences in accuracy of mortality prediction (Tables 6 and 7, and Figures 1 and 2). The highest accuracy of modified minor criteria in the retrospective cohort was illustrated by the highest AUC of 0.925. The validation cohort confirmed a similar paradigm. AUCs ranked in descending order were those of modified, simplified, and IDSA/ATS minor criteria. Modified minor criteria was performed similarly for the prediction of mortality in the retrospective cohort, but better in the validation cohort, compared with CURB-65 score.

TABLE 6

AUC Values for Different Scoring Systems

TABLE 7

Comparison of AUC Values Between the Scoring Systems

FIGURE 1

ROC curves for mortality prediction by the IDSA/ATS minor criteria, simplified minor criteria, modified minor criteria, and CURB-65 score in the retrospective cohort. CURB-65 = confusion, urea >7 mmol/L, respiratory rate ≥30 min−1, low blood pressure, and age ≥65 years, IDSA/ATS = Infectious Disease Society of America and the American Thoracic Society, ROC = receiver operating characteristic.

FIGURE 2

ROC curves for mortality prediction by the IDSA/ATS minor criteria, simplified minor criteria, modified minor criteria, and CURB-65 score in the validation cohort. CURB-65 = confusion, urea >7 mmol/L, respiratory rate ≥30 min−1, low blood pressure, and age ≥65 years, IDSA/ATS = Infectious Disease Society of America and the American Thoracic Society, ROC = receiver operating characteristic.

DISCUSSION

Our findings show that the increasing numbers of the 3 scoring systems present were positively associated with the mortalities in the 2 cohorts, showing significant increased ORs for mortality, that the sensitivity, specificity, PPV, and Youden index of modified minor criteria for predicting mortality were the best pattern in the 2 cohorts, and that the highest accuracies of the modified version for mortality prediction in the 2 cohorts were illustrated by the highest AUC values (AUCs in descending order were those of modified, simplified, and IDSA/ATS minor criteria).

Although the IDSA/ATS minor criteria scoring system was designed to guide ICU admission, not to predict mortality, Sibila et al[14] and we[4] reported that it was associated with mortality and then might serve as predictors of mortality. Our previous study revealed that the individual 2007 IDSA/ATS minor criteria for severe CAP were of unequal weight in predicting hospital mortality and some of the criteria had no predictive value.[4] Adding in sequence the 4 minor criteria (tachycardia > 125 bpm, arterial pH 7.30–7.34, sodium < 130 mEq/L, and glucose > 250 mg/dL) to the 2007 IDSA/ATS minor criteria to predict ICU admission, increased sensitivity from 41.7% to 53.8%, and AUC from 0.65 to 0.69.[15] These findings suggest that the minor criteria might be simplified to increase sensitivity and might also be modified to orchestrate an improvement in prediction. We found that the 4 noncontributory or infrequent variables (leukopenia, hypothermia, hypotension, and thrombocytopenia) could be removed and that the deletion improved PPV and AUC for the prediction of mortality in the 2 cohorts. Salih et al[7] recently discovered that a simplified score excluding 3 variables (leukopenia, hypothermia, and thrombocytopenia) was performed similarly for the prediction of mortality and ICU admission. The additional omission of hypotension criterion could effectively help to orchestrate an improvement in predicting mortality. Inclusion of nonpredictive variables might incur higher false positive rate (i.e., lower specificity), higher false negative rate (i.e., lower sensitivity), and lower PPV, negative predictive values, and AUC. Which mechanism might be envisaged to interpret the effectiveness of the additional omission in the current study? The cardiac output and the peripheral resistance produce blood pressure. Incorporation of the blood pressure criterion into a severity scoring system may lead to false negativity in the older people who have high prevalence of systolic hypertension owing to increasing age. The performance of scores did vary significantly between different studies in different healthcare systems. If the population of patients to which the score is being applied is significantly different from the original derivation it may be necessary to perform local recalibration of the score.[16] Therefore, further multicentre studies are warranted to assess the generalizability of the current findings.

We previously discovered that age ≥65 years showed independent relationship with mortality.[10] Chen et al[17] reported that the 30-day mortality rate was associated with age. Kelly et al[18] found that patients greater than 65 years of age had a higher incidence of altered mental status on presentation and that CURB scores and pneumonia severity index were higher in the older patients. Incorporation of age ≥65 years into our presently simplified minor criteria orchestrated a further improvement in sensitivity, Youden index, and AUC and then best predicted mortality. Future prospective clinical multicenter studies should also be performed to assess the generalizability.

CURB-65 score predicted mortality in CAP better than IDSA/ATS minor criteria in a low-mortality-rate setting.[9] Inclusion of nonpredictive variables might be envisaged to interpret the finding. PaO2/FiO2 ≤250 mmHg (a strong predictive value)[4] and multilobar infiltrates substituted for low blood pressure (nonpredictive variable),[4] which was the difference between modified minor criteria and CURB-65 score. This might be envisaged to interpret the reason why modified minor criteria was performed better than CURB-65 score for the prediction of mortality in the validation cohort. But why not in the retrospective cohort? CURB-65 scoring system performs well at identifying patients with pneumonia who have a low risk of death.[19] The mortality in the retrospective cohort was lower than that in the validation cohort. This might be the causation.

A big difficulty in the management of CAP is to screen patients who might rapidly develop adverse medical outcomes among those not meeting the IDSA/ATS major criteria. The current findings might have implications for the management of the disease, especially at the emergency department, which may improve survival. The patients with CAP might be triaged more appropriately according to simplified minor criteria, and most accurately by using the modified version. They might help us make more accurate clinical decisions about where these patients should be treated at (ICU vs non-ICU), and need for advance care. Sound site decision can optimize initial antibiotic treatment and is beneficial to reducing mortality.[20-23] Lack of ICU resources is a universal problem, especially in limited resources settings, but admission of CAP patients unlikely to benefit more from ICU care is not scarce. What we suggested might constitute 2 more pragmatic rules, which were more suitable for clinic and emergency department, to predict mortality from CAP.

LIMITATIONS

This study has 3 main limitations. First, this was a prospective 2 center, not a multicentre, validation cohort study. Second, there were relatively small samples. The findings might have been more robust with a greater number of patients met the criteria. Finally, the clinical outcomes data on ICU utilization were not collected. Accuracy of ICU admission prediction is pivotal to improve patient management.

CONCLUSIONS

The 2007 IDSA/ATS minor criteria could be simplified to 5 variables and then be modified to orchestrate improvements in predicting mortality in CAP patients. The modified version best predicted mortality. These were more suitable for clinic and emergency department.