Validation of the prediction rules identifying drug-resistant pathogens in community-onset pneumonia.

BACKGROUND: Appropriate initial antibiotic treatment and avoiding administration of unnecessary broad-spectrum antibiotics are important for the treatment of pneumonia. To achieve this, assessment of risk for drug-resistant pathogens (DRPs) at diagnosis is essential.
PURPOSE: The aim of this study was to validate a predictive rule for DRPs that we previously proposed (the community-acquired pneumonia drug-resistant pathogen [CAP-DRP] rule), comparing several other predictive methods. PATIENTS AND METHODS: A prospective observational study was conducted in hospitalized patients with community-onset pneumonia at four institutions in Japan. Pathogens identified as not susceptible to ceftriaxone, ampicillin-sulbactam, macrolides, and respiratory fluoroquinolones were defined as CAP-DRPs.
RESULTS: CAP-DRPs were identified in 73 (10.1%) of 721 patients analyzed. The CAP-DRP rule differentiated low vs high risk of CAP-DRP at the threshold of ≥3 points or 2 points plus any of methicillin-resistant Staphylococcus aureus specific factors with a sensitivity of 0.45, specificity of 0.87, positive predictive value of 0.47, negative predictive value of 0.87, and accuracy of 0.79. Its discrimination performance, area under the receiver operating characteristic curve, was 0.73 (95% confidence interval 0.66-0.79). Specificity of the CAP-DRP rule against CAP-DRPs was the highest among the six predictive rules tested.
CONCLUSION: The performance of the predictive rules and criteria for CAP-DRPs was limited. However, the CAP-DRP rule yielded high specificity and could specify patients who should be treated with non-broad-spectrum antibiotics, eg, a non-pseudomonal β-lactam plus a macrolide, more precisely.

Introduction

Pneumonia is one of the lethal infectious diseases.1 To achieve appropriate initial antibiotic treatment is essential for patients with pneumonia since inappropriate antibiotic treatment results in adverse outcomes.2,3 Identifying patients at risk for drug-resistant pathogens (DRPs) is therefore critical.

To classify patients at risk for DRPs, the 2005 American Thoracic Society (ATS) and Infectious Diseases Society of America (IDSA) guidelines for adult pneumonia proposed the concept of health care-associated pneumonia (HCAP).4 The HCAP criteria include nursing home residents, prior hospitalization, receiving home infusion therapy or home wound care, and chronic dialysis.4 However, previous studies have shown low prevalence of DRPs in patients with HCAP and low predictability of HCAP criteria for identifying patients with DRPs.5–9 Thus, there has been an increase of unnecessary broad-spectrum antibiotic use.10 In fact, our previous study revealed that overuse of broad-spectrum antibiotics significantly increased the risk of death in patients with non-DRPs.3 Therefore, it is crucial to identify patients with pneumonia who do not need broad-spectrum antibiotics.

Our previous study also suggested that the treatment strategy for both patients with community-acquired pneumonia (CAP) and HCAP could be unified because the risk factors for DRPs were identical between the two categories of pneumonia.11 Currently, the antibiotic treatment strategy should be considered combining CAP and HCAP, that is, community-onset pneumonia, based on their common risk factors for DRPs,5,11–13 and some investigators have proposed several prediction rules identifying patients with DRPs at the diagnosis of community-onset pneumonia.5,11,14–16 In our previous study, we elucidated six risk factors for CAP-DRPs that were not susceptible to antibiotics commonly used for patients with CAP. The risk factors for CAP-DRPs included prior hospitalization, immunosuppression, previous use of antibiotics, use of gastric acid suppressive agents, tube feeding, and nonambulatory status; and specific risk factors for methicillin-resistant Staphylococcus aureus (MRSA) included chronic dialysis, congestive heart failure, and positive MRSA history.11 We proposed a prediction rule based on the cumulative number of risk factors (the CAP-DRP rule).11 Practically, patients with no or one risk factor would be at a low risk for CAP-DRPs and those with three or more risk factors would be at a high risk. When patients have two risk factors for CAP-DRPs and no MRSA-specific risk factor, they could be also classified into the low risk group for CAP-DRPs.

The aim of this study was to validate the prediction rule, ie, CAP-DRP rule, and to compare its predictive performance with that of others. In addition, we considered it important to identify patients at low risk for CAP-DRPs in order to avoid unnecessary use of broad-spectrum antibiotics associated with worse outcomes.

Materials and methods

Study design and setting

This was a prospective observational study conducted in four institutions (a 1,000-bed university hospital and three major community hospitals with >500 beds) located in central Japan. Patient data were collected from April 1, 2013 to March 31, 2014. The protocol of this study adhered to the Declaration of Helsinki and the Japanese Ethics Guidelines for Epidemiological Studies. Obtaining informed consent of the participants was waived, but the opt-out method was adopted according to the ethics guidelines. Information about the study was disclosed to the target patients through the Internet, brochures, or bulletin boards at the participating institutions to give the candidates the opportunity to decline participation. This study was approved by the ethical committee of Nagoya University (No. 2012-0338) and the respective institutional review boards of the participating institutions; it was registered at University Hospital Medical Information Network in Japan (No. UMIN000009837; http://www.umin.ac.jp/).

Participants and categories of pneumonia

The study method was almost identical to that of our previous study.11 Briefly, all adult patients (age ≥20 years) who newly developed CAP or HCAP in their daily community living and needed inpatient treatment were enrolled in the study and followed up 1 month later.

Data collection and microbiologic evaluation

During the patient registration period, the study coordinator (DK) monitored patient enrollment in all institutions to decrease missing and contradiction of data. Microbiologic evaluation was performed by a method similar to a previous study.11 Briefly, microbiologic laboratories in all four institutions provided possible causative pathogens, which were cultured in a semiquantitative manner from respiratory tract samples (including sputum, tracheobronchial aspirates, and bronchoalveolar lavage fluid), pleural fluid, and blood. Serologic tests were performed to detect antibodies against Mycoplasma pneumoniae and Chlamydophila pneumoniae. Legionella pneumophila serogroup 1 antigen in urine was tested by immunochromatography. Microbiologic test results were independently reviewed by two investigators (DK and IY). Pathogens provided by the study institutions were re-cultured. Viruses, acid-fast bacilli, fungus, and anaerobes were not re-cultured. Antibiotic susceptibility tests were performed at a central laboratory (SRL, Inc., Tokyo, Japan).11 Microdilution was performed according to the guidelines of the Clinical and Laboratory Standards Institute.17 When no breakpoints were specified in these guidelines, clinical breakpoints for bacteria provided by the European Society of Clinical Microbiology and Infectious Diseases (version 1.2) were utilized.18

Definition of CAP-DRPs

Combination therapy with non-antipseudomonal β-lactam antibiotics plus a macrolide or monotherapy with fluoroquinolone has been recommended as the initial empirical antibiotic treatment in the international guidelines of CAP.19 Therefore, identified pathogens that were not susceptible to all of the following types of antibiotics: non-antipseudomonal β-lactam antibiotics (ceftriaxone or ampicillin–sulbactam), macrolides (azithromycin or clarithromycin), and fluoroquinolones (moxifloxacin, levofloxacin, or garenoxacin) were defined as CAP-DRPs.

Outcomes

In this observational study, we defined the drug resistance of identified pathogens (occurrence of CAP-DRPs) as the main microbiologic outcome. The 30-day and in-hospital mortality were also assessed.

Prediction rules for CAP-DRPs

Definitions of various prediction rules to identify patients with CAP-DRPs including the CAP-DRP rule are shown in Figure 1.5,11,15,20–22

Figure 1

Definitions of predictive rules for identifying patients with DRPs.

Notes: The definitions of DRPs in each previous report are as follows. Shorr et al: MRSA, Pseudomonas aeruginosa, Acinetobacter baumannii, and ESBL producing organisms; HCAP criteria by ATS and IDSA: P. aeruginosa, ESBL producing Enterobacteriaceae, Acinetobacter sp., and MRSA; Aliberti et al: MRSA, P. aeruginosa resistant to antipseudomonal penicillins, cephalosporins, carbapenems, and quinolones, Stenotrophomonas maltophilia, vancomycin-resistant Enterococcus sp., A. baumannii, ESBL producing Enterobacteriaceae, and other non-fermenting gram-negative bacilli; Brito and Niederman: not defined clearly; Schreiber et al: MRSA, P. aeruginosa, and ESBL producing organisms; Prina et al: MRSA, P. aeruginosa, and ESBL producing Enterobacteriaceae.

Abbreviations: DRPs, drug-resistant pathogens; CAP-DRPs, community-acquired pneumonia drug-resistant pathogens; MRSA, methicillin-resistant Staphylococcus aureus; HCAP, health care-associated pneumonia; NH, nursing home; LTC, long-term care facility; IV, intravenous; CVD, cerebrovascular disease; COPD, chronic obstructive pulmonary disease; ESBL, extended-spectrum β-lactamase; ICU, intensive care unit; ATS, American Thoracic Society; IDSA, Infectious Diseases Society of America.

Prediction rules for MRSA

In our previous study, we elucidated risk factors for MRSA that included three MRSA-specific factors (chronic dialysis during the preceding 30 days, congestive heart failure, and positive MRSA history within the previous 90 days) and three common risk factors for all CAP-DRPs including MRSA (prior hospitalization, prior antibiotic use, and use of gastric acid suppressive agents).11 The MRSA-specific risk score was determined on the basis of the cumulative number of risk factors for MRSA. Shorr’s MRSA score (range, 0–10) was also calculated according to the original article.14

Statistical analysis

Demographic, clinical, and microbiologic characteristics were described. Categorical data were summarized as frequencies in percentage and continuous data as median with interquartile range.

The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and area under the receiver operating characteristic curve (AUROC) of the prediction rules for CAP-DRP prevalence were evaluated.5,11,14,15,20–22 Moreover, we calculated the Youden indexes to compare different thresholds of Shindo’s CAP-DRP score.23 In this procedure, point estimates and their 95% confidence intervals (CIs) of the respective parameters were calculated. In addition, the predictive performance of the rules was assessed to identify patients with MRSA.11,14 Calculation of sensitivity, specificity, PPV, NPV, and accuracy was performed using the R statistical package (version 3.0.2; R Foundation for Statistical Computing, Vienna, Austria). PASW Statistics 20 (IBM Corporation, Armonk, NY, USA) was used for other statistical analyses. All tests were two-tailed and a P-value <0.05 was considered statistically significant.

Results

Demographic and clinical data

A total of 750 patients with community-onset pneumonia were assessed and 721 patients (480 CAP and 241 HCAP) were enrolled in this study as eligible patients (Figure 2). Among them, 355 patients in whom pathogens were identified and antibiotic susceptibility data were clarified were assessed to validate the prediction rules for CAP-DRPs. The baseline characteristics of the 721 eligible study patients are described in Table 1. Median age was 77 years and 66.9% were male. One hundred and twenty-one (16.8%) patients had a history of prior hospitalization; 265 (36.8%), prior antibiotic use; 51 (7.1%), immunosuppression; 146 (20.2%), nonambulatory status; 25 (3.5%), tube feeding; and 205 (28.4%), use of gastric acid suppressive agents.

Figure 2

Patient flow.

Abbreviations: CAP, community-acquired pneumonia; HCAP, health care-associated pneumonia; CAP-DRPs, community-acquired pneumonia drug-resistant pathogens.

Table 1

Patient characteristics

	Total (n=721)
Age, median (IQR) years	77	(69–84)
Men, n (%)	482	(66.9)
Health care-associated pneumonia, n (%)	241	(33.4)
Hospitalization for ≥2 days during the preceding 90 days	121	(16.8)
Resident in a nursing home or extended care facility	120	(16.6)
Home intravenous therapy (including antibiotics and chemotherapy)	37	(5.1)
Chronic dialysis during the preceding 30 days	12	(1.7)
Home wound care during the preceding 30 days	1	(0.1)
Comorbidities, n (%)
Chronic lung diseases	255	(35.4)
Congestive heart failure	123	(17.1)
Chronic renal diseases	57	(7.9)
CNS disorders	117	(16.2)
Diabetes	126	(17.5)
Immunosuppressiona	51	(7.1)
Use of antibiotics within the previous 90 days, n (%)	265	(36.8)
Nonambulatory status, n (%)	146	(20.2)
Dementia, n (%)	112	(11.5)
Tube feeding, n (%)	25	(3.5)
Use of gastric suppressive agents,b n (%)	205	(28.4)
Positive MRSA history within the previous 90 days, n (%)	17	(2.4)
Altered mental status,c n (%)	186	(25.8)
Fever (BT >37.8°C), n (%)	307	(42.6)
ICU admission, n (%)	52	(7.2)
PSI class,d n (%)
I–III	277	(39.2)
IV	270	(37.4)
V	160	(22.2)

Notes:

Immunosuppression included any immunosuppressive diseases, such as congenital or acquired immunodeficiency, hematological diseases, and neutropenia (<1,000/mm3), treatment with immunosuppressive drugs within the previous 30 days, or corticosteroids in daily doses of at least 10 mg/day of a prednisone equivalent for >2 weeks.

Gastric suppressive agents included histamine H2-receptor blockers and proton pump inhibitors.

Altered mental status was defined as a Glasgow coma scale <15.

PSI was evaluated in 707 patients.

Abbreviations: IQR, interquartile range; CNS, central nervous system; MRSA, methicillin-resistant Staphylococcus aureus; BT, body temperature; ICU, intensive care unit; PSI, Pneumonia Severity Index.

Microbiology and clinical outcomes

Pathogen distribution and clinical outcomes are shown in Table 2. Pathogens were identified in 369 (51.2%) patients. Non-CAP-DRPs (eg, Streptococcus pneumoniae, methicillin-sensitive S. aureus, Haemophilus influenzae, and antibiotic-sensitive enteric gram-negative bacilli) were isolated in 296 (41.1%) of 721 patients with community-onset pneumonia, and CAP-DRPs in 73 (10.1%). CAP-DRPs included MRSA (51 patients; 7.1%), Escherichia coli (12; 1.7%), and Pseudomonas aeruginosa (9; 1.2%). Among all the study patients, 30-day and in-hospital mortality were 8.9% and 10.8%, respectively.

Table 2

Microbiology and clinical outcomes

	Total (n=721)
Pathogens identified, n (%)	369	(51.2)
CAP-DRPsa	73	(10.1)
MRSA	51	(7.1)
Non-MRSA	27	(3.7)
Escherichia coli	12	(1.7)
ESBL+	8	(1.1)
Pseudomonas aeruginosa	9	(1.2)
Stenotrophomonas maltophilia	2	(0.3)
Others	4	(0.6)
Non-CAP-DRPsb	296	(41.1)
Streptococcus pneumoniaec	83	(11.5)
Methicillin-sensitive Staphylococcus aureus	56	(7.7)
Klebsiella pneumoniae	52	(7.2)
Haemophilus influenzae	43	(6.0)
Mortality, n (%)
30-day	64	(8.9)
In-hospital	78	(10.8)

Notes:

Identified pathogens which were not susceptible to β-lactam antibiotics (ceftriaxone or ampicillin/sulbactam), macrolides (azithromycin or clarithromycin), and fluoroquinolones (moxifloxacin, levofloxacin, or garenoxacin) were defined as CAP-DRPs.

25 isolates of P. aeruginosa and 7 of Acinetobacter baumannii were classified as non-CAP-DRPs.

Of 81 isolates with antibiotic susceptibility data, 80 were penicillin-susceptible S. pneumoniae and one was penicillin intermediate S. pneumoniae; and 75 and 74 were resistant to clarithromycin and azithromycin, respectively. Data are presented as n (%).

Abbreviations: CAP-DRP, community-acquired pneumonia drug-resistant pathogens; ESBL, extended-spectrum β-lactamase; MRSA, methicillin-resistant S. aureus.

Prediction for patients with CAP-DRPs

First, we examined the validity of the cutoff point of Shindo’s CAP-DRP rule. The CAP-DRP rule, ≥3 CAP-DRP risk factors or 2 CAP-DRP risk factors plus any of MRSA-specific risk factors, differentiated patients at high risk from low risk of CAP-DRPs with a sensitivity of 0.45, specificity of 0.87, Youden index of 0.32, PPV of 0.47, NPV of 0.87, and accuracy of 0.79. The simple cutoff to ≥2 and ≥3 CAP-DRP risk factors yielded sensitivity values of 0.59 and 0.33, specificity of 0.76 and 0.91, Youden indexes of 0.35 and 0.23, PPV of 0.37 and 0.46, NPV of 0.88 and 0.85, and accuracy of 0.72 and 0.79, respectively. Considering the Youden index and specificity, ≥3 CAP-DRP risk factors or 2 CAP-DRP risk factors plus any of MRSA-specific risk factors was most preferable.

Second, predictive performance was compared among the six prediction rules and criteria for CAP-DRPs including HCAP,4 Shorr,15 Aliberti,5 Brito and Niederman,20 Schreiber,22 Prina,21 and Shindo’s CAP-DRP rule (Table 3).11 When comparing original cutoff scores or criteria that are shown in Figure 1, sensitivity of the CAP-DRP rule (0.45) was lower than those of the other prediction rules, among which that of Aliberti’s rule was the highest (0.79). However, specificity of the CAP-DRP rule (0.87) was the highest followed by Brito and Niederman’s criteria (0.86). This trend of specificity was almost the same while comparing the prediction rules with changing of cutoff scores so as to have similar levels of sensitivity of the CAP-DRP rule (Table 3). In addition, PPV of the CAP-DRP rule was also the highest. NPVs of all tested rules and criteria were around 0.90.

Table 3

Comparison of predictive rules or criteria for CAP-DRPs

Prediction model	Cutoff score or criteria	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Accuracy (95% CI)
Shindo’s CAP-DRP	≥3 or2+1 MRSA-specific risk factor	0.45(0.34–0.57)	0.87(0.83–0.91)	0.47(0.35–0.59)	0.87(0.82–0.90)	0.79(0.74–0.83)
Health care-associated pneumonia	Yes	0.74(0.62–0.83)	0.73(0.67–0.78)	0.40(0.31–0.48)	0.92(0.88–0.95)	0.73(0.68–0.78)
Shorr
	≥1 (original)	0.72(0.60–0.82)	0.69(0.63–0.74)	0.36(0.28–0.45)	0.91(0.87–0.95)	0.70(0.65–0.74)
	≥3	0.71(0.59–0.81)	0.76(0.70–0.80)	0.42(0.33–0.51)	0.91(0.87–0.95)	0.75(0.70–0.79)
	≥4	0.38(0.27–0.51)	0.87(0.82–0.91)	0.42(0.30–0.55)	0.85(0.81–0.89)	0.77(0.73–0.81)
Aliberti
	≥3 (original)	0.79(0.68–0.88)	0.61(0.56–0.67)	0.33(0.26–0.41)	0.92(0.88–0.96)	0.65(0.60–0.70)
	≥4	0.64(0.51–0.76)	0.69(0.64–0.75)	0.31(0.23–0.40)	0.90(0.85–0.94)	0.68(0.63–0.73)
	≥5	0.44(0.32–0.58)	0.75(0.70–0.80)	0.28(0.19–0.38)	0.86(0.81–0.90)	0.70(0.65–0.75)
Brito and Niederman	Yes	0.45(0.33–0.57)	0.86(0.81–0.90)	0.43(0.32–0.55)	0.87(0.82–0.90)	0.78(0.74–0.82)
Schreiber
	≥2 (original)	0.51(0.39–0.63)	0.77(0.72–0.82)	0.36(0.26–0.46)	0.87(0.82–0.91)	0.72(0.68–0.77)
	≥3	0.27(0.18–0.39)	0.89(0.84–0.92)	0.37(0.46–0.51)	0.83(0.79–0.87)	0.76(0.72–0.81)
Prina
	≥2 (original)	0.96(0.89–0.99)	0.09(0.06–0.13)	0.21(0.17–0.25)	0.90(0.74–0.98)	0.26(0.22–0.31)
	≥5	0.67(0.55–0.78)	0.63(0.57–0.69)	0.31(0.24–0.39)	0.89(0.84–0.93)	0.64(0.59–0.69)
	≥6	0.51(0.39–0.63)	0.83(0.79–0.81)	0.43(0.32–0.54)	0.87(0.83–0.91)	0.77(0.72–0.81)
	≥7	0.33(0.22–0.45)	0.90(0.86–0.93)	0.45(0.32–0.60)	0.85(0.80–0.88)	0.79(0.74–0.83)

Abbreviations: PPV, positive predictive value; NPV, negative predictive value; CI, confidence interval; CAP-DRP, community-acquired pneumonia drug-resistant pathogen; MRSA, methicillin-resistant Staphylococcus aureus.

The CAP-DRP rule was predictive of CAP-DRPs with an AUROC of 0.73 (95% CI: 0.66–0.79) (Figure 3), which was almost identical to those of Shorr’s rule (AUROC: 0.73 [95% CI: 0.66–0.88]) and Aliberti’s rule (AUROC: 0.71 [95% CI: 0.65–0.77]), and somewhat larger than that of Schreiber’s rule (AUROC: 0.67 [95% CI: 0.60–0.77]).

Figure 3

Receiver operating characteristic curves for predicting CAP drug resistance.

Note: The definitions of each predictive rule are described in Figure 1.

Abbreviations: CI, confidence interval; CAP, community-acquired pneumonia; CAP-DRPs, community-acquired pneumonia drug-resistant pathogens.

Prediction for non-MRSA CAP-DRPs and MRSA

In our previous study, we proposed the threshold number of risk factors for CAP-DRPs to be three to identify patients with non-MRSA CAP-DRPs. In this validation study, the CAP-DRP rule registered a sensitivity of 0.29, specificity of 0.87, PPV of 0.15, NPV of 0.94, and accuracy of 0.83 (Table 4). This rule was less sensitive but more specific to non-CAP-DRPs than other rules.

Table 4

Predictive rules or criteria for non-MRSA CAP-DRPs and MRSA

Prediction model	Cutoff score or criteria	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Accuracy (95% CI)
Predictive rules or criteria for non-MRSA CAP-DRPs
Shindo’s CAP-DRP	>3	0.29 (0.13-0.49)	0.87 (0.83-0.91)	0.15 (0.07-0.28)	0.94 (0.90-0.96)	0.83 (0.78-0.86)
Health care-associated pneumonia	Yes	0.71 (0.51-0.87)	0.67 (0.61-0.72)	0.15 (0.09-0.22)	0.97 (0.93-0.99)	0.67 (0.62-0.72)
Shorr	>1	0.64 (0.44-0.81)	0.63 (0.58-0.68)	0.13 (0.08-0.19)	0.96 (0.92-0.98)	0.63 (0.58-0.68)
Aliberti	>3	0.68 (0.48-0.84)	0.56 (0.50-0.61)	0.11 (0.07-0.17)	0.96 (0.92-0.98)	0.57 (0.51-0.61)
Brito and Niederman	Yes	0.40 (0.22-0.59)	0.81 (0.77-0.86)	0.15 (0.08-0.25)	0.94 (0.91-0.97)	0.78 (0.74-0.82)
Schreiber	>2	0.54 (0.34-0.73)	0.74 (0.69-0.79)	0.14 (0.08-0.23)	0.95 (0.92-0.97)	0.72 (0.68-0.77)
Prina	>2	1.0 (0.82-1.00)	0.09 (0.06-0.12)	0.08 (0.06-0.12)	1.0 (0.83-1.00)	0.16 (0.12-0.20)
Predictive rules or criteria for MRSA
Shindo’s CAP-DRP	>3 or 2+1 MRSA-specific factor	0.53 (0.39-0.67)	0.86 (0.82-0.90)	0.38 (0.27-0.50)	0.92 (0.88-0.95)	0.82 (0.77-0.85)
Shindo’s MRSA-specific	>3	0.41 (0.28-0.56)	0.92 (0.88-0.95)	0.45 (0.30-0.60)	0.91 (0.87-0.94)	0.85 (0.81-0.88)
Shindo’s MRSA-specific	>4	0.22 (0.11-0.35)	0.99 (0.97-1.00)	0.73 (0.45-0.92)	0.89 (0.85-0.92)	0.88 (0.84-0.91)
Shorr’s MRSA score	>2	0.84 (0.71-0.93)	0.55 (0.49-0.61)	0.23 (0.17-0.30)	0.96 (0.92-0.98)	0.59 (0.54-0.64)

Abbreviations: PPV, positive predictive value; NPV, negative predictive value; CI, confidence interval; CAP-DRP, community-acquired pneumonia drug-resistant pathogen; MRSA, methicillin-resistant Staphylococcus aureus.

The predictive performance of the CAP-DRP rule varying the threshold of MRSA-specific risk factors was also compared to Shorr’s MRSA score (Table 4). The CAP-DRP prediction rule of ≥3 CAP-DRP risk factors or that of 2 points plus any of MRSA-specific risk factors differentiated patients with MRSA and non-MRSA with a sensitivity of 0.53, specificity of 0.86, PPV of 0.38, NPV of 0.92, and accuracy of 0.82. Furthermore, we also assessed the performance of our MRSA-specific risk estimation by changing the threshold number to 3 and 4. When the threshold of 4 was adopted, sensitivity and specificity were 0.22 and 0.99, respectively, with PPV of 0.73 and NPV of 0.89. Accuracy (0.88) was similar to that of the CAP-DRP rule. Compared to Shorr’s MRSA score, the CAP-DRP rule and the MRSA-specific risk factors indicated lower sensitivity and higher specificity.

Discussion

In this multicenter, prospective, observational study, we evaluated the predictive rules for CAP-DRP prevalence in patients with community-onset pneumonia. The CAP-DRP rule demonstrated high specificity in all the predictive rules tested and may reduce the use of unnecessary broad-spectrum antibiotics.

The definition of DRPs is important to discuss the predictive methods. MRSA, extended-spectrum β-lactamase-producing Enterobacteriaceae, P. aeruginosa, and other non-fermenting gram-negative rods including Acinetobacter baumannii are often defined as potential DRPs.24,25 However, some isolates of P. aeruginosa and A. baumannii may be susceptible to third-generation cephalosporin and fluoro-quinolones.26–28 In fact, the definition based on antibiotic susceptibility test results avoided overdiagnosis of DRPs with a reduction of 6.7%.29 Therefore, we defined CAP-DRPs as being not susceptible to antibiotics commonly used for patients with CAP (non-pseudomonal β-lactams, macrolides, and respiratory fluoroquinolones) with emphasis on drug susceptibilities.11,29 Our previous study revealed that the risk factors for CAP-DRPs were identical between both patients with CAP and HCAP.11 The 2016 IDSA/ATS guidelines for hospital-acquired and ventilator-associated pneumonia state that patients with HCAP should be treated in the same way as CAP.12 Thus, a unified treatment strategy for both CAP and HCAP (community-onset pneumonia) should be established, and one of the most important points is to predict patients with CAP-DRPs at diagnosis.11,13,30 However, validation studies on predictive rules for DRPs are limited.31,32 There fore, we evaluated the validity of several predictive rules for CAP-DRPs including our proposed rule (CAP-DRP rule) in patients with community-onset pneumonia.

Among the predictive rules tested in this cohort, the CAP-DRP rule differentiated patients at low risk from high risk of CAP-DRPs with high specificity (0.87 [95% CI: 0.83–0.91]). That is, the CAP-DRP rule may identify patients who do not require empirical broad-spectrum antibiotics. Our previous study revealed that antipseudomonal antibiotic use significantly increased the risk of death within 30 days after diagnosis in patients with non-CAP-DRPs such as S. pneumoniae and H. influenzae.3 Therefore, through the avoidance of broad-spectrum antibiotic overuse, patients with no or one risk factor for CAP-DRPs and those with two risk factors for CAP-DRPs plus no MRSA-specific risk factors might improve their outcomes.33 However, the AUROC of the CAP-DRP rule was 0.73, indicating moderate discriminability.34 This performance was quite similar to that of Shorr’s and Aliberti’s rules. Furthermore, even though some tested predictive models had relatively high specificities, their sensitivities tended to be low (Table 3). Therefore, predictive methods may not be suitable for discriminating patients at high risk for CAP-DRPs. For clinical application, predictive methods including the CAP-DRP rule should be refined to achieve higher predictive performance by adding some factors, eg, positive CAP-DRP history, and/or by combining with rapid diagnostic test results of antibiotic-resistant genes so that physicians can identify patients with CAP-DRPs in order to select appropriate initial antibiotics.

CAP-DRPs can be divided into MRSA and non-MRSA. Although the separation of MRSA from other CAP-DRPs is important for the selection of therapeutic regimens (ie, anti-MRSA or antipseudomonal antibiotics),35 many predictive rules have been created without any consideration for this separation.13 In this study, we additionally assessed the predictive performance for MRSA and non-MRSA CAP-DRPs separately. While the sensitivity of the CAP-DRP rule for non-MRSA CAP-DRPs was lower than other predictive rules, its specificity was higher than others. Therefore, when a patient has two or less risk factors for CAP-DRPs, antipseudomonal antibiotics should be refrained. Regarding prediction of MRSA, we assessed the predictive performance of the MRSA-specific risk score in addition to the CAP-DRP rule. When the cutoff point of ≥4 was adopted, the specificity was highest (0.99) with a PPV of 0.73 and NPV of 0.89. When patients have an MRSA-specific risk score of ≤3, anti-MRSA antibiotics would be unnecessary; similarly so in cases with no or one risk factor for CAP-DRPs and two risk factors for CAP-DRPs and none of three MRSA-specific risk factors. Thus, to ensure appropriate antibiotic treatment for patients with CAP-DRPs, it would be preferable to make a distinction between MRSA and non-MRSA CAP-DRPs such as P. aeruginosa.

There are some variations between countries or regions regarding risk factors for DRPs, proposed predictive rules, and results of validation studies.13,16,29,31,36,37 As Webb et al mentioned, risk factors for DRPs are composed of three factors including host (intrinsic) factors, environment (extrinsic) factors, and selective antibiotic pressure,16 the latter of which would differ by countries or regions.29 Therefore, predictive rules for DRPs should be created based on local data in the respective countries or regions. If a predictive rule created in a different region is applied, it should be locally validated before clinical use.

In this study, we did not include patients without identified pathogens. Labelle et al and Andruska et al demonstrated that patients with culture-negative pneumonia had lower severity of illness and hospital readmission rate compared with culture-positive patients.38,39 Therefore, we considered that patients without identified pathogens should ideally not be analyzed in the same manner as those with identified pathogens. Although prediction models for identifying patients with DRPs were developed among only patients with identified pathogens in most previous studies,5,11,15,21,22 physicians must determine the initial antibiotics for both patients with identified pathogens and those without. Therefore, researchers in future should discuss whether patients without identified pathogens (about half of all patients with community-onset pneumonia) should be included when developing the prediction model.

Limitations

This study had some limitations. First, outpatients were not included in this study. Therefore, the results of this study should be applied in only patients admitted to the hospital. Second, the pathogens identified in this study may not have been the cause of pneumonia in some cases because the cultures were performed semiquantitatively rather than quantitatively. Third, the prediction rules tested in this study were developed based on different definitions of DRPs. The difference between this study and the original ones might affect each predictive performance. Fourth, we could not compare with the Drug Resistance in Pneumonia (DRIP) score that was recently published and includes prior DRP history as a risk factor for DRPs.32 We will obtain prior non-MRSA CAP-DRP history and evaluate the DRIP score in future studies.

Conclusion

This multicenter prospective study examined the performance of several predictive rules or criteria for CAP-DRPs in patients with community-onset pneumonia. The predictability of the predictive rules is limited. The CAP-DRP rule yielded high specificity among the tested rules, and can reduce the overuse of broad-spectrum antibiotics and identify patients who can be treated by non-broad-spectrum antibiotics such as a non-antipseudomonal β-lactam plus a macrolide.

Data availability

The data sets used and analyzed for the current study are available from the corresponding author on request.