Trends in Illness Severity, Hospitalization, and Mortality for Community-Onset Pneumonia at 118 US Veterans Affairs Medical Centers.

BACKGROUND: Deaths from pneumonia were decreasing globally prior to the COVID-19 pandemic, but it is unclear whether this was due to changes in patient populations, illness severity, diagnosis, hospitalization thresholds, or treatment. Using clinical data from the electronic health record among a national cohort of patients initially diagnosed with pneumonia, we examined temporal trends in severity of illness, hospitalization, and short- and long-term deaths.
DESIGN: Retrospective cohort PARTICIPANTS: All patients >18 years presenting to emergency departments (EDs) at 118 VA Medical Centers between 1/1/2006 and 12/31/2016 with an initial clinical diagnosis of pneumonia and confirmed by chest imaging report. EXPOSURES: Year of encounter. MAIN MEASURES: Hospitalization and 30-day and 90-day mortality. Illness severity was defined as the probability of each outcome predicted by machine learning predictive models using age, sex, comorbidities, vital signs, and laboratory data from encounters during years 2006-2007, and similar models trained on encounters from years 2015 to 2016. We estimated the changes in hospitalizations and 30-day and 90-day mortality between the first and the last 2 years of the study period accounted for by illness severity using time covariate decompositions with model estimates.
RESULTS: Among 196,899 encounters across the study period, hospitalization decreased from 71 to 63%, 30-day mortality 10 to 7%, 90-day mortality 16 to 12%, and 1-year mortality 29 to 24%. Comorbidity risk increased, but illness severity decreased. Decreases in illness severity accounted for 21-31% of the decrease in hospitalizations, and 45-47%, 32-24%, and 17-19% of the decrease in 30-day, 90-day, and 1-year mortality. Findings were similar among underrepresented patients and those with only hospital discharge diagnosis codes.
CONCLUSIONS: Outcomes for community-onset pneumonia have improved across the VA healthcare system after accounting for illness severity, despite an increase in cases and comorbidity burden.

INTRODUCTION

Pneumonia is the leading cause of death from infectious disease in the USA.[1] While the COVID-19 pandemic saw a dramatic increase in deaths in 2020, studies prior to the pandemic had observed a decrease in pneumonia mortality over time,[2] particularly among hospitalized patients.[3] However, the cause of the observed decreases in death is unclear. Changes in patient characteristics, diagnostic labeling,[4] and care processes have all been proposed to contribute to changes in outcomes. Large population studies have previously been limited by characterizing patients with administrative claims data, which are imprecise.[5-7] The purpose of this study was to examine temporal trends in severity of illness, hospitalization, and mortality among patients initially diagnosed with pneumonia at 118 Veterans Affairs (VA) emergency departments (EDs) using clinical data from the electronic health record (EHR). We sought to determine the extent to which temporal trends in these three outcomes could be accounted by changes in illness severity between 2006–2007 and 2015–2016, and conversely.

METHODS

Setting

The VA network is the largest integrated healthcare network in the USA, serving nearly 8.5 million veterans at more than 1700 clinics, 152 VA Medical Centers (VAMCs), and 21 regional VA Integrated Service Networks (VISNs) with over 500,000 hospitalizations annually.[8] All health care settings in the VA share the same clinical electronic health record (EHR), Veterans Health Information Systems and Technology Architecture (VistA). All VA emergency departments have access to radiology and laboratory, and the ability to hospitalize patients to acute care beds at an adjoined VA hospital, with over 1M visits per year. Data were accessed through the Veterans Informatics and Computing Infrastructure, a computing environment that stores clinical data for research purposes.[9] The research was approved by VA and University of Utah Institutional Review Boards.

Participants

We identified all patient encounters at EDs from 1 January 2006 to 31 December 2016 that underwent chest imaging, including chest X-rays or computerized tomography (CT) scans within 24 h before and after the encounter time and with at least one clinical document signed by a physician, nurse practitioner, or physician assistant associated with the ED visit. To capture only new diagnoses, we included only the first encounter for a given patient within a 3-month period. To define cases of pneumonia, we avoided hospital discharge diagnosis codes because it limits the ability to examine outpatients among patients who were not hospitalized, and can lack confirmation of pneumonia in chest imaging up to 30% of the time;[10] however, we also conducted a secondary analysis among cohorts using this definition (Appendix C). We first defined a clinical diagnosis as one with either (1) an ED-assigned pneumonia diagnosis based upon established diagnostic codes [3,4] or (2) a clinical diagnosis of pneumonia within the assessment and plan section of the ED physician document identified by natural language processing (NLP), developed in previous work.[11] We then applied NLP of chest imaging reports to include only those cases with a report containing a radiographic assertion of pneumonia. Appendix A described NLP tool development and validation.

Measurements

Clinical Outcomes

We identified deaths from any cause occurring within 30, 90, and 365 days from the initial encounter using the VA Vital Status file. We defined a hospitalization as admission to an acute medical, surgical, or intensive care unit (ICU) that occurred within 24 h of the ED encounter.

Patient Characteristics Contributing to Illness Severity

We extracted baseline patient characteristics as well as clinical data from the EHR that have been identified as important to this construct for pneumonia by previous literature and clinical experience. We extracted demographics including age, gender, marital status, homelessness, nursing home residence, 38 comorbid conditions, 5 vital signs, and 21 laboratory values. To define each comorbidity, we extracted all International Classification of Disease (ICD)-9-CM[12] and ICD-10-CM[13] codes given to each patient within the year prior to the ED encounter. We classified them according to categories developed by the Agency for Healthcare Research and Quality.[14,15] We also classified psychosocial comorbidities including psychiatric diseases, substance use, and demographics including homelessness and marital status. Each comorbidity was treated as an independent covariate in the illness severity model. We extracted the first vital sign (temperature, respiratory rate, heart rate, systolic blood pressure, diastolic blood pressure) and pulse oximetry reading recorded between 6 h before and 12 h after the initial encounter time. We extracted the first laboratory result within 6 h before and 12 h after the initial encounter time, and included laboratory values previously proposed to predict pneumonia or sepsis severity,[16-20] including albumin, bilirubin, blood urea nitrogen, creatinine, C-reactive peptide, glucose, hematocrit, lactic acid, arterial pH, PaO2, PaCO2, serum bicarbonate, platelet count, potassium, sodium, troponin, white blood cell count, brain natriuretic peptide, erythrocyte sedimentation rate (ESR), and liver transaminases.

STATISTICAL ANALYSIS

Temporal Trends in Patient Characteristics, Hospitalizations, and Outcomes

Patient characteristics, hospitalizations, and outcomes were summarized overall, and results from the earliest 2 calendar years were compared to the latest 2 calendar years in the study period. The proportion and 95% CI of patient encounters with each comorbidity, median and interquartile range of each vital sign and laboratory result, and proportion and 95% CI of encounters meeting the criteria for systemic inflammatory response syndrome (SIRS—temperature > 100.4°F (38°C), pulse > 90 bpm, respiratory rate > 20 bpm, white blood cell count > 12 or < 4 K/μL) with the exception of mental status[21] were calculated for the combined years of 2006–2007 and for the years 2015–2016.

Temporal Trends in Illness Severity

We explored several representations of patient illness severity, which is generally accepted as an impression composed of observable clinical manifestations, including baseline patient vulnerability as well as physiologic measures, that portend risk of an undesired event for a given patient.[22,23] Therefore, we calculated the predicted risk for each event of interest—hospitalization and 30-day, 90-day, and 1-year mortality—based upon observable patient characteristics from the EHR for each patient.

We modeled temporal trends in illness severity by using a 2-step process. In the first step, we used the Extreme Gradient Boosting algorithm (XGBoost),[24] a machine learning approach that we previously validated to be highly accurate for our population,[25] to develop cross-sectional models to predict the probabilities of hospitalization, 30-day mortality, 90-day mortality, and 1-year mortality for each encounter based on extracted patient factors during a particular index time period. In the second step, we applied the predictive models developed in the first step to data extracted from patient visits occurring during successive 1-year intervals from 2006–2007 through 2015–2016. The changes in the average predicted probabilities of the respective outcomes (hospitalization, 30-day mortality, 90-day mortality, and 1-year mortality) over the successive 1-year intervals summarize the temporal trend in predicted illness severity using the extracted patient factors under the cross-sectional models developed in step 1.

For each event of interest, we developed cross-sectional predictive models using three different sets of extracted patient characteristics as predictor variables. The three sets of patient characteristics were as follows: (a) 69 extracted characteristics including demographics, comorbidities, and labs (the “complete” model); (b) a restricted set of 28 of the 69 extracted patient characteristics including age, sex, vital signs, and labs (“physiologic model”); and (c) a restricted set of 43 of the 69 extracted characteristics including demographics and comorbidities (“comorbidity model”). We considered the physiologic model to evaluate trends in predicted illness severity that are independent of drifts in coding of comorbidities over time. We considered the comorbidity model to provide predictions that do not require vitals and lab measurements, which are not always available. Because we observed changes in laboratory ordering (increase in ordering lactate, decrease in ABG ordering) and comorbidities across the time period, we explored whether the modeled trends in illness severity were robust to cross-sectional models derived from the beginning versus the end of the study period by selecting two extreme index time periods: for the development of the cross-sectional models in step 1: 2006–2007 and 2015–2016. Performance of all models is reported in Appendix C.

We jointly displayed the longitudinal changes in observed versus predicted mortality based on the six models in Figs. 2 and 3, with similar displays for other outcomes. Visual comparisons of the trends in observed versus predicted mortality indicate the extent to which changes in observed mortality were accompanied by changes in comorbidities and physiologic derangements over the same period. We conducted the same analysis among different case definitions of pneumonia and underrepresented patient subgroups (> 80 years, immunocompromised, self-identifying as black, rural, female, and presenting to small hospitals) in Appendix C.

Figure 2

Observed versus predicted 30-day mortality by three different models trained on early years (2006–2007—panel A) and later years (2015–2016—panel B). Complete model using all 69 patient factors (gray line); physiologic model using age, sex, vitals, and laboratory results (black dotted line); and comorbidity model using age, sex, and baseline patient demographics and comorbidities (gray dotted line). Observed 30-day mortality is represented as a black line.

Figure 3

Observed trends in 30-day, 90-day, and 1-year mortality and hospitalization (solid line) versus predicted trends in same outcomes based on the complete model using all patient features trained on early years (2006–2007—dotted line) and later years (2015–2016—dashed line).

We then decomposed the longitudinal change in observed mortality over the study period into two components: (1) the ratio of the change in predicted mortality to the change in observed mortality and (2) the difference between 1 and the above quantity. This decomposition was defined for each outcome (30-day mortality, 90-day mortality, 1-year mortality, and hospitalization) and for each of the six models defined by the three sets of extracted patient factors (69-variable, 28-variable, and 43-variable) and the two index time periods (2006–2007 and 2015–2016). The ratio for the first term in the decomposition is equal to: where (Predicted Mortality in 2015 – 2016 ∣ 2006 – 2007 Model) represents the average mortality that would result in 2015–2016 based on the changes in the extracted patient characteristics included in the model between 2006–2007 and 2015–2016 if the relationship between mortality and the predictor variables in the model remained unchanged between these two time periods. Thus, we refer to this first component of the decomposition as the proportion of the observed mortality change accounted for by change in patient characteristics under the model, and to the second component as the proportion of the observed mortality change that was not accounted for by change in extracted patient characteristics under the model. Similar interpretations apply for the other outcomes. Missing vital signs (occurred in <5% of all cases) were treated as missing at random and replaced with single imputation of median values; missing pulse oximetry (in 15%) and lab values (occurred in <10% for routine labs) were replaced with a normal value similar to previous work[25] and other studies,[26] since these were not missing at random and represented patients for whom providers perceived a lower illness severity. A secondary analysis using models that include an indicator variable for missing data[25] was also conducted (Appendix C).

RESULTS

Among 24M ED encounters during the study period, 3.98M had chest imaging; 297,498 had an initial diagnosis of pneumonia, of which 196,899 had a radiographic report consistent with pneumonia (Fig. 1). The median (interquartile range) age was 68 (61–80) years; 96% were male. Overall 30-day mortality was 8%, 90-day mortality 14%, 1-year mortality 26%, and hospitalization 67%.

Figure 1

Study population.

Several recorded comorbidities increased between 2006–2007 and 2015–2016 (Table 1, Appendix B), most notably renal disease (10% versus 16%), diabetes with (8% versus 15%) and without (27% versus 33%) complications, mood disorders (20% versus 27%), and coronary artery disease (20% versus 26%); notably, there was no change in myocardial infarction and only a slight increase in congestive heart failure. Despite the increase or stability in identified comorbidities, illness severity predicted by model estimates decreased. While most median physiologic variables were unchanged, median pulse and platelet count decreased, brain natriuretic peptide increased, and encounters meeting criteria for SIRS decreased from 54 to 39% (Table 1).

Table 1

Patient Characteristics Overall, During Years 2006–2007, and During Years 2015–2016. Comorbidities with >2% Change or Notable Stability and Most Commonly Used Physiologic Measures Shown. For Complete List of Patient Characteristics, Please See Appendix B. Frequency (Percent Confidence Intervals) and Median (Confidence Intervals) Are Shown

		All patients N=196,899	Year 2006–07: N=17,227	Year 2015–16: N=46,629
Demographics
Age		68 (68, 69)	68 (67, 68)	69 (69, 69
Male gender		188,522 (96–96%)	21997 (96–97%)	44333 (95–95%)
Married		8,9050 (45–45%)	10502 (43.5%, 45%)	21140 (45%)
Homelessness		1,0905 (5.4–5.6%)	1658 (3.1–3.6%)	3513 (7.3–7.8%)
Nursing home resident		3,398 (1.7–1.8%)	393 (1.4–1.8%)	775 (1.5–1.8%)
Comorbidities*
Diabetes without complications		6,0721 (31–31%)	4630 (26–28%)	15214 (32–33%)
Diabetes with complications		2,0957 (10–11%)	1308 (7–8%)	7194 (15–16%)
Renal disease		2,7610 (14–14%)	1778 (10–11%)	7482 (16–16%)
Coronary artery disease		4,5915 (23–24%)	3503 (20–21%)	11988 (25–26%)
Congestive heart failure		34,770 (18–18%)	2790 (16–17%)	8559 (18–19%)
Myocardial infarction		5,1304 (26–26%)	4372 (25–26%)	11822 (25–26%)
Chronic obstructive pulmonary disease		7,7968 (39–40%)	6828 (39–40%)	18383 (39–40%)
Tobacco use disorders		4,6332 (23–24%)	3578 (20–21%)	10774 (23–24%)
Mood disorders		50,064 (25–26%)	3443 (19–21%)	12592 (27–27%)
Anxiety		38,194 (19–20%)	2426 (14–15%)	10707 (23–23%)
Delirium disorders		20,785 (10–11%)	1478 (8–9%)	5598 (12–12%)
Physiologic measures	% missing
Temperature (°F)	3.4	98.7 (98.7–98.7)	98.9 (98.9–98.9)	98.6 (98.6–98.6)
Pulse (beats/min)	2.8	92.7 (92.6, 92.7)	94.4 (94.1, 94.7)	91.9 (91.8, 92.1)
Respiration rate (breaths/min)	3.1	20.4 (20.3, 20.4)	21.3 (21.3, 21.4)	19.8 (19.8, 19.9)
Systolic blood pressure (mmHg)	2.9	132.4 (132.3, 132.5)	130.3 (129.9, 130.7)	134.3 (134, 134.5)
Pulse oximetry (%)	15	93.6 (93.6, 93.7)	93.3 (93.3, 93.4)	93.8 (93.8, 93.9)
PaO2 (mmHg)	78.7	73.9 (73.5, 74.2)	71.4 (70.4, 72.4)	75.1 (74.3, 75.9)
Arterial pH	78	7.4 (7.4, 7.4)	7.4 (7.4, 7.4)	7.4 (7.4, 7.4)
Blood urea nitrogen (mg/dL)	8.7	23.3 (23.2, 23.4)	24.1 (23.8, 24.3)	22.7 (22.5, 22.8)
Creatinine (mg/dL)	8.3	1.4 (1.4, 1.4)	1.5 (1.5, 1.5)	1.4 (1.4, 1.4)
C-reactive peptide (mg/dL)	97.2	92.3 (89.7, 94.9)	136.9 (117, 156.9)	82.3 (78, 86.6)
Lactic acid (mmol/L)	78.7	2 (2, 2)	2.3 (2.2, 2.4)	1.9 (1.9, 1.9)
Bilirubin (mg/dL)	28.7	0.9 (0.9, 0.9)	0.9 (0.9, 1)	0.9 (0.9, 0.9)
Albumin (g/L)	29	3.4 (3.4, 3.4)	3.4 (3.4, 3.4)	3.4 (3.4, 3.5)
Glucose (mg/dL)	8.1	139.4 (139, 139.7)	139.8 (138.8, 140.9)	139.9 (139.2, 140.5)
Platelet count (K/μL)	8.7	237.3 (236.8, 237.8)	264.9 (263, 266.8)	232.3 (231.3, 233.3)
White blood cell (K/μL)	8	11.9 (11.8, 11.9)	12.6 (12.5, 12.7)	11.5 (11.4, 11.5)
Systemic inflammatory response syndrome (SIRS)*		87,738 (44–45%)	9249 (53–54.4%)	18228 (39–40)
Outcomes
30-day mortality		15,574 (7.8–8%)	1659 (9.2–10%)	3229 (6.7–7.2%)
90-day mortality		27,192 (14–14%)	2818 (16–17%)	5751 (12–13%)
1-year mortality		50,718 (26–26%)	4999 (28–30%)	11117 (24–24%)
Hospitalization		131,307 (67–67%)	12167 (70–71%)	29233 (62–63%)
7-day secondary hospitalization (among outpatients)		4,919 (7.3%, 7.7%)	405 (7.3–8.8%)	1165 (6.3–7.1%)
ICU admission (among inpatients)		19433 (14.6%, 15%)	1812 (14.2%, 15.5%)	4268 (14.2%, 15%)

*SIRS, systemic inflammatory response syndrome[27] with the exception of mental status changes. Patients deemed to have SIRS if any two of the criteria were met: temperature > 100.4°F (38°C), pulse > 90 bpm, respiratory rate > 20 bpm, white blood cell count > 12 or < 4 K/μL

Hospitalization and mortality decreased steadily between 2006–2007 and 2015–2016 and in excess of model predictions regardless of the patient characteristics used (Table 1, Figs. 2 and 3). Thirty-day mortality decreased from 10 to 7%, 90-day mortality 16 to 12%, 1-year mortality 29 to 24%, and hospitalization 71 to 63%. Predicted 30-day mortality risk decreased across years for the complete model and the physiologic model but increased for the comorbidity model (Fig. 2). This finding was similar for hospitalization as well as 90-day and 1-year mortality. The observed decreases exceeded the predicted decreases (Fig. 3): 30-day mortality decreased by an absolute 3%, compared to predicted decrease of 1.3%; 90-day mortality decreased by 4%, compared to predicted decrease of 1.5%; 1-year mortality decreased by an absolute 5%, compared to predicted decrease of 1.4–2.2%; and hospitalization decreased by an absolute 8%, compared to predicted decrease of 5.6–6.4%. Results were similar among patient subgroups, with the exception of a steeper decrease in observed mortality for patients identified by hospital discharge codes for pneumonia, smaller improvements for patients > 80 years of age and a rural address, and increases in severity of illness, hospitalization, and mortality for patients with immunocompromising comorbidities (Appendix C). Decomposition of time-varying covariates (Table 2) revealed that 45–47% of the decrease in 30-day mortality was explained for by changes in patient characteristics. Meanwhile, 32–34% of the decrease in 90-day mortality, 17–19% of the decrease in 1-year mortality, and 21–31% of the decrease in hospitalizations were explained by patient characteristics.

Table 2

Model-Predicted Outcomes and Estimated Proportion of Change Explained by Patient Characteristics

Outcome and Model	Probability of outcome predicted by 2006–2007 model		Probability of outcome predicted by 2015–2016 model		Percent of change explained by patient characteristics*
	Year		Year
	2006	2016	2006	2016	2006–2007 model	2015–2016 model
30-day mortality
Observed	9.6%	7.2%	9.6%	7.2%
Complete model	9.7%	8.4%	8.2%	6.9%	47%	45%
Physiologic model	9.7%	8.4%	8.0%	6.9%	44%	39%
Comorbidity model	9.5%	10.4%	6.2%	7.0%	−35%	−30%
90-day mortality
Observed	16.4%	12.3%	16.4%	12.3%
Complete model	16.5%	15.1%	13.7%	12.2%	32%	34%
Physiologic model	16.7%	13.7%	14.0%	12.3%	68%	39%
Comorbidity model	16.3%	17.2%	11.7%	12.8%	−28%	−32%
1-year mortality
Observed	29.0%	23.8%	29.0%	23.8%
Complete model	29.1%	28.1%	24.7%	23.6%	17%	19%
Physiologic model	29.2%	27.5%	25.4%	23.7%	31%	31%
Comorbidity model	28.8%	31.0%	21.3%	23.7%	−44%	−47%
Hospitalization
Observed	70.6%	62.7%	70.6%	62.7%
Complete model	71.0%	69.2%	65.1%	62.5%	21%	31%
Physiologic model	71.1%	68.6%	65.6%	62.6%	29%	36%
Comorbidity model	62.6%	62.8%	61.0%	62.6%	−12%	−79%

*Negative values denote that the model predicted an increase in the outcome

CONCLUSION/DISCUSSION

In a national cohort of Veterans presenting to emergency departments with initial diagnoses of pneumonia that were radiographically confirmed, we found a significant decrease in deaths and hospitalizations between 2006 and 2016. While illness severity, but not comorbidity burden, also decreased during this period, it accounted for less than half of the observed decrease in deaths and hospitalizations.

Prior to the COVID-19 pandemic, other studies had also observed overall declines in short-term mortality attributed to pneumonia over the past 20 years,[28,29] although the trend was not consistent for all settings.[30,31] Without detailed clinical data, these studies were unable to explore the mechanism of the observed reduced mortality. Most population studies in pneumonia use inpatient administrative claims data to identify cases, which are codes assigned retrospectively at hospital discharge, have only moderate accuracy identifying cases[5-7] and limit the ability to examine outcomes in patients treated as outpatients. Further, increased attention to hospital performance measures has caused a shift from coding pneumonia as a principal diagnosis to sepsis or respiratory failure.[4] Our study is the first population-based study that used only information available upon initial presentation, verified the population with radiographic confirmation of pneumonia, and had sufficient clinical detail to decompose trends in outcomes that may be associated with patient comorbidity and physiologic burden.

The comorbidity of the cohort increased slightly, suggesting a more chronically ill population. This is also suggested by other Veteran population studies,[32] specifically with increases in obesity and diabetes,[33] kidney disease,[34] and lung disease among younger Veterans,[35] although hospitalizations for cardiovascular events have decreased.[36] To interpret these trends, it is important to recognize that comorbidities identified in the EHR are a function of diagnosis. Certain comorbid illnesses may be increasingly identified and managed due to an increase in access and recognition. For example, coronary artery disease and congestive heart failure increased, but myocardial infarction remained stable, suggesting more diagnoses of heart disease, but not more heart attacks. Cardiovascular events are an increasingly recognized complication of pneumonia that contributes to long-term mortality.[37] Early recognition and management of heart disease could modify this risk, producing better outcomes in pneumonia. The mismatch between comorbidity-based predictions and clinical illness severity is similar to other studies[38] and has important implications for research and quality efforts: using risk predictions based only on comorbidities can lead to important inaccuracies when it comes to understanding changes in outcomes.

The decreased in illness severity could be explained by several trends. First, the use of VA care has increased dramatically since 2006, due to recession, loss of private insurance, increases in Veteran enrollment, and use of outpatient services.[32,39] ED use increased accordingly, which may also be subject to shifting preferences or availability of primary care.[40] Widespread vaccination or better controlled comorbidities may manifest in milder acute disease. Adoption of childhood pneumococcal vaccines[41] and more comprehensive influenza vaccination programs for adults have previously been proposed as an important mechanism of lower severity and decreased admissions and deaths.[42]

Illness severity accounted for less than half of the decrease in mortality we observed. Several improvements of care may have led to this change, including pneumonia-specific care processes,[43,44] implementation of performance measures for sepsis,[45,46] and bundled care including early mobilization.[47,48] The decrease in acute hospitalization was also not explained by lower illness severity and was more dramatic than the decrease in deaths. This finding may be due to greater provider and patient preference toward outpatient management of pneumonia, possibly motivated by quantitative assessments of illness severity,[49] more out-of-hospital care support systems,[50,51] or reduced inpatient capacity.[52] The potential consequences of this decrease are unclear, but 7-day secondary admissions were stable, suggesting that this trend toward outpatient management may be safe or even beneficial compared to hospitalization for many patients with pneumonia.

We recognize several limitations to the study. It was retrospective, using clinical data from the electronic health record. While our cohort selection approach avoided the instability of discharge ICD codes, has previously demonstrated high accuracy,[11] and included chest imaging reports, our results can still be subject to sampling, measurement error, and missing data compared to a prospective study. On the other hand, prospective studies requiring consent can under-represent those patients less likely to participate. The trends in illness severity were seen using models at both ends of the study period, across multiple subgroups, leading us to conclude that the findings were not simply a function of changes in data quality, attenuation bias, or patient populations. However, our decompositions did not attempt to identify and include all confounding factors in the predictive models and are not equivalent to causal analyses. More work must be done to shed light on the mechanisms of better outcomes in pneumonia, including which patient factors are modifiable, which acute interventions are critical, and which processes of care should be more widely adopted. However, this study indicates that rates of death and hospitalization for community-onset pneumonia have decreased across the VA healthcare system after accounting for illness severity, which adds to the existing evidence that pneumonia care and outcomes have been improving.

Appendix A. Summary of development and validation of natural language processing tools. (DOCX 36 kb)

Appendix B. Complete patient characteristics. (DOCX 28 kb)

Appendix C. Secondary analyses and model performance: Secondary analyses of patients with hospital discharge diagnosis codes for pneumonia and subgroups including patients >80 years of age, with immunocompromised conditions, self-identifying as black, rural, presenting to small hospitals (complexity 2 and 3), women. Secondary analysis using predictive models with an indicator variable for missing data. Model performance. (DOCX 4333 kb)