An Evaluation of Antimicrobial Prescribing and Risk-adjusted Mortality.

The Centers for Disease Control and Prevention recommends tracking risk-adjusted antimicrobial prescribing. Prior studies have used prescribing variation to drive quality improvement initiatives without adjusting for severity of illness. The present study aimed to determine the relationship between antimicrobial prescribing and risk-adjusted ICU mortality in the Pediatric Health Information Systems (PHIS) database, assessed by IBM-Watson risk of mortality. A nested analysis sought to assess an alternative risk model incorporating laboratory data from federated electronic health records.
METHODS: Retrospective cohort study of pediatric ICU patients in PHIS between 1/1/2010 and 12/31/2019, excluding patients admitted to a neonatal ICU, and a nested study of PHIS+ from 1/1/2010 to 12/31/2012. Hospital antimicrobial prescription volumes were assessed for association with risk-adjusted mortality.
RESULTS: The cohort included 953,821 ICU encounters (23,851 [2.7%] nonsurvivors). There was 4-fold center-level variability in antimicrobial use. ICU antimicrobial use was not correlated with risk-adjusted mortality assessed using IBM-Watson. A risk model incorporating laboratory data available in PHIS+ significantly outperformed IBM-Watson (c-statistic 0.940 [95% confidence interval 0.933-0.947] versus 0.891 [0.881-0.901]; P < 0.001, area under the precision recall curve 0.561 versus 0.297). Risk-adjusted mortality was inversely associated with antimicrobial prescribing in this smaller cohort using both the PHIS+ and Watson models (P = 0.05 and P < 0.01, respectively).
CONCLUSIONS: Antimicrobial prescribing among pediatric ICUs in the PHIS database is variable and not associated with risk-adjusted mortality as assessed by IBM-Watson. Expanding existing administrative databases to include laboratory data can achieve more meaningful insights when assessing multicenter antibiotic prescribing practices.

INTRODUCTION

Antimicrobial use in pediatric intensive care units (ICUs) is common; they are prescribed to 43%–79% of ICU patients.[1-4] The Centers for Disease Control and Prevention (CDC) cited improvement in antimicrobial prescribing practices as “perhaps the single most important action needed to greatly slow down the development and spread of antibiotic-resistant infections.”[5]

Multi-center databases such as the Pediatric Health Information System (PHIS) are increasingly used to evaluate antimicrobial use.[2,6] Previous PHIS studies identified variability in antimicrobial prescribing in pediatric hospital and ICU settings not explained by demographic characteristics.[2,3] These publications are being used to guide antimicrobial stewardship efforts at multiple centers.[7-9] As an administrative database, PHIS was not designed to assess antimicrobial prescribing. It does not contain microbiological data, or documentation of clinical decision-making pertaining antibiotic prescribing.[10]

Skillfully refining antimicrobial prescribing in the ICU setting is possible, but challenges exist when treating patients with suspicion of sepsis.[11] Antimicrobial courses are often prescribed empirically; approximately 40% of sepsis in the ICU is culture-negative.[12] Even when a pathogen is identified, current culture methods may not detect co-infection.[13] The CDC recently released an “Antimicrobial Use and Resistance (AUR) Module,” in which it recommends risk-adjusted antimicrobial benchmarking.[14] Including risk-adjusted mortality as a balancing metric in stewardship projects would improve compliance with these recommendations and ensure optimal targeting of antimicrobials to patients most likely to benefit. Prior studies demonstrating variability in antimicrobial prescribing in pediatric ICUs have been substantially limited by a lack of adjustment for illness severity.[2]

The PHIS database does not contain previously-validated severity-of-illness scores, nor does it contain the variables necessary to manually derive these scores.[15-18] However, it does contain a proprietary pediatric risk of mortality score developed by IBM Watson (Watson score) that could potentially facilitate the assessment of risk-adjusted mortality.[10] Additionally, the federated electronic health record data contained in PHIS+, an expansion of the PHIS database, provides an opportunity to construct a robust risk score reflecting severity of illness.[19] PHIS+ was developed to add robust laboratory data from 6 large children’s hospitals to the existing administrative data harbored by PHIS. A key aim of PHIS+ was to support development of “severity adjustment tools to study inpatient quality (eg, comparing hospitals, providers, systems) across multiple institutions using large administrative and clinical databases.”[20]

The present study sought to evaluate the discrimination and calibration of the Watson score in mortality prediction in PHIS over the past decade and compare hospital-level antimicrobial prescribing with risk-adjusted mortality. Because the Watson score is proprietary, its performance was evaluated against a novel severity of illness score developed using PHIS+.

METHODS

Data Source

Clinical and pharmacy data were provided by the PHIS database, an online, anonymized, quality controlled data warehouse containing clinical and resource utilization information maintained by the Children’s Hospital Association (CHA).[10,21] We obtained laboratory data from PHIS+.[19]

Study Design and Patient Population

This was a retrospective cohort study of patients discharged from each of the 51 PHIS participating centers between 1/1/2010 and 12/31/2019 (Total Study Cohort). We included patients admitted to a pediatric or cardiac ICU during the study period. We excluded patients admitted to a neonatal ICU. The primary exposure was prescription of 1 of the 20 common antibacterials (amoxicillin, ampicillin, ampicillin/sulbactam, azithromycin, cefazolin, cefepime, cefotaxime, cefoxitin, ceftazidime, ceftriaxone, ciprofloxacin, clindamycin, erythromycin, gentamicin, meropenem, metronidazole, piperacillin/tazobactam, sulfamethoxazole/trimethoprim, tobramycin, vancomycin) or 8 common antifungals (amphotericin, anidulafungin, caspofungin, fluconazole, itraconazole, micafungin, posaconazole, voriconazole). These medications were chosen because they represented the great majority of total antimicrobials prescribed during the study period. We defined antimicrobial exposure as prescription of any of these antibacterials or antifungals during the intensive care unit stay. We further divided antibacterials into “antipseudomonal” (cefepime, ceftazidime, ciprofloxacin, meropenem, and piperacillin-tazobactam) and “antimethicillin resistant Staphylococcus aureus (anti-MRSA)” (clindamycin, sulfamethoxazole/trimethoprim, vancomycin) modified from Brogan et al and in compliance with the Surviving Sepsis Campaign.[2,22] The primary outcome was in-hospital mortality. The local Institutional Review Board approved the study.

Statistical Analysis

We extracted clinical and administrative data from PHIS (age, sex, race, insurance status, admission priority, primary admission diagnosis, and billing charges for mechanical ventilation, extracorporeal membrane oxygenation, dobutamine, dopamine, epinephrine, milrinone, norepinephrine, vasopressin, discharge mortality, and Watson score). We also extracted complex chronic conditions based on Feudtner and renal replacement therapy usage based on ICD codes from Waikar.[23,24] The cohort was described using summary statistics. Variation in antimicrobial prescribing practices between centers was analyzed by chi-square test of proportions or 1-way analysis of variance (ANOVA).

We assessed Watson score discrimination using receiver operating characteristic and precision recall curves,[25] and calibration using the GiViTI (Italian Group for the Evaluation of Interventions in Intensive Care Medicine) belt.[26,27] The need to recalibrate risk scores for external datasets or subpopulations of interest is widely recognized.[28] Because the Watson score was poorly calibrated, we randomly divided the cohort into training (50%) and testing (50%) subsets and recalibrated the Watson score using isotonic regression.[29] We compared the performance of the recalibrated score against the original model. We performed linear regression to analyze the relationship between antimicrobial days per 1000 ICU days with differences in center-level actual and predicted mortality. All statistical analyses were performed using R (versions 3.5.2 & 3.6, R Foundation for Statistical Computing, Vienna, Austria) and RStudio (RStudio, Boston, Mass., versions 1.1.463 & 1.2.5033). An α value of 0.05 was considered significant.

PHIS+ Nested Subset Analysis

The PHIS+ Subset included all patients in the total study cohort discharged from 1 of the 6 PHIS+ participating ICUs between January 1, 2010 and December 31, 2012 (the entire duration of PHIS+). The PHIS+ subset was described with summary statistics as above. We trained a multivariable logistic regression mortality prediction model, using laboratory data from PHIS+ (arterial partial pressure of oxygen, arterial partial pressure of carbon dioxide, arterial pH, venous pH, lactate, creatinine, bicarbonate, potassium, white blood cell count, platelet count, bilirubin, activated partial thromboplastin time, international normalized ratio, c-reactive protein, and ferritin). Candidate variables were determined a priori based on similarity to previously validated models.[16-18,30,31] Missing data were imputed with normal values, as is the convention for other mortality prediction scores.[16-18,30] We generated diagnostic risk groups based on Appendix 1 of Straney et al, based on primary encounter diagnoses.[18] We combined arterial and venous pH values. Creatinine values were normalized for age, based on Leteurtre et al.[16] We randomly divided the cohort into training (80%) and testing (20%) subsets because of smaller numbers than in the main analysis. A multivariable, fixed-effects logistic regression model (without interaction terms) to predict discharge mortality was generated using the variables above using the training subset of the cohort. We considered variables significant if they were associated with discharge mortality with P < 0.1 (variables removed: age, potassium, c reactive protein, milrinone use, and renal replacement therapy). To reduce overfitting, we conducted automated backward stepwise analysis using Akaike Information Criteria, and removed additional variables based on best performance (variables removed: total parenteral nutrition use, bicarbonate, bilirubin, activated partial thromboplastin time, ferritin).[32] We evaluated model discrimination and calibration on the testing subset as described above.

RESULTS

Cohort Demographics

The total study cohort included 953,821 encounters, and the PHIS+ subset included 45,706 encounters. Both cohort demographics are listed in Table 1. Briefly, the total study cohort median age was 4.4 years (interquartile range 0.9–12.6 years), with 55.1% men, and 59.0% white. Of the total study cohort, 67.2% had at least 1 complex chronic condition, and 29.0% of children were technology-dependent. An estimated 35.5% of patients received mechanical ventilation, and 1.0% were supported with ECMO. Overall survival to discharge was 97.3%.

Table 1.

Cohort Demographics

	Total Study Cohort (%)	PHIS+ Subset (%)
No. Admissions	953821 (100%)	45706 (100%)
Median Age, y (IQR)	4.4 (0.9–12.6)	4.2 (0.8–12.3)
Men	525430 (55.1%)	25379 (55.5%)
White	562939 (59.0%)	30731 (67.2%)
Black	186006 (19.5%)	6584 (14.4%)
Asian	30660 (3.2%)	1201 (2.6%)
Pacific Islander	4869 (0.5%)	407 (0.9%)
American Indian	6176 (0.7%)	190 (0.4%)
Other race	127285 (13.3%)	5481 (12.0%)
Hispanic	174627 (18.3%)	4138 (9.0%)
Commercial insurance	362297 (38.0%)	22725 (49.7%)
Government insurance	548471 (57.5%)	20136 (44.1%)
Other payor	43053 (4.5%)	2845(6.2%)
Any complex chronic condition	640904 (67.2%)	33794 (73.9%)
Cardiovascular condition	261524 (27.4%)	13868 (30.3%)
Gastrointestinal condition	187406 (19.7%)	9810 (21.5%)
Hematologic or immunologic condition	58956 (6.2%)	2880 (6.3%)
Malignancy	78116 (8.2%)	4163 (9.1%)
Metabolic condition	99054 (10.4%)	3737 (8.2%)
Neurologic or neuromuscular condition	214816 (22.5%)	10978 (24.0%)
Congenital or genetic defect	133828 (14.0%)	8498 (18.6%)
Renal or urologic condition	69524 (7.3%)	3621 (7.9%)
Respiratory condition	107079 (11.2%)	6344 (13.9%)
Technology dependent	276843 (29.0%)	14831 (32.5%)
Transplant recipient	26991 (2.8%)	2510 (5.5%)
Median ICU length of service, d (IQR)	2 (1–5)	2 (1–5)
Received mechanical ventilation	338933 (35.5%)	18703 (40.9%)
ECMO	10027 (1.0%)	491 (1.1%)
Survived to discharge	927970 (97.3%)	44352 (97.0%)

Antimicrobial Prescriptions

The antimicrobials included in the study represented 86.3% of the total ICU antimicrobial prescriptions during the study period; their relative frequencies are shown in Supplemental Digital Content 1. (See figure 1, Supplemental Digital Content 1, which shows antimicrobial prescriptions by drug. Antimicrobials are listed in descending order of frequency. Bar heights represent antimicrobial days of therapy per 1,000 pediatric ICU days, http://links.lww.com/PQ9/A317.) Overall, 71.0% (individual center range 49.7% – 83.8%, P < 0.001) of the cohort received antimicrobials while admitted to the ICU, with 18.2% (7.0 – 31.7%, P < 0.001) receiving antipseudomonal antibacterials and 30.2% (16.5 – 56.1%, P < 0.001) receiving anti-MRSA antibacterials. Antimicrobial duration normalized to total number of pediatric ICU days at each center was heterogeneous. (See figure 2, Supplemental Digital Content 2, which shows antimicrobial prescriptions by ICU. ICUs are listed in rank order of total days of antimicrobials per 1,000 ICU days. Bar heights represent the sum of the days of therapy of the study antimicrobial days divided by the total ICU length of stay at a given hospital, http://links.lww.com/PQ9/A318). Vancomycin was the most used antimicrobial across all centers (mean 181 days of therapy per 1000 ICU days). The use of anti-MRSA agents by center varied by > 4-fold (range 131–635 days of therapy per 1,000 ICU days, P < 0.001). The use of antipseudomonal agents by center varied by >7-fold (range 99–705 days of therapy per 1000 ICU days, P < 0.001). The most used antifungals were fluconazole (mean 50 days of therapy per 1000 ICU days) and echinocandins (caspofungin and micafungin; mean 33 days of therapy per 1000 ICU days). Fluconazole use varied by 6-fold (range 21–131 days of therapy per 1000 ICU days, P < 0.001) and echinocandin use varied by 58-fold (range 2–117 days of therapy per 1000 ICU days, P < 0.001).

Watson Score Discrimination and Calibration

The Watson score discrimination and calibration are illustrated in Figure 1. Overall, before and after calibration, the Watson score showed a c-statistic of 0.909 (95% CI 0.907–0.911) and 0.913 (95% CI 0.911–0.915) and an area under the precision recall curve of 0.313 and 0.325, respectively. As shown in Figure 1, before isotonic regression, calibration was poor (P < 0.001), with overprediction of mortality throughout the cohort. After calibration, the model showed acceptable calibration on both the full cohort (P = 0.109) and the test subset (P = 0.261).

Fig. 1.

Watson score performance. A, Receiver operating characteristic curve. B, Precision recall curve. The x axis shows recall (sensitivity). The y axis shows precision (positive predictive value). C, D, GiViTI calibration belt before and after isotonic regression, respectively. D, Calibration belt for full cohort, with more stringent internal calibration methodology applied. The P value is a likelihood ratio test evaluating the deviation of the model from the observed result; a nonsignificant P value is desirable.

Correlation of Antimicrobial Prescribing and Risk-adjusted Mortality

As shown in Figure 2, there was no correlation between antimicrobials per 1000 ICU days and risk-adjusted mortality according to the uncalibrated (adjusted r2 = –0.019, P = 0.83) or calibrated Watson score (adjusted r2=–0.020, P = 0.87). One ICU was an outlier in antimicrobial prescriptions; removal of this ICU did not change the result (adjusted r2 = –0.005, P = 0.386, data not shown). A sensitivity analysis excluding amoxicillin and erythromycin did not significantly change the results (adjusted r2=–0.020, P = 0.87, data not shown). As shown in Figure 3, overall antimicrobial prescriptions decreased by 42.5 days per 1000 ICU days over the study period (P < 0.001). During the same time period, 25 of 51 ICUs had a statistically significant trend of improvement in risk-adjusted mortality by linear regression (P < 0.05). No ICU had a significant trend toward worsening risk-adjusted mortality, and 26 of 51 ICUs had nonsignificant trends.

Fig. 2.

Correlation between antimicrobial prescriptions and risk-adjusted pediatric ICU mortality in PHIS. A, Antimicrobial days of therapy per 1000 pediatric ICU days versus differences from Watson score predicted mortality. B, Antimicrobial days of therapy per 1000 pediatric ICU days versus differences from Watson score predicted mortality after recalibration with isotonic regression. For both panels, each point represents a single hospital, with antimicrobial prescriptions per 1000 ICU days plotted on the x axis, and difference from predicted mortality (% mortality – % predicted mortality) on the y axis. For both panels, the line represents linear regression with 95% confidence interval.

Fig. 3.

Antimicrobial prescriptions over time and risk-adjusted pediatric ICU mortality. Each thin line represents 1 PHIS pediatric ICU over time. Antimicrobial days of therapy per 1000 pediatric ICU days are graphed on the y axis against patients’ discharge year on the x axis. Line color is representative of changes in risk-adjusted mortality over time as assessed by linear regression. ICUs with statistically significant reductions in risk-adjusted mortality are graphed in blue. Those with nonsignificant changes are graphed in black. There were no ICUs with significant increases in risk-adjusted mortality. The thick black line represents the line of best fit for the data, or average among the ICUs.

The PHIS+ nested subset contained 45,706/953,821 (4.8%) of the patients in the total study cohort. The demographics of the PHIS+ nested subset are listed in Table 1. The novel multivariable logistic regression mortality prediction model coefficients are included as supplemental table. (See table, Supplemental Digital Content 3, which shows multivariable model for mortality prediction, http://links.lww.com/PQ9/A320.) On the testing subset, the model had a c-statistic of 0.943 (95% CI 0.930–0.957; See figure 3, Supplemental Digital Content 4, which shows PHIS+ novel mortality score performance. Panel S3A shows the receiver operating characteristic curve. Panel S3B shows the precision recall curve. The x-axis shows recall (sensitivity). The y-axis shows precision (positive predictive value). Panel S3C shows the GiViTI calibration belt. The P-value is a likelihood ratio test evaluating the deviation of the model from the observed result, a non-significant P-value is desirable. All panels show the model performance on the testing subset, http://links.lww.com/PQ9/A319) and area under the precision recall curve of 0.524. Model calibration was adequate (P = 0.175; See figure 3, Supplemental Digital Content 4, which shows PHIS+ novel mortality score performance. Panel S3A shows the receiver operating characteristic curve. Panel S3B shows the precision recall curve. The x-axis shows recall (sensitivity). The y-axis shows precision (positive predictive value). Panel S3C shows the GiViTI calibration belt. The P-value is a likelihood ratio test evaluating the deviation of the model from the observed result, a non-significant P-value is desirable. All panels show the model performance on the testing subset, http://links.lww.com/PQ9/A319). The discrimination of the PHIS+ model is compared against the Watson score on the entire PHIS+ cohort in Figure 4. The PHIS+ model had significantly better discrimination than the Watson score (c-statistic 0.940 [95% CI 0.933–0.947] versus 0.891 [95% CI 0.881–0.901], respectively P < 0.001 and area under the precision recall curve 0.561 versus 0.297, respectively). Both the PHIS+ model and the Watson score showed a significant inverse association between antimicrobials per 1000 ICU days and risk-adjusted mortality (adjusted r2 = 0.57, P = 0.05, and adjusted r2 = 0.88, P < 0.01, respectively, Figure 5).

Fig. 4.

Comparison of Watson score and Novel Mortality Model Performance on the PHIS+ Subset. A, Receiver operating characteristic curve. The P value is DeLong’s test for related receiver operating characteristic curves. B, Precision recall curve. The x axis shows recall (sensitivity).

Fig. 5.

Correlation between antimicrobial prescriptions and risk-adjusted pediatric ICU mortality in the PHIS+ Subset. A, Antimicrobial days of therapy per 1000 pediatric ICU days versus differences from novel PHIS+ score predicted mortality. B, Antimicrobial days of therapy per 1000 pediatric ICU days versus differences from Watson score predicted mortality. For both panels, each point represents a single hospital, with antimicrobial prescriptions per 1000 ICU days plotted on the x axis, and difference from predicted mortality (% mortality – % predicted mortality) on the y axis. For both panels, the line represents linear regression with 95% confidence interval.

DISCUSSION

Pediatric database analyses are becoming increasingly influential in the design and implementation of antimicrobial stewardship projects.[7,8] They have the potential to provide baseline data that can be used to drive change in antibiotic prescribing practices. However, previous analyses have been constrained by limited ability to adjust for severity of illness, as recommended by the CDC.[2,14] In this large, decade-long, multi-center, retrospective study of the PHIS database, we report substantial heterogeneity in antimicrobial use among pediatric ICUs. Our findings are similar to those previously published, with 4-fold and 7-fold differences in anti-MRSA and antipseudomonal antibacterial use, respectively.[2] For the first time, we report that this variation in antimicrobial use is not associated with hospital-level risk adjusted mortality determined by the Watson score.

We also report that the PHIS ICUs reduced overall antimicrobial use over the study period with no significant worsening of risk-adjusted mortality. Notably, as shown in Figure 3, ICUs with the highest antimicrobial use at the start of the study period experienced the greatest reduction in overall use. These trends may suggest that targeted reduction in antimicrobial use is not associated with harm. At our institution, antimicrobial stewardship efforts have been coupled with the implementation of electronic sepsis screening tools that trigger rapid response by physicians. Such tools have been shown to reduce mortality.[33] This work has been part of a larger effort instituted on a national level through the Improving Pediatric Sepsis Outcomes (IPSO) initiative, which has led to a 19% reduction in sepsis related mortality.[34] The approach of rapidly identifying sepsis, tailoring therapies based on culture results, and discontinuing antimicrobials at the earliest possible time point can ideally result in substantial reductions in antimicrobial use while also leading to reductions in mortality. However, there are many potential confounders that can influence patient-centered outcomes among children receiving antibiotics during an inpatient stay. Accordingly, developing robust risk adjustment models that can be calculated using multicenter data would allow for more accurate assessments of antimicrobial stewardship initiatives when constructing process control charts. Risk adjustment has been a key component of other database analyses that have identified best practices in the management of pediatric sepsis.[27] Stewardship efforts at curbing antimicrobial use must be carefully monitored to ensure that the impact on patient outcomes is favorable.

The above trends suggest that current antimicrobial stewardship efforts have effectively reduced overall antimicrobial use without negatively impacting population-level mortality. However, contradictory findings in the PHIS+ subset analysis highlight the vulnerabilities in using administrative databases to drive antimicrobial stewardship projects.[7-9] Viewed in isolation, Figures 2 and 5 would lead to contradictory conclusions. Figure 2 suggests that antimicrobial use is not associated with risk-adjusted mortality, whereas Figure 5 suggests that it may be. The reason for these disparate findings requires further investigation. Though the PHIS+ cohort in Figure 5 represented over 45,000 pediatric encounters, only 3 years of data from 6 large academic centers are included. Analysis of a small number of centers is prone to confounding by other center-specific practices. The inverse association between antimicrobial prescribing and mortality was largely mediated by a single center. This association appears related to specific centers, rather than risk-adjustment methods because it was present when using either the IBM-Watson model or the novel PHIS+ model.

The differing findings of the main analysis (Fig. 2) and the subset analysis (Fig. 5) highlight the need for specifically dedicated quality improvement databases. Ideally, databases used for antimicrobial stewardship projects would contain validated severity of illness metrics, culture data (including antimicrobial susceptibilities), data on antimicrobial choice before and after culture results, clinical diagnosis, rationale for treatment choice, and C. difficile rates (common metric sometimes associated with antibiotic overuse).[35] Given the remarkable success of the IPSO initiative,[34] the addition of such metrics to PHIS (or similar databases) is likely to offer a powerful platform to develop multi-center QI projects with an aim to safely reduce unnecessary antimicrobial use.

Because pediatric ICUs have heterogenous case-mix and low overall mortality, risk-adjustment is necessary to accurately compare performance between centers, or to assess performance of a single center over time. While the Watson score demonstrated very good discrimination in our cohort (c statistic 0.909 [95% CI 0.907–0.911]), this was likely overestimated by the overall low incidence of pediatric ICU mortality, as demonstrated by the area under the precision recall curve of 0.313. Additionally, without additional calibration, the model overpredicted mortality in most of the cohort, as shown in Figure 1C, where the calibration belt falls below the line of equivalence.

Proprietary algorithms such as the Watson score are also hampered by the lack of freely available derivation and performance data. To build an unambiguous model, we conducted a nested subset analysis of the PHIS+ database, which contains federated EHR data, and developed a mortality prediction model with improved discrimination compared with the Watson score. Our model had a c-statistic of 0.94 in this subset, making its performance comparable to the Paediatric Index of Mortality 3 score (0.88), Pediatric Logistic Organ Dysfunction Score (0.94), and Pediatric Risk of Mortality Score IV (0.90).[16-18,31] This improvement in performance highlights the value of capturing granular EHR data into a multi-center repository.[19]

This study has important limitations. First, while consistent with prior literature, antimicrobial days of therapy per 1000 ICU days is a general metric and does not account for differences in specific antimicrobial choices.[2] The PHIS database does not include provider notes or microbiological data. Specific culture data and resistance patterns likely informed many antimicrobial decisions. Additionally, as addressed in the nested subset analysis, the Watson score is proprietary, and the addition of transparent severity of illness metrics would substantially improve future database analyses.[16-18,31]PHIS+ has limited reliability of time-stamps on laboratory values, preventing our ability to censor laboratory values from within the last few hours of life. Lastly, all institutions were in the United States, limiting the generalizability out of this region.

Antimicrobial prescribing among the pediatric ICUs in the PHIS database is highly variable, and not associated with risk-adjusted mortality as assessed by the Watson score. At the ICU level, reductions in antimicrobial use have occurred simultaneously with improvement or no change in risk-adjusted mortality over time. We demonstrate that the addition of EHR-derived laboratory data to administrative registries offers a potentially powerful platform for targeted improvement in antimicrobial prescribing by adjusting for important confounders related to mortality, an important balancing measure for inpatient antibiotic stewardship initiatives that naturally intersect with the treatment of life-threatening serious bacterial infections and sepsis. In the future, sharing open-source code to perform risk adjustment at individual sites leveraging data from multicenter registries could allow sites to calibrate models to their own data while still performing standardized risk adjustment to monitor for improvement in antibiotic use over time. These approaches would allow sites to track the impact of quality improvement initiatives while adjusting for differences in patient population that may otherwise confound the relationship between antimicrobial prescribing and patient-centered outcomes such as mortality, an approach recommended by the CDC.

DISCLOSURE

The authors have no financial interest to declare in relation to the content of this article.

ACKNOWLEDGMENTS

The authors thank Erin Weslander, PharmD, for her assistance in selecting antimicrobials of interest, and access to hospital-level antimicrobial data. This study was supported by 5T32HD040686-20 (JHP), 5K23NS104133 (AKA), and 1K23HD099331-01A1 (CH).