Application of the Improving Pediatric Sepsis Outcomes Definition for Pediatric Sepsis to Nationally Representative Emergency Department Data.

To compare encounter estimates and demographics of pediatric patients (<18 years) meeting modified Improving Pediatric Sepsis Outcomes (IPSO) criteria for sepsis to cohorts obtained using other criteria for pediatric sepsis from administrative datasets.
METHODS: We analyzed data from the National Hospital Ambulatory Medical Care Survey for 2003-2018. We report encounter estimates, demographics, and treatments among pediatric sepsis events using 3 criteria: modified IPSO criteria for sepsis, explicit criteria using diagnostic codes, and implicit severe sepsis criteria requiring the presence of infection and organ dysfunction.
RESULTS: The modified IPSO, explicit, and severe sepsis criteria estimated the yearly encounter rates as 116,200, 27,900, and 56,000 respectively. The modified IPSO sepsis criteria accounted for 0.4% of emergency department encounters, with a high proportion of patients who received antibiotics (99.2%, 95% CI 97.8%-100.0%), intravenous fluids (100.0%, 95% CI 99.9%-100.0%), and blood cultures (98.7%, 95% CI 96.9%-100.0%). The explicit cohort had lower proportions with blood cultures (60.6%, 95% CI 40.4%-80.7%) and antibiotic use (77.0%, 95% CI 63.1%-90.8%), but a high proportion admitted (84.0% 95% CI 73.4%-95.7%). The severe sepsis definition had low proportions with blood cultures (12.7%, 95% CI 6.3%-19.1%) and admission (21.1%, 95% CI 14.5%-27.8%).
CONCLUSIONS: Pediatric sepsis estimates differed based on the criteria used for cohort ascertainment. The modified IPSO sepsis criteria group had higher acuity than the severe sepsis cohort but lower acuity than the cohort identified using the explicit sepsis criteria.

INTRODUCTION

Pediatric sepsis is associated with substantial mortality[1-4] and morbidity.[5] The current definition of pediatric sepsis is based on systemic inflammatory response syndrome (SIRS) accompanied by suspected infection,[6] though this definition is currently undergoing re-evaluation.[7] The evolution of clinical sepsis definitions has paralleled the development and evolution of sepsis criteria based on administrative coding. Clinical sepsis definitions can be applicable to epidemiologic investigations[8-10] and useful for tracking clinical operations.[11] Such operational criteria have important implications for future research, quality improvement, and public health efforts.

Increasing access to data from electronic health records (EHRs) has promoted the development of operational and epidemiologic disease criteria leveraging more granular clinical information. The Children’s Hospital Association’s Improving Pediatric Sepsis Outcomes (IPSO) criteria for pediatric sepsis were designed to identify sepsis episodes based on diagnostic criteria, orders, or resuscitative measures related to sepsis management using queries against EHR databases.[12] The IPSO collaborative uses separate criteria for suspected infection, sepsis, and critical sepsis to track performance as part of a multicenter quality improvement initiative to promote better pediatric sepsis recognition, treatment, and outcomes. A recent publication describing metric development using the IPSO sepsis definitions reported high overall adherence among a multicenter collaborative in reporting a set of core variables including hospital arrival times, volume of fluid boluses and related administration times, and timing between hospital units among other measures, but overall low reporting of measures including organ dysfunction, risk scores and high-risk conditions.[13] Although the IPSO criteria are intended to leverage EHR data, many instances of EHRs deployed by both large and small vendors rely on customized relational databases without robust mapping to standardized vocabularies. Benchmarks of sepsis incidence and outcomes generated using highly specific definitions may differ substantially from epidemiological investigations leveraging alternative indicators of sepsis. Accordingly, electronically determined definitions (also known as computable phenotypes) should balance parsimony to promote generalizeability, facilitate implementation and minimize maintenance requirements while capitalizing on the specificity that can be achieved with the granularity of the EHR.

It is unknown how the IPSO criteria compare to overall estimates and patient characteristics of other commonly used sepsis definitions applied to administrative datasets, such as previously utilized definitions of severe sepsis relying on paired evidence of infection and organ dysfunction (implicit criteria),[8-10] and strategies that leverage diagnostic codes for severe sepsis and septic shock, also referred to as explicit criteria.[3,4,14] In this investigation, we modify the IPSO criteria to assess the prevalence of sepsis in national survey data from emergency departments (EDs) and compare estimated presentation rates and cohort characteristics to those generated from previously used administrative and registry dataset sepsis criteria. We hypothesized that the modified IPSO criteria would generate an estimate of patients with higher acuity illness than the other implicit severe sepsis criteria and explicit sepsis criteria.

METHODS

Data Source

We performed a cross-sectional analysis of the National Hospital Ambulatory Medical Care Survey (NHAMCS), a nationally representative cross-sectional probability sample survey of visits to EDs of nonfederal and short-stay hospitals in the United States.[15] NHAMCS is conducted annually by the Centers for Disease Control and Prevention National Center for Health Statistics. Each record (ie, “count”) is de-identified and assigned a weight equal to the inverse of its probability of being included in the sample.[16] Statistical packages utilize the weights from each count to extrapolate survey-weighted population estimates for the US ED encounters. When present across multiple years, common data elements may be combined for the purposes of increasing sample size. The extrapolation of encounter level data to population estimates in the NHAMCS and the National Ambulatory Medical Care Survey, a similar survey used for outpatient encounters, has been used in greater than 1,000 publications to date.[17] Research performed using NHAMCS is approved by National Center for Health Statistics Ethics Review Board. For this study, we used NHAMCS public-use data for the period 2003–2018. We excluded adults (≥18 years) and patients classified as dead on arrival.

Sepsis Criteria

We applied 3 criteria of sepsis toward pediatric sepsis identification. Each was modified to suit the data available within the NHAMCS dataset.

We chose time periods for each selection based on the data elements required for each criteria. We assessed diagnosis codes using International Classification of Disease, ninth and tenth revision (ICD-9 and ICD-10) codes. Diagnoses were determined using those codes assigned in the ED and during subsequent hospitalization when provided (from years 2005 onward). Any inclusion of vital sign abnormalities utilized those assessed at triage.

We used the following sepsis criteria (Table 1):

Table 1.

Sepsis Definitions

Criteria	Definition	Years	Reference(s)
Adapted IPSO sepsis criteria	One of the following sets of criteria:1. Positive sepsis screen (≥2 SIRS vital signs for sepsis) PLUS treatment with antibiotic PLUS intravenous fluids or pressor PLUS blood culture performance2. Admission to ICU PLUS treatment with antibiotic PLUS intravenous fluids or pressor PLUS blood culture performance3. Performance of blood lactate PLUS treatment with antibiotic PLUS intravenous fluids or pressor PLUS blood culture performance4. Initiation of vasopressive agent PLUS treatment with antibiotic PLUS intravenous fluids or pressor PLUS blood culture performance5. Presence of a standalone ICD-9/ICD-10 code for sepsis (Table 1, Supplemental Digital Content 1, http://links.lww.com/PQ9/A304)6. Presence of another ICD-9/ICD-10 code for sepsis PLUS intravenous fluids or pressor PLUS blood culture performance (Table 1, Supplemental Digital Content 1, http://links.lww.com/PQ9/A304)	2012–2018	Scott et al[18]Children’s Hospital Association IPSO guidelines[12]
Explicit sepsis definition	Explicit documentation of sepsis as an ICD-9/ICD-10 diagnosis code (Table 2, Supplemental Digital Content 2, http://links.lww.com/PQ9/A305)	2003–2018	Filbin et al[14]
Severe sepsis, Angus/Wang criteria	Evidence of infection based on ICD-9/ICD-10 coding (Wang, et al) or the presence of fever (≥38.0 °C) or hypothermia (<36.0 °C) andEvidence of organ dysfunction based on ICD-9/ICD-10 coding (Wang et al), procedure of endotracheal intubation, or hypotension (systolic pressure ≤ 90 mm Hg)	2003–2018	Angus et al[10]Wang et al[9]Singhal et al[8]

ICD, International Classification of Disease; ICU, intensive care unit.

Modified IPSO sepsis criteria. There are three sepsis categories outlined by IPSO: “IPSO suspected infection,” “ISPO sepsis,” and “IPSO critical sepsis.”[18] In this investigation, we developed modified sepsis criteria based on the “IPSO sepsis” criteria to align with available NHAMCS survey variables.[18] Because we aimed to compare potentially similar sepsis approaches, we did not investigate the IPSO critical sepsis criteria. Furthermore, IPSO critical sepsis criteria require more granular data than are available in the NHAMCS dataset. A positive sepsis screen was defined as a patient meeting 2 of 3 vital sign criteria for SIRS.[6] The NHAMCS dataset does not provide a method to identify the use of a sepsis order set, sepsis huddle, nor the provision and quantity of boluses, which are key elements in IPSO sepsis criteria. The NHAMCS dataset provides dichotomous information on intravenous fluid provision but no differentiation between fluid ordered for bolus or maintenance indications. All laboratory tests performed in the ED were considered irrespective of the timing of tests. NHAMCS does not provide data concerning testing or medication administration following hospital admission. Although the IPSO sepsis criteria use only ICD-10 codes, we converted these to ICD-9 codes using Generalized Equivalence Mapping to extend the study period to encompass the use of ICD-9 codes. Generalized Equivalence Mappings are published by the Centers for Disease Control and provide bidirectional translation references between ICD-9 and ICD-10 codes.[19] These are provided in Table 1, Supplemental Digital Content 1, http://links.lww.com/PQ9/A304. As lactate was only available from 2012 onward, we assessed estimates for this sepsis criterion for 2012–2018.

Explicit documentation of sepsis as an ICD-9 or ICD-10 diagnosis using 2003–2018 time frame and included those for “sepsis,” “septic shock,” and “severe sepsis.” ICD-9 diagnosis codes for this set of criteria were previously utilized by Filbin et al[14] on the NHAMCS dataset using GEMs. We converted all diagnosis codes to ICD-10 codes (Table 2, Supplemental Digital Content 2, http://links.lww.com/PQ9/A305). We utilized codes with any baseline for “sepsis” to maximize overall compatibility with the modified IPSO sepsis definition, though we attempted to also provide estimates for “severe” and “nonsevere” sepsis within these definitions.

Severe sepsis. We used severe sepsis criteria first reported by Angus et al,[10] and subsequently developed by Wang et al[9] and Singhal et al,[8] to search NHAMCS. These criteria require the presence of both organ dysfunction and evidence of infection. For encounters after 2015, we converted ICD-9 codes to ICD-10 using GEMs. We also allowed the presence of fever or hypothermia (temperature <36 °C or ≥38 °C) to be a surrogate for the existence of infection, keeping with methods described by Wang et al[9] and Singhal et al[8] for severe sepsis criteria using NHAMCS. Organ dysfunction codes were identified using the same approach as Angus et al[10] and Wang et al.[9] We additionally included the presence of hypotension and endotracheal intubation as evidence of organ dysfunction.[8,9]

Data Acquisition

For included encounters, we acquired demographics, assessment, diagnostic testing, and treatment data. Demographics included age (groups of 0–11 and 12–17 years for children and adolescents, respectively), sex, race (classified as White, Black, and other), ethnicity, insurance status (private, public, and other), location in a metropolitan status area, and encounters in a pediatric hospital. We defined pediatric hospitals as those in which ≥75% of encounters were for patients younger than 18 years of age.[20] Clinical assessment data included vital signs for temperature, heart rate, respiratory rate (classified by SIRS criteria), and systolic blood pressure (classified according to American Heart Association age-based criteria[21,22]). Diagnostic testing included blood culture, complete blood count, urine culture, urinalysis, and radiography. Treatment factors included the provision of antibiotics, vasopressor agents, intravenous fluids, the performance of endotracheal intubation, and disposition. Antibiotics were assessed using medications classified as anti-infectives. NHAMCS contains fields for procedures of intravenous fluid provision and endotracheal intubation. Disposition was classified from the NHAMCS dataset into groups of admitted/transferred, discharged, and all others.

Statistical Analysis

We reported estimates using survey-weighting procedures accounting for the NHAMCS sampling design. We applied each of our 3 sepsis criteria to the included cohort and generated estimates of presentation during the time period, and evaluated demographics, assessments, testing, and treatment factors for each group. We calculated the percentage of survey-weighted encounters meeting each criteria out of all survey-weighted pediatric ED encounters. Survey-weighted estimates derived from fewer than 30 counts or a relative standard error greater than 30% were considered unstable and flagged. Descriptive variables were provided as estimates or percentages, using 95% confidence intervals (CIs). We conducted analyses using the survey package (version 3.36)[23] in R, version 3.6.1 (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Patient Inclusion

During the entire 16-year study period, there were a survey-weighted estimated 474 million ED encounters (95% CI 434–513 million) for pediatric patients. For each sepsis criteria, overall estimates are presented in Table 2, demographics in Table 3, and diagnostic testing, treatments, and dispositions are provided in Table 4.

Table 2.

Estimates of Sepsis Derived from Each Set of Criteria

Variable	Modified IPSO Criteria	Explicit Sepsis Definition	Severe Sepsis
Years assessed	2012–2018(7 y)	2003–2018(16 y)	2003–2018(16 y)
Encounters (count)	107	97	244
Survey-weighted estimate meeting sepsis criteria (95% CI)	813,268 (477,755–1,148,780)	446,338 (254,423–638,254)	895,752 (736,321–1,055,183)
Survey-weighted estimate of all pediatric ED encounters	215,131,677 (181,006,781–249,256,573)	473,563,211 (433,778,045–513,348,377)	473,563,211 (433,778,045–513,348,377)
Percent total included* survey-weighted encounters during time period (95% CI)	0.4 (0.2–0.5)	0.1 (0.1–0.1)	0.2 (0.2–0.2)
Yearly estimate (Estimate/y)	116,200	27,900	56,000

*Included encounters were pediatric patients not classified as dead on arrival.

Table 3.

Demographics

Variable	Adapted IPSO Criteria	Explicit Sepsis Definition	Severe Sepsis
	Percent (95% CI)	Percent (95% CI)	Percent (95% CI)
Age group
Child	71.2 (48.1–94.3)	65.5 (39.5–91.6)	75.5 (67.6–83.4)
Adolescent	28.8 (5.7–51.9)	34.5 (8.4–60.5)*	24.5 (16.6–32.4)
Male sex	52.6 (35.7–69.5)	66.0 (50.1–81.8)	42.9 (35.1–50.7)
Race
White	69.2 (60.6–78.1)	75.6 (61.4–89.8)	67.3 (58.8–75.8)
Black	20.0 (9.6–30.4)*	20.6 (7.1–34.1)*	26.2 (18.4–34.0)
Other	10.6 (2.1–19.1)*	3.7 (1.1–7.4)*	6.5 (2.2–10.8)*
Hispanic ethnicity	35.4 (21.5–49.3)*	37.8 (6.3–69.3)*	15.8 (7.6–24.0)*
Payment
Public	74.0 (63.1–84.8)	60.6 (41.5–79.8)	54.0 (43.6–62.4)
Private	18.8 (8.9–28.8)*	28.1 (11.5–44.6)	31.7 (23.7–39.7)
Other	7.2 (1.2–13.2)*	11.3 (2.0–20.7)*	15.2 (8.0–22.5)
Metropolitan status area	89.4 (79.2–99.7)	93.8 (86.0–100.0)	79.2 (68.3–90.1)
Pediatric hospital	40.4 (20.2–60.7)*	40.5 (17.5–65.5)*	12.6 (6.4–18.7)

All results are presented as survey-weighted percentages.

Hispanic ethnicity available from year 2007 onward; metropolitan status area data unavailable for year 2012.

*Calculated from a low number of raw counts, which may lead to estimate instability per National Center for Health Statistics guidelines.

Table 4.

Assessments, therapy, diagnostic testing, and disposition.

Variable	Adapted IPSO Criteria	Explicit Sepsis Definition	Severe Sepsis
	Percent (95% CI)	Percent (95% CI)	Percent (95% CI)
Heart rate abnormal for age	59.8 (48.0–71.6)	33.6 (22.6–44.6)*	17.4 (9.9–24.9)
Tachypnea for age	73.8 (64.4–83.3)	50.7 (29.4–72.0)*	43.2 (34.0–52.5)
Febrile or hypothermic	53.1 (34.4–53.9)	41.7 (29.6–53.7)	50.8 (41.4–59.2)
Hypotension for age	3.0 (0.0–6.6)*	3.1 (0.0–6.9)*	78.6 (70.6–86.6)
Therapy
Antibiotic use	99.2 (97.8–100.0)	72.3 (56.9–87.7)	54.3 (45.8–62.7)
Vasopressor use	6.6 (1.6–11.6)*	4.4 (0.0–8.8)*	3.4 (0.4–6.3)*
Intravenous fluids	100.0 (99.9–100.0)	77.0 (63.1–90.8)	33.3 (22.9–43.7)
Diagnostic testing
Blood culture	98.7 (96.9–100.0)	60.6 (40.4–80.7)	12.7 (6.3–19.1)*
Complete blood count	85.2 (77.8–92.6)	60.6 (48.9–72.3)	30.2 (21.6–38.8)
Urine culture	43.1 (34.0–52.2)	38.0 (25.2–50.8)	8.4 (1.5–15.3)*
Urinalysis	69.6 (56.5–82.8)	61.0 (48.9–73.0)	23.4 (16.9–29.9)
Radiography	69.2 (54.2–84.2)	54.1 (41.0–67.1)	38.2 (28.3–48.1)
Disposition
Admit/transferred	63.3 (43.7–82.9)	84.0 (73.4–95.7)	21.1 (14.5–27.8)
Discharged	30.7 (12.2–49.3)	15.0 (4.6–25.4)	75.4 (68.0–82.7)
Other	6.0 (0.0–15.6)*	0.6 (0.0–1.9)	3.3 (0.7–5.9)*

All results are presented as survey-weighted percentages.

Blood culture data available from encounters from 2003 to 2004 and 2007 to 2018; urine culture available only from encounters from 2003 to 2004 and 2012 to 2018; radiography available only from encounters from 2005 onward.

*Calculated from a low number of raw counts, which may lead to estimate instability per National Center for Health Statistics guidelines.

Modified IPSO Sepsis

0.4% of survey-weighted encounters met these criteria (from 107 counts), amounting to 116,200 pediatric sepsis patients per year. The majority of patients (83 of 107) met modified IPSO sepsis criteria based upon a positive SIRS-based sepsis screen with the provision of blood cultures and acquisition of intravenous fluids (Table 3, Supplemental Digital Content 3, http://links.lww.com/PQ9/A306). Most encounters were less than 12 years old (71.2%, 95% CI 48.1%–94.3%), with a high proportion with public insurance (74.0%, 95% CI 63.1%–84.8%). A high proportion received blood cultures (98.7% [95% CI 96.9%–100.0%]), antibiotics (99.2%, 95% CI 97.8%–100.0%), and intravenous fluids (100.0%, 95% CI 99.9%–100.0%), and were admitted to the hospital (63.3%, 95% CI 43.7%–82.9%).

Explicit Sepsis Criteria

The explicit sepsis criteria identified the fewest patients (0.1%, or 27,900 survey-weighted encounters per year). Estimates were generated from 97 counts. These criteria identified a high proportion of patients less than 12 years of age (65.5%, 95% CI 39.5%–91.6%). In this group, 75.0% of survey-weighted encounters did not have diagnosis codes for severe sepsis (ICD-9 codes 995.92, 785.52, or ICD-10 code R65.2); however, 60.6%, (95% CI 40.4%–80.7%) received blood cultures. A minority of these patients had heart rate (33.6%, 95% CI 22.6%–44.6%) or temperature (41.7, 95% CI 29.6%–53.7%) abnormalities in triage. A high proportion of encounters meeting explicit criteria were admitted (84.0%, 95% CI 73.4%–95.7%).

Severe Sepsis

The severe sepsis criteria were identified in 0.2% of patients, or 56,000 survey-weighted encounters annually, derived from 244 counts. Blood cultures were obtained in 12.7% (95% CI 6.3%–19.1%), approximately half received antibiotics (54.3%, 95% CI 45.8%–62.7%), and 21.1%, 95% CI 14.5%–27.8%) were admitted.

DISCUSSION

We performed a cross-sectional study of a nationally representative dataset to identify estimates for sepsis in the ED using modified IPSO Sepsis clinical criteria, as well previously established approaches for identifying sepsis in administrative datasets. The modified IPSO sepsis criteria identified a high-acuity population of patients, as evaluated using vital signs, diagnostic testing, and medications provided. This study’s findings demonstrate critical challenges in generating estimates of pediatric sepsis using combinations of administrative and clinical data and underscore the need for establishing careful a priori criteria when identifying pediatric patients with sepsis.

Using a modification of the IPSO sepsis criteria, we identified 116,000 pediatric sepsis patients per year. This figure is approximately 5-fold higher than the estimated figure identified using explicit sepsis criteria and almost twice as high as the estimate using the severe sepsis criteria. In the multicenter report by Scott et al,[18] 2.6% of pediatric inpatients met IPSO sepsis criteria. Our lower figure (0.4% of pediatric patients) likely reflects the use of ED as opposed to inpatient data. Patients who later met criteria during hospitalization were not included in the present work, whereas the results reported by Scott et al[18] include ED, inpatient, and intensive care unit estimates. Furthermore, the majority of encounters within NHAMCS are not from pediatric hospitals.

The modified IPSO sepsis criteria appeared to capture higher acuity disease than the other criteria, as indicated by the use of intravenous fluids, antibiotics, and vasopressors. The IPSO sepsis criteria differ from other criteria in that it encodes the presence of sepsis based on pragmatic parameters selected by the Children’s Hospital Association IPSO collaborative to facilitate performance improvement tracking at both the institutional and collaborative levels. Accordingly, the criteria may reflect institutional practice patterns and capture some cases of possible sepsis in the context of other high-acuity disease processes, in which antibiotics are prescribed in a “rule-out” process for noninfectious, life-threatening disease such as severe trauma, cardiac arrest of unknown cause, or gastrointestinal bleeding. Furthermore, the IPSO Sepsis criteria specifically utilize locally developed sepsis screening tools, which vary between institutions.[18]

Other differences are notable when comparing the modified IPSO sepsis criteria to our implementation of severe sepsis and explicit sepsis criteria within the NHAMCS dataset. A high proportion of patients with the modified IPSO Sepsis criteria and explicit sepsis criteria were evaluated at pediatric hospitals and were publicly insured. These findings may be because sepsis occurs more frequently in medically complex patients[3] and that such patients are more frequently publicly insured[24,25] and likely visit pediatric hospitals.[25] Additionally, a higher proportion of patients with severe sepsis had vital sign abnormalities; this is unsurprising as vital sign abnormalities are included as parts of these criteria. Despite this, these groups had lower proportions of blood cultures, antibiotic use, and hospital admission

While the modified IPSO Sepsis criteria appeared to capture a subset of patients with high-acuity disease, certain limitations of our criteria are apparent when used to generate ED estimates. Notably, some encounters meeting modified IPSO sepsis criteria were discharged from the ED, even though a large proportion were given antibiotics. Conceivably, some patients meeting our modification of the IPSO criteria may be more accurately characterized as serious bacterial infections without sepsis based on currently applicable consensus criteria. These could potentially include patients with infectious conditions but without concerns for organ dysfunction or tissue hypoperfusion, such as febrile infants with urinary tract infections, or patients at high risk of sepsis based on complex chronic conditions given antibiotics pre-emptively before all criteria for sepsis are evident.[6,26] Hospital admission is not required in the reference IPSO criteria.

Our findings related to the other sepsis definitions are generally comparable to previously published literature. One prior estimate for pediatric sepsis has been reported using data from NHAMCS. That investigation, which used a modified Angus criteria, suggested a rate of 95,000 cases annually for the years 2001–2009, which is higher than reported in the present study.[8] As noted in that study, a high proportion of encounters were discharged from the ED (84%), suggesting an overall low-acuity level. Investigators using other datasets have similarly determined that the Angus criteria may identify cohorts with overall lower acuity.[27] In contrast, explicit criteria may lack sensitivity and may underestimate the incidence and prevalence of sepsis encounters. Of note, not all encounters meeting the explicit criteria of sepsis received diagnostic testing, a finding compatible with previous reports indicating a blood culture is obtained in only 70% of pediatric sepsis cases.[28] Although this finding may partially relate to care provided in other settings (such as the intensive care unit or referring hospitals), it may also represent an opportunity to improve pediatric sepsis management in the ED. It is not unexpected that the cohort characteristics varied with different ascertainment strategies. A retrospective study utilizing data of patients admitted to an intensive care unit from a single specialty children’s hospital demonstrated that the use of explicit diagnosis codes for pediatric sepsis identified a smaller cohort compared to using modified Angus (269% larger) criteria.[29] An evaluation of administrative data from 44 children’s hospitals in the Pediatric Health Information Systems over the years 2004–2012 demonstrated a 7-fold difference in pediatric patients with explicit diagnosis codes for sepsis (0.45% of all hospital admissions) versus those using a modified Angus criteria (3.1% of all hospitalizations).[3]

Our study has substantial limitations intrinsic to any work aiming to identify populations of children with sepsis in large datasets. Though NHAMCS is widely used for epidemiological research, it is subject to limitations related to data abstraction, coding, and missing data.[30,31] NHAMCS relies on the extrapolation of survey-weighted encounters to generate national estimates; although this methodology is used extensively for medical research,[17] results are presented as estimates with CIs rather than exact values. Only 107 encounters were used for the generation of the modified IPSO sepsis estimates, and 97 for explicit sepsis estimates. Additionally, there are reported discrepancies between rates of endotracheal intubation and disposition in some NHAMCS encounters.[32] Not all variables of interest were available during the study period. Given the data available within the NHAMCS dataset, our modified IPSO Sepsis criteria have significant and important inherent limitations; for example, the present work used SIRS criteria as a surrogate for a sepsis screening tool, whereas many institutions have implemented screening tools that incorporate data that include SIRS vital sign thresholds, as well as risk factors such as complex past medical histories, mental status or perfusion assessments.[33]

We were unable to adapt the IPSO critical sepsis criteria to the NHAMCS dataset as this requires more detailed data related to the timing and frequency of intravenous fluid boluses. Furthermore, such an exploration would likely be limited by low counts: when assessing the counts of encounters meeting modified IPSO sepsis criteria who were also provided with antibiotics, intravenous fluids, and pressors, only 8 encounters met all of these criteria. We used intravenous fluids for our modified IPSO sepsis criteria but were unable to distinguish bolus intravenous fluids from carrier fluids. We were also unable to obtain data on the timing of events or events that may have occurred after admission, which is not available within the NHAMCS dataset. Our vital sign assessments were limited to only those at triage. Given the challenges with sepsis criteria, we were unable to compare our findings to a reference standard. Finally, we were unable to evaluate trends in the identification or treatment of pediatric sepsis over time using various criteria or the retrieval of measures such as mortality, where counts were few. This is particularly relevant for the explicit sepsis criteria.

It is important to note that the development of the IPSO clinical sepsis criteria are a hallmark of leveraging the breadth of EHR data to ascertain cohorts of interest with much greater specificity than previously possible in large pediatric datasets. Our need to modify the IPSO sepsis criteria is also emblematic of the relative complexity of these criteria compared to the severe sepsis and explicit sepsis criteria. Indeed, there are likely challenges to adapting the IPSO criteria to some EHR instances. While recent federal requirements imposed by the 21st Century Cures Act are promoting increasing interoperability between varied EHRs, including mapping salient variables for identifying clinical sepsis to standardized terminologies such as Logical Observation Identifiers and Codes, and facilitating standardized data transfer with HL7 fast healthcare interoperability resources, the majority of current EHR instances contain data with proprietary reference codes and noninteroperable, workflow-dependent data schemas. In the 46 participating IPSO hospitals from 1 prior report, 20 (43%) relied on manual or mostly manual chart review.[18] A separate report from the IPSO collaborative noted lags in data submission at sites coinciding with the departure of site champions, a finding compatible with the need for ongoing curation and maintenance of most EHR-integrated informatics pipelines.[13] Creating flexible definitions of clinical pediatric sepsis that can adapt to datasets of varied parsimony may eventually allow for more consistent reporting of sepsis incidence and outcomes.

In this cross-sectional study of a nationally representative dataset, we identified substantial differences between ascertained cohorts using 3 different sets of criteria for identifying pediatric sepsis in administrative registries. Compared to other algorithms, the modified IPSO Sepsis criteria appeared to identify a cohort of high acuity. The IPSO clinical sepsis criteria emblemize the potential of EHR-derived definitions of pediatric sepsis and informatics-driven quality improvement. However, further work is needed to construct definitions of pediatric sepsis that can serve the needs of institutional performance improvement efforts while also being used to generate consistent benchmarks from datasets of varied parsimony.

DISCLOSURE

The authors have no financial interest to declare in relation to the content of this article.