Predicting severe pneumonia in the emergency department: a global study of the Pediatric Emergency Research Networks (PERN)-study protocol.

INTRODUCTION: Pneumonia is a frequent and costly cause of emergency department (ED) visits and hospitalisations in children. There are no evidence-based, validated tools to assist physicians in management and disposition decisions for children presenting to the ED with community-acquired pneumonia (CAP). The objective of this study is to develop a clinical prediction model to accurately stratify children with CAP who are at risk for low, moderate and severe disease across a global network of EDs. METHODS AND ANALYSIS: This study is a prospective cohort study enrolling up to 4700 children with CAP at EDs at ~80 member sites of the Pediatric Emergency Research Networks (PERN; https://pern-global.com/). We will include children aged 3 months to <14 years with a clinical diagnosis of CAP. We will exclude children with hospital admissions within 7 days prior to the study visit, hospital-acquired pneumonias or chronic complex conditions. Clinical, laboratory and imaging data from the ED visit and hospitalisations within 7 days will be collected. A follow-up telephone or text survey will be completed 7-14 days after the visit. The primary outcome is a three-tier composite of disease severity. Ordinal logistic regression, assuming a partial proportional odds specification, and recursive partitioning will be used to develop the risk stratification models. ETHICS AND DISSEMINATION: This study will result in a clinical prediction model to accurately identify risk of severe disease on presentation to the ED. Ethics approval was obtained for all sites included in the study. Cincinnati Children's Hospital Institutional Review Board (IRB) serves as the central IRB for most US sites. Informed consent will be obtained from all participants. Results will be disseminated through international conferences and peer-reviewed publications. This study overcomes limitations of prior pneumonia severity scores by allowing for broad generalisability of findings, which can be actively implemented after model development and validation. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

In this large, prospective cohort study, we will develop a clinical prediction model to identify children with community-acquired pneumonia (CAP) at risk of severe disease recruited from a global network of paediatric emergency departments.

The results of this study will allow clinicians to better identify, among children with CAP, those at greatest risk of severe complications and need for hospitalisation.

This study will be the largest global prospective study of children with CAP occurring across several continents in upper middle-income and high-income countries, providing rigorous evidence to determine prognostic factors to assist with clinical management.

Study results may not be generalisable to children with CAP in lower middle-income and low-income countries.

Introduction

Community-acquired pneumonia (CAP) is the leading cause of death in young children worldwide. With an estimated 1 million children dying each year from CAP across the globe, CAP accounts for 15% of all deaths in children younger than 5 years.1 Most estimates of the burden of CAP, specifically mortality, come from low-income and lower middle-income countries, yet most resource utilisation for paediatric CAP occurs in high-income countries.2–4 In these high-income countries, although mortality is low, CAP is one of the most common serious childhood infections and a frequent and costly cause of emergency department (ED) visits and hospitalisations.4 5

Serious complications of CAP, including empyema, respiratory failure, sepsis and death, occur in a small proportion of children in high-income countries, yet early identification of children at risk of severe disease is critical for effective management. The incidence of empyema in children with CAP has increased over the last 20 years, with estimates of up to 40% of paediatric bacterial pneumonias complicated by empyema in high-income countries globally.4 6 Although some investigators have evaluated risk factors for empyema, which include ibuprofen administration, documented bacterial infection, chest pain, elevations in acute phase reactants and longer fever duration, most studies are small, and results are varied.7–9 The lack of evidence on CAP severity is emphasised by the US10 and British11 paediatric CAP guidelines that base severity classification on expert consensus or extrapolation of adult severity scores, rather than on data from children with CAP. These scores do not discriminate well when applied to children.12 13 For example, the severity criteria proposed by the Infectious Diseases Society of America and the Pediatric Infectious Diseases Society were derived from adult CAP criteria. When applied retrospectively to 518 children presenting to the ED with CAP, more than half of the children who were safely discharged home from the ED were misclassified as having severe disease warranting continuous monitoring or intensive care by these criteria.12 Predicting disease severity and complicated CAP in children worldwide would allow for appropriate disposition decisions and rapid, intensive therapies for those at highest risk, while minimising resource use for those at low risk.

Objectives

Severity criteria for children with CAP are largely adapted from adult studies and guidelines, despite significant differences in presentation, aetiology and pathophysiology between adults and children.10 11 Prior attempts to develop prognostic models of severity in children with CAP have been limited by small sample sizes and single-centre designs, a lack of generalisability to ED and outpatient settings and missing objective data due to retrospective study designs.14–16 The purpose of this study is to develop accurate, objective models of prognosis in paediatric CAP using a global cohort of paediatric EDs. The aims of this study are: (1) to identify predictors of disease severity in children with CAP, including the need for hospitalisation, empyema, respiratory failure, sepsis and death, in a large, global cohort of EDs; and (2) to develop a clinical prediction model that would accurately identify children with CAP who are at risk for low, moderate and severe disease and assess the predictive accuracy of this model. This study represents a substantive departure from current approaches in several important ways: (1) creation of a prospectively developed clinical prediction model for paediatric CAP severity for use in the ED, the setting for most initial management and disposition decisions in high-income countries; and (2) use of a large, global cohort of EDs that will enable both enrolment of a large sample size and ensure generalisability to a worldwide population of children with CAP.

Methods and analysis

Study overview

This study is a prospective cohort study using a convenience sample of children 3 months to 14 years of age diagnosed with CAP at EDs across the globe. Enrolment started in February 2019 and is slated to end in February 2021. This study is occurring within the Pediatric Emergency Research Networks (PERN; www.pern-global.com).17 PERN is a global association of paediatric emergency care research networks, including the Pediatric Emergency Care Applied Research Network (PECARN) and the Pediatric Emergency Medicine Collaborative Research Committee (PEMCRC) of the American Academy of Pediatrics in the USA, Pediatric Emergency Research Canada (PERC), Paediatric Research in Emergency Departments International Collaborative (PREDICT) in Australia and New Zealand, Paediatric Emergency Research in the UK and Ireland (PERUKI), Research in European Pediatric Emergency Medicine (REPEM), Research Network of the Spanish Society of Pediatric Emergency/Spanish Pediatric Emergency Medicine Research Group (RISeuP/SPERG) and Network for Research and Development of Pediatric Emergency Medicine in Latin America (Red de Investigacion y Desarrollo de la Emergencia Pediatrica Latinoamericana, or RIDEPLA). Together, the research networks within PERN have access to data from more than 5 million paediatric ED presentations annually and to more than 200 hospitals, in five of the six WHO regions.

Study population

Inclusion criteria

Children 3 months up to 14 years of age with a clinical diagnosis of CAP will be included. As this study is meant to be a pragmatic examination of CAP in the ED that is broadly generalisable and applicable in daily practice, inclusion and exclusion criteria are specifically designed to be inclusive rather than restrictive. In order to address the heterogeneity of definitions for CAP that exist, we will undertake sensitivity analyses using radiologist-confirmed pneumonia. We will exclude children younger than 3 months, as the young febrile infant has a distinct diagnostic and therapeutic approach. Likewise, we will exclude those 14 years and older to allow for consistent enrolment of paediatric patients, as not all EDs around the globe see patients older than 14 years of age. To account for the fact that CAP is often diagnosed without radiographic or laboratory evaluations in different regions of the world, we are including children diagnosed with CAP by the treating clinician, regardless of radiographic or laboratory findings.

Exclusion criteria

The goal of this study is to evaluate CAP severity in generally healthy children. Therefore, we will exclude children who were hospitalised in the 7 days prior to the study ED visit, have a diagnosis of hospital-acquired pneumonia or have a chronic complex condition, defined as chronic pulmonary disease (eg, cystic fibrosis, chronic lung disease of prematurity and tracheostomy dependent), chronic congenital cardiac disease (not included functionally insignificant murmurs), immunosuppression or immunodeficiency (eg, chronic corticosteroid use and oncological process on chemotherapy), sickle cell disease and neuromuscular disorders. Children with asthma are included in the study.

Study procedures

Participant screening and consent

Patients will be screened for eligibility during designated shifts for enrolment. Screening criteria will include any patient 3 months to 14 years of age who presents with a chief complaint of fever, cough, respiratory distress, shortness of breath, ‘rule out pneumonia’ or similar complaints based on local practice of recording chief complaints. Enrolment will occur during at least four designated ED shifts each month, with a specified window to allow for site variability in processes around shift length and shift change. The days chosen at each individual site will be determined by a combination of factors including site ED volume patterns, ability to enrol on certain days of the week and other site-specific factors. Consent will be obtained from the guardians of eligible participants before enrolling in the study.

In-person screening procedures can change in response to the COVID-19 pandemic. Screening and enrolment will occur retrospectively by telephone when in-person screening is not available. Each site will identify patients seen on the prior day (up to 3 days later to account for weekends) with a documented diagnosis of CAP. When necessary, research staff may enrol patients retrospectively by telephone with verbal consent.

Data collection

Once consent is obtained, study staff will ask parents and participants to complete a brief questionnaire about the participant’s clinical history. Clinicians caring for the patient will complete a questionnaire about the participant’s physical examination and their clinical impressions. For a subset of participants, a second clinician will perform a physical examination to determine the inter-rater reliability of predictors to be used in the clinical prediction model. Additional data collected about the ED visit will include laboratory and imaging results (if performed), antibiotics prescribed and disposition decisions.

For enrolments conducted by telephone, the physical examination findings will be abstracted from the medical record by local study personnel, with oversight by a study physician. These enrolments will not undergo a separate physical examination by a second physician, but a second data abstractor will complete medical record review on a sample of patients to assess inter-rater reliability of review.

Medical records of all enrolled participants who are hospitalised will be reviewed for clinical outcome data, including interventions (eg, respiratory support), complications of CAP (eg, empyema), relevant medications given (including antibiotics, corticosteroids and vasopressors) and duration of symptoms while an inpatient.

Participant follow-up

A follow-up data collection contact (telephone call or text) will occur approximately 7–21 days after the index ED visit or hospitalisation. Follow-up data collection will only occur for children discharged from the ED or hospitalised for less than 7 days from the study ED visit. Follow-up data collected will include return to medical care, changes in antibiotics and progression of disease. Medical records will be reviewed after the study visit to note if there were any revisits to the same institution, and if so, the above outcomes that occurred during these revisits.

Potential predictor variables

Potential predictors of disease severity in paediatric CAP were selected through extensive literature review, including a published systematic review and expert consensus.18 19 Selected predictors are listed in box 1. Definitions for all predictor variables are outlined in the study manual of operations that was distributed to all sites prior to commencement of enrolment.

Age.

Biological sex.

Prematurity.

Asthma.

Pneumonia.

Smoke exposure.

Vaccination status.

Duration of illness.

Illness symptoms.

Allergies.

Antibiotic use.

Vital signs (eg, heart rate, respiratory rate, blood pressure and temperature).

Oxygen saturation.

Appearance.

Mental status.

Perfusion.

Chest auscultatory findings (eg, wheezes, rales and rhonchi).

Work of breathing (eg, chest indrawing, grunting and nasal flaring).

Chest radiograph findings.

Chest ultrasound findings.

Chest CT findings.

White cell count.

Haemoglobin.

Electrolytes.

Blood gas.

Inflammatory markers (eg, C-reactive protein and procalcitonin).

Rapid viral testing.

Outcome measures

The primary outcome will be disease severity, represented by a three-tiered composite outcome, with outcomes occurring within 7 days of the index ED visit (box 2). Mild CAP will be defined as CAP treated in the outpatient setting. Moderate CAP will be defined as children requiring hospitalisation at any time within 7 days of the index ED visit but not having an outcome that is part of the definition of severe CAP. Severe CAP will be defined as CAP with the development of empyema or effusion requiring drainage procedures, intensive care unit admission more than 48 hours in duration, respiratory failure requiring positive pressure ventilation (invasive or non-invasive, including high-flow nasal cannula if used to treat distress due to CAP), septic shock, receipt of vasoactive infusions, receipt of extracorporeal membrane oxygenation or death. As the primary objective is the development of a predictive model, all outcomes included will be those that occur after the first 4 hours of management in the ED, including as an inpatient. Because predictive models aim to identify patients for whom an outcome is likely to occur, rather than to identify outcomes that are already present, we will exclude patients who meet the ‘severe CAP’ definition within 4 hours of presentation.

Community-acquired pneumonia (CAP) treated in the outpatient setting.

CAP requiring hospitalisation at any time within 7 days of the index emergency department visit but not having an outcome that is part of the definition of severe CAP.

Severe

CAP with:

Empyema or effusion requiring drainage procedures.

Intensive care unit admission >48 hours in duration.

Respiratory failure requiring positive pressure ventilation (invasive or non-invasive, including high-flow nasal cannula if used to treat distress due to CAP).

Septic shock.

Receipt of vasoactive infusions.

Receipt of extracorporeal membrane oxygenation.

Death.

Secondary outcomes will include variation in the management of CAP, assessed by rates of diagnostic testing, hospitalisation and antibiotic use, accounting for disease severity and global region. We will also assess antibiotic comparative effectiveness among antibiotic classes (β-lactams vs cephalosporins vs macrolides). The outcomes of interest will include duration of symptoms, hospital length of stay, development of CAP-associated complications and return to medical care after discharge (after adjustment or propensity analysis).

Data analysis

Sample size and power

We determined a sample size that would provide a sufficiently large number of children with the most severe CAP to ensure ample power. The primary objective to be met with this sample size is to allow our clinical prediction model to provide a clinically useful improvement in discriminative capacity for distinguishing children with the most severe CAP from the others. Based on prior data, we conservatively assume that 3% of the sample would be classified as severe in our three-level outcome.20 We also assumed that a clinically useful improvement in discriminative capacity would correspond to an area under the receiver operating characteristic curve (AUROC) for the severe versus low/moderate classification of 0.725. We will test the null hypothesis that the AUROC is 0.6 using a two-sided test with alpha of 0.05. According to the SAS ROCPOWER macro, an effective sample size of 2500 is required to provide at least 95% power for a model with 1 df, under the stated assumptions. To account for the consideration of numerous candidate predictors in model development, we derived a sample inflation factor of 1.88 by computing the ratio of the χ2 non-centrality parameters corresponding to 95% power, two-sided alpha 5%, with 10 and 1 df, respectively. Hence, we prescribe a minimum target sample size of 4700 (ie, 2500×1.88). Such a sample size would yield an estimated total of 141 children with severe CAP, which would permit estimating the sensitivity of a binary test for detecting the most severe cases with suitable precision such that the 95% CIs for this sensitivity parameter would be no wider than 17 percentage points. Prior to site engagement, sites were queried regarding their ability to enrol at least 50 children with CAP per year, which based on the number of EDs participating in PERN, would meet our sample size estimates.

Statistical analysis plan

The clinical risk prediction model for the three-level outcome will use ordinal logistic regression models, assuming a partial proportional odds specification. Essentially, the proportional odds specification amounts to modelling two separate dichotomisations of the three-level model: (1) low versus (moderate or severe) and (2) (low or moderate) versus severe—under the assumption that the multiplicative effects of individual predictors on the ORs are similar for both dichotomisations. Furthermore, the partial proportional odds specification allows some predictors to have separate multiplicative effects for the two dichotomisations.21 Model development will seek to achieve a clinical prediction model that provides high discrimination capacity and acceptable calibration in external samples, using principled approaches to guard against overfitting biases, the tendency of estimated models to find idiosyncratic patterns in the training sample that are not seen elsewhere. Discrimination capacity refers to the ability of risk predictions to sort patients according to their actual risks, so that persons with higher levels of severity are more likely to ‘outscore’ (ie, have higher risk predictions) than the persons who do not.22 Calibration, however, describes how closely the predicted risks match actual risks on an absolute basis. Model development will proceed systematically through Steyerberg’s checklist of steps, with appropriate adaptations for the ordinal logistic regression model.23 Our study will result in observations from a large number of children with CAP and is also well suited for exploratory analyses using novel statistical learning theory techniques.

As there is no universally accepted definition of CAP, we will examine children using a sensitive definition and additional definitions with increasing specificity. To promote recruitment across the broad spectrum of potential participants, the sensitive definition will be used for enrolment and will be the basis for the main analysis, but we will include all definitions in separate subanalyses. The following definitions will be considered, in order of most sensitive to most specific: (1) provider-diagnosed pneumonia (primary analysis): a clinical diagnosis of CAP, regardless of other factors; (2) radiographic pneumonia: a clinical diagnosis plus radiologist/ED physician diagnosed consolidation, empyema or parenchymal infiltrate on chest radiograph; and (3) definite pneumonia: having all four of the following: provider diagnosis of pneumonia, history of fever (≥38.0°C), evidence of respiratory illness by history or physical examination and chest radiograph suggestive of pneumonia (eg, infiltrate or consolidation).

Ethics and dissemination

This study poses minimal risk to participating children and their families. Ethics approval has been obtained at all participating sites. Cincinnati Children’s Hospital Medical Center Institutional Review Board (IRB) is serving as the central IRB for most US sites. Ethics approval for other sites occurred locally. Patients will receive standard care in ED. Participation in the study will not negatively impact or restrict care in the ED or hospital. A small potential risk exists around disclosure of confidential information. The analytical clinical database will contain no patient identifiers and will fulfil the definition of a deidentified dataset as defined by the Health Insurance Portability and Accountability Act in the USA. This analytical database will be the only one available for the analysis of the current and future derivative studies. Local sites will store identifying data necessary for follow-up contact and subject tracking in a password-protected spreadsheet file that will be maintained locally and not transmitted to any other site or investigator. All patients and families will provide written or verbal informed consent/assent and will have the ability to withdraw at any time without explanation. Results will be disseminated at international conferences and through peer-reviewed research publications.

Limitations

We anticipate several limitations of this study. First, this study is being performed in middle-upper income and high-income countries; thus, study results may not be generalisable to lower income nations. Second, given the scope of this study, it is challenging to enrol 24 hours per day, 7 days per week. We thus are enrolling a convenience sample of children with CAP when investigators and research staff are available. However, sites are predetermining enrolling shifts in order to track eligible patients who may be missed. Enrolled patients will be compared with missed patients to evaluate for selection bias. Finally, we are not requiring radiographic confirmation of CAP. Although children without radiographic CAP may be included, use of clinical diagnosis is consistent with actual practice in many PERN sites, allowing for the results of this study to be broadly generalisable to EDs where radiographs are not routinely obtained. We will perform sensitivity analyses to account for children with clinically diagnosed CAP, radiographic CAP and definitive CAP using definitions outlined in the previous section.

Conclusions

This study will enhance clinical care and future research by moving from use of subjective impressions that do not adequately predict outcomes in children with CAP towards an evidence-based approach. Accurate, globally derived objective models of prognosis in paediatric CAP will help physicians assess patients’ risks and improve resource allocation, hospitalisation and disposition decisions. In addition, this work has the potential to overcome previous barriers and limitations in understanding and predicting CAP severity and result in a new evidence-based, precision-oriented pathway of clinical care and research for childhood CAP.

Patient and public involvement

This research was planned without patient involvement. Patients were not invited to comment on the study design and were not consulted to develop patient relevant outcomes or interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy. We anticipate that parents and patients will be play an essential role in the implementation of the clinical prediction models that are developed as part of this study.