PEMCRC anaphylaxis study protocol: a multicentre cohort study to derive and validate clinical decision models for the emergency department management of children with anaphylaxis.

INTRODUCTION: There remain significant knowledge gaps about the management and outcomes of children with anaphylaxis. These gaps have led to practice variation regarding decisions to hospitalise children and length of observation periods following treatment with epinephrine. The objectives of this multicentre study are to (1) determine the prevalence of and risk factors for severe, persistent, refractory and biphasic anaphylaxis, as well as persistent and biphasic non-anaphylactic reactions; (2) derive and validate prediction models for emergency department (ED) discharge; and (3) determine data-driven lengths of ED and inpatient observation prior to discharge to home based on initial reaction severity. METHODS AND ANALYSIS: The study is being conducted through the Pediatric Emergency Medicine Collaborative Research Committee (PEMCRC). Children 6 months to less than 18 years of age presenting to 30 participating EDs for anaphylaxis from October 2015 to December 2019 will be eligible. The primary outcomes for each objective are (1) severe, persistent, refractory or biphasic anaphylaxis, as well as persistent or biphasic non-anaphylactic reactions; (2) safe ED discharge, defined as no receipt of acute anaphylaxis medications or hypotension beyond 4 hours from first administered dose of epinephrine; and (3) time from first to last administered dose of epinephrine and vasopressor cessation. Analyses for each objective include (1) descriptive statistics to estimate prevalence and generalised estimating equations that will be used to investigate risk factors for anaphylaxis outcomes, (2) least absolute shrinkage and selection operator regression and binary recursive partitioning to derive and validate prediction models of children who may be candidates for safe ED discharge, and (3) Kaplan-Meier analyses to assess timing from first to last epinephrine doses and vasopressor cessation based on initial reaction severity. ETHICS AND DISSEMINATION: All sites will obtain institutional review board approval; results will be published in peer-reviewed journals and disseminated via traditional and social media, blogs and online education platforms. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Largest multicentre cohort of children with anaphylaxis conducted to date.

Addresses important clinical questions including the prevalence of severe, persistent, refractory and biphasic anaphylaxis (as well as persistent and biphasic non-anaphylactic reactions), criteria for safe emergency department (ED) discharge and data-driven ED and inpatient observation periods.

The study is feasible, as similar methodology has already been successfully applied by the primary investigator in a single-centre cohort.

As with any retrospective chart review, there is potential for missing or inaccurate documentation.

These limitations will be mitigated by providing strict definitions and instructions regarding data extraction and interpretation, especially regarding subjective or time-sensitive data points.

Introduction

From 2008 to 2016, emergency department (ED) visits for children with anaphylaxis more than tripled.1 Most anaphylaxis episodes result from food exposures, which account for direct medical costs of $4.3 billion dollars annually in the US healthcare system, including anaphylaxis-related ED visits and hospitalisations.2 Families of children with allergies are burdened with the cost of missed work ($2838 annually per child), out-of-pocket medical expenses ($1405 annually per child), obtaining prescriptions for home and school, finding safe childcare and allergen-free foods and stress related to the risk of allergen exposures. Families also bear the burden of ED visits, hospitalisations and the risk of having severe, persistent or biphasic anaphylactic reactions.2 3

Children with anaphylaxis are frequently hospitalised due to concerns for persistent symptoms or to monitor for biphasic reactions.4 Recent studies report significant knowledge gaps regarding the incidence of severe, persistent and biphasic reactions (1%–15%)5–8 and potential predictors of such reactions.5 6 9 10 Furthermore, most prior studies are single centre and restricted to adults, which limit generalisability and applicability to children.7 11 12 Finally, there are no widely accepted evidence-based guidelines to support clinical decision-making following initial treatment with epinephrine.4 13 This likely contributes to wide practice variation regarding ED observation periods (reported from 4 to 24 hours) and hospitalisation rates (varying from 12% to 95% of children with anaphylaxis).7 14

We propose a large, multicentre study conducted in a network of North American EDs to achieve the following objectives: (1) determine the prevalence of and risk factors for severe, persistent, refractory and biphasic anaphylaxis, as well as persistent and biphasic non-anaphylactic reactions; (2) derive and validate prediction models of children who may be candidates for ED discharge rather than hospitalisation; and (3) determine data-driven lengths of ED and inpatient observation prior to discharge to home based on initial reaction severity. These data have the potential to inform and standardise ED clinical practice, reduce hospitalisation rates, optimise ED and inpatient lengths of stay and mitigate the burden on children, families and healthcare systems.

Methods and analysis

Overview

This is a multicentre, retrospective cohort study of children 6 months to less than 18 years of age presenting to 30 participating North American EDs from October 2015 to December 2019. The study was endorsed by and is being conducted through the PEMCRC of the American Academy of Pediatrics, a voluntary network of over 60 institutions in the USA and Canada. Cincinnati Children’s Hospital Medical Center (CCHMC) will serve as the data coordinating centre.

Patient and public involvement

No patient involved: this research was conducted without patient involvement; patients were not involved in study design or manuscript preparation.

Inclusion criteria

Anaphylaxis diagnosis:

Encounters that fulfil standard National Institute of Allergy and Infectious Disease/Food Allergy and Anaphylaxis Network (NIAID/FAAN) anaphylaxis criteria (box 1)15 based on documentation of symptoms from the pre-ED and ED settings.

ED encounters for allergic reactions that do not fulfil NIAID/FAAN anaphylaxis criteria but for which patients receive epinephrine (intravenous, intramuscular and subcutaneous (subQ)) in the pre-ED or ED settings. We are including these patients because administration of epinephrine may mitigate initial reaction severity and progression to anaphylaxis criteria.

Acute onset of an illness (minutes to several hours) with involvement of the skin, mucosal tissue or both (eg, generalised hives, pruritus or flushing and swollen lips-tongue-uvula) and at least one of the following:

Respiratory compromise (eg, dyspnoea, wheeze-bronchospasm, stridor, reduced peak expiratory flow and hypoxemia).

Reduced blood pressure (BP) or associated symptoms of end-organ dysfunction (eg, hypotonia (collapse), syncope and incontinence).

Two or more of the following that occur rapidly after exposure to a likely allergen for that patient (minutes to several hours):

Involvement of the skin-mucosal tissue (eg, generalised hives, itch-flush and swollen lips-tongue-uvula).

Respiratory compromise (eg, dyspnoea, wheeze-bronchospasm, stridor, reduced peak expiratory flow and hypoxemia).

Reduced blood pressure or associated symptoms (eg, hypotonia (collapse), syncope and incontinence).

Persistent gastrointestinal symptoms (eg, crampy abdominal pain and vomiting).

Reduced BP after exposure to known allergen for that patient (minutes to several hours):

Infants and children: low systolic BP (age specific) or greater than 30% decrease in systolic BP†.

Adults: systolic BP of <90 mm Hg or greater than 30% decrease from that person’s baseline.

*2006 NIAID/FAAN clinical criteria for diagnosing anaphylaxis. Used with permission.

†Low systolic blood pressure for children is defined as <70 mm Hg from 1 month to 1 year, <70 mm Hg + 2×age from 1 to 10 years and <90 mm Hg from 11 to 17 years.

NIAID/FAAN, National Institute of Allergy and Infectious Disease/Food Allergy and Anaphylaxis Network.

No receipt of epinephrine (intravenous, intramuscular and subQ) in the pre-ED or ED settings: although patients may have anaphylaxis symptoms that resolve by the time of ED arrival or there may be under-recognition or underdocumentation of anaphylaxis by the treating clinician, receipt of epinephrine is an objective and well-documented data point. We also seek to identify more severe cases for which there is no clinical uncertainty.

Transfer from another healthcare facility: it is difficult to accurately assess symptoms present at the time of initial outside healthcare facility evaluation. For the predictive models, we plan to include initial ED symptoms, and thus, it is important to ensure accuracy of data elements.

Anaphylactic reactions secondary to ED treatments: for anaphylactic reactions secondary to ED treatments (eg, anaphylaxis from ceftriaxone given to treat pneumonia), it is difficult to assess whether symptoms as well as disposition decisions are related to the primary diagnosis or anaphylaxis:

Significant comorbidities including hereditary angioedema, mast cell activation disorders and patients with tracheostomies or requiring home positive pressure ventilation: patients with these conditions may require specific management and disposition decisions and are thus not generalisable to the larger population of children with anaphylaxis.

Multiple ED encounters for anaphylaxis: for patients with multiple ED encounters for anaphylaxis, only data from the most recent ED encounter will be included in the data analysis to avoid including data from the same patient when deriving and validating the prediction model for safe ED discharge (objective 2).

Case ascertainment

The electronic medical record (EMR) will be queried for ED International Classification of Diseases, Tenth Revision (ICD-10) discharge codes and all subcodes starting from 2019 and then working backwards until reaching the target number of records as outlined in the sample size section of this protocol. We used ICD-10 codes for case ascertainment given they were adopted in the USA in 2015 and were used in clinical practice through the conclusion of the study period (December 2019).16 The following codes will be queried: ‘anaphylactic shock due to shell fish (crustaceans)’ (T78.01), ‘anaphylaxis to drug’ (T88.6), ‘anaphylaxis to serum’ (T80.5), ‘anaphylactic shock due to other fish’ (T78.02), ‘anaphylactic shock due to fruits and vegetables’ (T78.03), ‘anaphylactic shock due to tree nuts and seeds’ (T78.04), ‘anaphylactic shock due to milk and dairy products’ (T78.06), ‘anaphylactic shock due to eggs’ (T78.07), ‘anaphylactic shock due to other food products’ (T78.08), ‘anaphylactic shock due to unspecified food products’ (T78.09) and ‘anaphylactic shock, unspecified’ (T78.2) and subcodes (eg, T78.2XXA).5 Although anaphylaxis encounters for which there are no ICD anaphylaxis codes may be omitted, it is not feasible to perform manual reviews of encounters with non-anaphylactic diagnostic codes given it would triple the number of encounters to review5 and because our findings are not intended to describe population-level epidemiological trends in the prevalence of anaphylaxis-related ED visits.

Data abstraction

Trained coinvestigators will abstract variables from the EMR to a secure, Health Insurance Portability and Accountability Act (HIPAA)-compliant REDCap database hosted at CCHMC. Investigators will review each encounter to assess whether it fulfils inclusion criteria and does not collect any other data for encounters that do not fulfil these criteria or meet exclusion criteria. Variables include demographics, medical history (anaphylaxis, asthma and non-atopic comorbid conditions (eg, congenital heart disease)), time and type of allergen exposure and examination findings from the pre-ED and ED settings. Symptoms will be categorised by organ system involvement in accordance with NIAID/FAAN criteria (box 1).15 Additional variables include therapies received in the pre-ED, ED and inpatient settings such inhaled beta agonists, epinephrine (intramuscular, intravenous and subQ), racemic epinephrine, intravenous fluid boluses, vasopressors (dopamine, epinephrine, norepinephrine and vasopressin), positive pressure ventilation (high-flow nasal cannula, continuous positive airway pressure, bilevel positive airway pressure and intubation/mechanical ventilation), defibrillation/cardioversion, cardiopulmonary resuscitation (CPR) and extracorporeal membrane oxygenation (ECMO).

Demographic information, ED, inpatient admission and discharge times and vital signs will be abstracted through an automated query of the EMR or manually at sites that cannot feasibly perform the query. Each site will then manually review 10% of encounters to ensure accuracy of these data elements. Medical history, timing of allergen exposure and symptom onset, administered therapies and symptoms/exam findings (from the pre-ED, ED and inpatient settings) will be obtained through manual review of documentation in the medication administration record, ED and admission and discharge summary notes. The same process will be conducted for ED revisits within 72 hours of the index ED visit. Historical and physical examination findings, as well as pre-ED treatments, will be categorised as present/received or absent/not received based on documentation in the EMR. Variables not recorded will be categorised as not documented or missing based on criteria in the manual of operations (MOO).

Compliance

All data abstractors will be trained, and a detailed MOO will be available at each site. Teleconferences will be organised to discuss study progress and answer questions during the chart review process. The REDCap database will be created using best practices (data validation and minimisation of missing data) to accurately capture data. Routine data monitoring and compliance checks of the database will be performed to ensure data accuracy throughout the conduct of the study.

Outcome definitions

Outcome definitions were developed by group consensus among the authors, based on commonly cited definitions in the literature,5 17 18 biological plausibility and the feasibility/reliability of extracting data elements from the EMR to evaluate for the presence/absence of the outcomes. Data abstractors will not categorise patients as having anaphylaxis or specific anaphylaxis outcomes (severe, persistent or biphasic anaphylaxis, as well as persistent or biphasic non-anaphylactic reactions) other than that of refractory anaphylaxis; instead, they will be responsible for extracting data elements from the EMR (symptoms and the clinical course following epinephrine administration), which will be used to generate the composite outcome definitions described below.

Severe anaphylaxis: presence of at least one of the following two criteria:

Hypotension, shock, respiratory distress, respiratory failure, life-threatening arrhythmias, coma or death.

Receipt of any of the following therapies in the pre-ED, ED or inpatient settings: ≥3 doses of epinephrine (intravenous, intramuscular and subQ), ≥3 fluid boluses, vasopressors, antiarrhythmics, every 1 or 2 hours inhaled beta agonists, continuous albuterol, magnesium sulfate, intravenous terbutaline, positive pressure ventilation and life-saving interventions (tracheostomy, CPR, defibrillation/cardioversion and ECMO).

Persistent anaphylaxis18:

Persistent symptoms/examination findings after treatment with epinephrine (intravenous, intramuscular and subQ) that fulfil the 2006 NIAID/FAAN anaphylaxis criteria.15

Following treatment with each epinephrine dose, persistent symptoms will be assessed by asking, ‘Did symptoms completely resolve after treatment with epinephrine?’ Symptoms will be categorised as persistent if this variable is marked as ‘no,’ and all persistent symptoms will be recorded. Although the definition of persistent anaphylaxis would ideally include the duration of persistent symptoms, this time interval cannot be reliably assessed in the EMR.

Refractory anaphylaxis18: presence of bothof the following two criteria:

Presence of anaphylaxis following appropriate epinephrine dosing and symptom-directed medical management (eg, intravenous fluid bolus for hypotension and inhaled beta agonists for wheezing).

The initial reaction must be treated with three or more appropriate doses of epinephrine or initiation of an intravenous epinephrine infusion.

Reviewers will be asked to determine whether patients had refractory anaphylaxis at any point in the course of the reaction (no, yes and inconclusive), including in the pre-ED, ED or inpatient settings (among hospitalised patients), as well as for patients who have ED revisits.

Biphasic anaphylaxis18: presence of both of the following criteria:

Recurrent or new allergic symptoms after initial symptom resolution without allergen re-exposure.

Recurrent or new symptoms must fulfil the 2006 NIAID/FAAN anaphylaxis criteria and occur within 48 hours from initial symptom onset (eg, symptoms that occur beyond 48 hours will not be categorised as biphasic anaphylaxis).15

Symptoms will be categorised as resolved if this variable is marked as ‘yes’ or ‘not documented’. Although the definition of biphasic anaphylaxis would ideally include a minimum asymptomatic period, these time intervals cannot be reliably assessed in the EMR. To determine whether children had recurrent/new symptoms, we will ask, ‘Did the patient have recurrent or new symptoms after treatment with epinephrine?’ and all recurrent symptoms will be recorded. The upper time threshold for recurrent/new symptoms (48 hours) will be conditional on the onset, instead of resolution, of initial symptoms given the timing of the former is not reliably documented in the EMR. ED triage will serve as a proxy for symptom onset if the timing of which is not documented. Given patients transferred from outside healthcare facilities are excluded from the study, the time from symptom onset to ED triage is relatively brief and unlikely to change the outcome.

Biphasic and persistent non-anaphylactic reactions18: The only difference between the definitions of biphasic and persistent non-anaphylactic reactions and biphasic and persistent anaphylaxis is that the former definitions do not fulfil NIAID/FAAN anaphylaxis criteria.15

Patients who do not fulfil severe, persistent, refractory or biphasic anaphylaxis criteria, nor persistent or biphasic non-anaphylaxis criteria, will be categorised as having anaphylaxis not otherwise specified.18

Objective 2: safe ED discharge

Safe ED discharge is defined as no receipt of acute anaphylaxis medications and no hypotension beyond 4 hours from first administered dose of epinephrine (intravenous, intramuscular and subQ). Patients may have the outcome for safe ED discharge despite fulfilling criteria for severe, persistent or biphasic anaphylaxis. For example, a child may have persistent or biphasic anaphylaxis following first administered dose of epinephrine but not receive acute anaphylaxis medications (or have hypotension) beyond 4 hours from the initial epinephrine dose. Four hours was used as the time interval to evaluate for safe ED discharge based on guidelines that recommend observation periods of 4 hours to monitor for biphasic anaphylaxis.8

Acute medications are defined as epinephrine, vasopressors, antiarrhythmics, racemic epinephrine, receipt of inhaled beta agonists every 1–2 hours, continuous albuterol, magnesium sulfate or terbutaline. We did not include therapies such as non-invasive ventilation, intubation or CPR as the timing of these interventions is not reliably documented in the EMR and because it would be exceedingly rare for patients to receive these therapies without first receiving an acute medication. We did not include fluid boluses or receipt of inhaled beta agonists ≥every 4 hours as acute medications given there may be subjectivity around use of these therapies (eg, fluid bolus given for tachycardia without cardiovascular compromise). To account for the potential of children receiving medications such as epinephrine for mild, non-life-threatening symptoms (eg, subjective throat discomfort), we will record the severity of persistent and recurrent symptoms documented in the EMR following all epinephrine doses.

Objective 3: Time to last dose of epinephrine and vasopressor cessation following initial epinephrine dose.

These data will inform clinicians on data-driven observation periods in the ED and inpatient setting within predefined clinical subgroups (see the Predictors and subgroups section).

Secondary outcomes

Secondary outcomes include (1) ED and inpatient lengths of stay, (2) admission rates, (3) ED revisits related to the index encounter within 72 hours of ED or inpatient discharge, (4) the frequency of use of adjunctive anaphylaxis medications (antihistamines, antacids and corticosteroids) and (5) the frequency at which providers prescribe epinephrine auto-injectors and adjunctive medications and refer patients to allergists at the time of ED/inpatient discharge.

Predictors and subgroups

Candidate predictors by objective are outlined in table 1. Predictors were developed based on group consensus; biological plausibility; and, if applicable, previously cited risk factors for the outcome of interest. For each predictor, we sought to ensure data elements could be reliably and accurately extracted from the EMR. We will not include anaphylaxis biomarkers (eg, tryptase) as candidate predictors given they are not routinely obtained in clinical practice in the USA and Canada.8

Table 1

Candidate predictors stratified by objective, outcome and pre-ED epinephrine (intravenous, intramuscular and subQ)

	Predictors
	Demographics*	Med hx†	Allergen‡	Pre-ED sx§	Pre-ED intramuscular epinephrine¶	ED vitals**		Initial ED sx§	ED intramuscular epi¶
						Initial	>Initial
Objective 1
Severe
+Pre-ED epi	x	x	x
−Pre-ED epi	x	x	x
Persistent††
+Pre-ED epi	x	x	x	x
−Pre-ED epi	x	x	x	x		x		x
Biphasic††
+Pre-ED epi	x	x	x	x
−Pre-ED epi	x	x	x	x		x		x
Objective 2
Acute meds
+Pre-ED epi	x	x	x	x	x	x	x	x	x
−Pre-ED epi	x	x	x	x	x	x	x	x	x
Objective 3
Repeat epi
+Pre-ED epi	x	x	x	x
−Pre-ED epi	x	x	x	x		x		x

*Age and gender.

†Medical history: asthma, atopic dermatitis and anaphylaxis (including prior anaphylaxis to allergen or severe anaphylaxis, ICU admission, vasopressors, positive pressure ventilation and intubation).

‡Whether the allergen was unknown or known (food (peanut, tree nut, dairy and eggs), medication, (venom) and timing of allergen ingestion to symptom onset).

§Pre-ED symptoms/examination findings: designated by organ system involvement (dermatological, mucosal, respiratory, cardiovascular and gastrointestinal).

¶Epinephrine (intravenous, intramuscular and subQ) doses (none, one, two, three or more).

**ED vitals (initial=first vital signs obtained in the ED; >initial=all vital signs obtained after the initial vital signs): hypoxia (SpO2 ≤92%), wide pulse pressure (diastolic blood pressure ≤half of systolic blood pressure) and hypotension (low systolic blood pressure for children is defined as <70 mm Hg from 1 month to 1 year, <70 mm Hg + 2×age from 1 to 10 years and <90 mm Hg from 11 to 17 years).

††The same candidate predictors for the outcomes of persistent, refractory and biphasic anaphylaxis will be used to evaluate the outcomes of persistent and biphasic non-anaphylactic reactions.

ED, emergency department; epi, epinephrine; hx, history; subQ, subcutaneous; sx, signs/symptoms.

Objective 1: Predictors of severe, persistent, refractory and biphasic anaphylaxis, as well as persistent and biphasic non-anaphylactic reactions. We will only include variables present prior to first administered dose of epinephrine. Thus, we will not include symptoms present at the time of initial ED evaluation for patients who received pre-ED epinephrine.

Objective 2: Safe ED discharge. Given the outcome is time dependent (not receiving acute anaphylaxis medications and no hypotension beyond 4 hours from first administered dose of epinephrine), symptoms present after initial ED evaluation will not be included as predictors given there is unreliable documentation as to the timing of these variables. However, we will include the number of administered doses of epinephrine (intravenous, intramuscular and subQ) received within 4 hours of the first administered dose of epinephrine because this is an objective and frequently used criterion for admission. We will not include other medications because there may be an association between symptoms and therapies (eg, wheezing and receipt of albuterol). Additionally, we will not include intravenous fluid boluses as a predictor because there is often subjectivity around administration and because wide pulse pressure and hypotension will be included as predictors, and we feel they are more specific markers of cardiovascular involvement.

Objective 3: Time to last dose of epinephrine and vasopressor cessation following initial epinephrine dose. Observation periods likely do not need to be uniform for all children with anaphylaxis given patients with less severe reactions may not need to be observed as long as those with more severe reactions. Thus, we will assess the timing from initial epinephrine administration to receipt of last dose of epinephrine and vasopressor cessation, which may be a proxy for observation periods, by clinical subgroups based on initial reaction severity. Anaphylaxis severity will be categorised into three subgroups based on symptoms at the time of initial ED evaluation: (1) absence of hypotension and respiratory involvement, (2) respiratory involvement without hypotension and (3) presence of hypotension.15

Analyses

Objective 1: Prevalence of severe, persistent, refractory and biphasic anaphylaxis, as well as persistent and biphasic non-anaphylactic reactions. We will estimate the prevalence of severe, persistent, refractory and biphasic anaphylaxis, as well as persistent and biphasic non-anaphylactic reactions using proportions with 95% confidence intervals. Similar analyses will be performed for hospitalisation rate and rate of 72-hour ED revisits. To assess risk factors and their association with these outcomes, we will use GEEs to model each of the outcomes, on the risk factors in table 1, accounting for the between-hospital variations. The quasi-likelihood under the independence model criterion will be used for model selection.19

Objective 2: Safe ED discharge. We will derive and validate a predictive model and identify patients who do not need to be hospitalised, that is, at very low risk of receiving acute anaphylaxis medications or being hypotensive beyond 4 hours from first administered dose of epinephrine. The database will be randomly divided into a derivation set comprising 50% of encounters and a validation set including the remaining 50% encounters. The LASSO regression will be used to reduce the number of candidate predictors and build the predictive model.20–22 The LASSO method improves prediction accuracy and performs covariate selection to make the model more interpretable, by forcing the sum of the absolute values of the regression coefficients to be less than a fixed value, and, therefore, forces certain coefficients to be set to zero. This is achieved by penalising models that have more and larger coefficients. The penalty parameter will be chosen by 10-fold cross-validation. We will assess model ability to discriminate between those who received acute medications beyond 4 hours from first administered dose of epinephrine and those who did not use the area under the receiver operating characteristic (AUROC), using the validation dataset. A model with AUROC between 0.7 and 0.8 indicates fair model discrimination, and above 0.8 implies good model discrimination.23 Model calibration will be assessed using the Hosmer-Lemeshow test, which assesses whether the observed rates of the outcome match the expected rate in subgroups at the deciles of fitted risk values.24 We will aim to maximise the sensitivity and negative predictive value of the model to avoid miscategorising patients who receive acute medications or are hypotensive beyond 4 hours from first administered dose of epinephrine.

We will also use classification and regression tree (CART) analysis to develop models to predict patients who are at very low risk of receiving acute medications or being hypotensive beyond 4 hours from first administered dose of epinephrine. The CART analysis involves the segregation of different values of classification variables through a decision tree composed of progressive binary split, and the optimal split is selected based on the complexity parameter that corresponds with the lowest cross-validation error. As a result, CART analysis produces decision trees that are simple to interpret and readily used for risk stratification.25 26

Objective 3: Time to last dose of epinephrine and vasopressor cessation following initial epinephrine dose. We will assess (1) timing from first to second administered doses of epinephrine, as well as (2) timing from first to last administered doses of epinephrine and cessation of vasopressors depicted using the KM method in each subgroup: (1) absence of hypotension and respiratory involvement, (2) respiratory involvement without hypotension and (3) presence of hypotension. We will perform log-rank test to test whether the KM curves are different based on the severity of initial reaction. We will evaluate the cumulative risk of receiving the last dose of epinephrine and vasopressor cessation at different time intervals within the first 24 hours from first administered dose of epinephrine. The optimal length of ED and inpatient observation will be determined as <2% increases in cumulative risk of receiving additional epinephrine (intravenous, intramuscular and subQ) as the observation time increases by 1 hour.

Statistical analyses will be conducted using SAS V.9.4 (SAS Institute); the ‘rpart’ package in R will be used to fit the CART model.25

Data validity and reliability

Key variables for candidate predictors and outcomes

To minimise the potential for bias when abstracting variables for candidate predictors and outcomes, we will have a second abstractor at each institution perform an independent review of a random 10% of encounters. We will assess the inter-rater reliability expressed as the unweighted Cohen’s k for the following data elements: allergen, timing of allergen exposure and symptom onset, presence/absence of symptoms/examination findings, timing of pre-ED epinephrine administration, the clinical response to epinephrine administration (persistent and recurrent symptoms) and the outcome of refractory anaphylaxis.

Missing data

As with any retrospective chart review, there is potential for missing or inaccurate documentation. These limitations will be mitigated by providing clear definitions and instructions regarding data extraction and interpretation, especially regarding subjective data points. We will also record reasons for missing data and examine factors related to missingness. This will strengthen the missing at random assumption. In the event of excessive missing data (>10% of units with missing predictors or outcomes), we will perform multiple imputation or conduct sensitivity analyses using pattern-mixture models.

Sample size

We based our sample size on the ability to provide precise estimates of the sensitivities for the model to predict patients at very low risk of receiving acute medications or being hypotensive beyond 4 hours from first administered dose of epinephrine (objective 2). To be clinically useful, the clinical prediction rule must effectively identify a group of patients at near zero risk of receiving acute medications or being hypotensive beyond 4 hours of initial epinephrine. Therefore, the rule must have a sensitivity of near 100% to predict those who do receive acute medications beyond 4 hours of initial epinephrine administration. When applied to a validation sample, the lower end of the 95% CI for the sensitivity must be no <95% for the risk of receiving acute medications beyond 4 hours of initial epinephrine. If the rule is 100% sensitive, with a sample size of 150 patients receiving acute medications beyond 4 hours of initial epinephrine, the lower limit of the 95% CI will be 97.6%. If the rule is 98% sensitive, the lower end of the CI will be 94.3%.

It is estimated that 5%–15% of children will receive acute medications beyond 4 hours of initial epinephrine.4 To be conservative, we use 5% for sample size calculation. This would necessitate ~30 institutions each reviewing 400 records, of which approximately 200 would fulfil inclusion criteria, which we believe is feasible in the PEMCRC. We will ask each participating institution to review anaphylaxis encounters between January 2018 and December 2019. In case the target sample size is not reached, we will request participating sites to extend their review 12–36 months before January 2018, until the target sample size is reached. Half the sample will be used for model derivation, and the other half will be used for validation.

Ethics, reporting adverse events and dissemination

Cincinnati Children’s Hospital IRB has approved the study; for a complete list of sites that have received IRB approval at the time of article publication please, see online supplemental table 1. Not all ethical approvals are in place at the time of publication of the protocol; however, all participating sites will obtain local IRB approval prior to study initiation. Adverse events including breach of confidentiality will be reported to local IRBs and CCHMC. The results of the study will be published in peer-reviewed journals and shared via social media, traditional media, blogs and online education platforms. We anticipate the main study findings to be submitted for publication within 6 months of data analysis. Additionally, successful dissemination and application of our findings in clinical care require a diverse team of researchers, clinicians and public health professionals with expertise in clinical research, healthcare research, allergy/immunology and quality improvement. The primary investigators have broad expertise in these fields, and initial site recruitment has included coinvestigators and clinicians with extensive research expertise. Dissemination efforts at the study’s conclusion will include broad participation of key stakeholders in the areas of allergy, emergency care, healthcare policy, quality improvement and dissemination/implementation science.

Discussion

Strengths

This will be the largest retrospective, multicentre study of paediatric anaphylaxis conducted to date. The care received in PEMCRC EDs encompasses a broad group of North American EDs with differing practice patterns and acuity as well as diverse patient populations from wide geographical areas, which will account for potential regional differences in allergens, comorbidities and outcomes. Thus, we believe our findings will be generalisable to the care of children with anaphylaxis across the USA and Canada. When validated prospectively, our findings have the potential to standardise the ED management of children with anaphylaxis. Successful application of these findings may decrease ED lengths of stay and prevent unnecessary hospitalisations that contribute to escalating healthcare costs. Additionally, the data from this retrospective study will inform future prospective studies and support grant funding to refine and validate prediction models to improve the quality of care of children with anaphylaxis.

Limitations

This study is subject to limitations, primarily a result of the retrospective study design. There may be clinical information omitted from the medical record or inaccurate documentation about the timing of administered medications or symptoms. To account for this, we designed and pilot tested the data collection forms and produced a detailed MOO to define data elements and minimise ambiguity about how variables should be coded. We developed definitions of severe, persistent, refractory and biphasic anaphylaxis, as well as persistent and biphasic non-anaphylactic reactions that are clinically relevant and based on variables that are reliably and accurately documented in the EMR. Additionally, data abstractors are not responsible for diagnosing anaphylaxis or anaphylaxis outcomes (other than refractory anaphylaxis); instead, this designation will be made based on predefined composite outcome variables from data elements extracted from the EMR. In spite of these efforts, limitations remain regarding the presence/absence as well as timing and duration of persistent or recurrent symptoms.

Although this is the largest cohort of paediatric anaphylaxis, our findings may not be representative of national trends in the prevalence of anaphylaxis because most participating institutions are academic institutions and, therefore, the findings may not be generalisable to different care settings. We also excluded children who did not receive epinephrine in the pre-ED or ED settings. Although children with anaphylaxis may have complete symptom resolution without epinephrine administration, we sought to assess outcomes based on initial epinephrine administration, which is frequently used by clinicians to determine the length of ED observation or need for hospitalisation.