Multi-dimensional Measurements of Crowding for Pediatric Emergency Departments: A Systematic Review.

The absence of accepted crowding measurement tools that encompass the unique characteristics of pediatric emergency departments (EDs) creates a deficit in advancing efforts to identify and evaluate solutions for this growing problem. In this systematic review, we examined 4 studies that reported on the development and testing of multidimensional pediatric ED crowding measurements. Two investigations involved models (PEDOCS, SOTU-PED) that measured factors indicative or contributory to crowding. A third investigation developed a model mapping the flow of patients through the pediatric ED. The final study modeled the magnitude of physician's work load, particularly when this load is high when crowding is likely present, based on patient arrivals, presenting complaints and conditions, and tests ordered. These works from 4 studies on measuring crowding in pediatric EDs show promise, but this field is at an early stage. Future work should concentrate on comparing the utility of crowding measurements across multiple pediatric ED settings.

Introduction

Emergency department (ED) crowding is a global public health problem which probably will worsen as ED visits increase in many countries.[1,2] To address this threat to patient safety and quality of care, researchers have attempted to investigate the causes of crowding in adult and pediatric EDs,[3] proposed solutions to solve the problem,[4-6] studied safety concerns associated with crowding,[7-9] and analyzed the impact of crowding.[7,10,11] Research on this topic also has tried to establish an association between ED crowding and the quality of care provided to ED patients.[12,13] Despite a significant body of research on this topic, there remains no standard definition or measurement of ED crowding.[14] A commonly used definition for crowding was proposed by the American College of Emergency Physicians: “Crowding occurs when the identified need for emergency services exceeds available resources for patient care in the emergency department, or hospital, or both.”[15] This definition views crowding as situational, varying by institution and time.

Tools for measuring crowding in the adult ED have been developed and extensively studied.[16] Yet, there is no agreement about which tool should be standard, although there are many tools available to be evaluated in the adult ED setting across institutions in the hope of establishing validity for 1 standard tool in the future. The state of the art for measuring crowding is markedly different for pediatric EDs. There has been less research devoted for pediatric emergency medicine settings on techniques to measure crowding. Furthermore, in some countries, EDs dedicated solely for pediatric patients constitute new approaches to emergency care, so the phenomenon of pediatric ED crowding has had a shorter timeframe internationally for investigations on this topic. For these reasons, the tools to measure crowding and quality standards in pediatric EDs depend mainly on the work performed for adult EDs.

There are general principles and methods common to crowding measurements in adult and pediatric EDs, but the measurements designed for the adult settings fail to consider the uniqueness of pediatric ED operations and how the causes and effects of crowding differ greatly between the adult and pediatric emergency medicine settings. Rathlev et al,[17] compared operations featuring the patient acuity, volume, and length of stay between 52 adults and 17 pediatric patients in EDs in the United States (US), using data from the 2017 Academy of Administrators in Academic Emergency Medicine/Association of Academic Chairs in Emergency Medicine Benchmark Survey. The relevant differences observed in this study included that adult EDs had higher mean annual patient visits, more emergency medical system (EMS) arrivals, more EMS-transported patients who were hospitalized, more clinical staff, more behavioral health patients, higher left without being seen (LWBS) visits, more admissions, and higher acuity visits. Pediatric EDs had shorter lengths of stay, but adult and pediatric EDs were similar in regards to the patient arrival per treatment space (ie, number of patients per ED space to evaluate and treat patients), arrivals per clinician hours, arrival time until clinician evaluation, arrival time to admission, and average boarding time. Many of these components are relevant to crowding, which suggests that crowding measurement techniques should be commensurate with the uniqueness of pediatric ED operations.

Previous attempts to measure pediatric ED crowding have been based on unidimensional or proxy measurements or used single components selected from conceptual input-throughput-output frameworks. Although such frameworks consist of multiple, separate, unidimensional measures, the measures have not been combined or integrated to produce 1 quantifiable value that can indicate when an ED is crowded. The common unidimensional or single proxies that have been used to measure pediatric ED crowding are: LWBS rates,[18] mean ED length of stay,[19-21] number of ED boarders and ED boarding time,[19-21] ambulance diversion occurrence,[22] ED acuity (including total census, admission rate, total number of admissions, and proportion of nonurgent patients), ED daily patient volume and cumulative daily ED boarding time,[23] occupancy rate[21,24] length of stay > 6 hours,[25] and ED patient volume.[19] Despite the face validity of these single measurements, they might not fully incorporate the multidimensionality of the pediatric ED crowding phenomenon, and therefore might be at least limited in their scope, if not potentially inaccurate. Composite, multidimensional measurement approaches that involve multiple theoretical components of crowding appear preferable, given that the individual components could exemplify or even effect crowding in stronger or weaker ways across institutions and time.

The absence of accepted crowding measurement tools that encompass the unique characteristics of pediatric EDs creates a deficit in advancing efforts to identify and evaluate solutions for this growing problem. To help overcome this deficit and guide future research efforts, this systematic review was conducted to explore the extent of existing research on the development and evaluation of multidimensional or composite tools that measure crowding specific to pediatric EDs, and to consider what future research is needed for these tools.

Methods

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA, www.prisma-statement.org) guidelines were followed (Figure 1) for this systematic review. The search for relevant articles in English on pediatric ED crowding measuring tools involved searching 5 electronic databases from their inception through February 2020, namely: Medline (PubMed and PubMed Central), Cochrane Systematic Review, CINAHL, EMBASE, and Web of Science. The search terms, keywords, and keyword phrases used were related to “pediatric emergency department” and “crowding” (Table 1). In addition, manual searches were conducted to find relevant articles from the websites of 4 pediatric emergency journals, 1 general pediatric journal, and 13 adult and general emergency medicine journals. Further searches for additional articles were performed by reviewing the references of articles collected during the conduct of this investigation.

Figure 1.

Preferred reporting items for systematic reviews and meta-analyses (PRISMA) diagram of article searches through inclusion.

Table 1

Database Search Strategies.

	Emergency department concept		Crowding concept
	Vocabulary term	Keywords	Vocabulary term	Keywords
PubMed	“Pediatric emergency department” OR “pediatric emergency room” OR “pediatric emergency services”	“Pediatric” OR “pediatric” OR “children” AND “department” OR “room” OR “services”	“Pediatric emergency overcrowding” OR “pediatric emergency crowding” OR “pediatric emergency boarding” OR “pediatric emergency flow of patients” OR “pediatric emergency length of stay”	“Crowding” OR “overcrowding” OR “length of stay” OR “flow of patients” OR “boarding”
CINAHL	“Pediatric emergency department” OR “pediatric emergency room” OR “pediatric emergency services”	“Pediatric” OR “pediatric” OR “children” AND “department” OR “room” OR “services”	“Pediatric emergency overcrowding” OR “pediatric emergency crowding” OR “pediatric emergency boarding” OR “pediatric emergency flow of patients” OR “pediatric emergency length of stay”	“Crowding” OR “overcrowding” OR “length of stay” OR “flow of patients” OR “boarding”
EMBASE	“Pediatric emergency department” OR “pediatric emergency room” OR “pediatric emergency services”	“Pediatric” OR “pediatric” OR “children” AND “department” OR “room” OR “services”	“Pediatric emergency overcrowding” OR “pediatric emergency crowding” OR “pediatric emergency boarding” OR “pediatric emergency flow of patients” OR “pediatric emergency length of stay”	“Crowding” OR “overcrowding” OR “length of stay” OR “flow of patients” OR “boarding”
Cochrane database of systematic reviews	Not applicable	“Pediatric” OR “pediatric” OR “children” AND “department” OR “room” OR “services”	Not applicable	“Crowding” OR “overcrowding” OR “length of stay” OR “flow of patients” OR “boarding”

Studies initially considered for review had to satisfy the following inclusion criteria: full-text, English-language, original research articles studying crowding in the pediatric ED; studies that measured or defined pediatric emergency crowding, the causes of crowding, or the impact or consequences of crowding on patient safety or satisfaction; national, hospital, or departmental wide solutions or interventions to solve pediatric emergency crowding; or work on specific medical guidelines to solve crowding. Work not considered for review were abstracts, review articles, commentaries, non-research letters, opinion pieces or reports, as well as investigations centered in either adult EDs or combined adult and pediatric EDs. Consideration of this broad array of crowding research was purposeful, given that crowding-related investigations might have developed or tested crowding measurement tools as part of a larger study. In the final selection of articles for the systematic review, we concentrated on investigations that specified the creation, development, or testing of approaches or instruments that involved multidimensional or composite components related to crowding (instead of unidimensional or single proxy measurements), and those that produced specific limits, thresholds, or scales for crowding measurements.

For each database or source searched, the title and abstract were screened to exclude non-relevant articles. Articles which appeared to be relevant were screened independently by 2 reviewers for possible inclusion in the review. The 2 reviewers met and analyzed the articles to reach consensus regarding which would be considered for further analysis. The primary author summarized the data from the included articles and conducted an initial critical appraisal. The second reviewer assessed the summary and critical appraisal and made suggested edits; any differences in opinion were resolved through joint review of the articles. A meta-analysis was not possible given the differences in methodology and measured outcomes across studies; instead, a qualitative review was undertaken.

Results

Only 4 articles met study criteria for the present systematic review of multidimensional or composite crowding measurement tools evaluated for pediatric EDs (Table 2). These articles are summarized and critiqued individually as follows.

Table 2.

Summary of Pediatric Emergency Department Multidimensional Crowding Measurement Investigations.

References	Study setting, population, study year, pediatric ED LOS	Crowding measurement	Crowding measurement performance
Weiss et al[26]	Jackson Memorial Hospital pediatric ED, Miami	Pediatric ED Overcrowding Scale (PEDOCS), a scale ranging from 0 to 200 (0, not busy; 40, busy; 80, extremely busy but not overcrowded; 120, overcrowded; 160, severely overcrowded; 200, dangerously overcrowded)	Spearman correlation 0.81 between PEDOCS and pediatric staff (nurse and physician) perception of crowding, as compared to NEDOCS Spearman correlation of 0.70 with pediatric staff perception of crowding
	32 225 patient visits/year (2002)	PEDOCS = 33.3 * 0.11 + 0.07*(patients in the waiting room) + 0.04*(total registered patients)
	February 5 to 25, 2002
	Median LOS: 135 (IQR 120-330) minutes, longest LOS: 227 ± 189.4 (SD)
Noel et al[28]	North Hospital, Assistance Publique Hopitaux de Marseille pediatric ED	Linear model (SOTU-PED) to predict global hourly crowding perception	Correlation between global hourly crowding perception and SOTU-PED: 0.824 (P < .001)
	During model development period: mean LOS 160 (SD 13.1) minutes, median LOS 162 (IQR 152-169) minutes	SOTU-PED = 0.764 + 0.49 Census-H24 (number of admissions in the past 24 hours) + 0.496 Occ-Rate (occupancy rate) + 0.302 1-year infant (number of patients <1 year old) + 0.005 WT-Triage (waiting time for triage) + 0.002 WT-Med (waiting time for medical evaluation)	Prediction of global hourly crowding perception score >5 for SOTU-PED of 2 or greater, AUC: 0.957 (95% CI: 0.933-0.980), odds ratio: 51.88 (95% CI: 20.42-131.83), sensitivity 89.5% (95% CI: 0.79-0.95), specificity 85.9% (95% CI: 0.81-0.90), positive likelihood ratio: 8.16 (95% CI, 3.82-17.43), negative likelihood ratio: 0.157 (95% CI: 0.11-0.22), positive predictive value: 63.7% (95% CI: 60.9-66.4), and negative predictive value: 96.7% (95% CI: 94.3-98.7)
	During model validation period: mean LOS 153 (SD 14.6) minutes, median LOS 152 (140-165) minutes
	36 000 patient visits/year (2016)
	November 25, 2016 to January 25, 2017
Ajmi et al[30]	Regional University Hospital Center (CHRU), Lille, France, pediatric ED	Model of flow through the pediatric emergency department based on 3 primary stages: patient arrival and initial assessment, patient (re)orientation and treatment, and patient destinations	Three separate models were identified for summer, winter, and crisis periods (overcrowding)
	January 2011 to December 2012	The model produced minimum and maximum average waiting times for patients as they progress through stages of care
	23 150 patient visits/year (2011) and 24 039/year (2012)
	Summer period waiting times: 30 minutes to 2:30 hours
	Winter period waiting times: 1 to 4 hours
	Crisis (crowding) period waiting times: up to 10 hours
Chandoul et al[33]	Regional University Hospital Center (CHRU), Lille, France, pediatric ED	Model of healthcare treatment load (burden of care provided to patients by medical staff)	Model could predict during a day when total healthcare treatment load was high (75% and 95% upper limits of distribution of healthcare treatment load)
	January 2011-December 2012 for model development, January-November 2013 for model testing	Model used distributions of patient lengths of stay from 1.186 patient presentations (complaints and conditions) as influenced by number of tests performed
	23 150 patient visits/year (2011) and 24 039/year (2012)
	Median LOS (included cases only): 132 (IQR 87-196) minutes

Abbreviations: LOS, length of stay; IQR, interquartile range; SD, standard deviation; AUC, area under the curve; CI, confidence interval; SOTU-PED in French, Score Objectif de Tension dans les services d’Urgences pediatriques (English translation: quantitative scale for crowding in pediatric emergency department); NEDOCS, adult national emergency department overcrowding scale.

Weiss et al, Development of a Novel Measure of Overcrowding in a Pediatric Emergency Department

Article Summary

Weiss et al,[26] evaluated the utility of the adult National Emergency Department Overcrowding Scale (NEDOCS) at a single US pediatric ED.[27] The “gold standard” was a “composite expert consensus score” (EC score) of the degree of overcrowding, as measured through a 3-question survey per the opinions of the in-charge nurse and the ED physicians. The responses of the in-charge nurse and an unspecified number of ED physicians were solicited during the study period by a trained investigator. Their responses were averaged as a single value for the EC score on a 6-point scale (1 “not busy” to 6 “dangerously overcrowded”). When the EC score was obtained, the trained investigator also collected data on 20 items developed by an expert panel on patient census, throughput times, and ED staffing. Five of these items were used along with other values (eg, number of hospital beds) to compute the NEDOCS metric. Data were collected over a 3-week period at 42 randomly selected times during the study period according to 2 blocks of sample hours (day time: 9 am, 1 pm, and 5 pm) and night time (9 pm and 1 am). Each hour was sampled at least twice. Data at 5 am were not collected due to typical low patient censuses at that hour.

The investigators separately compared the NEDOCS metric and the 20 items they collected to the EC score using Spearman correlations. They also created a new composite measurement of crowding termed the Pediatric Emergency Department Overcrowding Scale (PEDOCS) through multivariable regression. The PEDOCS metric included 2 of the 20 items associated with the EC score: patients in the waiting room and total registered patients. They created a chart that used these 2 items to indicate the expected EC score (from “not busy” to “dangerously overcrowded”). Per the EC score, the ED was overcrowded on 44% of the 42 randomly measured time points.

Critique

Both NEDOCS and PEDOCS have good face validity in their inclusion of factors which are generally considered to be markers of crowding, but are relatively simple as composite measures, and thus might not fully account for the multi-dimensionality of crowding. Although the authors concluded that PEDOCS was superior to NEDOCS, their Spearman correlations (0.80 and 0.71) were close in value and were presented without corresponding intervals. The magnitude of the difference between them was not subjected to statistical testing. There are multiple limitations of this investigation and the composite measurements that were developed and tested. First, the EC score called as the “gold standard” for overcrowding was a subjective assessment that is most likely not replicable across institutions, and perhaps not within institutions. It is unclear if the ED in-charge nurse and physician(s) were aware of the objectives of the study as well as the measurements obtained (which could influence their opinion and score, thereby inflating its relationship to the PEDOCS and NEDOCS metrices); how many physicians were interviewed at each data time point to form the EC score (leading to variability in its measurement); if the ED in-charge nurse and physician(s) were queried independently or together (which affects their average score); what data or impressions they used to provide their opinion (and if they were related to the data collected for the study, which would have inflated the correlations measured); if they had training (and if so, the type and extent) to assess crowding; what were the experience, training, and knowledge level of the ED in-charge nurse and physician(s) regarding ED operations and crowding; and how divergent or convergent the ED in-charge nurse and physician’s opinions were on the crowding state when it was measured (ie, inter-reliability). Second, the investigation was conducted at a single institution, over a short-time period, and during the winter of a single year. Further, PEDOCS was derived and tested at the same location and time with the same dataset. These limitations reduce its external validity to other pediatric EDs with dissimilar operations, patient mix, and factors that influenced crowding. Third, PEDOCS used similar predictors as NEDOCS without consideration of pediatric-specific attributes, such as the age of the children under 1 year; as younger children usually require longer time for evaluation.

Noel et al, Real-Time Measurement of Crowding in Pediatric Emergency Department

Noel et al,[28] aimed to develop and test an objective, real-time, calculated composite measurement of crowding for the pediatric ED setting, they subsequently referred to as SOTU-PED (Score Objectif de Tension dans les services d’Urgences PEDiatriques). The study was conducted at a pediatric ED in Marseille, France. It was conducted in 2 phases, a 1-month model development period, followed by a 1-month validation period. The “gold standard” for SOTU-PED was a global hourly perception of crowding (termed “global HCP”). Global HCP was expressed on a 10-level Likert scale aimed to incorporate both crowding and dangerous conditions (quality and security of care). The scale ranged from “low activity, optimal care” to “high crowding, dangerous impairment of care.” Crowding was present for scores of >5. At each hour of the day (in an unspecified manner) using the global HCP, 2 triage nurses independently assessed the level of crowding perception when a patient presented for care and 2 pediatricians did likewise when discharging a patient. Global HCP scores were averaged; however, only scores that were similar across respondents were considered for analysis (when coefficient of variation was <0.5), which occurred in 50.7% of the development and 43.6% of the validation period. Only 30 hours over the study period included the hours between midnight and 8 am. The components of SOTU-PED were drawn from 10 crowding indicators (predictors) proposed by the study group through a Delphi method.[29] Two crowding indicators were substituted to reflect the pediatric ED setting: number of infants younger than 1-year-old present in the pediatric ED, and the number of children with serious conditions (level 1 and 2 according to the pediatric Canadian triage scale). The 10 crowding indicators were calculated hourly automatically by a tracking software system. The physician and nurses queried for the global HCP assessment were blinded to the crowding indicators that were calculated by the tracking software system (although it was not specified if they were unaware of the indicators themselves). The model for SOTU-PED was created using data from the 1-month development period and tested on the 1-month validation period. Univariable correlations between the 10 crowding indicators and global HCP were assessed using Spearman coefficients, and indicators that were statistically significant were considered for the multivariable models. The multivariable linear regression SOTU-PED model predicting global HCP scores was constructed using these statistically significant indicators through backwards elimination. The final multivariable SOTU-PED model (SOTU-PED) included 5 crowding indicators: waiting time for triage, waiting time before medical evaluation, occupancy rate, number of infants in ED younger than 1-year-old, and number of admissions during the last 24 hours.

A strength of the SOTU-PED model to measure crowding includes its consideration and inclusion of high face validity multiple indicators of crowding as determined by consensus, which were selected in a systematic manner from a group of experts. However, the predictors were intended to reflect crowding in adult and not pediatric EDs, and so might not be specific, relevant or in the proper form for the pediatric setting. The global HCP score as a “gold standard” also has face validity, but is a subjective standard that is without a clear definition, and can vary from person to person. Although it is a strength that the global HCP score used was assessed per visual analog scale with outlined and defined levels and was obtained hourly and independently from 4 nurses and physicians, there was not consensus about whether crowding was present for approximately half of the measurements. This high level of variability demonstrated the global HCP’s lack of reliability. Further, it was measured hourly and sequentially, and over time it probably included multiple measurements from the same participants, which lead to correlation between values. A significant problem of this correlation and with the limitation of using only values in the analysis that were similar across respondents was that the global HCP values probably were not representative of the true experience of crowding in this ED. Further, it is not known if respondents queried when obtaining the global HCP score considered any of the indicators of crowding that were included in the SOTU-PED score from their own review, which would result in a high as well as a potentially false association between global HCP scores and SOTU-PED scores (ie, contamination between the predictor and outcome). The use of backwards elimination modeling also can lead to model overfitting to the data. A model created in 1 setting that is overfit means that it does not include variables that might be less predictive in that setting, but could be more predictive in others, which decreases its utility when applied to other settings. Although it is a strength of the study that the SOTU-PED model was developed during 1 month and assessed during another month, the months were in succession, which would have led to correlation of global HCP scores and SOTU-PED scores, and hence overestimation of its utility. Further, the periods involved were both winter months and the development and validation were at the same site, which can reduce its external validity. Ignoring the concerns about an overestimate of the relationship between global HCP scores and SOTU-PED scores (correlation of values obtained, single site, contamination potential between global HCP scores and SOTU-PED scores, etc.), it is noteworthy that the model results suggested that it is a better predictor of when crowding is not present than when it is, based on the higher negative predictive value.

Ajmi et al, Mapping Patient Path in the Pediatric Emergency Department: A Workflow Model Driven Approach

Ajmi et al[30] and Zgaya et al,[31] reported on the work conducted as part of the HOST project (Hospital: Optimization, Simulation, and Avoidance of Strain), which one of its goals was to create and test models identifying crowded conditions using data from the Regional University Hospital Center pediatric ED in Lille, France. They based their work on the Business Process Model and Notation (BPMN),[32] which is a workflow model-driven framework used to describe complex processes in businesses. The investigators ultimately aimed to develop a workflow model that focused on the significant stages, activities, and actions that affected the patient journey through the pediatric ED from triage until disposition. They also endeavored to model patient pathways through the pediatric ED to identify bottleneck processes contributing to overcrowding. The multi-staged project included observations by the study staff and pediatric ED physicians to create a model of patient pathways, input from ED staff on perceived barriers and processed as well as review of the flow models, direct surveillance data of individual patients progressing through the pediatric ED according to the pathways, construction of the model using the BPMN format based on collected data, and model simulation and testing using real data from the pediatric ED.

The resultant flow model involved 3 stages (patient arrival and initial assessment, patient (re)orientation and treatment, and patient destinations) that include 6 decision points that direct when and where patients will receive care in the ED. Of note in this ED, selected patients were evaluated and underwent testing, then returned to the waiting room, or waited for procedures in a designated area (eg, casting, laceration repair), in contrast to remaining in a patient bed throughout their ED stay. The model considered the minimum, maximum, and average waiting times for activities during the ED stay. Patients were modeled based on arrival during 4 time slots from 8 to 1 am daily. The investigators included data from 2 years (2011 and 2012) to test their model. They identified 3 separate profile models for summer, winter, and crisis periods (overcrowding). Significant crowding determinants were related to availability of inpatient beds (surges in new presentations to the ED, and increase in medical staff work).

The strengths of this approach to measure crowding are in the use of a framework based on flow processes, inclusion of experts in flow modeling as well as front-line staff in the ED in the development and review of the model, direct observation of processes and re-evaluation of the model, use of 2 years of data, and separation of models by time periods (winter, summer, and crisis). The model identifies flow bottlenecks that contribute to crowding and indicate when overcrowding occurs. However, although the authors identify a crisis period indicative of overcrowding, they did not define in the manuscript at what threshold this occurs. They did provide examples of waiting times when a crisis profile existed, but not how this profile was determined. The model had also not been compared to other methods of measuring crowd. The model measures flow bottlenecks as indicators of crowding, and therefore is an approach that shows when crowding affects processes, as opposed to directly measuring determinants of crowding. The study was limited by its derivation and testing in a single ED that had unique features (eg, evaluation of patients who later wait in waiting areas for further care). Further, its application in real time assessment of crowding has not yet been evaluated.

Chandoul et al, A Robust Assessment of Effective HealthCare Demand in the Pediatric Emergency Department

This investigation was also a part of the HOST project at the Regional University Hospital Center pediatric ED in Lille, France. Chandoul et al,[33] developed and tested a different model than Ajmi et al,[30] that could indicate crowding occurrence. They modeled healthcare treatment load (ie, burden of care being provided to patients by medical staff) using an array of patient lengths of stay. The investigators reported that in a previous (uncited) study they obtained data on expected lengths of stay by “patient profile” (assumedly presenting complaint or condition) for 1186 presentations, of which 210 accounted for 88% of pediatric ED presentations (the study authors published a related article on arrays of pediatric ED lengths of stay).[34] For each presenting complaint or condition, they computed distributions of possible lengths of stay from low to high, according to factors that lengthened it, namely number of tests performed. Using data from 2011 and 2012 for this pediatric ED, they developed a model for healthcare treatment load over time of day. Data from the first 11 months of 2013 were used to validate the model. The model tracked each patient by presenting complaint and condition along with number of tests performed, accounting for arrival time in the ED, which enabled calculation of their expected length of stay. This calculation in turn lead to an estimate of total healthcare treatment load for all ED medical staff over time. From the model results, times during the day when total healthcare treatment load was high (75% and 95% upper limits of distribution of healthcare treatment load) could be predicted. The investigators noted that the number of patients present in the pediatric ED was not fully indicative of healthcare treatment load.

This approach to assess the time when crowding occurs has strength in its development on a large amount of data on pediatric length of stay as affected by tests ordered, and has strong face validity in its focus on burden of patient care based on workload assumptions by medical staff. Although reasonably set at higher limits of workload, crowding is inferred at these levels, yet not clearly defined as such. Further, other potential inputs and factors that contributed to crowding were not directly accounted for in the model. The model also appeared to be limited to determinations of crowding over a period of a single day, as opposed to forecasting it for future periods. The study was limited by its derivation and testing in a single ED, and so its approach needs replication and evaluation in other pediatric EDs.

Discussion

Measures of pediatric ED crowding have evolved from unidimensional to multidimensional. Multidimensional measures can enhance understanding of pediatric ED crowding and inform us of the concepts neglected by unidimensional measures. However, pediatric ED crowding in practice is commonly assessed by isolated proxy measurements, despite the awareness that crowding is a multidimensional, multifactorial, and integrated phenomenon. The potential advantage of multidimensional measures is that they not only provide a comprehensive assessment of crowding status, but could also identify specific factors that account for crowding. Knowing these factors could guide resource allocation and assist policy makers in developing appropriate solutions.

As demonstrated in this systematic review, there have been only a few multidimensional crowding measurements developed and tested, which suggests that research on this topic is at an early stage. We were able to uncover only 4 measurement tools that used integrated, multiple indicators (multidimensional) for the assessment of crowding in the pediatric emergency medicine setting. The divergence of approaches, their underlying concepts of crowding, the tools created, the components and metrics of the measurements, methods of tool application, and the assessments of the tools are noteworthy. This diversity, however, can be of concern in their applicability across institutions. However, the goal for all 4 tools was to quantify the crowding in solely in the pediatric ED and to distinguish crowding from non-crowding conditions and periods.

PEDOCS and SOTU-PED employed traditional modeling approaches that combined individual factors or metrices of crowding to provide an objective measurement of when crowding is occurring, although compared against a subjective standard. If implemented in practice, these 2 tools ideally would obtain data from real-time pediatric ED operations and inform ED staff and administrators that crowding is occurring presently, as well as the severity or stage of crowding. The 2 approaches by Ajmi et al and Chadoul et al, in contrast to PEDOCS and SOTU-PED, are highly complex models that inherently encompass factors that indicate or contribute to crowding, instead of specifying the metrices of crowding and examining their effects on crowding. However, crowding is measured as the result of either slowing of flow of patients through the ED (Ajmi et al) or greater work burden on pediatric ED physicians (Chadoul, et al). In both approaches, crowding’s existence was relative to its absence; crowding was present when the models showed that the pediatric ED was not functioning at typical levels, which constituted an internal comparator. Like PEDOCS and SOTU-PED, the 2 models when applied could signal when crowding is occurring, but differ from PEDOCS and SOTU-PED by potentially indicating when crowding will occur, if current trends continue.

All 4 tools do not per se predict future crowding periods (eg, that crowding that might occur the following day or week), unless temporal patterns (eg, day of week, time of day, month, season), or other inputs for the tools reliably repeat themselves to permit such predictions. Of note, all 4 tools established also do not explicitly or directly consider factors outside the pediatric ED that contribute to crowding, although the components included or the models themselves could indirectly reflect the effects of these external factors. External factors such as hospital bed occupancy,[35,36] which leads to an exit block and crowding, have been studied[37] and shown to have a significant effect on ED crowding. How well the 4 tools in this review address these factors is not known. Perhaps of greatest importance, how well these tools perform when used at other pediatric EDs and when compared to each other is not yet established. Given the early stage of pediatric ED crowding measurements, there are many opportunities for research. Undoubtedly the biggest challenge to this research is the lack of a “gold standard” comparator for measuring crowding, and applying such a standard without influencing how the new measurement is being evaluated. Future research should focus on advancing and improving existing pediatric emergency crowding measures in terms of their reliability and validity, within and across EDs. Other approaches and perspectives on crowding should also be considered, such as new modeling techniques simulation methods. The ideal goal is having a best performing, universally applicable, crowding measurement tool that is accurate, usable, and available for all pediatric EDs. However, if that goal is not possible, consideration should be given to having instead a suite of tools that can be adapted to local pediatric EDs needs and characteristics. Another goal for future research is for crowding measurement tools to predict when crowding might occur as far as advance as possible, as opposed to only verifying that it has already occurred. Commensurate with this work should be research on strategies which could be initiated when crowding is anticipated, under an appropriate time frame for action.

Limitations

As for any systematic review, we were limited to reviewing, summarizing, and critiquing only research that has been conducted on this topic, of which there has been only 4 studies. In addition, because research on this topic is at an early stage, the level of complexity of this review was limited to the type of research that has been performed—all single site demonstration studies without external validation and limited internal validation or reliability assessments. Further, the heterogeneous nature of the research and scarcity of the data available did not permit a quantitative analysis. A qualitative review, such as we performed, is also subject to the biases and perspectives of the authors of this review. Although we state that our review was limited to articles published in English, we did not uncover any non-English articles published on this topic, but might have inadvertently missed them and therefore they could not be included in our analyses.

Conclusion

For the pediatric ED, tools to measure crowding are not in as advanced a stage as they are for the adult ED setting. Currently, identified tools need to be evaluated for their reliability, validity, and efficacy across institutions. If they are not efficacious, other innovative multidimensional tools that incorporate the unique features of pediatric emergency medicine need to be developed and tested. By establishing valid pediatric ED crowding measurement tools, future efforts can be attempted to solve the problem of crowding to improve the ED work environment and patient safety.