Measures of Emergency Department Crowding, a Systematic Review. How to Make Sense of a Long List.

Emergency department (ED) crowding, a common and serious phenomenon in many countries, lacks standardized definition and measurement methods. This systematic review critically analyzes the most commonly studied ED crowding measures. We followed the Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) guidelines. We searched PubMed/Medline Database for all studies published in English from January 1st, 1990, until December 1st, 2020. We used the National Institute of Health (NIH) Quality Assessment Tool to grade the included studies. The initial search yielded 2293 titles and abstracts, of whom we thoroughly reviewed 109 studies, then, after adding seven additional, included 90 in the final analysis. We excluded simple surveys, reviews, opinions, case reports, and letters to the editors. We included relevant papers published in English from 1990 to 2020. We did not grade any study as poor and graded 18 as fair and 72 as good. Most studies were conducted in the USA. The most studied crowding measures were the ED occupancy, the ED length of stay, and the ED volume. The most heterogeneous crowding measures were the boarding time and number of boarders. Except for the National ED Overcrowding Scale (NEDOCS) and the Emergency Department Work Index (EDWIN) scores, the studied measures are easy to calculate and communicate. Quality of care was the most studied outcome. The EDWIN and NEDOCS had no studies with the outcome mortality. The ED length of stay had no studies with the outcome perception of care. ED crowding was often associated with worse outcomes: higher mortality in 45% of the studies, worse quality of care in 75%, and a worse perception of care in 100%. The ED occupancy, ED volume, and ED length of stay are easy to measure, calculate and communicate, are homogenous in their definition, and were the most studied measures.

Introduction

Emergency Department (ED) crowding is a common problem in the USA and around the world. Crowding can lead to the boarding of admitted patients in the ED. In a recently revised policy statement, the American College of Emergency Physicians lists several resulting problems. These include treating patients in areas not designated for treatment, such as hallways, treating boarded patients by ED nurses, increased ambulance diversion time, and decreased patient and ED staff satisfaction.1 Most authorities agree that ED crowding is a public health crisis that negatively impacts patient safety, worsens the quality of care, and increases mortality in some cases.2

Many EDs have in the last decade seen an increase in the number and acuity of illness of their patients.3 As the population ages, ED crowding is likely to continue to worsen. Furthermore, in some instances, the ED houses an observation unit for specific diseases or diagnoses, a process that can itself worsen crowding.

Accurate ED crowding measurement and evidence-based understanding of its impact are prerequisites before attempting to find solutions. At its core, crowding depends on three variables: the volume of patients arriving (input), the time to process and, or treat patients (throughput), and the volume of patients leaving the ED (output). Any combination involving a rise in input, a delayed throughput, or a decreased output can lead to crowding.

Despite substantial published research, there is still no consensus around the best metric to measure and define ED crowding. With the lack of standardized measuring methods, the comparison of research paper results cannot always generalize their applicability. This review of ED crowding measures literature attempts to provide a comparison of the different measures.

Methods

Study Design

A comprehensive systematic review of published medical literature from 1990 to 2020 revealed multiple ED crowding definitions. We focused on eight measures that were both commonly accepted and not specific to a country’s health system:

ED occupancy: The proportion of occupied ED beds, which is the number of occupied beds divided by the total number of ED beds, usually expressed in percentage.

ED length of stay (LOS): The time patients spend in the ED.

ED Volume: The total number of patients in the ED during a defined time.

ED boarding time: The time admitted patients spend in the ED waiting for transport to their assigned hospital bed.

Number of boarders: The number of patients waiting in the ED for a hospital bed.

Waiting room number: The number of patients in the waiting room of an ED.

The National ED Overcrowding Scale (NEDOCS) measure: A score based on a formula that requires the following variables: the number of ED patients, the number of ED beds, the number of hospital beds, the number of ventilators in use in the ED, the waiting time for the longest admission, the waiting room time of the last patient called to a bed, and the number of admits in the ED.

The Emergency Department Work Index (EDWIN) score: Is based on a formula that aims to measure the ratio of workload to work capacity. The formula is ∑niti/Na(BT- BA), where ni is the number of patients in the ED in triage category i, ti is triage category, Na is the number of attending physicians on duty, BT is the number of treatment bays, and BA is the number of admitted patients in the ED. The triage category is defined by the Emergency Severity Index (ESI).

The outcomes studied in the literature belonged to one of these three categories: mortality, quality of care, and perception of care. The quality-of-care outcome includes the timeliness of care. It often involves meeting pre-specified core measures, administering appropriate tests or medications in a timely fashion, or avoiding negative throughput events such as having patients leave without treatment (LWOT) or going on ambulance diversion. The perception of care outcome reflects the physicians’ and nurses’ impressions of the state of ED crowding.

This systematic review follows the Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) checklist for reporting. This study was exempt from IRB approval.

Search Strategy and Study Selection

On December 15th, 2020, three independent reviewers (CM, DC, TA) queried the PubMed (Medline) database with the keywords “ED, ER, Emergency Room, Emergency Department” and “Crowding, Overcrowding” analyzing titles and abstracts of all the articles published in English from January 1st, 1990 until December 1st, 2020. Our senior author (JSR) performed an initial review to guide the selection and inclusion of the crowding measures and outcomes. This search was not shared with the three other independent reviewers to avoid bias.

After reading the full text, the reviewers decided on the manuscripts meeting the inclusion criteria for the final analysis. The senior author (JSR) resolved disagreements. The reviewers included all types of studies (retrospective, prospective, qualitative, quantitative) and excluded simple surveys, review papers, opinions, letters to the editors, and case reports. They also included, when appropriate, additional studies found in the bibliographies of the manuscripts, using the same review process. In terms of content, the included studies needed to be relevant to the subject, with clear studied outcomes and measures of crowding.

The three independent reviewers (CM, DC, TA) manually extracted data into a standardized table provided by the senior author (JSR). The table’s categories are the date of publication, country of the studied population, type of study, population studied, ED crowding variable, outcome measured, type of hospitals, diagnoses, and type of patients (adult/pediatric/all).

Two independent reviewers (SB and either CM, DC, or TA) graded each of the manuscripts included in the final analysis using the National Institute of Health (NIH) Study Quality Assessment Tool.4 The possible grades based on the tool are Good, Fair, and Poor. The senior author (JSR) reviewed the ratings if the grading was not concurrent and weighed in. The authors had decided to eliminate papers they would grade as having a Poor Quality.

Results

The initial search yielded 2293 abstracts, 2184 of whom got excluded after title and abstract review. The bibliography search of the resulting 109 manuscripts yielded seven additional studies. Twenty-six of the 116 studies were later excluded after full review for the following reasons: ED crowding measures inclusion criteria not met (6 studies), selected outcome measures criteria not met (14), simple survey (5), incorrect time frame (1) (Figure 1). The final analysis, therefore, included 90 studies.

Figure 1

This figure summarizes the method used to select the studies of the review. Out of 2293 abstracts and titles reviewed, 109 were selected for full review. Seven more studies were later induced, and 26 later excluded, yielding a final number of 90 studies.

Study Characteristics

None of the chosen 90 papers got eliminated due to a grade of “Poor” based on the NIH tool. The reviewers graded most (72) of the selected studies “Good” with 18 studies graded as “Fair.” The studies mainly occurred after the year 2000. The majority of the studies occurred in the USA (55 out of 90 studies), followed by Canada (7 out of 90) and South Korea (4 out of 90). The studies were all observational, mostly retrospective cohorts. The majority of the studies were performed in tertiary care centers. We list details about each of the 90 studies in the ; we include the year and country of the study, the type of study, the population studied, the crowding measure and outcome each study addresses, a brief description of the findings, as well as the grading of the reviewers.

Analysis by ED Crowding Measure

The grouping of the papers based on the crowding measure studied (Table 1) showed that ED occupancy was the most studied measure (35 studies), followed by ED LOS (24 studies) and ED volume (21 studies). Studies about these measures were performed in community, tertiary hospitals, or both. They included adult and pediatric patients, as well as a multitude of diagnoses. The crowding measures that were the most heterogeneous were the boarding time and the number of boarders because patients were defined as boarders at different time points. Waiting room occupancy and boarding occupancy were eliminated due to a lack of studies.

Table 1

Description of Crowding Measures

Measure, Years of Publication	Type of Hospital	Type of Diagnoses	Type of Patients or Subjects	Description of Variable	Type of studies	Quality of Studies
ED occupancy 2006–20195–38	Tertiary5,8–10,13–15,17,19,20,24–34,36,37,39Community35Both6,7,11,16,18,21,23,38,40	Pain: abdomen,19 back,24 chest,22 all37Asthma7,23,29 CAP13,28MI7,10CVA, Stroke6,20,25Cardiac Arrest15Fracture12,30 Dislocation7Sepsis8,21,27Trauma36All patients5,9,11,13,14,16–18,26,31–35,38,39	Adults6,10,12,13,15,17–20,22–25,27,28,31,33,35–38Pediatrics29,30All patient7–9,11,16,21,26,32,34ED physicians and nurses14ED Nurses39	Quartiles5–8,19,22–26,33,37Continuous10,12,13,15,16,18,20,21,27–30,34,38,41Intervals of 1 hours9,17Nurse and physician survey14Overcrowding hazard Scale31	Retrospective5–9,12,13,17–19,21–30,32,33,36–38Prospective10,11,15,20,34,35,42Survey14,39	31 Good4 Fair
ED LOS 2003–20195,6,8,9,12,19,22–24,38,40,42–53	Tertiary5,8,9,12,19,22,43,45,48,50,53,54Community51Both6,23,24,38,40,42,44,46,47,49,52	Pain: abdominal pain,19 back Pain,24 chest22Asthma23 CAP50MI46,50,52CVA, Stroke6,43,44Intubated Trauma patients45Sepsis8,53Hip Fracture12All Patients5,9,38,40,42,47–49,51	Adults6,7,12,19,22–24,38,43–46,52,53All patients8,9,40,42,47–51	< or > 8 hours43Quartiles6,8,19,22–24,48,52<4, 4–8, > 8 hours46<2, ≥ 8 hours and by hours in between5Continuous: minutes,45 hours9,12,38,40,47,50,51>1, 1–2, 2–5, >5 hours42<9, 9–24, >24 hrs49	Retrospective5,6,8,9,12,19,22–24,38,40,42–53	22 Good2 Fair
ED volume 1999–20199,21,40–42,44,50,54–67	Tertiary9,41,50,54–67Both21,40,42,44	Asthma59 CAP, pneumonia50,56,57,67CVA, TIA, stroke44,63Cardiac arrest58 MI 50Pain54,60,62Sepsis21All patients9,41,42,55,61,65,66	Adults41,44,54,56–58,60,63,67Pediatrics59,62,66All patients9,21,40,42,50,55,61Physicians and nurses64,65	Daily,9,40,41,58 weekly,61 annual44,55,59 volume.ED volume when patient arrives21,42,54,56,57,60,62,63Quartiles67Continuous50	Retrospective9,21,40–42,44,50,54,56–59,61–63,67Prospective55,60,66Survey64,65	19 Good2 Fair
Boarding time 1999–201822,38,50,51,55,65,67–78	Tertiary22,50,55,65,67–78Both38Community51	Cellulitis72CAP, pneumonia50,67 Pneumonia72Critically-ill68,70Chest pain22,72,78MI50Necrotizing Fasciitis.71All patients38,51,55,65,69,73–77	Adults22,67,71,72,74,76–78All patients38,50,51,55,65,68–70,73,75Physicians and nurses65	Boarding hours38,50,51,69,72,74,77<6, 6–24, >24 hours68,70≤ 3, >3 hours;78 >8 hrs71Annual55Quartiles22,67Mean boarding time, beginning 8-hour interval75	Retrospective22,38,50,51,67–73,76,77Prospective55,75,78Survey65	14 Good3 Fair
Number of Boarders 1999–20196,8,21–24,54,57,60,64,65,69,75,79–82	Tertiary8,22,54,57,60,64,65,69,75,79–81Both6,21,23,24,82	Asthma23 CAP, pneumonia57Pain: back,24 chest,22 all54,60CVA, stroke6Critically ill82 Sepsis8,21Femoral fracture81All patients64,65,69,75,79,80	Adults6,22–24,54,57,60,81,82All patients8,21,64,65,69,75,79,80Physicians and nurses.64,65,80	Quartiles6,8,22–24,54,81> 4 hours,79 > 6 hours82Number on arrival21,57,60,69,80Average number of boarders, beginning 8-hour interval75	Retrospective.6,8,21,22,24,54,57,69,79,81,82Prospective75Survey64,65,80	13 Good3 Fair
ED waiting room census 2002–20176,8,9,18,19,22–24,37,60,64–67,80	Tertiary8,9,19,22,37,60,64–67,80Both6,18,23,24	Asthma.23 CAP67Pain: back,24 chest22 all19,37,60CVA, stroke6Sepsis8All patients9,18,64–66,80	Adults6,18,19,22–24,37,60,67,80All patients8,9,64Physicians and nurses64–66,80	Quartiles6,8,19,22–24,37,50,67Continuous18,60,80Hourly Average9	Retrospective6,8,9,18,19,22–24,37,67Prospective60Survey64–66,80	13 Good2 Fair
NEDOCS 2005-2018 11,14,34,39,66,83–91	Tertiary11,14,34,39,66,83–85,88–91Community86,87	All Patients11,14,34,39,66,83–91	Physicians and nurses14,66,83,84,86,87,89All patients11,34,85,88–91Pediatric66ED Nurses39	Quartiles14,39,84,87,90NEDOCS q10 min, AUC to predict ED diversion11,85<100, 100–140, >14034,91	Retrospective85Prospective.11,34,66,87–91Survey14,39,83,84,87,89	13 Good1 Fair
EDWIN 2003–20197,11,14,16,21,34,85,89,92–94	Tertiary11,14,16,34,85,89,92–94Both7,21	Asthma7Dislocation7MI7 STEMI94All patients11,14,16,21,34,85,89,92,93	Adults94All patients7,11,16,21,34,85,89,93Physicians and nurses14,92	Quartiles7<1.5 >1.514,93,94Continuous16,21,34q10 min, AUC for ED diversion11,85	Retrospective7,21,85,94Prospective11,16,34,89,93 Survey14,89,92	10 Good1 Fair

Notes: The measures are listed from the most to the least studied. The table provides details on the type of hospital, patients, or subjects in the studies. We provide a brief description of the variables used to measure crowding and aggregate the reviewers’ ratings per measure.

Abbreviations: CAP, community-acquired pneumonia; CVA, cerebrovascular accident; TIA, transient ischemic attack; MI, myocardial infarction.

In Table 2, we summarize findings from each crowding measure. Most of the variables were easy to calculate and easy to communicate, except for NEDOCS and EDWIN.

Table 2

Summary of Findings on ED Crowding Measures

Measure of Crowding	Ease of Calculation	Ease of Communication	Heterogeneity of Measure Definition	Number of Studies per Outcome, Number of Studies with a Positive Association in Parentheses
				All Outcomes, with Authors Ratings	Outcome: Mortality	Outcome: Quality of Care	Outcome: Perception of Care
ED occupancy	Easy calculation	Self-explanatory	Low	35; 31G, 4F	12 (5)	26 (15)	2 (2)
ED LOS	Easy calculation	Self-explanatory	Low	24; 22G, 2 F	8 (4)	21 (17)	0
ED Volume	Easy calculation	Self-explanatory	Low	21; 19G, 2 F	4 (2)	19 (17)	3 (3)
Boarding time	Easy calculation	Self-explanatory	High	17; 14G, 3 F	8 (5)	12 (7)	1 (1)
Number of boarders	Easy calculation	Self-explanatory	High	16; 13G, 3 F	4 (1)	14 (10)	3 (3)
Waiting room census	Easy calculation	Self-explanatory	Low	15; 13G, 2 F	2 (0)	10 (8)	4 (4)
NEDOCS	Complex formula	Difficult	Low	14; 13G, 1 F	0	7 (6)	8 (8)
EDWIN	Complex formula	Difficult	Low	11; 10G, 1 F	0	8 (8)	3 (3)
Percentage of studies with a positive association					45%	75%	100%

Notes: This table provides a high-level summary of our findings. All measures are easy to calculate and communicate, except for NEDOCS and EDWIN. All measures are relatively homogenous in their definition and utilization, except for the boarding time and number of boarders. The measures are listed from the most to the least studied. The table summarizes the reviewers’ ratings per measure and the number of studies with positive (worse outcome) associations.

Abbreviations: F, fair; G, good.

Outcome Measures

Quality of care was the most studied outcome. The EDWIN and NEDOCS had no studies with the outcome mortality. ED crowding was often associated with worse outcomes: higher mortality, worse quality of care, and worse perception of care in, respectively, 45, 75, and 100% of the studies. All the studies using the EDWIN score showed an association between increased crowding and worse outcomes. Table 2 provides a high-level summary and allows for at a glance comparison of the measures’ ease of calculation, ease of communication, and definition heterogeneity. It quantifies the number of studies per measure and outcome and the number of studies that showed worse outcomes. The supplementary offers a deeper, more detailed dive into the outcomes studied. In the , we describe the outcomes used in the 90 studies. We group them by measure and broad outcome category and clearly separate the studies that showed a positive association from those that did not.

Discussion

This review confirms the challenging aspect of measuring and defining ED crowding. The different ED crowding measures vary in the degree to which they were studied, the addressed outcomes, and the homogeneity of their definition. One should consider these factors if trying to choose one measure over another. Measures such as the ED occupancy, ED LOS, and ED volume are homogenous in their definition, well studied, easy to understand, measure and communicate and remain the obvious measure to which to go. The remaining measures are less studied. The two scores, EDWIN and NEDOCS, are very complex formulas. The advent and implementation of electronic medical records have eased the obtention of these scores with their ability to update the many required variables for the calculation constantly. The resulting score may still not be as easy to communicate to other health care providers and administrators who are not familiar with these two scores.

The studies reviewed had some variability in the definition of crowding measures, but the heterogeneity of the number of boarders and boarding time measures was higher comparably. Some standardization in the definition of boarding would be a helpful step given how valuable the measures of ED boarding are for a hospital system and how much boarding is a cause of patient dissatisfaction.

Unlike quality of care, mortality is not a widely studied outcome across the different measures. The mortality outcome is mainly studied with ED occupancy, and we would recommend this measure when addressing the outcome mortality. Note that mortality has been studied with neither the NEDOCS nor the EDWIN scores.

The perception of care, or how “busy or overwhelmed” the provider felt secondary to ED overcrowding, was less studied than the quality of care and mortality outcomes. The NEDOCS is the most studied measure on the perception of care. By nature, perception of care is more subjective than traditional outcomes. Yet, this perception of care is a more practical outcome that offers the “feel” of the providers working in a busy ED. Although difficult to quantify, this “feel” embodies a complex amalgam of the providers’ engagement, the acuity of patients, surges in volume, resource-intensive patients, and the help of ancillary services. The perception of care is even more crucial to study given the physicians, advanced practice providers, and nurses burnout epidemic. The vast majority of studies using NEDOCS and EDWIN found a positive association with perception of care.

This study has multiple limitations. As shown above, the ED crowding literature is quite large, though most of the studies were performed in US tertiary care hospitals. Extrapolating results to different countries and community hospitals requires caution. ED crowding research is limited by the retrospective nature of the studies done so far. Past research has not portrayed the dynamic and rapidly changing nature of ED crowding. For example, a hypothetical patient presents to an uncrowded ED. As their workup progresses, the ED becomes progressively more crowded. To what level of crowding can one reliably attribute potential adverse outcomes? Finally, our review weighed all studies that met our inclusion criteria equally even when the number of the studied patients or subjects varied considerably between them.

This paper highlights the inequalities between the different measures. If searching for the perfect measure, you will find none. Such findings have many implications on individual practicing staff and hospital systems but also at a population level. A hospital system might consider using a combination that includes at least one traditional measure, such as ED occupancy, ED LOS, or ED volume. Public health policies that decrease crowding can be challenging to write and implement without a consensus on the best evidence-based measure and the wide variability of measures amongst ED practices. Finally, additional studies are necessary to provide insight into this complex problem and compare the different measures.

Conclusion

Our systematic review of 90 studies of ED crowding highlights the variation between each measure, especially in terms of the outcome studied. The ED occupancy is the best-studied, followed by the ED LOS and ED volume. Newer scoring systems such as the NEDOCS and EDWIN have been studied less than the more traditional ones. It is essential to consider these findings when trying to answer a research question, influence clinical practice, or write policy.