The effectiveness and safety of emergency department short stay units: a rapid review.

Emergency department (ED) overcrowding has been defined as “a situation where the demand for emergency services exceeds the ability to provide care in a reasonable amount of time.” 1 ED overcrowding is a serious and ongoing issue across Canada; in a 2006 survey of Canadian ED directors, 62% of respondents reported that overcrowding had been a major or severe problem in 2004 and 2005.1

Short stay units (SSUs) have emerged as a potentially useful strategy for managing overcrowding in emergency departments. The theoretical benefit of SSUs is to “off-load” stable patients from the acute-care ED and to reduce the number of unnecessary hospital admissions. Typically, SSUs are focused on (1) expected short treatments such as blood transfusions; (2) further diagnostic investigations to finalize a medical diagnosis; and (3) safe discharge into the community, such as by involving a social worker. To prevent such units from being “dumping grounds,” most SSUs have strict inclusion/admission criteria. Part of the difficulty in evaluating the value of SSUs is terminology, since many terms have been used to describe such units (e.g., observation units, assessment units, and clinical decision units). Typically, however, SSUs are some type of extension of the ED whose overarching objective is to improve “the quality of medical care through extended observation and treatment, while reducing inappropriate admissions and healthcare costs.”2

Although the benefits of utilizing high-quality systematic reviews in the implementation of clinical practice guidelines are well recognized, the uptake of evidence-based decision-making has been slow in the field of health care administration. Policy-makers and administrators often work under strict time constraints, making traditional systematic reviews of the literature impractical. This paper presents an example of a streamlined approach to synthesizing evidence: a rapid review scanning the published literature in a timely manner and presenting the relevant information in a practical evidence summary.

We completed this review in response to a request by members of The Ottawa Hospital (TOH) senior management team. They were contemplating the introduction of an SSU to help alleviate ED overcrowding and wanted to know whether SSUs were effective and safe. They needed an answer urgently and approached our rapid response service (Box 1).

Box 1

Purpose and key messages of short stay unit rapid review

To frame the literature, we used the definition of SSUs as operationalized by the nominated stakeholder acting on behalf of the hospital’s senior management; specifically, we sought and summarized evidence that related to “an area of the hospital reserved for patients admitted directly from the ED who require a period of observation to resolve diagnostic uncertainty before being sent home or who are expected to recover within 48 hours or who require complex outpatient support arranged” (Dr. Alan Forster, Ottawa Hospital: personal communication, 2011).

Methods

Rapid reviews have emerged as a streamlined approach to synthesizing evidence quickly, typically for the purpose of helping decision-makers in health care and health services settings respond in a timely manner to urgent and emerging needs. Despite the proliferation of rapid review products, methods informing their production are disparate and underreported.3 In the absence of a standard protocol for the conduct of rapid reviews, our rapid review service has evolved its own eight-step approach, which was used to guide this review (Box 2; manuscript in preparation).

Box 2

The Knowledge to Action research programme eight-step approach to summarizing evidence

A needs assessment carried out by the Ottawa Hospital Research Institute (OHRI) and senior management of TOH identified SSUs as a priority topic for review (Step 1). The OHRI and a nominated TOH stakeholder then developed an answerable question to narrow the topic and guide the review (Step 2). A review proposal capturing the finalized review question, background information, proposed methods, deliverables, and timeline (10 January to 11 February 2011) was drafted by the research coordinator, and approval of the proposal was sought and obtained from the stakeholder (Step 3).

Using sample papers and the proposal document, an experienced Information Specialist developed and executed a detailed literature search (Step 4). The search strategy for published literature is listed in online Appendix A and was limited to the following databases: MEDLINE and EMBASE on OVID; the Cochrane Library on Wiley (including CENTRAL, Cochrane Database of Systematic Reviews, DARE, HTA, and NHS EED), and the Centre for Reviews and Dissemination (CRD) databases. Additional references were also sought by searching the bibliographies of relevant items. Grey (unpublished) literature was identified by searching the Web sites of relevant specialty societies (e.g., American College of Emergency Physicians, Canadian Association of Emergency Physicians, Society for Academic Emergency Medicine), organizations, and health technology assessment agencies; and through general Internet searching. The grey literature search was guided by and documented through Grey Matters, the Canadian Agency for Drugs and Technologies in Health grey literature search tool (our application of this tool is shown in online Appendix B).4 Retrieved records were imported into Reference Manager® (a bibliographic database software) for removal of duplicates, and then uploaded into DistillerSR© (an Internet-based systematic review software program) to be screened by reviewers.

Screening was conducted by 2 team members (1 with methodological expertise, 1 with ED clinical expertise) (Step 5) using questions developed by operationalized eligibility criteria. A hybrid approach of both a priori and iteratively defined eligibility criteria was used to maximize the efficiency (i.e., limit number of records to be screened) and meaningfulness (i.e., increase relevance or records) of the rapid review for our stakeholders. Thus, as in our previous rapid reviews (available at www.ohri.ca/kta), the included citations had to have been published in English and to be electronically available in full text (i.e., through a University of Ottawa library subscription). To prioritize evidence of contemporary ED practice, reports published before 2000 were excluded. Although primary studies are often included in rapid reviews, we chose to limit the evidence to systematic reviews, given the heterogeneity of the primary studies (determined during screening of title and abstracts) and the limited time available to reconcile findings and interpret the complexity of this evidence base.

Quality assessment, extraction, and narrative synthesis of the included studies was carried out by 1 member of the team (Step 6). The quality of included systematic reviews was assessed using AMSTAR, an 11-item measurement tool created to assess the methodological quality of systematic reviews (see online Appendix C). A higher score indicates increased methodological quality.5 For each systematic review, the primary objective, methods, findings, and relevant limitations was extracted and described narratively.

The extracted information was formatted into a cogent, user-friendly final report (Step 7). To help stakeholders get to the main points quickly, key messages were presented on the first page of the rapid review report and each subsection was summarized in a “bottom line” statement. The report was submitted to end-users for feedback and approval (Step 8). The approved report was then circulated by our stakeholder to the clinical and management knowledge users of TOH.

Results

A PRISMA flow diagram of the evidence identified by this rapid review is shown in Figure 1.

Figure 1

PRISMA flow diagram of study selection for qualitative analysis

Evidence on SSUs specifically

A 2003 systematic review conducted by Daly and colleagues2 (AMSTAR score 6/11) assessed the evidence on short stay observation units with respect to efficiency of care delivery and quality of services. Specifically, data from included studies were extracted according to the following domains: clinical outcomes, length of stay, re-presentation rates, ED efficiency and costs of care. Notwithstanding the fact that the review’s search date was over 10 years old, this was the best available synthesis of SSUs included in this rapid review. Twelve studies (1 Canadian) comparing observation units with routine care were included; between-study heterogeneity precluded quantitative meta-analyses, and findings could be presented only narratively. Table 1 from this report, summarizing the study characteristics and main conclusions, is included below. The authors concluded that “[SSUs] have the potential to increase patient satisfaction, reduce length of stay, improve the efficiency of EDs and improve cost effectiveness. However, [SSUs] have commonly been implemented alongside new clinical protocols, and it is not possible to distinguish the relative benefits of each. As demand increases, providing effective and cost-efficient care will become increasingly important. [SSUs] may help organizations that are attempting to streamline patient care while maintaining their quality of service delivery.”

Table 1

Comparative studies of short stay units

Bottom line

Evidence from 1 systematic review that assessed evidence up to 2000 and included 1 Canadian study suggested that SSUs may offer an effective and safe ED patient management option. Specifically, findings from the 12 studies reviewed suggested that SSUs may lead to improvements in patient satisfaction, length of stay, ED efficiency, and cost effectiveness. However, these findings should be interpreted cautiously, given the methodological limitations of the included studies and the need for an updated literature search.

Evidence on solutions for overcrowding

A 2006 systematic review by the Canadian Agency for Drugs and Technologies in Health (CADTH)1 assessed the evidence on interventions to reduce overcrowding in the ED (AMSTAR score 9/11). SSUs were captured in 2 before-and-after studies and were associated with positive outcomes; 1 study reported a decrease in ED length of stay for treat-and-release patients, while the other reported a decrease in the number of patients who left before being seen and in the number of ED diversions. On the basis of this evidence, the review authors categorize SSUs as one of the several interventions for which “limited evidence suggests that these efforts to address overcrowding at an institutional level should be encouraged and monitored; they have a high chance of success” (see Table 2 for an overview of the interventions assessed).

Table 2

Evidence-based interventions for emergency department (ED) overcrowding and clinical practice

Although the review attempted to assess the relative effectiveness of interventions aimed at improving ED overcrowding, the lack of direct comparisons and the general trend toward positive outcomes restricted this aim. Consequently, the reviewers could conclude only that “many interventions of varying complexity, intensity, and duration have been applied in an attempt to alleviate or control ED overcrowding. While most seemed to reduce overcrowding, it is difficult to determine the relative value of these interventions, and the lack of comparison studies makes it impossible to say which ones work best.”

A 2008 systematic review by Hoot and Aronsky6 assessed the evidence pertaining to the causes, effects, and solutions of ED overcrowding (AMSTAR score 2/11); 4 studies assessing observation units (grouped under “solutions”) were included. Study findings were summarized narratively and were generally positive with respect to process outcomes (e.g., decreased length of stay, rate of ambulance diversion, and number of patients leaving without being seen). Although systematic methods were employed, the literature search was not comprehensive (i.e., it searched a single database only and excluded non-English titles and grey literature) and extracted quality assessments were not used to frame study results. In view of the complexity of the included studies, the reviewers “refrain[ed] from making strong conclusions … based primarily on judgment rather than numeric inference” and considered their review to be of value more as a “structured overview of the relevant literature” to “guide interested readers to the original articles.”

Evidence from 2 systematic reviews published in 2006 and 2008, respectively, assessed interventions aimed at reducing ED overcrowding. Several studies assessing SSUs were included and resulted in generally positive process outcomes. Although, in light of the complexity of the studies included in their 2008 review, Hoot and Aronsky6 abstained from drawing conclusions, Bond and colleagues1 concluded in their 2006 CADTH report that there was sufficient (albeit limited) evidence to warrant implementation and further investigation of SSUs across institutions in Canada.

Other evidence

A 2006 systematic review by Boudreaux and colleagues7 assessed the evidence on performance improvement methods for increasing ED patient satisfaction (AMSTAR score 5/11). Observation units were captured as 1 of several interventions; they found “one supportive study (and no negative studies) demonstrating improvement in at least one indicator of satisfaction.”7 Only observation units for specific conditions (e.g., asthma and chest pain) were captured in this review.

Limited evidence from 1 systematic review indicates that SSUs may lead to improved patient satisfaction in specific clinical contexts.

Discussion

Emergency department crowding has been identified as a key concern for many hospitals across Canada. Two landmark studies published in 2006 demonstrated an association between ED/hospital crowding and patient mortality.8,9 Numerous other studies have demonstrated the significant adverse effect that ED crowding has on various clinical outcomes, including delayed time to thrombolysis in myocardial infarctions, delayed antibiotic administration in pneumonias, and decreased quality of pain care.10-12 It is now well understood that the problem extends beyond the ED, and that ED crowding is a symptom of much larger systemic problems within our hospital system. The American College of Emergency Physicians recently published a report recommending a series of possible solutions to help alleviate the problem, including the establishment of observation units/SSUs.13

Limitations of the available evidence

Our rapid review aimed to assess the evidence supporting the safety and effectiveness of SSUs in the Canadian context. Although the evidence addressing SSUs and their effect on ED crowding is limited, authors of the included systematic reviews are cautiously optimistic about the benefits of SSUs and suggest their further implementation and evaluation in Canadian hospitals.

The most notable limitation of SSU evidence is the lack of a sophisticated evaluative design—a problem common to evaluations of health services/administrative initiatives. CADTH provides valuable recommendations for the conduct of future studies, such as the need for comparable and representative comparison groups, blinded or unbiased outcome assessments, concurrent controls, comprehensive outcome assessment, and prospective design.1 Another significant limitation of current SSU evidence pertains to the unclear generalizability of results that arises from variability in how SSUs are defined and delivered and in their setup and target patient population(s) from one institution to another. Also important is the fact that the most recent systematic review identified by this rapid review is almost a decade old and thus may be outdated or lack applicability to today’s health care environment. Although this outdatedness does not inherently negate the findings of our rapid review, we do consider it a limitation of this evidence base and have begun the process of updating this systematic review. Finally, it is worth noting that there is no high-quality evidence reported in the literature on the cost-effectiveness of SSUs; this is an area where more information is sorely needed to support evidence-based decisions.

Benefits and limitations of rapid review

One distinct advantage of rapid reviews is their short preparation time and hence their ability to help management and policy users make timely decisions. However, although timeliness is an important issue for rapid reviews, there does not appear to be any consensus as to what “rapid” really means. In their scan of this literature, Ganann and colleagues noted that rapid reviews took anywhere from 1 to 9 months to complete.3 We believe that a time frame closer to 1 month is more appropriate. Rapidity, however, is also a potential limitation. To complete the process quickly requires truncating the traditional systematic review process to varying degrees, which can result in an increase of error and/or bias. It is reassuring to know that in their comparison of the results and conclusions of 4 rapid reviews and systematic reviews that addressed the same question, Watt and colleagues found few differences between the 2 types of reviews.14 However, more research on this topic is clearly needed, and cautious interpretation of the findings of our rapid review is warranted.

Another potential advantage of rapid reviews—at least those we are conducting—is their look and feel. Systematic reviews are often large documents that can be difficult to penetrate for key messages and the bottom line, both of which are important for management and policy decision-makers. Cognizant of this limitation, we initially based the format of our reports on that of the SUPPORT collaboration network15 and have iteratively made adaptations on the basis of feedback from end-users. Key messages appear on the first page, and the entire document is usually under 10 pages long. As a companion to this manuscript, the original format of the SSU rapid review is provided in online Appendix C.