Use of the SONET score to evaluate Urgent Care Center overcrowding: a prospective pilot study.

OBJECTIVES: To derive a tool to determine Urgent Care Center (UCC) crowding and investigate the association between different levels of UCC overcrowding and negative patient care outcomes.
DESIGN: Prospective pilot study.
SETTING: Single centre study in the USA. PARTICIPANTS: 3565 patients who registered at UCC during the 21-day study period were included. Patients who had no overcrowding statuses estimated due to incomplete collection of operational variables at the time of registration were excluded in this study. 3139 patients were enrolled in the final data analysis. PRIMARY AND SECONDARY OUTCOME MEASURES: A crowding estimation tool (SONET: Severely overcrowded, Overcrowded and Not overcrowded Estimation Tool) was derived using the linear regression analysis. The average length of stay (LOS) in UCC patients and the number of left without being seen (LWBS) patients were calculated and compared under the three different levels of UCC crowding.
RESULTS: Four independent operational variables could affect the UCC overcrowding score including the total number of patients, the number of results pending for patients, the number of patients in the waiting room and the longest time a patient was stationed in the waiting room. In addition, UCC overcrowding was associated with longer average LOS (not overcrowded: 133±76 min, overcrowded: 169±79 min, and severely overcrowded: 196±87 min, p<0.001) and an increased number of LWBS patients (not overcrowded: 0.28±0.69 patients, overcrowded: 0.64±0.98, and severely overcrowded: 1.00±0.97).
CONCLUSIONS: The overcrowding estimation tool (SONET) derived in this study might be used to determine different levels of crowding in a high volume UCC setting. It also showed that UCC overcrowding might be associated with negative patient care outcomes. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

The first prospective study on urgent care overcrowding.

The first study to report a link with overcrowding and patient outcome in the urgent care setting.

Derived a new overcrowding scoring system for urgent care crowding estimation which has not been reported earlier.

This is a single centre study that requires external validation.

Special population selection could lead to selection bias.

Introduction

As the demand for real-time access to care increases, emergency department (ED) overcrowding has become more and more common in recent years.1 2 One of the solutions to ED overcrowding is to reduce the numbers of low-acuity patients presenting for care.3 4 It is reported that hospitals are adding their own or partnering with existing non-hospital based urgent care centers (UCCs) to offset ED overcrowding.5 6 According to the report from the Urgent Care Association of America, the number of UCCs has increased over 12% within 3 years and it has provided care to over three million patient visits every week.7 UCCs are now recognised as providing convenient, less expensive access to care as compared with that experienced at an average ED.

In primary care settings, the gap of available providers is expected to continue to grow. The primary care setting workload is expected to increase by 29% from 2005 to 2025. Meanwhile, the number of primary care providers is expected to grow by only 2–7% during the same time frame.8 9 Given the prediction that both ED and primary care settings will continue to be resource constrained, a proactive approach to anticipating UCC overcrowding will offer a means to mitigate patient care risk. To the best of our knowledge, no UCC overcrowding estimation tool has been reported to date.

Accurately estimating UCC overcrowding will not only help reduce ED overcrowding but will also alert administrators to take action by mobilising resources as an overcrowded condition becomes imminent, thereby minimising the risk of undesirable patient care outcomes.10 11 The primary goal of this study is to derive a suitable tool we named SONET (Severely overcrowded, Overcrowded and Not overcrowded Estimation Tool) to evaluate overcrowding in a high volume UCC setting. A secondary goal is to determine the association between UCC overcrowding and negative patient care outcomes.

Materials and methods

Study design and patient population

This was a prospective pilot study designed to derive an estimation tool to determine overcrowding status in a moderate-to-high volume UCC setting. This study was carried out at a publicly funded health system that has both ED and UCC at different locations within the main campus and with separate triage systems. The annual volume of the study UCC is approximately 62 000 visits. Considering that no previous UCC overcrowding study has been reported and that no historical data are available for sample size estimation, the same study period used for the ED overcrowding study was used in this study.12 The John Peter Smith Health Network Institutional Review Board approved the study (IRB approval number: 110413.003ex).

All patients who registered initially at UCC were included in this study. Patients were triaged by dedicated nurses at the triage encounter point and individual patient acuity levels were then assigned by using the emergency severity index (ESI). ESI is a standardised ED/UCC triage system confirmed to be a reliable and valid triage system in the USA to determine the different acuity levels on each patient’s entry into the service.13 Patients with potentially higher levels of acuity (eg, ESI 1 and 2) are routed to a physician immediately. Physician discretion is employed to determine if these patients need to transfer to the ED for further emergent evaluation and treatment. Those patients at ESI levels 1 and 2 who were not sent to the ED remain in the urgent care workflow. Patients who had no overcrowding statuses estimated due to incomplete collection of operational variables at the time of registration were excluded in this study.

Study protocol

This study was carried out from 24 February 2014 until 16 March 2014. During these 21 days, all physicians, advanced practice providers (APP), charge nurses, flow coordinator nurses and triage nurses were called separately every 2 h by a dedicated UCC clerk and asked to report their perception of the current UCC crowding status. The UCC clerk was blinded to this study. The perceptions of UCC overcrowding were rated on a 0–100 mm visual analogue scale (VAS). UCC overcrowding was considered to be true if the score on the VAS ≥50 and was considered severely overcrowded if the score on the VAS ≥70. An average UCC overcrowding score was then calculated. Since no UCC overcrowding scale was reported earlier, our study overcrowding score was multiplied by a factor of 2 in order to match an ED overcrowding scale that is widely used nationally.12 A score ≥100 was considered overcrowded and ≥140 was considered severely overcrowded. Therefore, three different crowding statuses were considered: not -overcrowded, overcrowded and severely overcrowded.

A UCC opens at 6:00 and closes around 23:00 during weekdays. During the weekend, a UCC opens at 6:00 but closes at variable times depending on the volume of patients presenting during the course of the day. A UCC triage ends at 22:00. Patients who present after 22:00 are redirected to the ED for further evaluation and treatment. A UCC closes after the last patient's disposition, which is usually around 23:00. The perception of UCC crowding status was queried eight times each day during weekdays at 7:00, 9:00, 11:00, 13:00, 15:00, 17:00, 19:00 and 21:00 separately. During the weekend, queries occurred at 7:00 and then every 2 h until the UCC closed. Patients who registered between 6:00 and 7:00 were considered under the not-overcrowded category.

At the same time, provider perceptions of UCC crowding were asked by the UCC clerk, and all variables were also recorded simultaneously by that clerk who did not participate in this study. The clinical or operational variables considered to potentially affect UCC crowding were collected after discussion with a group of those with operational expertise. A scoring tool to determine UCC crowding was then derived from the study that we named SONET. Additionally, 1000 sample randomised data sets were employed to validate the study internally by using bootstrap methods.

Variables

The total number of UCC beds was used as a constant in this study. All the other clinical or operational variables, such as the total number of patients at UCC, the number of patients in the waiting room, the number of attending physicians, APPs and nurses on duty, the number of patients with different ESI levels, and the longest wait time of those patients in the waiting room at the time of scoring were also collected (see table 1). In order to potentially apply the SONET scoring system to different UCC settings, several indices were calculated as well. The total patient index was the total number of patients at UCC divided by the number of UCC beds. The waiting room patient index was the number of patients in the waiting room divided by the number of UCC beds. The results pending patient index was the total number of results pending for patients (eg, patients already seen by healthcare providers at UCC and then placed in the result pending area) divided by the number of active patients (eg, active patients were the total number of UCC registered patients less the number of patients in the waiting room). The physician index was the total number of patients at UCC divided by the number of physicians on duty. The nurse index was the total number of patients at UCC divided by the number of nurses on duty. The APP index was the total number of patients at UCC divided by the number of APPs on duty.

Table 1

Clinical and operational variables and indices collected in the UCC overcrowding study

Variables
The total number of patients at UCC	The number of patients already transferred to other facilities in the past 2 h	The number of physicians on duty
Total number of patients in the waiting room	The number of patients with ESI 1 or 2 transferred to other facilities in the past 2 h	The number of nurses on duty
The number of patients in the results pending area	The number of patients waiting to be transferred to other facilities	The number of APPs on duty
The number of patients with different assigned acuity levels (ESI 1, 2, 3, 4, 5)	The number of patients waiting to be discharged	The number of triage nurses on duty
The number of patients with different assigned acuity levels in the waiting room (ESI 3, 4, 5)	The longest wait time among patients in the waiting room expressed in hours
Index
Total patient index	Results pending patient index	Waiting room patient index
Nurse index	APP index	Physician index

The number of patients in the results pending area refers to the number of patients who had already been seen by a healthcare provider and were awaiting results of diagnostic testing. After the initial provider interview and physical examination, these patients are relocated from the examination room to the results pending area. The total patient index is the total number of patients at UCC divided by the number of UCC beds. The results pending patient index is the total number of patients residing in the results pending status (eg, patients already seen by healthcare providers at UCC and then transitioned to the results pending area) divided by the number of active patients (eg, active patients were the total number of patients in UCC less than those patients in the waiting room). The waiting room patient index was the number of patients in the waiting room divided by the number of UCC beds. The nurse index was the total number of patients at UCC divided by the number of nurses on duty. The APP index was the total number of patients at UCC divided by the number of APPs on duty. The physician index was the total number of patients at UCC divided by the number of physicians on duty.

APP, advanced practice provider; ESI, emergency severity index; UCC, Urgent Care Center.

Outcome measurement

The SONET score was derived after the study was completed and retrospectively entered into the study data. All patients during the study period were assigned to have SONET scores at the time of their registration in the UCC and stratified into three different crowding categories. Patients who registered at UCC with incomplete data were excluded from the study as their individual SONET scores could not be calculated.

In order to know whether UCC overcrowding potentially affects UCC operational efficiency, length of stay (LOS) and the number of left without being seen (LWBS) patients were used as markers for UCC efficiency measurements. UCC LOS refers to the interval of time starting with an initial UCC patient registration and ending at the point when a patient is physically discharged from the UCC track board. For LWBS patients, the LOS was calculated as the interval of time starting with an initial UCC registration and ending at the point that no response to a call for further service was documented. We performed three calls to every LWBS patient in a 20 min interval. If no response was received after the third call, the patient was considered LWBS and the time of the first call was recorded as the documented time of no response. All patients registered for UCC services during the study period were included in the data analysis. Patient care outcomes were compared among these three groups (not-overcrowded, overcrowded and severely overcrowded groups).

Data analysis and statistics

A linear regression model was applied and the independent operational variables that could affect UCC overcrowding status scores were determined. Correlation coefficient (r) was analysed on each operational variable with its scatter plot drawn. Variables that had strong correlation (r>0.6) with UCC crowding were chosen for linear regression analysis. Variance inflation factor (VIF) quantifies the severity of multicollinearity in the regression model analysis, thereby providing an index to estimate whether the regression coefficient is increased due to collinearity. Operational variables with high VIF (>10) were considered as having collinearity and were therefore excluded from the regression analysis.14 15 A formula was then generated based on the regression coefficient of each independent operational variable and an UCC crowding score was calculated. A bootstrap technique that randomised 1000 samples was used to internally validate the study score accuracy.

Considering the operational significance of determining UCC overcrowding status, the SONET score was divided into three categories: not overcrowded (score<100), overcrowded (score between 100 and 140, including 100 but not including 140), and severely overcrowded (score≥140). Patients were automatically assigned to three groups based on ED overcrowding scores at the time when a specific patient registered for services in the UCC. To compare the differences between LWBS, and LOS at UCC relative to the different UCC overcrowding status groups, analysis of variance (ANOVA) with Bonferroni correction was used to analyse differences between groups.

All statistical analysis was performed using STATA V.12 (College Station, Texas, USA) and a p<0.05 was considered a statistically significant difference.

Results

Derivation of SONET scoring system

The prospective pilot study was performed from 6:00 on 24 February 2014 until 19:00 on 16 March 2014, which included 15 weekdays and 6 weekend days. The UCC closes operations at different times during the weekends resulting in 36 data sets collected at different time points. Therefore, there were a total of 134 data sets collected, resulting in a data completion rate of 85.9% (134/156). Among these 134 time points, the UCC was determined by healthcare provider perceptions to be below the not-overcrowded threshold 57.46% (77/134) of the time. The UCC was determined to be below the overcrowded threshold 26.12% (35/134) of the time and below the severely overcrowded threshold 16.42% (22/134) of the time.

Results of linear regression showed only four variables that can be considered independent risk factors affecting the UCC crowding status. These are the total number of patients, number of results pending for patients, number of patients in the waiting room and longest wait time of patients in the waiting room. Other variables reached no statistical significance, had no correlation with overcrowding or had significant collinearity with a VIF (variance inflation factor) greater than 10. In order to suitably apply the tool with respect to different UCC settings, the total patient index and waiting room patient index were used. Therefore, a UCC crowding scoring formula (SONET) was derived and is defined as:

SONET Score=24.5×total patient index+58.1×waiting room patient index+2.7×number of results pending for patients+12.2×the longest time in hours of patient in the waiting room+32.4. (in short form: SONET Score=24.5T+58.1WI+2.7R+12.2L+32.4 (TWIRL), where T indicates the total patients index, WI indicates the waiting room patient index, R indicates the number of results pending for patients in a UCC, and L indicates the longest time in hours of patients in the waiting room). SONET score ≥100 is considered UCC overcrowded and score ≥140 is considered severely overcrowded.

Using the average perceptions of UCC crowding status among different healthcare providers as a ‘gold standard’ demonstrated strong inter-rater reliability between the SONET scores and provider perceptions when compared within the three different crowding statuses (not overcrowded, overcrowded and severely overcrowded, κ=0.6446). Internal validation using bootstrap methods showed similar results (data not shown).

A total of 3565 patients were registered to receive services in the UCC during the study period. Excluding patients who had no SONET scores calculated due to incomplete collection of operational variables at the time of registration, 3139 patients were enrolled in data analysis. The general information of these patients is shown in table 2.

Table 2

General information of patients in the study

Age (year±SD)	41.97±15.57 (95% CI 41.43 to 42.52)
Gender (male, %)	46.70
Level of acuity (% (n))
ESI-1	0.16 (5)
ESI-2	5.61 (176)
ESI-3	24.94 (783)
ESI-4	59.64 (1872)
ESI-5	8.00 (251)
Unknown	1.66 (52)
Disposition (% (n))
Discharged	89.58 (2812)
Admitted	4.17 (131)
LWBS	1.94 (61)
Average time intervals (min±SD)
From patient arrival to triage	7.9±7.0 (95% CI 7.6 to 8.1)
From patient arrival to placement in an examination room	42.6±41.4 (95% CI 41.2 to 44.1)
From patient arrival to patient initial encounter with a healthcare provider	75.9±56.6 (95% CI 74.0 to 77.9)
From patient arrival to disposition (discharge vs admit) rendered	132.5±82.7 (95% CI 129.6 to 135.4)
From patient arrival to patient departure from UCC	151.6±89.5 (95% CI 148.5 to 154.7)

ESI, emergency severity index; LWBS, left without being seen; UCC, Urgent Care Center.

In order to determine whether the UCC overcrowding status could affect UCC operational efficiency and safety, LOS and LWBS were investigated. Patients registered at a UCC triage during the study period were assigned to three different UCC crowding statuses determined by SONET scores (N: not-overcrowded; O: overcrowded; and S: severely overcrowded).

The average LOS at UCC under each crowding status determined by SONET reached statistically significant differences between groups. Similar results were found when patients were further subdivided into the different ESI level groups (table 3). The more severe the crowding score in the UCC, the longer the average LOS of all patients, especially those triaged to ESI levels 3, 4 and 5. When analysing only discharged patients, similar results were found with statistically significant differences among groups (see online supplementary appendix table). To more accurately determine the effects of UCC crowding status on delayed patient care, LOS was divided into several segments. The segments were time spent at triage, wait time for an available examination room, wait time to arrival of a healthcare provider, and wait time to disposition (table 4). The results of our study showed that the most significant delay in care occurred during the period while patients awaited an available examination room. No significant difference was noted after patients were initially seen by the healthcare providers.

Table 3

The average LOS of patients who registered at UCC as a function of relative acuity status and SONET score

	Average LOS of patients at UCC (min±SD, number of patients)
	All patients	ESI-1	ESI-2	ESI-3	ESI-4	ESI-5
Not-overcrowded	130±80 (1581)	57±67 (2)	59±56 (69)	136±95 (385)	139±73 (981)	93±47 (125)
Overcrowded	165±88 (957) *p<0.001	58±55 (3) *p=0.973	71±62 (58) *p=0.733	170±102 (242) *p<0.001	181±79 (549) *p<0.001	139±53 (87) *p<0.001
Severely overcrowded	186±97 (601) †p<0.001		57±48 (49) †p=0.699	192±114 (156) †p=0.112	208±75 (342) †p<0.001	187±83 (39) †p<0.001

*Comparison between not overcrowded and overcrowded groups.

†Comparison between overcrowded and severely overcrowded groups.

ESI, emergency severity index; LOS, length of stay (time interval from initial patient registration to departure from UCC); UCC, Urgent Care Center.

Table 4

Patient encounter average time intervals as a function of relative crowding status determined by the SONET score

	Average time spend of patients at different phases (min±SD)
	Arrival to triage	Triage encounter	Triage to patient placed in an examination room	Patient placed in an examination room to patient seen by a healthcare provider	Patient seen by a healthcare provider to disposition decision	Disposition decision to patient departure from UCC
Not-overcrowded	6.8±6.4	2.3±2.2	17.4±25.7	30.3±30.7	54.8±57.5	19.6±17.4
Overcrowded	7.9±6.8*p<0.001	2.5±2.2*p=0.097	38.5±35.6*p<0.001	39.4±33.8*p<0.001	56.1±63.0*p=1.000	21.0±20.8*p=0.267
Severely overcrowded	10.6±8.1†p<0.001	2.8±2.5†p=0.210	60.9±52.7†p<0.001	31.3±30.0†p<0.001	57.6±69.2†p=1.000	21.3±21.0†p=1.000

*Comparison between not overcrowded and overcrowded groups.

†Comparison between overcrowded and severely overcrowded groups.

UCC, Urgent Care Center.

LWBS data were collected every 2 h. The numbers of LWBS patients were 0.28±0.69 every 2 h if UCC was under a not-overcrowded status, 0.64±0.98 when at an overcrowded status, and 1.00±0.97 when at a severely overcrowded status. The results show that the numbers of LWBS patients were associated with the severity of UCC crowding as determined by the SONET scores but were not sufficiently powered to reach statistical significance (p>0.05).

Discussion

Providing urgent care services to meet the needs of the evolving healthcare consumer is gaining considerable interest in the industry. The number of UCC patients has increased substantially every year, resulting in the potential for UCC saturation and resultant overcrowding. To date, no UCC overcrowding estimation tool was available.16 17 In order to maintain a high standard of clinical and operational performance in the urgent care setting, assessment of UCC overcrowding is critical to effective management. Much research has been done on ED overcrowding, but minimal attention has been paid to overcrowding as it relates to UCC workflow.18–20 Our institution operates both an ED and a UCC at different locations with a different triage system providing us an opportunity to investigate overcrowding at each discreet location. In this study, a UCC overcrowding estimation tool (SONET) was derived that also showed the prolonged average LOS and increased number of LWBS patients linked closely with the severity levels of UCC overcrowding.

Since no UCC overcrowding tool has been reported, the operational variables chosen for deriving our UCC overcrowding tool were gleaned from either expert opinions or the experiences obtained from ED overcrowding studies.12 21 22 Twenty different operational variables and five indices were included in this derivation study (see table 1) in order to match the requirements of the different UCC settings. The majority of these variables were similar to the ones used in ED overcrowding studies,12 except (1) that the numbers of patients triaged to ESI levels 1 and 2 were not considered due to significantly fewer presentations of these patients to the UCC, resulting in insufficient power to perform statistical analysis; (2) for the numbers of critical care patients which would be transferred to intensive care settings relatively quickly. On the other hand, the number of APPs on duty and the number of patients waiting in the results pending area were added for investigation particularly in this study because (1) the majority of UCC settings have APPs and (2) the majority of UCC patients do not present with conditions requiring a monitored bed. The overwhelming majority of patients presenting to a UCC can be safely managed in a non-monitored area while awaiting diagnostic results and/or receiving medications, thereby releasing examination beds for new patients.

Our results showed that four different independent variables could affect the UCC overcrowding status. These variables include the total patient index, the number of results pending for patents, the waiting room patient index and the longest time in hours of patients in the waiting room. Two variables (total patient index and the longest time in hours of patients in the waiting room) have also been used to evaluate ED overcrowding in previous studies.12 The number of patients triaged at an acuity level of ESI-3 or its equivalent in the waiting room has been shown to affect ED overcrowding in previous studies. These patients accounted for the majority of patients waiting for an initial provider encounter when the ED was determined to be overcrowded.23 24 Different conditions may occur in the UCC setting. The majority of UCC patients in the waiting room will be ESI-4 and ESI-5 level patients. Considering that ESI-1 and ESI-2 patients will be transferred out of a UCC to a higher acuity setting, ESI-3 patients are therefore the highest priority patients to be seen in the average UCC. It is therefore appropriate to consider the waiting room patient index as an independent variable for UCC overcrowding evaluation. The number of results pending for patients is another variable that is similar with respect to vertical flow patients at an ED.25 Briefly, patients who present to an ED and are determined not to require a monitored bed are often processed through a pathway involving minimal time spent in an exam room followed by the majority of their time in a results pending area awaiting diagnostics, medications delivery and re-evaluation. This is a recognised method to reduce ED overcrowding and has been reported in other studies.25–27 In a busy UCC, this method is also employed to effectively manage UCC patient flow.

SONET was derived in this study to estimate UCC overcrowding. Three different levels of crowding were developed to include severely overcrowded, overcrowded and not overcrowded. The ranges of the SONET score for the different crowding statuses match those of NEDOCS (national emergency department overcrowding study), which is widely used nationally.12 UCC workflow is considered to be efficiently managed at an appropriate level when the SONET score falls under the not overcrowded threshold. When the overcrowding threshold is approached, UCC and hospital administrators are alerted of the high potential for severe overcrowding and to employ predetermined actions to avoid reaching a severely overcrowded status. When operational outcomes were measured in this study, it confirmed the importance of dividing relative overcrowding into these three categories. The number of LWBS patients and the average LOS of ESI levels 4 and 5 patients increased with the severity of UCC crowding. There was an average of 22 min increase in ESI level 3 patients when a UCC was deemed to be severely overcrowded as compared with a determination of overcrowded, though no statistically significant difference was appreciated. This was in part due to a tendency for more ESI level 3 patients being transferred out of a UCC under severely overcrowded conditions (data not shown). As previously mentioned, the average LOS among ESI levels 1 and 2 patients was not a contributing factor in this study as this cohort of patients is not treated in the lower acuity setting of an UCC. When total LOS is viewed as a function of relative crowding status, significantly prolonged delay to patient placement in an exam room was notable and is consistent with previous reports.26 27

Overall, a novel tool is derived to determine UCC overcrowding status and our findings also show that the severity of overcrowding could link to the negative patient outcomes. Based on the preliminary results of this study, a multicentre prospective study that focused on external validations and outcome measurements in different UCC settings has already been started.

Limitations

This study was performed in a single urban UCC affiliated with a publicly funded hospital system which could inevitably have population selection bias and limit its use in a more general setting. Considering that the study was performed in a relatively high volume UCC setting, this crowding estimation tool might only accurately reflect conditions typically encountered in a similar setting. In addition, the study facility has an emergency psychiatric unit which directly and indirectly accepts patients with urgent and emergent psychiatric conditions. As such, very few patients with psychiatric problems present to a UCC, resulting in a potential bias in terms of population selection. Therefore, the results of this study need to be validated in a multicentre study involving different UCC settings and populations. The operational variables chosen in this study were based on previous ED overcrowding studies and expertise recommendations, as such other variables that potentially affect UCC crowding might have been missed. During our study period, the process of triaging a low acuity (ESI levels 4 and 5) ED patient to a UCC when the ED is determined to be severely overcrowded was not yet initiated. Therefore, the number of patients transferred from an ED to a UCC was not considered a risk factor impacting UCC crowding. Furthermore, consideration of average LOS and numbers of LWBS patients as the only patient care outcome measurements may not be enough to determine the most accurate association to UCC crowding. Other patient care outcome variables such as 72 h UCC/ED returns, patient satisfaction, and nosocomial accidents will be included in our ongoing multicentre validation study.

Conclusion

An overcrowding estimation tool (SONET) derived in this study might be used to determine relative crowding status in a high volume UCC setting. The study also showed that UCC overcrowding might be associated with negative patient care outcomes.