Adherence to Universal Travel Screening in the Emergency Department During Epidemic Ebola Virus Disease.

BACKGROUND: During the 2014 West African Ebola Virus Disease (EVD) outbreak, the U.S. Centers for Disease Control and Prevention recommended that all emergency department (ED) patients undergo travel screening for risk factors of importing EVD.
OBJECTIVES: We sought to determine the overall adherence rate to the recommended travel screening protocol and to identify factors associated with nonadherence to the protocol.
METHODS: We conducted a multicenter, retrospective analysis of adherence to the travel screening program in an academic hospital and three affiliated community hospitals. A regression model identified patient and hospital factors associated with nonadherence.
RESULTS: Of the 147,062 patients included for analysis, 93.7% (n = 137,834) had travel screenings completed. We identified several characteristics of patients that were most likely to be missed by the screening protocol-patients with low English proficiency, patients who arrive via ambulance or helicopter, and patients with more severe illness or injury based on initial triage acuity.
CONCLUSIONS: These findings should be used to improve adherence to the travel screening protocol for future emerging infectious disease threats.

Background

United States emergency departments (EDs) are at the crossroads of medicine and public health. They are a vital component and safety net of the health care system and a frequent access point for those with serious illness (1). The ED has also become an important site for disease surveillance (2). Epidemics such as plague, pandemic influenza, severe acute respiratory syndrome, and human immunodeficiency virus have all demonstrated the catastrophic international spread of emerging infectious diseases (3). With increasing globalization, epidemics are no longer isolated, but instead free to cross borders and oceans with alarming ease (4). A substantial proportion of the care for travelers and immigrants is delivered in the ED (5). Consequently, the ED shoulders an important burden of rapidly identifying and isolating patients with the potential to import epidemic emerging infectious diseases—as was the case during the 2014 outbreak of Ebola Virus Disease (EVD) in West Africa (6).

EVD is extremely low in prevalence in the United States, but the implications for failure to identify a case are exceedingly grave. For example, a case of missed EVD that was discharged from an ED in Dallas, Texas in September of 2014 made national news when it placed a community at risk and spread EVD to two health care workers 7, 8, 9. Based on the U.S. Centers for Disease Control and Prevention (CDC) recommendations entitled “Identify, Isolate, Inform” during the fall of 2014, screening for at-risk travel and personal exposures should be performed on all patients presenting to the ED at the time of registration and triage (6).

As a result of the CDC recommendations, the University of Colorado Hospital and three affiliated community hospitals instituted a Universal Travel Screening (UTS) protocol to identify those patients at risk for importing EVD in the ED. The protocol mandates screening of all comers to the ED for 1) travel to affected countries in the previous 21 days and 2) contact with infected persons in the previous 21 days (Figure 1 ). Nursing or registration staff record responses to both travel screening questions at the time of triage, assisted by electronic medical record (EMR) reminders that prompt staff to complete the screening.

Figure 1

Emergency department Universal Travel Screening protocol based on CDC's “Identify, Isolate, Inform” guidelines. CDC = Centers for Diseases Control and Prevention; ED = emergency department; EVD = Ebola Virus Disease.

Objectives

Despite CDC recommendations and the presumed importance of this screening, there is little known regarding the success of such travel screening programs. In one multicenter study in New York City, mystery patients with simulated measles and Middle East Respiratory Syndrome who presented to the ED during unannounced drills were significantly less likely to be given a mask and then isolated when travel screening questions were not asked at triage (10). Due to the low prevalence of EVD in the United States, it has not been possible to quantify the accuracy of screening. However, even a test with 100% sensitivity will fail the system if the test is not correctly applied to all comers. Unless the UTS protocol is applied to every patient, we may fail to identify individuals who import and transmit emerging infectious diseases such as EVD in our hospitals and communities. Our preliminary observations suggest that a number of patients each day do not have completed travel screening during triage despite the protocol and computerized reminders in the EMR.

Although the global threat of the 2014 West African Ebola outbreak has abated, understanding the factors associated with nonadherence to travel screening will allow for process improvement and enhanced readiness for future emerging infectious disease threats. An analysis of the UTS program is therefore needed to understand its scope and limitations. The specific aims of this study are to 1) quantify the adherence to UTS protocol and 2) identify individual patient and hospital-level factors associated with nonadherence.

Methods

Setting

We performed a multicenter chart review of all adults and children who presented to the ED to determine the overall adherence to UTS and analyze factors associated with nonadherence. The study was conducted in four affiliated hospitals in Colorado between December 1, 2014 and May 31, 2015. Dates were chosen to correspond with the height of the EVD outbreak and the maturation of a stable UTS protocol at our centers. No specific patient populations were excluded because UTS by definition applies to all comers.

The four participating sites are located in three different cities and include an academic tertiary referral hospital and three community-level hospitals, with annual ED volumes ranging from 25,000 to over 110,000 encounters per year. Notably, the hospitals are staffed by three separate physician groups—one hosting a residency program—with different administrative structures. The catchment area of the academic hospital includes a large immigrant and refugee population and also routinely takes patients being transferred from a major international airport.

This study was approved at all sites by the Colorado Multiple Institutional Review Board.

Design

A customized, automated medical record abstraction report was generated using Epic (Madison, WI) to gather data on patient demographics (age, sex, and race/ethnicity), primary language, insurance, triage category, method of arrival, disposition from the ED, and presence of trauma, as well as the recorded answers to the two UTS protocol questions. Primary language was categorized into English, Spanish, and other. Insurance status was grouped by private insurance, public insurance (i.e., Medicare, Medicaid), and private pay. Hospital-level factors such as ED and hospital census, ED boarding statistics, and shift characteristics were matched with each patient record using the date, time, and site of encounter.

Patient encounters were deemed adherent if a valid “yes” or “no” answer was recorded for both UTS questions for that visit; encounters were coded nonadherent if one or both of the UTS questions were left blank or recorded as “unable to screen.” Due to the lack of subjectivity in our definition of nonadherence, the classification of adherence was automated from the report. One investigator reviewed 200 random encounters to ensure the accuracy of this method.

To achieve high precision (95% confidence limit of 1%), which would allow us to detect statistical separation of the de facto adherence rate from the a priori adherence rate of 100% (primary outcome), 4000 patients were needed. However, a much larger sample size was needed for logistic regression analysis of factors associated with nonadherence. To allow for a model with sufficient power for up to 10 variables with an average of three discrete observations per variable, 6000 nonadherent encounters would be needed. Based on an estimated adherence rate of 95% from previous observation, a minimum sample size of 120,000 encounters was chosen. With an expected monthly census of approximately 20,000 encounters for the four hospitals, a 6-month study period was selected.

All data management tasks, including data cleansing, recodes, distribution assumption checks, and descriptive statistics were performed using SPSS version 23 (IBM Corp., Armonk, NY). The adherence rate and the logistic regression analyses with logit link were performed in R 3.3.1 (R Core Team, 2016). The adherence rate was calculated by dividing all adherent ED encounters by the total number of ED encounters seen across all four hospitals.

The goal of the regression models was to identify a set of predictors that maximized prediction accuracy and model fit. Prediction accuracy was calculated using a proportion with correct predictions in the numerator and all predictions in the denominator. Model fit was assessed by comparing the residual (unexplained) variance difference between models as well as the Akaike information criterion (AIC). Model selection was also guided by predictor importance analysis using the varImp() function from the “caret” package in R (11). The vaImp() function uses absolute value of the t-statistic for each model parameter for importance score calculations that can vary from 0 to 100.

Additionally, an “effect size” measure (odds ratio) was added to provide context for the size of difference in addition to statistical significance. Statistical significance (p-value) is sensitive to sample size, and with large sample sizes, small differences between groups become statistically significant even if the difference between the groups is small (small effect size).

Results

The resulting sample size for the descriptive and inferential analysis was 147,062 ED encounters during the study period. Table 1 displays the demographics of ED patients at all four sites during the study period.

Table 1

Demographics of ED Patients During Study Period, December 2014–May 2015

Demographic	Adherent	Nonadherent	p Value
	n = 137,834 (93.7%)	n = 9228 (6.3%)
Sex
Female	77,160 (56.0%)	4648 (50.4%)	< 0.001∗
Race
White	86,412 (62.7%)	5774 (62.6%)	0.814
Black	21,313 (15.5%)	1381 (15.0%)	0.200
Asian	2218 (1.6%)	179 (1.9%)	0.015∗
Age
Median (IQR)	34.0 (30)	44.0 (33)	< 0.001∗

ED = emergency department; IQR = interquartile range.

Significant alpha < 0.05.

Adherence Rate

Out of 147,062 patients in our study, 9228 were not screened per the UTS protocol, which resulted in an overall adherence rate of 93.7% (95% confidence interval 93.6–93.8%). This was statistically significant (p < 0.0001) when compared with the a priori adherence threshold of 100% to ensure universal screening.

Individual Factors

Table 2 shows the association of the included individual factors with nonadherence to UTS. The nonadherent group had a significantly and meaningfully higher proportion of patients in Emergency Severity Index (ESI) 1 and 2—high acuity triage categories—whereas the adherent group had a higher proportion of individuals in the ESI 4 and 5 categories. The nonadherent group also had a larger proportion of individuals that were admitted to inpatient services.

Table 2

Univariate Analysis of Individual Factors Associated With Nonadherence to Universal Travel Screening

Individual Factors	Adherent	Nonadherent	Effect Size∗
	n = 137,834 (93.7%)	n = 9228 (6.3%)	(95% CI)
Insurance Status
Uninsured	11,119 (8.1%)	847 (9.2%)	1.15 (1.07–1.24)
Language
Non-English	8857 (6.4%)	839 (9.1%)	1.46 (1.35–1.57)
ESI Level
1	382 (0.3%)	288 (3.1%)	11.59 (9.93–11.53)
2	19,446 (14.1%)	2730 (29.6%)	2.56 (2.44–2.68)
≥ 3	118,006 (85.6%)	6210 (67.3)	0.35 (0.33–0.36)
ED Disposition
Discharged	114,796 (83.3%)	6684 (72.4%)	0.53 (0.50–0.55)
Admitted	19,531 (14.2%)	1953 (21.2%)	1.89 (1.81–1.99)
Arrival Method
Back door†	2160 (23.4%)	7068 (76.6%)	16.27 (15.48–17.11)
Trauma‡Alert/Activation	93 (0.1%)	73 (0.8%)	11.81 (8.69–16.06)

CI = confidence interval; ESI = Emergency Severity Index; ED = emergency department.

Arrival method: back door includes arrivals by ambulance, police, fire, or helicopter.

Odds ratio (OR) was used as a measure of effect size. OR 1.68 (small), OR 3.47 (medium), OR 6.71 (large).

Trauma designations: alert and activation criteria describe moderate to severe trauma.

The largest difference between groups in this study was the arrival method—the adherent cohort was more likely to enter via front door (e.g., walk-ins through triage) and the nonadherent cohort via the back door (e.g., ambulance, helicopter, police transfer) (odds ratio 16.2).

Hospital Factors

Table 3 shows the association of hospital-level factors with nonadherence to UTS. There was a significant and meaningfully higher proportion of adherent cases on the days with low ED daily census. The nonadherent cases occurred slightly more often on days with higher ED daily admissions and with longer boarding times, though the effect size is small for ED daily admissions. Also, one site (ED 3) had significantly and meaningfully more adherent cases than nonadherent cases, demonstrating variability of adherence between departments.

Table 3

Univariate Analysis of Hospital Factors Associated With Nonadherence to Universal Travel Screening

	Adherent	Nonadherent	Effect Size∗
	n = 137,834 (93.7%)	n = 9228 (6.3%)	(95% CI)
Site identifier
ED 1†	46,728 (33.9%)	3584 (38.8%)	1.24 (1.19–1.29)
ED 2	47,307 (34.3%)	3366 (36.5%)	1.09 (1.05–1.15)
ED 3	16,613 (12.1%)	417 (4.5%)	0.34 (0.31–0.38)
ED 4	27,186 (19.7%)	1861 (20.2%)	1.03 (0.98–1.08)
ED daily census
Low (M = 95.29, SD = 13.28)	(97.5%)‡	(2.5%)‡	0.35 (0.31–0.38)§
Medium (M = 161.41, SD = 15.69)	(93.6%)	(6.4%)	1.04 (0.98–1.09)
High (277.25, SD = 24.22)	(93.1%)	(6.9%)	1.4 (1.34–1.47)
ED daily admissions	27.3 (10.6)‖	29.2 (9.2)‖	0.20¶ (1.75–2.17)
ED border time (min)	3210 (3055)∗∗	3403 (2838)∗∗	N/A††

CI = confidence interval; ED = emergency department; N/A = not applicable.

Odds ratio (OR) was used as a measure of effect size. OR 1.68 (small), OR 3.47 (medium), OR 6.71 (large).

Reference site.

Percentage of patients out of the total for ED Daily Census category.

Odds ratio is per 1-point increase in cumulative score.

Mean (SD).

Cohen's d effect size (small 0.2, medium 0.5, large 0.8).

Median (interquartile range).

No widely accepted measure of effect size for median tests are known to the authors.

Modeling Nonadherence

Beyond the exploration of factors affecting nonadherence, an objective of this study was to investigate a combination of factors that best predict nonadherence. We developed a number of logistic regression models to maximize the correct prediction rate of nonadherent cases (Table 4 ). We selected the group of final predictors based on 1) the overall prediction rate of the model, 2) the reduction of residual (unexplained) variance, and 3) the AIC model fit statistic, with prediction rate being the most important factor overall.

Table 4

Comparison of the Two Best Models for Predicting Nonadherence to Universal Travel Screening

Model	Prediction Rate	AIC∗
Model 1†
ESI + Method of Arrival + Primary Language	73%	53,942
Model 2
ESI + Method of Arrival	79%	54,189

ESI = Emergency Severity Index.

Akaike information Criterion (AIC) is a relative measure of quality of models. The model with the lowest AIC is preferred.

Preferred model.

Although a model that included only ESI level and method of arrival had a prediction rate of 79%, our final model consisted of three best predictors of nonadherence: triage ESI level, method of arrival, and primary language. The prediction rate of this model (73%) was slightly degraded from the two-variable model, but had better reduction of unexplained variance as described by the lowest AIC (AIC = 53,942 vs. AIC = 54,187, respectively).

Discussion

Based on our analysis, the adherence rate to the current UTS protocol was less than the desired 100% threshold. This confirms previous observations that ED staff failed to screen ED patients for risks for importing EVD with some frequency. Based on this information, we recommend targeted revisions to the UTS protocol to enhance adherence for the inevitable future emerging pandemic threats.

Regression analysis identified several statistically significant differences between the adherent and nonadherent groups. Many of these lacked large effect sizes but have plausible explanations. At the hospital level, factors such as increased ED daily census, hospital census, and boarder status are all proxies for the workload placed on ED staff. When the staff and resources are strained, it logically follows that human errors increase and adherence to UTS subsequently decreases.

There were several significant and meaningful (high effect size) differences between the adherent group and the nonadherent group regarding individual (patient) factors. The most meaningful predictors of nonadherence were: ESI levels 1 and 2, arrival through the back door (i.e., via ambulance), and non-English primary languages. The effect size of these predictors may be confounded. For example, it is plausible that patients with high-acuity presentations (ESI level 1 or 2) are more likely to arrive via the back door.

Patients arriving via back door may require urgent interventions, causing delays in completing UTS questions. Unlike those that walk in through triage and are seen by registration prior to entering the triage process, these patients are often registered simultaneously while their evaluation begins. If stabilization procedures are needed, UTS questions are likely to be deferred.

The same applies to patients who are severely injured or ill upon arrival, as represented by an ESI ≤ 2. These patients may require immediate stabilization that prevents screening upon arrival. Alterations in mental status and need for intubation are undoubtedly barriers to verbal questioning.

We posit that language barriers between the patient and the staff completing the screening also influence screening adherence, especially if in-person or telephone interpretation is not available immediately upon arrival. It is plausible that staff forget to return to the UTS questions once an interpreter is reached. Additionally, some patients with low English-language proficiency will decline interpreters, leaving further impediments to screening questions.

Although an adherence rate of 93.7% might be considered adequate for some screening programs, we suggest two arguments as to why this is low for UTS during the 2014 West African EVD outbreak. First, the gravity of missing a case of imported EVD puts many lives at risk. Second, the individual factors that we identified as most meaningfully associated with nonadherence could conceivably be more common in those patients most at risk for importing EVD. For example, a patient who contracts EVD in West Africa and then travels to the United States may not speak English and require ambulance transport to the hospital, where he or she may be triaged with an ESI level 1 or 2, depending on the severity of their presentation. This would imply that the patients most at risk for importing EVD are the ones that are least likely to be screened by the current protocol. Nonadherence may introduce a selection bias against high-risk groups in our ED population.

Of course, there may be other factors outside of a travel screening program that may prompt nurses and clinicians to rapidly identify and isolate a case of imported EVD. For example, we noted that of the 8866 nonadherent encounters that had a documented triage temperature, only 1.3% (n = 112) had a fever > 38.0°C, whereas 1.8% (n = 2493) had a fever out of the 137,824 adherent encounters with a documented triage temperature. That said, a protocol serves to standardize the approach so that the system does not rely precariously on clinician gestalt.

Recommendations

The development of a prediction model for nonadherence is useful for creating recommendations that will have the most impact on improving UTS adherence in our institutions. By focusing on the three variables contained in our model, we recommend:

Training staff to use interpreters as soon as feasible to complete UTS questions for patients with low English-language proficiency.

Requesting that Emergency Medical Services report both screening questions at the time of handoff to the ED team.

Adding an option to question friends and family who accompany those patients who are too injured or ill to answer for themselves, when available.

We also recommend further investigation of alternative criteria for screening those patients that are incapacitated and are not accompanied by another party who can answer on their behalf. To date, we are not aware of a protocol to do this.

Limitations

In this retrospective study, we were limited to data that were readily recorded and available in our EMR. There may be other, equally important factors that influence adherence to UTS that were not assessed. The authors defined variables of interest a priori prior to creating the automated chart abstraction tool to minimize this effect. It is also possible that screening was truly performed, but not recorded in the EMR in some cases, which would falsely lower our adherence rate. Because other triage data were largely complete in the EMR, unrecorded responses are likely negligible.

Although our multicenter study was conducted at four separate sites, the protocols for UTS were nearly identical, with only slight variations to accommodate different ED staffing, triage, and intake structures. This could affect generalizability to hospital systems using alternative protocols. However, because our sites used the recommendations of the CDC's “Identify, Isolate, Inform” guidelines that influence many other hospitals, it is reasonable to assume that these factors influence adherence across the country. Notably, there was variability even among our institutions in this study, with the hospital reporting the highest adherence also having the lowest annual ED volumes.

Lastly, we did not investigate the efficacy of the UTS protocol for identifying actual EVD cases. As EVD is very low prevalence, evaluation of the accuracy of screening would not be possible at our centers in the timeframe studied. Our findings are strictly related to the adherence to a UTS protocol, supposing that without applying the test to all comers, the accuracy of the screening tool is undermined. Therefore, our recommendations are directed at how the screening questions are applied but not at how the two screening questions function.

Conclusion

The Emergency Department serves an important surveillance role during times of emerging infectious disease threats. A UTS protocol based on CDC recommendations during the 2014 West African EVD outbreak must be applied to all comers to the ED to identify and isolate possible imported cases of EVD. The UTS protocol is only effective with improved adherence to screening. We recommend protocol improvements directed at those patients that were least likely to be screened—those that are acutely ill upon triage, that do not speak English, and that arrive via ambulance, helicopter, or other nonprivate transport.

Article Summary

Emergency departments (EDs) shoulder an important burden of identifying and isolating cases of emerging infectious disease that pose pandemic threats to our communities.

We conducted a study to assess adherence to standard ED travel screening protocol and to identify factors associated with nonadherence to the protocol in an effort to improve the process for future pandemic threats.

Although the EDs included had an overall adherence rate of 93.7%, several high-risk populations were selectively missed by the screening at undesirable rates—those with high-acuity triage scores, those that arrived by ambulance, and those that did not speak English.

Our findings can be used to improve travel screening protocols during future emerging infectious disease threats to avoid the unnecessary spread of disease to health care workers and our communities.