Adverse events in patients with return emergency department visits.

OBJECTIVES: This study describes the proportion of emergency department (ED) returns within 7 days due to adverse events, defined as adverse outcomes related to healthcare received.
DESIGN: Prospective cohort study.
SETTING: We used an electronically triggered adverse event surveillance system at a tertiary care ED from May to June 2010 to examine ED returns within 7 days of index visit. PARTICIPANTS: One of three trained nurses determined whether the visit was related to index emergency care. For such records, one of three trained emergency physicians conducted adverse event determinations. MAIN OUTCOME MEASURE: We determined adverse event type and severity and analysed the data with descriptive statistics, χ(2) tests and logistic regression.
RESULTS: Of 13,495 index ED visits, 923 (6.8%) were followed by ED returns within 7 days. The median age of all patients was 47 years and 52.8% were women. After nursing review, 211 cases required physician review. Of these, 53 visits were adverse events (positive predictive value (PPV)=5.7%, 95% CI 4.4% to 7.4%) and 30 (56.6%) were preventable. Common adverse event types involved management, diagnostic or medication issues. We observed one potentially preventable death and 58.5% of adverse events resulting in transient disability. The PPV of a modified trigger with a cut-off of return within 72 h, resulting in admission was 11.9% (95% CI 6.8% to 18.9%).
CONCLUSIONS: Our electronic trigger efficiently identified adverse events among 12% of patients with ED returns within 72 h, requiring hospital admission. Given the high degree of preventability of the identified adverse events, this trigger also holds promise as a performance measurement tool. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

Following an emergency department (ED) visit, patients often return with the same presentation. Previous studies have documented that up to 9% of patients discharged from the ED return within 14 days.1 2 Return visits are often not preventable because they are due to scheduled follow-up or progression of disease.3 4 There are some ED return visits, however, that are preventable because they are due to inadequate symptom control, misdiagnosis, inappropriate management or failure in the follow-up plan.3 5–7 Return ED visits are also important from the healthcare system perspective because they lead to increased ED crowding and healthcare costs. Thus, addressing factors associated with return ED visits could improve patient satisfaction, clinical outcomes and reduce costs.8 9

As a result of their importance, return ED visits have been proposed as triggers for quality assessment.10 ‘Triggers’ are clinical events indicating the possibility of an adverse event defined as a poor outcome caused by medical care.11 By scanning triggers, those responsible for monitoring quality and safety can identify cases for subsequent investigation. While it has been proposed for use, the trigger ‘return ED visits’ may be limited in its utility. As noted, many return visits are due to non-preventable reasons. From a quality perspective, only those return visits caused by adverse events are important. Thus, until research has been done to determine the proportion of visits due to adverse events, the utility of this trigger is unknown.

We recently implemented an electronic system enabling automated detection, investigation and statistical analysis of ED return visits. The objectives of this study were to determine what proportion of triggered return ED visits represented adverse events and what patient and visit factors predicted these adverse events. We felt this information would help validate the use of return ED visits as a quality indicator. It would also help us refine the use of our trigger system for monitoring quality in the ED.

Methods

Study design and setting

We conducted this prospective cohort study at a multi-facility academic hospital that has two tertiary care academic EDs which together treat more than 134 000 patients per year. The study was approved by The Ottawa Hospital Research Ethics Board and the board waived the need for written informed consent from the participants.

The study used our institution's Patient Safety Learning System. The Patient Safety Learning System is an electronic system designed to help identify and manage adverse events. It incorporates data from the following sources: (1) voluntary incident reporting from front line healthcare workers; (2) prospective surveillance by clinical observers; (3) morbidity and mortality rounds; and (4) electronic triggers such as those described by the Institutes of Health (eg, transfer to intensive care unit).10 For the purpose of this study, we used the Patient Safety Learning System's capability to electronically capture all return ED visits within 7 days and analysed these for adverse events. This created a comprehensive and automated electronic trigger. In this study, we did not use voluntary incident reporting as a data source.

Eligibility criteria

All patients having an ED encounter between 9 May and 13 June 2010 were eligible for the study. We programmed the Patient Safety Learning System to identify patients returning to the ED within 7 days regardless of the site of their index visit. This could include patients admitted on the index visit but who were discharged prior to the 7-day evaluation period. We also evaluated the electronic trigger for patients who returned to the ED within 72 h and were subsequently admitted as recommended by a national consensus panel on ED quality indicators.12 We did not require informed consent for this study as we did not contact patients or impact their care.

Data collection procedures

We used an integrated event engine to continually scan the hospital's patient registration system for ED visits meeting our study criteria. When an event occurred, an HTML file was created and forwarded to the Patient Safety Learning System. The file contained the patient's name, medical record number, gender, date of birth and visit information for both encounters (dates, times, presenting complaints and discharge diagnoses).

Adverse event determination

We defined an adverse event as an adverse outcome related to the care received during the index visit. We considered the following types of problems as ‘related to ED care’: complications related to treatment or procedures, missed diagnoses, management errors at the time of discharge, suboptimal follow-up, and communication errors. We defined a diagnostic issue as not acting on documented signs, symptoms, laboratory tests or imaging or not ordering an indicated diagnostic test. A management issue was defined as a suboptimal management plan despite accurate diagnosis or based on an inaccurate diagnosis. We considered an unsafe disposition decision to be when a patient was placed at an unnecessary risk of experiencing death or major disability by being sent home. Suboptimal follow-up included problems with follow-up arrangements that led to the development of new symptoms, unnecessary prolongation of symptoms, an unscheduled return visit to the ED or a subsequent unscheduled hospitalisation. This could be due to inadequate availability of a follow-up appointment or due to inappropriate follow-up arrangements. We defined a medication adverse effect as occurring when a patient experienced a symptom related to a medication regardless of whether the medication was appropriately prescribed or taken. Finally, a procedural complication occurred when a patient experienced adverse consequences of a procedure.

The case file was initially reviewed by one of three senior ED nurses. These nurses received training on adverse event definitions and a structured approach to case review. The nurse reviewed the cases to identify those that were scheduled for medications or follow-up procedures, or which were due to completely unrelated injuries or ailments. These cases were removed from subsequent investigations and classifications. The nurse also identified patients who were seen upon return to the ED directly by a non-emergency physician (eg. a consulting service). These cases were excluded because the sparse documentation by these services at the time of the return ED visit was insufficient to conduct adverse event review. The nurse used the ED record of treatment for this categorisation and completed a free-text response to the prompt: ‘Please describe what happened’.

If the case was not rejected by the nurse reviewer, it was promoted to one of three ED physicians (LC, AP, SR) for further classification. These physicians received training in the Patient Safety Learning System functionality and the classification task. The reviewers used the medical record and their clinical judgement to determine whether the return visit represented an adverse event. The physician peer reviewers used a standardised, previously used, 6-point Likert scale that ranged from 1 (definitely due to the patient's underlying condition entirely) to 6 (definitely due to the medical management).1 13 14 Peer reviewers were required to demonstrate a concrete reason for the association with healthcare management in order for an outcome to be considered an adverse event. Those patients who experienced progression of disease were not considered to have experienced an adverse event.

Subsequently, the physician determined whether in their opinion the adverse events were preventable. We defined a preventable adverse event as an adverse event caused by a healthcare management problem such as a diagnostic issue, management issue, unsafe disposition decision or suboptimal follow-up. If a physician reviewer was uncertain of her review, then the case was discussed by all reviewers until consensus was achieved.

For cases representing an adverse event, the record underwent further classification as to the event's severity and type. This classification was performed by a single physician (NM) using standard approaches.15 All adverse events were reviewed by three physicians (NM, LC, AF) to ensure appropriate classification.

Statistical analysis

All Patient Safety Learning System information was linked for our statistical analysis to other hospital data using the patient's medical record number and ED visit date. Specifically, we obtained patient information describing past visit history and encounter information including encounter duration, Canadian Triage and Acuity Score (CTAS) score, tests performed and disposition.16

We described the baseline characteristics of our entire study population, those experiencing a return visit within 7 days and those experiencing an adverse event. We used medians and IQRs for continuous variables and frequency distributions for categorical variables. We report the proportion of adverse events with 95% CIs.

We performed two multivariable analyses comparing patients with and without adverse events. The first analysis was to determine risk factors associated with adverse events from the perspective of the index visit. This analysis could be useful for clinicians who wish to predict which patients are at higher risk of an adverse event or for health system analysts who wish to perform risk adjustments when comparing institutions. The second analysis was to determine risk factors associated with adverse events from the perspective of the return visit. This analysis could be useful for health system analysts for distinguishing return ED visits due to adverse events from non-care-related factors.

For both analyses, we followed the same general approach. We first performed a univariate analysis using the χ2 statistic (binary and categorical variables) and t statistics (continuous variables). We conducted multiple logistic regression using statistically significant variables and clinically sensible variables against the occurrence of an adverse event. The only difference between these analyses was the population. For the first analysis, we included the entire cohort. For the second, we only included the patients experiencing a return ED visit. All analyses were performed using SAS V.9.2.

Results

Figure 1 depicts study flow. There were 13 495 ED visits during the study. Of these, 923 (6.8%) were followed by a return visit within 7 days and were reviewed by a nurse. After nursing review, 712/923 (77.1%) records were excluded for various reasons, including: return to ED for unrelated reasons (n=182, 25.6%), clear progression of disease (n=163, 22.9%), patient left without being seen on return visit (no data available to evaluate outcome) (n=143, 20.1%) and returned direct to another clinical service (n=117, 16.4%). Overall, 211 of 923 cases (22.9%) underwent ED physician review for adverse event determination.

Figure 1

Patient flow for 13 495 emergency department (ED) visits, 9 May 9–13 June 2010.

Table 1 describes baseline characteristics of patients at the index ED encounter. Overall, the patients were middle aged and evenly distributed between men and women. The population was of high acuity (n=10 774, 79.8% CTAS 1–3) and a quarter arrived by ambulance. The majority were discharged home (n=10 616, 78.7%). Painful conditions involving the chest, abdomen and back were the most frequent discharge diagnostic categories.

Table 1

Baseline characteristics of patient population for all 13 495 patients and those who returned to the ED and those who suffered adverse events and preventable adverse events

Patient characteristic	All patientsN=13 495 (%)	Patients with return ED visits N=923 (%)	Patients with adverse events N=53 (%)	Patients with preventable adverse events N=30 (%)
Age
Median (IQR)	47 (31–65)	49 (32–67)	54 (34–73)	54 (34–74)
Sex
Female, n (%)	7124 (52.8%)	456 (49.4%)	25 (47.2%)	14 (46.7%)
CTAS
1	96 (0.7%)	10 (1.1%)	0 (0.0%)	0 (0.0%)
2	3615 (26.8%)	279 (30.2%)	21 (39.6%)	12 (40.0%)
3	7063 (52.3%)	491 (53.2%)	24 (45.3%)	15 (50.0%)
4	2377 (17.6%)	130 (14.1%)	8 (15.1%)	3 (10.0%)
5	344 (2.5%)	13 (1.4%)	0 (0.0%)	0 (0.0%)
Campus
Civic	6804 (50.4%)	461 (49.9%)	26 (49.1%)	16 (53.3%)
General	6691 (49.6%)	462 (50.1%)	27 (50.9%)	14 (46.7%)
Arrived by ambulance	3489 (25.9%)	287 (31.1%)	15 (28.3%)	8 (26.7%)
Previous inpatient/ED visit last 6 months	5246 (38.9%)	540 (58.5%)	29 (54.7%)	15 (50.0%)
Any radiology during ED visit*	5860 (43.4%)	360 (39.0%)	34 (64.2%)	20 (66.7%)
Consult requested on index ED visit	3759 (27.9%)	247 (26.8%)	15 (28.3%)	9 (30.0%)
ED disposition home/residence	10 616 (78.7%)	742 (80.4%)	47 (88.7%)	28 (93.3%)
Most common ED discharge diagnostic categories
Abdominal pain	667 (4.9%)	54 (5.9%)	4 (7.5%)	3 (10.0%)
Chest pain	571 (4.2%)	30 (3.3%)	3 (5.7%)	1 (3.3%)
Back pain	353 (2.6%)	24 (2.6%)	0 (0.0%)	0 (0.0%)
Cellulitis	292 (2.2%)	35 (3.8%)	2 (3.8%)	1 (3.3%)
Wrist/hand wounds	207 (1.5%)	10 (1.1%)	1 (1.9%)	1 (3.3%)
Mental and behavioural disorders	190 (1.4%)	28 (3.0%)	0 (0.0%)	0 (0.0%)
Congestive heart failure	87 (0.6%)	6 (0.7%)	3 (5.7%)	1 (3.3%)
Pharyngitis	82 (0.6%)	7 (0.8%)	2 (3.8%)	0 (0.0%)

*Radiology includes: chest X-ray, abdominal X-ray, X-rays of extremities, CT head, CT chest, CT abdomen and pelvis, ultrasound abdomen and pelvis.

CTAS, Canadian Triage and Acuity Scale; ED, emergency department.

Of the 923 return ED visits detected by the Patient Safety Learning System, 53 visits (5.7%, 95% CI 4.4% to 7.4%) were due to patients experiencing an adverse event. Over half of these (30, 56.6%) were deemed preventable. Table 2 describes adverse event type and severity. The most common types of adverse events were management issues, diagnostic issues and medication adverse effects. In terms of severity, most adverse events resulted in transient disability and there was one death. The adverse event resulting in death occurred in a patient with end-stage renal disease who was seen in the ED with abdominal pain. The discharge diagnosis was ‘renal failure secondary to fluid overload’. The patient returned 24 h later with septic shock due to ischaemic colitis. The patient died shortly thereafter as a result of sepsis. Figure 2 provides a narrative summary of a sample of adverse events. When we adjusted the electronic trigger to examine return ED visits within 72 h resulting in admission to hospital, we found 126 patients met these criteria and of these, 15 experienced adverse events (positive predictive value 11.9%).

Table 2

Adverse event (AE) types and severity for 53 AEs

	No. (and %*) of events
	AE	Preventable AE
	N=53	N=30
Event classification
Management issue	16 (30.2)	12 (40.0)
Diagnostic issue	15 (28.3)	13 (43.3)
Medication adverse effect	11 (20.8)	2 (6.7)
Procedural complication	10 (18.9)	3 (10.0)
Unsafe disposition decision	1 (1.9)	0 (0)
Severity of harm
Symptoms only	21 (39.6)	9 (30.0)
Transient disability	31 (58.5)	20 (66.7)
Death	1 (1.9)	1 (3.3)

*Denominator is total number of events for each column.

Figure 2

Examples of adverse events cases.

Our univariate analyses demonstrated that patients who suffered adverse events were older than the overall population (54 vs 47 years) and more likely to be men (n=28, 52.8% vs n=7099, 47.2%) (see online supplementary appendices A and B). A greater proportion of patients with previous inpatient or ED encounters within the last 6 months had adverse events compared with the overall population. A similar trend was noted for those who had received some form of radiological investigation during their index visit. Time of day of arrival was statistically significantly associated with the occurrence of an adverse event with a greater likelihood of an adverse event occurring if the patient arrived during the daytime period.

We used a multivariable logistic regression model to simultaneously control for several factors associated with adverse events in the entire cohort (n=13 495). The only variables independently associated with adverse events in this group were previous ED visit in the preceding 6 months (OR=1.77, 95% CI 1.00 to 3.13) and any radiology test performed during the index visit (OR=2.17, 95% CI 1.31 to 3.90). The following variables were not associated with adverse events: patient age, Charlson score and presence of diabetes. We further conducted a multivariate logistic regression analysis to determine predictors for adverse events among patients who experienced a return ED visit. The only variables independently associated with adverse events in this group were imaging studies at both index and return ED visit (OR=1.89, 95% CI 1.00 to 3.55) and being admitted following return visit within 72 h of index visit (OR=2.17,95% CI 1.12 to 4.20). Patient age was not associated with adverse events among those patients with return visits.

Discussion

In all, 7% of ED encounters at our institution were followed by a return visit within 7 days. We noted 6% of these return visits were adverse events (ie, they were related to healthcare provided in the ED). When we further refined the trigger to include only patients returning within 72 h resulting in hospital admission, the positive predictive value of the trigger increased to 12%. The most common contributors to these adverse events were management and diagnostic issues. Many adverse events were of moderate severity but one was associated with death. The preventability of these adverse events was high with more than half being deemed preventable upon peer review. We found that any radiological investigation upon index or return ED visit and returning within 72 h resulting in admission were associated with the occurrence of an adverse event. Possible reasons for this finding include that those patients who received radiological investigations may have been sicker or the emergency physician may have had less certainty as to the diagnosis on the index visit.

The purpose of an electronic trigger is to more efficiently identify events where patients experience healthcare-related harm. This results in a focused identification of cases where more indepth review can be conducted to identify quality of care issues. Returning to the ED after an initial visit has been described as a ‘life event’ which is important to patients and represents a potential gap in high quality care.11 Our results indicate that while in the majority of cases patients who return to the ED do not experience adverse events, those that do have a high proportion of preventability. Our data suggest that this electronic trigger is effective in terms of decreasing the burden of adverse event surveillance via health records review. We only need to review eight return ED visits within 72 h resulting in admission in order to identify one adverse event. And further, our findings are consistent with other studies evaluating electronic triggers such that we found a low flag rate (1% in our study) with a good positive predictive value (12%).17 Thus, for those who are interested in improving quality of care in the ED, this modified electronic trigger may serve as a more efficient method of adverse event detection.

We believe this population is representative of a Canadian tertiary care academic ED. The themes of adverse event type are similar to what has been documented in past ED based adverse event studies.1 13 18 19 In general, the prominence of diagnostic and management issues rather than procedural complications has been previously described as characteristic of emergency medicine practice, distinct from inpatient care.20–22

We chose 7 days as the cut-off for our electronic trigger in order to be more inclusive and enhance the sensitivity of the trigger tool. This was also based on our previous data which showed that most adverse events occurred within this time period for high acuity ED patients.13 The proportion of patients returning to the ED has been reported in a variety of different ED settings around the world. American academic and community EDs have observed 0.5%–3.0% of patients return to the ED within 72 h whereas for those patients discharged from observation units, the proportion who return increases to 10.0%.2 3 23 Internationally, a similar proportion of return ED visits within 72 h have been observed, from 0.8% in Spain to 5.5% in Singapore.6 7 When we adjusted our electronic trigger to the 72 h cutpoint resulting in admission to hospital as per previous national consensus recommendations, we found that 12% of returns were adverse events, thus increasing the positive predictive value of the tool.12 With the high degree of preventability identified for these adverse events, this trigger tool may also hold promise as a performance measure; however, this is yet to be evaluated.

Limitations

There were several limitations to this study. First, the number of adverse events detected was low and this decreases the likelihood of finding statistically significant associations with patient and ED visit predictor variables. This was a pilot study aimed at determining feasibility of the electronic trigger; hence, we screened a relatively high number of cases (13 495) over a short period of 2 months. This may be subject to unmeasured influences of staffing and secular trends. Second, it is possible that during the nursing review, the cases they rejected did in fact represent missed adverse events. This might imply that adverse event determination may require physician review of more cases as we have done in previous studies which is more resource intensive. Third, because this was a dual centre study, we were unable to detect returns to other institutions or deaths in the community. Our previous ED based adverse event studies have shown that the numbers of patients in these groups are very small.1 13 Fourth, we relied upon what was documented in the electronic health record to perform our adverse event determinations. This likely resulted in an undercounting of adverse events. In the case of patients who left without being seen, we did not have any data to evaluate the outcomes. When evaluating adverse events through examining the health record, we are subject to hindsight and outcome biases. Finally, preventability assessments can be subjective and we did not perform kappa values to determine inter-rater variability.

Clinical and research implications

If hospitals are considering active surveillance for ED quality indicators, it is clear that this method is superior to incident reporting for identifying important adverse events. During this study, the hospital's system for the voluntary reporting of patient safety incidents did not record a single event detected by the electronic trigger methodology, including the identified patient death. The electronic trigger system is a less resource intensive method than manual chart review and telephone interviews.

Further development of this system will allow us to pursue quality assurance initiatives for the ED which can provide timely feedback to ED clinicians and administrators on the proportion of return ED visits and the occurrence of preventable adverse events. By identifying high risk patient groups, we will be able to more strategically plan future patient safety interventions. Promising interventions for these populations include optimising discharge instructions, telephone follow-up and regular adverse event audits.3 8 9 24 Future research will be required to find the optimal time frame for return visits in order to have a manageable number of cases to review. We also plan to implement this on a broader scale where most emergency physicians practicing will be trained in adverse event determination and will have a rotating schedule of review of return ED visits.

Conclusions

Using the trigger of patients who return to the ED within 72 h resulting in admission allowed us to efficiently identify adverse events with a high degree of preventability. While the causes of these adverse events were varied, they do offer potential opportunities for quality improvement. Future work should focus on the evaluation of this tool's reliability and utility for making comparisons between EDs.