MONitoring Knockbacks in EmergencY (MONKEY) - An Audit of Disposition Outcomes in Emergency Patients with Rejected Admission Requests.

Background: Emergency Department (ED) clinicians commonly experience difficulties in referring patients to inpatient teams for hospital admission. There is limited literature reporting on patient outcomes following these complicated referrals, where ED requests for inpatient admission are rejected - which study investigators termed a "knockback". Purpose: To identify disposition outcomes and referral accuracy in ED patients whose admission referral was initially rejected. Secondary objectives were to identify additional patient, clinician and systemic factors associated with knockbacks. Selection and Methodology: Emergency clinicians prospectively nominated a convenience sample of patients identified as having knockbacks over two time periods (Jan-Feb 2020 and Aug 2020 to Jan 2021) at a tertiary Australian ED. Data were analyzed with a mixed-methods approach and subsequent descriptive and thematic analyses were performed.
Results: A total of 109 patients were identified as knockbacks. The referrals were warranted, with 89.0% of cases (n = 97) ultimately requiring a hospital admission. In 60.6% (n = 66) of the admissions, patients were admitted under the inpatient team initially referred to by the ED, suggesting referrals were generally accurate. The number of in-hospital units involved in the admission process and ED length of stay were positively correlated (0.409, p < 0.001). Patient factors associated with knockbacks include pre-existing chronic medical conditions and presenting acutely unwell. Analysis of clinicians' perspectives yielded recurring themes of disagreements over admission destination and diagnostic uncertainty.
Conclusion: In this patient sample, emergency referrals for admission were mostly warranted and accurate. Knockbacks increase ED length of stay and may adversely affect patient care. Further focused discussion and clearer referral guidelines between ED clinicians and their inpatient colleagues are required.

Introduction

Emergency Department (ED) doctors are routinely required to work alongside their inpatient counterparts when referring patients for hospital admission, a process known as referrals or handoffs. Despite these interactions being commonplace, it is often reported to be a frustrating process with difficulties being reported as frequently as 56% of the time.1 Ineffective and incongruent communication between physicians has been identified repeatedly as a primary contributor to adverse patient events.2 Studies conducted in Australia, the United States and the United Kingdom identified various factors such as poor communication skills, competing priorities and clinical uncertainty around the patient contributed significantly towards these disagreements.3–6 In particular, several studies have highlighted a lack of trust on the part of in-hospital teams regarding the clinical decisions of emergency clinicians, and would not accept the patient until they had either reviewed the patient personally or received further investigation results.1,5–7

Problems in the consultation and referral process may carry significant implications not only for clinical outcomes such as length of stay, but also overcrowding in the ED while decisions are made surrounding patients’ ultimate in-hospital destination.8 There has been an increasing emphasis on referral practices and communication skills as part of emergency physician training to improve this process.1,4,9 However, there remains a lack of understanding in the current Australian healthcare context on disposition outcomes and contributing factors regarding patients whose referral process was complicated by a rejected admission request – which the study investigators termed a “knockback”.

The aim of this exploratory study was to describe knockbacks in a tertiary Australian ED, its impact on patient disposition outcomes, and other patient, clinician and systemic contributing factors. The primary outcome examined was whether the referrals for admission were warranted. Secondary outcomes included whether the initial referral from ED was directed at the correct inpatient team. The study was titled “MONitoring Knockbacks in EmergencY” (The “MONKEY” Audit).

Methods

Aim

The MONKEY audit was undertaken to fulfil the following outcomes:

To identify patients who were initially knocked back for admission, and yet subsequently required inpatient care within one week, such as an inpatient admission, or hospital-based procedure or intervention (ie, disposition outcomes), thereby constituting a warranted referral.

To determine how frequently patients were ultimately admitted under the team first contacted by ED (ie, “accuracy” of referral), and;

To uncover and thematically detail patterns and practices around patient knockbacks (ie, contributing patient, clinician and systems factors).

Setting

The MONKEY audit was undertaken in the Austin Hospital Emergency Department (ED) which is a tertiary-level teaching institution located in North-Eastern suburban Melbourne, Australia. The Austin ED receives more than 90,000 presentations annually. The Austin ED is staffed by ED clinicians comprising nurse practitioners (NPs), hospital medical officers (HMOs), Emergency registrars (Emergency specialist trainees, otherwise known as Emergency Residents in the United States) and Emergency consultants (Emergency Physicians). Referrals for admission are generally made to registrars (ie, specialty trainees, or specialty resident physicians-in-training) of inpatient teams, with approximately 40% of ED patients being admitted daily.

Recruitment

Patients were prospectively recruited by ED clinicians who identified their patient as having experienced a knockback. This process was conducted over two separate periods which were denoted as MONKEY1 and MONKEY2. MONKEY1 was conducted over 4 weeks from January 2020 until February 2020 while MONKEY2 was conducted over six months from August 2020 until January 2021. The second study period also intended to cover a similar 4-week period but was complicated in the context of a significant SARS-CoV-2 (COVID-19) outbreak in Melbourne, Australia. As such, the study period was extended for a 6-month hospital medical officer term to allow for competing priorities for emergency clinicians at this time while simultaneously ensuring consistency among the referring and accepting clinicians.

Figure 1 shows patient inclusion, where a total of 109 cases were included (MONKEY1, n = 67; MONKEY2, n = 42). Three patients were initially excluded from MONKEY1 because they were duplicates or ineligible. One patient was initially excluded from MONKEY2 because of insufficient data obtained.

Figure 1

Patient inclusion flowchart.

Study Design

ED clinicians were asked to anonymously nominate their patients who were knocked back by submitting the patient’s Unit Record (UR) number to a secure collection box located in the Austin ED. In cases where the same patient represented to the Emergency Department within one week for the same complaint, these were classified as one admission. Convenience sampling at the Emergency clinician’s discretion, with an element of judgment sampling, was the primary method of recruitment. At the end of each study period, investigators (the second, third and fourth authors) collected the patient UR numbers and conducted a retrospective chart review of electronic patient records.

Details of each knockback were recorded in an Excel database, including:

The seniority of the referring ED clinician

In-hospital teams contacted, and in what order

Final admission unit and length of ED and hospital stay, if known

Basic patient demographics and clinical outcome of the patient within the first week of admission

Free-text section to detail reasons for disposition disagreement, if identifiable

The study was reviewed by the Austin Health Office for Research and was approved as a clinical audit project with the reference number Audit/20/Austin/19.

Thematic and Statistical Analysis

A mixed-methods approach was undertaken to analyze the data, primarily by the first and second authors. Detailed qualitative analysis was performed with the constant comparative method used in the grounded theory approach to identify themes based on demographical data and the free-text section providing context for the disposition disagreement. Codes were generated and refined over multiple iterations of the data to develop key themes until saturation of themes was reached. Basic quantitative analysis was performed using IBM SPSS Statistics 26 comparing correlations between variables, with a p-value of 0.01 taken as significant.

Results

Descriptive Analysis

Patient demographics are shown in Table 1. Patient median age was 64 years old (range from 16 to 97 years old). Patient comorbidities were defined as existing medical issues detailed on hospital records. Patients had a median of 5 comorbidities (range from 0 to 17).

Table 1

Patient Demographics

Age
Median	64
25th Quartile	43
75th Quartile	80
Minimum	16
Maximum	97
Comorbidities
Median	5
Minimum	0
Maximum	17

Demographics of referring clinicians who nominated patients are shown in Table 2. 32.1% (n = 35) of cases were submitted by junior hospital medical officers (HMOs), 53.2% (n = 58) by Emergency registrars, 11.0% (n = 12) by Emergency physicians and 3.7% (n = 4) by nurse practitioners.

Table 2

Referring Clinician Demographics

Seniority of ED Clinician (%)	Number of Cases	Accurate Referrals, n (%)
HMO	35 (32.1%)	22 (62.9%)
Registrar	58 (53.2%)	35 (60.3%)
Physician	12 (11.0%)	6 (50.0%)
Nurse Practitioner	4 (3.7%)	3 (75.0%)
Total	109 (100%)	66 (60.6%)

The median number of teams involved in the admission process was 2, ranging from 1 to 4 (Figure 2). The teams most frequently contacted were General Medicine (Gen. Med), General Surgery (Gen. Surg) and the Short Stay Unit (SSU). NB A full list of teams contacted is available in the . Regarding knockback patterns, the most common inter-unit knockbacks occurred between Gen. Med–Gen. Surg and Gen. Surg–Gastroenterology (Table 3).

Figure 2

Number of teams involved in admission.

Table 3

Description of Inpatient Teams

Top 5 Teams contacted	Frequency
General medicine	48
General surgery	30
SSU	26
Cardiology	13
Orthopedics	12
Top 3 Knockback patterns	Frequency
Gen. Med. v Gen. Surg.	6
Gen. Surg. v Gastroenterology	6
Gen. Med. v Cardiology	5

Patient Disposition Outcomes

89.0% of cases (n = 97) required a hospital admission with 24.7% (n = 24) of these admissions lasting over 7 days, and 12.7% (n = 12) of admissions requiring an inpatient procedure (such as surgery or procedures requiring anesthetics involvement), and 3.7% (n = 4) requiring outpatient care (such as imaging, outpatient consultations or follow-up procedure). 4.6% (n = 5) of patients represented after an initial discharge from the hospital (Table 4).

Table 4

Primary Outcomes

Overall Accuracy of Referrals	Accurate Referrals (Total)	Percentage
MONKEY1	46 (67)	68.7%
MONKEY2	20 (42)	47.6%
Total	66 (109)	60.6%
Patient Outcomes	Frequency
Hospital Admission	97
Extended Admission (>7 days)	24
Inpatient Procedure	12
Representation after ED discharge	5
No admission	12
Outpatient Procedure	4
ED Length of Stay (hours)
Median	9
Minimum	0
Maximum	37

The ED length of stay was positively skewed with a median of 9 hours and a range from 0 to 37 hours (Table 4). Note that patients whose ED length of stay exceeded 24 hours were those admitted to the Short Stay Unit and therefore the total time included ED and SSU length of stay. Notably, there was a statistically significant link between the number of in-hospital units involved in the admission process and the ED length of stay (Spearman Correlation 0.409, p < 0.001) (Table 5).

Table 5

Correlations Between Patient Variables, Number of Inpatient Units Involved and ED Length of Stay

Patient Factors:	No. of Units Involved
Age
Pearson Correlation	−0.063
Sig. (2-tailed)	p = 0.517
Comorbidities
Pearson Correlation	−0.062
Sig. (2 tailed)	p = 0.520
Outcomes:	ED Length of Stay
No. of Units Involved
Spearman Correlation	0.409
Sig. (2-tailed)	p < 0.001

“Accuracy” of Initial Referral

An “accurate” referral was defined as the patient ultimately being admitted under the unit which was initially contacted by the emergency clinician, indicating that the emergency clinician’s referral for admission under the first unit contacted was justified. In 60.6% of cases (n = 66), the patient was eventually admitted under the team who was initially contacted. 68.7% of referrals (n = 46) in MONKEY1 and 47.6% of cases (n = 20) in MONKEY2 were deemed “accurate” (Table 4). Information regarding the referral level of care (eg, whether the patient was suitable for a general ward vs required intensive care) is available in the .

Factors Contributing to Knockbacks

Patient Factors

Regarding patient factors that may have contributed to the knockback, as shown in Table 5, there was no clear association between patient age and the number of units involved in admission (Pearson Correlation −0.063, p = 0.517), or between the number of patient comorbidities and the number of units involved in admission (Pearson Correlation −0.062, p = 0.520).

Qualitative analysis revealed several recurring themes revolving around patient presentations. Ninety-one patients in the audit had 1 key presenting complaint, 17 patients had 2 active issues and there was one patient whose presenting complaint was unclear. The most common presenting complaints were fever or suspected infection, abdominal pain in females and central neurological issues such as delirium and headache (Table 6).

Table 6

Patient-Related Knockback Themes

Patient Presentation Distribution	Frequency
1 Presenting Complaint	91
2 Presenting Complaints	17
Unclear Presenting Complaint	1
Top 5 Presenting Complaints	Frequency
Fever	14
Abdominal Pain (female)	13
Central Neurology	13
Shortness of Breath	12
Falls	12
Top 5 Patient Contributing Factors	Frequency
Chronic Medical Condition	22
Acutely Unwell	14
History of Malignancy	11
Representation to ED	11
Suspected COVID-19	10

Individual patient contributing themes, defined as factors not directly related to their presenting complaint but likely complicating the diagnostic process, were also identified and thematically analyzed. Patients had between 0 and 3 contributing factors, with the most frequent themes being the presence of co-morbid conditions requiring ongoing care/medication (not including cancer), an acutely unwell state in the ED (eg, in significant pain or clinically unstable) and a medical history of cancer. Representation to the ED and patients who were suspected COVID-19 patients rounded out the top 5 contributing factors to patient knockbacks (Table 6). There were only 2 instances (out of 109) where a knockback may have occurred because the patient needed a higher level of care (eg, Intensive Care Unit) and thus may have been inappropriate for a general ward at the time of first referral.

NB A full list of patient presenting complaints and patient contributing factors is available in the .

Clinician and System Factors

Referrals made by HMO doctors involved the greatest number of teams with a mean of 2.43 (SD 0.94). This was followed by Emergency registrars (2.21 – SD 1.20), Emergency physicians (2.08 – SD 1.16) and NPs (1.25 – SD 1.00) (Table 7).

Table 7

Clinician and System-Related Knockback Themes

Seniority	Mean Teams Contacted	Standard Deviation
HMO (n=35)	2.43	0.94
Registrar (n=58)	2.21	1.20
Physician (n=12)	2.08	1.16
Nurse Practitioner (n=4)	1.25	1.00
Top 5 Themes in Knockback	Frequency
Refer Elsewhere	77
Review In-person First	36
Clinician Disagreement	35
Request Investigations	31
Communication Issues	15

Systems-related factors, which were derived from free-text written submissions, were also examined with thematic analysis. Individual cases featured between 1 and 6 clinician-related themes. The most frequently occurring was in-hospital teams directing the admission to another team. Other prominently recurring themes included requests to review the patient in person first, disagreement regarding clinical decisions, requesting further investigation results, interpersonal/communication issues (such as lack of professionalism or difficulties in establishing contact) and administration issues (such as unclear guidelines for referral or being outside contact hours for specific services) (Table 7). NB A full list of clinician and system-related themes is available in the .

Discussion

The MONKEY audit evaluated disposition outcomes in a cohort of 109 ED patients where referral for hospital inpatient admission was initially rejected (a knockback) and investigated common themes which would have contributed to this outcome.

Predicting Patients Who Required Admission

The findings of the MONKEY audit bear much clinical significance, with 9 out of 10 patients subsequently requiring an admission, some of them lengthy inpatient stays. As almost two-thirds of patients were eventually admitted to the team where the referral was first made, this suggests that the majority of requests for referrals to multiple other teams were not warranted. Therefore, most of these knockbacks may be regarded as an unnecessary delay to the patient care journey once the need for admission has been identified by ED clinicians. This outcome appears to partially support the notion that ED decision-making regarding patient disposition is mostly accurate and that knockbacks may be associated with poorer patient outcomes, including increased length of stay in ED. This is noted by the correlation between the number of inpatient units involved and the prolonged stay in ED, demonstrating the previously identified complex ED-inpatient team interface.10

Prior studies regarding ED admissions have focused primarily on predicting the need for admission and accuracy of diagnosis.11,12 A retrospective study by Chiu et al conducted in a Hong Kong hospital found that ED diagnoses matched final discharge diagnoses 71.4% of the time. A prospective cohort study of experienced physicians conducted by Vlodaver et al in the United States found that ED clinicians predicted the correct admission 88.6% of the time.13 While these studies were conducted in overseas institutions with likely different admission protocols, scopes of practice and patient demographics, results from the MONKEY audit nonetheless seem to reinforce the notion that ED clinicians are capable at predicting both clinical diagnoses and the appropriate admitting team. The comparatively lower accuracy reported in the MONKEY audit could be attributed to the study only investigating cases where there was a disposition dilemma while including clinicians of varying levels of seniority.

Referral Accuracy

There was a disparity in accuracies of referrals between the two study periods (MONKEY1 and MONKEY2). MONKEY1 also involved an overlap between acquainted inpatient registrars with incoming new registrars including ones from other hospital networks and states. Study investigators initially proposed comparisons between the two study periods, however this was not performed, as the two samples were not comparable in time period, size and situation (N.B. MONKEY2 was limited by the Victorian COVID-19 “second wave”, impacting on patient recruitment). It is possible that referral accuracy rates were lower due to COVID-19 necessitating additional testing requirements and inhibiting patient flow processes.

In analyzing the teams involved, most knockbacks encountered in the study involved teams such as general medicine, general surgery and the short stay unit (SSU). This reflects that these are often the busiest units within the hospital and does not necessarily reflect any predisposition on their behalf to knock back referrals. Another potential explanation for these findings is the increasing emphasis on sub-specialization units, increased workload, and patient acuity, all of which may further complicate the referral-admission process.

Factors Involved in Knockback

Exploring patient factors linked to knockbacks, there was no significant association between patient age or number of comorbidities and knockbacks. This suggests that knockbacks are universal and not simply related to older and more complex patients. While certain presentations (such as fever or abdominal pain in females) and patient factors (such as complex medical history) were associated with disposition dilemmas, of particular concern was that multi-morbid, acutely unwell patients featured prominently with patient safety connotations. Specifically, these factors can make it difficult for ED clinicians to arrive confidently at a clinical diagnosis that is agreed upon by receiving inpatient units.

Examining clinician-related factors, the weak negative association between the seniority of the doctor making the admission with its accuracy implies that junior clinicians may be more likely to face knockbacks from in-hospital teams due to their relative inexperience, as claimed by Reid et al.1 Of note, the two cases where the admitting team asked for the referral to come from a more senior role both involved junior doctors placing the referral. One factor that may confound these results is that more complex patients are usually assigned to more senior doctors in the emergency department. The most frequent clinician-sided themes, apart from deferment of responsibility to another unit, were the need for external teams to review first and clinician disagreements, both suggesting lack of reciprocal trust.3 Another recurring theme identified among clinicians was interpersonal issues. Communication-related issues during referrals have been well documented previously,14,15 and the MONKEY audit highlights that referring ED clinicians also experience interpersonal conflict which may impact on their patient care.4,6 Likewise, Staib et al discussed various barriers in ED to hospital admissions including both avoidable and unavoidable delays, competing priorities, as well as confusion over clinical territories.10 The findings from the MONKEY audit support these as key systemic issues that require addressing.

In only 2 cases was there disagreement in patient disposition because of the level of care, ie, where a patient was too critically unwell for a general ward, requiring intensive care instead (for details refer to the ). Future research with a larger patient sample could include patient observations at the time of referral to determine if clinical deterioration or instability impacted on inpatient doctors’ reluctance to accept the referral for admission. In addition, future studies could aim to investigate all admissions to the ED within a smaller time frame to be able to compare outcomes with a matched cohort of patients whose admission processes were more streamlined. Furthermore, while there were several identified factors linked to state-wide health policy such as the sub-specialization of hospital teams, it is difficult to generalize the findings of the MONKEY audit to other health-care settings. Thus, it would be of benefit to conduct future similar audits across multiple sites to compare and confirm these findings.

Limitations

The main limitation to the MONKEY audit was its limited size and scope. The original purpose of the study was not to perform a wide-ranging analysis of ED admission outcomes but instead was exploratory in nature, identifying outcomes and contributors to difficult admission processes, and hence was not designed or powered for detailed quantitative analysis. Due to both the complex nature of the admission process as well as the limited sample size of this audit, concrete conclusions on the correlations and interactions between the investigated variables cannot be drawn with the current data and the nature of this study. In other words, any attempt at correlation analysis may be impacted by low power, especially when other potential confounding variables were not controlled for (such as patient characteristics, state of busyness of the ED, or clinician characteristics). In addition, it is difficult to determine causality such as whether patient outcomes were impacted by factors such as knockbacks or delayed reviews, or vice versa. To this end, future studies may explore the importance of these factors relative to one another.

There were also significant limitations to recruitment for the two studies. As mentioned previously, the two study periods were not homogenous and not comparable in either scope, length or study environment due to the impacts of the COVID-19 pandemic. These differences also led to possible reporting bias. In both cases, clinicians were simply asked to submit patients who met the study criteria at their own discretion, and in addition, not all knockbacks would have been captured suggesting likely selection and performance biases. Furthermore, patients in the MONKEY audit were not matched to control groups, so it is difficult to generalize these findings to the broader ED environment.

Conclusion

Despite its limited scope, this audit sheds light on an important process in the patient journey. Its mixed methods analysis identifies several key patterns and recurring themes around referral-admission complications; it is not unreasonable to propose that this is a reoccurring phenomenon in similar tertiary Emergency Department contexts. The salient findings are that generally, ED clinicians initially refer to the correct team, and that requests for admission are warranted in most cases. The themes around knockbacks identified both patient and systems-centered issues such as patient complexity and clinician disagreement, which match the existing literature. This does not, however, invalidate the fact that sometimes knockbacks may be an eventual outcome with complicated patient presentations. There is a need for focused discussion and clearer referral guidelines between in-hospital clinicians as well as emergency clinicians.