A Realist Case Study of a Regional Hospital's Response to Improve Emergency Department Access in the Context of Australian Health Care Reforms.

INTRODUCTION: Major health-care reforms have extended across all Australian public hospitals in recent years. Improving emergency department (ED) access has been a focus of these reforms.
OBJECTIVE: This study evaluates how the national reforms have led to improvement in ED access in a regional hospital in remote Australia.
METHODS: Assessing a complex scenario such as national reforms and the challenges faced by the regional hospital to implement these reforms requires in-depth analysis. A realist evaluation theory-based approach was employed, allowing investigation of what, how, why, and for whom change occurred. A case study mixed methods design was adopted within the realist framework to answer these questions about change. RESULTS AND
CONCLUSION: The study identified moderate improvement in ED access as a result of the reforms (investment in infrastructure and workforce and the introduction of ED targets). Clinical leadership and support from management were essential for the improvement. Without ongoing investment and clinical redesign activities, however, sustainability of the improvement may prove difficult.

Introduction

Australian Health-Care Reforms

Governments worldwide are engaged in health service reform.[1] The main intention of these reforms is to improve the performance and sustainability of health services. In 2009, a report by the National Health and Hospital Reform Commission identified that the Australian public hospital system was, in practice, “fragmented, poorly responsive, underfunded and in dire need of reform.”[2] Following the report’s release, Australia’s States and Territories agreed to a series of National Agreements under the auspices of the Council of Australian Governments.[3,4] The agreements stipulated the Federal Government would provide additional funding for public hospitals to increase access to essential hospital services such as emergency department (ED) and elective surgery services.[2] Public hospitals with EDs would be funded to enable timely access for patients presenting to EDs. As part of the national health reform agreements, performance indicators and targets were established to increase ED access.[5] Also, clinical redesign activities to improve access were initiated along with major infrastructural redevelopments in ED’s across the country.[6-9]

Emergency Department Reform

The principal reason for ED overcrowding in Australia is Access Block—the delay patients experience in ED when an inpatient bed is unavailable.[5] Access Block is an indicator of ED performance, measured as an ED length of stay of greater than 8 hours for patients requiring hospital admission.[10] Access Block, however, is not just an ED problem. It reflects systemic issues: relative lack of inpatient bed capacity or efficient mechanisms to admit inpatients from ED. Therefore, multifactorial evidence-based solutions that improve capacity and processes throughout the hospital are required to improve performance.[10,11] The national agreements therefore encouraged hospital-wide solutions, allocating funding for non-ED areas. They also introduced additional ED targets:

Emergency department waiting times: This performance indicator monitors the proportion of patients seen within the clinically recommended time for their triage category.[12] It is a reliable indicator of how accessible hospital emergency services are to patients.[10,12]

There are no current national ED waiting time targets for public hospitals.[13,14] Until it expired, the most recent target was “80 percent of emergency department presentations are seen within clinically recommended triage times as recommended by the Australian College of Emergency Medicine” by 2012 to 2013.[13]

National Emergency Access Target (NEAT): This reflects ED attendance outcomes, where “90 percent of all patients presenting to a public hospital ED will either physically leave the ED for admission to hospital, be referred to another hospital for treatment, or be discharged within four hours.”[15] The relevant agreement introduced phased targets for each calendar year from 2012 to 2016 with the final target being 90% (note 1). Reward funding associated with achievement of NEAT targets ceases in 2015 to 2016.

Emergency Department Reform in Regional Hospitals

The Australian Government reforms were intended to cover all Australian public hospitals, including the 569 hospitals in nonmetropolitan areas.[2,5,16] Many of these 569 hospitals are regional hospitals.[17,18] Regional hospitals refer to those hospitals located in government-defined regional areas (nonmetropolitan areas beyond major capital cities and their immediate areas).[18-20] Regional hospitals in Australia have different contextual challenges compared with metropolitan hospitals.[21,22] These challenges include workforce shortages, inadequate infrastructure, limited professional support, and services not aligned with population growth. Emergency departments in these locations face additional challenges, including an ever increasing load of complicated cases, and patients seeking nonurgent care because they cannot afford or access paid primary care services.[21,23,24]

Alice Springs Hospital (ASH) is the regional hospital for Central Australia.[25] ASH is a 186-bed hospital providing a range of services including general medicine, nephrology, surgery, psychiatry, pediatrics and obstetrics, and gynecology among other acute and subacute specialist services.[26] The present hospital was established in 1977 but has been undergoing continuous renovations and redevelopments since then.[27-29] Located in a remote area (note 2), the hospital faces significant challenges in delivering quality and comprehensive specialist services to its patients.[31,32]

The ASH ED currently sees over 42 000 presentations per annum and services a population of 60 000 spread over 830 000 km[2].[26,33-35] A significant proportion of ASH patients are Aboriginal, and Aboriginals form 30% of the Northern Territory population (note 3).[31,32,36] ASH ED has previously been known to experience significant challenges including large numbers of patients with chronic disease complications (such as diabetes mellitus complications and chronic kidney disease in adults, and chronic suppurative otitis media, and chronic suppurative lung disease in children),[31,32,37] alcohol-related admissions, and inadequate infrastructure. When compared to the other major regional hospitals, ASH ED performance has previously fared poorly in terms of access.[23] However, following the recent introduction of ED targets and clinical redesign activities and major infrastructural redevelopments such as a new ED and expanded workforce, local and national reporting has indicated recent improvement in ASH ED access (see Figure 1).[38-40] Explanations for these improvements have been attributed to the national health care reform funding, capital projects, and performance targets.[27,28,38,41]

Figure 1.

ASH emergency department (ED) patients departing within 4 hours (National Emergency Access Target [NEAT]) compared to other major regional hospitals, 2013 to 2014. Adapted from National Health Performance Authority, 2015.

Improvement in ASH ED access in the context of the national reforms and associated investment suggested an association. However, the complexity of the context and outcomes necessitates formal assessment prior to making any firm conclusions about association. Further, improvements must also be considered in the context of the hospital’s remote location and heavy disease burden, which have an impact on hospital service delivery.[24,31,32] The current study aimed to verify ASH’s ED access improvement and establish how improvement was achieved in the face of known and emerging challenges.

Methods

Study Framework

The reform context, remote location, and hospital patient profile present a contextually rich and complex research setting. A robust methodology allowing in-depth analysis was chosen to answer the research question. Realist evaluation, a theory-based evaluation, used to describe and analyze complex phenomena was chosen.[42,43] When evaluating organizations, realist evaluation offers distinct advantages over nontheoretical evaluation approaches by analyzing why changes occur, under which conditions, and in which situations.[43] An initial program theory is developed to explain how the program has worked in a particular setting, and this theory is then used to focus the research questions and select appropriate data collection methods. A range of data are then collected to heuristically test and refine the program theory as the evaluation progresses.[42,44] The construction, exploration, and refining of program theories is expressed in the form of Context-Mechanism-Outcome configurations (CMOCs). Different contexts and mechanisms triggering change are identified and hypothesized to explain variations in program outcomes. The final research product of realist evaluation is not a determination of the effect size but a refinement of the initial program theory to more accurately represent what, for whom, why, and how change has occurred.[44]

Case studies also allow a rich understanding of the context of the research and the processes being enacted. They are especially important where a planned change is occurring in a complex setting, and it is important to understand why a planned change or intervention succeeds or fails.[45,46] Case studies help in setting boundaries around the phenomenon under study while establishing units to be researched.[47,48] They are therefore widely used in theory-driven health research,[43,45] and a case study design has been adopted in the current study to complement the realist framework. ASH was selected as a single case because of the unique nature of the hospital (the only major regional hospital in remote Australia)[18] and the complex environment under study (national health care reforms being implemented in a remote regional hospital).

Study Design

This study employed a realist case study design, utilizing a mixed method approach, implemented across several phases (see Table 1). Ethics approval was received from the Central Australia Human Research Ethics Committee (HREC-14-266), and approval covered all phases of the study including interviews in the latter part of the study.

Table 1.

Data Collection and Analysis Phases.

	Step	Methods	Analysis and Expected Outcome
Phase 1	Construction of program theory	Literature review, interview with the main hospital clinicians and managers, and review of theories concerned with organizational change and performance.	Initial program theory and a preliminary CMO configuration—the hypothetical pathway linking intervention strategies to outcomes.
Phase 2	Data collection-quantitative	Correlation and regression analysis of select ED access indicators	Quantification and confirmation of an improvement trend in ED access over time.
Phase 3	Data collection-qualitative	Review of hospital and government documents pertaining to study questions and context.	Themes identified and coded using computer-assisted qualitative data analysis software. The themes were used to refine the program theory and preliminary CMO configuration
Phase 4	Data collection-qualitative	Purposive sampling of hospital staff and semistructured interviews with selected participants.	Themes were identified from interview data and coded using computer-assisted qualitative data analysis software.
Phase 5	Data collection-qualitative	A second round of interviews with select participants.	The interviews were used to explore or confirm themes that emerged from the previous round.
Phase 6	Validation and refining the theory	Analysis of findings from phases 3, 4, and 5 to outline the relationship between various components of the CMOC and the role of the context in the study.	The analysis assisted with further refinement of the program theory in order to provide a credible and dependable explanation of what components of the reform initiated interventions worked, for whom, and under what conditions.

Abbreviations: CMO, Context-Mechanism-Outcome; ED, emergency department.

Data Collection and Analysis

Program Theory

Initial program theory and preliminary CMOCs were developed in phase 1 on the basis of a literature review incorporating academic, hospital, and media documents as well as pilot interviews with senior hospital clinicians and managers. Interview participants provided verbal informed consent.

Quantitative Data

The preliminary program theory proposed that national health-care reforms (funding and introduction of targets) contributed to the increase in ED access. In phase 2, to test the increase over time, ED performance indicator data for 7 years commencing 2008 (the year the first national ED target was introduced) and ending 2014 were obtained from the hospital patient care information system. These data were split by monthly results and examined using bivariate scatter plot and correlation analysis.

Qualitative

Thematic analysis of 20 official documents (Federal and Territory Government documents, national agreements, and hospital reports) was undertaken in phase 3 to extract contextually relevant themes. Thirty interviews in phases 4 and 5 followed this with relevant hospital executives, hospital managers, medical heads of departments, senior clinicians, and junior clinicians. Interview participants provided written informed consent.

Participant groupings correspond to units of analysis as prescribed in case study design. Themes from the interview transcripts and document analysis were laid out in the form of CMOCs to analyze the hospital’s program to improve ED access and to test and refine the program theory.

Results

Preliminary Program Theory

Based on the literature review and pilot interviews, a preliminary program theory was developed as follows:

National health-care reforms and introduction of ED access targets have led to improvement in ASH ED access because of the resources provided through reforms and the willingness of management and clinicians to support implementation.

Quantitative Findings

Initially, annual results for the 3 performance indicators considered in this study (ED waiting times, Access Block, and NEAT) were charted over 2008 to 2014 (see Table 2). The data indicate moderate improvement (increase in percentage of ED patients seen within clinically recommended time and decrease in Access Block). To analyze the results further, the performance indicator data were broken down by month and scatter plots constructed to explore the improvement over time (see results in Table 3). The scatter plots indicated a positive relation between time and ED seen within clinically recommended time and NEAT results, and a negative relation between time and Access Block results. To obtain strength and significance of these relationships, correlation analysis was performed. The correlation analysis (Table 3) indicates a strong association (P <.001) between the month of the year and ED presentations seen within time and a moderately strong inverse relationship between Access Block and the month of the year (P < .05). There was no statistically significant improvement over time for NEAT. The NEAT results should be interpreted in light of its recent introduction in 2012: the other 2 performance indicators have been in place for some time.[10,13,15,49]

Table 2.

ASH ED Performance Indicators, 2008 to 2014.a

ED Performance Indicators	2008	2009	2010	2011	2012	2013	2014
Number of ED presentations	33 528	37 273	39 210	41 931	39 962	42 107	42 873
ED percentage of patients seen within clinically recommended triage times	51.5%	58.7%	49.1%	53.3%	57.0%	56.5%	61.5%
ED Access Block	46.0%	45.0%	44.0%	43.7%	38.9%	42.4%	42.4%
NEAT	60.2%	64.6%	60.2%	62.4%	63.0%	61.2%	62.9%

Abbreviations: ED, emergency department; NEAT, National Emergency Access Target.

aSource Data: Alice Springs Hospital.

Table 3.

Scatterplot and Correlation Analysis Between Month of the Year and ED Performance Indicators, 2008 to 2014.a

ED Performance Indicators	Linear Equation (y = mx + c)	R2 Value	Correlation With Month
ED presentations seen within time	y = 0.1037x + 51.184	0.1209	0.348b
Access Block	y = −0.0649x + 45.96	0.0763	−0.276c
NEAT	y = 0.0159x + 61.42	0.0223	0.149

Abbreviations: ED, emergency department; NEAT, National Emergency Access Target.

an = 84.

b P < .001.

c P < .05.

Qualitative Findings

The key CMOC pathways that emerged from document and interview transcript analysis are outlined in Table 4. The CMOCs and quantitative results identified nuances that led to revision and refinement of the program theory as follows:

Table 4.

Context-Mechanisms-Outcome Configuration Patterns.

Context	Intervention	Mechanisms	Outcome
National health care reform	Local investment in hospital infrastructure and workforce	Management wants to support clinicians + clinicians feel supported by management + clinicians feel encouraged to develop new protocols and pathways to improve patient flow.	Improved patient flow
National and local health care reforms	Introduction of ED performance indicators	Management wants to focus on specific factors such as Access Block + management feel compelled to communicate with clinicians more regularly and effectively + some inpatient clinicians are keen to support and collaborate with ED clinicians + clinicians feel encouraged to develop new protocols and pathways to improve patient flow	Improved ED performance
		Management does not want to meet targets at the cost of patient safety and relationship with clinicians.	Recent establishment of a local program to respond to national reforms
National health care reform	Investment in specific areas of hospital infrastructure and workforce only	Inpatient clinicians are not particularly motivated to support ED or meet targets + some clinicians feel unsupported	Potential increase in Access Block
Alice Springs Context (remote + patient profile)	Introduction of ED performance indicators	Management does not want to meet targets at the cost of patient safety and relationship with clinicians + management and clinicians frustrated with inadequate community services + management and clinicians feel powerless to address patient flow issues + ED and inpatient clinicians feel patient safety comes before patient flow or targets	Potential decrease in ED access

Abbreviation: ED, emergency department.

Introduction of contextually relevant reform programs have led to improvement in ASH ED access through involvement of both management and clinicians. The changes have been aided by infrastructure and workforce investment acquired through national and local funding. The funding has provided resources and motivation for clinicians to implement reforms. However, unequal investment across different areas of the hospital and continuing social challenges will have an impact on the motivation of hospital staff to sustain improvement.

Discussion

The study aimed to determine whether ASH ED access has improved in the context of national reforms, and if so, how improvement was achieved in the face of known and emerging challenges for regional hospitals. Combining a case study design with a realist evaluation framework enabled rich exploration of this complex context and potential extrapolation of results to other hospitals experiencing reforms. Consistent with a realist approach, the study has highlighted the importance of contextual factors, program resources, and staff reasoning for the realization of change.

The quantitative component of this study charted ED performance indicator results over time. Analyses identified moderate improvement in some ED access indicators (increase in ED presentations seen within time and decrease in Access Block) over the reform period 2008 to 2014. While these indicators capture only a single dimension of ED performance, they are still an important representation of the outcome of targeted ED reform.[10,50,51]

The qualitative phase of this study probed how and why these outcomes were achieved. Analysis indicated that strong clinical leadership ensured improvement in ED access, despite contextual challenges. Reform investment into infrastructure and workforce were also key drivers to improvement. While reform targets were important and a driver for improvement in ED access, patient safety was paramount for both management and clinicians. Sustainability of performance improvement will be difficult without changes in other hospital departments (infrastructure and workforce), improvement in support services, and a long-term clinical redesign program.

One of the limitations of this study is that not all reform variables (such as reform funding and workforce parameters) were considered in the quantitative analysis to identify association with improvement in ED access. This is because this study is meant to be first of the studies that will explore the impact of national reforms on regional ED access. Further studies, as realist standards[52] suggest, will be required to probe the association and perhaps causative factors. While we cannot assume from the results of this study that national health care reform, in isolation, led to improvement in ASH ED access, we can confidently note there has been moderate improvement in ED access following the introduction of reform components.

Another limitation is that a single site was selected to undertake the evaluation. However, the realist case study design doesn’t discriminate against single site selection and allows intraprogram comparison and in-depth analysis within a single site. This level of analysis may not have been feasible to complete (in reasonable time frames) if multiple sites were involved. While we cannot assume the program theory derived from this study applies to all regional hospitals, we can safely understand the mechanisms (positive and negative) and reform components (infrastructure and workforce funding, introduction of targets, and clinical redesign activity) identified through this study are vital considerations when implementing ED reforms in other regional hospitals.