Controlled pre-post, mixed-methods study to determine the effectiveness of a national delirium clinical care standard to improve the diagnosis and care of patients with delirium in Australian hospitals: a protocol.

INTRODUCTION: Delirium, an acute confusional state, affects up to 29% of acute inpatients aged 65 years and over. The Australian Delirium Clinical Care Standard (the Standard) contains evidence-based, multicomponent interventions, to identify and reduce delirium. This study aims to: (1) conduct a controlled, before-and-after study to assess the clinical effectiveness of the Standard to improve diagnosis and treatment of delirium; (2) conduct a cost-effectiveness study of implementing the Standard and (3) evaluate the implementation process. METHODS AND ANALYSIS: The study will use a controlled, preimplementation and postimplementation mixed-methods study design, including: medical record reviews, activity-based costing analysis and interviews with staff, patients and their family members. The study population will comprise patients 65 years and over, admitted to surgical, medical and intensive care wards in four intervention hospitals and one control hospital. The primary clinical outcome will be the incidence of delirium. Secondary outcomes include: length of stay, severity and duration of delirium, inhospital mortality rates, readmission rates and use of psychotropic drugs. Cost-effectiveness will be evaluated through activity-based costing analysis and outcome data, and the implementation process appraised through the qualitative results. ETHICS AND DISSEMINATION: Ethics approval has been received for two hospitals. Additional hospitals have been identified and ethics applications will be submitted once the tools in the pilot study have been tested.The results will be submitted for publication in peer-reviewed journals and presented to national and international conferences. Results seminars will provide a quality feedback mechanism for staff and health policy bodies. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

This is the first study in Australia and among few internationally to measure both the cost-effectiveness and clinical effectiveness of a National Clinical Care Standard.

This novel evaluation approach uses a controlled, pre–post design, including both quantitative and qualitative data collection, to measure changes in delirium rates in acute care.

The methods outlined in this study have the potential to be applied to the assessment of other Clinical Care Standards.

Limitations of the study include recruitment in five publicly funding acute care facilities within two states in Australia, and a lack of longer-term follow-up for affected patients.

The Standard is not mandatory and implementation may be interpreted differently at each facility.

Introduction

The increasing average age of patients in Australian hospitals is associated with greater levels of cognitive impairment in the inpatient population.1 Patients in the 65 years and over age group, even those with normal cognition, can experience a short-term reduction in their cognitive function and become acutely confused during admission. The term delirium is used to describe this state and is generally characterised by: its temporary and variable nature, the presence of precipitation factors, and resolution once these factors are removed or treated.2 Symptoms and signs of delirium range from patients being agitated and hyperactive, to being sleepy and hypoactive. Common to all manifestations is a change in attention, awareness, and cognition and varying levels of confusion.2

Delirium is a significant problem in acute care. Using published incidence rates of 3%–29%,3 we estimate delirium affected 116 731–1 128 400 inpatients aged 65 years and over, applying Australian 2013–2014 admissions data.4 Higher delirium rates of 47%–63% have been observed in surgical patients,5 and critically ill patients with delirium stay, on average, 6.5 days longer in hospital.6 Furthermore, other national Australian data indicate delirium was a principal diagnosis in 11 232 separations (0.29%) of patients aged 65 years and over during 2013–2014, and that 28% of these patients had existing dementia.7 These figures are below the incidence range of 3%–29% collected from record reviews and targeted assessment,3 but do not include the number of patients developing delirium secondary to other risk factors such as surgery or treatment in an intensive care unit (ICU). Prevalence rates (10%–31%) are higher than for hospital-acquired delirium (3%–29%),8 with a prospective cohort study (n=10 014) showing on-admission delirium rates of 24.6% for patients aged 65 years and over.9 Although delirium is by definition a transient issue, patients developing the condition in hospital are 2.6 times more likely to die during the admission.10 Patients diagnosed with delirium have a higher risk of developing dementia (adjusted relative risk (RR) of 5.7, 95% CI 1.3 to 24.0), and the presence of dementia increases the risk of developing delirium two to five times.10 11

The Australian Commission for Safety and Quality in Health Care (the national agency for initiatives in this domain) published the National Delirium Clinical Care Standard (the Standard) in 2016,12 which includes a multicomponent intervention for reducing delirium in acute care.13 These strategies for preventing and treating delirium were developed in the USA as part of the Hospital Elder Life Program (HELP),14 and were influential in informing the Delirium Care Pathway developed by the Australian Government in 2011.15 HELP targets patients with high-risk factors for delirium: existing cognitive impairment, sleep deprivation, immobility, hearing and visual impairment, and dehydration. The HELP has been updated to reflect the guidelines from National Institute for Heath and Care Excellence in the UK, and includes protocols for medication reviews, pain management, constipation, infection control, hypoxia and aspiration pneumonia.16 A recent Cochrane review described strong evidence to support a multicomponent approach to reducing delirium in both medical and surgical wards versus usual care (RR 0.69, 95% CI 0.59 to 0.81),17 although this strategy was less effective for those with pre-existing dementia (RR 0.9, 95% CI 0.59 to 1.36). The evidence for whether these programmes reduced the length of a delirium episode was inconclusive.

Despite research on the costs of delirium,18 and separately on the effectiveness of interventions,19 20 the cost-effectiveness of multicomponent interventions in acute care has been less widely studied.10 The voluntary nature of the Standard means hospitals need a compelling reason to invest the time, resources and clinical governance infrastructure required to implement the Standard.21

Given the low levels of reported delirium rates,7 we hypothesise that introducing the Standard will improve detection rates and enable patients to be more accurately diagnosed and treated. The aims of are to: (1) conduct a controlled before and after study to assess the clinical effectiveness of the Standard to improve diagnosis and treatment of delirium in acute inpatients aged 65 years and over in Australia; (2) conduct a cost-effectiveness study of implementing the Standard and (3) evaluate the implementation process. The economic evaluation will include the perspective of patients and their families and carers, as well as the health system. The study design will incorporate both programme evaluation and implementation science principles to support the sustainability of the Standard within the acute care health system.22

Methods

Study design

The study will use a mixed-methods, controlled, pre–post design, comprising medical record reviews, activity-based costing analysis and interviews with hospital staff, patients, and their carers and relatives. The study will be conducted during the period 2017–2019.

Study population

The study population for the medical record reviews will comprise all patients aged 65 years and over admitted to selected surgical, medical and intensive care wards in five acute care facilities in New South Wales (NSW) and the Australian Capital Territory (ACT) during the medical record review periods (see table 1). In addition, we will conduct interviews with nursing staff on the study wards (n=10 per hospital), patients who have recovered from an episode of delirium (n=10 per hospital), their relatives and carers (n=10 per hospital), and hospital management.

Table 1

Project timeline, study design and data collection periods

Month	Intervention hospital	Control hospital
1	First medical record review period to measure delirium incidence and secondary outcomes (2–4 weeks). First assessment of current status of hospital compliance against Standard.
2–3*	Implementation model development. Preimplementation activities completed.	Note: Standard not implemented in control hospital
4*	Standard implementation.
5–6	Interviews with nursing and quality control staff. Interviews with implementation teams and hospital management.* Interviews with patients and their carers and relatives.
7–8	Second medical review period (2– 4 weeks). Second assessment of hospital compliance against Standard.
9–11	Clinical and cost-effectiveness analyses completed.
12	Translation activities. Appraisal of implementation process and preparation of summative report.

*These activities will only be undertaken by the intervention hospitals.

Intervention

The Standard comprises a hospital-wide, multicomponent strategy for detecting and reducing delirium.12 A key component is the development of a safety and quality pathway (Pathway) for patients with cognitive impairment (see table 2 for summary). The Pathway includes patients with delirium and dementia due to the causal relationship between the two clinical states.23

Table 2

Safety and quality pathway for patients with cognitive impairment in hospital

Step	Actions	Explanation
Step 1	Identify patients at high risk for developing delirium, and screen for cognitive impairment	Risk factors include: Age 65 and over Known cognitive impairment Severe illness (risk of dying) Hip fracture Cognitive concerns raised by others
Step 2	Identify and monitor risk factors	Falls and pressure injury screening Medicines review Nutrition and dehydration screening Assessment of communication difficulties Identification of treatment not wanted by patient, for example, through advanced care plans
Step 3	Implement individual, integrated prevention and management plans in partnership with patients, carers and family

Table derived from Standard publications.12

Comparison

Four of the study hospitals (intervention hospitals) will implement the Standard. Medical record review data from these hospitals will be analysed at the ward and hospital level to compare the level of diagnosis and treatment of delirium before and after implementing the Standard. A fifth hospital, with similar demographics, will act as the control hospital in order to assess underlying trends in delirium recognition and treatment (see table 1).

Outcomes

For aim 1 (clinical effectiveness), the primary clinical outcome will be the incidence of hospital-acquired delirium before and after implementing the Standard. Secondary outcomes will include length of stay, severity and duration of delirium, inhospital mortality rates, readmission rates and Standard-related indicators.24 Primary and secondary clinical outcomes will be identified using medical record audits and indicator data collected by the hospitals.24

For aim 2 (cost-effectiveness), we will use activity-based costing analysis to determine the incremental cost of implementing the Standard. We will assess the change in resource use resulting from improved detection and treatment of delirium,18 25 and use outcome data and published health utilities relating to delirium to perform a cost-effectiveness analysis.10 Implementation will be assessed using the RE-AIM framework: reach, effectiveness, adoption, implementation consistency and maintenance.26

Recruitment and consent

Medium to large regional and metropolitan public hospitals (n=5) in two jurisdictions (ACT and NSW) will be invited to participate. A waiver of consent for the medical record reviews has been approved for two hospitals and will be included in the ethics submission for the remaining hospitals.

Consenting nursing staff (n=10 for each hospital) on the study wards will be invited to participate in the qualitative part of the study to assess their perceptions/views of the treatment and diagnosis of delirium (all hospitals) and implementation process (study hospitals). Patients (n=10 at each hospital), and their relatives and carers (n=10 at each hospital), who had a resolved episode of delirium during their hospital stay will be identified by the senior nursing staff on the study wards and approached to take part in the study. Additional management, quality and finance staff at each hospital will be identified for consent to be interviewed for the costing analysis.

Sample size calculations

Our main outcome of interest will be the incidence of delirium. We hypothesise that delirium may be underdiagnosed at baseline,27 and that implementing the Standard protocols will result in an increased incidence rate. A Cochrane review estimated prevalence rates on admission of 10%–31%, and hospital-acquired incidence of 3%–29%.8 We estimate a weekly admission rate of 0.84 patients aged 65 years and over per bed,7 and an average of 18 beds per ward over the four intervention hospitals. Using a review period of 4 weeks for the first intervention hospital, and a 2-week period for the remaining three intervention hospitals, we estimate 1506 records will be reviewed (753 records for each of the preimplementation and postimplementation arms of the study). This is above the sample size required (345 records per arm) to detect a change in reported delirium rates of 0.3% to a conservative 3% incidence rate using 80% power and 95% CI, for the pooled hospital data.

Data collection

Medical record reviews

The medical records of all patients aged 65 years and over and admitted to the study wards during the medical record review period in the preimplementation and postimplementation phase in each hospital will be included (see table 1). Patient demographics, diagnosis, length of stay, inhospital mortality, delirium risk factors, cognitive screening and delirium diagnostic testing will be abstracted from the records using a purpose-designed tool (online supplementary file 1).

Additional data collected for those patients who developed delirium will include: precipitating factors, and the severity and duration of delirium. Data will also be collected to assess compliance with protocols that form part of the Standard indicators. These protocols include: hydration and nutrition, medication reviews, pain management, risk of falls and pressure injuries.28 The medical record review will collect several of the Standard indicators (see table 3) in the study wards, including falls and pressure injury risk assessments. All the indicators will be collected by the intervention hospitals as part of each hospital’s normal indicator collection.

Table 3

Delirium Clinical Care Standard Indicators24

Indicator	Description
1a	Evidence of local arrangements for cognitive screening of patients presenting to hospital with one or more key risk factors for delirium
1b*	Proportion of older patients undergoing cognitive screening within 24 hours of admission to hospital using a validated test
2a	Evidence of training sessions undertaken by staff in the use of a validated diagnostic tool for delirium
2b*	Proportion of patients who screen positive for cognitive impairment at admission who are assessed for delirium using a validated diagnostic tool
2c*	Rate of delirium among acute admitted patients
2d*	Rate of delirium among acute admitted patients with onset during the hospital stay
3a	Evidence of local arrangements for implementing interventions to prevent delirium for at-risk patients
4a*	Proportion of patients with delirium who have a comprehensive assessment to investigate cause(s) of delirium
4b*	Proportion of patients with delirium who receive a set of interventions to treat the causes of delirium, based on a comprehensive assessment
5a	Evidence of local arrangements for patients with delirium to be assessed for risk of falls and pressure injuries
5b*	Proportion of patients with delirium assessed for risk of falls and pressure injuries
5c*	Proportion of patients with delirium who have had a fall or a pressure injury during their hospital stay
6a	Evidence of local arrangements to ensure that patients with delirium are not routinely prescribed antipsychotic medicines
6b*	Proportion of patients with delirium prescribed antipsychotic medicines in hospital
7a*	Proportion of patients with current or resolved delirium who have an individualised care plan
7b*	Proportion of older patients with current or resolved delirium who are readmitted for delirium within 28 days

*Indicators collected from the medical record review.

Activity-based costing analysis

Each intervention hospital will be responsible for implementing the Standard through development of an implementation model. Such models include the programme logic model approach,29 and help identify: (1) resources and approvals required, (2) implementation activities such as staff training or physical changes to the wards, (3) outputs to measure implementation activities, (4) short-term to medium-term outputs in terms of length of stay and (5) indicators to measure impact on longer-term patient outcomes. The resources and activities identified in the model will be assessed and costed through assessment of the time, grade and numbers of staff involved. Interviews with the hospital management team will be used to measure other costs of implementation.

Standard implementation analysis

To evaluate the implementation process, we will use the five dimensions of the reach, efficacy, adoption, implementation and maintenance (RE-AIM) framework.26 The RE-AIM checklist will provide a structured approach to analysing the implementation through discussions with the implementation team.26 This will include a preaudit and postaudit of each hospital to determine the level of compliance with the Standard for each step of the Pathway (see table 2) and with Standard Indicators (see table 3). In addition, the results of the interviews with nursing staff on the intervention hospital wards, and with the implementation teams will be assessed using the three mechanisms for change outlined in the Standard: (1) establish responsive systems; (2) ensure a skilled and informed workforce and (3) enable partnerships between clinicians, patients, carers and families.12

Interviews with staff

Registered nurses working on the study wards during implementation will be interviewed (n=10 per hospital), in the postimplementation period (months 5 and 6 in table 1). These interviews will assess the initial and medium-term impact of the Standard on their working practices, their views on the implementation process, and whether implementing the Standard has impacted the diagnosis, treatment and prognosis of inpatients with delirium. Questions relating to the Standard will be removed from the interview questionnaire at the control hospital, and replaced with questions relating to current practice about the identification and management of delirium. The interview format and questions are included in (online supplementary file 2). Interviews relating to costs will be conducted using an open-ended question format. Question topics will relate to the resources, activities and indicators identified in the implementation model.

Interviews with patients, carers and families

Consenting patients who had a resolved episode of delirium during admission (n=10 per hospital) in the postimplementation phase, and their families and carers (n=10 per hospital) will be interviewed to assess the impact of delirium (online supplementary file 2). Patients, and their relatives and carers will be identified by the staff and interviewed in person during their stay or by telephone after discharge. The interviews will be electronically recorded, professionally transcribed, de-identified and analysed with NVivo software using a framework analysis approach.30

Analysis and evaluation

Aim 1: clinical effectiveness

The descriptive statistics from the medical record reviews will be analysed and multilevel modelling techniques used to determine whether implementing the Standard was associated with a change in the incidence of delirium.31 This type of statistical modelling will allow for clustering at the hospital and ward level to account for the differences in implementation strategies and for differences in delirium incidence rates in medical, surgical and ICU environments. The incidence of hospital-acquired delirium will be reported as a percentage of total study admissions both preimplementation and postimplementation, and by hospital and ward. Hospital-acquired delirium will be differentiated from delirium present on admission through the use of the medical record review and condition onset codes. Delirium rates will also be presented on a per patient per day basis due to the evidence linking length of stay and delirium.13 Primary and secondary outcomes will be adjusted for variables collected in the medical record review including: demographic data, risk factors for developing delirium, admission ward and evidence of reduced cognitive function admission. Under the terms of the Standard, each hospital will determine the most appropriate tests to screen and diagnose delirium. We will collect the scores for these tests and construct severity scores for those tests that have been validated to assess severity.

Aim 2: cost-effectiveness

The incremental costs of implementing the Standard, including the changes in resource use resulting from the intervention, will be determined through analysis of the implementation model, activity-based costing analysis and interviews with hospital management. The impact on patient outcomes will be modelled through the change in discharge disposition, length of stay and changes in health utilities associated with delirium.10 Incremental cost-effectiveness ratios will be calculated by dividing the mean incremental costs by the mean difference in outcomes, and a sensitivity analysis will be performed for the main parameters.32 Resources and outcomes will be considered within a 1-year time frame. Adjustment rates of 5% will be used where costing analysis is performed outside a common 1-year period. A sensitivity analysis will be performed using 1%, 5% and 10% changes for the main cost parameters.

Aim 3: implementation effectiveness and summative evaluation

The results of the staff interviews and analysis of the implementation model development process will be used to assess both the resources required to design the individual components of the Standard, and the overall effectiveness of the Standard implementation, using the RE-AIM framework and checklist.26 A summative evaluation report will be compiled to combine these results and be presented to stakeholders.33 Implementation science techniques and feedback tools will be used to investigate the core challenges in effective translation of the Standard into clinical practice. This will incorporate both quantitative measures, for example, medical record review data, and qualitative outcomes, for example, hospital staff perceptions of implementation challenges. Implementation science components include: broad inclusion criteria, ongoing consumer and stakeholder engagement, a participatory research approach with stakeholders, and the use of process and outcome indicators. The report will provide validation of the generalisability of the results.29 34

Implications of this research

Delirium has been shown to have a significant impact on patient outcomes but most importantly up to 30%–40% of cases are deemed preventable using evidence-based guidelines for implementing changes to inpatient care.10 35 36 Given the national and international significance of the condition, it is critical to have a better understanding of whether interventions to detect, prevent and treat delirium are effective. We hypothesise that the results of this study will: (1) show an increase in the incidence of delirium due to a higher level of vigilance and screening by trained staff, (2) provide prevalence and incidence rates of delirium in Australian acute care, (3) use process indicators and qualitative analysis to illustrate any issues surrounding implementation of the Standard, including identifying criteria within the Standard that have been more challenging to implement and (4) use clinical indicators and cost-effectiveness analysis to determine the longer-term impact of the Standard on patient outcomes. This study therefore has important implications for health policy-makers, aged care agencies, health quality bodies and health funding bodies both nationally and internationally. The research will have direct translational impact in terms of assessing the incidence and impact of delirium in the acute care sector.

Ethics and dissemination

The results from the study will be submitted for publication in peer-reviewed journals, and to national and international conferences relating to health policy development and implementation, cognitive function and deterioration, and patient safety and quality. An implementation report will be compiled for each hospital and presented to clinical staff and management. The summative evaluation report will be presented to the ACSQHC.