Deepening our Understanding of Quality in Australia (DUQuA): a study protocol for a nationwide, multilevel analysis of relationships between hospital quality management systems and patient factors.

INTRODUCTION: Despite the growing body of research on quality and safety in healthcare, there is little evidence of the association between the way hospitals are organised for quality and patient factors, limiting our understanding of how to effect large-scale change. The 'Deepening our Understanding of Quality in Australia' (DUQuA) study aims to measure and examine relationships between (1) organisation and department-level quality management systems (QMS), clinician leadership and culture, and (2) clinical treatment processes, clinical outcomes and patient-reported perceptions of care within Australian hospitals. METHODS AND ANALYSIS: The DUQuA project is a national, multilevel, cross-sectional study with data collection at organisation (hospital), department, professional and patient levels. Sample size calculations indicate a minimum of 43 hospitals are required to adequately power the study. To allow for rejection and attrition, 70 hospitals across all Australian jurisdictions that meet the inclusion criteria will be invited to participate. Participants will consist of hospital quality management professionals; clinicians; and patients with stroke, acute myocardial infarction and hip fracture. Organisation and department-level QMS, clinician leadership and culture, patient perceptions of safety, clinical treatment processes, and patient outcomes will be assessed using validated, evidence-based or consensus-based measurement tools. Data analysis will consist of simple correlations, linear and logistic regression and multilevel modelling. Multilevel modelling methods will enable identification of the amount of variation in outcomes attributed to the hospital and department levels, and the factors contributing to this variation. ETHICS AND DISSEMINATION: Ethical approval has been obtained. Results will be disseminated to individual hospitals in de-identified national and international benchmarking reports with data-driven recommendations. This ground-breaking national study has the potential to influence decision-making on the implementation of quality and safety systems and processes in Australian and international hospitals. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

This is an in-depth study examining quality management systems and patient factors in a large sample of hospitals, substantially advancing earlier work.

We consulted widely with Australian and international stakeholders in order to operationalise definitions and measures in acute myocardial infarction, stroke and hip fracture.

The study informs the design of future interventions and quality management systems in Australia and internationally.

Key challenges include sufficient sample sizes, the recruitment of hospitals (n=∼60), complex ethical, governance and site approval processes and attrition of recruited hospitals during the approval process.

Introduction

Despite decades-long efforts to improve healthcare delivery systems worldwide, there remain deficiencies in the quality and safety of care patients receive.1–4 Hospitals are continuously assessed for levels of quality, safety and performance using a range of methods, bolstered by policies and regulatory activities at national, state or regional, local, and hospital levels.5 Assessments can be resource-intensive, and are often conducted with invalidated tools, or in a piecemeal manner.6 Results are typically used to demonstrate policy compliance, rather than to stimulate improvement.

Few studies in healthcare research systematically examine relationships between quality interventions and patient outcomes.7 Despite the growing body of research, there is little evidence of the association between the way hospitals are organised for quality and patient outcomes, limiting our understanding of how to effect large-scale change.8 9 In recognition of this, a large multinational study ‘Deepening our Understanding of Quality Improvement in Europe’ (DUQuE) was undertaken between 2009 and 2014.10 This landmark study, undertaken in 188 hospitals across seven countries (Czech Republic, France, Germany, Poland, Portugal, Spain and Turkey), examined the effectiveness of quality management systems (QMS) and strategies at organisation, department, professional and patient levels for stroke, acute myocardial infarction (AMI), hip fracture and obstetrics. DUQuE aimed to improve the way quality strategies and their effects on performance and outcomes were measured in European hospitals11–14 by uncovering the relationships between them. This study also aimed to identify factors influencing the uptake of quality management activities by hospitals, and establish the feasibility of using routinely collected data to compare hospital performance.10

DUQuE developed and validated several tools for European acute care hospitals, assessing the implementation of QMS,15 compliance with quality processes and the extent of implementation of clinical quality strategies at hospital level,16 departmental performance and the implementation of quality management activities across specific departments and care pathways,17 and levels of clinical management provided by doctors and nurses.18 Key published findings from DUQuE include:

Low baseline performance and high variations were found on a wide range of evidence-based quality and safety indicators; levels of patient involvement in quality management were found to be low and appeared tokenistic;19 20

Quality management was associated with clinical effectiveness of care, and to a lesser extent for patient safety culture; however, no link with patient-reported experience of care was established;19 21

Greater variation of evidence-based organisation and patient safety strategies was found within, rather than between, countries;22

An organisational management structure which supports innovation in care was associated with a more developed QMS;23

Discussions of quality performance at executive board-level meetings were associated with higher levels of quality management;24 and

Accreditation and certification were associated with clinical leadership, systems for patient safety and clinical review, but not with clinical practice.25

DUQuE produced an evidence-based guide on effective quality and safety strategies to assist purchasing agencies and hospital managers with decisions on implementation and improvement.26 However, despite providing evidence of associations between hospital quality and a range of processes and outcomes, generalisability of DUQuE findings are limited to Europe. Extending this work to other healthcare systems will provide in-depth understanding of the relationships that exist within these systems, and will enable hospital benchmarking using standardised measures both within and between countries—an international first.

While DUQuE took a pathway analysis approach, it did not distinguish between QMS and clinical treatment processes delivered in emergency departments (EDs) and those of inpatient departments treating patients with stroke, AMI and hip fracture. Given that the ED is a common point of entry for all three conditions, inclusion of ED factors in the data for each condition is likely to bias the study towards ED performance. It is also likely that differences will exist between the quality of care delivered in the ED and other hospital departments providing care for stroke, AMI and hip fracture following admission, thereby biasing the findings for those departments.27 Although there is work distinguishing the quality of care provided in the ED, and other inpatient department care,28 the relationship between quality strategies used in the ED, and quality of care provided in the ED, has not been assessed. In addition, no distinction has been made between quality strategies and quality of care provided in the ED and quality strategies and quality of care provided in subsequent inpatient departments for any acute medical condition. Understanding the components of ED care in relation to specific care pathways can help identify which aspects of care are attributable to the ED versus other wards or departments. It is important to monitor the critical ED segment of care and distinguish this from care provided elsewhere,28 so that barriers to the delivery of guideline-concordant, evidence-based care can be identified and addressed using relevant, context-specific interventions.29

The Deepening our Understanding of Quality in Australia (DUQuA) study involves substantial modification to the DUQuE study design, to advance the international evidence base. DUQuA aims to measure and examine relationships between organisation, department, and patient level factors within Australian hospitals. Specific research questions are:

What department-level factors (ie, QMS, leadership, culture) are associated with patient-level factors (ie, clinical treatment processes, patient perceptions of care and nationally recorded clinical outcomes)?

What organisation-level factors (ie, QMS, size, location) including ED factors (ie, QMS, leadership, culture) are associated with patient-level factors (ie, clinical treatment processes, patient perceptions of care and nationally recorded clinical outcomes)? How much does each factor contribute to the total variation in patient factors?

Methods and analysis

Conceptual framework

Working with DUQuE research leads (RS and OG), members of the DUQuA team (RC-W, NT and JB) assessed the evidence from DUQuE to refine the conceptual model. The DUQuA team (RC-W, NT, VP, EH and JB) developed the analytical framework iteratively at strategic and methodological meetings, streamlined the original model, and built the final main constructs and the relationships we aim to test.

In line with DUQuE, we aim to further our understanding of the characteristics of Australian hospitals and hospital departments, and examine links to the quality of care provided to patients.9 10 The DUQuA conceptual model (figure 1) outlines the expected relationships between organisation (hospital) and department-level constructs with patient-level factors (ie, clinical treatment processes, patient perceptions of care and nationally recorded outcomes) in three care pathways: stroke, AMI and hip fracture. Selection of these conditions is addressed below.

Figure 1

DUQuA conceptual model (DUQuA, Deepening our Understanding of Quality in Australia; QMS, quality management systems).

The specific objectives of DUQuA are to assess relationships between: (1) department-level QMS, leadership, culture and patient-level factors; (2) organisation-level QMS and patient-level factors; and (3) ED QMS, leadership, culture and patient-level factors.

Research design

The DUQuA project is a multilevel, cross-sectional study with data collection at organisation, department, professional and patient levels in hospitals across Australia. Organisation-level measures will consist of external assessment and self-reported QMS, and department-level measures will consist of external assessment of QMS and self-reported perceptions of leadership and culture. Patient-level measurements based on clinical treatment processes, patient perceptions of care and nationally recorded clinical outcomes are nested in hospital departments, which are in turn nested in hospitals in each of the six Australian states and two territories.

Sample size

Given our research design requires multilevel analysis, we used the following equation30 to determine the design effect of the study and inflate the sample size accordingly: where nh is the number of hospitals, mp the number of departments needed in a one-level regression according to a standard sample size calculation, np,h the number of departments per hospital (in this case three), and ICC the intraclass correlation coefficient.

The sample size for the DUQuA multilevel model is the design effect multiplied by the sample size required for a standard linear regression. To identify which ICC to use to calculate our sample size, we used an α of 0.05 and power of 80%, in line with convention, and undertook a literature search of similar studies and ICCs. Taljaard et al31 published a set of ICCs for maternal health outcomes from the 2005 WHO Global Survey on Maternal and Perinatal Health. The ICC for the only continuous process variable analysed was 0.081, the median ICC for all process measures was 0.161, and the median for all continuous measures was 0.05. The same study indicated that ICCs for institutions tended to be larger for clinical treatment processes than patient outcomes. Similarly, for process variables in primary care settings, Campbell et al32 found that the ICC tended to range between 0.05 and 0.15, compared with outcome variables which were generally less than 0.05. James33 reported a median ICC of 0.12 for measures of organisational climate, and Forbes and Taunton34 reported a hospital-level ICC of 0.03–0.07 for three organisational phenomena. Smits et al35 also published hospital-level ICC for patient safety culture-related outcomes, all of which are under 0.07.

Taking this into account, an ICC of 0.16, higher than most ICCs in the literature, was used to calculate a conservative sample size. It reflects the median from the maternal health process measures, the most similar group of available reported ICCs. For this ICC, the minimum sample size required to answer our primary research question is 43 hospitals. However, in order to increase the power of the study, to be in line with simulation papers (which have shown that standard errors are estimated to be inadequate by 15% with <50 clusters),36 and to allow for hospital refusal to participate or subsequent withdrawal, a decision to oversample at the organisation level was made.

Selection of clinical conditions

DUQuE originally tested relationships between QMS and four conditions (AMI, stroke, hip fracture and obstetrics). These conditions were selected because they cover a broad range of hospital care, there are evidence-based standards for clinical treatment processes against which compliance could be assessed, and there is demonstrated variability in both compliance with clinical treatment process measures and nationally reported outcomes of care, such as complications or mortality, allowing for analysis of associations between these measured constructs.10 DUQuE conditions were retained except obstetrics; in Australia, maternity care is often provided in dedicated hospitals. Replacement conditions were considered but none were deemed suitable due to a lack of evidence-based standards for clinical treatment processes, or a lack of variability in compliance with processes or outcomes.

Selection and recruitment of hospitals

Hospitals will be approached in all jurisdictions: the Australian Capital Territory (ACT), New South Wales (NSW), the Northern Territory (NT), Queensland (QLD), South Australia (SA), Tasmania (TAS), Victoria (VIC) and Western Australia (WA). To guarantee adequate data for collection, the inclusion criteria are (1) general public hospitals with approximately 200 beds or more, (2) with an ED, and (3) that regularly admit more than 30 each of stroke, AMI and hip fracture patients over a 3-month period. Preliminary work indicated that 70 hospitals meet these criteria and will be invited to participate. Table 1 lists the number of hospitals identified as eligible to participate by state.

Table 1

Number of eligible hospitals by state or territory

State/territory	Number of eligible hospitals
Australian Capital Territory	2
New South Wales	24
Northern Territory	1
Queensland	13
South Australia	4
Tasmania	2
Victoria	18
Western Australia	6

The Director of Medical Services (DMS), or an equivalent senior hospital leader, will be the initial point of approach for recruiting hospitals. Following discussion with the DMS and other relevant hospital stakeholders, a letter will be sent to the hospital executive team of eligible hospitals inviting their participation. The DMS or equivalent will be recruited as a ‘Local Principal Investigator’ (LPI) at each hospital that accepts this invitation, acting as a hospital coordinator. For each hospital, one or more healthcare professionals will be recruited to support the LPI and coordinate data collection. The Royal Australasian College of Medical Administrators (RACMA) has agreed to support the DUQuA team during recruitment and data collection. Other recruitment methods, including engaging a hospital executive network, or personal invitation, will be employed as necessary. LPIs and healthcare professionals will assist in operationalising the standardised recruitment strategy for staff and patients.

Selection and recruitment of participants

The source population for the study includes the Quality Manager at each participating hospital; all clinicians treating patients with stroke, AMI and hip fracture; and all patients with stroke, AMI and hip fracture meeting the study inclusion criteria. Where possible, we have adopted the same inclusion/exclusion criteria as DUQuE (The DUQuE Project team. Deepening our Understanding of Quality Improvement in Europe: Manual for Hospital Coordinators [unpublished material], 2011.). Specific details on selection of participants follows.

Quality Manager

Description

This is the person responsible for the coordination of quality improvement activities in the hospital. He or she should have a good overview of all activities supporting quality improvement. This may be a formally appointed Quality Manager, or another professional in charge of managing aspects of quality in the hospital.

Recruitment

The Quality Manager will be invited by the DUQuA team to complete the Quality Management Systems Index (QMSI) questionnaire (see ‘Organisation-level measures’). If more than one professional meets the description, then the most experienced person will be invited to participate until one questionnaire is completed.

Clinicians

Clinicians will be invited to complete leadership and culture questionnaires. Clinician inclusion criteria are doctors, nurses and allied health professionals practising in the clinical area of the ED or a particular pathway (stroke, AMI, hip fracture) for at least 50% of their work time.

All clinicians that meet the inclusion criteria will be invited to participate in the leadership and culture questionnaire (see ‘Department-level measures’) until a minimum of 20 questionnaires for each department are completed at each hospital. The DUQuA team will invite clinicians to complete the questionnaire, by coordinating the dissemination of a web link to the online questionnaire.

Patients

There are two patient-level measures that involve the collection of patient data. The inclusion criteria for each condition are listed in table 2. Patients who are admitted to the hospital with a primary diagnosis other than those listed in table 2 will be excluded.

Table 2

Patient inclusion criteria

Condition	Inclusion criteria
Stroke	Patients aged 18 years and older, with a principal diagnosis code of acute ischaemic stroke OR not specified stroke. Include patients with a principal diagnosis code of: (1) ICD 10 I63 or (2) ICD 10 I64
AMI	Patients aged 18 years and older, with a principal diagnosis code of AMI according to: (1) ICD 10 I21 or ICD 10 I22 and (2) ECG changes associated with STEMI: new LBBB or persistent ST-segment elevation ≥1 mm in two or more contiguous ECG leads and (3) blood sampling shows elevated serum markers of myocardial necrosis for creatine kinase MB form and troponins
Hip fracture	Patients aged 65 years and older, with at least one of the primary diagnosis criteria of: (1) fractura colli femoris (ICD 10 S72.0) or (2) fractura pertrochanterica (ICD 10 S72.1) or (3) fractura subtrochanterica femoris (ICD 10 S72.2)

AMI, acute myocardial infarction; ICD, International Classification of Diseases; LBBB, left bundle branch block; MB, myocardial band; STEMI, ST-segment elevation myocardial infarction.

Recruitment for patient perceptions of care measure

All patients meeting the inclusion criteria who present to the hospital during the study period will be invited to complete the questionnaire. Additional criteria and screening will be applied to ensure that participation is voluntary and that participants are able to give informed consent. Healthcare professionals at each hospital will be trained to invite patients to complete the questionnaire. Recruitment will continue until a minimum of 30 questionnaires are completed for each condition at each hospital.

Clinical treatment processes indicator audit

The inclusion criteria (table 2) will be applied to records of patients admitted to the hospital between September 2014 and February 2015. Records of patients admitted to the hospital with a primary diagnosis other than those listed in table 2 will be excluded. For each condition in each hospital, 60 consecutive patient medical records will be extracted and reviewed until 30 audits of patients meeting the inclusion criteria have been completed. Healthcare professionals trained by the DUQuA team will undertake the medical record reviews.

Development, selection and collection of measures

Measures developed and validated for DUQuE9 1015–17 22 37 were initially considered for DUQuA. During a 3-day consultation with the DUQuE lead researchers (RS and OG), we agreed on which constructs should be excluded from the DUQuA study based on a lack of relevance to the Australian healthcare system, poor variance demonstrated by a particular measure, an absence of a relationship with any other variables measured in DUQuE, or misalignment of the construct with our revised theoretical framework. Excluded constructs are: external pressure, hospital governance, hospital culture, hospital professional involvement, patient involvement in quality management and professionalism.

To establish new constructs not assessed as part of DUQuE (ie, department-level leadership and culture), literature reviews were performed to identify existing measures. Explicit criteria were used to assess and select measures, including psychometric properties, level of evidence and appropriateness for the Australian healthcare system. In particular, the significance of interhospital transfers in the care of patients with stroke, AMI and hip fracture was identified. Australian studies indicate that 14% of patients with hip fracture and a quarter of patients with AMI are transferred between hospitals, and 63% of Australian hospitals have ED protocols for interhospital transfers of patients with stroke.38–40 Consequently, variables regarding the potential impact of interhospital transfer were incorporated to allow for their measurement.

Where the DUQuE measures or items were unsuitable for the Australian context, or where it was felt that newer evidence-based measures were available, we reviewed the literature for new measures, consulted with expert clinicians and refined specific items to ensure measures were appropriate and relevant. Stakeholder groups of healthcare professionals and researchers with expertise in clinical practice, quality or both were established to inform revisions to DUQuA measures and the introduction of new measures. New measures have been piloted in at least one hospital (June 2015 and September 2015) to assess suitability and feasibility of use. The final list of DUQuA measures, content descriptions, evidence of reliability and validity, and data collection methods, is provided in table 3.

Table 3

DUQuA measures: content, evidence of reliability and validity, and collection methods

Measures	Content	Reliability and validity	Collection methods
Organisation-level QMS
QMSI	Ten subscales: quality policy; hospital governance board activities; quality resources; quality management; evidence-based medicine protocols; preventive protocols; internal quality methods (for general activities, personnel, clinical practice, patients)	Satisfactory internal consistency (0.72–0.82) was demonstrated for eight scales. The scale with the low coefficient—analysing feedback and patient experiences (α=0.48)—was retained due to the theoretical importance of this topic15*	Self-report questionnaire completed by the hospital's Quality Manager or equivalent (n=1)
QMCI	Four subscales: quality planning; monitoring patient and professional opinions; quality control and monitoring; improving quality of care	Cronbach's reliability coefficients were satisfactory (0.74–0.78) for the four scales16*	Quality assessment by experienced hospital surveyor (site visit)
CQII	Seven subscales: preventing and controlling healthcare associated infections; medication safety; preventing patient falls; preventing pressure injuries; routine assessment and diagnostic testing of patients in elective surgery; safe surgery that includes an approved checklist; recognising and responding to clinical deterioration in acute healthcare	Cronbach's reliability coefficients were satisfactory (0.82–0.93) for the seven scales16*	Quality assessment by experienced hospital surveyor (site visit)
Department-level QMS
SER	Assignment of clinical responsibilities for a condition	Factor loadings and Cronbach's α values reported as: AMI (0.58–0.63, α=0.69), stroke (0.29–0.50, α=0.46), and hip fracture (0.65–0.69, α=0.76)17*	Quality assessment by experienced hospital surveyor (site visit)
EBOP	Organisation of department processes (admission, acute care, and discharge to facilitate evidence-based care recommendations)	It was not possible to build one generic scale for the EBOP, because it consists of different items across pathways17*	Quality assessment by experienced hospital surveyor (site visit)
PSS	Use of international consensus based patient safety recommendations	Despite the same items being used across pathways for PSS, factor analysis did not produce a generic scale for the four pathways17*	Quality assessment by experienced hospital surveyor (site visit)
CR	Integration of audit and systematic monitoring in departmental quality management mechanisms	Factor loadings and Cronbach's α values reported as: AMI (0.64–0.91, α=0.86), stroke (0.65–0.93, α=0.84), and hip fracture (0.36–0.91, α=0.76)17*	Quality assessment by experienced hospital surveyor (site visit)
Department-level culture and leadership
SAQ and Shipton and colleagues’ Leadership Effectiveness Scale	Three subscales: teamwork climate (the perceived quality of collaborating between personnel); safety climate (perceptions of a strong and proactive organisational commitment to patient safety); perceptions of management (administrative and managerial support, staffing levels, and managerial style in the workplace)Leadership effectiveness (staff perceptions of the effectiveness of healthcare leaders in their workplace)	Composite scale reliability for the SAQ was 0.90 (Raykov's ρ coefficient), indicating strong reliability41Validated in a survey of approximately 18 000 employees of the National Health Service in the UK (Cronbach's α=0.92)42	Combined into a self-report questionnaire completed by doctors, nurses, and allied health professionals working in stroke, AMI, and hip fracture wards, and the ED (n=80)
Patient-level clinical treatment processes
Clinical audit tools	Nationally recognised process composite indicators based on evidence of impact on patient outcomes	NA	Patient data retrieved from national registries and/or by medical record review (n=90)
Patient-level outcomes
Nationally collected audit data	Includes readmission and mortality rates, and length of stay	NA	Collected from publicly available national data
Patient perceptions of care
PMOS	Eight domains: communication and team work; organisation and care; access to resources; ward type and layout; information; staff roles and responsibilities; staff training; equipment	Reliability was established using Cronbach's α (0.66–0.89) and test-retest reliability (r=0.75). The positive index significantly correlated with staff reported ‘perceptions of patient safety’ (r=0.79) and ‘patient safety grade’ (r=−0.81) outcomes from the Agency for Healthcare Research and Quality Safety Culture Survey, demonstrating convergent validity43	Self-report or assisted questionnaire completed by patients meeting the inclusion criteria (AMI, stroke, and hip fracture) (n=90)

n, participants per hospital.

*Reliability and validity details for original DUQuE measures, not the adapted versions.

AMI, acute myocardial infarction; CR, Clinical Review; CQII, Clinical Quality Implementation Index; DUQuE, Deepening our Understanding of Quality Improvement in Europe; EBOP, Evidence-Based Organisation of Pathways; ED, emergency department; NA, not available; PMOS, Patient Measure of Safety; PSS, Patient Safety Strategies; QMS, quality management systems; QMCI, Quality Management Compliance Index; QMSI, Quality Management Systems Index; SAQ, Safety Attitudes Questionnaire; SER, Specialized Expertise and Responsibility.

Organisation-level measures

Development and selection

The three measures of organisation-level QMS are the QMSI, the Quality Management Compliance Index (QMCI) and the Clinical Quality Implementation Index (CQII). These measures, validated by DUQuE, have been modified by the DUQuA team in consultation with national quality assessment experts (eg, the Australian Council on Healthcare Standards) for the Australian context and to reflect changes made to the condition-specific indicators. The adapted QMSI is a seven-subscale survey which provides a proxy measure for the managerial aspects of quality management that might influence the implementation of quality systems in hospitals.15 The adapted QMCI is a four-subscale proxy measure for compliance with procedures used to plan, monitor and improve quality of care.16 The adapted CQII is a seven-subscale proxy measuring implementation of QMS, including whether specialised quality management committees and hospital policies exist, and the level of compliance monitoring, system sustainability and improvement focus.16

Collection

The Quality Manager will complete the self-report paper QMSI questionnaire. The QMCI and CQII will be completed in a quality assessment site visit by experienced external hospital surveyors who will be trained by the DUQuA team.

Department-level measures

Quality management systems

Development and selection

A modified version of a checklist developed and validated by DUQuE17 will assess department-level QMS. The four measures (see table 3) are Specialized Expertise and Responsibility (SER), Evidence-Based Organisation of Pathways (EBOP), Patient Safety Strategies (PSS) and Clinical Review (CR).17

Collection

Department-level QMS assessment will be completed in a quality assessment site visit by experienced external hospital assessors drawn from a pool of Australian Council on Healthcare Standards surveyors who will be trained by the DUQuA team.

Culture and leadership

The culture measure consists of two domain subscales from the Australian version44 of the validated Safety Attitudes Questionnaire (SAQ)41 ‘Teamwork climate’ and ‘Safety climate’. Two scales, each previously validated and measured on a four-point Likert scale, were combined to form the leadership measure: the ‘Perceptions of management’ domain subscale from the SAQ,41 and the Leadership Effectiveness Scale.42 These measures are combined in one questionnaire.

Clinicians will complete the self-report questionnaire online.

Patient-level measures

Clinical treatment processes indicators

Selection

The DUQuA team started with indicators for each condition used in the DUQuE study,10 and refined these following consultation with expert stakeholder groups for each condition and the Australasian College for Emergency Medicine (ACEM), and examination of indicators used by condition-specific Australian national registries. Box 1 presents the final list of DUQuA clinical treatment process indicators.

▸ Clinical treatment process indicators (acute ischaemic stroke OR not specified stroke)

1. Patient screened for stroke according to validated stroke screening tool (eg, FAST or ROSIER scale)

2. Patient screened for eligibility* for thrombolysis (intravenous recombinant tissue plasminogen activator (Rt-PA))

3. Patient received a brain scan within 60 min of arrival to the hospital

4. Eligible* patient received intravenous rt-PA†

5. Eligible* patient received intravenous rt-PA† within 60 min of arrival to the hospital

6. Swallow screen or swallow assessment performed before the patient was given oral food or fluids, or oral medication

7. Patient admitted to stroke unit

8. Patient spent at least 90% of their acute hospital admission on a stroke unit

9. Patient discharged on statin therapy or lipid-lowering medication unless contraindicated‡

10. Patient without atrial fibrillation discharged on antiplatelet therapy unless contraindicated‡

11. Patient with atrial fibrillation discharged on oral anticoagulants unless contraindicated‡

12. Patient discharged on an antihypertensive agent unless contraindicated‡

13. Patient received risk factor modification advice before leaving the hospital unless patient had severe cognitive impairment or communication impairment, or otherwise contraindicated‡

14. Ongoing care plan developed with and provided to the patient and/or family prior to discharge

▸ Clinical treatment process indicators (acute ST-segment elevation myocardial infarction)

1. ECG performed and interpreted before or within 10 min of arrival to the hospital for patient with chest pain or other acute coronary syndrome symptoms

2. Patient received aspirin prior to arrival at the hospital

3. Patient received aspirin prior to being sent to the catheterisation laboratory

4. Eligible§ patient received fibrinolysis or percutaneous coronary intervention (PCI)¶

5. Eligible§ patient received fibrinolysis before or within 30 min of hospital arrival

6. Eligible§ patient, treated with PCI,¶ had a door-to-device time of 90 min or less for PCI-capable hospital, or 120 min or less if transferred from a non PCI-capable hospital

7. Patient discharged with a written, individualised care plan (such as graded physical activity, smoking cessation therapies and addressing psychosocial needs) unless discharged on palliative care pathway or where adherence to a secondary prevention plan is not indicated

8. Patient discharged on aspirin or dual antiplatelet therapy unless contraindicated‡

9. Patient discharged on a statin or lipid-lowering therapy unless contraindicated‡

10. Patient referred to cardiac rehabilitation or other secondary prevention programme unless contraindicated‡

11. Discharge summary provided to general practitioner or ongoing clinical provider of patient within 48 h of discharge

▸ Clinical treatment process indicators (hip fracture)

1. Patient initial pain score recorded within 30 min of arrival to the hospital

2. Patient received analgesia or nerve blocks within 30 min of arrival to the hospital, unless patient declined pain relief

3. Patient pain reassessed within 60 min of arrival to the hospital

4. Orthopaedic team notified within 60 min of patient arrival to the hospital

5. Surgery performed within 48 h of arrival to the hospital

6. Patient received prophylactic antibiotic treatment within 60 min prior to surgical incision

7. Patient received prophylactic thrombolytic treatment within 48 h of arrival to the hospital

8. Surgery performed with the aim of allowing patient to fully weight bear without restriction in the immediate postoperative period

9. Patient mobilisation started day after surgery unless contraindicated‡ or patient declined

10. Patient offered a dedicated mobilisation session to regain function at least once per day until discharge

11. Patient received a specialist falls assessment (including falls history, cause of fall index, and risk factors for falling and injury) from a trained clinician prior to discharge

12. Patient received bone protection medication for secondary fracture prevention at discharge

*Eligibility for intravenous rt-PA: exclude patients who present at first emergency clinical contact more than 4.5 h after symptom onset, patients with unknown time of onset or patients where thrombolysis is contraindicated.

†Intravenous rt-PA should only be given in hospitals with appropriately qualified clinicians and adequate infrastructure and facilities.48 Evidence of transfer policies and procedures will be sought in hospitals without the capability to perform intravenous rt-PA.

‡Contraindications include advanced care directives, being on a palliative care pathway and clinical judgement. Contraindication must be documented.

§Eligibility for reperfusion: exclude patients who present at first emergency clinical contact more than 12 h after symptom onset, or where reperfusion is contraindicated.

¶PCI should only be given in hospitals with appropriately qualified clinicians and adequate infrastructure and facilities.49 Evidence of transfer policies and procedures will be sought in non PCI-capable hospitals.

Development of stroke indicators

DUQuE clinical treatment process indicators for stroke were presented to an expert stakeholder group that included members of the National Stroke Foundation (NSF). The NSF collects cross-sectional data in a biennial review of 40 consecutive eligible patient medical records by trained auditors of stroke clinical treatment processes over a 6-month period, from approximately 177 hospitals across Australia. Indicators developed for this registry align with the ‘Indicator Specification: Acute Stroke Clinical Care Standard’, developed by the Australian Commission on Safety and Quality in Health Care (ACSQHC) and certified by the Federal Minister for Health.45 Therefore, the indicators were amended to align with the NSF Clinical Audit and the national ACSQHC standards.

Development of AMI indicators

DUQuE AMI clinical indicators were compared against the ‘Indicator Specification: Acute Coronary Syndromes Clinical Care Standard’, developed by the ACSQHC and certified by the Federal Minister for Health.46 Final indicators chosen for DUQuA align with the relevant indicators in this document.

Development of hip fracture indicators

DUQuE hip fracture clinical indicators were modified for DUQuA in consultation with the Co-Chair of the Australian and New Zealand Hip Fracture Registry (ANZHFR), and the guidelines produced by the ANZHFR.47

An audit of clinical treatment process indicators will be undertaken. For hospitals that submit data to the NSF Clinical Audit, data will be directly retrieved from the most recent audit (September 2014 to February 2015) within the NSF database.[i] In hospitals not contributing to the NSF Clinical Audit, data will be collected via medical record review. As no national AMI or hip fracture registries were active during the NSF data collection period, AMI and hip fracture clinical treatment process data will also be collected via medical record review.

Patient perceptions of care

Development and selection

A modified version of the eight-scale, self-report UK-validated Patient Measure of Safety (PMOS) questionnaire,43 50 which assesses patients’ perceptions of the factors contributing to patient safety, will measure subjective patient factors. The instrument presents 43 items on a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree).

The PMOS has been piloted in one hospital (ethically approved, HREC reference LNR/14/157) with stroke, AMI, and hip fracture patients. Stage 1 involved four patients who participated in a ‘think aloud’ study to provide an indication of the suitability of the content, length, and presentation of the measure. Stage 2 tested the refined measure with 31 patients to establish the suitability of the PMOS for use with a cohort of patients similar to patients likely to participate in DUQuA, and demonstrate the level of variability produced by the measure.

Collection

Patients will complete the self-report paper questionnaire. Patients may request assistance to complete the questionnaire from a family member or the trained clinician responsible for administering the questionnaire.

Nationally recorded clinical outcomes

Development and selection

Clinical outcomes will be determined by analysing routine data collected by hospitals and submitted to the Australian Institute of Health and Welfare (AIHW), a national Australian Government data collection agency. Data will include the number of cases of each condition discharged by diagnosis, (including stratification by age and gender), in-hospital mortality, length of hospital stay, readmission, and mortality. Data will also be collected for the number of: percutaneous coronary interventions (AMI), thrombolysis procedures (stroke), and hip fracture operations performed in each hospital over the study period.

The DUQuA team will retrieve the required data from the AIHW.

Data analysis

The analysis plan for each research question is presented in table 4. Descriptive statistics will summarise the spread, frequencies or distributions of the variables of interest. Simple correlations, linear and logistic regression and multilevel modelling will be undertaken. The exact approach to the statistical analysis will depend on the distribution of the data, the variables that will need to be adjusted for (eg, case mix), and the final sample size achieved.

Table 4

DUQuA statistical analysis plan

Research question	Statistical analysis	Test
What department-level factors are associated with patient-level factors?	Test the relationship between department-level factors and patient-level factors, adjusting for higher level hospital variation	Multilevel modelling
	Test the relationship between the ED variables and patient-level factors	Regression modelling
	Test the relationship between patient perceptions, clinical treatment processes and clinical outcomes	Correlations
	Test the effect of transfers on patient-level factors	Regression, correlation
What organisation level-factors are associated with patient-level factors?How much to organisation factors contribute to the total variation?	Test the relationship between organisation-level QMS and department-level QMS, leadership, and culture	Regression, correlation,
	Test the relationship between organisation-level factors and patient-level factors	Multilevel modelling, regression
	Identify the amount of variation in patient-level factors attributable to organisation-level and department-level variables	Multilevel modelling

DUQuA, Deepening our Understanding of Quality in Australia; ED, emergency department; QMS,quality management systems.

Multivariate modelling will take the form of a multilevel or hierarchical model in the first instance; with hospitals at the top level.[ii] This type of regression will adjust for both hospital-level and department-level covariates. Subject to the results of this modelling, the levels may be adjusted to select individual patient impact measures as level 1 or, alternatively, a single-level model utilised. Employing multilevel methods will allow for the amount of variation in patient-level factors attributed to the hospital and department level, as well as which factors contribute to this variation, to be identified.

Ethics and dissemination

Regionally based Human Research Ethics Committees (HRECs) in Australia have approved the DUQuA project for NSW (14/206)[iii], VIC (15/36)iii, ACT (15/131)iii, SA (15/260)iii, WA (15/118)iii and QLD (15/361)iii. Remaining regionally based ethical applications are under review, and governance requests for participating hospitals will be submitted following confirmation of the final sample.

Results of the DUQuA project will be disseminated to participating hospitals in the form of confidential national and international benchmarking reports with a set of data-driven recommendations. An extensive publication plan will support the dissemination of results including reporting on the international validation of measurement tools, associations between measures using linear and multilevel models, the extent to which hospital benchmarking can identify consistent high and low performers, and the impact of the ED and patient transfers on the delivery of care for the three selected conditions. This ground-breaking study, with international implications, has the potential to build a coalition of hospitals for subsequent interventional work, influence policymaker decisions on the implementation of quality and safety systems and processes, and ultimately improve the delivery of patient care in Australian hospitals.