From QASC to QASCIP: successful Australian translational scale-up and spread of a proven intervention in acute stroke using a prospective pre-test/post-test study design.

OBJECTIVES: To embed an evidence-based intervention to manage FEver, hyperglycaemia (Sugar) and Swallowing (the FeSS protocols) in stroke, previously demonstrated in the Quality in Acute Stroke Care (QASC) trial to decrease 90-day death and dependency, into all stroke services in New South Wales (NSW), Australia's most populous state.
DESIGN: Pre-test/post-test prospective study.
SETTING: 36 NSW stroke services.
METHODS: Our clinical translational initiative, the QASC Implementation Project (QASCIP), targeted stroke services to embed 3 nurse-led clinical protocols (the FeSS protocols) into routine practice. Clinical champions attended a 1-day multidisciplinary training workshop and received standardised educational resources and ongoing support. Using the National Stroke Foundation audit collection tool and processes, patient data from retrospective medical record self-reported audits for 40 consecutive patients with stroke per site pre-QASCIP (1 July 2012 to 31 December 2012) were compared with prospective self-reported data from 40 consecutive patients with stroke per site post-QASCIP (1 November 2013 to 28 February 2014). Inter-rater reliability was substantial for 10 of 12 variables. PRIMARY OUTCOME MEASURES: Proportion of patients receiving care according to the FeSS protocols pre-QASCIP to post-QASCIP.
RESULTS: All 36 (100%) NSW stroke services participated, nominating 100 site champions who attended our educational workshops. The time from start of intervention to completion of post-QASCIP data collection was 8 months. All (n=36, 100%) sites provided medical record audit data for 2144 patients (n=1062 pre-QASCIP; n=1082 post-QASCIP). Pre-QASCIP to post-QASCIP, proportions of patients receiving the 3 targeted clinical behaviours increased significantly: management of fever (pre: 69%; post: 78%; p=0.003), hyperglycaemia (pre: 23%; post: 34%; p=0.0085) and swallowing (pre: 42%; post: 51%; p=0.033).
CONCLUSIONS: We obtained unprecedented statewide scale-up and spread to all NSW stroke services of a nurse-led intervention previously proven to improve long-term patient outcomes. As clinical leaders search for strategies to improve quality of care, our initiative is replicable and feasible in other acute care settings. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

Evidence of successful ‘scale-up and spread’ of a complex, proven intervention across an entire state within a short 8-month time frame.

An example of large systems transformation involving multidisciplinary clinicians.

Collaboration between researchers who conducted the original trial, clinicians and quality improvement experts.

Our tight time frame may not have allowed enough time for full protocol implementation, as some barriers (eg, treatment of hyperglycaemia with insulin) require further attention.

Use of self-reported processes of care audit data.

Background

Implementation science has emerged as a rigorous field of enquiry aiming to generate better evidence about efforts to embed clinical evidence into routine healthcare practice.1 Since it can take an average of 17 years for implementation of evidence into standard practice,2 evidence to inform selection of strategies is sorely needed to accelerate the pace towards evidence-based practice in a more predictable manner. The Cochrane Effective Practice and Organisation of Care (EPOC) group has conducted systematic reviews of interventions aiming to improve knowledge translation.1 3 4 Active, targeted implementation strategies have been shown to be effective in closing evidence-practice gaps and changing clinician behaviour. However, no single implementation strategy is effective in all circumstances for all healthcare settings.1 Further research is required into effective models of care;5 6 in addition, the context and barriers to practice change known to influence the success of intervention implementation should be addressed effectively.1

One of the difficulties in achieving rapid uptake is that evidence generated from clinical trials is difficult to implement more widely, often due to lack of published details about precisely how the trial was conducted7 and a lack of process evaluation, in particular for complex interventions.8 9 ‘Scalability’ of interventions to promote evidence-based practice in healthcare is worthy of greater scientific study. ‘Scalability’ is ‘the ability of a health intervention shown to be efficacious on a small scale and or under controlled conditions to be expanded under real-world conditions to reach a greater proportion of the eligible population, while retaining effectiveness’.10 There are relatively few examples in the published literature where efforts to ‘scale-up and spread’ beyond elite academic centres have been systematically studied.10 Those that have been published have focused predominantly on population health initiatives such as mass immunisation programmes.11 12 Identification of these studies from the literature also is difficult due to the lack of an agreed taxonomy to classify and systematically report them. Furthermore, scalability is often limited by insufficient detail by researchers of the ‘nuts and bolts’ of a successful intervention.13

One example of a large-scale intervention involving scale-up and spread in the acute care setting is the Michigan Keystone project.14 This cohort study demonstrated improvements in rates of central venous catheter bloodstream infections in 103 intensive care units (ICUs) in the USA following introduction of a checklist of five evidence-based practices for management of central venous catheters, implemented using clinical champions, education and coaching. Simultaneously, ICUs implemented daily goal sheets to improve communication, an intervention to reduce ventilator-assisted pneumonia and a programme to improve the safety culture. Elements of the project subsequently were adapted and introduced into 200 ICU settings in the UK in the Matching Michigan study.15 While central venous catheter bloodstream infections dropped in the UK ICUs, the Matching Michigan study was not an exact replica of the original Michigan Keystone project, particularly in terms of implementation of the intervention.16

As reported elsewhere,17 our team previously developed a successful implementation intervention that aligned clinical practice in participating stroke units more directly with the evidence. This complex healthcare intervention resulted in significant improvements in patient outcomes. In brief, our Quality in Acute Stroke Care (QASC) trial showed that a multidisciplinary, nurse-initiated intervention focused on three clinical protocols to manage FEver, hyperglycaemia (Sugar) and Swallowing dysfunction (the FeSS protocols; box 1) in the first 72 h of patient admission significantly decreased death and disability by 16% (p=0.002). These dramatic improvements in death and dependency were larger than any pharmacological18 or organisational19 initiatives for acute stroke known at that time. To design that intervention, we had incorporated best practice from the field of implementation science to design a standardised intervention comprising systematic local barrier identification,20 reinforcement of multidisciplinary teamwork,21 local adaptation22 and use of site champions.23 This cluster randomised controlled trial also showed that this intervention changed process of care as well as patient outcomes by significantly reducing fever, glucose levels and improved swallow screening practices.24

Fever

Temperature readings 4–6 hourly for the first 72 h

If temperature>37.5°C, treat with paracetamol

Sugar (hyperglycaemia)

Formal venous glucose on admission to the emergency department or stroke service

Blood sugar readings 4–6 hourly for the first 72 h for people with known diabetes

Blood sugar readings 4–6 hourly for the first 48 h for people not known to have diabetes

If glucose>10 mmol/L, treat with insulin

Swallowing

Swallow screen or swallow assessment within 24 h of admission and prior to being given oral food, drink or medications

Referral to speech pathologist for full assessment for those who fail the swallow screen

Our next challenge as a team of clinicians, academics and health service managers was how to ‘scale-up and spread’ this effective intervention beyond original trial sites to reach all hospitals in New South Wales (NSW), the most populous Australian jurisdiction. We did not want to leave the uptake of this intervention to chance. Instead, we aimed to test our success in scaling up these three clinical protocols to all 36 NSW stroke services using those intervention elements demonstrated to be effective in the QASC trial. Known as the QASC Implementation Project (QASCIP), this statewide scale-up of our proven intervention was evaluated rigorously by measuring impact on clinical care for fever, hyperglycaemia and swallowing dysfunction management as described below. We report the findings based on the SQUIRE (Standards for Quality Improvement Reporting Excellence) guidelines.25

Methods

All NSW stroke services (n=36) were invited to participate in this prospective pre-test/post-test study: these comprised 31 sites with dedicated stroke units in large hospitals and five sites without dedicated stroke units but with integrated hospital stroke services based on agreed hospital service delineations.26 Those sites which had previously participated in the QASC trial were also eligible to participate (irrespective of whether they had been allocated to the intervention or control group).

A personal invitation letter and study summary were sent to senior clinical and management executives in each stroke service including chief executives of the respective local health district; clinical directors of all 36 stroke services; directors of nursing; directors of allied health; stroke clinical nurse consultants/coordinators and, where applicable, stroke service/stroke unit nurse unit managers. In our invitation, sites were asked to consent to nominate up to three clinical stroke champions to act as local change agents for ‘scale-up and spread’ of the intervention. Our study then faithfully replicated the intervention from the QASC trial as described below (box 2).

The QASCIP implementation intervention consisted of:

Informing Local Health District (LHD) chief executives and key health service managers

Engaging multidisciplinary clinicians and clinical champions at each participating hospital

A 1-day multidisciplinary training workshop for clinical champions in order to assess barriers and enablers, provide education and reinforce teamwork

Interactive educational meetings and provision of educational resources

Support in the form of site visits, telephone and email contact

Scale-up and spread initiative

Clinical champions from each participating hospital attended a 1-day educational workshop where education and training were provided about: (1) the FeSS protocols, including ASSIST swallow screening training (see below); (2) barrier and enabler identification; and (3) reinforcement of multidisciplinary teamwork. A small change to the Sugar protocol used in the QASC trial was made whereby the treatment point for raised glucose was lowered from 11 to 10 mmol/L to align with the newly released Australian Diabetes Society Guidelines for Routine Glucose Control in Hospital.27 The ASSIST swallow screening training package consisted of online education with case scenarios and a knowledge test to train non-speech pathologists (ie, nurses and medical staff) to competently perform a swallow screen for patients with acute stroke. Patients who failed the swallow screen were to be kept nil by mouth and referred to a speech pathologist for a swallow assessment. Clinical champions were also provided with implementation tools and educational materials including a prepackaged PowerPoint presentation for use in their sites; a ‘barrier and enabler’ assessment tool, an implementation plan template, and a suggested implementation and evaluation timeline (all are freely available for download at http://www.acu.edu.au/qasc). The aim of this suite of clinical and educational tools was to facilitate efforts by clinical champions at their own sites to lead implementation in their local stroke services based on an effective intervention.

Clinical champions were charged with targeting all clinicians in their stroke services. They were not obliged to engage their respective emergency department or ICU but rather to concentrate their efforts on changing practice in their stroke service. We allowed 1 month for clinical champions to return to their sites and begin implementation. Prior to the start of the postimplementation audit and consistent with the QASC trial,17 we also allowed for an additional 3-month bedding down period to establish the FeSS clinical protocols into routine care. All participating sites were visited by the project coordinator and the NSW Agency for Clinical Innovation (ACI) Stroke Network Manager at least once during this ‘bedding down’ period. The project coordinator provided monthly proactive and reactive ongoing support to the clinical champions via email and telephone.

Retrospective medical audit pre-QASCIP

To establish pre-QASCIP practice, following consent from stroke services, the National Stroke Foundation (NSF) provided the researchers with self-reported data previously collected independently of our study as part of the NSF National Clinical Audit (using patient data for stroke admissions between 1 July 2012 and 31 December 2012)28 and the NSF Organisational Survey (data collected 1 April to 31 May 2013).29 Using the established NSF audit web-based tool,30 clinical champions had conducted this retrospective audit of the records for the first 40 consecutive patients with a primary diagnosis of stroke admitted to the stroke unit between 1 July 2012 and 31 December 2012, excluding patients with subarachnoid haemorrhage, subdural and extradural haematoma, and transient ischaemic attacks. Any sites which had not participated in either of these previous NSF audits provided self-reported retrospective preintervention data directly to the researchers using the NSF web-based audit tool in an identical manner and time period.

All clinical champions received training at the QASCIP workshop on the auditing process. In addition, online audit training was made available to the site auditors through the NSF website with monthly support teleconferences.

Instrument

For the preimplementation and postimplementation audit, we used 19 existing relevant items from the 2013 NSF Clinical Audit as follows: fever: n=4; hyperglycaemia: n=5; swallowing: n=4 and patient demographics: n=6. As collected through the NSF National Clinical Audit, these questions assessed frequency of temperature and glucose monitoring and treatment for the first 72 h after stroke admission. The swallowing questions examined the time and date the swallow screen was completed, if patients were kept nil by mouth before a screen and prior to food, fluids or oral medications, and if patients were kept nil by mouth after a failed screen and subsequent referral to a speech pathologist. In addition, we also accessed eight existing relevant items from the 2013 NSF Organisation Survey regarding existing hospital and stroke service characteristics, namely location (metropolitan/rural); presence of dedicated stroke unit (yes/no); regular multidisciplinary team meetings (yes/no); whether there were existing clinical care pathways for: stroke, fever management, hyperglycaemia management, swallowing management (yes/no for each); and current use of the ASSIST swallowing screening tool (yes/no).

Prospective medical audits post-QASCIP

Participating sites were also required to conduct a self-reported prospective audit for the first 40 consecutive patients from the postimplementation period, namely 1 November 2013 to 28 February 2014, using identical NSF tools and equivalent inclusion criteria. The postimplementation audits were started 4 months after the one-day educational workshop, with a total of 8 months between this workshop and completion of the postimplementation audits.

Using the postimplementation cohort, inter-rater reliability was undertaken through repeat medical record audits for four key outcome measures for fever, five for hyperglycaemia and three for swallowing dysfunction at each site using different auditors as per the standard NSF data collection method to measure data reliability.28 Stroke services were asked to reaudit the first five consecutive patients; however, those stroke services with a lower volume of patients with stroke (<20 in the postimplementation cohort) were only required to reaudit records for three patients.

Data analysis

De-identified data were analysed by an independent statistician. Aboriginality, age group, sex, history of diabetes and premorbid modified Rankin Score (mRS) were compared between patients included in the preimplementation and postimplementation audits using a logistic regression model, and length of stay was compared between the precohorts and postcohorts using a linear model; these models included pre-post as the explanatory variable and used a generalised estimating equations  (GEEs) approach to adjust for correlation of observations within hospitals.

The number and proportion of patients with each of the monitoring and treatment elements for all three of the FeSS protocols were compared between the preimplementation and postimplementation periods. In addition, we calculated overall monitoring adherence, overall treatment adherence and a composite measure of appropriate monitoring and treatment for all three of the FeSS protocols.

To compare monitoring and treatment outcomes from preimplementation to postimplementation, logistic regression analyses were undertaken, which included audit period (preimplementation or postimplementation) as the primary predictor variable of interest. The logistic regression models were fit within a GEEs framework to adjust for correlation of patients within hospitals. ORs and 95% CIs are presented for the key composite outcomes of appropriate monitoring and treatment for each of the three FeSS protocols.

For each of the preintervention FeSS monitoring and treatment practices, we also report the corresponding proportion for the post-QASC trial findings from the 10 QASC trial intervention sites; however, small sample sizes in the audit precluded significance testing.

We also examined associations between change in adherence to the three clinical protocols from preimplementation to postimplementation and the following factors: (1) volume of stroke admissions (<100 vs ≥100 patients with stroke/year); (2) hospitals with a dedicated stroke unit and hospitals with a stroke service; (3) hospitals that participated in the original QASC trial and hospitals that only participated in the QASCIP; (4) hospitals randomised to the original QASC trial intervention group and hospitals that only participated in the QASCIP; and (5) hospital location (rural (population<25 000) vs urban (population≥25 000)). We generated a separate model for each of the five factors which included time (preimplementation/postimplementation), the factor of interest and the interaction between these two variables. The p values for the interaction term was used to determine whether the factor was associated with change in protocol adherence.

Patients admitted for <24 h were excluded from all analyses as outcomes were assessed in 24 h blocks of time. In addition, where patients were only admitted for 48 h, observations for days 1 and 2, but not day 3, were included in the analyses. Patients not known to have diabetes, with no episode of hyperglycaemia (blood glucose level (BGL)>10 mmol/L) in the first 48 h, were excluded from the monitoring element 4—M4 in the analysis as per the clinical protocol. Data recorded as ‘not documented’ and ‘unknown’ were assumed to be negative and included in the relevant denominator.

κ Values with 95% CIs were calculated to determine the inter-rater reliability.

Adherence to clinical audit standards

Adherence to 29 of the 30 relevant standards from the then proposed UK standards for design and conduct of a national clinical audit or quality improvement study was also documented.31

Results

Service characteristics

All 36 stroke service sites agreed to participate. All 32 sites who had participated in the routine 2013 NSF Clinical Audit data consented to the QASCIP researchers having access to their precollected hospital NSF audit data. The remaining four stroke services provided data directly to the researchers. In addition, consent was provided by the majority of participating sites (n=35) to access data from the routine 2013 NSF Organisational Survey to provide hospital characteristics data. The one site that had not participated in the 2013 NSF Organisational Survey provided hospital characteristics directly to the project team.

Most hospitals participating in the study had a dedicated stroke unit (n=31, 86%); most were located in the metropolitan region (n=32, 89%). Those self-reporting already having existing protocols were as follows: management of fever (n=32, 89%), hyperglycaemia (n=30, 83%), swallowing (n=35, 97%) and having a clinical care pathway for stroke (n=32, 92%; table 1).

Table 1

Hospital characteristics

	Total hospitals in study(n=36)
Hospital location
Metropolitan	32 (89%)
Rural	4 (11%)
Hospitals with a dedicated stroke unit	31 (86%)
Hospitals with a clinical care pathway for managing stroke	33 (92%)
Hospitals with regular stroke multidisciplinary team meetings	34 (94%)
Hospitals with an agreed management (including assessment and monitoring) protocol for fever	32 (89%)
Hospitals with an agreed management (including assessment and monitoring) protocol for hyperglycaemia	30 (83%)
Hospitals with an agreed management (including assessment and monitoring) protocol for swallow	35 (97%)
Hospitals that use the ASSIST tool	28 (78%)

Pre-QASCIP and post-QASCIP audit results

We obtained data for 1062 patients treated in the preintervention period and 1082 patients in the postintervention period (some hospitals did not have 40 stroke admissions during the audit periods). There were no significant differences between patients in the preimplementation and postimplementation cohorts in terms of aboriginality (p=0.09), age group (p=0.15), gender (p=0.88), diabetes status (p=0.24) and premorbid mRS (p=0.89; table 2).

Table 2

Patient demographics

	Statewide aggregate			QASC trial
	Preimplementation audit (n=1062)	Postimplementation audit (n=1082)	p Value	Intervention hospitals(n=603)
Aboriginal or Torres Strait Islander
Yes	33 (3.1%)	23 (2.1%)	0.09	5 (0.8%)
No	983 (93%)	1034 (96%)		506 (84%)
Refused/Don’t know	46 (4.3%)	25 (2.3%)		92 (15%)
Age group
<65	264 (25%)	241 (22%)	0.15	197 (33%)
65 to 74	252 (24%)	252 (23%)		141 (23%)
75 to 84	350 (33%)	350 (33%)		171 (28%)
Over 85	193 (18%)	232 (22%)		94 (16%)
Gender
Male	589 (55%)	599 (55%)	0.88	358 (60%)
Diabetes
Yes	271 (26%)	254 (23%)	0.24	108 (18%)
Premorbid mRS (prior to admission to hospital)
0 or 1 (None or minimal disability)	672 (66%)	692 (66%)	0.89	476 (93%)

mRS, modified Rankin Score; QASC, Quality in Acute Stroke Care.

Length of stay data were available for 901 patients in the preimplementation audit and 857 in the postimplementation audit. Length of stay was similar for the preimplementation cohort (median 6.0 days; minimum 1.0, maximum 76 days) and for the postimplementation cohort (median 5.0 days; minimum 1.0, maximum 53 days; p from the GEE linear model=0.18).

Significantly increased proportions of patients received care according to the fever protocol from preimplementation to postimplementation (pre: 69%; post: 78%; p=0.003; OR 1.6, 95% CI 1.2 to 2.2). Specifically, significantly higher proportions of patients postimplementation were monitored for fever on days 1–3. However, of the 135 patients with a febrile episode in the postimplementation cohort, only 64 (47%) received paracetamol within 1 h, a non-statistically significant increase from preimplementation (38%; table 3).

Table 3

Number and proportion of patients monitored and treated according to fever protocol

		Statewide aggregate				QASC trial
Element	Monitoring and treatment for fever	Preimplementation audit(n=1062)	Postimplementation audit(n=1082)	OR (95% CI)	p Value	Intervention group (n=603)
	Monitoring
M1	Temperature recorded at least four times on day 1	924 (87%)	1025 (95%)	2.68 (1.60 to 4.50)	0.0002	545 (93%)
M2	Temperature recorded at least four times on day 2	861 (84%)	940 (91%)	1.89 (1.30 to 2.76)	0.0009	482 (82%)
M3	Temperature recorded at least four times on day 3	764 (82%)	833 (88%)	1.56 (1.11 to 2.20)	0.011	379 (64%)
	Monitoring adherence*
	Monitored according to protocol for fever	802 (76%)	906 (84%)	1.66 (1.18 to 2.34)	0.0033	337 (56%)
	Treatment
	At least one febrile event (temperature≥37.5°C)	149 (14%)	135 (12%)	0.89 (0.65 to 1.21)	0.45	105 (17%)
T1	Received paracetamol within 1 h of their first febrile event (temperature≥37.5°C)	57 (38%)	64 (47%)	1.45 (0.95 to 2.20)	0.08	19 (18%)
	Protocol adherence: monitored and treated†
	Monitored and treated according to the protocol for fever	729 (69%)	845 (78%)	1.62 (1.18 to 2.24)	0.0031	258 (44%)

Day 1 indicates first 24 h since admission to hospital. The protocol recommends that observations should be taken at least six hourly, so there should be at least four separate temperature recordings during the first 24 h of admission.

Statistically significant p values are shown in bold.

*Must meet all M1, M2 and M3 to be deemed as having been monitored according to protocol.

†Must meet all M1, M2, M3 and T1 to be deemed as having been monitored and treated according to protocol.

QASC, Quality in Acute Stroke Care.

There were significantly increased proportions of patients who received care according to the hyperglycaemia (sugar) protocol from preimplementation to postimplementation (pre: 23%; post: 34%; p=0.0085; OR 1.8, 95% CI 1.2 to 2.7). Specifically, significantly increased proportions of patients from preimplementation to postimplementation had their BGL monitored on days 1–3. However, of the 205 patients in the postimplementation cohort who had a finger-prick glucose level >10 mmol/L, only 56 (27%) received insulin within the 1 h recommended time interval, with no clinical or statistically significant improvement from preimplementation (22%; table 4).

Table 4

Number and proportion of patients monitored and treated according to the hyperglycaemia (sugar) protocol

		Statewide aggregate				QASC trial
Element	Monitoring and treatment	Preimplementation audit (n=1062)	Postimplementation audit (n=1082)	OR (95% CI)	p Value	Interventio n group(n=603)
	Monitoring
M1	Formal VBG measurement in the ED	678 (64%)	754 (70%)	1.28 (0.83 to 1.97)	0.27	184 (31%)
M2	Finger-prick blood glucose level recorded at least four times on day 1	533 (50%)	749 (69%)	2.50 (1.66 to 3.75)	<0.0001	362 (60%)
M3	Finger-prick blood glucose level recorded at least four times on day 2	442 (43%)	679 (66%)	2.81 (1.89 to 4.16)	<0.0001	314 (52%)
M4	Finger-prick blood glucose level recorded at least four times on day 3	179 (60%)	222 (79%)	2.41 (1.54 to 3.78)	0.0001	311 (52%)
	Monitoring adherence*
	Monitored according to the protocol for hyperglycaemia	301 (28%)	424 (39%)	1.66 (1.11 to 2.48)	0.014	61 (10%)
	Treatment
	At least one finger-prick glucose level of >10 mmol/L	187 (18%)	205 (19%)	1.11 (0.85 to 1.43)	0.44	135 (22%)
T1	Insulin received within 1 h of their finger-prick glucose level of >10 mmol/L	41 (22%)	56 (27%)	1.32 (0.79 to 2.21)	0.30	19 (14%)
	Protocol adherence: monitored and treated†
	Monitored and treated according to the protocol for hyperglycaemia	240 (23%)	363 (34%)	1.76 (1.16 to 2.69)	0.0085	21 (3.5%)

Day 1 indicates first 24 h since admission to hospital. The protocol recommends that observations should be taken at least six hourly, so there should be at least four separate finger-prick blood glucose levels taken during the first 24 h of admission to hospital. Formal VBG defined as: blood glucose sample sent to laboratory for analysis.

Statistically significant p values are shown in bold.

*Must meet all M1, M2, M3 and M4 to be deemed as having been monitored according to protocol if the patient is known to have diabetes or is not known to have diabetes but has one or more episodes of hyperglycaemia (glucose >10 mmol/L). Must meet all M1, M2 and M3 and have no episode of hyperglycaemia (glucose>10 mmol/L) to be deemed as having been monitored according to protocol if the patient is not known to have diabetes.

†Must meet all M1, M2, M3 and M4 (if applicable, see*) and T1 to be deemed as having been monitored and treated according to protocol.

ED, emergency department; QASC, Quality in Acute Stroke Care; VBG, venous blood glucose.

The proportion of patients receiving care according to the swallowing protocol increased from preimplementation to postimplementation (pre: 42%; post: 51%; p=0.033). Specifically, increased proportions of patients with acute stroke received a swallow screen or swallow assessment within 24 h, and prior to receiving oral food or drink or medications. Similar high proportions of patients, preimplementation (97%) and postimplementation (95%), who failed a swallow screen subsequently received a swallow assessment by a speech pathologist as recommended (table 5).

Table 5

Number and proportion of patients monitored and treated according to swallowing protocol

		Statewide aggregate				QASC trial
Element	Monitoring and treatment for swallowing	Preimplementation audit(n=1062)	Postimplementation audit(n=1082)	OR (95% CI)	p Value	Interventiongroup(n=603)
	Monitoring
	Received a swallow screen within 24 h of admission to hospital	453 (43%)	562 (52%)	1.57 (1.15 to 2.15)	0.0047	284 (47%)
	Received a swallow assessment within 24 h of hospital admission	404 (38%)	418 (39%)	1.01 (0.84 to 1.22)	0.91	330 (55%)
M1	Received a swallow screen or a swallow assessment within 24 h of hospital admission	733 (69%)	814 (75%)	1.38 (1.09 to 1.74	0.0068	491 (81%)
M2	Received a swallow screen or a swallow assessment before they were given food or drink (orally)	605 (57%)	736 (68%)	1.60 (1.11 to 2.23)	0.013	135 (22%)
M3	Received a swallow screen or a swallow assessment before they were given oral medications	550 (52%)	670 (62%)	1.53 (1.10 to 2.13)	0.011	222 (37%)
	Monitoring adherence*
	Monitored according to protocol for swallow dysfunction	454 (43%)	565 (52%)	1.50 (1.05 to 2.14)	0.03	65 (11%)
	Treatment
	Failed the swallow screen	178 (17%)	230 (21%)	1.40 (1.05 to 1.86)	0.02	95 (16%)
T1	Failed the swallow screen and received a swallowing assessment by a speech pathologist	173 (97%)	218 (95%)	0.52 (0.17 to 1.57)	0.25	74 (78%)
	Protocol adherence: monitored and treated†
	Monitored and treated according to the protocol for swallowing dysfunction	450 (42%)	556 (51%)	1.47 (1.03 to 2.09)	0.033	62 (10%)

Day 1 indicates first 24 h since admission to hospital.

Statistically significant p values are shown in bold.

*Must meet all M1, M2 and M3 to be deemed as having been monitored according to protocol.

†Must meet all M1, M2, M3 and T1 to be deemed as having been monitored and treated according to protocol.

QASC, Quality in Acute Stroke Care.

Of note, statewide overall monitoring and treatment practices preimplementation were higher when compared with data collected from the 10 intervention hospitals at conclusion of the original QASC trial which had been completed some 24 months earlier, for example, monitoring adherence for fever (QASCIP preimplementation: 69% vs QASC trial: 44%); hyperglycaemia (QASCIP preimplementation: 23% vs QASC trial: 3.5%) and swallowing (QASCIP preimplementation: 42% vs QASC trial: 10%).

There were no significant differences in change in whether patients were monitored and treated according to the protocol from preimplementation to postimplementation and number of stroke admissions (<100 vs ≥100 patients with stroke/year) for fever (p=0.75), hyperglycaemia (p=0.95) and swallowing (p=0.28). Similarly, there were no significant differences in preimplementation to postimplementation change in adherence between hospitals with a dedicated stroke unit compared with hospitals without (ie, with only a stroke service) for the fever protocol (p=0.81), the hyperglycaemic protocol (p=0.31) and the swallowing protocol (p=0.09).

No significant differences were found between rural and urban hospitals and change in whether patients were monitored and treated according to fever protocol (p=0.11) or the swallowing protocol (p=0.052). There were, however, statistically significant improvements for monitoring and treatment according to the hyperglycaemic protocol in urban sites (preimplementation: n=228 (23%); postimplementation; n=356 (35%)) when compared with rural sites (preimplementation: n=12 (14%); postimplementation; n=7 (11%; p=0.0006)).

There were no significant differences between hospitals that participated in the original QASC trial and hospitals that only participated in the QASCIP in pre-post change in monitoring and treatment adherence to: the fever protocol (p=0.48), the hyperglycaemic protocol (p=0.85) and the swallowing protocol (p=0.57). Likewise, there was no difference between hospitals randomised to the original QASC trial intervention group and hospitals that only participated in the QASCIP and change from preimplementation to postimplementation in whether patients were monitored and treated according to the fever protocol (p=0.54) or the swallowing protocol (p=0.77). However, adherence to the hyperglycaemic protocol in those hospitals which received the QASC intervention in the original trial remained consistent from preimplementation (37%) to postimplementation (35%), but increased from preimplementation (16%) to postimplementation (33%) for the hospitals which did not receive the QASC intervention as part of the original trial (p=0.02).

Inter-rater reliability data were provided for a total of 148/1082 (14%) postimplementation patients from all participating hospitals. For 10 of the 12 patient variables in the postimplementation clinical audit, the κ values indicated substantial inter-rater reliability (ie, >0.6; figure 1).32

Figure 1

Inter-rater reliability for 12 key individual variables.

Adherence to audit standards

Of the 30 proposed UK standards for design and conduct of a national clinical audit or quality improvement study, 29 standards were relevant to our audit. One standard was not applicable as our audit did not involve any electronic data linkage. We adhered to 29 of the 29 (100%) proposed standards for the design and conduct of a national clinical audit or quality improvement study relevant to our context.

Discussion

The reporting of ‘scale-up’ at the national or state level of an implementation intervention proven effective in changing clinical practice and improving patient outcomes is limited. There is a pressing need for high-quality studies to assess mechanisms by which interventions, shown to be effective in academic centres or hospitals inclined to participate in health services research, can be ‘scaled up’ to every relevant clinical setting.33

This is one of the few studies to examine a systematic effort to ‘scale-up and spread’ an effective implementation strategy in acute healthcare. Furthermore, the pace and geographical scale-up and spread were notable. Since successful evidence translation can take decades,2 the fact we achieved these significant and clinically important changes within a short 8-month time frame, only 4 years since publication of our original trial, and in all stroke services across an entire state, is laudable. Having tested resources and tools available from the QASC trial, combined with evidence from the rigorous process evaluation,24 enabled the rapid replication in the real world and reduced the evidence-to-practice translation timeline by many years.

Our implementation study demonstrated significant improvements in adherence to all three FeSS clinical protocols. Having proven the effectiveness of implementation strategies in the earlier QASC trial provided an incontrovertible foundation for QASCIP itself. Additional strengths of QASCIP included 100% participation of all NSW stroke services. That this study embraced smaller stroke services as well as hospitals with dedicated stroke units was also noteworthy.

The significant improvement in monitoring for all three elements was encouraging, with the exception of formal venous blood glucose measurement, which warrants further attention. Routine collection of this measure is potentially achievable through the embedding of serum glucose in electronic pathology orders for patients with stroke on admission to the emergency department. However, although treatment practices did not significantly improve for fever (administration of paracetamol), the number of patients with fever was small.

Use of insulin for hyperglycaemia remained poor with less than a third of patients who required insulin not receiving it. As in the QASC trial, QASCIP did not supply insulin administration protocols to sites. This was a deliberate and pragmatic approach as new treatment protocols of this nature require extensive local consultation and time to implement and it was decided that hospitals would use their locally agreed current inpatient protocols while adhering to the principles of best practice for glucose management. Anecdotal evidence gathered at the site support visits highlighted that not all hospitals had an agreed inpatient insulin administration protocol for management of hyperglycaemia and that these sites may have benefited by provision of such insulin administration guidelines which may then have improved adherence.

Swallowing surveillance in the form of a swallowing screen by a non-speech pathologist or a swallowing assessment by a speech pathologist within 24 h was also less than optimal even in the postimplementation cohort (75%). This requires further attention. Patients continue to be fed and given oral medications prior to a swallow screen or assessment. In contrast, the percentage of patients who failed the swallowing screen and correctly received a swallowing assessment by a speech pathologist was encouragingly high preimplementation (97%), remaining so postimplementation (95%).

Of note, the majority of sites reported already having fever, hyperglycaemia and swallowing management practices and protocols prior to participation in the study. Acknowledging that these protocols may have varied somewhat from the FeSS clinical protocols, higher preimplementation adherence rates might reasonably have been expected.

Of interest, preimplementation practices for the protocol adherence (ie, monitoring and treatment practices) for fever, hyperglycaemia and swallowing across the state were already higher in comparison to postimplementation data collected from the 10 intervention hospitals in the original QASC trial. This improvement over time was also reflected in national data from the 2013 NSF audit (fever: monitoring adherence 71%, paracetamol within an hour 36%; hyperglycaemia: monitoring adherence 18%, insulin treatment 25%; swallowing: monitoring adherence 39% (no swallowing treatment data available)).34 We speculate that this could potentially be due to widespread publicity, media reporting and dissemination at conferences and seminars of the results of the original QASC trial following its publication in the Lancet in 2011, that is, passive dissemination. The significant improvement from pre-to-post for those QASCIP hospitals which did not receive the intervention as part of the original QASC trial was potentially due to lower adherence preimplementation for the non-QASC trial intervention hospitals.

In view of concerns among policymakers of the intractable differences in health outcomes between rural and metropolitan populations,35–37 our postaudit data showed equivalence between rural and urban hospitals in their adherence to the fever protocol (p=0.11) and was marginally so for the swallowing protocol (p=0.052). This is encouraging, given the likely limited staffing at rural speech pathology services, particularly after hours and at weekends. There were significant postimplementation improvements for monitoring and treatment according to the hyperglycaemia protocol in urban sites (preimplementation: n=228 (23%); postimplementation: n=356 (35%)) when compared with rural sites (preimplementation: n=12 (14%); postimplementation: n=7 (11%; p=0.0006)), but further exploration of this difference was beyond the scope of our study. However, hyperglycaemia management was poorly attended in all sites regardless of location.

Our study had several limitations. First, we only had the resources to evaluate processes of care and not patient 90-day outcomes as in the original QASC trial. However, the QASC trial provided robust evidence that improvements to processes of care, even minimal ones, can dramatically result in better patient outcomes. Second, audit data were self-reported, possibly introducing selection bias as well as responder bias. However, the potential for these biases was present throughout the study and, having been obtained from a national clinical audit initiative, unlikely to be unique to our study alone. Furthermore, any biases would be expected to be consistent for the preintervention and postintervention cohorts and our primary outcome was improvement over time, rather than absolute values, further boosted by high inter-rater reliability. Despite these shortcomings, we successfully adhered to all the relevant proposed standards for the design, and conduct of a national clinical audit or quality improvement study31 invites greater confidence. Adherence to rigorous standards for quality improvement studies likely to inform clinical practice change is essential.38 To the best of our knowledge, this was the first systematic application of UK Audit Standards in a study of this type internationally. Reporting on the quality of audit should be imperative in all large-scale audits.

Collaboration with the NSF was essential for the timely completion of the study. Use of established data collection tools and existing training methods significantly reduced the cost and timeline for the study. Importantly, in future, hospitals will be able to reaudit the same processes of care easily and efficiently in the future to measure self-sustainability of improvements made. Sustainability of practice change is the next frontier in quality improvement.39 We have also provided valuable benchmarking data for other Australian states. The value of using existing data sources (ie, registries and routinely collected audit data) to measure change over time cannot be underestimated. Data linkage projects and the creation of funded, mandated national data sets with uniform and agreed data definitions are the way forward for multisite large-scale statewide or national quality improvement activities such as this.

We connected researchers with clinicians to develop a pragmatic, replicable intervention.13 The key to our success and what makes this implementation study so unique from other studies16 was the commitment from the collaborators only to use the proven implementation strategy from the QASC trial, resisting adopting the ‘kitchen sink’ approach, criticised elsewhere for often including untested implementation strategies.1 Involving the researchers who undertook the seminal trial was key to ensuring that this approach was maintained. Researchers are rarely involved in evaluation of scale-up, possibly due to the difficulty in securing dedicated funding for implementation and the need to deliver on other traditional academic key performance indicators such as publications and grant income.13 Should research impact become a metric for researcher performance, implementation research will flourish as an academic field and clinical translational initiatives such as QASCIP will increase in number and focus. Despite this, there is a role for researchers in the scale-up of interventions,13 and particular consideration could be given specifically to involving those who conducted the original research in order to promote implementation fidelity and provide advice. Dedicated funding for implementation is seriously overdue in Australia and other countries.13 40

Conclusion

Our study is one of the few to successfully and systematically replicate methods from a positive implementation trial. We show, for the first time, significant statewide improvements in clinical management of fever, hyperglycaemia and swallowing for patients with stroke through our clinical translational initiative conducted within a short 8-month time frame. There was, however, room for improvement in the proportion of patients receiving care according to these protocols. Protocol uptake may have been improved with a longer duration between implementation and postintervention audit. Barriers to the hyperglycaemia protocol, in particular, warrant future attention. Our results clearly demonstrate the benefits to patients of funding ‘scale-up’ of a proven implementation strategy across an entire statewide health system. Nonetheless, further research is recommended to illuminate the complexity of clinical evidence translation on a large scale and at pace.