Impact of an Expanded Hospital Recognition Program for Stroke Quality of Care.

BACKGROUND: In 2009, the Get With The Guidelines-Stroke (GWTG-Stroke) program offered additional recognition if hospitals performed well on certain stroke quality measures. We sought to determine whether quality of care for all hospitals participating in GWTG-Stroke improved with this expanded recognition program. METHODS AND
RESULTS: We examined hospital-level performance on 6 quality of care (process) measures and 1 defect-free composite quality measure for stroke following expansion of the existing performance measure recognition program. Compliance with all measures improved following launch of the expanded program, and this rate increased significantly for all 9 measures. When evaluated as the relative rate of increase in use over time, process improvement slowed significantly (P<0.05) following launch of the program for 2 measures, and accelerated significantly for 1 measure. However, when evaluated as a gap in care, the decrease in the quality gap was greater following launch of the program for 5 of 6 (83%) measures. There was no evidence that other processes of stroke care suffered as the result of the increase in measures and expanded recognition program.
CONCLUSIONS: While care for stroke continues to improve in this country, expanded hospital process performance recognition had mixed results in accelerating this improvement. However, the quality gap continues to shrink among those participating in provider performance programs.

Introduction

The American Heart Association's Get With The Guidelines‐Stroke (GWTG‐Stroke) program has been developed to measure and improve the quality of care and outcome for patients hospitalized with stroke.1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 The program provides via performance achievement awards public recognition of hospitals with high performance on select performance measures for acute ischemic stroke. These measures of achievement include intravenous (IV) tissue plasminogen activator (tPA) within 3 hours (if symptoms onset to door is within 2 hours), use of antithrombotics, timing of antithrombotics, anticoagulation for atrial fibrillation, deep venous thrombosis prophylaxis, low‐density lipoprotein (LDL) <100 mg/dL or statin treatment for patients with LDL ≥100 mg/dL, and smoking cessation counseling. Performance on these achievement measures has reached a high level.1, 2 In contrast, performance on several other quality measures that were not utilized as part of the hospital recognition criteria has been poor.1

In order to further improve stroke care, the GWTG‐Stroke program expanded its recognition program by creating the Plus Awards in 2009. This program provides an added incentive by recognizing hospitals meeting 75% compliance on any 4 additional quality measures. The additional measures are dysphagia screening, stroke education, consideration of rehabilitation, door to tPA time within 1 hour, documentation of LDL cholesterol, intensive statin therapy, last known well to IV tPA time within 4.5 hours if onset to door is less than 3.5 hours, and reporting of the National Institutes of Health Stroke Severity Score.

We sought to evaluate the impact of the Plus awards on quality of stroke care for hospitals participating in GWTG‐Stroke. We tested the hypothesis that performance on the quality measures for stroke improved at a faster rate following implementation of the Plus award program than prior to the Plus awards. In addition, we sought to exclude any unintended, negative impact on the pre‐existing stroke performance measures following the launch of the PLUS awards.

Methods

GWTG‐Stroke is a national voluntary stroke registry and performance improvement initiative from the American Heart Association. In GWTG‐Stroke, participating hospitals use an internet‐based patient management tool (Quintiles, Cambridge, MA) to collect data on consecutive acute ischemic stroke patients. The methods and quality auditing for GWTG‐Stroke have been previously described in detail.

Plus Award Intervention

Prior to the introduction of the Plus Awards,12 the recognition program for the GWTG‐Stroke (Achievement Award) publically acknowledged hospitals reaching 85% compliance with each of the following measures: IV tPA within 3 hours (if last known well to IV tPA time is within 2 hours), early antithrombotics, appropriate antithrombotics, anticoagulation for atrial fibrillation, deep venous thrombosis prophylaxis, LDL cholesterol <100 mg/dL or statin treatment for patients with LDL ≥100 mg/dL, and smoking cessation counseling. For a hospital to also be recognized by the new Plus Award Program, they must both receive the established Achievement Award, and demonstrate 75% compliance for 12 consecutive months on 4 out of 8 stroke quality measures: dysphagia screening, stroke education, consideration of rehabilitation, door to tPA time within 1 hour, documentation of LDL cholesterol, intensive statin therapy, use of IV tPA by 4.5 hours if last known well to IV tPA time is within 3.5 hours, and reporting of the National Institutes of Health Stroke Severity Score. Prior to the initiation of the Stroke Plus awards, hospitals were provided details of their performance on these measures but there was no public recognition of high performers.

Study Population

A total of 2 480 993 patients with stroke were identified from January 2006 to December 2013. We excluded patients during the Plus award transition period (July 2009–December 2009, N=159 494), patients from sites that did not have patients in both the pre‐ (January 2006–June 2009) and postaward periods (January 2010–December 2013, N=239 190), patients who died prior to discharge (N=140 460), and patients who were transferred to other healthcare facilities or left against medical advice (N=792 117). Patients who died, transferred, or left against medical advice were older, were more likely to be female, and more likely to have comorbid conditions (Table S1). The primary analysis included 1 149 732 acute ischemic stroke patients from 1224 participating hospitals.

Outcomes

The primary outcomes were use of the quality measures in appropriate candidates. We excluded 2 measures with 12 months or less of pre‐award data available (stroke education and intensive statin therapy). The 6 measures included in the analysis were dysphagia screening, consideration of rehabilitation, door to tPA time within 1 hour, documentation of LDL cholesterol, use of IV tPA by 4.5 hours if onset to door is within than 3.5 hours, and reporting of the National Institutes of Health Stroke Severity Score. A defect‐free composite quality measure was also created.

All GWTG‐Stroke participating institutions were required to comply with local regulatory and privacy guidelines and, if required, to secure institutional review board approval. Sites were granted a waiver of informed consent under the common rule as data were used primarily at the local site for quality improvement. The Duke Clinical Research Institute (Durham, NC) served as the data analysis center, and institutional review board approval was granted to analyze aggregate deidentified data for research purposes.

Statistical Analysis

Patient and hospital characteristics were summarized descriptively for the preprogram and postprogram periods. Standardized mean differences were calculated for the pre‐ and postaward periods. Piecewise (or segmented) logistic multivariable regression models were performed to track the trends over time of achievement measures in pre‐, and post‐Plus periods. The adjusted models account for differing hospital and patient characteristics over time. Characteristics included were (1) patient demographics: age, sex, race; (2) medical history: atrial fibrillation, prosthetic heart valve, previous stroke or transient ischemic attack, coronaPry artery disease or prior myocardial infarction, carotid stenosis, peripheral vascular disease, hypertension, dyslipidemia, and smoking; (3) hospital characteristics: annual stroke admission, bed size, region, hospital type (academic versus not), primary stroke center, urban/rural location. For each outcome, we provide the odds ratio (with 95% CI and P‐value) per 3 calendar months as the rate of improvement during the preprogram period, the odds ratio (with 95% CI and P‐value) per 3 months after program initiation, and a P value comparing these to evaluate if the rate of improvement significantly changed after program initiation. Both unadjusted and adjusted odds ratios and CIs are reported. The generalized estimating equation method with exchangeable working correlation matrix was applied to provide valid inference after accounting for the within‐site correlation.13

Hospital characteristics were missing in <1%, and patients from these hospitals were excluded in multivariable models. All P values are 2‐sided, with P<0.05 considered statistically significant. Analyses were performed using SAS software (version 9.2; SAS Institute, Cary, NC).

Results

The primary analysis compared treatment for 1 149 732 stroke patients (from 1224 hospitals) who were hospitalized in the preprogram period (January 2006–June 2009, N=429 491) or program (Plus Award) period (January 2010–December 2013, N=720 241). Patient and hospital characteristics for both groups are displayed in Tables 1 and 2. In general, differences in patient and hospital characteristics over time were small though often statistically significant because of the large sample size.

Table 1

Patient and Facility Characteristics of Patients Before and After Initiation of the Enhanced Recognition Program (Plus Awards)

Variable	PostMean	PreMean	Standardized Differencea
N	720 241	429 491
Age, y	67.7	67.6	0.9
Female, %	51.0	51.7	−1.4
Race—White, %	70.4	72.0	−3.5
Race—Black, %	15.6	14.3	3.6
Insurance—Private, %	41.5	25.8	31.4
Insurance—Medicaid, %	8.6	4.2	15.6
Insurance—Medicare, %	30.0	18.8	23.3
No Insurance/Self, %	7.0	4.0	11.0
Emergency medical service, %	35.5	45.1	−19.3
Atrial fibrillation/flutter, %	13.8	12.7	3.2
Prosthetic heart valve, %	1.3	1.5	−1.3
Previous stroke/TIA, %	29.1	28.7	1.0
Coronary artery disease, %	23.3	24.4	−2.5
Carotid stenosis, %	3.7	3.9	−0.9
Diabetes mellitus, %	29.7	27.3	5.5
Peripheral vascular disease, %	4.0	4.0	0.3
Hypertension, %	72.5	71.0	3.5
Smoker, %	19.3	19.6	−0.7
Dyslipidemia, %	44.0	39.1	9.9
Heart failure, %	6.6	3.1	16.4
Sickle cell disease, %	0.1	0.0	1.8
Current pregnancy, %	0.1	0.0	1.2
Ambulate independently prior to current event, %	75.0	61.4	34.4
Stroke type—IS, %	60.6	52.7	15.9
Stroke type—TIA, %	29.0	36.0	−14.9
NIH Stroke Scale	4.0	4.0	0.7

IS indicates ischemic stroke; NIH, National Institutes of Health; TIA, transient ischemic attack.

Some consider a standardized difference of 10% or more to be “clinically significant.”

Table 2

Comparison of Hospital Characteristics Before and After Launch of the New Quality Metrics (Plus Awards)

Variable	PostMean	PreMean	Standardized Difference
Outcome
Discharge home, %	92.8	94.5	−6.9
LOS	3.8	4.0	−3.0
Ambulate independently at discharge, %	62.0	75.6	−33.2
Meeting all achievement measures, %	92.1	81.8	31.0
Hospital characteristics
Annual volume of ischemic stroke admissions	261.6	261.2	0.3
Number of beds	456.4	460.1	−1.2
Region
Northeast, %	27.6	25.8	4.0
Midwest, %	19.0	18.3	2.0
South, %	36.1	38.7	−5.4
West, %	17.3	17.2	0.2
Teaching hospital, %	61.9	62.0	−0.2
Rural location, %	3.5	3.2	1.6
PSC sites, %	53.7	55.9	−4.4

LOS indicates length of stay; PSC, primary stroke center certification.

Quality Metrics and Plus Awards

Use of the 6 quality metrics are shown over time in Figure 1A and 1B. Use increased for all measures from before (2006–2009) to after initiation of the Plus Award program (2010–2015). Following the launch of the Plus Awards, 132 674 patients were admitted to hospitals receiving Plus Awards compared to 587 567 admitted to non‐Award hospitals.

Figure 1

A and B, Trends in use of the quality metrics targeted as part of an expanded hospital recognition program are shown from 2006 to 2013. The program was announced in 2008 and launched in July 2009. All metrics increased over time. Timely reperfusion increased dramatically 1 year after the launch of the Plus awards, which was more closely linked to the launch of an additional program (Target Stroke) that targeted door‐to‐needle times (DTN). LDL indicates low‐density lipoprotein; NHSS, National Institutes of Health Stroke Severity Score; tPA, tissue plasminogen activator.

Compliance with all quality metrics use was considerably higher for patients hospitalized at the Plus Award facilities (Table 3). For hospitals recognized with the Plus Award compared to those not recognized, the absolute difference in quality measure performance ranged from 24.7% for use of IV tPA within 4.5 hours in patients arriving within 3.5 hours of last known well to 4.3% for assessed for rehabilitation (Table 3). Performance improved for all quality metrics after initiation of the Plus Award program.

Table 3

Performance on Newer Quality Stroke Measures and Established Achievement Measures for Hospitals That Did (Plus Sites) and Did Not (Non‐Plus Sites) Receive the Expanded Plus Award

Variable	Plus SitesMean	Non‐Plus SitesMean	Standardized Difference
Quality measures (new)
Dysphagia screen, %	85.1	69.9	36.9
Rehabilitation considered, %	96.7	92.4	18.9
Door‐to‐IV tPA time ≤1 hour, %	44.0	35.2	18.0
LDL documented, %	91.2	83.8	22.7
Onset IV tPA by 4.5 hours (if onset to door <3.5 hours), %	67.1	42.4	51.2
NIHSS reported, %	75.7	59.6	35.1
Defect‐free measure quality, %	70.7	52.9	37.3
Achievement measures (established)
Onset to IV tPA by 3 hours (if onset to door <2 hours), %	87.9	60.5	65.9
Early antithrombotics, %	97.7	96.2	8.5
Antithrombotics, %	98.4	97.2	8.4
Anticoagulation for AF, %	95.3	89.3	22.7
DVT prophylaxis, %	97.8	96.8	6.5
LDL 100 mg/dL or ND—statin, %	95.2	92.0	13.2
Smoking cessation, %	98.3	95.2	17.4
Defect‐free measure, %	93.1	87.6	18.9

DVT indicates deep venous thrombosis; IV, intravenous; LDL, low‐density lipoprotein; ND, not determined; NIHSS, National Institutes of Health Stroke Scale; tPA, tissue plasminogen activator.

Quality Measured by the Relative Increase in Use

The unadjusted and adjusted rates of increase per quarter for the preprogram period (January 2008–June 2009) were compared with the established program period (January 2010–December 2013) for the 6 quality measures (Table 4). Adjustment had little impact on the observed odds ratios and confidence intervals. The odds ratio for receiving the recommended care increased significantly more rapidly for 1 measure in the post‐Award period than the pre‐Award period (IV tPA within 60 minutes), at a similar rate for 3 measures, and more slowly during the post‐Award period than during the pre‐Award period for 2 measures. For the defect‐free composite measure, the rate of improvement was less in the post‐Award period.

Table 4

Unadjusted and Adjusted Changes in Quality Measures in Pre‐ and Post‐Plus Program

		Unadjusted				Adjusteda
Outcome	Variable	OR	Lower 95% CI	Upper 95% CI	P Value	OR	Lower 95% CI	Upper 95% CI	P Value
Dysphagia screen	Calendar time: Pre‐Plus (per quarter)	1.078	1.068	1.088	<0.0001	1.066	1.055	1.078	<0.0001
	Calendar time: Post‐Plus (per quarter)	1.049	1.043	1.056	<0.0001	1.053	1.045	1.060	<0.0001
	Post vs Pre‐Plus	0.974	0.962	0.986	<0.0001	0.987	0.973	1.001	0.0697
Rehabilitation considered	Calendar time: Pre‐Plus (per quarter)	1.260	1.239	1.282	<0.0001	1.302	1.278	1.327	<0.0001
	Calendar time: Post‐Plus (per quarter)	1.022	1.017	1.027	<0.0001	1.031	1.023	1.039	<0.0001
	Post vs Pre‐Plus	0.811	0.796	0.827	<0.0001	0.792	0.774	0.809	<0.0001
Door‐to‐IV tPA time ≤1 hour	Calendar time: Pre‐Plus (per quarter)	0.996	0.981	1.010	0.5708	0.997	0.983	1.010	0.6335
	Calendar time: Post‐Plus (per quarter)	1.108	1.099	1.117	<0.0001	1.108	1.099	1.117	<0.0001
	Post vs Pre‐Plus	1.113	1.091	1.134	<0.0001	1.112	1.092	1.132	<0.0001
LDL documented	Calendar time: Pre‐Plus (per quarter)	1.072	1.066	1.078	<0.0001	1.085	1.076	1.094	<0.0001
	Calendar time: Post‐Plus (per quarter)	1.043	1.038	1.048	<0.0001	1.053	1.047	1.060	<0.0001
	Post vs Pre‐Plus	0.973	0.965	0.981	<0.0001	0.970	0.959	0.982	<0.0001
Onset IV tPA by 4.5 hours (if onset to door <3.5 hours)	Calendar time: Pre‐Plus (per quarter)	1.137	1.124	1.150	<0.0001	1.134	1.122	1.146	<0.0001
	Calendar time: Post‐Plus (per quarter)	1.120	1.112	1.128	<0.0001	1.125	1.116	1.133	<0.0001
	Post vs Pre‐Plus	0.985	0.970	0.999	0.0423	0.992	0.978	1.006	0.2603
NIHSS reported	Calendar time: Pre‐Plus (per quarter)	1.082	1.071	1.092	<0.0001	1.073	1.062	1.084	<0.0001
	Calendar time: Post‐Plus (per quarter)	1.077	1.070	1.084	<0.0001	1.082	1.073	1.090	<0.0001
	Post vs Pre‐Plus	0.996	0.983	1.008	0.5048	1.008	0.994	1.022	0.2566
Defect‐free quality measure	Calendar time: Pre‐Plus (per quarter)	1.094	1.086	1.101	<0.0001	1.084	1.076	1.091	<0.0001
	Calendar time: Post‐Plus (per quarter)	1.051	1.047	1.056	<0.0001	1.054	1.049	1.059	<0.0001
	Post vs Pre‐Plus	0.961	0.953	0.970	<0.0001	0.973	0.964	0.981	<0.0001

IV indicates intravenous; LDL, low‐density lipoprotein; NIHSS, National Institutes of Health Stroke Scale; OR, odds ratio; tPA, tissue plasminogen activator.

Variables in the model—age, sex, white race, insurance, medical history of atrial fibrillation, atrial flutter, chronic obstructive pulmonary syndrome or asthma, diabetes mellitus, hyperlipidemia, hypertension, peripheral vascular disease, prior myocardial infarction, cerebrovascular accident/transient ischemic attack, stroke, anemia, renal insufficiency, smoking, ischemic history, hospital size, hospital type, region, heart transplant, urban/rural location.

Quality Measured by the Relative Decrease in Gap in Care

When the yearly decrease in the quality gap (100%‐baseline use) was averaged over the 3 pre‐Plus years and compared with the 4 post‐Plus years (Figure 2), 5 of 6 measures (as well as the composite measure) showed an acceleration in the average annual quality gap reduction following the launch of the Plus Awards.

Figure 2

The average annual decrease in the quality gap (between baseline rate and 100%) for each quality measure is shown for the periods before and after the launch of the Plus Awards. LDL indicates low‐density lipoprotein; NHSS, National Institutes of Health Stroke Severity Score; tPA, tissue plasminogen activator.

Impact on Established Measures of Quality

There was no evidence of adverse impact on the established achievement measures (Figure 3A and 3B). Those hospitals receiving the new award (Plus Sites) had better performance on the established Achievement Measures (Table 3).

Figure 3

A and B, Trends in use of existing achievement measures that form the primary basis for hospital recognition. There was no evidence that the launch of the new quality measures drew attention away from the established measures. AF indicates atrial fibrillation; DVT, deep vein thrombosis; LDL, low‐density lipoprotein; TPA, tissue plasminogen activator.

Discussion

We evaluated the impact of expanding the recognition program of the American Heart Association's GWTG‐Stroke Program. We found that performance on the targeted measures improved after launch of the program and the performance of those hospitalized recognized with the Plus Award was considerably better than those not recognized. The rate of change in improvement did not increase for most measures. In fact, we found that the rate of improvement over time was slower for 2 measures and faster for 1 measure following the launch of the Plus awards than in the period before the Plus Awards. However, when measured as the relative decrease in the gap in care, the program was associated with most of the processes improving at an accelerated rate following launch of the program.

These findings demonstrate that the Plus Awards were effective in providing recognition for hospitals with superior performance on the quality measures that were the focus of the awards criteria. Those hospitals receiving the Plus Awards provided higher quality care as measured by all the quality measures compared to those hospitals not recognized. During the postaward period, clinically relevant improvements were observed in the performance for each quality measure. However, the relative rates of improvement in the postaward period were increased only for the IV tPA within 60‐minute measure, which was the focus of a separate focused performance improvement initiative Target Stroke,14, 15 which remained similar for 3 measures, and actually decelerated for 2 quality measures.

There are several potential explanations for the decrease in the rate of improvement following the launch of the awards. Each of the process metrics has a ceiling at 100%, and “room” for additional improvement continually decreases as care improves. If changes in performance over time were small compared to the gap in care (current versus ideal state), then the examination of rates of change over time may reflect an impact of an intervention. However, we found that care was improving rapidly prior to the launch of the Plus awards. Thus, the impact of ceiling effects may have occurred regardless of the initiation of Plus awards.

Our findings provide guidance for those wishing to evaluate a new intervention when contemporaneous controls are not possible. If quality is stable or only slowly improving, then an analysis of rate of change over time may detect moderate or greater effects of an intervention. However, if the gap in care is rapidly decreasing at baseline, it may not be possible to detect an incremental effect of any intervention. Measuring the decrease in the “care gap” may be more revealing if the process use is already over 50%.

Our results are consistent with a prior evaluation of the GWTG‐Heart Failure enhanced Award program.16 After the American Heart Association expanded their recognition program by adding additional heart failure quality measures, the investigators noted that care improved. However, as with the current analysis of stroke care, the rate of improvement in heart failure care did not increase after enhancement of the heart failure recognition program. Those metrics that were at a relatively low level prior to launch increased more rapidly than those metrics that were at a higher baseline level.

Other reasons for a slowed rate of increase following the launch of the Plus awards include that the GWTG‐Stroke Performance Achievement Award remained the primary motivator for quality improvement even though compliance was already at a high level. Achievement measures usually have a stronger evidence base compared to quality measures that may also contribute to the hospitals’ higher level of compliance for Achievement than quality measures. In contrast to the Performance Achievement Award recognition, the Plus awards may not have provided sufficient incentive for hospitals to focus additional meaningful performance improvement efforts on these processes. The hospitals may have felt that an additional award had insufficient value to devote resources to change practice. It is also possible the way the Plus Award recognition program was structured, with the option of which measures to select for recognition, was less effective for facilitating process improvement. The impact of any recognition program may be weakened by strong financial incentives being implemented by many payers including Medicare that began during the study (eg, readmissions reduction), but did not specifically involve patients hospitalized with stroke.17

One concern of expanding the number of measures used for recognition or pay for performance is that they will detract from existing measures. In our study there was the potential that hospitals would focus less on the more established (and important) quality measures for stroke care by redirecting resources toward improving the quality measures that were part of the expanded recognition program. However, our results do not provide any evidence of such an effect, as the existing Achievement measures also improved with the launch of the expanded program.

Limitations

There are several potential limitations of this study. The American Heart Association's use of the Plus Award program was nonrandomized, nor did it have a contemporaneous control. The baseline rates of use were often rapidly increasing at the time of the program launch, making it difficult to determine the incremental impact of the program. In addition, the time duration between pre and post measurement may be too short to detect important differences because of the Plus Award Program. The quality measures evaluated by the new recognition program were already reported, privately, to the individual hospitals. Hospitals participating in GWTG‐Stroke may be more interested in quality improvement than other hospitals, and these hospitals may have already focused on many of the quality metrics. An additional award may have been a minimal incentive for these higher‐performing hospitals. The GWTG‐Stroke program is voluntary, and it is not clear whether an award program would have a similar impact if hospitals were mandated to participate.

In summary, we found that an expanded recognition program for the quality of stroke care, while providing recognition to hospitals with superior performance on quality measures, did not have a clear impact on accelerating improvements in care. While care improved compared to baseline, the rate of care improvement slowed for some measures. Importantly, the assessment of the program's impact was different if quality was measured as the relative increase in use or the relative decrease in nonuse (gap in care). Our findings demonstrate the difficulty in interpreting the impact of hospital or provider incentives when contemporaneous controls are not feasible.

Sources of Funding

The Get With The Guidelines–Stroke (GWTG‐Stroke) program is provided by the American Heart Association. GWTG‐Stroke has been funded in the past through support from Medtronic, GlaxoSmithKline, Ortho‐McNeil, and the American Heart Association Pharmaceutical Roundtable. Dr Heidenreich is supported by a grant from the Veteran Administration's Quality Enhancement and Research Initiative 04‐326.

Disclosures

Dr Schwamm reports research for PCORI (significant), NINDS (significant), Other: unpaid chair of GWTG Stroke clinical workgroup. Dr Fonarow reports research for PCORI (significant). The remaining authors have no disclosures to report.

Table S1. Comparison of Patient Characteristics Between Those Excluded and Those Included in the Study Population.

Click here for additional data file.