Perception Versus Actual Performance in Timely Tissue Plasminogen Activation Administration in the Management of Acute Ischemic Stroke.

BACKGROUND: Timely thrombolytic therapy can improve stroke outcomes. Nevertheless, the ability of US hospitals to meet guidelines for intravenous tissue plasminogen activator (tPA) remains suboptimal. What is unclear is whether hospitals accurately perceive their rate of tPA "door-to-needle" (DTN) time within 60 minutes and how DTN rates compare across different hospitals. METHODS AND
RESULTS: DTN performance was defined by the percentage of treated patients who received tPA within 60 minutes of arrival. Telephone surveys were obtained from staff at 141 Get With The Guidelines hospitals, representing top, middle, and low DTN performance. Less than one-third (29.1%) of staff accurately identified their DTN performance. Among middle- and low-performing hospitals (n=92), 56 sites (60.9%) overestimated their performance; 42% of middle performers and 85% of low performers overestimated their performance. Sites that overestimated tended to have lower annual volumes of tPA administration (median 8.4 patients [25th to 75th percentile 5.9 to 11.8] versus 10.2 patients [25th to 75th percentile 8.2 to 17.3], P=0.047), smaller percentages of eligible patients receiving tPA (84.7% versus 89.8%, P=0.008), and smaller percentages of DTN ≤60 minutes among treated patients (10.6% versus 16.6%, P=0.002).
CONCLUSIONS: Hospitals often overestimate their ability to deliver timely tPA to treated patients. Our findings indicate the need to routinely provide comparative provider performance rates as a key step to improving the quality of acute stroke care.

Stroke is the fourth leading cause of mortality and a leading cause of disability in the United States.1 Early detection and treatment of acute stroke symptoms with intravenous tissue plasminogen activator (tPA) has been demonstrated to improve both short- and long-term outcomes including functional status and quality of life.2 Despite widely disseminated guidelines on patient eligibility criteria and process-of-care metrics, many patients do not receive these recommended interventions in a timely fashion.3 In an effort to optimize stroke care, evidence-based management guidelines and protocols for the establishment of primary stroke centers have been benchmarked nationally through voluntary national registries, such as the American Heart Association’s (AHA’s) Get With The Guidelines (GWTG)–Stroke.4–8 In addition, best practices in stroke care are publicly disseminated to help lagging hospitals improve their care to meet guideline recommendations. Despite these efforts, significant variation and gaps are still present in hospitals’ use and application of AHA process guidelines.3

An element identified as critical to successfully transforming health care delivery is having the impetus to change.9 Overestimation of the quality of care that an institution provides may perpetuate suboptimal performance, whereas accurate measurements of current performance and realistic comparison to other more successful sites might provide the needed motivation to fuel quality improvement.9–12 Prior studies have found that without access to accurate, reliable, and timely comparative data, providers have a tendency to believe they are at least as good as others.9–12 To date, no studies have looked at how hospitals perceive their quality of stroke care relative to other hospitals.

The purpose of this study was to compare stroke teams’ perceptions of their performance against known metrics associated with tPA administration. We surveyed GWTG-Stroke registry institutions to examine the extent to which hospital staff accurately reported both the percentage of eligible patients who received tPA at their site and the percentage of treated patients who received tPA within 60 minutes of arrival. We also assessed the stroke teams’ perceptions of how their performance on guideline-based tPA administration compared with their peers. Finally, we sought to identify hospital factors associated with overestimating “door-to-needle” (DTN) performance.

Methods

The GWTG-Stroke registry is an initiative of the AHA and the American Stroke Association, with the goal of improving the care and outcomes of patients with stroke and transient ischemic attacks. Details of the design and conduct of the GWTG-Stroke program have been published previously.6 GWTG-Stroke uses a Web-based patient management tool (Outcome Sciences, Inc) to collect clinical data on consecutively admitted patients, to provide decision support, and to enable real-time online reporting features.6,13 Hospital personnel collect data on patients admitted with the principal clinical diagnosis of acute stroke or transient ischemic attack by prospective clinical identification, retrospective identification through the use of discharge codes, or a combination thereof. Outcome Sciences, Inc, serves as the data collection and coordination center for GWTG-Stroke. The Duke Clinical Research Institute serves as the GWTG data analysis center and has an agreement to analyze the aggregate deidentified data for research purposes. The Duke University Health System institutional review board determined that this project met the criteria for a declaration of exemption under the Code of Federal Regulations 45 CFR 46.101(b)(2).

Patient data, such as demographics, medical history, onset time of stroke symptoms (recorded as last known well time), arrival time, treatments and procedures, tPA treatment initiation time, and tPA complications, were abstracted from the GWTG-Stroke registry. Data on hospital-level characteristics (eg, bed size, geographical region, teaching status, and annual stroke volume) were gathered from the AHA hospital database.6

Survey Tool Creation and Implementation

A structured survey tool consisting of 70 items regarding components of stroke care management, including communication and teamwork, current process of care, organizational culture, performance monitoring and feedback, and overcoming of barriers was used for this study. The survey was a standardized telephone questionnaire; each item had 5 Likert-type answer options, and respondents chose the option that most accurately fit the site’s status. The survey questions were developed based on a previous qualitative study conducted by our research group and content experts.14 The questions were then reviewed by 10 stroke neurologists, stroke advanced practice nurses, and coordinators—all considered experts in the field. Based on this feedback, the survey was refined and then given to 5 stroke nurse coordinators for additional input. The survey was field tested through telephone interviews with medical and nursing staff at community hospitals, further refined through factor analysis, and then finalized.15 Three questions were created to elicit each site’s perception of tPA administration and DTN performance, which was defined as the percentage of patients who received tPA within 60 minutes of arrival among all patients who received tPA at the institution. Respondents were asked to quantify both their tPA administration rate among eligible patients and their DTN performance as 0% to 20%, 20% to 40%, 40% to 60%, 60% to 80%, or 80% to 100%. They were further asked to rank themselves as a low, below-average, average, above-average, or top performer in terms of their DTN performance on a national level. Accuracy of perception was defined as the respondent’s ability to correctly identify their performance status relative to all GWTG-Stroke hospitals (below average, average, above average). Responses overestimating or underestimating performance were scored as inaccurate (eg, perceiving hospital performance as average when it is actually below average). A factor analysis of the other survey items has been published separately.15

Survey targets were key personnel in stroke care at eligible GWTG-Stroke sites. Contact was established through information from the GWTG-Stroke database (as available) or from AHA quality improvement directors or by direct calling of the hospital. We interviewed a representative from each institution who identified himself or herself as a person familiar with the hospital’s tPA administration process. These interviewees included stroke neurologists, stroke coordinators, and hospital-based quality-improvement staff who were considered experts in the field. Interviews were ≈15 minutes long and were conducted by telephone between January 2011 and July 2011.

Statistical Analysis

Patient sociodemographic and clinical variables, hospital characteristics, and quality-of-care measures were compared among hospitals classified according to performance ranking. Performance rank, categorized as top, middle, and low performers, was considered to be ordinal. Categorical variables were reported as percentages, and continuous variables were reported as medians and 25th and 75th percentiles. Categorical variables were compared using χ2 rank-based group mean-score statistics. Continuous variables were compared using Cochran–Mantel–Haenszel statistics.

Further analysis of sites focused on the dichotomized ranking of their relative performance into overestimation versus no overestimation. Top-performing hospitals were excluded for this analysis because their ranking could not be overestimated. Middle-performing hospitals that answered “above average” or “top performer” were categorized as overestimation, whereas those that answered “average,” “below average,” or “low performer” were categorized as no overestimation. Low-performing hospitals that answered “average,” “above average,” or “top performer” were labeled as overestimation, whereas those that answered “below average” or “low performer” were labeled as no overestimation. Sociodemographic factors, clinical variables, hospital characteristics, and quality-of-care measures were compared between these 2 groups. Pearson χ2 tests and Wilcoxon rank-sum tests were used to compare categorical and continuous variables, respectively. A multivariable logistic regression model was used to determine which factors were independently associated with DTN performance overestimation. Variables that were likely potential confounders were identified a priori to be included in this model based on literature review and clinical judgment. These variables were Joint Commission Primary Stroke Center status, geographic region, annual volume of tPA, annual volume of ischemic stroke, percentage of DTN time within 60 minutes, and years that the survey respondent had been with the site’s stroke team.

All tests were 2-sided with P<0.05 considered as the level of statistical significance. All statistical analyses were performed using SAS version 9.3 (SAS Institute).

Results

All 1395 hospitals participating in GWTG-Stroke between October 1, 2009, and September 30, 2010, were included as potential survey sites. Low-volume sites, defined as those treating <6 acute ischemic stroke patients with tPA during the study period, were excluded from the sampling frame. The remaining hospitals (n=704) were ranked by DTN performance. A total of 300 hospitals were identified for the telephone survey. The 100 highest and 100 lowest performing hospitals from this ranked list were selected, with DTN performance ranging from 45.2% to 92.3% and from 0.0% to 0.0%, respectively. Fifty additional hospitals above and below the median DTN performance of the ranked list (20.0%) were selected to form 100 middle-performing hospitals (DTN performance range 16.7% to 25.0%). Of these 300 sites, 157 responded to initial site contact. Among these, 16 sites were excluded: 2 sites refused to complete a telephone interview, and an additional 14 sites did not respond to key questions or had incomplete data (>10% of responses missing). The final sample represented 141 hospitals that treated 48 201 stroke patients during the study period.

Patient- and hospital-level characteristics were compared between surveyed and nonsurveyed hospitals. Surveyed hospitals tended to have a higher number of eligible patients receiving tPA (P=0.001) and a higher annual tPA volume (P=0.006) but did not have significant differences in other hospital characteristics, including annual stroke volume or rate of DTN ≤60 minutes (Table1). Most patient demographics queried showed statistically significant differences, including race/ethnicity, insurance status, and components of medical history, although many of these differences were modest (Table2).

Table 1

Hospital Characteristics of Surveyed Versus Nonsurveyed Sites

	Surveyed Hospitals (n=141)	Nonsurveyed Hospitals (n=159)	P Value
Hospital region, %
West	24.1	14.5	0.023
South	27.7	43.4
Midwest	17.7	17.0
Northeast	30.5	25.2
Rural, %	1.4	5.0	0.079
Academic, %	49.7	48.4	0.732
TJC Primary Stroke Center, %	45.4	46.5	0.842
Annual tPA volume, median (25th to 75th P)	10.7 (7.5 to 16.0)	8.2 (5.6 to 15.0)	0.006
Annual ischemic stroke volume, median (25th to 75th P)	170.8 (115.0 to 225.6)	160.7 (102.0 to 230.1)	0.260
Number of eligible patients treated with tPA, median (25th to 75th P)*	12.0 (8.0 to 20.0)	9.0 (6.0 to 15.0)	0.001
% DTN ≤60 minutes in treated patients, median (25th to 75th P)	20.0 (0.0 to 50.0)	16.7 (0.0 to 44.0)	0.213

25th to 75th P indicates 25th to 75th percentile; DTN, door-to-needle; TJC, The Joint Commission; tPA, tissue plasminogen activator.

Data from October 1, 2009, to September 30, 2010.

Table 2

Patient Characteristics of Surveyed Versus Nonsurveyed Sites

	Surveyed Hospitals (n=141)	Nonsurveyed Hospitals (n=159)	P Value
Age, median (25th to 75th P), y	72 (59 to 82)	71 (59 to 82)	0.357
Male, %	47.6	46.8	0.010
Race/ethnicity, %
White, non-Hispanic	70.0	70.5	<0.001
African American	15.5	16.1
Hispanic	6.3	6.2
Asian	3.4	2.1
Other	1.8	0.9
Insurance status, %
Medicare	34.7	36.6	<0.001
Medicaid	8.6	9.5
Self/none	6.6	6.1
Private/VA/others	38.0	38.2
Medical history, %
Atrial fibrillation/flutter	17.3	17.1	0.490
Prosthetic heart valve	1.4	1.5	0.026
Previous stroke/TIA	31.9	32.6	0.033
CAD/prior MI	26.5	28.3	<0.001
Carotid stenosis	4.2	5.3	<0.001
Diabetes mellitus	31.7	32.7	0.002
Peripheral vascular disease	4.3	4.8	<0.001
Hypertension	80.3	81.0	0.011
Smoker	18.5	18.7	0.478
Dyslipidemia	43.1	43.3	0.605
Heart failure	7.7	8.2	0.010

25th to 75th P indicates 25th to 75th percentile; CAD, coronary artery disease; MI, myocardial infarction; TIA, transient ischemic attack; VA, veterans affairs.

The study population consisted of 49 top performers, 52 middle performers, and 40 low performers. Most personnel interviewed from each hospital were nursing staff (85.0%). On average, respondents had been working with their current stroke teams for 4 years (25th to 75th percentile 2 to 6 years). Comparisons of hospital and patient characteristics among these performance groups are shown in Tables3 and 4. Significant hospital-level factors that varied among the 3 performance groups were volume of tPA administration during the performance evaluation period (October 1, 2009, to September 30, 2010; P<0.001), annual volume of tPA administration (P<0.001), and annual ischemic stroke admissions (P=0.045). Top-performing hospitals tended to have more ischemic stroke admissions per year and administered tPA to a greater number of patients. In addition, most patient-level variables showed statistically significant differences among the 3 performance groups. These variables included median age, gender, race/ethnicity, insurance status, and comorbidities such as previous stroke or transient ischemic attack (Table4).

Table 3

Hospital Characteristics by Performance on DTN ≤60 Minutes

	Top Performers (n=49)	Middle Performers (n=52)	Low Performers (n=40)	P Value
Hospital region, %
West	32.7	25.0	12.5	0.125
South	24.5	25.0	35.0
Midwest	12.2	19.2	22.5
Northeast	30.6	30.8	30.0
Rural, %	0.0	1.9	2.5	0.309
TJC Primary Stroke Center, %	46.9	42.3	47.5	0.998
Annual tPA volume, median (25th to 75th P)	14.0 (8.5 to 23.3)	11.3 (8.2 to 17.2)	7.8 (4.6 to 10.3)	<0.001
Annual ischemic stroke volume, median (25th to 75th P)	199.0 (107.5 to 286.4)	174.2 (128.2 to 234.5)	145.9 (109.5 to 189.9)	0.045
Number of eligible patients treated with tPA, median (25th to 75th P)*	15.0 (9.0 to 30.0)	14.0 (9.0 to 23.0)	8.0 (5.0 to 12.0)	<0.001
% DTN ≤60 minutes in treated patients, median (25th to 75th P)	57.1 (50.0 to 65.5)	18.5 (14.9 to 23.1)	0.0 (0.0 to 0.0)	<0.001
Title, %
Nurse	77.6	94.2	82.5	0.216
Neurologist	4.1	0.0	5.0
Administrator	2.0	0.0	7.5
Other	14.3	5.8	5.0
Years with current stroke team, median (25th to 75th P)	4.0 (3.0 to 6.0)	4.0 (2.5 to 6.0)	3.0 (2.0 to 5.0)	0.124

25th to 75th P indicates 25th to 75th percentile; DTN, door-to-needle; TJC, The Joint Commission; tPA, tissue plasminogen activator.

Data from October 1, 2009, to September 30, 2010.

Table 4

Baseline Patient Characteristics by Performance on DTN ≤60 Minutes

	Top Performers (n=49)	Middle Performers (n=52)	Low Performers (n=40)	P Value
Age, median (25th to 75th P), y	70 (58 to 81)	73 (60 to 83)	72 (59 to 82)	<0.001
Male, %	49.0	47.3	46.0	<0.001
Race/ethnicity, %
White, non-Hispanic	64.1	75.6	70.2	<0.001
African American	15.8	12.7	19.1
Hispanic	7.9	5.8	4.8
Asian	4.6	2.9	2.6
Other	3.1	1.1	1.0
Insurance status, %
Medicare	38.5	31.0	34.7	<0.001
Medicaid	8.6	8.9	8.4
Self/none	7.5	6.4	5.6
Private/VA/others	31.4	46.5	34.7
Medical history, %
Atrial fibrillation/flutter	15.9	18.4	17.5	<0.001
Prosthetic heart valve	1.1	1.5	1.5	0.002
Previous stroke/TIA	29.3	33.0	33.9	<0.001
CAD/prior MI	26.1	26.6	26.8	0.217
Carotid stenosis	3.4	4.5	4.8	<0.001
Diabetes mellitus	31.6	31.3	32.5	0.182
Peripheral vascular disease	3.8	4.6	4.4	0.003
Hypertension	78.7	81.3	80.9	<0.001
Smoker	20.2	17.5	17.6	<0.001
Dyslipidemia	39.5	47.4	41.8	<0.001
Heart failure	7.1	8.3	7.7	0.013

25th to 75th P indicates 25th to 75th percentile; CAD, coronary artery disease; DTN, door-to-needle; MI, myocardial infarction; TIA, transient ischemic attack; VA, veterans affairs.

Almost two-thirds of respondents (65.3%) correctly estimated the percentage of eligible patients at their institution who received tPA. Figure1 shows a significant trend toward lower accuracy on this measure as DTN performance declined (P=0.002). Regardless of performance category, respondents reported less accurately on their performance on the DTN ≤60 minutes metric (Figure1). Overall, 29.1% of hospitals were able to accurately quantify their DTN ≤60 minutes ability, and this rate of accuracy was not statistically different among performance groups (P=0.853).

Figure 1

Perception of performance on tPA administration and rates of DTN ≤60 minutes. Percentage of accurate responses regarding tPA administration and rates of DTN ≤60 minutes at respondent’s home institution. DTN indicates door-to-needle; tPA, tissue plasminogen activator.

Although the median DTN ≤60 minutes rate among top-performing hospitals was 57.1% of treated patients, the majority (67.4%) of top-performing hospitals estimated their performance on DTN ≤60 minutes to be >60% (Figure2). Similarly, 67.5% of low-performing hospitals perceived their performance to be >20% (actual median DTN ≤60 minutes rate 0.0%). The majority (81.6%) of top performers recognized themselves as above average or top in their ability to deliver timely tPA relative to other sites (Figure3). In contrast, low performers tended to overestimate their relative ability to provide timely tPA compared with others: 85.0% reported their DTN ≤60 minutes performance to be average, above average, or top in comparison to other US hospitals.

Figure 2

Perception of performance on door-to-needle (DTN) ≤60 minutes by quartile for hospital ranking. Distribution of responses regarding perceived achievement rate of DTN ≤60 minutes at respondent’s home institution divided by hospital ranking, based on actual performance data, and comparison of response distribution with actual median within performance subgroups. *Indicates category for median rate of actual DTN ≤60 minutes for each classification of performers.

Figure 3

Perceived relative performance on door-to-needle (DTN) ≤60 minutes by hospital ranking. Distribution of responses regarding perceived success rate of DTN ≤60 minutes at respondent’s home institution relative to other US institutions divided by hospital ranking based on actual performance data.

Among all middle- and low-performing hospitals surveyed (n=92), respondent overestimation occurred at 56 sites (60.9%). Patient and hospital characteristics differed slightly for sites that overestimated their relative performance versus those that did not. Hospitals with staff that overestimated performance had higher percentages of minority patients; slightly younger median age populations; and higher combined percentages of patients on Medicaid, Medicare, or with no insurance. Hospitals with staff that reported accurately or that underestimated their performance relative to others had a slightly higher percentage of patients with almost every medical condition collected. Statistically significant percentages between the 2 groups were found for patients with a history of atrial fibrillation or flutter, coronary artery disease or previous myocardial infarction, peripheral vascular disease, smoking, dyslipidemia, and heart failure. At hospitals with staff that were accurate or that underestimated their performance, higher percentages of patients were on antiplatelet medications, cholesterol-lowering medications, and anticoagulation medications prior to admission.

Sites with overestimated DTN performance tended to have lower volumes of tPA administration during the performance evaluation period (median 10.0 [25th to 75th percentile 8.0 to 15.5] versus median 14.5 [25th to 75th percentile 9.0 to 22.5], P=0.022), smaller percentages of eligible patients receiving tPA (84.7% versus 89.8%, P=0.008), and smaller percentages of DTN ≤60 minutes (10.6% versus 16.6%, P=0.002) (Table5). Other hospital characteristics, such as hospital size, Joint Commission Primary Stroke Center status, academic affiliation, geographic region, and hospital location (rural versus urban) were not found to be associated with accuracy of performance perception.

Table 5

Hospital Characteristics of Overestimation Versus No Overestimation

	Overestimation (n=56)	No Overestimation (n=36)	P Value
Hospital region, %
West	17.9	22.2	0.415
South	35.7	19.4
Midwest	19.6	22.2
Northeast	26.8	36.1
Rural, %	1.8	2.8	0.736
TJC Primary Stroke Center, %	46.4	41.7	0.656
Annual tPA volume, median (25th to 75th P)	8.4 (5.9 to 11.8)	10.2 (8.2 to 17.3)	0.047
Annual ischemic stroke volume, median (25th to 75th P)	163.1 (111.9 to 219.2)	166.5 (128.4 to 209.6)	0.609
Administered tPA among eligible patients	84.7	89.8	0.008
DTN ≤60 minutes in treated patients, %	10.6	16.6	0.002
Title, %
Nurse	85.7	94.4	0.339
Neurologist	3.6	0.0
Administrator	5.4	0.0
Others	5.4	5.6
Years with current stroke team, median (25th to 75th P)	4.0 (2.0 to 5.5)	4.0 (2.5 to 6.0)	0.731
Percentage of hospitals accurately reporting eligible patients who receive tPA, %	58.9	58.3	0.955
Percentage of hospitals accurately reporting treated patients who receive tPA <60 minutes, %	23.2	33.3	0.290

25th to 75th P indicates 25th to 75th percentile; DTN, door-to-needle; TJC, The Joint Commission; tPA, tissue plasminogen activator.

In a multivariable analysis of factors associated with relative DTN performance overestimation, sites were less likely to overestimate their performance for every 10-case increase in annual tPA volume (odds ratio 0.31, 95% CI 0.10 to 0.94) and for every 10-percentage-point increase in the percentage of DTN times within 60 minutes of arrival (odds ratio 0.59, 95% CI 0.36 to 0.96). Other factors were not significantly associated with overestimation of relative performance. These results are shown in Table6.

Table 6

Multivariable Model of Overestimation

Variable	OR	95% CI	P Value
TJC Primary Stroke Center	1.15	(0.41 to 3.17)	0.792
Geographic region
Northeast	1.00 (ref)	—	—
Midwest	1.03	(0.28 to 3.71)	0.970
South	1.76	(0.50 to 6.22)	0.380
West	1.62	(0.40 to 6.46)	0.498
Annual volume of tPA administration, per 10-case increase	0.31	(0.10 to 0.94)	0.039
Annual volume of ischemic stroke admissions, per 50-case increase	1.38	(0.91 to 2.10)	0.128
Percentage of DTN within 60 minutes during the study period, per 10-percentage-point increase	0.59	(0.36 to 0.96)	0.035
Number of years survey respondent had been with the site’s stroke team	1.01	(0.89 to 1.15)	0.849

DTN indicates door-to-needle; OR, odds ratio; TJC, The Joint Commission; tPA, tissue plasminogen activator.

Discussion

Our study demonstrates that staff perception of their own hospital’s performance in terms of timeliness of tPA administration in acute stroke care is generally more optimistic. Less than one-third of identified hospital-based stroke-care quality leaders could accurately quantify their institution’s rate of DTN ≤60 minutes. Furthermore, hospital personnel tended to overestimate their DTN performance relative to peers. Among low-performing hospitals, 85% overestimated their institution’s performance of DTN time, with almost 5% of low performers believing their DTN performance was “superior” on a national level. Sites with higher annual volumes of tPA administration and higher percentages of DTN times within 60 minutes were less likely to overestimate their relative DTN performance.

We found that institutions often did not recognize their underperformance relative to other sites, and that is consistent with prior studies. A survey of US hospital board chairs found that two-thirds of respondents rated their institution’s Joint Commission or Hospital Quality Alliance performance as better or much better than that of other hospitals. Among low-performing hospitals, 58% reported their performance to be better or much better than the national average; none of these hospitals reported their performance to be worse or much worse than their US peers.10 A Swedish survey by Pukk et al found similar results; respondents tended to rate their institution as equal to or better than others.12 Nonetheless, previous studies have noted that the impetus to change is 1 of the top 5 elements critical to the successful and sustainable transformation of patient care.16 Consequently, addressing misperceptions that one’s performance is average or above par when it actually is not is an important step in addressing this motivation for change. In addition, previous studies have demonstrated that having leadership and a culture of continuous quality improvement, as well as a financial commitment to quality, are highly associated with hospital improvement in the care of patients with acute coronary syndromes.17 Providing service-line directors, nursing analytic tools, and comparative data further assist in perpetuating interest in continuous quality improvement.18

A study of hospital board chairs showed that chairs of top-performing hospitals were more likely to report being familiar with quality measures and to regularly review performance reports.10 Prior systematic reviews have also demonstrated the impact of audit and feedback on health care professionals’ compliance with desired practice and shown that feedback is more effective when baseline performance is low.19 Consequently, it appears that with access to comparative data, respondents are more realistic about their performance, whereas in instances in which no or little data were available, respondents tended to overestimate their own performance.12,20

In our study, overall stroke volume and tPA volume were related to performance of DTN time, and top performers were more accurate in their perceptions of performance, suggesting that they may track this information and that these metrics are important. In contrast, lower performers overestimated their institution’s performance of tPA use relative to peers. This gap between performance and perception could be the result of better communication of internal and relative performance at high-performing hospitals; this communication may increase awareness and engage all stroke-care staff in quality improvement. Understanding these differences will help improve stroke care for all hospitals.

Our study provides further insights into the challenges of feedback and provider self-reflection. Critical to the success and interest in GWTG-Stroke as a voluntary quality-improvement initiative is the ability for hospitals to submit their institutional stroke care data and to have this information benchmarked against similar peers. The basis for the program is to provide hospital physicians and leadership with the ability to assess their quality-improvement efforts relative to secular and policy trends that influence all hospitals’ care of stroke patients. All institutions in our study were able to download routine site-specific performance data, yet it is unclear how individual hospitals used this information and shared metrics among personnel in stroke care for assessment of quality gaps and case review. The striking discordance among all hospitals and the care provided suggests that opportunities exist to focus communication of individual performance and relative performance across personnel involved in stroke care. Even though our survey efforts selected a single member of a larger team or a person with limited experience, training or exposure to these data and the fragmentation in the understanding of relative performance are still reflected. Further studies are needed to evaluate practices used by stroke teams to stay abreast of their performance on a national scale and will help implementation of targeted interventions to address the misperceptions identified through our research.

Limitations

Several limitations of this study need to be acknowledged. First, only GWTG-Stroke hospitals were included in this survey. Participation in the GWTG-Stroke registry is voluntary; therefore, despite participation from >1000 hospitals across the United States, this study could select centers with greater interest in stroke-care quality improvement. Consequently, the accuracy of perception of quantitative performance may not be applicable to non–GWTG-Stroke hospitals. Extrapolation of these results on perception of relative performance may also be inaccurate. Second, as with all surveys, the data presented are dependent on the subjective interpretation and biases of interview participants. Furthermore, only 1 representative was interviewed from each institution; obtaining perceptions from a larger number of team members from each hospital may have produced different findings. Surveys and interview formats were standardized to minimize variance among interviews. Third, although our response rate was quite high among those we were able to successfully contact (141 of 157; 89%), we failed to find correct contact information for many sites that were originally selected. Finally, although hospitals were identified based on their performance in 2009–2010, interviews were conducted between January 2011 and July 2011, and that may have created some historical bias. Quality-improvement initiatives implemented during this time lag may have affected both measured and perceived performance.

Conclusion

Personnel at low-performing hospitals had a tendency to overestimate their performance in timely tPA administration in stroke care relative to other sites participating in GWTG-Stroke. Such overestimation may have important implications for the low motivation to investigate improvement barriers and to implement site-specific quality-improvement initiatives. Further research is needed to identify the reasons for this misperception in performance. Potential changes in benchmarked performance feedback reports should be considered to ensure broader distribution of improvement initiatives.