Association of Magnet Status With Hospitalization Outcomes for Ischemic Stroke Patients.

BACKGROUND: It is not clear whether Magnet recognition by the American Nurses Credentialing Center (nursing excellence program) is associated with improved patient outcomes. We investigated whether hospitalization in a Magnet hospital is associated with improved outcomes for patients with ischemic stroke. METHODS AND
RESULTS: We performed a cohort study of patients with ischemic stroke from 2009 to 2013, who were registered in the New York Statewide Planning and Research Cooperative System database. Propensity-score-adjusted multivariable regression models were used to adjust for known confounders, with mixed effects methods to control for clustering at the facility level. An instrumental variable analysis was used to control for unmeasured confounding and simulate the effect of a randomized trial. During the study period, 176 557 patients were admitted for ischemic stroke, and met the inclusion criteria. Of these, 32 092 (18.2%) were hospitalized in Magnet hospitals, and 144 465 (81.8%) in non-Magnet institutions. Instrumental variable analysis demonstrated that hospitalization in Magnet hospitals was associated with lower case-fatality (adjusted difference, -23.9%; 95% CI, -29.0% to -18.7%), length of stay (adjusted difference, -0.4; 95% CI, -0.8 to -0.1), and rate of discharge to a facility (adjusted difference, -16.5%; 95% CI, -20.0% to -13.0%) in comparison to non-Magnet hospitals. The same associations were present in propensity-score-adjusted mixed effects models.
CONCLUSIONS: Using a comprehensive all-payer cohort of patients with ischemic stroke in New York State, we identified an association of treatment in Magnet hospitals with lower case-fatality, discharge to a facility, and length of stay. Further research into the factors contributing to the superiority of Magnet hospitals in stroke care is warranted.

Introduction

Public reporting is at the core of recently enacted legislation aimed at improving quality, and empowering shared decision‐making.1, 2, 3, 4, 5, 6 The Magnet Recognition Program of the American Nurses Credentialing Center7 is one such initiative designed to identify healthcare facilities with a commitment to quality improvement, and excellent nursing care delivery. This program involves rigorous documentation and site visits to evaluate institutions across 5 core principles: transformational leadership, a structure that empowers staff, an established professional nursing practice model, support for knowledge generation and application, and robust quality improvement mechanisms (benchmarking, morbidity review, etc).7 The stated goal of the Magnet program is to “improve patient care.”7 Through recent inclusion in US News and World Report rating,8 endorsement by the Leapfrog Group,9 and media attention, these initiatives are increasingly recognized by the public.

Prior studies have investigated the association of Magnet recognition with outcomes for different patient groups. Some researchers have shown improved outcomes in Magnet hospitals for elderly Medicare medical and surgical patients.10, 11 However, others failed to demonstrate a similar benefit of Magnet status.12, 13, 14 These and other retrospective analyses failed to control for unmeasured confounding, stemming from the nonrandom patient allocation to particular hospitals. There has been no previous investigation attempting to answer this question in a comprehensive all‐payer cohort, while controlling for unmeasured confounding.

We used the New York Statewide Planning and Research Cooperative System (SPARCS)15 to study the association of being hospitalized in a Magnet hospital with case‐fatality, discharge to a facility, and length of stay (LOS) for patients with ischemic stroke.

Methods

New York Statewide Planning and Research Cooperative System

This study was approved by the Dartmouth Committee for Protection of Human Subjects. The study was based on de‐identified data and the consent process was waived. All patients who were hospitalized for acute ischemic stroke, and were registered in the SPARCS (New York State Department of Health, Albany, NY)15 database between 2009 and 2013 were included in the analysis. For these years, SPARCS contains patient‐level details for every hospital discharge, ambulatory surgery, and emergency department admission in New York State as coded from admission and billing records. More information about SPARCS is available at https://www.health.ny.gov/statistics/sparcs/.

Magnet Recognition Program

The Magnet Recognition program of the American Nurses Credentialing Center was established in 1994 by a subsidiary of the American Nurses Association.7 Magnet recognition is a voluntary program that lasts for 4 years. As of 2015, 402 facilities in the United States were recognized by the program. More information on this process can be found at http://www.nursecredentialing.org/Magnet. The program's website was used to identify hospitals in New York State that obtained Magnet recognition and the year this was achieved. Hospitals were classified as having Magnet recognition in the corresponding year of the analysis. Classifications were updated each year of the study period in case of mergers or closures.

Cohort Definition

In order to establish the cohort of patients, we used International Classification of Disease‐9‐Clinical Modification codes to identify patients in the database who were hospitalized for acute ischemic stroke (International Classification of Disease‐9‐Clinical Modification code 433.x1, 434.x1) between 2009 and 2013.

Outcome Variables

The primary outcome variable was case‐fatality during the initial hospitalization for ischemic stroke. Secondary outcomes were LOS during the initial hospitalization, and the rate of discharge to a facility. Discharge to a facility was defined as discharge to any location with services other than primarily to the patient's home.

Exposure Variables

The primary exposure variable was whether the stroke patient was admitted to a Magnet hospital for their care.

Covariates (Table S1) used for risk‐adjustment were age, sex, race (black, Hispanic, Asian, white, other), insurance (private, Medicare, Medicaid, uninsured, other), patient location during the stroke (inpatient versus outpatient setting), and stroke intervention either via administration of intravenous tissue plasminogen activator (International Classification of Disease‐9‐Clinical Modification 99.10, V45.88) or mechanical thrombectomy (International Classification of Disease‐9‐Clinical Modification 39.74). The comorbidities used for risk adjustment were diabetes mellitus, smoking, chronic lung disease, hypertension, hypercholesterolemia, peripheral vascular disease, congestive heart failure, coronary artery disease, history of stroke or transient ischemic attack, alcohol abuse, obesity, chronic renal failure, and coagulopathy. Only variables that were defined as “present on admission” were considered part of the patient's preadmission comorbidity profile.

We additionally controlled for hospital characteristics including primary stroke center or comprehensive stroke center status, hospital size, and Get with the Guidelines program participation (70.8% of Magnet hospitals, and 30.4% of non‐Magnet institutions).16

Statistical Analysis

The association of Magnet recognition with our outcome measures was examined in a multivariable setting. Patients admitted to a Magnet hospital or a non‐Magnet institution were nonrandomly directed to either facility (depending on presentation, ambulance protocols, etc). In order to account for this unmeasured confounding, and to simulate the effect of randomization, we used an instrumental variable analysis, an econometric technique.17 The differential distance of the patient to the closest Magnet hospital (distance to a Magnet hospital minus distance to a non‐Magnet institution) was used as an instrument for the treatment facility. This advanced observational technique has been used before by clinical researchers, to answer comparative effectiveness questions for different interventions, including the value of primary stroke centers.18 The goal is to simulate randomization, especially when the baseline functional characteristics of the patients (including the functional status of patients with stroke) are unknown (similar to our application).19, 20, 21 The instrumental variable analysis is utilized to minimize the impact of such unmeasured confounders. This analysis uses the differences across regions to simulate the structure of a randomized trial, in an observational setting. It attempts to create balance of unmeasured covariates among treatment groups.

A good instrument is not associated with the outcome other than through the exposure variable of interest (a requirement known as the exclusion restriction criterion).22 In our case it is unlikely that the differential distance to the closest Magnet hospital would be associated with case‐fatality in any way other than the choice of treatment facility. A 2‐stage least‐squares method was used for the calculation of the coefficients. The value of the F statistic in the first stage of the 2‐stage least‐squares approach was 112, which is consistent with a strong instrument (F statistic >10), based on a practical rule.17

A probit regression was used for the categorical outcomes (case‐fatality, and discharge to a facility),23 and a linear regression for the linear outcomes (LOS). The covariates used for risk adjustment in these models were as follows: age, sex, race, insurance status, and all the comorbidities and hospital characteristics mentioned previously. Since the coefficients produced by the probit function are not interpretable, we used the marginal effects of our independent variables instead. The marginal effects are the partial derivatives of the coefficients, and reflect the change in the probability of the dependent variable, for 1 unit change in the independent variable, at the average value of all other covariates.

In order to demonstrate the robustness of our data in a sensitivity analysis, we used standard techniques to account for measured confounding, while accounting for clustering at the hospital level. For categorical outcomes, we used a probit regression model with hospital ID as a random effects variable, while controlling for all the covariates mentioned previously. In an alternative way to control for confounding, we used a propensity‐adjusted (with deciles of propensity score) probit regression model. We calculated the propensity score of admission to a Magnet hospital with a separate probit regression model, using all the covariates mentioned previously. For continuous outcomes, we performed similar analyses using linear models. Logarithmic transformation of the values of LOS yielded identical results and is therefore not reported further.

Regression diagnostics were used for all models. Number needed to treat was calculated when appropriate. All results are based on 2‐sided tests, and the level of statistical significance was set at 0.05. This study, based on 176 557 patients, has sufficient power (80%) at a 5% type I error rate to detect differences in case‐fatality, as small as 0.7%. Statistical analyses were performed using Stata version 13 (StataCorp, College Station, TX).

Results

Patient Characteristics

In the selected study period, there were 176 557 patients hospitalized for acute ischemic stroke (mean age was 71.3 years, with 53.0% females) who were registered in SPARCS. There were 32 092 (18.2%) patients hospitalized in Magnet hospitals, and 144 465 (81.8%) in non‐Magnet institutions. The characteristics of the 2 cohorts at baseline can be seen in Table 1.

Table 1

Patient Characteristics

	All Patients		Patients Treated in Magnet Hospitals		Patients Treated in Non‐Magnet Hospitals		P Value
	N=176 557		N=32 092		N=144 465
	Mean	SD	Mean	SD	Mean	SD
Age	71.32	14.88	72.07	14.76	71.16	14.90	<0.0001
	N	%	N	%	N	%	P Value
Female sex	93 511	53.0	17 014	53.02	76 483	52.94	0.810
Race
White	106 983	60.8	22 898	71.57	84 083	58.38	<0.0001
Black	33 802	19.2	4624	14.45	29 170	20.25	<0.0001
Hispanic	15 520	8.8	1738	5.43	13 780	9.57	<0.0001
Asian	4974	2.8	816	2.55	4157	2.89	0.001
Other	14 763	8.4	1916	5.99	12 846	8.92	<0.0001
Insurance
Medicare	105 307	59.7	18 887	58.9	86 418	59.94	0.001
Medicaid	12 133	6.9	1621	5.05	10 510	7.29	<0.0001
Private	50 514	28.7	9278	28.93	41 228	28.6	0.182
Uninsured	6766	3.8	2107	6.57	4657	3.23	<0.0001
Other	1536	0.9	175	0.55	1361	0.94	<0.0001
Transient ischemic attack	12 725	7.2	2472	7.7	10 253	7.1	0.0001
Coronary artery disease	54 593	30.9	10 546	32.86	44 047	30.49	<0.0001
Chronic obstructive pulmonary disease	25 169	14.3	4546	14.17	20 623	14.28	0.610
Congestive heart failure	31 906	18.1	5893	18.36	26 013	18.01	0.133
Diabetes mellitus	59 964	34.0	9990	31.13	49 974	34.59	<0.0001
Coagulopathy	5506	3.1	1213	3.78	4291	2.97	<0.0001
Chronic renal failure	24 413	13.8	4746	14.79	19 667	13.61	<0.0001
Hypertension	135 560	76.8	24 077	75.02	111 483	77.17	<0.0001
Smoking	19 568	11.1	3469	10.81	16 096	11.14	0.086
Hypercholesterolemia	79 928	45.3	14 807	46.14	65 119	45.08	0.001
Obesity	10 811	6.1	1829	5.7	8980	6.22	0.001
Alcohol	6460	3.7	945	2.94	5515	3.82	<0.0001
Peripheral vascular disease	13 112	7.4	2705	8.43	10 407	7.20	<0.0001
Treated with IV tPA	10 160	5.75	1928	6.00	8232	5.70	0.031
Received mechanical thrombectomy	1308	0.74	154	0.48	1154	0.80	<0.0001
Received both treatments	717	0.41	68	0.21	649	0.45	<0.0001

IV tPA indicates intravenous tissue plasminogen activator.

Inpatient Case‐Fatality

Overall, 2525 (7.9%) inpatient deaths were recorded in Magnet hospitals and 12 855 (8.9%) in non‐Magnet institutions (Table 2). Hospitalization in a Magnet hospital for acute ischemic stroke was associated with lower case‐fatality in comparison to non‐Magnet institutions (difference, −6.7%; 95% CI, −8.9% to −4.5%) in unadjusted analysis. Likewise, using a probit regression with instrumental variable analysis, we identified that Magnet hospitals were associated with a 23.9% decreased case‐fatality (95% CI, −29.0% to −18.7%), in comparison to non‐Magnet institutions (Table 3). This persisted in a mixed effects probit regression model (adjusted difference, −10.5%; 95% CI, −12.8% to −8.1%) and a propensity score adjusted probit model (adjusted difference, −9.7%; 95% CI, −11.9% to −7.4%). This corresponded to 5 patients with stroke needing to be treated in a Magnet hospital to prevent 1 death.

Table 2

Magnet Status and Unadjusted Outcomes

	Inpatient Mortality	Discharge to Rehabilitation	Length of Stay
Patients treated in Magnet hospitals	2525 (7.9%)	14 390 (48.7%)	8.8 days (SD 14.1)
Patients treated in non‐Magnet hospitals	12 855 (8.9%)	658 589 (52.1%)	9.1 days (SD 15.9)

Table 3

Multivariable Models Examining the Association of Magnet Status With Outcomes

	Inpatient Mortalitya		Discharge to Rehabilitationa		Length of Stayb
	Adjusted Difference (95% CI)	P Value	Adjusted Difference (95% CI)	P Value	Adjusted Difference (95% CI)	P Value
Instrumental variable analysisc	−23.9% (−29.0% to −18.7%)	<0.001	−16.5% (−20.0% to −13.0%)	<0.001	−0.4 (−0.8 to −0.1)	<0.001
Mixed effects regressiond	−10.5% (−12.8% to −8.1%)	<0.001	−10.1% (−11.7% to −8.4%)	<0.001	−0.2 (−0.3 to −0.1)	<0.001
Propensity score adjusted regressiond	−9.7% (−11.9% to −7.4%)	<0.001	−9.9% (−11.6% to −8.3%)	<0.001	NA	NA

NA indicates not applicable.

Regressions based on probit models.

Regressions based on linear models.

Differential distance to Magnet hospital was used as an instrument of treatment hospital.

Hospital ID was used as a random effects variable.

Length of Stay

The average LOS was 8.8 days (SD 14.1) in Magnet hospitals, and 9.1 days (SD 15.9) in non‐Magnet institutions (Table 2). Magnet hospitals were associated with lower LOS than non‐Magnet institutions (difference, −0.3; 95% CI, −0.5 to −0.1) in the unadjusted analysis. Using a linear regression with instrumental variable analysis, we demonstrated (Table 3) that hospitalization in a Magnet hospital was associated with 0.4 days shorter LOS in comparison with non‐Magnet institutions (95% CI, −0.8 to −0.1). We found similar results in a mixed effects linear regression model (adjusted difference, −0.2; 95% CI, −0.3 to −0.1).

Discharge to a Facility

Overall, 14 390 (48.7%) patients were discharged to a facility from Magnet hospitals and 658 589 (52.1%) from non‐Magnet institutions (Table 2). Hospitalization in a Magnet hospital for acute ischemic stroke was associated with lower rate of discharge to a facility in comparison to non‐Magnet institutions (difference, −8.6%; 95% CI, −10.2% to −7.0%) in unadjusted analysis. Likewise, using a probit regression with instrumental variable analysis, we identified that Magnet hospitals were associated with a 16.5% lower rate of discharge to a facility (95% CI, −20.0% to −13.0%), in comparison to non‐Magnet institutions (Table 3). This persisted in a mixed effects probit regression model (adjusted difference, −10.1%; 95% CI, −11.7% to −8.4%) and a propensity score adjusted probit model (adjusted difference, −9.9%; 95% CI, −11.6% to −8.3%). This corresponded to 6 patients needing to be treated in a Magnet hospital to prevent 1 discharge to a facility.

Discussion

Using a comprehensive all‐payer cohort of patients in New York State with acute ischemic stroke, we identified an association of hospitalization in a Magnet hospital with lower case‐fatality, LOS, and discharge to rehabilitation.1, 2, 3, 4, 5, 6 These results were consistent across statistical techniques used to control for measured and unmeasured confounders. Although we can only speculate about the reasons behind the more profound effect observed with the instrumental variable analysis, we hypothesize that controlling for unmeasured confounders (ie, possibly sicker patients being preferentially hospitalized in magnet hospitals) results in a more clear association of Magnet hospitals with improved outcomes. The clinical significance of the observed differences should be assessed based on the individual patient and practice characteristics of every provider. In recent years, there is increasing emphasis on public reporting, center of excellence recognition, and patient engagement in treatment decisions. Magnet recognition is heavily advertised by hospitals.7 These facilities have been found to have lower rates of nursing burnout and improved overall financial performance.7, 24, 25, 26, 27, 28 However, the association of this recognition with improved outcomes in ischemic stroke has not been studied before.

Prior investigations have demonstrated conflicting results regarding the association of Magnet recognition and patient outcomes. The first study linking Magnet status to improved outcomes was published in 1994.29 However, it was based on the original 1983 cohort of Magnet hospitals recognized by reputation, and not the formal review process currently used.29 Most recently, Friese et al10 in a longitudinal analysis of elderly surgical Medicare patients demonstrated that hospitalization in a Magnet hospital was associated with lower 30‐day mortality and rates of failure‐to‐rescue. Similar results were demonstrated in a regional analysis in Pennsylvania.11 Other groups have identified superior outcomes for Magnet facilities in terms of falls,30 mortality after trauma,31 and outcomes of very low‐birth‐weight infants.32 On the contrary, several researchers were not able to demonstrate a benefit of Magnet status.12, 13, 14 The lack of adjustment for clustering, and rigorous control for measured and unmeasured confounders (especially the fact that patients were nonrandomly selected for either treatment center), significantly limit the interpretation of the results of these investigations. A meta‐analysis by Petit Dit Dariel and Regnaux reached the same conclusions.33

Our current study purposefully addresses many of these methodologic limitations. First, we created a cohort of all patients in a major state, giving a true picture of practice in the community. Second, we used advanced observational techniques to control for confounding. Propensity score stratification was used to adjust our analyses for known confounders. The possibility of clustering, which can bias the results of multicenter national studies, was accounted for by using mixed effects methods. Most importantly, an instrumental variable analysis was used to control for unmeasured confounders (mainly the a priori selection of treatment facility), and simulate the effects of randomization. The instrumental variable analysis is expected to control for such factors and report results for patients of similar functional status. Results were consistent across techniques, supporting the validity of the observed associations.

Further research into the factors contributing to the potential superiority of Magnet hospitals in stroke care is warranted. Previous work has demonstrated that Magnet hospitals have less organizational hierarchy and increased nursing autonomy, invest in quality benchmarking and reporting, and have higher nursing satisfaction.7, 24, 25, 26, 27, 28, 34, 35, 36 Such institutional commitment to quality improvement empowers nurses and physicians to deliver evidence‐based care, establish effective communication, and identify patient problems more quickly,34, 35, 36 which is particularly critical in the largely protocol‐driven stroke care. A culture that encourages nursing participation in the decision‐making additionally supports effective interdisciplinary care.34, 35, 36 From a policy perspective, it is important to recognize effective quality‐reporting initiatives, given the growing body of such reports, and the resulting hesitancy of the public to adopt them. Some of the most prominent such efforts such as Hospital Compare, and the ProPublica Surgeon Scorecard have been criticized for their accuracy.37, 38, 39

Our study has several limitations. Residual confounding could account for some of the observed associations. However, this is minimized to the extent that we are using a good instrument for treatment facility. The F statistic in our analysis suggests a strong instrument. In addition, coding inaccuracies will undoubtedly occur and can affect our estimates. However, in several reports the coding for stroke has been shown to have a near perfect association with medical record review.40, 41 Although SPARCS includes all hospitals from the entire New York State, the generalization of this analysis to the entire US population is uncertain. SPARCS does not provide any clinical information on the treatment times, or the functional status of the patients (National Institutes of Health Stroke Scale), which can affect the choice of institution. However, the use of the instrumental variable analysis is attempting to control for unknown confounders such as these, and has been used before in patients with stroke from this database.19, 20, 21

Additionally, we were lacking posthospitalization and long‐term data, or timing of inpatient mortality on our patients. Quality metrics (ie, modified Rankin score) are also not available through SPARCS, and therefore we cannot compare the 2 hospital settings on these outcomes. The definitive comparison of the 2 hospital settings on functional outcomes can only be done in prospective registries. In this direction, the NeuroPoint Alliance has created the first module for a cerebrovascular registry, with results expected in the near future.42 Finally, causality cannot be definitively established based on observational data, despite the use of advanced techniques, such as the instrumental variable analysis.

Conclusions

It is not clear whether Magnet recognition by the American Nurses Credentialing Center is associated with improved patient outcomes. We investigated the association of Magnet recognition with case‐fatality, LOS, and discharge to a facility for ischemic patients with stroke. Using a comprehensive all‐payer cohort of patients with ischemic stroke in New York State, we identified that an association of treatment in Magnet hospitals was associated with lower case‐fatality, discharge to a facility, and LOS. Further research into the factors contributing to the superiority of Magnet hospitals in stroke care is warranted.

Sources of Funding

This work was supported by grants from the National Institute on Aging (PO1‐ AG19783), the National Institutes of Health Common Fund (U01‐AG046830), and the National Center for Advancing Translational Sciences (NCATS) of the NIH (Dartmouth Clinical and Translational Science Institute—UL1TR001086). The funders had no role in the design or execution of the study.

Disclosures

None.

Table S1. Coding Definitions

Click here for additional data file.