Post-Acute Care Data for Predicting Readmission After Ischemic Stroke: A Nationwide Cohort Analysis Using the Minimum Data Set.

BACKGROUND: Reducing hospital readmissions is a key component of reforms for stroke care. Current readmission prediction models lack accuracy and are limited by data being from only acute hospitalizations. We hypothesized that patient-level factors from a nationwide post-acute care database would improve prediction modeling. METHODS AND
RESULTS: Medicare inpatient claims for the year 2008 that used International Classification of Diseases, Ninth Revision codes were used to identify ischemic stroke patients older than age 65. Unique individuals were linked to comprehensive post-acute care assessments through use of the Minimum Data Set (MDS). Logistic regression was used to construct risk-adjusted readmission models. Covariates were derived from MDS variables. Among 39 178 patients directly admitted to nursing homes after hospitalization due to acute stroke, there were 29 338 (75%) with complete MDS assessments. Crude rates of readmission and death at 30 days were 8448 (21%) and 2791 (7%), respectively. Risk-adjusted models identified multiple independent predictors of all-cause 30-day readmission. Model performance of the readmission model using MDS data had a c-statistic of 0.65 (95% CI 0.64 to 0.66). Higher levels of social engagement, a marker of nursing home quality, were associated with progressively lower odds of readmission (odds ratio 0.71, 95% CI 0.55 to 0.92).
CONCLUSIONS: Individual clinical characteristics from the post-acute care setting resulted in only modest improvement in the c-statistic relative to previous models that used only Medicare Part A data. Individual-level characteristics do not sufficiently account for the risk of acute hospital readmission.

Stroke is a leading cause of disability and the fifth leading cause of death in the United States, with an estimated $54 billion annually in direct costs.1,2 Stroke-associated morbidity results in a persistent risk for hospital readmission. Previous estimates reveal as many as 21% of stroke patients are readmitted within 30 days, and >55% are readmitted by 1 year.3,4 Reducing preventable hospital readmissions is a key initiative for the Centers for Medicare and Medicaid Services (CMS) under guidance of the Affordable Care Act.5–7

Several studies have described the characteristics of acute care hospitals and stroke patients to establish risk factors for readmission.3,4,8–16 However, a systematic review did not reveal any publications meeting American Heart Association criteria for reporting health care provider outcome data.17,18 The model developed under contract by CMS has limited predictive accuracy (c-statistic 0.60) and relies solely on Medicare Part A claims. Others have expressed concerns that the current CMS model does not adequately adjust the risk of readmission for stroke severity.7 A solution to these limitations may exist in the post–acute care setting. Large stores of patient-level data have not been applied to predicting readmission after ischemic stroke.

The Minimum Data Set (MDS) is a comprehensive clinical assessment of all nursing home residents in the United States, which enables the measurement of patient-level outcomes among stroke patients. Ischemic stroke often leads to long-term disability and need for placement in nursing homes, making the MDS a useful tool for tracking stroke-related morbidity.1,19,20 At acute hospital discharge, one-third of stroke patients require long-term care placement.21 Among more severe ischemic stroke patients, nearly 70% require admission to a nursing home.22 As a result, the MDS represents a previously untapped resource of nationwide patient outcome data applicable to a large proportion of stroke patients. Key MDS domains of relevance to stroke patients, such as detection of cognitive impairment, have a high degree of predictive ability (c-statistic 0.93).23 Functional measures lacking from Medicare Part A data, such as activities of daily living, are plentiful in the MDS and have high levels of internal consistency over time.24 We hypothesized that post–acute care patient-level variables from the MDS will improve prediction model performance of acute hospital readmission among ischemic stroke patients discharged to nursing homes.

Methods

Selection of Study Subjects

The process used for selecting the analytical sample is presented in Figure 1. Patients qualified for inclusion in the cohort based on age >65 years and discharge diagnosis from an acute care hospital after an ischemic stroke between January 1, 2008, and December 31, 2008. Fee-for-Service Medicare Part A Inpatient Standard Analytical Files were used to identify cases as defined by International Classification of Diseases, Ninth Revision (ICD-9) principal diagnosis codes for Acute Ischemic Stroke (433.x and 434.x). The ICD-9 codes chosen for case identification are identical to those used in previous publications under contract with CMS.25,26 Unique individuals were identified by matching Medicare health insurance claim number, Social Security number, sex, and date of birth. New-onset ischemic stroke cases were selected by excluding Medicare Part A claims from January 2007 to December 2007 containing the same primary or secondary ICD-9 codes as just given. Given the 30-day time dependence of the primary outcome of readmission after acute hospital discharge, only direct admissions to CMS certified (>95% of all US nursing homes) nursing homes were chosen. Medicare Part A claims were then linked to MDS records within 1 year of the index stroke admission date. The MDS assessment is performed at the time of nursing home admission by a trained clinician and at regular intervals thereafter. The assessment covers 13 domains including cognition, behavior, motor, and global physical functioning with a high degree of reliability and validity.24,27–30 Only complete MDS assessments performed within 14 days of acute hospital discharge were considered for linkage with Medicare Part A claims files. This resulted in a mean time between acute hospital discharge and MDS assessment of 5 days (SD ±3.2) for this analysis.

Figure 1

Flow diagram of study subject selection. *Part A claims for ICD-9 codes 433.x, 434.x. †MDS indicates Minimum Data Set; no MDS case, no admission to nursing home. ‡Nursing home admission not temporally associated with the index stroke admission.

Outcomes

“Acute hospital readmissions” were operationally defined as any inpatient stay generating a Medicare Part A claim after the index stroke hospitalization. Planned readmissions for further diagnostic or procedural care were excluded by predetermined ICD codes from the primary outcome analysis (Data S1). Hospitalizations billed as observation-level and emergency department visits were not considered readmissions in this analysis. Readmission within 30 days was the primary time interval of analysis because it is a hospital quality measure now mandated by CMS.25 Crude readmission rates at time intervals of 90 and 365 days were also calculated for comparison. To address the competing risk of mortality and readmission, rates of mortality were calculated separately. If a patient died before readmission, this was considered a non-readmission event and was not included in the readmission prediction model. This approach was chosen to allow for comparisons with previously published readmission models.25

Variable Selection and Statistical Analysis

Demographic characteristics of the study sample were obtained through linkage of inpatient claims with Medicare enrollment files and MDS files. Ascertainment of deaths and dates of death was made with the use of CMS vital status variables located on the Medicare enrollment file. Medicare Part A claims data were used to obtain acute hospital length of stay and the Elixhauser Comorbidity Measure.31 All other variables considered in the development of prediction models for readmission at 30 days were derived from the MDS. Screening of variables available in the MDS occurred on the basis of clinical relevance as determined by a panel of cerebrovascular experts and prior publications with the use of Medicare part A claims and MDS data.4,17,25,32–35 Variables considered to be of particular clinical relevance but not found to be statistically associated were included in the final model. Composite variables derived from MDS measures included the activities of daily living index, cognitive performance scale, pressure ulcer stage, social engagement, and the Changes in Health, End-stage disease, Symptoms and Signs (CHESS) comorbidity index.36–38 The cognitive performance scale is a 7-category hierarchical scale, which correlates closely with the Mini-Mental State Exam, as well as diagnoses of dementia.37 The CHESS score is a 6-point ordinal scale ranging between 0 (no instability) to 5 and is an indicator of mortality and clinical stability among nursing home residents.38

Crude readmission rates were calculated at 30, 90, and 365 days. Ninety-day and 1-year readmission rates are cumulative. Frequency distributions of missing data were examined for each variable included in the final model. The most common reason for missing observations (3.4%) related to different forms of the MDS assessment administered (Figure 1). A complete case analysis approach was used. For the demographic variables of race and education level, >10% missing values from the initial assessmen were populated with values from the next complete MDS assessment. This method was not used for time-dependent clinical variables such as pneumonia, seizure disorder, and others. Logistic regression was used to construct risk-adjusted models of readmission and death at 30 days post acute care hospitalization. Model fitness was tested via residual plots and pseudo R2. Regression model performance was assessed by model χ2 and calculation of the c-statistic for 30-day readmission and mortality models. Comparisons of c-statistics confidence intervals were performed by using nonparametric methods described by DeLong et al39 To assess the potential for facility-level contributions to risk of readmission, a fixed-effect sensitivity analysis was performed by using variables from Medicare’s Online Survey, Certification and Reporting (OSCAR) system. Covariates included in the model were nursing home bed capacity, ownership, resident payer status, weighted health inspection deficiency score, and physician, specialist, and nurse staffing ratios.40,41 Given the possibility of a readmission event occurring before MDS assessment, a sensitivity analysis was performed using a second model to exclude subjects readmitted before 6 days. The study was approved by our institutional review board, and all files accessed for the analysis are covered by a Data Use Agreement with CMS. All analyses were performed within the Brown University Center for Gerontology and Healthcare Research using SAS version 9.3 and Stata version 13.

Results

Subject Characteristics

Among 252 569 Medicare fee-for-service new-onset ischemic stroke admissions aged ≥65 years in 2008, there were 87 094 (35.5%) individuals admitted to nursing homes (Table S1). Among 39 178 patients who were admitted directly to nursing homes after acute stroke hospitalization, there were 29 338 (75%) patients with complete MDS assessments performed at a mean of 5 days (SD ±3.2) from discharge. Individuals with partial MDS assessments not included in the analytical sample had a mean age of 80 years (SD ±7.9), were mostly female (56%), and had fewer medical comorbidities. Table 1 provides a complete description of the study sample followed by crude odds of readmission and death at 30 days. The average patient age was 83 years old, 65% were female, and 85% were white. Stroke-related disabilities included hemiparesis among 35%, aphasia in 16%, and 12% with dysphagia requiring a feeding tube. Depression was diagnosed among 22% of patients. More than 37% of patients had an active do not resuscitate order.

Table 1

Study Sample Description and Crude Odds of Readmission and Death at 30 Days

Characteristics	MDS Cases (N=39 178), % (No.)	Crude Odds of Readmission at 30 Days	95% CI	Crude Odds of Death at 30 Days	95% CI
Age, mean (SD)	83 (7.7)	0.99	0.98 to 0.99	1.06	1.06 to 1.07
Age group to y
65 to 74	17 (6632)	Ref.		Ref.
75 to 84	39 (15 388)	0.94	0.88 to 1.00	1.57	1.35 to 1.81
85 to 94	40 (15 528)	0.79	0.73 to 0.84	2.70	2.35 to 3.10
≥95	4 (1630)	0.64	0.56 to 0.74	4.99	4.08 to 5.92
Sex (male)	35 (13 681)	1.17	1.11 to 1.22	1.01	0.94 to 1.10
Race
White	85 (32 859)	Ref.		Ref.
Black	10 (3865)	1.53	1.42 to 1.65	0.38	0.32 to 0.46
Asian	1.57 (608)	1.48	1.23 to 1.77	0.67	0.47 to 0.95
Hispanic	3.2 (1253)	1.28	1.11 to 1.46	0.64	0.49 to 0.82
Native American	0.3 (129)	0.99	0.65 to 1.53	0.69	0.32 to 1.47
Education level
No schooling	0.76 (296)	0.95	0.71 to 1.27	1.15	0.70 to 1.88
Grade 8 or less	13 (4965)	1.07	0.99 to 1.16	0.94	0.81 to 1.09
Grades 9 to 11	10 (3998)	1.03	0.95 to 1.13	0.96	0.82 to 1.13
High school	40 (15 825)	Ref.		Ref.
Technical	4 (1476)	0.90	0.78 to 1.03	0.91	0.71 to 1.17
Some college	10 (4055)	0.91	0.83 to 1.00	0.85	0.72 to 1.00
Bachelor’s degree	6 (2406)	0.93	0.83 to 1.04	0.90	0.73 to 1.10
Graduate degree	3 (1264)	0.89	0.77 to 1.04	0.81	0.61 to 1.07
Acute hospital LOS, median (IQR)	6 (5 to 9)	1.04	1.04 to 1.05	0.98	0.98 to 0.99
Elixhauser score, mean (SD)	3 (1.3)	1.05	1.04 to 1.08	1.16	1.13 to 1.20
Heart disease, n (%)	15 (5877)	1.08	1.00 to 1.15	1.00	0.88 to 1.13
Dysrhythmia, n (%)	27 (10 578)	1.06	0.99 to 1.15	1.36	1.19 to 1.56
CHF, n (%)	17 (6660)	1.26	1.19 to 1.34	1.42	1.29 to 1.57
Hypertension, n (%)	82 (32 125)	1.00	0.93 to 1.08	0.74	0.65 to 0.85
Comatose, n (%)	0.67 (257)	0.43	0.29 to 0.64	58.2	43.3 to 78.2
Aphasia, n (%)	16 (6175)	1.17	1.10 to 1.25	1.78	1.61 to 1.96
Hemiparesis, n (%)	35 (13 452)	1.22	1.16 to 1.28	1.62	1.49 to 1.76
Paraplegia, n (%)	0.16 (60)	1.22	0.68 to 2.19	2.39	1.13 to 5.03
Quadriplegia, n (%)	0.05 (18)	1.04	0.34 to 3.17	3.09	0.90 to 10.7
Feeding tube, n (%)	12.1 (4658)	2.25	2.11 to 2.40	1.58	1.41 to 1.77
Tracheostomy, n (%)	0.44 (170)	3.11	2.30 to 4.22	0.87	0.44 to 1.70
Dementia, n (%)	21 (8063)	0.84	0.79 to 0.90	0.74	0.67 to 0.83
Depression, n (%)	22 (8448)	0.88	0.83 to 0.94	0.61	0.54 to 0.69
Anxiety disorder, n (%)	9 (3571)	0.84	0.77 to 0.92	0.53	0.45 to 0.63
Seizure disorder, n (%)	5 (1727)	1.16	1.02 to 1.30	0.78	0.58 to 1.03
Current tobacco, n (%)	5.5 (1883)	1.01	0.90 to 1.14	0.71	0.55 to 0.91
COPD, n (%)	14 (5269)	1.26	1.18 to 1.35	0.90	0.79 to 1.02
Cancer, n (%)	8 (2501)	1.05	0.95 to 1.17	1.88	1.58 to 2.24
Renal disease, n (%)	6 (2512)	1.29	1.18 to 1.42	0.92	0.78 to 1.09
C. difficile, n (%)*	0.9 (325)	2.04	1.63 to 2.57	1.02	0.65 to 1.60
Pneumonia, n (%)*	7 (2674)	1.46	1.34 to 1.60	1.98	1.74 to 2.26
Active DNR order, n (%)	37 (14 216)	0.64	0.60 to 0.67	6.15	5.58 to 6.79
Active DNH order, n (%)	2 (787)	0.31	0.25 to 0.41	13.7	11.8 to 15.9
Hospice care, n (%)	0.8 (322)	0.20	0.12 to 0.33	24.2	19.3 to 30.3
Non–English first language, n (%)	4.5 (1.586)	1.29	1.15 to 1.46	0.50	0.37 to 0.68

CHF indicates congestive heart failure; COPD, chronic obstructive pulmonary disease; DNH, do not hospitalize; DNR, do not resuscitate; LOS, length of stay; MDS, Minimum Data Set.

Infections are documented if they occurred within 7 days of the assessment.

Crude and Adjusted Mortality and Readmission Estimates

Crude rates of mortality and readmission at 30, 90, and 365 days are provided in Figure 2. At 30 days post acute hospital discharge, there were 8448 (21%) readmissions and 2791 (7%) deaths. Results from multiple logistic regression models predicting the odds of readmission and mortality at 30 days are displayed in Table 2. Notably, variables relating to stroke disability were not statistically significant predictors of readmission. The presence of hemiparesis (34%) or aphasia (16%) was not a significant predictor of 30-day readmission. The need for a feeding tube was associated with a 21% increased odds of readmission (odds ratio [OR] 1.21, CI 1.08 to 1.35). Bowel incontinence increased odds of readmission by 16% (OR 1.16, CI 1.06 to 1.28). Medical comorbidities associated with increased odds of readmission included congestive heart failure (OR 1.17, CI 1.08 to 1.27), chronic obstructive pulmonary disease (OR 1.26, CI 1.16 to 1.38), renal disease (OR 1.26, CI 1.13 to 1.42), and stage IV pressure ulcers (OR 1.33, CI 1.09 to 1.62). Advanced directives and goals of care all had the largest effect size in reducing odds of readmission. Active do not resuscitate orders were associated with a 25% reduction in odds of readmission (OR 0.75, CI 0.696 to 0.800). There was a low prevalence of hospice care and do not hospitalize orders (0.8% and 2% respectively); however, variables were associated with large reductions in odds of readmission (OR 0.357, CI 0.182 to 0.701; and OR 0.526, CI 0.378 to 0.731, respectively).

Figure 2

Crude cumulative mortality and readmission rate at 30, 90, and 365 days (N=39 178). The same sample was used for mortality and readmission estimates.

Table 2

Risk-Adjusted Odds of Readmission and Death at 30 Days

Characteristic	Odds of Readmission at 30 Days	95% CI	Odds of Death at 30 Days	95% CI
Sex (male)	1.09	1.02 to 1.18	1.20	1.00 to 1.44
Married	1.05	0.98 to 1.01	0.96	0.80 to 1.15
Active DNR order	0.75	0.70 to 0.80	2.46	2.06 to 2.93
Active DNH order	0.53	0.38 to 0.73	2.58	1.89 to 3.53
Hospice care	0.36	0.18 to 0.70	1.78	1.12 to 2.82
Acute hospital LOS	1.01	1.00 to 1.02	0.90	0.88 to 0.92
Feeding tube	1.21	1.09 to 1.35	0.69	0.54 to 0.89
Bowel incontinence	1.16	1.06 to 1.28	1.44	1.16 to 1.80
Bladder incontinence	0.92	0.85 to 1.00	1.09	0.90 to 1.34
Urinary tract infection	1.10	1.02 to 1.19	0.92	0.77 to 1.11
Dementia	0.92	0.85 to 0.99	0.90	0.75 to 1.07
Alzheimer dementia	0.87	0.76 to 0.99	0.90	0.69 to 1.18
CHF	1.17	1.08 to 1.27	1.30	1.08 to 1.57
COPD	1.26	1.16 to 1.38	0.83	0.65 to 1.06
Depression	0.90	0.83 to 0.98	0.789	0.647 to 0.961
Diabetes	1.09	1.01 to 1.16	1.04	0.87 to 1.25
Renal disease	1.26	1.13 to 1.42	1.52	1.17 to 1.55
C. difficile infection*	1.32	0.97 to 1.77	0.80	0.32 to 2.03
Elixhauser score	1.03	1.00 to 1.06	1.02	0.96 to 1.08
Urinary catheter	1.13	1.04 to 1.22	1.33	1.09 to 1.61
Hypotension	1.36	1.04 to 1.78	1.37	0.72 to 2.64
Stage 4 pressure ulcer	1.33	1.09 to 1.62	0.93	0.58 to 1.50
CHESS score
0 (Low intensity)	Ref.
1	0.54	0.50 to 0.60	0.91	0.70 to 1.19
2	0.56	0.52 to 0.61	1.08	0.86 to 1.38
3	0.57	0.52 to 0.63	1.53	1.21 to 1.94
4	0.59	0.50 to 0.68	2.12	1.61 to 2.79
5 (High intensity)	0.23	0.05 to 1.04	3.22	1.45 to 7.18
Social engagement score
0 (Not engaged)	Ref.
1	0.85	0.77 to 0.95	0.65	0.53 to 0.79
2	0.81	0.73 to 0.91	0.45	0.35 to 0.57
3	0.77	0.68 to 0.86	0.36	0.27 to 0.48
4	0.68	0.59 to 0.77	0.37	0.26 to 0.53
5	0.67	0.55 to 0.82	0.32	0.15 to 0.66
6 (Very engaged)	0.71	0.55 to 0.92	0.37	0.15 to 0.92

Covariates included were age, race, sex, education level, non-native English speaker, active tobacco use, do not resuscitate order, do not hospitalize order, hospice care, guardianship status, acute hospital length of stay, bowel incontinence, bladder incontinence, indwelling urinary catheter, body mass index, marital status, type 2 diabetes mellitus, congestive heart failure, hypotension, hypertension, arthritis, hip fracture, Alzheimer dementia, dementia, aphasia, hemiparesis, paraplegia, seizure, anxiety disorder, depression, bipolar disorder, schizophrenia, chronic obstructive pulmonary disease, retinopathy,anemia,cancer, chronic renal disease, recent infection, sepsis, C. difficile infection,pneumonia, urinary tract infection, gastric feeding tube, tracheostomy, activities of daily living index, cognitive performance scale,37 communication scale, pressure ulcer scale, CHESS comorbidity index,38 Elixhauser score,31 social engagement scale36 (italicized variables not significant in univariate analysis). CHESS indicates changes in health, end-stage disease, symptoms, and signs score; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; DNH, do not hospitalize; DNR, do not resuscitate; LOS, length of stay.

Infections are documented if they occurred within 7 days of the assessment.

Composite variables examined included the Elixhauser Comorbidity Measure, CHESS comorbidity index, cognitive performance scale, and social engagement score.34,36,40 The Elixhauser Comorbidity Measure, composed of ICD-9 codes from Medicare Part A inpatient claims, was not a strong predictor of 30-day readmission (OR 1.03, CI 1.00 to 1.06) or death (OR 1.02, CI 0.956, 1.08). Among MDS-derived composite variables examined, lower scores on the CHESS comorbidity index were associated with decreased odds of readmission. However, higher scores on the CHESS index were reliably associated with increasing odds of death at 30 days. The cognitive performance scale was not a significant predictor of readmission. Higher levels of social engagement, a marker of nursing home quality, were associated with progressively lower odds of readmission (OR 0.71, CI 0.55 to 0.92).36

Predictive Model Performance

The c-statistic for readmission at 30 days was 0.65 (CI 0.640 to 0.657) (Figure 3). Variability explained by the readmission model as measured by coefficient of multiple determination (pseudo R2) was 0.042. Although unable to be directly compared given differences between cohorts, the standard model that used data restricted to Medicare Part A has a c-statistic of 0.60.41 With use of the MDS-derived model with death as the outcome, c-statistic and pseudo R2 were 0.88 (CI 0.871 to 0.892) and 0.281, respectively. The MDS model performance was similar at predicting readmission at 90- and 365-day time points, with c-statistic of 0.63 and 0.61, respectively.

Figure 3

Receiver operator characteristic (ROC) curves for mortality and readmission at 30 days (N=39 178). AUC indicates area under the curve (c-statistic); 95% confidence interval for mortality ROC (0.87 to 0.89), readmission ROC (0.64 to 0.66).

Sensitivity analyses were performed to quantify the effect of early readmission and facility level characteristics on model performance. With use of the same set of predictor variables, the exclusion of 4029 subjects readmitted within 6 days of nursing home admission did not alter the c-statistic (0.65). The second sensitivity analysis, using a facility fixed-effect model, resulted in a minimal increase in c-statistic (0.66).

Discussion

Among 39 178 new-onset ischemic stroke patients admitted to nursing homes, 21% were readmitted to acute care hospitals within 30 days. At 1 year, 55% of the original cohort had a readmission event. Using 29 338 subjects with complete MDS records, we identified multiple patient-level independent predictors of readmission at 30 days. Despite the addition of a wealth of patient characteristics, including stroke-specific functional markers, we achieved only a modest increase in c-statistic (0.65) compared with models using only Medicare Part A data (c-statistic 0.60).25 Higher levels of social engagement, a composite marker of nursing home quality, independently reduced odds of readmission. The use of post–acute care data for prediction modeling has not been performed previously, and the best models currently available have poor reliability. These results have critical implications in the context of looming penalties against hospitals for stroke-related readmissions.7 We conclude that individual patient characteristics within the MDS do not account for all determinants of readmission after stroke. Future efforts may target improving stroke severity adjustment and post–acute care quality metrics for improving the accuracy of stroke readmission prediction modeling.

Few Stroke-Specific Predictors of Readmission Using the MDS

We calculated similar readmission rates compared with earlier studies restricted to Medicare Part A inpatient claims.3,4,8–16,21,22 By using the MDS, we identified the presence of a feeding tube, bowel incontinence, congestive heart failure, renal disease, and chronic obstructive pulmonary disease to be associated with increased odds of readmission. Increasing severity on the National Institutes of Health Stroke Score (NIHSS) was the strongest indicator for hospital readmission in an analysis of American Heart Association’s Get With The Guidelines–Stroke database.4 We did not detect an association with readmission from nursing homes for stroke characteristics such as aphasia or hemiparesis. Further, measures of functional status derived from the MDS such as activities of daily living and cognitive performance indexes were not reliable predictors of readmission. The CHESS comorbidity index, a measure of a patient’s clinical instability, was associated with mortality but paradoxically also associated with lower rates of readmission. Trends were consistent at 90 and 365 days. MDS composite measures were only rarely derived from other variables used in the regression model. An analysis using only composite variables resulted in the same lack of association with readmission. The observed associations likely relate to characteristics specific to the substantial proportion of stroke survivors who require nursing home care and may be influenced by residual measured and unmeasured confounding.

Reliability of Mortality Prediction Better Than Readmission

The MDS-derived model is a more robust predictor of mortality (c-statistic 0.87) than of readmission (c-statistic 0.65) (Figure 3). Mortality, especially among this cohort of nursing home patients, is more directly associated with a patient’s mix of comorbidities. Although not the focus of this analysis, the mortality model could be applied as a clinical decision support tool for families and nursing home providers. On the contrary, it appears that individual patient characteristics do not account for all of the key determinants of readmission. Facility-level and socioeconomic factors may be among the most important contributors to risk of readmission. Level of social engagement was a significant predictor of readmission, which relates to many factors specific to nursing home quality.35 Others have established the importance of socioeconomic and facility-level characteristics in readmission and mortality prediction models.11,14 Payer status was found to be a key determinant of readmission in a comparison of health maintenance organization enrollees and Medicare recipients.11 Medicare-designated critical access hospitals treat greater proportions of older, medically complex uninsured patients and are associated with higher risk standardized mortality rates, but not higher readmission rates, compared with traditional Medicare-reimbursed inpatient facilities.14 We performed a fixed-effect sensitivity analysis of nursing facility-level characteristics without significant change in c-statistic. Multilevel modeling of post–acute care facility data in combination with stroke-specific disability measures has yet to be performed.

Although the MDS-derived model did not significantly improve accuracy of prediction, multiple independent predictors were identified and should be considered in future models. The medical comorbidities of congestive heart failure, chronic obstructive pulmonary disease, renal disease, and pressure ulcers each reliably increased odds of readmission. Notably, advanced directives, such as do not resuscitate orders, were associated with the largest reduction of odds of readmission and very narrow CIs (OR 0.75, CI 0.696 to 0.800). Beyond risk modeling, these are actionable targets for readmission reduction programs.

Limitations to this analysis include the restriction of study subjects to age >65 years. Medicare beneficiaries aged <65 years meet the criteria for a high degree of permanent disability and differ significantly from the majority of the Medicare population. Patient location before index stroke admission was not included in the analysis, and as a result patients residing in nursing homes before stroke admission were not stratified in the analytical sample. Medicare managed care beneficiaries were excluded. These patients still generate MDS assessments when admitted to nursing homes, introducing a source of potential selection bias to this analysis. Stroke-specific measures of stroke severity and disability were not measured. A complete case analysis approach was used, which can bias estimates when missing data are nonrandom. Partial MDS records excluded from the analysis had similar demographic characteristics as the analytical sample. Despite multiple validated measures of individual functional status contained within the MDS, validated stroke-specific measures such as the NIHSS or modified Rankin Scale were not available. The NIHSS is projected for inclusion in ICD-10-CM, which may greatly enhance future prediction models.42 Although similar methods were used to derive the patient cohort used in the MDS model, a direct comparison to the Medicare Part A–only prediction model could not be made given the use of condition categories as predictors in the Part A–only model.

Temporal trends in the version of MDS assessment performed presented a key challenge. The MDS assessment varies in length depending on the time point of administration. We utilized the first full MDS assessment within 14 days of nursing home admission to examine all possible variables. Mean time from acute care hospital discharge to first full MDS assessment was 5 days. This time delay between discharge and full MDS assessment may alter time-sensitive predictors of readmission. To assess the impact of subjects with an early readmission event on model performance, a separate sensitivity analysis was performed. With the exclusion of 4029 (35% among those readmitted within 30 days) subjects readmitted within 6 days, there was no significant change in the predictive capacity of our model.

Conclusions

Results from this study suggest the need for a broader scope to readmission prediction models. Impending health care reform measures will levy financial penalties on acute care hospitals for stroke readmission events.7 Patient-specific severity adjustments that use the MDS do not account for all key determinants of readmission among stroke patients. The readmission model’s lack of reliability limits its clinical utility and warrants a cautionary approach before implementing its use in CMS payment reforms. Validated stroke severity measures and nursing home quality of care metrics may be important variables to be considered in future models of readmission after ischemic stroke.