Performance Of Safety-Net Hospitals In Year 1 Of The Comprehensive Care For Joint Replacement Model.

The Comprehensive Care for Joint Replacement (CJR) model introduced in 2016 aims to improve the quality and costs of care for Medicare beneficiaries undergoing hip and knee replacements. However, there are concerns that the safety-net hospitals that care for the greatest number of vulnerable patients may perform poorly in CJR. In this study we used Medicare's CJR data to evaluate the performance of 792 hospitals mandated to participate in the first year of CJR. We found that in comparison to non-safety-net hospitals, 42 percent fewer safety-net hospitals qualified for rewards based on their quality and spending performance (33 percent of safety-net hospitals qualified, compared to 57 percent of non-safety-net hospitals), and safety-net hospitals' rewards per episode were 39 percent smaller ($456 compared to $743). Continuation of this performance trend could place safety-net hospitals at increased risk of penalties in future years. Medicare and hospital strategies such as those that reward high-quality care for vulnerable patients could enable safety-net hospitals to compete effectively in CJR.

Introduction:

Hip and knee replacements are the most commonly performed inpatient surgeries for older Medicare beneficiaries and the largest surgical expenditure for Medicare.[1] In 2014, there was more than a threefold variation in hospital complication rates following hip and knee replacements, and spending for these surgeries ranged from $16,500 to $33,000 across geographic regions.[2] To reduce this variation, the Centers for Medicare and Medicaid Services (CMS) introduced the Comprehensive Care for Joint Replacement (CJR) model in April 2016. CJR is a five-year bundled payment reform that holds hospitals accountable for quality and spending during the inpatient stay and ninety days of postacute care for Medicare beneficiaries undergoing hip and knee replacements. In 2016, 794 acute care hospitals from sixty-seven Metropolitan Statistical Areas (MSAs) were mandated to participate in CJR.[2] With recent updates to hospital participation requirements, 491 hospitals will continue to participate in the CJR until 2020.[3]

To determine rewards and penalties, CJR calculates a composite quality score that is capped at twenty points and is a weighted average of a hospital’s performance on complications (50 percent), patient satisfaction (40 percent), and successful submission of patient-reported outcome data (10 percent).[4] Although this score is primarily based on a hospital’s performance relative to that of other hospitals (for example, hospitals at the ninetieth percentile receive the maximum number of quality points), hospitals can increase their scores by an additional two points if their quality performance improves significantly compared to that in the previous year. This scoring methodology remains constant through the five years of CJR. Hospitals are also assigned a prospective target price at the start of each performance year. For the first and second years, the price was two-thirds of a hospital’s historic spending for hip and knee replacements and one-third of the regional spending. The weight of the hospital’s spending gradually decreases in the third and fourth years, and the target price in the fifth year is determined solely by regional spending. Hospitals with expenditures below the target price, adjusted by the composite quality score, can earn back the difference as a reward payment. Beginning in the second year, hospitals that exceed the quality-adjusted target price are expected to pay the difference back to CMS as a penalty.

While different target prices are set based on Medical Severity Diagnosis-Related Groups, CMS does not adjust a hospital’s target price for other health risks or social risk. Because of this, safety-net hospitals that care for disproportionate numbers of vulnerable patients (patients with high medical and social needs) are likely to fare poorly under CJR.[5-7] A study using data from the pre-CJR period found that among Medicare beneficiaries in Michigan who underwent hip and knee replacements in the period 2011–13, an increase in the medical complexity of patients was associated with lower rewards per episode for hospitals.[6] In our study, we used data from the first year of CJR to examine the performance of safety-net hospitals, with the goal of identifying quality and payment disparities that could inform future policy evaluations.

Study Data And Methods

Data Sources And Study Population

We used the first year of CJR (April–December 2016) performance results published online[8] and 2017 payment results published on the CMS.gov website.[2] We linked the information in these sources with the 2016 Medicare Impact File[9] to determine a hospital’s safety-net status and with data from the American Hospital Association Annual Survey[10] to obtain hospital characteristics. While 794 hospitals were mandated to participate in the first year of CJR, safety-net status data were available for only 792 hospitals. Hence the final analytic cohort for the study included 792 CJR hospitals located in sixty-seven MSAs.

Outcomes

The outcomes of interest were CJR-specified metrics for postoperative complications, patient satisfaction, submission of patient-reported outcomes data, overall quality performance, receipt of rewards in the first year of CJR, and the reward amount per episode (inpatient stay and 90-day post-acute care).[4] Data for these hospital-level outcomes were obtained from the first-year CJR performance[8] and payment[2] results published by CMS.

Postoperative complications were defined using risk-standardized complication rates that capture complications in the ninety-day period following surgery.[11] Patient satisfaction was defined using the Hospital Consumer Assessment of Healthcare Providers and Systems’ linear mean roll-up score, which reflects provider communication, staff responsiveness, discharge information, care transitions, and patient satisfaction.[12] Quality performance was categorized based on the composite quality score as excellent (a score of >15), good (≥6.9–≤15), acceptable (≥5–<6.9), or below acceptable (<5). Hospitals with performance that is acceptable or better earn rewards through all years of CJR. Hospitals with performance that is below acceptable need to repay CMS in the second through the fifth years, contingent on their spending performance. Complication rates were measured for surgeries in the period 2013–16, patient satisfaction scores in 2015–16, and submission of patient-reported outcomes in July–August 2016.

Key Explanatory Variable

The key explanatory variable was hospital safety-net status. We used the disproportionate patient percentage from the 2016 Medicare Impact File as our measure of hospital safety-net status.[13,14] This percentage reflects a hospital’s degree of dependence on disproportionate share hospital payments, which are given to hospitals to partially offset the cost of caring for clinically complex and socially disadvantaged patients. A hospital’s disproportionate patient percentage is calculated as the sum of two parts: the inpatient days for patients entitled to both Medicare Part A and Supplemental Security Income divided by the total Medicare days, and the inpatient days for patients eligible for Medicaid but not for Medicare divided by the total inpatient days.[15]

For the main analysis, we categorized hospitals into quintiles based on the disproportionate patient percentages for all CJR hospitals.[16] Quintile 1 included hospitals with the lowest percentage (non-safety-net hospitals), quintiles 2–4 included hospitals with increasing percentages, and quintile 5 hospitals were classified as safety-net hospitals.

Covariates

We controlled for hospital-level characteristics that were likely to confound the association between safety-net status and CJR-specified outcomes or that could partially explain the variation in these outcomes. These included hospital ownership, medical school affiliation, geographic region, number of beds, and volume of hip and knee replacements. The empirical specifications for the outcomes, key independent variable, and covariates are provided in online appendix exhibit A.1.[17]

Statistical Analysis

We used chi-square, Kruskal Wallis, and equality-of-medians tests to study the unadjusted variation in hospital characteristics and outcomes across quintiles of disproportionate patient percentages. To examine the adjusted variation in outcomes across these quintiles, we estimated hierarchical linear regression models for continuous outcomes such as complication rates and patient satisfaction scores. We estimated hierarchical logistic regression models for categorical outcomes such as submission of patient-reported outcomes, quality performance, and receipt of financial rewards. We used two-part Poisson models with log link functions for the amount of financial rewards per episode.[18,19] Each multivariate model included quintiles of disproportionate patient percentages as the key explanatory variable, and each model controlled for hospital-level confounders discussed above and MSA-level random effects to account for the clustering of hospitals within an MSA.

Sensitivity Analysis

We conducted additional analyses to check the robustness of the results. First, since a hospital’s performance in CJR is closely aligned with its performance in previous years, we accounted for baseline quality by constructing difference-in-differences[20] models that estimated the change in quality for safety-net hospitals compared to their baseline levels and relative to changes at other hospitals in CJR. To obtain information on the pre-CJR performance, we used hospital-level data from Hospital Compare for the complication rates and patient satisfaction scores. The details and results of this estimation strategy are presented in appendix exhibit A.6.[17]

Second, we estimated models using alternative specifications of disproportionate patient percentages, such as quartiles and tertiles.

Third, we specified an alternative measure of a hospital’s safety-net status using the percentage of inpatient stays insured by Medicaid, which we obtained from the American Hospital Association Annual Survey data.[21]

Fourth, we reestimated the multivariate models for complication rates, patient satisfaction scores, and submission of patient-reported outcomes among hospitals that received rewards by controlling for the surgical volume.

Limitations

Our study had several limitations. First, since it was intended to determine the variation in performance and payment across hospitals in the first year of CJR, the unit of analysis was the hospital. Hence the findings from this study can be used only to draw inferences about hospital performance and not about a patient’s likelihood of a particular outcome.

Second, CMS assigns the median score for quality measures of low-volume hospitals. Because of this provision, many safety-net hospitals which are also low-volume, may receive higher scores than what their performance may have warranted. However, our findings demonstrate that despite this provision, safety-net hospitals have lower-quality performance than non-safety-net hospitals.

Third, the patient satisfaction score summarizes the experience of all patients in a hospital and is not specific to patients who have hip and knee replacement surgery. However, since this measure is a valid indicator of the overall patient satisfaction in a hospital, it is also a meaningful representation of the experience of patients with hip and knee replacement.

Fourth, the volume of hip and knee replacements is an important determinant of surgical outcomes.[22] However, the data on surgeries eligible for inclusion in CJR are publicly reported only for hospitals that receive financial rewards, and hence surgical volume was not controlled for in the multivariate models of our main analysis. We addressed this limitation by reestimating the complication rates, patient satisfaction scores, and submission of patient-reported outcome models for hospitals that received rewards.

Study Results

Characteristics Of Hospitals With Varying Disproportionate Patient Percentages

Of the 792 hospitals, 159 qualified as safety-net hospitals (exhibit 1). The mean disproportionate patient percentage of safety-net hospitals was nearly eleven times that of non-safety-net hospitals (69.39 compared to 6.51). Appendix exhibit A.2 presents a histogram of the percentages.[17] Compared to non-safety-net hospitals, safety-net hospitals were more likely to be government-owned, of medium size (200–<400 beds), and affiliated with a medical school (exhibit 1). The median hip and knee replacement volume was 80 percent lower for safety-net hospitals than for non-safety-net hospitals.

Exhibit 1:

Descriptive statistics and unadjusted outcomes for hospitals mandated to participate in the Comprehensive Care for Joint Replacement (CJR) model

	DPP quintiles
	1	2	3	4	5	All
Number of hospitals	158	158	159	158	159	792
Safety-net metrics
Mean DPP****	6.51	17.68	25.68	36.28	69.39	31.15
Mean Medicaid discharges as percentage of all discharges****	9.16	15.55	18.74	23.47	35.81	20.61
Hospital ownership (%)****
Private for profit	36.08	19.62	19.50	19.62	28.30	24.62
Private not for profit	59.49	69.62	62.89	63.29	48.43	60.73
Government	4.43	10.76	17.61	17.09	23.27	14.65
Number of beds (%)****
Less than 200	72.78	61.39	45.28	41.14	42.14	52.53
200–less than 400	24.05	32.91	33.33	32.91	40.25	32.70
400 or more	3.16	5.70	21.38	25.95	17.61	14.77
Medical school affiliation (%)****	33.54	31.65	42.77	54.43	64.78	45.45
Median hip and knee replacement volume****	65	76	70	57	13	59
Hospitals receiving rewards in the first year of CJR (%)****	56.96	47.47	55.35	48.10	33.33	48.23
Mean complication rate****	2.58	2.88	2.82	2.80	2.78	2.78
Mean patient satisfaction score****	87.56	86.33	85.86	84.88	82.27	85.37
Patient-reported outcomes submission (%)****	23.72	29.75	23.90	17.09	10.06	20.89
Quality performance category and spending (%)****
Below acceptable[b]	43.04	52.53	44.65	51.90	66.67	51.77
Acceptable, good, or excellent[c]	56.96	47.47	55.35	48.10	33.33	48.23
Mean reward amount per episode ($)**	742.58	507.88	584.63	598.00	456.31	577.73

SOURCE Authors’ analysis of data for 2016 from the following sources: Data.Medicare.gov . Comprehensive Care for Joint Replacement model—provider data (note 8 in text), CMS.gov. Historical impact files for FY 1994 through present (note 9 in text), and American Hospital Association. AHA Annual Survey Database™ (note 10 in text). NOTES Non-safety-net hospitals are in quintile 1 of disproportionate patient percentages (DPPs; explained in the text); hospitals with increasing DPPs are in quintiles 2–4; and safety-net hospitals are in quintile 5. The patient satisfaction score is the Hospital Consumer Assessment of Healthcare Providers and Systems’ linear mean roll-up score (explained in the text).The complication rate is risk-standardized. Hip and knee replacement volume is reported only for hospitals receiving financial rewards; all other statistics are reported for all hospitals in CJR. Complication rate, patient satisfaction score, patient-reported outcomes submission, and improvement in these quality metrics are used to determine a composite quality score (ranging from 0 to 20), which in turn determines the quality performance category (below acceptable, acceptable, good, or excellent). The quality performance category and whether the hospital spending per episode is above or below the target price determine the amount a hospital will earn or will need to repay to Medicare. In the first year of CJR, hospitals earned financial rewards and were not required to repay Medicare.

p < 0.05

p < 0.001

Below acceptable OR higher quality but not meeting target price.

Acceptable, good, or excellent quality and meeting target price.

Unadjusted Outcomes For Hospitals In The First Year Of Comprehensive Care For Joint Replacement Model

Only 33 percent of safety-net hospitals qualified for financial rewards, compared to 57 percent of non-safety-net hospitals. The complication rate for safety-net hospitals was 0.20 percentage point higher than for non-safety-net hospitals, and the mean patient satisfaction score was 5.29 percentage points lower (82.27 versus 87.56) (exhibit 1). The mean financial reward per episode at safety-net hospitals was $456, a little more than half that of non-safety-net hospitals (appendix exhibit A. 3 graphically presents the differences in the mean financial reward and quality indicators for the five hospital groups). [17]

Adjusted Outcomes For Hospitals In The First Year Of Comprehensive Care For Joint Replacement Model

Compared to non-safety-net hospitals, safety-net hospitals had a complication rate that was 0.26 percentage point higher and a patient satisfaction score that was 4.43 percentage points lower (exhibit 2). [17] Safety-net hospitals had 0.34 times the odds of submitting patient-reported outcomes data, compared to non-safety-net hospitals. This lower-quality performance could have resulted in lower composite quality scores for safety-net hospitals. Full model estimates are presented in appendix exhibit A.4, and the adjusted marginal estimates are presented in appendix exhibit A.5.[17]

Exhibit 2:

Adjusted measures of complication rates, patient satisfaction scores, and patient-reported outcomes submission for hospitals mandated to participate in the Comprehensive Care for Joint Replacement (CJR) model, by quintile of disproportionate patient percentage (DPP)

	Complication rate	Patient satisfaction score	Patient-reported outcomes submission (odds ratio)
Number of hospitals	593	740	791
DPP quintile (ref: 1)
2	0.31****	−1.74****	1.30
3	0.28****	−2.25****	0.83
4	0.26****	−2.87****	0.54*
5	0.26***	−4.43****	0.34***

SOURCE Authors’ analysis of data for 2016 from the following sources: Data.Medicare.gov. Comprehensive Care for Joint Replacement model—provider data (note 8 in text), CMS.gov. Historical impact files for FY 1994 through present (note 9 in text), and American Hospital Association. AHA Annual Survey Database™ (note 10 in text). NOTES The DPP quintiles are explained in the notes to exhibit 1. The estimates for the complication rate (which is risk-standardized) and the patient satisfaction score (explained in the notes to exhibit 1) are from hierarchical linear regression models, and the estimates for the patient-reported outcomes submission are from hierarchical logistic regression models. All models controlled for hospital ownership, medical school affiliation, hospital region, number of beds, and Metropolitan Statistical Area random effects.

p < 0.10

p < 0.01

p < 0.001

The odds of safety-net hospitals’ having below-acceptable performance were 3.56 times those for non-safety-net hospitals (exhibit 3). Consequently, safety-net hospitals had lower odds of receiving financial rewards. Moreover, the adjusted reward amount per episode of $424.59 for safety-net hospitals was approximately half that for non-safety-net hospitals.

Exhibit 3:

Adjusted measures of quality performance, likelihood of financial reward, and financial reward per episode for hospitals mandated to participate in the Comprehensive Care for Joint Replacement (CJR) model, by quintile of disproportionate patient percentage (DPP)

	Odds ratio		Dollars
DPP quintile	Below-acceptable quality performance [a]	Receipt of financial reward	Reward per episode
1	Ref	Ref	843.85
2	1.72**	0.58**	577.65***
3	1.39	0.72	609.87***
4	1.98***	0.51***	601.96***
5	3.56****	0.28****	424.59****

SOURCE Authors’ analysis of data for 2016 from the following sources: Data.Medicare.gov. Comprehensive Care for Joint Replacement model—provider data (note 8 in text), CMS.gov. Historical impact files for FY 1994 through present (note 9 in text), and American Hospital Association. AHA Annual Survey Database™ (note 10 in text). NOTES There were 790 hospitals. The DPP quintiles are explained in the notes to exhibit 1. Hospitals in the quality performance categories of acceptable, good, or excellent are eligible to receive rewards conditional on their spending performance. Hospitals in the category of below acceptable are not eligible for rewards regardless of their spending performance. The estimates for the quality performance category and receipt of rewards are from hierarchical logistic regression models, and the estimates for the reward per episode are from a two-part Poisson model with a log link function. For the reward per episode, numbers represent the adjusted marginal estimates. All models controlled for the variables listed in the notes to exhibit 2.

p < 0.05

p < 0.01

p < 0.001

Below acceptable quality performance OR higher quality but not meeting target price.

Appendix exhibit A.6 presents estimates for changes in key outcomes after CJR implementation and demonstrates both that safety-net hospitals had significantly higher complication rates and lower patient satisfaction scores in the pre-CJR phase and that CJR implementation did not produce significant changes in these metrics.[17]

Appendix exhibits A.7–A.13 present findings from the sensitivity analysis that used different specifications for disproportionate patient percentages and used the percentage of Medicaid stays in a hospital as a proxy for safety-net status.[17] These findings are consistent with those from the main analysis.

Discussion

Payment reforms such as CJR are intended to improve care for all patients, regardless of their clinical or socioeconomic background. However, our findings demonstrate that compared to non-safety-net hospitals, 42 percent fewer safety-net hospitals received rewards in the first year of CJR, and rewards per episode for safety-net hospitals were 39 percent smaller than those for non-safety-net hospitals. These trends persisted when we adjusted for hospital characteristics in the multivariate analysis. Furthermore, our sensitivity analysis demonstrated that safety-net hospitals had poorer quality in the pre-CJR years and that this pattern continued in the first year of CJR. Collectively, these findings suggest that in future years, safety-net hospitals will be less likely to receive rewards and more likely to pay penalties to CMS.

There are several explanations for these findings. First, safety-net hospitals care for sicker patients, and higher proportions of their patients are socioeconomically disadvantaged and members of racial minority groups.[23] These patients lack robust social support systems[24] and are less likely to adhere to prescribed treatment regimens.[16] Consequently, patients in safety-net hospitals are more likely to experience complications and have lower patient satisfaction. Second, achieving the prescribed quality benchmarks requires hospitals to make substantial investments in infrastructure and personnel to improve care quality and patient experiences.[25] However, safety-net hospitals have fewer resources to allocate to quality improvement initiatives,[13] which limits their ability to have effective care coordination programs in place—especially in the first year of CJR. Third, safety-net providers are known to lag behind others in their information technology capabilities,[26] which are vital for collecting and reporting the patient-reported outcomes that contribute to the composite quality score. Finally, some of the quality indicators used for assessing performance in the first year of the CJR were measured prior to the announcement of the CJR. Thus, the indicators may not fully represent the quality improvement steps that hospitals may have adopted after the public announcement and implementation of the CJR. These factors may have contributed to safety-net hospitals’ having received lower composite quality scores.

Importantly, caring for complex patients is expensive, and safety-net hospitals spend more on patient care compared to other hospitals.[23] In the first year of CJR, safety-net hospitals were required to limit their spending to levels below their historic expenditures. Given the clinical and social profiles of their patients, safety-net hospitals would have struggled to reduce spending below historic levels in that year, reducing their likelihood of qualifying for rewards. Furthermore, in later years, the CJR formula for setting target prices assigns greater weight to regional spending and less weight to a hospital’s own spending improvement, which makes it increasingly difficult for safety-net hospitals to achieve spending targets.

Our findings should be viewed in light of recent changes to CJR that made participation voluntary for low-volume hospitals and for all hospitals in thirty-three of the sixty-seven MSAs included in the program, effective January 2018.[3] While this update reduced the overall number of participating hospitals to 491, the proportion of safety-net hospitals that will continue in CJR remains similar to that in the first year. This underscores the importance of finding ways to support safety-net hospitals in CJR.

Our findings are consistent with impact evaluations of other alternative payment models on safety-net hospitals. A recent study that examined the characteristics of hospitals receiving rewards in CJR found that only 21.9 percent of hospitals receiving rewards were safety-net hospitals and that 37.3 percent of hospitals not receiving rewards were safety-net hospitals.[27] Furthermore, other work has shown that safety-net hospitals were more likely to be penalized under reforms such as the Hospital Readmissions Reduction Program.[7]

In its current form, CJR rewards sustained improvement and assigns substantial weight to hospital-specific historic spending in calculating a hospital’s target cost. We consider this to be an excellent opportunity for lower-performing and higher-spending hospitals to improve on both quality and spending, thus increasing their chances of earning rewards. However, our results also suggest that safety-net hospitals still face an uneven playing field in CJR. CMS should consider introducing and measuring disparity-reduction metrics that incentivize hospitals to improve care for the most vulnerable patients and rewarding better outcomes for these patients.[5] CMS could also consider adjusting for sociodemographic risk in CJR performance metrics to give safety-net hospitals more credit for the clinical and social profiles of their patients.[28] Failure to recognize and address these challenges in CJR is likely to increase the clinical and financial burden on safety-net providers, many of which may be reluctant to care for vulnerable patients given concerns about financial penalties. This would reduce the access of vulnerable patients to safety-net providers.

Conclusion

Safety-net hospitals were less likely to receive rewards in the first year of CJR due to their quality and spending performance. Continuation of this trend into subsequent years will place these hospitals at higher risk of CJR penalties. Strategies that address the added complexity of patients treated in safety-net hospitals are necessary to reduce the disadvantage of safety-net hospitals and the vulnerable patients they serve.