Lessons from the Camden Coalition of Healthcare Providers' First Medicaid Shared Savings Performance Evaluation.

Accountable Care Organizations (ACOs) aim to reduce health care costs while improving patient outcomes. Camden Coalition of Healthcare Providers' (Camden Coalition) work already aligned with this aim before receiving state approval to operate a certified Medicaid ACO in New Jersey. Upon its formation, the Camden Coalition ACO partnered with UnitedHealthcare and, through state legislation, Rutgers Center for State Health Policy (CSHP) was established as its external evaluator. In evaluating the Camden Coalition ACO, Rutgers CSHP built on the Medicare Shared Savings model, but modified it based on the understanding that the Medicaid population differs from the Medicare population. Annual savings rate (ASR) was used to measure shared savings, and was calculated at the Medicaid product level and aggregated up to reflect a single ASR for the first performance year. The calculated performance yielded a range of shared savings from an ASR of 0.4% to 5.3%, depending on which dollar amount was used to create the outlier ceiling (limit at which a subset of members with expensive utilization patterns are excluded) and how the appropriate statewide trend factor (the expected percentage increase in Medicaid costs across the state) was chosen. In all scenarios, the ASR resulted in less cost savings than predicted. The unfavorable results may be caused by the fact that the evaluation was not calibrated to capture areas where Camden Coalition's ACO was likely to make its impact. Future ACO evaluations should be designed to better correlate with the patient populations and practice areas of the ACO.

Introduction

As communities across the country confront rapidly rising health care costs, the Accountable Care Organization (ACO) model is being adopted as a payment reform strategy with the potential to improve patient outcomes and bend the cost curve. Among the mix of payers investing in ACOs, a small number of state Medicaid programs are implementing various approaches to the ACO model to keep patients healthy while managing costs. Each state's unique contexts have produced different variations on how each program has been implemented. Several of the early state adopters, including Colorado,[1] Minnesota,[2] and Oregon,[3] have demonstrated savings during their first few years of implementation.

The New Jersey Medicaid ACO Demonstration Project, inspired by the Camden Coalition of Healthcare Providers' (the Camden Coalition) early efforts to engage individuals with significantly high levels of hospital utilization as well as build an organizational and governance structure that would foster communication and collaboration across health care providers and the broader community, was enacted through the New Jersey state legislature in August 2011. The New Jersey Medicaid ACO model is unique among ACO models in that it established a geographic-based approach in which all Medicaid recipients living in a defined catchment area would be assigned to the ACO. The ACOs are required to have at least 5000 Medicaid recipients residing in the catchment area and establish themselves as nonprofit entities whose membership is comprised of all of the area's hospital systems, 75% of the community's primary care providers (PCPs), as well as representation from community organizations. Each ACO is ultimately held responsible for the health, service utilization, and costs of all Medicaid enrollees in its geographic catchment area. Given that only approximately 5% of the state's Medicaid population is fee for service, because of additional legislation that drove the majority of New Jersey's Medicaid population to be covered by managed care, there were limited opportunities for gainsharing unless each ACO additionally contracted with New Jersey's managed care organizations (MCOs), which manage approximately 95% of Medicaid enrollees in the state. Ultimately, 7 communities applied for ACO certification, 3 of which (Camden, Trenton, and Newark) received approval.

The Camden Coalition's ACO has long-standing relationships with MCOs. The ACO established 2 MCO contracts before state certification: the first was with UnitedHealthcare New Jersey in December 2013, and the second was with Horizon Blue Cross Blue Shield of New Jersey in January 2014. As part of its array of strategies for accomplishing the ACO's goals of improving the quality of health care delivery and bending the cost curve, the Camden Coalition operates: (1) a regional Health Information Exchange that collects real-time data from the 4 large health systems in the region; (2) a multidisciplinary, community-based care management program that engages approximately 250 individuals per year who have high numbers of hospital admissions and significant medical and social complexity; and (3) clinical redesign programs that provide structured capacity building and enhanced reimbursement to the majority of primary care practices in the city.

The New Jersey ACO legislation established Rutgers Center for State Health Policy (CSHP) as both a technical assistance provider during the ACO implementation phase as well as the external evaluator of the demonstration project. Rutgers CSHP's role included analyzing and reporting on Medicaid data to help guide ACO strategies, as well as developing and refining a new evaluation methodology to identify whether quality improved and shared savings was produced in each ACO community.

Creating evaluations for shared savings agreements is complicated, and this article addresses the lessons that the Camden Coalition, Rutgers CSHP, and UnitedHealthcare learned while evaluating the Camden Coalition's ACO performance in year 1. Specifically, it discusses the effects of including all MCO spending in the evaluation, and the results of examining the patients who accrued the highest costs over performance in year 1.

Establishing the ACO's first shared savings contract

While the Medicaid ACO demonstration legislation and regulations were being finalized, the Camden Coalition had already begun conversations with several MCOs to understand potential opportunities for collaboration. The earliest partnership emerged through the Coalition's relationship with UnitedHealthcare, the MCO with the second largest share of Medicaid enrollees in Camden. In December 2013, the Camden Coalition and United Healthcare executed a 2-year contract that would became the Coalition's first ACO shared savings contract, providing a unique platform to learn and disseminate shared savings experiences to the broader statewide ACO efforts. The shared savings contract covered United members across all Medicaid product lines who were assigned to the network of PCPs who were members of the Camden Coalition (12 practices), totaling approximately 8500 members. Although the Camden Coalition's state-certified ACO's population was based on all Medicaid members within Camden's 5 zip codes, the Camden Coalition's MCO contract included non-Camden residents who received care from Camden PCPs and did not include Camden members who sought care outside the Camden Coalition's network of PCPs.

In the first contract year, from December 2013 through November 2014, the Camden Coalition agreed to start applying its care management intervention for United's highest risk patients and begin implementing a range of clinical redesign efforts, including creating a citywide quality improvement plan and the “7-day pledge” program to ensure that hospitalized members are connected with primary care as soon as possible following their discharge. Both the care management intervention and the 7-day pledge program were still in the early stages during the first year of the contract: Camden Coalition provided care management services to 52 United members and successfully connected an additional 337 members to primary care services following a hospitalization.

Methods

Shared savings evaluation for Medicaid ACOs is relatively new and still maturing. Rutgers CSHP's evaluation approach, first articulated in their 2012 report, built on the evaluation foundation developed through the Medicare Shared Savings Program's methodology.[4] Understanding that there are significant differences between Medicare's and Medicaid's membership, there remained many questions and the expectation that the process of evaluating would generate new insights and additional questions. In keeping with the newly developing ACO Demonstration Project's evaluation, United and the Coalition decided to forgo United's preexisting shared savings methodology and, instead, developed by Rutgers CSHP. The Institutional Review Board (IRB) at Cooper University Hospital determined that this work constitutes a performance improvement/quality assurance project and therefore did not not require IRB approval.

Overview of shared savings methodology

The initial New Jersey Medicaid ACO Demonstration evaluation framework was built on the Medicare Shared Savings Program's methodology, which included a 3-year baseline period (with more recent years weighted more heavily), risk adjustment using the IPRO risk score, and a trending factor that would be drawn from the statewide growth rate in per capita Medicaid spending. Key deviations from the Medicare model included no minimum threshold for savings and no truncation of outlier spending (because of the expectation that ACOs would be targeting the most expensive members). The ACO savings rate (ASR) would be calculated at the Medicaid product level and aggregated up to reflect a single ASR for the performance year.

Although the broad evaluation methodology was established prior to the first year performance evaluation, because of the uncertain nature of conducting shared savings analysis within the Medicaid population, there was an expectation that new insights may emerge and that partners may need to adapt and evolve the methodology based on the learning. Two particular components of the evaluation required significant reconsideration upon embarking on the evaluation: the decision to not create an outlier ceiling and the methodology for selecting the trend factor, the expected percentage increase in Medicaid costs across the state.

Outlier ceiling

The initial approach of deviating from the Medicare Shared Savings Methodology and not truncating outlier spending originated from the understanding that a core strategy of the ACOs would be the targeting of high-cost Medicaid members through community-based care management. This small subset of members with expensive utilization patterns likely would have the greatest potential return on investment, and enforcing a cap on a member's contribution, intended to safeguard against the potential of a few extreme cases having a disproportionate effect on the savings rate, would have the unintended consequence of limiting the potential savings that could be detected. Upon commencement of the evaluation, however, it became evident that a large majority of the most expensive members were children and their high costs often were the result of non–hospital-based care. Including the entire universe of their spending – sometimes in excess of several hundred thousand dollars toward outpatient and pharmacy care – was having a detrimental impact on the findings. As a result, all parties agreed to revise the methodology to include an outlier ceiling. Although the Medicare model creates a spending ceiling at 99% of the distribution ($100,000), Medicaid's differing population necessitated a different level of truncation. For the purpose of this early evaluation, the study team highlights how different truncation levels impact the savings rate estimates.

Trend factor

The trend factor accounts for the expected increases in Medicaid spending that are attributable to external factors driving health care spending. The initial approach to trend factor involved using Mercer's trend factor. Each year, Mercer is contracted by the New Jersey Department of Human Services to develop this number in order to serve a variety of purposes, including budgetary planning and assisting with rate negotiations with MCOs. The Mercer estimated trend factor for the performance period was 2.8%. However, over the course of the evaluation, the Coalition learned that the Mercer trend factor was not a straightforward estimate of expected cost growth, but instead had a lot of additional assumptions and components included in it. Among these was an “efficiency adjustment,” which is intended to reflect the proportion of anticipated cost growth that could be attributable to inefficiencies at the MCO level. When further examining the trend's methodology, it was discovered that the original Mercer estimate equaled 5.8% and that it was reduced to 2.8% because of the efficiency adjustment. After numerous attempts to deconstruct the Mercer methodology to arrive at a fair and transparent trend factor, the study team was ultimately unsuccessful, and instead modeled the 2.8% and 5.8% scenarios, as well as modeling the halfway point of 4.3%.

Methodology for calculations

The methodology for evaluating shared savings is based on calculating the ASR for each Medicaid product line during the performance year and multiplying it by expected spending. Savings are calculated by product line on a per member per month (PMPM) basis to account for different membership rates across the baseline and performance period. The formulas for calculating savings are as follows:

ASR = (Adjusted Baseline – Performance)/(Adjusted Baseline)

Total Savings PMPM = (ASR) * (Adjusted Baseline PMPM)

Total Overall Savings = (Total Savings PMPM) * (Total member months in the performance year)

“Adjusted Baseline” equals the sum of the risk-adjusted spending during the baseline period and the allowable spending growth (trend factor) and “Performance” equals the true spending in the performance period.

Population estimates

Because the majority of the Camden Coalition's interventions during the first performance period were focused on Camden residents (the 7-day pledge, which targets all members regardless of residency, was still getting started), the study team only evaluated the Camden resident subset of United members at the selected practices, which totaled 3716 residents. For this subset of members, costs were highly concentrated: 0.9% of patients accounted for more than 32% of spending (Figure 1).

Members by cumulative percent of spending.

Results

Savings estimates

Rutgers CSHP's evaluation of performance year 1 yielded a range of shared savings estimates (Table 1). The shared savings estimates ranged from an ASR of 0.4% to 5.3%, depending on 2 key decision points: at which dollar amount to create the outlier ceiling and how to choose the appropriate statewide trend factor. Because the Coalition received an upfront shared savings payment from the MCO that was commensurate with these estimates, even in the most favorable scenario, no additional savings payment was triggered by the evaluation.

1.

Total Annual Savings Rate (Percent) and Savings Amount ($) Across Variable Trend Factors and Outlier Ceilings

	Outlier Ceiling
Trend Factor	100k	80k	63k
2.80%	0.4%	1.7%	2.5%
	$27,962	$114,368	$163,991
4.30%	1.8%	3.1%	3.9%
	$127,773	$212,427	$259,366
5.80%	3.2%	4.5%	5.3%
	$227,583	$310,487	$354,742

Spending by service type

Because variation in savings estimates is expected to be related to overall spending on specific services, the study team explored spending by service type. Nearly half (47%) of United's total spending was driven by outpatient services. Included in outpatient services is emergency department (ED) spending, which is billed as an outpatient service, but ED spending represented only about 10% of total spending. The remaining 37% includes other outpatient services. Inpatient spending was the second most expensive service type, but only reflected 19% of aggregate spending and was nearly equal with pharmacy-related spending, which comprised 18% of total spending (Figure 2).

Percent of spending by service type. ED, emergency department.

Cost drivers by patient type

The majority of high-cost members were not the archetypical patient that the Coalition has sought to engage through its ACO – that is, patients experiencing high inpatient utilization. Rather, the top 1% of members based on cost tended to fall into 2 distinct groups: pediatric members receiving expensive outpatient-based (predominantly home-based) services, and older members whose high costs were being driven by expensive pharmacy and/or outpatient services. More than 50% (18 out of 32) of the high-cost members were younger than 20 years old. On average, only 19% of their spending was driven by hospital-based care. Hospital-based costs accounted for more than 25% of spending for only 8 of the 32 members. For the remaining individuals, high levels of spending were driven in large part by a few key factors: private duty nursing, cancer-related therapies (radiation and chemotherapy drugs), other pharmacy spending, and Healthcare Common Procedure Codes (HCPCs). (HCPCs include a range of codes used for billing purposes. Because of the granularity of the data that were shared for the evaluation, the study team could not determine which specific HCPCs were driving these specific patients' trends. However, across all of the MCO membership in the performance period, the 2 most expensive HCPC categories were T1024 - “Evaluation and treatment by an integrated, specialty team contracted to provide coordinated care to multiple or severely handicapped children” - and J7192 - “Factor VIII Recombinant” [hemophilia treatment]).

Table 2 displays the top 1% of members (by total spend) and highlights each member's primary cost driver as well as the percentage of their total spend attributable to hospital-based services.

2.

Most Expensive United Members (>$50k in Annual Spend) and Primary Cost Driver

2014 cost rank	Age bin	2014 paid	Primary service type spending	Primary cost driver(s)	Primary costs as % of total costs	Hospital-based costs as % of total costs
1	0–4	$409,159	Outpatient	Home health – private duty nursing	61.6%	14.4%
2	15–19	$275,279	Outpatient	Home health – private duty nursing	69.7%	12.4%
3	10–14	$245,347	Outpatient	Home health – private duty nursing	41.4%	24.1%
4	50–59	$240,987	Inpatient		86.4%
5	0–4	$233,973	Outpatient	Home health – private duty nursing	60.5%	6.4%
6	5–9	$194,297	Outpatient	Home health – private duty nursing	78.9%	0.0%
7	15–19	$146,531	Outpatient	Pharmacy and dialysis	90.3%	2.3%
8	10–14	$142,647	Outpatient	Radiation therapy and pharmacy	75.6%	4.9%
9	5–9	$139,932	Outpatient	Home health – private duty nursing	75.8%	5.9%
10	20–29	$132,243	Inpatient		67.0%
11	0–4	$97,149	Outpatient	HCPC (non-DME)	68.9%	0.0%
12	20–29	$96,799	Outpatient	Chemotherapy drugs	90.7%	0.3%
13	50–59	$94,898	Inpatient		39.8%
14	50–59	$94,738	Pharmacy	Chemotherapy drugs	51.6%	2.2%
15	50–59	$79,291	Pharmacy	Pharmacy	86.6%	6.8%
16	50–59	$75,310	Pharmacy	Pharmacy	25.6%	42.0%
17	0–4	$72,815	Outpatient	HCPC (non-DME)	91.5%	0.2%
18	0–4	$71,903	Outpatient	HCPC (non-DME)	89.8%	1.0%
19	0–4	$70,202	Outpatient	HCPC (non-DME)	97.4%	0.2%
20	60+	$69,451	Pharmacy	Pharmacy	29.6%	23.4%
21	0–4	$64,023	Outpatient	HCPC (non-DME)	85.4%	0.4%
22	50–59	$61,885	Pharmacy	Pharmacy	78.2%	0.0%
23	0–4	$61,860	Outpatient	HCPC (non-DME)	92.1%	5.6%
24	30–39	$58,819	Outpatient	Home health – private duty nursing	52.0%	13.7%
25	0–4	$58,522	Inpatient		97.7%
26	0–4	$57,653	Outpatient	HCPC (non-DME)	82.3%	0.3%
27	50–59	$56,493	Pharmacy	Pharmacy	76.3%	0.1%
28	5–9	$56,480	Inpatient		80.7%
29	40–49	$56,200	Pharmacy	Pharmacy	32.7%	38.9%
30	50–59	$53,800	Physician	HCPC	80.5%	0.0%
31	40–49	$51,671	Pharmacy	Pharmacy	59.3%	10.7%
32	0–4	$51,588	Inpatient		39.4%

DME, durable medical equipment; HCPC, Healthcare Common Procedure Code.

Cost drivers by service type

Ultimately, the biggest cost driver was increasing pharmacy costs. Between 2011 and 2014, each successive year witnessed a significant increase in PMPM spending on pharmacy services. Across all 3 product lines, costs increased by greater than 50%, as shown in Table 3.

3.

Growth in Per Member Per Month Pharmacy Spending by United Medicaid Product Line Between Baseline and Performance Year 1

	Supplemental Security Income with Medicare (Duals)	Supplemental Security Income without Medicare (non-Dual)	Temporary Assistance for Needy Families, Family Health Plan, Children's Health Plan
Baseline PMPM spending	$4.46	$228.28	$19.61
Performance year PMPM spending	$6.87	$383.70	$30.25
Percent growth	53.90%	68.10%	54.20%

PMPM, per member per month.

The study team expected the ACO to influence pharmacy spending in a few scenarios. For example, pharmacy costs could be affected through the Coalition's efforts to perform post-hospitalization home-based medication reconciliation for all patients in its high-touch community-based intervention, or through office-based medication reconciliation for patients receiving 7-day pledge appointments. However, upon further review, it became clear that many of the drivers of PMPM growth in pharmacy spending were being impacted by a number of factors that were exogenous to the ACO's sphere of work (Table 4). The Community ACO model was constructed to facilitate care coordination and care management across disparate health systems. Although some of these care coordination and care management strategies might lead to increased uptake in pharmacy utilization, the Coalition and the other community ACOs in the demonstration project were not equipped to tackle the much larger problem of pharmacy costs becoming an increasingly large share of overall health care spending.[5]

4.

Major Pharmacy Cost Drivers by Medicaid Product Line, Comparing Baseline and Performance Year 1

SSI non-Dual population
	PMPM increase in performance year	Total increase in spending	Cost drivers
Antivirals (Hepatitis C, HIV)	$62.02	$306,459	Sovaldi, Stribild, and Incivek
Psychotherapuetic Drugs	$18.59	$91,857	Cymbalta, Invega, Abilify and Seroquel XR
Antiasthmatics	$16.33	$80,682	Advair Diskus
CNS Drugs	$14.75	$72,903	Copaxone
Antihyperglycemics	$10.07	$49,733	Lantus and Lantus Solostar

CHP, Child Health Plan; CNS, central nervous system; FHP, Family Health Plan; PMPM, per member per month; SSI, Supplemental Security Income; TANF, Temporary Assistance for Needy Families.

Discussion

The evaluation of the Coalition's first performance year with United Healthcare highlights that the shared savings conversation is complex, especially when including all MCO spending in the evaluation model. Using this framework is an attempt to account for the total impact that the ACOs have on spending, particularly because many ACOs' strategies will have cost implications outside of inpatient and ED spending as they shift care toward more appropriate settings that will drive costs up elsewhere. However, the evaluation of performance year 1 shows the potential vulnerabilities of including all costs in shared savings models. For example, the major drivers of the pharmaceutical spending surge observed during the performance period were a combination of new expensive therapies and increases in the unit cost of existing, commonly-prescribed therapies, rather than increases in disease prevalence or an uptick in utilization of these therapies. Using the performance year 1 evaluation framework, these cost increases, which the ACO cannot control, are included in shared savings estimates.

Selecting a trend factor—the number that allows ACOs to estimate growth that is outside of their control—will continue to be a challenge. More work must be done to arrive at a fair and objective trend factor, including looking beyond the Mercer trend methodology. In forthcoming evaluations, the Coalition and its partners are exploring using actual statewide trends in Medicaid spending over a longer period of time and averaged out per year as a potential alternative to the Mercer methodology.

Similarly, the per member ceiling for outlier spending has broad implications, especially for Medicaid ACOs. The Medicare approach to shared savings includes a $100,000 ceiling on individual spending, based roughly on the highest 1% of spending for Medicare members. As the Medicaid population is much healthier, on average, than the Medicare population, the most expensive 1% is significantly lower. The range of shared savings estimates detailed in this article highlights the impact that outlier ceilings can have on outcomes.

Lessons for future shared savings agreement evaluations

As Medicaid shared savings strategies continue to mature, effective partnerships among payers, providers, ACOs, and evaluators are crucial. Each stakeholder brings relevant experience and insight that is critical for understanding the stories underlying the data. Through additional data analysis and better understanding of statewide Medicaid trends, ACOs can better understand exogenous spending trends and articulate the impact of these trends on shared savings estimates. Sharing these lessons with stakeholders in real time can allow ACOs to iterate on their evaluations, so approaching evaluation as a collaborative process will shed light on victories as well as highlight opportunities.

Evaluations should be calibrated to capture where the ACO likely will make its impact. For instance, the Camden Coalition's ACO was evaluated using all measures for performance year 1. In subsequent years, the Coalition will aim to work collaboratively at the beginning of its relationships with MCOs to develop evaluation plans that map closely onto the strategies and interventions it employs. As the Camden Coalition moves forward, it and its partners will reflect on adjusting the shared savings evaluation methodologies to better delineate and account for spending trends that are within and outside the scope of the ACO's work.

ACOs and their partners also will be well served to approach evaluation with a realistic expectation of timing—it is unlikely that an ACO will achieve savings immediately. In part, this will be affected by the ability of the ACO to set realistic savings targets that are tied to its strategy. Likewise, the ACO needs time to test, iterate upon and scale tailored interventions, and gather enough data (which often comes with a time lag) to properly evaluate its impact.

In line with its overall strategy, the Camden Coalition's ACO was designed to focus on reducing avoidable hospitalizations. Generally, the study team makes a distinction between patients with mitigable health care needs, such as untreated chronic physical and behavioral health conditions, and patients who have non–mitigable needs, such as congenital conditions and cancer. The team's findings show that, using the performance year 1 methodology, the patients who have the highest health care costs in the ACO are patients who have managed conditions that require a high level of spending on health care. Additionally, more than half of the patients with the highest costs are children, who are served by only one of the 2 interventions in the ACO (the 7-day pledge). Through process and quality improvement, there may be opportunities to reduce costs for these patients, but they are not the population the ACO is currently targeting. Following the evaluation of the Coalition's first performance year, questions remain as to how to best account for these members. The study team will need to work collaboratively with the State, MCOs, and Rutgers CSHP to better understand additional subgroups of high-cost members and set realistic expectations over potential impact, and the shared savings methodology will need to evolve to reflect these conversations.

This shared savings evaluation taught the study team that though they expected to use methodology intended for a Medicare population, they needed to consider the unique population the Camden Coalition serves and the interventions being tested. Additionally, shared savings is 1 avenue of driving collaboration between MCOs and community-based organizations, but it is not the only way. Focusing solely on shared savings will limit opportunities to explore other creative ways to collaborate and realign incentives—and it will be critical to properly evaluate those efforts, as well.