Voluntary partial capitation: the Community Nursing Organization Medicare demonstration.

In a recently concluded Medicare demonstration, Community Nursing Organizations (CNOs) received capitated payment to provide a subset of Medicare services through a nursing case management delivery system. Demonstration participation was voluntary, both for CNOs and recruited beneficiaries, raising several challenging issues associated with selection. We investigate provider and beneficiary selection, as well as Medicare costs, using multiple evaluation methodologies. We find that CNO enrollment is associated with increased payment by Medicare for CNO-covered services. Results showing CNO enrollees to be more costly to Medicare for non-CNO services are consistent with cost shifting, but could also be accounted for by biased provider selection into the demonstration.

RCT Entities:   Population Interventions Outcomes

Introduction

Many Medicare demonstrations have the following structure: a solicitation is issued for organizations to provide or assume risk for care to beneficiaries. Organizations voluntarily apply for participation and CMS selects, based on the merits of application, final participating organizations. Interested beneficiaries then voluntarily seek out the participating providers or plans, sometimes after receiving marketing material from those providers or plans, e.g., CNO demonstration, preferred provider organization demonstration (Federal Register, 2002), home health prospective payment demonstration (Federal Register, 1999), and Program of All-Inclusive Care for the Elderly (PACE) (Public Law 105-33, section 4801). In demonstrations with randomized designs, these interested beneficiaries are then randomly assigned to treatment and control groups.

Evaluations of demonstrations with voluntary participation on the part of providers and beneficiaries face two sources of bias due to selection: the familiar selection effect due to voluntary participation of beneficiaries and the less considered selection effect due to voluntary participation of providers. To thoroughly study these issues of selection, it is not sufficient to compare measures of the treatment and control groups as both groups were formed after providers and beneficiaries self-selected. One needs also to consider a third comparison group, composed of beneficiaries with no contact with the participating organizations. Note that this comparison group differs from the control group because the control group consists of beneficiaries who did have contact with the organizations. In fact, they may know about the demonstration precisely because they frequent the participating providers and/or received their marketing material.

In recent years, the primary focus of demonstrations has been to test models for evidence of cost control, with the principal cost management tool being capitation. In some cases, the participating organizations are only at risk under capitation for a subset of Medicare benefits. We call the latter arrangement partial capitation; it is also known as a carve-out. The financial incentive for organizations paid under partial capitation is to minimize utilization covered under the capitated payment. There is no incentive for the organizations to minimize utilization of services that are not covered under partial capitation (i.e., those for which another entity, perhaps traditional Medicare, is at risk). Therefore, considering these financial incentives alone, one might hypothesize that partially capitated models of care would be associated with shifts in utilization and costs from the services under capitation to those that are not. However, as this article will show, potentially biased selection of providers into the demonstration must be accounted for before a conclusion of cost shifting can be supported.

The Medicare CNO demonstration (1994-2001) was an example of a voluntary, partially capitated demonstration. This demonstration tested an innovative approach to the provision of community nursing and ambulatory care services for Medicare beneficiaries. Structured around the two fundamental concepts of nurse case management and capitated payment, CNOs attempted to promote the timely and appropriate use of community health services and to reduce the use of costly acute care services. Early experience in the demonstration suggested that CNO participation might reduce utilization and cost and improve outcomes for Medicare beneficiaries (Ethridge, 1997; Lamb and Zazworsky, 1997; Schraeder and Britt, 1997; Storfjell, Mitchell, and Daly, 1997). Evidence supporting these claims, however, was anecdotal and not quantitative (Burtt, 1998).

Our quantitative evaluation of the CNO demonstration focused principally on the implications of the CNO treatment model for cost to the Medicare Program. Specifically, our analyses compared the costs to Medicare of services utilized by those treated under the CNO model (CNO treatment group) to those generated by a randomized control group and a population reference group (the latter consisting of beneficiaries with no known contact with a CNO). Our main finding is that Medicare spending was higher for members of the CNO treatment group as compared with both the control group and to the population reference group. In particular, expenditures for CNO-covered services were higher for the treatment group as compared with the control or population reference groups. Comparison of the CNO treatment group to the population reference group suggested that CNO participation also induced higher utilization of non-CNO services at some sites.

Our results, while most pertinent to the cost implications of nurse-managed care, have broader relevance. In addition to a demonstration of the CNO service delivery model, the CNO demonstration was one of partial capitation because CNOs received a fixed payment per member for a subset of services normally covered by traditional fee-for-service (FFS) Medicare. Thus, our results bear on a significant issue associated with evaluation of a partial capitation model: how does one interpret evidence of cost shifting (Anderson and Weller, 1999; Glass and Sappington, 1999)? As will be explored, only by analyzing results from a treatment, control, and population reference group can cost shifting be unambiguously differentiated from biased selection of providers into the demonstration.

The remainder of this article is organized as follows. The next section provides background on the origins and history of the demonstration and evaluation, details about the selection and operation of the CNO sites, the services they provided, and beneficiary eligibility and enrollment in the CNOs. The evaluation design and data are then described, including a discussion of the original randomized design, departure from that design, and supplementation of the analysis with a population reference group. Next we provide analysis of expenditures based on the randomized design. Following that is an analysis of expenditure based on a risk-adjusted treatment-to-population comparison. The article concludes with a policy discussion in which some additional findings from the study are summarized.

Background

History of the Demonstration and Evaluation

The impetus for developing the CNO model stemmed from limitations in traditional FFS Medicare. Parts A and B of Medicare reimburse only care that is ordered by a physician and supplied by certain providers under certain specified conditions. Medicare generally has no provision for reimbursing preventive care, health promotion, or care not authorized by a physician-services that might lead to lower medical costs and improved health outcomes for Medicare beneficiaries. Since 1985, Medicare health maintenance organizations (HMOs) have aimed to address these limitations by providing a broader and more flexible array of services, in return for a fixed monthly payment for each subscriber. However, many Medicare beneficiaries are reluctant to join HMOs, because they typically restrict members' choice of providers.

The CNO concept thus provided an alternative to both traditional FFS Medicare and Medicare HMOs. Like HMOs, CNOs were funded by flat monthly Medicare payments for each enrolled member and could exercise substantial discretion in organizing care in the most efficient and productive way. Because only a limited range of services were covered by the capitation payment, beneficiaries were still able to choose their providers, notably physicians, hospitals, and other facilities in the same manner as all other traditional FFS Medicare beneficiaries. However, CNO nurses were to coordinate the provision of health care services for each enrollee with a focus on prevention and disease management, thus attempting to avoid higher future health care costs (Storfjell, Mitchell, and Daly, 1997; Schraeder and Britt, 1997; Ethridge, 1997).

In order to explore the impact of the CNO model of care delivery on utilization, cost to Medicare, and outcomes, the CNO demonstration was mandated by the Omnibus Budget Reconciliation Act (OBRA) of 1987. To carry out this demonstration, CMS entered into cooperative agreements in 1993 with the following four eligible organizations to serve as demonstration providers: (1) Carondelet Health Care (Tucson, AZ), (2) Carle Clinic (Urbana, IL), (3) Living at Home/Block Nurse Program (Minneapolis, MN), and (4) Visiting Nurse Service (New York, NY). Originally, the demonstration was to last 3 years, 1994-1996, inclusive. In 1996, CMS extended the CNO demonstration an additional year. The Balanced Budget Act of 1997 subsequently granted a further 2-year extension. Most recently, the Balanced Budget Refinement Act of 1999 granted an additional 2-year extension, authorizing the sites to continue operating until December 31, 2001. This final 2-year period (2000 and 2001) is not covered in this article.

Early reports from nurses at the CNO demonstration sites suggested that the CNO model might result in lower hospital and home health utilization, cost savings, and improved health care outcomes for participants (Ethridge, 1997; Lamb and Zazworsky, 1997; Schraeder and Britt, 1997). However, CNO nurses and administrators cautioned that these benefits might be evident only after long-term CNO participation (Storfjell, Mitchell, and Daly, 1997). Most claims of cost savings and improved outcomes made in the early years of the demonstration were anecdotal and not based on quantitative analysis (Burtt, 1998). The quantitative analyses reported in the literature on the utilization and cost implications of nurse case management of care for the elderly are contradictory and, taken as a whole, inconclusive (Gagnon et al., 1999). This lack of consensus suggests that benefits of nursing case management are not obvious or guaranteed and may depend on precisely how the intervention is implemented and what incentives exist for the organizations contracted to provide case management.

When the CNO demonstration began, CMS also funded the first of what would be two quantitative evaluations. The first evaluation covered the period January 1994-July 1997, was conducted by several of the authors, and was based on randomized assignment of applicants to either CNO participation or traditional Medicare. The study found only small and generally insignificant differences in functional status between treatment and control groups at 15, 27, and 39 months after randomization. Total Medicare expenditures were found to be significantly higher among treatment group beneficiaries than among those assigned to the control group. This result held regardless of whether the treatment group was defined as all beneficiaries randomly assigned to treatment—the intent-to-treat model—(Hollis and Campbell, 1999) or was defined as beneficiaries assigned to treatment and actually enrolled in a CNO.

Age-adjusted mortality rates computed during this first evaluation were lower for CNO applicants than for eligible non-applicants residing in the same localities. These mortality results were in close agreement with those of studies of Medicare risk HMOs carried out during the 1980s and 1990s (Brown et al., 1993) and suggested favorable selection into the CNOs (Abt Associates Inc., 1998; 2000). However, in a subsequent evaluation, we learned that selection into the CNOs was not favorable.

The Medicare, Medicaid, and State Children's Health Insurance Program (SCHIP) Benefits Improvement and Protection Act (BIPA) of 2000 directed the Secretary of Health and Human Services to investigate whether results from the CNO demonstration might change in the long run, because gains from the preventive efforts of the CNOs might take time to materialize. At the same time, BIPA reduced the inflation-adjusted capitation rates to be paid to the sites (15 percent in New York and 10 percent at the other sites) and called for a second evaluation. Results from this second evaluation, covering the period 1994-1999 were published (Abt Associates Inc. 2001) and are the topic of this article.

CNO Sites

Through a competitive selection process, CMS chose four diverse sites to set up CNOs for the demonstration. Each site had considerable freedom in how it chose to organize itself. As long as the mandatory services were provided and the basic OBRA guidelines were followed, sites could individually determine the most efficient and productive ways to serve their members.

Each CNO site was unique in the characteristics of its operating organization, the population it served, or the health provider market in which it operated: The Carle Clinic CNO (Carle) was operated by a for-profit, private physician group in rural Illinois and served a population relatively unfamiliar with managed care; the Carondelet Health Care CNO (Carondelet) was run by a Catholic, non-profit corporation in southern Arizona and served a population with a relatively high degree of seasonal migration; the Living at Home/Block Nurse Program CNO (LAH/BNP) was a community-based initiative in Minnesota with a contractual relationship with HealthSpan, the largest Medicare-certified home care agency in the State and an experienced player in the robust managed care market in Minnesota; the Visiting Nurse Service of New York CNO (VNSNY) was operated by the largest non-profit Medicare-certified home health care agency in the Nation and served enrollees who tended to be older, sicker, more in need of psychological services, and more resistant to managed care than enrollees at other sites.

Covered Services, Eligibility, and Enrollment

OBRA 1987 required that certain services be provided as part of the CNO service package: home health care, outpatient physical therapy, durable medical equipment, prosthetics/orthotics, and supplies. Non-CNO services, or those the CNOs were not obligated to provide and were covered by traditional Medicare were inpatient hospitalization (short and long stay), hospital outpatient services, skilled nursing facility stays, hospice, physician office visits, and other Part B services (e.g., lab, ancillary).

With a few exceptions all Medicare beneficiaries residing in catchment areas close to the CNOs, entitled to Part A benefits, and enrolled in Part B were eligible to enroll in a CNO. Each CNO site was required to hold at least one open enrollment period during the demonstration and to accept any eligible beneficiary who applied for membership. Initially, those accepted into the demonstration were randomly assigned to treatment and control groups for the evaluation. In later phases of the demonstration, direct enrollment into the treatment group and switching from control to treatment group became possible.

Each site developed its own strategy for marketing and recruitment of eligible beneficiaries. All sites relied on physician referrals, direct mail, and word of mouth. Some sites also used brochures, fliers, group presentations, television and newspaper advertising, and telemarketing efforts. The number of participants by site and treatment/control status is summarized in Table 1.

Table 1

Number and Percentage of Treatment and Control Group Participants, by Site: 1994-1999

Site	Treatment Group		Control Group
	Number	Percent	Number	Percent
Carle1	3,191	76	1,000	24
Carondelet2	3,691	74	1,322	26
LAH/BNP3	2,196	72	868	28
VNSNY4	1,988	77	594	23
Total	11,066	75	3,784	25

Operated by a for-profit, private physician group in rural Illinois and served a population with relatively unfamiliar with managed care.

Run by a Catholic, non-profit corporation in southern Arizona and served a population with a relatively high degree of seasonal migration.

The Living at Home/Block Nurse Program was a community-based initiative in Minnesota.

The Visiting Nurse Service of New York was operated by the largest non-profit Medicare-certified home health agency in the Nation and served enrollees who were older, sicker, in need of psychological services, and more resistant to managed care.

SOURCE: Abt Associates Inc.: Data from the 1999 Community Nursing Organization Evaluation Analytic Files.

Evaluation Design and Data

Randomized Design

In order to develop the most precise estimates possible of the impacts of the CNO intervention, the demonstration was originally structured as a social experiment in which individuals were randomized to either CNO participation (the treatment group) or to continue to receive their traditional Medicare benefits (the control group). However, given that participation in the CNO was voluntary, the decision to apply was likely to be influenced by hard-to-measure factors that also influence health outcomes and cost. The subset of the Medicare population that wished to participate in the CNO was likely to differ from those who had no interest in joining the CNO. Thus, the only way to create a valid comparison strategy was to do so after the decision to participate in the evaluation had been made so that only those who wished to participate could be compared. Applicants were randomly assigned to treatment or control status after the decision to apply had been made, a consent statement had been signed, and collection of baseline data had occurred.

To accommodate the program's need to build up enrollment quickly, two applicants were assigned to the treatment group for every applicant assigned to the control group. The fact that the control group was smaller than the treatment group reduced the statistical power of the evaluation, increasing the size of the minimum impact that could be detected reliably.

Departures from Randomized Design

Though the original evaluation design was based on random assignment of individuals to treatment (CNO participation) and control (traditional Medicare only) groups, the actual implementation of the demonstration deviated from this design. In several instances, beneficiaries who were randomly assigned to the control group were provided services intended for the treatment group. In addition, during certain periods of the demonstration, no beneficiaries were assigned to the control group.

Starting October 1, 1995, all new applicants were assigned to the treatment group, because, according to the original schedule, no followup assessments allowing comparisons between treatments and controls would have been performed on applicants randomized after that date. In early 1996, CMS modified the original contract allowing the evaluation to continue for another year. At that time, it was decided that the pool of control group members was already sufficiently large and that randomizing a small number of new controls would contribute little to the analysis. Throughout 1996, all applicants were assigned to the treatment group. However, when the demonstration was again extended for 2 more years, the randomization of new applicants to both treatment and control groups in a 2:1 ratio was resumed.

Overall, the hiatus in randomization to the control group lasted from October 1995-December 1996. This period is referred to as Wave 2 of the evaluation, with the initial phase with intact randomization labeled Wave 1. When it had been decided to continue the demonstration for 2 additional years, randomization in a 2:1 ratio was resumed from January 1, 1997, until October 2, 1997 (Wave 3). After October 3, 1997, sites again were allowed to enroll applicants without randomization and members of the control group were permitted to switch to the treatment group (Wave 4). By the end of 1999, approximately 18 percent of applicants (697 out of 3,784) originally assigned to the control group had enrolled in the CNO and received the same services as those in the treatment group.

Population Reference Group

Due to these breakdowns in the experimental design, estimates computed using data from Wave 2 (when nobody was randomized to the control group) and Wave 4 (when nobody was randomized to the treatment group and control group members were allowed to switch to the treatment group), would suffer from bias due to self-selection. Moreover, no estimates of controls can be made during these two waves because no beneficiaries were assigned to the control group. Hence, the need arose for the construction of a population reference group to serve as a comparison group in lieu of a randomized control group. In this article, we present results comparing Medicare expenditures for the treatment group to that of the control group and a population reference group.

The population reference group consisted of all eligible Medicare beneficiaries who lived in a CNO catchment area. The Medicare population differed in several respects from demonstration participants: the former tended to be younger because the non-elderly disabled were less likely to participate in the demonstration; they tended to have lower spending and utilization, due in part to the fact that more members of the population reference group had zero Medicare claims; and they were more likely to be non-white and female. Because of these differences, comparison of the average Medicare payment for CNO treatment group members to that of the population reference group would suffer from selection bias. Therefore, we applied several statistical techniques to reduce the effects of bias on estimates, including a risk-adjusted difference-in-difference approach within a two-step switching model.

Analytic File

Our analytic file consisted of data on 11,066 CNO applicants who were randomized to the treatment group, 3,784 who were randomized to the control group (Table 1), and 239,382 person-year records representing the population reference group. The file was constructed using data from three sources: (1) Medicare Enrollment and Claims Files from CMS, (2) CNO enrollment and payment files maintained by CMS, and (3) Hierarchical Coexisting Conditions (HCC) files constructed by Fu Associates. Medicare enrollment and claims files (specifically the Medicare Enrollment Database and the National Claims History Database) provided beneficiary demographics and entitlement data, beneficiary enrollment dates, beneficiary and provider identifiers, service dates and units, service types and locations, charges and reimbursements, and diagnosis and procedure codes. We obtained service and payment data from the inpatient, skilled nursing facility, outpatient, home health agency, hospice, and physician/supplier claims files. Claims records were collected from 1993 to 1999 for all demonstration participants and the population-based reference group.

The CNO enrollment and payment files were maintained by CMS to determine CNO eligibility. These files contained capitation rates, group cell categories to which enrollees were assigned, and corresponding assessment dates. The enrollment files allowed CMS to keep accurate eligibility records, and were necessary for CMS to determine capitation payments for the CNOs each month.

Because of potential differences in average health status between the treatment group and the population reference group, an HCC-based risk-adjustment technique was used to separate the potential demonstration effect from these background differences. HCCs are groups based on International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) (Centers for Disease Control and Prevention, 2004) diagnoses that are predictive of future utilization of health care (Ellis et al., 1996). HCC scores for both demonstration participants and the population-based reference group were calculated by Fu Associates, under a direct contract with CMS, and made available to the project team.

Analysis of Randomized Beneficiaries

Analytic Approach

We defined the treatment group as all individuals randomized to CNO enrollment, even if they later disenrolled from the CNO. In so doing, we followed the principle of intent-to-treat under which study subjects are analyzed according to their initial treatment/control assignment regardless of compliance or non-compliance with the experimental intervention (Hollis and Campbell, 1999). Under intent-to-treat, the difference in mean monthly expenditures for the treatment and control groups does not estimate the expected saving to Medicare of a month spent in the CNO, but rather the expected saving (positive or negative) associated with the CNO averaged over months in which those assigned to the treatment group could have been in the CNO. This measure of CNO effect is appropriate for evaluation of a demonstration intervention that might be implemented by the Medicare Program as a voluntary option for the beneficiary. If a CNO program were to become part of the Medicare benefit, then it is highly unlikely that beneficiaries would be compelled to remain in the CNO, once they had joined. Rather, beneficiaries would be permitted to leave the program and some would do so, just as they did in the demonstration. The treatment/control difference thus answers the question: “For beneficiaries likely to enroll in a CNO program, what is the average monthly saving to Medicare of giving them the option to do so?” Cumulative expenditures per person/per month were computed for both groups by month of enrollment. All dollar amounts were expressed in 1999 dollars using the Consumer Price Index for discounting.

Results

Over the first 72 months of operation of the demonstration, total monthly expenditures per person for all services were higher for the treatment group in all four sites (Table 2). The relative difference in total expenditures per month between the treatment and control groups varied from 5 percent at Carondelet, to more than 14 percent at Carle and LAH/BNP. Mean expenditures for CNO-covered services were greater for treatments than for controls at every site by amounts ranging from $44 to $55 per person/per month. These amounts represented relative differences of between 54 and 180 percent. Expenditures for non-CNO services were comparable for treatment and control groups at Carle and LAH/BNP, $20 per person/per month lower for treatments than controls at the Carondelet site, and $10 per person/per month higher at VNSNY.

Table 2

Medicare Expenditure Per Person/Per Month, 72 Months After Random Assignment, by Site: 1994-1999

All Services	Carle1	Carondelet2	LAH/BNP3	VNSNY4
Treatment	**$357	$479	**$416	**$830
Control	**311	455	**364	**765
CNO-Covered Services
Treatment	83	103	84	156
Control	36	59	30	101
Non-CNO Services
Treatment	274	376	332	674
Control	275	396	334	664
Inpatient Hospitalization
Treatment	129	161	181	355
Control	134	165	187	337
Emergency Room
Treatment	1.07	1.01	1.06	1.77
Control	0.92	0.97	0.98	1.4
Physician Office Visits
Treatment	14	25	14	37
Control	14	26	14	38

Denotes treatment-control differences significant at p<0.05.

Operated by a for-profit, private physician group in rural Illinois and served a population with relatively unfamiliar with managed care.

Run by a Catholic, non-profit corporation in southern Arizona and served a population with a relatively high degree of seasonal migration.

The Living at Home/Block Nurse Program was a community-based initiative in Minnesota.

The Visiting Nurse Service of New York was operated by the largest non-profit Medicare-certified home health agency in the Nation and served enrollees who were older, sicker, in need of psychological services, and more resistant to managed care.

NOTES: All figures are in 1999 dollars. Means describe beneficiaries randomized between January 1994 and September 1995, and between January 1997 and October 1997. CNO is Community Nursing Organization.

SOURCE: Abt Associates Inc.: Data from the 1999 Community Nursing Organization Evaluation Analytic Files.

By construction, the results of Table 2 more heavily weight the early experience of program participants. Because CNO effects may have appeared gradually over time among enrollees and because the number of months 1 and 2 observations will invariably exceed the number of later (e.g., month 35 or 36) observations, the estimation of CNO treatment group expenditures may understate true CNO effects. Also, treatment/control differences may be depressed by the inclusion of early years of CNO operation when program staff had not yet learned how best to provide services to a diverse clientele. It is reasonable to expect that a number of lessons would be learned during the initial phase of operation that would result in better identification of beneficiary needs and improved service delivery in later years.

To address these issues, expenditures per person/per month were recomputed after deleting all data for the first 6 months after randomization for all beneficiaries and after deleting data for 1994, the first year of CNO operation. If startup effects on either beneficiaries or the CNOs themselves were of substantial magnitude, then treatment/control differences computed on this pared-down sample would be a more accurate estimate of long-term CNO effects. These deletions, however, had almost no effect on mean Medicare expenditures per person/per month (results not shown). Mean expenditures for the treatment group exceeded that for the control group in every site, by amounts ranging from 7 to 15 percent—nearly identical to the relative difference between treatments and controls shown in Table 2.

Medicare expenditures per person/per month for hospital inpatient care, emergency room care, and physician office visits are also shown in Table 2. Mean monthly expenditures for inpatient care were similar for the treatment and control groups, ranging from 4 percent less for the treatment group at the Carle site, to 5 percent more for the treatment group at VNSNY. Expenditures for emergency room visits were slightly higher for treatment group members at all sites, relative to control group members. Expenditures for physician office visits were similar for treatment and control groups at all sites.

Discussion

The treatment/control differences presented are biased toward zero due to the previously described contamination of the control group (namely, some controls received treatment during the demonstration). Despite this bias, the results indicate that per person/per month expenditures for CNO-covered services were higher for treatments than controls at all sites. In principle, these differences in expenditure for CNO-covered services could have been eliminated by reducing the CNO capitated payment rate. However, doing so might have increased incentive to shift utilization and cost to non-CNO services. Differences in per person/per month expenditures for non-CNO services were less consistent across sites and no clear conclusion about cost shifting under the implemented payment rates can be drawn.

Treatment-to-Population Comparison

This section describes our quasi-experimental approach in which we constructed a comparison group based on the traditional FFS Medicare population in the CNOs' catchment areas. As discussed, CNO applicants differed from the population in the CNO catchment areas in age/disability, race, sex, health care services utilization, and spending. A straight comparison of means would confound any treatment effect with these observable (as well as any unobservable) differences and would provide a biased estimate of the effect of CNO participation on our spending measures. Thus, adjustment techniques that account for observable and unobservable differences between the treatment group and the population reference group were necessary.

Our risk-adjustment approach was based on a risk score, defined as the current year value of the quantity under analysis (e.g., this year's total expenditure, CNO-covered expenditure, etc.) as predicted by demographic characteristics, last year's expenditures and diagnoses from Medicare inpatient, outpatient, and physician claims. The overarching strategy for risk adjustment was to predict expenditures with a very rich model separately and then use the predicted value from this first model as a single risk score variable in the regression models that tested the treatment effect (Needleman et al., 2001). In other words, the treatment effect models investigated to what degree CNO participation could explain the deviation of the actually observed expenditures from the estimated expenditures, which were predicted on the basis of beneficiary characteristics. We estimated a very comprehensive model based on demographic characteristics, past utilization, past diagnoses, and interaction terms. Demographic characteristics were captured by a variable for sex and four variables for age groups (under 65, 65-74, 75-84, 85 or over). Past utilization was captured by including (both linearly and quadratically) the expenditures incurred during the previous year. It is both plausible and well established by research that such prior utilization measures are highly predictive of future resource use (van Vliet and van de Ven, 1993; Pope et al., 1998), making them ideal candidates for the proposed adjustment model.

To capture past diagnoses, we included 118 indicator variables for the prospective HCCs. HCCs have been created to calculate risk-adjusted capitation payments for Medicare beneficiaries enrolling in HMOs (Ellis et al., 1996). They are groupings of ICD-9-CM diagnostic codes into non-mutually exclusive categories based on clinical logic that have been shown to be good predictors of utilization of medical care.

Our analyses of expenditure variables relied on many different techniques and specifications. We began with the simplest, unadjusted difference in means and then applied other techniques to yield a sequence of models of increasing sophistication. So, for example, after comparing unadjusted means, we compared means risk adjusted according to the method previously described. Next, we adjusted differences for risk and year effects. Following that, we applied a difference-indifferences approach that examines not the treatment versus population levels of expenditure over time, but the changes in the difference in levels between treatment and population groups. The advantage of this approach is that baseline differences, which may be a consequence of non-random (observable or unobservable) selection cancel, and thus, cannot bias the results. Finally, we combined this difference-in-differences approach with a two-part switching model to further control for unobservable selection (Heckman, 1974; Maddala, 1983). Throughout our analysis, all dollar amounts were converted to 1999 dollars using the Consumer Price Index for discounting.

Differences in average total Medicare spending per person/per month are reported for all four sites in Table 3. The table displays the unadjusted estimates which, as previously discussed, reflect both a treatment and a selection effect. In all four sites, average spending was significantly higher in the treatment group than in the population. The difference ranged from $228 at the VNSNY site to $46 at Carle. Risk-adjusted differences in average monthly spending, accounting for observable beneficiary risk factors, are presented in the table. Although the treatment group still had higher expenditures, these adjusted differences were considerably lower and were statistically insignificant for two sites (Carle and LAH/BNP). Between 67 percent (Carondelet) and 72 percent (Carle) of the unadjusted differences could be explained by differences in case-mix, demonstrating the power and and LAH/BNP). Between 67 percent (Carondelet) and 72 percent (Carle) of the unadjusted differences could be explained by differences in case-mix, demonstrating the power and importance of the risk-adjustment model. In addition to observable factors of beneficiary risk, we also adjusted for the impact of random events that occurred in 1 or more years of the observation period, such as a flu epidemic, by using year-fixed effects. As shown in Table 3, adjusting for year-fixed effects had almost no effect on the estimated differences between spending for treatment group members and spending for the population. This is not surprising, because it is unlikely that such events would have affected one of the two groups more than the other.

Table 3

Estimated Differences in Per Month Spending Between the Community Nursing Organization (CNO) Treatment Group and the Population Reference Group for All Services in All Years: 1994-1999

Risk	Carle1	Carondelet2	LAH/BNP3	VNSNY4
Unadjusted	***$46	***$91	***$52	***$228
Adjusted	9	***30	14	***66
Adjusted and Year Effects	8	***29	15	***66

p<0.01.

Operated by a for-profit, private physician group in rural Illinois and served a population with relatively unfamiliar with managed care.

Run by a Catholic, non-profit corporation in southern Arizona and served a population with a relatively high degree of seasonal migration.

The Living at Home/Block Nurse Program was a community-based initiative in Minnesota.

The Visiting Nurse Service of New York was operated by the largest non-profit Medicare-certified home health agency in the Nation and served enrollees who were older, sicker, in need of psychological services, and more resistant to managed care.

NOTES: All dollar amounts were converted to 1999 constant dollars. The CNO treatment group consists of all those randomized to treatment or directly enrolled in the CNO.

SOURCE: Abt Associates Inc.: Data from the 1999 Community Nursing Organization Evaluation Analytic Files.

Despite the predictive power of our risk-adjustment method, it is possible that part of the remaining spending differences might be traceable to unaccounted for baseline differences between CNO participants and the population. To investigate this possibility, we examined differences in spending changes over time, employing the previously described difference-in-difference approach. Starting from the base-line year of 1994 (the first year of the demonstration) we examined how average spending changed differently for participants as compared to the population. By analyzing changes over the years of the demonstration, we also addressed the question of whether the sites became more efficient over time. The underlying hypothesis is that CNO sites might not have been cost effective in the early years since enrollees had not had time to benefit from preventive services and the sites were still improving their models of care, but that later years would show more positive health effects. The results of this analysis are summarized in Table 4.

Table 4

Estimated Differences Between the Community Nursing Organization (CNO) Treatment Group and the Population Reference Group: Changes in Per Month Spending, Relative to 1994

Year	Carle1	Carondelet2	LAH/BNP3	VNSNY4
All Services
1995	-$1	$26	***$121	***$153
1996	3	3	***116	**153
1997	*22	45	***111	***277
1998	11	45	**101	***377
1999	***60	**75	***192	***373
CNO-Covered Services
1995	1	***17	***31	**21
1996	***15	-1	**18	*17
1997	***14	11	***33	***55
1998	**12	***22	***36	***105
1999	0	*14	***52	***96
Non-CNO Services
1995	-1	3	***87	**128
1996	3	-9	**79	**128
1997	22	22	**67	***201
1998	11	12	62	***267
1999	**60	42	***142	***255

p<0.1.

p<0.05.

p<0.01.

Operated by a for-profit, private physician group in rural Illinois and served a population with relatively unfamiliar with managed care.

Run by a Catholic, non-profit corporation in southern Arizona and served a population with a relatively high degree of seasonal migration.

The Living at Home/Block Nurse Program was a community-based initiative in Minnesota.

The Visiting Nurse Service of New York was operated by the largest non-profit Medicare-certified home health agency in the Nation and served enrollees who were older, sicker, in need of psychological services, and more resistant to managed care.

NOTES: All dollar amounts were converted to 1999 constant dollars. All differences are adjusted for beneficiary risk and year effects. The CNO treatment group consists of all those randomized to treatment or directly enrolled in the CNO. Note that CNO-covered and non-CNO components were estimated in separate statistical models, they do not necessarily add up to the differences in total spending.

SOURCE: Abt Associates Inc.: Data from the 1999 Community Nursing Organization Evaluation Analytic Files.

The results of Table 4 clearly demonstrate that average monthly Medicare spending increased much faster in the treatment group than in the population. In two sites (LAH/BNP and VNSNY), the difference in changes was already statistically significant in 1995, whereas it became significant only in 1999 at Carondelent, and in 1997 and 1999 at Carle. All four sites showed a nearly steady increase in estimated differences through time, indicating that average spending in the treatment group kept increasing relative to the population over the course of the demonstration. Not only are many of the estimates statistically significant, but also of a substantial magnitude: By 1999 average spending increased from $60 (Carle) to as much as $373 (VNSNY) more per CNO participant per month.

The same methodology was used to assess the hypothesis that a beneficiary had to be enrolled in a CNO over a certain period for the positive effects of case management on cost to materialize. For this analysis, we removed those treatment group members who had less than 6 months of enrollment. This definition of CNO treatment selectively removes approximately one-third of the person-years from the treatment group, while the population reference group remains unchanged. The results (not shown) reveal the same pattern as the ones presented in Table 4: Differences in monthly spending between CNO participants and the reference group grew over time, with the exception that the differences at Carle were not statistically significant in any year. Finally, we obtained nearly identical results (not shown) using a two-part switching model.

Having established that CNO participation was associated with higher average monthly Medicare expenditures, we analyzed which components of spending were responsible for this difference. We again used the risk-adjusted difference-in-differences methodology, as previously described, to account for observable and unobservable characteristics other than CNO participation that might have contributed to differences in spending. The first step was to decompose total Medicare spending into covered an non-CNO services. These results are presented in Table 4.

For CNO-covered services, the estimates show that average monthly Medicare expenditures in the treatment group increased substantially faster than in the population, and did so steadily over time. The effect is more marked at the LAH/BNP and VNSNY sites, where this difference was statistically significant in every single year and of much higher magnitude than at the two other sites. However, even at Carle and Carondelet, treatment group expenditure increased significantly faster than that for the population in 3 of the 5 years. These results are consistent with those from the analysis of randomized treatments and controls: payment rates for the CNO sites were set too high to achieve budget neutrality.

By contrast, the results for non-CNO services do not match up with the findings from the analysis of the randomized portion of the demonstration. When comparing CNO applicants randomly assigned to treatment or control status, we found that average spending for those services was higher for the treatment group at one site, lower at another, and approximately the same at the other two. In contrast, Table 4 shows that by 1999 non-CNO spending had increased disproportionately and statistically significantly for CNO participants at three of the four sites.

To shed further light on this discrepancy, we examined spending for three major components of non-CNO spending: inpatient care, emergency room visits, and physician office visits (Table 5). These results suggest that the more rapid expenditure increase in the treatment group can mainly be attributed to hospital services. Average spending for hospital services increased substantially more in the treatment group, with the largest increases at LAH/BNP and VNSNY. In addition, expenditure for emergency room visits increased disproportionately for the treatment group at all sites, whereas changes in spending for physician office visits showed no differential pattern, though these results are not significant in all years and at all sites.

Table 5

Estimated Differences Between the Community Nursing Organization (CNO) Treatment Group and the Population Reference Group: Changes in Per Month Spending, Relative to 1994

Year	Carle1	Carondelet2	LAH/BNP3	VNSNY4
Inpatient Hospitalization
1995	$3	$5	**55	*$82
1996	2	-15	**58	**104
1997	12	12	42	***132
1998	0	5	42	***175
1999	*4	14	**104	***193
Emergency Room
1995	0.28	0.01	0.11	0.42
1996	*0.44	0.17	0.13	0.45
1997	0.12	0.15	0.0	*0.4
1998	**0.44	0.13	0.04	0.03
1999	**0.3	0.19	0.0	0.2
Physician Office Visits
1995	*-0.60	1.04	0.63	-1.18
1996	**0.72	-0.42	0.15	0.94
1997	0.4	0.43	*0.82	0.8
1998	0.32	0.03	0.51	-0.42
1999	0.3	0.83	-0.78	-0.96

p<0.1.

p<0.05.

p<0.01.

Operated by a for-profit, private physician group in rural Illinois and served a population with relatively unfamiliar with managed care.

Run by a Catholic, non-profit corporation in southern Arizona and served a population with a relatively high degree of seasonal migration.

The Living at Home/Block Nurse Program was a community-based initiative in Minnesota.

The Visiting Nurse Service of New York was operated by the largest non-profit Medicare-certified home health agency in the Nation and served enrollees who were older, sicker, in need of psychological services, and more resistant to managed care.

NOTES: All dollar amounts were converted to 1999 constant dollars. All differences are adjusted for beneficiary risk and year effects. The CNO treatment group consists of all those randomized to treatment or directly enrolled in the CNO. Note that CNO-covered and non-CNO components were estimated in separate statistical models, they do not necessarily add up to the differences in total spending.

SOURCE: Abt Associates Inc.: Data from the 1999 Community Nursing Organization Evaluation Analytic Files.

Conclusion and Policy Discussion

In summary, the results suggest that the CNO as a model of care provision was associated with increased cost to the Medicare Program compared with traditional FFS Medicare payment. This conclusion is based on very robust findings that were consistent across several analytic approaches. The differences persisted after the application of increasingly complex risk-adjustment methods so one can be confident that they were not due to baseline differences between the treatment group and the traditional Medicare population reference group or control group. Moreover, these differences increased over time and were robust to changes in the way CNO participation was defined. Lastly, according to analysis of randomized beneficiaries not included in this article, there is no statistically significant evidence of increase in physical or social functioning of the treatment group, as compared with the control group (Abt Associates Inc., 1998; 2000).

If one were to interpret the results comparing the treatment group to the population reference group without the context provided by the findings from the randomized portion of the demonstration, the conclusion would be unambiguous: Care for enrollees was shifted to the inpatient setting, and the magnitude of this shift increased over time. This was particularly true at the LAH/BNP and VNSNY sites, and to a lesser degree at Carle, and is consistent with the incentive that the CNO demonstration created: In any payment arrangement in which the provider bears the burden of the marginal cost of selected areas of care there is a strong incentive to avoid cost by shifting the provision of care away from those areas. Because the CNOs would not have to provide any care to a beneficiary while hospitalized, but would still collect the monthly payments, hospital care would have been a particularly appealing option for cost shifting. Our findings are consistent with this cost-shifting incentive.

The main challenge to this interpretation is the fact that a comparison of average spending of demonstration applicants randomized to treatment or control status did not show consistent differences in non-CNO or hospital spending. Two explanations for this discrepancy can be hypothesized. First, the randomized design of the demonstration was not maintained throughout the study period. In particular, in the final years of the demonstration, control group members were permitted to receive the benefits available to the treatment group. This contamination of the control group during the demonstration reduces the minimum detectable difference between treatment and control group members. Hence, if there were a small and consistent difference in non-CNO or hospital spending, perhaps it was not detectable due the breakdown in the randomized experimental design.

The second hypothesis is that the shifting of care to the inpatient sector was a provider effect rather than an incentive effect. In other words, staff at LAH/BNP and VNSNY, and to a lesser degree at Carle and Carondelet, were more inclined to send beneficiaries to the hospital than the average health care provider for the Medicare population. They would do so regardless of treatment or control status, as this was just their prevailing pattern of care provision. This is not meant to imply that those additional hospitalizations were strictly unnecessary, but only that a beneficiary with contact with a CNO was more likely to be hospitalized with a marginal indication. The provider effect hypothesis is also consistent with the temporal pattern observed in the data. During the demonstration period, there was a strong secular trend in the Medicare population toward fewer hospitalizations and shorter hospital stays, resulting in substantial shifting from the inpatient to the outpatient sector. If site staff maintained their established practice patterns despite this trend, their clientele would increasingly have higher hospital spending than the population.

Regardless of whether they affected provider decisions, the financial incentives built into the design of the CNO demonstration are the same as would accompany any partially capitated Medicare demonstration or initiative. Shifting of costs to the subset of benefits not covered under capitation benefits the organization receiving payment and increases costs to the Medicare Program. Unfortunately, the primary financial tool available to Medicare, the capitated payment rate, is ill-suited to address this problem. Decreasing capitated payment in order to recover some of the cost only increases the cost-shifting incentive. If the Medicare Program continues toward increased privatization of risk it would be advantageous, in terms of financial incentives, to require private organizations to provide all (or as many as feasible) Medicare benefits rather than to divide benefits among different entities. Only when all benefits (or, at least, all those that could be substituted for one another) are bundled together can the capitated payment induce cost containment without cost shifting.