Adjusted clinical groups: predictive accuracy for Medicaid enrollees in three states.

Actuarial split-sample method were used to assess predictive accuracy of adjusted clinical groups (ACGs) for Medicaid enrollees in Georgia, Mississippi (lagging in managed care penetration), and California. Accuracy for two non-random groups--high-cost and located in urban poor areas--was assessed. Measures for random groups were derived with and without short-term enrollees to assess the effect of turnover on predictive accuracy. ACGs improved predictive accuracy for high-cost conditions in all States, but did so only for those in Georgia's poorest urban areas. Higher and more unpredictable expenses of short-term enrollees moderated the predictive power of ACGs. This limitation was significant in Mississippi due in part, to that State's very high proportion of short-term enrollees.

Introduction

While States continue to expand their use of Medicaid managed care, the percent of States' enrollees in capitated arrangements varies from under 3 to 80-100 percent across States (Holahan, Rangarajan, and Schirmer, 1998). Medicaid managed care has helped States to manage and lower program outlays and yet, there are continuing concerns. Low provider participation, lack of provider and patient education and certain aspects of Medicaid populations have made translation of private into Medicaid managed care sometimes difficult. Due to low capitation rates relative to costs (McCue et al., 1999) the interest of commercial plans in Medicaid may be waning (Felt-Lisk, 1999; Forbes et al., 1998).

As States become purchasers under managed care, the issue of risk selection arises. Given one premium, plans have an incentive to encourage enrollment of better health risks and discourage enrollment of worse ones, thereby competing on risk selection, not price. Access, especially for those most ill can become compromised. Diagnosis-based risk adjustment has the potential to alter these incentives. Some States (e.g., Maryland and Minnesota) use ACGs in their Medicaid Programs while others use the Disability Payment System (e.g., Colorado and Oregon) or Chronic Illness and Disability Payment System (CDPS) (Delaware and Michigan) and more are in the planning stages (Kronick et al., 2000).

The purpose of this study is to assess the predictive accuracy of risk adjustment within Medicaid welfare and poverty-related populations using the ACGs system, Version 4.0, developed at Johns Hopkins University. This study adds to information on the prediction errors for these systems (Kronick et al., 2000) by examining predictive accuracy for ACGs within California, Georgia, and Mississippi. We also examine how the predictive accuracy of the ACG system is affected by short-term enrollees and potential selection based on urban location within Medicaid populations.

We chose to look at Georgia and Mississippi as southern States that have not successfully implemented capitated Medicaid managed care and that lag behind in private health maintenance organization (HMO) penetration. They serve poorer, perhaps sicker, Medicaid enrollees in generally more rural environments that are perhaps less amenable to capitation. California, by contrast, leads the country in private managed care and is currently expanding Medi-Cal managed care using a two-plan model. In this model, Medi-Cal enrollees in 12 counties across the State have a choice of a local initiative versus private commercial plan(s); the local network is required to retain safety-net providers. California is considering the implementation of risk adjustment based on ACGs (California Department of Health Services, 2002); a recent report noted that risk adjustment could address potential risk selection within the two-plan model (Adams, Bronstein, and Becker, 2000).

Risk-adjustment methods should improve the accuracy of predicted costs. Traditional actuarial methods are used here to examine the predictive accuracy of ACGs for random groups among the Medicaid and privately insured and for non-random groups within Medicaid. Including private sector data allows for comparison to earlier studies of risk adjustment (Dunn et al., 1996) and places the analysis within the context of the study States' broader insurance pool.

Background Studies

As States seek to induce commercial plans to integrate Medicaid enrollees, a key question is whether Medicaid enrollees pose a different insurance risk than privately insured. A study of staff-model HMOs reported an initial increase in utilization for Medicaid beneficiaries, but no permanent increase (Hart et al., 1997). Another found costs per Aid to Families with Dependent Children (AFDC) and poverty-related Medicaid enrollee 23 percent higher than for the commercially insured after enrollment in (nine) HMOs (Welch and Wade, 1995). The present study is part of a larger one that examined relative use and costs of Medicaid versus private insured (Adams et al., 2001). Those results indicated that Medicaid enrollees cost more based on price-standardized measures and these higher costs reflected either higher case mix or higher use of certain services (e.g., inpatient days) or both, across the study States. If States do not risk adjust, plans will have an incentive to enroll healthier Medicaid enrollees, avoid Medicaid markets, or underserve Medicaid beneficiaries.

Risk adjustment of rates can help address these issues if they improve the accuracy of rates paid to plans in relation to expenses. A study of children under age 13 found risk adjusters (e.g., health status, prior use) increased predictive ability from 5 percent with age and sex to as much as 70 percent (Newhouse et al., 1993). Another study found ACGs increased the ratio of payments to expected expenses from less than 25 to almost 70 percent for Medicaid children with chronic health conditions (Fowler and Anderson, 1996).

A comprehensive study (Dunn et al., 1996) of existing risk-adjustment methods based on privately insured, compared ACGs with others using prospective and retrospective models. Prospective risk adjustment, the most prevalent in research and practice, is modeled by developing risk-adjusted payments from one year of data to predict actual expenses during the next year. Retrospective or concurrent risk adjustment is modeled by developing adjusted payments from one-half of a sample to predict actual expenses for the other one-half of the sample in the same year. All risk-adjustment methods studied outperformed age and sex alone and relative to no risk adjustment, would reduce, but not remove, incentives for risk selection (Dunn et al., 1996). While retrospective models have generally been used for payment, the high turnover characteristic of State Medicaid welfare and poverty-related groups may justify the use of concurrent models for profiling or making retrospective adjustments to payments.

The study by Dunn et al. (1996) did not include Medicaid data. One study of disabled Medicaid enrollees found the DPS markedly improved on age and sex models (Kronick et al., 1996). Researchers of the Maryland ACG-based system also noted significant improvement with ACGs, but the final payment system made special adjustments for pregnant females, short-term enrollees, those with rare and expensive diseases, and those residing in the city of Baltimore (Weiner et al., 1998). The study by Kronick et al. (2000) adds significant new information for Medicaid populations. This work shows that the CDPS improves on the DPS and ACG systems, but not the ambulatory diagnostic groups (ADGs), in terms of predictive accuracy for AFDC adults and children. As they note, information on the usefulness of diagnostic classification systems for these enrollment groups has been less readily available.

The present analysis adds to our understanding by using data on both Medicaid and privately insured and examining predictive accuracy for welfare and poverty-related eligibility groups within three States for one diagnostic classification system. In Georgia and Mississippi, where capitated managed care remains low, plans will likely compare Medicaid with private insurance risks as they expand clientele. We consider (ACGs) for both Medicaid and private insured and within Medicaid, for those with high-cost conditions or living in poorer urban areas; both groups could be potentially selected against. Finally, we consider how short-term enrollees, or turnover, might affect the use of risk adjustment. Earlier work by Adams et al. (2000) found short-term enrollees more costly on a monthly basis.

Data and Methods

Data for our analysis are drawn from the Medicaid enrollment and claims data for 1994 for Georgia, Mississippi, and California. Data for Georgia and Mississippi were raw files from the States while the California data were from the State Medicaid Research Files (SMRF) maintained by CMS. All 1994 enrollees in Georgia and Mississippi were included. Due to the size of California's Medi-Cal files data, enrollees in seven urban and rural counties were used: Alameda, Los Angeles, San Francisco, San Diego, Humboldt, Butte, and Tulare. Enrollment in these counties represent approximately 58 percent of total State enrollees.

The Medicaid enrollment files contained monthly eligibility, raw eligibility codes, and age and sex. In a broader study by Adams et al. (2001), eligibility was collapsed into four groups: (1) AFDC (now Temporary Assistance to Needy Families [TANF]) or welfare-related groups; (2) poverty-related or expansion groups; (3) disabled; and (4) foster care children. Medically needy in the study States with this program (Georgia and California) were included in their respective categorical eligibility group. We omitted those age 65 or over since they are often not included in States' Medicaid managed care Programs and we compared Medicaid to private working age populations in parts of the study. Data on detailed diagnoses, procedure codes, and amounts paid were drawn from the claims files for each enrollee.

Data for the privately insured are from the 1994 Medstat MarketScan data. The MarketScan data are a non-random sample of large, self-insured employers nationwide and include claims for more than 7 million individuals and dependents. These individuals work in a range of occupational settings (e.g., manufacturing, retail, finance, insurance, and government) including some wage levels ($8 in retail) more comparable to Medicaid populations. Individuals are generally in firms that self-insure; this type of coverage applies to almost one-half (48.9 percent) of workers according to the Medical Expenditure Panel Survey (MEPS). MarketScan claims data also include detailed diagnoses, procedure codes, and amounts paid (inclusive of copay and deductibles) which allow us to complete comparable analyses on the publicly and privately insured, a comparison often lacking in the literature. A separate file on covered lives by age and sex was used to identify the number of non-users; dummy records were created for these individuals.

We exclude those enrolled in a capitated plan since full claims would not be available for them. We also omit outpatient prescription drug claims. While drug costs are increasing rapidly, differences across Medicaid and privately insured would largely reflect coverage differences. Furthermore, the Medstat MarketScan system does not contain drug data for all covered lives. Finally, because drug claims lack diagnoses, their omission does not affect ACG assignment. However, their omission could have a small effect on the comparative predictive analysis if the differences in average costs across ACGs varies without, versus with, drug costs in a different manner for the two insured groups. We also omit payments for durable medical equipment, direct reimbursements to providers for case management services and nursing home claims. Overall sample sizes for each study State are shown in Table 1.

Table 1

Background Data on Study States and Sample Sizes: 1994

State	Population			Overall Medicaid Program			Study Enrollee Numbers
							Private Insured	Medicaid1
								Welfare and Poverty Related
	Total1	Percent		Enrollees1	Dollars Paid	Percent Capitated Managed Care		6 Months or Less	6 Months or More
		Poor	Capitated Managed Care
California	31,617	17	45	6,778	$2,686	17	188,506	2,584	550
Georgia	7,138	15	16	1,170	2,788	2	147,775	1,040	152
Missippippi	2,600	25	3	570	2,582	1	27,700	228	199

Number in millions.

SOURCE: Adams, E.K., Emory University, 2002.

California, a State known for its efficiencies in the Medi-Cal Program (Zuckerman et al., 1998) spends less per enrollee ($2,686) than Georgia ($2,788), but more than Mississippi ($2,582). Approximately 45 percent of California's total population is in capitated managed care, far higher than Georgia (16 percent) or Mississippi (3 percent). Both Georgia and Mississippi have virtually no capitated MMC while California, as noted, is implementing the two-plan model. They are currently considering the implementation of ACGs (California Department of Health Services, 2002). We also note, as shown in the Medicaid Enrollment column in Table 1, the portion of those enrolled for less than 6 months is far greater in Mississippi than in the other two study States. The majority of our analysis uses data for those enrolled in Medicaid for 6 months or more.

Methods

We use ACGs, Version 4.0 to assess relative health risks of the Medicaid and privately insured. This system clusters the diagnoses (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM]) recorded on claims for inpatient and ambulatory care observed for an individual over a given time period (e.g., a year). The system first assigns claims for individuals to 1 or more of 34 ADGs. The system then categorizes individuals into 1 of 52 mutually exclusive risk groups (ACGs) based on their observed health problems, age, and sex. The value of the ACG system is that it identifies categories of individuals that are relatively homogeneous within ACG and relatively distinct across them in terms of expected health care utilization and expense (Weiner et al., 1998; 1991). The ACG system is built on a hierarchy of all problems for which an individual seeks care. For example, a person with one or more minor acute conditions (such as minor infection) is assigned to one ACG, while a person with multiple serious conditions (such as diabetes or asthma) is assigned to another. Non-users of any services during the time observed are assigned to their own ACG.

Actual per member per month expenditures are calculated by dividing total health care expenditures by number of months enrolled (this is the measure of resource used throughout the article). Monthly enrollment is available in Medicaid data. For the privately insured we estimated the member months using MarketScan data on mean length of enrollment by age and sex for all covered lives. While we found little variation in the number of months enrolled by age and sex groups this method may understate or overstate months enrolled for a particular individual.

We use one year of data to test the predictive ability of ACGs with a concurrent and retrospective model. It is known that these models would not perform as well if applied prospectively, but as previously noted, they may be particularly useful for the Medicaid population. We use the split-sample actuarial technique (Dunn et al., 1996; Fowler et al., 1996; Ellis et al., 1996) that first splits the sample into two random groups. Fifty percent of the estimation sample is used to estimate the relationship of age, sex, and ACG groups to per member per month expenditures. The remaining 50 percent, the prediction sample, is used to generate random samples (50 groups of 2,500 each) of enrollees for whom expenditures are predicted using the regression coefficients. We sampled with replacement. We chose the group size of 2,500 following Dunn et al. (1996); while predictive accuracy will increase with the size of the group they found this effect levels off beyond a group size of 2,000 to 3,000 individuals. Given that outliers can affect predictive accuracy, we tested with and without truncation (taking into account the similarity of results, we present results without truncation).

We used two sets of predictor variables: (1) age and sex and (2) age, sex, and ACG. The difference in accuracy gained with the inclusion of ACGs is our focus. The models were fit using data for only welfare (then AFDC) and poverty-related enrollees who are enrolled 6 months or more. We have fuller claims experience for these long-term enrollees and hence, observation of diagnoses, their ACG assignment, and measurement of resource use is more reliable; most applications of the ACG system use resource weights based on this long-term enrollee sample (Weiner, 1998). We do, however, provide one test of predictive accuracy for those enrolled less than 6 months by using an ACG assignment based on their shorter observation period.

We focused on the welfare and poverty-related groups as those most States enroll in managed care first. These groups are also more comparable with the privately insured which, by and large, does not include elderly or disabled. Even though the disabled Medicaid populations are high cost and perhaps amenable to cost savings, States did not initially mandate their participation (Holahan, 1999). Still, this landscape is rapidly changing. Regenstein and Schroer (1998) note that approximately one in four non-elderly disabled are enrolled in managed care, the majority of which are capitated arrangements although certain services—behavioral health, pharmacy, dental, hospice and long term care—are often carved out. However, other researchers have thoroughly addressed the issue of risk adjusters for the Medicaid disabled and we were interested in the effect of turnover, more characteristic of the welfare and poverty-related groups, on risk adjustment.

There are two types of resource measures used in parts of this analysis. These are: (1) State-specific Medicaid and private-specific expected resource measures (per member per month) by age, sex and ACG, and (2) pooled Medicaid and private relative resource weights (based on per member per month) by ACG within each State. The first is used to gauge predictive accuracy while the latter is used to derive a case-mix index.

The resource measures for the predictive accuracy analysis are derived from the estimation random sample using regression analysis. Our use of a simple regression model to derive expected resource use follows earlier work (Dunn et al., 1996; Fowler and Anderson, 1996; Ellis et al., 1996) and is the equivalent of taking mean actual expenditures for each (mutually exclusive) risk group. This is akin to what States do to set capitated rates unless they were willing to use averages by ACG from other States as suggested by Kronick et al. (1996).

The first resource measure, predicted per member per month expenditures, is the predicted value from the regressions. This predicted value becomes the simulated payment for individuals based on their specific age, sex and ACG category. For example, two females in the age group 35-44 in Georgia Medicaid, assume one is not pregnant, but has asthma (ACG = 700) and the other has an uncomplicated pregnancy (ACG = 1710). Their predicted expense and payment would be based on the same age and sex coefficient, but a different one for their specific ACG; the predicted per member per month for the first in Georgia Medicaid was $23.28 and for the second, $197.29. (The full set of regression results is available on request from the authors.)

The second resource measure—a set of resource weights—is used to derive a summary case-mix index for private and Medicaid insured within each State. These indices are created by first using average resource measures by ACG to create the relative weights. These weights are the simple mean expenditures per member per month within each ACG divided by the grand mean (per member per month) across all ACGs; these are based on pooled Medicaid and private sample data in each State. To derive the case-mix index, these weights are multiplied by the number of cases in each ACG within the insured group, summed and divided by the total number of persons in the insured group, as follows:

Where n = the number of cases in the ith ACG. Note that the ACG system assigns non-users to their own ACG and hence, non-users are included in the total N. An insured group's index will be higher if there are more enrollees in the ACGs with relatively higher weights. Given the sensitivity of these weights to geographic variation in coding, payment, and completeness of data we also estimated the weights (and indices) based on pooled (across the three States) Medicaid and, in turn, privately insured data.

To assess the issue of turnover in the Medicaid population we repeat the predictive accuracy analysis in several ways. Since assignment of ACG for enrollees in the Medicaid system for less than 6 months is less reliable we first use only age and sex regression coefficients to predict per member per month expenses for them. We test age and sex coefficients derived from the greater than 6-month sample and in turn, coefficients by age and sex specific to less than 6-month enrollees. We then test the use of ACG assignments for those enrolled less than 6 months, but use age and sex and ACG coefficients derived from those enrolled more than 6 months to predict their expected resource use. In each instance, we derive summary measures for random groups including both short- and long-term enrollees.

While the regression R2 provides a summary measure of the predictive power (addition of the ACGs increased it from around 0.01 to 0.35 for the privately insured and from 0.04 to 0.32 for Medicaid insured, across the study States) for individuals, it can be affected by a small number of cases with large errors. Further, it does not provide measures of typical error size or summary measures across plans. To address these issues we use a set of individual and group measures of predictive accuracy (Dunn et al., 1996).

In general, these measures use the simulated payments for individuals (based on the regression coefficients) and compare these with their actual expenditures. Comparisons are summarized using a: (1) mean absolute prediction error, (2) mean absolute percentage prediction error, and (3) predictive ratio. The mean absolute prediction error averages (across the 50 groups) the absolute difference between mean actual and mean predicted expenditures for individuals in the group. The percentage error is this prediction error reported as a percentage of the predicted value, and averaged. The predictive ratio is the ratio of the total sum of predicted dollars for individuals in a group to the sum of their actual dollar expenses, averaged across the 50 groups. As such, it indicates whether the average plan will receive more or less in total payments than expenses incurred.

We use census data for analysis of potential risk selection based on Medicaid enrollee's neighborhood. Distressed neighborhoods (Kasarda, 1993) simultaneously exhibit disproportionately high levels of poverty (20 percent or more), joblessness, female-headed families and welfare receipt; severely distressed neighborhoods have 40 percent or more in poverty, these characteristics, plus high teenage dropout rates (Kasarda, 1993). To approximate such neighborhoods, we identified enrollees in the predictive sample in urban counties and ZIP Codes characterized by the poverty cut offs. We omit Mississippi from this analysis since it is predominantly rural and, therefore, the urban population from which to draw repeated samples was small. Further, plans could not easily select on this basis since the majority of Mississippi ZIP Codes met the 20 percent poverty level cut-off.

Results

One of the key questions for plans entering the Medicaid market is whether the expected resource use of the poorer populations enrolled in Medicaid managed care, generally the younger females and children in welfare and poverty-related eligibility groups, differ from those seen for similar age and sex groups in the private sector. To further describe the samples and the differences across the two insured groups, we show data by age, sex, and pregnancy status. We also include the mean total expenditures for each of these groups (Table 2).

Table 2

Mean Total Expeditures per Enrollee, by Sex and Age in Study States: 1994

State	Private			Medicaid
	N	Percent	Mean	N	Percent	Mean
California
Males	85,093	45.2	$2,046.26	812,548	39.9	$367.51
0-17 Years	21,386	25.1	1,153.60	665,715	81.9	316.80
18-34 Years	16,805	19.7	1,329.70	83,584	10.3	400.62
35-44 Years	16,590	19.5	1,703.36	38,952	4.8	647.26
45-54 Years	15,400	18.1	2,461.70	18,642	2.3	885.78
55-64 Years	14,912	17.5	4,086.45	5,656	0.7	1,262.08
Females	103,365	54.8	2,216.30	1,222,503	60.1	672.66
0-17 Years	20,136	19.5	1,047.15	655,909	53.7	317.53
18-34 Years	23,739	23.0	2,052.35	393,074	32.2	1,402.94
35-44 Years	22,481	21.7	2,325.61	131,024	10.7	1,047.16
45-54 Years	20,254	19.6	2,571.56	35,589	2.9	975.32
55-64 Years	16,755	16.2	3,277.55	6,907	0.6	1,181.49
Pregnant	4,929	4.8	5,833.07	144,753	11.8	3,660.24
Georgia
Males	67,488	45.7	1,596.95	328,206	36.9	754.62
0-17 Years	19,370	28.7	994.58	313,135	95.4	748.85
18-34 Years	12,401	18.4	1,115.29	10,031	3.1	643.22
35-44 Years	13,425	19.9	1,484.56	3,353	1.0	1,279.95
45-54 Years	13,604	20.2	2,021.60	1,416	0.4	1,494.50
55-64 Years	8,688	12.9	3,136.19	271	0.1	1,184.58
Females	80,223	54.3	1,665.99	560,607	63.1	1,281.22
0-17 Years	17,664	22.0	762.81	320,939	57.2	753.17
18-34 Years	18,294	22.8	1,647.17	196,467	35.0	2,096.13
35-44 Years	18,761	23.4	1,819.55	36,039	6.4	1,520.30
45-54 Years	18,993	23.7	2,005.93	6,261	1.1	1,387.68
55-64 Years	9,511	11.9	2,505.06	901	0.2	1,379.49
Pregnant	3,014	3.8	5,949.27	87,500	15.6	4,481.79
Mississippi
Males	12,204	44.1	1,329.68	84,949	37.3	367.51
0-17 Years	3,442	28.2	666.24	82,580	97.2	362.07
18-34 Years	2,105	17.2	633.89	1,838	2.2	460.50
35-44 Years	2,205	18.1	1,347.18	381	0.4	768.65
45-54 Years	2,763	22.6	1,590.58	116	0.1	1,111.59
55-64 Years	1,689	13.8	3,099.18	34	0.0	1,506.03
Females	15,488	55.9	1,358.68	142,797	62.7	672.66
0-17 Years	3,327	21.5	634.32	90,816	63.6	368.85
18-34 Years	3,025	19.5	1,208.05	40,897	28.6	1,271.74
35-44 Years	3,761	24.3	1,549.25	8,836	6.2	945.46
45-54 Years	3,153	20.4	1,572.12	1,868	1.3	947.09
55-64 Years	2,222	14.3	2,022.94	380	0.3	1,110.12
Pregnant	398	2.6	4,478.27	17,760	12.4	2,631.90

NOTE: Expenditures are mean annual expenses for all inpatient and outpatient services a for those enrolled 6 months or more.

SOURCE: Adams, E.K., Emory University, 2002.

These data show many of the expected patterns. Medicaid enrollees are far more likely to be younger, female, and to have higher pregnancy rates than privately insured. Most of the males enrolled in Medicaid are children (age 0-17), whereas males in the privately insured sample are distributed fairly evenly across the age groups (Table 2). Of the study State's Medicaid enrollees 60–65 percent are female as opposed to the 55 percent that are privately insured. Only 3-5 percent of the privately insured females in our samples have a pregnancy diagnoses during the year. This is in stark contrast to the percentages seen in the Medicaid sample. In our study States. 17-24 percent of the welfare-related females have pregnancy diagnoses, compared with 8-11 percent of the poverty-related groups.

While Table 2 provides insight on the basic differences between Medicaid and the privately insured, it is difficult to summarize what these mean for relative health risks. One way to measure and compare health risks across these groups is the ACG-based case-mix index (Table 3).

Table 3

Comparison of Privately and Publicly Insured Groups, by Study States: 1994

State	Privately Insured	Medicaid Enrollees1
		All	Welfare and Poverty Related
California	1.19	0.99	0.97
Georgia	0.69	1.05	1.02
Mississippi	0.69	0.94	1.02

Enrolled 6 months or more.

NOTE: Indices are derived based on relative resource weights by adjusted clinical group.

SOURCE: Adams, E.K., Emory University, 2002.

These indices indicate the Medicaid welfare and poverty-related groups exhibit a higher case mix than their privately insured counterparts in Georgia and Mississippi, but not California. The latter is surprising, but may reflect the very high penetration of managed care in that State. That is, those privately insured and not in an HMO in California may be disproportionately less healthy, avoiding managed care. While the Medstat data are not necessarily representative of all privately insured, a large portion of the privately insured in California that are not in capitated managed care are likely in the types of large, self-insured employer plans sampled by Medstat.

As noted, the use of State-specific pooled weights might affect these case-mix indices. We therefore re-estimated them using relative weights derived by pooling data across the three States' private (and separately, Medicaid) samples. Using these relative weights, the California private index was higher (1.20) than Georgia's (.78) or Mississippi's (0.73) while their Medicaid index lie in between (0.98 versus 1.16 in Georgia and 0.88 in Mississippi) (Adams, Bronstein, and Becker, 2000). This suggests our California sample of privately insured has a higher case mix than their privately insured counterparts in Georgia or Mississippi, consistent with our hypothesis. California's Medi-Cal has a slightly higher case-mix index than Mississippi. These findings indicate that State variation in the penetration of private sector managed care may be a factor to consider when making such comparisons and/or as States seek to expand Medicaid managed care.

One way to gauge the predictive power of ACGs is to simulate their use in setting per member per month rates for random groups. Table 4 results indicate that the use of ACGs improves predictive power, but risk adjustment based on ACGs was more effective for our privately versus Medicaid insured sample.

Table 4

Summary of per Member per Month Predictive Accuracy—Random Groups, by Study States: 1994

Group	California		Georgia		Mississippi
	Private	Medicaid	Private	Medicaid	Private	Medicaid
	Mean Absolute Error (Mean Percent Error)	Mean Absolute Error (Mean Percent Error)	Mean Absolute Error (Mean Percent Error)	Mean Absolute Error (Mean Percent Error)	Mean Absolute Error (Mean Percent Error)	Mean Absolute Error (Mean Percent Error)
Age, Sex	$15 (6.7)	$4 (6.8)	$13 (8.7)	$4 (3.5)	$9 (7.1)	$4 (7.0)
Age, Sex, and ACG	13 (5.9)	4 (6.5)	9 (6.1)	3 (3.3)	8 (5.8)	4 (6.1)
			Predictive Ratios
Age, Sex	0.97	0.99	1.04	1.00	1.00	1.02
Age, Sex, and ACG	1.00	0.99	1.01	1.00	0.97	1.01

NOTES: Measures are derived for 50 random groups of 2,500 each sampled with replacement from the predictive one-half of the privately insured and Medicaid samples (enrolled 6 months or more) in each State. ACG is adjusted clinical group.

SOURCE: Adams, E.K., Emory University, 2002.

For the random groups all predictive ratios are generally close to one, indicating that the per member per month capitated rates (or predicted amounts) would be very close to actual expenses across a large number of plans (50) to which enrollees are randomly assigned. This simply illustrates that individual prediction errors tend to cancel out in large groups. That is, even simple models using age and sex predict relatively well when individuals are randomly enrolled across plans.

In reality, individuals are not distributed randomly and sicker enrollees may concentrate in a few plans. We assess the predictive accuracy of ACGs in light of potential selection processes or non-random enrollment of Medicaid enrollees across plans: high-cost cases, and those residing in poorer urban neighborhoods.

The high-cost, chronic conditions included in the analysis are: cancer, heart disease, HIV, diabetes, sickle cell anemia, and congenital anomalies. (A full list of the of the ICD-9-M codes used to define these high-cost conditions is available on request from the authors.) The per member per month expenses for those welfare and poverty-related with any of the high-cost conditions range from $279 in Mississippi to $335 in California, and $387 in Georgia. These ranged from almost four times the per member per month for welfare and poverty-related enrollees overall in Georgia, to almost six times as much in California.

Risk-adjusted payments for 50 groups, randomly selected from the total pool of individuals with high-cost conditions, result in far greater accuracy than payments based on age and sex alone (Table 5). In each State the mean absolute error is reduced by more than $100 per member per month and mean absolute percent error is at least cut in one-half. While reinsurance would address the issue of unpredictable, high-cost cases, it would not address the additional, predictable expenses (less than a $50,000-cap) associated with these conditions. Of those with high-cost conditions, less than 1 percent had annual expenses more than $50,000 in any study State. Results based on truncated data (more than $50,000) still show a marked improvement in predictive power when risk adjusters are included.

Table 5

Summary of per Member per Month Predictive Accuracy—Non-Random, High-Cost Medicaid Groups, by Study States: 1994

State	Mean Absolute Error	Mean Absolute Percent Error	Predictive Ratio
California
Age, Sex	$259	77.10	0.23
Age, Sex, and ACG	119	35.30	0.65
Georgia
Age, Sex	273	69.40	0.30
Age, Sex, and ACG	100	24.60	0.74
Mississippi
Age, Sex	201	72.10	0.28
Age, Sex, and ACG	92	35.00	0.67

NOTES: ACG is adjusted clinical group. Measures are derived for 50 samples of 2,500 each drawn from the predictive one-half of the Medicaid sample enrolled 6 months or more.

SOURCE: Adams, E.K., Emory University, 2002.

The predictive ratios (Table 5) across the 50 plans average from 0.23 in California to 0.30 in Georgia for those with high-cost conditions when payments are based on age and sex alone. While they improve markedly with ACGs, the predictive ratios still indicate underpayment, averaging from 0.65 in California to 0.74 in Georgia.

Other characteristics could also lead to selection or non-random distribution of enrollees across plans. Geographic characteristics, such as location in poorer neighborhoods, for example, could be readily used to select out certain enrollees thought to be higher users. Geographic location may also correlate with access problems such as the lack of office-based physician practices physically located there (Goldstein, 1994). Some States have adjusted their rates to reflect unmeasured characteristics that lead to higher costs in these areas (Weiner, 1998). These two characteristics may work together if, for example, an urban teaching hospital located in a poor neighborhood is either part of a Medicaid managed care network or is forming its own, as in the California two-plan model. If premiums for providers specializing in care to low-income areas are not risk adjusted, they may find themselves at a financial disadvantage.

In Table 6 we show the mean absolute and mean absolute percent error for 50 groups of 2,500 drawn from the poorest urban ZIP Codes in California and Georgia. We note that these analyses are based on coefficients from an urban subsample of the estimation sample.

Table 6

Summary of per Member per Month Predictive Accuracy—Non-Random, High-Poverty Medicaid Groups, by Study States: 1994

State	Mean Absolute Error	Mean Absolute Percent Error	Predictive Ratio
California1
Age, Sex	$7	10	1.01
Age, Sex, and ACG	6	9	1.01
Georgia1
Age, Sex	5	5	1.01
Age, Sex, and ACG	5	5	1.05
California2
Age, Sex	7	12	1.07
Age, Sex, and ACG	7	11	0.91
Georgia2
Age, Sex	6	6	0.95
Age, Sex, and ACG	3	3	1.02

Enrollees in urban ZIP Code residence that have 20 percent or more of the population under 100 percent of Federal poverty level.

Enrollees in urban ZIP Code residence that have 40 percent or more of the population under 100 percent of Federal poverty level.

NOTES: ACG is adjusted clinical group. Measures are derived for 50 samples of 2,500 each drawn from the predictive one-half of the Medicaid sample enrolled 6 months or more.

SOURCE: Adams, E.K., Emory University, 2002.

First, the mean absolute and percentage error is somewhat higher for these non-random groups than for the random groups as shown in Table 4. For those in ZIP Codes with 20 percent or more poverty, the mean absolute error equals $5 in Georgia and $7 in California. There is only a slight improvement in predictive accuracy with the use of ACGs for the California enrollees and no improvement in Georgia. In these areas, actual expenditures are somewhat lower than those predicted by age and sex, but the difference does not appear related to risk status (risk adjustment based on ACGs does not improve predictions).

For enrollees in the 40 percent or more poverty areas, the mean absolute and percentage error, without ACG adjustment is higher which suggests expenses are less predictable for enrollees in these poorest areas. While the results indicate there is not an underpayment issue for the 20 percent or more poverty areas in either State, plans would be paid only 95 percent of expected costs in the poorest areas (40 percent or more) of Georgia without risk adjustment. Given relatively low margins among commercial plans (Rogal and Gauthier, 1998) this could provide enough incentive for them to avoid these areas. Adding an ACG adjustment increases predictive accuracy significantly there. In the higher poverty areas in California, however, expenses are lower than those predicted by age and sex alone, but higher than those predicted when risk adjusted using ACGs. This suggests that this high-poverty group uses more resources than we would expect, given their risk profile. This may relate to unmeasured differences in health risk or higher costs of providers (e.g., public teaching hospitals) located in these areas. Each would call for a different State policy response.

Turnover is a key issue for Medicaid Programs. As enrollees come in and out of the system, program managers and physicians will likely have difficulty providing continuity of care. Turnover can pose a particular problem for managed care and for setting capitated rates since States will not have the necessary diagnostic information to set risk-adjusted rates for short-term enrollees. While concurrent and retrospective rate adjustments may help in this regard, those newly enrolling may need and use more resources than others in the same ACG due to potential postponement of needed medical care. Thus, if short-term or new enrollees account for a large portion of the plans' caseload this can cause financial problems.

Our data indicate short-term enrollment is a major issue in Mississippi. Mean months enrolled for all (short- and long-term) welfare and poverty related enrollees in Mississippi was 6.6 versus 9.3 -9.9 months in the other two States; short-term enrollees accounted for almost one-half (47 percent) of Mississippi's welfare and poverty related enrollees as opposed to 15 percent in Georgia and 21 percent in California. Further, the differences in per member per month were greater in Mississippi. Short-term welfare and poverty-related enrollees cost $179 per member per month (versus $59) for long-term enrollees in Mississippi. In Georgia, short-term enrollees cost $117 per member per month (versus $94) and in California, $141 (versus $58).

Up to this point in the analysis we have included only those enrolled more than 6 months in our tests of predictive accuracy. To address the turnover issue we present the age, sex, and ACG models for random groups of enrollees inclusive of the short-term (less than 6 months) enrollees (Table 7). First, we used age and sex coefficients based on those enrolled 6 months or more for the short-term enrollees; we apply ACG risk adjustment to only the long-term enrollees. The only change in the second bank is that we used age and sex coefficients for the short-term enrollees unique to them, or based only on data for those enrolled less than 6 months. This can be viewed as a blended rate. In the third bank we use the ACG assignment of short-term enrollees (based on their limited observation time) and apply the age, sex, and ACG regression coefficients from the long-term enrollees. Together, these illustrate possible options States might use to adjust payments for individuals for which they have little time to observe utilization and expenses.

Table 7

Summary of per Member per Month Predictive Accuracy for Alternative Models—Random Medicaid Groups, by Study States: 1994

State	Mean Absolute Error1	Mean Absolute Percent Error1	Predictive Ratio1
California2
Age, Sex	$14	18	0.83
Age, Sex, and ACG	14	18	0.83
Georgia
Age, Sex	13	12	1.02
Age, Sex, and ACG	13	12	1.02
Mississippi
Age, Sex	45	30	0.77
Age, Sex, and ACG	45	30	0.77
California3
Age, Sex	8	11	1.03
Age, Sex, and ACG	8	11	1.02
Georgia
Age, Sex	13	12	1.02
Age, Sex, and ACG	13	12	1.02
Mississippi
Age, Sex	45	52	1.30
Age, Sex, and ACG	45	51	1.30
California4
Age, Sex, and ACG	18	23	0.77
Georgia
Age, Sex, and ACG	11	9	0.92
Mississippi
Age, Sex, and ACG	52	40	0.61

No truncation.

Based on 6 months or more.

Unique to less than/more than 6 months.

Based on more than 6 months.

NOTES: ACG is adjusted clinical group. Measures are derived for 50 samples of 2,500 each drawn from the predictive one-half of the Medicaid sample enrolled 6 months or more.

SOURCE: Adams, E.K., Emory University, 2002.

With the inclusion of the short-term enrollees, the absolute mean error and percentage errors in the first bank are markedly higher than those for random groups of long-term enrollees; the mean errors range from $13-$45, compared with mean errors of $3-$4 (Table 4). The highest errors are in Mississippi where the percent error averages 30 percent. The effects on the predictive ratios are also presented in Table 4. In both California and Mississippi, payments based on age and sex coefficients for those enrolled more than 6 months would underpay plans. In California, the predictive ratio is 0.83 and in Mississippi, it equals 0.77. Only in Georgia does the random distribution of short- and long-term enrollees across plans result in a predictive ratio close to one.

We use the age and sex coefficients and simulate payment based on data for short-term enrollees (Table 4). We note that this is the approach taken by Maryland in their adaptation of the ACG system (Weiner, et al., 1998). The results show that the mean and percentage errors fall for California, but not for the other two States; errors are still dramatically higher in Mississippi. The predictive ratio, however, for California is closer to one as is Georgia's. This blended rate approach seems to work well in these States. In Mississippi, however, the mean percentage error is actually higher and the predictive ratio indicates that plans would be over paid. Short-term enrollees in this State apparently have a highly unpredictable expenditure pattern and, as previously noted they also comprise a significantly larger portion of total welfare and poverty related enrollees in this State.

We tested the potential of assigning an ACG to short-term enrollees despite the short observation period and using age, sex, and ACG coefficients and simulated payments from those enrolled for 6 months or more (Table 7). In all three States this results in an underpayment of plans. None of the three approaches tested here appear to be viable for welfare and poverty related groups in Mississippi.

Discussion

Our analysis indicates that ACGs can be useful in addressing potential risk selection under managed care even within the younger, welfare and poverty-related Medicaid enrollee groups who exhibit significant turnover, in Georgia and California. As with most policies, the adaptation of risk adjustment must be State-specific. In Georgia and California, for example, ACG risk adjustment with a blended rate for short- and long-term enrollees appears viable. However, they differ in terms of their ability to use ACGs to account for higher costs seen in urban poor areas. Mississippi's welfare and poverty-related enrollees exhibit such high levels of turnover that they would likely need special policies to accompany any risk-adjustment system.

As with earlier studies, we find the use of ACGs improve Medicaid payment accuracy relative to those based on only age and sex in all three States. While risk adjustment will not fully reduce the incentives of plans to select those without higher-cost chronic conditions, it offers other advantages such as the ability to monitor patterns of enrollment and utilization across plans and payment negotiations. The implementation of risk adjustment, however, requires that States put more effort and incur expenses to obtain reliable, current encounter data on which to monitor and update payment rates and policies.

As previously noted, we found risk adjustment improved accuracy for selection based on location in the poorest urban areas in Georgia only. Difficulty in predicting for these groups may reflect unmeasured differences in health status related to poorer socioeconomic environments. Some States, including Maryland, have used simple adjustments for their most urbanized areas (Baltimore) to “…compensate for what are believed to be unmeasured health status differences…” there (Weiner, 1998). These results may also reflect that access is an issue—provider systems in these inner urban areas are often emergency rooms, safety-net hospitals, and community clinics where the costs of care are higher and continuity of care is harder to assure. This points to the fact that risk adjustment alone cannot be used to address all aspects of payment policy. State policies to assure appropriate use of emergency room and adequate payment of higher-cost safety-net providers need to be coordinated with their risk-adjustment methods.

We did find a lower percent—2 versus 10—of urban Medicaid enrollees in the poorest ZIP Codes in California than Georgia, consistent with findings of limited concentration of the poor in western versus southern cities (Kasarda, 1993). This may make policies to address selection in urban poor areas more important in Georgia, but our inability to predict for enrollees in distressed California neighborhoods is relevant to that State's two-plan model if local initiatives disproportionately serve poorest urban areas. California is now considering the implementation of risk adjustment based on ACGs for these counties.

A challenge for implementing risk adjustment in Medicaid can be turnover in the population. A churning of enrollees in and out of the Medicaid Program is well documented (Carrasquillo et al., 1998). In general, this raises the issue of prospective versus concurrent and retrospective payment adjustment. While States might assume that short-term enrollees are replaced by other short-term enrollees of similar risk and, therefore, that plans' profiles are stable over time, it is the relative distribution of short-term enrollees across plans that matters for payment equity. If new enrollees are distributed disproportionately among plans, this may mean that concurrent and retrospective adjustment is justified and/or needed in Medicaid Programs.

It does appear that States are making special adjustments to their payments to address issues related to short-term enrollees even when using diagnoses-based risk adjustment. Although some States recognize the potentially higher costs of new enrollees (Schwalberg, 1997) or plan to update rates more frequently (Weiner et al., 1998) to address this issue, Washington found new enrollees cost more per member per month and increased Medicaid capitation rates accordingly (Schwalberg, 1997). Our results indicate that our study States may also need such adjustments. Lacking adequate information on health risks for short-term enrollees, they may under or over pay plans. While age and sex adjusters specific to short-term enrollees worked fairly well in California and Georgia, the high proportion of short-term enrollees (almost 50 percent) in the welfare and poverty-related groups in Mississippi and the apparent unpredictable nature of their expenses appears to limit that State's ability to set accurate rates.

Mississippi has had historically high poverty rates, and high unemployment as well as relatively low Medicaid eligibility levels. This means there are relatively more poor, likely uninsured, families that cannot qualify for Medicaid until they lose whatever source of income they may have, develop severe health problems, or become pregnant. Our data show there is significantly more turnover, or at least short-term enrollees, in this State as well. This would indicate there is a significant movement of needy, perhaps less healthy, enrollees in and out of the Mississippi Medicaid Program. While this deserves further attention, we do not find the results inconsistent with this State's background. To the extent these are pregnant females enrolling late in their pregnancy, Mississippi (along with other States) can use supplemental or other payment adjustments to deal with these expenses (Holahan et al., 1999).

We note several study limitations. First, we have only examined the ACG system. While it performed comparably to others in predictive accuracy (Dunn et al., 1996), some note that the ACG system separates the very healthy from the sick but may not distinguish well among persons with different degrees of illness (Kronick et al., 1996). Also, the Medstat data are a large, convenience sample of the privately insured and are not necessarily representative of the non-capitated privately insured nationally, nor within the study States. We also note, as did Kronick et al. (2000), that while our data precede the passage of welfare reform, the relationship of diagnoses to expenditures and the type of diagnoses seen across large numbers will be applicable to current enrollees in these eligibility groups. Finally, the availability of only 1 year of data meant we tested only concurrent and retrospective adjustment—risk adjusters would not work as well prospectively as our analysis implies.

While we only examine three States, Medicaid policy does require State-specific information. Further, the differences seen here across States are informative. Our finding that even the younger females and children in Medicaid have a higher case mix than privately insured in southern States, but not in California implies that the privately insured not enrolled in an HMO in California may be less healthy. While this might lead us to conclude that California can more easily attract commercial plans into Medi-Cal markets, it is likely that the differences in Medicaid managed care across our States is due to a more fundamental issue. When Georgia and Mississippi tested capitated managed care, for example, enrollment was voluntary and plans were not able to enroll sufficient numbers to remain viable.

Even if these States move to mandatory Medicaid managed care, however, plans will make decisions regarding the Medicaid market based on relative health risks and expected costs across the public and private sectors. The private sector does not generally view risk adjustment as warranted, given its costs and payment errors in the 5-7 percent range (Rogal and Gauthier, 1998). This may not hold for Medicaid markets, however, where profit margins are lower (Felt-Lisk, 1999). Medicaid agencies in these States should consider how attractive their rates would be under the limitations imposed by current Federal policy (Holahan et al., 1999) and the higher case-mix found here for their Medicaid enrollees.