Public reporting of quality information on Medicaid health plans.

Transparency through public reporting of quality data is key to achieving the Institute of Medicine's (IOM) vision for 21st century health care. This article reviews the status of States' voluntary public reporting of Medicaid managed care (MMC) quality data, and analyzes these data. Twenty-one States, including 17 of the 20 largest managed care States, have made plan-level data publicly available online, although the data are sometimes thin, with few measures reported, hard-to-access, and old. We conclude that CMS could better leverage the power of public reporting for quality improvement (QI) by increasing the visibility of health plan employer data and information set (HEDISV) data that States already collect.

Introduction

The IOM (2001) identified transparency as the seventh rule for redesigning the health care system for the 21st century, stating that information should be made available to patients and their families that describes the system's performance on safety, evidence-based practice, and patient satisfaction. While information on performance of health plans with respect to Medicare beneficiaries is readily available nationwide from the CMS Web site, and hospital, nursing home, and home health data are available nationally, the same is not true for Medicaid health plans. The National Committee for Quality Assurance (2004) reports annually national statistics on HEDIS® effectiveness of care measures for Medicaid, but does not regularly report access or use-of-services measures, critical for the Medicaid population. The American Public Human Services Association (2002) last reported national data on a core set of tracking measures for Medicaid for 2001, including some access and use of services measures. Yet full-risk MMC plans are the dominant vehicle for Medicaid service delivery in many States, and these plans served over 15 million beneficiaries for comprehensive medical benefits in 2004, or 36 percent of Medicaid enrollees (Felt-Lisk, Barrett, and Verdier, 2005).

This article identifies the extent to which Medicaid health plan quality information is available online. We reviewed the available data and offers here a first look at major patterns in it by type of health plan: for-profit versus non-profit plans, larger versus smaller plans, Medicaid-dominated (those with Medicaid members comprising more than 75 percent of total enrollees) versus other Medicaid-serving plans, and within the Medicaid-dominated category, provider-owned versus other ownership types.

In brief, we find most of the large MMC States have made data publicly available online, although the data are sometimes thin, with few measures reported, hard-to-access, and old. Non-profit plans had significantly higher mean HEDIS® rates for several of the eight studied indicators relative to for-profit plans, and among Medicaid-dominated plans, provider-owned plans outperformed other Medicaid-serving plans on several measures. Larger plans out performed smaller plans on several measures as well. We found no pattern with respect to performance of Medicaid-dominated versus other Medicaid-serving plans.

Study Methods

Data Sources

Plan-level data on quality measures were obtained from reports available on State Medicaid Web sites or other (e.g., State health department) Web sites with plan-level quality data in fall 2004 and again in fall 2006. In late 2004, calls were made to each State Medicaid agency to verify that we had found all the available plan-level data for any publicly reported quality indicators. State officials who could respond to that inquiry were also asked if they would like to comment briefly on the audience for publicly reporting. Of the 21 States that publicly reported quality-related data, 9 offered a comment on the audiences for the online data. The people who offered the comments were typically referred to us by the Medicaid director's office as knowledgeable about the public availability of the State's MMC quality data.

Plan characteristics are from a data set developed by Mathematica Policy Research, Inc. The Henry J. Kaiser Family Foundation funded the development of a 2004 database that merged CMS data on full-risk Medicaid enrollment by plan for June 30, 2004, with health maintenance organization (HMO) industry data on total enrollment and plan characteristics for many of the plans as of July 1, 2004, from the InterStudy Competitive Edge HMO Directory. Partially capitated plans and plans that do not provide comprehensive medical services (e.g., behavioral health and dental plans) were excluded from the database. There are many Medicaid-dominated plans whose characteristics cannot be obtained from the InterStudy Competive Edge HMO Directory, because they are licensed differently from HMOs in their States, although they operate as full-risk plans. For purposes of the analysis, Medicaid-dominated plans are defined as those for whom Medicaid enrollees comprise 75 percent or more of the plan's total enrollment. We researched these plans' characteristics on the Internet and called plans directly when necessary. The NCQA provided data on which health plans were accredited in 2004.

Analysis

We first arrayed the most recently available clinical quality-related measures found online for all the States, and counted the number that reported the same and similar measures, noting differences between State-specified measures and HEDIS® and consumer assessment of healthcare providers and systems (CAHPS®) measures. For purposes of the analysis, we considered quality-related measures to be those examining clinical quality, access to care measures commonly used for the Medicaid population (including HEDIS® measures of utilization that bear on access to primary care), and patient satisfaction or experience with care. Next, we compared selected characteristics of State Medicaid Programs that report and do not publicly report any quality data. We identified themes in the comments of State officials regarding the audience for publicly reported data based on review of the near-verbatim notes.

Finally, for all measures where at least seven States and 60 health plans publicly reported the identical measure in the same year, we compared mean values of the measure by plan size (total health plan enrollment), tax status, ownership type, and the extent to which the plans also served commercial populations. Although our analytic approach was opportunistic in terms of which measures were reported by a sufficient number of plans to be reviewed, the eight measures included a range of types of measures applicable to Medicaid beneficiaries of varying ages. The measures include (1) chronic care (two diabetes and asthma), (2) preventive (breast cancer screenings, and prenatal care), and (3) utilization that bear on access to care (well-child visits 3-6 years, ambulatory care for adults 20-44 years, and 45-64 years). T-tests were used to identify statistically significant differences. While the publicly reported data we use in this analysis do not include all States and plans, they are typically required of all Medicaid plans within the reporting States, and therefore the data do not risk the bias that may occur with voluntary reporting when only higher-scoring plans choose to report their data. We could not perform a similar analysis using the CAHPS® data because of the slightly varying ways States reported their data: for example, some States reported the percentage of respondents who gave a certain rating or range of ratings, while other States reported the mean rating of all respondents.

Study Results

Extent and Content of Public Reporting

Twenty-one States, or 60 percent of States with full-risk MMC programs in 2004 publicly reported at least some HEDIS®-like or CAHPS®-like data for their contracted health plans in 2003, 2004, or 2005. Seventeen States reported both HEDIS® and CAHPS®-like data, while 2 States reported only HEDIS®-like data, and 2 States reported only CAHPS®-like data (Table 1).

Table 1

Online Reporting of Quality Data, by State

State	Size of Full-Risk Managed Care Program2
States Reporting Both HEDIS® and CAHPS®
California	Large
Colorado	Small
Florida	Large
Iowa	Small
Maryland	Large
Michigan	Large
Minnesota	Large
Missouri	Large
New Mexico	Large
New York	Large
North Carolina	Small
Ohio	Large
Pennsylvania	Large
Texas	Large
Virginia	Large
Washington	Large
Wisconsin	Large
States Reporting Either HEDIS® or CAHPS®
Arizona (HEDIS® Only)	Large
Massachusetts (HEDIS® Only)	Large
Oregon (CAHPS® Only)	Large
Rhode Island (CAHPS® Only)	Small
States Not Reporting HEDIS® or CAHPS®
Connecticut	Large
Delaware	Small
District of Columbia	Small
Hawaii	Small
Illinois	Small
Indiana	Large
Kansas	Small
Kentucky	Small
Nebraska	Small
Nevada	Small
New Jersey	Large
North Dakota	Small
South Carolina	Small
West Virginia	Small

The following States had no fully-capitated managed care program: Alabama, Alaska, Arkansas, Georgia, Idaho, Louisana, Maine, Mississippi, Montana, New Hampshire, Oklahoma, South Dakota, Tennessee, Utah, Vermont, and Wyoming.

Large programs have more than 200,000 full-risk Medicaid managed care enrollees.

NOTES: HEDIS® is Health Plan Employer Data and Information Set. CAHPS® is Consumer Assessment of Healthcare Providers and Systems.

SOURCE: Felt-Lisk, S., Barrett, A., Mathematica Policy Research, Inc., and Nyman, R., 2006.

Which States Report—Most of the States with large full-risk MMC programs are now publicly reporting some quality-related data by plan (17 of the 20 with at least 200,000 enrollees). Conversely, States with smaller programs tended not to report (11 of 15). As a group, the States that publicly report include 87 percent of all Medicaid enrollees in full-risk plans, and 85 percent of Medicaid-serving plans. The States that publicly report quality data also tend to have more of their plans accredited by the NCQA, with 56 percent accredited compared with 11 percent accredited in other States. NCQA reports that about one-half of health plans nationally are accredited (including those that do and do not serve Medicaid). The only States with more than 200,000 beneficiaries in full-risk plans that do not report publicly are New Jersey, Connecticut, and Indiana. Indiana plans to publish plan-specific data within the next year, while New Jersey and Connecticut have no specific plans to do so.

Several of the other States that did not publicly report data online only had one health plan in their programs, so did not view publication as a priority (Kentucky, Kansas, and North Dakota). Another State (Hawaii) that does not publish quality-related data online produces a flier with plan-specific HEDIS® data that is included in enrollees' packets at the time of re-enrollment.

Types of Measures Reported—The most frequently reported HEDIS®-like measures focus on women and children's health and chronic care measures. Prenatal care, appropriate medications for asthma, well-baby visits, well-child visits, well-adolescent visits, and one or more measures of comprehensive diabetes care were the most common, reported in 13 States (Table 2). The types of HEDIS® measures that are less often publicly reported either relate to a relatively smaller proportion of most States' Medicaid populations (measures such as blood pressure controlled, beta blocker after heart attack, and cholesterol management after major cardiovascular events); relate to mental illness or substance abuse treatment, which is often carved out and/or under different internal management within the Medicaid agency (six States report something in this category); relate to dental care where coverage may be very limited (nine States report annual dental visit for one or more age groups); use self-report survey methodology (smoking cessation is asked as an additional CAHPS® question by four States); or represent selectivity among similar measures—e.g., choosing to report one of two optional immunization combinations, or fewer than all seven rates for well-baby visits at 15 months of age.

Table 2

HEDIS® Measures Most Frequently Reported Online: 2003-2004

HEDIS® or Modified HEDIS® Measure	Number of Reporting1		Number of Reporting States Using Modified HEDIS® and Type of Modification
	States	Plans
Appropriate Medications for Asthma (5-9 Years, 10-17 Years, 18-56 Years, and Combined Rate)	13	121	2: No Combined Rate; Age Group 0-20/21+
Timeliness of Prenatal Care (Visit in First Trimester)	13	106	—
Adolescent Well-Care Visits (12-21 Years)	13	103	2: Age Group 11-20; Age Group 6-20
Well-Child Visits (3-6 Years)	13	99	1: Age Group 3-5
Well-Baby Visits (15 Months Old) 0, 1, 2, 3, 4, 5, and 6+ Visits	13	93	7: Only Reported 6+ Visits (3); Only Reported Zero and 6+ Visits (2); Only Reported 3 Visits (1); Age Group 2 Years (1)
Comprehensive Diabetes Care (18-75 Years), At Least One Component Measure2	13	66	1: Age Group 18-64
HbA1cTest	10	54	1: Age Group 18-64
Eye Exam	9	47	—
LDL Screening	9	51	—
Monitoring for Diabetic Nephropathy	7	40	—
HbA1c Controlled	5	30	—
Lipids Controlled	5	30	—
Cervical Cancer Screenings (18-64 Years)	12	106	4: Age Group 21-64 (3); Age Group 21-65 (1)
Breast Cancer Screenings (50-69 Years)	11	103	3: Age Group 50-64; Age Group 52-69; Age Group 40+
Postpartum Care	10	93	0
Adult Ambulatory Care Visits (20-44 Years)	9	80	0
Childhood Immunizations (4 DTaP/DT, 3 IPV, 1 MMR, 3 influenza type B, 3 hepatitis B, and 1 chicken pox vaccination)	9	56	4: Unclear Combination of Vaccines (2); No Chicken Pox and Two Hepatitis B (1); No Chicken Pox (1)
Adult Ambulatory Care Visits (45-64 Years)	8	73	0

Out of 149 plans and 19 States that reported some HEDIS® measures.

Only 5 States reported all component measures.

NOTE: HEDIS® is Health Plan Employer Data and Information Set.

SOURCE: Felt-Lisk, S., Barrett, A., Mathematica Policy Research, Inc., and Nyman, R., 2006.

Some global CAHPS® ratings of member satisfaction were reported by plan by 19 States, although States varied in which ratings were given: doctor or nurse, specialist, health plan, and/or health care (Table 3). Fewer States reported on the child survey per se—only 10 of the 19 reported separate responses for a child survey; one more combined the child and adult survey responses. In addition to the global ratings, 17 of the 19 States reported data on at least one of the five CAHPS® composite measures and/or their individual question components: (1) getting needed care, (2) getting care quickly, (3) how well doctor communicates, (4) courteous and helpful office staff, and (5) customer service. Several States also reported on smoking cessation.

Table 3

Number of States Reporting CAHPS® Ratings of Member Satisfaction Online

CAHPS® Composite Measure	Number of Reporting
	Adult Survey		Child Survey
	States	Plans	States	Plans
Global Ratings
Doctor or Nurse	15	119	8	64
Specialist	15	121	8	64
Health Care	16	109	8	64
Health Plan	19	151	10	81
Getting Needed Care	17	145	8	73
Getting Care Quickly	14	120	7	63
How Well Doctor Communicates	16	134	8	73
Courteous and Helpful Office Staff	12	87	7	63
Customer Service	15	112	8	73

NOTE: CAHPS® is Consumer Assessment of Healthcare Providers and Systems.

SOURCE: Felt-Lisk, S., Barrett, A., Mathematica Policy Research, Inc., and Nyman, R., 2006.

Standardization of Measures—As their developers intended, the availability of HEDIS® and CAHPS® has promoted standardization in quality measurement across States. States commonly used the precise HEDIS® measure definition, although varying the age group for the measure was not uncommon (Table 2). All States publicly reporting data on member satisfaction/experience used the CAHPS® survey instrument as the basis for reporting, although the way that the data are reported varies. For example, while all the relevant States reported at least one global rating measure for personal doctor or nurse, specialist, health plan, and/or health care, the response categories varied, with some States reporting a proportion of members giving a rating or range of ratings, while others reported the average rating.

Survey response rates were not always reported together with the CAHPS® results (10 States did not report the response rates). The response rates that were reported for the adult survey ranged from a low of 25 percent in Maryland to a high of 48 percent in Washington. Data based on low response rates may be biased, with information missing from the hardest-to-survey population, and thus should be interpreted with caution. Rates of 45-50 percent may be acceptable, however, even though such rates are still low by commercial and Medicare CAHPS® survey standards. NCQA protocols have included a target response rate of 45 percent for CAHPS® for Medicaid, recognizing the difficulty and cost of achieving high response rates in this population. Further, a study by Lurie and colleagues (2003) finds that CAHPS® survey results pertaining to assessments of care by race/ethnicity based on a commercial population with a 43 percent response rate were largely similar to those from a Medicare population with a high response rate (80 percent), indicating that the lower response rate did not create substantial bias.

How Many Measures?

States are selective in which HEDIS® measures they require from plans and publicly report. This is not surprising, since States must consider the level of effort required of plans to collect the measures; many plans still abstract from medical records in addition to using administrative data to support the measures. On average, in 2004, the States that publicly reported HEDIS® measures reported 11 of the 48 measures with relevance to effectiveness of care or access to primary care for the Medicaid population, although the number varies. The degree of selectivity varies by State, with 10 States reporting 10 or more HEDIS® measures and 4 reporting only 6 or less. Florida, Arizona, and Colorado have rotated which measures they reported in between one year and the next, another strategy for reducing burden.

How Old and How Easy to Find?

The search for publicly available quality information on Medicaid, while fruitful in many respects, highlighted two major drawbacks of the way the data are currently handled. First, it took considerable effort to locate the data in many States. In two States, quality data for all managed care populations (commercial, Medicaid, and Medicare) were located on the State health department Web site, without obvious linkage from the Medicaid Web site. In two other States, links to the reports were either broken or had been erased from the Web site; these reports were accessible via an Internet search engine, but not directly from the Medicaid Web site itself. In several other States, the reports were located in the publications and reports section of the Medicaid agency Web site, buried among numerous unrelated reports. In contrast, Web sites where the data were particularly easy-to-find (the report or the links were clearly displayed on the main Medicaid MMC page) included Michigan and New Mexico.

Second, the data are generally fairly old. The lag varied, but in November 2006, the most recent HEDIS®-like data available for most States was for calendar year 2004, although a few States, such as Minnesota, had published data for 2005. Clearly, publicly reported data must be held to a high standard of accuracy, so the lag may at least partly reflect time for auditing and ensuring accuracy. Survey data should be able to be made available sooner, since no audit process is required. In fact, there was more variation in the lag for reporting survey data. This may partly reflect that some States do surveys on a less frequent timetable than annually. By November 2006, four States had 2005 survey data available, while others lagged more, with seven States having 2004, four States having 2003, two States having 2002, and one State having 1999-2000 data as the most recent available.

Audience for the Publicly Reported Data

States see health plans, legislators, and State government officials inside or outside their agency as the prime audience for the online public reported data—not so much consumers, based on the nine publicly reporting States that commented on the audience for their online public reporting. They do not believe consumers generally access the data online, although several States (Colorado, Maryland, and Wisconsin) mentioned they use the same data to create report cards that are included in enrollees' packets when information is sent about re-enrollment. Beneficiary advocates, practitioners, and researchers were also mentioned by a couple of States as primary audiences.

Patterns in Selected HEDIS® Scores by Type of Plan

Three patterns of interest appear from a descriptive analysis of the publicly available HEDIS® data for eight measures for calendar year 2004, by type of plan. The patterns are based on analysis of differences in average plan rates across the different types of plans that serve Medicaid enrollees: non-profit versus for-profit; larger versus smaller plans based on total enrollment above/below 100,000 members; Medicaid-dominated versus other Medicaid-serving plans; and within the Medicaid-dominated category, provider-owned versus non-provider-owned plans.

Non-Profit Versus For-Profit—Non-profit plans perform significantly better than for-profit plans on three of the eight measures analyzed, with all of the remaining measures except well-child visits also better, but with differences that were not statistically significant (Table 4). The largest differences were for diabetic eye exam (55 versus 48 percent) and diabetic HbA1c test (84 versus 76 percent). Given the limited number of plans available for analysis (60 to 76, depending on the measure), we cannot use multivariate methods that would confirm whether it is nonprofit status that is driving this result as opposed to some other factor or factors that are also prevalent in nonprofit plans.

Table 4

Mean HEDIS® Rates for Medicaid Managed Care Plans, by Tax Status

Measure	Non-Profit Plans		For-Profit Plans		Difference in Means with 95 Percent Confidence Interval
	Plans	Mean	Plans	Mean
Prenatal Care	33	81.1	34	78.3	1.8	+/-	4.7
Breast Cancer Screening	50	60.7	26	57.1	3.5	+/-	4.2
Ambulatory Care (45-64 Years)	49	85.4*	25	82.6	2.8	+/-	2.3
Ambulatory Care (20-44 Years)	40	78.7	31	78.1	0.6	+/-	2.9
Diabetic Eye Exam	40	54.6*	30	48.3	6.4	+/-	5.2
Diabetic HbA1c Test	34	83.6*	27	76.4	7.2	+/-	5.7
Appropriate Medications for Asthma	31	66.1	31	63.4	2.6	+/-	3.7
Well-Child Visits (3-6 Years)	27	62.5	34	66.4	-3.9	+/-	5.5

p < 0.05.

NOTES: Measures must be reported by at least 60 plans in at least 7 States, and be collected using HEDIS® specifications (unmodified) during calendar year 2004 to be included in this analysis. HEDIS® is Health Plan Employer Data and Information Set.

SOURCE: Felt-Lisk, S., Barrett, A., Mathematica Policy Research, Inc., and Nyman, R., 2006.

State location did not seem to be driving the results. Michigan, New York, and California are the only three States that had the potential to influence these figures substantially, since they contain many non-profit plans. In terms of other characteristics, nonprofit plans in the database tended to be smaller (236,000 versus 250,000), have fewer Medicaid enrollees (76,000 versus 114,000), and have a lower percentage of enrollment that is Medicaid (67 versus 71 percent)—not factors which one would expect to be correlated with higher scores, given the following analysis regarding patterns by plan size and proportion of Medicaid enrollees. The same pattern of higher non-profit plan scores was evident from this same analysis using 2002 data, and was noted in another prior analysis that used data for 1999 for all plans nationally that reported HEDIS® data (Felt-Lisk, Dodge, and McHugh, 2001).

Provider-Owned Versus Other Medicaid-Dominated Plans—Within the group of plans that focuses exclusively or almost exclusively on the Medicaid population, provider-owned plans performed significantly better on three of the eight measures, and three of the five remaining measures were better although the differences were not statistically significant (Table 5). The significant differences were for the diabetic HbA1c test measure, appropriate medications for asthma, and breast cancer screening. Provider-owned plans in this analysis include 18 plans owned at least in part by hospitals or hospital systems, 10 owned at least in part by federally qualified health centers, 6 owned at least in part by an academic medical center, and 4 at least in part owned by physicians. Other Medicaid-dominated plans included 16 plans affiliated with multi-State managed care firms, 10 independent firms, 8 government-owned plans, and 1 other plan. There were not enough plans publicly reporting these data to further distinguish averages between plans with different types of ownership.

Table 5

Mean HEDIS® Rates for Medicaid-Dominated Plans That Are Provider-Owned Versus Other Ownership Types

Measure	Provider-Owned Plans		Other Medicaid-Dominated Plans		Difference in Means with 95 Percent Confidence Interval
	Plans	Mean	Plans	Mean
Ambulatory Care (20-44 Years)	27	76.4	23	76.6	-0.2	+/-	3.0
Prenatal Care	11	81.4	24	79.6	1.7	+/-	6.0
Ambulatory Care (45-64 Years)	26	84.2	19	82.7	1.5	+/-	2.3
Breast Cancer Screening	21	66.4*	23	54.8	11.6	+/-	5.1
Diabetic Eye Exam	21	54.9	23	50.4	4.5	+/-	6.2
Diabetic HbA1c Test	20	84.4*	16	77.8	6.6	+/-	4.1
Well-Child Visits (3-6 Years)	12	63.0	24	66.3	-3.3	+/-	7.6
Appropriate Medications for Asthma	11	68.9*	22	63.4	5.5	+/-	4.2

p < 0.05.

NOTES: Measures must be reported by at least 60 plans in at least seven States, and be collected using HEDIS® specifications (unmodified) during calendar year 2004 to be included in this analysis. HEDIS® is Health Plan Employer Data and Information Set.

SOURCE: Felt-Lisk, S., Barrett, A., Mathematica Policy Research, Inc., and Nyman, R., 2006.

Larger Versus Smaller Plans—Larger plans with 100,000 or more enrollees outperformed smaller plans on three measures: prenatal care and access to preventive/ambulatory care for adults 20-44 years as well as adults 45-64 years (Table 6). Since they also had higher rates for three of the five other measures, though not significantly higher, this pattern may be worth followup in the future as more data become available increasing the power to use more sophisticated analytic methods.

Table 6

Mean HEDIS® Rates for Medicaid Managed Care Plans, by Plan Size

Measure	<100,000 Total Plan Members		>100,000 Total Plan Members		Difference in Means with 95 Percent Confidence Interval
	Plans	Mean	Plans	Mean
Prenatal Care	35	77.6*	32	83.0	-5.4	+/-	4.5
Breast Cancer Screening	42	60.1	34	58.7	1.4	+/-	4.1
Ambulatory Care (20-44 Years)	39	76.5*	32	80.9	-4.4	+/-	2.7
Ambulatory Care (45-64 Years)	38	83.3*	26	85.9	-2.6	+/-	2.3
Diabetic Eye Exam	38	51.5	32	52.4	-0.9	+/-	5.4
Appropriate Medications for Asthma	36	64.3	26	65.4	-1.1	+/-	3.8
Diabetic HbA1c Test	35	81.1	26	79.4	1.7	+/-	6.0
Well-Child Visits (3-6 Years)	33	62.3	28	67.5	-5.3	+/-	5.4

p < 0.05.

NOTES: Measures must be reported by at least 60 plans in at least seven States, and be collected using HEDIS® specifications (unmodified) during calendar year 2004 to be included in this analysis. HEDIS® is Health Plan Employer Data and Information Set.

SOURCE: Felt-Lisk, S., Barrett, A., Mathematica Policy Research, Inc., and Nyman, R., 2006.

Medicaid-Dominated Versus Other Medicaid Plans—Medicaid-dominated plans did not show a significant pattern one way or the other relative to other Medicaid-serving plans, with slightly higher rates for Medicaid-dominated plans, and only one significant difference (Table 7). This is consistent with qualitative research that found only limited differences in QI strategies between Medicaid-dominant and commercial managed care organizations (Felt-Lisk and Gold, 2003).

Table 7

Mean HEDIS® Rates for Medicaid Managed Care Plans, by Proportion of Plan Enrollees Who Are Medicaid Enrollees

Measure	Medicaid-Dominated Plans1		Other Medicaid-Serving Plans2		Difference in Means with 95 Percent Confidence Interval
	Plans	Average	Plans	Average
Prenatal Care	35	80.2	32	80.2	0.0	+/-	4.7
Ambulatory Care (20-44 Years)	50	76.5*	21	83.3	6.8	+/-	2.7
Ambulatory Care (45-64 Years)	45	83.6	19	86.1	2.5	+/-	2.8
Breast Cancer Screening	44	60.3	32	58.3	-2.0	+/-	4.1
Diabetic Eye Exam	44	52.6	26	50.7	-1.8	+/-	5.5
Diabetic HbA1c Test	36	81.5	25	78.9	-2.5	+/-	6.0
Well-Child Visits (3-6 Years)	36	65.2	25	63.9	-1.3	+/-	5.6
Appropriate Medications for Asthma	33	65.2	29	64.2	-1.0	+/-	3.7

p < 0.05.

> 75 percent of enrollees are Medicaid.

>0 but <75 percent of enrollees are Medicaid.

NOTES: Measures must be reported by at least 60 plans in at least 7 States, and be collected using HEDIS® specifications (unmodified) during calendar year 2004 to be included in this analysis. HEDIS® is Health Plan Employer Data and Information Set.

SOURCE: Felt-Lisk, S., Barrett, A., Mathematica Policy Research, Inc., and Nyman, R., 2006.

Discussion

This study points to both the tremendous progress and remaining shortcomings of public reporting of quality data for Medicaid. A decade ago, States typically collected health plan-level HEDIS data on only a few quality indicators, if at all, and the data were rarely reported publicly (Landon et al., 2004). Today, we showed that individuals with some persistence can find some publicly available clinical quality or access data online for plans in 21 States, including 17 of the 20 largest programs. Usually, both HEDIS® and CAHPS® measures are reported and include both access-related and clinical measures, which reflect a balanced view of quality that encompasses both the consumer's perspective and clinical measures. Over time, as intended, the availability of HEDIS® and CAHPS® instruments have led to relatively standardized methods of data collection and reporting.

However, the full transparency envisioned by the IOM has yet to be achieved. The data were difficult to find for about one-third of the States. The data were sometimes relatively old, which is particularly surprising for CAHPS® data, which do not require extensive validation. The data were also relatively thin for some States (few measures), probably reflecting a combination of different levels of sophistication at the plans and different views on the importance of having a broad set of data by the State.

Further, CMS could do more to leverage the power of public reporting to improve the Medicaid Program nationally, by increasing the visibility of HEDIS® data that are already being collected. A growing consensus and new research finds that public reporting of hospital data has generated substantial QI by hospitals in the targeted areas (Laschober and Maxfield, 2005; Laschober et al., 2005). At present, States' public reporting of their plans' quality data is voluntary and requires effort to access. CMS could increase the visibility of these data by, for example, linking to the information from the CMS Medicaid Web site, and annually posting a highlights document reflecting analysis of the available data. This could include, for example, identifying national benchmarks and highest-scoring States and plans for the more commonly collected measures that follow the standard definition. Minor State-specific age group variations from the standard HEDIS® definitions and different ways of reporting on the CAHPS® data currently hamper the ability to combine all the relevant data for analysis at the national level to some extent; to maximize public reporting long-term CMS could encourage standardization.

One way to encourage standardization without a mandate would be to recognize high-performing States and plans, with eligibility for recognition contingent on following the HEDIS® definitions exactly. One of the most widely known models for recognition is the U.S. News and World Report (2006) “Best Health Plans” list which includes the top five Medicaid health plans based on NCQA analysis of HEDIS® data used in accreditation scoring. However, this type of model has drawbacks including (1) relatively few Medicaid plans report enough measures to create this type of composite score so as to be included in the analysis, (2) the number of plans identified as excellent is extremely small, and (3) because the list is designed for consumers rather than to facilitate improvement, no information is provided about how the plans achieved their high scores. NCQA's (2006) quality profile case studies provide examples, by clinical topic area, to show how plans can work to improve quality, but they are not linked to plan scores, and do not name the plan. One possible model for CMS would be a combination of these concepts that would name the plans with the highest scores in each clinical area, and allow those plans to contribute information about the factors it believes have helped it achieve the high scores, thereby promoting both achievement of high scores through recognition and furthering the QI purpose of the recognition.

The pattern of better HEDIS® scores among non-profit relative to for-profit plans may suggest a tendency for nonprofit plans to invest more resources at the margin in improving access for the Medicaid population. However, without more complete and more recent data that would allow more sophisticated analytic methods, we cannot confirm that it is non-profit status driving this result rather than some other factor present in non-profit plans. Performance among plans with respect to their profit status bears watching particularly as for-profit multi-State managed care firms have been increasing their health plan holdings (Felt-Lisk, Barrett, and Verdier, 2005).

A pattern of better HEDIS® scores among Medicaid-dominated provider-owned plans relative to other Medicaid-dominated plans is consistent with an earlier set of case studies that found integration with a delivery system was a common characteristic to high-performing health plans based on their clinical HEDIS® scores broadly (not limited to Medicaid) (Felt-Lisk and Kleinman, 2000). Plans that are heavily integrated with providers may have two possible advantages—the providers may be more willing to support plan efforts to improve, since their fates are closely aligned, and it is possible that beneficiaries may pay more credence to reminders sent by a health plan that they associate with their provider rather than one they think of as an insurance company. Again, more complete data and multivariate analysis are needed to confirm that provider ownership rather than another co-occurring characteristic is driving the higher scores.

In sum, the vast majority of States with sizable MMC programs are providing some transparency by reporting health plan quality data online, but the potential power of transparency could be better leveraged by CMS in the instance of MMC.