Medicare Managed Care CAHPS®: A Tool for Performance Improvement.

The Medicare+Choice (M+C) program, created by the 1997 Balanced Budget Act (BBA), expands Medicare's health insurance options to include a wider range of health plan options. In this article, we describe the Consumer Assessment of Health Plans Study (CAHPS®) survey and its use with beneficiaries receiving care through Medicare managed care (MMC) plans. We also discuss the implications of these efforts for future quality improvement efforts.

Introduction

In recent years, HCFA has revised its strategic plan, reorganized its operations, and initiated several programs aimed at ensuring that Medicare beneficiaries receive the highest possible quality of health care. The M+C program, created by the 1997 BBA, expands Medicare's health insurance options to include a wider range of health plan options. The BBA, the Government Performance and Results Act, and other government-wide initiatives have all emphasized the themes of accountability and a renewed focus on Medicare beneficiaries as customers.

The M+C program has developed programs to monitor and improve the quality of ambulatory care. In an effort to better understand the experiences and needs of Medicare beneficiaries, it now surveys annually a probability sample of all Medicare beneficiaries and asks about their health care experiences using the CAHPS® survey.

Background

To ensure that beneficiaries have access to the best possible health care, HCFA has initiated several quality monitoring systems, including the CAHPS® surveys, the Health Outcomes Survey, and the collection of measures in the Health Plan Employer Data Information Set (HEDIS®). These measurement activities allow HCFA to identify quality improvement opportunities and to disseminate information to beneficiaries to help them make better health care choices.

The advent of M+C presented HCFA with the formidable task of educating the Nation's 40 million Medicare beneficiaries about changes in the Medicare program. HCFA established the National Medicare Education Program to inform beneficiaries, caregivers, and partners, about features of the Medicare program and, in particular, about the expanded options with M+C. As part of the National Medicare Education Program, HCFA provides comparative information on health plans, including information about enrollee experiences through the Medicare & You Handbook, mailed annually to Medicare beneficiaries before the open enrollment period, through a telephone hotline (1-800-MEDICAR[E]), and through HCFA's Web site http://www.medicare.gov.

CAHPS® Survey

Since 1998, HCFA has conducted an annual nationwide survey of Medicare beneficiaries enrolled in managed care organizations to collect information about their experiences with their health plans. This survey was developed as part of the Medicare CAHPS® project. CAHPS® is the first comprehensive effort to assess consumer experiences with their health plans and services. The Medicare CAHPS® effort consists of: the managed care survey, a disenrollment survey, and the Medicare (fee-for-service) survey. This article focuses exclusively on the survey of beneficiaries enrolled in the MMC plans, presents background information on the CAHPS® survey, provides an overview of the first 3 years of data collection and analysis, and a discussion of the long-run potential for using the managed care data for performance improvement.

Description of MMC CAHPS®

This survey is part of a group of consumer surveys developed by a consortium of researchers from the Research Triangle Institute, the RAND Corporation, and the Harvard Medical School through cooperative agreements with the Agency for Healthcare Research and Quality. The goal of the CAHPS® initiative, which began in 1995, was to develop and test standardized questionnaires and report formats that could be used to collect and report meaningful and reliable information about health plans. The surveys contain core items that can be used with any population and additional items targeted to particular groups. Thus, they can be used with all types of health insurance consumers, including Medicaid recipients, Medicare beneficiaries, those who are commercially insured, and across the full range of health care delivery systems.

The core CAHPS® survey asks respondents for four overall ratings of their doctor, specialist (if used), overall health care, and health care plan. Information from most of the specific questions is combined to create composite scores on getting needed care, getting care quickly, doctor communication, courtesy and helpfulness of office staff, and paperwork, information, and customer service.

In 1996 HCFA funded the CAHPS® consortium to develop a special version of CAHPS® for beneficiaries enrolled in the MMC plans. The initial MMC CAHPS® survey, which was only used in the first administration of the survey, included all items in the CAHPS® 1.0 adult core instrument and 28 additional items specific to the MMC version. The CAHPS® 2.0 survey instrument has been used in subsequent administrations. Items unique to the MMC CAHPS® version ask about ease of obtaining needed medical equipment, physical, occupational or speech therapy, home health services, and prescription drugs. Other MMC items concern advice to quit smoking, receipt of flu shots, pneumonia vaccinations, personal doctor's knowledge about important medical decisions, and the health plan's handling of complaints. Most of the Medicare-specific items were added to the core to capture the experience of beneficiaries with services that are used more often by patients with chronic conditions or limitations in activities of daily living. The questions related to smoking counseling, flu shots, and pneumonia vaccinations were added to obtain information from managed care enrollees to fulfill HEDIS® reporting requirements.

We present data on the ratings and composite scores here because those are the data that currently are publicly available. Factor analyses of data from the first year survey confirm that these composites generally correspond to the dimensions of performance along which plans vary (Zaslavsky et al., 2000a). Because CAHPS® 1.0 was used the first year and CAHPS® 2.0 in subsequent years of the MMC CAHPS® surveys, some of the composite scores are not directly comparable across years.

Survey Procedures

HCFA contracted with the Barents Group of KPMG Consulting Inc. and its subcontractors Westat, Data Recognition Corporation, Picker Institute, and the Harvard Medical School to implement the MMC CAHPS® survey in fall 1997. The first survey was fielded in February 1998 and subsequent surveys have been fielded every fall, beginning in 1998.

For the most recently completed survey (fall 1999), eligible plans included all M+C organizations and continuing cost contracts with contracts in effect as of July 1, 1998. In some cases, a single contract that covers a large geographic area was divided into several geographically defined parts, which were then treated separately in the survey and analysis. HCFA drew a random sample of up to 600 Medicare beneficiaries who had been enrolled in an eligible plan for at least 6 months and who were non-institutionalized. If the organization had less than 600 Medicare beneficiaries, everyone in the organization was included in the sample.

Numerous attempts are made to reach respondents in English and Spanish by mail, telephone, Federal Express, or Priority Mail. A pre-survey notification letter is sent to all sample members followed by the first survey mailing. Thank you and reminder postcards are sent to all sample members. Non-respondents receive a second survey mailing and telephone interviews are conducted with sample members who have not responded to the first or second mailing. Non-respondents for whom no telephone numbers are available receive a third survey by Federal Express or Priority Mail.

For February of 1998, there were 237 organizations sampled and 89,804 beneficiaries responded for a response rate of 75 percent. For September of 1998, there were 311 organizations sampled and 138,354 beneficiaries responded for a response rate of 81 percent. For September of 1999, there were 367 organizations sampled and 166,072 beneficiaries responded for a response rate of 82 percent.

CAHPS® Survey Results

Mean scores on each of the global ratings and composites are presented for each of the 10 HCFA regions for the third year of the MMC CAHPS® survey in Table 1. There is significant interregional variation on almost all the measures in all the years. For example, in year 3, the regional average doctor ratings ranged from 8.7 to 9.1 (Table 2). The highest doctor scores each year tend to be in the Northeast and Mid-Atlantic and the lowest doctor scores tend to be in the Pacific and Northwest regions. Plan ratings tended to have less interregional variation than ratings of doctor, specialist, and health care.

Table 1

Average of Plan Means by the Plan's Primary Federal Region: Year 3

Federal Region	Getting Needed Care	Getting Care Quickly	Doctor Communicate Well	Courteous and Helpful Staff	Paperwork Information and Customer Service
Northeast	2.852	3.553	3.699	3.825	2.585
North Mid-Atlantic	2.798	3.433	3.632	3.746	2.487
Mid-Atlantic	2.831	3.511	3.678	3.788	2.599
South Atlantic	2.789	3.413	3.625	3.749	2.575
East Mid-West	2.813	3.510	3.646	3.788	2.58
Southwest	2.758	3.414	3.626	3.755	2.573
Mid-West	2.831	3.519	3.646	3.796	2.624
Mountain	2.790	3.513	3.655	3.781	2.629
Pacific	2.747	3.360	3.561	3.716	2.61
Northwest	2.819	3.519	3.651	3.806	2.656
Significance Level	***9.58	***16.02	***14.65	***12.24	***4.05

PR>F is 0.0001

SOURCE: (Zaslavsky, A.M., Landon, B.E., Beaulieu, N.D. and Cleary, P.D., 2000.)

Table 2

Average of Plan Means by the Plan's Primary Federal Region: Year 3

Federal Region	Number of Plans in Region	Doctor	Specialist	Health Care	Health Plan
Northeast	23	9.075	9.031	9.15	8.737
North Mid-Atlantic	39	8.878	8.738	8.885	8.352
Mid-Atlantic	31	9.023	8.967	9.059	8.683
South Atlantic	64	8.862	8.845	8.881	8.639
East Mid-West	55	8.874	8.859	8.982	8.656
Southwest	42	8.809	8.858	8.871	8.614
Mid-West	16	8.807	8.789	8.925	8.704
Mountain	13	8.771	8.761	8.911	8.621
Pacific	57	8.657	8.56	8.673	8.535
Northwest	27	8.756	8.827	8.93	8.682
Significance Level		***12.61	***11.41	***15.34	**3.24

PR>F is 0.0001.

PR>F is 0.001.

SOURCE: (Zaslavsky, A.M., Landon, B.E., Beaulieu, N.D. and Cleary, P.D., 2000.)

The absolute differences in CAHPS® scores among regions may appear modest, but they are substantial compared with typical differences among plans and represent average differences over thousands of beneficiaries. Analyses of data from the first year of the survey showed large variations among market areas within regions and large interplan variations within market areas (Zaslavsky et al., 2000b). That is, the data confirm that the CAHPS® survey has adequate precision to detect differences in consumer experiences across plans and geographic areas.

We do not know what accounts for the variations observed, but several aspects of the results are consistent with other information we have about health care quality. The finding that scores are highest in the Northeast, and lowest in the Pacific and Mountain regions, is consistent with patterns in other data, such as, other HEDIS® indicators. Also, the differential patterns in the measures suggest that they reflect different aspects of plan performance. For example, the relatively large differences in doctor scores may reflect well-known regional differences in practice styles (Table 2). Analyses of results within regions and geographic markets (Zaslavsky et al. 2000b) show complementary patterns. For example, perhaps because many plans in the same market share common physicians, there is relatively little interplan variability in doctor scores. On the other hand, there are significant differences in plan scores, probably because they have distinct policies and administrative procedures.

Comparative information can be useful to plans as they try to identify areas in which to focus quality improvement efforts (Figure 1). Although a lack of differences among plans does not mean there is not a need for improvement, areas in which there are large interplan differences on a rating or composite score provide tangible evidence of the possibility of better performance and can motivate plans to improve those areas of performance.

Figure 1

Getting Care Quickly

Implications for Performance Improvement

Data from the MMC CAHPS® survey provide useful information on variation among plans overall and by dimensions of plan performance. While the initial purpose of the MMC CAHPS® was to obtain information that could be used by Medicare beneficiaries to compare plan options, the information also can be a valuable tool for monitoring health plan performance and managed care quality improvement efforts.

The documented variations in health plan performance, previously described, provide evidence that some plans are more effective than others in meeting the needs of their Medicare enrollees. The ratings of individual plans encompass overall performance, provider performance, processes of and access to care. These data offer the opportunity to identify specific dimensions of health plan performance that are of concern and to develop mechanisms to encourage and assist these plans to improve.

As an initial step in that direction, HCFA is currently providing detailed information on individual health plan ratings to Medicare health plans, peer review organizations, (PROs) and to HCFA staff with responsibilities for monitoring and oversight of Medicare health plans.

Health Plans—Receive a yearly detailed report of their MMC CAHPS® scores on overall ratings and specific measures. In addition, to assist the health plans on how to evaluate their scores, State means of the MMC CAHPS® scores are provided. This information can be used by each plan to assess areas of good performance and areas where members are indicating that they are less successful. Ideally, health plans would use these data to identify problems and areas for improvement and would develop strategies that would result in improvements in areas where they are less successful in meeting the needs of Medicare beneficiaries. Since the MMC CAHPS® is only in its third year (and Medicare plans have received just two reports), there is little evidence to date about how much and in what ways the CAHPS® information is encouraging health plans to change.

PROs—Receive an annual MMC CAHPS® report that provides detailed information on the CAHPS® ratings of each Medicare health plan in its region. PROs can use these data to identify problems in service to Medicare beneficiaries enrolled in health plans, both in individual health plans and overall plans. Identification of these problems could then provide a basis for developing targeted approaches for encouraging improvements in health plan performance.

HCFA Central and Regional Offices— Individual health plan ratings and summary ratings, by region, are provided to central and regional office staff who have responsibility for oversight and monitoring of the M+C program. The MMC CAHPS® information, in combination with the knowledge that HCFA staff have about specific Medicare health plans, offers the opportunity to assess and compare health plan performance and to identify potential problem areas in specific health plans. This information, in turn, could be used by HCFA staff to focus on particular organizational components and their operations during periodic site visits to Medicare health plans, in order to evaluate the extent to which they are in compliance with HCFA requirements. Additionally, HCFA staff could encourage these health plans to make changes that would improve performance, overall and by component.

In addition to providing information of Medicare health plan ratings to plans, PROs, and internal staff, HCFA also is using MMC CAHPS® data to better understand the relationships between Medicare health plan CAHPS® ratings and factors that affect these ratings (Landon et al., 2001). Results of these analyses may be used, in the longer term, to assist health plans, PROs, and HCFA to develop strategies for better meeting the needs of all Medicare beneficiaries enrolled in M+C plans.

Finally, HCFA is also supporting extensive research on differences in the health plan experiences of subgroups of the Medicare population, including those with exceptional needs for health care (e.g., the Medicare disabled under age 65, those in fair or poor health and/or with limited independence), racial and ethnic minorities (e.g., black persons, Hispanics, and Latinos), and Medicare-Medicaid dually eligible beneficiaries (Cox and Langwell, 2000). The purpose of these analyses is to gain insight into the performance of Medicare health plans in meeting the needs of those beneficiaries who may have atypical health care needs or who may face cultural, linguistic, or other barriers to care. Results may be used by HCFA to identify problems faced by beneficiaries who may require additional efforts by health plans to facilitate their effective use of services. Health plans could then be encouraged to make changes that would improve access, service, and outcomes for all Medicare enrollees.

Implications for Health Policy

The MMC CAHPS® has been in place for only 3 years. Thus, its potential for use as a tool to assist health plans, HCFA, and PROs to improve performance is still evolving. It is clear, however, that the MMC CAHPS® could be an extremely useful means for identifying areas for Medicare health plan improvement. Evidence to date, suggests that HCFA and PROs will use this information to design strategies for encouraging health plans to make changes in their organization and operations that will offer increasing value and service to their Medicare enrollees.

HCFA is moving toward value-based purchasing, which is a way to encourage and financially reward managed care organizations that provide better than average quality to M+C beneficiaries. MMC CAHPS®, in combination with other indicators compiled by HCFA, will be used to more fully manage the M+C program and to reward plans that demonstrate high performance in serving and meeting the needs of the Medicare population. To date, value-based purchasing has not been implemented by HCFA, but several possibilities are being explored. For example, HCFA could design an incentive rewards system that would offer M+C plans higher payments, reduced regulatory burden, or other rewards for maintaining high performance and for demonstrating significant improvements in key areas of quality and service. The MMC CAHPS® offers the opportunity to design incentive rewards to improve health plan performance based on consistent, reliable, and continuously collected data.