Disenrollment information and Medicare plan choice: is more information better?

To help Medicare beneficiaries and their intermediaries select the best health plan, CMS publicly reports comparative plan information. Using a laboratory version of Medicare Health Plan Compare that involved a simulated plan choice by 359 Medicare intermediaries, we experimentally investigated plan recommendations with and without disenrollment information and time constraints for viewing materials. Results indicated that the presence of disenrollment information reduced time spent on other measures of plan performance. It also reduced decision quality for less educated intermediaries. Designers and sponsors of consumer-oriented materials should recognize that more information is not always better.

Introduction

The 1997 Balanced Budget Act (BBA) required CMS to provide information on Medicare and comparative health plan information, including information on cost sharing, basic and supplemental benefits, beneficiary satisfaction, plan performance, and disenrollment. The BBA directed CMS to provide this information to beneficiaries through the mail, telephone, Internet, and partner organizations (Gogloff, 2003).

Shortly after the BBA was enacted, CMS created the National Medicare and You Education Program (NMEP) “…to help people become more active participants in their heath care decisions.” The primary initial objectives were ensuring that beneficiaries (1) receive accurate, reliable, and relevant information; (2) can access such information when they need it; (3) understand it well enough to make informed choices; and (4) perceive the sources of information as trustworthy and credible. Toward these ends, NMEP implemented an ambitious set of activities, including the development and dissemination of new information through print (Medicare and You Handbook), toll-free hotline (1-800-Medicare), Internet (www.medicare.gov) and partners (e.g., the National Partners Alliance Network and the State Health Insurance Assistance Programs); a national publicity campaign; and research and assessment (Gogloff, 2003).

Helping Medicare beneficiaries make informed plan choices is a formidable task (Hibbard et al., 2001; 1998). Medicare beneficiaries face complex health plan choices (McCormack et al., 2001), their understanding of plan options is limited (McCormack et al., 2002), and many beneficiaries face physical, literacy, and cognitive impairments to reading, processing, and using health information (Sofaer et al., 2001).

Beneficiaries often rely on family and friends to help them with health-related choices, including plan choices (Sofaer et al., 2001). An estimated 27 percent of the 178,000 daily visitors to www.medicare.gov in 2000 were friends or relatives of beneficiaries (Centers for Medicare & Medicaid Services, 2001). But we know little about how such intermediaries use the Medicare Web site. In this study, we investigated the behavior of intermediaries, defined as relatives of Medicare beneficiaries who aid in their decisions regarding Medicare, by asking them to explore a laboratory version of the www.Medicare.gov Web site and recommend a plan for a family member on Medicare.

We also investigated whether including plan disenrollment information, which CMS started publicly reporting in 2000, affects plan recommendations. Disenrollment information includes the overall rate of voluntary disenrollment from plans, as well as the rates due to specific reasons. CMS was already publicly reporting information on beneficiary experiences, technical quality of care, and cost and benefits, all of which have been shown to affect plan choices in sensible ways (Atherly et al., 2004; Beaulieu, 2002; Schoenbaum et al., 2001; Spranca et al., 2000; Uhrig and Short, 2002/2003). If consumers view disenrollment information as redundant with or inferior to these other measures of plan performance (Lied et al., 2003), disenrollment information might overload or distract them. However, if they regard it as valuable new information, or as a convenient summary of these other measures of performance, it might facilitate their decisionmaking (Harris-Kojetin et al., 2002).

Finally, we investigated the effects of time pressure on decisionmaking. Time pressure is ubiquitous in real life, yet often ignored when designing and testing educational materials or choice environments (Schwartz, 2004; Teleki et al., 2007). Decisionmakers under time pressure often (1) accelerate their processing of information (Ben Zur and Breznitz, 1981); (2) simplify their choice strategy, shifting from compensatory to non-compensatory strategies (Payne, Bettman, and Johnson, 1993); or (3) focus on a subset of available information (Edland, 1994; FallHauser, Urban, and Weinberg, 1993). Each of these responses can compromise decision quality. The effects of time pressure on real life decisionmaking might explain the fact that educational materials often work well in the laboratory, where subjects typically face no time constraints, but less well when they are disseminated in the real world.

Method

Subjects

A random digit dialing method was used to recruit participants for the study and three pilot testing sessions from the Los Angeles area. Eligible participants were adult U.S. citizens age 65 or under who were comfortable reading and writing English and using a computer, and who had a family member and/or a partner on Medicare whom they had assisted with health-related decisions or whom they intended to assist in the future.

Table 1 shows the characteristics of the 359 Medicare intermediaries who participated in the study. Fifty-eight percent were female. Nearly one-half were age 45-54 (43 percent); only 10 percent were age 18-34. Fifteen percent were Black or African-American and 9 percent were Hispanic. Most had college degrees (62 percent), used the Internet more than once a day (51 percent), and had used it to find health information (80 percent).

Table 1

Self-Reported Characteristics of the Sample of Medicare Intermediaries Used in Study

Characteristic	Percent
Sex
Female	58
Male	42
Age
18-24 Years	1
25-34 Years	8
35-44 Years	30
45-54 Years	43
55-65 Years	17
Race/Ethnicity
American Indian or Alaskan Native	<1
Asian or Pacific Islander	5
Black or African American	15
Hispanic or Latino	9
White	67
Other	3
Education
Some High School	<1
High School Graduate or GED	7
Some College (1 -3 Years)	31
College Graduate	28
Some Graduate Training	13
Graduate Degree	21
Household Income
< $20,000	1
$20,001-$40,000	9
$40,001-$60,000	20
$60,001-$80,000	23
$80,001-$100,000	19
> $100,000	27
Relationship to Beneficiary
Child	74
Spouse/Partner	1
Other Family Member	24
Frequency of Computer Use
Never Used a Computer	0
Between Once a Year and Once a Week	4
Once a Day	19
More than Once a Day	76
Frequency of Internet Use
Never	<1
Between Once a Year and Once a Week	20
Once a Day	30
More than Once a Day	51
Ever Used Internet to Find Health Information?
Yes	80
No	20

N=359.

SOURCE: Spranca, M., Elliott, M., Shaw, R., and Kanouse, D., RAND Corporation, Santa Monica, CA, 2001.

We also asked intermediaries to describe the Medicare beneficiary they had in mind while looking at the Medicare Health Plan Compare Website and completing the survey. Table 2 shows beneficiary characteristics as reported by the intermediary. Intermediaries most often said that they were thinking of their mother (51 percent) or father (24 percent) when recommending a health plan. A majority described the beneficiary's health as being fair or poor (55 percent), with 56 percent saying that the beneficiary had high blood pressure. Intermediaries reported that 50 percent of beneficiaries had been enrolled in Medicare for 2 to 10 years.

Table 2

Characteristics of the Medicare Beneficiaries, as Reported by the Medicare Intermediaries

Characteristic	Percent
Sex1
Female	51
Male	24
Unknown	25
Overall Health Status
Excellent	3.9
Very good	12.4
Good	28.4
Fair	37.1
Poor	18.3
Medical Illnesses or Conditions
High Blood Pressure	55.7
Diabetes	22.3
Alzheimer's Disease	8.1
Stroke	9.5
Cancer	12.8
Asthma	8.9
Heart Disease	22
Length of Time on Medicare
23 Months or Less	9.3
2-5 Years	18
6-10 Years	31.5
> 10 Years	41.3
Health Plan Changes
Switched from Medicare to Medicare Managed Care	12.8
Switched from one Medicare Managed Care plan to Another	13.9
Switched from Medicare Managed Care to Medicare	4.5
Enrolled in Medicare Managed Care
Yes	26.1
No	40.9
Don't Know	33.1

Inferred from relationship to intermediary.

SOURCE: Spranca, M., Elliott, M., Shaw, R., and Kanouse, D., RAND Corporation, Santa Monica, CA, 2001.

Study Design

The laboratory experiment used a 2 × 2 factorial design of disenrollment information and time constraint, within which there was a fractional factorial design of cost/benefit levels, disenrollment levels, and quality for the four plan alternatives. Eight versions of the laboratory experiment were used, since there were two sets of four choices in each of the four conditions (Table 3).

Table 3

Full Design of Laboratory Experiment Showing Conditions and Plan Combinations

Design1	Version1	Managed Care Plan2	Costs/Benefits	CAHPS®/HEDIS®	Disenrollment Rate Percentage
					Overall	Due to
						Costs	Quality3
Condition 1
No Disenrollment Information with No Time Constraint	1a	Coastal	High	Low	—	—	—
		Valley	Low	Low	—	—	—
		Canyon	High	High	—	—	—
		Mountain	Low	High	—	—	—
	1b	Coastal	Low	Low	—	—	—
		Valley	High	Low	—	—	—
		Canyon	Low	High	—	—	—
		Mountain	High	High	—	—	—
Condition 2
No Disenrollment Information with Time Constraint	2a	Coastal	High	Low	—	—	—
		Valley	Low	Low	—	—	—
		Canyon	High	High	—	—	—
		Mountain	Low	High	—	—	—
	2b	Coastal	Low	Low	—	—	—
		Valley	High	Low	—	—	—
		Canyon	Low	High	—	—	—
		Mountain	High	High	—	—	—
Condition 3
Disenrollment Information with No Time Constraint	3a	Coastal	High	Low	17	6	11
		Valley	Low	Low	8	3	5
		Canyon	High	High	8	3	5
		Mountain	Low	High	4	1	3
	3b	Coastal	Low	Low	17	6	11
		Valley	High	Low	8	3	5
		Canyon	Low	High	8	3	5
		Mountain	High	High	4	1	3
Condition 4
Disenrollment Information with Time Constraint	4a	Coastal	High	Low	17	6	11
		Valley	Low	Low	8	3	5
		Canyon	High	High	8	3	5
		Mountain	Low	High	4	1	3
	4b	Coastal	Low	Low	17	6	11
		Valley	High	Low	8	3	5
		Canyon	Low	High	8	3	5
		Mountain	High	High	4	1	3

Each condition shows one of the four factorial combinations of disenrollment information and time constraint. Each version shows the set of four plans that intermediaries reviewed and selected from.

Although realistic in appearance, the plan designs shown are all hypothetical Medicare Managaed Care Plans.

Disenrollment due to costs included concerns about the plan costing too much and not paying for particular benefits. Disenrollment due to quality included problems with the care or services they received, such as not seeing the doctors that they wanted to or other non-financial issues.

NOTES: CAHPS® is Consumer Assessment of Health Plans Study. HEDIS® is Health Plan Employer Data and Information Set.

SOURCE: Spranca, M., Elliott, M., Shaw, R., and Kanouse, D., RAND Corporation, Santa Monica, CA, 2001.

We created four versions of the Medicare Health Plan Compare Web site that closely matched the original in design and content. The site provided information on four hypothetical Medicare managed care plans that could have been offered in the participants' geographical area. The neutral names (Coastal, Valley, Canyon, and Mountain) of the four hypothetical health plans were intended to avoid associations with actual health plans.

The Web site described the costs and covered services associated with the four plans, and displayed comparative data on quality as measured by CAHPS® and the Health Plan Employer Data and Information Set (HEDIS®). In order to test the impact of disenrollment information, two versions of the laboratory Web site also included disenrollment data.

Relationships Among Plan Characteristics

To enhance realism, we built into the design certain relationships among the independent variables, which are summarized as the following:

Within plans, cost and benefits are perfectly positively correlated (two low and two high on both characteristics).

Cost/benefits and CAHPS®/HEDIS® (quality) are uncorrelated within each version and overall.

Cost/benefits and disenrollment rates have a moderately positive correlation in versions 3a and 4a and a moderately negative correlation in versions 3b and 4b.

Quality and disenrollment rates are moderately negatively correlated.

Costs and Benefits

The specific costs and benefits for each hypothetical plan approximated the 20th and 80th percentiles of what was offered at the time by Medicare managed care plans operating in southern California. The large difference between the low and high cost/benefit plans was intended to force participants to make a clear tradeoff.

We displayed this information using the layout from the then current Medicare Health Plan Compare site, but with simplified and standardized coverage and cost descriptions that highlighted differences in boldface. Displaying somewhat less information than is on the Web site allowed users to view all information within the experimental session.

Quality

The quality section of the Web site showed plan ratings for six HEDIS® measures and eight CAHPS® measures. Two of the plans in each set were given low quality ratings and two were given high quality ratings. For each of the two low-quality plans, the score for each measure was generated as the average of 10 randomly selected plans below the median on that measure. Scores for high-rated plans were similarly derived, except that the random plans were drawn from those above the median.

Disenrollment Rates and Reasons

We assigned a high disenrollment rate (17 percent) to one plan, a moderate rate (8 percent) to two plans, and a low rate (4 percent) to the fourth plan. These rates corresponded to the 80th, 50th, and 20th percentiles, respectively, in CMS' database of disenrollment rates for Medicare managed care plans. At each of the three levels, the proportion who gave specific reasons for disenrollment corresponded to the overall national distribution of reasons among those who disenrolled.

Time Constraints

Participants in one-half of the sessions were given only 15 minutes to explore the laboratory version of the Medicare Health Plan Compare Web site before being asked to recommend a health plan. This time was determined in pretesting to be short enough that participants would feel pressure to review the site quickly and make deliberate decisions about the order in which to view information and how much time to allocate to each section. We did not want the viewing time to be so short, however, that participants would feel that the task was impossible and thus not worth taking seriously. Team members reviewed the original Medicare Health Plan Compare Web site and found that it took approximately 30 minutes to read all content on every page. CMS' testing of the Medicare Health Plan Compare site found that most visitors took 15 to 30 minutes to review the site and most focus group participants reported that they would not spend more than 30 minutes reviewing the site in the real world.

Procedure

Each laboratory session was conducted in RAND's computer classroom in groups of up to 10 participants. Sessions began with instructions and a brief explanation of the purpose of the study. Participants were told to imagine that their family member on Medicare had asked them for advice on choosing a new Medicare managed care plan. One-half of the groups were told that they could take as much time as they wanted to look at the site and learn about the plans. The others were told that they had a 15-minute time limit to review the plan information.

All were told that they could take notes while viewing the site, and that after finishing, they would fill out a two-part questionnaire. They were not allowed to look at the Web site while answering questions about their recommendations in order to assure that these plan recommendations were based on what they learned during the allowed period. For the second part of the questionnaire, they were allowed to go back to the site.

In time-constrained conditions, facilitators directed participants to an end page link that marked when the participant completed the session in the server log at the end of the 15-minute time period. Facilitators passed out part one questionnaires, verified that the computers were on the correct Web page, and reminded participants that they should remain on the end page while completing the first part of the questionnaire. Those without time constraints indicated that they were finished viewing the site by raising their hand, prompting facilitators to individually directed them to the end page and provide the questionnaire.

After participants completed part one, they raised their hands and received the second part. After participants completed both parts they received a $75 payment.

Measures

Data came from questionnaires completed by participants and server logs. Questionnaires included questions about participants' health plan choice and preferences, usability and usefulness of the Web site, knowledge questions to test if they understood certain types of information, and demographic questions. Questionnaires for participants who received disenrollment information included questions about disenrollment information.

The server logs contain the exact Web pages viewed, the order in which pages were viewed, and the exact time at which viewing started and ended.

Method for Correcting Time Spent Reviewing Site

The session began when the facilitator said to begin reviewing the site; the server log recorded the start as when subjects clicked on the first page of the site. The session ended when the facilitator said to end (in the time constraint conditions) or when the subject raised her hand to indicate that she was done (in the no time constraint conditions); the server log recorded the end as when the subject or facilitator clicked on the end session button. Although the start time was accurate, the end time is systematically overestimated because of the time between when participants stopped viewing materials and when they arrived at the end page, following the facilitator's instructions. We corrected for this bias by estimating and subtracting the amount of time it took subjects to end their session.

For those assigned to a time constraint condition, we defined the end of a session as 15 minutes if the last section they visited was a non-introductory section, or less than 15 minutes, if the last section they visited was the introduction. For those assigned to a condition without time constraints, we defined the end of a session as the point at which subjects clicked the end session button (per the server log) minus the median amount of time over 15 minutes it took constrained subjects to click the end session button.

Analytic Approach

Evaluations of the Web Site

We performed descriptive statistics of measures regarding the evaluation of the site and the importance of the topic.

Analyses of Time Allocation

We computed the time participants spent viewing the site, overall and for specific sections. A series of two-sample t-tests compared viewing times on the basis of time constraints and the presence of disenrollment information.

Analyses of Plan Recommendations

We modeled choice of recommended plans using McFadden's (1974) conditional logit model. Conditional logistic regression models multicategorical choice as a function of characteristics of the choices themselves. The influence of characteristics that do not vary within a given set of choices for a single subject (e.g., characteristics of the chooser or of the choice set) may be incorporated via interactions with characteristics of choices.

In the present application, each subject chooses among four plans. The experimental conditions that apply to a given subject's set of four choices represent one of the four factorial combinations of time constraint (present or absent) and disenrollment information (present or absent). The time constraint characteristic can be incorporated as an interaction with choice characteristics. The presence of disenrollment information changes the characteristics of the choices, and so this factor is incorporated by using two parallel conditional logit models: one for choice sets without disenrollment information and one for choice sets with disenrollment information. For the former set, cost/benefit and CAHPS®/HEDIS® ratings are the choice characteristics. For the latter set, disenrollment information (three levels, as two dummies) is added. Thus, Models 1- 4 are two pairs of models—a pair limited to main effects of choice characteristics, with and without disenrollment information (Models 1 and 2), and a pair that included interactions of these characteristics with time constraints (Models 3 and 4). Models 5 and 6 removed non-significant interactions from Models 3 and 4.

Finally, we considered four individual characteristics: (1) the education of the intermediary, (2) the intermediary-reported health status of the beneficiary, (3) duration of beneficiary Medicare enrollment, and (4) the importance of cost to the beneficiary (each treated linearly, as 1 degree of freedom). We used mean imputation to fill in what was <1 percent missingness in each of the four individual measures. These were investigated in a series of 20 Models (Models 7-26): 8 models involving those with no disenrollment information added interactions one at a time to Model 3; 12 models each added one interaction to Model 4 for those with disenrollment information. The interactions correspond to all possible interactions of each of the four intermediary or beneficiary characteristics with a single main effect choice characteristic (CAHPS®/HEDIS®, disenrollment, and cost/benefit).

Results

Importance of Disenrollment Information

We asked a series of questions to determine how participants valued having disenrollment data. Of 179 responding participants, 55 percent felt that it was very important and 34 percent said that it was somewhat important to know the proportion of people who chose to leave a plan. Of 153 responding participants, 48 percent felt that the disenrollment information was very useful and 39 percent felt that it was somewhat useful.

We also asked participants about the understandability of the site's disenrollment section. Fifty-eight percent of 154 respondents said that the information was very easy to understand, and 36 percent thought that it was somewhat easy to understand. Forty-six percent of 156 respondents felt that the Web site contained about the right amount of disenrollment information, 34 percent would have liked a little more. The two most common suggestions about what additional disenrollment information could be added were including sample sizes and more detailed reasons why people chose to leave (e.g., examples of specific problems people had with their plan).

Time Constraints and Disenrollment Information

As expected, Table 4 shows that the introduction of a time constraint reduced the total time intermediaries spent reviewing the site by more than 3 minutes (14:46 versus 18:22 without disenrollment information, 14:48 versus 17:59 with disenrollment information, p<0.01 for each). Thus, the introduction of a 15-minute time constraint reduced average time spent by more than 20 percent.

Table 4

Effects of Time Constraints and Addition of Disenrollment Data on Time Allocated by Intermediary, Overall, and by Section of Site

Disenrollment Information	No Time Constraint	Time Constraint	Change in Time Spent on Each Section When Time Constraints Are Added
No Disenrollment Data Present	B	A	A – B
Total	18:22	14:46	-3:362
Introduction	1:29	1:28	-0:01
Cost/Benefits	8:19	7:06	-1:131
CAHPS®/HEDIS®	8:34	6:12	-2:222
Disenrollment Data Present	D	C	C – D
Total	17:59	14:48	-3:112
Introduction	2:17	1:33	-0:441
Cost/Benefits	6:43	6:27	-0:16
CAHPSO/HEDIS®	6:24	5:05	-1:192
Disenrollment	2:35	1:43	-0:522
Change in Time Spent on Each Section When Disenrollment Data Are Added	D – B	C – A	—
Total	-0:23	+0:02	—
Introduction	+0:482	+0:05	—
Cost/Benefits	-1:362	-0:39	—
CAHPS®/HEDIS®	-2:102	-1:072	—

p<0.05.

p<0.01.

NOTES: CAHPS® is Consumer Assessment of Health Plans Study. HEDIS® is Health Plan Employer Data and Information Set.

SOURCE: Spranca, M., Elliott, M., Shaw, R., and Kanouse, D., RAND Corporation, Santa Monica, CA, 2001.

How did intermediaries allocate their time when they had only 15 minutes to review the site, as compared to those who were unconstrained (Table 4)? When disenrollment information was not available, they budgeted less time to cost/benefits (-1:13 minutes, p<0.05) and CAHPS®/HEDIS® (-2:22 minutes, p<0.01) sections under time constraints. When disenrollment information was available, they budgeted less time to the introduction (-44 seconds, p<0.05), CAHPS®/HEDIS® (-1 minute and 19 seconds, p<0.01) and disenrollment (-52 seconds, p<0.01) sections under time constraints.

The addition of disenrollment information did not affect the overall amount of time intermediaries spent reviewing the site, even without time constraints (p>0.05), but it did affect the allocation of time across sections of the site beyond the disenrollment section itself. Intermediaries allocated about 1 minute less time to the cost/benefits (p<0.01 with and without time constraints). In the absence of time constraints, the addition of disenrollment information further resulted in about 2 minutes less devoted to CAHPS®/HEDIS® and about 1 minute more devoted to the introduction (p<0.01 for each).

Plan Preferences

Main Effects

Model 1 tested the effect of CAHPS®/HEDIS® and cost/benefit on plan choice in the absence of disenrollment information. As shown in Table 5, intermediaries strongly preferred high CAHPS®/HEDIS® plans (OR=6.11, p<0.01) and weakly preferred low cost/benefits plans (OR=1.33, p<0.05).

Table 5

Odds Ratios of the Effects of Plans Characteristics and Experimental Conditions on Plan Recommendations

Disenrollment Information Present	Model 1	Model 2	Model 3	Model 4
	No	Yes	No	Yes
	(N=712)	(N=716)	(N=712)	(N=716)
Independent Variables
CAHPS®/HEDIS®—High	6.112	4.012	6.492	3.672
Cost/Benefits—Low	1.331	0.95	0.96	0.89
Disenrollment—Low	—	1.42	—	1.45
Disenrollment—Medium	—	0.76	—	0.60
Time*CAHPS®/HEDIS®—High	—	—	0.89	1.17
Time*Cost/Benefit—Low	—	—	2.002	1.12
Time*Disenroll—Low	—	—	—	1.67
Time*Disenroll—Medium	—	—	—	1.01

p<0.05.

p<0.01.

NOTES: CAHPS® is Consumer Assessment of Health Plans Study. HEDIS® is Health Plan Employer Data and Information Set.

SOURCE: Spranca, M., Elliott, M., Shaw, R., and Kanouse, D., RAND Corporation, Santa Monica, CA, 2001.

Model 2 tested the effect of CAHPS®/HEDIS®, cost/benefit, and disenrollment on plan choice in the presence of disenrollment information (OR=4.01, p<0.05). Intermediaries' strong preference for high CAHPS®/HEDIS® plans persists, but their preference for low cost/benefit plans is no longer significant (p>0.05). The disenrollment information also had no significant effect on choice (p>0.05).

Time Constraint

Model 3 tested the interaction of time constraint with CAHPS®/HEDIS® and cost benefit. The presence of the time constraint did not affect preference for CAHPS®/HEDIS®, but did affect preference for cost/benefit (p<0.01). Intermediaries preferred low cost/benefit plans when time constrained (OR=2.00*0.96=1.92, p<0.05) and otherwise showed no preference (OR=0.96, p>0.05).

Model 4 tested the interaction of time constraint with CAHPS®/HEDIS®, cost benefit and disenrollment information. When disenrollment information was present, the time constraint had no effect on preference for CAHPS®/HEDIS®, cost/benefit or disenrollment (p>0.05 for each).

Beneficiary and Intermediary Characteristics

One of the eight interactions was significant at p<0.05 for the intermediaries who did not see disenrollment information. Here, intermediaries who rated cost as important to their family member on Medicare on a three-category scale (very, somewhat, or not important) were considerably more likely to recommend low cost/benefit plans (OR=2.63 per level, p<0.001, results not shown.).

Two of the 12 interactions were significant at p<0.05 for intermediaries who did see disenrollment information (results not shown, p<0.01 for each). Higher education (measured on a six-category scale as shown in Table 1) was strongly associated with a smaller chance of choosing a low CAHPS®/HEDIS® plan (OR=0.54 per level of education, such as high school degree versus some college) and a smaller chance of selecting a high (OR=0.40 per level of education) or medium (OR=0.44 per level of education) disenrollment plan. The magnitude of these coefficients suggests that any statistically significant preference (p<0.05) for higher CAHPS®/HEDIS® were only evident among intermediaries with at least a high school diploma and that statistically significant preferences for lower disenrollment was evident only among those intermediaries with at least a college degree.

Conclusion

Intermediaries strongly preferred plans that scored high on CAHPS® and HEDIS®. This was expected, since these plans always dominated the others, so that intermediaries never had to compromise on other valued plan features. Most intermediaries recommended one of the two dominant health plans5, even when they faced a time constraint and had disenrollment information to process.

However, most of the intermediaries in this study were highly educated, and higher education was associated with recommending a dominant plan. Specifically, when disenrollment information was present, those with less than a college education recommended objectively inferior low CAHPS®/HEDIS® plans at rates indistinguishable from superior plans, suggesting that the added cognitive burden of processing disenrollment exceeded their capacity. This is supported by other results suggesting that adding disenrollment information significantly reduced time studying CAHPS®/HEDIS® information.

How important is disenrollment information in a decision aid that already provides information on CAHPS®/HEDIS®, costs, and benefits? Intermediaries describe the disenrollment information as important and useful. Intermediaries quickly reviewed the disenrollment information, spending about 2-1/2 minutes, compared to more than 6 minutes each for CAHPS®/HEDIS® and cost/benefits. This difference may reflect a judgment that it was less important, or easier to comprehend quickly.

It must be borne in mind that in the absence of explicit tradeoffs, consumers typically endorse additional information as beneficial. However, intermediaries were generally unwilling to trade off their preferred cost/benefit structure for a plan with lower disenrollment rates.

The absence of a disenrollment effect on plan recommendations might have been a consequence of our experimental design. Since disenrollment was moderately negatively correlated with CAHPS® and HEDIS®, limiting one's choice to plans with high CAHPS®/HEDIS® scores automatically excluded plans with the highest (worst) disenrollment rate. Once intermediaries had eliminated the dominated choices, they effectively had a choice of two plans varying along cost/benefit (high versus low) and disenrollment (low versus moderate). The preference for a plan with a 4 versus 8 percent disenrollment rate might not have been strong enough to overcome intermediaries' preference for a low or high cost/benefit plan. Future research could examine the effects on choice of (1) plans with conflicting indicators of quality, and (2) a stronger disenrollment manipulation or weaker cost/benefit manipulation. Nonetheless, given a realistic association between disenrollment and CAHPS®/HEDIS®, this may suggest that disenrollment information adds relatively little beyond CAHPS®/HEDIS® to inform plan choice.

Various types of time constraints, including self-imposed limitations on effort, are typical of the real world circumstances under which people often examine consumer materials. Our experimental manipulation introduced an externally imposed time constraint whose effects may differ from certain other types of naturally occurring constraints. Nonetheless, our results show that even mild time constraints can affect how time is proportionately allocated to different parts of the task and can also affect recommendations, e.g., by encouraging a focus on attributes already considered important or those that are more familiar.

Policy Implications

More information is not always better and is sometimes worse. In deciding whether to report additional information to Medicare beneficiaries and intermediaries, policymakers should carefully consider (1) how conceptually and empirically similar the additional information is to the information already being reported, and (2) the audiences' ability and motivation to process the additional information. As new information becomes available, policymakers should also consider whether it might replace existing information, effectively reducing decisionmaking effort without compromising quality.

On the one hand, there is evidence that disenrollment information is partially redundant with CAHPS® and HEDIS® information and that less-educated intermediaries choose objectively inferior plans in the presence of this information.

On the other hand, disenrollment information has potential to be a simple summary measure of plan performance. Although its empirical relationship to cost/benefit design has not been examined, it should be noted that dissatisfaction with a plan's cost and benefits is a major reason that consumers voluntarily leave health plans.

The target audience may determine which of these contrasting views of disenrollment information is more accurate. More educated audiences and those willing to devote time to materials might effectively use disenrollment information along with the other plan information, even if the marginal benefit is small. Less educated audiences and those unwilling or unable to spend much time might not be well served by additional information, but might prefer the disenrollment information in lieu of the other plan information, even if there is some cost to decision quality. Decision tools could accommodate the information preferences of these very different audiences. However, it is important to note that offering choices of which information to view imposes its own cognitive burden and time costs, since the informational options must be described well enough for users to make an informed choice.

As we contemplate a future with even more plan information, beneficiaries will need more assistance from decision tools and intermediaries. Well-designed decision tools may extend beneficiaries' ability and motivation, effectively enabling them to process more information in less time and with fewer errors. Intermediaries, whether professional (e.g., State Health Insurance Assistance Programs counselors) or lay (e.g., trusted family members), might also help beneficiaries understand their choices and confidently select options that meet their needs.

In conclusion, disenrollment information may draw time and attention away from other plan performance measures, including CAHPS®/HEDIS®, especially for less-educated intermediaries. In our study, disenrollment information reduced the time intermediaries spent on other performance measures, even in the absence of time constraints. Moreover, lower-quality decisions resulted by those with less education. Designers of consumer-oriented comparison materials should recognize that more information is not always better. How much and what kind of information to present should be guided by careful testing that specifically examines the usability and additional value of proposed additions.