Potential unintended consequences due to Medicare's "no pay for errors" rule? A randomized controlled trial of an educational intervention with internal medicine residents.

BACKGROUND: Medicare has selected 10 hospital-acquired conditions for which it will not reimburse hospitals unless the condition was documented as "present on admission." This "no pay for errors" rule may have a profound effect on the clinical practice of physicians.
OBJECTIVE: To determine how physicians might change their behavior after learning about the Medicare rule.
DESIGN: We conducted a randomized trial of a brief educational intervention embedded in an online survey, using clinical vignettes to estimate behavioral changes. PARTICIPANTS: At a university-based internal medicine residency program, 168 internal medicine residents were eligible to participate. INTERVENTION: Residents were randomized to receive a one-page description of Medicare's "no pay for errors" rule with pre-vignette reminders (intervention group) or no information (control group). Residents responded to five clinical vignettes in which "no pay for errors" conditions might be present on admission. MAIN MEASURES: Primary outcome was selection of the single most clinically appropriate option from three clinical practice choices presented for each clinical vignette. KEY
RESULTS: Survey administered from December 2008 to March 2009. There were 119 responses (71%). In four of five vignettes, the intervention group was less likely to select the most clinically appropriate response. This was statistically significant in two of the cases. Most residents were aware of the rule but not its impact and specifics. Residents acknowledged responsibility to know Medicare documentation rules but felt poorly trained to do so. Residents educated about the Medicare's "no pay for errors" were less likely to select the most clinically appropriate responses to clinical vignettes. Such choices, if implemented in practice, have the potential for causing patient harm through unnecessary tests, procedures, and other interventions.

RCT Entities:   Population Interventions Outcomes

INTRODUCTION

On 01 October 2008, Medicare’s “no pay for errors” rule went into effect for every acute care hospital in the United States1. This policy is designed to improve the care of Medicare patients by incentivizing hospitals to prevent designated hospital-acquired adverse events. Medicare will not reimburse the hospital for increased costs due to any of these adverse events unless documented by the physician as having been present on admission.

While it is important to understand how physicians may change their practices as a result of these new reimbursement rules, it may be particularly important to understand the impact on residents. The Accreditation Council for Graduate Medical Education (ACGME) has mandated that trainees achieve competency in systems-based practice, which is described as “awareness of and responsiveness to the larger context and system of health care”2. The implementation of this new Medicare rule provides the ideal conditions for a natural experiment to understand whether residents are achieving this competency. Residents care for a large number of Medicare inpatients3,4 and potential changes in their practices as a consequence of this rule may have clinical ramifications for these patients. Moreover, characterizing the resident physician response to the rule provides a framework to understand the potential for “no pay for errors” rules to result in unintended consequences throughout our healthcare system.

We performed a randomized trial of a brief educational intervention, seeking to determine whether more knowledge of the changes in Medicare reimbursement might lead resident physicians to provide care that sought to maximize revenue while conflicting with evidence-based guidelines.

METHODS

Sites and Subjects

On October 1st, 2008, the Medicare rule on hospital-acquired conditions went into effect. We conducted our study at the University of California, San Francisco (UCSF) between December 2008 and March 2009. None of the three clinical sites at which residents rotate made a systematic effort to educate residents about the rule (except for one lecture attended by 7% of the residents) or encourage changes in documentation or practices because of the rule.

All UCSF internal medicine residents were eligible to participate (N = 168). The study was approved by UCSF’s institutional review board.

Randomization

Eighty-four residents were assigned to each group by equal size block-randomization using the Microsoft Excel © computerized random number generator.

Intervention

We designed our brief educational intervention to resemble common ways that new billing, reimbursement, and documentation information is conveyed to clinicians, both in content and in appearance. Clinicians often receive emails and brief communications from various stakeholders seeking to improve reimbursement, providing information meant to influence documentation and billing practices. At our institution, physicians frequently receive emails from medical staff leaders on important patient safety, quality, and reimbursement issues comprised of a single paragraph summary of the issue followed by a bulleted list of salient data. Following this format, the initial part of our intervention was a one paragraph explanation of the rule, followed by a list of Medicare’s ten “no pay for errors” conditions. This statement was integrated into the survey for the intervention group and was visible on a standard sized computer monitor without scrolling.

Additionally, to simulate clinical reminders that aim to influence both the documentation and clinical practices of physicians5,6, we inserted a single sentence prior to each clinical vignette in the intervention group, reminding the subject of the applicable “No pay for error” condition.

Measurements

To measure potential changes in behavior as a result of the intervention, we used an experimental vignette design7. In this study design, two groups of subjects are presented with identical vignettes and asked to make selections on multiple-choice clinical questions. One group is presented with additional information (intervention group) in order to determine how that information may influence their choices in comparison to the control group (Fig. 1).

Figure 1

Study flow of participants.

Experimental vignettes were initially developed by the principal investigator (SM), and then reviewed and modified by UCSF experts in the fields of medical education, outcomes research, and quality improvement. Questions measuring resident knowledge and attitudes underwent the same iterative process. The survey and vignettes were felt to represent strong content validity, and were pre-tested on non-internal medicine housestaff and recent internal medicine residency graduates.

Each vignette was designed to have a single most clinically appropriate answer. This correct clinical choice, however, might place reimbursement at risk. Two other answers were presented that were clinically inappropriate but were designed to protect hospital reimbursement in the face of the new policy. For example, for hospital-acquired “vascular catheter-associated infection,” a vignette described a patient without evidence of infection who needed a central line. Three options were presented: one option represented appropriate care (place the central line) and two reflected unnecessary or inappropriate care (place the line and send blood cultures or place the line and start prophylactic antibiotics) that might be seen as helping to defend hospital reimbursement in the context of the new Medicare policy. The final survey including the clinical vignettes is shown in the Online Appendix.

The primary outcome for this study was the selection of the pre-designated most clinically appropriate choice for the clinical vignettes.

Data Collection

The pre-randomized groups received identical email invitations, and were then directed to either the intervention or control group online survey, which used the proprietary Zoomerang © platform. A coffee card was offered as incentive to complete the survey. The survey was available from December 2008 through March 2009, and periodic reminders were sent to residents who had not completed it. Participation was voluntary, and consent was implied by completing the survey.

Statistical Analysis

In addition to direct comparisons between groups, we performed multivariate logistic regression to adjust for confounding baseline characteristics. Residents were dichotomized by baseline knowledge, by their sense of personal responsibility to “know new Medicare rules regarding hospital reimbursement,” and by the amount of teaching they had received about Medicare reimbursement prior to the survey. We used logistic regression to adjust for these baseline characteristics and determined odds ratios and 95% confidence intervals (95% CI). To compare the attitudes of the control and intervention groups, the means of responses on a five point Likert scale of the control group were compared to the means of the intervention groups’ responses using t-tests, and p-values were obtained. All analyses were carried out using SAS, version 9.2 (SAS Institute, Inc., Cary, NC).

RESULTS

Respondent Characteristics

Of 168 internal medicine residents surveyed, 119 responded (71%). There were no significant differences in the control and intervention groups in age breakdown, post-graduate year, or number of hours of teaching about Medicare reimbursement. The respondents were equally distributed across post-graduate year (Table 1).

Table 1

Demographic Information of Responding Internal Medicine Residents

Characteristic	Control group, N (%) n = 58	Intervention group, N (%) n = 61	P =
Age
<30	28 (48%)	25 (41%)	0.4237
>30	30 (52%)	36 (59%)
PGY
1	21 (36%)	20 (33%)	0.5462
2	16 (28%)	22 (36%)
3	21 (36%)	18 (36%)
4	0	1 (2%)
Hours of teaching about Medicare
None	23 (40%)	24 (39%)	0.9826
1–2	26 (45%)	26 (43%)
3–4	8 (14%)	10 (16%)
5 or more	1 (2%)	1 (2%)

Changes in Responses to Clinical Vignettes as a Result of Being Informed of the New Rule

Residents in the intervention group tended to be less likely than those in the control group to select the most appropriate clinical practice choice in the clinical vignettes. In four out of five clinical vignettes, a lower percentage of residents in the intervention group selected the most appropriate choice; two of these four differences were statistically significant (Table 2). These results were unchanged after a multivariate logistic regression adjusting for baseline knowledge, self-described responsibility to know about new Medicare reimbursement rules, self-described awareness of new Medicare rule, and hours of teaching about Medicare reimbursement (not shown).

Table 2

Responses to Clinical Vignettes-Percentage Selecting Most Appropriate Clinical Practice Choice

	Control Group, N (%)	Intervention Group, N (%)	P =
Case 1: Catheter-Associated Urinary Tract Infection (1).	55 (94.8%)	53 (86.9%)	0.13
Case 2: Vascular Catheter-Associated Infection.	52 (89.7%)	48 (78.7%)	0.10
Case 3: Catheter-Associated Urinary Tract Infection (2).	54 (93.1%)	57 (93.4%)	0.94
Case 4: Deep Vein Thrombosis following total knee replacement.	38 (65.5%)	28 (45.9%)	0.03
Case 5: Stage III and IV Pressure Ulcers.	32 (55.1%)	20 (32.8%)	0.01

Baseline Knowledge Regarding Medicare Reimbursement Rules

There were no significant differences between the control and intervention groups in the baseline knowledge of residents regarding Medicare reimbursement rules. The large majority of residents were aware of the basics of Medicare, but most harbored misconceptions about the scope of the new “no pay for errors” policy, with 83% [n = 99] believing incorrectly that hospitals would receive no reimbursement at all if a patient had a preventable adverse event, and 87% [n = 103] believing that Medicare has forecasted over one billion dollars in cost savings from the policy, a vast overestimate8.

While residents tended to know that documenting that a “preventable adverse event” was present on admission allowed for full Medicare payment for that condition [n = 108, 91%], they were unsure as to what the specific conditions were. For example, residents knew that conditions such as “Foreign Object Retained after Surgery,” [n = 117, 98%] and “Falls and Trauma Occurring in the Hospital,” [n = 114, 96%] were considered preventable complications by Medicare. However, only 52% [n = 62] knew tham “Manifestations of Poor Control of Blood Sugar Levels” was a “no pay” condition and only 67% [n = 80] knew that “Pulmonary Edema from Excessive Intravenous Fluids” was not.

Attitudes Around Medicare Reimbursement Rules and Preventable Complications

Residents strongly agreed that it was their responsibility to reduce hospital-acquired infections and preventable complications in their patients [using a five-point Likert scale with 5 = strongly agree, 1 = strongly disagree; mean score = 4.8, SD 0.6]. They also agreed that it was their responsibility to know new Medicare rules affecting hospital reimbursement [mean 3.8, SD 0.9] and to change their documentation practices based on new Medicare reimbursement rule [mean 3.7, SD 0.9]. Residents did not feel, however, that they had been adequately trained in reimbursement, disagreeing with the statement “I have received sufficient training about Medicare reimbursement during residency to document appropriately in the medical record,” [mean 1.8, SD 0.8] (Table 3).

Table 3

Internal Medicine Residents’ Attitudes Around Medicare Reimbursement Rules and Preventable Complications

	Control, n = 58	Intervention, n = 61		Total, n = 119
“Please describe your agreement or disagreement with the statements listed below”	Mean [SD]	Mean [SD]	p =	Mean [SD]
It is my responsibility to reduce hospital acquired infections and preventable complications in my patients	4.8 [0.6]	4.8 [0.6]	0.94	4.8 [0.6]
It is my responsibility to know new Medicare rules affecting hospital reimbursement	3.8 [1.0]	3.9 [0.9]	0.66	3.8 [0.9]
It is my responsibility to change my documentation practice based on new Medicare rules affecting hospital reimbursement	3.6 [1.0]	3.8 [0.8]	0.31	3.7 [0.9]
Medicare’s new “No pay for errors” policy will unfairly penalize hospitals	3.5 [1.0]	3.4 [1.0]	0.57	3.4 [1.0]
Medicare’s new “No pay for errors” policy will unfairly increase physician workload	3.3 [1.0]	3.1 [1.0]	0.32	3.2 [1.0]
It is my responsibility to improve reimbursement for the hospital	3.0 [1.1]	3.1 [1.0]	0.81	3.1 [1.1]
I have received sufficient training about Medicare reimbursement during residency to document appropriately in the medical record	1.6 [0.8]	1.9 [0.8]	0.03	1.8 [0.8]

5 = Strongly agree, 4 = Agree, 3 = Neutral, 2 = Disagree, 1 = Strongly disagree

DISCUSSION

This randomized trial of a brief educational intervention of internal medicine residents showed that residents who were informed about the Medicare “no pay for errors” rule tended to be less likely to choose the most appropriate clinical practice choices in response to clinical vignettes. While actual behaviors were not measured in this study, if these clinical practice choices were to be implemented in practice, they could result in patient harm through unnecessary tests, procedures, and other interventions. Most residents were aware of the new rule, but most had important misconceptions regarding the scope and content of the rule, revealing an important gap in resident competence in systems-based practice.

There is a growing trend in the United States for insurers to use incentives and disincentives in an attempt to lower costs and increase quality of care9. One result of this movement is that clinicians now frequently receive brief communications seeking to modify their documentation and billing practices. It is critical for practicing physicians to be aware of these new regulations, as well as to appropriately respond to the steady stream of information, which is often limited and may well be misleading, meant to influence their clinical practices.

It has long been recognized that societal systemic “purposive” changes can result in unintended consequences10. This phenomenon has been described in other large health systems changes, such as the implementation of the “time to antibiotics for pneumonia” performance measure, which may result in the inappropriate early use of antibiotics in an attempt to excel on a performance measure11–13. Similarly, the physician response to the new “no pay for errors” rules could have major consequences, both intended and unintended14,15. Our study supports concerns that giving clinicians information regarding “no pay for errors” and other reimbursement rules may lead to unintended consequences with the potential to harm patient care unless such education balances individual patient needs with a more systems and reimbursement-based emphasis.

These findings have implications for both graduate medical education and for the care of hospitalized patients in general. Most importantly, in the face of financial incentives and disincentives meant to influence their behavior, physicians must continue to make patient-centered, evidence-based clinical decisions. Our study demonstrates how physicians may infer that they are expected to perform certain actions simply as a result of being informed about a new rule. Although physician professionalism should act as a countervailing force when inevitable tensions between reimbursement and patients’ best interests arise, history tells us that reimbursement-centered decisions sometimes prevail16,17. Therefore, when providing information to clinicians about changes in payment policies, the underlying goal should be emphasized: increased patient safety and quality of care, not necessarily increased reimbursement.

This study has several limitations. Most importantly, since clinical vignettes variably predict true clinical behavior18–21, it is not possible to generalize these results to actual clinical practices. Based on this study, we are unable to determine whether residents changed their actual behaviors as a result of being informed of new reimbursement rules, and if so, whether this had a deleterious effect on patients. Secondly, the clinical vignettes were reviewed by experts in medical education, outcomes research, and quality improvement and were felt to have strong content validity, however, they did not undergo external validation. Finally, this study was performed with internal medicine residents at a single university-based training program. Therefore, these results may not be applicable to practicing clinicians or to training programs in community-based hospitals.

“No pay for errors” rules and similar incentives meant to lower costs and improve quality and safety are here to stay. Our data support concerns that these rules can result in unintended consequences. Therefore, education in systems-based practice should teach trainees about these and other reimbursement rules, while strongly emphasizing the need to follow evidence-based medicine and a patient-centered approach. Further clinical research will be needed to determine if physician practices actually change after the implementation of “No pay for errors” rules, and whether these changes positively or negatively impact the care of patients.

Below is the link to the electronic supplementary material.

(PDF 84 kb)