Physician Perspectives on Comparative Effectiveness Research: Implications for Practice-based Evidence.

Comparative effectiveness research (CER) is defined by the Institute of Medicine as "the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care." The goal of CER is to provide timely, useful evidence to healthcare decision makers including physicians, patients, policymakers, and payers. A prime focus for the use of CER evidence is the interaction between physician and patient. Physicians in primary practice are critical to the success of the CER enterprise. A 2009 survey suggests, however, that physician attitudes toward CER may be mixed-somewhat positive toward the potential for patient care improvement, yet negative toward potential restriction on physician freedom of practice. CER methods and goals closely parallel those of practice-based research, an important movement in family medicine in the United States since the 1970s. This article addresses apparent physician ambivalence toward CER and makes a case for family medicine engagement in CER to produce useful practice-based evidence. Such an effort has potential to expand care options through personalized medicine, individualized guidelines, focus on patient preferences and patient-reported outcomes, and study of complex therapeutic interventions, such as integrative care. Academic medical researchers will need to collaborate with experienced family physicians to identify significant practice-based research questions and design meaningful studies. Such collaborations would shape CER to produce high-quality practice-based evidence to inform family and community medicine.

Comparative effectiveness research (CER) is defined by the Institute of Medicine as “the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care.” Its goal is to provide timely, useful evidence to a full range of healthcare decision makers, including physicians, patients, policymakers, and payers. A prime focus for this timely, useful evidence is the interaction between the physician and the patient. In this relationship, the one holding chief responsibility to access, understand, evaluate, explain, and apply this evidence is the practicing physician, who is core to the success of the CER enterprise. Despite the centrality of clinicians to the CER effort, however, there is some evidence of physician ambivalence. CER methods and goals closely parallel those of practice-based research (PBR), an important movement in family medicine in the United States since the 1970s., The purpose of this article is to address possible reasons for physician ambivalence to CER and to make a case for family medicine engagement in CER as an important contributor to a body of practice-based evidence. Drawing on outcomes from a 2009 stakeholder symposium, the article also addresses the potential for CER to inform physician-patient assessment of integrative care.

The term CER came sharply into public awareness in the debate over the American Reinvestment and Recovery Act of 2009 (ARRA). ARRA provided $1.1 billion in funding for a broad new CER effort and mandated an Institute of Medicine study to determine CER priorities for national focus. Debate was intense during this time, as summarized by Chen in The New York Times in an article titled, “Does Oversight Threaten the Doctor-Patient Bond?”:

Supporters of the provision believe that such “comparative effectiveness” research will help to identify ineffective therapy, improve quality of care and ultimately decrease the time and effort, not to mention the dollars, spent on treatments that don't work well. But critics warn that such research could ultimately lead to a one-treatment-fits-all approach and that the government could become an unwelcome third party, dictating “appropriate” decisions in the doctor-patient relationship.

In the midst of robust public argument, how did physicians themselves view CER? How might physician perspectives affect the development of practice-based evidence?

PHYSICIAN PERSPECTIVES ON COMPARATIVE EFFECTIVENESS RESEARCH

During the 2009 healthcare reform debate, Keyhani et al carried out a survey, the results of which suggest that physicians—core users of CER—might not be strongly supportive of the CER effort. Findings indicate that, while a large majority (77.8%) supported inclusion of CER data in national guidelines for clinical practice, only a much smaller majority (56.5%) favored the development of such guidelines. Moreover, though almost 2 out of 3 physicians agreed that CER data would be used to restrict their freedom of practice (65.7%), more than half of the sample agreed that CER would improve patient care (55.2%). These paradoxical findings may indicate either that in mid 2009 physicians had not yet heard enough about CER to form a clear opinion or that they were genuinely conflicted about what they had heard. In either case, the findings raise a question: If physicians are uncommitted to the CER effort, can it succeed in its purpose of generating practice-based evidence to improve care delivery?

A fundamental tenet of CER is that it embraces the heterogeneity of real-world clinical populations. CER is carried out in settings more diverse, using study methods more practice-based, than would be typical of a randomized controlled drug trial., CER utilizes the full range of study methods cited by Westfall regarding PBR: “observational studies, physician and patient surveys, secondary data analysis, and qualitative research.” Additional CER approaches include pragmatic controlled trials conducted by physicians and staff in community clinical practices and the analysis of data from patient records, registries, or insurance claims generated in such nonacademic settings.,,– Because the CER effort is dependent on data obtained through the engagement of physicians actively practicing in the community, it is clear that significant ambivalence would be of concern. Keyhani et al did not elicit reasons why many survey respondents did not express support for guidelines or why a majority foresaw restriction of their freedom of practice. Other research may be suggestive.

Practice Guideline Concerns

Currently, physicians have online access to a database of almost 2500 evidence-based clinical guidelines. Additional guidelines based on CER, or even current guidelines revised with new CER data, would add further to the load of information necessary for the practice of standard medicine. Given this burden, Nellesen et al have observed that “limits in the ability of individuals to process data may prove to be among the most difficult barriers to translating evidence from CER to practice.” Beyond the volume of available guidelines, physicians have expressed reservations about guideline content. A synthesis of qualitative research on physician attitudes showed that practice guidelines have been perceived as too general or too inflexible to apply to individual patients and not always relevant in demographically diverse settings.

Epstein and Teagarden have made a similar observation about CER—namely, that “classic comparative effectiveness research focuses on the average benefit across wide populations.” Doubt regarding the relevance of CER findings to individual patients and groups would be reasonable. Keyhani's ambiguous survey results may point to physician concern about whether the CER effort will simply produce more difficult-to-access, difficult-to-apply guidelines, without providing evidence of practical value for patient care.

Restriction of Practice Concerns

A further reason for physician mistrust of practice guidelines is the perception that some are aimed more at cost containment than patient well-being. The need to slow the growth of US healthcare costs looms large in public discussion, even leading on occasion to talk of care rationing. In news and online debate, which lack peer review, CER has at times been incorrectly associated or even equated with cost-effectiveness research., Moreover, because regional variations in clinical practice often are cited as a contributing factor in excessive healthcare costs,, CER might reasonably be viewed as an effort to generate comparative cost data, with negative downstream effects on freedom of practice.

If CER is seen as a movement to treat patients according to a cost-cutting “average benefit” and not according to physician judgment and patient preference, then clinicians may well doubt its value. But does CER promise more than discovery of least expensive best practices for the peak of a bell curve? Can an expanded set of research methods, based in clinical practice, yield evidence for individualized and improved patient care, while decreasing the risks of inappropriate treatment and unnecessary cost?

COMPARATIVE EFFECTIVENESS RESEARCH AND OPTIONS FOR INDIVIDUALIZING CARE

The current move toward effectiveness research is aimed at generating evidence that has “external validity,” ie, direct clinical relevance to patient populations seen in the community. Effectiveness research calls for study designs that reflect conditions of ordinary clinical practice. Such designs may engage heterogeneous populations from different types of settings; collect a wide range of patient data, including data on individual patient values and preferences; evaluate combinations of therapies delivered together; and carry out subgroup analysis to identify characteristics of patients most and least likely to experience positive outcomes., By contrast, the standard randomized controlled trial (RCT) is an “efficacy” study aimed at determining “the extent to which a specific intervention is beneficial under ideal conditions” rather than under conditions of ordinary clinical practice. In a standard RCT, participants are recruited or excluded according to specified criteria, and an intervention is carried out according to a well-defined protocol, often in an academic medical setting with specially trained personnel. Efficacy trials measure 1 or 2 primary endpoints and generally do not support subsequent analysis to identify differential effects on participant subgroups, nor do they typically measure patient-centered outcomes such as quality of life or care satisfaction. Though commonly thought to produce the “gold standard” for reliable clinical evidence, efficacy trials invoke Green's question: “If it is an evidence-based practice, where's the practice-based evidence?” Efficacy trials are not focused on external validity, whereas that is explicitly the goal of effectiveness research.

CER does not, however, abandon rigorous study design. Pragmatic clinical trials, basic to the CER portfolio, can include elements such as randomization and controls, while more nearly approaching conditions of actual community practice (eg, heterogeneous patient populations with varying degrees of compliance, from a range of practice settings).,, Handley et al make the same point regarding the use of quasi-experimental designs in PBR. Far from aiming at an average benefit that accrues to a minority in the middle, CER uses a wide range of valid, practice-based methods to support clinical care in a diverse and actual clinic population. In short, CER can be considered a form of PBR.

Personalized Medicine

Disease prevention, diagnosis, and treatment based on genetic testing is often referred to as “personalized medicine.” The heterogeneity of patients in CER studies allows researchers to tease out clinically important distinctions in patient subgroups, providing physicians in family practice with information—potentially including data on genetic variants—for personalizing treatment options. For example, a comparative study of 2 statins showed that participants given the stronger of 2 drugs had 16% fewer negative cardiovascular events than did the comparison group. When these results were reanalyzed according to genetic characteristics, new findings showed that among those using the stronger statin, the ones carrying a specific gene variant had a 40% reduction in adverse events, while those lacking the variant showed no statistically significant effect. Subgroup analysis in CER has the potential to inform choices, improve outcomes, and support appropriate use of resources.

Individualized Guidelines

Another CER strategy for patient-centered care is the development of “individualized guidelines.” In this approach, a wide range of longitudinal data from a single patient record can be analyzed to estimate the risk reduction expected from a particular treatment. Individualized guidelines can also be used to identify patients for treatment, stratified by degree of expected benefit. Using data on myocardial infarctions (MIs) and strokes from the Atherosclerosis Risk in Communities study, Eddy et al analyzed patient-specific longitudinal data and concluded that “individualized guidelines could prevent the same number of MIs and strokes [as standard guidelines] at savings that are 67% greater … or it could prevent 43% more MIs and strokes for the same cost as treatment according to [standard] guidelines.” Such results again suggest the potential for CER methods to individualize care and improve outcomes, while at the same time reducing cost.

Patient Preferences and Patient-reported Outcomes

CER supports use of a wide range of personal data, including patient preferences and patient-reported outcomes (PROs). Though rarely used as primary endpoints in standard RCTs, PROs carry considerable weight in family medicine. Participants in a 2009 CER stakeholder symposium concluded that where standard clinical measures of treatment effectiveness do not adequately reflect patient experience, further outcome measures should be developed to document “meaningful changes in a patient's life.” Clinician stakeholders emphasized the importance of tracking subjective yet existentially important outcomes such as global well-being and unanticipated positive treatment effects, eg, a sense of increased personal peace. CER studies can elicit PRO data through questionnaires, interviews, and online patient portals in clinical care settings, correlate findings with data obtained from electronic medical records, and thereby analyze the association between patient experience and outcomes. Such CER approaches are intended to yield evidence of value for effective physician-patient communication and collaborative decision making.

Research design for Complex Therapeutic Interventions

In clinical practice, multiple interventions often are used in combination to address a given condition. A family physician treating a patient with anxiety may prescribe an anxiolytic, refer to family therapy, and recommend a form of stress management such as meditation or yoga. CER provides methods for study of such therapeutic “bundles.” In one example, patients undergoing in vitro fertilization were randomized to receive either usual care or usual care plus a 10-session bundle of therapeutic approaches, including cognitive behavior therapy, relaxation training, negative health behavior modification, and social support. The pregnancy rate in the combination group was significantly higher than that in the usual care group. Where the goal is to optimize clinical outcomes, rather than test the efficacy of a single intervention, effectiveness research provides valid methods for studying therapeutic combinations.,

Surveys indicate that complementary and alternative medicine (CAM) is used by approximately 4 out of 10 Americans in a given year., Approximately 1 out of 30 Americans reported using a mind-body modality (eg, yoga, tai chi, meditation) in a single year (2007) as a result of a recommendation from a conventional medical provider. A majority of patients using CAM do so in combination with standard medical care, though most often without disclosing the combination to a physician.,, Surveys of primary care clinicians show that large majorities of respondents believe their patients are using CAM, though only a minority routinely ask patients about their CAM use., Family physicians are thus frequently unaware of the combinations of therapies used by their patients, which presents a dilemma. If a combination is potentially harmful, the physician has no opportunity to address safety issues with the patient. If a combination is potentially helpful, the physician has missed an opportunity to learn from patient experience and acquire practice-based evidence. Large majorities of primary care clinicians express a desire for more continuing education in CAM., In response to this pervasive need, CER offers ways to address both the evidentiary gap and the communication gap. CER methods provide a framework for the study of combinations of conventional and complementary modalities, which can yield scientifically valid data to inform physician-patient dialogue about safe and effective integrative care.

Research design for Health Services delivery

CER offers flexible methodology that supports the design of community-level studies, not only for management of disease conditions but for optimal health services delivery. Disease prevention and health promotion are 2 issues particularly salient in family and community medicine. The Health Improvement and Prevention Study (HIPS) protocol is a CER design involving 30 community clinics cluster-randomized into experimental and control groups. Clinicians in the experimental practice groups received training in motivational interviewing and health behavior counseling. HIPS proposed a framework for efficient yet individualized physician-patient dialogue, leading to preventive action as appropriate. Primary endpoints for comparison in experimental and control groups included not only patient physiological data and health behaviors at 12 months but also volume of referrals to nutrition, exercise, and education interventions and clinician self-reported attitudes and practices regarding prevention and risk factor management. CER studies carried out by PBR networks can yield evidence regarding multiple aspects of clinical care delivery, integrating the perspective and experience of physicians themselves.

Comparative Effectiveness Research and Freedom of Practice

CER, as a resource for generating scientific, yet practice-based evidence, offers both methodological rigor and design flexibility, such that patient individuality and physician-patient relationship can be valued, supported, and optimized. While comparative effectiveness data may inform decisions that allocate resources more efficiently,, CER as framed in the Patient Protection and Affordable Care Act of 2010 (ACA) is explicitly not focused on cost reduction. Language in the ACA prohibits Medicare from using a cost-effectiveness threshold in making coverage decisions. The ACA establishes an independent body, the Patient-Centered Outcomes Research Institute, to coordinate the CER effort, but sets limits on the kind of research that the panel may support. It is explicitly prevented from funding research that uses cost-effectiveness measures such as the standard “quality-adjusted life year,” or QALY, a means of combining length of life, health status, and patient preferences in a single measure. These restrictions specify an approach to CER that is not aimed at limiting freedom of practice to the least costly treatment for the statistically average patient.

CONCLUSION

CER holds potential to expand treatment options using approaches such as personalized medicine and individualized guidelines; engaging patient preferences and patient-reported outcomes; and developing research designs appropriate for testing benefits and harms of complex therapeutic interventions, including both conventional and complementary or integrative therapies. Physicians in family and community medicine are key stakeholders in the development and use of CER. The expertise of physicians engaged in community-based clinical care is of particular importance. Academic medical researchers will work in partnership with clinicians steeped in the complexity and diversity of “real world” practice. Experienced family physicians can most appropriately identify significant research questions and collaborate in the conduct of CER. Such a research enterprise would shape CER to produce high-quality practice-based evidence that, alongside efficacy RCTs, will reliably inform family and community medicine.