Personalized medicine and comparative effectiveness research in an era of fixed budgets.

For personalized medicine to be widely adopted in clinical practice, stakeholders need evidence of effectiveness, cost effectiveness and financial viability. Comparative effectiveness research (CER) using population based, retrospective data can inform assessments of personalized medicine. The purpose of this paper is to explore the potential and the limitations of CER. While the analytic methods and data used for CER overcome many of the disadvantages of randomized controlled trials, there are significant barriers, including lack of routinely collected genetic information, patient-reported outcomes and information on new and emerging technologies. Recommendations for using CER include augmenting current data with genetic information, promoting the collection of uniform health outcomes, using value of information analysis to guide development of new technologies, and greater use of decision analysis. Finally, in order to address stakeholder concerns regarding short term financial viability, additional emphasis should be devoted to cost analysis of implementation costs and overall financial impact.

Introduction

Personalized medicine is facing an uncertain future. On one hand, it is touted as the future of the healthcare system, an approach that will yield safer, more effective, and less costly medications and medical interventions [1]. On the other hand, there are few clinical applications of personalized medicine currently being implemented, and the prognosis for future developments is unclear [2]. There are a variety of reasons for the lack of widespread clinical adoption [3], but the most immediate and compelling is that the stakeholders—clinicians, funders, governments and patients—do not have enough evidence that personalized medicine approaches will yield significant health benefits or reduced costs to justify investing or funding current applications [4]. In an era of fixed budgets with limited resources, the lack of clear evidence of the costs and health gains is a significant barrier to the widespread development and adoption of personalized medicine.

The lack of evidence on health outcomes is in part due to the types of genetic markers that have been identified to date. Whereas early proponents of personalized medicine and genetic testing envisioned a world where people’s future disease and health states could be assessed by taking a sample of saliva (the ‘genetic scorecard’ [5]), the recent progress in genetic influences on disease has had greater success identifying multiple genetic predictors of disease, often with low predictive power, than single gene links. These low-penetrance, multiple genetic predictors complicate research efforts to identify the effectiveness of new treatments since large samples are needed to identify the contribution of the genetic component to poor health outcomes. In addition, the presence of multiple genetic predictors makes it less likely that there will be simple and clear interventions or pharmacogenetic treatments that will dramatically improve health outcomes. To date, it has proven difficult to assess or demonstrate the effectiveness of treatments based on personalized medicine using randomized controlled trials [6].

In addition to the lack of evidence on the effectiveness of new treatments, the economics of personalized medicine have proven to be problematic [7]. Developing personalized approaches can be more costly than traditional approaches because personalized approaches must include the cost of identifying genetic markers, of developing a genetic test, and integrating the test into clinical practice. On the other side, the expected revenue can be less than population-based treatments since the market for personalized treatments will be smaller (in the most extreme case, only the individual) [3, 7], although population based treatments may be faced with increased market pressure unless they develop more effective treatments. For biotechnology and pharmaceutical companies to invest in developing personalized approaches, there needs to be a clear indication from third party payers (insurers or the government) or the public that they will be willing to pay for these new approaches, and perhaps subsidize their development. Given the lack of evidence of increased effectiveness and/or cost savings, it is not surprising that payers have been reluctant to commit to funding the development of these new approaches [2, 8].

The lack of evidence of clinical utility and financial viability has been identified as a significant barrier to the widespread development and adoption of personalized medicine [2, 4]. Previous researchers have also noted that randomized controlled trials have significant limitations in assessing the benefits of personalized medicine, including insufficient sample sizes for conducting subgroup analysis, limited time for follow-up, restrictive exclusion criteria and limited ability to compare various treatment modalities [3]. Instead, researchers have advocated for the use of an alternative methodology to assess the costs and effectiveness of personalized medicine: Comparative effectiveness research (CER) [1, 9–12]. CER has been defined as a “rigorous evaluation of the impact of different options that are available for treating a given medical condition for a particular set of patients” [13]. While this definition is flexible enough to incorporate most study methodologies, in practice (at least in the United States, where funding has significantly increased over the past several years) it has primarily involved using retrospective data (e.g., SEER-Medicare data) to examine the effectiveness (including, perhaps, cost effectiveness) of alternative treatments in actual practice. The evidence from CER tends to differ from evidence from traditional randomized controlled trials in that it includes a broader population (e.g., not subject to exclusion criteria as in many RCTs), reflects the effectiveness of how care is actually delivered in the real world (a factor often highly controlled in most RCTs), can be used to compare a variety of different treatment modalities, and can include data for a longer period of time. Thus, CER has the potential to overcome many of the limitations of RCTs in providing evidence for the effectiveness and cost of personalized medicine.

The purpose of this paper is to explore the potential and the limitations of CER for providing evidence on the effectiveness and financial viability of personalized medicine. The discussion will not focus on the challenges and opportunities from the standpoint of identifying new types of genetic markers, but rather on the potential for identifying evidence on effectiveness and cost of personalized medicine. As such, particular attention is devoted to identifying financial barriers to developing and implementing personalized medicine, how comparative effectiveness might provide evidence on long term costs and consequences, and the limitations of CER research. The paper concludes with recommendations for using CER to facilitate the adoption of personalized medicine in the future.

Personalized medicine

Personalized medicine refers to ‘a tailored approach to patient treatment based on the molecular analysis of genetics, proteins and metabolites’ [3], or ‘care that relies on any diagnostic method (genomic, phenotypic, proteomic, imaging, etc.) that identifies personal characteristics to target therapeutic interventions to those patients who may disproportionally benefit from the treatment and/or improved benefit-risk profile’ [14]. In other words, personalized medicine aims to use information specifically about the patient (as opposed to a broader information category such as family history or ethnic group) to determine the appropriate healthcare intervention. The intervention could range from a common treatment given to many (such as Warfrin) [15] to a medication tailored for the individual’s genetic profile.

For a personalized treatment to be financially viable, the expected revenue must outweigh the cost of developing and administering the treatment. To illustrate the types of personalized medicine and how they differ in potential costs and revenue streams, consider the five cases outlined in Table 1.

Table 1

Overview of types of personalized medicines

	Benefits to patient		Costs						Who pays? Testing and treatment
			Development		Testing and treatment		Cost savings
	At time of genetic test	In the future	Genetic test	Treatment	Genetic test	Treatment	Adverse events	Future healthcare costs	Third party Payer	Patient	Private company
Screening/risk assessment
(i) Preventive healthcare treatments available	+	+	+		+	+		−	+
(ii) No preventive healthcare treatments available	+?		+		+			+	+a	+b
Diagnosis and treatment
(iii) Better use of existing treatment	+	+	+		+	+	−	−	+
(iv) New drug or treatment	+	+	+	+	+	+	−	−	+
Drug development
(v) Facilitate development of other treatments			+	−	+						+

aPay for future health costs

bPay for genetic test

Risk assessment/screening (i and ii)

Tests for the purpose of identifying disease risk (rather than diagnosis of an exciting condition or determining treatment) are available for over 1700 conditions [16], with many available directly to consumers. Consumers can purchase information on their genetic risk for both future health conditions (e.g., breast cancer and diabetes) and physical characteristics (e.g., eye color, muscle control and hair thickness [17]) of themselves and predictions for their offspring. This has raised concerns about the extent to which the public will understand this information, how it might be used, and whether a misunderstanding will cause distress and lead to unnecessary diagnostic and follow-up tests [18, 19].

For third-party payers (health insurers and government funders), the value of this type of test depends upon whether it will lead to reductions in future disease. If there are preventive actions that can be taken (i), then the cost of developing and administering the test might be offset by future health savings and avoidance of future illness. For instance, tests for the genetic variant associated with breast cancer might lead to preventive measures ranging from lifestyle (e.g., stop smoking) to a preventive mastectomy. The benefit of these tests, and the extent to which payers will cover the cost of the test and subsequent treatments, depends upon the evidence of effectiveness in preventing occurrence, the savings in associated future healthcare costs, and the cost of developing and administering the test. In theory, this might also lead to the development of preventive genetically based treatments such as more effective vaccines or medications to delay or prevent onset of disease.

For many tests, however, there are no direct preventive measures that can be taken. For instance, although genetic tests are available for dementia, there are few effective preventive treatments available to consumers. The value to consumers from understanding their future risk is a topic of much research. In some cases, the individual may value the certainty associated with knowing that a condition will or will not occur in the future (e.g., people with family history of Huntington’s Disease finding out they do not have the genetic marker). Payers (e.g., Medicare) have proven reluctant to pay for these types of risk assessments to date, so the primary source of revenue has come directly from consumers [20]. That said, payers may be responsible for subsequent diagnostic tests associated with individuals worrying about the condition.

Diagnosis and treatment (iii and iv)

The third and fourth types of personalized medicine involve the use of genetic information to help guide treatments of existing conditions or estimate the prognosis of an existing disease. As opposed to risk assessment or screening, which might (in principle) be performed on the entire population, personalized treatments are only for a subset of individuals with a condition. This type of personalized medicine aims to identify existing treatments that will have the greatest impact (iii) or developing treatments tailored to the individual (iv).

The financial viability of this type of personalized approach depends upon the extent to which the new treatments lead to fewer adverse events (reducing current health spending) and better future health outcomes (future cost savings) [21]. The most successful examples to date have used genetic tests to determine which individuals respond best (and worse) to an existing treatment (case iii). For instance, a number of studies have examined using a genetic test to identify patients with acute coronary syndromes (ACS) with a reduced function allele CYP2C19*2 (*2 allele) [22, 23]. As patients treated with thienopyridines have an increased risk of adverse cardiac events with clopidogrel, the genetic test is used to determine who should get treated with prasugrel (a more expensive treatment). If these savings are large enough to offset the cost of giving the genetic test to patients with ACS, then it will result in current cost savings.

It has been argued that this type of personalized medicine is the most likely to yield significant health gains and/or cost savings [7, 15]. Because this type of personalized medicine uses existing treatments, there are no treatment development costs. This approach offers the potential of new life for drugs that have been eclipsed by a newer class or found inadequate for general usage.

In contrast, there are relatively few examples of personalized medicines involving treatments developed specifically for a genetic condition or biomarker (case iv). Unlike treatments involving existing drugs or interventions, this type of personalized medicine requires not only developing the genetic marker and test, but also incurring the cost of developing a genetic or molecular based treatment. Because the developmental costs can be significant, the number of potential patients small, and current payment mechanisms not designed to reward innovative tests or development of genetically based treatments, this approach is seen as unlikely to yield a significant number of new interventions in the near future [14].

Drug development (v)

Finally, the fifth type of personalized medicine involves using genetic or molecular information to help guide selection of participants for treatment testing. It is hoped that this approach will yield significantly faster and less expensive testing of new treatments [7]. However, as the primary benefits from this research are likely to fall to companies who have developed the drugs, it is unlikely to be paid for by payers. To date, this type of pharmacogenetic application has been more efficient for some drugs, but has not lead to a paradigm shift in the way drugs are developed and tested [14].

Summary

The framework presented in Table 1 is not intended as a comprehensive assessment of all types of personalized medicine. Rather, it is intended to highlight the types of information required to identify the benefits from the personalized medicine, the factors influencing the costs, and the sources of revenue. From this perspective, there are a number of points to note:

The benefits of preventive treatments emerging from risk assessments (i) might require significant amount of follow-up time in order to assess their effectiveness. While this is a feature of many screenings, genetic tests of conditions that will not be expressed until a much later time (i.e., low immediacy) present special challenges;

Risk assessments with no or ineffective preventive treatments (ii) will not result in cost savings even in the long run, and may be associated with other health care costs when patients are distressed about their genetic information. The benefits to the patient are unlikely to be captured by traditional health outcomes;

The benefits of personalized medicine using biomarkers or genetic information to identify the most appropriate treatment (iii and iv) can be assessed by comparing the outcomes with existing (non-personalized) treatments;

If personalized treatments are to result in cost savings (in the short run), it will require significantly reducing the number of adverse events and/or the cost of treatments. As it is unlikely that personalized treatments (iv) will cost less to develop and implement than existing treatments, only those medical conditions associated with a large numbers of people who experience significant adverse events are likely to result in cost savings.

Comparative effectiveness research

At first glance, there would appear to be a conflict between personalized medicine and CER. Whereas the goal of personalized medicine is to produce a treatment tailored for each individual, CER yields global assessments of the average effectiveness of treatments across populations. Since people respond differently to various treatments, the goal should be to find the treatment that is right for them, not the treatment that is right ‘on average’. For instance, consider two drugs (A and B) with average levels of effectiveness of 4 and 5, respectively (Fig. 1). If the treatment decision was based on the average level of effectiveness, the conclusion would be that B should be recommended over A. But because there is variation in the effectiveness, there are some individuals treated with B who did worse than 4, and other individuals treated with A who did better than 5. This raises the possibility that some people might be better off being treated with A, others with B, and looking for one favored treatment ignores the potential benefits from personalizing treatment.

Fig. 1

Comparative effectiveness of two drugs

Advocates would say that such a conclusion (treat some with Drug A, others with Drug B), is entirely consistent with CER, it only requires performing subgroup analysis or controlling for confounding variables during the analysis [1, 9, 11, 24]. In fact, conducting CER with secondary, claims data might actually augment and support personalized medicine. Many of the challenges of assessing the costs and benefits of personalized medicine using randomized controlled trials are not present in CER analysis of secondary datasets, including:

Data is often available for many years before and after events, thus allowing lengthy follow-up times;

Datasets often contain large sample sizes, as is required for subgroup analysis of personalized medicine;

Information on type and cost of adverse events requiring medical care is routinely collected;

Population based data contains information on more diverse populations than is usually available through RCTs, thus making the results more reflective of the population; and

Information is typically available on a variety of treatment modalities, not just the ones chosen for the RTCs.

But CER of retrospective data does have its limitations for use in assessing the costs and effectiveness of personalized medicine. First, there is often inadequate information on health outcomes, particularly patient reported outcomes. Most studies using secondary care data are limited to outcomes measures that can be assessed with diagnostic codes or mortality. More general quality of life measures, such as the SF36 or disease-specific measures, are not routinely collected. While this is an issue for nearly all studies that utilize secondary care data, it is especially pertinent for personalized medicine where the perceived value of genetic information to the patient (e.g., from risk assessments) often goes beyond health outcomes [25]. Ideally, these datasets would include information on preference based assessments of the information and intervention, including willingness-to-pay or outcomes measured through discrete choice experiments.

Second, if comparative effectiveness is to be used for subgroup analysis, then each individual’s data must be linked to their genetic information. At present, genetic information is not routinely recorded and it is difficult to imagine that it be routinely available at anytime in the near future. Not only would routine collection of genetic information be costly and potentially inaccurate, but it would raise significant privacy issues as genetic information (by definition) is uniquely identifiable. While the FDA in the US has recently encouraged manufacturers to collect biomarkers data as part of the testing process [2], there are significant hurdles that would need to be overcome before recording of genetic information is routine.

Finally, using data from actual practice limits the scope of CER to examining the effectiveness of treatments and interventions that are already in use, not new or rarely used treatments. In addition, claims-based data only record interventions if they are reimbursed. New procedures may not yet have specific reimbursement codes and thus may not be recorded. Thus, CER may not be very useful for examining the effectiveness of new treatments.

Analysis of retrospective data is not the only form of CER, which can also include prospective trials and systematic reviews. But prospective trials or data collection will be subject to many of the same limitations as RCTs, including restrictive samples and small sample sizes. Systematic reviews and meta-analyses, while useful in consolidating and summarizing the results of previous studies are unlikely to provide systematic evidence on the costs and health outcomes associated with many types of personalized medicines. While CER methods can assist in providing evidence for personalized medicine, CER is unlikely to provide sufficient evidence of the effectiveness and financial impacts of personalized medicine to convince stakeholders to support widespread adoption.

Summary

CER has the potential to provide evidence for assessing the effectiveness and cost effectiveness of personalized medicine, but there are significant limitations that exist. The usefulness of CER using secondary data will vary across countries (which will have different types of data available), but all will face the challenges of:

Lack of patient reported outcomes;

Lack of linked genetic or biomarker information; and

Lack of evidence on new or not recorded personalized approaches.

Recommendations

For personalized medicine to be adopted by clinicians, recognized by payers, invested in by private companies, and accepted by consumers, there needs to be compelling evidence of its effectiveness and financial viability. As was argued above, such evidence is unlikely to come from current research modalities. Developing evidence will require both long and short term strategies and actions.

Ideally, evidence needed to determine the effectiveness and cost effectiveness of personalized medicine, including patient recorded and disease specific health outcomes, genetic information, and characteristics of the clinical environment, would be routinely collected and available to researchers interested in developing new personalized approaches and identifying the effectiveness of existing methods. Specifically, developers of new personalized approaches would benefit from being able to identify conditions where there is currently sufficient variation in clinical effectiveness, cost and/or adverse events from existing treatments, and this variation is related to genetic markers in individuals.

The information provided by this analysis could inform value of information (VOI) analysis to guide the future development. VOI is intended to identify the net benefit that would be received if all uncertainty regarding clinical utility and costs was resolved. That is, it can be used to identify the value from undertaking additional study or analysis. Previous researchers have pointed out that this would be especially useful for personalized medicine since the ultimate effectiveness and cost effectiveness of the genetic markers depends upon information on clinical outcomes (such as rate and cost of adverse events) that is often not incorporated by basic scientists when deciding which genetic marker to investigate [10, 20]. Although VOI is widely used by biotechnology and pharmaceutical companies, as well as by NICE in the UK, formal recognition of its importance for personalized medicine has been only recent.

The type of data described above would allow researchers to identify areas where development of genetic markers or tests might lead to improved health outcomes and be cost effective. The data would also allow researchers to examine the effectiveness and cost effectiveness of existing personalized medicine approaches. However, there are two areas were additional evidence will be needed. First, regardless of how comprehensive the datasets or effective the information gathering, CER research using clinical data will not supplement RTC in providing evidence on the effectiveness of new interventions. But CER can augment RTCs by identifying the types of genetic markers associated with adverse events and long term health outcomes. Decision modeling using information from RTCs and CER can then explore the long term effectiveness and cost effectiveness of the new treatments or tests.

Second, CER does not provide reliable evidence on the cost of implementing personalized medicine into clinical practice. It has been argued that implementation issues, including the development of guidelines to facility change in clinical management, coordination of laboratory testing and communication of results, integration into clinical practice and ways of communicating results to patients, are substantial barriers to widespread adoption of personalized medicine in practice [4]. Information on the short term net cost is particularly of interest to stakeholders operating under budget constraints. Thus, CER research may need to be accompanied by more regular analysis of the costs and barriers to implementing personalized medicine in practice.

Of course, while the above describes the ideal data that would be available to conduct CER on personalized medicine treatments and tests, no country currently has large datasets on diverse populations that includes genetic markers. So in the short term, researchers will need to find ways to augment existing CER data will other types of data. For instance, CER can be used to identify conditions with significant adverse events, diverse treatment patterns and having differential effects on specific populations that might suggest the procedures are candidates for developing personalized procedures. The outcomes from CER might be liked with genetic data for specific populations and information on the effectiveness of interventions from RTC to inform decision modeling of outcomes and long term results. While no substitute for more comprehensive datasets, a combination of methods (CER to identify long term outcomes, genetic information and RTCs to identify short term response of specific populations, and decision modeling to examine the long term effectiveness, cost effectiveness and robustness of results) would provide a way to enhance the evidence available on the effectiveness and cost effectiveness of personalized medicine.

As countries will vary in the data that are available, the recommendations will need to be tailored to each country. But in general, recommendations for prompting assessments of personalized medicine include:

Link existing datasets to genetic information, ensuring appropriate safeguards. While these might eventually be collected for the population, in the short term, researchers might seek to augment CER datasets on a case-by-case basis;

Promote the collection of uniform health outcomes, including disease specific and patient reported outcome measures. Again, in the short term, this might require augmenting CER datasets on a case-by-case basis;

Value of information analysis using information from CER is a useful tool for identifying personalized medicine approaches that are likely to be accepted by stakeholders and should be used more widely by basic scientists;

Decision (e.g., Markov) models can be used to explore the effectiveness and cost effectiveness of personalized medicine by combining information from CER, RCTs and systematic reviews; and

To highlight the financial viability and impact on stakeholders, researchers should look to include a cost analysis focusing on short term costs or costs savings to clinicians and other stakeholders.

Conclusion

For personalized medicine to fulfill expectations and alter healthcare in fundamental ways, stakeholders need to evidence that their investments in personalized medicine will yield new procedures that dramatically improve health outcomes and/or reduce adverse events. The purpose of this paper was to examine the potential for CER to provide evidence of effectiveness and financial viability. The discussion highlighted the advantages that CER has over other sources of information (RTCs), but also the limitations of most current datasets that are used for CER. Until comprehensive datasets linking genetic information, patient reported outcomes, diagnostic codes or medical records, and healthcare usage becomes available, researchers might use decision modeling populated by CER and genetic information to explore the impact of personalized medicine.

Calls for the use of CER to examine the impact of personalized medicine have gained momentum in recent years [1, 9, 11, 24], and previous researchers have highlighted the potential for VOI to augment existing data [10, 20]. The discussion here builds upon those studies by examining in more detail the strengths and weaknesses of using CER in conjunction with personalized medicine. The overall conclusion here, that CER is valuable but is unlikely to provide compelling evidence, is less optimistic than previous studies. But the conclusions should not be viewed as conflicting. Rather, this discussion is intended to highlight the challenges that can be overcome by combining CER with decision modeling and auxiliary data collection.

Implicit in these discussions is an assumption about how the evidence from these studies will be used. Much of the debate and controversy about CER stems from fears/hopes (depending on one’s perspective) that it will be used to determine which treatments are best for society, and that only these will be made available to clinicians and patients. This top-down approach to funding is actually very rare in developed countries, even in the United Kingdom (where NICE has recently been relegated from having authority in determining treatments to merely making recommendations for guidelines [26]). More common is for the results from CER (or RTCs) to be used to inform guidelines or clinical practice, but to leave clinicians and patients with some degree of autonomy with regards to their treatment decisions.

A consequence of decentralizing decision making to the individual/provider level (be it clinicians, patients, hospitals or insurance companies) is that they may have a more limited perspective. In a time of constrained budgets and limited resources, providers and private insurers may have fixed budgets and thus be unwilling to invest in procedures that have significant short term financial costs and only long term benefits (be it health gains or reduced future spending). For this reason, providers may be less interested in the results from cost effectiveness analysis than cost analysis. Monitoring and reporting the cost of implementing personalized medicine into clinical practice, including the cost of training staff, changing information systems and communicating complicated results to patients, are likely to be more important than future health gains and savings in implementing personalized medicine into regular clinical practice. Though information from CER can inform the discussion, a more formal cost analysis may ultimately need to accompany studies examining the effectiveness and cost effectiveness of personalized medicine.