Differences in Incremental Cost-Effectiveness Ratios for Common Versus Rare Conditions: A Case from Oncology.

BACKGROUND: Incremental cost-effectiveness ratios (ICERs) are used to assess the value for money of new drugs. Many believe that ICERs for drugs that treat rare diseases are much higher than those of common drugs. Our objective was to compare the proportion of ICERs that are cost effective for rare and common cancers.
METHODS: We used the Tufts Medical Center Cost-Effectiveness Analysis (CEA) Registry to identify cost-effectiveness studies of pharmaceutical interventions for cancers. Studies that assessed FDA-approved 'orphan drugs' were categorized as assessing rare cancers. The proportion of common and rare cancer drugs that were cost effective at various ICER thresholds were compared along with study characteristics. Logistic regressions were conducted to assess important predictors of cost effectiveness.
RESULTS: We identified 303 studies that reported 701 ICERs. Seventy nine percent (n = 240) of studies evaluated drugs for common cancers. At a threshold of US$50,000/QALY, 58% (n = 321) of ICERs for drugs treating common cancers and 64% (n = 94) of ICERs for drugs treating rare cancers were cost effective (p = 0.23). At US$100,000/QALY, 74% (n = 409) of ICERs for common cancers and 78% (n = 115) of ICERs for rare cancers were cost effective (p = 0.35). Results from the logistic regressions demonstrated that rarity was not a statistically significant predictor of cost effectiveness at both thresholds with publication year, study sponsorship, and cancer type as covariates.
CONCLUSIONS: The proportion of ICERs that were cost effective at both thresholds does not appear to be significantly different between the two groups. Rarity is not statistically significantly associated with cost effectiveness, even when adjusted for important covariates.

Key Points for Decision Makers

Introduction

Cost-effectiveness analyses (CEAs) are commonly used to compare the health outcomes and cost of an intervention relative to the standard of care [1]. Results are often summarized using the incremental cost-effectiveness ratio (ICER). The ICER is the difference in costs between the intervention and the standard of care divided by the difference in health outcomes. For countries that use Health Technology Assessment (HTA) processes and rely on pharmacoeconomic models to assess the cost effectiveness of the drug, interventions with ICERs below a certain threshold value are more likely to be covered by drug plans and other insurers [2, 3]. A subset of CEAs, cost-utility analysis (CUA), measures health outcomes in a standardized way, using the quality-adjusted life-year (QALY) as the outcome measure. QALYs provide a way of encapsulating improvements in both quality and quantity of life into a single outcome. When measured in this way, the ICERs of interventions that treat different diseases can be compared against a common benchmark.

Due to the weak clinical efficacy that is often available for orphan drugs, it is often argued in the literature that standard economic techniques are difficult to apply to orphan drugs and that a greater degree of uncertainty surrounds the cost effectiveness of orphan drugs [4]. The limited number of patients that are affected by a rare disease makes it difficult to gather enough patients for clinical trials and demonstrate statistically significant clinical differences. Moreover, rare diseases tend to affect multiple organs, are complex in nature, and lead to highly variable clinical courses [5]. If the therapy is evaluated on the basis of a single outcome that only a fraction of the patient population experiences, then the clinical benefit of the drug can be further diminished [5]. All these factors could result in high uncertainty when assessing drugs that treat rare diseases.

Orphan drugs are also particularly expensive. One possible explanation is that orphan drugs have higher average research and development and production costs per unit sold because of the small unit sales volumes [6]. In the US, 15 orphan drugs were commercialized between 2006 and 2008, with six of these drugs costing more than US$100,000 per patient per year [6]. Due to the high prices associated with drugs that treat rare diseases, it is believed that conventional methods that are used to assess cost effectiveness will result in largely unfavorable ICERs [7, 8]. Clinical uncertainty in combination with high prices presents a challenge for payers when faced with making funding decisions for these drugs [6].

In the US, the Orphan Drug Act (ODA) was introduced in 1983 with the goal of facilitating the development of drugs for rare diseases through tax credits and market exclusivity. Since its introduction, 362 orphan drugs have received market approval and oncology comprises the largest proportion of these drugs [9]. In the EU, the same is true for the proportion of oncology drugs that comprises approved orphan drugs [10]. The economic impact of cancer-related interventions has received increased attention in the medical literature and media due to the high costs associated with these new treatments [11]. Treatments focused on rare cancers are therefore under special scrutiny at the time of budget allocations and adoption of new therapies.

Although it is believed that ICERs for drugs assessing rare diseases are higher than those of drugs assessing common diseases, no studies to date have formally examined the evidence for this belief. Our objective in this study was to assess whether there were differences in point estimates of ICERs for rare and common cancers using published CUA studies.

Methods

Data Source

The primary source of data was the Tufts Medical Center Cost-Effectiveness Analysis Registry (http://www.cearegistry.org), a database with detailed information on CEAs published in peer-reviewed medical and economic journals since 1976 (data retrieved September, 2012). The information for the database was extracted from English publications that were searched on MEDLINE using the keywords QALYs, quality-adjusted, and cost-utility analysis with an original cost per QALY estimate [11]. The methodology for literature search and data extraction has been described in great detail elsewhere [12].

Data Elements

For each study, the following characteristics were available: publication year, journal, author, disease area, prevention stage, country, sponsorship, time horizon, and study rating. For each ICER reported for each study, the following information was captured: target population, intervention, comparator, ICER reported by authors, ICER converted to 2012 US dollars. Publications that focused on ‘pharmaceutical interventions’ intended to treat cancer were included; studies that focused on diagnosis and/or prevention of diseases were excluded. Cancer types included breast, hematological, lung, cervical, ovarian, colorectal, prostate, malignant neoplasms, and other neoplasms. Some studies were categorized under more than one cancer type. Sponsorship sources for studies were categorized as follows: government, foundation, pharma or device, healthcare, professional membership organization, none, not determined, and other. The database also includes a quality score, assigned by the Tufts group, based on the study’s methodological rigor, quality of presentation, and potential value to decision makers [11]. This is a subjective rating score by reviewers from 1 (lowest quality) to 7 (highest quality). Scores reflect the following considerations: (1) correct computation of the ICER, (2) comprehensive characterization of uncertainty of the results, (3) correct use and specificity of assumptions, and (4) appropriate estimation of utility weights [13].

We distinguished studies assessing rare versus common cancers as follows. First, we identified the drugs that were compared in each study by searching the title, abstract, and the full text. We identified the therapeutic indication(s) assessed for each of these comparators within the publication. Next, we determined if each of these comparator-indication pairs appeared in the FDA Orphan Drug Product Database. The FDA defines orphan drugs as drugs and molecules that are intended for the safe and effective treatment, diagnosis, and prevention of rare disorders that affect fewer than 200,000 people in the US or that affect more than 200,000 persons but are not expected to recover the cost of developing and marketing a treatment [14]. If any of the comparators were FDA-approved for the same indication as examined in the publication, then the study was categorized as treating ‘rare’ cancers. If none were, then the study was categorized as treating ‘common’ cancers (Fig. 1).

Fig. 1

Study flow chart

Data Analysis

Our primary analysis included comparing study characteristics between the common and rare groups. Specifically, we compared publication year, cancer type studied, country, study sponsorship, and study rating.

Our secondary analysis involved determining whether ICERs assessing common and rare cancers were different. In order to understand this difference, we used willingness-to-pay (WTP) thresholds of US$50,000/QALY and US$100,000/QALY and computed the proportion of ICERs that were less than the thresholds for both study groups. The proportions of ICERs less than the thresholds were compared using two sample t tests.

In order to understand what factors would contribute to the proportion of ICERs that are cost effective at the two above-mentioned thresholds, we performed multiple logistic regressions. Specifically, we modeled the probability that an ICER was less than US$50,000/QALY and US$100,000/QALY as a function of rarity type (common or rare), indicators of cancer type, publication year, and sponsorship type. The multiple models of logistic regressions included the inclusion and exclusion of groups of covariates. Because the unit of observation in this regression was the ICER, not the study, we estimated study-level clustered standard errors.

In order to determine if there is an association between the quality of the cost-effectiveness analysis and the rarity of cancer reported, another regression model was run with study quality as the dependent variable and rarity type as the independent variable. All analyses were performed using Stata 13 (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX, USA: StataCorp LP).

Results

From the CEA registry, 303 relevant studies were identified. These 303 studies yielded 701 ICERs. Out of the 303 studies, 240 (79.2%) were categorized as studies assessing common cancers and 63 (20.8%) as studies assessing rare cancers. Out of the 701 ICERs reported, 78.9% (n = 553) were from studies assessing common cancers and the remaining 21.1% (n = 148) were from studies assessing rare cancers. The number of studies for both common and rare groups increased with time (Fig. 2).

Fig. 2

Publication year for cost-utility analyses for rare and common cancers

The majority of common studies were categorized under malignant neoplasms. Malignant neoplasms and other neoplasms contributed to the majority of rare studies. Some studies were categorized under more than one cancer type. Three studies were categorized under three types of cancer, 64 studies under two types of cancer, and the remaining 236 studies were categorized under only one type. The distribution of cancer types for studies assessing common and rare cancers can be seen in Appendix 1 (see electronic supplementary material [ESM]).

The majority of the CUAs were from the US, UK, and Canada as seen in Appendix 2 (see ESM). With regards to study sponsorship, pharmaceutical companies sponsored 42% of all studies. Numbers of studies categorized under each type of sponsorship and their proportions can be seen in Table 1. Some studies were categorized under more than one sponsorship category.

Table 1

The sponsorship type for studies assessing common and rare cancers

Sponsorship type	Studies assessing rare cancers		Studies assessing common cancers		All studies
	%	n	%	n	%	n
Government	17.8	13	19.3	53	19.0	66
Foundation	8.2	6	9.8	27	9.5	33
Pharmaceutical companies	57.5	42	37.8	104	42.0	146
Healthcare	2.7	2	3.6	10	3.4	12
Professional membership organization	0.0	0	2.9	8	2.3	8
None	1.4	1	4.7	13	4.0	14
Not determined	9.6	7	16.4	45	14.9	52
Other	2.7	2	5.5	15	4.9	17
Total	100	73	100	275	100	348

The study quality rating for rare and common cancers are compared in Fig. 3 using a box and whisker plot. The ends of the box are the upper and lower quartiles and the median is marked by the vertical line inside the box. The median study quality for the common group and rare group was 4.5 and 5, respectively. A larger range in study quality was observed for studies assessing common cancers.

Fig. 3

Study quality rating for rare versus common cancers

The distribution of all 701 ICERs in different quadrants of the cost-effectiveness plane can be seen in Appendix 3 (see ESM). Most of the ICERs for both common and rare groups were in quadrant 2 (more costly, more effective). The distribution of ICERs for common and rare cancer groups can be seen in Fig. 4.

Fig. 4

The distribution of ICERs for common and rare cancers. ICERs incremental cost-effectiveness ratios, QALY quality-adjusted life-year

The proportion of ICERs for common and rare groups that are below the thresholds of US$50,000/QALY and US$100,000/QALY can be seen in Table 2. The difference in proportions between the two groups was not statistically significant at either threshold. The proportion of ICERs that were cost effective at a range of WTP values can be seen in Fig. 5. At a threshold of US$0K/QALY, the proportion of ICERs that are cost effective for each group was statistically significant (p = 0.049). For the remaining threshold values tested (US$25,000/QALY to US$200,000/QALY) the proportion of ICERs that are cost effective in each group was not statistically significant.

Table 2

Proportions of ICERs for common and rare cancers under US$50,000/QALY and US$100,000/QALY

Threshold	Proportion of ICERs that assess common cancers that are cost effective (%)	Proportion of ICERs that assess rare cancers that are cost effective (%)	p value
US$50,000/QALY	58	64	0.23
US$100,000/QALY	74	78	0.35

ICERs incremental cost-effectiveness ratios, QALY quality-adjusted life-year

Fig. 5

Proportion of ICERs that are cost effective at different willingness-to-pay thresholds. ICERs incremental cost-effectiveness ratios, QALY quality-adjusted life-year

In order to determine predictors of cost effectiveness at thresholds of US$50,000/QALY and US$100,000/QALY, multiple logistic regressions were performed with dummy variables for rarity, publication year (1992–2012), study sponsorship (government, foundation, pharma, healthcare, professional membership organization, none, not determined, and other) and cancer type (breast, hematological, lung, cervical, ovarian, colorectal, prostate, malignant neoplasms, and other neoplasms) and study quality as covariates. In all the models, the odds ratios for rarity were not statistically significant (Appendix 4, see ESM). The coefficient for study quality was statistically significant in seven of the eight models. Sponsorship by pharma was statistically significant in five out of the six models. No sponsorship was also statistically significant in all three models for cost effectiveness at US$50,000/QALY. The results of the linear regression for study quality and rarity resulted in a coefficient of 0.27 which was statistically significant (Appendix 5, see ESM).

Discussion

The findings of our study suggest that there are no significant differences in ICERs for drugs that treat common and rare cancers using published literature. The increasing trend of published CEAs over time seen in our analysis has also been observed in other studies [11]. The CEA registry also shows a similar trend in published CUAs over time for all therapy areas [15]. This trend is not surprising since payers across the world now require economic evaluations to make reimbursement decisions [16, 17].

The cancer types in the majority of studies used for this analysis were categorized as malignant and other neoplasms. This is not surprising since these two categories are defined very broadly by the CEA registry reviewers. For both common and rare groups, the majority of studies were funded by pharmaceutical companies followed by government. This is different from a previous study that assessed CUAs in cancer care where only 21.5% of studies were industry sponsored [11]. However, that study looked at CUAs until 2007 while our study included a longer publication period up to 2012.

The mean quality score seen for all CEAs in the Tufts CEA registry is 4.57 [13]. The median study quality scores for the common and rare groups in our analysis were similar to this mean score for all studies. When a linear regression with study quality as a dependent and rarity as an independent variable was performed, the coefficient on rarity was statistically significant. However, the estimated 0.27 unit increase is minimal in the spectrum of study quality which ranges from 1 to 7.

The graphical distribution of ICERs for common and rare groups appeared to be similar visually. The trend seen for these groups aligns with a previous study that assessed CUAs in cancer care [11]. Moreover, they are also similar to CUA distributions in other healthcare fields where the majority of ICERs are below US$50,000/QALY [18, 19]. The highest proportions of ICERs falling below a US$50,000/QALY threshold could be due to CUAs with favorable results being published more often than the ones with unfavorable results.

The proportions of ICERs that are cost effective below WTP thresholds of US$25,000/QALY to US$250,000/QALY for the common and rare groups were not statistically significant. Perhaps, in oncology, the price difference between drugs for common and rare conditions and the clinical effect difference between the comparators is minimal, resulting in ICERs that are similar between the two groups. Moreover, it is possible that studies assessing ultra-rare diseases result in extremely high ICERs and they’re known to have significant price tags associated with them [4]. Point estimates of ICERs specific to ultra-rare diseases would be an interesting topic of research.

There are several limitations with our study. First, we only included studies from the CEA registry where health benefits are quantified using QALYs. All other types of economic evaluations were not included. Secondly, our assessment did not include HTA reports. It is possible that the ICERs believed to be reasonable or appropriate by HTA agencies are different from the ICERs reported in the literature. Next, our categorization of studies into common and rare groups was based on the FDA orphan drug database and thus used the FDA definition of a rare disease. Internationally, different definitions of rare diseases are used which could allow for different categorization of studies and potentially different results. Moreover, one must take caution before extending these results to other therapeutic areas as the observations from this study are restricted to oncology. It is possible that drugs that treat rare cancers and common cancers are similar in nature in terms of treatment effect observed in clinical trials, the types of outcomes used to measure treatment effect, and in drug prices, which could give rise to the results that stem from this study. Lastly, we were not able to compare the uncertainty associated with the ICERs for different studies among the two groups. Clinical uncertainty is an important factor that affects the results of a CEA and can lead to high uncertainty in a computed ICER. If the ICERs for orphan drugs were more uncertain, it is possible that the ICER point estimates between rare and common drugs might not differ appreciably, but there might be a much greater probability that rare drugs are not cost effective.

Conclusions

In summary, we did not observe any significant differences in point estimates of ICERs for drugs that treat common versus rare cancers in a sample of published CUAs. Our analysis suggests that the perception that ICERs for rare diseases are higher than for common diseases may be not be true for cancers. More work is needed to understand whether the uncertainty associated with ICERs assessing interventions for rare diseases is higher than that for common diseases.

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 45 kb)

Many believe that drugs that treat rare diseases that have been appraised to date have incremental cost-effectiveness ratios (ICERs) that are higher than those of drugs that treat common conditions.

Our results shows that when comparing ICERs for drugs that treat rare and common cancers, the proportion of ICERs that are cost effective at US$50,000/QALY and US$100,000/QALY is not different between common and rare groups.

This study does not find evidence that rarity is associated with cost effectiveness based on published cost-utility studies in oncology.