A Critical Appraisal and Recommendations for Cost-Effectiveness Studies of Poly(ADP-Ribose) Polymerase Inhibitors in Advanced Ovarian Cancer.

BACKGROUND: Ovarian cancer is the fifth leading cause of cancer death in women in the US. With poly(ADP-ribose) polymerase (PARP) inhibitors having shown promising results in ongoing trials, there is interest in better understanding their economic value.
OBJECTIVE: This study aimed to review and evaluate the quality of published cost-effectiveness analyses (CEAs), and provide recommendations for CEAs in this setting.
METHODS: A systematic literature review of the MEDLINE and EMBASE databases was conducted in June 2019 to identify CEAs of PARP inhibitors in treating advanced ovarian cancer from peer-reviewed journals and conferences. Key information from the identified publications were extracted and reviewed. The quality of full-text studies was assessed using the Quality of Health Economic Studies instrument. Recommendations for future CEAs were developed based on the findings from the literature review.
RESULTS: Eighteen CEAs (five in full texts) met the inclusion criteria. Most adopted a US healthcare or societal perspective. The majority of the studies did not clearly display the economic model structure. No studies reported the validation of model projections based on internal or external data. Surrogate outcomes such as incremental costs per progression-free life-year gained were the most common outcomes reported. The majority of studies drew their conclusions based on surrogate outcomes, even with no theoretical or empirical threshold for cost effectiveness. All five full-text studies included some type of sensitivity or scenario analyses. The key drivers of the incremental cost-effectiveness ratio were treatment duration, effects, and costs, health utility, and prevalence of BRCA mutations.
CONCLUSION: In the existing CEAs for PARP inhibitors, there were uncertainties and challenges leading to variation in quality. We provided recommendations to improve consistency and quality of CEAs in this setting, which will help to better understand the value of PARP inhibitors, improve decision making, and reduce potential misallocation of resources.

Key Points for Decision Makers

Introduction

Approximately 21,000 women in the US are diagnosed with ovarian cancer each year, with the majority of patients diagnosed in the late stages [1]. The 5-year survival rate among patients with ovarian cancer is estimated at 47%, and with nearly 14,000 deaths each year, the cancer is the fifth most deadly among women in the US [2]. Germline mutations in the BRCA1 and/or BRCA2 genes, which code for deoxyribonucleic acid (DNA) repair, are an important risk factor for ovarian cancer. A recent study showed that 44% of women with a BRCA1 mutation and approximately 17% of women with a BRCA2 mutation will develop ovarian cancer by the age of 80 years [3].

Treatment of ovarian cancer typically consists of surgery and platinum chemotherapy with or without bevacizumab. Following primary therapy, maintenance with poly(ADP-ribose) polymerase (PARP) inhibitors can help manage the disease and delay disease progression, especially among women with BRCA mutations. PARP inhibitors, including olaparib, rucaparib, and niraparib, block the PARP pathway and make it hard for tumor cells with BRCA mutations to repair DNA, causing cell death [4]. In the US, olaparib was initially approved as a fourth-line option in ovarian cancer patients with a BRCA mutation [5]. In recent studies, olaparib monotherapy has demonstrated benefit as a maintenance therapy for adult patients with a BRCA mutation who are in a complete or partial response to primary therapy [6], and also as a maintenance therapy for patients with platinum-sensitive relapsed (PSR) disease [7, 8]. Olaparib has also shown benefit when used in combination with bevacizumab as a maintenance therapy in newly diagnosed ovarian cancer patients [9]. Rucaparib and niraparib are currently recommended as options for PSR disease [10]. Niraparib also demonstrated benefits in progression-free survival (PFS) among patients with newly diagnosed advanced ovarian cancer who had a response to platinum-based chemotherapy in a recent study [11].

Cost-effectiveness analysis (CEA) is a commonly used and well-established method of evaluating the comparative effectiveness and costs of alternative treatment options. CEA is becoming increasingly important in the US, as stakeholders further use it to determine drug value and access in recent years. Given the emergence of the PARP inhibitor class, there is much interest in their economic value, and many CEAs have been published or presented. For example, the Institute for Clinical and Economic Review estimated the cost effectiveness of PARP inhibitors as maintenance therapy for women with relapsed ovarian cancer and found that the PARP inhibitors would provide gains in quality-adjusted survival and overall survival (OS) over alternative therapies, and only olaparib is priced in alignment with these benefits [12]. CEA is a reliable reference for stakeholders only if it is of sufficient quality and unbiased; otherwise, it may lead to the misallocation of healthcare resources [13, 14]. In this study, we aimed to review and evaluate the quality of existing CEAs for PARP inhibitors in advanced ovarian cancer and identify the challenges in conducting high-quality CEAs. Based on the key drivers, issues, and uncertainties found in the literature review, this study also aimed to provide recommendations for the proper conduct of high-quality CEAs in this setting to better understand the value of PARP inhibitors, improve future CEAs and decision making, and reduce potential misallocation of resources.

Methods

Systematic Literature Review

A systematic literature review was conducted to identify CEAs of PARP inhibitors in treating advanced ovarian cancer, according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Relevant disease and study design terms were compiled for searches in the MEDLINE and EMBASE databases (Online Appendix 1). In addition, proceedings of 2016–2019 annual meetings of the American Society of Clinical Oncology (ASCO), Society of Gynecologic Oncology (SGO), International Society for Pharmacoeconomics and Outcomes Research (ISPOR), and the European Society for Medical Oncology (ESMO) were screened. The systematic literature search was conducted in June 2019. The titles and abstracts of all citations returned from the systematic literature search were screened using a predefined set of inclusion criteria (Online Appendix 2), i.e. CEAs of PARP inhibitors in the treatment of adult patients with advanced ovarian cancer that reported incremental cost-effectiveness ratio (ICER) results and were published in English. Studies that assessed the cost effectiveness of other ovarian cancer treatments (e.g. bevacizumab) were eligible for inclusion if they also reported the cost-effectiveness of PARP inhibitors. Other types of studies, such as clinical trials, observational studies, reviews, or meta-analyses were excluded. The full texts of studies and posters/presentations that passed title and abstract screening were retrieved (if available) and were reviewed using the same criteria.

Data from included publications were extracted into a prespecified extraction grid available in Online Appendix 3. Extracted information included study designs (e.g. model structure, model perspective, model outputs, and target population), efficacy inputs (e.g. OS and PFS data source, extrapolation methods, model fit evaluation, and model validation), utility and cost inputs (e.g. utility tool and data source, costs components and data source), and study results (e.g. base-case results and sensitivity analysis results). All screening and data extraction steps were conducted independently by two reviewers in parallel, and any disagreements were resolved via independent arbitration by a third reviewer.

Quality Assessment of Economic Analysis

The quality of identified full-text studies was assessed using the Quality of Health Economic Studies (QHES) instrument [15]. This instrument evaluates the appropriateness of study methods as reported in the papers, and the validity, transparency, and comprehensiveness of study results. It has 16 items with dichotomous response (yes/no), and each item has a weighted point value. The quality of a study is measured by the sum of points across items with ‘yes’ responses. The quality score ranges from 0 to 100 and a higher score indicates better quality. Two reviewers independently reviewed each study using the QHES checklist. If the score differed between reviewers, co-authors discussed the score in order to reach a consensus.

Studies that were published as abstracts, and their corresponding posters (if available), were reviewed and summarized by the same reviewers. Specifically, we reviewed the abstracts in reference to the domains listed in Table 1 and the more detailed extraction grid available in the Online Supplementary Material (Appendix 3). These studies were not scored using the QHES instrument, as many components of reporting cannot be fairly assessed from abstracts.

Table 1

Identified studies from the systematic literature review, and their study design

Study, year	Study type	Source	Target population	Perspective	Time horizon	Comparator	Model structure	Health states	Model outputs
Guy et al., 2019 [22]	Full-text	PharmacoEconomics; March, 2019	Adult patients who had platinum-sensitive, recurrent, high-grade, serous ovarian cancer, received at least two platinum-based regimens, were sensitive to last platinum-based chemotherapy. Population split into BRCA and non-BRCA	US payer	Lifetime	Niraparib; olaparib; rucaparib; routine surveillance	Decision-analytic	Three (progression-free disease, progressed disease, dead)	ICER per QALY
Dottino et al., 2019 [19]	Full-text	Obstetrics and Gynecology; April 2019	Patients with platinum-sensitive, recurrent ovarian cancer	Societal perspective	< 24 months	Niraparib for all patients; niraparib for patients with BRCA mutations; niraparib for patients with BRCA mutations or homologous recombination deficiency-positive tumors; routine surveillance for all patients	Decision-analytic	NR	ICER per PF QALY
Zhong et al., 2018 [33]	Full-text	Journal of Managed Care and Specialty Pharmacy; December 2018	Adult patients with recurrent epithelial ovarian, fallopian tube, or primary peritoneal cancer who have received previous treatments of platinum-based chemotherapy and were in a complete or partial response to the most recent chemotherapy	US health care sector perspective	Until disease progression	Olaparib; niraparib; routine surveillance	Decision-analytic	NR	ICERs per PFS LY ICERs per PFS QALY
Smith et al., 2015 [26]	Full-text	Gynecologic Oncology; October 2015	Patients with platinum-sensitive recurrent ovarian cancer with and without germline BRCA1/2 mutation	A third-party payer	NR	Olaparib; routine surveillance	Decision-analytic	NR	ICERs per PF LYS
Alvarez-Secord et al., 2013 [17]	Full-text	International Journal of Gynecological Cancer; June 2013	Patients with recurrent platinum-sensitive high-grade serous cancers after a partial or complete response to a platinum-containing regimen	Societal perspective	12 months	Olaparib for all patients; BRCA1/2 mutation testing followed by olaparib and treat mutation carriers only; routine surveillance	Markov model	NR	ICERs per PF LYS
Wolford et al., 2019 [32]	Abstract	ASCO	Patients with germline BRCA mutations who had newly diagnosed or platinum-sensitive recurrent ovarian cancer	NR	NR	Olaparib as first-line maintenance therapy; olaparib as second-line maintenance therapy; routine surveillance	Markov model	Five (response, hematological complications, non-hematological complications, next-line therapy or progression, and death)	ICERs per PFS month of life gained ICERs per PFS LYS
Wolford et al., 2018 [30]	Abstract	ASCO	Patients with advanced/recurrent ovarian cancer	NR	NR	Niraparib; olaparib; rucaparib; bevacizumab; pembrolizumab; paclitaxel	Markov model	Five (heme complications, respond, non-heme complications, next-line onwards, death)	ICERs per PFS month ICERs per QAL month
Wolford et al., 2017 [29]	Abstract	ASCO	Patients with recurrent ovarian cancer	NR	NR	Niraparib; rucaparib; olaparib; non-platinums (doxil, topotecan, pemetrexed, abraxane, trabectedin/doxin), bevacizumab (bevacizumab/taxol, bevacizumab/doxil, bevacizumab/topotecan), platinums (carboplatin/doxil, carboplatin/gemcitabine, carboplatin/taxol, cisplatin/taxol, cisplatin/gemcitabine, oxaliplatin)	Markov model	Five (heme complications, respond, non-heme complications, next-line onwards, death)	ICERs per PFS month
Wolford et al., 2016 [28]	Abstract	ASCO	Patients with recurrent ovarian cancer	NR	NR	Olaparib; platinum-based regimen; non-platinum-based regimen; bevacizumab-based regimen	Markov model	Five (response, hematological complications, non-hematological complications, progression, and death)	ICERs per quality-adjusted PFS month
Chin and Carlson, 2018 [18]	Abstract	ISPOR	Patients with BRCA mutations associated with advanced ovarian cancer who have been previously treated with two or more platinum-based chemotherapy regimens	US health system perspective	Lifetime	Rucaparib; routine surveillance	Partitioned survival model	Three (PFS, progressed, death)	ICERs per LYG ICERs per QALY
Fernandez-Santos and Andreas, 2017 [21]	Abstract	ISPOR	Patients with relapsed ovarian cancer	Spanish National Healthcare System	NA	PLD; topotecan; ter anti-VEGF therapy (PLD + bevacizumab); trabectedin + PLD followed by platinum; anti-VEGF therapy (carboplatin + gemcitabine + bevacizumab followed by bevacizumab), PARP inhibitor maintenance (BRCA mutation) [carboplatin + paclitaxel followed by olaparib]; trabectedin + PLD	No model	NA	ICERs per LYG
Mylonas et al., 2016 [25]	Abstract	ISPOR	Patients with BRCA-mutated platinum-sensitive recurrent ovarian cancer	A third-party payer	Lifetime	Olaparib; routine surveillance	Markov model	NR	LY QALY Years spent free of chemotherapy Total cost ICERs
Wolford et al., 2019 [31]	Abstract	SGO	Patients with newly diagnosed ovarian cancer	NR	NR	Olaparib; bevacizumab	Markov model	NR	ICERs per PF LYS
Dottino et al., 2018 [20]	Abstract	SGO	Patients with platinum-sensitive recurrent ovarian cancer	US health system perspective	NR	Niraparib for all patients; BRCA germline mutation testing and niraparib for carriers; BRCA germline and tumor HRD testing and niraparib of both BRCA carriers and those with HRD + tumors; routine surveillance for all patients	Decision-analytic	NR	ICERs per PF QALY
Liu et al., 2018 [23]	Abstract	SGO	Patients with platinum-sensitive recurrent epithelial ovarian carcinoma	NR	NR	Olaparib; niraparib; rucaparib; routine surveillance	Decision-analytic	NR	ICERs per PF LYS
Liu et al., 2017 [24]	Abstract	SGO	Patients with platinum-sensitive recurrent epithelial ovarian carcinoma with germline BRCA1/2 mutations, somatic HRD, and wild-type	NR	NR	Olaparib; niraparib; rucaparib; routine surveillance	Decision-analytic	NR	ICERs per PF LYS
Smith et al., 2015 [27]	Abstract	SGO	Patients with platinum-sensitive recurrent ovarian cancer	NR	NR	Olaparib; routine surveillance	NR	NR	ICERs per PF LYS
Alvarez-Secord et al., 2012 [16]	Abstract	SGO	Patients with recurrent platinum-sensitive high-grade serous cancers following response to chemotherapy	NR	12 months	Olaparib for all patients; HRD testing and treat mutation carriers with olaparib; routine surveillance for all patients	Markov model	NR	ICERs per PF LYS

ASCO American Society of Clinical Oncology, HRD homologous recombination deficiency, ICER incremental cost-effectiveness ratio, ISPOR International Society for Pharmacoeconomics and Outcomes Research, LY life-year, LYG life-year gained, LYS life-year saved, NR not reported, PF progression-free, PFS progression-free survival, PLD pegylated liposomal doxorubicin, QALY quality-adjusted life-year, SGO Society of Gynecologic Oncology, VEGF vascular endothelial growth factor

Results

Overview

Of 193 publications returned from the search, 18 cost-effectiveness studies met the inclusion criteria (Fig. 1) [16-33]. Five of the 18 studies were published as full-text articles [17, 19, 22, 26, 33] and the remaining were abstracts presented at conferences. The 18 studies identified by the systematic literature review and the key elements of their study designs are summarized in Table 1. The study population was patients with newly diagnosed ovarian cancer for two studies [31, 32], and patients with relapsed/recurrent advanced ovarian cancer for the remaining 16 studies. Additionally, subsets of ovarian cancer patients with BRCA mutations were considered in 6 of the 18 studies [18, 21, 24–26, 32]. Most of the studies adopted a US healthcare or societal perspective.

Fig. 1

PRISMA flow diagram for cost-effectiveness analyses of PARP inhibitors. (1) Study population: studies in non-human populations (level 1: n = 13) and non-ovarian cancer populations (level 1: n = 38) were excluded. (2) Study treatment: studies that did not include PARP inhibitors (level 1: n = 41) were excluded. (3) Publication type: guidelines and recommendations (level 1: n = 1), reviews (level 1: n = 24), and letters (level 1: n = 5) were excluded. (4) Study design: clinical trials (level 1: n = 2) and observational studies (level 1: n = 4; level 2: n = 1) were excluded. (5) Outcome of interest: studies that did not report the incremental cost-effectiveness ratio (level 1: n = 1), budget impact analyses (level 1: n = 13), and cost–benefit analyses (level 1: n = 1) were excluded. ASCO American Society of Clinical Oncology, ESMO European Society for Medical Oncology, ISPOR International Society for Pharmacoeconomics and Outcomes Research, PARP poly(ADP-ribose) polymerase, PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analyses, SGO Society of Gynecologic Oncology

The majority (n = 14) of the included abstracts and full-text studies reported the incremental costs per progression-free life-year (PF LY) gained or per progression-free quality-adjusted life-year (PF QALY) gained [16, 17, 19, 20, 23, 24, 26–33]. Five studies reported incremental costs per LY or QALY gained [18, 21, 22, 25, 29]. The incremental cost per PF LY gain reported for olaparib versus placebo ranged from €63,046 [25] to $600,000 [26], depending on the country, line of therapy, and BRCA mutation status. Similarly, the incremental cost per PF LY gain reported ranged from $248,992 [23] to $496,157 [24] for rucaparib versus placebo, and from $197,000 [33] to $304,775 [23] for niraparib versus placebo.

Objective, Perspective, Subgroups, and Time Horizon for Cost-Effectiveness Models

Of the five full-text studies, QHES scores were 71 [26], 77 [19], 79 [17], 85 [33], and 92 [22] points of a possible 100 (Table 2). All five full-text studies clearly stated their objective (QHES item 1) and perspective of analysis (QHES item 2). All studies used evidence from randomized controlled trials (RCTs); however, there was variation in evidence used as inputs to cost-effectiveness estimation (QHES item 3). Specifically, the majority (four of five) of the full-text studies relied on the surrogate measure of PFS and did not include OS evidence or any attempt at extrapolation beyond the trial time horizon [17, 19, 26, 33]. As expected, studies that relied solely on PFS evidence were, by definition, restricted to shorter time horizons (QHES item 8) of < 24 months or not reported [17, 19, 26, 33], whereas the one full-text study that generated OS from a PFS/OS relationship assessed outcomes over a 20-year time horizon [22]. This trend was consistent in the abstracts we reviewed but did not score; of the abstracts assessed, two studies included a longer time horizon, with the remaining studies focused on shorter time horizons [18, 25]. Only 3 of 18 studies reported discounting outcomes, with two studies reporting a 3% discounting rate and one study reporting a 3.5% discounting rate (QHES item 8) [18, 22, 25]. All full-text studies met QHES items 4 and 6 (incremental analysis between alternatives). QHES item 4 (prespecified subgroups) was particularly important, given that economic evaluations were informed by clinical evidence specific to patients’ BRCA status.

Table 2

Quality of Health Economic Studies assessment for six full-text studies

Quality of Health Economic Studies assessment	Points	Guy et al., 2019 [22]	Dottino et al., 2019 [19]	Zhong et al., 2018 [33]	Smith et al., 2015 [26]	Alvarez-Secord et al., 2013 [17]	Wolford et al., 2020 [41]a
1. Was the study objective presented in a clear, specific, and measurable manner?	7	✓	✓	✓	✓	✓	✓
2. Were the perspectives of the analysis (societal, third-party payer, etc.) and reasons for its selection stated?	4	✓	✓	✓	✓	✓	✓
3. Were variable estimates used in the analysis from the best available source (i.e. randomized control trial—best, expert opinion—worst)?	8	✓	✓	✓	✓	✓	✓
4. If estimates came from a subgroup analysis, were the groups prespecified at the beginning of the study?	1	✓	✓	✓	✓	✓	✓
5. Was uncertainty handled by (1) statistical analysis to address random events; or (2) sensitivity analysis to cover a range of assumptions?	9	✓	✓	✓	✓	✓
6. Was incremental analysis performed between alternatives for resources and costs?	6	✓	✓	✓	✓	✓	✓
7. Was the methodology for data abstraction (including the value of health states and other benefits) stated?	5	✓	✓	✓	✓	✓	✓
8. Did the analytic horizon allow time for all relevant and important outcomes? Were benefits and costs that went beyond 1 year discounted (3–5%) and justification given for the discount rate?	7	✓
9. Was the measurement of costs appropriate and the methodology for the estimation of quantities and unit costs clearly described?	8	✓	✓	✓	✓	✓	✓
10. Were the primary outcome measure(s) for the economic evaluation clearly stated and did they include the major short-term, long-term, and negative outcomes?	6	✓	✓	✓		✓	✓
11. Were the health outcomes measures/scales valid and reliable? If previously tested valid and reliable measures were not available, was justification given for the measures/scales used?	7	✓	✓	✓	✓	✓	✓
12. Were the economic model (including structure), study methods and analysis, and the components of the numerator and denominator displayed in a clear, transparent manner?	8			✓		✓	✓
13. Was the choice of economic model, main assumptions, and limitations of the study stated and justified?	7	✓	✓	✓	✓	✓	✓
14. Did the author(s) explicitly discuss direction and magnitude of potential biases?	6	✓	✓	✓	✓
15. Were the conclusions/recommendations of the study justified and based on the study results?	8	✓					✓
16. Was there a statement disclosing the source of funding for the study?	3	✓	✓	✓	✓	✓	✓
Total score	100	92	77	85	71	79	78

aThe full-text article by Wolford et al. was published after the systematic searches were conducted and is not included in the review, but was scored nonetheless [41]. Five conference abstracts authored by Wolford et al. are included in the review [28–32]

Model Structure

In 16 of the 18 identified studies, model types were described as decision-analytic models [19, 20, 22, 23, 26, 33], Markov models [16, 17, 25, 28–32], and a partitioned survival model [18]. For the remaining two studies, one did not report the model structure [27] and the other relied on reported median OS for the CEA and did not estimate life expectancy through simulation or evidence synthesis [21]. Health states considered in the models were reported in six studies; two of the studies considered three states (progression-free, progressed disease, and death) [18, 22], while the other four studies considered five states (response, hematological complications, non-hematological complications, next-line therapy or progression, and death) [28–30, 32].

Among the five full-text studies, only two (40%) clearly displayed the economic model structure, study methods, and components of the numerator and denominator (QHES item 12) [17, 33]. It was unclear how components of the numerator and denominator were calculated in the remaining three full-text studies. Across all full-text articles and abstracts, four studies extrapolated results beyond the trial time horizon to report LYs and/or QALYs gained [18, 21, 22, 25]. However, the majority of studies did not clearly define model structures through text or figure descriptions. For example, two studies did not include a model figure or explain in detail the model structure [19, 26]. Three of the full-text studies included an upfront decision tree on BRCA testing or BRCA prevalence for targeting therapy to subgroups by BRCA status [17, 19, 26]. Two full-text studies stratified results by BRCA status [22, 33].

Data Abstraction, Cost Measurement, and Outcome Measures

All five full-text studies stated the methodology for data abstraction (QHES item 7) and cost sources and estimation (QHES item 9). Five studies (abstracts) compared PARP inhibitors with therapies that are not approved by the US FDA for the treatment of ovarian cancer [21, 28–31]. These studies, along with all full-text studies and abstracts, did not attempt any indirect treatment comparisons or meta-analyses when comparing PARP inhibitors with other active comparators. While all outcome measures were stated across the full-text studies, one study did not include negative outcomes such as adverse events (QHES item 10) [26]. Of the five full-text studies, two reported utilities using the EuroQoL 5-dimensions [17, 22], one reported deriving scores from the Functional Assessment of Cancer Therapy–Ovarian (FACT-O) scale [33], and the remaining full-texts did not use any scales (QHES item 11).

Model Uncertainty, Validation and Limitations

All five full-text studies included some type of sensitivity or scenario analyses (QHES item 5) [17, 19, 22, 26, 33]. Within the full-text papers, only two performed a probabilistic analysis [22, 33], and of the abstracts assessed, four reported a one-way or probabilistic sensitivity analysis [16, 18, 25, 27]. Within the full-text papers, only two performed a probabilistic analysis [22, 33]. All five full-text studies listed limitations (QHES item 13), but only four of the five studies discussed magnitude and/or direction of bias (QHES item 14) in the cost-effectiveness findings [19, 22, 26, 33]. We found no studies that provided comprehensive validation, as suggested by good practice guidelines, using both internal and external data [34]. Guy et al. provided a description of validation procedures with partial validation information such as Kaplan–Meier plots with parametric distribution fits for visual inspection [22]. There was no reported validation of modeled OS, which is instrumental to the prediction of both LYs and QALYs.

Key Drivers of Cost and Cost Effectiveness

Key drivers of cost and cost-effectiveness results are input parameters that produce the largest impact on model outputs (e.g. costs, QALYs, ICERs) from sensitivity analyses, scenario analyses, or modeling assumptions. Across all reviewed model-based economic evaluations, key drivers of ICERs for PARP inhibitors against comparators include inputs to the denominator such as treatment effect assumptions related to PFS and OS [18, 22, 26, 33] and health-related quality of life (HRQoL) utility scores for health states [18, 22]. Other key drivers of cost-effectiveness ratios include treatment discontinuation [22], prevalence of germline BRCA mutations [19, 26], and acquisition price [16, 19, 26, 33].

As reported in sensitivity and scenario analyses, increases (decreases) in PFS, while holding the comparator PFS and other parameters fixed, decreased (increased) ICER estimates [22, 26, 33]. Smith et al. and Zhong et al. both reported cost per PF LY ratios above $100,000 per PF LY gained for PARP inhibitors, even with considerable increases in PFS over routine surveillance in both patients with and without BRCA mutations [26, 33]. Alternatively, Guy et al. found that niraparib was cost effective at $100,000 per QALY in the base-case versus olaparib, rucaparib, and routine surveillance [22]. In this analysis, increases in PFS for niraparib were assumed to translate to increases in OS, thus reducing the ratios below $50,000 per QALY versus routine surveillance. Conversely, decreases in PFS increased the ratios above $140,000 per QALY versus routine surveillance. Time on treatment, which may be related to PFS, was associated with higher ICERs, and shorter time on treatment was associated with lower ICERs, as compared with the base-case ICER [22]. While this seems counterintuitive, it is plausible that as patients stay alive they may continue to take maintenance therapy, resulting in higher overall costs that may not necessarily be outweighed by the benefit of therapy.

The four studies that extrapolated results beyond the trial time horizon to report LYs and QALYs gained, found that increases (decreases) in OS were also associated with decreases (increases) in ICERs [18, 21, 22, 25]. Chin and Carlson varied the OS hazard ratio for rucaparib versus placebo, resulting in a range of ICERs from $325,000 per QALY at upper values and over $500,000 per QALY at lower values, compared with a base-case ICER of $361,000 per QALY [18]. Through scenario analyses, Guy et al. found that ICERs were most sensitive to the PFS/OS relationship, as well as distributional assumptions for OS extrapolation [22]. Assuming a 1:1 relationship between PFS and OS, compared with the 1:2 relationship in the base-case for niraparib versus routine surveillance, increased ICERs from $68,287 to $126,500.

Variation in utility scores had a small impact on ICERs across four studies that used utility scores to estimate a QALY or PF QALY gained [18, 19, 22, 33]. However, incorporation of HRQoL utility scores varied by study, including differences in utility scores by health state and differences in utility scores within health state, by treatment. For example, one study added utility values as a scenario analysis and found an increase from $226,000 per PF LYG in the base-case to $297,000 per PF QALY gained for niraparib versus placebo [33], whereas Chin and Carlson [18] and Guy et al. [22] found small ranges of ICER per QALY estimates (i.e. less than $25,000 change from base-case) after varying utility scores by their upper and lower bounds. Other small nuances led to differences in cost per QALY estimates. For example, Guy et al. applied a utility of 0.849 for niraparib, 0.820 for routine surveillance, and 0.769 for olaparib, all within the same PFS health state. Therefore, the study found that olaparib and niraparib had equivalent LYs but differences in total QALYs [13].

Trial evidence has suggested differences in response to PARP inhibitors by BRCA mutation status. Model evaluations found differences in ICER estimates by BRCA mutation status [22, 33]. Given improvements in PFS and OS were larger in patients with BRCA mutations, ICER estimates across studies were lower compared with patients without BRCA mutations [22, 33].

As expected, the price of therapy was also a significant driver of ICER estimates across studies [17, 19, 26, 33]. For studies that reported suggested price ranges, prices per month to align with benefit ranged from $1000 [17] up to $5000 [26]. There are two caveats to these price suggestions [17, 26]: the studies used PF LYs or PF QALYs gained to calculate threshold prices, and some studies used prior dosing strategies that have since been updated.

Input parameters that did not have a considerable impact on ICERs were adverse events and their associated costs [17, 19, 22, 33], hospice care [17], relapse chemotherapy [17, 22], and cost of BRCA testing [17, 19].

Study Conclusions and Disclosures

Four full-text studies drew their conclusions based on incremental costs per PF LY or PF QALY gained [17, 19, 26, 33]. Given there is no theoretical or empirical threshold for cost effectiveness using PFS as an effectiveness measure, coming to conclusions on whether a PARP inhibitor was or was not cost effective was not appropriate based on reviewer assessment (QHES item 15). None of the abstracts reviewed reported QALY or LY gained results. All full-text studies stated the funding source, with one study funded by industry and all others either unfunded or government funded (QHES item 16).

Discussion

The majority of studies identified in the review reported on the cost effectiveness of PARP inhibitors from a US healthcare or societal perspective, despite there being reluctance in the US to the widespread acceptance of CEAs and ICERs. It is not our intention to wade into that debate, but rather to focus on quality issues that should be embraced as a prerequisite to the use of CEAs more broadly. The QHES was designed as a minimum benchmark of quality to improve comparability across CEAs. Cost-effectiveness modeling approaches involve nuanced analytic decisions that can be the source of much debate, which at minimum should be reported clearly, and this is what the QHES judges. Therefore, the QHES is necessary but not sufficient for assessing the quality of cost-effectiveness studies. Here, we discuss overall findings on the strengths and limitations in the quality of published CEAs of PARP inhibitors in advanced ovarian cancer, together with recommendations for future studies.

Evidence Synthesis, Model Structure, and Extrapolations

The quality of evidence synthesis includes analytic decisions surrounding appropriate model comparators, inputs, model structure, and PFS/OS relationships. Appropriate model comparators include therapies indicated for the population with head-to-head or indirect treatment comparison evidence. However, in numerous studies, non-indicated therapies for ovarian cancer were included in model comparisons [21, 28–31]. Furthermore, comparisons across PARP inhibitors were made without a network meta-analysis, and using potentially biased, naive cross-trial comparisons of effect [22]. Including non-indicated therapies may be relevant given wide use in practice; however, without appropriate head-to-head evidence in practice or through a network meta-analysis may result in biased cost-effectiveness estimates.

Additionally, inputs should be clearly defined and use a consistent approach across therapies. For example, one paper assigned differential utility values for the same health state of PFS [22], which led to the same extrapolation estimate of OS across therapies, but different QALY findings. Given significant differences in tolerability or safety profiles, assigning differential utility values may be appropriate, but in this case, there was no clear justification or evidence provided in support of this assumption.

There were inconsistencies in both the reporting and application of model structures that reflect the disease course of advanced ovarian cancer. For example, seven of the studies provided generic descriptions of model structure, such as ‘decision-analytic model’ [19, 20, 22–24, 26, 33]. While the remaining studies were more specific, with reporting model structures such as Markov model [16, 17, 25, 28–32] or partitioned survival model [18], there were further inconsistencies on reporting the number of health states among those same studies [16, 17, 25, 31]. Of the remaining studies that reported the number and description of health states, the majority included five health states that included complications, next lines of therapy, and death [28-31], while others included three health states that included progression-free, progressed, and death [18, 22]. Clear descriptions of model structure that reflect the disease course given available data would improve the comparability and replicability of modeling studies for the treatment of ovarian cancer with PARP inhibitors.

Other evidence synthesis and input inconsistences were observed for the PFS/OS relationship. PFS is a key surrogate outcome for OS and is generally defined as the length of time from random assignment in a trial to disease progression or death from any cause [35]. Where OS is uncertain or immature, the assumed relationship between PFS and OS is crucial for cost-effectiveness modeling because these analyses are attempting to establish the time spent in each health state, which contributes to incremental differences in quality and quantity of life between comparators. However, there are multiple challenges to describing this relationship, including, but not limited to, trials often being powered for PFS and not OS, leading to greater uncertainty in OS extrapolations; PFS includes death as an event within survival analyses, leading to difficulties in assigning death in models at the appropriate time point without patient-level data; patients may receive multiple lines of therapy, making it difficult to estimate survival based on one therapy; and patients in the placebo arm may switch to the treatment being evaluated after disease progression, making the OS benefit of the treatment underestimated [36].

Of the four studies that extrapolated results beyond the trial time horizon to report LYs and QALYs gained, two included the source of the relationship between PFS and OS [18, 22]. Both studies used data from the phase II Study 19 trial [7, 8, 37] of olaparib versus placebo for OS data, along with PFS sourced from niraparib and rucaparib trials. However, the rationale behind the relationship between the PFS of the treatment being modeled (niraparib or rucaparib) and OS from another treatment (olaparib) was not clearly delineated in either study, and no evidence of the validity of the predicted OS curve from this relationship was reported. Results from both studies are therefore difficult to validate against other clinical and economic evidence [13]. This underscores the importance of modeling treatment effects on the correlated outcomes of PFS and OS in a way that adequately accounts for uncertainty related to parameters describing the surrogate relationship (e.g. a multivariate meta-analysis [38]), in order to fully describe the range of possible LY and QALY estimates.

Outcome Measures

Without extrapolation of OS, the majority of full-text studies were left with reporting PF LYs or PF QALYs gained [16, 19, 26, 33]. Studies that present PFS outcomes alone will be missing important changes in quality and quantity of life over time that contribute to the benefits and risks of therapy. While QALYs and LYs gained also require assumptions such as extrapolation with the relationship between PFS and OS, QALYs and LYs are easily comparable with other oncologic and non-oncologic conditions. An ISPOR Task Force on defining elements of value in healthcare noted that despite their limitations, QALYs remain the most accepted measure for capturing the incremental benefit of a treatment, although QALYs may not reflect all elements of a treatment’s value [39].

Drawing Conclusions on Value

The majority of studies reported PF LYs or PF QALYs gained, and drew surprisingly strong conclusions on value, given no widely accepted threshold for cost effectiveness exists for these effectiveness outcomes. Even with limitations on extrapolated evidence to estimate QALYs and LYs gained, many countries around the world make coverage and reimbursement decisions based on cost per QALY (or similar metric) ratios. Even in the US, value assessment bodies use cost per QALYs or LYs, alongside benefits, disadvantages and contextual considerations, to identify high- and low-value interventions [40]. Regardless of use in coverage and reimbursement decisions, modelers should focus on what they can control through their analytic decisions and not draw conclusions as to the cost effectiveness of a modeled intervention where no threshold is stated or exists for the ICER metric employed in the study. Modelers who extrapolate beyond trial horizons can attempt multiple uncertainty analyses that vary the relationship between PFS and OS. Where extrapolation may not be possible, we recommend discussing clinical benchmarks on OS that would meet commonly cited cost-effectiveness thresholds.

Since running the searches, we identified a relevant study published in full text by Wolford et al. [41]. This study evaluated the cost effectiveness of PARP inhibitors for platinum-resistant ovarian cancer from a US patient and third-party payer perspective in terms of cost per median PFS month, through a five-state Markov model. The relationship of this study and the five conference abstracts authored by Wolford that were included in this review is unclear [28-32]. The QHES score for the full-text study is 78 (Table 2), within the range of full-text studies included in the review (71–92). The conclusions of our critical appraisal and recommendations for future studies are unaltered by our additional consideration of this publication.

Conclusion and Key Recommendations for Future Cost-Effectiveness Analyses for Poly(ADP-Ribose) Polymerase Inhibitors in Ovarian Cancer

The key discussion points above translate into key recommendations for future CEAs (Table 3). Recommendations generic to all CEAs include appropriate analytic decisions around time horizon, population, model structure, and input parameters. In order to capture full benefits and costs, lifetime horizons (i.e. life expectancy of the population) should be used in the base-case analysis. This requires extrapolation beyond trial duration, and we provide additional recommendations to validate these steps below. There is strong evidence that supports differential response to therapy by mutation status. Subgroup analyses and appropriate comparators within those subgroups should be clearly defined.

Table 3

Key recommendations for future CEAs for PARP inhibitors in ovarian cancer

Category	Recommendation	Rationale
Time horizon	Time horizon should capture all relevant costs and effects, and reflect the nature of ovarian cancer	The majority of studies did not estimate the full life expectancy of ovarian cancer and thus cannot suggest value-based pricing to inform coverage and reimbursement decisions
Population and comparator	Subgroup analyses and appropriate comparators within those subgroups should be clearly defined. Define comparator therapy and dosing used in the indicated population	Multiple studies used comparators not indicated for ovarian cancer. Multiple studies compared PARP inhibitors within the class and outside of the class without indirect treatment comparison evidence
Model structure	Model structure and number of health states should be based on the disease course (e.g. second progression-free survival and time to second subsequent treatment) and available data	The majority of studies did not accurately describe the model structure used or justify the model structure based on the disease course of ovarian cancer
Costs	Drug price estimates should be reflective of prices paid by public and/or private payers depending on the perspective chosen. Adverse event costs should be included if possible. Indirect costs (e.g. work productivity losses) should be included in countries where the societal perspective is required	Price and cost estimates varied widely, even within the same perspective; results were difficult to compare within the same setting and perspective
Outcomes	Utility parameters should be consistent across health states; vary utility to identify the impact on outcome	One study used differential utility values across the same health states for different treatments without evidence on tolerability or safety differences
	Include generic outcomes (QALYs, LYs) and cancer-specific outcomes (PF LYs)
Model validation	Validate model outcomes (e.g. OS at specific time intervals) against long-term outcomes shown in RCTs and/or real-world effectiveness studies	None of the reviewed studies validated model outcomes against clinical evidence and therefore it was difficult to assess model accuracy and bias
Uncertainty	Attempt and report multiple sensitivity analyses of cost-effectiveness estimates to structural and parametric uncertainties associated with the extrapolation of immature outcomes such as OS; and/or estimate clinical benchmarks on OS that would meet commonly cited cost-effectiveness thresholds	The majority of the studies did not extrapolate survival or justify links between PFS and OS relationships
Conclusions	Conclusions from the cost effectiveness of PARP inhibitors should align with uncertainty in the evidence over lifetime horizons	The majority of the studies concluded cost-effectiveness results without consideration of uncertainty analyses and lifetime endpoints such as QALYs or LYG

CEAs cost-effectiveness analyses, PARP poly(ADP-ribose) polymerase, QALYs quality-adjusted life-years, LYs life-years, PF progression-free, PFS progression-free survival, OS overall survival, RCTs randomized controlled trials, LYG life-years gained

Future studies should clearly describe the model structure and number of health states, along with justifying the structure chosen based on the disease course and available data. Specifically, modelers should consider the role of intermediate clinical endpoints, such as second PFS and time to second subsequent treatment, in supporting such relationships [42]. Accurate descriptions of model structure would improve the comparability and replicability of model-based outcomes across studies. Further efforts to improve comparability and replicability of models can be informed by open-source modeling efforts [34, 43–45]. All inputs should be consistent with modeling assumptions, clinical evidence, and country-specific healthcare system dynamics. For example, if a lifetime horizon is chosen, costs and outcomes related to subsequent treatment should be included. Inputs such as drug price should include both the appropriate price that is paid by public and/or private payers depending on the perspective chosen [46]. Negative inputs such as adverse events and the cost of those adverse events for active comparators and the standard of care should be included in all model analyses. Additionally, utility inputs should be consistent across health states and treatments unless there is evidential support for them to vary. In other words, health states such as PFS or progressed disease should have the same mean starting utility across therapies, and any reductions in utility would come from tolerability or safety considerations.

Recommendations specific to PARP inhibitor evaluations are related to outcomes, model validation, uncertainty, and conclusions on value. Generic outcomes (QALYs) and cancer-specific outcomes (PF LYs) should both be included to increase comparability across past and future studies. Model validation should follow good research practice guidelines that include both internal and external validation [47]. Where uncertainty exists in the relationships between PFS and OS, it is important to validate modeled OS at relevant time intervals against RCT and/or real-world data. Finally, conclusions from the cost effectiveness of PARP inhibitors should align with uncertainty in the evidence. We recommend two paths, given that mature OS data are not available: attempt multiple uncertainty analyses that link different assumptions between PFS and OS; and/or estimate clinical benchmarks on OS that would meet commonly cited cost-effectiveness thresholds.

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 38 kb)

Supplementary material 2 (DOCX 31 kb)

Supplementary material 3 (XLSX 41 kb)

With poly(ADP-ribose) polymerase (PARP) inhibitors having shown promising results in ongoing trials, there is interest in better understanding their economic value.

The key drivers of the incremental cost-effectiveness ratio were treatment duration, effects, and costs, health utility, and prevalence of BRCA mutations.

Recommendations specific to PARP inhibitor economic evaluations are related to outcomes, model validation, and conclusions on value.