What is the quality of economic evaluations of non-drug therapies? A systematic review and critical appraisal of economic evaluations of radiotherapy for cancer.

BACKGROUND: Breast, cervical and colorectal cancers are the three most frequent cancers in women, while lung, prostate and colorectal cancers are the most frequent in men. Much attention has been given to the economic evaluation of pharmaceuticals for treatment of cancer by the National Institute for Health and Care Excellence (NICE) in the UK and similar authorities internationally, while economic analysis developed for other types of anti-cancer interventions, including radiotherapy and surgery, are less common.
OBJECTIVES: Our objective was to review methods used in published cost-effectiveness studies evaluating radiotherapy for breast, cervical, colorectal, head and neck and prostate cancer, and to compare the economic evaluation methods applied with those defined in the guidelines used by the NICE technology appraisal programme.
METHODS: A systematic search of seven databases (MEDLINE, EMBASE, CDSR, NHSEED, HTA, DARE, EconLit) as well as research registers, the NICE website and conference proceedings was conducted in July 2012. Only economic evaluations of radiotherapy interventions in individuals diagnosed with cancer that included quality-adjusted life-years (QALYs) or life-years (LYs) were included. Included studies were appraised on the basis of satisfying essential, preferred and UK-specific methods requirements, building on the NICE Reference Case for economic evaluations and on other methods guidelines.
RESULTS: A total of 29 studies satisfied the inclusion criteria (breast 14, colorectal 2, prostate 10, cervical 0, head and neck 3). Only two studies were conducted in the UK (13 in the USA). Among essential methods criteria, the main issue was that only three (10%) of the studies used clinical-effectiveness estimates identified through systematic review of the literature. Similarly, only eight (28%) studies sourced health-related quality-of-life data directly from patients with the condition of interest. Other essential criteria (e.g. clear description of comparators, patient group indication and appropriate time horizon) were generally fulfilled, while most of the UK-specific requirements were not met.
CONCLUSION: Based on this review there is a dearth of up-to-date, robust evidence on the cost effectiveness of radiotherapy in cancer suitable to support decision making in the UK. Studies selected did not fully satisfy essential method standards currently recommended by NICE.

Key Points for Decision Makers

Introduction

Economic evaluations are increasingly used to inform decisions about the efficient allocation of healthcare resources. To date, these methods have mainly been applied to evaluate pharmaceutical interventions. Less evidence is available for other types of intervention, such as medical devices, surgical techniques and public health interventions, and decision making in these cases appears more challenging [1, 2]. The National Institute for Health and Care Excellence (NICE) in the UK represents an independent organisation tasked with assessing the clinical and cost effectiveness of health technologies and, to date, has mainly focused on new pharmaceutical and biopharmaceutical products. However, the scope of their responsibilities continues to expand and now includes diagnostics, public health interventions and social care.

NICE has issued a Reference Case for cost effectiveness in technnology appraisal, which specifies the methods standards considered to be consistent with a UK National Health Service (NHS) objective of improving population health given limited resources and a fixed budget. An element that distinguishes NICE from most other agencies is the explicit statement of a cost-effectiveness threshold, in terms of cost per quality-adjusted life-year (QALY) gained (£20,000–30,000), to classify interventions as good value for money.

Cancer is a leading cause of morbidity and mortality worldwide. An estimated 12.7 million new cases of cancer were diagnosed worldwide in 2008, with 7.6 million cancer-specific deaths (around 13 % of all deaths) [3]. Breast, cervical and colorectal cancers are the three most frequent cancers in women, while lung, prostate and colorectal cancers are the most frequent in men [3]. Much attention has been given to the economic evaluation of pharmaceuticals for treatment of cancer by NICE and similar authorities internationally. This contrasts markedly with the level of economic analysis developed for other types of anti-cancer interventions, including radiotherapy and surgery. For example, of the 35 NICE appraisals of breast cancer interventions, 28 (80 %) were technology appraisals of pharmaceuticals and seven (20 %) were non-pharmaceuticals.

Originally undertaken to inform the UK Department of Health, the aim of this review was to assess the quality of published economic evaluation studies of radiotherapy in breast, cervical, prostate, head and neck and colorectal cancers. Critical assessment of the studies was in terms of their reporting and conduct, including whether or not they adhered to methods standards for economic evaluation. There was also an assessment of whether the studies would provide relevant evidence for decisions in a UK setting.

Methods

Five systematic literature reviews were undertaken of the published evidence on the economic evaluation of radiotherapy in breast, cervical, prostate, head and neck, or colorectal cancer in July 2012. To identify relevant evidence, the following databases were searched: MEDLINE, EMBASE, Cochrane Database of Systematic Reviews (CDSR), NHS Economic Evaluation Database (NHSEED), Health Technology Assessment (HTA), Database of Abstracts of Reviews of Effects (DARE), and EconLit. Additionally, the Cost-Effectiveness Analysis (CEA) registry (https://research.tufts-nemc.org/cear4/), the NICE website (www.nice.org.uk), recent conference proceedings and reference lists of the included studies and existing reviews were also searched (see Appendix 1, with breast cancer as an example).

Citations retrieved by the searches were filtered by title and abstract, and the full manuscripts were retrieved if they satisfied the inclusion criteria. Studies were included in the review if they assessed the cost effectiveness of a radiotherapy intervention in individuals with breast, colorectal, cervical, head and neck or prostate cancer, and presented results in the form of either an incremental cost per QALY gained or an incremental cost per life-year (LY) gained. No language, publication or date restrictions were applied to the searches. Evidence was reviewed by a single researcher (MB) in close consultation with another author (HW) and, where there was any uncertainty, the issues were discussed within the review team until a consensus was achieved.

A data extraction template was designed to include relevant information from the studies identified including setting, patient population, intervention and comparators, type of economic analysis (e.g. model-based vs. based on a clinical study) and results. Included studies were appraised on the basis of satisfying essential, preferred and UK-specific methods requirements. The checklist draws heavily on the NICE Reference Case for economic evaluations. In addition, it draws on the authors’ experience of undertaking economic evaluations, working with decision makers to use economic evaluations to inform decisions and authors’ knowledge of the published literature on checklists and methods guidelines issued by other decision-making organisations. The aim was to review the report and conduct of included economic evaluations, focusing on the quality of each study to inform decision making. As such, the checklist is concise and designed as a decision aid, clearly distinguishing where essential and preferred requirements are met, as explained below. Those methods standards for economic evaluation that are generally recommended as best practice for undertaking a high-quality study were considered as essential requirements, and include the following seven:

Comparators clearly defined and justified: the study should provide a transparent and detailed description of the interventions considered, justifying the choice of comparators with relevance to, for example, clinical practice or best practice. For example NICE guidelines [4] state that, in order to define the decision problem “a definition and justification of the technologies being compared” (page 31) is needed. Similarly, the Canadian guidelines issued by the Canadian Agency for Drugs and Technologies in Health (CADTH) [5] state that “it is crucial to select the appropriate comparators for the analysis, as the choice will be important in determining the cost-effectiveness of the intervention” (page 16). Other guidelines, for example those of the Academy of Managed Care Pharmacy (AMCP) [6] in the USA and the Pharmaceutical Management Agency (PHARMAC) [7] in New Zealand, specify which comparator to use in the analysis (for example, “the treatment that most prescribers would replace in New Zealand,” page 8).

Patient group/indication clearly described: a clear description of the characteristics of the patient group receiving the interventions should be provided. For example, NICE states that the decision problem is supported by “a definition and justification of the relevant patient group (s) to be treated” (page 31). Some guidelines (e.g. NICE, CADTH, those issued by the Pharmaceutical Benefits Advisory Committee [PBAC] [8] in Australia) also highlight the importance of defining potential subgroups.

Effectiveness evidence based on a systematic review: evidence of a full literature search and, where appropriate, a synthesis of evidence on health effects informed by systematic review, are recommended by most economic guidelines (e.g. CADTH, PBAC, NICE, AMCP, the Healthcare Insurance Board (CVZ) [9] in the Netherlands). This ensures the data used represent all the relevant published evidence available at the time of the study, and helps to minimise bias. Data synthesis methods such as meta-analysis are recommended as appropriate. For example, the NICE reference case states that “evidence on outcomes should be obtained from a systematic review, defined as the systematic location, inclusion, appraisal and synthesis of evidence to obtain a reliable and valid overview of the data related to a clearly formulated question…. Synthesis of outcome data through meta-analysis is appropriate provided there are sufficient relevant and valid data that use measures of outcome that are comparable.”

Data for measurement of health-related quality of life (HRQoL) reported directly by patients and/or carers: it is generally recommended (e.g. NICE, Spanish guidelines [10], PHARMAC) that information on changes in HRQoL should be reported directly by patients with the condition under analysis, or by their carers when it is not possible to obtain these data directly from patients. The NICE reference case states that the “measurement of changes in HRQL should be reported directly from patients.”

Appropriate time horizon: the time horizon should be long enough to reflect all important differences in costs or outcomes between the interventions being compared. In the case of cancer, a lifetime horizon is typically the most appropriate, as the interventions could potentially have an effect on survival duration.

Relevant one-way sensitivity analyses: this is necessary to capture the full range of uncertainty that is relevant for key (model) inputs. In some circumstances, alternative scenarios and worst/best cases should be considered. Essentially, all guidelines for economic evaluation recommend the use of at least a one-way sensitivity analysis for key uncertain parameters.

Probabilistic sensitivity analysis (PSA) used to quantify full uncertainty: PSA is considered the most appropriate technique to analyse joint uncertainty in all parameters. Although only NICE recommends PSA for submissions, this is also encouraged by other agencies (AMCP, CADTH, CVZ).

Two methods features are more controversial but preferred for economic evaluation in many settings:

CEA using QALYs: although the use of a CEA based on QALYs is often stated as the preferred option to account for the impact of an intervention on HRQoL and survival duration using a consistent generic measure of outcome, this is not always seen as mandatory by all international agencies.

Preference data for valuation of HRQoL health states by a representative sample of the public: although some agencies (e.g. NICE, CADTH) recommend that the valuation of HRQoL should be based on the preferences of a representative sample of the general public, this is not always seen as mandatory, and valuation from patients, carers or health professionals are accepted alternatives by some agencies.

Finally, a secondary objective of the review was to assess the relevance of the studies identified from a UK decision-making perspective. The NICE Reference Case defines five areas of methods that can be considered specific to decision making in the NHS:

Comparators used in the UK NHS at the time each study was undertaken

Decision problem and analysis relating to a UK setting

Cost perspective of UK NHS and personal social services costs

Discount rate of 3.5 % per annum for costs and outcomes

Cost per QALY below the NICE threshold value (£20,000–30,000 per QALY).

The last point assesses if the intervention(s) analysed would be considered cost effective in the UK context

Results

Results of the Review

A flow diagram of the results of the searches is presented in Fig. 1. A total of 3,358 titles and abstracts were initially identified (breast 794, colorectal 623, prostate 707, cervical 325, head and neck 909). After excluding duplicates and studies that did not meet inclusion criteria, 116 full papers were retrieved (breast 18, colorectal 18, prostate 37, cervical 13, head and neck 30). Of these, 29 satisfied the inclusion criteria and were included in the review (breast 14 [11-24], colorectal 2 [25, 26], prostate 10 [11, 27–35], cervical 0, head and neck 3 [11, 29, 36]).

Fig. 1

Flow diagram for study inclusion

Of the studies, 13 (45 %) were undertaken in the USA, and only two (7 %) studies were set in the UK. A total of 14 (48 %) of the studies selected were published a decade or more before this review. Most analyses expressed cost effectiveness in terms of incremental costs per QALY. There was some variation in the interventions considered in the analyses (especially for breast and prostate cancer). Study characteristics and the main results for each study are presented in Table 1. To judge the relevance of these findings to the UK setting and to assess the quality of the studies included, the numbers of studies that met essential, preferred and UK-specific requirements were examined. Figure 2 illustrates findings for each study included.

Table 1

Characteristics of studies included

References	Setting	Population	Intervention	Comparator	Study design	Main findings
Breast cancer
Alvegard et al., 2005 [22]	Sweden	Stage I–II BC	Postoperative RT	No RT	Model (not explicit)	Post-operative RT is cost effective when used as adjunction to no medical adjuvant tx and in high-risk pts
Dunscombe et al., 2000 [20]	Canada	Premenopausal node-positive BC	Adjuvant loco regional RT (RT plus surgery and chemotherapy)	No RT	Spreadsheet-based activity-costing model	Adjuvant RT is a cost-effective tx
Hayman et al., 1998 [13]	USA	Stage I–II BC	Adjuvant RT	No RT	Markov model	RT is cost effective following conservative surgery
Hayman et al., 2000 [14]	USA	Stage I–II BC	Adding an electron-beam boost to tangential RT	RT without tangential boost	Markov model	Boost is not cost effective using a threshold of $50,000 per QALY
Lee et al., 2002 [15]	USA	High-risk premenopausal node-positive BC	PMRT	No PMRT	Markov model	PMRT is cost-effective; results of model were robust
Lievens et al., 2005 [24]	Belgium	Stage I–III BC	Post-operative RT to the internal mammary and medial supraclavicular lymph node region	No RT	Markov model	RT was cost effective when evaluated over a long time period in tumours with a slow natural history and systemic tx for relapse
Liljegren et al., 1997 [23]	Sweden	Unifocal stage I BC post sector resection and axillary dissection	Post-operative RT	No RT	Model (decision tree)-based	Cost of RT is high; results show the importance of identifying risk factors for local recurrence
Lundkvist et al., 2005 [11]	Sweden	BC (not specified)	Proton RT	Conventional RT	Markov model	Likely that proton RT is more appropriate for individuals with higher than norm risk of CVD
Marks et al., 1999 [16]	USA	Local regional relapse node-positive BC	RT	No RT	EE using data from several clinical studies	Cost per local regional relapse prevented decreases as the number of positive axillary nodes increases
Patrice et al., 2007 [17]	USA	Early-stage BC	RT plus tamoxifen	Tamoxifen alone	Markov model	RT post conservative surgery was cost effective in older women
Prescott et al., 2007 [12]	UK	Minimum-risk (elderly women post breast-conserving surgery)	Whole breast RT	No RT	Markov model	While RT was well tolerated with no impairment on overall QoL at 3 years, the no RT intervention was cost effective
Samant et al., 2001 [21]	Canada	High-risk (postmenopausal) node-positive BC (post mastectomy)	Loco regional RT	No RT	Update of Dunscombe et al., 2000 [21]	RT appears cost effective, but further analyses needed
Sher et al., 2009 [18]	USA	Early-stage estrogen-receptor positive BC	EB-PBI; MS-PBI	WBRT	Markov model	EB-PBI is cost effective vs. WBRT, but MS-PBI is not and is unlikely to be cost effective unless the QoL after MS-PBI is superior
Suh et al., 2005 [19]	USA	Ductal carcinoma in situ	RT	No RT	Markov model	Addition of RT following BCS for pts with ductal carcinoma in situ should not be withheld because of concerns regarding its cost effectiveness
Colorectal cancer
Dahlberg et al., 2002 [26]	Sweden	Resectable rectal cancer	Preoperative RT followed by surgery within the next week	Surgery without RT	EE alongside clinical trial	Preoperative RT was cost effective even using their most pessimistic assumptions
van den Brink et al., 2004 [25]	The Netherlands	Resectable rectal cancer	Preoperative RT with TME	TME without preoperative RT	Model-based TME study	Preoperative RT was both effective and cost effective
Prostate cancer
Basu and Meltzer, 2005 [31]	USA	Early PC	RT	Surgery (not specified), WW	Model (NS)	WW was the most cost-effective option given a threshold of $50,000 per QALY
Hummel et al., 2003 [28]	UK	Early localised (TNM stages 1 and 2) PC	Traditional RT	WW, RP, brachytherapy, 3D-CRT, cryotherapy	Markov model	Brachytherapy and 3D-CRT appear the most cost-effective options but high variability was found
Konski and Watkins-Bruner, 2004 [32]	USA	Hormone-refractory PC with bone metastases	SFX, MFX	Pain medication only, chemotherapy (mitoxantrone or prednisone)	Markov model	SFX was the most cost-effective option for palliative tx
Konski et al., 2005 (RTOG) [34]	USA	Treatment of locally advanced with clinical stage T2b, T2c, T3, T4 PC without distant metastasis	RT plus hormone therapy	RT	Markov model	Adding hormone to RT provided good value for money
Konski and Konski, 2005 [33]	USA	Clinically localised PC	I-RT	3D-CRT	Markov model	RT generally cost effective, although it could be less good value for money for old pts and over a short-term time horizon
Konski et al., 2006 [35]	USA	PC eligible for surgery	I-RT	3D	Markov model	I-RT could be considered cost effective at the upper limit of acceptability
Konski et al., 2007 [29]	USA	Intermediate-risk PC	I-RT	PPT	Markov model	PPT is not cost effective in most pts
Lundkvist et al., 2005 [11]	Sweden	Unspecified PC	Proton therapy	Conventional RT	Markov model	Proton therapy appears a cost-effective option but high variation around mean values was found
Neymark et al., 2002 [27]	France	PC candidate for surgery	Hormonal therapy plus radiotherapy (COMB)	Conventional RT	EE with data from a clinical trial	COMB was dominant when mean survival time was estimated by a restricted means analysis
Samant et al., 2003 [30]	Canada	Locally advanced PC	Adjuvant goserelin in addition to RT	RT	EE with data from a clinical trial	Long-term adjuvant goserelin provided good value for money
Head and neck cancer
Higgins, 2011 [37]	Canada	Early glottis (T1 or T2) cancer	TOL	XRT	Decision tree model- based EE	TOL is dominant over XRT for this group of pts with early cancer
Konski and Watkins-Bruner, 2004 [32]	USA	Locally advanced HNSCC	AFXC, AHFXS, HFX	Standard fractionated RT	Markov model	HFX and AFXC appear the most cost-effective options
Lundkvist et al., 2005 [11]	Sweden	H&N (unspecified)	Proton therapy	Conventional RT	Markov model	Proton therapy was the cost-effective option

AFXC accelerated fractionated radiotherapy with concomitant boost, AHFXS accelerated and fractionated radiotherapy with split, BC breast cancer, BCS breast-conserving surgery, CVD cardiovascular disease, EB-PBI external beam partial breast irradiation, EE economic evaluation, HFX hyper fractionated RT, HNSCC head and neck squamous cell carcinoma, H&N head and neck, I-RT intensity-modulated RT, MFX multifraction RT, MS-PBI MammoSite partial breast irradiation, NS not specified, PC prostate cancer, PMRT post-mastectomy RT, PPT proton-beam therapy, pt(s) patient(s), QALY quality-adjusted life-year, QoL quality of life, RP radical prostatectomy, RT radiotherapy, SFX single fraction RT, TNM tumour, node, metastasis, TME total mesorectal excision, TOL trans oral CO2 laser incision, tx treatment, WBRT whole breast RT, WW watchful waiting, XRT external beam radiation, 3D-CRT three-dimensional conformal radiation therapy

Fig. 2

Quality of included studies compared with the NICE reference case. HRQoL health-related quality of life, NHS UK National Health Service, NICE National Institute for Care and Excellence, QALY quality-adjusted life-year

Essential Requirements

Among the seven criteria that were defined as essential requirements, four were met in the majority of studies; these were comparators clearly defined, patient group/indication clearly defined, appropriate time horizon and relevant one-way sensitivity analyses. In 27 (93 %) of the studies, the interventions/comparators of the analysis were clearly defined. The two studies without a clear description of the interventions compared were both in prostate cancer (Neymark et al. [27] and Basu and Meltzer [31]). These studies focused on methods as opposed to generating results to inform decision making. As such, the failure to specify the options being compared can perhaps be understood.

The interventions compared could be grouped into eight broad categories: radiotherapy techniques in early-stage breast cancer patients following conserving surgery; post-mastectomy radiotherapy versus no radiotherapy in breast cancer patients; the comparison of alternative radiotherapy techniques in patients with breast cancer; radiotherapy options for colorectal cancer; radiotherapy versus surgery in early prostate cancer; different radiotherapy techniques in patients with prostate cancer; radiotherapy plus hormone therapies in locally advanced prostate cancer and palliative treatments; and comparison of radiotherapy techniques for head and neck cancer. Only in the methods paper by Basu and Meltzer [31] was there no clear description of the patient population (defined only as prostate cancer patients). In the other studies, patients’ general characteristics were reported, at least in terms of disease severity. Given the potential impact of radiotherapy on mortality risk, the most appropriate time horizon is likely to be patients’ lifetime. A long-term time horizon (longer than 10 years) was used in 23 (79 %) studies, while six (21 %) studies [12, 26, 31, 27, 32, 38] adopted a relatively short time horizon of between 1 and 8 years or did not clearly specify the time horizon of the analysis. Finally, one-way sensitivity analyses were conducted in the majority of studies (24 [83 %]), and the impact of variations in key parameters on cost-effectiveness results was reported.

Other accepted requirements for methods standards were not generally met. First, clinical data were obtained from a systematic review of the literature in only 3 of the 29 (10 %) studies (Hummel et al. [28] for prostate cancer; Konski and Watkins-Bruner [32] for both prostate and head and neck). In one study [15], data were obtained from a published meta-analysis; 13 studies (45 %) obtained clinical data from a single study (generally a clinical trial), while the remaining analyses were informed by data from various sources either selectively chosen by the authors or not fully described.

The data for measurement of HRQoL were reported directly by patients and/or carers in 5 of 26 studies (19 %). In three studies, HRQoL was not considered. In 7 of 26 (27 %) cases, data were taken from a mixture of patients’ and experts’ judgements. In the remaining 14 of 26 (54 %) cases, data were taken from previously published analyses and it was not possible to tell who was involved in the measurement task.

Finally, PSA was performed in only 10 of 29 studies. Six of the 19 studies without a PSA were conducted before 2001 when this technique was not commonly used, but the remaining 13 studies were published more recently.

Preferred Requirements

QALYs were used as the main outcome measure in 26 of 29 cases (90 %). The inclusion of HRQoL preference data valued by a representative sample of the public was rare, with only 3 of 26 (10 %) analyses [12, 25, 28] using these general population weights. In most cases (14 of 26, 54 %), little detail on sources of valuation of HRQoL data was provided and it is unclear whether these were taken from the general public.

UK-Specific Requirements

Only two studies were conducted in the UK (Hummel et al. [28] for prostate cancer and Prescott et al. [12] for breast cancer). Therefore, the UK NHS and personal social service costs and the UK setting were used in these analyses. Prescott et al. [12] adopted a 3.5 % per annum discount rate both for costs and for benefits while, in the Hummel et al. [12] study, the NICE discount rate of 6 % for costs and 1.5 % for benefits was used as they based their analysis on an earlier NICE Reference Case (2001). The majority of studies (24 of 29 [83 %]) used comparators relevant to the UK, even though they were not developed for UK decision makers specifically. Exceptions include studies by Lundqvist et al. [11] (both for prostate and for head and neck cancer) and Konski et al. [29] (who assessed the cost effectiveness of proton-pump therapy, which are not frequently used in the UK), and two studies on colorectal cancer [25, 26], which considered interventions whose use in the NHS currently varies widely.

As a number of the costs for the analyses were not undertaken using British pounds sterling, the currency for the cost-effectiveness estimates were converted using standard currency conversion exchange rates (year 2012). NICE’s cost-effectiveness threshold value was applied to the studies to assess the value for money on the interventions in the UK NHS. Incremental cost-effectiveness ratios were not inflated since the objective of the analysis was to assess whether these estimates were below the £20,000–30,000 threshold at the time of each study. The main intervention under analysis would be considered cost effective in 16 (55 %) studies using a £20,000 threshold and 19 (65 %) studies using a £30,000 threshold (Suh et al. [19], Konski et al. [35] and Lundqvist et al. [11] for prostate are those studies with cost-effectiveness ratios between £20,000 and £30,000). On the basis of the NICE threshold values, the following results could be synthesised: in the case of breast cancer, whole breast radiotherapy following breast conservation would be considered cost effective compared with no radiotherapy, but further studies are required to identify a very low-risk group of patients who derive minimal benefit and can, therefore, safely avoid radiotherapy. Post-mastectomy radiotherapy would be considered cost effective compared with no radiotherapy in patients at higher risk of recurrence. However, the definition of this level of risk is not clear. For colorectal cancer, preoperative radiotherapy followed by surgery compared with surgery alone would be considered cost effective. For prostate cancer, brachytherapy and 3-dimensional (3D) conformal radiation would be considered cost effective compared with traditional radiotherapy, but intensity-modulated radiotherapy would be cost effective compared with 3D conformal radiation; radiotherapy plus hormone therapy would be considered cost effective compared with radiotherapy alone in locally advanced patients; single or multi-fraction radiotherapy appears cost effective compared with palliative treatments (pain medication or chemotherapy) in hormone-refractory metastatic prostate cancer patients. Finally, for head and neck cancer, accelerated fractionated radiotherapy with concomitant boost, and hyper fractionated radiotherapy would be considered cost effective compared with standard fractionated radiotherapy in patients with local advanced disease.

Discussion

The objective of this review was to identify and summarise the evidence on the cost effectiveness of radiotherapy for various cancers, and to assess methods quality in those studies based on the NICE Reference Case. We observed substantial heterogeneity in the methods used. Overall, three key methods requirements considered essential for a good-quality evaluation (effectiveness evidence based on a systematic review, data for measurement of HRQoL reported directly by patients and/or carers, use of PSA to quantify the uncertainty) were generally not fulfilled. One of these criteria, the failure to use PSA or equivalent using individual-patient data from trials to characterise fully evidential and other forms of uncertainty, may reflect that a proportion of these studies was undertaken prior to the general use of these methods. Although there are also few examples in the later studies, it may be expected these methods will be more commonly used in the future.

The failure of a large proportion of studies (18 of 26 [69 %]) to source HRQoL measurements from patients probably reflects, in part, evidential weaknesses in the underlying clinical data being used by the economic evaluations in this review. In other words, when these studies were undertaken, few HRQoL data were collected in trials, particularly generic preference-based instruments suitable for estimating QALYs. More recently, methods to ‘map’ HRQoL weights from disease-specific instruments or clinical measures have been developed (Hernàndez Alava et al. [36]), and it may be expected they will be used in future studies where there are no direct estimates of suitable HRQoL weights. Where HRQoL weights are not available to estimate QALYs, use of high-quality mapping strategies can provide an informative approach to address the gap [43].

Arguably the most concerning method limitation is the failure of most studies (26 of 29 [90 %]) to incorporate the full extent of the available evidence on clinical effectiveness into economic analyses using systematic review. A general principle of evidence-based medicine, as well as economic evaluation, is not to be selective with evidence. However, in the review, many studies (12 of 29 [41 %]) used a single source of clinical-effectiveness evidence such as a randomised trial. It is possible that single clinical studies represent the only clinical evidence available in some cases, but this was not discussed explicitly in the papers. Where more than one clinical study has been identified through a systematic review, evidence on effectiveness for economic evaluation should be synthesised using, for example, meta-analysis. At the very least, a range of scenarios should be considered in the economic analysis using those different sources of clinical evidence. However, in this review, a minority of studies used meta-analysis or scenario analysis to bring in a fuller range of clinical evidence.

In terms of relevance for UK decision makers, only two studies were conducted in the UK and fully met the NICE UK-specific requirements. It is difficult to judge the relevance of the results of studies conducted in other jurisdictions, especially due to differences in clinical practice and sometimes in methods criteria specified by reimbursement and pricing agencies. Although most studies included interventions that were relevant to UK clinical practice at the time of publication, they might be dated due to improvements in clinical techniques and less important for current policy making. Radiotherapy techniques used in the cancer areas investigated have evolved rapidly over the last decade and many newer techniques are currently being evaluated in clinical trials. In addition, the clinical evidence in many of the studies is old and a number of new trials describing novel radiotherapy interventions have been reported, or are currently ongoing (e.g. CRO7 trial, SUPREMO trial, TARGIT-A trial, Warde et al. [37-40]).

Essential and preferred methods requirements were mainly taken from the NICE Reference Case, supported by other guidelines (e.g. AMCP, CADTH, PBAC). NICE represents an oft-cited independent, centralised authority where decision making is informed by CEA. A strength of NICE is the provision of an explicit cost-effectiveness threshold to judge the value for money of the intervention considered. Another point that is not explicitly defined in the NICE Reference Case but is of considerable importance is the impact of heterogeneity in the cost effectiveness of treatments across different subgroups of patients. In the case of cancer, the identification of specific types of patients in whom radiotherapy is cost effective appears a key issue. For example, in the case of breast cancer, an on-going meta-analysis by the Early Breast Cancer Trialists’ Collaborative Group (EBCTCG) [41] of over 10,000 women treated in clinical trials with and without radiotherapy, has shown that, overall, breast radiotherapy reduces any cancer recurrence by 50 % [38]. However, although the relative advantage for breast radiotherapy is constant, the absolute benefit for individuals varies depending on their risk of recurrence and this, in turn, drives the cost effectiveness of radiotherapy in these patients. It is likely that, in a very low-risk group of patients, radiotherapy would not be cost effective. The same can be stated of prostate cancer, in which the relative merits of radiotherapy versus watchful waiting for low-risk disease has not been tested but is unlikely to demonstrate improvements in disease-specific survival up to 10 years of follow-up, since a comparison of radical prostatectomy and watchful waiting has shown no survival benefit [42].

In conclusion, based on this review, there is a dearth of up-to-date, robust evidence on the effectiveness and cost effectiveness of radiotherapy in cancer. The number of published economic evaluations of radiotherapy using QALYs or LYs saved appears quite low. For example, only two studies were found for colorectal cancer, three for head and neck cancer and none for cervical cancer. Some essential methodological standards were generally not met, especially in the identification of the clinical evidence. Consequently, the full uncertainty associated with the clinical benefits of the interventions was rarely captured. Finally, few of the studies were based in the UK, and international costs of interventions and healthcare use do not generally transfer to the UK. As new, relevant evidence becomes available, additional cost-effectiveness evaluations should be undertaken to inform decision makers for current and future clinical practice in this important area.

Few studies were found that examine the cost effectiveness of radiotherapy for breast, cervical, colorectal, head and neck and prostate cancer, and only two analyses were conducted in the UK.

Additional, high-quality evidence is required to inform decision making on the effectiveness and cost-effectiveness of radiotherapy in cancer.

Many recent publications identified in the review did not satisfy essential methods requirements. Forthcoming economic evaluations of radiotherapy in cancer should adhere to such requirements to better inform decision makers.