Practical issues in handling data input and uncertainty in a budget impact analysis.

The objective of this paper was to address the importance of dealing systematically and comprehensively with uncertainty in a budget impact analysis (BIA) in more detail. The handling of uncertainty in health economics was used as a point of reference for addressing the uncertainty in a BIA. This overview shows that standard methods of sensitivity analysis, which are used for standard data set in a health economic model (clinical probabilities, treatment patterns, resource utilisation and prices/tariffs), cannot always be used for the input data for the BIA model beyond the health economic data set for various reasons. Whereas in a health economic model, only limited data may come from a Delphi panel, a BIA model often relies on a majority of data taken from a Delphi panel. In addition, the dataset in a BIA model also includes forecasts (e.g. annual growth, uptakes curves, substitution effects, changes in prescription restrictions and guidelines, future distribution of the available treatment modalities, off-label use). As a consequence, the use of standard sensitivity analyses for BIA data set might be limited because of the lack of appropriate distributions as data sources are limited, or because of the need for forecasting. Therefore, scenario analyses might be more appropriate to capture the uncertainty in the BIA data set in the overall BIA model.

Introduction

Health technology assessment (HTA) is “a multidisciplinary field of policy analysis, studying the medical, economic, social and ethical implications of development, diffusion and use of health technologies” [1]. Historically, HTA agencies have focussed on producing high quality assessment reports to inform decision-makers. Nowadays such organisations are increasingly undertaking or commissioning HTAs to inform a particular resource allocation decision, i.e. appraisals such as: listing a drug on a national or local formulary, defining coverage and insurance plans, and issuing mandatory guidance on the use of health care technologies [2]. For many health care systems, three goals can be identified: cost containment, cost-effective implementation of technologies, and a sustainable health care system requiring instruments encouraging innovations. These three goals are often in conflict with each other. Therefore, a successful HTA approach for health care technologies constitutes a balance between three goals.

Escalating costs have become a major concern for healthcare professionals, decision-makers and the public, prompting the implementation of new cost containment measures over the last decade, especially for new pharmaceuticals. Reimbursement of new pharmaceuticals up to now was based simply on registration data (efficacy, safety and quality parameters). In recent years we have learned to distinguish various additional data requirements, especially relating to the use of innovations in real daily practice. The most important new data requirements are effectiveness, cost-effectiveness and budgetary impact. Other considerations might also be taken into account depending on the specific indication, e.g. equity in the case of life-style drugs or orphan drugs.

The focus of this paper is on offering some guidance for the execution of a budget impact analysis (BIA). Budget impact data from a financial analysis are able to present the impact of an innovation on a national annual drug budget as well as on overall health care budgets. The purpose of a BIA is to estimate the affordability of a new health care intervention for specific health care decision-makers. A deeper insight into budgetary consequences is necessary in making choices within the context of unavoidable budgetary restrictions. BIA might become an essential part of comprehensive economic assessments of health care programs, along with cost-effectiveness analyses (CEA), before national or local formulary approval or reimbursement.

The main difference between CEA and BIA is in their respective goals. CEA looks at a representative individual or cohort and estimates costs and benefits of the new intervention over a time horizon that captures all relevant costs and benefits. CEA thus represents the traditional approach to health economic evaluation, combining clinical and economic outcomes. In contrast, BIA looks at the current and future population of interest to the decision maker and estimates the impact of the new intervention on short- or longer-term annual healthcare budgets.

There are also other differences, aside from the obvious, between a health economic and a financial analysis:

The perspective in a health economic study in most countries is a societal one, including non-medical costs and indirect costs, and costing ought to be based on opportunity costs. A financial analysis is performed from the perspective of the payer, includes mainly medical costs, and costing is based on actual prices and tariffs.

The time horizon in a health economic study depends on the disease and may vary from only an episode (e.g. treatment of infections) to a life-time in chronic diseases (e.g. multiple sclerosis). The time horizon for a financial analysis is based on the perspective (or needs) of the payer and corresponds to budgeting periods varying from 1 year up to 5 years.

The target audiences for health economic studies are usually central reimbursement authorities, which have an interest in aggregated outcomes. The target audiences of financial analyses are also central authorities, but may also include regional decision-makers (insurers, hospitals), where organisational structure often results in a silo-mentality leading to a higher interest in disaggregated outcomes. For example, a hospital pharmacist might be interested only in the impact of a new pharmaceutical on his drug budget, regardless of other cost savings for the hospital, e.g. a reduction in length of stay.

The development of prescription restrictions is another policy tool used to control the costs of pharmaceuticals that is closely linked to BIA. While traditionally reimbursement decisions were based on the registered indication, authorities have recently started issuing restrictions on claims made for a drug. Through the creation of prescription guidelines for certain conditions, healthcare authorities can exercise control over physicians’ prescribing. These recommendations are seen as an extension of non-reimbursement lists. The primary goal of a prescription restriction is to reduce and control the budget impact of the reimbursement of a new pharmaceutical. Therefore, prescription restriction and BIA are closely related: a prescription restriction is often developed based on the outcome of a BIA for the registered indication, if the assumed budget impact is too high.

The primary purpose of a BIA should be to provide information on the organisational and economic consequences of the implementation of a new technology in order to create space and budgets for the introduction of new cost-effective technologies. Similarly, BIA should also be used to identify corresponding decreased budgets due to withdrawal of non cost-effective technologies. However, our experience is that the purpose of a BIA is often misunderstood because it has been used mainly as a tool for slowing down the speed of the uptake of new technologies. Examples of the use of BIA as a tool for analysing withdrawal of non cost-effective treatments are rare.

If BIA is used for reimbursement issues by authorities or third-party payers, it is vital that such analyses are performed according to generally accepted and standardised methods. This increases the transparency of a financial analysis, allowing for more rational interpretation of the BIA results and a better comparison of different analyses for new pharmaceuticals. This is especially important when budget constraints allow the reimbursement of only a limited number of new pharmaceuticals. Guidelines for cost-effectiveness studies were developed initially in Australia [3] and Canada [4], but are currently available for most Western countries, e.g. Germany [5]. However, national authorities in the past rarely published any BIA guidelines. To fill this gap, an International Society For Pharmacoeconomics and Outcomes Research (ISPOR) Task Force published the first international guideline for the execution of a BIA, also describing a general approach for the handling of uncertainty [6].

The objective of this paper is to address in more detail the importance of dealing systematically and comprehensively with the uncertainty in a BIA. The handling of uncertainty in health economics is used as a point of reference for addressing the uncertainty in a BIA, as a substantial number of methodologies have already been developed for CEA that may be applicable for BIA. The first part of this paper describes the modelling design of a BIA and the required data input for the BIA model and its data sources. The second part describes the handling of uncertainty in the BIA model.

Design of a BIA

Model design

The design of a new innovative drug will usually be a modelling study based on decision analytic techniques, as a population-based cohort study is not feasible. Therefore, the handling of uncertainty in cost-effectiveness modelling studies may be applicable to a BIA model. A cost-effectiveness model should, as much as possible, reflect the real life situation of the disease, incorporating current treatment patterns with input values (probabilities and items of healthcare utilisation) deviating as little as possible from population values [7]. This requirement for a cost-effectiveness model applies also to BIA, as the aim of BIA is to show the impact of a new drug on the annual national drug budget and/or the overall health care budget in daily practise after launch. Models may take the form of simple decision-analytic trees or may be very complex Markov models of underlying disease processes and treatments. Building on those findings, the BIA model might be an extension of a cost-effectiveness model [8].

To illustrate this approach, a model was constructed for a new hypothetical drug in Parkinson’s disease, which allowed us to determine the budgetary impact and the cost-effectiveness of a new drug from the society perspective in The Netherlands. This study presents a model for an appropriate assessment of the budgetary impact of a new drug, which can be used simultaneously for traditional CEA. [8] The use of a Markov model allowed both a financial analysis and a CEA to be performed. The advantage of combining BIA and CEA is that the analyses are based on the same underlying assumptions and data, which will increase the consistency between the outcomes of both analyses.

However, the BIA model may also be developed independently from the cost-effectiveness model, which could result in different structures for the two models. For example, a cost-effectiveness model often requires a complex Markov structure, whereas a BIA model may often be based on a more simple structure.

A BIA model depends on the pathology, including underlying pathophysiological mechanisms, prognosis, disease progression, and existing treatment options, all of which are relevant to the design of the BIA study.

Prognosis

Many diseases will not lead to a complete recovery. There might be: (1) an increased risk of relapse (e.g. depression), (2) incomplete recovery (e.g. stroke), or (3) other morbidities (e.g. myocardial infarction due to angina pectoris). In addition, disease progression may be characterised by (1) episodes with intermittent symptom-free periods; (2) exacerbations, from which there may be only partial recovery, leaving the patient in a worse health state after each episode (e.g. multiple sclerosis); or (3) a more continuous chronic character (e.g. Parkinson’s disease). A patient may also have a higher risk of co-morbidities than the general population (e.g. infections) regardless of his/her disease progression. This information is needed to decide on the type of BIA model (prevalence-based BIA model, incidence-based BIA model or a BIA model based on both prevalence and incidence) and its characteristics (need of health states, pathways, follow-up period) and effectiveness measure(s). Prevalence-based and incidence-based models are often developed independently from the cost-effectiveness model, because those models usually do not require the complexity of a cost-effectiveness model. However, a BIA model that includes prevalence and incidence often does require the complexity of a cost-effectiveness model, and therefore can be based on an extension of the underlying cost-effectiveness model.

Choice of comparator

The choice of the comparator is based primarily on country-specific treatment patterns and clinical guidelines. Relevant comparators may include other drugs, other medical care such as surgery or watchful waiting, or no treatment. One difference compared to a cost-effectiveness study is that a BIA evaluation does not consider the comparator as a single treatment modality, and consists of a comparison with separate homogenous comparators. The comparator in a BIA study consists of a mix of available treatment modalities in a health care setting from the perspective of a specific target audience. The comparator mix might also be used to decide on model characteristics. Finally, information on the type of physician (GP or specialist) and setting (in-patient or out-patient) associated with each treatment modality helps to define the perspective of the study and the selection of economic outcomes. For example, the comparator mix from a hospital perspective may be different from the insurer’ perspective; the comparator mix may also differ between regions.

Cost effectiveness model

Three different types of data can be distinguished in a cost-effectiveness model:

Clinical probabilities. These mostly cannot be influenced by a physician and are derived from clinical measures. Examples of clinical probabilities are response to a treatment, relapse after a previous response, and mortality.

Treatment patterns. In addition to standard therapy being the first choice of therapy for the total cohort entering the model, the following decisions on the choice of therapy may occur in the model: (i) therapy after a response (termination of treatment or continuation of treatment) (ii) therapy after a treatment failure in response to the initial treatment; and (iii) therapy after a treatment failure, for example, a change to second-choice therapy.

Costing information. Costing involves estimating the units of healthcare utilisation used and their prices/tariffs (product of unit and price).

The data for a cost-effectiveness model may come from a variety of sources and are subject to varying degrees of uncertainty. Data sources for the variables being used in a cost-effectiveness model may be clinical trials, literature (e.g. meta-analysis), databases, medical records, Delphi panels and/or official tariff lists. These data sources will yield, for each variable, a fixed input value and a range.

The cost-effectiveness model calculates costs at the patient level, whereas the BIA calculates costs at a population level. Therefore, a BIA model can be considered an extension of the cost-effectiveness model. The translation from cost per patient to budget impact at the population level requires an additional step. The cost-effectiveness model may be used to calculate the average cost per patient over each subsequent year after initiation with the new treatment. The next step is to calculate the budget impact by multiplying the average cost per patient by the number of patients by the incorporation of epidemiology data and other specific data (e.g. diffusion curves) for a financial analysis (Table 1).

Table 1

Input parameters in a budget impact analysis (BIA) for cost per patient, number of patients, and time horizon

BIA	Input parameters
Cost per patient	Probabilities
	Treatment patterns
	Costing information: resource utilisation and prices and tariffs
Number of patients	Prevalence
	Incidence
	Proportion of identified patients
	Proportion of eligible patients
	Proportion of patients in clinical trials
	Annual growth rate for utilisation of the technology
	Existing mix of available treatment modalities
	Information on dosing
	Treatment sequencing
	Diffusion (uptake)
	Substitution effects
	Off-label use
Time horizon	As demanded by the research question at hand

The potential number of candidates of the target population for a new pharmaceutical depends on epidemiology (prevalence and incidence of the pathology), prescription restrictions, growth of the target population, off-label use, and the existing treatment mix and diffusion curves.

Handling of uncertainty in BIA

Measurement of uncertainty

Sensitivity analyses aim at providing information on the degree of uncertainty in economic evaluations, and are currently the most widely applied method of dealing with uncertainty in economic evaluations [9]. The methods used, the choice of parameters and the range of these parameters must be stated and substantiated.

The conventional approach is a ‘one-way’ sensitivity analysis, which varies one of the input variables from its baseline values while observing the effect on the outcome of the model. In a multivariate sensitivity analysis more than one input variable is varied. A probabilistic sensitivity analysis (PSA) permits the analyst to assign a range and distribution to input variables [10]. The results of a PSA are presented in a cost-effectiveness acceptability curve, which displays the probability that a new treatment is the most cost-effective treatment considered in the analysis at a range of different threshold incremental cost-effectiveness ratio (ICER) values representing what society might be willing to pay to gain one, e.g., quality-adjusted life year (QALY).

The approach of handling uncertainty in cost-effectiveness models is also applicable to components of the BIA model that relate to the above-mentioned types of data (clinical probabilities, treatment patterns, resource utilisation and prices/tariffs). Although the structure of the BIA model may differ from the cost-effectiveness model, the required data on clinical probabilities, treatment patterns, resource utilisation and prices/tariffs might be derived from the cost-effectiveness model.

In the remainder of this manuscript, we will consider the use of the above-mentioned sensitivity analyses for dealing with uncertainty in BIA.

Uncertainty in input data for the BIA model corresponding with the health economic data set

The handling of uncertainty of the input data of the BIA model, which corresponds with the standard data set for a cost-effectiveness model (clinical probabilities, treatment patterns, resource utilisation and prices/tariffs), can be based on the standard methods of sensitivity analysis that are used for CEA.

The use of prescription restriction requires cautious handling of the clinical probabilities. The primary objective of a phase III clinical trial is to obtain registration by the official registration authorities (e.g. EMEA, FDA) for a new drug. Clinical trials for new drugs are usually designed only with sufficient sample size and power to demonstrate a clinically meaningful and statistically significant difference (today mostly superiority) for the primary efficacy outcome in the overall study population. Therefore, a prescription restriction, which reduces the registration indication to a smaller subpopulation for reimbursement purpose, suffers from various methodological weaknesses. The use of clinical data from subgroup analysis in a BIA analysis often suffers not only from statistical constraints, but also from that major methodological weakness that the trial was designed only for the overall trial population (e.g. the hypothesis, objective and selection of the primary clinical outcome). As a consequence, a standard sensitivity analysis based on the distribution of the clinical probabilities does not capture the total uncertainty associated with the clinical probabilities. Therefore, additional scenario analyses must be performed, for example by the extension of the range or the confidence interval of the clinical input variables, or the use of clinical probabilities from the overall study population in the clinical trials.

Other scenarios may also have to be considered, but the selection of these additional scenario analyses remains subjective, in contrast to standard sensitivity analyses, which are based on statistical distributions. Therefore, a justification of the rationale of each scenario analysis is important for interpretation of the results of the base case analysis of a BIA and the associated uncertainty in BIA outcomes.

Uncertainty in input data for the BIA model beyond the health economic dataset

Other input data in the BIA model (BIA data) differ from the standard health economic dataset. In cases where there is no valid data from national data sources, the use of a Delphi panel is the only option for either validation of international data or adaptation of international data to the study country of the BIA. In addition, this approach allows estimates to be elicited when there are no data available or when the required input information relates to forecasts.

For BIA analyses in The Netherlands the use of data from Delphi panels in a BIA has been accepted in previous submissions. Examples of handling missing data by means of a Delphi panel, as detailed below, were applied in these BIA analyses. The handling of the following data subsets is ordered in a step-by-step way to correspond with the topic sets in the first section of this paper describing epidemiologic data needs. In this way, we hope to give the reader a structured look at the practical issues surrounding incorporation of uncertainty in a BIA.

Prevalence and incidence

Prevalence can be the main driver in BIA for chronic diseases, where the annual inflow (incidence) equals the annual outflow of patients (e.g. improvement, progression or mortality) within the registered indication. Incidence can be the main driver for episodic diseases with full recovery or death (e.g. infectious diseases), or diseases where the initiation of treatment starts at diagnosis and selection of treatment is based on staging (e.g. adjuvant or advanced treatment in oncology). Prevalence and incidence data should be derived from the published literature—ideally country-specific literature, e.g. a previous BIA for medications with similar indication. That data should allow for the construction of statistical distributions for the standard sensitivity analysis. If no country-specific epidemiology data is available, additional scenario analyses may be performed in order to capture this additional uncertainty. International epidemiology data may also be validated by a Delphi panel of national clinical and epidemiology experts in order to generate a national estimate with a distribution.

Proportion of patients

Proportion of patients actually identified by a general practitioner (GP) or a specialist and really being treated

As an example, the total number of registered rheumatoid arthritis (RA) patients was 148,000 in The Netherlands at the time-of-launch (1999) of currently available biological agents. However, the total number of registered adult patients being treated by rheumatologists ranged between 30,000 and 60,000 patients [11]. As biologicals can be prescribed only by rheumatologists in The Netherlands, the subpopulation of RA patients being treated by a rheumatologist yields the potential number of candidates for treatment. The remaining RA patients are treated by a GP and would not fulfil the criteria for biologicals, because GPs are not allowed to prescribe these drugs. On the other hand, there may be a pool of eligible patients with previous failure on current standard treatment, who may have been lost to follow-up and may return to their specialist when these patients or their GPs become aware of biologicals as a new treatment option. This pool of patients should be considered as a separate cohort of patients in the BIA because of different patient characteristics and because this pool has only a temporary budget impact, as patients with treatment failures before the introduction of new drugs will disappear over time.

The proportion of patients who actually have been registered by the GP or specialist and are being treated might be derived from national databases or published literature, e.g. a previous BIA for medications with similar indication, which allows the construction of statistical distributions for the standard sensitivity analysis. However, this data often is not available. A national Delphi panel may be used to elicit an estimate with a distribution. For example, in a BIA for biologicals in psoriasis, it was not possible to derive the number of psoriasis patients treated by a dermatologist from the total number of psoriasis patients in The Netherlands. Therefore, a Delphi panel estimated the number of psoriasis patients treated by dermatologists, which was based on the number of dermatologists and the number of psoriasis patients treated per dermatologist. There are approximately 300 dermatologists in The Netherlands. Every dermatologist treats approximately 75 (minimum 50, maximum 100) psoriasis patients. Consequently, approximately 22,500 (300 × 75) psoriasis patients are treated by dermatologists.

Proportion of patients eligible for treatment with the new pharmaceutical

This proportion has been based traditionally on the registered indication for the new pharmaceutical but, beginning in the last decade, reimbursement authorities have increasingly imposed prescription restrictions. For example, biologicals in RA in The Netherlands and Germany are restricted to patients with a failure to two disease-modifying antirheumatic drugs (DMARDs) instead of the broader registered indication, which required only a failure to one DMARD.

This input variable may be derived from the published literature—ideally country-specific literature, e.g. a previous BIA for medications with similar indication, which allows the construction of statistical distributions for the standard sensitivity analysis. For example, the proportion of oncology patients eligible for standard 1-line or 2-line treatment is usually well documented for oncology indications with a high prevalence, e.g. colon carcinoma, lung carcinoma, and breast carcinoma. However, this epidemiological information might not be available when an existing indication, e.g. 2-line treatment in oncology, is further restricted leading to a new niche and therefore lacking existing epidemiology data. National databases, which contain prescription data, can sometimes provide the proportion of eligible patients. However, a limitation of most prescription databases is that they provide only utilisation data for all drugs, but without the option of categorisation according to different indications. Therefore, this type of database is an appropriate data source only when the current standard treatment for the indication of the new pharmaceutical consists of medication that is not prescribed in other indications. For example, biologicals were prescribed initially only in RA and therefore actual use of biologicals from a database would be a reliable estimate of the proportion of eligible patients for a BIA for new pharmaceuticals in this restricted indication. However, most current biologicals for RA are now also prescribed in other disease areas and therefore prescription data on biologicals from a database would currently not provide information of the proportion of eligible patients for this new pharmaceutical (of course depending maybe on other information recorded in the corresponding database). The standard treatment for a new pharmaceutical may not always be another medication. For example, a new pharmaceutical may be indicated for patients who are candidates for a cardiovascular intervention [e.g. coronary artery bypass grafting (CABG) or percutaneous transluminal coronary angioplasty (PTCA)]. In this case, the proportion of eligible patients may be derived from national hospitalisation statistics. However, often there is no data available on the proportion of eligible patients, and the use of a Delphi panel is required to elicit an estimate of the proportion of eligible patients. For example, in the above-mentioned BIA analysis for biologicals in psoriasis, the panel estimated that 7% of patients treated by a dermatologist (minimum 5%, maximum 10%) are estimated to meet the prescription criteria. This percentage leads to 1,575 (7% of 22,500) candidates for biologicals.

Proportion of eligible patients actually treated with the new pharmaceutical

The majority of patients have clinical grounds for treatment, but reasons for non-treatment with a biological may also be patient-related, especially considering the risk of adverse events and contra-indications. The main reason for a patient’s refusal is the fear of adverse events or a dislike of the mode of administration. A Dutch survey showed that only 75% of all potential RA patients are treated with a biological, implying that nearly 25% of RA patients are not treated for various reasons. Clinical reasons might include risk of infection, planned pregnancy, expected low compliance, and malignancy. RA patients themselves may dislike the subcutaneous route of administration or the short hospital stays required for intravenous administration of biologicals [12].

This facet of the BIA is a prediction (or forecast) rather than an actual figure, and therefore no real data is available for this input variable. Sometimes a previous BIA for medications with similar indication may provide relevant information, but a BIA is often performed for a new treatment that has no clinically equivalent existing medication. Clinical trial data may provide some information based on inclusion and exclusion criteria and reported adverse events, but those data may not reflect daily practise. Therefore, the only option seems to be to use a Delphi panel in order elicit forecasts on the actual proportion of patients that might be candidates for treatment with the new pharmaceutical. Drug information on contra-indications, drug interactions and safety may be provided to the Delphi panel on order to elicit a clinical justified estimate. In the BIA for biologicals in psoriasis referred to above, the panel members estimated that about 25% of all patients meeting the prescription criteria would not actually be treated with biologicals.

Proportion of patients included in clinical trials

Clinical trials can be categorised into phase III trials for registration and phase IV trials after market approval. This distinction is important because of the differential impact on the utilisation of new medications. In a phase III trial, patients are randomised to the study drug and its comparator, which are provided by the study sponsor. Consequently, phase III trials will reduce the potential total national budgetary impact if those trials are going to be performed for another new pharmaceutical with a similar target indication. In phase IV studies, patients are using standard treatment and their medication costs contribute to the budgetary impact. However, phase IV studies may have an impact on the distribution of the current treatment mix, which may be especially relevant for biologicals.

Information on ongoing clinical trials for new competitive pharmaceuticals may be derived from the FDA or the EMEA. This information (study countries and number of sites, sample size) may be used to estimate the potential number of patients in the country of the BIA. New competitive pharmaceuticals, which are currently in phase II, may also lead to clinical phase III trials. However, as phase III trials may not yet have been submitted to registration authorities for approval (in the country of interest), it is difficult to estimate a potential number of patients. The use of a Delphi panel for this type of information is also not a reliable source, as clinical experts may also not have this information, with the exception of some clinical opinion leaders who might be involved as investigators and may provide estimates. On the other hand, they may have confidentiality conflicts prohibiting the use of such input.

Growth in utilisation

This refers to the annual growth in utilisation of the new pharmaceutical over the time horizon of the BIA. The actual number of eligible patients might be based on the assumption that the annual increase of eligible patients equals the annual growth rate of the total population. However, changes in prescription restriction guidelines may lead to a different growth of the actual number of eligible patients. For example, the indication for reimbursement might be broadened, leading to a relative growth rate of eligible patients higher than that of the total population. On the other hand, there may also be a pool of eligible patients with previous failures to current standard treatment, which will lead to a temporary higher growth than the annual growth of the total population. Published information in the public domain and information spread by patient societies contribute to this inflow of “old” patients. Changes in treatment patterns or guidelines may also lead to differential growth of annual utilisation. For example, continuous treatment with biologicals in RA patients may be changed to cyclic treatment, leading to lower annual drug use per patient.

This input variable is also a prediction (or forecast) instead of an actual figure, and therefore no real data is available for this input variable of the BIA. Historical data on annual growth over the previous 5 years might be extrapolated for an additional 3 years. This approach could be validated with a Delphi panel or a previous BIA for medications with similar indication, if the anticipated drug costs for the new pharmaceutical are not very different.

Changes in prescription restriction and guidelines

Possible changes in prescription restriction and guidelines may also lead to different drug utilisation, which requires two estimates: (1) probability of change in guidelines, (2) likely reduction in drug utilisation following from that regulatory step. As discussed earlier, the pool of patients with previous treatment failures is difficult to obtain from real data. However, recent BIA for medications that have been reviewed for reimbursement may provide useful information. An example might be a recent BIA report from the Netherlands for a potential new medication for patients with similar indications to cinacalcet (for secondary hyperparathyroidism in renal failure) [13]. The temporary pool of patients consists of patients previously failing on cinacalcet because of lack of response or adverse events. This report for cinacalcet mentions that only 50% of patients starting cinacalcet will continue treatment. Hence, based on the patient population of 1,040 patients, there is a pool of 520 patients that might be eligible for the new medication. If such information is not available, a Delphi panel can also be used to estimate this figure. It should be noted that not all of these patients will actually switch to the new treatment. These figures also have to be derived from a Delphi panel.

Distribution of existing mix of available treatment modalities at time of launch

Relevant comparators may include other drugs, other kinds of medical care such as surgery or watchful waiting, or no treatment. Because of the time horizon of a BIA, the future distribution of the available treatment modalities is also required, which may also include the inclusion of new treatment modalities at the phase III clinical trial stage that are highly likely to be registered after launch. Because of the time horizon of the BIA, new treatment modalities in earlier stages of clinical development are probably not relevant. It is often difficult to explain the distribution of existing treatment modalities from a clinical point of view. The treatment decision often is rather subjective, sometimes also depending on non-clinical criteria, e.g. patient preferences or financial grounds. For example, reimbursement of infliximab is based on the regulation of expensive inpatient drugs in the Netherlands: 80% of the drug is reimbursed, with the other 20% being the responsibility of the hospital. Hence, there could be a financial incentive to prescribe etanercept or adalimumab, because those biologicals are fully reimbursed. Another non-clinical determinant of treatment selection is that some patients participate in clinical studies (cohort studies) and the study protocol determines the treatment patterns.

These data may be derived from the published literature—ideally country-specific literature, e.g. a previous BIA for medications with similar indication, which allows the construction of statistical distributions for the standard sensitivity analysis. National databases that contain prescription data may also provide the distribution of existing treatments. However, a limitation of most prescription databases is that they provide only utilisation data for all drugs, but without the option of categorisation according to different indications as mentioned before. In addition, the mix may also include non-medication, which is not included in prescription databases. As a result, the use of a Delphi panel might also be required to elicit an estimate of the distribution of existing treatments.

Dosing

Dosing may be based on labelling information. However, dosing in daily practise may differ from dosing in labelling information. For example, a Dutch study by van den Bemt showed that the mean ratio for etanercept was 0.92, implicating that etanercept was, on average, used slightly less then once weekly [14].

For existing treatment, actual dosing in daily practise may be preferred instead of labelling information, if published data is available. Dosing for the new medication can be based only on labelling information at time of launch.

Future distribution of available treatment modalities

This input variable is also a prediction (or forecast) instead of an actual figure and therefore no real data is available for this input variable. Historical data cannot be applied for treatment modalities, which was an option for forecast on annual growth, although previous BIA for medications with similar indication may provide relevant information. As a result, also in this case the use of a Delphi panel is required to elicit an estimate of the distribution of existing treatments or to validate estimates from another BIA.

Treatment sequencing

If the indication for the new pharmaceutical is a first-line treatment, patients with treatment failure may switch to a second-line treatment that is currently also used as a first-line treatment. When similar drugs are used as first-, second- or third-line treatments, relevant data from daily practice will be very hard to find. As mentioned above, databases that contain prescription data may not categorise drugs according to indication, and also may not register the treatment sequence. Checking whether a reimbursement database is able to deliver that kind of data is always advisable. Nevertheless, in most cases the use of a Delphi panel is still required for obtaining this type of information.

Diffusion (or uptake) curves over the follow-up period in the BIA

Diffusion (or uptake) curves reflect the annual proportion of patients switching from each treatment modality of the treatment mix to the new pharmaceutical. In the financial analysis, diffusion generally increases from time of launch over the follow-up period. This input variable is also a prediction (or forecast) instead of an actual figure and therefore no real data should be available. Ideally, the uptake curve for each separate treatment option should be estimated instead of a general uptake from the total treatment mix. One reason for this might be that deviations in uptake for expensive, frequently applied, treatment options will have a much higher impact than a similar uptake from cheap but rare treatment options. Historical data on diffusion curves from recent BIA analyses for a medication with a similar indication may be considered. However, if the new medication differs substantially in clinical properties or price, these historical uptake curves may not be applicable, but instead may be used as starting point for a Delphi panel. In all cases, the anticipated uptake for the new medication in relation to previous medications is, of course, not available in existing BIAs. In particular, if the uptake relates to an expensive treatment requiring hospitalisation, the impact on the total budget might be substantial, making this a highly sensitive input factor.

Substitution effects

The switch from an existing treatment modality to the new pharmaceutical may also lead to other substitution effects, for example a change in dosage and/or type of co-medication, or a change in other resource utilisation (consultations, hospitalisation). For example, a switch from adalimumab or etanercept to infliximab in RA treatment would lead to additional hospitalisation costs because of the need for inpatient intravenous administration of infliximab. Substitution effects might have an impact on the drug budget, but more usually on the total health budget. One economic study showed that the total annual drug costs per patient do not differ substantially between etanercept and infliximab, with costs of € 15,012 and € 14,921, respectively [15]. However, other medical costs (i.e. excluding the costs of the two drugs themselves) are substantially higher for infliximab, which is due to the additional costs associated with day-care and use of methotrexate (MTX; € 6,010).

These effects might be derived from differences in the treatment patterns associated with the previous and the new medication. For example, a switch from adalimumab or etanercept to infliximab in rheumatoid arthritis therapy would lead to additional hospitalisation costs caused by the need for inpatient intravenous administration of infliximab. A difference in clinical properties of the new medication (safety, warnings, contra-indications and route of administration) compared with existing treatment options might be followed by different resource utilisation, which could be evaluated by real practice data or alternatively as a starting point for a Delphi panel.

Off-label

Off-label use means that the drug is used outside its registered indication, usually in subpopulation(s) where officially registered drugs are not efficacious or lead to intolerable side effects. Off-label use can have substantial budgetary implications. The risk of off-label use is especially high when registration trials are running for the same drug in other indications. Based on preliminary findings, physicians may already start to prescribe the drug before its official registration, especially when the new indication is related to the currently registered indication. For example, biologicals were initially approved for RA, but early clinical trial data already showed clinical evidence for biologicals in related rheumatoid disorders like ankylosing spondylitis. Therefore, health authorities often proactively incorporate off-label use in BIAs. It is important to note that restricted use and the risk of off-label use are not mutually exclusive. For example, a new innovative drug may be limited within the registered indication, but the risk of off-label use still exists. An example is the prescription restriction of biologicals to a severe subpopulation within the RA indication (failure to two DMARDs instead of one DMARD). On the other hand, there was also a risk of off-label use in related diseases like ankylosing spondylitis. The financial analysis of off-label use usually cannot rely solely on an existing cost-effectiveness study for the new pharmaceutical, which focusses only on the registered indication. Therefore, separate submodels for off-label indications need to be developed. The use of a drug that is reimbursed only with restricted use, within its registered indication, but outside the restricted indication, is a special form of off-label use. Off-label data is also a prediction (or forecast) instead of an actual figure and therefore no real data is available. These data may be derived from a previous BIA for medications with a similar indication, with a Delphi panel as the last remaining option in the absence of any alternative.

Discussion

The objective of this paper was to address the importance of dealing systematically and comprehensively with uncertainty in a BIA in more detail. The handling of uncertainty in health economics was used as a point of reference for addressing the uncertainty in a BIA.

The overview showed that standard methods of sensitivity analysis, which are used for standard data set in a cost-effectiveness model (clinical probabilities, treatment patterns, resource utilisation and prices/tariffs), cannot always be used as input data for the BIA model beyond the cost-effectiveness data set for various reasons:

The data set in a cost-effectiveness model comes mainly from actual data sources: clinical trials, literature (e.g. meta-analysis), databases, medical records, and/or official tariffs, which allows the construction of a statistical distribution of an input variable. This distribution can be used for 1-way sensitivity analysis (using a confidence interval in addition) or a PSA. In some cases, a Delphi panel will be required to obtain missing data, which usually relates to health care utilisation. A statistical distribution may also be constructed for data from a Delphi panel, although this uncertainty captures only the variance in the estimates, not the uncertainty of a Delphi panel compared with actual data. It was shown that the availability of appropriate data sources for BIA data will often be limited for the part of the BIA model that does not capture the health economic component. Data on prevalence and incidence may often be derived from the literature, but most other BIA data can be derived only from a Delphi panel. Whereas in a cost-effectiveness model, only limited data may come from a Delphi panel, a BIA model often relies on a Delphi panel for the majority of data. Although a statistical distribution may also be constructed for such BIA, the impact of the uncertainty of a Delphi panel might be much larger compared with actual data.

In addition, the data set in a cost-effectiveness model relates only to existing data, whereas the data in a BIA model also includes forecasts, e.g. annual growth, uptakes curves, substitution effects, changes in prescription restrictions and guidelines, future distribution of the available treatment modalities, and off-label use. In some cases, mathematical methods of extrapolation may be used, for example using data on historical trends, which could help in the estimation of distributional properties. Nevertheless, this variance captures only the statistical uncertainty, but not the uncertainty in the assumptions for the forecast. In the majority of cases the use of a Delphi panel seems to be the only option for obtaining this type of information.

Summarising, one conclusion could be that given the estimation of statistical distributions for all input variables of a BIA analysis, it is possible to capture the uncertainty in the variance, but the impact of the level in uncertainty could be much larger than in a health economic data set of the same BIA model. As a consequence it might be questioned if a sensitivity analysis on the BIA data set sufficiently captures uncertainty in the outcome of the model.

The cost-effectiveness model calculates costs at the patient level, whereas the BIA calculates costs at a population level. Therefore, separate sensitivity analyses may be performed on the cost-effectiveness data set and the results may be reported for costs at patient level. This analysis does not depend on the additional BIA dataset, and therefore standard sensitivity analyses can be applied. A PSA can also be performed, exploring the probability of whether the cost difference at patient level exceeds a predefined threshold. As decision-makers for a BIA analysis might differ from the target audience for a CEA, the relevance of a PSA might be questioned. Cost-effectiveness in most jurisdictions might be a criterion at a more central level, whereas most decision-makers are at regional level.

Decision-makers with budget responsibility often are not familiar with the specific terminology from health economics, for example PSA and cost-effectiveness acceptability curves. The use of PSA for calculating budget-acceptability curves might provide relevant information regarding outcome uncertainty, but may not be relevant for the target audience. In addition, their set of decision criteria might be different and their handling of uncertainty may rely on methods other than the sensitivity analyses used in health economics. Therefore, different scenario analyses could be a more useful approach for this type of decision-maker. While a sensitivity analysis is based on the modification of clinical and economic estimates of input variables over a plausible range of values, a scenario analysis is based on the modification of the underlying therapeutic strategies of the model: e.g. society perspective, life-time horizon.

The use of standard sensitivity analyses for the BIA dataset might also be limited because of the lack of appropriate distributions as data sources or the need for forecasting. Again, scenario analyses might be more appropriate to capture the uncertainty in the BIA dataset of the overall BIA model. But, having said that, sensitivity analysis for the BIA set may not be useful; one-way sensitivity analyses may still be informative, i.e. testing for sensitivity to input variables, e.g. by using exploring extreme values, ±25% or a 1-way sensitivity analysis based on minimum and maximum values of the distribution.

Another option to estimate the level of uncertainty would be to use point-sensitivity analysis. The advantage of this approach would be that this type of sensitivity analysis provides information on the sensitivity of the model to the input variable regardless of the range. This could be especially appropriate for the specific BIA dataset, as particularly the range is uncertain. To take account of the fixed input value, sensitivity is best calculated by comparing the percentage change in input value to the percentage change in outcome value [16]. Therefore, this equation can provide a measure of the responsiveness of one item in relation to the other. In the end, point-sensitivity shows how critical a variable really is for the model and allows a comparison between all variables that is not confounded by the range of each variable.

Summarising, because BIA is used for reimbursement issues by authorities or third-party payers, it is vital that these analyses are performed according to generally accepted and standardised methods. This increases the transparency of any financial analysis, allowing for a more rational interpretation of the BIA results and a better comparison of different analyses for various new pharmaceuticals.