A methodological review of national and transnational pharmaceutical budget impact analysis guidelines for new drug submissions.

INTRODUCTION: Budget impact analysis (BIA) in health care, sometimes referred to as resource impact, is the financial change in the use of health resources associated with adding a new drug to a formulary or the adoption of a new health technology. Several national and transnational organizations worldwide have updated their BIA guidelines in the past 4 years. The aim of the present review was to provide a comprehensive list of the key recommendations of BIA guidelines from different countries that may be of interest for those who wish to build or to update BIA guidelines.
METHODS: National and transnational BIA guidelines were searched in databases including MEDLINE, EMBASE, Cochrane, EconLit, CINAHL, Business Source Premier, HealthSTAR, and the gray literature including regulatory agency websites. Data were reviewed and abstracted based on key elements in a standard BIA model (analytical model structure, input and data sources, and reporting format).
RESULTS: Eight national (Australia, UK, Belgium, Ireland, France, Poland, Brazil, and Canada) and one transnational (International Society for Pharmacoeconomics and Outcomes Research) BIA guidelines were included in this review, and a comprehensive list of BIA recommendations was identified. The review showed that certain recommendations such as patient population assessment, drug-related direct costs, discounting, and disaggregated results were common across the various jurisdictions. BIA guidelines differed from each other in terms of the number and scope of recommendations, the terminology used (eg, the definition of comparators or cost offsets) and the direction of the recommendations (ie, to include or not to include with respect to such items as off-label indications, indirect costs, clinical outcomes, and resource utilization).
CONCLUSION: While there was a common purpose for all of the BIA guidelines that were identified, substantial differences did occur in the specific recommendations. The pharmaceutical financing system structure might explain why guidelines from the UK, Australia, and Canada have more country-specific recommendations. The desire to be consistent with adopted economic evaluation assumptions might be another reason for some observed differences between countries. Further research is required to assess the source of the heterogeneity between BIA recommendations are identified in different guidelines.

Introduction

The first budget impact analysis (BIA) analytic framework was published by Mauskopf1 in 1998. In 2001, Trueman et al2 provided essential suggestions for conducting a BIA, and the Polish BIA guidelines in 20043 followed the initial framework of BIA proposed by Trueman et al.2 In 2005 in Canada, the Patented Medicine Prices Review Board initiated the development of the Canadian BIA guidelines which were subsequently published in 2007.4 The International Society For Pharmacoeconomics and Outcomes Research (ISPOR) task force published the first transnational guidelines for the execution of a BIA in 2007,5 followed by Germany6 and France7 in 2008.

During the past decade, many jurisdictions around the world have updated their BIA guidelines, including the Ire-land (2018),8 France (2018),9 UK (2017),10 Australia (2016),11 Poland (2016),12 and Belgium (2015).13 ISPOR published their second task force report on good practices for conducting BIA in 2014.14 In Asia (ie, Iran,15–17 Thailand18) and Latin America (ie, Brazil,19 Chile, Colombia, Cuba, and Mexico), there have been initiatives regarding drug reimbursement decision making based on standard economic evaluation and BIA guidelines. Brazil has published their BIA guidelines in 2012, and Chile, Colombia and Mexico require BIA as part of their Health Technology Assessment (HTA) process.20

A number of systematic reviews of BIA empirical studies have recently been published,21–25 and literature reviews of national and transnational BIA guidelines have been conducted as part of national BIA guidelines development (eg, France [2018],9 Belgium [2015],13 and Canada [2008]26). However, the Belgian and the Canadian guidelines did not systematically review the BIA literature. In contrast, the French BIA guidelines provides a comprehensive review of the BIA literature, including 9 national BIA guidelines, 5 recommendations of good practices developed by national and international societies for health economics, and 14 methodological publications on existing BIAs, published between 2000 and 2016.9 Nevertheless, the French review did not provide sufficient details regarding the individual guidelines reviewed and cannot be used as a foundation for constructing a new set of BIA guidelines or updating existing versions. To illustrate, the results were briefly listed in a table in an aggregated form rather than providing a complete detailed list of the BIA recommendations. The present study has been designed to identify and abstract all guideline recommendations relating to three key aspects in designing a standard pharmaceutical BIA (analytical model structure, input data and sources, and reporting format). This paper presents a comparative review of the BIA key element recommendations that are discussed in national and transnational BIA guidelines and, also, provides a list of the relevant components that are needed in order to conduct a comprehensive pharmaceutical BIA.

Methods

Data sources

A systematic search of the literature was undertaken to identify BIA guidelines published from 1998 to June 30, 2018. The following bibliographic databases were searched through the Ovid interface: MEDLINE, EMBASE, Cochrane, Econ-Lit, CINAHL, Business Source Premier, and HealthSTAR. We also searched the gray literature (Supplementary material S1) including International Network of Agencies for Health Technology Assessment (INAHTA) and non-INAHTA members (eg, National Institute for Health and Care Excellence, Pharmaceutical Management Agency as well as EUnetHTA, Health Technology Assessment International, International Health Economics Association, and International Society for Pharmacoeconomics and Outcomes Research). The search strategy included a combination of text words and Medical Subject Headings terms and synonyms of budget/financial analysis, guidelines, and methodology/modeling. The keywords used for the searches are shown in Supplementary material S1.

Inclusion and exclusion criteria

The inclusion criteria were limited to BIA guidelines published since 1998 by different countries or international organizations (eg, ISPOR) that presented recommendations on all three key elements of designing a BIA (ie, analytical model structure, input and data sources, and reporting format).14 The titles and abstracts identified in these searches were screened to find eligible published national and transnational BIA guidelines (peer-reviewed or online multimedia). When a country or transnational BIA guideline was updated, we only included the latest updated version of the BIA guidelines for each organization in order to avoid duplication in data abstraction.

Citations that reported BIA for any specific drug or medical device (empirical studies), or review articles of empirical BIAs, abstracts, and conference proceedings and methodological publications other than guidelines for conducting a pharmaceutical BIA were excluded. National guidelines were excluded if they did not explicitly discuss the key elements of a BIA model or if they did not add any additional information beyond the guideline that had been adopted from, and where the latter was already included in the review.

Study selection, data abstraction, and synthesis

Titles and abstracts of all articles were screened (level 1 screening) for inclusion by one reviewer. Following level 1 screening, the full text of the selected articles was retrieved (level 2 screening) and assessed by two independent reviewers for eligibility for final inclusion. The disagreement was resolved through consensus and, if persistent, arbitrated through discussion with a third person.

Using a data abstraction template, all included guidelines were reviewed by two independent reviewers to abstract key elements which were discussed in each BIA guideline. An Excel-based data abstraction form was developed based on the predetermined BIA key elements in accordance with ISPOR BIA guidelines (For sake of simplicity and consistency with other BIA guidelines, in the present review, “ISPOR II Task Force report on BIA Good Practice” was abbreviated to “ISPOR BIA guidelines”).14 All the listed recommendations were for a base-case BIA model. The Excel-based data abstraction form was initially tested using two (Irish and Belgian) BIA guidelines before being used to abstract the data/recommendations from all the included BIA guidelines.

For the purpose of this paper, the BIA key elements were categorized into three groups: analytic model structure, input and data sources, and the reporting format. In each category, we defined primary and secondary elements. The primary elements were the main components within each category (eg, perspective, time horizon, target population, scenarios to compare, costing, modeling, and uncertainty), and secondary elements were more specific and detailed considerations related to the primary elements (eg, off-label use, the degree of implementation, and scenario analysis). The analytic model structure contains a discussion of twelve primary BIA elements (eg, model design, model validation, perspective, time horizon, target population, costing, comparators, discounting and inflation, and handling the uncertainty). The data input category mainly addresses data sources for market-share estimation and epidemiologic analyses. The reporting format section describes details for reporting BIA results based on the payer’s requirements and the standard practices in conducting and reporting BIAs (eg, aggregated and disaggregated results in each year of the time horizon and outcomes are presented in natural and monetary units). All terminologies, categories, and BIA key elements were defined in accordance with ISPOR BIA guidelines.14

Results

Literature search results

A total of 3,804 potential citations were identified through the systematic and the manual searches (having removed duplicates). Fifty-two citations were included after the title and abstract review, of which 43 were excluded for not meeting the eligibility criteria, resulting in a total of 9 national and transnational BIA guidelines published between 1998 and 2018.8–14,19,26 Figure 1 shows the detailed study selection process, and a summary of the included guidelines in the review is shown in Table 1.

Figure 1

PRISMA flow diagram of search results.

Abbreviations: BIA, budget impact analysis; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Table 1

Summary of nine included guidelines in the review

Countries	Financing system	Year	Organization	Title
Ireland8	Publicly funded health and social care system	2018	The Health Information and Quality Authority (the authority)	Guidelines for the Budget Impact Analysis of Health Technologies in Ireland 2018
France9	French statutory social insurance scheme	2018	HAS	The HAS guidelines for conducting BIA
UK10	NHS	2017	NICE	Assessing resource impact process manual: guidelines
Australia11	PBS	2016	PBAC	Guidelines for preparing a submission to the PBAC (version 5.0)
Poland12	National Health Fund (NHF)	2016	The Agency for Health Technology Assessment and Tariff System	HTA guidelines
Belgium13	Federal government, communities, patients	2015	Belgian Health Care Knowledge Centre	Guidelines for BIAs
ISPOR14	NA	2014	ISPOR	ISPOR taskforce report: Budget Impact Analysis – Principles of good practice: Report of the ISPOR 2012 Budget Impact Analysis good practice II task force
Brazil19	Unified Health System	2012	Ministry of Health, National Committee for Health Technology Incorporation	Diretriz para análises de impacto orçamentário de tecnologias em saúde no Brasil (guidelines for budget impact analysis of health technologies in Brazil)
Canada26	Federal, provincial and territorial drug plans, private payers, patients	2007	Patented Medicine Prices Review Board	Guidelines for conducting pharmaceutical budget impact Analyses for submission to public drug plans in Canada

Abbreviations: BIA, budget impact analysis; HAS, French National Authority for Health; HTA, health technology assessment; ISPOR, International Society for Pharmacoeconomics and Outcomes Research; NHS, National Health System; NA, not applicable; NICE, National Institute for Health and Care Excellence; PBAC, Pharmaceutical Benefits Advisory Committee; PBS, Pharmaceutical Benefits Scheme; RIA, resource impact assessment.

Country-specific (national) guidelines from eight countries (Australia, UK, Belgium, Ireland, France, Poland, Brazil, and Canada) were included. The guidelines from five countries were excluded. Germany (2008),6 Thailand (2014),18 and the USA27 each adopted the ISPOR BIA guidelines, while the Wales28 and Scotland29 guidelines were derived from the UK NICE recommendations.10 None of these five countries provided any additional methodological information beyond the source guidelines that they had adopted (which were already included in this review as a primary guideline). A summary of the countries that have developed national BIA guidelines and their associated drug plans is provided in Supplementary material S2.

Guideline recommendations pertaining to the BIA key elements

A comprehensive list of all the BIA guideline recommendations was derived from the nine reviewed guidelines and is presented in Table 2. Figure 2 shows the number of guidelines that have made specific recommendations. The following sections provide a synthesis of the key similarities and differences among the nine guidelines.

Table 2

BIA categories and recommendations of nine national and transnational BIA guidelines

BIA primary elements	BIA secondary elements	ISPOR (2014)	Canada (2007)	Belgium (2012)	France (2018)	Ireland (2018)	Poland (2016)	UK (2017)	Australia (2016)		Brazil (2012)
Perspective
	The recommended perspective is that of the budget holder range from a single payer covering an entire health care system through specific providers	Yes	Yes (federal, provincial, and territorial drug plans, private payers)	Yes (health care payers, patients, provider)	Yes (French statutory social insurance scheme, patient, provider)	Yes (publicly funded health and social care system)	Yes (public payers, patients, hospitals)	Yes (commissioner, provider)	Yes (PBS/RPBS; federal government; NIP)		Yes (public and private systems; nation, states, or municipalities)
Technology
	The technology should be described in sufficient detail to differentiate it from its comparators and to provide context for the study					Yes
Population size and characteristics
	Definition of patient population	Yes (all patients eligible for the new intervention during the time horizon of interest, given any access restrictions)	Yes (defined as individuals insured by drug plans of interest and have the condition of interest)		Yes (target and expected population)	Yes (target population should be defined based on the approved indication for the technology and also defined as those with a specified disease who may avail of the technology being considered in the defined time horizon)	Yes (all patients in whom a given health technology can be used in accordance with the assessed medical indications)	Yes (resident and registered population)	Yes (number of patients will be treated and number of unit doses will be dispensed over the time horizon)		Yes
	Top-down population approach: estimation of the number covered by the locally approved indications for the new technology which needs to reflect uptake, and changes in patterns of use	Yes	Yes (epidemiologic approach)			Yes	Yes	Yes (incidence and prevalence-based approach)	Yes (epidemiologic approach)		Yes (incidence and prevalence-based approach)
	Bottom-up approach: this starts from the number of individuals likely to avail of the technology. It includes the number of individuals that will switch from an existing technology and the number of newly treated patients. These estimates may be informed by existing claim-based data		Yes (claim- based approach)	Yes (define population for all indications of the intervention under study)	Yes (define population for all indications of the intervention under study)	Yes			Yes (market- share approach)		Yes (claim- based approach)
	Open (dynamic) population	Yes		Yes	Yes (population should be described for each year of the BIA, and expected changes in their sizes should be taken into account)	Yes	Yes
	Subgroups	Yes		Yes	Yes	Yes (based on biologically plausible and justified evidence but not based on treatment response)			Yes (stratify by beneficiary)		Yes
	Catch-up effect	Yes				Yes
	Access restrictions	Yes	Yes				Yes				Yes
	Unit of analysis (per patient or episode)					Yes (per patient or per episode of care)			Yes (per unit dispensed)		Yes (per episode)
	Off-label indications in the eligible population may also be included	Yes (for the current treatment mix)	No	Yes	Yes	No					Yes
	Degree of implementation of the new intervention (substitution, combination, and expansion)	Yes		Yes	Yes		Yes		Yes		Yes
Comparators
	Definition	Yes (BIA compares scenarios defined by sets of, rather than specific individual, interventions)	Yes (two scenarios, reference and new drug scenario, should be compared for the treatment strategy)	Yes (current situation that would change if the intervention under consideration is introduced in the health care system; most cost- effective alternatives)	Yes (BIA compares sets of interventions [scenarios] rather than individual interventions)	Yes	Yes (the assumptions concerning the “current scenario” and the “new scenario” should be described and justified in the analysis)		Yes (PBS medicines that will be affected by the proposed listing; mixed treatment comparisons)		Yes (comparison of two or more scenarios, which are representations of different market conditions)
	Current intervention mix for the eligible population	Yes (the current mix may include no intervention as well as interventions that might be replaced by the new one)	Yes (forecast version of the current market without the new drug)	Yes	Yes (scenarios without the intervention under study)	Yes (baseline scenario that reflects the current mix of technologies and forecasts the situation should the new technology not be adopted)	Yes (takes into account the interventions currently used in a given population including no intervention or interventions used in different conditions)				Yes (set of therapeutic options currently available for the treatment of the disease)
	New intervention mix	Yes (the introduction of a new intervention sets in motion various marketplace dynamics, including product substitution and possibly market expansion)	Yes (new drug scenario is forecast version of the current market with introduction of the new drug)	Yes	Yes (scenarios with the intervention under study)	Yes (new technology scenario, where the new drug is adopted)	Yes (reflects the market after the introduction of the new technology)			Yes (cost of each intervention included in the analysis will reflect the cost of the entire therapeutic package associated with that intervention)
Costs and outcomes
	Direct cost consequence of implementing NICE guidelines							Yes
	Cost of the current and new intervention mix is determined by multiplying the budget holder’s price for each intervention by proportion of the eligible population using that intervention and by the number of people in the eligible population	Yes	Yes (treatment strategy-based approach)		Yes	Yes	Yes
	Actual acquisition cost of the intervention for the budget holder includes any discounts, rebates, or other adjustments that may apply	Yes	Yes			Yes
	Opportunity costs are the costs that arise when implementing the technology or clinical guidelines that might not being reflected in the “actual costs” at the time of doing BIA analysis	Yes				Yes	Yes (the cost of additional outlays in the health care system, related to the implementation of the assessed technology)
	The costs included should be limited to direct costs associated with the technology that will accrue to the relevant payer(s)	Yes	Yes (direct drug cost)			Yes (drug administration costs, the cost of drug wastage and the cost of drug monitoring)	Yes (actual payments and actual savings achieved by a public payer/patient; taking into account the existing risk sharing schemes)	Yes	Yes (direct drug cost)	Yes (costs of the new drug and those directly associated with its use, as adjuvant medications or treatment of adverse events)
	Cost of clinical outcomes and disease complication	Yes	No	No (health outcomes are not included; however cost consequences of health outcome, eg, treatment cost of adverse events, are included)		Yes (efficacy, effectiveness, and safety, cost offsets)		Yes (direct clinical consequences)
	Cost of health care utilization (eg, hospital days or physician visits)	Yes	No	Yes (eg, cost of treatment of adverse drug reactions)		Yes (cost offsets)	Yes
	Indirect costs: the impact of the new intervention on productivity, social services, and other costs outside the health care system	No (should not be included routinely in a BIA [except for the private payers or employers])	No	Maybe (can be quantified in a separate analysis)		No		Yes (eg, productivity cost)		No
	Cost of supplies: the analytic framework should allow for cost-relevant details of how accompanying devices for the proposed medication are used	Yes				Yes	Yes			Yes
	The annual depreciation of any capital costs should be included in the analysis					Yes
	Labor costs					Yes	Yes (eg, staff training cost)
	Value-added tax					Yes
	Proposed drug cost based on unit drug price and average dose for average duration of time	Yes	Yes			Yes (technology cost)	Yes	Yes	Yes (changes in the number of units dispensed and costs over the time horizon)	Yes (per patient, per time period)
	The BIA should also estimate the impact of adherence or persistence on intervention effectiveness and safety if condition-related costs are included in the BIA	Yes
	Calculate both the global budget impact and separately the budget impact for the different health care payers (this implies that potential transfers of budgets between different levels of governments and/or patients)			Yes					Yes
	Application of the therapeutic equivalence method in the comparison of costs is recommended									Yes
Time horizon
	BIAs should be presented for the time horizons of relevance to the budget holder	1–5 years	3 years	3 years	3–5 years	5 years	2 years	5 years	6 years	1–5 years (subjected to budget holder’s needs)
Modeling
	Modeling may be needed to calculate the budget impact for bringing together the best available data from different sources	Yes (if an economic evaluation was performed, the BIA model should be consistent with the clinical and economic assumptions in EE)		Yes	Yes (according to the characteristics and the management of the disease of interest in France)	Yes (based on the good modeling practice)		Yes	Yes	Yes
	Assumptions should be the same as EE	Yes (the justified comparator in an economic evaluation may be different from the comparator in the BIA)		Yes		Yes		Yes	Yes
	The computing framework for a BIA can be a simple cost calculator programmed in an Excel-based spreadsheet	Yes	Yes	Yes	Maybe (transparent and accessible to the decision maker)			Yes (a resource impact template is an Excel spreadsheet)	Yes	Yes
	More complicated software	Yes		Yes (decision tree, Markov model)		No (simplest design)				Yes (decision tree, Markov models, discrete event simulation)
Handling uncertainty and scenario analyses
	Sensitivity analysis: parameter uncertainty in the input values	Yes	Yes		Yes	Yes	Yes			Yes
	One-way and/or multi-way sensitivity analysis, analysis of extremes	Yes	Yes		Yes	Yes	Yes
	PSA is recommended in BIA		No	Yes		Yes
	Scenario analysis: structural uncertainty introduced by the assumptions made in framing the BIA	Yes		Yes	Yes	Yes			Yes
	Important parameters to be assessed in the sensitivity and scenario analyses have been provided in the guidelines	Yes	Yes			Yes	Yes (population size [eg, the degree of possible abuse], costs of use and reimbursement conditions)		Yes	Yes
	Describe the direction and magnitude of the impact of uncertainty on the overall estimates								Yes
Discount rate
	An attempt should be made for forecasting changes in the value of the currency used the BIA over the time horizon	Yes	Yes			Yes (in certain circumstance)				Yes (in certain circumstance)
	Discounting is generally not required	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Validation
	The computing framework and input data used for a BIA must be sufficiently valid to credibly inform the budget holder’s decisions	Yes	Yes	Yes (face validity)	Yes	Yes			Yes (the template workbook enables the PBAC to validate the presented estimates)	Yes
	The process of the validation is required		No			Yes (should be documented)
	Value of the information analyses (the cost of extra data collection vs improved model precision)		Yes
	The programming created by the developer of the budget impact model to perform the analysis (source code) should be made available for review (on the condition that property rights are respected)		Yes		Yes					Yes
	Model code should be provided to reviewers		Yes
	Postmarket reassessment: the observed costs in a health plan with the current interventions should be compared with the initial-year estimates from a BIA	Yes	Yes					Yes	Yes
	Quality assurance and publication							Yes (for all resource impact products)	Yes (quality use of medicines)
Inputs and data sources
	Recommended data sources	Yes	Yes		Yes	Yes		Yes (quite comprehensive list)		Yes
	Search strategy; inclusion criteria for data selection and source selection; strengths and weaknesses of the used sources, and methods of analysis should be presented						Yes
	Use data from another jurisdiction where the intervention has been introduced	Yes	Yes			Maybe (might not be realistic in Ireland)				Yes (health systems comparable to the Brazilian system)
	Use estimates of expected market share from the producer	Yes
	Extrapolate from experience on product diffusion with similar interventions in the budget holder’s setting	Yes	Yes
	Data could be sourced from clinical trials	Yes	Yes			Yes
	Unpublished data sources, such as expert panels	Yes	Yes			Yes	Yes (taking into account the existing risk sharing schemes)			Yes
	Original cost survey, obtaining primary data, by sampling, involving interviews with health professionals under study									Yes
Presenting results
	The estimated annual total and incremental budget impacts should be reported separately for each year of the time frame	Maybe (a table should show the total and disaggregated costs for each time period reported in the BIA)	Only incremental impact		Yes	Yes	Yes		Yes
	Results should be reported in terms of their natural units and financial cost					Yes
	Introduction, study design and methods, results, conclusions, and limitations	Yes (very detailed)	Yes (not described in detail)			Yes (not described in detail)
	All results should be presented in their disaggregated and aggregated forms for each year of the timeframe	Yes (results should be presented in a disaggregated manner)	Yes (results should be presented in a disaggregated manner)	Yes (results should be presented in a disaggregated manner)	Yes	Yes	Yes (results should be presented in a disaggregated manner)	Yes (results should be presented in a disaggregated manner)	Yes (according to the PBS and the RPBS, and for beneficiary type)
	Inclusion of graphics and figure of the analytical framework, schematic representation of uncertainty analyses	Yes
	Table of assumptions, tables of inputs and outputs, appendices, and references	Yes					Yes			Yes
	The addition of relevant appendices to the main report is encouraged. The appendices may cover literature search strategies, evidence summaries, intermediate results (eg, of individual Delphi panel rounds, and the names and addresses of participating experts and investigators)	Yes								Yes
	Resource impact products: resource planner; resource impact reports and templates; resource impact statement							Yes

Abbreviations: BIA, budget impact analysis; EE, economic evaluation; ISPOR, International Society for Pharmacoeconomics and Outcomes Research; NICE, National Institute for Health and Care Excellence; NIP, National Immunization Program; PBAC, Pharmaceutical Benefits Advisory Committee; PBS, Pharmaceutical Benefit Scheme; PSA, probabilistic sensitivity analysis; RPBS, Repatriation Schedule of Pharmaceutical Benefits.

Figure 2

A schematic list of BIA recommendations in the reviewed guidelines.

Note: The positive and negative recommendations are illustrated in different colors.

Abbreviations: BIA, budget impact analysis; EE, economic evaluation; NICE, National Institute for Health and Care Excellence.

Analytical model structure

Perspective

In most BIAs, using the perspective of the primary health care budget holder is recommended. However, in the French,9 Polish,12 and Canadian26 BIA guidelines there is a recommendation to use the patient’s perspective as complementary analysis to the base-case analysis. In contrast, Australia11 explicitly requires the exclusion of any copayment from any other source beyond the identified budget.

Time horizon

It is recommended in the Polish12 and Belgian13 guidelines to present the budget impact up to the steady state, with a minimum time horizon of 2–3 years. The minimum time horizon in the Canadian BIA guidelines26 is 3 years, whereas in the updated NICE10 and Australian11 guidelines a longer time duration is recommended (6 and 5 years, respectively). France9 and ISPOR14 recommend a BIA time horizon varying from 3–5 and 1–5 years in the base-case analysis, respectively. The Brazilian guidelines have also taken a time horizon from 1–5 years.19 The base-case analysis should estimate the annual financial impact over a minimum timeframe of 5 years in the recently updated Irish guidelines.8 A comparison of the time horizon recommended in different guidelines is shown in Figure 3.

Figure 3

Time horizon recommended by nine reviewed guidelines.

Note: A range of time horizon is illustrated (in different color) for the guidelines/countries, if applicable.

Abbreviation: ISPOR, International Society for Pharmacoeconomics and Outcomes Research.

Target population

Some guidelines have defined the target population as the “entire population of patients affected by the assessed indications, targeted by the proposed medicine, over a specified time horizon.8,12,14 French guidelines have introduced two population groups to be included in the analysis, “the target population and the expected treated (forecasted population to be actually treated by the intervention in the real-life practice) population for all indications.”9 Based on the Canadian BIA guidelines, the target population is defined as “all drug plan beneficiaries who are expected to be diagnosed and treated for the conditions of interest and are eligible to use the new drug.”26

Subpopulation analyses can be performed for BIA if there are appropriate justifications: by beneficiary, differences in safety, treatment effect, baseline risks, costs, or market share.8,9,11,13,14,19 For the target population estimation, there are two approaches: top-down or epidemiological and bottom-up or market-share (claim-based analyses). An epidemiological approach is usually preferred if the submission indicates a superior therapeutic conclusion in clinical studies, whereas a market-share approach might be preferred if the submission indicates a noninferior therapeutic conclusion.11 In the epidemiological approach, disease severity shifts, incidence, and prevalence are required, and it is usually inevitable to use data from different sources.26 Apart from the UK,10 Poland,12 and ISPOR14 (which only ask for the epidemiologic approach), other guidelines recommend BIA results obtained from both epidemiologic and market-share approaches for all new drug submissions.

The degree of implementation (full replacement or partial substitution of existing technologies or shifts in the target population, market growth, or expansion) is essential in both approaches and recommended by most guidelines. In the Canadian guidelines, it is advised that the treatment displacement assumptions regarding the changes to the market share of each competitor after the introduction of the new drug be tested in the sensitivity analysis.26 The population is dynamic in the Irish, Polish, Belgian, ISPOR and Brazilian guidelines, meaning that patients could be added to or removed from the analysis based on whether they meet the inclusion criteria or not over time.8,12–14,19 In some cases, when the technology applies to a well-defined group of patients, the BIA may require a defined closed population.12

In addition, the French, Belgian, ISPOR (for the current treatment mix) and Brazilian BIA guidelines9,13,14,19 recommend consideration of off-label usage in all indications for the assessed medicine as complementary to the base- case analysis; this is especially relevant if there is available evidence for cost-effectiveness and, more importantly, it is noted by the payer.9 In the Canadian BIA guidelines, the off-label use is only considered in the sensitivity analysis.26 The catch-up effect which applies to the chronic conditions for patients who switch to the new drug is recommended in the Irish and ISPOR guidelines.8,14 Any planned local regulations and legislations which would limit new drug access in a subpopulation should be considered.12,14,19,26

Scenarios to compare (comparators)

In most of the reviewed guidelines, the current scenario/practice (including “no intervention”) should be “routine care” or the best clinical practice, including the most cost-effective alternatives. The new scenario is the “current scenario” with the new intervention added to or replacing the current interventions entirely or partially.13,14 NICE considers a broader picture of budget impact and defines the current and new scenarios as current and future clinical practice activities (at activity levels) resulting from adopting the NICE guidelines in the NHS.10 In Canada, the comparator definition is more market-oriented. According to the Canadian BIA guidelines, reference scenario is the current market-share distribution of all comparators without new drug, whereas new drug scenario is forecast market share of same comparators with the inclusion of the new drug.26 Multidrug treatment (ie, treatment mix or set,14 treatment set,9 treatment mix,11 and strategy-based treatment26) rather than individual interventions is recommended in most of the guidelines.8,9,11,12,14,19,26

Cost analysis

Ireland, France, Australia, Poland, ISPOR, Brazil and Canada consider costing based on multi-drug treatment strategy (including adjunct therapies).8,9,11,12,14,19,26 The BIA should, therefore, identify all medicines likely to be affected by the new drug.

Most of the guidelines agree on the fact that direct health care-related costs for the most relevant perspective should be included in the base-case, similar to the guidelines for economic evaluations.8–10,12–14,19 However, the Australian11 and Canadian26 BIA guidelines exclude the costs associated with changes in outcomes, costs associated with clinical consequences/complications (eg, adverse drug reactions), and resource utilization (eg, hospitalization, emergency room admission), while other guidelines suggest to review such nondrug related costs. In the latest version of the Irish guidelines, for pharmaceuticals, direct costs include the cost of the drug and any other drug-related costs (concomitant therapies, adverse events, and infusion-related costs such as consumables and staffing).8 The impact on indirect, non-health care-related costs (eg, productivity, transport, capacity, and workforce) are not usually included in a BIA base-case analysis, except for the NICE guidelines (Table 2).8,9,13,14

Other differences between BIA guidelines were related to the scope of costs (eg, costs related to personnel training, budget transfers between different governments and patients).8,9,13,14 According to the Irish, Polish, and ISPOR guidelines, it is important to consider additional resources that must be taken from the existing services when implementing a new technology, which are called “opportunity costs.” Opportunity costs are the costs that arise when implementing the technology or clinical guidelines that might not being reflected in the “actual costs” at the time of doing BIA analysis.8,12,14 In the case of including condition-related costs (ie, health outcomes and resource use), the actual opportunity costs are relevant in the ISPOR guidelines. In such cases analysts may use cost accounting approaches if actual opportunity costs are not available for a particular jurisdiction.14 According to the Irish guidelines “actual costs” are cash payments which occur from implementing the technology or clinical practice guidelines.8 The BIA should clearly state which unit of analysis is adopted in measuring the outcomes. There are two possible units of analysis: per patient or episode of care. Specified interventions may range from once-daily, repeated, periodic, or continuous interventions; it needs to be clear the number of times or the length of time people might experience the intervention or how many treatment events might arise.8,11,19

Cost of the treatment should be adjusted to consider markups, discounts, inventory allowance,8,14,26 business-related costs to the pharmacy covered by the drug plans, and dispensing fees and patient copayments, as requested by drug plans in Canada.26 In the Canadian BIA guidelines, drug prices can be obtained from provincial formulary websites, public drug plan databases, and manufacturers’ market access department for preparing BIA reports.26 There are also recommendations on how to deal with New Chemical Entities and generic drug prices for BIAs in the Canadian BIA guidelines.26 In Australia, Pharmaceutical Benefits Advisory Committee (PBAC) also recommends “dispensed price for maximum amount” for BIA.11 It is recommended that uncertainties regarding the drug reimbursement price should be targeted through a sensitivity analysis.26

In the Irish guidelines, the value-added tax could be considered if applicable,8 and in the Belgian and Canadian BIA guidelines, protocol-driven costs should be excluded (eg, costs related to the patient enrollment process and additional laboratory tests specific to the clinical trial design).13,26 None of the guidelines recommends inflation and discount rates; however, in the Canadian, Brazilian, Irish, and ISPOR BIA guidelines, they are permitted in the certain circumstances and if there is justification for being included (eg, confirmed information on pricing policy, implementation of an approved new policy rule in the near future, or price changes after patent expiration).

Modeling

Transparency, validity, simple, and user-friendly design along with explicit definitions and assumptions are the most favorable features of a BIA model. It is recommended that the model be designed based on the projected disease condition and be flexible enough to capture long-term outcomes/costs in the chronic diseases.9 Similar to cost-effectiveness analyses, in the Belgian and Brazilian BIA guidelines, decision trees or Markov models can be helpful to be consistent with the economic evaluations.13,19 Most guidelines recommend using an Excel-based model (rather than more complicated software) to calculate the budget impact.9–11,14,19,26 This allows for extending the analysis to the appropriate time horizon and using different data sources. Face, internal, and external validities have to be checked and documented. The model validity and transparency could be assessed using recommendations provided by ISPOR and the Society for Medical Decision Making task force report.30

Handling the uncertainty

Decreasing the uncertainty is an essential consideration in BIA. Although probabilistic sensitivity analysis is not recommended in the Canadian BIA guidelines, one-way, univariate deterministic sensitivity analysis or multivariate scenario analysis are acceptable for the most important variables such as prices, population and market shares.26 Sensitivity analysis of data obtained from clinical trials,11 drug dosage,26 price,26 and market data from other jurisdictions14 are also recommended.8,9,11,12,14,19,26

Scenario analysis is recommended by Ireland, France, Australia, Belgium, and ISPOR.8,9,11,13,14 PBAC11 has provided a very detailed list of recommended scenarios to be considered in reporting the budget impact results, eg, the effects of promotional efforts on prescriber and consumer behavior. Risk sharing agreements with the manufacturers and a more extended introduction phase for the proposed drug have also been recommended by the UK and Australia for managing uncertainty in early BIA results.10,11

Input and data sources

National statistics and registries are recommended sources for epidemiologic data (eg, disease prevalence and incidence).8,9,12,14,19,26 The best sources for the claim-based and market research information are the payer database14 and the manufacturer’s marketing department.14,26 In the Irish, ISPOR, Brazilian and Canadian guidelines, data from foreign markets are acceptable if local information are not available (Table 2).8,14,19,26 The BIA reports from manufacturers with clear supporting data could also be helpful.14,26 Consensus expert opinion is an option when market intelligence for forecasting the new drug market share is not available.8,12,14,19,26

Reporting format

There are specific requirements for reporting the results in the reviewed guidelines. Newly updated guidelines have put more attention to the details and the manner BIA results are reported, mainly based on the policymakers’ interest and requirements.

Total and incremental impact on the primary payer’s budget should be presented in the Polish, Irish, French, and Australian guidelines.8,9,11,12 The Canadian guidelines only require the incremental impact on the annual budget.26 Results should be both aggregated and disaggregated in each year of the time horizon in the Irish, French and Australian guidelines.8,9,11

The budget impact can be presented in natural (eg, number of unpaid working days) and monetary units separately for the different health care payers.8 A table of assumptions, inputs, and outputs, a schematic representation of any uncertainty analyses (eg, Tornado diagram), appendices, and references should be included.9,14,19 Estimated financial implications for the health budget (other health sectors), the impact of uncertainty (quantify how precise are the results), activities to support the quality use of medicines, and postmarketing surveillance amendments are recommended by PBAC.11 In their new resource impact assessment (RIA) manual, NICE classifies results as “substantial” if the implementation of a single recommendation in the UK costs higher than a specific threshold.10

NICE recommends publishing the resource planner, a word file of resource impact reports, resource impact statements, quality assurance and publication, as well as making postpublication amendments. RIA results should be published at the same time as NICE evidence-based guidelines and performed in parallel with economic evaluations.10

Discussion

In the present review, we identified BIA guidelines from Ire-land, France, UK, Australia, Poland, Belgium, ISPOR, Brazil and Canada reviewed and all their recommendations related to the analytical model structure, input and data sources, and reporting format of BIAs.8–14,19,26 It is the first peer-reviewed evidence in the health literature in which a systematic review of national and transnational BIA guidelines was published as robust and comprehensive basis for the future research.

There are some similarities in guidelines recommendations (eg, using drug-related direct costs from the primary payer’s perspective, top-down or bottom-up approaches for population assessment, simple [not complicated] modeling techniques, and deterministic sensitivity analysis as the minimum requirements for a BIA base-case analysis). Differences between guidelines were related to number, scope, and direction (yes/no) of recommendations (eg, inclusion of off-label indications, indirect costs, clinical outcomes, and health care resource utilization; duration of time horizon; dealing with uncertainty [eg, deterministic analysis vs PSA], and reporting format). Moreover, there are differences in the terminologies which are used in different guidelines/countries for defining specific concepts in designing a BIA (eg, multidrug treatment in assessing the comparators, target population definition such as “open population”, or cost offsets).

Some guidelines were closely aligned in their recommendations (eg, French, Australian, Belgian, and ISPOR BIA guidelines), while others had included more country-specific recommendations (eg, Canada, Australia, and the UK). In some guidelines/countries such as ISPOR, UK, Belgium, Ireland, and Australia, if an economic evaluation was performed, the BIA model should be consistent with the clinical and economic assumptions in economic evaluation. In the UK, BIA is called RIA and the estimation of costs and savings is based on direct consequence of implementing NICE guidelines (not just drug comparators).10

The results of our review are similar to the French literature review9 of BIA guidelines in terms of key aspects in designing BIA. However, our review used BIA categories more aligned with the ISPOR BIA guidelines.14 The literature review that was conducted as part of the Belgian guidelines was not published with sufficient detail,13 and the literature review results in the French guidelines were summarized in an aggregated format.9 Thus, there were insufficient details to provide a complete taxonomy of BIA guideline recommendations. A previous Canadian BIA literature review26 included the older versions of the Polish (2004), Australian (2002), and ISPOR (2007) BIA guidelines. Our literature review was different in terms of 1) the review design (systematic), 2) the scope (focused on only BIA guidelines recommendations), 3) inclusion criteria (all BIA guidelines published since 1998, excluding any versions that were replaced by newer updates), and 4) reporting format (applicable details for future research).

The present review is the most recent systematic review of published national and transnational BIA guidelines that have been created or updated since 1998. A potential limitation of this study includes having only one reviewer for the level 1 (title and abstract) screening which we believe that did not contribute to considerable bias. We did not include results from countries that simply adopt BIA guidelines from other jurisdictions (Germany, Thailand, USA, Scotland, and Wales) which might be considered a limitation in that it would underestimate the frequency of use for some recommendations. We also did not include published BIA methodologic papers as we were only interested in reviewing BIA guideline recommendations.

Conclusion

To maintain sustainability in financing the health care systems, it is increasingly important to improve informed pricing and reimbursement decision making at national and transnational levels. Our literature review showed that over last 20 years, countries have become actively interested in comprehensive financial and economic evaluations and have tried to keep their BIA guidelines updated. Through a systematic review of national and transnational BIA guidelines published or updated since 1998 following Mauskopf’s1 publication, we provided a full list (not a summary) of the details for conducting a standard pharmaceutical BIA in accordance with the most up-to-date national and transnational BIA guidelines recommendations. The remaining challenge is how to embrace the heterogeneity of recommendations and terminologies that is evident across different guidelines. Further research is required to analysis each countries’ pharmaceutical financing system in more detail to assess any true relationship between country-specific health care parameters and BIA recommendations. The results of this review can be a starting point for countries who are initiating the development of national standard BIA guidelines based on their pharmaceutical reimbursement requirements. The present review can provide useful practical methodological information for BIA users and producers and provide a contribution to the future research in the field of pharmaceutical BIA.

Supplementary materials

Supplementary material S1

Systematic literature review process

MEDLINE, EMBASE, Cochrane, EconLit, CINAHL, Business Source, Ovid HealthSTAR, and the gray literature including International Network for Agencies for Health Technology Assessment (INAHTA) and non-INAHTA members (eg, NICE, PHARMAC) as well as European network for health technology assessment (EUnetHTA), Health Technology Assessment International (HTAi), iHEA, and International Society for Pharmacoeconomics and Outcomes Research (ISPOR) were searched using a combination of text words and Medical Subject Headings terms and synonyms of budget/financial analysis, guidelines, and methodology/modeling. The keywords used for the searches are as following:

Search strategy

MEDLINE

The gray literature list

Websites of health technology assessment or regulatory agencies

Supplementary material S2

Countries with developed budget impact analysis (BIA) guidelines and the types of drug programs where they are applied.

In Australia, there is a government-run Pharmaceutical Benefits Scheme that subsidizes prescription medication, and there is a copayment for patients at the point of dispensing.1 The BIA guidelines as a part of the Australian guidelines on the preparation of new drug submissions to Pharmaceutical Benefits Advisory Committee (PBAC) (2016) is the first full revision of PBAC guidelines since 2006. After 2010, any recommendation by PBAC that has a financial impact on the Federal government’s budget is reviewed by the cabinet.2 There is a close relationship between the estimated financial impact of a drug on the Australian drug budget and the rate of PBAC positive recommendations for reimbursement.3

Belgium has a Bismarck-type social insurance system (multipayer) in which the insurers, called Sickness Funds, are financed by both employers and employees.4 In Belgium, since 2002, Health Care Knowledge Centre (KCE) under the supervision of the Minister of Public Health and Social Affairs is in charge of conducting studies that support the political decision making on health care and health insurance.5 The Belgian guidelines for economic evaluations now include guidance for a BIA in an updated version (2015). The Belgian official Health Technology Assessment institute, KCE, and Belgian stakeholders from both government and industry contributed to improving their recent national economic evaluations and BIA guideline.5

In Brazil, the Unified Health System provides free universal care for all Brazilians as well as vaccinations and pre-natal care. A highly decentralized system has led to complex patterns of funding and service provision with the Federal, State, and Municipal governments involved. Brazil’s system remains highly privatized with the private sector receiving substantial funds from all levels of government.6 Brazil (Ministry of Health [CONITEC]) has been developing the necessary analytical instruments for the evaluation of new technologies for health. In this context, the development of national recommendations for budget impact studies in the health area became more important. The methodology for the development of budgetary impact studies in the health area was adapted to the Brazilian needs, through several presentation and discussion sessions among the professionals of the institutions involved.7

Canada is an example of a “National Health Insurance” model. Canada’s publicly funded health care system is called “Medicare” in which ten provincial and three territorial health care insurance plans share roles and responsibilities for health care services with the Federal government.8 Drug benefit funding is primarily a composite of provincial/territorial governments and private insurance programs. Federally, the Patented Medicine Prices Review Board sets ex-factory price ceilings for patented medications. Although a BIA had been required to be submitted to most provincial public drug plans in the 1990s, before 2007, there was no standardized method of conducting a BIA in Canada. In 2005, Patented Medicine Prices Review Board initiated the development of the Canadian BIA Guidelines on behalf of the National Prescription Drug Utilization Information System, and this was published in 2007.9

In France, the pharmaceutical reimbursement decision-making process consists of two steps: 1) the technical assessment by French National Authority for Health La Haute Autorité de Santé (HAS) and 2) enlisting the drug with price-fixing by the “health care products pricing committee” of the Ministry of Health (Comité Economique des Produits de Santé [CEPS]).10 Since January 2016, cost-effectiveness analysis and BIAs are required to be submitted by manufacturers to HAS and CEPS for highly specialized medicines with an expected 2-year sales revenue more than €50 million.11 In France, BIA for new drug submissions should be prepared for the French statutory social insurance scheme. HAS updated the French BIA guidelines for new drug submissions in December 2017, however, it is not still clear that how BIA results would be applied in the reimbursement price negotiation process.

The Republic of Ireland has a new NHS which was launched in 2005 and is controlled by the Health Service Executive.12 The Irish “Health Information and Quality Authority” (The Authority) has the responsibility to evaluate the clinical and cost-effectiveness of health technologies, and provides evidence-based reports to the Minister of Health and Health Service Executive and develops guidelines for doing HTA in Ireland. The latest updated version of the Irish BIA guidelines on health technologies was published by The Authority in 2018.13

Health care in Poland is primarily financed by the National Health Fund (Narodowy Fundusz Zdrowia) and state budget or local government budgets. The state budget plays a complementary role to National Health Fund in the system. The primary role of the local governments is to ensure access to the services, mostly by performing ownership functions toward health care institutions. In Poland, the BIA guidelines are a part of the latest updated Health Technology Assessment guidelines which initially issued by the Agency for Health Technology Assessment and Tariff System in 2007 and were updated in 2009 and 2016.14

National Health Service (NHS) in the United Kingdom is an example of a single-payer health care system for a country. In the UK, the NHS institution in England and Wales pays for medicines if NICE provides a favorable recommendation. NICE published their updated guidelines on the resource impact (budget impact) assessment process on May 2017. It is proposed that a cap called “budget impact test”15 of £20 million, in any of the first 3 years, be considered to signal the need for negotiation with manufacturers for special arrangements to better manage the introduction of new technologies recommended by NICE.16 Moreover, NICE has recently proposed a Fast Track technology Appraisal process for the new technologies which fall below an incremental cost-effectiveness ratio of £10,000 per quality adjusted life years. The budget impact test would be removed as a criterion for entry into the Fast Track Technology Appraisal process.16,17

Budget impact/budgetary impact/resource impact/financial impact analysis/assessment/studies

1. “budget impact*”.m_titl.

2. “budgetary impact*”.m_titl.

3. budget impact analy*.mp. [mp = title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms]

4. budgetary impact analy*.mp. [mp = title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms]

5. budget impact stud*.mp. [mp = title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms]

6. financial impact*.mp. [mp = title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms]

7. “economic impact*”.m_titl.

8. “economic analy*”.m_titl.

Review; guidance; guidelines; methods

9. review.mp. [mp = title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms]

10. limit 9 to “review articles.”

11. “Review Literature as Topic”/

12. “review*”.m_titl.

13. “guideline*”.m_titl.

14. limit 13 to abstracts

15. “guidance*”.m_titl.

16. limit 15 to abstracts

17. Methods/

18. “method*”.m_titl.

19. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8

20. 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 or 17 or 18

21. 19 and 20

#H I Ts: 120

Countries	Agencies
Inter/multinational	International Network for Agencies for Health Technology Assessment (INAHTA); Health Technology Assessment International (HTAi); International Society For Pharmacoeconomics and Outcomes Research (ISPOR); WHO Health Evidence Network; European Information Network on New and Changing Health Technologies (EUROSCAN); the University of Birmingham; National Horizon Scanning Centre; European network for health technology assessment (EUnetHTA)
Australia	Department of Health and Aging (https://pbac.pbs.gov.au/)
Austria	Institute of Technology Assessment (ITA); Ludwig Boltzmann Institute for Health Technology Assessment (LBI-HTA)
Belgium	Federal Kenniscentrum voor de Gezendheidszorg (KCE)
Canada	Canadian Agency for Drugs and Technologies in Health (CADTH)Provincial drug plans:•http://www.health.gov.on.ca/en/pro/programs/drugs/drug_submissions/guideline_templates.aspx• https://www.ab.bluecross.ca/dbl/pdfs/bia-form.docx• https://www.gov.mb.ca/health/mdbif/sub.html• http://www.inesss.qc.ca/fileadmin/doc/INESSS/Inscription_medicaments/Fiches_inscription/en/Submission_guidance_document.pdf
Republic of China	National Health Development Research Center (NHDRC); Key Lab of Health Technology Assessment
Denmark	Danish Centre for Evaluation and Health Technology Assessment (DCEHTA); Danish Institute for Health Services Research and Development (DSI)
Finland	Finnish Office for Health Care Technology and Assessment (FinOHTA)
France	L’Agence Nationale d’Accréditation et d’Evaluation en Santé (ANAES); Ministere de la Santé, de la Famille, et des Personnes handicappés; Committee for Evaluation and Diffusion of Innovative Technologies (CEDIT); French National Authority for Health (HAS) Department of Economics and Public Health Assessment
Germany	German Institute for Medical Documentation and Information (DIMDI)
Israel	Israel Center for Technology Assessment in Health Care (ICTAHC)
Netherlands	College voor Zorgverzekeringen/Health Care Insurance Board (CVZ); Health Council of the Netherlands
New Zealand	New Zealand Health Technology Assessment Clearing House for Health Outcomes and Health Technology Assessment (NZHTA)
Norway	Norwegian Centre for Health Technology Assessment (SMM)
Poland	Agency for Health Technology Assessment (AHTAPol)
Sweden	Centre for Medical Technology Assessment (CMT); Swedish Council on Technology Assessment in Health Care (SBU)
Switzerland	Swiss Network for Health Technology Assessment; Institute for Innovation and Valuation in Health Care (INNOVAL)
Thailand	Health Intervention and Technology Assessment Program (HiTAP)/International Health Policy Program (iHPP)
UK	National Health System (NHS)National Institute for Clinical Excellence (NICE)• https://www.nice.org.uk/about/what-we-do/into-practice/resource-impact-assessment• https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance/budget-impact-test• https://www.nice.org.uk/about/what-we-do/into-practice/forward-planner• https://www.nice.org.uk/about/what-we-do/into-practice/forward-planner#view
USA	Agency for Healthcare Research and Quality (AHRQ); ECRI Institute; Institute for Clinical Systems Improvement (ICSI); Blue Cross and Blue Shield Association’s Technology Evaluation Center (TEC)