HTA Implementation Roadmap in Central and Eastern European Countries.

The opportunity cost of inappropriate health policy decisions is greater in Central and Eastern European (CEE) compared with Western European (WE) countries because of poorer population health and more limited healthcare resources. Application of health technology assessment (HTA) prior to healthcare financing decisions can improve the allocative efficiency of scarce resources. However, few CEE countries have a clear roadmap for HTA implementation. Examples from high-income countries may not be directly relevant, as CEE countries cannot allocate so much financial and human resources for substantiating policy decisions with evidence. Our objective was to describe the main HTA implementation scenarios in CEE countries and summarize the most important questions related to capacity building, financing HTA research, process and organizational structure for HTA, standardization of HTA methodology, use of local data, scope of mandatory HTA, decision criteria, and international collaboration in HTA. Although HTA implementation strategies from the region can be relevant examples for other CEE countries with similar cultural environment and economic status, HTA roadmaps are not still fully transferable without taking into account country-specific aspects, such as country size, gross domestic product per capita, major social values, public health priorities, and fragmentation of healthcare financing.

Introduction

Public healthcare resources are limited all over the world; hence, encouraging the use of evidence‐based and transparent decision‐making processes to inform the allocation of scarce resources is essential at both global and national levels. Multidisciplinary tools and methods need to be established and implemented in order to improve the efficiency of resource allocation. Regardless of current global and national health policy agendas, decision‐making should rely on strong scientific evidence (Hoomans and Severens, 2014; Rudmik and Drummond, 2013). Additional research is needed in different jurisdictions on the factors influencing the uptake of research findings into health policymaking (Innvaer et al., 2002; Oliver et al., 2014), for example, how political and institutional dimensions affect the use of health evidence in decision‐making (Liverani et al., 2013).

Health technology assessment (HTA) is a multidisciplinary field of policy analysis, which studies the medical, social, ethical, and economic implications of the development, diffusion, and use of health technologies (INAHTA, 2014). The purpose of HTA is to support policymakers with the best available information about new and already widely used health technologies in order to inform resource allocation decisions in a variety of settings, including low‐income and middle‐income countries (Grutters et al., 2011; Lopert et al., 2013; Mathew, 2011; Velasco‐Garrido and Busse, 2005). Furthermore, the formal implementation of HTA can improve allocative efficiency (Henshall 2012; Tunis and Turkelson 2012) and have positive spillovers beyond sound reimbursement decisions, such as strengthening the dialogue between relevant stakeholders and focusing the public debate on patient‐level outcomes (Gerhardus et al., 2008).

The vast majority of methodological work on HTA, to date, builds on the experiences of high‐income countries, especially in Western Europe, North America, and Australia. However, many country‐specific factors can play an important role in the relative value of a technology to the decision‐maker. The demand from one side, the affordability on the other, and the gap between them highly depend on socioeconomic factors. Added therapeutic benefits, baseline risks, and local treatment practices reflect on the effectiveness of the technologies. Differences among countries can be observed in the basic benefit packages and formulary lists, and consequently, in the unmet medical need and standard care for the comparator in HTA analyses. Variations in philosophies and techniques across national and regional HTA systems in Europe – as well as worldwide – are the result of health, social, financial, and political differences (Allen et al., 2013). As such, the evaluations and recommendations issued by different HTA bodies over the same technologies differ (Kristensen and Gerhardus, 2010; Nicod and Kanavos, 2012) because of differences in local data in addition to differences in methodological and institutional approaches.

Our study is focusing on the application of HTA in Central and Eastern European (CEE) countries. The exact countries included in the term ‘Central and Eastern Europe’ could be contentious, as it does not refer to an official definition or predefined group of countries (e.g., by OECD, see http://stats.oecd.org/glossary/detail.asp?ID=303). It rather attempts to cover countries in Europe, which share both a similar economical, demographical patterns and a history of socialist rule as well. These could have important implications for resource allocation and decision‐making process in healthcare systems. Without making any declarations as to regional boundaries, we believe that our framework and analysis are broadly relevant to the widest definitions of the CEE (from the Baltic to the Balkan region and from Central to Southeastern Europe), as many of these countries also share low per capita gross domestic product (GDP) and/or low total GDP, which severely limits the national resources available for HTA. For example, according to the International Monetary Fund World Economic Outlook Database October 2014 Edition (see http://www.imf.org/external/pubs/ft/weo/2014/02/weodata/index.aspx), even higher income CEE countries such as Poland and Hungary have more than three times lower GDP per capita compared with France, Germany, or UK. Further, many of the countries in the region are small, so while the GDP per capita of Baltic states (Estonia, Latvia, and Lithuania) is relatively high (>16.000 USD in 2014), they all have populations under 2–3 million, resulting in very limited national resources for country‐specific HTA research.

These countries also share similarities in healthcare decision‐making processes as they face particular challenges in providing highly developed health services compared with more affluent countries: the delays in pricing and reimbursement decisions may be substantial (Ades et al., 2014), patient access to high cost technologies in rare diseases is more limited (Iskrov et al., 2012), the resources to provide complex biological therapies are fewer (Péntek et al., 2014), expensive oncological therapies are rarely affordable (Vladescu et al., 2009), and several isolated ethnic groups face access barriers even to basic services (Rechel et al., 2009).

When comparing the CEE region to the wealthier Western European (WE) countries, five contextual factors particularly influence HTA development and the associated decision‐making process: (1) Financial resources for improving health through innovative and expensive technologies are more limited; (2) The health status of the population is worse; hence, the actual needs for effective medical technologies could be higher; (3) The reliance on public financing and provision of health care is heavier in post‐communist countries. Historically free and easy access to healthcare technologies and services makes rationalization a sensitive issue in the public domain; (4) The pricing rationale of new healthcare technologies (especially pharmaceuticals) in Europe is primarily driven by requirements and needs of large WE countries;. and (5) Capacity for HTA is insufficient, for example, there is a deficit of trained HTA doers (especially health economists), and budgets for conducting HTA are low (Boncz et al., 2014; Inotai et al., 2012; Jakovljevic, 2013; Kaló et al., 2013).

In this paper, our objective was to summarize the major types of HTA implementation practices and to propose an HTA implementation scorecard that can support the formulation of HTA roadmaps in CEE countries.

Methods

Authors are involved into HTA implementation and capacity building in their routine work. The paper summarizes the current conclusions of an ongoing consultation process conducted by authors during their work at international levels in fora, which included EUnetHTA, ISPOR CEE Network, and ISPOR HTA Roundtable Europe. Draft components of the HTA roadmap have been identified earlier and have already been presented and discussed at several meetings. Feedback and input was collected from relevant stakeholder groups in an iterative process.

The situation analysis was informed by a pragmatic literature review of current HTA practices in CEE countries. Initially, we searched for articles listed on Pubmed and Scopus with keywords of HTA in combination of CEE or CEE country names. Additionally, we looked at the reference list in publications included in the literature review. As scientific papers in English language are limited or out‐dated with the exception of few CEE countries, such as Poland or Hungary, we also reviewed the gray literature, including web pages of Ministries of Health and HTA bodies in each CEE country.

Two major outputs were prepared for this paper as a result of the abovementioned methods. Firstly, a summary on major types of HTA implementation practices was presented to understand how CEE countries adjusted their decision‐making process to the specific economic and political conditions. Based on this overview, we prepared a draft HTA roadmap scorecard to support HTA implementation strategies in CEE countries through an assessment of the current status of HTA implementation and future long‐term objectives in seven different categories.

The draft HTA scorecard was applied at the Adriatic Pharmacoeconomic Congress in April 2015 in Sibenik, Croatia. Forty‐one congress participants from 12 different countries filled in the scorecard, and then aggregated responses were discussed by a roundtable panel. Based on the feedback of participants, the scorecard has been extended with one additional category related to international collaboration on HTA, and certain items of the scorecard have been amended to improve the clarity of statements.

The Political Economy of Health Technology Assessment Implementation in Central and Eastern European Countries

Although CEE countries share a common history as well as similarities in the post‐communist transition, there are still significant political and economic differences among countries with important constraints on HTA implementation.

Firstly, CEE countries are heterogeneous in terms of size and wealth. When large multinational blocks dissolved (Soviet Union, Yugoslavia, and Czechoslovakia), smaller countries were re‐established or created, especially in the Baltic and Adriatic regions. Such small countries face more limitations in full HTA implementation because of correspondingly fewer resources and larger difficulties in building large capacities for HTA. Although collaboration among small countries in HTA and health services research with neighboring countries may be highly beneficial, some of the political and ethnical tensions that eventually resulted in separation of these countries remain. At a higher level, successful pan‐European collaborations among European HTA agencies are possible (Woodford and Cook, 2014), but some barriers should still be overcome in terms of communication, identifying collaborative partners, differences in the methodology of assessment, for example, outcome selection, and the level of acceptance of different types of comparison and study type (Huić et al., 2013; ten Have et al., 2013).

Secondly, the economies of CEE countries have evolved differently from the setback following the political transition. Financial recovery was rapid in countries with peaceful transitions, while others have not recovered for many years. In addition, the political and economic transition became European Union (EU) accession oriented in selected CEE countries. EU institutions guided reforms in these countries by setting intermediate and final objectives, permanently monitoring the process and providing financial and technical assistance for the timely accomplishment of necessary steps. The economic performance of EU member states from CEE is generally stronger than non‐EU countries (European Transition Compendium, 2015). Although poorer countries have a greater need to minimize the opportunity cost of inappropriate health policy decisions, they cannot allocate sufficient resources for HTA capacity building; therefore, they are less advanced in HTA implementation.

Lastly, policy leaders in CEE countries had no intention to apply transparent decision‐making processes before the political and economic transition, thus the evidence base on healthcare resource allocation decisions was limited. After the transition, non‐transparent privatization processes and the expansion of the informal sector resulted in affluent economic and political segments in several CEE countries. Many individuals in these segments may have an interest in preserving status quo, thus have disincentives to support transparent resource allocation algorithms in public sector decisions, including public healthcare financing. As such, there has been little political support towards HTA implementation, which would clearly reduce the scope for individual decisions lacking strong scientific evidence‐based and verifiable criteria. In addition, several previous healthcare policymakers established consulting companies in the region, benefiting from personal insights to non‐transparent HTA, pricing, and reimbursement processes. Although this problem is documented only in Poland (Ozieranski et al., 2012a; Ozieranski et al., 2012b), similar conclusions could be drawn in many other CEE countries. This particular lack of political will coupled with underdeveloped accountability mechanisms must be distinguished from the wider political resistance towards formalizing HTA components or processes, which high‐income countries may also experience, albeit for more substantive reasons such as centralization‐decentralization tensions (Artells et al., 2014) and the reluctance to rationalize care (Neumann and Weinstein, 2010).

The aforementioned factors make the implementation of HTA difficult in CEE countries, and they generate a wide range of HTA implementation experience.

Patterns of Health Technology Assessment Implementation Practices in Central and Eastern European Countries

The initial question of HTA implementation is whether the process should start with mandating the use of technology assessment in pricing and reimbursement decisions or with investing into HTA capacity building. If mandating HTA evidence comes first, it poses an additional question on the degree of HTA institutionalization. HTA activities, especially the critical appraisal of technology assessment, can be coordinated either by public HTA offices/institutes or by special committees.

Figure 1 describes the major types of HTA implementation practices in CEE countries across the two key dimensions: the extent to which HTA implementation is a public priority and the focal HTA development area (capacity building or mandate to inform decisions). In extreme cases, HTA implementation has not yet started, or, on the contrary, has already been fully implemented. In the first case, the decision‐making process hardly includes any HTA evidence, and policymakers do not intend to pursue the application of this tool; therefore, they pay no attention on HTA capacity building. This is typically the case of small countries with health systems in transition, such as the lately established states in Southeast Europe, like Kosovo or Montenegro (Jakovljevic, 2013). In the second extreme case, HTA implementation is already a recognized public priority, accompanied by heavy reliance on local HTA evidence in the decision‐making process. Consequently, these countries, such as Poland and Hungary, have already seen investments in the development of versatile educational system and established public HTA organizations. Permanent HTA graduate and postgraduate programs supplemented with short courses contribute to the quantity and the quality of the HTA capacities (Gulácsi et al., 2014; Nizankowski and Wilk, 2009).

Figure 1

Major types of health technology assessment (HTA) implementation practices in Central and Eastern European countries

In between these extreme types, countries can be clustered according to their current focus either on capacity building or policy implementation. In many CEE countries, the mandatory use of HTA evidence in decision‐making is not supported by adequate human resource capacities and legal framework. In such cases, the decision‐making process may dominantly rely on international HTA evidence, for example, in Romania (Lopert et al., 2013). However, transferring the data and the results from other jurisdictions without any local adaptation and adjustment may do more harm than good (Kaló et al., 2012). Because of limitations of HTA transferability, this practice could be only a transient state of HTA implementation. Despite general recommendations for local adaption of HTA evidence, smaller countries have more constraints to advance their HTA system to capture local evidence.

Mandating the use of local HTA evidence without adequate human resources and permanent educational programs may also negatively influence the quality of technology assessment processes. In such cases (e.g., Bulgaria), stakeholders may not fully rely on HTA recommendations in policy decisions (Iskrov et al., 2013).

When permanent HTA graduate courses appear in the education system, there is a good opportunity to improve the quality and the quantity of national HTA capacity, which could also raise the quality of HTA appraisals. In Serbia, HTA has slowly gained momentum, and the key reason for its slow implementation was the lack of health economic experts in the country (Zah, 2011).

Institutionalization of the HTA process, including establishment of public HTA offices, is an important element of implementation (Oortwijn et al., 2013). In Croatia, the Agency for Quality and Accreditation in Health Care and Social Welfare has been covering health technology assessment since 2009 (Huić, 2011; Mittermayer et al., 2010). Because of the limited human and financial HTA resources, and being subject to a legal framework in which HTA is still not mandatory, the Croatian HTA Department from the begining of the establishment of an HTA process recognized the importance of international collaboration through EUnetHTA within capacity building and joint collaborative work on core HTAs.

In Slovenia, limited human resources led to plans of setting up a structured HTA network integrating existing national institutions in order to overcome capacity limitations (Turk et al., 2010). Networking national HTA agencies makes special sense in relatively small countries. In Slovakia, a methodological guideline was published with a strong commitment for economic evaluations to contribute to efficient resource allocation decisions. However, without sufficient capacity, the role of HTA was not accurately defined within the healthcare system (Tesar, 2012). These examples clearly show that at this stage of implementation, more capacity is needed for a well‐functioning HTA system.

Making HTA a public priority implies the continuous development of capacity building and HTA education. For example, the Baltic countries had already shown an interest in HTA capacity building more than a decade ago (Danguole, 2009; Gibis et al., 2001). Even less developed countries, such as Albania, pursued initiatives to modernize the health system in which a number of medical professionals received HTA training (Jakovljevic, 2013). The Canadian Society for International Health (CSIH) provided an initial HTA mentoring program in Kazakhstan with the partnership of Kazakhstan Ministry of Health (MoH). A seven‐member CSIH team provided HTA materials, short courses, and one‐on‐one support for MoH HTA staff over a 2.5‐year project (Muratov et al., 2014).

Investing in capacity building alone does not guarantee the use of HTA evidence in health policy decisions even after sufficient human resources become available if appropriate legal framework and public HTA organizations are not established. If the demand for HTA evidence from decision‐makers is limited, academic centers sometimes conduct technology assessments mainly on pharmaceuticals without strong influence on pricing and reimbursement decisions.

Health Technology Assessment Implementation Scorecard – Comparison of Current Status and Future Directions

Our research indicates that the current practice of HTA implementation is highly heterogeneous in the CEE region. Some countries are still in the initial phases and several others are in transition towards full HTA implementation. Although there are some countries with full HTA implementation, our experience suggests that the even these countries have room for improvement, for example, in the transparency of the HTA process (Kaló et al., 2013; Kolasa and Wasiak, 2012).

An HTA implementation scorecard can be a useful tool to assess the current status of HTA implementation and to set up long‐term objectives. Based on our experience and review of the scientific literature on HTA implementation practices in CEE, we developed such a scorecard focusing on eight key components (Table I).

Table I

HTA implementation scorecard – comparison of current status and future directions

	Current status	Preferred status in 10 years
1. HTA capacity building
Education (single choice)
– No training
– Project‐based training and short courses
– Permanent graduate program with short courses
– Permanent graduate and postgraduate program with short courses
2. HTA funding
Financing critical appraisal of technology assessment (single choice)
– No funding for critical appraisal of technology assessment reports or submissions
– Dominantly private funding (e.g. submission fees) by manufacturers for the critical appraisal of technology assessment reports or submissions
– Dominantly public funding for critical appraisal of technology assessment reports or submissions
Financing health technology assessment (i.e., HTA research) (single choice)
– No public funding for technology assessment; private funding is not needed or expected
– No or marginal public funding for research in HTA; private funding is expected
– Sufficient public funding for research in HTA; private funding is also expected
– HTA research is dominantly funded from public resources
3. Legislation on HTA
Legislation on the role of HTA process and recommendations in decision‐making process (single choice)
– No formal role of HTA in decision‐making
– Dominantly international HTA evidence is taken into account in decision‐making
– International and additionally local HTA evidence is taken into account in decision‐making
– Local HTA evidence is mandatory in decision making
Legislation on organizational structure for HTA appraisal (single choice)
– There is no public committee or institute for the appraisal process
– Committee is appointed for the appraisal process
– Committee is appointed for the appraisal process with support of academic centers and independent expert groups
– A public HTA institute or agency is established to conduct formal appraisal of HTA reports or submissions
– Public HTA institute or agency is established to conduct formal appraisal of HTA reports or submissions with support of academic centers and independent expert groups
– Several public HTA bodies are established without central coordination of their activities
– Several public HTA bodies are established with central coordination of their activities
4. Scope of HTA implementation
Scope of technologies (multiple choice)
– HTA is not applied to any health technologies
– Pharmaceutical products
– Medical devices
– Prevention programs and technologies
– Surgical interventions
– Other (please specify): ......................................
Depth of HTA use in pricing and/or reimbursement decision of health technologies (single choice)
– HTA is not applied to any health technologies
– Only new technologies with significant budget impact
– Only new technologies
– New technologies + revision of previous pricing and reimbursement decisions
5. Decision criteria
Decision categories (multiple choice)
– None of the below categories are applied
– Unmet medical need
– Health care priority
– Assessment of therapeutic value
– Cost‐effectiveness
– Budget impact
– Other (please specify): …………………………………….
Decision thresholds (single choice)
– Thresholds are not applied
– Implicit thresholds are preferred
– Explicit soft thresholds are applied in decisions
– Explicit hard thresholds are applied in decisions
Multi criteria decision analysis (single choice)
– Explicit multi criteria decision framework is applied
6. Quality and transparency of HTA implementation
Quality elements of HTA implementation (multiple choice)
– None of the below quality elements are applied
– Published methodological guidelines for HTA/economic evaluation
– Regular follow‐up research on HTA recommendations
– Checklist to conduct formal appraisal of HTA reports or submissions exists but not available for public
– Published checklist is applied to conduct formal appraisal of HTA reports or submissions
Transparency of HTA in policy decisions (single choice)
– Technology assessment reports, critical appraisal and HTA recommendation are not published
– HTA recommendation is published without details of technology assessment reports and critical appraisal
– Transparent technology assessment reports, critical appraisals and HTA recommendations
Timeliness (single choice)
– HTA submission and issuing recommendation have no transparent timelines
– HTA submissions are accepted/conducted following a transparent calendar, but issuing recommendation has no transparent timelines
– HTA submissions are accepted continuously and issuing recommendation has transparent timelines
7. Use of local data
Requirement of using local data in technology assessment (single choice)
– No mandate to use local data
– Mandate of using local data in certain categories without need for assessing the transferability of international evidence
– Mandate of using local data in certain categories with need for assessing the transferability of international evidence
Access and availability of local data (single choice)
– Limited availability or accessibility to local real world data
– Up‐to‐date patient registries are available in certain disease areas, but payers' databases are not accessible for HTA doers
– Payers' databases are accessible for HTA doers, patient registries are not available or accessible in the majority of disease areas
– Up‐to‐date patient registries are available in certain disease areas and payers' databases are accessible for HTA doers
8. International collaboration
International collaboration, joint work on HTA (joint assessment reports) and national/regional adaptation (reuse) (multiple choice)
– No involvement into joint work; and no reuse of joint work or national/regional HTA documents from other countries
– Active involvement in joint work (e.g. EUnetHTA Rapid REA, full Core HTA)
– National/regional adaptation (reuse) of joint HTA documents
– National/regional adaptation (reuse) of national/regional work performed by other HTA bodies in other countries
International HTA courses for continuous education on HTA (single choice)
– Limited interest in (1) developing / implementing of and (2) participating at international HTA courses
– Interest only in regular participation at international HTA courses
– High interest in (1) developing / implementing of and (2) participating at international HTA courses

HTA capacity building

HTA is a multidisciplinary field, and several of its components, especially those related to synthesizing clinical evidence, are included in the traditional curriculum of medical or pharmacy graduate training. However, CEE countries have limited capacity for delivering postgraduate HTA courses, especially in the field of developing skills in economic modeling. Until such training courses are established in the region, potential CEE researchers have to study in WE academic centers. This is a very costly alternative to local education, and actually, a high proportion of CEE experts with Master or PhD degrees obtained abroad do not return to their own country to support HTA implementation.

Undergraduate courses are not an option for in‐depth training, especially in economic evaluation methodology and transferability; however, they are important to raise the awareness, positive attitude, basic knowledge, and understanding on the potential contribution of medical professionals.

Local project‐based or regular short courses may be useful options to alleviate the need for trained HTA professionals, as well as HTA courses provided by international networks and organizations, such as EUnetHTA, HTAi, and ISPOR. Such short courses are also valuable for HTA users. However, the fluctuation of top‐level policymakers is relatively high in CEE countries, which necessitates the repetition of these courses.

HTA funding

Health technology assessment processes have clear associated costs proportional with their scope (ten Have et al., 2013); therefore, funding is needed to conduct both the technology assessment (i.e., the research part) and the appraisal of evidence submitted by researchers or manufacturers (i.e., the appraisal part).

If HTA becomes a mandatory element of the pricing and reimbursement process of new technologies, manufacturers of new technologies are able to fully or partially fund the research activity of HTA doers from private resources. Also, in such cases, submission fees related to pricing and reimbursement dossiers of new technologies may be the primary funding source of the critical appraisal process of technology assessment, and therefore governments do not need to allocate significant budget for HTA implementation. However, such a development would allow limited budgets to revise previous HTA recommendations, conduct technology assessment in areas with no new products, or development of HTA guidelines, standards and databases, and communication and stakeholder engagement. That is why it is desirable to invest public resources in both the assessment and appraisal components of HTA.

HTA legislation and governance

In several CEE countries, HTA has no formal role in health policy decisions, such as the pricing and reimbursement of new technologies. Some countries (e.g., Romania) predominantly rely on international HTA guidance in policy decisions without formally assessing the transferability of international HTA evidence, especially conclusions of economic evaluations (Drummond et al., 2009; Welte et al., 2004). Countries like Poland, Slovakia, and Hungary request HTA evidence based on local input data or the translation of international evidence to local practice before approving the price and reimbursement of new pharmaceuticals or medical devices (Kaló et al., 2013; Kolasa and Wasiak, 2012).

The organizational structure related to the critical appraisal of HTA evidence provided by HTA doers and researchers or submitted by manufacturers is crucial. If only a committee reviews the quality of HTA submissions, there is no room for in‐depth appraisal. If a public HTA institute or agency is established, in addition to a more detailed appraisal process, there is also more opportunity for revising previous decisions or conducting HTA in areas with no new products to support disinvestment decisions or optimization of healthcare delivery. The HTA organization may rely on external expertise, for example, the Scottish Medicines Consortium with limited internal staff relies heavily on the contribution by academic centers (Stafinski et al., 2011).

In large countries, especially those with multi‐payer health care financing, establishing multiple HTA agencies may reduce a potentially monopolistic position of a central HTA office. However, inappropriate coordination among several HTA bodies may result in duplication of efforts when generating HTA evidence or recommendations.

Scope of HTA implementation

If capacity for HTA is limited, it makes sense to introduce its use only for selected health technologies. In several countries, HTA evidence is used only for pricing and/or reimbursement of pharmaceuticals. Countries, however, may have significant differences in applying HTA evidence only for new products with significant budget impact, all new products, or even revision of HTA evidence for products already on the reimbursement list (Carone et al., 2012).

Health technology assessment cannot be limited to pharmaceutical technologies; therefore, it is increasingly used for policy decisions of new medical devices or medical prevention (Drummond et al., 2008). As HTA capacities are limited, priority setting process may facilitate the selection of topics for assessment in a period of time.

Decision criteria

Health technology assessment evidence can be mandated in policy decisions by requesting all or only some components of HTA. Healthcare priority and the assessment of therapeutic value are commonly used criteria even in countries without formal HTA processes. Budget impact is an increasingly used criterion in countries with highly limited public resources. Cost‐effectiveness evidence, however, is requested only in countries with more advanced HTA implementation. In addition to financial criteria, policymakers need to consider the program feasibility in terms of the availability of quality workforces, efficient management strategy, and social acceptability of new technologies.

The decision rule for most criteria is implicit. For example, it is difficult to judge how much added therapeutic value is needed to justify reimbursement or how much is the acceptable budget impact, if the new technology cannot save public resources (e.g., oncology drugs with significant survival benefit).

Several CEE countries, however, established and published thresholds for the cost‐effectiveness criterion. Cost‐effectiveness thresholds can be soft or hard depending on how much deviation is allowed in policy decisions compared with conclusions of the economic evaluation (Kolasa et al., 2012). In Poland and Hungary, cost‐effectiveness thresholds are soft, that is, new technologies with incremental cost‐effectiveness ratio (ICER) above the published threshold still can be reimbursed, especially if they provide solutions to high unmet medical needs or inequities and their budget impact is relatively small, such as orphan drugs in rare diseases (Grzywacz et al., 2014; Szegedi et al., 2014). On the other hand, Slovakia applies a hard threshold, that is, reimbursement cannot be granted for new pharmaceuticals with an ICER above the threshold determined by the Parliament. However, based on the Slovak legislation, drugs in rare diseases (i.e., with prevalence below 1:100 000) can be reimbursed without considering the threshold, and patient access schemes for highly innovative pharmaceuticals can be applied if the ICER is above the threshold.

Different components of HTA can be assessed jointly and explicitly in multi‐criteria decision analysis (MCDA). In 2010, MCDA was introduced for new hospital technologies in Hungary, primarily for medical devices (Endrei et al., 2014).

Quality and transparency of HTA implementation

The quality of HTA implementation can be improved by several tools, such as published methodological guidelines for technology assessment. A detailed critical appraisal checklist can standardize the appraisal process of HTA submissions and consequently improve the consistency of reimbursement decisions (Inotai et al., 2012). Publishing a critical checklist provides indirect guidance for HTA doers and therefore may prevent unnecessary errors or mismatches in the interpretation of scientific evidence. The learning process on HTA quality can be improved by regular follow‐up research on previous HTA recommendations.

Transparency is essential to successful HTA implementation as clear requirements and published evidence can formally justify previous policy decisions and create verifiable criteria for future decisions. Despite sophisticated HTA methodology in Hungary, technology assessment and critical appraisal reports, or HTA recommendations, are not published and not even available for public scrutiny. In Poland and Croatia, HTA reports with recommendations issued by national HTA agency are published, therefore researchers can review the appropriateness of previous recommendations (Kolasa et al., 2011).

Another component of transparency is the timeliness of the HTA process. Ideally, HTA submissions are accepted continuously, and issuing recommendations has transparent timelines.

Use of local data

If human and financial resources for HTA are limited and the availability and quality of local data on the effectiveness of health technologies and costs of care are low, HTA users may not request local evidence and data in the decision‐making process. For example, the current Romanian pharmaceutical reimbursement process relies heavily on HTA recommendations issued in Germany, France, and UK, while policymakers cannot take into account local cost‐effectiveness studies. On the other hand, several CEE countries with more advanced HTA implementation mandated the use of local data in economic evaluations (Skoupá et al., 2014).

Local effectiveness and cost data can be more easily generated in countries where patient registries in the most important disease areas and payers' databases are available and accessible for HTA doers.

International collaboration

According to the new European HTA Network Strategy, international collaboration in HTA may prevent duplication of efforts. Although individual CEE countries may just choose the reuse or adaptation of international HTA reports or models, active contribution to joint assessment reports represents an increased level of commitment to HTA implementation.

International collaboration in continuous education on HTA can reduce the scarcity of human resources in CEE, including development of training course materials and delegation of participants at international HTA training programs.

Conclusion

Because of poorer population health and more limited healthcare resources, the opportunity costs of suboptimal health policy decisions are greater in CEE compared with WE. CEE countries could pay an even higher price for inappropriate reimbursement and resource allocation decisions, especially in difficult economic periods. CEE decision‐makers, in their capacity as HTA users, must improve the appropriateness of reimbursement and resource allocation decisions to increase the allocative efficiency of healthcare financing (Kaló et al., 2012).

As resources for HTA are insufficient in CEE countries compared with WE countries, they need more creativity and special skills to develop and implement HTA. CEE HTA doers should rely more on collaborative efforts, such as the EUnetHTA reports in relative effectiveness and core HTA (Kristensen et al., 2009) and ISPOR HTA Roundtable Europe. Active CEE participation in international collaborations and joint HTA work ensures that advancement of HTA tools takes into account CEE needs and constraints. Therefore, increased funding and involvement of CEE researchers in European projects – focusing on public health provision and methodological development of evidence‐based health policy – are essential. However, the transferability of pricing and reimbursement practices and HTA recommendations from WE countries may be limited. Consequently, the national adaptation of international HTA evidence has vital importance, which still necessitates local HTA capacity, budget, and well‐functioning organizations.

Although CEE countries with similar cultural environment and economic status can learn from each other in improving their healthcare systems, there is no generalizable gold standard for HTA implementation. HTA roadmaps are not fully transferable without taking into account country‐specific aspects, such as country size, GDP per capita, major social values, public health priorities, and the fragmentation of healthcare financing. Countries need to focus on identifying and formulating their own values, health policy objectives, and constraints in order to develop their own HTA systems. A creative and consistent implementation process delivers success over years. We believe that the availability of and adherence to a clear roadmap can support this process and improve the efficiency of HTA implementation in each country. This may eventually result in better health outcomes and more equitable access to health technologies in parallel with improving the sustainability of health care financing in the CEE region.

The proposed HTA implementation scorecard is designed to set up long‐term objectives for the roadmap. It is informed by empirical observations and a pragmatic review of scientific and gray literature and stakeholder consultations. Limited research has looked into the applicability and political feasibility of such tools; therefore, we propose that future research address this aspect. Our scorecard must be viewed as an initial step in a multi‐stakeholder dialogue on best HTA implementation practices in CEE countries and should be updated as needed.

We plan to use the scorecard at international training courses (such as the ISPOR Health Technology Assessment Training Program – see http://www.ispor.org/Education/HTATraining/Index) to facilitate workshops on HTA implementation. In addition, we aim to collect information on current and preferred status on local HTA implementation in CEE countries and publish results in scientific journals.

The HTA implementation scorecard may also be generalizable to other emerging regions with similar constraints as CEE, such as limited human resources, budget, and yet unestablished organizational structure for HTA.

Conflicts of Interest

The authors declare that they have no conflict of interest.

Ethics statement

No specific ethical approval was needed for this study.