Health technology assessment in Switzerland: a descriptive analysis of "Coverage with Evidence Development" decisions from 1996 to 2013.

OBJECTIVES: To identify factors associated with the decisions of the Federal Department of Home Affairs concerning coverage with evidence development (CED) for contested novel medical technologies in Switzerland.
DESIGN: Quantitative, retrospective, descriptive analysis of publicly available material and prospective, structured, qualitative interviews with key stakeholders.
SETTING: All 152 controversial medical services decided on by the Federal Commission on Health Insurance Benefits within the framework of the new federal law on health insurance in Switzerland from 1997 to 2013, with focus on 33 technologies assigned initially to CED and 33 to evidence development without coverage. MAIN OUTCOME MEASURES: Factors associated with numbers and type of contested services assigned to CED per year, the duration and final outcome of the evaluations and perceptions of key stakeholders.
RESULTS: The rate of CED decisions (82 total; median 1.5/year; range 0-9/year), the time to final decision (4.5 years median; 0.75 to +11 years) and the probability of a final 'yes' varied over time. In logistic regression models, the change of office of the commission provided the best explanation for the observed outcomes. Good intentions but absence of scientific criteria for decisions were reported as major comments by the stakeholders.
CONCLUSIONS: The introduction of CED enabled access to some promising technologies early in their life cycle, and might have triggered establishment of registries and research. Impact on patients' outcome and costs remain unknown. The primary association of institutional changes with measured end points illustrates the need for evaluation of the current health technology assessment (HTA) system. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Comprehensive analysis of all medical technologies submitted to ‘Coverage with Evidence Development’ (CED) within one country over a defined time frame.

Additional structured qualitative interviews with the key stakeholders in the process in order to understand the mechanisms associated with the decision process and changes over time.

The finding that institutional changes provided the best explanation for an association with the many major changes in the process in logistic regression models underlines the need for scientific analyses of CED as a valuable tool in health technology assessment (HTA).

The retrospective nature of the study and the absence of data on patients’ outcome and costs limit assessment of the real value of CED.

It might be difficult to generalise the results to countries with other healthcare systems.

Introduction

Health technology assessment (HTA) is considered essential in any solidarity-based healthcare system for supporting funding decisions. The rising gap between unlimited requests and limited resources requires transparent assessment of allocation of funds. Traditionally, HTA has used the instruments of evidence-based medicine such as a systematic search of high-quality research. The rapid development of novel medical services (including drugs, devices, diagnostics and interventional procedures) increasingly requires funding decisions before sufficient evidence has been generated.1 On the one hand, there is a desire not to commit to a technology that may ultimately prove to be ineffective and/or unsafe. On the other hand, there is a wish to provide patient access to promising innovative approaches early in their life cycle. For such situations, in many jurisdictions around the world, funding has been linked with the requirement of further evidence development typically with the help of a registry or a clinical trial. Different terms have been used, the most prominent being ‘Coverage with Evidence Development’ (CED), defined as a type of managed entry agreement between manufacturers or service providers and the paying healthcare system.2–7 Despite its many deficiencies and its dependency on political decisions, CED has been considered by many to be the tool to evaluate evolving technologies, but the best approach remains unknown.8–10

Switzerland has used the CED concept for non-drug technologies since 1996, when the new Federal Law on Basic Health Insurance (KVG/LAMal) came into force. With this new law, it became mandatory for each resident in the country to buy a basic health insurance package from one of about 60–70 competing health insurance companies. The law stipulates that individual medical technologies have to be covered when they are considered ‘effective’, ‘appropriate’ and ‘efficient’. These three terms are central in Swiss legislation and are preconditions for coverage by the Swiss statutory health insurance scheme. In the case of medical services (including in vivo diagnostics and devices but not drugs and not in vitro diagnostics) provided by physicians or hospitals, it is assumed that these criteria are fulfilled by default and no formalised HTA process is necessary for reimbursement. This is called the ‘principle of trust’. In case of doubt, however, anyone with a legitimate interest, for example, a health insurance provider, can challenge the medical service and a formalised HTA process is triggered.11 Such a potentially controversial medical service has to be reported to the Federal Office of Public Health (FOPH), which is responsible for, among other things, the supervision of the health benefit catalogue.

The provider or the manufacturer has then to submit full documentation of the available evidence on effectiveness (including a systematic review), appropriateness and cost-effectiveness. The FOPH checks the submission for completeness and writes a summary including critical issues. All information on the case is assembled in a dossier which is handed over to the Federal Commission for Medical Benefits and Principles (ELGK) for appraisal. The final decision lies with the Federal Department of Home Affairs (EDI), the parent organisational unit of the FOPH, and it is published in the procedures ordinance (KLV/OPAS) related to the health insurance law.

Since its introduction on 1 January 1996 the possible decision was not limited to ‘yes’ and ‘no’ but also ‘yes, in evaluation’ for novel and promising medical technologies where the existing evidence was incomplete. Under this status, the medical service was reimbursed but with the stated goal of further evidence collection. This status comes with an initial period of time which is frequently extended if the evidence is still incomplete. Hence, CED had been used in Switzerland for many years without being formally labelled as such.

Finally, before 2004 a number of services were listed as ‘no, in evaluation’ in the procedures ordinance. Those services could be provided. However, they were not reimbursed by the health insurance scheme and had to be financed by other means, for example, a research grant or private insurance. Therefore, they cannot be labelled CED. However, they could be upgraded to ‘yes’ or ‘yes, in evaluation’ later in the process. The decision option ‘no, in evaluation’ was abandoned in 2004 since it did not have any practical significance from a governance perspective.

A recent descriptive analysis provided some insight into incidence, duration and final outcome of the CED decisions in Switzerland. No structured evaluation was made; no factors associated with decisions or outcome were looked for.12 There is increasing awareness of a need for decisions based on HTA, a rising concern about ‘the second gap’ in translation (the ‘first gap’ exists in the translation of knowledge from benchmark to new medical interventions, and the ‘second gap’ from new medical interventions to clinical application), but little information on evaluation of HTA decisions in the literature in general.13 14 We aimed to learn more about the relative frequency of CED decisions compared to the total number of decisions on contested medical services and on potential factors associated with the final reimbursement decision.

Methods

Study design

A mixed-methods approach was used. In a longitudinal retrospective quantitative analysis of publicly available data we searched for factors associated with initiation, duration and outcome of CED decisions; with focused qualitative interviews of key stakeholders we searched for soft factors within the multilevel decision process.

No individual patient data were analysed; no ethics committee approval was required.

Data collection

The study followed the principle of a previous analysis but looked at all 152 initial decisions by the Federal Department of Home Affairs (EDI/DFI) regarding contested medical services since 1996 (figure 1). The decisions are published in online supplementary annex 1 of the procedures ordinance to the law on Health Insurance (KLV/OPAS) which is updated at least once a year and is publicly available on the webpage of the FOPH (http://www.bag.ch). All decisions on new procedures year after year from 1996 until 2013 were looked up manually by the research team. All decisions with a formal ‘yes, in evaluation’ or ‘no, in evaluation’ were selected for detailed analysis. Information on decision, duration of CED state, restrictions and requirements was extracted.

Figure 1

Multistate Model of 152 contested medical technologies decided on by the Federal Commission on Health Insurance Benefits in Switzerland from 1996 to 2012. The figure depicts the transitions from one evaluation state to another to a final ‘yes’ or final ‘no’.

Information on the number of decisions per year that directly lead to a ‘yes’ or ‘no’ was provided by the FOPH along with additional information on the sequence of decisions on reimbursement, as illustrated in figure 1.

Decisions with a formal ‘yes’, but an additional requirement for, for example, a registry or reanalysis after a specified time interval were not included, despite their ‘conditional’ strings. All contested medical services were grouped by their type of technology as defined by the EuroScan database (http://euroscan.org.uk/) into diagnostics, procedures, devices and programmes.15 Analysis concentrated on factors associated with incidence, duration and outcome of the process (see online supplementary tables S1).

Factors analysed for an association with the CED process

We analysed the association of restrictions and requirements imposed on the evaluations with the final reimbursement decision and the duration of the evaluation. Explanatory variables encompass restrictions to specified centres or specialists, previous approval by a medical advisor of the health insurer or the requirement for a registry. No public information was available to us concerning the submitter, the amount and quality of available evidence on efficacy, safety and cost impact at the time of the decision, burden of disease or unmet needs.

The association of institutional factors with the incidence and the final reimbursement decision was also analysed. As factors reflecting the institutional environment, we considered the influence of the president of the appraisal committee, the Federal Commission for Medical Services and Policy Issues (ELGK/CFPP), and the decisionmaker, the federal councillor of the Federal Department of Home Affairs.

The president of the appraisal committee has agenda setting power and is therefore the most important member of the ELGK. The federal councillor in charge is the final decisionmaker. He or she is in the role of the health minister although formally Switzerland does not have one. His or her decisions are based on, but are independent from, the recommendation by the ELGK. Most decisions however follow the recommendations by the commission. A further important institutional factor could have been the federal office that the ELGK was assigned to. While originally it belonged to the federal social insurance office, it became part of the FOPH in 2004.

Structured qualitative interviews

Focused interviews following Merton and Kendell16 were conducted with past and current members of the appraisal committee (ELGK/CFPP) and representatives of the FOPH who also chaired the committee until 2011, when this was changed for governance reasons. The selection of the interviewees was carried out by way of theoretical sampling in order to represent as many different perspectives as possible.17 Ten interviews were conducted with 11 people because one interview was carried out with two people. Seven interviewees were members of the appraisal committee (current members: 4; past members: 3). Three of the interviewees were current or former presidents of the committee. Five of the interviewed committee members were representing a stakeholder group (health insurers: 2; service providers: 2; patients: 1). Five interviewees were representing the FOPH and included both current and past members. Eight interviewees were medical doctors by training whereas four were lawyers (one had a double degree). Individuals can have multiple characteristics.

The focused interviews were designed to validate the results generated in the statistical analysis, to ease interpretation of the results, to better understand the context and the nature of the decision-making process, to shed light on the decision-making dynamics hidden in the ‘black box’, and to learn about different individual perspectives and interpretations of the situations by the experts. The interviews were based on a semistructured questionnaire (see online supplementary material); they were all carried out face-to-face, they lasted between 50 and 120 min and they were audio taped and transcribed before analysis. The analysis was performed by iterative reading by one author (UB) and free codes were applied. Statements on the same emerging themes were grouped in tabulated form. A second author (AG) read all the interviews and crosschecked the results.

Statistical data analysis

Factors associated with the incidence of new evaluations and with the final decision, respectively, were identified using logistic regression. We tested the influence of the explanatory variables by means of deviance tests. Variable selection was based on Akaike's Information Criterion (AIC).18 19 Time to event was evaluated using cumulative incidence functions estimated by a proportional cause-specific hazard model.20 All analyses were conducted in R 3.0.2.21

For the detailed analysis, we excluded technologies of alternative medicine (such as homeopathy, acupuncture, anthroposophical medicine, traditional Chinese medicine or phytotherapy) due to the strongly political nature of decisions in this field. In a referendum, the Swiss population decided in May 2009 to keep alternative medicine within the mandatory health insurance scheme. The approval rate was 67%.

Results

Use of ‘CED’ in Switzerland from 1996 to 2012

We distinguish between two related concepts. First, we consider the number of medical services evaluated. Second, we evaluate the number of decisions on the reimbursement of these services. Medical services that go through a period of CED change their state at least twice, some even more often. Consequently, two or more decisions are made until a final reimbursement decision has been reached. Over time, a total of 152 contested medical services were evaluated and 234 decisions were made by the commission (figure 1). For 86 (57%) of the medical services, a direct decision for acceptance (N=50; 33%) or rejection (N=36; 24%) was made. No further details were collected on these decisions. For 66 services (43%), the requirement ‘in evaluation’ was added by the commission at their first decision, for 33 each as ‘yes, in evaluation’ or ‘no, in evaluation’ (figure 1).

‘In evaluation’ was added in total for 82 (35%) of the 234 decisions (table 1). Of these, 46 medical services (20%) were assigned with ‘yes, in evaluation’ and consequently became CED (figure 2A). They concerned all types of contested services: alternative medicine procedures (N=10; 22%; concerning 5 services, all evaluated twice), therapeutic procedures (N=24; 52%), diagnostics (N=8; 17%), medical devices (N=8; 17%) and programmes (N=3; 7%). The frequency of these different types varied over time. In the beginning many services were diagnostics, while in more recent years there were none. Evaluation of programmes in turn was only taken up in more recent years (see figure 2A). Slightly fewer decisions were designated with ‘no, in evaluation’ (36 of 234; 15%; figure 2B).They concerned primarily therapeutic procedures (N=24; 67%), with a few diagnostics (N=4; 11%), medical devices (N=7; 19%) and programmes (N=1; 3%).

Table 1

Numbers of CED decisions in Switzerland from 1996 to 2012 by type of medical technology* and additional requirements†

Technology	‘Yes, in evaluation’				‘No, in evaluation’				Total
	Total	Thereof with restriction of			Total	Thereof with restriction of
		Centre	Specialist	Registry		Centre	Specialist	Registry
Diagnostics	8	4	4	6	7				15
Devices	10	7	1		4				14
Procedures	25	8	13	7	24	3		3	49
Programmes	3	1		1	1				4
Total	46	20	18	14	36	3	0	3	82

*According to EuroScan database.9

†Restriction to certain specialists, centres, devices, or linked with the requirement for a registry.

CED, coverage with evidence development.

Figure 2

(A) Sequence of decisions by the commission by type of technology. Depiction of the 46 coverage with evidence development ‘CED’ decisions ‘yes, in evaluation’. Colours are according to the type of medical technology as defined by EuroScan15(devices (grey), diagnostics (orange), procedures (blue) and programmes (green)). Bars represent time from initial decision to final decision. Intercepts indicate prolongation of the initial evaluation period. Symbols to the right of bars show the decision that ended the evaluation period. (B) Sequence of decisions by the commission by type of technology. Depiction of the 36 ‘CED’ decisions ‘no, in evaluation’. Colours are according to the type of medical technology as defined by EuroScan15(devices (grey), diagnostics (orange), procedures (blue) and programmes (green)). Bars represent time from initial decision to final decision. Symbols to the right of bars show the decision that ended the evaluation period.

For the majority of the initial ‘yes, in evaluation’ decisions, CED was linked with one or more additional requirements. The procedure was either restricted to a specialist physician (N=18; 39%), or a specialised centre (N=20; 43%) or required a registry (N=14; 30%). Similarly, some ‘no, in evaluation’ decisions were restricted to a specialised centre (N=3, 8%) or to the requirement for a registry (N=3, 8%; see online supplementary tables S1a–c). Decisions may impose multiple restrictions and often do so, as can be seen from table 1. The number of restrictions imposed clearly exceeds the number of decisions made.

Factors associated with initial decisions

The number of annual initial decisions by the commission changed significantly over time and ranged from 3 to 28 (figure 3). The number of initial decisions on new services to proceed with ‘yes, in evaluation’ ranged from 0 (1997, 2003, 2004, 2008, 2010, 2011) to 6 (2002) with an average of 2.6 per year. New decisions for ‘no, in evaluation’ were only made from 1996 to 2004; they ranged from 1 (1996–1999) to 17 (2002) per year with an average of 3.4 per year during these 9 years. There is a noticeable difference in new ‘CED’ decisions by the commission before and after 2005 with a mean number of new decisions of 3 (2.44, when alternative medicine is excluded) before, 1.78 (1.22, when alternative medicine is excluded) after 2005. However, these differences were statistically significant only at a 10% level (p=<0.1; with and without alternative medicine).

Figure 3

Numbers of new evaluations 1996 to 2013. Number of total new evaluations and number of new coverage with evidence development evaluations. Vertical lines show changes in the institutional environment, either a new office (green dotted line), a new president (red solid lines) or a new federal councillor (blue dotted lines).

We found no association between the share of decisions for CED and the organisational unit of the commission, the president of the commission or the federal councillor.

Outcome of ‘CED’ evaluation

A decision was made for 37 out of 46 ‘yes, in evaluation’ cases (80%) and for 35 out of the 36 ‘no, in evaluation’ cases (97.2%) by the end of 2013. Final reimbursement (‘yes’) was granted in 59.4% and 42.9% of all decisions respectively. The average duration of the evaluation was a total of 5.36 years (4.3 years initial and +1.07 years extension) for the 37 ‘yes, in evaluation’ cases that were already decided with a high variation (0.5–11 years) and most decisions (23 out of 37; 62.2%) were made between 5 and 8 years after initiation of the evaluation. The respective duration of the evaluation was 7.25 years for the 35 ‘no, in evaluation’ cases with again a high variation (0.5–21 years).

Most potential paths in the multistate model occurred, the exceptions being transitions from ‘yes’ to ‘no’ or ‘no, in evaluation’ and from ‘no’ to ‘no, in evaluation’ (figure 1).

Factors associated with final decisions

Looking at all evaluations classified as ‘yes, in evaluation’ that had already arrived at a final decision, no association was found between final outcome and requirements and restrictions, such as restriction to specialist physician, restriction to specialised centres or conduct of a registry. In contrast, the probability of a positive final decision changed significantly when the ELGK became associated with the FOPH (p<0.01), when the heads of the Federal Department of Home Affairs (p<0.05) were replaced and when the concurrent chair of the appraisal committee (ELGK/CFPP) changed (p<0.05). The strong correlation between these three factors does not permit identification of a unique source of this variation. However, according to the AIC, the association with the FOPH provides the most parsimonious model that fits the data well.

Analysing the association of restrictions and requirements with time until decision, we found that duration was significantly longer when any restriction or requirements did apply (p<0.01). This finding should not be necessarily interpreted in a causal way, since it may just indicate that more difficult cases were accompanied by technical requirements.

Registries

For a total of 14 (30.4%) cases classified as ‘yes, in evaluation’ conduct of a registry was required. No criteria were specified on how and by whom the registry had to be established or how the registry was financed. No public data of any of the registries is available.

There is one exception. ‘CED’ for certain haematopoietic stem cell transplants was linked with the requirement for ‘JACIE’ accreditation of the transplant centre as a prerequisite for reimbursement.22 Adherence to the ‘JACIE’ quality management system (Joint Accreditation Committee of the International Society for Cellular Therapy and the European Group for Blood and Marrow Transplantation; http://www.jacie.org) implies reporting of all haematopoietic stem cell transplants, those on ‘CED’ as well as all other indications, to the Swiss and the European data registry.

Qualitative interviews

The standardised qualitative interviews with key experts and past and present committee members identified several highly consistent findings. All participants believed in the value of CED, were convinced that this strategy did provide early access to promising therapies before final evidence was established, and did take their task seriously. They noted that the appraisal committee should give a recommendation but at the same time provide neutral expertise. The appraisal committee faced the challenge of considering efficacy and cost-effectiveness while the pricing for a medical service was decided on elsewhere. The interviewed realised that the durations between the evidence generation and the final decision-making varied considerably and were sometimes too long. They recognised the enormous workload associated with the documentation and the impossibility for each member of the commission to judge details. They considered the criteria to become listed as a contested medical procedure in part erratic, dependent on the presiding chair, the composition of the committee, and the documentation by the applicants.

They mentioned the lack of criteria to arrive at a ‘yes’, a ‘yes, in evaluation’, a ‘no, in evaluation’ or a ‘no’ and the lack of standardised criteria on when to link a decision with additional requirements, such as restriction to specified providers or the conduct of a registry. They noted the major, in part divergent conflicts of interest but all agreed on the need for an evaluation of the evaluation (table 2).

Table 2

Contradictory elements as emerging themes in the CED process in Switzerland

	Positive elements	Negative elements
Key problems	▸ Office can/must decide▸ ‘WZW’ at center of decision-making▸ CED integrates HTA (evidence generation) and decision-making	▸ Should provide expertise but remain neutral▸ Pricing (in Switzerland) independent from evaluation▸ Time frame varies too much
The role of rules	▸ Safeguard against arbitrariness and randomness▸ Guarantees accountability and reasonableness (clear processes as eg, NICE in England has it)	▸ Random variations over time are reality▸ Rigid process blocks flexibility; pragmatic and potentially very efficient decisions in individual situation not possible
Transparency vs confidentiality	▸ Transparency essential for fair process, reasonableness and accountability▸ Confidentiality permits members to be honest and open during the meeting	▸ Transparency induces public pressure on committee members and lobbying▸ Transparency can violate the interests of manufacturers▸ Confidential information cannot be used for other purposes, for example, economic assessment, price negotiations.▸ Confidentiality carries risk of inefficiency
Efficiency and resources	▸ Commissioners are devoted to task▸ Swiss process is lean and efficient	▸ Risk of work overload for committee members through time constraints, poor preparation of meetings, broad range of topics and language barriers▸ Not all technologies get the same attention▸ Decision can be arbitrary
Political pressure	▸ Department in principle follows recommendation of commission	▸ Pressure on commissioners less severe than in drug commission (no individual products; rather class products)▸ Pressure by pressure groups, med tech industry, media
CED as a struggle	▸ CED for controversial medical technologies is part of the reimbursement decision-making process and should improve ‘WZW’▸ Different interests are represented in commission▸ Commission and FOH realise key deficiencies in process	▸ Evidence frequently not better after CED phase but difficult to say ‘no’ at the end of a CED process.▸ No rules yet in Swiss CED process: (a) when to use CED; (b) how to define and what methodology to use for open questions; (c) how to guarantee the quality of the evaluation (compliance of service providers, financing)▸ Bad compromises, not necessarily the most competent experts are chosen▸ Time constraints, Transparency, Resources, Process definition, Feedback to the commission (evaluation of the evaluation)
Changes over time	▸ Decisions more based on evidence, more scientific▸ More realistic perception of CED (and its possibilities and limits)▸ More diverse commission	▸ More cases, more documents, more work (over)load,▸ High turnover of people at BAG (loss of knowledge)▸ More heterogeneous commission
Different interests need to be balanced	▸ Patients demand access▸ Physicians want to use novel, promising therapies▸ Industries (researchers) want to sell products▸ Payers need to control costs▸ Federal office follows laws▸ Commissioners strive for correct decisions

Based on qualitative structured interviews (for details see methods).

CED, coverage with evidence development; FOH, Federal Office of Public Health; HTA, health technology assessment; NICE, National Institute for Health and Care Excellence; WZW, decision criteria (effectiveness, appropriateness and cost-effectiveness).

Discussion

This comprehensive overview on the use of CED in one country over nearly two decades illustrates a major challenge to HTA: institutional factors dominated the use of CED and final decisions. Granting access to novel but contested medical procedures via CED in Switzerland varied significantly over time; so did the result of final decisions. The factors identified as significantly associated with input to and output from the system were the organisational units of the commission, the heads of the Federal Department of Home Affairs and the concurrent head of the appraisal committee (ELGK/CFPP). The strong correlation between these three precluded further identification; still, change of the department did provide the most plausible explanation.

The role of politics in the decision-making process is not necessarily bad in a democratic country but it is a fact that should be recognised. There is little scientific research on this aspect, an exception being a recent qualitative analysis of expert interviews.9 The Swiss population demanded re-access in a referendum when alternative medicine failed to stand the test of evidence and was waived from the list of standard insurance benefits.23 In any solidarity-based or democracy-based Health Care System, participants should have the right to express their values. However, assessment and appraisal should be clearly separated.24

The absence of clear criteria for a CED decision and a definition of the decision pathway, the arbitrariness in decision-making, and the lack of scientific evaluation of what was performed were key comments from the interviews. In the related ordinance (KLV/OPAS), definitions varied, sometimes from one edition to the next. Additional requirements such as conduct of a registry or limitation to defined centres or specialists followed in part erratic patterns, despite the establishment of a series of handbooks for the commission. Impact of these instructions could not be assessed in this study. The specific label ‘no, in evaluation’ was abandoned without evaluation in 2004. It was considered to have no practical meaning, since providers or producers of any medical service had the possibility to resubmit a file as soon as new evidence was generated. Of note in this context, the initial decision ‘no, in evaluation’ for autologous haematopoietic stem cell transplantationin autoimmune disorders was crucial to obtaining research funds and did stimulate initiation of a multicenter prospective randomised study with an ultimately successful outcome.25–28

Similarly, the use of registry was required in 14 cases; no specific recommendations or support structures were linked to these requests. Unsurprisingly, information on status of registries was minimal at best, with few exceptions. With the introduction of a Swiss law on transplantation, reporting of all transplants to the Swiss registry and adherence to the quality management system ‘JACIE’ became mandatory in Switzerland in order to be reimbursed. Reporting was reimbursed as well.29 Hence, haematopoietic stem cell transplantation in Switzerland presents a successful model with comprehensive reporting and documented improvement in outcome.22 Changes from CED to acceptance (eg, multiple myeloma, autoimmune disorders) or rejection (eg, lung cancer or melanoma) were based on national and international scientific criteria. As a tool, CED could specifically apply to the emerging diagnostic and therapeutic services of personalised medicine, where standard phase III trials no longer suffice.30

The qualitative data of the structured interviews supported the quantitative findings. All persons asked confirmed the seriousness of the participants, the willingness ‘to do their best’ but were concerned about the erratic structure of the evaluation process. They felt informed about CED as an evaluation tool and strongly believed in the concept. They were convinced that CED did indeed permit early access to novel therapies for patients in need and generate new evidence. They expressed their concern about the lack of scientific and administrative criteria and the absence of evaluation of the evaluation process itself. They criticised in part the absence of academia from the HTA appraisal process. A review of the literature reveals that less than 10% of all the publications which were searched for the term ‘CED’ appear in general medicine journals, a minute amount in high-ranking medical journals.31 32 This lack of interest is historical and can vary between benign neglect and interest-driven aversion by the medical profession.33 In the Swiss context, no medical faculty in Switzerland holds a chair on HTA. This lack of interest of academia in HTA has recently been discussed.28

This report has limitations and weaknesses. We concentrated on publicly available material, for example, the KLV official publications. The inconsistency in these reports precludes an unbiased analysis. Some medical services were listed in a different format from year to year. For the sake of the analysis we defined them as presented. We did not evaluate the decisions with a direct ‘yes’ or ‘no’. We could not evaluate the potential impact of the internal learning process of the commission, of the related structural changes, or of changes in the Swiss Health Care System in general. The commission and related structures are in a constant learning process, which is highlighted by background documents available on the website of the FOPH. The setup of HTA and decision-making was audited by the parliament in 2008/2009 and is due to be developed further with the plans of the government to establish permanent structures for HTA and quality which at present are under public consultation. Still, some clear findings can be described and are useful when developing structures and processes further.

Conclusion

This analysis of 17 years of CED in Switzerland describes its potential benefits and deficiencies. The introduction of CED enabled access to some promising technologies early in their life cycle, and might have triggered the establishment of registries and research. Impact on patients’ outcome and costs remains unknown. Furthermore, CED increases the complexity of the decision-making process; CED recommendations should be made with care. They should follow internationally agreed principles8 and be integrated into a clear and structured process and consistent decisions. The primary association of institutional changes with measured end points illustrates the need for evaluation of the current HTA system.