Elicitation of Norwegian EQ-5D-5L values for hypothetical and experience-based health states based on the EuroQol Valuation Technology (EQ-VT) protocol.

INTRODUCTION: Norway is one of several European countries that lacks a national value set and scoring algorithm for the EuroQol five dimensions (EQ-5D). Recent studies have found differences between countries in terms of health values or preferences for health states described by instruments such as the EQ-5D. The project aims to model a national value set for the five level version of the EQ-5D based on values elicited from a representative sample of the Norwegian adult general population in terms of region, age, sex and level of education. Using a sampling strategy supporting the collection of values for both hypothetical and experienced health states, the study will have the additional aim of assessing the feasibility of collecting experience-based values in accordance with the latest EQ-5D valuation study protocol, and comparing values with those given for hypothetical health states. METHODS AND ANALYSIS: Multistage random sampling and quota-sampling will contribute to representativeness. To increase the number of valuations of experienced health states, those with less than perfect health will be oversampled, increasing the total number of interviews from 1000 to 1300-1500. The most recent EQ-5D valuation protocol will be followed which includes computer assisted face-to-face, one-to-one interviews and use of composite time trade-off and discrete choice experiments. ETHICS AND DISSEMINATION: The study has been reviewed and found to be outside of the scope of the ethics committee and thus not in need of ethical approval. The study findings will be disseminated through peer-reviewed publications, conference presentations and summaries for key stakeholders and partners in the field. The scoring algorithms will be available for widely used statistical software. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first Norwegian valuation study with composite time trade-off and discrete choice experiment undertaken on a scale large enough to meet the recommendations of the most recent EuroQol (EQ) five dimensions protocol.

Sampling strategy designed to both ensure representativeness of the final sample according to geographical region, age, sex and educational level and increase the number of experience-based valuations.

Data collection restricted to EQ protocol, primarily developed for hypothetical health state valuation, but with the additional aim of collecting experience-based valuations. Study design does not allow for the assessment of methods other than those described in the EQ-valuation technology protocol.

Restricted samples for comparisons of experience-based valuations.

High respondent burden experienced in interviews limits the scope for addressing additional methodological questions.

Introduction

Economic evaluation undertaken by the Norwegian Institute of Public Health (NIPH) and the Norwegian Medicine Agency increasingly informs decisions about the introduction of new drugs and other health technologies in Norway.1 2 The Norheim Committee3 and Magnussen Working Group4 proposed methods to enhance the quality of economic evaluation, thereby further strengthening the role of economic evaluation in decision-making. The Ministry of Health followed up these proposals in a 2016 White Paper to Parliament on principles for priority setting in healthcare.5

Given the important role and impact of economic evaluation, it is important that the methods it incorporates, including cost–utility analysis, are consistent with societal values regarding publicly financed healthcare. Economic evaluation, when taking into account societal values, often takes the form of cost–utility analyses with the estimation of the incremental cost per quality-adjusted life year (QALY) gained.6 QALY takes the integral of health-related quality of life (HRQoL) over time, with HRQoL represented on a scale where 1 indicates a preference equal to that for full health and 0 implies a health state equal to that of being dead. Values are typically derived using general population surveys where respondents consider the relative undesirability of different health states described using instruments such as the EuroQol five dimensions (EQ-5D).7 After assigning values to health states described by an instrument, QALYs are calculated by multiplying the health state value by the length of time spent in each. Evaluation of alternative technologies then involves comparison of incremental QALYs gained over incremental costs for new vs existing technologies.

Several instruments are available to calculate QALYs, of which the EQ-5D is by far the most widely applied both internationally and in Norway.8–10 The EQ-5D, a trade mark of the EuroQol Research Foundation, is available in over 150 languages11 in the self-complete paper version,12 and national value sets and normative data exist for over 20 countries.7 13–25 It is brief, widely tested, and includes five important aspects of health (mobility, self-care, usual activities, pain/discomfort and anxiety/depression), with the most recent version having five levels (5 L) from no problems to severe problems. The EQ-5D is considered highly acceptable to most patient groups and feasible for application where a short-form general measure of health is required. The instrument has had widespread application in research including clinical trials, population health surveys, in both Norwegian26 and Swedish National Quality Registries,27 and more recently as a healthcare quality indicator as part of the National Health Service for England and Wales Patient-Reported Outcomes Measures programme.14

The Norwegian Medicines Agency recommends the use of EQ-5D in all technology assessments and the use of a 5 L tariff for studies where the 5 L version has been used.2 In the absence of a Norwegian tariff, the 2018 Agency guidelines currently recommend the use of the EQ-5D-5L tariff for England14 where EQ-5D-5L has been used. However, criticism has been levelled at the English tariff including concerns with data quality in which serious deficiencies were revealed.28 The English 5 L tariff followed an early protocol, which has since been updated with the aim of improving data quality and interview techniques. Following these concerns, and in contrast to recommendations of the Norwegian Medicines Agency, National Institute for Health and Care Excellence continues to recommend the use of the 3 L tariff over the 5 L tariff, with 5 L values mapped onto 3 L where appropriate.29

The EQ-5D is widely used in Norway, including the national quality registers where it is the most widely used patient-reported outcome measure. A national EQ-5D-5L value set and scoring algorithm is highly anticipated and will enhance the validity of economic evaluations in Norway. Norwegian EQ-5D users have largely relied on the EQ-5D-3L scoring algorithm from the UK,30 with a crosswalk-based approach31 for studies that have used the 5 L version. Crosswalk-based approaches have several limitations related to issues with data dependency and differences in scale range, and are an interim solution pending a national 5 L value set.31–33 Cross-national comparisons of national EQ-5D-5L value sets also suggest that there might be substantial differences across countries13 34 with culture and values having a role.35 Values for health for the 5 L version of the EQ-5D, that are representative for the Norwegian general population, will enhance the validity and legitimacy of economic evaluation in Norway.

With few exceptions,36–38 existing EQ-5D value sets are based on the general population valuing hypothetical health states, which follows recommendations that economic evaluation should include societal preferences.39 In recent years, there has been some criticism levelled at this approach, questioning the validity of health state valuations from a general population lacking the adequate experience or knowledge of the health states which they value in the form of hypothetical health states.40 41 An alternative approach, as recommended by Sweden’s Dental and Pharmaceutical Benefits Agency,42 involves individuals valuing their own health state to give experience-based values or basing their valuations on other forms of experience. The debate on whether to use hypothetical or experience-based values is to a certain extent a normative issue, relating to what we aim to maximise.43 However, there are a number of empirical questions pertaining to experience-based values. Arguably, patients have a better understanding of the consequences of reduced health on quality of life.41 44–46 On the other hand, they may have trouble imagining life in full health, may under-report impact of disease due to adaptation or changes in expectations over time,44 47 or may be less inclined to value their current health state as a state that is worse than being dead. Experience-based valuations, if better understood and elicited from representative samples of the general population may, however, be suitable for inclusion as societal values. The feasibility of collecting experience-based values, the assessment of how those with less than perfect health value their current health state and other health states in general, and how different forms of experience may influence health state valuations, are new areas for research to which this study will contribute.48

The project will derive a Norwegian EQ-5D-5L value set representative of region, age, sex and level of education composition in the Norwegian adult general population. Furthermore, the study will allow for comparisons of experience-based and hypothetical health state valuation.

Methods and analysis

Values for EQ-5D-5L health states will be obtained by electronic data collection including computer assisted face-to-face, one-to-one interviews and the use of composite time trade-off (cTTO) and discrete choice experiments (DCE).49 50 The latest EQ-5D-5L protocol will be followed including EQ-valuation technology (EQ-VT).

Sampling

Respondents must be aged 18 years or older, resident in Norway and proficient in Norwegian. Following EQ-VT protocol, sample size is set to a minimum of 1000 individuals with each valuing 10 health states, which gives the recommended 10 000 responses.50 Additional 300–500 interviews, based on the oversampling of those with less than perfect health, will increase the number of valuations of experienced health states.

Norway is a Northern European country with a population of slightly more than 5 million, and a universal healthcare system. The population covers a comparatively large land mass, and for many there may be several hours travel time to the nearest hospital or large city. Urbanisation has further contributed to variation in demographic characteristics at the regional level. These factors combined with local culture, politics and traditions mean that geographical considerations are important to the design of the study.

The study will use a combination of multistage random sampling and quota sampling ensuring representativeness according to geography, age, sex and educational level. The first stage of sampling will be of geographical areas, here defined as municipalities within each acute care hospital catchment area. Norway’s four regional health authorities include Northern, Central, Western and South-Eastern, with more than half the population residing in the South-Eastern health region. The catchment areas served by the 54 acute care hospitals cover all Norwegian residents (see figure 1). They vary considerably in the number of residents that they serve, from 15 000 to 500 000 residents. One acute care hospital will be randomly selected from each health region with the exception of the South-Eastern region, where three will be randomly selected to account for the disproportionate number of people residing in this region. Hospital catchment areas within each region will be sampled with proportional allocation, ensuring equal probability proportionate to the number of people residing in each area within the region.

Figure 1

Hospitals with acute care function in Norway.

Within each sampled geographical area, the possible locations for data collection will constitute the sample frame for the second stage of sampling (table 1). Locations will include public places (eg, public libraries, town halls), workplaces, recreational organisations (eg, sports clubs) and healthcare providers (eg, hospitals, rehabilitation institutions). The bodies concerned must be willing to grant the study permission for data collection and cooperate with provision of a suitable space for completion of the interviews. The locations will act as clusters of possible respondents, stratified into groups based on the characteristics of target respondents, for example, age and educational level. Stratification will increase homogeneity per cluster and ensure the representation of specific groups less likely to participate including those with poorer health, lower socioeconomic status, or faced with time constraints, including those with young children or in full-time employment.51 Locations within each group in the sample frame will be randomly selected. The number of locations selected within each sample frame will be based on the size of the area and quotas. Response rates, recruitment and data quality will be assessed for the different location strata and compared across catchment areas.

Table 1

Locations for recruitment of participants, by age group and health status

Healthy				Reduced health
Young	Middle aged	Elderly	All ages	All ages
Places of higher education	Workplaces	Eldery homes	Public library	Hospitals
Child daycare facilities/Primary schools	Recreational organisations (sports teams)	Recreational organisations (choirs/orchestras)	Town hall	Rehabilitation centres
Social welfare*	Social welfare*	Community volunteer centres		Health centres
	Adult education*

*Locations chosen to increase participation of those with lower socioeconomic status.

Within each catchment area and at the respondent level, quota sampling will be applied according to age, sex and level of education (see tables 2 and 3). The total quota will first be allocated to each region proportionate to the number of people residing in each region. For the three regions with one sampled hospital catchment area, the quota for each of these hospital catchment areas will correspond to the regional quota. In the South-Eastern region, the regional quota is further allocated to each hospital catchment area proportionate to the number of people residing in each of these areas. The quota is then allocated to groups according to gender, age group (young adults: age 18–34, middle-aged adults: age 35–64, elderly: age 65+) and level of education (lower education, no higher than high school education, higher education-bachelor, masters or PhD) equivalent to the distribution of these attributes in the respective regions. The quotas for each group are calculated using data available from http://microdata.no (see table 4), a national platform in Norway giving researchers direct access to national registries for which Statistics Norway has processing authority, such as the Norwegian National Registry, National Education Database, labour market data, register for Personal Tax Payers and FD-Trygd (event history database).52

Table 4

Reference data for the calculation of quotas, data for 2018 (http://microdata.no, statistics Norway, data accessed: 12 March 2019)

Region	Sex	Highest attained educational level	Age group
			18–24	25–34	35–44	45–54	55–64	65–74	75+
South-Eastern region	Male	Primary or secondary	117 220	130 448	133 470	143 252	119 278	94 473	62 167
		Tertiary	13 603	72 661	77 273	66 785	51 553	40 368	19 650
	Female	Primary or secondary	100 571	94 904	99 033	120 226	114 228	107 739	103 859
		Tertiary	24 196	104 395	101 833	79 908	55 565	34 126	17 126
Western region	Male	Primary or secondary	48 863	54 616	52 141	54 172	44 925	34 032	23 977
		Tertiary	5129	26 041	27 176	21 446	17 552	12 302	5291
	Female	Primary or secondary	40 743	35 932	34 778	42 701	40 672	36 866	38 127
		Tertiary	9928	39 550	36 796	27 107	18 494	9777	4750
Central region	Male	Primary or secondary	32 425	33 771	32 095	36 110	32 525	26 289	18 441
		Tertiary	3674	15 730	15 703	13 497	11 291	8664	3521
	Female	Primary or secondary	26 707	21 526	21 130	28 292	29 998	28 218	28 275
		Tertiary	6456	23 177	22 577	18 320	12 267	6980	3024
Northern region	Male	Primary or secondary	22 976	23 320	21 793	25 812	23 582	20 282	13 464
		Tertiary	1736	7895	8724	9427	7450	5273	1845
	Female	Primary or secondary	18 357	15 382	14 562	19 478	20 589	20 492	19 767
		Tertiary	3470	13 212	14 402	13 721	8872	4300	1707

Table 2

Example sampling of hospital catchment areas and quotas per catchment area

Region	Population in region	Catchment area	Population in catchment area	Quota per catchment area
Northern	381 907	Hospital 1	130 000	140
Central	560 690	Hospital 2	60 000	205
Western	843 899	Hospital 3	330 000	309
South-Eastern	2 299 890	Hospital 4	500 000	448
South-Eastern	‘’	Hospital 5	160 000	143
South-Eastern	‘’	Hospital 6	280 000	251

The study will largely rely on recruitment of potential participants by contact persons at each sampled location. Contact persons will assist in identifying and recruiting potential respondents to the study. Prior to data collection, contact persons will receive information and materials for publication in local newspapers and social media designed to enhance participation. In addition to the recruitment of respondents through locations, potential respondents will be able to contact the project group for more information about the study and enquire about participation. Potential respondents will be informed of a gift incentive. Cash has been found to be more effective than other incentives for increasing response rates and following the interview, respondents will receive a cash card equivalent to €30.

Data collection will take place from November 2019 to June 2020. Depending on the final sampling, and with an estimate of a minimum of four interviews per interviewer per day, a minimum of 55–80 working days are required for data collection. The recruitment strategy will be piloted in the catchment area sampled closest to Oslo. Necessary adjustments will follow before data collection proceeds in the rest of the country.

Interviewer training

Interviewers with Masters education level or equivalent will receive training in accordance with EuroQol Foundation guidelines and recommendations, with initial training prior to, and revised training after, the first phase of data collection.53 Based on existing studies and recommendations from EQ (Elly Stolk, personal communication), eight to twelve interviewers are required.

Quality control (QC) reports will help monitor progress and data quality throughout.54 55 The reports will include assessments of protocol compliance, face validity of data collected and value distributions per interviewer. QC reports have been found to further the homogeneity of interviewer performance and reduce protocol violations and the number of inconsistent responses.54 Interviewers not meeting predefined standards will be flagged, recommended for retraining and ultimately excluded. Evaluation of the data collected and interviewer performance will be regularly discussed with interviewers in face-to-face group meetings throughout data collection, and with EQ contact persons.

EuroQol valuation technology

EQ-VT was developed to meet the challenges involved with valuation of the 5 L version of the EQ-5D, with emphasis on improving data quality and cross-country comparability.49 The standard protocol includes digital representation of visual aids to assist the respondent throughout the interview (see figures 2 and 3). The study will use the portable version of the software, EQ-PVT, which for the respondent has the same functionality and for the most part resembles the standard EQ-VT software package.

Screenshot of visual aid for cTTO task in EQ-VT. cTTO, composite time trade-off; EQ-VT, EuroQol valuation technology.

Screenshot of visual aid for DCE task in EQ-VT. DCE, discrete choice experiments; EQ-VT, EuroQol valuation technology.

The interview will start with administration of the EQ-5D-5L questionnaire, including the Visual Analogue Scale (VAS), followed by background questions for age, sex and experience with serious illness. Next, cTTO is administered, beginning with an explanation of the task demonstrated with ‘the wheelchair example’ including an introduction to the ‘worse than dead’ part of the task. This is followed by practice tasks for three states described with the EQ-5D-5L descriptive system, selected to reflect a mild, moderate and severe health state, familiarising the respondent further with the cTTO, the concept of health states worse than being dead and the use of lead-time in the cTTO for the valuation of such states. Lastly, respondents are administered their current health state as a cTTO task, allowing for the comparison of how respondents value their own health state with both cTTO and VAS.

Respondents are randomised to 1 of 10 TTO blocks of EQ-5D-5L health states, each consisting of 10 health states, one of which is always the worst state (level 5 on each dimension, state 55555), and one among the 5 mildest states (11112, 11121, 11211, 12111 and 21111), for a total of 86 unique EQ-5D-5L health states for direct valuation.49 Respondents get the opportunity to review their responses in a feedback module (see figure 4), where individual task responses can be removed. On completion of the TTO tasks, respondents are randomised to 1 of 28 state pair blocks for discrete choices, each block consisting of seven state pairs. In both the TTO and DCE parts of the interview, the order of presentation is randomised. The randomised TTO and DCE tasks do not explicitly include a valuation of the respondents own health state, however, respondents can by chance be presented their own health state as a choice, in which case the task will be completed as normal.

Figure 4

Screenshot of the feedback module in EQ-VT. EQ-VT, EuroQol valuation technology.

The interview ends with further background questions specific to this study relating to variables known to be associated with valuations of health states including caregiver status, educational level and marital status.56–59 The influence of such variables will be assessed for the final value set.

Analysis

The demographic characteristics and health status, that is, EQ-5D-5L profile, of respondents will be assessed and compared with national data. Parallel to this study, the NIPH has initiated data collection for a postal survey assessing the health status of the Norwegian population using the EQ-5D-5L, allowing for comparison of the health status of study populations. Health state values for EQ-5D-5L will be estimated through statistical modelling of the survey data. The EQ-5D protocols are not prescriptive with regard to modelling and approaches will depend on the characteristics of the data obtained.50 Following previous research, different models will be assessed including either the cTTO data, or combining the cTTO and DCE data in a hybrid model, and the results compared for adequacy with those for existing national value sets.14–25 Modelling of values for the national value set will exclude valuations from respondents recruited from locations specifically for the collection of experience-based values and the valuations of respondents’ own health state. Subgroup analysis will identify variables contributing to health state valuation in the Norwegian population. Values for health states defined as respondents’ ‘own health today’ will be compared with values estimated for the same health states by the general population. In addition, all experienced-based valuations by those with serious illness and/or less than perfect health will be compared with valuations based on the total general population sample and, given sufficient data, those without experience of serious illness and/or with perfect health today. To assess experience-based valuations, and explore both the wider and more narrow concepts of experience-based valuations,48 three potential profiles will be assessed: (1) respondents’ valuation of own health state, (2) valuations given by respondents recruited from locations specifically chosen to target those with poorer health, that is, health services, (3) valuations given by respondents who have indicated that they have experience with serious illness.

Patient and public involvement

Patients and members of the public were not invited to comment on the study design or contribute to the writing or editing of this document for readability or accuracy.

Strengths and limitations

This is the first Norwegian valuation study with both cTTO and DCE undertaken on a scale large enough to meet the most recent EQ-5D protocol. The study intends to complete 1300–1500 face-to-face computer-assisted interviews across a country with a relatively dispersed population of citizens and large geographical distances between them. Data collection involves a small number of interviewers working over an 8-month period.

Both the duration and magnitude of the tasks involved make the interview demanding. It is important that data collection is cost-effective, which includes considerations of data quality, representativeness and total number of valuations. Given the strategy of sampling locations and organisations rather than individuals, the assessment of its effectiveness in terms of number and representativeness of respondents will be important following the initial data collection period. Poor recruitment and data collection in remote geographical locations will be costly. The number and characteristics of respondents per location will be monitored throughout data collection. Adaptive sampling will allow for inclusion of additional locations where response rates are low and quotas are not met. Additional locations will be chosen at random from the predefined frame of possible locations within the selected geographical area.

Due to the need for extensive training, interview experience and understanding of the task, only 8–12 interviewers will be included. This will give more control over the data collection and the quality of the data collected. However, this and potential costs saved in terms of interviewer recruitment, training and travel costs, must be balanced against the increased impact of any loss of interviewers through illness or resignation during data collection. Norway has a harsh winter climate and apart from the Southern and Eastern region, where the interviewers are based, the interviews will primarily take place outside the winter months to reduce the risk of travel delays and interviewer illness. The NIPH, which is conducting the research, has several experienced interviewers familiar with the study who will be able to complete training and contribute to data collection if needed.

The main justification for the strategy of sampling stratified locations and the use of quotas on the respondent level is to ensure representativeness of the final sample according to geographical region, age, sex and educational level. An additional quota will be used to recruit those with less than perfect health, through locations such as hospitals and rehabilitation centres. Locations will also be selected to directly seek out others who are typically harder to reach and are less likely to participate in research studies, such as those with different ethnic backgrounds or with young children. Studies have found that some attributes, such as marital and caregiver status/having young children, may influence the respondents response to the task, such as their willingness to trade time in the TTO task, despite showing similar preferences for given health states when using other types of valuation tasks.56 58 Hence, it is important that respondents with such attributes are included in the study and locations such as day care facilities for young children and primary schools will be selected to facilitate this. Questions relating to these attributes will be included in the background questions closing the interview, and as such will allow for subgroup analysis of the effect of these attributes on the valuation of health in the Norwegian sample.

The derivation of values based on experienced health states is a recent development in the field of health state valuation.48 In recent years, there have been major developments in the field of standardised protocols for health state valuation, including EQ-VT. Such standardisation is a long way off for experienced health state valuation and, as was the case for hypothetical health state valuation up until the last decade, there is considerable variation in the choice of methods.60 In Norway and other countries, the feasibility of collecting such data is still in its infancy, including choice of sampling strategies, recruitment and how to minimise respondent burden. This study builds on existing methodology in the form of EQ-VT protocol, to assess the feasibility of recruiting potential respondents (including from healthcare settings) for experience-based health state valuation, respondent burden in the form of completed interviews and data quality. The study design is constrained by the EQ-VT protocol, but the results of the study will inform the development of more appropriate methodology in the future. Furthermore, the design will allow the comparison of results with those for hypothetical health state valuation.

Ethics and dissemination

The study was reviewed by The Regional Health Authority Research Ethics Committee and found to be outside of the scope of the ethics committee thus not in need of ethical approval. All study participants will give informed consent.

The final scoring algorithm will contribute to the quality and relevance of the results of EQ-5D applications in Norway, and it is highly likely that, when available, the EQ-5D-5L with a Norwegian scoring algorithm will be the recommended instrument of choice for future economic evaluations undertaken in Norway by the pharmaceutical industry and other important users. Application of the same instrument and scoring across the health services and industry will further enhance decision-making relating to scarce healthcare resources. Moreover, scores based on Norwegian preferences will further enhance the appropriateness of the EQ-5D in clinical and health services research and quality indicators work, including the national quality registers.

The study results will be published in peer-review scientific journals, presented at appropriate forums, including national and international conferences, and scoring algorithms made publicly available for R, Stata and other widely used statistical software. Presentations will be given to users of the research, including research centres that widely use the EQ-5D in clinical, health services and health economics research in Norway.

Table 3

Example of quotas within a sampled catchment area based on the composition of sex, age and educational level in the general population of the respective region (source: official statistics for 2017 generated from microdata.no)

Sex	Highest attained educational level	Age groups							Total quota per sex and educational level
		18–24	25–34	35–44	45–54	55–64	65–74	75+
Male	Primary or secondary	8	9	8	9	9	7	5	56
	Tertiary	1	3	3	3	3	2	1	16
Female	Primary or secondary	7	6	5	7	8	8	7	47
	Tertiary	1	5	5	5	3	2	1	22
Total quota per age group		17	22	22	25	22	18	14	140

Example given sampling scenario and catchment area for Hospital 1 in table 2.