Valuation of EQ-5D-3l Health States in Slovenia: VAS Based and TTO Based Value Sets.

INTRODUCTION: The two primary objectives of this paper were (a) to develop first logically consistent TTO based EQ-5D-3L value sets for Slovenia and (b) to revisit earlier developed VAS based EQ-5D-3L value sets.
METHODS: Between September 2005 and April 2006, face-to-face interviews with 225 individuals in Slovenia were conducted. Protocols from the Measurement and Value of Health study were followed closely. Each respondent valued 15 health states out of a total of 23. Model selection was informed by the criteria monotonicity/logical consistency. Predictive accuracy was assessed in terms of mean square difference between out-of-sample predictions and corresponding observed means, as well as Lin's Concordance Correlation Coefficient.
RESULTS: Modelling was based on 2,717 VAS and 2,831 TTO values elicited from 225 respondents. A 6-parameter constrained regression model with a supplementary power term was selected for VAS and TTO value sets, as it produces monotonic values, and proved superior in terms of out-of-sample predictive accuracy over the tested alternatives.
CONCLUSION: This is the first EQ-5D-3L TTO based value set in Slovenia and the second in Central and Eastern Europe (besides Poland). It is also the first monotonic and logically consistent VAS value set in Central and Eastern Europe. Comparisons with Polish and UK TTO values show considerable differences, mostly due to mobility with having a substantially greater weight in Slovenia. The UK value set generally produces lower values and the Polish value set higher values for mild states.

Introduction

Slovenia passed the regulation that required economic evaluation to inform drug and health technology reimbursement decision-making in the 1990s. Health technologies are assessed by various bodies (1, 2). The latest evaluation guidelines by the Health Insurance Institute of Slovenia recommend that the benefits of the treatment are expressed as quality-adjusted life years (QALYs).

QALY is a measure that encapsulates a treatment’s impact on a patient’s life length and also on their health-related quality of life (HRQOL), which is recognized as a key indicator of treatment outcomes (3). The QALY requires data that expresses health-related quality of life (HRQOL) in the form of a single value, sometimes known as a health state utility value, which is scored on a scale that assigns a value of 1 to a state equivalent to full health and 0 to a state equivalent to death (4). Weinstein and Stason (1977) connected QALYs with utilities, specifically expected utility, rather than the “weights” of the earlier literature; and this connection has remained although, not everybody agrees with the concession of the term “quality” to refer only to expected utility-based measures (5). Anyhow, in health economics, utilities (values) are typically combined with survival estimates and aggregated across individuals to generate quality-adjusted life years (QALYs) for use in cost–utility analyses of healthcare interventions (6).

There are many methods available regarding how the health states can be valued and grouped into two broad categories of measures: direct and indirect methods of measurement. The direct valuation methods include standard gamble (SG), time trade-off (TTO), DCE (discrete choice experiment), rating scales, equivalence technique, ratio scaling and person trade-off. The SG approach is the classic method of measuring preferences in economics under conditions of uncertainty, and is based on von Neumann Morgenstern utility theory (7). The theoretical underpinnings of all other methods are less clear. TTO valuation methodology does not conform to utility-under-uncertainty requirements under expected utility theory, but is still a dominant method in the valuation sets across countries (8). Regarding VAS values, there are a lot of criticisms and opposing views on their suitability for use in cost utility analysis. Mostly, criticisms consist of VAS values not being choice based and their lack of theoretical foundation (5). Due to these issues, most health economists would recommend a choice-based value set, derived from TTO or DCE data, especially for economic studies where cost-utility analysis is anticipated. If a choice-based value set is not available for the country/region, a choice-based value set can be selected from a country/region that most closely approximates the country where the study is being conducted. Alternatively, a VAS-based value set can be used if that is available for the country/region (9). Due to these issues, most health economists would recommend a choice-based value set, derived from TTO or discrete choice experiment (DCE) data to be used in studies that estimate the value of health states of any population. If a choice-based value set is not available for the country/region, a choice-based value set can be selected from a country/region that most closely approximates the country where the study is being conducted. Alternatively, a VAS-based value set can be used if that is available for the country/region (9).

Utilities (values representing preferences) for healthcare priority setting are typically obtained indirectly by asking the general population (or patients) to fill in a questionnaire and attach value to hypothetical heath state, later on converting the results to a value set for all health states, using population (or patient) values. There are at least two advantages that contributed to the popularity of the indirect methods: the pool of health states is already defined and so are their values (value set). When a patient defines his own health state in subsequent studies, a value can thus be attached to his/her health state from the value set.

Some of the established questionnaires are the Health Utility Index, the Short Form 6D, 15D instrument, Assessment of Quality of Life (AQOL) and the EuroQol 5D (EQ-5D). The EQ-5D is a prominent example of preference-based measures developed by the EuroQol Group (9). It has been suggested that these are the most widely used preference-based measures in the world (10). To improve the instrument’s sensitivity and to reduce ceiling effect, EuroQoL Group developed a new version in 2009, called EQ-5D-5L. The new version kept its original 5 dimensions, but expanded the response options from 3 to 5 levels. As there are a lot of existing 3L value sets in many countries, for comparison reasons a non-parametric model was set up to transform any EQ-5D-3L values into EQ-5D-5L values. In this way, 5L values can also be used in cases when 5L preferences directly elicited from representative general population samples are not yet available (11).

The EQ-5D-3L descriptive system has been formally translated and validated into the Slovenian language in 1999/2000 (12). The two primary objectives of this paper were (a) to develop first logically consistent EQ-5D-3L TTO-based value sets for Slovenia and (b) to revisit earlier developed VAS-based EQ-5D-3L value sets for Slovenia (13). Some issues that went undetected with the previous VAS value set have been identified, and methodological advances seem to make it possible to improve on earlier modelling.

Methods

Study Overview

The study was a multicentre, population-based study, using face-to-face interviews. The sample was prepared by the Statistical Office of Slovenia using the Central Population Register. In the sample, 1,000 individuals aged 18+ from 40 Slovenian municipalities were included. At the first level, 40 municipalities were randomly selected and later on 25 individuals were selected from each municipality. Each person carried a name, last name, address, house number, postcode, municipality, age and gender. The investigators started the interviews in September 2005 and finished in April 2006. Participant recruitment was conducted primarily through landline telephone numbers for each participant in the sample. 225 participants agreed to participate in the survey. Interviews were conducted by three interviewers, who underwent one-day training on the health state valuations, the purpose of the interviews and TTO procedures. To facilitate the training, the interview book prepared by Gudex (14) was translated into Slovenian language and used for training of the interviewers and in the pilot training. Each investigator conducted 5 test interviews.

EQ-5D

EQ-5D consists of a descriptive system and EQ visual analogue scale (EQ-VAS). The EQ-5D descriptive system consists of 5 dimensions: mobility (MO), self-care (SC), usual activities (UA), pain/discomfort (PD), and anxiety/depression (AD). Each dimension has 3 levels: no problems, some problems, and extreme problems (9). The respondent is asked to indicate his or her health state by ticking the box that marks the most appropriate level of problems in each dimension. A unique health state can be described by using a 5-digit vector formed according to the responses to the 5 questions. For example, no problems in MO and SC, some problems in UA and PD and extreme problems in AD can be referred to as “11223.” Health states defined in this way may be converted into a single summary index by applying a formula that attaches values to each of the levels in each dimension. A total of 243 possible health states can be defined.

Health State Selection

In the valuation task, each investigator had 3 sets of health states, and investigators decided randomly which set to use with each respondent. The sets were named A, B and C. Each set contained 15 health states. Some health states were included in all three sets, but some were not. Health states in each set represent the complete scale of health states, from worst to best health states. Sets B and C were developed in 2000 (16). The number of all various directly valued health states in all three sets is 23 plus unconscious and dead. These states are 11211, 11111, 21111, 12111, 11112, 11121, 11122, 11113, 11131, 11133, 11312, 13311, 32211, 22222, 21232, 22323, 22233, 32223, 32313, 23232, 33321, 33323, 33333, unconscious and dead. Health states can also be divided into mild, moderate and severe states (17) in such a way that all the categories were represented in all three sets.

Interview Process

The questionnaire consists of four parts. In the first part, the respondent indicated his/her own health state on the day of the interview using an EQ-5D descriptive system. Furthermore, the respondent marks how good or bad his/her health state is on a visual analogue scale (VAS) from 0 to 100 (where 0 represents the worst health state imaginable and 100 represents the best health state imaginable).

The second part of the questionnaire is a valuation of the 15 selected health states. Once the respondent had familiarised himself/herself with the health states by reading them, he/she ranked the selected states from worst to best. After ranking he/she attached the value from 0 to 100 to all 15 health states.

The third part of the interview is the valuation of the same selected 15 health states using time trade off (TTO) method. The interviewers follow the adapted Measurement and Value of Health study (MVH) protocol (14). The MVH study was a large exercise, in which 3,395 respondents valued 13 different health states. Because of the limited budget, we included 23 health states altogether. Out of 23 health states, we made three different sets of 15 health states (sets A, B and C) as described in Chapter 2.3.

The objective of the TTO is to determine the length of lifetime the respondent would be willing to forego to live in a better health state (typically ‘full health’) and to avoid living in a bad health state. This is achieved by presenting respondents with a series of choice tasks, each involving two alternative hypothetical lives. The two lives are presented so that the respondent is forced to choose between a longer life in the health state of interest and a shorter life in better health (15).

The last part of the interview collects social demographic data: gender, age, education, work experience, smoking habits, experience with illness and postcode.

Preference Elicitation Techniques

In the TTO procedure, the interviewers used a TTO board and a set of health state cards. A TTO board was made of three layers of thick cardboard and incorporated a sliding scale from 0 to 10 years. Both sides of the board were used, one for states better than dead and the other for health states worse than dead. The respondent was taken through each of 15 health states to be valued, one at a time, with the interviewer moving the scale as appropriate. The respondent needed to make a series of choices between two hypothetical lives: one involving x years of healthy life, followed by death (Life A) and the other involving t years in a worse health state (where x≤t), followed by death (Life B). Time t was fixed, whereas time x was varied until the respondent reached their point of indifference. This iterative procedure continued until the respondent was unable to choose between the two lives. In our study, the respondent started with a choice between living Life A (health state 11111) for 10 years followed by death and living Life B (worse health state) for 10 years followed by death. Life A was chosen – the next choice was between Immediate Death (x=0 for Life A) and 10 years of Life B, followed by death. In the next choice, x was set at 5 years; in case Life A was chosen, x was decreased and the opposite until the point of indifference was found. The value of Life B was calculated according to how much healthy time the respondent was willing to forgo at this point of indifference – the utility value of the health state was at this point calculated as x/10. In case of states worse than dead, the respondent was again given two alternatives, but this time Life A was a combination of health state I for y number of years followed by full health for x number of years (x+y=10), followed by death. Life B was a certain outcome of immediate death. Time x was again varied until the respondent was indifferent between both alternatives. At this point, the utility value for health state I was calculated as –x/(10-x). Respondents were allowed to trade time in months and weeks.

In VAS procedure, the respondents ranked the health states and, in the second phase, attached them a value from 0 to 100. VAS values were later rescaled using the mean observed values for state 11111 and death. Health state “Unconscious” was not used.

Statistical Analyses

Historically, values for EQ-5D-3L have been modelled by use of ordinary least squares (OLS) regression, using dummy variables representing the presence or absence of different levels of problems on each of the five dimensions. Built on this framework, different interaction terms have been added in different national valuation studies. More recently, the introduction of the EQ-5D-5L has resulted in a range of innovations in terms of modelling, including hybrid models combining TTO and DCE data (24), random intercept models, censored/interval regression to account for censoring at −1, and use of constrained, non-linear regression models (18, 22, 23).

For EQ-5D-3L, the standard, additive 10-parameter model, hereafter referred to as ADD10, has parameters representing levels 2 and“ 3 for each dimension. Let y represent the observed disutility of a health state, represent x the dummy variable indicating the presence of problems on dimension d at level l and β the coefficient representing the estimated disutility of having problems on dimension d at level l (e.g. β3 representing the disutility of having moderate problems on mobility). The mathematical function of ADD10 is as follows:

An EQ-5D-3L variant of the constrained model approach used in the Chinese and Malaysian EQ-5D-5L valuation studies employs six primary parameters: one for each dimension, representing the disutility of having problems at level 3 (β), which for level 2 are multiplied by parameters for level 2 (L2). Thus, the disutility of having moderate problems on mobility is β × L2. The mathematical function of this model, hereafter MULT6, is as follows (note that x still represents the dummy variable representing the presence of problems on dimension d at level l):

This constrained model assumes that the relative severity of level 2, “moderate problems”, is similar across dimensions. This assumption reduces the number of parameters to be fitted, and thereby provides more robust results than unconstrained models, particularly with smaller samples of data.

We tested the ADD10 and MULT6 models, with and without the inclusion of a constant term (intercept) representing any deviation from full health. The model variants including intercept are denoted with an “i”, e.g. ADD10i. We also tested an extension of MULT6 in which the full expression is exponentiated by a separately fitted parameter P:

This model, hereafter MULT6P, was included under the assumption that respondents may display diminishing sensitivity to health problems when combined, so that the perceived disutility of problems on two separate dimensions at the same time may be smaller than the sum of the disutility of each problem in isolation.

Standard error estimation is non-trivial in regression models involving multiplication of two or more (presumably normally distributed) parameters. Consequently, standard errors for model parameters and modelled values were estimated for MULT6 and MULT6P using bootstrapping (22). Briefly, 10,000 bootstrap samples were drawn (with replacement) at the level of individual study participants, each subsample of the same size as the observed data. The regression models were fitted to each bootstrap sample, and standard errors were calculated by taking the standard deviation for the resulting coefficients and the predicted health state values.

Given the limited number of valued health states, and the relatively small sample size used in this study, we were concerned that regular regression models might produce results that were highly susceptible to random error. We, therefore, tested the included model variants using penalised regression, including Lasso (20), Ridge regression (17,18,19), and Elastic net (21).

Model selection was informed by two primary criteria, being monotonicity/logical consistency. Modelled state values should reflect the hierarchical structure of the EQ-5D descriptive system, so that further problems on any dimension should always result in worse (lower) values. Monotonic models were compared in terms of out-of-sample predictive accuracy for observed means. This was compared using leave-out-by-state cross-validation (18, 22). Predictive accuracy was assessed in terms of mean square difference between out-of-sample predictions and corresponding observed means, as well as Lin’s Concordance Correlation Coefficient.

The final Slovenian TTO model was compared visually to the Polish EQ-5D-3L value set (25) and the UK MVH value set (26), and the final VAS model was compared visually to the EU VAS value set (27).

All statistical analyses were performed in the R statistical package, version 3.3.2, in the RStudio environment, using ggplot for graphical output (28,29,30). Regression models were run in the xreg package (31).

Results

In total, 225 respondents completed the interview, of which 126 (56%) were female. Distribution of the respondents according to social and demographic variables is shown in Table 1. The sample was well representative of the Slovenian population in terms of age, educational level and activity with students being slightly underrepresented and unemployed being slightly overrepresented. Regarding gender distribution, women were overrepresented in the sample. The majority of problems reported in the EQ-5D descriptive system were pain/discomfort, followed by problems with usual activities and mobility. A really small share of the sample had problems with self-care (9.3%). The mean health state recorded on the EQ-VAS was 72.15 (SD 20.2) and the mean estimated interview difficulty was 2.87 (1 is very easy and 5 is very difficult).

Table 1

Study sample characteristics compared with the Slovenian general population data 2005.

Group	Mean pre-test scores (SD)	Mean post-test scores (SD)
Age
18–24	27 (12%)	190,239 (11.5%)
25–34	48 (21.3%)	300,793 (18.2%)
35–44	43 (19.1%)	304,490 (18.5%)
45–54	39 (17.3%)	310,757 (18.9%)
55–64	28 (12.5%)	229,580 (13.9%)
65+	40 (17.8%)	312,874 (19%)
Mean age (SD)	45.3 (17.4)	n/a
Gender
Male	99 (44%)	981,465 (49%)
Female	126 (56%)	1,021,893 (51%)
Educational level
Primary	53 (23.6%)	494 (28.8%)
Secondary	147 (65.3%)	952 (55.5%)
High	25 (11.1%)	267 (15.6%)
Work
Employed	111 (49.3%)	813,100 (47.3%)
Retired	62 (27.5%)	529,622 (30.8%)
Housewife	8 (3.6%)	n/a
Student	20 (8.9%)	112,228 (6.5%)
Unemployed	18 (8%)	92,575 (5.4%)
Other	6 (2.7%)	n/a
EQ-5D dimension problems (%)		n/a
Mobility	68 (30.2%)
Self-care	21 (9.3%)
Usual activities	69 (30.7%)
Pain/discomfort	101 (44.9%)
Anxiety/Depression	64 (28.4%)
EQ VAS own health (SD)	72.15 (20.2)	n/a

Cross-validation fit statistics can be found in Table 2. The fitted parameters of ADD10 were not monotonic. MULT6 and MULT6P with no intercept were monotonic for both VAS and TTO, while the version with an intercept was monotonic for TTO only. MULT6 displayed poor fit, both in direct estimation and in cross-validation. Ridge regression improved out-of-sample predictive accuracy for ADD10 and MULT6, but not for MULT6P. By comparison, MULT6P had substantially improved fit, outperforming all other tested variants in terms of out of sample predictive accuracy, both for VAS and TTO data. MULT6P with an intercept did not improve predictions for TTO, and did not converge for VAS. MULT6P was selected for generating VAS and TTO value sets. Coefficients and bootstrap-based SE estimates for the two models can be found in Table 3.

Table 2

Cross-Validation fit statistics.

TTO	ADD10	ADD10i	MULT6	MULT6i	MULT6P	MULT6iP
Monotonicity	-	-	✓	✓	✓	✓
R	0.920	0.941	0.934	0.930	0.966	0.966
CCC	0.894	0.938	0.893	0.929	0.966	0.966
MSE	0.046	0.022	0.048	0.024	0.012	0.012
MAE	0.181	0.114	0.182	0.126	0.087	0.087
VAS	ADD10	ADD10i	MULT6	MULT6i	MULT6P	MULT6iP
Monotonicity	-	-	✓	✓	✓	-
R	0.926	0.919	0.891	0.923	0.971	-
CCC	0.879	0.897	0.886	0.883	0.941	-
MSE	0.02	0.015	0.015	0.021	0.01	-
MAE	0.123	0.094	0.096	0.102	0.082	-

R – Pearson’s correlation coefficient, CCC – concordance correlation coefficient, MSE – mean squared error, MAE – mean absolute error

Table 3

Coefficients and bootstrap-based SE estimates.

	TTO		VAS
	Coefficient	SE	Coefficient	SE
MO	0.943	0.126	0.424	0.070
SC	0.243	0.052	0.105	0.029
UA	0.202	0.039	0.103	0.028
PD	0.448	0.049	0.180	0.012
AD	0.239	0.037	0.137	0.021
L2	0.125	0.043	0.176	0.025
P	0.551	0.044	0.423	0.020

Discussion

The Slovenian VAS and TTO based value sets are presented in Annex 1 and 2. The first VAS value set for Slovenia was calculated back in year 2000, however, the values of the health state were not monotonic: some of the logically superior health states displayed lower values (12). In 2012, a new improved set was published (13), however, again due to some methodological issues discovered later, it cannot be recommended for Slovenia’s priority setting. With the advanced methodology, for the first time in Slovenia it was possible to obtain a logically consistent and monotonic VAS based value set as well as a 3L TTO based value set, which is also the second 3L TTO based value set in Central and Eastern Europe.

Annex 1

Slovenian EQ-5D-3L VAS value set.

Annex 2

Slovenian EQ-5D-3L TTO value set.

Figure 1 displays the TTO value set compared to observed mean values, along with TTO-based values from a UK MVH study and the Polish TTO-based EQ-5D-3L valuation study. Figure 2 presents the Slovenian VAS value set, observed mean values, and the EU VAS value set.

Figure 1

Graphical comparison of Slovenian EQ-5D-3L TTO value set versus (a) UK TTO and (b) Polish TTO value sets.

Figure 2

Graphical comparison of Slovenian VAS value set versus EU VAS value set.

For the TTO value set, there are two main drivers of differences between the national value sets: first, mobility has a substantially greater weight in the Slovenian value set. Second, the UK value set produces lower values for mild states, while the Polish value set produces higher values. The VAS value set is more in line with the EU VAS value set, but generally produces somewhat higher values. Due to considerable differences between TTO value sets in Slovenia in comparison to the UK and Poland, its use should be recommended for Slovenian studies.

After testing various modelling approaches, the Slovenian TTO and VAS value sets were fitted using a 6-parameter constrained regression model with a supplementary power term, which produces monotonic values and was superior in terms of out-of-sample predictive accuracy over the tested alternatives.

The Slovenian TTO-based value set, being a choice-based method, is recommended for use in all studies, including economic analysis. Systematic pairwise comparison across all conditions and value sets in previous studies (32) revealed the greatest differences between the TTO and VAS-based value sets, as well as the varying sensitivity of the disease burden evaluations of chronic disease conditions to the choice of value sets. Therefore, using a VAS value set in the presence of newly developed TTO value set in Slovenia would unnecessarily produce incomparable results. However, in order to allow for comparisons with previous studies where VAS values were used due to the absence of a TTO-based value set, it is suggested to present VAS-based results in parallel. Another option is also the presentation of the results in parallel with the UK TTO value set, given that it has been the most used value set in the region (33).

Further analysis of the differences between the first two TTO based value sets in Central and Eastern Europe (CEE) is recommended – it has always been claimed that CEE countries display more similar values of health states, which differ from value sets in Western European countries, however, the first glance at both value sets does not confirm such speculations.

The main limitation of the study is the year of the data collection: the completion of the valuation study has been substantially delayed (from the data collection in 2005), as earlier modelling attempts produced non-monotonic values. Attempts at ameliorating these modelling issues through the application of exclusion criteria failed, indicating that the observed non-monotonicities were not reflective of a small subgroup of respondents displaying conflicting preferences. The improved fit of the chosen model, which included a power term below 1, indicates that respondents display substantially diminishing sensitivity to increasing health problems. Whether or not this diminishing sensitivity is unique to this study population may warrant further investigation.

Besides the modelling issues, sample size and the low number of health states valued were additional reasons why it was difficult to obtain the monotonic value set. Currently, the EuroQol Group Association recommends the sample size of 1,000 units to complete the valuation study with sufficient statistical power. Back in 2005, such recommendations were not in place and our data collection was limited in financial terms as well as timewise.

Conclusion

The article presents the first TTO-based EQ-5D value set for Slovenia. There have been two previous attempts to present an EQ-5D VAS based value set in Slovenia, once in 2000 (12) and once in 2012 (13), however, those value sets either lacked logical consistency or consistent modelling techniques. The use of a constrained ordinary least squares approach built upon experiences from EQ-5D-5L valuation studies in China, but extended to handle diminishing sensitivity to increasing health problems, allowed us to generate logically consistent value sets for VAS and TTO. The two value sets presented in this paper are recommended for use in EQ-5D-3L studies in Slovenia.