Parameterising User Uptake in Economic Evaluations: The role of discrete choice experiments.

Model-based economic evaluations of new interventions have shown that user behaviour (uptake) is a critical driver of overall impact achieved. However, early economic evaluations, prior to introduction, often rely on assumed levels of uptake based on expert opinion or uptake of similar interventions. In addition to the likely uncertainty surrounding these uptake assumptions, they also do not allow for uptake to be a function of product, intervention, or user characteristics. This letter proposes using uptake projections from discrete choice experiments (DCE) to better parameterize uptake and substitution in cost-effectiveness models. A simple impact model is developed and illustrated using an example from the HIV prevention field in South Africa. Comparison between the conventional approach and the DCE-based approach shows that, in our example, DCE-based impact predictions varied by up to 50% from conventional estimates and provided far more nuanced projections. In the absence of observed uptake data and to model the effect of variations in intervention characteristics, DCE-based uptake predictions are likely to greatly improve models parameterizing uptake solely based on expert opinion. This is particularly important for global and national level decision making around introducing new and probably more expensive interventions, particularly where resources are most constrained.

Introduction

In the early stages of introducing new health interventions such as novel products or services, there is often considerable uncertainty around their potential uptake, impact and cost‐effectiveness. In such cases, mathematical modelling studies are frequently used to decide whether these new interventions could be cost‐effective, with the results of these analyses critical for ultimately deciding whether or not to introduce the intervention. As such, it is crucial that models make realistic assumptions about levels of intervention uptake and how new interventions might affect the use of existing services or products. Before data on real‐life uptake become available, models generally rely on trial data, expert opinion, observed uptake of comparable interventions, or model a range of uptake scenarios. Such uptake measures are likely to be highly uncertain and fail to account for the dynamic and heterogeneous manner in which individuals make decisions, for example, how users value a new product's characteristics differently such as reduction in risk (efficacy) or price and how this affects uptake and substitution from similar services or products. Even real‐life uptake data will be highly context‐specific and intervention‐specific, and will not be useful for understanding how uptake could vary if the intervention was delivered differently.

The conventional modelling approach assumes uptake is uniform, that is, the same proportion of potential users will take up an intervention regardless of its characteristics such as efficacy or cost, and often apply a number of uptake assumptions. This assumption is currently the norm for economic evaluations in the HIV prevention field, with a given level of uptake assumed based on at best expert opinion or comparable products or services, but normally just assumed with no data to back up the assumption (Cremin et al., 2013; Dimitrov et al., 2011; Gomez et al., 2012; Gray et al., 2011; Terris‐Prestholt et al., 2014; Verguet et al., 2013; Verguet and Walsh, 2010; Walensky et al., 2012; Williams et al., 2011).

Cost‐benefit analyses can incorporate user preferences to value non‐market impacts (Fujiwara and Campbell, 2011); however, the use of stated preference methods to explore the dynamic effect of intervention characteristics on uptake, and therefore intervention impact and cost‐effectiveness, is novel. This letter illustrates the benefit of integrating user preferences into impact and cost‐effectiveness models using an example from the HIV prevention field, where modelling is often used to inform decisions for health policy both at global and national levels. We propose the use of empirically collected preference data such as those generated through discrete choice experiments (DCEs). In DCEs, potential users make repeated choices between intervention scenarios. Varying the characteristics of these scenarios allows for explicit estimation of the magnitude of their effect on uptake. Importantly, DCEs generate data on how users may substitute to using new products or services away from existing behaviour, a critical model parameter.

The new approach allows the modelling of synergies among intervention attributes and both uptake and substitution between new and existing interventions. This letter proceeds as follows: Section 2 describes our theoretical model, and Section 3 uses the model to illustrate how DCE‐based uptake projections can affect the modelled impact of introducing new HIV prevention technologies in South Africa. The discussion in Section 4 seeks to draw out the key implications of our letter. Box 1 provides an overview of terminology used.

Theoretical model

To demonstrate this approach, we compare the impact prediction of the conventional method, where uptake is assumed to be independent of intervention characteristics (i.e. uniform uptake) with those of a model using uptake and substitution predictions from DCE data. We use a simple model to estimate the short‐term impact of two HIV prevention products on the average level of protection that an individual has. For a single product x, we assume the average protection against HIV, P, from product, x, is a function of its efficacy, E, and uptake (or use), U,

A second concern when introducing new products is that substitution may occur between effective (and likely cheaper) existing options and new potentially less effective or more expensive options. In the context of introducing a new prevention product, y, we assume that a proportion, U, of those using existing product x substitute for the new product y and a proportion U of those not using any product start using product y. If the efficacy of the new product is E, then the total protection provided, P by the new (y) and existing (x) products is as follows:

The additional protection provided by introducing the new product (P) will depend on the efficacy of the new product and how the uptake and substitution away from existing products are related to this efficacy. In the following section, we illustrate this numerically in terms of HIV prevention impact, then consider the relevance for economic evaluations in LMIC.

An Example

This letter aims to illustrate the value of incorporating DCE‐based uptake predictions into economic evaluations using an example from the HIV prevention field: the introduction of topical pre‐exposure prophylaxis (TPrEP) in South Africa. Also known as microbicides, TPrEP is a relatively new HIV prevention technology. A recent trial showed TPrEP could be effective for reducing the risk of HIV acquisition among HIV‐negative women (Karim et al., 2010) but with wide confidence intervals, estimating a per sex‐act efficacy of 54%, ranging from 8 to 83% (Terris‐Prestholt et al., 2014). Note that this is less than the efficacy of condoms, and so substitution from condoms to TPrEP could result in increased HIV transmission. We use the model in Equation 2 to compare the projected additional HIV protection provided by TPrEP using conventional uptake assumptions with DCE–based uptake estimates from South Africa (Terris‐Prestholt et al., 2013). A household survey of 1017 adult women collected data on women's preferences for HIV prevention products including the male condom and TPrEP. It measured women's preferences for product characteristics including HIV and pregnancy prevention efficacy (see Box 2 for more detail).

As shown in Figure 1, predicted uptake of TPrEP increased with its assumed efficacy and was greater among women who had not used condoms at their last sex‐act (non‐condom users) compared with those who had used a condom (condom users) (Terris‐Prestholt et al., 2013). For example, if TPrEP is only 55% efficacious then the DCE predicts 11% of condom users will use the product, whereas 16% of non‐condom users will use the product. However, if TPrEP is 95% efficacious, then predicted uptake increases to 30% among condom users and 41% among non‐condom users. Thus, using individual preference data permits prediction of overall uptake and substitution between new and existing HIV prevention products by user and product characteristics. The method can further be refined to include additional intervention and user characteristics.

Figure 1

Predicted uptake of TPrEP by assumed HIV efficacy among condom and not‐condom users (adapted from Terris‐Prestholt et al., 2013)

Using these uptake projections, we are able to model the incremental impact of introducing TPrEP into a population of adult women in Gauteng, South Africa, where the DCE data were collected. We start with the estimated average protection provided by condoms at different levels of use applying Equation (1). We introduce these predictions into the simple theoretical model of HIV protection in Equation (2) and estimate the incremental impact as P. Assuming condoms are 85% effective (Pinkerton and Abramson, 1997; Pinkerton et al., 1998) then at 20% condom use, the average per sex‐act protection is 18% before TPrEP introduction and 51% at 60% condom use.

Figure 2 presents the estimates of the incremental HIV protection provided from adding TPrEP into the method mix in two ways: firstly using uniform uptake predictions (the left bar in each pair) and secondly, using uptake and substitution parameters from the DCE (right bar in each pair). The uniform uptake assumption based on expert opinion assumes that 30% of non‐condom users would use TPrEP and 5% of condoms users would switch to TPrEP, regardless of TPrEP efficacy (Terris‐Prestholt et al., 2014).

Figure 2

Comparison of additional HIV protection estimated using conventional uniform uptake assumptions and the DCE‐based uptake predictions: variation by baseline condom use and the efficacy of TPrEP

By allowing for variation in uptake according to TPrEP efficacy, the model predicts that introducing 55% efficacious TPrEP into a population with 20% condom use results in 6% additional population protection compared with when just condoms were used—half the impact predicted (12%) with the uniform uptake assumption. However, when a 95% efficacious TPrEP is considered, higher uptake is predicted than from expert opinion, resulting in 32% additional population protection, nearly 50% more than was predicted (23%) with the uniform uptake assumption. The difference between the TPrEP impact projections for the uniform and DCE uptake predictions is smaller at 75% efficacy but is similar across different levels of baseline condom use.

Discussion

To inform model based cost‐effectiveness analyses, this study proposes the use of DCEs to estimate the likely uptake of new products as well as substitution away from existing products. It illustrates how economic evaluation estimates that rely on modelled impact projections could be severely biased, off by up to 50% in our example, if simple uptake assumptions are applied that fail to account for the synergistic relationship between uptake and substitution and the intervention characteristics. DCEs provide one approach to estimating uptake that can inform modelling in the absence of observed uptake data.

An important assumption of our proposed method is that DCEs (and other stated preference approaches) have sufficient external validity to accurately predict real‐world behaviour. A number of studies in the health literature have compared stated‐preference measures with participants' actual behaviour, or revealed preferences (Lancsar and Swait, 2014). Generally, these have found that stated‐preference techniques are strongly associated with the direction in which people value different products (Ryan and Watson, 2009) and are often not significantly different from revealed preference estimates of magnitude (Mark and Swait, 2004; Kesternich et al., 2013). However, more research is needed to better understand how well DCEs predict not only uptake and failure to take up new health interventions as well as how to design and analyse DCEs to strengthen their external validity. Lancsar and Swait (2014) recently proposed key study designs to evaluate external validity of DCEs, providing a framework upon which to build future validation studies. Early exploratory work is starting to emerge providing more nuanced evidence on their external validity: for example, the positive predictive value may be far better than the negative predictive value (Lambooij et al., 2015; Salampessy et al., 2015), and aggregated uptake predictions are likely to obscure individual level variability in preference (Krucien et al., 2014). The use of labelled DCEs, where alternatives are named explicitly (for example ‘male condom’, ‘TPrEP’ rather than ‘Alternative A’, ‘Alternative B’) has been shown to increase external validity (De Bekker‐Grob et al., 2010), as has carrying out experiments in populations with experience of making decisions relevant to the experiment's context (Groom et al., 2004; Blomquist and Whitehead, 1998). In this case, condom users have already made a proactive and informed decision to use HIV and pregnancy protection, perhaps making their indications of substitution to a less effective new product more reliable.

Stated preference methods such as DCEs are used to explore preferences at a single time point (i.e. whether or not an individual changes his or her behaviour), and are not generally predictive of time‐variant or long‐term, repeated activities. Many interventions require long‐term adherence to produce a substantial effect, but stated preference methods are generally unable to describe if, and how much, individuals will adhere to an intervention. However, initial uptake is a necessary, albeit insufficient, condition for long‐term impact.

The introduction of a new single‐purpose technology such as TPrEP has given rise to concerns that people will stop using existing multi‐purpose technologies such as condoms, decreasing pregnancy protection as well as protection for other sexually transmitted infections (STIs) (Karim et al., 2010; Underhill, 2013). Data from DCEs allow us to interrogate the degree to which people may switch methods. Expert opinion suggested that around 5% of people would swap condoms for TPrEP, irrespective of efficacy, whereas the DCE data suggest a more nuanced view with the degree of substitution being dependent on intervention efficacy. For instance, the data predict that 11% of condom users would switch to a 55% efficacious TPrEP but up to 30% for a 95% efficacious TPrEP. This is important because substitution may have effects on wider STI disease burden and ultimately temper the additional benefit new technologies could provide, worsening their overall cost‐effectiveness. Within the HIV prevention field, this analysis could be expanded to show how synergies with other product characteristics can be modelled explicitly such as how uptake and substitution may change with the cost of the product or with the addition of protection against pregnancy or other STIs.

User preferences can also be used to model the potential impact and cost‐effectiveness of other interventions where uptake or use of an intervention is driven by its characteristics, and/or observational data are not yet available, such as stimulating demand for treated bed nets to prevent malaria, malaria vaccinations and voluntary medical male circumcision (Aigbogun et al., 2015; Desrochers et al., 2014; Thirumurthy et al., 2014). Consideration of consumer preferences for different characteristics of these interventions, such as efficacy or aesthetic appeal, may better inform modelling their potential cost‐effectiveness and optimal design of demand‐creation strategies.

At present, infectious disease models tend to use probabilistic sensitivity analysis (PSA) to account for uncertainty in model predictions. However, PSAs do not commonly consider the interdependence of uptake, efficacy and substitution parameters. It is possible that PSA could consider these relationships, but without data to parameterize these relationships, the considerable uncertainty this method would incorporate into the model estimates could reduce their usefulness. The method proposed in this letter could indeed be used in conjunction with PSA: DCE data could be used to parameterize a PSA, giving distributional information for the parameters and the interactions between these factors to ultimately reduce uncertainty in the model and its PSA.

As quantitative data on drivers of uptake are rare for new interventions, this study proposes the use of DCE data to strengthen economic evaluations. We suggest that assuming uniform uptake in economic evaluations could erroneously support or reject the adoption of a new technology. This is of particular concern in low‐income and middle‐income settings due to considerable strains on healthcare and research spending, alongside an often high burden of ill health.

Conflicts of Interest

The authors have no conflicts of interest to declare.

Ethics statement

This letter uses only secondary data publically available in the literature, and no specific ethical approval was required.

Acknowledgements

We thank Edina Sinanovic, Gabriela B. Gomez, and Catherine Pitt for their helpful comments on an earlier draft of the manuscript.

Funding

Support for this work was provided by: Programme for Appropriate Technology for Health (PATH), Economic and Social Research Council 1 + 3 Studentship, Faculty of Public Health and Policy Fellowship, Microbicide Development Programme, and the Bill and Melinda Gates Foundation project titled “ARV‐Based Prevention Technologies: Developing the Capacity and Needed Tools to Deliver New Prevention Products” (2011‐2015).

Efficacy:	The extent to which an intervention produces a benefit under ideal circumstances
Uptake:	The extent to which potential users adopts a new intervention
Adherence:	The extent to which a user, who has adopted an intervention, complies with a given regime as prescribed by the intervention
Use:	A function of uptake and adherence. The extent to which individuals sufficiently abide by an intervention's requisite behaviours
Effectiveness:	A function of efficacy, uptake and use. The extent to which an intervention produces a benefit under ‘real‐world’ circumstances. Includes non‐uptake and improper use.
Uniform Uptake:	The same proportion of potential users will uptake an intervention regardless of intervention characteristics such as efficacy or cost.

Data & Methods: A DCE was conducted via a random household survey among 1017 women in urban Gauteng Province, South Africa. Women were presented with choices between potential women's NPTs (microbicides, diaphragm, female condom) and ‘what I did last time’ (use or not use a condom) with different HIV and pregnancy prevention effectiveness and prices. Choice probabilities were estimated using the nested logit model and used to predict uptake. Results: In this high HIV prevalence setting, HIV prevention effectiveness is the main driver of uptake followed by pregnancy prevention effectiveness. For example, a microbicide with poor effectiveness would have niche appeal at just 11% predicted uptake, while a highly effective microbicide (95% effective against HIV and pregnancy) would have far wider appeal (56% predicted uptake). Although women who reported not using condoms were more likely to choose the NPTs, at current very high rates of male condom use in South Africa (60%), about half of microbicide uptake is projected to be among those currently not using condoms. Conclusions: Women are very interested in new HIV prevention technologies, especially if highly effective in preventing HIV and pregnancy. Women in greatest need were also most likely to switch to the new products.