Human papillomavirus vaccine introduction in low-income and middle-income countries: guidance on the use of cost-effectiveness models.

BACKGROUND: The World Health Organization (WHO) recommends that the cost effectiveness of introducing human papillomavirus (HPV) vaccination is considered before such a strategy is implemented. However, developing countries often lack the technical capacity to perform and interpret results of economic appraisals of vaccines. To provide information about the feasibility of using such models in a developing country setting, we evaluated models of HPV vaccination in terms of their capacity, requirements, limitations and comparability.
METHODS: A literature review identified six HPV vaccination models suitable for low-income and middle-income country use and representative of the literature in terms of provenance and model structure. Each model was adapted by its developers using standardised data sets representative of two hypothetical developing countries (a low-income country with no screening and a middle-income country with limited screening). Model predictions before and after vaccination of adolescent girls were compared in terms of HPV prevalence and cervical cancer incidence, as was the incremental cost-effectiveness ratio of vaccination under different scenarios.
RESULTS: None of the models perfectly reproduced the standardised data set provided to the model developers. However, they agreed that large decreases in type 16/18 HPV prevalence and cervical cancer incidence are likely to occur following vaccination. Apart from the Thai model (in which vaccine and non-vaccine HPV types were combined), vaccine-type HPV prevalence dropped by 75% to 100%, and vaccine-type cervical cancer incidence dropped by 80% to 100% across the models (averaging over age groups). The most influential factors affecting cost effectiveness were the discount rate, duration of vaccine protection, vaccine price and HPV prevalence. Demographic change, access to treatment and data resolution were found to be key issues to consider for models in developing countries.
CONCLUSIONS: The results indicated the usefulness of considering results from several models and sets of modelling assumptions in decision making. Modelling groups were prepared to share their models and expertise to work with stakeholders in developing countries. Please see related article: http://www.biomedcentral.com/1741-7007/9/55.

Background

Cervical cancer is the second most common cause of cancer-related death in women worldwide, with about 500,000 newly diagnosed cases and 250,000 related deaths occurring every year, mostly in low-income and middle-income countries [1]. Two prophylactic vaccines against human papillomavirus (HPV), the necessary cause of cervical cancer, have been shown to be efficacious against HPV infection and precursors of cervical cancer [2,3]. Both of these protect against HPV types 16 and 18, which are responsible for about 70% of invasive cervical cancer cases [4]. The two vaccines differ in aspects not related to their protection against cervical cancer due to vaccine-type HPV, such as their valency, the non-cervical disease endpoints indicated in their licensure as well as the degree of protection against non-vaccine type HPV infection reported in clinical trials [5].

Policymakers are faced with decisions about the introduction of one of the two HPV vaccines in routine immunisation programmes. The World Health Organization (WHO) recommends that the cost effectiveness of introducing a new vaccine to the national immunisation programme is considered before such a strategy is implemented [6], and has reiterated this advice for the case of HPV vaccination [7]. In recent years, health economic evaluation and modelling have played an important role in decisions about HPV vaccination in high-income countries [8,9]. However, there is less evidence that economic evaluations have had such a role in low-income and middle-income countries, despite the availability of results from several studies based in these countries [10-14]. A major difficulty is that these countries often lack the technical capacity to perform and interpret results of economic appraisals of vaccines, which, in the absence of country contextualisation, is believed to have sometimes led to suboptimal decision making about vaccine introduction [15].

Existing literature reviews of models of HPV-related preventative strategies have mainly [16-20] or solely [21-24] focused on high-income countries. In addition, the reviews mainly discuss the results of previous adaptations of cost-effectiveness models rather than their potential for application to new settings. Review authors have usually not had access to most of the models that they were reviewing, and hence have not been able to compare model results when applied to a common data set and assumptions. Hence they may provide useful summaries to policymakers in (mainly high-income) countries where multiple analyses have already been conducted, but are less useful in exploring the feasibility of applying models to low-income or middle-income settings.

Decision makers in low-income and middle-income countries have requested WHO guidance on the strengths and potential limitations of existing mathematical models produced by both public and private sector analysts to inform HPV vaccine introduction decisions. The purpose of this paper is to assess existing economic models of HPV vaccination in terms of their capability, requirements, robustness and limitations, particularly when applied to a low-income or middle-income country setting. This information is meant to provide policymakers in these countries with information about the feasibility of applying these models to inform their own decision making, as well as guidance in interpreting their results. The objective is to provide a menu of potentially useful models and their key characteristics, rather than to recommend a particular model.

Methods

Literature review and model selection

In order to survey existing economic models of HPV vaccination, the Ebscohost, Embase, PubMed and ScienceDirect databases were searched for economic evaluations of HPV vaccination. Articles were included if they were full research papers describing an economic evaluation (cost effectiveness, cost utility or cost-benefit analysis) of vaccination against HPV to reduce the incidence of cervical cancer and/or its precursor lesions, were published before February 2010 and were written in English, Dutch, German, Spanish, Italian or French. Additional file 1 gives the search terms used and number of studies found.

A subset of the models identified in the literature review, together with an additional model discussed in the grey literature [25] and later published [26], were selected for a more detailed comparison. Models were chosen based on the willingness of model developers to participate in the study as well as to ensure representation of models with the following characteristics: model development by authors from institutions in low-income or middle-income countries, previous model application to low-income or middle-income settings, both public sector and pharmaceutical industry provenance, as well as being representative of different model structures (static/dynamic, aggregate/individual based) represented in the literature. Industry models were included in order to provide guidance to decision makers in low-income and middle-income countries who are often confronted with results from such models, and sometimes have no other source of cost-effectiveness information.

Evaluation of hypothetical scenarios

In order to compare the robustness and applicability of the models, model developers were provided with standardised scenarios consisting of input data sets for two hypothetical countries: a low-income country and a middle-income country. For the hypothetical low-income country, parameters for demographics, HPV epidemiology and treatment costs were assembled from low-income countries in the WHO Africa region. For the hypothetical middle-income country, the parameters were assembled from lower-middle-income countries in the WHO South-East Asia and Western Pacific regions (details are given in Additional file 2). Model developers were asked to use their models to evaluate the cost effectiveness of adding adolescent HPV 16/18 vaccination either to a situation with no organised cervical screening (for the hypothetical low-income country), or to a situation with some limited screening (in the hypothetical middle-income country). Screening was assumed to be performed by either visual inspection with acetic acid (VIA) or Pap smear (see Additional file 2). The analysis of the impact of vaccination was limited to cervical cancer outcomes only.

The standardised data sets that were provided to model developers included demographic, epidemiological and economic parameters (see Additional file 2). Natural history parameters (such as transition rates between different precancer neoplasia stages) are expected to be largely independent of country, ignoring the possible effects of factors such as HIV coinfection and nutrition. Hence model developers were asked to use the original parameter set that they used for what they considered to be a general 'base case' model, either in a representative low-income or middle-income setting, or in what they considered to be their main modelling publication. They were also asked to use their original model assumptions for cancer diagnosis, treatment, staging, recovery and recurrence.

Developers were asked to provide results of the evaluation within a month of receiving the standardised data set. They were requested to provide both epidemiological (for example, HPV prevalence, cervical cancer incidence) and economic (for example, discounted and undiscounted costs and disease-weighted life years saved) results. One-way sensitivity analyses on discount rates, duration of vaccine protection, key costs and utilities as well as the prevalence of HPV infection were also requested. Lastly, model developers were asked to fill in a questionnaire on how long the exercise took, the level of expertise needed to conduct it, and the extent to which they were willing to share model software, code and documentation with potential partners in low-income and middle-income countries. Developers then met with HPV vaccinologists, epidemiologists and policy advisors in order to discuss the findings.

Results

The literature review identified 58 articles on economic evaluations of HPV vaccination. Based on the World Bank classification of countries by income level, 41 single-country studies were set in high-income countries; these describe the cost effectiveness of vaccination in 18 different high-income countries. In addition, six single-country studies were set in three upper-middle-income countries (Brazil, Mexico and South Africa), while two single-country studies were set in lower-middle-income countries (India and Vietnam). There were no identified single-country studies set in low-income countries. There were also nine multicountry studies; three of these were set in high-income countries, four in low-income or middle-income countries and two in both high-income and low-income/middle-income countries. Hence high-income countries were far more strongly represented in the overall literature. Indeed, some countries (USA, UK and The Netherlands) had the benefit of three or more economic evaluations conducted by separate groups. In contrast, in low-income and middle-incomecountries (apart from Brazil and Mexico), evidence is scarce, and at best relies on either a single study or part of a multicountry or regional analysis. For instance, the cost effectiveness of HPV vaccination in all 72 Global Alliance for Vaccines and Immunisation (GAVI)-eligible countries is described in a single publication [14]. Details of the identified articles are given in Additional file 3.

Despite the large number of publications, the majority of studies were conducted based on a model that had been previously described; either the same model in a different context, or an adapted version of a previously used model. It was explicitly mentioned in 35 of these articles that the model used in the study had been used previously.

Six models were selected for inclusion in the comparison exercise: (i) the Harvard model [10], (ii) the WHO-CHOICE ('CHOosing Interventions that are Cost Effective') model [13], (iii) the Thai model [26], (iv) the South African model [12], (v) the Merck model [27] and (vi) the GlaxoSmithKline (GSK) model [28]. The selection includes two with wide applicability to many countries including low-income and/or middle-income countries (Harvard and WHO-CHOICE models), two developed by modellers based in middle-income countries with specific application to their own countries (Thai and South African models), and one from each of the manufacturers of the two licensed HPV vaccines (Merck and GSK models). All six models have been applied to low-income and/or middle-income countries. They also encompass the range of structures used in models of HPV vaccination including Markov (GSK, South Africa, WHO-CHOICE), semi-Markov (Thai), compartmental dynamic (Merck) and static individual based (Harvard). In the cases where groups had multiple modelling tools, only one was selected for the exercise. Key characteristics of the models are shown in Additional file 4.

Figure 1 shows the different model estimates of age-specific HPV 16/18 prevalence and cervical cancer incidence for the two hypothetical countries before and after vaccination. None of the models perfectly reproduced either HPV prevalence or cervical cancer incidence in the standardised data set provided to the model developers. In particular, they had difficulty reproducing very low HPV 16/18 prevalence levels in the hypothetical middle-income country. However, most models agreed that large (up to 100%) decreases in type 16/18 HPV prevalence and cervical cancer incidence are likely to occur following vaccination.

Figure 1

Human papillomavirus (HPV) prevalence and cervical cancer incidence from each model. Pre/post-vaccination HPV prevalence and cervical cancer incidence for type 16 and 18 HPV from each of the six models compared to figures in the standardised data set, for the hypothetical low-income country (top graph) and middle-income country (bottom graph). The Thai model shows HPV prevalence for all high-risk HPV types but cancer incidence for only type 16 and 18 related HPV infection.

The relationship between HPV prevalence and cervical cancer incidence in a model depends on its representation of the natural history of cervical disease. Developers used different approaches to adapt their models to different outcomes. The South African model used HPV prevalence data as a direct input. The GSK and Merck models adjusted the force of HPV infection in order to fit to cervical cancer data. Since natural history parameters were kept constant, this sometimes led to a worse fit to HPV prevalence data. The Thai model grouped all (vaccine and non-vaccine type) HPV-related outcomes, and assumed that 70% of cervical cancer cases were the result of HPV 16/18 infection to fit to data. For the Harvard model, developers selected the profile from an existing suite of prefitted models that best fit the data. This resulted in a fit that was less good than if the model had been independently fitted. The WHO-CHOICE model used cervical cancer incidence data as a direct input into the model, but adjusted this using a regional specific estimate of the proportion of cancer cases caused by HPV 16/18, then applied pre-existing vaccine efficacy estimates.

Figure 2 shows model estimates of the incremental costs, utilities and cost-effectiveness ratios of vaccination for both settings. Most models predicted the ratio to be around US$1,000 to US$2,000 per disease-weighted life year gained for both countries. However, the South African model indicated lower incremental cost-effectiveness ratios than the others, while the Thai model indicated higher incremental cost-effectiveness ratios for the hypothetical low-income country. The most influential factors affecting cost effectiveness across all models were the rate of discounting, duration of vaccine protection, vaccine price and HPV prevalence. These parameters have also been found to be influential in reviews of HPV models in high-income countries [21,22].

Figure 2

Incremental cost-effectiveness ratios for vaccination from each model. Discounted incremental cost-effectiveness ratios for vaccination in the hypothetical low-income (top graph) and middle-income (bottom graph) countries, using each of the six models. Dotted lines show gross domestic product (GDP) per capita for the hypothetical country. Missing bars indicate that results were not available for that model.

Key issues for low-income and middle-income countries

The model comparison exercise and subsequent discussions highlighted a number of key issues that affect model-based analyses based in low-income or middle-income settings.

Demographic change

The absolute incidence of cervical cancer predicted by many of the models for the hypothetical low-income country exceeds that of most of the low-income countries in the world, even though the age-dependent incidence of cervical cancer in the hypothetical data set is typical of many of these countries. The main reason is because the population structure of low-income countries is not currently in equilibrium since the number of births far exceeds the number of deaths each year. For instance, prevaccination cancer incidence predicted by many of the models for the hypothetical low-income country is higher than that of most low-income African countries. An open population model may be able to capture the current population structure and its likely changes in the future. However, a closed population model (such as a cohort model) may predict a higher absolute incidence of cancer compared to an open model since the cohort will be larger than older cohorts currently observed. Both kinds of models may predict a higher absolute incidence of cancer than current statistics indicate, and hence overestimate the cost effectiveness of vaccination.

Data resolution

HPV-related outcome data for many low-income and middle-income countries are poor, and often highly aggregated. For example, cervical cancer incidence available from the WHO/Institut Català d'Oncologia Information Centre on HPV and Cervical Cancer [29] is stratified into the age groups 0-14, 15-44, 45-54, 55-64 and ≥65 years. Models with very simple natural history representations may thus predict that 15-year-old women have the same cancer incidence as 44-year-old women, hence overestimating the projected incidence of cancer in younger women due to the large size of the 15-44-year-old age band. Overestimating the proportion of cancers occurring in younger women may cause the cost effectiveness of vaccination prior to sexual debut to be overestimated (since more life years are lost by cancer occurring at a young age). However, it may cause the cost effectiveness of vaccination of older women to be underestimated (because vaccination is assumed to occur too late to protect many of these women). More sophisticated models may be able to capture the time lag between the peak incidence of HPV infection and of cervical cancer by using progression rates based on model fits to countries with more data, and hence correctly predict that cervical cancer incidence is higher in adults than in adolescents.

Access to treatment

HPV vaccination models in high-income countries generally assume that all women with diagnosed cancer receive treatment. This assumption may not be accurate in many low-income and middle-income countries. Lower access to treatment will increase both the net cost of vaccination (because fewer health care savings are made by reducing HPV-related outcomes), and the net benefit of vaccination (because more utilities are lost for each cancer case). This could affect the incremental cost-effectiveness ratio in either direction. For instance, the WHO-CHOICE model estimated that reducing treatment coverage in the hypothetical low-income country from 100% to 20% would decrease the incremental cost-effectiveness ratio of HPV vaccination from US$647 to US$507 per disease-weighted life year gained.

Accessibility of models

Table 1 summarises the accessibility of the models used in the exercise in terms of their complexity and availability to new users. Models containing a user interface were generally more available to new users; developers of more complex models were more reluctant to publicly release their models without the opportunity to train new users.

Table 1

Accessibility of the models

	GSK	Harvard	Merck	South African	Thai	WHO-CHOICE
Complexity of evaluation for this exercise:
System used	Desktop PC	Linux cluster	Desktop PC	Desktop PC	Desktop PC	Desktop PC
Coding platform	Excel	C++	Mathematica	TreeAge	Excel VBA	Excel VBA
Time taken (person h)	30	15	20	12	25	32
Availability to new users:
Source code	X					X
Front end	X		X (web based)		X	X
Documentation	X		X		X
Availability	1	3	2	3	2	1

Accessibility of the models in terms of their complexity (based on the evaluation conducted for this exercise) and availability to new users. Availability of the model to new users is rated according to the following: (1) freely available for anyone to use, (2) may be available as part of collaboration under supervision by developers, (3) may be available to apply to new settings provided the evaluation is conducted by developers.

WHO-CHOICE = World Health Organization 'CHOosing Interventions that are Cost Effective' model; VBA = visual basic application.

Discussion

Six models were used to evaluate the cost effectiveness of HPV vaccination of adolescent girls in two hypothetical scenarios typical of a low-income and middle-income country. Despite differences in model type, structure, assumptions and complexity, the models reached conclusions that were qualitatively similar about the cost effectiveness of vaccination, although they displayed diverse quantitative features particularly in sensitivity analyses.

The data sets given to model developers were hypothetical and not meant to inform decision making for any actual country. Furthermore, a number of simplifications were made to the exercise. For example, the exercise focused on the most important proven benefit of HPV vaccination, that is, the prevention of cervical cancer and its precursor lesions due to HPV type 16 and 18 infection. Protection against other endpoints (including genital warts, non-cervical HPV-linked cancers and crossprotection against non-vaccine type infection) were not considered. In a real-world setting, these endpoints may provide substantial additional benefits to vaccination. Also, model developers were given a month to conduct the exercise, following receipt of the standardised data set. They were asked to restrict themselves to a single model, even though some groups had several models that they could deploy, often in order to crossvalidate results. This simulates a typical real-world situation in low-income and middle-income countries, where there may be constraints on the time and resources available to conduct an economic evaluation. The limited time to conduct the exercise required other trade offs of the investigators. For example, they were asked not to alter their existing natural history parameters to better fit outcome data provided, which may account for the variation in model predictions of HPV prevalence and cervical cancer incidence, as well as the difficulty some models had in accurately reproducing both outcomes in the same model. Also, only one-way sensitivity analysis was performed. In practice, investigators may typically conduct multidirectional, probabilistic and scenario sensitivity analyses to capture uncertainty in HPV natural history and epidemiology.

Only a subset of available HPV models was evaluated. The models selected were some of those perceived to be potentially useful for deployment in low-income and middle-income settings; however, this does not imply that other available models (either from the same or from other model developers) could not also be used. Also, the type of model to be used will depend on the kind of question that needs to be answered. For example, a dynamic model would be needed to investigate the impact of herd immunity, and the incremental benefit of interventions that rely on this effect, such as catch-up campaigns. However, the input and computational power needed for such a model may not be available in all settings.

While unrealistic for directly informing policy, the results of this exercise are useful for identifying features of models and model comparisons that are useful to policymakers wishing to commission cost-effectiveness evaluations. The results of the exercise suggest that even when using a standardised data set, there are important differences between model predictions, although broad qualitative features are common to most model results. This finding highlights the importance of not relying on a single model type or set of modelling assumptions for decision making. If independent models using different structures and assumptions reach similar qualitative results (such as HPV vaccination being cost effective under a particular threshold) there may be greater confidence in the robustness of such conclusions. This is particularly the case for low-income and middle-income countries that are limited by lack of input data, and hence may have to make assumptions where data are not available.

Specific recommendations for model developers and policy makers are listed in the Appendix.

Conclusions

Evidence about the cost effectiveness of HPV vaccination in low-income and middle-income countries is scarce. Policymakers in these countries often do not have the benefit of using results from an array of cost-effectiveness studies conducted in their country, unlike their counterparts in countries such as the USA, UK and The Netherlands. Furthermore, they often lack the technical capacity to use such models and interpret their results. However, the response to this exercise demonstrates that modelling groups are prepared to share their models and expertise to work with stakeholders in low-income and middle-income countries. This exercise and its findings should improve knowledge about the range of models available as well as many of the features to consider when using them.

Appendix

Recommendations

For model developers

(1) Model developers evaluating vaccination in low-income and/or middle-income countries should ideally account for (or, at a minimum, discuss) the possible effects of demographic change, low data resolution and poor access to cancer treatment where relevant.

(2) Models with a range of types, structures, assumptions and complexity may give broadly similar results about the cost effectiveness of routinely vaccinating girls in early adolescence, although they may not all be able to perfectly fit prevaccination outcome data.

For decision makers

(1) The models evaluated here appear to be suitable for evaluating the cost effectiveness of routinely vaccinating girls in early adolescence.

(2) Decision making should ideally be informed by several different models; agreement among models gives greater confidence in the robustness of conclusions.

(3) Model developers are prepared to share their models and expertise, and hence could be approached by decision makers in low-income and middle-income countries wishing to evaluate vaccination options.

Competing interests

ND is an employee of GSK Biologicals, which developed and is currently marketing the HPV vaccine Cervarix® (Bivalent Human Papillomavirus (Types 16, 18) Recombinant Vaccine). EE is an employee of Merck & Co, Inc. which developed and is currently marketing the HPV vaccine GARDASIL® (Quadrivalent Human Papillomavirus (Types 6, 11, 16, 18) Recombinant Vaccine). RH is a staff member of the World Health Organization. RH alone is responsible for the views expressed in this publication and they do not necessarily represent the decisions, policy or views of the World Health Organization. All other authors declare that they have no competing interests.

Authors' contributions

RH conceived the study. MJ and RH conducted the literature review and constructed the standardised data sets. ND, EE, GG, JK, NP and ES used the data sets as input into their own models, and analysed the results. MJ integrated the results from different models and drafted the manuscript with input from the other authors. All authors read and approved the final manuscript.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1741-7015/9/54/prepub

Additional file 1

Literature review search strategy. Search strategy used for the literature review of economic evaluations of human papillomavirus (HPV) vaccination.

Click here for file

Additional file 2

Standardised data set and data sources used to inform it. Details on the standardised data sets for the hypothetical low-income and middle-income countries, as well as the data sources used to inform it.

Click here for file

Additional file 3

Articles on economic evaluations of human papillomavirus (HPV) vaccination. Articles identified as economic evaluations of HPV vaccination during the literature search published before February 2010.

Click here for file

Additional file 4

Key characteristics of the models involved in the exercise. Details on the provenance, type, structure, parameters, capabilities and outcomes presented of the models involved in the exercise.

Click here for file