HPV-FRAME: A consensus statement and quality framework for modelled evaluations of HPV-related cancer control.

Intense research activity in HPV modelling over this decade has prompted the development of additional guidelines to those for general modelling. A specific framework is required to address different policy questions and unique complexities of HPV modelling. HPV-FRAME is an initiative to develop a consensus statement and quality-based framework for epidemiologic and economic HPV models. Its development involved an established process. Reporting standards have been structured according to seven domains reflecting distinct policy questions in HPV and cancer prevention and categorised by relevance to a population or evaluation. Population-relevant domains are: 1) HPV vaccination in pre-adolescent and young adolescent individuals; 2) HPV vaccination in older individuals; 3) targeted vaccination in men who have sex with men; 4) considerations for individuals living with HIV and 5) considerations for low- and middle-income countries. Additional considerations applicable to specific evaluations are: 6) cervical screening or integrated cervical screening and HPV vaccination approaches and 7) alternative vaccine types and alternative dosing schedules. HPV-FRAME aims to promote the development of models in accordance with an explicit framework, to better enable target audiences to understand a model's strength and weaknesses in relation to a specific policy question and ultimately improve the model's contribution to informed decision-making.

Introduction

Infection with oncogenic types of the human papillomavirus (HPV) has been causally linked to cervical cancer, the fourth most common cancer in women worldwide, with an estimated 569,847 new cases being diagnosed in 2018 [1]. HPV is also associated with a varying proportion of cancers at other anogenital and oropharyngeal sites in both women and men, with the total HPV-attributable cases in 2012 estimated as 630,000 [2]. HPV is additionally responsible for anogenital warts, a debilitating and embarrassing condition for many, as well as recurrent respiratory papillomatosis (RRP), an uncommon condition of recurrent growth of wart-like lesions in the respiratory tract. Although it is a rare disease {0.07–1.0 case per 100,000 children (juvenile onset RRP) [3,4]; 0.54–4.0 cases per 100,000 adults (adult onset RRP) [5,6]), in a small number of cases, RRP results in respiratory obstruction which can be life-threatening [7].

In the past decade, HPV vaccination of girls has been introduced in many countries, with the objective of preventing cervical cancer development later in life for women while also preventing HPV-related cancers in men via herd protection (i.e. indirect protection of unvaccinated individuals from HPV infection due to others in the population being protected by being vaccinated against HPV). Some countries have also introduced universal vaccination of boys and/or targeted vaccination of men who have sex with men (MSM). In parallel, there have been recent developments in screening methods (e.g. HPV testing) and triage technology advances for screen-positive women, which are leading to more effective secondary cervical cancer prevention programs. However, the full health gains in preventing cancer diagnoses and deaths through screening and vaccination will only be observed several decades after the initial implementation phase. Decision-analytic models of health prevention strategies have therefore been key to supporting long-term predictions of health outcomes, resource utilization and economic value for new interventions in primary and secondary prevention of HPV and related disease.

Due to the role of models and cost-effectiveness evaluations in guiding health technology assessment and healthcare policy decision-making, quality frameworks have been proposed to enable transparency, interpretation and credibility of results, and comparison between similar studies. A quality framework attempts to address poorly designed models, over-simplification, misinterpretation and failure to appreciate the degree of uncertainty in a decision process [8]. Existing frameworks for modelling, e.g. the International Society of Pharmacological Outcomes Research (ISPOR) guidelines, provide a set of best practices and relevant guidelines for general modelling [9]. However, the publication of an increasingly large volume of modelling/cost-effectiveness analyses of HPV vaccination programs as well as screening programs [[10], [11], [12], [13], [14], [15], [16], [17], [18]], a growing number of independent model types, and the need to address different policy questions has resulted in intense research activity in the area of HPV modelling over the last decade. In addition, modelling HPV presents unique complexities including sexual transmission, multiple HPV types with different natural histories, a variety of HPV-related diseases, difficulty in studying the progression of the disease to cancer (pre-cancerous lesions are detected by screening and treated), multiple primary and secondary prevention tools, herd protection effects resulting from vaccination, modification of natural history due to screening and many levels of heterogeneity in sexual and health-seeking behaviour. Therefore, in order to address distinct issues related to HPV modelling a specific framework is required in addition to existing guidelines.

Here, we present HPV-FRAME, an initiative to develop a consensus statement and a quality framework in the form of a CONSORT-style itemised checklist encapsulating agreed reporting standards for epidemiological and economic models of interventions in HPV and HPV-related cancer prevention. The framework does not consider models for other purposes, such as biological models or conceptual models which do not use real-world data. HPV-FRAME is specifically oriented to policy evaluations, with the aim of allowing modelers, peer-reviewers, journal editors and policy decision-makers to understand a model's strengths and weaknesses in relation to a specific policy question, thus improving the model's contribution to informed decision-making. HPV-FRAME is not intended to be prescriptive about modelling methods, but is designed to promote transparency in reporting those methods, improve the completeness of reported evaluations, prevent oversimplification and limit the number of poorly designed and reported studies.

General principles of good modelling practice as they apply to HPV evaluations

A number of key general principles of good modelling practice have previously been agreed upon and disseminated, and some of these specifically consider infectious diseases and HPV [[9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19]]. These general principles provide a backbone for a high quality modelling study or economic evaluation, and are briefly summarised in Table 1.

Table 1

Summary of general principles of good modelling practice.

Principle	Reference
The type and scope of the economic or epidemiological evaluation should fit the requirements of the decision maker.	[20,21]
Dynamic models should be used unless it can be demonstrated that herd effects are unimportant.	[[19], [20], [21]]
Publications should include: results stratified by subgroup (e.g. age or sex), health outcomes in natural units, intermediate outcomes (e.g. pre-cancerous lesions) and sexual mixing assumptions.	[21,23]
Goodness of fit to data should be shown where appropriate.a	[22,23]
Sensitivity analysis (considering whether a probabilistic approach is appropriate) should be used and should include the discount rate.	[[20], [21], [22], [23]]
All diseases that are relevant to the intervention should be incorporated.	[23]

How the authors describe and document these aspects should be a decision they make. For example, some authors choose to develop online webpages which provide an enduring reference for subsequent publications. Another alternative is to utilise technical appendices and/or reference previous work as applicable.

Model-based decision and cost-effectiveness analyses are usually focused on evaluating policy-relevant questions. They need to explicitly describe what decision-making process they inform, and accordingly, justify their approach, including the range of options considered, the type of evaluation, and methodological assumptions such as the perspective used. If there are more than two intervention options, a full incremental analysis should be conducted. National and international guidelines for economic evaluations should generally be followed, with justification for any deviations.

The choice of a model should be determined by best practice guidelines. In general terms, model structures and parameterization should be as parsimonious as possible, while being able to capture the essential elements of the decision and mindful of the availability of data. Dynamic models should be used unless it can be demonstrated that herd effects are unimportant to the decision being informed [[19], [20], [21], [22], [23]]. This is because dynamic transmission models explicitly simulate changing (declining) HPV incidence and prevalence for vaccine-included types over time, and the consequently declining risk of new infections, after population-wide vaccination initiatives are introduced. Other types of models should ensure that declining incidence/prevalence of vaccine-included types after vaccination are effectively simulated. Static models should only be used when both (i) a conservative (i.e. under-estimated) measure of impact or value for money is sufficient (e.g. a static model may underestimate total health and monetary benefits of vaccination in which case it cannot be used to rule out a vaccination intervention that is found not to be cost-effective); and (ii) when the analysis does not include comparisons between multiple interventions with herd effects (which cannot be captured by static models), such as is often the case in HPV vaccine evaluations – for example, catch-up vaccination versus no catch-up, both sexes versus female-only, 3-dose versus 2-dose, nonavalent versus bivalent HPV vaccines. Examples of relevant policy questions in HPV prevention, and model types which are likely to be needed to adequately address each question, are shown in Fig. 1.

Fig. 1

Model type and data requirements for example policy questions.*

*This schematic summarises general aspects of model construction. In practice, all model types can vary considerably in their complexity and data requirements.

A thorough review of available data should be performed to inform model calibration. This review should span data on sexual behaviour, HPV prevalence studies, cancer incidence and mortality data as appropriate to the model, with a focus on extracting data from high-quality studies only (e.g. IARC-reported registry data on cancer rates should be used over unvalidated local hospital data on cancer rates), and model fits across all relevant data sources should be provided. If model parameters were fitted to observed data, then both graphical (e.g. model outputs compared to data points) and numerical (e.g. deviance, Akaike Information Criterion) measures of goodness of fit should be presented. If Bayesian methods were used then the posterior distributions of fitted parameters should be well-described. Wherever possible, fitted or otherwise parameterized models should be validated by comparing predictions to outcomes not used for parameterization.

Reported outcomes should include main disease outcomes and disease-specific outcomes in natural units (e.g. cancer cases), intermediate endpoints if modelled (e.g. HPV prevalence, pre-cancerous lesions) [21,23], long-term health outcomes [e.g. quality-adjusted life-years (QALYs) or disability-adjusted life years (DALYs)] and economic outcomes, if relevant (e.g. costs). Disaggregated outcomes by time and age (both with and without discounting if appropriate) should be shown at a minimum for the base case (in an appendix if necessary). The timing of interventions, time horizon of the analysis, and indication of population-based versus cohort-based outcomes should also be specified.

Probabilistic sensitivity analysis (PSA) is considered the preferable approach for economic evaluations to assess the joint influence of uncertain parameters on costs and cost-effectiveness outcomes. Multivariate sensitivity analysis (MSA) for epidemiological models or a best/worst case analysis at a minimum, should be considered for health outcomes. Although univariate sensitivity analysis can provide some insight into uncertainty, it does not take into account the total effect of a combination of uncertain parameters and is therefore less preferable than PSA/MSA for economic components. The choice of distributions for PSA or prior distributions for Bayesian fitting algorithms should at a minimum capture sampling uncertainty and not violate best practice guidelines without justification.

Overall, it is important to note that models predict possible rather than actual outcomes, ranging from best case to worst case scenarios and that the predicted outcomes are dependent on the policy question and the parameters defining it as well as the validity and rigour of the data used in the model.

HPV-FRAME development process

The development of HPV-FRAME was informed by an established process involving a 5-phase strategy with distinct activities (Table 2) [24]. The need for a framework for HPV prevention modelling was first identified at an International Papillomavirus Conference modelling workshop in 2009 in Malmö, Sweden [23], and was also prompted by a comprehensive review of the HPV modelling literature conducted for a 2012 Vaccine Monograph [25]. The HPV-FRAME Steering Committee (KC, MJ, JK, SK, JB, RB and MB) met before the satellite session at the International Papillomavirus Conference (IPVC) 2014 and identified the preliminary domains of HPV-FRAME which were presented at information sessions held at international conferences (IPVC 2014, Seattle, USA; EUROGIN 2015, Seville, Spain; IPVC 2015, Lisbon, Portugal). Interested parties were identified via these conference information sessions and from a dedicated website established in 2015 (www.hpv-frame.org), and were asked to sign-up to a mailing list to be kept informed of the progress of this initiative and contribute input. The HPV-FRAME Steering Committee held a meeting at IPVC 2015 where the structure of the framework sections was decided. These sections were presented at a dedicated HPV-FRAME session at EUROGIN 2016 (Salzburg, Austria) and abstracts summarising the scope of each section were posted on the website. Input from attendees was minuted and a call for further submissions via the website was made. As a result of the work at this stage, the domains were expanded in response to increasing interest in certain areas (e.g. vaccination at older ages). The draft framework was developed and presented at a dedicated HPV-FRAME session at IPVC 2017 (Cape Town, South Africa). The final framework is presented here after interested parties (via the mailing list) were invited to provide feedback on the content of this manuscript.

Table 2

The 5-step development process for HPV-FRAME.

Step	Action	Timeline
Initial steps	Identify the need for a guidelineReview the literatureThe need for a framework was identified via an expert discussion followed by a comprehensive review of the HPV modelling literature conducted for a 2012 Vaccine Monograph.	2009–2013
Pre-meeting activities	Identify participants• HPV-FRAME information sessions held at IPVC 2014 (Seattle), EUROGIN 2015 (Seville) and IPVC 2015 (Lisbon) • Interested parties asked to sign up to mailing list • Website and mailing list were established Generate a list of items for consideration at the meeting• HPV-FRAME Steering Committee held a workshop at IPVC 2015 (Lisbon), to decide upon the structure of the framework	2014–2015
Meeting – EUROGIN 2016 (Salzburg)	Presentation of draft framework• Dedicated HPV-FRAME session held at EUROGIN – Input from attendees. • Abstracts summarising the scope of each section were posted on website.	2016
Meeting – IPVC 2017 (Cape Town)	Presentation of the framework• Dedicated HPV-FRAME session at IPVC 2017	2017
Publication and post-publication activities	Dissemination• Public consultation on manuscript draft: emailed to interested parties for comment. • Submission to journal • Inform other journal editors Review framework regularly• Set up capacity for comments/input for next revision (via website)	2018 - ongoing

The focus of the framework is on the analysis of the effectiveness and/or cost-effectiveness of interventions in cervical screening and HPV vaccination which are designed to inform decision-making. There are two key components to each domain in the framework: reporting key structural inputs/parameters and reporting key outputs. Although the framework is not prescriptive about which outputs should be reported, researchers are encouraged to provide clarity on the outputs reported and which subset of outputs are calibration or validation targets and therefore presented against observational data. Modelers are encouraged to report by age and sex. An example application of the framework to a previously published analysis has been provided for selected domains.

The reporting standards for HPV models have been structured according to seven domains which reflect policy questions of topical interest in HPV-related interventions. Domains have also been categorised by relevance to populations or evaluations (Table 3). A) Population relevant domains are: HPV vaccination in pre-adolescent and young adolescent females and/or males (Domain 1); additional issues regarding HPV vaccination in older women and/or men (Domain 2); additional issues regarding targeted vaccination in men who have sex with men (MSM) (Domain 3); HPV vaccination and/or cervical cancer screening for individuals living with HIV (Domain 4); and specific considerations regarding HPV prevention in low and middle income countries (LMIC) (Domain 5). B) Domains that are additional considerations for evaluations applicable to specific interventions are: cervical screening or integrated cervical screening and HPV vaccination approaches (Domain 6); and additional issues regarding alternative vaccine types (e.g. nonavalent), and alternative schedules (Domain 7). Overall, reporting standards in all relevant domains should be addressed when reporting model inputs and outputs. For example an evaluation of vaccination of pre-adolescent HIV positive men who have sex with men in a low income country should consider the reporting standards in Domain 1 (vaccination in adolescents), Domain 3 (targeted vaccination in MSM), Domain 4 (evaluations for individuals living with HIV), and Domain 5 (HPV prevention in LMIC). Furthermore, HPV-FRAME core reporting standards (Table 4) should additionally be reported for all models.

Table 3

Summary information on HPV-FRAME.

Domain No	Target population	Modelled strategy	No of reporting standards
1	Adolescent females or/and males	Vaccination	10 + CRS
2	Adult females and/or males	Vaccination	6 + CRS
3	Men who have sex with men (any age)	Vaccination	7 + CRS
4	Females or/and males living with HIV	Vaccination and/or cervical cancer screening	6 + CRS
5	Females and/or males in low and middle income countries	Vaccination and/or cervical cancer screening	3 + CRS
6	Females and/or males	Cervical screening.Integrated cervical screening and HPV vaccination	6 + CRS5 + CRS
7	Females and/or males	Vaccination using alternative vaccine types (e.g. nonavalent) or/and alternative dose schedules	5 + CRS

CRS: core reporting standards (see Table 4).

Table 4

HPV-FRAME core reporting standards.

a) Inputs	Reported by age? (Y/N)	Report by sex (F-only, M-only or both)?	Comments
Target population for intervention	Y	Y	Age group(s), female/male
Sexual behaviour	Y (for dynamic models)	Y	Sexual behaviour inputs used e.g. number of partners, rate of partner change, heterogeneity, duration of partnerships, number of sex acts per partnership or per unit of time where modelled.Inputs stratified by appropriate risk groups (e.g. by sexual activity, sex, age, sex of partners, urban/rural if applicable) where modelled.Assortativity of sexual mixing where modelled. Probability of HPV transmission per partnership or per sex act
Cohort examined for evaluation/time horizon	N	N	Are results presented for first vaccinated cohort, cohort at post-vaccination equilibrium and/or what time horizon has been considered?
Quality of life assumptions	If available	If available
Calibration	Y	Y	Calibration method(s) usedValues of goodness-of-fit metric(s)Comparison of model outcomes with data points used to fit the model to (in either tabular or graphical format), including credibility/uncertainty intervals or distributions where relevant
Validation (where possible)	Y	Y	Comparison of model outcomes with data points used to validate the model (in either tabular or graphical format), including credibility/uncertainty intervals or distributions where relevant
Costs	If applicable	If applicable	Related to detection of disease and treatment. Parameter applicable only to economic evaluations. Currency and year; methods for inflation adjustment.
b) Outputs	Reported by age? (Y/N)	Report by sex (F-only, M-only or both)?	Report as calibration or validation target? (Y/N)
Cancer incidence, mortality, life years, QALYs/DALYs (as appropriate)	Y	Y	Y
HPV prevalence, pre-intervention	Y	Y	As appropriate to evaluation
CIN2 detected	Y	Y	N
Sensitivity analysis on key inputs	Y	Y	N
Incremental cost-effectiveness ratios and costs saved			Only for economic evaluations

QALYs: quality-adjusted life-years.

Domain 1. reporting modelled evaluations of HPV vaccination in adolescent females and/or males

To date, a number of reviews of modelling/cost-effectiveness analyses of HPV vaccination programs [25] and a few guidelines for modelling vaccination or infectious diseases in general [19,21,26] and HPV vaccination in particular [20] have been published, which summarize key conclusions about model results and methodological recommendations. Vaccinating girls prior to sexual debut has been reported to be cost-effective at list prices for vaccines and commonly used cost-effectiveness criteria [[14], [15], [16], [17], [18],27]. However, in middle-income countries with high vaccine prices, improving screening coverage has been found to be more cost-effective than vaccination if lower prices are not negotiated [27]. As vaccination and screening target different age-cohorts, some argue that best practice to maximise impact and ensure equity would be to introduce both strategies, even in LMIC [29,30].

A number of studies have shown that extending vaccination to males is unlikely to be cost-effective unless prices (including delivery costs) are extremely low and/or overall vaccine uptake in females is low [11,12,15,17,28]. Other recent studies have found that when all HPV-related diseases are taken into account, cost-effectiveness estimates are more favourable for both-sex vaccination [31,32]. However, while we have a reasonable understanding of the natural history of HPV infections leading to high-grade intraepithelial neoplasia in the cervix based on evidence accumulated through multiple prospective studies, this is less true for other sites. There are unanswered questions related to the development of natural immunity, latency and reactivation, the duration from HPV infection to invasive cancer, the prognostic relevance of intermediate stages such as anal intraepithelial neoplasia (AIN) and uncertain intermediate stages for oropharyngeal cancer. However, if the intervention to be considered is vaccination, the interim natural history before development of invasive disease may not need to be specified in detail. When considering both-sex vaccination evaluations, it should be noted that the context is usually female-only vaccination as the comparator because most countries introduced female-only vaccination first; cost-effectiveness will thus be dependent on the vaccine coverage in girls and vaccine price. Recent evidence shows that list prices are rarely paid in national or regional vaccination programs [33]. The actual price paid for vaccines at a regional or country level is often an outcome of confidential tender negotiations and may not be available to use in an analysis; in this situation it is recommended that threshold analysis be performed to determine the maximum price that should be paid.

The key drivers of uncertainty in cost-effectiveness evaluations of HPV vaccination are duration of vaccine protection [14,16,34], natural immunity duration [34,35], model type (static/dynamic) [17,28], vaccine effectiveness [28] (measured by how effective the vaccine is in preventing persistent HPV-infection or HPV-related disease, or as applicable to the particular research question), comparators (screening/no screening) [27,28,36], vaccine price [27,36], discount rate [14] and HPV prevalence [14]. Furthermore, a recent meta-analysis of models predicting the long-term population-level effectiveness of quadrivalent vaccination in women and men found that key drivers of vaccination impact include how models capture heterogeneity in mixing between risk groups (e.g., age, level of sexual behaviour) and the level of natural immunity among women [35]. Furthermore, involving relevant experts (clinicians, policy makers, etc.) to help inform model assumptions would be key to making sure that certain events are being modelled appropriately and that the questions being answered are genuinely policy-relevant.

HPV-FRAME reporting standards for models of adolescent female and male vaccination in individuals ≤19 years of age, are presented in Appendix Table A1. An economic evaluation of HPV vaccination in the UK [37] is used as an example application (Appendix Table A2).

Table A1

HPV-FRAME reporting standards for models of vaccination in adolescent individuals.

a) Inputs	Reported? (Y/N)	Reported by age? (Y/N)	Report by sex?	Comments
Vaccine uptake	Y	Y	Y	Uptake by number of doses, age, sex
Vaccine efficacy	Y	Y	Y	Efficacy by number of doses, age, sex, HPV type
Vaccine cross-protection	Y	N/A	N/A	Level and duration of protection by dose and type.
Duration vaccine protection and waning	Y	Y	Y (report by sex and site/disease if applicable)	Duration of vaccine protection and waning assumptions. Report base case assumption and sensitivity analysis on duration of protection
Vaccine and delivery costs	Y	Y (if applicable)	Y (if applicable)	Vaccine-related cost assumptions (by sex if they differ) including infrastructure (overhead), administration, cost per dose. Delivery mechanism
Pre-vaccination disease burden (including population attributable fractions for HPV)	Y	Y	Y (if applicable)	Pre-vaccination disease burden included when assessing effect on infections and cancer or pre-vaccination cancer burden for population attributable fractions.
Duration of natural immunity	Y	Y	Y	Comment on waning function
b) Outputs	Reported? (Y/N)	Report by age? (Y/N)	Report by sex? (B, F, or M)	Report as calibration or validation target? (Y/N)
Absolute reductions in HPV infections, and/or warts, post-vaccination	Y	Y	As appropriate to evaluation	Y
Absolute reductions in CIN2+ post-vaccination	Y	Y	As appropriate to evaluation	Y. Suggested comparison to vaccine trial data; consideration should be given to trial-based modelling to support this validation process, and/or comment should be made on the applicability and external validity of the trial data to the situation being modelled.
Absolute reductions in invasive cancer (cervical and other HPV cancers, as relevant) post-vaccination	Y	Y	As appropriate to evaluation	Y

B: both sexes; CIN2+: cervical intraepithelial neoplasia grade 2 and above; F: female; male; N: no; Y: yes.

Table A2

Application to Jit et al. (2008) [37].

a) Inputs	Reported?	Reported by age? (Y/N)	Reported by sex?	Comments
Vaccine uptake	Y	Y	Y
Vaccine efficacy	Y	Y (implicitly)	Y (implicitly)	Assumed 100% independent of sex and age
Vaccine duration and waning	Y	Y (implicitly)	Y (implicitly)	Assumed independent of sex and age
Vaccine and delivery costs	Y	Y	Y
Pre-vaccination disease burden (including PAFs)	Y	Y	Y	In accompanying paper (Jit et al., 2010; Choi et al., 2010)
Heterogeneity in sexual behaviour	Y	Y	Y	Reported by age, sex and risk group (Choi et al., 2010)
Duration of natural immunity	Y	Y	Y
b) Outputs
Absolute reductions in HPV infections, cervical and other HPV-related cancers and/or warts, post-vaccination	Y	N	Implicitly (cervical)N (warts/non-cervical cancers)	Outputs considered: cervical (F); warts (F + M); non-cervical cancers (F + M; sensitivity analysis)
Absolute reductions in CIN2+ post-vaccination	Y	N	Y (implicitly)
Absolute reductions in invasive cancer post-vaccination	Y	N	Y(Implicitly)

F: female; M: male; N: no; PAFs: population attributable fractions; Y: yes.

Domain 2 - additional issues in reporting vaccination in older individuals

Vaccinating a larger segment of the population than has previously been routinely performed, with a focus on vaccinating up to older ages [38] is an emerging area of evaluation. Two sub-categories of this area will be addressed in HPV-FRAME: i) untargeted vaccination of females and/or males to older ages (similar in concept to an ‘extended catch-up’) and ii) targeted strategies in HPV-negative individuals or those who are negative for particular HPV types e.g. HPV16. The latter could be achieved by integrating vaccination with HPV-based cervical screening i.e. vaccinating only HPV negatives after screening while referring HPV positive women for further management. This strategy has the potential to improve cost-effectiveness by targeting vaccination to those in the population who are most likely to benefit as they are not infected with vaccine-included HPV types. This concept is sometimes called “screen and vaccinate” [39,40].

Very little work modelling vaccination of older people has been performed thus far. However, the original evaluations of female HPV vaccination in relation to the optimal age of catch-up are very relevant to evaluations of untargeted vaccination to older ages [33]. Some of the original catch-up vaccination modelling literature was difficult to interpret because a wide age range of vaccination was modelled. This averaged out the costs and benefits across ages, and the specific age at which catch-up became cost-ineffective was not apparent. The cost-effectiveness of catch-up campaigns to older ages is variable and depends on the age distribution of sexual partnerships, vaccine price, uptake and the threshold for cost-effectiveness [41]. Furthermore, an evaluation will need to be done separately for females and males, since firstly, the cost-effectiveness in males differs from females (given lower overall disease burden) [13]; and secondly, cost-effectiveness depends on herd effects from established vaccination programs in females.

A previous review of the literature in relation to modelling catch-up noted that the validity of the conclusions about catch-up vaccination and vaccination of older women depends on assumptions made about exposure of each of the age cohorts to type-specific HPV infection, since vaccination against a particular HPV type is maximally effective in women who are HPV DNA negative for that type [13]. Also, appropriate modelling of naturally-acquired immunity to infection strongly impacts the calculation of vaccine effectiveness in catch-up cohorts; therefore, if naturally-acquired immunity is not included in the model, then catch-up vaccination appears more cost-effective. Therefore, well-calibrated dynamic models should be equipped to simulate and explore various assumptions around naturally-acquired HPV immunity, especially when addressing the differential effects of catch-up vaccination.

HPV-FRAME reporting standards for evaluations of vaccinating older people are presented in Appendix Table A3. Overall, for evaluations of vaccinating older people it will be critical to understand a model's assumption around the possibility of new infection in older individuals progressing, naturally-acquired immunity, and re-emergence of latent infections and their clinical meaning. These assumptions will profoundly impact the modelled simulation of vaccine impact if administered at older ages and hence impact cost-effectiveness. Since this is an emerging area, no example application is presented.

Table A3

HPV-FRAME reporting standards for evaluations of vaccination at older ages∗.

Inputs	Detail	How to report	Report by sex?
Natural history	Nature history structure used in the model (including progression and regression from high grade to low grade disease/productive HPV infection).	Report progression status by age and/or by time since infection and HPV type. (Comment on whether progression rates change according to time since HPV infection or by age).	As appropriate to evaluation
	Rate of clearance of HPV infection	Report by age and/or by time since infection and HPV type	As appropriate to evaluation
	Rate of loss of naturally acquired immunity	Report by age and/or by time since infection and HPV type	As appropriate to evaluation
	Simulation of latency by HPV type (handling of apparently new infections in older women – is the possibility that some are reactivated latent infections explored in sensitivity analysis?)	Report proportional reduction in true HPV incidence, by age or time since infection and HPV type. Also report assumptions about re-activated latent infection (i.e. whether there is differential progression to precancer or cancer)	As appropriate to evaluation
Vaccination∗	Vaccine coverage at older ages	Report by age	As appropriate to evaluation
	Whether screen-and-vaccinate is being modelled, or just vaccination at older ages (without linking to screening/HPV status)	Report screening strategy (if any including screening and triage test, interval and age-range and management strategy for those who are vaccinated)	As appropriate to evaluation

HPV-FRAME core reporting standards and reporting standards for models of vaccination in adolescent individuals (Table A1) are also applicable.

Domain 3. additional issues for reporting modelled evaluations of targeted HPV vaccination in men who have sex with men (MSM)

Models of HPV prevention in MSM differ from modelling HPV infection in heterosexuals. HPV epidemiology has distinct features among MSM with implications for transmission and control (e.g. higher probability of co-infection with HIV, different pattern of age-dependency in HPV prevalence). Selective vaccination of MSM may only be realistically achieved after sexual debut which has implications for vaccine delivery, effectiveness and impact. Thus, estimating the cost-effectiveness of targeted vaccination of MSM requires additional considerations compared to assessing the cost-effectiveness of vaccinating all adolescent boys.

The burden of the disease needs to be assessed in advance and needs to be made clear at the onset of any modelling analysis. However, estimating disease burden among MSM is hindered by sexual orientation not being captured by cancer registries. The burden of anal cancer is certainly higher among MSM than heterosexual males [42,43]. Currently, data on the natural history and epidemiology of HPV and HPV-related disease among MSM are more limited than that for women. Overall, there is still a large amount of uncertainty regarding the vaccine-preventable burden of disease in MSM.

Considerations for evaluations of HPV vaccination in males and in older individuals (Domain 2) will also apply to evaluations of targeted vaccination of MSM. In practice, the burden of disease that can be prevented by vaccinating at an older age is a function of both an individual's past history of infection (and possible immunity), as well as their risk of being infected in the future and of that infection progressing to cancer. The MSM population in many high-income countries with relevant data is typified by sustained high levels of partner change and thus prior exposure to HPV may be high in a sexually active population but risk of future infection is also potentially higher. Furthermore, MSM have a higher risk of exposure to HIV and there is evidence that the prevalence of oncogenic HPV is higher among HIV-positive compared to HIV-negative MSM [44]. There is also evidence that HIV-positive MSM have increased anal and penile oncogenic HPV incidence rates, decreased anal clearance rates compared to HIV-negative MSM [[45], [46], [47], [48]], and increased risk of anal cancer [49]. For models which include HIV, reporting standards outlined in Domain 4 are also applicable.

There are some key issues that need to be considered in the transmission dynamics of HPV models of MSM. These include consideration of whether the infection of multiple anatomic sites and/or the sex role separation (insertive vs receptive intercourse) should be made explicit, the typically high variance in MSM contact rates, how to define viral transmissibility in terms of mode and type (per-partnership or per-contact transmissibility) and how to model prevalence by age. Modelling networks including consideration of bisexual men is required to fully capture the potential effects in MSM of some herd protection which may be derived from female vaccination.

Only a few trials to date have assessed vaccine efficacy in MSM [[47], [48], [49]]. Models considering the incremental benefit of universal male vaccination in addition to universal female vaccination should ideally take into account the disease burden in MSM versus exclusively heterosexual men and lower herd protection (from female vaccination) for MSM compared to exclusively heterosexual men. Ignoring this would yield a conservative estimate for the incremental benefit of both universal boys and targeted MSM vaccination (compared to female-only vaccination). Models of targeted MSM vaccination should likewise consider whether universal male vaccination is also taken into account as part of the comparator - this will depend on the vaccination program in place, the age groups targeted for immunization and the time horizon of the analysis. The route by which vaccines can be delivered to MSM are decidedly different from those for vaccination of adolescents or adult women, with implications in terms of vaccine coverage by age that need to be considered, possibly drawing on delivery mechanisms and coverage levels achieved for other selective MSM vaccination programs such as hepatitis B vaccination.

HPV-FRAME reporting standards for cost-effectiveness models of targeted HPV vaccination in MSM are presented in Appendix Table A4. An example application to an evaluation of targeted HPV vaccination of MSM in the UK [50] is presented in Appendix Table A5.

Table A4

HPV-FRAME reporting standards for cost-effectiveness models of vaccination in MSM.∗

a) Inputs	Reported? (Y/N)	Report by Age? (Y/N)	Comments
MSM-specific disease burden	Y	Y	State where incidence rates have been derived from
Interaction between HIV and HPV	Y	Y	If HIV is modelled state what interactions are included: risk of infection, risk of progression etc.
HIV prevalence	Y	Y	Report for HIV-positive and negative individuals
Vaccine coverage	Y	Y
Prior exposure	Y	Y	If modelled or assumed
b) Outputs	Reported? (Y/N)	Report by Age? (Y/N)	Report as calibration or validation target? (Y/N)
Reduction in disease incidence	Y	Y	Y
Impact assessment (e.g. relative and absolute reductions in disease outcomes)	Y	N	Y. State outcomes by cancer type or whether composite outcome measures

MSM: men who have sex with men; N: no; Y: yes.

HPV-FRAME core reporting standards are also applicable.

Table A5

Example application of HPV-FRAME guidelines to Lin et al., 2016 [50].

Inputs	Reported? (Y/N)	Report by Age? (Y/N)	Comments
MSM-specific disease burden	Y	Y	Incidence rates by age derived from registries combined with population attributable fractions and relative risks
HIV prevalence	Y	Y	Split by diagnosed and undiagnosed
Time from HPV infection to cancer	Y	N	Natural history model for anal cancer giving rate of progression to cancer precursors and cancer. Proportionate outcomes for other cancers
Vaccine coverage	Y	Y	Age bands 16–25, 16–30, 16–35, 16-40y, 50% among GUM attenders (16.7% of all MSM)
Prior exposure	Y	Y (implicitly)	Estimated by transmission-dynamic model

GUM: genitourinary medicine clinics; PP: per protocol; N: no; Y: yes.

N: no; Y: yes.

Domain 4. modelled evaluations of HPV vaccination or cervical cancer screening in individuals living with HIV (ILWH)

Women living with HIV have higher HPV acquisition risks [51,52], decreased clearance of HPV and precancerous lesions [52,53], increased progression to high-grade pre-cancer [54,55], higher cervical cancer incidence [56] and higher cervical cancer-related mortality compared to HIV-negative women [57]. Antiretroviral therapy (ART) for HIV can decrease the risk of cervical cancer, but this benefit is likely dependent on early ART initiation and sustained viral suppression [58,59]. Both the bivalent and quadrivalent vaccine have been evaluated for efficacy in HIV-infected women [[60], [61], [62]].

Similarly, in men infected with HIV, HPV acquisition and prevalence of anal oncogenic HPV infections is higher [63,64], HPV persistence is greater [65], HPV clearance is decreased [64] and there is an increased progression to high-grade anal intraepithelial neoplasia (AIN) [66] and an increased risk of anal cancers [67]. The evidence regarding the impact of ART on AIN in men living with HIV is contradictory. Some studies have not found greater protection against AIN [68] or an impact on the natural history of AIN with use [[68], [69], [70]] whereas others have found that prolonged use of ART (>2 years) is associated with a reduced risk of high-grade AIN [71,72].

Key considerations for modelling HPV-associated cancers in ILWH are presented in Appendix Table A6 and include capturing differences in HPV pathogenesis by HIV status. Relative risks can be used as multipliers to increase rates of HPV acquisition, persistence and progression for ILWH [73]. This approach would have the benefit of also capturing disparities in access to care; ILWH with low counts of CD4 T lymphocytes are at the highest risk of precancer and HPV-associated cancers but are likely to have the lowest access to HIV care including cancer screening. Protocols for cervical and anal cancer screening and HPV vaccination can differ by HIV status; and, for outcomes among ILWH, these should be reflected in the model design and parameterization. Evaluations involving men living with HIV may also need to consider reporting standards presented for Domain 3 (MSM).

Table A6

HPV-FRAME reporting standards for models of HPV-associated cancers among individuals living with HIV (ILWH).∗

a) Inputs	Reported? (Y/N)	Report by age? (Y/N)	Report by sex? Y/N (B, M, F)
HPV prevalence, CIN prevalence and cervical cancer incidence by HIV status	Y	Y	Y (implicitly)
HPV disease multipliers on HPV acquisition, progression from HPV infection to cancer (or relevant precursors, if modelled) for HIV-infected women/men	Y	If applicable	Y (implicitly)
HPV-associated cancer mortality by HIV status (and CD4 count if modelled)	Y	Y	Y
Relevant co-morbidities	Y	N	Y
HPV-associated screening sensitivity/specificity by HIV status	Y	If applicable	Y (implicitly)
b) Outputs
Reduction in cervical cancer incidence over time by HIV status (and CD4 count and ART status if modelled)	Y	Y	Y (implicitly)

ART: Antiretroviral therapy; CIN: cervical intraepithelial neoplasia; CD4: CD4 T lymphocytes; HIV: human immunodefiency virus; N: no; Y: yes.

HPV-FRAME core reporting standards are also applicable.

This domain is an emerging area in the literature; although some evaluations have been performed [74,75] an example has not been provided for this type of evaluation.

Domain 5. additional issues in reporting modelled evaluations in LMIC

Questions regarding the comparative and cost-effectiveness of HPV prevention strategies are most pressing in low- and middle-income countries (LMIC), where the burden of HPV-related disease is greatest. These questions have motivated a growing number of model-based analyses to date, which have focused primarily on cervical cancer screening strategies and evaluations of HPV vaccination alone or with screening. Screening 1–3 times per lifetime between the ages of 30–50 years has been reported to be cost-effective in many LMIC evaluated and primary HPV testing is predicted to be more effective when compared to cytology or visual inspection with acetic acid [[76], [77], [78], [79]]. Furthermore, HPV vaccination of adolescent girls with bivalent, quadrivalent or nonavalent HPV vaccine is likely to be cost-effective in LMIC at low vaccine prices (e.g. prices available to Gavi, the Vaccine Alliance and the Pan American Health Organisation Revolving Fund) [27]. The opportunity to pair vaccination with some screening in adulthood has also been shown to be promising in some countries [18,[80], [81], [82], [83], [84], [85], [86], [87], [88], [89], [90], [91], [92], [93], [94], [95]]. Screening has also been found to be cost-effective in several analyses of HIV-infected women and populations with a high burden of HIV [[96], [97], [98]].

For models evaluating HPV prevention in LMIC there is still uncertainty regarding the burden of disease, as well as the health impact and the costs of delivering health interventions. In terms of costs, the WHO states that a fixed cost–effectiveness threshold should never be used as a stand-alone criterion for decision-making; although cost-effectiveness ratios are informative they also need to be considered alongside affordability, budget impact, fairness, feasibility and any other criteria considered important in the local context [99].

In addition to the rigorous reporting guidelines specified in the general framework, additional key parameters and assumptions for models of HPV prevention in LMIC are listed in Appendix Table A7. An example application to an evaluation of the comparative and cost-effectiveness of HPV-based cervical cancer screening algorithms in El Salvador [77] is presented in Appendix Table A8. Outputs were also reported as per core reporting standards (Table 4). It should be noted that the WHO is currently drafting a strategic plan for cervical cancer elimination, which specifies goals and targets for vaccination, cervical screening (twice in a lifetime) and pre-cancer and cancer treatment scale-up in LMIC. Ongoing modelling is supporting the estimates of impact and cost-effectiveness of the WHO strategic elimination plan.

Table A7

Additional HPV-FRAME reporting standards for models of HPV prevention in LMIC.#∗

Inputs	Reported? (Y/N)	Report by Age? (Y/N)	Report by sex? (Y/N)	Comment
HIV prevalence rates, if endemic in country	Y	N	Y
Description of any opportunistic or pilot/demonstration screening projects ongoing	Y	Y	Y
Costs	Y	N	N	Currency conversion (e.g., PPP, tradeable versus non-tradeable), indicative willingness to pay threshold to help inform decision making.

Abbreviations: PPP: purchasing, power, parity.

Models need to specify whether data came from the country for which the evaluation is conducted or whether the data was extrapolated from another setting. If data is extrapolated, the method used to do so needs to be described.

HPV-FRAME core reporting standards are also applicable.

Table A8

Example application of HPV-FRAME guidelines to Campos et al. (2015) [77].

Inputs	Reported? (Y/N)	Reported by Age? (Y/N)	Reported by sex? (B, F, M)	Comment
HIV prevalence rates	N	N	N
Description of any opportunistic or pilot/demonstration screening projects ongoing	Y	Y	F-only	Pap every 2 years
Costs	Y	N	F-only	Currency conversion: NA (local currency costs = US$)
Outputs†	Report by Age?(Y/N)	Report by sex? (B, F, M)	Report as calibration or validation target? (Y/N)	Comments
HPV prevalence, pre-intervention	Y	F-only	Y	Used for calibration
Prevalence of HPV16 and HPV18 in cervical cancer	N	F-only	Y	Used for calibration
Cervical cancer incidence	Y	F-only	Y	Used for validation
Relative reduction in lifetime risk of cervical cancer	N	F-only	N	HPV screen-and-treat reduced lifetime cancer risk by 60%.
Total lifetime cost	N	F-only	N	2012 US$
Life expectancy	N	F-only	N	Discounted life expectancy
Incremental cost-effectiveness ratio	N	F-only	N	2012 US$/year of life saved.

B: both sexes, F: female only; M: male only.

Outputs reported here for illustrative purposes.

Domain 6. framework for models of cervical screening, in unvaccinated and vaccinated populations and integrated cervical screening and HPV vaccination approaches

Cervical screening using cytology has been the basis for reducing cervical cancer incidence and mortality rates in high-income countries over the last several decades. However, two developments have required screening to be re-evaluated in many countries. The first is the emergence of a wealth of evidence on the improved sensitivity of HPV DNA testing compared to cytology for detection of precancerous lesions and the second has involved the large-scale deployment of prophylactic HPV vaccination [100].

Cervical screening evaluations consider many policy alternatives including cytology technologies (conventional vs. liquid-based cytology), HPV testing vs. cytology vs. co-testing using both technologies, different test technologies for HPV screening, different triaging approaches for screen-positive women, and the role of self-collection for HPV testing as a screening approach. A newer challenge has involved the need to evaluate cervical screening in the context of HPV vaccination, which requires the complex interactions between the two preventative mechanisms to be simulated. Cohorts of women who have been vaccinated against HPV in various countries are entering the target cervical screening age range. Over the next few decades, the risk of CIN2+ in vaccinated age cohorts will be substantially lower than in unvaccinated age cohorts and will additionally decrease over time because of increasing herd effects, the introduction of new vaccines and potentially increased vaccine uptake in the future. Prospective studies alone do not provide sufficient guidance to policy makers, and integrated vaccination-screening models are needed to project the long-term effects and value of vaccination with continued screening.

A recent systematic review of model-based cervical screening evaluations [10] identified only a small number of screening evaluations conducted for vaccinated women. Some evaluations have found that cervical screening below the age of 25 years in vaccinated cohorts is unlikely to be cost-effective [[101], [102], [103], [104], [105], [106]].

Primary HPV screening is likely to be cost-effective in screening cohorts vaccinated with either the bivalent or quadrivalent HPV vaccine when the screening interval is at least 5 years [[101], [102], [103], [104], [105], [106]]. It has also been reported that cervical lesions arising from HPV infections from types other than HPV16/18 have a lower progression rate than HPV16/18 [107] and are associated with a low recurrence disease rate [108]. For certain countries, evidence in vaccinated women of reduction in cervical pre-cancer and/or cancer is likely to be the impetus for changing their approach to screening for these women.

Natural history; algorithms for screening, surveillance, treatment, and post-treatment; adherence to screening and follow-up recommendations; and test characteristics are key areas to consider in cervical screening evaluations. It is recommended that the following parameters are reported: screening, triage, surveillance, and treatment algorithms; screening and follow-up adherence; test characteristics; calibration and/or validation with screening turned on (against screened population outcomes); and comparisons to other models answering a similar question as a form of model corroboration.

HPV-FRAME reporting standards for models that examine the impact of cervical screening alone, in either vaccinated or unvaccinated women, adapted from previous work [109], are presented in Appendix Table A9. The Reported Data should clarify that each reporting standard should encompass both primary and secondary tests, when relevant. Additional reporting standards for models that examine the combined impact of combinations of cervical screening and vaccination are presented in Appendix Table A10. An example application of the framework to an evaluation of primary HPV testing versus cytology-based cervical screening in women in Australia, both vaccinated for HPV and unvaccinated [110] is presented in Appendix Table A11.

Table A9

HPV-FRAME reporting standards for models of cervical screening.∗

Inputs	Reported? (Y/N)	Report by age? (Y/N)	Comment∗∗
Routine screening behaviour (routine and follow-up and test-of-cure)	Y	Y	Screening age-group, percentage of women who are never screened, and percentage of women not screened.Clinician-collected vs self-collected sampleOrganisation of screening (opportunistic vs programmatic)Proportion attending (or rate) screening for each age group at specified interval cycles (on-time screening)Proportion attending (or rate) screening beyond screening intervals i.e. over-screening (important for costs)
Screening test(s) and colposcopy accuracies	Y	N	Report by test result: cytology grade for cytology or HPV type for HPV tests; report on sensitivity and specificity or equivalent specification if test accuracy.Proportion unsatisfactory tests
Abnormal test management (primary and triage)	Y	N	Distinction of result grade (e.g. cytology grade).Separate management by test resultSensitivity and specificity values of cytology or other test result
Diagnostic follow-up of abnormal tests	Y	N	Indication of diagnostic follow-upDistinct colposcopy and biopsy outcomes
			Sensitivity and specificity values of colposcopy
Management by disease grade (confirmed disease)	Y	N	Indication of HSIL managementDistinct HSIL management approachesTreatment efficacy of treated HSIL
			Follow-up of treated disease
Sources of information for screening structure and parameterization	Y	N	GuidelinesSurvey or surveillanceExpert opinion/personal communication
			Assumption

Abbreviations: HSIL: high grade squamous intraepithelial neoplasia

HPV-FRAME core reporting standards are also applicable.

Each reporting standard should encompass both primary and secondary tests, where relevant.

Table A10

Additional HPV-FRAME reporting standards for integrated models of cervical screening and vaccination.∗

Inputs	Reported? (Y/N)	Report by age? (Y/N)	Report by sex? (B, F, M)	Comments
HPV type incidence, clearance and progression rates	Y	Y	F	HPV16 and HPV18 need to be modelled separately. HPV31/33/45/52/58 need to be modelled separately when evaluating screening in cohort vaccinated with 9-valent vaccine or when cross-protection from 2/4-valent vaccine is assumed.
Herd effect	Y	Y	F
Association between vaccination and screening uptake	Y	Y	F
Screening test(s) and colposcopy accuracies	Y	N	F	Report by test result: by grade for cytology; by HPV type for HPV tests; report on sensitivity and specificity or equivalent specification if test accuracy.Proportion unsatisfactory tests
Fixed – variable costs∗	Y	N	F	Report all vaccine cost assumptions including system-level infrastructure, administration, cost per dose, cost per vaccinated individual

B: both sexes; F: female; M: male; N: no; Y: yes.

HPV-FRAME core reporting standards are also applicable.

Table A11

Example application of HPV-FRAME guidelines to Lew/Simms et al., 2017 [110].

Inputs	Reported? (Y/N)	Reported by age? (Y/N)	Comments
Routine screening behaviour	Y	Y	Screening age-group: the strategies evaluated involve:-2y conventional cytology-based screening for women 18-69 years (comparator); 3y cytology screening for women aged 25-49 years combined with 5-yearly cytology screening for women aged 50-64 years; 5-yearly primary HPV screening for women aged 25-64 years
			- The modelled age-specific screening attendance rate for comparator strategy (i.e. current program) was based on the analysis of Victoria Cytology Cervical Register data; rates modelled for other screening strategies were derived from the currently observed Australian data and informed by other countries’ experience
			- 2.1% for conventional Pap smear and 1.8% for manually-read or image-read cytology.
Abnormal smear management	Y	N	- Cytology results was separated into 5-grades: negative, ASC-US (pLSIL), LSIL, ASC-H (pHSIL), HSIL+
			- Different management was modelled for negative, low-grade and high-grade cytology results
			- For CIN2+ detection at ASC-US threshold:Conventional cytology: 74.1% (sensitivity), 95.7% (specificity)Manually-read LBC: 74.1% (sensitivity), 94.7% (specificity)Image-read LBC: 81.2% (sensitivity), 94.5% (specificity).For CIN3+ detection at ASC-US threshold:Conventional cytology: 75.6% (sensitivity), 95.3% (specificity)Manually-read LBC: 83.9% (sensitivity), 94.3% (specificity)Image-read LBC: 80.0% (sensitivity), 96.6% (specificity)
Diagnostic follow-up of	Y	N	-Indication of diagnostic follow-up stated
abnormal cytology			- Colposcopy outcomes including unsatisfactory, satisfactory and normal, and satisfactory and abnormal. Biopsy outcomes including negative/HPV infected, CIN2, CIN2/3 and cervical cancer.
			- The modelled colposcopy positive rate are 40.2% for women without CIN, 76.5% for women with CIN1, and 88.4% for women with CIN 2 or worse
			- Follow-up of treated disease: Annual follow-up with HPV and cytology co-testing until the woman obtained negative outcome on both test in two consecutive 12 months follow-up visit (test-of-cure)
Inputs	Reported?	Reported by age (Y/N)	Lew/Simms et al, 2017
Management by CIN grade	Y	N	- Indication of CIN management stated
			- Distinct CIN management approaches: women with negative histology outcome or CIN1 were referred to follow-up; women with CIN2/3 were referred to precancer treatment.
			- Treatment efficacy of treated CIN: 100% (assumed a 10% failure rate in the first treatment but all women with unsuccessful treatment were assumed to be identified and had successful second treatment)
Sources of information for screening structure and parameterisation	Y	N	Australian National Health and Medical Research Council (NHMRC) 2005 guideline and the new clinical management guidelines that developed in 2015–16 to support the new National Cervical Cancer Screening Program were modelled.
			Survey/ surveillance:- The Victorian Cytology Cytology Register data was analysed to inform the screening coverage, and the proportion of early-screening, ‘on-time’ screening and late-screening.
			- Colposcopy data collected at Royal Women Hospital was used to inform the compliance rate to colposcopy follow-up and colposcopy accuracy.- Screening outcomes reported in the annual National Cervical Cancer Screening Program monitoring report (i.e. age-specific proportion of cytology test with low-grade and/or high-grade outcomes, age-specific proportion of women with abnormal high-grade histology per 1,000 women screened etc) were used to calibrate/validate model’s screening outcome.- Age-specific cervical cancer incidence and mortality rate reported by the Australian Institute of Health and Welfare was used for model calibration.- Cost was assumption was informed by Australian Medical Benefit Scheme; assumption were made where applicable
			- HPV vaccination coverage was modelled based on the National HPV Vaccination Program Register data.
			- Expert opinion of the Renewal Steering Committee was seek for management that was not specified in the guideline for cervical screening.
			Cost of manually-read LBC, image-read LBC and HPV test (when it was used as primary screening test) were based on assumption

ASC-H: atypical squamous cells cannot exclude high-grade squamous intraepithelial lesion; CIN: cervical intraepithelial neoplasia; HSIL: high-grade squamous intraepithelial lesion; LBC: liquid based cytology; LSIL: low-grade squamous intraepithelial lesion; pHSIL: possible high-grade squamous intraepithelial lesion; pLSIL: possible low-grade squamous intraepithelial lesion.

Domain 7. Additional issues for reporting models of alternative HPV vaccines and reduced-dose schedules

With the availability of alternative HPV vaccines, strong evidence for the efficacy of two-dose schedules and increasing signs of promise of one-dose schedules, model-based analyses have emerged to address these questions. Key drivers of reduced dose evaluation outcomes include their lower costs and the duration of vaccine protection which is a function of waning as protection must be maintained during key ages of sexual activity. A long duration of vaccine efficacy is required in order for a reduced-dose schedule to yield substantial benefits, although these may differ by country, depending on sexual behaviour, age-mixing and other cofactors. Analyses for reduced-dose schedules are subject to the same reporting guidelines as specified in the general framework; however, HPV-FRAME provides additional reporting guidelines for comparative evaluations of different vaccine types and reduced-dose schedules. Comparative vaccine or dose evaluations that have been performed to date include bivalent versus quadrivalent HPV vaccines [14,111]; bivalent or quadrivalent HPV vaccines versus nonavalent [112]; and 1-dose versus 2-doses versus 3-doses of the HPV vaccine [113,114]. These evaluations have concluded that the cost threshold for the additional amount that could be paid for higher valency vaccines in high-income countries like the USA or the UK is US$50 per dose for the quadrivalent HPV vaccine (versus bi-valent) [14,111] and ~US$18–28 per dose for the nonavalent HPV vaccine (versus the bi- or quadrivalent vaccine) [115,116] or ~$3 per dose in low to middle income countries (LMIC) [83]. Fewer doses can be more cost-effective than a 3-dose regimen but that is dependent on assumptions regarding the duration of vaccine protection of a reduced-dose schedule, which needs to be at least 20 years [114,117].

HPV-FRAME reporting standards for models of alternative HPV vaccines and reduced-dose schedules are presented in Appendix Table A12. An example application of an evaluation presenting a comparison of two dose and three dose HPV vaccine schedules [114] is presented in Appendix Table A13.

Table A12

HPV-FRAME reporting standards for evaluations assessing alternative vaccine types or reduced-dose schedules.∗

a) Inputs	Reported? (Y/N)	Report by Age? (Y/N)	Report by sex? (B, F, M)	Comments
Vaccine efficacy/waning	Y	Y	Y	Level and duration of protection (waning function) by dose, type, age. (For 2-dose regimen: specify if each dose received is modelled and the efficacy modelled for each dose).
Timing between doses (for 2-dose)	Y	N/A	Y
Vaccine cross-protection	Y	N/A	N/A	Level and duration of protection by dose and type
Cost	Y	N/A	N/A	Cost per dose, cost per vaccinated individual, and all vaccine cost assumptions including infrastructure, administration.
b) Outputs	Reported? (Y/N)	Report by Age? (Y/N)	Report by sex? (B, F, M)	Report as calibration or validation target? (Y/N)
Threshold cost per dose	Y	N/A	N/A

B: both sexes, F: female only; M: male only; N: no; N/A: not applicable; Y: yes.

HPV-FRAME core reporting standards are also applicable.

Table A13

Example application of HPV-FRAME guidelines to Jit et al., 2015 [114].

Inputs	Reported? (Y/N)	Report by Age? (Y/N)	Report by sex? (B, F, M)	Comments
Vaccine efficacy/waning (by dose, type)	Y	Y (implicitly)	N/A	Full protection against 16/18 (2- and 3-dose)Waning: 10, 20 or 30 years for 2-dose with exponential or uniform waning functionsAge dependency: 2-dose regime only given to 12-year olds in modelled scenarios.
Timing between doses (for 2-dose)	Y	N/A	N/A	2-dose regime only given to 12-year olds in modelled scenarios.
Vaccine cross-protection (by dose, type)	Y	Y (implicitly)	N/A	Partial efficacy against 31/33/45/52/58 assumed for 2- and 3-dose; sensitivity analysis with no cross-protection for 2-dose
Cost per dose/per vaccinated individual	Y	N	N/A	Cost per dose assumed constant
Output	Reported? (Y/N)	Report by Age? (Y/N)	Report by sex? (B, F, M)	Report as calibration or validation target? (Y/N)
Threshold cost per dose	N	N	N

B: both sexes, F: female only; M: male only; N: no; Y: yes.

Conclusions and next steps

The HPV-FRAME initiative aims to complement general principles of good modelling practice as articulated by ISPOR, and other modelling guidelines via the provision of specific guidance on issues of relevance in HPV modelling. Ensuring clear reporting of HPV prevention models through the use of HPV-FRAME will allow the end-user to judge a model's validity, ascertain areas of simplification and whether these were appropriate to the decision question, and assess the degree of uncertainty in a decision process. Overall, the use of the framework should increase the quality and standard of reporting evaluations of HPV prevention.

HPV-FRAME was developed by a core steering committee, with input from contributors from the wider HPV expert community who expressed an interest in participating in the initiative. It is possible that under a different panel of contributors some of the parameters included in the current framework may have varied to a certain degree. However, the framework was based on the identification of sources of bias and quality issues from systematic review of the relevant literature and has been developed by modellers within the HPV modelling community using a well-established process.

HPV modelling is a dynamic field with new data and evidence emerging. There is still a large amount of uncertainty in many of the inputs to models of HPV prevention. For example, in relation to models of alternative HPV vaccines and reduced-dose schedules the relative properties among the different vaccines and the level of protection with reduced-dose schedules has not been fully determined. Model-based analyses can integrate the existing evidence and explore these uncertainties to provide timely input to decision making. However, it is incumbent upon the investigators to clearly specify the assumptions regarding these uncertainties in order to improve transparency, which has been the motivation for HPV-FRAME. To account for changes and other issues such as the possibility of a specified repository for models’ documentation, we aim to review and update HPV-FRAME regularly through a review of the recent literature and input received from interested parties through the dedicated HPV-FRAME website (http://www.hpv-frame.org).

Funding sources

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflicts of interest

KC is co-principal investigator of an investigator-initiated trial of cytology and primary HPV screening in Australia (“Compass”), which is conducted and funded by the VCS Foundation, a government-funded health promotion charity. VCS has received equipment and a funding contribution for the Compass trial from Roche Molecular Systems and Ventana USA. However neither KC, nor her institution on her behalf (Cancer Council NSW) has received direct or indirect funding from industry for Compass Australia or NZ or any other project. JB has received grants from the European Community during the conduct of the study and personal fees from GSK and Merck and non-financial support from DDL outside the submitted work. The fees from GSK and Merck were collected by the employer. JJK, SK, RB, JAB, NC, CJ, MS, KTS, MAS, LSV, MB and MJ have nothing to disclose.