Methods and quality of disease models incorporating more than two sexually transmitted infections: a protocol for a systematic review of the evidence.

INTRODUCTION: Disease models can be useful tools for policy makers to inform their decisions. They can help to estimate the costs and benefits of interventions without conducting clinical trials and help to extrapolate the findings of clinical trials to a population level.Sexually transmitted infections (STIs) do not operate in isolation. Risk-taking behaviours and biological interactions can increase the likelihood of an individual being coinfected with more than one STI.Currently, few STI models consider coinfection or the interaction between STIs. We aim to identify and summarise STI models for two or more STIs and describe their modelling approaches. METHODS AND ANALYSIS: Six databases (Cochrane, Embase, PLOS, ProQuest, Medline and Web of Science) were searched on 27 November 2018 to identify studies that focus on the reporting of the methodology and quality of models for at least two different STIs. The quality of all eligible studies will be accessed using a percentage scale published by Kopec et al. We will summarise all used approaches to model two or more STIs in one model. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework will be used to report all outcomes. ETHICS AND DISSEMINATION: Ethical approval is not required for this systematic review. The results of this review will be published in a peer-reviewed journal and presented at a suitable conference. The findings from this review will be used to inform the development of a new multi-STI model. PROSPERO REGISTRATION NUMBER: CRD42017076837. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

This review will summarise the methodology that was used to model more than two sexually transmitted infections (STIs) in a single disease model.

This review is not limited to a certain kind of modelling approach or intervention.

Focus on summarising different techniques to model interacting STIs, excluding the potential interactions of STIs with non-STIs.

Introduction

Disease modelling

Disease models attempt to simplify a complex topic to a single aspect of interest. Computational disease models, for example, examine the spreading of diseases within a population of interest and extrapolate the economic effect a potential intervention might have on this population.1 2 With increasing computational power, disease modelling has become an important approach to inform healthcare decisions.3

To set up a disease model either specialised modelling software (eg, TreeAge4) or more general software (like Excel5) can be used. Specialised modelling software comes with a greater functionality, whereas non-expert users are more familiar with general software, which comes to the cost of longer calculation times to obtain results from the model.6

In general, disease model can be described using different dimensions. The most important dimensions to describe disease models are explained in the following paragraphs.

There are two types of approaches: individual-based disease models and compartment-based disease models. Individual-based models are computationally more intense as they simulate each person within the modelled cohort, whereas compartmental models look at proportions of the cohort that are in the same health state within the disease model. The modelled individuals of the model can be able to interact with each other, which is helpful to model infectious diseases but computationally more intense than the non-interacting modelling of population.1 7

The cohort of a disease model can either be open or closed. A closed cohort is defined at the beginning of the modelling process, and no new individuals can enter a closed cohort model during the simulation process. An open cohort model allows new individuals to enter the simulation, that is, keeping the simulated cohort at the same size as modelled individuals might die during the modelling time.8

Disease models can handle time in various ways. Markov-type models, for example, simulate time in a calendar-based manner, so that time always proceeds in steps of fixed length, also called cycles. Time could also be handled event-based, which means that the model skips periods when nothing happens and proceeds from one event to the next one.8–10

Different modelling approaches might be suitable to answer various kind of questions depending on the modelling setting.1

Sexually transmitted infections (STIs)

STIs are infections that are primarily transmitted through sexual contact. There are many demographic, behavioural and biological risk factors for acquiring STIs, including rate of partnership change, condoms use, and age.11 As these risks apply to all STIs in the same way, people with one STI may have another one simultaneously.

There is also biological evidence that the presence of one STI can harm the tissue integrity and therefore make a patient more susceptible of catching another infection at the same time.12 13

Disease modelling and STIs

Disease models for STIs have been used since the mid-1980s. Systematic reviews of single STI models, for example, for chlamydia and gonorrhoea, have already been undertaken.14 15 Some of these single STI models have informed government policy like the National Chlamydia Screening Programme in the UK.16

Rationale and research aims

Many disease models exist that examine single STIs, which is why there is good evidence on the methodology to develop such a model.1 3 However, some interventions may impact on several STIs at once. For example, interventions to increase condom use have shown to decrease the prevalence of STIs,17 18 whereas Cochrane did not find significant evidence that increased condom use will result in decreased transmission rates for STIs.19 This ambiguous situation underlines the importance to include more than one infection in an STI model to further examine potential effects of interventions targeting more than one STI at the same time. We aim to summarise the literature on the simultaneous modelling of at least two different STIs and report on the methodology and quality of these multi-STI models.

Registration

This review is registered with the international database of Prospectively Registers Systematic Reviews in health and social care under the registration number: CRD42017076837 (available at https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=76837).

Methods and analysis

Eligibility criteria

Inclusion criteria

Articles will be included if they:

Report on a disease model as one of the main aims of the paper.

Examine STIs at a population or cohort level to describe the spreading of the disease.

Cover two or more different STIs.

Contain an English title and abstract.

Type of study: governmental documents, journal articles, clinical trials with modelling component and theses.

Populations: sexually active population (or subgroups of it), examining at least horizontal STI transmission.

Interventions: any kind of intervention.

Exclusion criteria

Articles will be excluded if they:

Do not provide enough detail to extract the relevant output (online supplementary appendix A) to reproduce the modelling approach.

Focus on conditions other than STIs, for example, cancer, diabetes or tuberculosis.

Examine the interaction of a STI with a non-STI.

Examine only one STI, even if the model covers different strains of the same STI.

Examine the connection of a STI and its sequelae, for example, the progression from HIV infection to AIDS, without taking other STIs into account.

Type of study: qualitative work and case reports.

Populations: solely regarding vertical transmission.

Type of study

The search focuses on modelling studies, which also includes health economic analyses. As we want to extract much information, only articles that aim to report in a detailed way on a disease model and its development can be included. At least one of the objectives of the studies to be examined should be the detailed description of the model or the model development process.

The references of any review looking at multi-STI modelling studies will be included. We will add the mentioned modelling studies into the set of articles to be screened.

Clinical trials that have a modelling component and report on this with sufficient detail will be included.

Governmental documents and theses will be included.

Any other type of publication, for example, case reports or qualitative work, will be excluded.

Populations

Models have to look at the sexually active part of a population. Models that only look at subgroups of the sexually active part of the population, such as homosexual men, sex workers or young people will be included as well.

The review focuses on articles that examine horizontal transmission, for example, through sexual contact. If an article only simulates vertical transmission, that is, mother-to-child transmission (congenital transmission), it will not be included in the review. If an article considers horizontal and vertical transmissions, it will be included in the review. We will include studies looking at any kind of horizontal transmission. This could also be non-sexual transmission of STIs, for example, through needle sharing.

Interventions

Relevant modelling studies could examine a variety of different interventions, for example, screening, treatment or behaviour change approaches. This review does not aim to examine a certain type of intervention. It will look at models that are able to simulate interventions for at least two STIs at the same time. Therefore, articles reporting on models covering any intervention will be included.

If studies do not look at any specific intervention, but only introduce a generic model with the ability to examine several STIs at the same time, these studies will be included.

Outcomes

We want to get an overview over multi-STI disease models as well as the methodology used to implement these models. Therefore, we will extract the following information:

Modelling approach.

Entity level.

Open cohort versus closed cohort.

Interacting versus non-interacting population.

Time handling.

Data origin.

Cohort size.

Time horizon.

Modelling software.

List of included STIs.

Interaction.

List of sequelae of STIs.

Interventions.

Economic component.

Year in which the study has been conducted.

Input.

Country.

Output.

Customisability.

The data item ‘output’ will capture the different outputs the given model can calculate. These can be economic outcomes, like ‘cost per infection prevented’ or ‘costs per QALY gained’ or other numeric outcomes such as the ‘total number of infections’. All parameters that can be inputted in the model or have been used by the researchers are captured using the data item ‘input’. We will also capture the degree to which ‘input’ parameters can be modified if/when additional evidence becomes available or to modify the model to examine a different research question. This will be captured in the ‘customisability’ data item. All data items and the reasons for including them are reported more detailed in the online supplementary appendix A. Additionally, data identifying the study as year of publication, authors, title and journal will be captured.

Information sources

The following databases will be used to search for disease models: Cochrane, Embase, PLOS, ProQuest, Medline and Web of Science. Grey literature will be searched to find additional material using OpenGrey and New York Academy of Medicine Grey Literature Report. Conference proceedings will be found using Web of Science and Embase. PhD theses will be searched for using ProQuest, Web of Science, OpenGrey and the DART-Europe portal.

We will not contact authors to understand papers with incomplete information, as we regard the completeness of information given in an article as a quality indicator. The details provided in an article should be sufficient to understand and evaluate the described model.

Before starting the title and abstract screening, pilot searches were carried out to see whether the search terms yielded all known key articles. If the potentially relevant articles were not found we amended the search strategy.

Search strategy

The search strategy was adapted for different search engines and database to fit their syntax. All versions of the search strategy can be found in online supplementary appendix B. The general search strategy is split into three main fields: ‘disease models’, ‘sexually transmitted infections’ and ‘the interacting feature’. For each field, an individual search term was developed. These search terms were combined using ‘AND’.

The search terms are set up to have a high sensitivity to avoid missing potentially important articles. However, the search term has a lower specificity, which will be compensated by manually sieving out the irrelevant search results.

The search was carried out on 27 November 2017.

Selection process

All researchers involved in article screening and data extraction will attend a meeting before starting the screening/data extraction to develop a common understanding of the inclusion and exclusion criteria and to harmonise their understanding of the matter.

Two reviewers will independently conduct the title and abstract screening, with FS screening all articles and another reviewer screening 20%. During the title and abstract screening, the reviewers will not know the year of the study, the authors and the journal the study was published in. Arising conflicts will be solved by RH. If the second and third reviewer find that the first reviewer is overexclusive and has missed some papers, we will increase the percentage of papers to be reviewed by two reviewers by 10% and repeat the process.

All articles eligible for full-text screening will be independently screened by two reviewers, with FS screening all articles and another reviewer screening 20%. RH will solve any conflicts.

The data extraction will be done independently by two researchers. If any conflicts arise, a meeting will be set up to find consensus; if necessary, this will be moderated by RH.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses framework will be used to systematically report the results.20

Quality assessment

We will examine the quality of the included disease models. The quality will be assessed using a percentage scale.21 In this percentage scale, the quality of the model is examined in 17 dimensions, which are grouped in five categories:

Conceptual model.

Parameters.

Computer implementation.

Evidence from examining model performance.

Evidence from examining the consequences of model-based decisions.

Each dimension can be scored as ‘none’ (=0 points), ‘partial’ (=1 point) or ‘complete’ (=2 points). If some of the dimensions of the score are not applicable, this particular dimension will not be included in the calculation. The sum of all points over all applicable dimensions for a model is divided by the total points a model could have reached to calculate the percentage as a quality indicator.

Bias assessment

We will examine, using standard statistical methods, whether published models tend to report a positive effect of the examined intervention, so that we can uncover a potential publication bias.

Analyses

All reviewed studies will be reported in the final report, including their calculated percentage scale of the quality assessment value.

We will report how often each modelling approach has been used and how high the average percentage scale for each modelling approach was. We will report on the distribution of years in which the study has been conducted to understand potential trends in multi-STI modelling.

To understand which STI interaction have been the most relevant, we will set up a graph to show which STIs have been modelled together most frequently.

Subgroup analysis

We will use the percentage scale of the quality assessment to differentiate between models with higher and lower quality. We will compare these subgroups individually to examine the differences between those.

We will examine whether articles obtained through grey literature searches differ from articles obtained through searches in published literature databases.

To examine trends in the usage and variations in quality of modelling approaches, we will examine all modelling approaches separately.

It might be possible to use the same methodology to model STIs in a low-income country and in a high-income country or in a low prevalence versus high prevalence setting. Whereas the input of these models might and most certainly will differ, the technological approach in both settings could be the same. This is why we do not focus the search on a particular setting but examine the different income subgroups separately later on.

Study records and data management

The results from different search engines and databases will be downloaded. We will import these search results into a new and empty Endnote database. To guarantee the reproducibility of results, a backup of this database will be saved.

Endnote22 will be used to remove duplicates. An automated check for duplicate titles and year of publication will be applied. Each possible duplicate will be deleted manually to prevent deleting non-duplicates. A backup after the duplicate elimination will be saved.

All unique articles will be imported into the same Microsoft Access database.23 After title and abstract screening and after full-text screening, backups of the database will be saved.

Electronic input forms to capture the information retrieved from title and abstract screening, full-text screening and data extraction will be developed. These forms will be used by all researcher involved in article screening and data extraction. The forms will be developed in Microsoft Excel and VBA programming. The information extracted with the help of these forms are also stored in Excel workbooks, one for each researcher. These workbooks will then be imported into the Microsoft Access database for further processing, quality assessment, bias assessment and analyses.

Patient and public involvement

This review will not involve patients or any other member of the public. No patients or members of the public were involved in the development of the research question and outcome measures, the design of the study and the conduct of the study.

Ethics and dissemination

No patient level data is included or used in this systematic review.

The aim of this review is to describe the quantity and quality of published multi-STI models. A limitation of this is that we will not be able to conduct a meta-analysis of the findings. We will summarise all results, but we will not be able to produce aggregate figures such as funnel plots as it is likely that the models included in the review will report a range of outcomes with no single identifiable outcome to evaluate.

We will summarise the methodology that has been used to model STIs and assess the quality of existing multi STI models. We will not assess whether the most suitable approach to answer the research question of interest has been chosen by the authors of those disease models.

The modelling of STIs interacting with non-STIs, for example, HIV and tuberculosis, although being clinically important,24 will not be examined in this review, as it will not answer our research question and is beyond the remit of the review.

We will publish the results in a peer-reviewed journal and present them at a suitable conference. The findings of this review will be used to inform the development of a multi STI disease model, incorporating the most important STIs in a UK setting.

Amendments

This is the first version of the protocol. No amendments have been made to this version so far. If the protocol has to be amended, all amendments will be listed in a table in the final report on the results of the review.