Modeling of novel diagnostic strategies for active tuberculosis - a systematic review: current practices and recommendations.

INTRODUCTION: The field of diagnostics for active tuberculosis (TB) is rapidly developing. TB diagnostic modeling can help to inform policy makers and support complicated decisions on diagnostic strategy, with important budgetary implications. Demand for TB diagnostic modeling is likely to increase, and an evaluation of current practice is important. We aimed to systematically review all studies employing mathematical modeling to evaluate cost-effectiveness or epidemiological impact of novel diagnostic strategies for active TB.
METHODS: Pubmed, personal libraries and reference lists were searched to identify eligible papers. We extracted data on a wide variety of model structure, parameter choices, sensitivity analyses and study conclusions, which were discussed during a meeting of content experts. RESULTS & DISCUSSION: From 5619 records a total of 36 papers were included in the analysis. Sixteen papers included population impact/transmission modeling, 5 were health systems models, and 24 included estimates of cost-effectiveness. Transmission and health systems models included specific structure to explore the importance of the diagnostic pathway (n = 4), key determinants of diagnostic delay (n = 5), operational context (n = 5), and the pre-diagnostic infectious period (n = 1). The majority of models implemented sensitivity analysis, although only 18 studies described multi-way sensitivity analysis of more than 2 parameters simultaneously. Among the models used to make cost-effectiveness estimates, most frequent diagnostic assays studied included Xpert MTB/RIF (n = 7), and alternative nucleic acid amplification tests (NAATs) (n = 4). Most (n = 16) of the cost-effectiveness models compared new assays to an existing baseline and generated an incremental cost-effectiveness ratio (ICER).
CONCLUSION: Although models have addressed a small number of important issues, many decisions regarding implementation of TB diagnostics are being made without the full benefits of insight from mathematical models. Further models are needed that address a wider array of diagnostic and epidemiological settings, that explore the inherent uncertainty of models and that include additional epidemiological data on transmission implications of false-negative diagnosis and the pre-diagnostic period.

Introduction

The last decade has seen a dramatic increase in the development of novel diagnostic tests for active tuberculosis (TB) [1]–[4]. As a result policy makers must decide what novel diagnostics to implement with the limited resources available to them. To date the contribution of modeling to decision-making processes has been limited; despite the rapid growth of interest in TB diagnostics, in many cases models are not available to aid decision-making, particularly within country. As policy makers face questions about implementation of novel diagnostic strategies with substantial budgetary implications, the demand for these models will increase. Given the wide variety in modeling approaches, methods and objectives, both those developing and those using models can benefit from a synthesis and evaluation of current TB diagnostic modeling practices.

The development of novel diagnostics is not limited to technological advancements, but includes novel diagnostic algorithms and systems of diagnosis that allow novel or existing diagnostic tools to affect changes in clinical practice leading to improved patient outcomes[1]–[4]. Novel tests may boast a range of improved attributes. However the impact of any novel test depends on how that test is implemented and for what intended population. Novel tools cannot benefit TB patients unless they lead to earlier or more appropriate initiation of treatment. Likewise, evaluations of novel tests should account for the role of empiric therapy as individuals may still be treated in the absence of a positive test result, and this can have important implications on the impact of novel tools. New technologies are often (although not always) accompanied by increased costs, relating to the test itself and/or the systems into which they are implemented. In the face of multiple technological options and even more potential algorithms and implementation plans, deciding which novel diagnostic(s) to implement, and how, is a very complex process [5]. Such decisions must involve careful consideration of available strategies, requiring an understanding of the potential costs, population level impact for each test upon implementation in a given setting and uncertainty around these factors. Mathematical modeling, including transmission modeling, health systems modeling and economic evaluation can help to make these decision-making processes more transparent and data-driven; in the absence of such models, it is difficult to utilize existing data in a systematic fashion for decision-making.

The objectives of this paper were to systematically review all studies employing mathematical modeling to explore the potential epidemiological impact and/or cost-effectiveness of novel diagnostic tests for TB and to identify and discuss key methodological challenges and limitations of existing models and gaps in empirical evidence required to populate such models. This review was undertaken within the context of the TB Modelling and Analysis Consortium (TB MAC) meeting on modeling of novel TB diagnostics, where preliminary results were discussed.

Methods

We carried out a systematic literature review to identify existing TB modeling papers that evaluated novel tools or algorithms to diagnose active TB. Our focus was to identify models exploring novel diagnostic technology or models concerned with understanding the TB diagnostic pathway. Models that explored the impact of alternative case finding approaches not related to new diagnostic tools (e.g. active versus passive case finding) were not included.

Search strategy

The medical literature was searched for relevant studies in PUBMED; the detailed search string is included in the supplementary material. Personal libraries of TB MAC members were searched for modeling papers (free text search for ‘model’), and mathematical modeling journals were scanned for any papers describing TB models. Within this set of retrieved citations, potentially relevant records were identified through a text search for ‘diagn’ OR ‘test’ OR ‘screen’. Citations of all selected original articles [6] published after the 1st of January 2000 to March 1st 2013, were reviewed. The database of relevant citations is regularly updated by TB MAC and available online at: http://www.tb-mac.org/Resources/.

Inclusion and exclusion criteria

Papers were eligible for inclusion if they were written in English and published after the 1st of January 2000 to restrict to recent diagnostic strategies with relevance to current policy decisions. Papers were excluded if the model focused on diagnosing latent TB infection, or evaluated only diagnostic tools that fell within the existing standard of care at the time of publication, based on our understanding of TB diagnostic practices. Therefore this review was focused on modelling novel diagnostic tools or algorithms for active TB that are NOT currently in use as the standard of care. For the purposes of this review, the Xpert MTB/RIF test (Xpert, Cepheid Inc., CA, USA) was not yet considered as standard of care in any setting. The selected papers were independently assessed by two reviewers (AZ and RMGJH) for inclusion and all included studies underwent double data extraction by the same two reviewers; disagreements were resolved through consensus. In defining ‘mathematical model’ we followed Garnett et al [7] and included decision analytic, transmission, operational or within-host models, but excluded purely statistical models and studies using models to only estimate resource requirements without corresponding measures of health-related outcomes [8].

Papers were grouped into three subject areas that were not mutually exclusive: 1) population level impact/transmission, defined as models including a transmission component or measuring epidemiological impact at the population level; 2) health systems, defined as models including compartments that represent points of interactions between patient and health care providers or institutions; and 3) model based cost-effectiveness and cost-utility analyses, following the definitions by Drummond et al. [8]. To further clarify these definitions, transmission modeling generally assesses the population level impact of interventions in terms of changes in TB incidence, TB prevalence or TB mortality over time and can estimate the potential future benefit of introducing a novel diagnostic intervention on those population-level outcomes. Health systems models, which may include transmission and economic evaluations, involve explicitly modeling the interaction between patients and the health system or health care provider. These models are important for evaluating TB diagnostic interventions in the local context and can improve our understanding of operational elements such as patient and diagnostic delay or other key elements of the patient–provider interaction that may influence a test's potential impact and costs. We followed the definitions of Drummond et al. for cost-effectiveness and cost-utility analyses, including any type of model (transmission, health system, cohort, etc.) that includes a comparison of two or more alternative strategies both in terms of costs and consequences (efficacy or effectiveness) [8]. Economic evaluations can rely on many different types of models, but economic evaluations are defined by the study purpose or question they are answering (cost-effectiveness, budget impact, etc.), using different types of models to answer different economic questions. Decision analysis and Markov models are the most common approach for evaluating cost-effectiveness of novel diagnostics; however cost components can also be added to transmission or health systems models.

Data extraction and synthesis

For all papers information was extracted pertaining to the research question and result. Extracted data included: details of the population and diagnostics pathways that were explored, model methodology (model type, sensitivity analyses, details of model structure) and, where appropriate, details on how the cost-effectiveness analysis was conducted, including questions about the costing perspective employed and scope of the costing parameters. A complete summary of all data extracted is available in the supplementary material. The results were discussed by a broad group of experts during the TB MAC meeting on “Impact and cost-effectiveness of current and future diagnostics for TB” in Amsterdam, April 24–25th 2013 (www.tb-mac.org/WorkAreas/WorkArea/4).

Results and Discussion

Out of 5619 unique records that were found in the literature search, 91 records were selected for evaluation, of which 46 underwent full-text evaluation. After full-text screening 31 papers were selected. Consultation with experts in the field yielded five additional papers; therefore 36 papers were included for data extraction (see Figure 1). The complete summary of all included papers and extracted data is available in Table S1.1 thru Table S3.4 in Data S1.

Figure 1

Systematic review flowchart for selection of papers.

Is enough modeling being done?

Since the approval of Xpert in 2010, a sharp increase in TB diagnostic modeling has been seen both among papers modeling Xpert and among papers evaluating alternative nucleic acid amplification tests (NAATs). Compared with only 8 TB modeling papers published from 2000–2007 inclusive (approximately one per year), 10 TB diagnostic modeling papers were published in 2012, 6 of which focused on Xpert.

The 36 included papers were grouped into the three non-exclusive categories: 16 papers included population impact/transmission modeling, 5 were health systems models, and 24 included cost-effectiveness models. Novel tests evaluated across the 35 papers included: several NAATs, light-emitting diode (LED) microscopy, Determine TB-LAM (Alere Inc., MA, USA) a lateral-flow urine lipoarabinomannan detection assay, FastPlaque TB and FastPlaque-Response (Biotec Labs Ltd, UK), serological tests including Anda TB (Anda Biologicals, Strasbourg), MTT assay, a non-commercial colorimetric assay (ICN Biomedicals) and sputum processing methods including bleach sedimentation and sample dilution. A table of novel TB diagnostic tests included in the evaluated studies with brief descriptions is included in Table 1. Novel diagnostic strategies included the expansion of culture, drug sensitivity testing (DST), chest x-ray and mass miniature radiography (MMR). Several hypothetical diagnostic tests, scenarios or algorithms were also evaluated including: hypothetical NAATs with improved speed, sensitivity and specificity; a dipstick style test; rapid DST; tests resulting in reduction in diagnostic delay; and rapid point of care with improved sensitivity, specificity and reduced time to results compared to smear microscopy or conventional drug susceptibility testing (Tables S1.2, S2.2 & S3.3 in Data S1).

Table 1

Novel diagnostic tests for active TB disease evaluated in included modeling review.

Novel diagnostic tests evaluated	Type of Diagnostic	Method	Sample type
Nucleic Acid Amplification Tests (NAATs)
GeneXpert MTB/RIF (Xpert) (Cepheid, USA)	NAAT	Real-time PCR based technique to detect both presence of MTB and RIF resistance using an automated cartridge based design	Resp. spec.
INNO-LiPA Rif. TB (Immunogenetics, Belgium)	Line probe	A reverse hybridization-based line probe assay that detects mutations is the rpoB “hotspot” gene region (RIF resistance)	Resp. spec. and/or liquid/solid culture
MTBDRplus (Hain Lifescience, Germany)	Line-probe	Molecular genetic assay for identification of resistance to rifampicin and/or isoniazid. Mycobacterial DNA is extracted from the specimen, specifically amplified via PCR and detected on a membrane strip using reverse hybridization and an enzymatic color reaction.	Resp. spec. and/or liquid/solid culture samples
GenProbe Amplified Mycobacterium Tuberculosis Direct Tests (MTD) (Gen-Probe Inc., USA)	NAAT	Transcription mediated amplification of rRNA, detects Mycobacterium tuberculosis rRNA directly and rapidly	Resp. spec.
Cobas Amplicor (Roche Diagnostics)	NAAT	PCR amplification of 16s rRNA	Resp. spec.
Non-NAAT based tests
LED Microscopy	Microscopy	Light Emitting Diode (LED) based fluorescence microscope, energy-efficient, does not require a dark room	Resp. spec.
FastPlaque-Response (Biotech, UK)	Phage-based assay	Allows for detection of Rifampicin resistance; Phage-based assay that detects live MTB in a plaque assay in a lawn of rapidly growing detector cells. Samples are incubated with and without RIF overnight	Resp. spec.
FastPlaque TB (Biotech, UK)	Phage-based assay	Mycobacteriophage (virus that infect TB) - Phage-based assay that detects live MTB in a plaque assay in a lawn of rapidly growing detector cells.	Resp. spec.
MTT assay (ICN Biomedicals)	Colorimetic assay	Noncommercial colorimetric assay that uses an indicator of cellular growth and viability whose oxidized yellow form becomes purple after reduction to formazan by the dehydrogenases of live bacterial cells	Liquid/solid culture samples
Bleach sedimentation on sputum samples	Sputum processing to increase dx yield	Bleach digestion of sputum, followed by specimen concentration step such as sedimentation	Resp. spec.
Anda TB (Anda Biologicals, Strasbourg)	Serological ELISA test	ELISA based detection of antibodies elicited by antigens of MTB that are recognized by humoral immune system	Blood samples
Diluting sputum before MTD Gen-probe	Specimen processing	1∶10 dilution of the processed specimen	Resp. spec.
Determine TB-LAM (Alere, USA)	Lateral-flow immune-chromatographic assay	Lateral-flow immune-chromatographic assay detects lipoarabinomannan (LAM) – an immunogenic glycolipid in the cell way of MTB- in urine. First true POC test, excellent specificity and higher sensitivity than sputum smear microscopy in immuno-compromised adults	Urine samples

Abbreviations: MTB: Mycobacterium tuberculosis, RIF: rifampicin, PCR: polymerase chain reaction, POC: point-of care, ELISA: enzyme linked immune-sorbent assay, Resp. Spec.: respiratory specimens, dx: diagnostic.

The majority of studies (83%, 30/36) modelled high TB incidence settings; 39% (14/36) of evaluated studies modelled the South African setting specifically, reflecting the large burden of TB in South Africa and the recent move and investments towards novel technologies in improving TB control. Only 6/36 (17%) studies -all cost-effectiveness analyses- modelled low TB incidence settings including the United States, United Kingdom, and Finland. While the majority of studies are modelling settings with the highest TB burden, many important epidemiological settings (e.g., China, Southeast Asia outside of India) have few or no models to provide insight into potential cost-effectiveness or epidemiological impact of novel diagnostics or diagnostic strategies. As a result decisions with important repercussions for the TB epidemic and TB control budgets are frequently being made in the absence of supporting insight that models could provide. As tests and diagnostic algorithms are developed and evaluated, models are needed that represent a wide range of diagnostic tests, strategies and settings to help support data-driven policy decisions.

What questions are modelling studies trying to answer?

Models covered a wide range of objectives in evaluating novel diagnostics. Transmission and health system models investigated the impact of diagnostic delay (n = 5), and demonstrated several key insights including a threshold for average delay to diagnosis beyond which an epidemic will escalate [9], the importance of applying novel diagnostics in both private and public health systems to reduce diagnostic delay [10], and that test sensitivity was a key determinant of diagnostic delay [11]. In prison populations it was shown screening at entry and annually with MMR could keep TB prevalence below 1% [12]. In the case of drug-resistant TB, it was demonstrated that current strategies with long delays to diagnosis will probably not halt the spread of MDR-TB, and reducing the delay through improved access to DST and second-line treatment may reduce transmission of drug-resistant TB [13], [14]. Transmission models highlighted the importance of considering different elements of the diagnostic pathway (n = 4) including the pre-diagnostic infectious period; others linked operational and transmission models, demonstrating that operational context elements such as patient diagnostic delay, access to care and loss to follow-up can inform setting specific models useful for data-driven policy decisions [15]–[18].

In terms of economic evaluations, models provided comparisons of the cost-effectiveness and impact on TB incidence and mortality of various novel or hypothetical diagnostics and strategies (n = 24) [16], [19]–[41]. Many studies demonstrated the cost-effectiveness of novel diagnostics, often highlighting the settings and implementation strategies and/or algorithms in which those tests were most cost-effective. Among cost-effectiveness models, the most frequently studied diagnostic assays included Xpert MTB/RIF (n = 7) and alternative NAATs (n = 4). Most (n = 16) of the cost-effectiveness models compared new assays to an existing baseline and generated an incremental cost-effectiveness ratio (ICER), of which 69% (n = 11) concluded that novel TB diagnostics were likely to be cost-effective relative to that baseline. Some technologies including NAATs implemented in low TB prevalence settings were judged not to be cost-effective [27],[30], and serology was shown to be more costly and less effective than existing diagnostics such as sputum smear microscopy [28]. Cost-effectiveness studies also highlighted barriers or challenges associated with roll-out of novel diagnostic tests including operational barriers or increased indirect costs, associated with HIV or multi-drug resistance (MDR) care [19], [25]. A transmission study demonstrated that DST for all retreatment cases could be highly cost-effective [40]. Evaluations of hypothetical point of care (POC) tests suggested that a highly specific, low cost POC test would be highly cost-effective, with the greatest impact in settings with poor infrastructure [37].

Do models include the relevant features?

In each of the three model categories we evaluated relevant features associated with modelling methods, structure and parameters including: outcomes of interest, type of model, sensitivity and uncertainty analyses, costing methods and data sources, how drug susceptibility and HIV status were incorporated in the model along with false negatives and positives.

Population impact/transmission models

Sixteen (44%) of the selected papers employed a transmission model to assess the population impact of a novel TB diagnostic [9]–[20], [40], [42]–[44] (Table S1.1 in Data S1). The main outcomes of interest included: TB incidence, TB mortality, incidence of multi-drug resistant TB (MDR-TB) or extensively drug resistant TB (XDR-TB), and transmission rate. All papers focused on implementation in a population with a high TB burden and/or low or middle income setting, with the exception of one theoretical analysis that assessed high, medium and low TB burden settings [15]. Seven of 16 studies (44%) explored methods of TB diagnostic modeling; for example: linking transmission modeling and discrete event simulation (n = 2) [16], [18], including patient–provider interactions (n = 1) [10], assessing implications of periods of pre-diagnostic transmission (n = 1) [15] or modeling details of the diagnostic pathway or diagnostic delay (n = 3) [9], [11], [17]. A dynamic compartmental model was employed in 7/16 (44%) of the transmission studies evaluated [9], [12], [15], [17], [42]–[44]; the remaining studies linked a compartmental model with an operational model (n = 3) [13], [16], [18], a Markov model (n = 1) [10], a state transition model [40], a cost-effectiveness analysis (n = 1) [19], two decision analysis models [11], [40], a cohort model (n = 1) [9] or a Markov model with a cost-effectiveness component (n = 1) [20]. Drug susceptibility and HIV status were each considered in 8/16 (50%) studies and 7/16 studies respectively, 4/16 (25%) considered both in the same model [13], [16], [19], [43].

Health system models

Only 5/36 (14%) of included papers explicitly modelled interactions between patients and health care providers or health care facilities [10], [13], [16]–[18]. All models assessed populations in settings with a high TB burden and all were published in the last five years (Table S2.2 in Data S1). Outcomes of interest included TB incidence and prevalence, TB mortality and rates of transmission. One study also assessed costs and number of patients cured [16]. This was the only study from this group to evaluate novel diagnostics (i.e.: LED and Xpert) as opposed to a hypothetical test or a decrease in the delay to diagnosis [16]. All 5 studies employed a transmission model; two employed discrete event simulation [16], [18], one [10] employed a Markov structure and one [13] a queuing structure. Four of 5 studies (80%) [10], [13], [16], [18] explicitly modelled the period of transmission pre-diagnosis (Table S2.3 in Data S1). Two studies [13], [16] included both HIV status and drug susceptibility in the model while a further study considered only HIV status [17].

Two of 5 studies (40%) were calibrated to empirical data [10], [17]: one employing epidemiological characteristics of the TB and HIV epidemic in the modelled country [19] and the other employing study data evaluating patient-provider interactions [17]. One study (20%) considered false positives [16], while 3/5 (60%) explicitly modelled false negatives allowing re-entry into the same diagnostic pathway [10], [16], [17].

Future transmission and health systems models would benefit from careful consideration of the transmission patterns in the pre-diagnostic phase of disease development as well as among individuals receiving false-negative diagnoses. Several factors might contribute to differential transmission during these periods, including the duration of continued infectiousness in each time period, the trajectory of infectiousness over time and changes in contact structure over time. For example, if the majority of the transmission occurs prior to the patient seeking care, improved sensitivity and specificity of a novel diagnostic may not impact upon transmission and therefore TB incidence. Molecular epidemiology studies and emerging technologies including whole genome sequencing coupled with intensive contact tracing studies, may serve to provide improved understanding of these factors in the future, and to ultimately to help better model the impact of novel diagnostics on transmission. Modellers should also carefully consider the trajectory of false negative patients through the diagnostic pathway across different operational contexts. Do false negative diagnoses result (on average) in many months of high-level transmission, just a few weeks of contact only with people who are already exposed, or death with very little on-going transmission? Studies involving intensive follow-up of representative groups of people testing negative for TB in given settings could provide helpful data in understanding how many false negatives ultimately become diagnosed with TB, when diagnosis occurs and what happens to those who are not diagnosed with TB. Data collection in these areas should be encouraged to better inform models seeking to appropriately address these challenging considerations.

Cost-effectiveness models

Twenty-four out of the 36 selected papers (67%) included a cost-effectiveness or cost-utility analysis [16], [19]–[41]. Twenty out of 24 studies (83%) were cohort models, including either a decision analysis framework [21]–[23], [25]–[38], [41], or Markov approach [20], [24], [26]; 3/24 (12.5%) were transmission models [16], [19], [40], one study linked a cohort (Markov) and transmission model [20], and one transmission model also utilized a health systems approach [16]. The majority of cost-effectiveness models, 16/24 (67%), took their study population as people with TB symptoms or in the case of DST individuals diagnosed with TB [21]–[23], [25]–[37]; remaining studies assessed HIV populations initiating ART [24], [39], [41], the general population [16], [19], [40], or a prison population with high MDR prevalence [20], [38] (Table S3.1 in Data S1). Most models assessed populations in high TB burden settings, however 5/24 (21%) were set in low TB burden countries [27], [29], [30], [33], [38]. Eight of 24 (33%) studies evaluated Xpert [16], [19]–[21], [24], [32], [34], [36], while 9/24 (37.5%) evaluated an alternative NAAT [22], [27], [29]–[34], [36]. Effectiveness measures in these analyses included health utility measures [quality-adjusted life years (QALYs), 4/24 (17%), or disability-adjusted life years (DALYs), 7/24 (29%)] and cases detected; 16/24 studies (67%) generated an ICER (Table S3.2 in Data S1). Several studies also calculated total program and/or implementation costs. The cost-effectiveness studies were performed from either a health care provider, health system, TB program or TB laboratory perspective. Only one study included patient costs [25], and these were restricted to an approximate estimate of cost of travel to the health facilities. No study reported taking a societal perspective. Less than half (11/24, 46%) employed an empirical costing component in the analysis [16], [20], [22], [25]–[27], [29], [33], [36], [40], [41]. Fifteen of 24 studies (62.5%) included HIV in the model [16], [19]–[21], [23], [24], [26], [28], [32], [34]–[39], [41], and 11/24 (46%) accounted for drug susceptibility status [16], [19], [22], [24], [30], [32]–[34], [36], [38], [40].

There exists a scarcity of models that include the costs incurred by TB patients throughout the diagnostic and treatment process, and no cost-effectiveness studies were conducted from the patient or society perspective. Omission of the patient or societal perspective may underestimate the costs and/or impact associated with earlier or more effective diagnosis. Estimation of patient costs may be important for other models as well; for example, if patient costs of transport and TB diagnosis are catastrophically high, the incremental impact of point-of-care diagnostics (i.e., that do not require multiple visits to healthcare facilities to initiate treatment) may be much greater than if patient costs are low. As TB is a disease of poverty, closer evaluation of the patient-level costs of TB diagnosis may reveal that these costs are a critical barrier to the impact of novel diagnostic tests [45]. Increased effort should be made to collect empirical cost data from the patient and societal perspective in a variety of settings; and/or incorporate existing data into current models.

Moreover, many cost-effectiveness analyses claiming a health systems, health care provider or TB program perspective did not include all relevant costs. Some studies included only test costs and salaries, omitting overhead costs; likewise costs associated with HIV care post-TB diagnosis and MDR-TB treatment post-diagnosis with Xpert or other new diagnostics, were not systematically included across relevant studies. Reviewed studies demonstrated that the inclusion or omission of indirect costs, particularly costs associated with HIV and MDR care, can have important implications for estimated ICERs and model conclusions, and should be carefully considered in cost-effectiveness studies. While often challenging to distinguish the extent to which direct provider costs and higher-level program costs are incremental, omission of these costs may lead to overly optimistic conclusions surrounding the cost-effectiveness of novel interventions. Cost-effectiveness analyses should carefully consider relevant costs and endeavour to include clear explanations of which costs are and are not included and why.

Is uncertainty properly captured?

Sensitivity analyses were performed in 14/16 transmission studies (87.5%), yet only 6 considered variation in two or more variables at a time [12], [15], [17], [19], [40], [43] (Table S1.3 in Data S1). Half of the papers calibrated their models to existing real data (e.g. incidence trends or point estimates). Among the 16 transmission studies evaluated, 13 (81%) explicitly accounted for the impact of transmission before diagnosis. The impact of individuals with false negative diagnoses (i.e. patients with active TB who were misdiagnosed as TB negative and therefore continued to transmit) was estimated in 6/16 studies (37.5%) [10], [11], [16], [17], [19], [43], of which 5 assumed that these patients would re-enter into the same diagnostic pathway at the same rate as ‘pre-diagnostic’ individuals with active TB. Three health systems studies (60%) performed sensitivity analyses [13], [16], [17], only one of which considered more than one variable at a time [17].

All economic studies performed sensitivity analyses, and 13/24 (54%) assessed sensitivity to more than one variable at a time [19], [20], [22], [23], [26]–[28], [33]–[37], [39], [40] (Table S3.4 in Data S1). Fourteen of 24 (58%) explicitly modelled false positives [16], [19], [22], [24], [26]–[28], [30], [31], [33], [35], [39], [41] (which may have an impact on costs as patients then receive unnecessary treatment and follow-up), 14/24 (58%) studies explicitly modelled false negatives [16], [19], [20], [22]–[24], [26], [28], [30], [31], [34], [36], [39], [41] (which may have an impact on health outcomes as they remain undiagnosed, untreated and potentially infectious), Only three studies allowed false negatives to re-enter the diagnostic pathway, in two cases entering the same diagnostic pathway as those entering for the first time [16], [26] and in the third a second pathway for false negatives was modelled [39].

Sensitivity analyses were frequently limited to one- or two- way analyses. More recent models increasingly include multi-way sensitivity analyses as well; however, thorough investigation and discussion of both uncertainty in model structures and parameter values (both point estimates and associated ranges) remains a largely unmet need. More transparent approaches to uncertainty may help to build understanding and trust in modelling approaches among policy makers.

Most models evaluated do not generate predictions that can be validated against data collected in the future; therefore this systematic review is limited in its ability to assess whether model predictions could be validated in real life. This review is also limited by its ability to review only English language publications.

Conclusion

As highlighted in this review, models have provided helpful insight both in terms of pragmatic policy implications concerning the cost-effectiveness of novel diagnostics and methodological insights. Models have highlighted the scenarios and algorithms where novel technologies are most cost-effective, demonstrating that while Xpert and other NAATs are highly cost-effective in high TB burden settings, they are not cost-effective in settings with low TB prevalence [19], [20], [27], [30], [36]. Similarly, serology in India was shown to be both more costly and less effective than standard smear microscopy contributing to important policy implications to discourage use of serology at the national and global levels [28]. Models have provided insight on the potential impact of novel diagnostics on transmission of TB over time; implementation of Xpert in five African countries was shown to have the potential to reduce TB incidence and mortality over ten years but increase costs associated with MDR treatment and HIV care [19]. The importance of explicitly modelling the pre-diagnostic infectious period was demonstrated in transmission models that further assessed how different characterizations of this infectious period may have important implications on model results and interpretations. Diagnostic delay was examined in several transmission models, along with the identification of key determinants, such as test sensitivity and the health system's ability to return results to the patient and initiate timely treatment. Finally, models have highlighted the importance of the operational context stressing the need for setting-specific operational or health systems models to guide and support data-driven policy decisions.

The literature on novel TB diagnostic modeling remains limited but is growing rapidly with the on-going development and introduction of novel diagnostic assays and diagnostic systems. We have identified and reviewed a group of models that may help to inform decisions related to implementation of novel TB diagnostics, but further models are needed that include additional epidemiological settings and compare additional diagnostic algorithms. These models would benefit tremendously from epidemiological data on the transmission implications of false-negative diagnosis and the pre-diagnostic period. Improvement and expansion of mathematical models describing transmission, health systems, and cost-effectiveness of novel TB diagnostics will ensure more rational implementation and resource allocation of these tools in order to realize their potential to improve human health in high TB burden settings.

Prisma Checklist.

(DOC)

Click here for additional data file.

Supplementary tables. Table S1.1, General overview of population impact/transmission model. Table S1.2, What was modeled (diagnostics and scope of model). Table S1.3, Modeling methods. Table S2.1, Health System models: General overview. Table S2.2, Cost-effectiveness specific considerations. Table S2.3, What was modeled (diagnostics and scope of model). Table S2.4, Modeling methods (including which mixed methods were applied). Table S3.1, Cost-effectiveness models: General overview. Table S3.2, Cost-effectiveness specific considerations. Table S3.3, What was modeled (diagnostics and scope of model). Table S3.4, Modeling methods.

(DOCX)

Click here for additional data file.