Validation of a spatial agent-based model for Taenia solium transmission ("CystiAgent") against a large prospective trial of control strategies in northern Peru.

BACKGROUND: The pork tapeworm (Taenia solium) is a parasitic helminth that imposes a major health and economic burden on poor rural populations around the world. As recognized by the World Health Organization, a key barrier for achieving control of T. solium is the lack of an accurate and validated simulation model with which to study transmission and evaluate available control and elimination strategies. CystiAgent is a spatially-explicit agent based model for T. solium that is unique among T. solium models in its ability to represent key spatial and environmental features of transmission and simulate spatially targeted interventions, such as ring strategy. METHODS/PRINCIPAL
FINDINGS: We validated CystiAgent against results from the Ring Strategy Trial (RST)-a large cluster-randomized trial conducted in northern Peru that evaluated six unique interventions for T. solium control in 23 villages. For the validation, each intervention strategy was replicated in CystiAgent, and the simulated prevalences of human taeniasis, porcine cysticercosis, and porcine seroincidence were compared against prevalence estimates from the trial. Results showed that CystiAgent produced declines in transmission in response to each of the six intervention strategies, but overestimated the effect of interventions in the majority of villages; simulated prevalences for human taenasis and porcine cysticercosis at the end of the trial were a median of 0.53 and 5.0 percentages points less than prevalence observed at the end of the trial, respectively.
CONCLUSIONS/SIGNIFICANCE: The validation of CystiAgent represented an important step towards developing an accurate and reliable T. solium transmission model that can be deployed to fill critical gaps in our understanding of T. solium transmission and control. To improve model accuracy, future versions would benefit from improved data on pig immunity and resistance, field effectiveness of anti-helminthic treatment, and factors driving spatial clustering of T. solium infections including dispersion and contact with T. solium eggs in the environment.

Introduction

Cysticercosis is a neglected tropical disease (NTD) that exacts a substantial health and economic burden in low-income countries. The global burden of cysticercosis includes approximately 5 million people with neurocysticercosis (NCC) or epilepsy due to NCC [1] and hundreds of millions of dollars in annual livestock losses from discarded pork [2].

Although global eradication of T. solium transmission is unlikely in the short-term, local control or elimination is now possible [3,4] due to the availability of new tools that can be deployed to interrupt transmission. These include effective treatment of taeniasis [5,6] and porcine cysticercosis [7,8], improved diagnostic tests [9,10], and a vaccine to prevent pig infection [11,12]. In 2012, shortly after the success of a large-scale elimination demonstration in Peru that effectively implemented many of these tools [3], the World Health Organization (WHO) declared ambitious targets for global control and elimination of T. solium. They called for validated control strategies to be identified by 2015, and for these strategies to be scaled and implemented in several countries by 2020 [13]. Due, in part, to the high cost and lengthy time required to conduct prospective trials, optimal strategies for T. solium control have not yet been identified, and the 2020 targets for large-scale implementation have not been met. More recently, WHO called for transmission models to be deployed to optimize control strategies and reach agreed-upon targets for control [14], a goal that was recently reinforced in the WHO’s 2030 goals for T. solium control [15].

To this end, six models of T. solium transmission have been published to date that attempt to fill this gap [16-21]. Despite preliminary use of these models to compare and contrast available control strategies, there are significant concerns about the validity of these models that must be addressed before reliable policy recommendations can be made. The most important deficit of existing T. solium models is that none have been validated against data from prospective trials carried out in the field. Comparison of model predictions with observed outcomes is a critical step for ensuring the validity of a model, and up until recently, a lack of data from prospective trials has prevented such validation.

Beyond a lack of validation, prior models have not incorporated certain features of T. solium transmission that may be important for model accuracy. First, no prior model has a spatially explicit structure, meaning that pigs and humans are assumed to mix homogenously to transmit the parasite, a limitation that was highlighted in a recent systematic review of available T. solium models [22], and in a report outlining key modeling improvements needed to meet the WHO 2030 goals [15]. Spatially clustered transmission patterns have been well-documented in endemic villages [23-26], and spatially targeted control strategies (e.g., “ring strategy”) have shown success in field trials [27]. Second, the lack of an open population structure that allows for human travel and migration is a key limitation of existing T. solium models. The lack of such a structure prevents existing models from capturing rebounds in transmission that may occur after elimination has been achieved due to movement of infected people or pigs in to an area with successful elimination [28,29]. Together, these design features may lead existing models to over-estimate intervention effectiveness and over-estimate the likelihood of achieving control and elimination targets [30,31].

In order to address the above limitations, we developed a novel agent-based model (ABM) called “CystiAgent” [21]. Notably, CystiAgent includes a spatial structure and open population, which allows it to represent key aspects of the T. solium life-cycle, and facilitates the evaluation of spatially targeted control interventions. In the current study, we present results from our validation of CystiAgent using data from the RST, a cluster-randomized trial conducted in northern Peru that included 23 villages and 6 unique control strategies: four spatially targeted ring interventions and two mass applied interventions [32]. Our objective was to evaluate the ability of CystiAgent to replicate baseline levels of transmission through model calibration and to evaluate the accuracy of the model for predicting observed reductions in transmission when control strategies were applied.

Methods

Model description

Model structure

CystiAgent is a spatially explicit ABM developed in NetLogo 6.0.4 (Northwestern University, Evanston, IL), an open-access ABM software ideal for representing spatial processes in a graphical user interface. The full model code is available at http://modelingcommons.org/browse/one_model/6268, and an in-depth model description with sensitivity analyses of model parameters was previously published by Pray et al. [21].

Agents

CystiAgent has two classes of agents–humans and pigs–that interact and transmit T. solium in a spatial environment. Humans may be infected with the adult-stage intestinal tapeworm (i.e., T. solium taeniasis) through consumption of infected pork, and pigs may be infected with larval-stage metacestodes (i.e., porcine cysticercosis) through contact with T. solium eggs or proglottids in the environment. Exposure to eggs may cause light cyst infection (<100 cysts), and exposure to proglottid segments may lead to heavy cyst infection (≥ 100 cysts); either may lead to seropositivity [33-35]. The different exposure-infection pathways for heavy and light cyst infection in pigs were based on a combination of biologic plausibility and consistency with prior transmission models [18,19]. Pig infection is a categorical state variable (susceptible, light infection, heavy infection), and individual T. solium eggs or cysts are not agents in the model. Therefore, the cut-off value of 100 cysts between light and heavy cyst infection serves only to allow for translation of prevalence from serological studies and to fit the conceptual distinction between transmission routes for light and heavy infection, but is not numerically represented in the model. Human cysticercosis, including NCC or NCC-related seizure disorders, is not included in this model.

Environment

The model environment consists of households in villages that are spatially distributed according to a set of input coordinates that can be assigned to represent real villages if household coordinates are known. Household characteristics such as the presence of latrines or corrals to contain pigs are assigned at baseline and the houses are populated with humans and pigs.

Processes and flow

CystiAgent consists of a sequence of biological and behavioral processes that cycle continuously to propagate the T. solium life-cycle (Fig 1).

Fig 1

Transmission of Taenia solium in CystiAgent.

Panel A (top): 1) Live pig trade: infected (live) pigs are sold to other households inside and outside the village, and imported from outside; 2) in-house slaughter and sale of pork to other households 3) pork consumption: potentially infected pork is consumed by household members; 4) human infection (see panel B for details); 5) human travel: humans travel to other endemic villages where they may acquire tapeworm infections through pork consumption; 6) open defecation: humans may practice open outdoor defecation; 7) pig roaming and foraging: free-roaming pigs consume potentially infectious eggs present in the environment; 8) pig infection (see panel B for details). Panel B (bottom): Susceptible pigs (S) may be heavily exposed (EH) by consuming T. solium proglottid segments in the environment or through the stochastic parameter heavy-all, which is a probability of exposure applied to all pigs, or be lightly exposed (EL) by consuming T. solium eggs in the environment or through light-all. Pigs with heavy or light exposure may develop heavy (IH) or light (IL) cyst infection through the stochastic tuning parameters light-inf and heavy-inf. Pigs with heavy or light exposure may become seropositive (IS) based on the stochastic tuning parameters light-sero and heavy-sero. Susceptible humans (S) may be heavily exposed (EH) or lightly (EL) exposed to cysts after eating infected pork from a heavily or lightly infected pig. Consumption of heavily or lightly infected pork cause tapeworm infection (I) in humans based on the stochastic tuning parameters ph2h and pl2h, respectively. Icons provided by Noun Project and created by Pariphat Sinma, Ikopah, Dennis, Berkah, Giorgiana Ionescu, Nabilauzwa and Alexsandr Vector.

Live pig trade. Infected pigs that are due for slaughter may be sold live within the village and then slaughtered at home by the buyer, or exported out of the village. Infected pigs from external villages may also be imported into the village through the live pig trade.

In-house slaughter and pork sale. Infected pigs may be slaughtered by their owners. The resulting pork meat may then be consumed at home and/or sold to other households.

Pork consumption. When consumed pork is infected with T. solium cysts, all members of the consuming households are exposed to potential tapeworm infection.

Human infection. Humans who consume infected pork may acquire a tapeworm infection (or remain in a susceptible state), a stochastic process determined by two locally calibrated tuning parameters (one each for consuming lightly and heavily infected pork). The intestinal tapeworm reaches maturity after 8 weeks [36], and begins expelling infectious eggs at that time. Tapeworm infections naturally clear after a stochastically determined duration [36,37].

Human travel. Humans that are designated as travelers leave the community at regular intervals, may contract tapeworm infections through consumption of infected pork while traveling to other endemic areas, return to the village after travel and resume contamination of their environment.

Open defecation. Human tapeworm carriers that do not own or use a latrine release T. solium eggs and proglottid segments into the environment surrounding their household location. When tapeworm infections clear, humans stop releasing proglottid segments, but contamination of the environment with eggs persists until the eggs naturally degrade according to an exponential decay function [38].

Pig roaming and foraging. Pigs that are designated as free-roaming (i.e., not contained in corrals) are exposed to T. solium proglottids and eggs that are present in their home-range areas.

Pig infection. Pigs that are exposed to proglottid segments may develop heavy cyst infection, while pigs exposed to eggs in the environment may develop light cyst infection. Either may result in seropositivity. Free-roaming pigs are exposed to an additional risk of exposure that is proportional to the number of tapeworm carriers in the village and naïve to the pig’s location. This represents exposure to eggs that results from roaming and consumption of human feces from open defecation sites outside of the home area. Progression of T. solium exposure (eggs or proglottids) to seropositivity or cyst infection (or return to a susceptible state) is a stochastic process determined by a set of four locally calibrated tuning parameters (see below). Mature cyst infection occurs immediately after exposure without a pre-patent maturation period. Pigs that are lightly infected may progress to heavy cyst infection if exposure to proglottid segments occurs; otherwise, infected or seropositive pigs remain in that state until slaughter, with no option for recovery unless a treatment intervention is applied.

Parameters

Each of the above processes is defined mathematically by a probability distribution and corresponding parameter(s) (Table 1). The majority of parameters were derived from observational studies conducted in northern Peru [21,32], but some were based on expert opinion and literature review. Model parameters, source data, and in-depth sensitivity analyses on model parameterization are described in detail in Pray et al [21].

Table 1

CystiAgent model parameters.

Parameter Description	Parameter Name	Distribution	Value (range, if applicable)	Source
Village input features (village-specific) *
Humans per household	humans-per-hh	Poisson	3.8 (3.32–4.94)	RST
Proportion of households raising pigs	prop-pig-owners	Binomial	0.49 (0.25–0.75)	RST
Pigs per pig-raising household	pigs-per-hh	Exponential	2.43 (1.74–4.21)	RST
Corral prevalence among pig-owner households	prop-corrals	Binomial	0.50 (0.23–0.92)	RST
Latrine prevalence	prop-latrines	Binomial	0.64 (0.19–0.97)	RST
Live pig trade
Proportion of pigs sold prior to slaughter	pigs-sold	Binomial	0.51	HH
Proportion of sold pigs exported	pigs-exported	Binomial	0.73	HH
Rate of pigs imported from endemic areas(imports / pig / week)	pig-import-rate	Constant	0.00105	HH
Prevalence of cyst infection among imports	import-prev	Binomial	0.134	HH
Proportion of infected imported pigs with light cyst burden	light-to-heavy	Binomial	0.76	HH
In-house slaughter and pork sale
Pig slaughter age (months)	slaughter-age	Log-normal†	Log-mean = 2.279 (median = 9.8 months);Log-SD = 0.515	RST
Proportion of pork consumed by owner	hh-only-pork	Binomial	0.40	HH
Proportion of pork sold after slaughter	sold-pork	Binomial	0.12	HH
Proportion of shared pork eaten by owner	shared-pork-hh	Binomial	0.8	HH
Human infection
Incubation time to reach tapeworm maturity	tn-incubation	Constant	8 weeks	[36]
Tapeworm lifespan (years)	tn-lifespan	Normal†	Mean = 2 yearsSD = 0.96 years	[36,37]
Human travel
Proportion of households with a frequent traveler	traveler-prop	Binomial	0.42	HH
Frequency of travel to other endemic areas(every X weeks)	travel-freq	Constant	8 weeks	HH
Duration of travel	travel-duration	Exponential†	1.75 weeks	HH
Incidence of T. solium taeniasis during travel(risk / person / week)	travel-incidence	Constant	0.00023	[24]
Open defecation
Latrine-use (prop. of households that “always” use latrine)	latrine-use	Binomial	0.25	[39]
Radius of environmental contamination (meters from home)	cont-radius	Log-normal†	Log-mean = 3.27 (median = 26 meters)Log-SD = 0.547	[39]
Rate of egg decay in environment (mean survival duration)	decay-mean	Exponential	8 weeks	[38]
Pig roaming and foraging
Proportion of pig households with corrals that “always” corral pigs	corral-always	Binomial	0.05	[39]
Proportion of pig households with corrals that “sometimes” corral pigs	corral-sometimes	Binomial	0.57	[39]
Proportion of pigs in “sometimes”-corral-households that are corralled	prop-corral-some	Binomial	0.32	[39]
Radius of pig roaming “home-range”(meters from home)	home-range	Log-normal†	Log-mean = 3.79 (median = 44 meters)Log-SD = 0.552	[39]
Tuning parameters (village-specific) *
Probability of human taeniasis upon slaughter of lightly infected pig	pl2h	Binomial	0.00435 (0.0023–0.0099)	Calibration
Probability of human taeniasis upon slaughter of heavily infected pig	ph2h	Binomial	0.00473 (0.00314–0.0075)	Calibration
Probability of exposure to T. solium eggs per human with taeniasis	light-all§	Binomial	0.00944 (0.00445–0.0141)	Calibration
Probability of exposure to T. solium proglottids per human with taeniasis	heavy-all§	Binomial	0.0161 (0.0123–0.0209)	Calibration
Probability of light cyst infection upon contact with to T. solium eggs	light-inf	Binomial	0.286 (0.0137–0.0582)	Calibration
Probability of heavy cyst infection upon contact with to T. solium proglottids	heavy-inf	Binomial	0.00654 (0.0041–0.0116)	Calibration
Probability of pig seropositivity upon exposure to T. solium eggs	light-sero	Binomial	0.316 (0.095–0.67)	Calibration
Probability of pig seropositivity upon exposure to T. solium proglottids	heavy-sero	Binomial	0.286 (0.031–0.82)	Calibration

* Village input values were measured and applied separately in each village; tuning parameters were calibrated separately for each village; parameter values in these categories are displayed in this table as the mean and range among villages included in the analysis

† Values for these five parameters (slaughter-age, tn-lifepan, travel-duration, cont-radius, and home-range) were applied at the individual pig, human, or household level by randomly assigning a value from the specified statistical distribution (all other parameter values were assigned at the village level).

§ Exposure probabilities (“light-all” and “heavy-all”, x) scaled to the current number of tapeworm carriers (HT) according to 1 − (1–x)HT

Abbreviations: RST = Ring Strategy Trial; HH = Household Survey; SD = Standard Deviation

Fitting

Apart from the biological and behavioral parameters included in the model, CystiAgent utilizes a set of tuning parameters to adjust the probabilities of infection to match the observed prevalence of human and pig infection in each trial village. To do this, the model includes eight tuning parameters that represent different probabilities of exposure or infection for pigs or humans. Since these parameters represent complex sequences of unknown probabilities and cannot be determined through primary data collection or literature review, we adopted an approximate Bayesian computation (ABC) algorithm [40] to computationally derive their values for this analysis (this is described in detail below).

Model validation—ring strategy trial

Trial design

The RST was a cluster-randomized trial comparing six unique population-level interventions designed to control T. solium transmission. The trial was carried out in 23 villages in the Piura region of northern Peru over a period of 24 months (2015–2017) [32]. Trial villages were randomly divided into six arms each receiving unique intervention designs (Table 2). Four of the six arms received some variation of ring strategy. This is an approach that targets anthelminthic treatment to humans and pigs that live within 100 meters of pigs found to be heavily infected through non-invasive tongue palpation [27,41]. Between the four ring-strategy arms, the approach varied based on the intervention applied within 100-meter rings. For humans, the two options included presumptive treatment of human taeniasis with two doses of oral niclosamide (NSM), or stool screening for taeniasis with the enzyme-linked immunosorbent assay for copro-antigen detection (CoAg-ELISA) and repeated follow-up testing and treatment with oral NSM until cure. Some villages in each of these categories received treatment of pigs with a single oral dose of oxfendazole (OFZ), and others had no pig treatment interventions. All ring interventions were administered in trial villages every 4 months throughout the 2-year trial-period. The remaining two trial arms received mass treatment of humans with a single oral dose of NSM every 6 months with and without the addition of porcine mass treatment every 4 months. Eligibility criteria for RST included all humans two years of age or older, and all pigs greater than two months of age and not pregnant.

Table 2

Summary of interventions and populations in Ring Strategy Trial–Peru, 2015–2017.

Strategy	Interventions	Population
Ring treatment	Pig tongue screening, human treatment in rings(q4 months, 7x)	2 villages*(~1200 humans, ~400 pigs)
Ring treatment w/ pig treatment	Pig tongue screening, human and pig treatment in rings (q4 months, 7x)	4 villages(~1600 humans, 500 pigs)
Ring screening	Pig tongue screening, human screen-and-treat in rings(q4 months, 7x)	4 villages(~1500 humans, 600 pigs)
Ring screening w/ pig treatment	Pig tongue screening, human screen-and-treat, pig treat in rings (q4 months, 7x)	4 villages(~1500 humans, 400 pigs)
Human mass treatment	Human MDA(q6 months, 5x)	4 villages(~1300 humans, ~400 pigs)
Human and pig mass treatment	Human and pig MDA(q6 months, 5x)	3 villages(~1400 humans, ~500 pigs)
TOTAL		21 villages (~19000 humans, ~8000 pigs)

*2 villages excluded from ring treatment due to small size and lack of observed transmission

Abbreviations: MDA = Mass drug administration, q = frequency; x = repetitions.

Trial outcomes

Serum samples from all eligible pigs in the trial communities were collected every four months over the 24-month trial period (seven rounds total) and tested with the enzyme-linked immune-electro transfer blot (EITB). We considered a pig to be seropositive if two or more of the bands on the EITB assay were reactive [42]. Because necroscopic examination of pigs to assess true cyst infection was not conducted in the trial, we estimated the prevalence of porcine cysticercosis at baseline and trial-end using EITB serology and conversions from prior necropsy studies in the region. EITB serology is highly sensitive but non-specific for identifying pigs with active larval cyst infection (seropositivity without infection may be due to prior resolved cyst infection, exposure without established infection, or cross-reactivity), thus conversion is needed account for EITB positive pigs without active cyst infections [33,42]. We estimated that a total of 43.9% of seropositive pigs would be infected with larval cysts (31.1% of seropositive pigs with a light cyst burden of < 100 cysts, and 12.8% with a heavy cyst burden of ≥100 cysts), figures that were based on the average of two large necropsy studies conducted in Peru [34,35]. In addition to estimating the prevalence of cyst infection at baseline and trial end, we measured the cumulative incidence of EITB seroconversion for each of four-month intervals throughout the trial. This measure of seroincidence represented the proportion of new or previously seronegative pigs that became seropositive during the previous four-month period.

For human taeniasis, all human participants were offered NSM at the conclusion of the trial, and post-treatment stool samples were analyzed with CoAg-ELISA to evaluate the final-round prevalence of human taeniasis. Baseline prevalence of human taeniasis, however, was not directly measured in the RST. We estimated this value using a regression equation that was developed from a prior cross-sectional study of human taeniasis and porcine cysticercosis in the region [24]. For this, seroprevalence of (2+ EITB bands) in pigs was used a predictor for the log-prevalence of human taeniasis. A complete description of statistical methods for outcome estimation and other validation settings are included in S1 Appendix and village-level RST outcome are available in S1 Data.

Model validation

Validation attempts were performed individually for each village in the RST. Two of the 23 villages were excluded from validation because they were either too small to achieve stable transmission (<10 pigs in the village) or had no heavily infected pigs identified at baseline, meaning that no ring interventions were applied during RST. This left 21 villages for model validation. For each village, a unique set of household coordinates and input characteristics were applied based on RST census data and the model was fit with a unique set of tuning parameters generated by ABC calibration. Other than these village-specific settings, the same model structure and parameter values were used in all validation attempts. After calibration was performed, factors influencing calibration accuracy across villages were assessed using multivariable linear regression with relative errors at baseline as the dependent variable.

Calibration of model tuning parameters

We used an ABC method to estimate values for the six tuning parameters that define the probabilities of human and pig infection in the model (pl2h, ph2h, light-inf, heavy-inf, light-all, and heavy-all; see Table 1 for details) and two serological parameters (light-sero and heavy-sero) defining the probability of antibody response after exposure for pigs. A unique set of tuning parameters was estimated for each of the 21 villages. Our ABC method followed a two-stage sequential rejection sampling approach [40,43,44]. We used a Sobol’ quasi-random sequence [45,46] to sample 5,000 values from a uniform prior distributions (0 to 1) of each parameter. For each combination of parameter values, we ran the model though 1000 weeks without interventions and recorded the average prevalence of taeniasis (humans), light and heavy cyst infection (pigs) and seroincidence (pigs) across the simulation period. The “sensitivity” package in R was then used to calculate the Euclidean distance [47,48] between summary statistics and the baseline prevalence observed in the field data. We selected the top 1% of model runs that minimized the Euclidean distance and extracted posterior distributions from the selected parameter sets. With the new generated posterior distribution, we repeated the rejection sampling algorithm with 10,000 tuning parameters sets and selected our final parameter values based on the median values of the new posterior distributions produced for each parameter in this final step. All model simulations for calibration and validation were executed in R (The R Foundation for Statistical Computing, Vienna, Austria) and NetLogo on the Amazon Web Service EC2 cloud computing platform (aws.amazon.com, Seattle, WA). Model simulations were distributed across multiple 72-core parallel processors using the “parallel” R-package [49] and executed on the EC2 cloud using the R-Studio Shiny server [50].

Simulation of control strategies

For model validation, all simulations began with a 1000-week burn-in period, followed by the corresponding intervention sequence that was applied in the RST, and 100-week post-intervention observation period. Intervention sequences were repeated 1000 times per village for validation. Participation in interventions among humans and pigs was applied separately in each village and reflected the proportions that were observed in that village during the RST. The sensitivity and specificity of the CoAg-ELISA for detecting T. solium taeniasis was set to 96.4% and 100%, respectively based on the method described by Guezala et al. [9], and the efficacy of NSM for treatment of human taeniasis was set at 76.6% for one dose, 86.6% for two doses, and 93.3% for post-screening follow-up. Treatment of pigs with OFZ was assumed to have an efficacy of 100%. Detailed model settings used for validation are available in S1 Appendix and participation rates used for each village are available in S1 Data.

Evaluation of model accuracy

For each RST village, we compared the model-predicted prevalence of human taeniasis, porcine cysticercosis, and pig seroincidence with the values observed in the trial (or values estimated from trial observations). For graphical comparison, the plots of model-predicted outcomes show the absolute and relative error of the median of the 1000 model runs compared to the values observed in the trial or generated from trial observations. The relative error and absolute errors between an observed P and the median simulated prevalence P is defined as follows:

Results

Baseline calibration for ring strategy trial

Among the 21 villages evaluated in the RST, the median observed prevalence of human taeniasis and porcine cysticercosis at baseline were 2.2% (range: 0.9% to 3.8%) and 19.2% (range: 6.3% to 27%), respectively. Median porcine seroincidence per 4-month period at baseline was 44.2% (range: 5.3% to 78%).

The ABC calibration step allowed CystiAgent to accurately replicate observed baseline prevalence for all villages and disease outcomes (Fig 2). The median absolute difference between simulated and observed taeniasis prevalence at baseline was +0.09 percentage points (range: -0.20 to +0.63 percentage points), corresponding to a median relative error of +4.9% (range: -18.5% to +33.3%). For porcine cysticercosis, the median absolute difference between simulated and observed prevalence at baseline was +0.58 percentage points (range: -1.6 to +3.1 percentage points), with a median relative error of +3.8% (range: -9.3% to +22.8%). For porcine seroincidence, median absolute differences and relative errors between simulated and observed at baseline were +1.8 percentage points (range: -3.4 to 5.3 percentage points) and +3.2% (range: -4.7% to +27.9%), respectively. Overall, calibration accuracy was high across all villages, and no village factor in the model, including number of households, use of corrals and use of latrines were associated with an improvement or decline in the accuracy of baseline calibration after evaluating calibration errors in a multivariable linear regression model.

Fig 2

Relative (top) and absolute (bottom) errors during validation of CystiAgent against results from the Ring Strategy Trial for human taeniasis (left) and porcine cysticercosis (right) at baseline (month 0) and trial-end (month 24) (n = 21 villages). For ring treatment (n = 6 villages), ring screening (n = 8 villages) and mass treatment (n = 7 villages villages), interventions with (white) and without (grey) pig treatment are designated by the background color of the plot area. Three villages (570, 578, and 585) had no human taeniasis identified at trial-end and relative errors were not able to be calculated.

Accuracy of intervention effects

When RST intervention strategies were simulated in CystiAgent, the model overestimated the intervention effects for taeniasis and porcine cysticercosis in the majority of villages. The median prevalence of taeniasis observed at the conclusion of the RST was 0.55% (range: 0 to 4.3%), with elimination of taeniasis by trial-end in 3 of the 21 villages. CystiAgent predicted elimination of taeniasis at trial-end in 12 of 21 villages and median prevalence across all villages of 0% (range: 0 to 0.37%), which was 0.53 percentage points less (range: 4.3 to +0.36 percentage points) than the observed prevalence at trial-end.

The median prevalence of porcine cysticercosis observed at the conclusion of the RST was 8.8% (range: 2.0 to 23.3%). CystiAgent predicted a trial-end median prevalence of 2.2% (range: 0% to 6.4%), which was a median of 5.0 percentage points less (range: 18.9 to +4.1 percentage points) than the observed prevalence of porcine cysticercosis at trial-end. None of the villages achieved complete elimination of taeniasis and porcine cysticercosis by the end of the trial; however, CystiAgent predicted T. solium elimination in two of the 21 villages at the end of the trial, and six villages by the end of the 100-week post-intervention period.

Between the three primary intervention types compared in RST (ring screening, ring treatment, and mass treatment), CystiAgent predicted similar and substantial reductions in taeniasis and porcine cysticercosis by trial-end (Fig 3). This differed from observed results of the trial, which found that ring-screening and mass-treatment led to larger reductions in prevalence compared to ring-treatment, although all three intervention types were effective control strategies against human taeniasis and porcine cysticercosis. Prediction errors for taeniasis and porcine cysticercosis at trial-end were greatest for ring treatment (-1.0 and -7.4 percentage points for taeniasis and porcine cysticercosis, respectively), and less for ring screening (-0.47 and -3.1 percentage points), and mass treatment (-0.45 and -5.2 percentage points).

Fig 3

Simulation outcomes versus observed rates of human taeniasis, porcine cysticercosis and porcine seroincidence in the Ring Strategy Trial, by intervention type (n = 21 villages).

Plots display village-specific median simulated outcomes in CystiAgent across 1000 simulations per village, and observed outcomes from RST (median value within intervention type in bold). For ring treatment (n = 6 villages), ring screening (n = 8 villages) and mass treatment (n = 7 villages villages), intervention and without pig treatment are combined. See S1 Fig for sub-arm comparisons of villages with and without the addition of pig treatment.

In villages where antihelminthic treatment of pigs was added to each of the three primary intervention types, CystiAgent predicted more dramatic reductions in the prevalence of porcine cysticercosis compared to interventions with human screening or treatment alone. The predicted effect of pig treatment on human taeniasis prevalence was minimal. Graphs of model predictions versus RST outcomes split by interventions with and without pig treatment are available in S1 Fig. Despite model predictions, an additive effect of pig treatment was not observed in RST, which led to increased prediction errors in villages where pig treatment was applied.

The accuracy of CystiAgent with respect to porcine seroincidence as a measured outcome in RST was similar to other outcomes, with the model slightly overestimating the intervention effect in most villages. Across all 21 RST villages, the median observed porcine seroincidence in the final 4-month study interval was 12.6% (range: 3.3% to 45%), which corresponds to a median decline of 24.8 percentage points (range: 1.6 to 71 percentage points) from baseline. In CystiAgent, the median simulated seroincidence in the final 4-month study interval was 6.1% (range: 0% to 25.7%), which corresponds to a median absolute difference of 6.6 percentage points lower (range: -36.2 to +18.7 percentage points) than the observed seroincidence during that interval.

Comparing the accuracy of seroincidence simulations across intervention types in RST, all three overestimated intervention effects by a similar degree by trial-end (Fig 4). CystiAgent predicted seroincidences during final 4-month study interval that were a median of 6.2, 7.0, and 6.6 percentage points less than observed seroincidence in ring screening, ring treatment, and mass treatment villages, respectively. Despite this consistency at trial-end, model accuracy varied by intervention type at mid-study time-points 2 and 3 (months 4 and 8 of RST), when CystiAgent failed to replicate a steep early decline in seroincidence that was observed only in ring screening villages.

Fig 4

Relative errors of porcine seroincidence during validation of CystiAgent against result from the Ring Strategy Trial, by intervention type–ring treatment (n = 6 villages), ring screening (n = 8 villages), and mass treatment (n = 7 villages).

Each time point represents the 4-month cumulative seroincidence across the 24-month trial period, as measured by enzyme-linked immune-electro transfer blot (EITB) in pigs. Error bars show inter-quartile range for relative errors among village in each intervention. Within each intervention type, villages with and without pig treatment are combined.

Discussion

Development and validation of a T. solium transmission model was identified as a key goal in WHO’s 2014 framework for intensified control [14], yet none of the currently available models have been validated against data from field trials of T. solium control interventions. In this research, we developed a novel transmission model for T. solium called CystiAgent, and validated the model using data from a large cluster-randomized trial of ring and mass-applied pharmaceutical intervention conducted in northern Peru. Validation results showed that CystiAgent could be accurately calibrated to reflect baseline levels of T. solium transmission, but overestimated the magnitude of intervention effects in most villages. Despite this, the current validation allowed for identification of potential gaps in the model that may be addressed in future versions, and represented an important step towards delivering an accurate and validated model that can be used to answer key questions about T. solium transmission and control.

Baseline calibration

For this validation, CystiAgent was calibrated to each study village individually using the model’s eight tuning parameters and a complex ABC estimation method. Using this method, baseline calibration was highly accurate in nearly all villages (median calibration errors of +0.09 and +0.58 percentage points for taeniasis and porcine cysticercosis, respectively), which provided assurance that the model could reproduce observed levels of endemic transmission. Two villages were excluded from the study because their small population size and lack of infected pigs led to unstable transmission in the model; therefore, very small villages (<30 households) may not be suitable for CystiAgent. Estimation of tuning parameters separately for each village with ABC estimation is computationally demanding, and may not be feasible for larger scale use of CystiAgent to compare or evaluate T. solium control interventions. To address this limitation, updates to CystiAgent that may be considered for future versions include reducing the number of tuning parameters and fitting a single set of parameter values to prevalence levels averaged across a larger geographic region. Such adjustments would benefit from additional sensitivity analysis and validation to evaluate the impact of these changes on model performance.

Validation of intervention strategies

Validation of CystiAgent against RST interventions showed that the model was able to execute all six intervention types (ring screening, ring treatment, and mass treatment; with and without pig treatment) and reproduce observed declines in transmission; however the model overestimated the magnitude of intervention effects in the majority of villages tested. The median trial-end prevalence for taeniasis and porcine cysticercosis simulated in CystiAgent were 0.53 percentage points lower (0.55% observed vs. 0% simulated) and 5.0 percentage points lower (8.8% observed vs. 2.2% simulated) than the observed prevalences in RST, respectively.

There are a few possible explanations for this error that could be addressed with additional data collection to improve CystiAgent performance. First, the efficacy of NSM for curing taeniasis may have been overestimated in the model. While the applied single-dose efficacy of 76.6% has been reported elsewhere [51], variations in manufacturer formulation and shelf-life could impact the treatment efficacy. Such a difference would reduce the effectiveness of control interventions and could have contributed to the overestimated intervention effects predicted in CystiAgent. Second, heterogeneous non-participation in control intervention, particularly if concentrated among persons at high-risk for T. solium infections, would reduce the effectiveness of control interventions. In CystiAgent, treatment applications among simulation participants was randomly assigned among eligible individuals without accounting for systematic non-participation, which could have led to overestimation of intervention effectiveness. Third, acquired immunity and resistance to T. solium infection in pigs was not included in CystiAgent due to a lack of data to accurately parameterize this process. Pig immunity is highly dynamic [33] and the number of pigs with acquired immunity in the population would likely decrease as control interventions are applied due to lower levels of exposures to T. solium eggs in the environment, which could lead to more rapid rebounds in transmission among susceptible pigs after treatment applications. CystiAgent was not able to replicate these dynamic changes in population-level resistance among pigs, which may have caused overestimation of the intervention effects predicted by CystiAgent.

Model performance with respect to porcine seroincidence by round showed that the model predicted steady declines in seroincidence after each successive intervention application for all three intervention types. While this trajectory matched the observed patterns of decline in villages that received ring treatment and mass treatment interventions (despite overestimating the magnitude of decline), villages that received the ring screening intervention experienced a sharp initial decline in seroincidence followed by a leveling off at low levels of transmission that was not replicated in the model. The mechanism underlying this sharp decline and subsequent leveling off in ring screening arms is not definitively known. Possible explanations for not capturing this pattern in CystiAgent include a higher efficacy of NSM when used post-screening (91.8% post-screening efficacy was applied in CystiAgent), or a higher degree of spatial clustering among T. solium-infected humans and pigs than modeled.

Strengths and limitations

There are a few important strengths of this validation to highlight. First, we had access to all data collected during RST, a unique opportunity that allowed for this validation to be conducted. Using these data, the model was tested independently in 21 separate villages and 6 unique intervention types, leading to a robust assessment of the model’s accuracy in a variety of transmission settings. The consistency of the model’s performance across all villages provides initial support for the generalizability of the model to other transmission settings. Second, the spatially explicit structure of CystiAgent allowed us to evaluate ring interventions in this validation. Ring interventions are spatially targeted strategies for T. solium control that have shown promising results in field studies [27,32], and are not able to be evaluated in other non-spatial T. solium models.

There are two main limitations of this validation study. First, baseline and trial-end measures of porcine cysticercosis prevalence were limited by the diagnostic methods used in RST. CystiAgent estimated the prevalence of true cyst infection in pigs based on EITB serology because necroscopic examination to definitely diagnose cyst infection in pigs was not conducted in RST. EITB serology is highly sensitive but poorly specific for detecting true cyst infections [33,52]. Therefore, to approximate the prevalence of true cyst infection at baseline and trial-end, we estimated that 44% of seropositive pigs based on EITB would be infected, a figure based on two necropsy studies previously conducted in the region [34,35]. These necropsy studies were conducted in different pig populations in different regions of Peru that had not received intervention programs, and may not have the same immune responses as pigs in this trial. Also, we used the same conversion factor for EITB serology at baseline and trial-end, but the diagnostic performance of EITB may change after interventions are applied due to changing levels of immunity in the pig population. This effect was observed in a previous trial in Peru [3], and could indicate that trial-end prevalence of cyst infection in RST was lower than estimated, which would indicate that model predictions were more accurate than the measured error-rates indicated. More research is needed to determine how treatment and disease control impacts population immunity and diagnostic performance such that these features can be accounted for in future models. Secondly, parameters used for this validation were primarily sourced from a single region of northern Peru (Piura region). While the depth of data available from this region made it possible to develop and validate this detailed ABM, some key parameters may differ in other endemic areas of the world, which could impact the model’s accuracy when applied to these regions. Application of CystiAgent outside of northern Peru, therefore, would benefit from local data to set appropriate values for key model parameters such as latrine use, pig roaming ranges, and pig trading, which were identified as high priority parameter groups in prior sensitivity analyses conducted with CystiAgent [21].

Future research priorities

CystiAgent and other T. solium models will always be limited by uncertainty in the mechanisms and dynamics of T. solium transmission, which could impact model accuracy and validity. Some of the more important gaps in knowledge that may impact transmission but are not yet incorporated into CystiAgent include age-related differences in pig roaming and environmental exposure to T. solium [39,53,54], distribution patterns of infected pork through black market channels [55], and the possibility of vector-borne transmission of T. solium eggs via dung beetles and flies [56,57]. Perhaps the most important aspect of T. solium transmission not included in the model, however, is immunity. Due to insufficient knowledge of the mechanisms of immunity, resistance, and susceptibility to T. solium infection in humans and pigs, we were not able to incorporate these features into the model. As described above, this may have contributed to the observed overestimation of intervention effects. Since probabilities of infection in the model (i.e., tuning parameters) were estimated using levels of transmission observed at baseline, the model is not able to account for changes in susceptibility or resistance that may occur in response to control pressure when an intervention is applied. This feature could be added to the model if appropriate data from experimental and/or field studies becomes available. While these and many other factors are not yet features of CystiAgent, a key advantage of its ABM structure is that it has the flexibility to incorporate these novel features when data become available, and can serve as an accessible platform to develop and test hypotheses about T. solium transmission dynamics.

Conclusion

In this research, we developed and validated a novel transmission model for T. solium called CystiAgent. In this first large-scale validation of a T. solium transmission model, CystiAgent was able to simulate levels of transmission observed at baseline in the validation villages, and adequately replicate the effects of control interventions in the majority of villages. Overall, the model overestimated intervention effects in many of the villages tested. These imperfect results, however, represent important data-points that can be used to adjust and improve future versions of the model. Moving forward we will continue to test and improve the CystiAgent model using data available from interventions in Peru, and evaluate the generalizability of the model through validation against data from other endemic regions. Ultimately, we aim to use a final validated version of CystiAgent to evaluate available strategies for T. solium control and deliver evidence-based recommendations that will meet the need for validated strategies emphasized by WHO.

Additional methods for comparison of CystiAgent simulations with observed results from the Ring Strategy Trial.

(DOCX)

Click here for additional data file.

Data file including CystiAgent simulation results, observed results from the Ring Strategy Trial, and all intervention settings for each village evaluated (n = 21 villages).

(CSV)

Click here for additional data file.

Performance of CystiAgent model across six intervention strategies in RST (ring screening, ring treatment, and mass treatment; each with and without the addition of pig treatment) (n = 21 villages).

Plots display village-specific median simulated outcomes in CystiAgent across 1000 simulations per village, and observed outcomes from RST (median value within intervention type in bold). For ring treatment (n = 6 villages), ring screening (n = 8 villages) and mass treatment (n = 7 villages villages), interventions with (grey) and without (white) pig treatment are designated by the background color.

(TIF)

Click here for additional data file.

30 Mar 2021

Dear Mr. Pray,

Thank you very much for submitting your manuscript "Validation of a spatial agent-based model for Taenia solium transmission (“CystiAgent”) against a large prospective trial of control strategies in northern Peru" for consideration at PLOS Neglected Tropical Diseases. As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. In light of the reviews (below this email), we would like to invite the resubmission of a significantly-revised version that takes into account the reviewers' comments.

We cannot make any decision about publication until we have seen the revised manuscript and your response to the reviewers' comments. Your revised manuscript is also likely to be sent to reviewers for further evaluation.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Please prepare and submit your revised manuscript within 60 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email. Please note that revised manuscripts received after the 60-day due date may require evaluation and peer review similar to newly submitted manuscripts.

Thank you again for your submission. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Kendall McKenzie

Staff Admin

PLOS Neglected Tropical Diseases

Makedonka Mitreva

Deputy Editor

PLOS Neglected Tropical Diseases

***********************

Reviewer's Responses to Questions

Key Review Criteria Required for Acceptance?

As you describe the new analyses required for acceptance, please consider the following:

Methods

-Are the objectives of the study clearly articulated with a clear testable hypothesis stated?

-Is the study design appropriate to address the stated objectives?

-Is the population clearly described and appropriate for the hypothesis being tested?

-Is the sample size sufficient to ensure adequate power to address the hypothesis being tested?

-Were correct statistical analysis used to support conclusions?

-Are there concerns about ethical or regulatory requirements being met?

Reviewer #1: (No Response)

Reviewer #2: The methods present the published CystiAgent spatially explicitly model, parameterised for Northern Peru and fitted using an ABC methodology (probability of exposure/infection "tuning" parameters) to baseline village-level prevalence. Interventions applied in the ring strategy trial (RST) are used to validate CystiAgent, by comparing the relative/absolute errors of model projections vs observed data (porcine seroincidence) or estimated outcomes (porcine cysticercosis "true" cyst prevalence or human taeniasis prevalence). The methods are well written, and appropriate to address to address the study objectives. I have a number of questions which are outlined in the "summary/general comments" section.

--------------------

Results

-Does the analysis presented match the analysis plan?

-Are the results clearly and completely presented?

-Are the figures (Tables, Images) of sufficient quality for clarity?

Reviewer #1: (No Response)

Reviewer #2: The results address the analysis plan, are well presented (including the figures), showing the ability of CystiAgent to accurately capture the baseline endemic prevalence in each village following fitting of the "tuning" probabililty parameters. Model intervention projections were overestimated compared to observed data (porcine seroincidence) or estimated outcomes (porcine cysticercosis "true" cyst prevalence or human taeniasis prevalence), represented by the visual model projections comapred to data, and relative/absolute error plots. I have a number of questions and recommendations for improvements outlined in the "summary/general comments" section.

--------------------

Conclusions

-Are the conclusions supported by the data presented?

-Are the limitations of analysis clearly described?

-Do the authors discuss how these data can be helpful to advance our understanding of the topic under study?

-Is public health relevance addressed?

Reviewer #1: (No Response)

Reviewer #2: The authors provide a rich discussion into the discussion section, in particular describing possible explanations for the observation of overestimation model intervention impact projections compared to data/estimated outcomes. The wider implications are described, showing the utility of using a spatially-explicit model to consider the contribution of spatial epidemiological processes and testing spatial interventions. I have outlined some key considerations and recommendations in the "summary/general comments" section.

--------------------

Editorial and Data Presentation Modifications?

Use this section for editorial suggestions as well as relatively minor modifications of existing data that would enhance clarity. If the only modifications needed are minor and/or editorial, you may wish to recommend “Minor Revision” or “Accept”.

Reviewer #1: (No Response)

Reviewer #2: Please see further comments for the results section under the "Summary and General comments" section.

--------------------

Summary and General Comments

Use this section to provide overall comments, discuss strengths/weaknesses of the study, novelty, significance, general execution and scholarship. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. If requesting major revision, please articulate the new experiments that are needed.

Reviewer #1: The manuscript provides a very valuable evaluation of the CystiAgent Taenia solium disease transmission model, and generates avenues for further research. I only have some minor suggestions:

1. L418, "prevent.. from accurately replicating observed patterns": one could argue if this is really the goal of any transmission model, or if it would even be possible. To replicate observed patterns, one should have a perfect understanding of reality, while the role of simulation models is rather to better understand reality through simplification. So the lack of a spatially explicit structure, or of an open population structure, would not be a "structural deficit" per se, but rather a design choice, ie, simplification.

2. Table 1: it is not always clear how exactly the distribution and parameter should be understood. Do all distributions represent variability, and not uncertainty? For Poisson, Exponential and Binomial (assuming n=1), one parameter would indeed suffice, but for other parameters it would not. I would therefore appreciate some more info on how to interpret the uniform, log-normal and normal distributions.

3. P12L3: the exclusion of 2 villages could be seen as a possible source of bias in the validation results, and could merit some discussion. Does it mean the model is not able to deal with low prevalence settings?

4. P10L2: Would it be possible to include a reference to the manuscript in review?

5. The terms "ring strategy" and "ring strategy trial" are sometimes capitalized, especially in the beginning of the manuscript. The abbreviation "RST" is introduced on page 9 but could be introduced earlier on.

6. In the author summary, "T. solium" should be written in full at first occurrence.

Reviewer #2: Intro section:

Minor comments:

1. Pg. 3 Lines 19-20: presumably the “over 5 million people with epilepsy from NCC” figure comes from Table 1 of Ref[1]; if this is the case, the average figure is 4.85 million cases and relates to persons infected with NCC or epilepsy due to NCC, so this should be written as “approximately 5 million people with neurocysticercosis (NCC) or epilepsy due to NCC”.

2. Pg. 5 Lines 2-3 : This is not correct, existing transmission model with closed population structure would be able to capture rebound in prevalence following intervention cessation (if interventions did not achieve local elimination). The authors would be more accurate in stating existing models would not be able to capture rebound following elimination, due to movement of infected people into an area with successful elimination.

3. Pg.5 lines 3-5: I do not think the authors can word this so strongly. These deficits *may*result in over-estimation of predictions, but really cross-validation of models on the same datasets will be required to determine differences in predictive capacity (i.e. cross-validation/comparison efforts are more common-place in modelling areas for other NTDs).

=============

Methods section:

1. It is not clear from the methods text, and tables 1 and Figure 1, how long pigs remain in the “exposed” state, or how this is represented in the model. Does the represent the pre-patent period in pigs (before cyst maturation?)- from Table 1 and description in the methods section text, my impression is that pigs progress immediately after exposure based on the tuning parameter light-inf or heavy-inf without including a cyst maturation period in pigs? Please can the authors make this clearer in the text and describe the way progression from exposed to infected is parameterised in the model.

2. I am a little unclear regarding the implications of lines 10-13 on page 11; so 42.9% are identified as infected if seropositive (30.1% with light infections plus 12.8% with heavy cyst infections), does this imply that 57.1% are false-positive or antibody positive from the EITB but not harbouring active larval infection (i.e. recovered)?

3. Can the authors indicate whether the copro-antigen ELISA protocol used matched the Guezela et al. 2009 protocl (ref 4 in S1 appendix). (and). This is important as 96.4% sensitivity suggests this protocol was used, whereas sensitivity of copro-antigen ELISA from other literature estimates is generally lower (for example see the modelled estimate from Praet et al. 2013 with a sensitivity of 84.5%). Other copro-antigen protocols based on the original Allan et al. 1999 protocol) give specificity estimates of <100% (not species specific), whereas the Guezela et al. 2009 protocol gave a specificity of 100% (so presumably the authors are assuming 100% specificty, although it is not clear?)- more discussion on the different protocols for copro-antigen ELISA tests can be found in Lightowlers et al. 2016.

Praet N, Verweij JJ, Mwape KE, Phiri IK, Muma JB, Zulu G, van Lieshout L, Rodriguez-Hidalgo R, Benitez-Ortiz W, Dorny P, Gabriël S. Bayesian modelling to estimate the test characteristics of coprology, coproantigen ELISA and a novel real-time PCR for the diagnosis of taeniasis. Trop Med Int Health. 2013 May;18(5):608-14.

Lightowlers, M W et al. “Monitoring the outcomes of interventions against Taenia solium: options and suggestions.” Parasite immunology vol. 38,3 (2016): 158-69. doi:10.1111/pim.12291

Minor:

1. Can the authors provide a reference and explain the rationale for the cut-off for light infections with <100 cysts and heavy infections with > 100 cysts (pg.6 lines 6-7).

2. Can the authors please state in the table which parameters refer to light-sero and heavy-sero from the description in the text (lines 11-12, page 12) - presumably this is the last row in table 1, but this should be made explicit.

3. Please explicitly state that min/max and other quantities are available to specify statistical distributions in the Pray et al. 2020 CystiAgent publication (and that these are the same as presented in this manuscript), as this is not clear.

============

Results section:

1. Is there a reason for not estimating the prevalence of cyst infection from the EITB seroincidence data every 4 months, so that prevalence estimates could be more closely compared to simulated dynamics on porcine cysticercosis prevalence throughout the trial (i.e. middle panel in Figure 3)? Equally, why was human taeniasis prevalence not estimated for each timepoint (from the regression model using EITB data) to compare taeniasis prevalence estimates with modelled estimates (throughout the trial)? I think the analysis would be enhanced if these estimates are presented vs. model projections in Figure 3 (rather than just comparing baseline and end-of trial estimates to modelled projections) - the relative/abolsute error plots can still just show differences at baseline vs. end of trial.

2. Where the authors state that various factors are not associated with an improvement or decline in the accuracy of baseline calibration (lines 15-17 pg.14), how was this tested - from the sensitivty analysis in the original Pray et al. 2020 CystiAgent model paper?

3. The authors state in the legend of Figures 2- 4 that the intervention with and without pig treatment are combined, is there a reason for this? I would recommend showing the model projections and observed data broken down by +/- pig treatment (at least in the supplementary information) to show whether pig treatment has a systematic impact on difference between observed data and projections between ring treatment/ring screening/mass treatment (and the relative error in figure 4).

4. There seems to be a trial-end point missing for the human taeniasis relative error panel in figure 2 (village 576 for ring screening and village 570 for mass treatment)?

Minor:

1. Is the median porcine seroincidence a % per 4 months (lines 6 on page 14)? Can the authors explicitly state this if so i.e. 44.2% per 4 months at baseline. I recommend in Figure 3 that the seroincidence units on the y-axis (of the 3rd column/panel) should be “proportion per 4 months”.

2. If I am understanding correctly, the median difference referenced on line 8 (for taeniasis) and 11 (porcine cysticercosis) of pg.14 relate to the median (absolute) differences? Can the authors explicitly state whether they refer to the absolute or relative differences throughout the manuscript please.

3. The abbreviations for porcine cysticercosis (PC) and taeniasis (HT) are first introduced in line 22 on page 15, however should be introduced much earlier, with abbreviations used on line 24 of page 3 (if the authors wish to use abbreviations).

4. On line 17-18 of page 16, should this read as range: -36.2 to +18.7% percentage points or 36.2 lower to 18.7% higher to improve clarity

==============

Discussion section:

1. The authors indicate that they have used data to suggest that 44% of EITB-seropositive pigs would be infected (should this not be 43% from 30.1% light infected + 12.8% heavy infected?) – I think an improved approach would be to adjust observed prevalence data/estimates to 'informed' or 'true' prevalence with the methodology proposed by Speybroeck et al. 2013 (using a Bayesian framework, incorporating uncertainty in diagnostic sensitivity and specificity)? If evidence from previous intervention programmes suggest that the false positive rate (specificity) changes (as the authors suggest in lines 1-3 of page 21), the observed results in the RST could be adjusted based on the method proposed by Spebroeck et al. 2013 with varying specificity throughout the programme? I recommend this as an area for the authors to consider for adjusting their field data, especially if the diagnostic parameters change throughout intervention (which, as the authors suggest may change the error between simulated prevalence and prevalence estimates from the data).

Speybroeck N, Devleesschauwer B, Joseph L, Berkvens D. Misclassification errors in prevalence estimation: Bayesian handling with care. Int J Public Health. 2013 Oct;58(5):791-5.

Minor:

1. Could the authors explain the statement that the number of pigs with acquired immunity would decrease as control interventions are applied (lines 9-10 on page 19); is this because reduction in environmental contamination of the environment (through treatment of taeniasis carriers) would result in lower exposure of pigs (and therefore development of immunity)? Does this then assume that susceptibility in the pig population is higher when pigs are subsequently exposed (following imported cases of human taeniasis carriers moving into the area?) I think the authors could expand on this point further.

--------------------

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Brecht Devleesschauwer

Reviewer #2: No

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.

Reproducibility:

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

21 Jun 2021

Submitted filename: CystiAgentValidation_ResponseToReviewers.docx

Click here for additional data file.

3 Aug 2021

Dear Mr. Pray,

Thank you very much for submitting your manuscript "Validation of a spatial agent-based model for Taenia solium transmission (“CystiAgent”) against a large prospective trial of control strategies in northern Peru" for consideration at PLOS Neglected Tropical Diseases. As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. In light of the reviews (below this email), we would like to invite the resubmission of a significantly-revised version that takes into account the reviewers' comments.

There are a few issues arising which need to be addressed. Please see comments from the reviewer

We cannot make any decision about publication until we have seen the revised manuscript and your response to the reviewers' comments. Your revised manuscript is also likely to be sent to reviewers for further evaluation.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Please prepare and submit your revised manuscript within 60 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email. Please note that revised manuscripts received after the 60-day due date may require evaluation and peer review similar to newly submitted manuscripts.

Thank you again for your submission. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Paul R. Torgerson

Associate Editor

PLOS Neglected Tropical Diseases

Makedonka Mitreva

Deputy Editor

PLOS Neglected Tropical Diseases

***********************

There are a few issues arising which need to be addressed. Please see comments from the reviewer

Reviewer's Responses to Questions

Key Review Criteria Required for Acceptance?

As you describe the new analyses required for acceptance, please consider the following:

Methods

-Are the objectives of the study clearly articulated with a clear testable hypothesis stated?

-Is the study design appropriate to address the stated objectives?

-Is the population clearly described and appropriate for the hypothesis being tested?

-Is the sample size sufficient to ensure adequate power to address the hypothesis being tested?

-Were correct statistical analysis used to support conclusions?

-Are there concerns about ethical or regulatory requirements being met?

Reviewer #1: (No Response)

Reviewer #2: Thank you for you responses, they certainly helped and the improvements to the manuscript are noticeable. I have a few remaining questions to follow-up:

In response to question & answer (1):

Original Q.1: It is not clear from the methods text, and tables 1 and Figure 1, how long pigs remain in the

“exposed” state, or how this is represented in the model. Does the represent the pre-patent period in pigs (before cyst maturation?)- from Table 1 and description in the methods section text, my impression is that pigs progress immediately after exposure based on the tuning parameter light-inf or heavy-inf without including a cyst maturation period in pigs? Please can the authors make this clearer in the text and describe the way progression from exposed to infected is parameterised in the model.

New question:

Thanks for this. I think the section could do with a little more explanation/clarification, as an exposed compartment (as seen in Fig 1) would normally refer to individuals exposed but not infectious (in this context egg establishment and cyst maturation to a mature cyst, governed by a rate of transition). But I understand the exposed compartment, within this agent-based model formulation, represents an instantaneous transition governed by the light-all or heavy-all stochastic tuning (probability of exposure) parameters, and then an instantaneous transition to light infection (based on the probability of light infection upon contact parameter light-inf), or to heavy infection (based on the probability of heavy infection upon contact parameter heavy-inf). To this end, I think Figure 1 should be adjusted to provide more clarity on the transmission pathways: for example I think, if my understanding is correct, that there should be two pig exposure compartments, with light-all and heavy-all parameters to Exposed_light (EL) and Exposed_heavy (EH), which independently lead to the IL or IH compartments (and then two transitions from for EL and EH to IS based on light-sero and heavy-sero tuning parameters, which will improve clarity (if my understanding here is correct). Furthermore, I think Figure 1 could be improved by detailing the tuning parameters between compartment transitions, so that Figure 1 and Table 1 align more closely (also please see Figure 2 in the CystiSim Braae et et al. 2016 paper for example model schematic including transition probabilities within an IBM).

In response to question & answer (3):

Original Q.3: I am a little unclear regarding the implications of lines 10-13 on page 11; so 42.9% are identified as infected if seropositive (30.1% with light infections plus 12.8% with heavy cyst infections), does this imply that 57.1% are false-positive or antibody positive from the EITB but not harbouring active larval infection (i.e. recovered)?

New question: Thanks for this clarification, I think some text here with the relevant reference to briefly outline that the EITB is highly sensitive but not highly specific (hence the high false positive proportion due to measuring pigs that are exposed rather than only infected), would be useful in the first paragraph after Trial Outcomes (and to put the 56.1% FP proportion into context).

Minor:

Original Q:Can the authors provide a reference and explain the rationale for the cut-off for light infections

with <100 cysts and heavy infections with > 100 cysts (pg.6 lines 6-7).

New question: Thanks, I think the authors may have missed a reference here in the updated text after “Exposure to eggs may cause light cyst infection (<100 cysts), and exposure to proglottid segments may lead to heavy cyst infection (≥ 100 cysts); either may lead to seropositivity” if this cut-off is based on data, or if not please state whether this is an assumption based on expert-opinion?

--------------------

Results

-Does the analysis presented match the analysis plan?

-Are the results clearly and completely presented?

-Are the figures (Tables, Images) of sufficient quality for clarity?

Reviewer #1: (No Response)

Reviewer #2: Again, thank you for your responses which were very clear, and the corresponding edits to the manuscript. I have one remaining question:

Original Q. If I am understanding correctly, the median difference referenced on line 8 (for taeniasis) and 11 (porcine cysticercosis) of pg.14 relate to the median (absolute) differences? Can the authors explicitly state whether they refer to the absolute or relative differences throughout the manuscript please.

New question: Thanks for the amendments and clarifications. I am not sure I fully understand the axis and accompanying results text for Figure 2. Does the x-axis on figure 2 for absolute error (bottom two panels) refer to (% points difference), as I am not sure how a median absolute error (difference) is therefore estimated from Figure 2 (i.e., the absolute error difference at baseline for taeniasis seems to have a max lower limit across villages of -0.03 and max upper limit across villages of +0.065)?

--------------------

Conclusions

-Are the conclusions supported by the data presented?

-Are the limitations of analysis clearly described?

-Do the authors discuss how these data can be helpful to advance our understanding of the topic under study?

-Is public health relevance addressed?

Reviewer #1: (No Response)

Reviewer #2: (No Response)

--------------------

Editorial and Data Presentation Modifications?

Use this section for editorial suggestions as well as relatively minor modifications of existing data that would enhance clarity. If the only modifications needed are minor and/or editorial, you may wish to recommend “Minor Revision” or “Accept”.

Reviewer #1: (No Response)

Reviewer #2: (No Response)

--------------------

Summary and General Comments

Use this section to provide overall comments, discuss strengths/weaknesses of the study, novelty, significance, general execution and scholarship. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. If requesting major revision, please articulate the new experiments that are needed.

Reviewer #1: Thank you for addressing my comments.

Reviewer #2: (No Response)

--------------------

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Brecht Devleesschauwer

Reviewer #2: Yes: Matthew Dixon-Zegeye

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.

Reproducibility:

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

6 Oct 2021

Submitted filename: CystiAgentValidation_PLOS_ResponseToReviewers.docx

Click here for additional data file.

8 Oct 2021

Dear Mr. Pray,

We are pleased to inform you that your manuscript 'Validation of a spatial agent-based model for Taenia solium transmission (“CystiAgent”) against a large prospective trial of control strategies in northern Peru' has been provisionally accepted for publication in PLOS Neglected Tropical Diseases.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us now if you or your institution is planning to press release the article. All press must be co-ordinated with PLOS.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Neglected Tropical Diseases.

Best regards,

Paul R. Torgerson

Associate Editor

PLOS Neglected Tropical Diseases

Makedonka Mitreva

Deputy Editor

PLOS Neglected Tropical Diseases

***********************************************************

I think the authors have satisfactorily addressed the reviewers comments and the manuscript can now go forward for publication

22 Oct 2021

Dear Mr. Pray,

We are delighted to inform you that your manuscript, "Validation of a spatial agent-based model for Taenia solium transmission (“CystiAgent”) against a large prospective trial of control strategies in northern Peru," has been formally accepted for publication in PLOS Neglected Tropical Diseases.

We have now passed your article onto the PLOS Production Department who will complete the rest of the publication process. All authors will receive a confirmation email upon publication.

The corresponding author will soon be receiving a typeset proof for review, to ensure errors have not been introduced during production. Please review the PDF proof of your manuscript carefully, as this is the last chance to correct any scientific or type-setting errors. Please note that major changes, or those which affect the scientific understanding of the work, will likely cause delays to the publication date of your manuscript. Note: Proofs for Front Matter articles (Editorial, Viewpoint, Symposium, Review, etc...) are generated on a different schedule and may not be made available as quickly.

Soon after your final files are uploaded, the early version of your manuscript will be published online unless you opted out of this process. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.

Thank you again for supporting open-access publishing; we are looking forward to publishing your work in PLOS Neglected Tropical Diseases.

Best regards,

Shaden Kamhawi

co-Editor-in-Chief

PLOS Neglected Tropical Diseases

Paul Brindley

co-Editor-in-Chief

PLOS Neglected Tropical Diseases