Fundamental limits on inferring epidemic resurgence in real time using effective reproduction numbers.

We find that epidemic resurgence, defined as an upswing in the effective reproduction number (R) of the contagion from subcritical to supercritical values, is fundamentally difficult to detect in real time. Inherent latencies in pathogen transmission, coupled with smaller and intrinsically noisier case incidence across periods of subcritical spread, mean that resurgence cannot be reliably detected without significant delays of the order of the generation time of the disease, even when case reporting is perfect. In contrast, epidemic suppression (where R falls from supercritical to subcritical values) may be ascertained 5-10 times faster due to the naturally larger incidence at which control actions are generally applied. We prove that these innate limits on detecting resurgence only worsen when spatial or demographic heterogeneities are incorporated. Consequently, we argue that resurgence is more effectively handled proactively, potentially at the expense of false alarms. Timely responses to recrudescent infections or emerging variants of concern are more likely to be possible when policy is informed by a greater quality and diversity of surveillance data than by further optimisation of the statistical models used to process routine outbreak data.

Introduction

Real-time estimates of the transmissibility of an infectious disease [1,2] are crucial for informed outbreak responses. Timely detection of salient changes in the effective reproduction number (R) of the disease of interest, which measures the average number of secondary cases likely caused by a typical primary case, can provide important evidence for policymaking and public communication [3,4], as well as improve forecasts of disease burden [5] (e.g., hospitalisations and deaths). Two critical changes of interest are resurgence and control. Resurgence, which we define as an increase from subcritical (R ≤ 1) to supercritical (R > 1) transmissibility, can warn of imminent waves of infections, signify the emergence of pathogenic variants of concern and signal important shifts in the behavioural patterns of population [6,7]. Alternatively, control (or suppression) describes a switch from supercritical to subcritical spread and can indicate the effectiveness of interventions and the impact of depleting susceptibility (including that due to vaccine-induced immunity) [8,9].

Identifying these transmissibility changes in real time, however, is an enduring challenge for statistical modelling and surveillance planning. Inferring a transition in R from stochastic time series of incident cases necessitates assumptions about the differences among meaningful variations (signal) and random fluctuations (noise) [10-12]. Modern approaches to epidemic modelling and monitoring aim to maximise this signal-to-noise ratio either by enhancing noise filtering and bias correction methods [13-15], or by amplifying signal fidelity through improving surveillance quality and diversity [16-18]. While both approaches have substantially advanced the field, there have been few attempts to explore what, if any, fundamental limits exist on the timely detection of these changes. Such limits can provide key benchmarks for assessing the effectiveness of modelling or data collection and deepen our understanding of what can and cannot be achieved by real-time outbreak response programmes, ensuring that model outputs are not overinterpreted and redirecting surveillance resources more efficiently [19-21].

While studies are examining intrinsic bounds on epidemic monitoring and forecasting [22-25], works on transmissibility have mostly probed how extrinsic surveillance biases might cause R misestimation [14,26-28]. Here we address these gaps in the literature by characterising and exposing fundamental limits on detecting resurgence and control, from a perfectly ascertained incidence time series, using effective reproduction numbers. This presents new and useful insights into the best real-time performance possible and blueprints for how outbreak preparedness might be improved. We analyse a predominant, flexible real-time epidemic model [1,2] and discover stark asymmetries in our intrinsic ability to detect resurgence and control, emerging from the noisier, low-incidence data underlying possible resurgence events. While epidemic control or suppression change-points are inferred robustly and rapidly, the data bottleneck caused by subcritical spread forces inherent delays (potentially 5–10 times that for control and on the order of the mean disease generation time) that inhibit real-time resurgence estimation.

We show that these innate constraints on resurgence detection worsen with smaller epidemic size, steepness of the upswing in R and spatial or demographic heterogeneities. Given these limitations to timely outbreak analysis, which exist despite perfect case reporting and the use of optimal Bayesian detection algorithms [15,29], we argue that methodological improvements to existing models for analysing epidemic curves (e.g., cases, hospitalisations or deaths) are less important than designing enhanced and integrated surveillance systems [30,31]. Such systems, which might fuse multiple data streams including novel ones (e.g., wastewater [32]) to triangulate possible resurgences, could minimize some of these fundamental bottlenecks. We conclude that early responses to suspected resurging epidemics, at the expense of false alarms, might be justified in many settings, both from our analysis and the consensus that lags in implementing interventions can translate into severely elevated epidemic burden [33-36]. While such decisions must, ultimately, be weighed against the cost of those interventions, the bottlenecks we expose, hopefully, bolster the evidence base for decision-making. Using theory and simulation, we explore and elucidate these conclusions in the next section.

Results

Epidemic resurgence is statistically more difficult to infer than control

We first provide intuition for why resurgence and control might present asymmetric difficulties when inferring transmissibility in real time. We consider an epidemic modelled via a renewal branching process [37] over times (usually in days) 1≤s≤t. Such models have been widely applied to infer the transmissibility of many diseases including COVID-19, pandemic influenza and Ebola virus disease. Renewal models postulate that the incidence of new cases at time s, denoted I, depends on the effective reproduction number, R, and the past incidence, as in Eq ( [2]. Here means the set or time series {I, I,…,I} and ≡ indicates equality in distribution.

In Eq (, Pois represents Poisson noise and Λ is the total infectiousness, which summarises the weighted influence of past infections. The set of weights w for all u define the generation time distribution of the infectious disease with [38]. We assume that all w are known. If this distribution changes across the epidemic [39], we can recompute the Λ terms after that change to model its effects. Applying Bayesian inference techniques (see for all derivations) [2,40] under the assumption that transmissibility is constant over a past window of size m days, , we obtain the gamma (Gam) posterior distribution given the incidence data , with sums across the window of and .

Here (a, c) are prior distribution (P(R)) parameters, which are set so the prior mean of R is above 1 but uninformative. This maximises sensitivity to resurgence since the model, in the absence of data, favours E[R]>1. The approximations above and later emerge from the window assumption and underpin popular real-time R-inference methods [2,41]. Using this renewal formulation, we define the relative change in the epidemic size as . This measures the perturbation to the past incidence (summarised by λ) that the most recently observed incidence, i, causes over τ(s). Normalising by λ is sensible as the posterior mean estimate of R is roughly , so Δλ approximates R−1.

This posterior distribution only uses data up until time s and defines our real-time estimate of R at that time. We can analyse its properties (and related likelihood function ) to obtain the Fisher information (FI) on the left side of Eq (. We derive this expression in the . This FI captures how informative is (here approximated by ) for inferring R, with its inverse defining the smallest asymptotic variance of any R estimate [10,42]. Larger FI implies better statistical precision.

As resurgence will likely follow low-incidence periods, we might expect λ to be small, while R rises. This effect will reduce the FI in Eq (, making these changes harder to detect. In contrast, the impact of interventions will be easier to infer since these are often applied when cases are larger (so λ will be big) and reduce R. This observation applies for any τ(s) and is fundamental as it delimits the best estimator performance under our renewal model (Cramer-Rao bound) [43].

We expand on this intuition, using the R posterior distribution to derive (see ) the real-time resurgence probability , as on the right side of Eq (. We plot its implications in , corroborating our intuition. In panel A we find that larger past epidemic sizes (λ) improve our ability to detect transmissibility shifts from fluctuations in incidence (the posterior distributions for R overlap less). Panel B bolsters this idea, showing that when λ is smaller (as is likely before resurgence) we need to observe larger relative epidemic size changes (Δλ) for some increase in than for an equivalent decrease when aiming to detect control (where λ will often be larger). This detection asymmetry holds for arbitrary window sizes and indicates that data bottlenecks translate into real-time detection delays. We assess the magnitude of these delays next.

Relative sensitivity to perturbations in incidence.

Panel A plots posterior real-time distributions for time-varying reproduction numbers R, given incidence data , at different relative incidence perturbations, (increasing from blue to red). Here τ(s) represents some arbitrary window size used in computation (see Eq (). The degree of distribution separation and hence our ability to uncover meaningful incidence fluctuations from noise, improves with the current epidemic size, λ (i.e., as this increases from 25–400 overlap among the distributions decreases). Panel B shows how this sensitivity modulates our capacity to infer resurgence () and control (). If epidemic size is smaller, larger relative incidence perturbations are required to detect the same change in R (curves have steeper gradient as we traverse from blue to red). Resurgence (likely closer to the blue line in the top right quadrant) is appreciably and innately harder to detect than control (likely closer to the red line, in the bottom left quadrant).

Fundamental delays on detecting resurgence but not control

The intrinsic asymmetry in sensitivity to upward versus downward shifts in R (see ) implies that it is not equally simple to infer resurgence and control from incident cases. We investigate ramifications of this observation by comparing our real-time R-estimates to ones exploiting all the future incidence information available. We no longer consider window-based approximations (which we only use to extract analytic insights) but instead apply formal real-time Bayesian inference and detection algorithms [29]. We investigate two foundational posterior distributions, the filtered, p, and smoothed, q, distributions, defined as in Eq (.

Here p considers all information until time s and captures changes in R from in real time. Estimates of R using this posterior distribution minimise the mean squared error (MSE) given . In contrast, q extracts all the information from the full incidence curve , providing the minimum MSE R-estimate given [29]. This smoother MSE is never larger and may be substantially smaller than the filtered MSE due to its use of additional information (i.e., ) [29,44]. The differential between p and q, summarised via the Kullback-Liebler divergence, D(p|q), measures the value of this additional ‘future’ information.

Bayesian filtering and smoothing are central formalisms across engineering, where real-time inference and detection problems are common [29,45]. We compute formulae from Eq ( via the EpiFilter package (see [15,28]), which uses optimal forward-backward algorithms, improves on the window-based approach of the last section and maximises the signal-to-noise ratio in R-estimation. We further obtain filtered and smoothed probabilities of resurgence as and . The probability that the epidemic is controlled (i.e., R ≤ 1) is the complement of these expressions. Our main results, which average the above quantities over many simulated Ebola virus and COVID-19 epidemics, are given in and Fig A in the , respectively. The simulated incidence curves are also provided in Figs B-C in the and illustrate the expected differences in case numbers associated with both upward and downward shifts in R. We uncover striking differences in the intrinsic ability to infer resurgence versus control in real time.

Resurgence and control dynamics of Ebola virus.

Using renewal models with the generation time from [46], we simulate 1000 realisations of Ebola virus epidemics (t = 300) with step (A panels) and seasonally (B panels) changing transmissibility (true R in black). Top panels show posterior mean estimates from the filtered (E[R], blue) and smoothed (E[R], red) distributions from every realisation (computed using EpiFilter [15]). Middle panels average the Kullback-Liebler divergences from those simulations, D(p|q), and bottom panels display the overall filtered (, blue), and smoothed (, red) probabilities of resurgence. We find fundamental and striking delays in detecting resurgence, often an order of magnitude longer than those for detecting control or suppression in transmission (see lags between red and blue curves in all relevant panels). Note that the initial rise in of panel A, which precedes the transition in R, is due to the influence of the prior distribution (which has a mean above 1) in a period with very few cases. We present the incidence curves that underlie the simulations here in Fig C in the .

Upward change-points are significantly harder to detect both in terms of accuracy and timing. Discrepancies between p- and q-based estimates (the latter benchmark the best realisable performance) are appreciably larger for resurgence than control. While decreases in R can be pinpointed reliably, increases seem fundamentally more difficult to detect. These limits appear to exacerbate with the steepness of the R upswing. We confirm these trends with a detailed simulation study across five infectious diseases in . There we alter the steepness, θ, of transmissibility changes and map delays in detecting resurgence and control as a function of the difference in the first time that p- and q-based probabilities cross 0.5 (Δt50) and 0.95 (Δt95), normalised by the mean generation time of the disease. We find that lags in detecting resurgence can be at least 5–10 times longer than for detecting control and are of the order of the average intrinsic generation time of the disease.

Delays in detecting upward and downward changes in R.

We characterise the discrepancies between detecting resurgence and control against the steepness or rate, θ, of changes in transmissibility (R), which we model using logistic functions (panel A, steepness increases from blue to red). We compare differences in the probability of detecting resurgence (P(R>1)) or control (P(R≤1) under filtered and smoothed estimates (see main text) first crossing thresholds of 0.5 (Δt50) and 0.95 (Δt95) for five infectious diseases (panel B plots their assumed generation time distributions from [2,46,47]). We simulate 1000 epidemics from each disease using renewal models and estimate R with EpiFilter [15]. Panels C and D (here colours match panel B, Δt is normalised by the mean generation times of the diseases) show that delays in detecting resurgence (dots with colours indicating the disease) are at least 5–10 times longer than for detecting control (diamonds with equivalent colours). Our ability to infer even symmetrical transmissibility changes is fundamentally asymmetric, largely due to the differences in case incidence at which those changes usually tend to occur.

Fundamental delays worsen with spatial or demographic heterogeneities

In previous sections we demonstrated that sensitivity to changes in R is asymmetric, and that intrinsic, restrictive limits exist on detecting resurgence in real time, which do not equally inhibit detecting control. While those conclusions apply generally (e.g., across diseases), they do not consider the influence of spatial or demographic heterogeneity. We examine this complexity through a simple but realistic generalisation of the renewal model. Often R-estimates can be computed at small scales (e.g., at the municipality level) via local incidence or more coarsely (e.g., countrywide), using aggregated case counts [3,13]. We can relate these differing scales with the weighted mean in Eq (, where the overall (coarse) R at time s, , is a convex sum of finer-scale R contributions from each group (R[j] for the jth of p groups) weighted by the epidemic size of that group (as in Eq ( we use windows τ(s), of some size m, to derive analytic insight).

Our choice of groupings is arbitrary and can equally model demographic heterogeneities (e.g., age-specific transmission), where we want to understand how dynamics within the subgroups influence overall spread [7]. Our aim is to ascertain how grouping, which often occurs naturally due to data constraints or a need to succinctly describe the infectious dynamics over a country to aid policymaking or public communication [48], affects resurgence detection. Eq ( implies that . Since resurgence will likely first occur within some specific (maybe high risk) group and then propagate to other groups [7], this expression suggests that an initial signal (e.g., if some R[j]>1) could be masked by non-resurging groups (which are, from this perspective, contributing background noise).

As the epidemic size in a resurging group will likely be smaller than those of groups with past epidemics that are now being stabilised or controlled, this exacerbates the sensitivity bounds explored earlier via Eq (. We can verify this further loss of sensitivity by examining how the overall posterior distribution depends on those of the p component groups as follows, with ⊛ as a repeated convolution operation and Ω as some generic posterior distribution for the jth group.

While Eq ( holds generally, we assume gamma posterior distributions, leading to statistics analogous to Eq (. We plot these sensitivity results at p = 2 and 3 in , where group 1 features resurgence and other groups either contain stable or falling incidence. We find that as p grows (and additional distributions convolve to generate ) we lose sensitivity (posterior distributions overlap more for a given relative change in incidence (). Reductions in either the weight (α1), epidemic size (λ[1]) or other R[j≠1], further desensitise the resurgence signals i.e., decrease the gradient of the detection probability curves. This is summarised by noting that if R[1] = max R[j], then the sensitivity from Eq ( is only matched when the resurging group dominates (α1≈1) or if other groups have analogous R i.e., R[1]≈R[j]. Delays in detecting resurgence can therefore be severe. Heterogeneity on its own, however, does not force asymmetry between detecting control and resurgence.

Influence of heterogeneities in transmission.

We investigate how differences in transmissibility among groups (e.g., due to demographic or spatial factors) fundamentally limit the ability to detect resurgence from a specific group (in this example group 1 with reproduction number R[1]). Panel A shows that the grouped posterior distribution becomes less sensitive to a fixed relative change in group 1 incidence, (the level of change increases from blue to red). Posterior distributions over (the overall reproduction number across groups) are more overlapped (and tighter in variance) as p rises, for fixed R[1] (top). Panel B plots how overall resurgence detection probability depends on the weight (α1, top, 0.05–1) and epidemic size (λ[1], middle, 20–80, p = 2) as well as changes in R[3] (bottom, 0.5–1.2, p = 3). Decreases in α1 (red to blue) or λ[1] mean other groups mask the resurging dynamics in group 1, reducing sensitivity (curves become less steep). In the latter case the (green with solid line at median of λ[1] range) is always more conservative than P(R[1]>1) (black with solid median line). As R[3] falls (red to blue) the ability to detect resurgence also lags relative to that from observing group 1 (black).

Discussion

Probing the performance limits of noisy biological systems has yielded important insights into the real-time estimation and control of parameters in biochemistry and neuroscience [49-51]. Although models from these fields share dynamic similarities with those in infectious disease epidemiology, there has been relatively little investigation of how real-time estimates of pathogen transmissibility, parametrised by R, might be fundamentally limited. This is surprising since R is among key parameters considered in initiatives aiming to better systematise real-time epidemic response [41,52]. Here we explored what limits may exist on our ability to reliably detect or measure the change-points in R that signify resurgence and control. By using a combination of Bayesian sensitivity analyses and minimum MSE filtering and smoothing algorithms, we discovered striking asymmetries in innate detection sensitivities. We found that, arguably, the most crucial transitions in epidemic transmissibility are possibly the most inherently difficult to detect.

Specifically, resurgence, signified by an increase in R from below to above 1, can possibly be detected only 5–10 times later than an equivalent decrease in R that indicated control (Figs and , and Fig A in ). As this lag can be of the order of the mean generation time of the pathogen under study, even when case reporting is perfect and optimised detection algorithms are applied, this represents a potentially sharp bottleneck to real-time responses for highly contagious diseases. Intuition for this result came from observing that sensitivity to R change-points will weaken (due to noise masking the signal) with declining epidemic sizes or case incidence, and increasing ‘true’ R, both of which likely occur in resurgent settings due to periods of subcritical spread (Eq ( and ). The converse applies to control, which is usually enforced in larger (and less intrinsically noisy) incidence regimes. Furthermore, we found that these latencies and sensitivity issues would only exacerbate when heterogeneous groupings across geography or demography (Eqs ( and ) are considered.

An interesting corollary of these results occurs if we consider the detection of an upward shift in R at large incidence. If this increase affects the majority of cases (i.e., Eq ( applies), then we would detect it without significant delay because epidemic curves are now inherently less noisy. However, if incidence is large and a resurgence occurs in some subset of the cases (i.e., the upward R-shift is localised to group j and Eq ( applies) then we would still face the innate delay of a mean generation time together with further loss of sensitivity due to the cases in groups other than j acting as background noise. This scenario might realistically occur when a new pathogenic variant emerges (e.g., the alpha COVID-19 variant appeared during a high incidence period in the UK [53]) or when specific age groups sustain resurgence (e.g., the 20–49 age group for COVID-19 in the USA [7]). These detection delays limit our ability to rapidly identify and target interventions at resurgent groups. Our work emphasises that the correlations among incidence, transmissibility parameters underlying this incidence and heterogeneous groups contributing to that incidence can fundamentally constrain our response sensitivity and timeliness.

Practical real-time analyses often involve grouping or data aggregation [9,13] and are subject to reporting and other latencies (e.g., if notifications, hospitalisations or deaths are used as proxies for infection incidence), which introduce additive delays on top of those we uncovered [14,54]. Consequently, we argue that while case data may provide robust signals for pinpointing when epidemics are under control (and assessing impacts of interventions), they are insufficient, on their own, to sharply resolve resurgence at low incidence. This does not devalue methods seeking to better characterise real-time R changes [1,2,13,28], but instead contextualises how such inferences should be interpreted when informing policy. Given the intrinsic delays in detecting resurgences, which might associate with critical epidemiological changes such as variants of concern or shifts in population behaviours [6,7], there might be grounds for conservative policies (e.g., those of New Zealand and Australia for COVID-19 [55]) that trade off early interventions against the expense of false alarms. While the value of such policies ultimately depends on many complex economic, political and socio-behavioural factors, our study, together with works that show how lags in enacting interventions can induce drastic costs [33-36], provides a first step towards dissecting some of these trade-offs.

Moreover, our analyses suggest that designing enhanced surveillance systems, which can comprehensively engage and integrate diverse data sources [30,31] may be more important than improving models for processing case data. Fusing multiple and sometimes novel data sources, such as wastewater or cross-sectional viral loads [18,32], may present the only truly realistic means of minimizing the innate bottlenecks to resurgence detection that we have demonstrated. Approaches aimed at improving case-based inference generally correct for reporting biases or propose more robust measures of transmissibility, such as time-varying growth rates [14,41,56]. However, as our study highlights limits that persist at the gold standard of perfect case reporting and, further it is known that under such conditions growth rates and R are equally informative [57], these lines of investigation are unlikely to minimise the detection limits that we have exposed.

There are three main limitations of our results. First, as we only considered renewal model epidemic descriptions with assumed generation times, which predominate real-time R studies, our work necessarily neglects the often-complex contact network structures that can mediate infection spread [58] or lead to intervention-induced generation time changes [39]. However, other analyses using somewhat different approaches to ours (e.g., Hawkes processes [59]) show apparently similar sensitivity asymmetries. There is evidence that renewal models may be as accurate as network models for inferring R [60], while being easier to run and fit in real time. They are also known to be equivalent to various compartmental models [61]. We do not examine the influence of generation time changes, as data on those are rarely available for routine, real-time analyses. However, as the ratio of the resurgence to control lags is 5–10, we expect this asymmetry to be robust to generation time changes, which are relatively smaller [39]. Given the flexibility of our model and that the asymmetry we discovered is contingent on low-incidence data being noisier and typical of resurgence settings, which is a model agnostic point, we expect that the intrinsic limits we have exposed are general and not model artefacts.

Second, while we analysed one common and important definition of resurgence that depends on effective reproduction numbers, other more recent definitions of epidemic re-emergence exist that are linked to complex dynamic characteristics of diseases such as critical slowing down [62]. Our aim was to understand and expose limitations of the most common surveillance data types (incidence) and the most prominent epidemic summary statistics (time-varying or effective reproduction numbers), which are among those informing policy [41], so we did not examine such metrics. Testing to see if these other characteristics also show asymmetry could be an interesting follow-up study but would require different modelling approaches. Last, we did not include any explicit economic modelling. While this is outside the scope of this work it is important to recognise that resurgence detection threshold choices (i.e., how we decide which fluctuations in incidence are actionable) imply some judgment about the relative cost of true positives (timely resurgence detections) versus false alarms [12]. Incorporating explicit cost structures could mean that delays in detecting resurgence are acceptable. We consider this the next investigative step in our aim to probe the limits of real-time performance.

Methods

We derive some of the mathematical formulae central to the main text. Eq ( describes the renewal model [37], which simulates the spread of an epidemic, characterising how incidence at some time s, I, depends on the effective reproduction number at that time, R, and the total infectiousness, Λ. Inference under this model commonly assumes that an incidence window of size m defined as contains all the information about R [2]. Consequently, we have the Poisson joint log-likelihood over this window, l, (see Supplement of [40]), with grouped sums and , as follows.

In Eq (, is independent of R. The maximum likelihood estimate under this model solves i.e., . The Fisher information, FI[R], defines the best achievable precision (i.e., smallest variance) around this estimate [42], and is computed from Eq ( as [40,42]. This gives . Substituting E[I] = ΛR from Eq (, then yields the key result in the left side of Eq (.

Widely used real-time methods, such as EpiEstim [2] and related approaches, often calculate the posterior distribution . This approximation is a consequence of the m-window assumption and is conventionally obtained by setting a conjugate gamma prior distribution i.e., P(R)≡Gam(a, c−1). Hyperparameters (e.g., a = 1, c = 1/5) are often selected to ensure this prior distribution is uninformative. Applying Bayes law with the Poisson likelihood from Eq ( yields .

We can compute the resurgence probability as . This approximation also proceeds from the window-based formulation. The cumulative distribution function of the gamma posterior distribution, F(x), can be written as below for some x.

Eq ( results from standard properties of gamma distributions. We compute the resurgence probability as 1−F(1), which gives the right side of Eq (. The above formulae are useful both for providing analytic insight and measuring performance of realistic estimators used in outbreak analysis, which adhere to this formulation [2,13,41,63].

These equations all feature a dependence on the choice of window size m. As investigated in [40] large m can mean that we are slower to detect transmissibility changes, while small m can lead to oversensitivity to noise. We avoid this m-dependence by simply using this approach to gain general, theoretical insights into detection asymmetries and latencies. Specifically, in the main text we prove that the lag in inferring resurgence is larger than that when estimating a corresponding control signal, for arbitrary window sizes (due to smaller historical incidence across suspected periods of resurgence). We then perform more detailed (but less tractable) investigations to discern the likely magnitude of these asymmetric lags.

These investigations (in Figs and Fig A in ) apply the EpiFilter method [15], which largely circumvents window size issues. EpiFilter exploits formal signal processing theory to minimise the mean squared error in the estimation of R. Its sequential predictive accuracy (i.e., it has small generalisation error) and its ability to detect change-points in real time have been verified on extensive simulations [15,28], and suggest it as a tool suitable for exploring fundamental limits on resurgence and control. This difference in methodology is signified in our notation in Eq (, which no longer uses window approximations (τ(s)). There our results are direct outputs of EpiFilter.

Derivations for the inference equations behind the filtering and smoothing in EpiFilter are in [15,29]. This more general formulation allows us to go beyond the analytic insights from the EpiEstim type models above and limits the influence of prior distributions on results (which is particularly strong when incidence is small) since R is a-priori uniformly distributed over some wide range ([0.01, 10] here). Consequently, we examine the problem of resurgence detection from multiple angles. The prior distributions used in all methods have mean and median above 1 so that any delays we find in detecting resurgence are the minimum possible.

The trends uncovered in Eq (, where heterogeneity or grouping is explored, are within the EpiEstim framework, but will be valid for EpiFilter and general R-estimation methods, since they result from the properties of convex sums and averages only. Last, while our conclusions may appear limited due to their dependence on renewal models, we note that renewal models (i) can describe realistic transmission patterns for many diseases with accuracies comparable to that of more detailed network-based models [60] (ii) are the dominant model for measuring real-time outbreak changes [1,41,60] and (iii) are able to equivalently represent the dynamics of prevailing compartmental models, such as the SEIR model, depending on the form of the generation time distribution considered [61].

This provides additional Figs A-C.

Fig A: Resurgence and control dynamics of COVID-19. We repeat the simulations from but for realisations of COVID-19 epidemics. Fig B: Incidence curves for COVID-19. We present the simulated counts of daily new cases that underlie the results of Fig A. Fig C: Incidence curves for Ebola virus disease. We present the counts of daily new cases that underlie the results of of the main text.

(PDF)

Click here for additional data file.

4 Feb 2022

Dear Dr Parag,

Thank you very much for submitting your manuscript "Fundamental limits on inferring epidemic resurgence in real time using effective reproduction numbers" for consideration at PLOS Computational Biology. As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. The reviewers appreciated the attention to an important topic. Based on the reviews, we are likely to accept this manuscript for publication, providing that you modify the manuscript according to the review recommendations.

Please prepare and submit your revised manuscript within 30 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to all review comments, and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Thank you again for your submission to our journal. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Joseph T. Wu

Associate Editor

PLOS Computational Biology

Virginia Pitzer

Deputy Editor-in-Chief

PLOS Computational Biology

***********************

A link appears below if there are any accompanying review attachments. If you believe any reviews to be missing, please contact ploscompbiol@plos.org immediately:

[LINK]

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #1: In "Fundamental limits on inferring epidemic resurgence in real time using effective reproduction numbers", Parag and Donnelly examine out ability to detect changes in epidemic growth, as characterized by the reproductive number R, in real time. They find that there are delays in our ability to detect changes in the reproductive number, particularly changes around the critical threshold of R being above or below 1, that are inherent in the generating process and hence cannot be overcome by improved analysis of epidemic curves. In many ways there results are obvious, shifts in R cannot be detected without data, and since that data comes from subsequent generations of infection that are necessarily delayed by the generation time of the disease, there is an upper limit to how quickly we can detect these shifts. Likewise, as more cases are present when epidemics shift to a decline than when they shift to a resurgence, statistical theory tells us the former will be easier to detect than the latter.

My statement that these conclusions are in retrospect obvious is in no way a negative comment on this work…much of the best science seems obvious in retrospect. The thorough and comprehensive way in which the authors prove their point and quantify many of these delays is compelling and invaluable, and I found no technical problems with their work. I found the use of classical statistical measures, such as Fisher information, to characterize many of the results a particularly compelling. Hence, I think this analysis is a strong and important contribution to the literature.

As detailed below, I do think the presentation (particularly the figures) could be made more clear and that the authors could be a little bit more nuanced in their discussion of the practical impact of these results. However, these are simply improvements to what is already and interesting and informative paper.

Specific Comments:

1 Abstract, "This belies epidemic…"

I don't think this means quite what the authors intend, or I am misunderstanding the rest of the sentence.

2 Abstract, "Responses to recrudescent…"

I think this misses the nuance that is around a similar statement in the main text. I.e., it does not come across that what is being argued is that improvements to analysis of epidemics curves are less important than improving surveillance.

3 Introduction, "…enhancing syndromic…Such systems…"

I feel like the argument here is broader than syndromic surveillance, as highlighted by the reference to wastewater surveillance later. I.e., the authors appear to be arguing that novel methods beyond syndromic surveillance might be useful.

4 Figures 1-4

I will admit to finding the figures a bit hard to decode, and wonder if something can be done to make the meaning of the colors, axes, etc. a bit more clear.

5 Discussion, "…there are grounds….that enact interventions…at the expense of false alarms"

I wonder if this argument goes too far beyond what the analysis in the paper supports, and might warrant a more nuanced discussion. Without an analysis of the cost of false alarms, it is hard to say if a conservative approach is warranted or not. Further, this decision will depend on the cost of an intervention over the short and long term, which will likely be specific to intervention and context. As to the latter point, my perception is that in the United States public health agencies had the political capital to implement interventions only a limited number of times, while in some other countries there was more acceptance or repeated imposition of control measures during the COVID-19 pandemic. I certainly don't think it is incumbent on the authors to wade into all such thorny issues, but I think it would be worthwhile for them to acknowledge they exist and that their analysis represents only a first step in figuring out how to better optimize responses. [I now see that this was addressed a bit in the last paragraph, but still think it warrants a bit more discussion here].

6 Discussion, "…enhancing syndromic surveillance…."

See above comments about limiting to syndromic.

Reviewer #2: Please see attachment.

**********

Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example in PLOS Biology see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.

Reproducibility:

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

References:

Review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript.

If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Submitted filename: Parag_review_upload.pdf

Click here for additional data file.

11 Feb 2022

Submitted filename: responses.docx

Click here for additional data file.

8 Mar 2022

Dear Dr Parag,

We are pleased to inform you that your manuscript 'Fundamental limits on inferring epidemic resurgence in real time using effective reproduction numbers' has been provisionally accepted for publication in PLOS Computational Biology.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us now if you or your institution is planning to press release the article. All press must be co-ordinated with PLOS.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Computational Biology.

Best regards,

Joseph T. Wu

Associate Editor

PLOS Computational Biology

Virginia Pitzer

Deputy Editor-in-Chief

PLOS Computational Biology

***********************************************************

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #1: The authors have done a good job of addressing most of my concerns, and I think this paper will be a valuable contribution to the literature. I do still find the figures a touch challenging, but understand that the concepts being conveyed are complex.

Reviewer #2: Thank you for your comprehensive response to the points raised.

**********

Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

6 Apr 2022

PCOMPBIOL-D-21-02218R1

Fundamental limits on inferring epidemic resurgence in real time using effective reproduction numbers

Dear Dr Parag,

I am pleased to inform you that your manuscript has been formally accepted for publication in PLOS Computational Biology. Your manuscript is now with our production department and you will be notified of the publication date in due course.

The corresponding author will soon be receiving a typeset proof for review, to ensure errors have not been introduced during production. Please review the PDF proof of your manuscript carefully, as this is the last chance to correct any errors. Please note that major changes, or those which affect the scientific understanding of the work, will likely cause delays to the publication date of your manuscript.

Soon after your final files are uploaded, unless you have opted out, the early version of your manuscript will be published online. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.

Thank you again for supporting PLOS Computational Biology and open-access publishing. We are looking forward to publishing your work!

With kind regards,

Livia Horvath

PLOS Computational Biology | Carlyle House, Carlyle Road, Cambridge CB4 3DN | United Kingdom ploscompbiol@plos.org | Phone +44 (0) 1223-442824 | ploscompbiol.org | @PLOSCompBiol