Using observational data to quantify bias of traveller-derived COVID-19 prevalence estimates in Wuhan, China.

BACKGROUND: The incidence of coronavirus disease 2019 (COVID-19) in Wuhan, China, has been estimated using imported case counts of international travellers, generally under the assumptions that all cases of the disease in travellers have been ascertained and that infection prevalence in travellers and residents is the same. However, findings indicate variation among locations in the capacity for detection of imported cases. Singapore has had very strong epidemiological surveillance and contact tracing capacity during previous infectious disease outbreaks and has consistently shown high sensitivity of case-detection during the COVID-19 outbreak.
METHODS: We used a Bayesian modelling approach to estimate the relative capacity for detection of imported cases of COVID-19 for 194 locations (excluding China) compared with that for Singapore. We also built a simple mathematical model of the point prevalence of infection in visitors to an epicentre relative to that in residents.
FINDINGS: The weighted global ability to detect Wuhan-to-location imported cases of COVID-19 was estimated to be 38% (95% highest posterior density interval [HPDI] 22-64) of Singapore's capacity. This value is equivalent to 2·8 (95% HPDI 1·5-4·4) times the current number of imported and reported cases that could have been detected if all locations had had the same detection capacity as Singapore. Using the second component of the Global Health Security index to stratify likely case-detection capacities, the ability to detect imported cases relative to Singapore was 40% (95% HPDI 22-67) among locations with high surveillance capacity, 37% (18-68) among locations with medium surveillance capacity, and 11% (0-42) among locations with low surveillance capacity. Treating all travellers as if they were residents (rather than accounting for the brief stay of some of these travellers in Wuhan) contributed modestly to underestimation of prevalence.
INTERPRETATION: Estimates of case counts in Wuhan based on assumptions of 100% detection in travellers could have been underestimated by several fold. Furthermore, severity estimates will be inflated several fold since they also rely on case count estimates. Finally, our model supports evidence that underdetected cases of COVID-19 have probably spread in most locations around the world, with greatest risk in locations of low detection capacity and high connectivity to the epicentre of the outbreak. FUNDING: US National Institute of General Medical Sciences, and Fellowship Foundation Ramon Areces.

Introduction

During the outbreak of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), infections in travellers have been used to estimate the size of the epidemic in Wuhan, Hubei province, China, which was the epicentre of the outbreak. This approach is similar to that used for the 2009 H1N1 influenza pandemic, for which infections in tourists returning from Mexico were used to estimate the time-specific risk of infection (incidence or cumulative incidence) with the novel pandemic H1N1 influenza strain in Mexico (or parts thereof). The idea was that surveillance for H1N1 influenza virus was not intense during the early days of the pandemic in Mexico, the source location, and that detection would be far more sensitive in travellers leaving Mexico, who would be screened when returning home as a means of preventing introductions of cases into destination locations.2, 3 Reports that health systems in Wuhan were overwhelmed, and that many cases of coronavirus disease 2019 (COVID-19) were not being counted, led to the use of outgoing traveller data to estimate the time-specific risk of COVID-19 in Wuhan. This estimate, in turn, has been used to estimate the cumulative incidence of infection by a specific date in Wuhan and, from there (typically assuming exponential growth and no appreciable depletion of susceptible people), the cumulative number of cases. Two important assumptions underlie this calculation. First, it assumes that detection of cases in the destination location has been 100% sensitive and specific, whether cases are detected at the airport (with symptoms) or later after arrival at their destination (incubating during travel). Second, it assumes that travellers have the same prevalence of infection as does the average resident of Wuhan, so the prevalence inferred in travellers can be directly applied in Wuhan. Here, we consider the extent to which these two assumptions are justified and the expected effects they will have on our understanding of the current outbreak of COVID-19.

We have previously reported variability between locations in the world in the relation between the number of travellers from Wuhan to each international destination and the number of imported cases detected in that destination. On average, across locations presumed to have high surveillance capacity, an increase of 31 passengers per day in estimated travel volume from Wuhan to each destination was associated with one additional imported case reported over the period Jan 8, 2020, to Feb 4, 2020. However variation was reported around this average. Among destinations with substantial travel volume, Singapore showed the highest ratio of detected imported cases to daily travel volume, a ratio of 1·0 cumulative case count per 10·3 daily travellers. Singapore is known for exceptionally sensitive detection of cases (eg, during the 2003 outbreak of severe acute respiratory syndrome [SARS]) and has had very detailed case-reporting during the COVID-19 outbreak. Therefore, to test the first assumption, that case-detection has been 100% sensitive, we use Singapore as an example location with very strong case-detection capacity, and we estimate the capacities of other locations relative to Singapore.

Evidence before this study

We searched Google Scholar and PubMed on Feb 12, 2020, with the terms (“COVID-19” OR “SARS-CoV 2” OR “SARS coronavirus 2”) AND (“Wuhan” OR “Hubei”) AND “incidence traveler”. We searched for any type of article published in English between Dec 1, 2019, and Feb 12, 2020. Current work to estimate the incidence of coronavirus disease 2019 (COVID-19) in Wuhan uses cases detected outside of China. From this work, other estimates are derived, such as case-fatality rates and risk of exportation to locations without yet-detected cases. Assumptions are made that the detection capacity of cases in destination locations is 100% and that travellers from Wuhan have the same prevalence of infection as does the average resident of Wuhan.

Added value of this study

We tested both these assumptions and quantified the bias that they introduce. Using a Bayesian modelling approach with WHO case counts of imported cases, estimates of passenger volume from Wuhan to destination locations, and the Global Health Security index of epidemic surveillance strength, we have provided the first estimates of the global ability to detect imported cases of COVID-19. Importantly, we have also shown the variation of this ability between regions with different surveillance strength. Finally, we have provided the first mathematical model to estimate the infection prevalence of Wuhan visitors relative to residents as a function of key variables, such as the visit duration.

Implications of all the available evidence

Our study will allow better estimates to be produced of the global burden of COVID-19, in view of the large underdetection of cases. Our findings will support rapid deployment of outbreak surveillance and control capacities in regions at high risk of case importation paired with low surveillance capacity. Our research implies that existing estimates, which assume perfect detection of cases outside of China, should increase values for infection prevalence in the epicentre of the epidemic and reduce estimates of case-fatality based on our prevalence estimates.

To test the second assumption, that travellers and residents of Wuhan have the same prevalence of infection, we built a simple mathematical model of the point prevalence of infection in visitors relative to that in residents. This model allows us to study the expected discrepancy between visitor and resident prevalence for various scenarios, such as different durations of visits, growth rates of infection, or times to recovery.

Methods

Data

From 195 worldwide locations (reflecting mainly countries, without taking any position on territorial claims), we included 194, excluding the epicentre of mainland China. Data for imported cases aggregated by location were obtained from the WHO technical report (dated Feb 4, 2020); a case count of 0 was assumed for all locations not listed. We used case counts up to Feb 4, 2020, because after this date the number of imported cases dropped rapidly, probably because of the lockdown of Hubei province implemented from Jan 23, 2020.

We defined imported cases as people with known travel history from China; 82% (124 of 152) had travel history from Hubei province and 18% (28 of 152) from unknown locations in China. Estimates of daily air travel volume were obtained from Lai and colleagues and are based on historical (February, 2018) data from the International Air Travel Association, including estimates for 27 locations that are most connected to Wuhan. These data capture the daily average number of passengers traveling via direct and indirect flight itineraries from Wuhan to destinations outside of China. For 167 locations not listed by Lai, we set the air travel volume to three passengers per day, which is half the minimum reported by Lai. Because the relative (rather than absolute) flight connectivity of Wuhan with different locations matters for our model, we assumed that this relative connectivity was only weakly affected by early timing of the Lunar New Year in 2020.

Surveillance capacity was assessed using the Global Health Security Index, which is an assessment of health security across 195 locations agreeing to the International Health Regulations (IHR 2005). Specifically, we used the second category of the index, Early Detection and Reporting Epidemics of Potential International Concern published in 2019, henceforth referred to as simply the GHS2 index. We classified locations with GHS2 index above the 80th percentile as high surveillance locations, those with GHS2 index below the 20th percentile as low surveillance locations, and all others as locations with medium surveillance capacity. In view of Singapore's high rate of COVID-19 case-detection per expected case, we treated this location as a special case for surveillance of COVID-19 and we assigned it its own category of most strong surveillance.

We did not need to obtain ethics approval for this study because we did not enrol individuals and we used data available publicly.

Estimating detection probability relative to Singapore

We considered detection of 18 cases by Feb 4, 2020, in Singapore to reflect the highest surveillance capacity among all locations, and we estimated the probability of detection in other locations relative to Singapore according to the following model. We modelled case-detection across i= 1, . . ., n worldwide locations, with 194 locations (n); Singapore was indexed i= 1, with the rest of the locations following in order of decreasing GHS2 index. Using the notation of McElreath, we assumed that the observed case count across n locations followed a Poisson distribution and that the expected case count was linearly proportional to daily air travel volume and a random variable θlevel[i] reflecting the ith location's capacity of detecting cases relative to Singapore.

In this equation,

Y i denotes the reported case count in the ith location, λi is the expected case count in the ith location, β denotes the regression coefficient, x i is the daily air travel volume of the ith location, and θlevel[i] denotes the ith location's capacity of detecting cases relative to Singapore.

For θhigh, θmedium, and θlow, we assigned a uniform prior over [0,1] and for log(β) we assigned a weakly informative normal prior with mean 0 and SD 50. We considered the global average detection probability θglobal to be a transformation of θlevel[i]. In practice, having fitted the model, we took the weighted mean of posterior draws of θlevel[i] for i = 2, . . ., n, where weights are proportional to daily air travel volume, x i. Exclusion of Singapore (i = 1) enabled estimation of the global detection probability relative to Singapore. Conversely, 1/θglobal was the multiplier of the case count that could have been detected globally under a capacity equivalent to that of Singapore.

We calculated the mean and 95% highest posterior density interval (HPDI; the narrowest interval containing a given probability mass) of the numerical approximation of the posterior distribution of θglobal, and the mean and 95% HPDI of the numerical approximation of the posterior distribution of 1/θglobal. Note that the estimate of 1/θglobal and its 95% HPDI is not simply the inverse of the estimate for θglobal and its bounds, because the inverse of a mean is not equal to the mean of the inverse, and similarly for the HPDIs.

We fitted our model using Stan version 2.19.1, and we drew 80 000 samples from the joint posterior distribution of θhigh, θmedium, θlow and β using four independent chains (20 000 samples each), and discarded for each chain the first 500 samples (burn-in). Diagnostic plots of the Markov Chain Monte Carlo sampler for each of the inferred random variables (θhigh, θmedium, θlow and β) are shown in the appendix (p 2). All analyses are fully reproducible with the code available online.

Testing the effect of length of stay in point prevalence of travellers

In 2009, during the influenza pandemic which originated in Mexico, most travellers leaving Mexico were assumed to be tourists, or other temporary visitors, with relatively short stays in Mexico, and the risk that they were infected was assumed to represent a cumulative hazard over the period of their stay.2, 3 The basic assumption was that short-term visitors faced the same hazard of infection as did residents of Mexico, but, in view of their shorter stay, visitors had a somewhat lower prevalence of infection when returning to their home location. Many estimates in 2019–20 for COVID-19 have, instead, made the assumption of equal prevalence in travellers leaving Wuhan and in residents, which is equivalent to assuming that either all travellers are Wuhan residents or (if travellers are a mix of residents and visitors) all visitors had stayed long enough during the epidemic that their prevalence was similar to that of residents.

To quantify the difference between these two scenarios, assuming that all travellers are short-term visitors versus assuming that all travellers are residents or long-term visitors, we considered a simple model of an exponentially growing epidemic, in which the hazard of infection at time t is denoted λ(t) and is increasing at rate r. At the beginning of the epidemic, which we call time 0, the hazard of infection is λ(0) and thereafter λ(t) = λ(0)e rt. Then the point prevalence of infection at time u in residents who have stayed in Wuhan for the duration of the epidemic will be the probability that they have become infected and not recovered by time u, assuming that the cumulative hazard remains small enough by that point that there has been no appreciable depletion of susceptible people.

This equation assumes exponentially distributed time to recovery, with mean duration y −1, when we assume that the infection is only detectable until the time of recovery (for other cases, time to recovery should be duration of detectable infection). The same quantity for a visitor who had only been in Wuhan for d days before departure would be as follows:

In this equation, we assume that visitors differ from residents only in the duration of exposure, not in the intensity of exposure. Under these assumptions, the ratio of prevalence in visitors to that in residents, which we call V, would be as follows:

Once u and, thus, the number of cases in the exponential phase of the epidemic is substantial, this term can be well approximated as V ≈ 1–e −(r + y)d. We plot this approximation of V since doubling times aligned with a range of times to recovery and a range of lengths of stay (appendix p 3). The expression for V can also be expressed in terms of the transmission rate by replacing r + γ by β denoting the transmission rate. This step yields the expression V ≈ 1 – e −βd. We also include an analysis of V under the relaxed assumption that the transmission rate for residents and visitors might be different (appendix p 1).

Role of the funding source

The funders had no role in study design, data collection, data analysis, data interpretation, or writing of the report. All authors had full access to all data in the study and had final responsibility for the decision to submit for publication.

Results

Global ability to detect imported cases of COVID-19, weighted by flight volume from Wuhan, was estimated at 38% (95% HPDI 22–64) of Singapore's capacity (figure 1 ). Equivalently, an estimated 2·8 (95% HPDI 1·5–4·4) times the current number of imported and detected cases could have been detected if all locations had had the same detection capacity as Singapore, leading to 1·8 (0·5–3·4) undetected cases per detected case. Globally, detection capacity varied widely: the ability to detect imported cases among, according to GHS2 index, locations with a high surveillance capacity was 40% (95% HPDI 22–67), among locations with medium surveillance capacity it was 37% (18–68), and among locations with low surveillance capacity it was 11% (0–42; figure 1).

Figure 1

Posterior distributions of detection probabilities relative to Singapore

Upper panel is a density plot of θglobal. Lower panel shows posterior distributions of θlow, θmedium, θhigh. Solid vertical lines show median estimates. Shaded areas show 80% HPDI. Curved lines show the 95% HPDI. HPDI=highest posterior density interval.

The prevalence ratio between Wuhan's temporary visitors and residents approached 1·00 as the epidemic growth rate, the duration of stay, and the recovery rate increased, and it approached 0·00 for short duration of stay, long time to recovery, and slower epidemic growth (figure 2 ). For example, for a visiting duration of 1 week (7 days), an epidemic doubling time of 5 days, and a time to recovery of 11 days, the prevalence in visitors is predicted to be 0·80 (ie, 80% of that in residents). Instead, for a visiting duration of 3 days, the prevalence in visitors would be 0·50 (50% of that in residents).

Figure 2

Ratio of infection prevalence in temporary visitors relative to that in residents

Plot shows the ratio over a range of durations of visit (in days) and a range of durations of detectable infection (time to recovery in days). Upper panel shows an epidemic doubling time of 5 days; lower panel shows an epidemic doubling time of 7 days. Ratios are shown as decimals rounded to 2 decimal places, with lighter areas as the ratio approaches 1·00. In this base case, we assume that the hazard of infection is the same for residents and visitors.

Discussion

In our study, we tested two assumptions underlying the estimation of incidence at the epicentre of the COVID-19 outbreak. The first of these was that the capacity for detection of international imported cases is 100% sensitive and specific across locations. Although we know of no reason to doubt specificity of detection, we tested the assumption of 100% sensitivity. Based on findings of our previous study, we assumed Singapore has the highest capacity for surveillance with respect to COVID-19. We regressed cumulative cases against Wuhan-to-location air travel volume, considering Singapore to have the greatest detection capacity, and estimating relative underdetection compared with Singapore in remaining locations (classified according to the GHS2 index). Although it is unlikely that the GHS2 index reflects the true ranking of locations for any specific epidemic, we assume that it can capture roughly different levels of surveillance capacity. We, therefore, grouped locations (apart from Singapore) into three classes of surveillance capacity (high, medium, and low) instead of using exact ranking. Although possibly important for detection, our model does not account for differences in typical flight duration to different destinations.

We estimated that detection of exported cases from Wuhan worldwide is 38% (95% HPDI 22–64) as sensitive as it has been in Singapore. Put another way, this estimate implies that the true number of cases in travellers is at least 2·8 (95% HPDI 1·5–4·4) times the number that has been detected. Equivalently, for each detected exported case there are at least 1·8 (95% HPDI 0·5–3·4) undetected cases. If the model is correct, this is a lower bound on the frequency of undetected cases for two reasons. First, Singapore's detection capacity is probably not 100% efficient. Singapore had, as of Feb 12, 2020, eight documented cases of COVID-19 transmission, for which there were no known epidemiological links to China or other known cases, implying that imported cases in Singapore could have gone undetected (although it is not certain that these imports came from Wuhan or elsewhere in China, and links might still be found). Second, Singapore's detection capacity, similar to that in other locations, has relied largely on symptoms and travel history, so the number of asymptomatic or low-severity cases missed by such a strategy is unknown.

The second assumption we tested is that the true prevalence in visitors to the epicentre is similar to that of residents: it might be different for either of two reasons. First, the true prevalence could be less if people who visit are less well integrated into the social mixing that produces infection (eg, if they have stayed in specific parts of the city or in hotels), or it could be more if travellers are engaging more intensely in relevant social mixing (eg, through welcoming ceremonies). This aspect could, therefore, increase or decrease prevalence in visitors relative to residents (appendix p 3). In our study, we focused on quantifying a second difference, which is that some visitors will have been in the city only for a short time and, thus, have had less exposure to the infection than residents. We find this effect is most pronounced when the epidemic is growing slowly, when visitors have stayed only briefly, and when the duration of detectable infection is short. We found that for plausible parameters for COVID-19, prevalence in visitors staying only 3 days could be as little as half that of residents, but for longer stays of more than a week the visitor prevalence should be 80% or more that of residents (figure 2). Assuming the population of travellers out of Wuhan is a mix of visitors of various durations and residents, this finding suggests that underestimation of source population prevalence because of the presence of short-stay visitors could be appreciable but more modest than the effect of imperfect detection. For example, in combination with our estimates of underdetection, the total underestimation of cases in Wuhan from only studying visitors could be around 70% for 7-day visits, 5-day doubling time, and 11 days of detectable infection or, with a 3-day visit, as much as 81%. The infection exposure of visitors in high-risk venues (eg, aeroplanes or airport toilets) could be important, and additional variation in exposure between travellers of different international destinations due to, for example, different dominant types of reasons for travel (eg, group holidays vs business trips).

Based on our model, the risk of undetected importation and subsequent circulation correlates with air travel connectivity. Indeed, at the time of writing (March 9, 2020), nine locations have reported more than 500 confirmed cases of COVID-19 (France, Germany, Iran, Italy, Japan, Singapore, South Korea, Spain, and the USA), suggesting local transmission. All locations (apart from Iran) are among the 27 most connected locations to Wuhan through air travel, which supports the important role of undetected importation through air travellers (the probability of drawing uniformly at random eight or more highly connected locations to Wuhan under the null assumption that draws are independent of flight connection is <0·001). However, as new COVID-19 epicentres evolve, the role of air travel from China in the transmission of SARS-CoV-2 is expected to decline. Finally, our model predicts that locations with high connectivity to Wuhan paired with a relatively low surveillance capacity (eg, India, Maldives, New Zealand, Pakistan, Russia, Sri Lanka, and United Arab Emirates) are probably underdetecting imported cases and potentially also self-sustained transmission.

Our finding that imported cases detected among travellers probably under-represents the source population prevalence has two important implications for the public health response to COVID-19. First, this finding has implications for approaches to case burden and severity estimation that use cases in travellers to impute cases in Wuhan, which are then compared (for severity estimation) against deaths in Wuhan. If the true number of imported cases is underestimated, then there are more cases in Wuhan and a larger denominator, resulting in reduced estimates of severity compared with severity estimates assuming 100% detection in travellers. Future studies should account for our evolving understanding of detection capacity when estimating case numbers and severity in source population on the basis of traveller case numbers. Second, our model predicts the scenario in which the virus has been imported from Wuhan and remained partly undetected in nearly all locations. An important corollary of this possibility is that, despite large efforts to detect and stop the virus from entering new locations, many undetected imported cases can occur and cause hidden local transmission until a sizable number of cases accumulates, leading to international spread of COVID-19 beyond locations' detection and reporting capacities.