Trend in Sensitivity of Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Serology One Year After Mild and Asymptomatic Coronavirus Disease 2019 (COVID-19): Unpacking Potential Bias in Seroprevalence Studies.

A key aim of serosurveillance during the coronavirus disease 2019 (COVID-19) pandemic has been to estimate the prevalence of prior infection, by correcting crude seroprevalence against estimated test performance for polymerase chain reaction (PCR)-confirmed COVID-19. We show that poor generalizability of sensitivity estimates to some target populations may lead to substantial underestimation of case numbers.

Background

During the current severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic, public health agencies have used serology to investigate the clinical spectrum, distribution, and determinants of coronavirus disease 2019 (COVID-19) across time and place to inform a range of interventions [1]. Serology is the preferred method for determining past infection with SARS-CoV-2 because pathogen-specific antibodies are detectable in serum long after clearance of viral RNA or antigen from accessible sites [1]. The prevalence of prior infection (PPI) has been estimated from cross-sectional sero-surveys, corrected for test sensitivity and specificity against polymerase chain reaction (PCR)-confirmed infection [2]. However, there are 2 obvious limitations of this approach.

First, serological assay sensitivity estimates have primarily been derived using samples obtained from symptomatic—usually hospitalized—patients [3-6]. Yet a large proportion of cases have only mild illness, and more than one-third remain asymptomatic [7]. Given the strong correlation between COVID-19 severity and magnitude of antibody response [8, 9], sensitivity estimates derived from moderately and severely ill patients may not represent the broader SARS-CoV-2-infected population.

Second, most sensitivity estimates are based on samples obtained in early convalescence [3-5], yet serum antibodies have been shown to decline substantially within a few months of infection [10]. The kinetics of antibody decay may differ between mild and severely ill patients. Nevertheless, antibody decay is not considered in most published seroprevalence estimates.

In light of these limitations, we hypothesized that estimates from seroprevalence studies would underestimate PPI because of differences between test sensitivity in target populations and sensitivity estimates used for correction. To assess the possible magnitude of this bias, we investigated the longitudinal trend in results of one commercial serological assay in a cohort of individuals with mild and asymptomatic COVID-19 over 1 year, and modelled changes in sensitivity.

Estimation of Test Sensitivity Over Time

The cohort consisted of 48 older adults (median age 67 years, range 36–81), recruited from a previously described group exposed to SARS-CoV-2 on an Antarctic cruise [11], with SARS-CoV-2 infection confirmed by PCR and/or serology (Supplementary Table 2). Notably, 21/48 (44%) remained asymptomatic during 14 days of active monitoring. We collected 207 serum samples between 16 April 2020 and 14 April 2021; after excluding 3 samples collected after COVID-19 vaccination, 204 samples were available for analysis (median per participant: 4.5; range: 1–5). All participants provided informed consent. The study was approved by the Human Research Ethics Committee of the Department of Health and Human Services, Victoria (HREC 05-20).

Longitudinal analysis of antibody data requires estimating time since infection or disease onset, a challenging prospect for asymptomatic individuals. This cohort were almost certainly all exposed to SARS-CoV-2 over a 4-week period, between boarding the ship on 15 March 2020 and entering a managed quarantine facility in Australia on 12 April 2020 [11]. All symptomatic participants became ill between 20 March and 3 April 2020, with the median date of onset 24 March (Figure 1A). For symptomatic participants we considered date of disease onset to be the same as symptom onset. For asymptomatic participants we set the date of disease onset to the median date of symptom onset, an assumption which is necessarily imprecise at the individual level but which we expected, on average, to hold true for the cohort.

Figure 1.

Estimation of the impact of antibody decay on assay sensitivity. A, Distribution of symptom onset in 27 symptomatic participants, vertical dashed line shows the median date of onset used to infer disease onset in the remaining 21 asymptomatic participants. B, OD ratios obtained using the EUROIMMUN Anti-SARS-CoV-2 IgG kit on samples from all 48 SARS-CoV-2 infected participants. C, OD ratios for a simulated population of 1000 infected individuals. Thick line shows the median, with the shaded region showing the corresponding 95% confidence bands. D, Test sensitivity of the EUROIMMUN kit in the simulated population (line) with 95% confidence band (shaded region). Horizontal dashed lines in panels B and C show the test positivity threshold (1.1). Vertical dashed lines in panels C and D are drawn at 1 year, the approximate time from which ongoing estimates are based on assumed ongoing exponential decay. Abbreviations: IgG, immunoglobulin G; OD, optical density; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

We tested samples using the EUROIMMUN (EI) Anti-SARS-CoV-2 enzyme-linked immunosorbent assay (ELISA) kit for the detection of immunoglobulin G (IgG), as per the manufacturer’s instructions. The EI kit uses recombinant S1 domain of SARS-CoV-2 spike as antigen [3]. Results are expressed as the ratio of the measured optical density (OD) for the sample to that of a supplied calibrator, with ratios ≥1.1 considered positive. Published sensitivity estimates for samples obtained >14 days after symptom onset range from about 70–100%, and specificity 97–100% [3-6].

We modeled OD ratios obtained using the EI IgG kit in a hierarchical mixed-effects framework, using a nonlinear model proposed by Simonsen et al [12]. The model accommodates an initially rapid but gradually diminishing increase in antibody from disease onset, followed by exponential decay toward a steady state level [12]. To simulate the longitudinal trend in test sensitivity, we used parameter estimates from the fitted model to construct a hypothetical population of 1000 infected individuals. We simulated OD ratios for these individuals over a 3-year period from disease onset, and calculated sensitivity as the proportion of individuals with an OD ratio ≥1.1. We constructed 95% confidence intervals (CI) by repeating the model fitting and simulation procedures on 1000 bootstrap resamples of the original data set. Additionally, we investigated trends in sensitivity under simulated epidemic scenarios. Details of this analysis, as well as model specification, fit, and simulation methods, are provided as Supplementary Material.

Figure 1B shows the observed OD ratios in our sample. Our simulation procedure and data were compatible with a maximum sensitivity of 81% (95% CI: 74–84%) 40 days post-disease onset in the simulated population (Figure 1C–1D). Simulated sensitivity declined to 76% at 3 months (95% CI: 69–79%) and 53% at 1 year (95% CI: 47–57%). Based on extrapolation of exponential decay, our simulation was consistent with a decline in sensitivity to 37% at 3 years (95% CI: 27–42%).

Example Application to Serosurveillance

When the proportion of previously infected individuals in the target population is high, correction based on biased sensitivity estimates may have a non-negligible effect on estimated PPI and, by extension, decisions affecting public health. We used Murhekar et al [13] to demonstrate this problem. They conducted a serosurvey of 28 598 individuals between 18 December 2020 and 6 January 2021, to estimate the proportion of the Indian population previously infected with SARS-CoV-2. Their population-weighted, but unadjusted, seroprevalence was 21.7% using the Siemens S1-RBD IgG assay, with a PPI of 21.5% after correcting for the manufacturer-reported sensitivity of 100%, and specificity of 99.9% [13]. Based on this adjusted seroprevalence, they estimated the true number of infected individuals in the Indian population to be 242 124 000, indicating that there had been 23.8 infected individuals for each reported case as of 19 December 2020 [13].

In order to roughly estimate the possible degree of bias in these findings, we considered the following simplistic assumptions: (1) All infected individuals had onset of disease on 15 September 2020, the date of peak reported cases in India’s first wave [14]. (2) All individuals in the study were sampled on 24 December, 100 days later. (3) The temporal profiles of the true sensitivities of the EI and Siemens assays were equivalent. (4) The reported specificity for the Siemens assay was generalizable to the target population.

Our simulation was compatible with a true sensitivity of 74% 100 days post-disease onset. Correcting the crude seroprevalence for this value and the reported specificity of the Siemens assay gave a PPI of 29.0% (equation 3 in the Supplementary Material), equivalent to 326 426 000 infected individuals, or an infection-case ratio of 32.1, 1.35 times higher than the initial estimate. Interestingly, Murhekar et al noted that seropositivity among a subgroup of 664 participants who reported testing positive for SARS-CoV-2 by PCR was only 64%, a finding they suggested might be due to antibody decay [13].

Limitations

Our study had several limitations. The age and sex structure of our cohort differed substantially from most serosurveillance target populations. We have previously discussed the possibility of selection bias due to differential participation based on symptoms [11]. However, given population estimates of a relatively low infection: hospitalization ratio [15], and high asymptomatic proportion [13], we believe our sample to be more applicable to seroprevalence studies than most others used to derive estimates of serological assay sensitivity. Longitudinal trends in sensitivity may vary depending on the assay and target antigen. Our model has not been validated and relies on unverified assumptions including multivariate log-normality in parameter distribution, and continuing exponential decay past one year. Consequently, we intended to illustrate the potential for biased incidence estimates arising from serosurveillance studies, rather than to provide a precise quantification of this bias or a specific correction method.

Conclusion: Improving the Utility of Serosurveillance for COVID-19

In attempting to infer population prevalence of prior COVID-19 from seroprevalence studies, careful consideration should be given to bias due to non-generalizability of assay sensitivity estimates to target populations, leading to substantial underestimation. The effect of this bias increases with time from infection to sampling.

At a minimum, seroprevalence studies seeking to estimate SARS-CoV-2 infection should include sensitivity analyses allowing for much lower test sensitivity than those reported by kit manufacturers. The utility of seroprevalence surveys to infer patterns of COVID-19 might be further improved by the application of methods to recover incidence by applying models of test kinetics to cross-sectional data [12, 16]. Such methods currently require either: (1) Single cross-sectional surveys using multiple assays with distinct kinetics, with results reported on continuous rather than binary scales, coupled with longitudinal models of test kinetics derived from population-representative cohorts [16] or (2) multiple cross-sectional surveys coupled with accurate mortality data [17, 18]. Studies seeking to estimate infection rather than population immunity must now distinguish between vaccine- and infection-induced antibody responses. This might be achieved by the development of new multiplex assays, but in the interim may require collection of vaccination status for sensitivity analyses [19].

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Click here for additional data file.