Will all SARS-CoV-2 seroprevalence surveys provide the right picture?

In various countries experiencing the coronavirus disease 2019 (COVID-19) epidemic, surveys of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) seroprevalence are conducted to assess the actual spread of the SARS-CoV-2 infection in defined populations since the beginning of the epidemic. Real-time reverse-transcriptase polymerase chain reaction (RT-PCR) test for SARS-CoV-2 in upper respiratory specimens collected through swabs is recommended by the World Health Organization for clinical management and outbreak control purposes and is currently the gold standard for the etiological diagnosis of SARS-CoV-2 infection. However, incidence and prevalence data based on those tests are not representative of the general population experience for a number of reasons. First, swabs are only collected from either symptomatic persons or from people with increased risk of infection (e.g. close contacts of COVID-19 cases, healthcare professionals, and so on). In addition, the intensity or capacity of swab collection, which may be highly variable from one geographical area to another, affect the likelihood of detecting people who are infected at a certain time. Finally, RT-PCR can detect viral RNA in a respiratory specimen at the time of swab collection, thus results of the test only apply to that specific moment.

In Italy, 60 million inhabitants, the Ministry of Health and the National Statistical Institute ISTAT, in collaboration with the Italian Red Cross, planned a nationwide seroprevalence survey: a sample of 150,000 people was to be tested in 2000 towns and cities, split by sex, occupation, and six age classes, from May 25, 2020 and for approximately 10 days. The sample was planned to be representative of each of the 20 Italian Regions. For example, in the 1,200,000-inhabitant Region Friuli Venezia Giulia (FVG), 6232 subjects had to be surveyed in approximately ten days. The survey was established through Legislative Decree 30 of May 10, 2020 and funded with public money for more than 4 million Euro. The theoretical benefits of this survey are extremely important from a public health point of view and results might influence policy makers’ decisions in the upcoming months. There are two main issues, however, that should be considered when interpreting results: serological test performance and survey participation bias.

Serological test performance

In the absence of published research on the validity of serological tests, we retrospectively analyzed data on tests conducted in the 530,000-inhabitant province of Udine, constituting approximately half FVG Region, at the University Hospital of Udine. The Virology Laboratory of our Hospital conducted more than 90,000 RT-PCR tests on upper respiratory specimens collected through swabs in either symptomatic persons or asymptomatic close contacts of cases or subjects at high risk of infection (more than 40,000 persons from March 1, 2020). For different reasons (clinical, screening, and so on), some of those persons also underwent serological testing, either at the same Hospital or at other public Hospitals of the Region. We observed that in 274 persons with at least one positive RT-PCR test for SARS-CoV-2 and at least one subsequent serological test as of June 10, 93.8% (95% confidence interval: 90.4–96.2%) tested positive for Immunoglobulin G (IgG) and 46.5% (40.6–52.4%) for Immunoglobulin M (IgM). In 153 persons with at least two negative RT-PCR tests (to minimize the risk of false negative RT-PCR tests) and no positive RT-PCR tests, 89.5% (83.9–93.7%) tested negative for IgG and 94.0 (89.3–97.0%) for IgM. Table 1 shows time intervals from first swab collection to blood withdrawal for discordant cases. Keeping in mind timing and the fact that RT-PCR on specimens collected through swabs cannot be considered a real gold standard because it may also be fallacious, our data suggest that evidence provided by serological tests seems to be less than perfect because false negative and false positive results likely exist. Nonetheless, at least for population-level epidemiological purposes, serological test performance may be acceptable, in the absence of other reliable sources of information.

Table 1

Days from the first swab collection to blood withdrawal for the serological test in persons with discordant RT-PCR test and serological test for SARS-CoV-2.

Discordant results pattern	Mean	Standard deviation	25th percentile	Median	75th percentile	Minimum	Maximum
RT-PCR + IgG- (N = 17)	18.3	16.1	2	15	31	1	42
RT-PCR + IgM- (N = 145)	26.8	13.0	17	29	36	1	59
RT-PCR- IgG+ (N = 16)	27.2	17.6	14	24.5	42	3	63
RT-PCR- IgM+ (N = 9)	18.0	6.3	14	15	23	9	29

RT-PCR, reverse-transcriptase polymerase chain reaction; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Participation bias

After the start of the nationwide Italian survey, in FVG, 1835 blood samples were collected from May 27 to June 10, less than 30% of the planned sample size. Potential participants had to be invited by the Italian Red Cross through a telephone call from a number, not well advertised before the beginning of the survey, starting as 065510, which could have been misinterpreted as a spam phone call, reducing responses. In addition, attending ambulatories for blood withdrawal might be considered unsafe by part of the population. Given the scarce adherence, the Red Cross prosecuted the telephone calls to potential participants beyond the originally planned 10 days and further 446 subjects were enrolled as of June 26. Thus, 2281 subjects were enrolled overall, 36.6% of the planned sample. In FVG, participation was low, reducing precision of the estimates in the Region. In addition, selection bias was likely, affecting validity. In fact, participation differed by geographical area (the relative distribution in three regional subareas was, respectively, 40.4%, 25.8%, and 33.7% for the potential survey sample vs 45.5%, 28.7%, and 25.7% for actual participants). In addition, because subjects with positive serological tests are temporarily isolated (as reported in the survey information sheet), those who had just restarted working after several weeks of lockdown might have not consented to participate, to avoid the risk of a new period of inactivity and lost income. Finally, in FVG, the population was additionally requested to participate, when presenting for the blood withdrawal, in an independent social survey promoted by the FVG Regional Administration, aiming at linking serological results with personal behaviors, as advertised in the local media. The burden of an additional survey and the possibility that one's personal data and sensitive information could be communicated to others might have further discouraged participation, in a non-random fashion.

Thus, in the FVG Region, despite test accuracy may still be acceptable to assess the spread of SARS-CoV-2 infection in the population, participation bias may distort estimates. Even in an emergency situation as the one caused by COVID-19, good communication and careful planning are crucial for effective interventions. When interpreting results of seroprevalence surveys, particular attention should be devoted to the assessment of potential sources of bias.