Using Prenatal Blood Samples to Evaluate COVID-19 Rapid Serologic Tests Specificity.

INTRODUCTION: Background cross-reactivity with other coronaviruses may reduce the specificity of COVID-19 rapid serologic tests. The vast majority of women attend prenatal care, which is a unique source of population-based blood samples appropriate for validation studies. We used stored 2018 serum samples from an existing pregnancy cohort study to evaluate the specificity of COVID-19 serologic rapid diagnostic tests.
METHODS: We randomly selected 120 stored serum samples from pregnant women enrolled in a cohort in 2018 in Tegucigalpa, Honduras, at least 1 year before the COVID-19 pandemic. We used stored serum to evaluate four lateral flow rapid diagnostic tests, following manufacturers' instructions. Pictures were taken for all tests and read by two blinded trained evaluators.
RESULTS: We evaluated 120, 80, 90, and 90 samples, respectively. Specificity for both IgM and IgG was 100% for the first two tests (95% confidence intervals [CI] 97.0-100 and 95.5-100, respectively). The third test had a specificity of 98.9% (95% CI 94.0-100) for IgM and 94.4% (95% CI 87.5-98.2) for IgG. The fourth test had a specificity of 88.9% (95% CI 80.5-94.5) for IgM and 100% (95% CI 96.0-100) for IgG. DISCUSSION: COVID-19 serologic rapid tests are of variable specificity. Blood specimens from sentinel prenatal clinics provide an opportunity to validate serologic tests with population-based samples.

Significance

What is already known on this subject? The validity of COVID-19 serologic rapid tests is not established, although they are becoming widely available. Sentinel prenatal clinics provide a unique opportunity to collect population-based blood samples for validation studies.

What this study adds? We used stored serum samples collected in a sentinel prenatal clinic before the COVID-19 pandemic in Tegucigalpa, Honduras. We evaluated four rapid serologic tests and found specificities varying from 88.9 to 100%. The high number of false positives for some tests would make them less suitable for field studies.

Introduction

There is an urgent need to obtain serologic data for SARS-CoV-2, the virus causing COVID-19, for populations worldwide. Rapid diagnostic tests (RDTs) could play a key role in serologic surveys, but their validity is often not well documented for COVID-19 tests (Sethuraman et al. 2020). Blood routinely collected during prenatal care is a unique source of population-based samples which could be used to perform COVID-19 serologic studies (Buekens et al. 2020). UNICEF estimates that 87.4% of pregnant women in the world had at least one prenatal visit in 2014–2019 (UNICEF 2019). Data from sentinel prenatal clinics are used to generate population-based estimates of infectious diseases seroprevalence. For example, HIV/AIDS seroprevalence is often estimated from prenatal care data (Eaton et al. 2014). Additionally, prenatal care offers a unique opportunity to collect data from a population of both asymptomatic and symptomatic women. Most women receive prenatal care to some extent, providing a better estimate of population seroprevalence than samples based on blood banks, hospital biobanks, or volunteers.

The specificity of COVID-19 serologic tests has been questioned because of possible background cross-reactivity with other coronaviruses. This has been an issue for serological testing of SARS-CoV and MERS (Meyer et al. 2014). Specificity corresponds to the proportion of true negative among healthy subjects. We used stored serum samples collected before the pandemic from the Zika in Pregnancy in Honduras (ZIPH) cohort study to evaluate the specificity of COVID-19 serologic RDTs available for field studies in Honduras.

The first COVID-19 case in Honduras was reported on March 10, 2020 (Gobierno de Honduras 2020). Nine weeks later, the country has reported 2646 COVID-19 cases and a low testing rate of 572 tests per million inhabitants (Worldometer 2020). Soon after the start of the COVID-19 pandemic in Honduras, the government independent group named Plataforma Todos Contra el COVID-19 (all against COVID-19 platform) was formed. One of the working aims of this group has been to guide and scale up the laboratory testing for confirmation of individual cases and for epidemiologic surveillance studies (Plataforma Todos Contra el COVID-19 2020).

Methods

The current study leveraged stored serum samples from an existing prospective pregnancy cohort. The ZIPH study enrolls women at their first prenatal visit at the Alonso Suazo Health Center (Tegucigalpa, Honduras) and follows them up until delivery (Buekens et al. 2016). The health center is a large urban prenatal clinic with more than 1,100 new prenatal visits a year. We enrolled 3991 women from July 2016 to March 2020. Data management is centralized at the Institute for Clinical Effectiveness and Health Policy (Instituto de Efectividad Clínica y Sanitaria, IECS), Buenos Aires, Argentina.

The Data Center selected at random 120 women among 977 women who were enrolled in 2018 and had authorized their blood samples to be stored for 10 years for additional studies. All samples were collected more than 1 year before the first case of COVID-19 was reported in Honduras. The sample size was based on the number of RDTs available for evaluation. The Data Center prepared a list of study numbers (with control digits), which was emailed to Honduras where study labels were printed and pasted on data forms and on rapid tests immediately before use (Fig. 1). Data were collected on paper forms and entered in REDCap (Harris et al. 2009). Additionally, scans of each data form were taken and were sent (encrypted) to IECS. This system allows for a digital backup of all study data forms.

Fig. 1

a–c Digital picture of COVID-19 rapid serological test (RDT#1) with negative result (a) and RDT#3 with IgG (b) and RDT#4 with IgM (c) positive results

Four lateral flow RDT were obtained by the Plataforma Todos Contra el COVID-19 through the Honduran Sanitary Regulatory Agency (Agencia Regulatoria Sanitaria, ARSA,) or directly as a donation. We used the following lateral flow RDTs:

RDT#1: Hightop COVID-19 IgM/IgG Ab Rapid Test Kit (Qingdao Hantang Biological Technology Co., Ltd., Qingdao, China);

RDT#2: COVID-19 IgG/IgM Rapid Test Kit (Nantong Egens Biotechnology Co. Ltd., Nantong, China);

RDT#3: Orient Gene COVID-19 IgG/IgM Rapid Test (Zhejiang Orient Gene Biotech Co. Ltd, Huzhou, China).

RDT#4: Standard Q COVID-19 IgM/IgG Duo Test (SD Biosensor, Gyeonggi-do, Republic of Korea).

Laboratory procedures were performed at the Departamento de Laboratorio Clínico, Hospital Escuela, Tegucigalpa, Honduras. One aliquot of frozen serum was thawed and kept at 4 ℃ for up to 3 days. Samples were brought to room temperature before testing. RDT packages were opened immediately before use. RDTs were performed following the manufacturers’ instructions. Rapid tests were read at 10 to 20 min according to the respective RDT instructions, and digital pictures were immediately taken using a cell phone camera. Pictures were taken under constant lighting conditions using two light sources. We used a 3 × magnification to allow the picture to be taken from enough distance to avoid shadows. The pictures were uploaded to an encrypted cloud site and were immediately available to the Data Center. The pictures were then uploaded to a secure website and were read by two blinded trained evaluators and compared to the initial reading in the laboratory. Discrepancies between the initial reading and readings from the blinded evaluators, or discrepancies between blinded evaluators, were resolved by an independent third blinded trained evaluator. This is similar to the approach we used earlier for Chagas RDTs (Buekens et al. 2013, 2018). We used OpenEpi version 3.01 to calculate 95% Fisher exact confidence intervals (CI).

This study was approved by Tulane University Institutional Review Board (893652; Amendment approval April 22, 2020), and by the Facultad de Ciencias Médicas, Universidad Nacional Autónoma de Honduras ethics committee (CEIB-079-2016; Amendment approval April 24, 2020). All women provided written informed consent at enrollment and authorized their blood samples to be stored for up to 10 years.

Results

Table 1 shows the characteristics of the selected women enrolled in the ZIPH cohort.

Table 1

Maternal characteristics

	N = 120
	n/N	%
Maternal age
Between 15 and 19	31/120	25.8
Between 20 and 34	80/120	66.7
35 or higher	9/120	7.5
Years of Education*	9	(6–12)
Gestational age at first visit (weeks)
Between 1 and 13	69/93	74.2
Between 14 and 27	19/93	20.4
28 or higher	5/93	5.4
Symptoms during pregnancy
Eruption	0/116	0.0
Fever	0/116	0.0
Arthralgia	0/116	0.0
Arthritis	0/116	0.0
Conjunctivitis	0/116	0.0
At least one symptom	0/116	0.0

*Median (interquartile range IQ)

We tested 120 RDTs #1, 80 RDTs #2, and 90 RDTs #3 and #4 each. All tests had a clearly visible control band for all samples, except one for RDT#4. Pictures were taken for all tests (Fig. 1). Both blinded evaluators agreed with the initial reading in 100% (120/120) of RDT#1 s, 96.3% (77/80) of RDTs#2, 95.6% (86/90) of RDTs#3, and 93.3% (84/90) of RDT#4. The third independent blinded evaluator made the final decision in 13 cases.

Table 2 shows that the specificity for both IgM and IgG was 100% for both RDT #1 and RDT #2, with 95% CIs lower limits of 97.0% and 95.5%, respectively. Specificity for RDT #3 was 98.9% for IgM and 94.4% for IgG. Specificity for RDT#4 was 88.9% for IgM and 100% for IgG with a 95% CI lower limit of 96.0%.

Table 2

Serologic rapid diagnostic tests (RDTs): positive (+) tests for IgM and IgG and specificity (% and 95% confidence intervals [CI])

	IgM+	IgM specificity% (95% CI)	IgG+	IgG specificity% (95% CI)
RDT #1	0/120	100 (97.0–100)	0/120	100 (97.0–100)
RDT #2	0/80	100 (95.5–100)	0/80	100 (95.5–100)
RDT #3	1/90	98.9 (94.0–100)	5/90	94.4 (87.5–98.2)
RDT #4	10/90	88.9 (80.5–94.5)	0/89	100 (96.0–100)

Discussion

Our results suggest that the COVID-19 serologic RDTs available in Honduras are of variable specificity when used in a general population. Specificity should have been 100% for all tests in these pre-COVID-19 samples. RDT #1 and #2 was 100% specific for both IgM and IgG, but specificity was as low as 88.9% for IgM for RDT#4 and 94.4% for IgG for RDT #3. None of the RDTs we tested are approved by the United States Food and Drug Administration (FDA), but all of them have a CE Mark for use in the European Union (Weissleder et al. 2020).

The specificities we found are comparable to the ones reported in the recent literature. A study of 10 serologic rapid tests found specificities varying from 91.6 to 100% among 108 blood donor plasma specimens collected in the United States before July 2018 (COVID-19 Testing Project 2020). A review of mostly unpublished data about nine serologic RDTs found specificities of 98.7% to 100%, including RDT #1, for which reported specificities were of 96.0% for IgM and 97.5% for IgG (Zainol Rashid et al. 2020). Hoffman et al. (2020) evaluated RDT #3 and found higher specificities, 100% for IgM and 99.2% for IgG. A meta-analysis of regulatory data from serologic RDTs approved in Brazil found a pooled specificity of 97% for IgM and 98% for IgG (Castro et al. 2020). A study evaluating a RDT in China included 128 “clinical negative samples” and found one sample positive for IgG and 10 positive for IgM (Li et al. 2020). Of note, “clinical negative samples” in Li. et al. (2020) were from unconfirmed suspected COVID-19 cases, which could explain the high frequency of positive IgM. Another study of a COVID-19 serologic RDT found no positive among 26 healthy blood donors, with a specificity of 100% (Shen et al. 2020).

Data from SARS-CoV and MERS suggested that background cross-reactivity with other human coronaviruses was an issue (Meyer et al. 2014). However, this was mostly the case for tests detecting antibodies against the whole virus rather than the ones detecting antibodies against specific recombinant antigens. More than 90% of adults have been reported to have antibodies against the four human coronaviruses (Gorse et al. 2010). Coronavirus infection among children with influenza-like illness occurs in Central America as in other countries of the Americas (Taylor et al. 2017). An unexpected finding from a study from Vietnam showed that human coronavirus infection was less frequent among children with influenza-like illness when housing was overcrowded (Nguyen et al. 2016). The 2012 Honduras Demographic and Health Survey (DHS) shows that in Tegucigalpa, 38% of the households have > 4 people (Secretaría de Salud de Honduras, Instituto Nacional de Estadística e ICF International 2013). It is thus possible that circulation of other human coronaviruses is relatively low in our setting.

Rapid tests used for seroprevalence surveys need to be highly specific. The low specificity for IgG of RDT #3 is a concern, as it would falsely classify more than 5% of those tested as positive. The other three RDTs were highly specific for IgG and would be recommended if further studies would show a high sensitivity. We conclude that among those tested, COVID-19 serologic RDTs specificity varies from a low 88.9 to 100% in unaffected populations from Tegucigalpa, Honduras. If further studies confirm their sensitivity in that setting, highly specific tests would be the instrument of choice for serological surveys. The validity of all RDTs should be carefully evaluated, and blood specimens from sentinel prenatal clinics provide an opportunity to test them with population-based samples.