Validation of rapid antibody (IgG-IgM) test kit for SARS-CoV-2 infection in Qatar.

BACKGROUND: In response to the growing coronavirus disease 2019 (COVID-19) pandemic and the shortage of laboratory based molecular testing capacity and reagents, multiple diagnostic test manufacturers have developed rapid and easy to use devices to facilitate testing outside laboratory settings. These kits are either based on detection of proteins from SARS-CoV-2 virus or detection of antigen or human antibodies generated in response to the infection. However, it is important to understand their performance characteristics and they must be validated in the local population setting. DESIGN AND METHODS: The objective is to assess the validity of the rapid test for IgG and IgM immunoglobulins compared to the current gold standard reverse transcription polymerase chain reaction (RT-PCR) test. A total of 16951 asymptomatic individuals were tested by the Ministry of Public Health track-and-trace team using both rapid immunodiagnostic test and RT-PCR as part of screening across various random settings with potential risk of community interaction prior to gradual lifting of restrictions in Qatar.  Rapid test was considered to be posiive if both IgG and IgM are positive, while only IgG/IgM positive was considered as rapid test negative. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were calculated.
RESULTS: The sensitivity of rapid test kit was found to be 0.9%, whereas the specificity was found to be 97.8%. the PPV was found to be 0.3% whereas the NPV was found to be 99.4%.
CONCLUSIONS: Based on the outcome and results of the study, it appears that the sensitivity and PPV of the rapid antibody test are low. As such, this test is not recommended for use to assist in taking clinic-based decisions or decisions related to quarantine/isolation.

Introduction

Severe acute respiratory syndrome coronavirus -2 (SARSCoV- 2) and the resulting coronavirus disease 2019 (COVID-19) pandemic has presented many diagnostic challenges. Molecular testing of upper or lower respiratory tract samples for virus nucleic acid by reverse transcription polymerase chain reaction (RTPCR), is the golden standard test and the recommended method for detecting SARS-CoV-2.[1] RT-PCR allows earlier detection and isolation of confirmed cases, which in turn helps in reducing household and community transmission.

The RT-PCR has a lot of limitations like requirements for certified laboratories, expensive equipment and skilled technicians and long turnaround time,[2] shortages of laboratory reagents and testing capacity due to the growing pandemic. This has led to the felt need for a faster and more convenient testing method to complement nucleic acid detection. Several diagnostic tests are available to identify current infection, past infection and immune response. These kits are either based on detection of proteins from SARS-CoV-2 virus or detection of antigen or human antibodies generated in response to the infection, in the blood or serum.

Antibody detection tests detect the body’s immune response to the infection caused by the virus rather than detecting the virus itself. The serologic assays to detect antibodies against SARSCoV- 2 are of great interest as high levels of IgM and IgG can be detected from the second week of symptom’s onset. In the early days of an infection when the body’s immune response is still building, antibodies may not be detected. IgM, the first antibody against any new virus infection will only be detectable after 3–6 days, mainly in the blood and lymph fluid; while IgG, the most abundant type of antibody, will be detectable only after 8 days.[3,4] This limits the test’s effectiveness for diagnosing COVID-19 in the early days of infection. If IgM and IgG antibodies for SARSCoV- 2 are detected in the blood sample it is likely that the individual became infected recently. If only IgG is detected, then it is probable that the person had an infection sometime in the past or is in the later stages of infection. Therefore, it is recommended to use combined IgG/IgM detection kits to be applicable for different stages of COVID-19 infection and serology tests should not be used as the sole basis to diagnose COVID-19.

However, these tests can play a role in the fight against COVID-19 by helping healthcare professionals identify individuals who may have developed an immune response to SARS-CoV- 2 and thus aid in determining who may donate a part of their blood called convalescent plasma, which may serve as a possible treatment for those who are seriously ill from COVID-19. These antibody tests are likely to have a useful role for detecting previous SARS-CoV-2 infection if used 15 or more days after the onset of symptoms. However, the duration of antibody rise is currently unknown, and very little data has been found for titres beyond 35 days post-symptom onset. We are therefore uncertain about the value of these tests for seroprevalence surveys to assist in public health guidelines and management plans purpose.[5]

Classical serology immunoassays have a relatively long turn around time, which is not suitable for emergency situations to take swift decisions.[2] Whereas, the qualitative detection of SARS-CoV- 2 IgG/IgM antibodies in human serum, developed based on gold immunochromatography assay (GICA), configured like a home pregnancy test kit, can generate results in less than 15 minutes, without laboratory equipment or skilled personnel or sample transportation. However, it is important to understand their performance characteristics and they must be validated in the local population and settings before these tests can be recommended for use.

The aim of this study was to evaluate diagnostic indexes including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of a particular combined IgM/IgG antibody detection kit, to determine its diagnostic usefulness in our local setting and aid in the decision making, as to when and how to use such kits.

Design and methods

Objective

To evaluate the sensitivity, specificity, PPV, NPV of the rapid IgG-IgM combined antibody test compared to the current gold standard RT-PCR test.

Study setting

Screening conducted across various random settings (like schools, universities, banks, hotels, restaurants and other food outlets, shops and supermarkets, petrol stations, cleaning companies, gyms and other sports academies) in Qatar with potential risk of community interaction as part of different phases of gradual lifting of restrictions in Qatar (Figure 1).

Figure 1.

Distribution across various settings.

Study population

All staff aged 18 years and above, working in the selected settings, who were asymptomatic were tested using both rapid immunodiagnostic test and molecular RT-PCR as part of the screening activities performed by the Ministry of Public Health’ (MOPH) track-and-trace team.

Sample size and sampling technique

All individuals aged above 18 years who were tested by the MOPH track-and-trace team as part of screening from August 1st 2020 to November 30th 2020 were included.

Nasopharyngeal and throat swabs were collected for detection of SARS-CoV-2 RNA particles using RT-PCR at the national laboratory under Hamad Medical Corporation and the samples were declared either positive/negative.

Capillary blood samples were collected to test for SARS-CoV- 2 IgG/IgM antibody using BioMedomics COVID-19 IgM-IgG rapid test kit. This kit is intended to test IgM and IgG separately and takes about 15 minutes. There are three detection lines on this kit. The appearance of either G or M or both lines shows presence of anti-SARS-CoV-2 IgG or anti-SARS-CoV-2 IgM or both antibodies in the sample. If the control (C) line does not appear in any test, the test will be invalid.

If both IgM and IgG for SARS-CoV-2 antibodies were detected, it is likely that the individual became infected with SARS-CoV- 2 recently. If only the IgM was positive, then it is likely that the individual became infected with SARS-CoV-2 very recently. If only the IgG was positive, then it is probable that the person had an infection sometime in the past or is in the later stage of infection. If both IgG and IgM were found to be negative, then it was considered as negative for infection.

Data management and analysis

Retrospective review of the database maintained by the MOPH track-and-trace team was done. De-identified details of eligible individuals who were tested using both RT-PCR and rapid tests for antibody detection by the team as part of screening from August 1st 2020 to November 30th 2020 was extracted from the database. The age, gender, nationality, setting where they were tested, rapid immunodiagnostic test results and RT-PCR results were extracted without any personal identifiers. IgM positive test results with or without IgG positivity was considered as rapid test positive for current infection and if only IgG was positive then it was considered as rapid test negative for current infection but may have had a previous infection. If both IgG and IgM were negative, then it was considered as rapid test negative. Sensitivity, specificity, PPV and NPV, were calculated.

Ethical considerations

The ethical clearance (exempt research certificate ERC-108-3-2020) was issued by the Health Research Governance Department prior to conducting the research. All data was kept in an encrypted password-protected laptop and stored in locked cabinets at the Principal Investigator’s office at HP-CDC, MOPH. Moreover, there is no foreseen risk associated with this study. The data sets are anonymous and personal identifiers have been removed. Only the principal investigator and co-investigators will have access to the study data. There is no collection of bio-specimens involved in this study. There is no issue of subject withdrawal/withdrawal of consent as we are using secondary data.

Results and Discussion

A total of 16951 individuals were tested using both RT-PCR and rapid tests for antibody detection as part of screening by the team from August 1st 2020 to November 30th 020. Nearly threefourths (71.9%) were males (Figure 2). Majority were in their second/ third decade of life and only 0.3% were aged below 19 years while 3.6% were aged above 55 years (Figure 3).

Figure 2.

Gender distribution.

Figure 3.

Age wise distribution.

Figure 4 depicts the major nationalities that were screened and remaining not identified were just one or two per nationality. As shown in the figure, majority were Indians (can be explained by the population distribution in Qatar) followed by Nepalis, Filipinos and Qataris.

Figure 4.

Nationality wise breakup.

Out of the 16951, 103(0.6%) tested positive with RT-PCR. More than three fourths(77.7%) were males and 63.1% were aged between 20-34 years. None of those aged below 19 years were detected to be positive for COVID using RT-PCR while 0.5% of the above 54 years turned out to be positive. Out of the 103 that tested positive with RT-PCR, 18.4% were Filipinos, followed by Nepalese (15.5%), Indians (11.6%), Sri Lankans (11.6%), Bangladeshis (10.7%) Qataris (4.8%) and other nationalities (27.2%). As such the positivity rate among each nationality was similar and very low i.e, <1% as only asymptomatic staff were screened. A higher rate of positivity (3% and 10%) was found among the Algerians and Mauritians respectively only because the number screened was very low. But as such the distribution of nationalities of the screened population is representative of the distribution of population in the state of Qatar.

Three hundred and sixty-seven (367) individuals (2.2%) were identified to have current infection (positive for IgM); 1401 individuals (8.3%) were IgG positive (probable previous infection with SARS-CoV-2) (Figure 5). Out of those 367 found as having a current infection by the rapid test only 1 was found to have a positive result by RT-PCR (Table 1). The remaining 102 SARS-CoV2 positive cases identified using RT-PCR were found to be negative with the rapid test kit.

Figure 5.

Summary of rapid kit test results.

Table 1.

Comparison between the rapid test kit results and RT-PCR.

Rapid test result	RT-PCR result
	Negative	Positive	Grand total
IgG(-ve) / IgM(-ve) (Not infected with SARS CoV-2)	15095	88	15183
IgG(+ve)& IgM(-ve) (probable previous infection with SARS CoV-2)	1387	14	1401
IgG(-ve)& IgM(+ve) (Current infection with SARS CoV-2; probably early stages)	275	1	276
IgG(+ve)& IgM(+ve) (Recent / Current infection with SARS CoV-2)	91	-	91
Grand total	16848	103	16951

Grand total

A 2*2 matrix (Table 2) was made based on the set criteria, i.e. IgM positive test results with or without IgG positivity was considered as rapid test positive for current infection and if only IgG was positive then it was considered as rapid test negative but previous infection. If both IgG and IgM were negative, then it was considered as rapid test negative. Based on this sensitivity, specificity, PPV and NPV, were calculated as shown below.

Table 2.

2*2 matrix for calculation of diagnostic indexes.

Rapid test result	RT-PCR result
	Negative	POSITIVE	Negative
POSITIVE (current infection) IgM positive test results with or without IgG positivity	366	1	367
NEGATIVE only IgG was positive (probable previous infection) AND both IgG and IgM were negative	16482	102	16584
Grand total	16848	103	16951

In our study the sensitivity was very low and specificity slightly lower than what was described by previous studies. The manufacturer of BioMedomics COVID-19 IgM-IgG rapid test, displayed a combined sensitivity of 100% (95% confidence interval (CI): 86.77%-100%) and a combined specificity of 98.75% (95% CI: 93.23%-99.97%).[6] Cassaniti et al. compared a rapid IgM/IgG test with RT-PCR in the emergency department and reported that 8.3% exhibited a positive result for IgM/IgG lateral flow immunoassay (LFI) while RT-PCR was negative.[2] Other study by Döhla et al. found similar rates of 11%.[7] Li et al. found the sensitivity, specificity, PPV, NPV, and accuracy of the test to be 85.6%, 91%,95.1%, 82.7%, and 88.3%, respectively; while Shen et al. found sensitivity and specificity to be71.1% and 96.2%.[8,9]

Sensitivity has mainly been evaluated mostly in hospitalized patients, so it is unclear whether the tests are able to detect lower antibody levels likely seen with milder and asymptomatic COVID- 19 disease. It is very important to calculate diagnostic indexes of the test kits stratified by the time from onset of illness or infection or by time of sample collection. Prazuck et al. investigated the COVID-PRESTO1 and COVID-DUO1 in comparison with RTPCR testing and the sensitivity of both increased with the duration from symptoms onset, reaching 100% in patients experiencing first symptoms of COVID-19 more than 15 days ago. The specificity of both tests was found to be 100%, no false positive results having been obtained.[10] But in this study the testing was done as part of screening, among asymptomatic apparently healthy people, hence could not be analyzed across different time periods.

The sensitivity of antibody tests is too low in the first week from the onset of symptoms, to have a primary role in the diagnosis of COVID-19. If both IgG and IgM were found to be negative, the probability of infection can not be ruled out. There is always a possibility of not having enough detectable antibodies in the very early stages of infection. For the accurate evaluation of the test kit, if samples from previously PCR confirmed COVID-19 cases, obtained during disease or convalescence, were used as true positives, then in the PCR-positive cases for which antibodies may not yet had time to develop, or in potential cases with immune defects, it is possible that the negative IgM or IgG results were in fact true negatives. A lower sensitivity, means more false-negative cases; who can transmit the infection to people they come in contact with. To effectively use such rapid test kits, they should be used together with RT-PCR in order to have a lower false negative number of patients. False-positive cases can be further confirmed by other detection methods.

Serological cross-reactions have earlier been observed between SARS-CoV and SARS-CoV-2 which is another possibility to be ruled out.[11] Hence this must also be considered while reading results of the rapid test kits for detecting antibodies. For a more optimal evaluation of the sensitivity of the assay, a gold standard for SARS-CoV-2 specific antibodies would have been needed. This is, however, unfortunately not available.

Conclusions

In our study the sensitivity and PPV of the rapid antibody kits are lower than what was described by previous studies and therefore we recommend not to use them alone for testing or for the purpose of taking clinic-based decisions or decisions related to quarantine/ isolation. It may have a role by complementing other tests.

The high NPV indicates that the rapid test will be useful for detecting past infections and possible immunity, which may be crucial for restoring social functions after lockdown. Since NPV is above 99% this kit can be used for testing for travel purposes, but can not be recommended for testing and quarantine purpose as the sensitivity and PPV are very low.

World Health Organization does not recommend the use of antibody-detecting rapid kits for patient care but encourages its use in disease surveillance and epidemiological research.[12] It may be used for screening purpose to assess antibody profiles in a large population. Large-scale screening programs using antibody tests are currently under evaluation by different governments, to give a notion of the magnitude of the spread in different geographical areas.