Performance assessment of seven SARS-CoV-2 IgG enzyme-linked immunosorbent assays.

The pandemic of COVID-19 has caused enormous fatalities worldwide. Serological assays are important for detection of asymptomatic or mild cases of COVID-19, and sero-prevalence and vaccine efficacy studies. Here, we evaluated and compared the performance of seven commercially available enzyme-linked immunosorbent assay (ELISA)s for detection of anti-severe acute respiratory syndrome corona virus 2 (SARS-CoV-2) immunoglobulin G (IgG). The ELISAs were evaluated with a characterized panel of 100 serum samples from qRT-PCR confirmed COVID-19 patients, collected 14 days post onset disease, 100 SARS-CoV-2 negative samples and compared the results with that of neutralization assay. Results were analysed by creating the receiver operating characteristic curve of all the assays in reference to the neutralization assay. All kits, were found to be suitable for detection of IgG against SARS-CoV-2 with high accuracy. The DiaPro COVID-19 IgG ELISA showed the highest sensitivity (98%) among the kits. The assays demonstrated high sensitivity and specificity in detecting the IgG antibodies against SARS-CoV-2. However, the presence of IgG antibodies does not always correspond to neutralizing antibodies. Due to their good accuracy indices, these assays can also aid in tracing mild infections, in cohort studies and in pre-vaccine evaluations.

enzyme‐linked immunosorbant assay

plaque reduction neutralization test

receiver operating characteristic curve

INTRODUCTION

The corona virus disease of 2019 believed to have been originated in Wuhan, Hubei province in China, is caused by severe acute respiratory syndrome corona virus 2 (SARS‐CoV‐2). , The disease was declared a pandemic by the World Health Organization and has since infected more than 178 million people and caused over three million fatalities worldwide. The SARS‐CoV‐2 belongs to the Nidovirales order of the Coronaviridae family, which includes SARS‐CoV and MERS‐CoV, which caused outbreaks in 2003 and 2012 respectively.

The SARS‐CoV‐2 causes respiratory infections of varying severity. The most common symptoms include fever, dry cough, tiredness and the more severe symptoms include acute respiratory distress syndrome (ARDS), coagulation disorders, multiorgan dysfunction and central nervous system infection. The diagnosis of SARS‐CoV‐2 is mainly dependent on the detection of viral RNA by real time reverse transcriptase polymerase chain reaction (qRT‐PCR). The viral RNA can be detected from 72 h before onset of symptoms  to more than 80 days post first detection. , In patients with mild and asymptomatic infection, low PCR positivity rate has been reported in samples collected 8 days after onset of symptoms. As the recent trend indicates an increase in the number of asymptomatic cases, there is a pressing need for serodiagnosis of the SARS‐CoV‐2. IgM and IgG antibodies against the virus can be detected in the serum of the patients as early 4–7 days post onset of disease (POD) , , and up to 95% of infected individuals may show seropositivity after 8 days POD. , , The currently available enzyme immunoassays for the detection of exposure to SARS‐CoV‐2 are based on the detection of IgA, IgM, IgG, or total antibodies against the virus. , ,

The SARS‐CoV‐2 contains four structural proteins—spike (S), nucleocapsid (N), envelope (E) and membrane (M). Of the four proteins, S and N proteins are most immunogenic. While the N protein facilitates viral replication, assembly and release, , the S protein, mediates binding of the virus to the ACE‐2 cellular receptors. The S protein comprises two sub units, S1 and S2, responsible for binding to host cell receptor (ACE‐2) and fusion of cellular and viral membranes respectively. , Majority of the serological assays have S or N proteins as their target antigens.

The serodiagnosis of the SARS‐CoV‐2 is still being explored for accurate and reliable diagnosis. Several ELISAs and other antibody testing assays such as chemiluminescence based immunoassay and lateral flow (rapid diagnostic) assays are now available from different manufacturers. The detection accuracy of IgG ELISA may considerably vary among the test kits, highlighting the need of validation before using them in field settings. Here, we have evaluated the performance of seven commercially available anti‐SARS‐CoV‐2 IgG ELISA kits, which can be used to address different requirements. We analysed their performance in correlation to neutralization assay, which is a gold standard in assessing immunity against SARS‐CoV‐2.

MATERIALS AND METHODS

Ethical statement

All the samples collected were with informed consent from patients and the study was approved by ICMR‐ NIV Institutional Ethics Committee.

Sample panel

The test kits were evaluated with a panel of 100 serum specimens from qRT‐PCR confirmed COVID‐19 patients, collected 14 days POD. These samples were tested by the in‐house plaque reduction neutralization test (PRNT) for the presence of antibodies against the SARS‐CoV‐2.

A total of 90 SARS‐CoV‐2 PRNT negative serum samples collected before beginning of the COVID‐19 pandemic. An additional 10 samples positive for other respiratory viruses such as human corona virus (HCoV) OC43, influenza A(H1N1)pdm09, influenza A(H3N2), parainfluenza virus 4, measles and rubella viruses were included to assess cross‐reactivity.

Tests performed

The evaluation was performed on seven commercial SARS‐CoV‐2 anti‐IgG ELISAs: COVID Kawach Anti SARS‐CoV‐2 human IgG ELISA (J Mitra and Co. Pvt. Ltd.); Anti SARS‐CoV‐2 ELISA (IgG) (Euroimmun Medizinische Labordiagnostika AG); Aspen SARS‐CoV‐2 IgG ELISA (Aspen Laboratories); GA CoV‐2 IgG (Generic Assays [GA], GmbH); COVID‐19 IgG ELISA (Dia.Pro Diagnostic Bioprobes Srl), PANBIO™ SARS‐CoV‐2 IgG ELISA (Abbott) and SCoV‐2 Detect™ SARS‐CoV‐2 IgG ELISA (InBios International). For conciseness, we refer to the kits by the name of the manufacturer. All the tests were performed according to manufacturer's instructions. Repeat testing of the samples was done only when there were inconclusive/equivocal results obtained. To analyse user/operational variability, every test included 4 intra‐assay and inter‐assay replicates (two positives and two negatives). The test characteristics are compared in Table 1.

Table 1

Detailed kit specifications for seven commercial ELISA kits

Sr. No.	Kit manufacturer	Product name	Sample type	Sample dilution	Time to result	No of samples/kit	Antigen type	Country of origin	Nature of assay type
1	Euroimmun	Anti SARS‐CoV‐2 ELISA (IgG)	Serum	1:101	120 min	93	Recombinant S1 protein	Germany	Semiquantitative
2	Abbott	PANBIO SARS‐CoV‐2 IgG ELISA	Serum	1:5	120 min	90	Recombinant protein	USA	Semiquantitative
3	J Mitra & Co. Pvt. Ltd.	COVID Kawach Anti SARS‐CoV‐2 Human IgG ELISA	Serum	1:100	130 min	94	Inactivated Whole Antigen	India	Qualitative
4	Generic Assays	GA CoV‐2 IgG	Serum	1:20	105 min	91	Recombinant antigen	Germany	Qualitative
5	Aspen Laboratories Pvt. Ltd.	Aspen SARS‐CoV‐2 IgG ELISA	Serum	1:100	80 min	92	N protein	India	Qualitative
6	InBios International Inc.	SCoV‐2 Detect™ SARS‐CoV‐2 IgG ELISA	Serum	1:100	110 min	90	Not specified	USA	Semiquantitative
7	DIA. PRO Diagnostic BioProbes Srl	COVID‐19 IgG ELISA	Serum	1:20	105 min	91	Recombinant N+S protein	Italy	Qualitative

Abbreviation: ELISA, enzyme‐linked immunosorbent assay.

Plaque reduction neutralization test

The PRNTs were performed as described by Deshpande et al (2020). Briefly, all the sera were heat inactivated and serially diluted 4‐fold starting at a dilution of 1:10. Further these samples were mixed with an equal amount of virus suspension containing 50–60 plaque‐forming units (PFU) in 0.1 ml. After incubating the mixtures at 37°C for 1 h, each virus‐diluted serum sample (0.1 ml) was inoculated onto a 24‐well tissue culture plate containing a confluent monolayer of Vero CCL‐81 cells. After incubating the plate at 37°C for 60 min, an overlay medium consisting of 2% carboxymethyl cellulose with 2% fetal calf serum in 2× MEM was added to the cell monolayer and the plate was further incubated at 37°C in 5% CO2 for 5 days. Plates were stained with 1% amido black for an hour. Antibody titers were determined as the highest serum dilution that resulted in >50 (PRNT50) reduction in the number of plaques.

Statistical analysis

The statistical analysis was performed by GraphPad Prism 9 and SPSS statistics 20. The coefficient of variation (CV) of the controls were calculated. Intra‐ and inter‐assay precision were analysed to check the repeatability of the tests. The sensitivity, specificity, PPV and NPV were calculated for all kits by Fischer's exact test. The inter‐rater agreement (Cohen's Kappa coefficient (κ)) of the tests was also calculated. The κ values were interpreted as very good (0.81–1.00), good (0.61–0.80), moderate (0.41–0.60), fair (0.21–0.40) or poor (<0.20). The receiver operating characteristic (ROC) curves were plotted to check the accuracies of the assays. The resulting antibody titres of the PRNT were calculated with probit analysis.

RESULTS

Assessment of kit controls

The percent coefficient of variation (%CV) for the kit controls (positive and negative) were analysed for all the7 kits. The %CV of all the commercial anti SARS‐CoV‐2 IgG ELISA kits are represented in Table 2.

Table 2

Performance of kit controls of seven ELISA kits

Kit	Euroimmun	Abbott	J Mitra COVID Kawach	GA	Aspen	InBios	Dia. Pro
Positive controls	7.81	1.38	4.94	6.82	3.32	0.56	13.33
Negative controls	3.68	4.95	3.16	0.87	13.55	5.64	12.64

Abbreviation: ELISA, enzyme‐linked immunosorbent assay.

The Inbios IgG ELISA had the lowest %CV of 0.56 for positive controls whereas GA IgG had a lowest %CV of 0.875 for negative controls indicating high reproducibility. Dia.Pro.'s IgG ELISA had the highest % CV for positive control (13.33) and Aspen IgG showed highest %CV for negative control (13.855). An overall low variation was observed among the negative controls of all the kits. The CV of all the kits were within the limit of acceptance of 15% (Bioanalytical Method Validation Guidance for Industry, US Food and Drug Administration, 2018). The overall % CV of Euroimmun, J Mitra, GA, Abbott and InBios was ≤10 indicating good performance of these assays.

Repeatability assessment of the kits

Inter and intra assay precision for each kit was analysed with 4 serum samples (two positive and two negatives) tested on each run of all the kit manufacturers. Each of these samples was tested in three different kit lots for inter‐assay assessment and four replicates within each plate were taken for intra‐assay precision. The %CV for all the replicates of one kit was calculated to assess the repeatability of the kit (Table 3). The data indicated that, with an exception to two assays, all the assays showed low inter‐ and intra‐assay %CV. While, the Euroimmun IgG showed the least intra‐assay variation (2.491) and GA showed the least inter assay (2.757) variation, Dia.Pro. IgG showed highest variation in the intra assay (17.03) and Aspen IgG showed highest variation in inter assay (12.88).

Table 3

Precision and repeatability assessment of the commercial ELISA kits

% CV	Euroimmun	Abbott	J. Mitra COVID Kawach	GA	Aspen	InBios	Dia. Pro
Intra assay	2.491	6.525	11.429	9.977	4.766	3.702	17.033
Inter assay	2.888	6.239	8.364	2.757	12.881	3.979	9.295

Abbreviation: ELISA, enzyme‐linked immunosorbent assay.

Measures of diagnostic accuracy

The measures of diagnostic accuracy of all the kits were analysed using the panel of 200 specimens (Table 4). For IgG detection, the J. Mitra COVID Kawach IgG ELISA showed 96% sensitivity, whereas the test kits of Euroimmun, Dia.Pro., InBios, Abbott, GA, and Aspen showed sensitivities between 91% and 98%. The DiaPro IgG had the highest sensitivity (98%), whereas, Euroimmun IgG ELISA had the lowest sensitivity of 91%. The specificity for IgG were relatively low (95%) for Euroimmun IgG ELISA, while rest of the assays displayed higher specificities (96%–99%).

Table 4

Sensitivity, specificity, and agreement of kits compared to confirmed reference results

Manufacturer	Sensitivity %	Specificity %	PPV %	NPV %	Area under curve	Kappa
Aspen	96	98	97.96	96.08	0.984	0.958
Abbott	97	98	97.98	97.03	0.999	0.950
J. Mitra	96	99	98.97	96.12	0.999	0.960
Dia. Pro	98	98	98	98	0.999	0.977
Euroimmun	91	95	94.79	91.35	0.997	0.845
GA	98	97	97.03	97.98	0.988	0.911
InBios	97	96	96.04	96.97	0.999	0.943

Figure 1 represents the distribution of optical density (OD) values of the kits for the testing panel. Within the seven assays, seven assays (J. Mitra, Euroimmun, Abbott, GA, InBios, DiaPro, and Aspen) had consistently negative or low ODs for the negative panel. No cross‐reactivity was observed with sera known to be positive for other respiratory viruses and all gave values below the defined cut‐off point for all kits. Similarly all the assays had OD values in similar range for the positive panel.

Figure 1

Distribution of OD values yielded by 200 samples in the validation panel. The panel consists of confirmed 100 samples positive in PRNT for SARS‐CoV‐2 and 100 samples which tested negative in the same. OD, optical density; SARS‐CoV‐2, severe acute respiratory syndrome corona virus 2

The ROC area under the curve (AUC) measures the accuracy of the assay. The AUC for all the IgG assays were between 0.984 and 0.999 (Table 3 and Figure 2). The Aspen IgG had the lowest (0.984), while the Abbott, J. Mitra, DiaPro, and InBios IgG ELISA kits had the highest AUC of 0.999. The data suggest overall good accuracy of all kits for detection of true positive and true negative samples.

Figure 2

ROC analysis of the results yielded by the seven test kits. A panel of 200 samples characterized by the neutralization test were tested by all the kits and the results obtained were compared with the results of the neutralization test. ROC, receiver operating characteristic

Inter rater agreement of the kits

The Cohen's kappa (κ) analysis was performed to understand the inter rater agreement between the test kits and the standard assay. In the evaluation of the agreement, the N+S based DiaPro ELISA had the highest agreement (κ = 0.977) with the standard assay. The whole antigen‐based J. Mitra assay had an agreement 0.960, followed by the N based Aspen (κ = 0.958) and InBios (antigen not specified) assay having κ of 0.943. The recombinant antigen based GA and Abbott ELISA had the κ values of 0.911 and 0.950. The S1 based Euroimmun assay had the agreement (κ) of 0.845. The findings suggest differences in the agreements of the results within the assays based on similar antigen.

Agreement between the kits

The agreement between the kits was analysed by percent concordance of results for all the kits with each other (Figure 3). The results of individual samples were compared for evaluating concordance. Although the sensitivity and specificity of the assays can be similar, the results for the individual samples may vary. Our results indicated a similar observation. The concordance of InBios and GA with Euroimmun was relatively lower (between 86% and 89%). Rest of the assays showed good agreement with all the assays (between 90% and 99%). The findings also suggest that samples with probable high IgG titers (which yielded high OD values) were detected by all the assays, while results varied among the kits for samples which yielded low OD values (probably containing low levels of IgG). This might indicate variation resulting from the antigen used and the inherent sensitivity of the assay.

Figure 3

Head on comparison of percent concordant results of each kit. All the kits were compared head on for the concordance of results for the panel of 200 samples. The kit to kit concordance ranged from 86% to 99%

DISCUSSION

The nucleic acid detection test is currently the gold standard method for the diagnosis of SARS‐CoV‐2 infection, but ELISAs, CLIA and rapid tests for detection of antibodies, might also be useful in for detecting exposure to the virus. In this study, we compared seven ELISA kits for detection of SARS‐CoV‐2 IgG using a well‐characterized panel of 200 samples. The detection of IgG is useful in the sero‐epidemiological studies, and therefore the performance comparison of these kits may guide the contextual use of these assays in the field studies. Euroimmun assay measure IgG antibody response against S protein, J. Mitra assay antibody against inactivated whole protein, DiaPro against a combination of N and S protein and Aspen Lab against N protein. InBios, Abott, and Generic Assays did not provide details of the target antigen used in their assays.

Ease of performance is a distinct advantage of ELISAs. A few manufacturers employed coloured reagents as a pipetting guide in the assay. All the kits detected IgG antibody to the virus in serum. The time required for the assay performance ranged from 80 min (Aspen ELISA) to 130 min (J. Mitra). All the assays allowed testing of more than 90 samples per assay.

All the assays displayed sensitivity and specificity in range of 91%–98% and 95–98%, respectively. The variation in the sensitivity and specificity between the different assays may be attributed to the different assay formats and target antigen. The high negative predictive value of these assays also suggests their usefulness in detection of past infections.

Studies suggest that serum antibody levels against N and S proteins in the COVID‐19 patients tend to increase after 10–17 days post onset of symptoms. , Liu W, et al. reported comparable sensitivities of S and N antigen based ELISAs, which was also observed in our study. The inter kit agreement between the kits are also high suggesting usefulness of the kits in detection of the SARS‐CoV‐2 IgG.

The findings of this study are consistent with similar recent studies which suggest overall good performance of the SARS‐CoV‐2 IgG assays. , , , , , , , Detailed studies are necessary to evaluate the performance of these assays on samples collected during different phases of infection. Also, cross‐reactivity of the assays should be determined using a larger number of samples positive for closely related viruses.

The results of this study may guide the use of various commercial assays in the field studies and help in public health decisions related to COVID‐19. We are also studying the antibody responses to different antigens and emergence of antibodies against different viral proteins, which will help in understanding the response curve as well as to formulate a serological testing algorithm for detection of antibodies against SARS‐CoV‐2.

CONCLUSION

In conclusion, the antibody tests have an important role in diagnosis and add value to the molecular diagnosis. This study provides background for the utility of these commercial IgG assays in screening, contact tracing and sero‐prevalence studies for the SARS‐CoV‐2. The assays evaluated were highly specific and sensitive in detecting the IgG antibodies against SARS‐CoV‐2. However, it is also to be kept in mind that the presence of IgG antibodies does not always correspond to neutralizing antibodies. Due to their high sensitivity and specificity, these assays can be readily used in tracing those who had asymptomatic/mild infections, in longitudinal studies and in vaccine studies.

CONFLICT OF INTERESTS

The authors declare that there are no conflict of interests.

AUTHOR CONTRIBUTIONS

Conceptualization: Gajanan Sapkal, Ketki Deshpande, Ullas PT, Priya Abraham. Analysis of data: Ketki Deshpande, Gajanan Sapkal, Aparana Rakhe; Ojas Kaduskar, Varsha Potdar, Pragya D. Yadav, Gururaj Deshpande. Sample collection and methodology: Gurav YK, Bipin Tilekar, Ketki Deshpande, Ojas Kaduskar, Aparana Rakhe, Sanskruti Saka, Kshitija Gadekar, Roshni Patil, Shankar Vidhate, Kirti Khutwad. Writing, review and editing: Ketki Deshpande, Gajanan Sapkal, Ullas PT, Neetu Vijay, Harmanmeet Kaur, Priya Abraham, Pragya D. Yadav; Gururaj Deshpande. Supervision: Gajanan Sapkal, Priya Abraham, Varsha Potdar, Neetu Vijay, Priyanka Gupta, Harmanmeet Kaur, and Jitendra Narayan.