Evaluation of seven commercial RT-PCR kits for COVID-19 testing in pooled clinical specimens.

There are more than 350 real-time polymerase chain reaction (RT-PCR) coronavirus disease-2019 (COVID-19) testing kits commercially available but these kits have not been evaluated for pooled sample testing. Thus, this study was planned to compare and evaluate seven commercially available kits for pooled samples testing. Diagnostic accuracy of (1) TRUPCR SARS-CoV-2 Kit (Black Bio), (2) TaqPath RT-PCR COVID-19 Kit (Thermo Fisher Scientific), (3) Allplex 2019-nCOV Assay (Seegene), (4) Patho detect COVID-19 PCR kit (My Lab), (5) LabGun COVID-19 RT-PCR Kit (Lab Genomics, Korea), (6) Fosun COVID-19 RT-PCR detection kit (Fosun Ltd.), (7) Real-time Fluorescent RT-PCR kit for SARS CoV-2 (BGI) was evaluated on precharacterised 40 positive and 10 negative COVID-19 sample pools. All seven kits detected all sample pools with low Ct values (<30); while testing weak positive pooled samples with high Ct value (>30); the TRUPCR Kit, TaqPath Kit, Allplex Assay, and BGI RT-PCR kit showed 100% sensitivity, specificity, and accuracy. However, the Fosun kit, LabGun Kit, and Patho detect kit could detect only 90%, 85%, and 75% of weakly positive samples, respectively. We conclude that all seven commercially available RT-PCR kits included in this study can be used for routine molecular diagnosis of COVID-19. However, regarding performing pooled sample testing, it might be advisable to use those kits that performed best regarding positive identification in samples' pool, that is TRUPCR SARS-CoV-2 Kit, TaqPath RT-PCR COVID-19 Kit, Allplex 2019-nCOV Assay, and BGI Real-time RT-PCR kit for detecting SARS CoV-2.

INTRODUCTION

The on‐going coronavirus disease‐2019 (COVID‐19) pandemic caused by severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) started in December 2019 from Wuhan, China. World Health Organization (WHO) declared a pandemic on March 11, 2020 and urged that the most effective way to prevent infections and save lives is breaking the chains of transmission, and to do that escalation of COVID‐19 testing is urgently required.

The test available for laboratory diagnosis of COVID‐19 includes point of care antigen detection, Conventional real‐time polymerase chain reaction (RT‐PCR), cartridge‐based test molecular test like GeneXpert (Cepheid) and Truenat (Molbio), and automated high throughput molecular assays like Roche Cobas, Abbot molecular SARS CoV‐2 assay, Glenmark ePlex assay, and so forth. Amongst them, conventional RT‐PCR is the preferred and most widely used test for COVID‐19 diagnosis. ,

There are more than 350 Conventional RT‐PCR COVID‐19 testing kits available commercially, of which 29 kits have been approved by the United States food and drug administration (US‐FDA). RT‐PCR is an expensive test and requires a well equipped molecular laboratory with trained manpower. Many countries are experiencing acute shortages of diagnostic kits and manufacturers of molecular testing kits and consumables are also struggling to keep with the demand. It has become important to come up with novel ideas to conserve the reagents used for molecular tests. However, at the same time, the disease is new it is important to validate modifications to the testing protocol before universal adoption.

Several researchers are advocating that it is time to reintroduce the Dorfman theory of sample pooling in the era of molecular testing. , In a recent study from the University of Nebraska Medical Centre, Omaha, the authors have used a web‐based application and determined the most efficient pool size to be five samples when the incidence rate of SARS‐CoV‐2 infection is 10% or less and concluded that group testing will result in saving of reagents and increase in the testing capability of at least 69%. Several countries are performing pooled sample testing for COVID‐19, however, none of the available RT‐PCR kits has been tested for pooled sample either by kit manufacturer or research groups. Thus, this study was planned to compare and evaluate seven commercially available COVID‐19 RT‐PCR kits for pooled samples' testing (Table 1).

Table 1

Overview of RT‐PCR kits evaluated in the study

S No	Name of Kit	Manufacturer	Regulatory clearance	Target genes	Limit of detection	Kit interpretation
1	Allplex 2019‐nCoV assay	See gene	US‐FDA	E, N, RdRP	4167 copy/ml	C t < 40 positive
2	Patho Detect RT‐PCR kit	Mylab	ICMR, India	E, RdRP	‐	‐
3	FOSUN COVID‐19 RT‐ PCR Kit	Fosun	US‐FDA	E, N, ORF1ab	300 copy/ml	C t < 36 positive
4	TRUPCR SARS‐CoV‐2 RT‐qPCR kit	Black Biotech	US‐FDA	E, N, RdRP	10 copy/μl	C t < 35 positive
5	TaqPath COVID‐19 Combo Kit	Thermo Fisher Scientific	US‐FDA	S, N ORF1ab	2 copy/μl	C t < 40 positive
6	Lab Gun Real‐Time PCR Kit	Lab Genomics	US‐FDA	E, RdRP	20 copy/μl	C t < 40 positive
7	Real‐Time Fluorescent RT‐PCR Kit for 2019‐ nCoV	BGI Genomics	US‐FDA CE‐IVD	ORF1ab	150 copy/μl	C t < 37 positive

MATERIALS AND METHODS

Study design and sample collection

This prospective observational study was designed and conducted at the Department of Microbiology, Sanjay Gandhi Post Graduate Institute of Medical Sciences (SGPGIMS), Lucknow, India. The swabs were collected by healthcare workers at Rajdhani Corona Hospital, SGPGIMS in a 3 ml viral transport media (VTM) and transported to the COVID‐19 laboratory in a cold chain.

Selection of RT‐PCR kits

Before conducting the study, a survey was done on commercially available RT‐PCR kits regarding necessary approvals (US‐FDA/CE/ICMR, India), the lower limit of detection, usage, and availability in India, compatibility with different PCR platforms, cost, and so forth. Based on the survey results, the following kits were selected and procured: (1) TRUPCR SARS‐CoV‐2 Kit (Black Bio Biotech), (2) TaqPath RT‐PCR COVID‐19 Kit (Thermo Fisher Scientific), (3) Allplex 2019‐nCOV Assay (Seegene), (4) Patho detect COVID‐19 qualitative PCR kit (My Lab), (5) LabGun COVID‐19 RT‐PCR Kit (Lab Genomics), (6) Fosun COVID‐19 RT‐PCR detection kit (Fosun Ltd.), and (7) Real‐time Fluorescent RT‐PCR kit for detecting SARS CoV‐2 (BGI Genomics). None of the manufacturers were involved in the assessment and interpretation of the study results.

Performance of RT‐PCR on direct unpooled clinical specimens

The samples (VTM) were opened in biosafety cabinet class‐II and 300 μl of the VTM was further processed for viral nucleic acid extraction by a chemagic Viral DNA/RNA Kit on a chemagic™ 360 instrument (Perkin Elmar) as per the manufacturer's protocol. To avoid RNA degradation, the study was planned in such a manner that the entire experiment was completed in 24 h.

A 25 μl reaction was prepared for qualitative detection of SARS Corona virus‐2 by RT‐qPCR utilizing 5 μl of extracted RNA, 12.5 μl of 2× PCR buffer and 1 μl of AgPath One‐Step RT‐PCR Reagents (Thermo Fisher Scientific) and primer and probe sequences targeting E genes, RdRP and RnaseP as per WHO protocol. All oligonucleotides were synthesized and provided by Thermo Fisher Scientific and thermal cycling was performed at 55°C for 10 min for reverse transcription, followed by 95°C for 3 min and then 40 cycles of 95°C for 15 s, 58°C for 30 s using an Applied biosystem 7500 Real‐Time PCR system (Thermo Fisher Scientific). All samples that were initially screened for the E gene and positive samples were confirmed by detection of specific RdRP gene. The cut‐off threshold (C t value) for each sample was recorded and samples with C t value < 40 were considered as positive. All the samples were tested in triplicate before including them in the study. Once the positive and negative samples were identified by WHO protocol; we tested 40 positive samples and 10 negative samples by all seven commercial kits selected for study following kits protocol on an Applied biosystem 7500 Real‐Time PCR system.

Pooling of clinical samples for the RT‐qPCR before RNA extraction

Positive pools were created using 60 μl VTM from an RT‐PCR confirmed COVID‐19‐positive patient specimen added to 60 μl VTM from each of four negative patient samples to prepare a final volume of 300 μl. Similarly, negative sample pools were also created. The pooled 300 μl VTM was used as starting material for RNA extraction and nucleic acid extraction was performed on each pool using viral nucleic acid extraction by a chemagic Viral DNA/RNA Kit on a chemagic™ 360 instrument (Perkin Elmar). Finally, 100 μl of RNA was eluted and the same RNA was used in further experiments. The study protocol was designed in such a way that the positive sample with a low cut‐off threshold (C t value < 30) and high C t value (>30) were included. All RT‐PCR tests were performed using RNA extracted from pooled samples on an Applied biosystem 7500 Real‐Time PCR system and thermocycling settings and results interpretation was performed as per manufacturer's instructions.

RESULTS

A total of 500 samples were tested for SARS Cov‐2. Sixty samples were positive and the rest negative; we retested all positive and 10 negative samples in triplicate and finally 40 positive samples and 10 negative samples were selected for the study. Twenty positive samples had high viral loads with a low cut‐off threshold (<30) and twenty samples had low viral loads with a high C t value (>30). The selected 50 samples were tested by RT‐PCR for SARS CoV‐2 by using seven commercial kits. All seven commercially RT‐PCR kits could correctly identify 40 positive and 10 negative samples, as a weak positive sample with a cut of threshold in the range of 35–38 was also detected by these kits, it can be safely concluded that they can be used for routine diagnostics of COVID‐19.

All seven commercial kits were evaluated with 40 positive and 10 negative sample pools in duplicate. Results showed that all kits performed well when strongly positive samples were tested and all seven kits detected all sample pools with low C t value (<30). When testing weak positive pooled samples (C t value > 30), TRUPCR Kit, TaqPath Kit, Allplex Assay, and BGI RT‐PCR kit showed 100% sensitivity, specificity, and accuracy. (Table 2) The Fosun kit, LabGun Kit, and Patho detect kit could detect only 18 (90%), 17 (85%), and 15 (75%) of weakly positive samples respectively. The C t value of all samples that could not be detected on pool testing by RT‐ PCR was more than 37. (Table 3) The C t value of three pool by WHO protocol that was not detected by LabGun kit was 37.8, 38.2, and 38.4, among them the LabGun kit could detect all three with the good sigmoid graph at C t value > 40 but was interpreted as negative as per manufacturer recommendations, the LabGun RT‐PCR kit demonstrated a sensitivity, specificity, and accuracy of 93.0%, 100%, and 94% respectively.

Table 2

Overall diagnostic efficacy of kits used in the study

	Lab Gun	Fosun	BGI	Thermo Fischer Scientific	Black Bio	My Lab	Seegene
Sensitivity	93.02% (80.9%–8.5%)	95.2% (83.8%–99.4%)	100.0% (91.1%–100.0%)	100.0% (91.1%–100.0%)	100.0% (91.1%–100.0%)	88.8% (75.9% – 96.2%)	100.0% (91.1%–100.0%)
Specificity	100% (69.1%–100%)	100% (69.1%–100%)	100% (69.1%–100%)	100% (69.1%–100%)	100% (69.1%–100%)	100% (69.1%–100%)	100% (69.1%–100%)
Positive predictive value	100.0%	100.0%	100.0%	100.0%	100.0%	100.0%	100.0%
Negative predictive value	76.9% (52.8%–90.8%)	83.3% (56.3% ‐95.0%)	100.0%	100.0%	100.0%	66.6% (46.6%–82.0%)	100.0%
Accuracy	94.3%	96.15%	100.0%	100.0%	100.0%	90.9%	100.0%

Table 3

Showing Cut off threshold (C t value) of positive sample pools not detected by few commercial kits

S. No.	E gene (In house)	Lab Gun	Fosun	BGI	Thermo Fischer Scientific	Black Bio	My Lab	Seegene
1	38.4	Negative (41.4)	Negative (38.1)	35.4	37.2	34.2	Not detected	38.0
2	37.8	Negative (40.6)	35.6	36.6	35.8	32.8	Not detected	38.4
3	37.5	38.9	Negative (37.0)	35.2	36.8	35.0	Not detected	38.4
4	38.2	Negative (41.0)	35.8	36.2	37.2	33.6	Not detected	38.8
5	37.8	38.8	34.8	37.0	36.4	34.0	Not detected	38.2

The C t value by WHO protocol of two sample pools tested negative by FOSUN kit was 37.5 and 38.4, the FOSUN RT‐PCR kit detected them at a C t value of 37.0 and 38.1, respectively, but was interpreted negatively as per manufacturer recommendations (C t value > 36 Negative). The Patho detect kit by MyLab could not detect five (25%) sample pool and there was no sigmoid shaped graph even at a higher C t value. The sensitivity, specificity, and positive predictive value of the Mylab kit is 88.8%, 100%, and 100.0%, respectively, but documented a low negative predictive value of 66%.

DISCUSSION

Amidst the ongoing COVID‐19 pandemic, the World Health Organization has globally emphasized the importance of the molecular diagnosis of SARS CoV‐2 to limit the spread as well as to appropriately treat those patients who have a serious infection. WHO has further emphasized that an urgent increase in laboratory testing, isolation, and contact tracing should be the backbone of the pandemic control strategy. Accurate and timely results are important for decision making during this current outbreak, both in the inpatient and OPD settings. For patients admitted to the hospital, the results are critical for medical management, patient cohorting and infection control measures. Likewise, the results are also very important in the outpatient setting; as community prevalence data and identification of disease hot spots help the decision‐makers to form the basis for social distancing, the lockdown of infected areas, and a disease combating strategy.  A short turnaround time of COVID‐19 test reports is also critical for judicious use of limited resources, such as the availability of holding area beds, isolation rooms, and real‐time cohorting decisions. In addition, timely test results are required to ensure the safety of healthcare workers and minimize their exposure as levels of personal protective equipment required by healthcare professionals also vary depending on whether a patient is COVID‐19 positive or negative.

The recommended test for diagnosis of COVID‐19 is RT‐PCR, and to conduct this test, a fully functional molecular laboratory is required equipped with specialized equipment like biosafety cabinets, automated RNA extractors, a Real‐time PCR machine, and trained manpower to process the samples while ensuring biosafety and biosecurity. It is difficult to set up a new molecular testing laboratory in the midst of the COVID‐19 pandemic and increasing the testing capacity of the existing laboratory by sample pooling strategy is a practical solution. The pandemic reached India in March 2020 and the testing capacity of the nation was less than 5000 samples/day. On 13.04.2020, WHO in collaboration with the Indian Council Of Medical Research, New Delhi issued an advisory recommending pool testing of five‐sample pools in an area where COVID‐19 prevalence is <5%. With time the molecular laboratory network in India has been strengthened and using the sample pooling strategy currently 110,000 samples are tested every day and to date, 60 million samples have been tested with 5 million COVID‐19 positive cases. Sample polling conserves PCR Kits and consumables and significantly decreases manpower requirement. At our center, we have performed 300,000 COVID‐19 RT‐PCR to date and data suggest that five‐sample pooling saves 55% reagents and increases the testing capacity 2.5 times using the same infrastructure and manpower (unpublished data).

Currently, there is no literature on the evaluation of the efficacy of various commercially available Real‐Time PCR kits for pool sample testing, thus this study was planned to compare and evaluate seven commercially available RT‐PCR kits for the detection of SARS‐CoV‐2 in clinical sample pools. We compared the kits by performing testing using the same quantified RNA; thus, allowing parallel evaluation.

All seven RT‐PCR kits evaluated in this study could correctly identify all 40 positives and 10 negative samples, including a weak positive sample. A similar study evaluated the clinical performance of selected RT‐PCR kits from seven different manufacturers—Altona Diagnostics, BGI, CerTest Biotec, KH Medical, Primer Design, R‐ Biopharm AG, and Seegene and PCR efficiency was found to be ≥96% for all assays and the authors concluded that all tested RT‐PCR in the study may be used for routine diagnostics of COVID‐19 in patients by experienced molecular diagnostic laboratories. In another study evaluating the diagnostic performance of RT‐PCR kits provided with emergency use authorization by US‐FDA, the New York SARS‐CoV‐2 Real‐time PCR Diagnostic Panel (modified CDC) assay, the Simplexa COVID‐19 Direct (Diasorin Molecular) assay, GenMark ePlex SARS‐CoV‐2 (GenMark) assay, and the Hologic Panther Fusion SARS‐CoV‐2 (Hologic) assay were tested and the study results suggested that all 4 kits performed similarly. Thus, based on these studies' results and existing literature, it can be concluded that US‐FDA/CE kits can be used for laboratory diagnosis of COVID‐19.

To date, to the best of our knowledge, no study has compared the diagnostic performance of commercially available RT‐PCR kits for pooled sample testing; although there are few studies that have evaluated pool testing efficacy using a single RT‐PCR kit. In a study on evaluation of RT‐PCR for pool testing using the Seegene Allplex 2019 nCoV assay, the authors pooled up to 32 samples and reported that the kit could detect all positive samples and concluded that using standard protocols sample pooling can be applied immediately in current clinical testing laboratories. In another study on pool sample testing, the authors performed pool testing using a Real Star SARS‐CoV‐2 RT‐PCR Kit (Altona Diagnostics) and suggested that pooling of up to 30 samples per pool can increase test capacity with existing equipment and test kits and detects positive samples with sufficient diagnostic accuracy.

This study results show higher analytical sensitivities, specificity, and accuracy of the TRUPCR SARS‐CoV‐2 Kit, TaqPath RT‐PCR COVID‐19 Kit, Allplex 2019‐nCOV Assay, and Real‐time Fluorescent RT‐PCR kit for detecting SARS CoV‐2 (BGI) assays when compared to the FOSUN COVID‐19 RT‐PCR detection kit, LabGun COVID‐19 RT‐PCR Kit and Patho detect COVID‐19 qualitative PCR kit assays. The C t value of samples that could not be detected by three RT‐PCR was more than 37. The Lab Gun kit and Fosun kit could detect all samples with a good sigmoid graph at high C t value but were interpreted as negative as per manufacturer recommendations. Thus, in pooled samples, RT‐PCR graphs should be analyzed for sigmoid curve even beyond the manufacturer‐recommended cut‐off threshold, and in case of the appearance of any graph, the RT‐PCR should be repeated with deconvoluted samples. Further unpublished data from our centre suggest that less then <2% of our RT‐PCR‐positive samples had C t value > 37, and by following the above‐mentioned precautions, we picked the maximum possible positive cases.

We conclude that all seven commercially available RT‐PCR kits included in this study can be used for the molecular diagnosis of COVID‐19. When performing pool sample testing, it might be advisable to use those kits that performed best regarding positive identification in the samples pool that is the TRUPCR SARS‐CoV‐2 Kit, TaqPath RT‐PCR COVID‐19 Kit, Allplex 2019‐nCOV Assay, and Real‐time Fluorescent RT‐PCR kit for detecting SARS CoV‐2 (BGI).

CONFLICT OF INTERESTS

The authors declare that there are no conflict of interests.

AUTHOR CONTRIBUTIONS

Atul Garg designed the study, Ujjala Ghoshal prepared and edited the manuscript, Sangram S. Patel, D.V. Singh, and Akshay K. Arya performed the molecular lab work and Shruthi Vasanth performed data analysis.