High concordance in SARSCoV-2 detection between automated (Abbott m2000) and manual (DaAn gene) RT-PCR systems: The EDCTP PERFECT-Study in Cameroon.

Molecular diagnosis of COVID-19 is critical to the control of the pandemic, which is a major threat to global health. Several molecular tests have been validated by WHO, but would require operational evaluation in the field to ensure their interoperability in diagnosis. In order to ensure field interoperability in molecular assays for detection of SARS-CoV-2 RNA, we evaluated the diagnostic concordance of SARS-CoV-2 between an automated (Abbott) and a manual (DaAn gene) realtime PCR (rRT-PCR), two commonly used assays in Africa. A comparative study was conducted on 287 nasopharyngeal specimens at the Chantal BIYA International Reference Centre (CIRCB) in Yaounde- Cameroon. Samples were tested in parallel with Abbott and DaAn gene rRT-PCR, and performance characteristics were evaluated by Cohen's coefficient and Spearman's correlation. A total of 273 participants [median age (IQR) 36 (26-46) years] and 14 EQA specimens were included in the study. Positivity was on 30.0% (86/287) Abbott and 37.6% (108/287) DaAn gene. Overall agreement was 82.6% (237/287), with k=0.82 (95%CI: 0.777-0.863), indicating an excellent diagnostic agreement. The positive and negative agreement was 66.67% (72/108) and 92.18 % (165/179) respectively. Regarding Viral Load (VL), positive agreement was 100% for samples with high VLs (CT<20). Among positive SARS-CoV- 2 cases, the mean difference in Cycle Threshold (CT) for the manual and Cycle Number (CN) for the automated was 6.75±0.3. The excellent agreement (>80%) between the Abbott and DaAn gene rRTPCR platforms supports interoperability between the two assays. Discordance occurs at low-VL, thus underscoring these tools as efficient weapons in limiting SARS-CoV-2 community transmission. ©Copyright: the Author(s).

Introduction

Coronaviruses are a large family of viruses, which may cause disease in animals or humans. Seven coronaviruses can lead to infection, but four of them are most common in human, namely 229E, NL63, OC43, and HKU1.[1] They usually cause a wide range of respiratory infections varying from the common cold to more severe diseases such as the Middle East Respiratory Syndrome (MERS) and the Severe Acute Respiratory Syndrome (SARS) of which the most recently discovered is the SARS-CoV- 2, responsible for the coronavirus disease in 2019 (COVID-19).[1,2] Time from exposure to SARS-CoV-2 and symptom onset is generally between 2-14 days, with an average of 5 days. Common symptoms include fever, cough, sneezing and shortness of breath. Complications may include pneumonia, throat pain, acute respiratory distress syndrome, cytokine storm, blood clotting, and even death.[2,3]

As of February 23th 2022, there have been 426,624,859 confirmed cases of COVID-19 worldwide, including 5,899,578 deaths with a total of 10,407,359,583 vaccine doses administered,[4] with the USA reporting the highest number of cases all around the world.[4] In Africa, 53 countries were declared affected with 11,129,305 cases and 247,310 deaths. Of all African countries, South Africa still records the highest number of cases.[5] In Cameroon, we recorded 119,107 confirmed cases on February 16th, 2022 including 1,920 deaths and 1,066 951 received at least one dose of vaccine (representing 7.7% of the target population). With a case fatality rate of 1.6%, the country remains among the African countries with the highest number of infections.[6]

To ensure an effective control and surveillance system of COVID-19, the World Health Organization (WHO) provides Emergency Use Authorization (EUA) to molecular and non-molecular techniques for the diagnosis of SARS-CoV-2 infection. Non-molecular techniques such as rapid diagnostic tests, involving antigens and antibodies have been authorized for the large scale detection of SARS-CoV-2 and sero-surveillance within the context of massive spread of the pandemic.[7-9] Although non-molecular techniques are very effective (high specificity) in slowing the spread, they are also less sensitive in identifying cases with low-level viremia as compared to molecular diagnostics.[10] Interestingly, molecular assays involving real-time amplification (rRT-PCR) of the nucleic acid (RNA) are the standard methods for diagnosing SARS-CoV-2 infection,[11-13] but they might have some disparities in their ability to accurately detect the viral target gene(s), especially in specimens with very-low viral titers or with emerging variants.[10-14] Of note, diagnostic trials have been conducted extensively prior to the EUA, but the routine use of these assays in real-life remains poorly documented. Furthermore, the lack of evidence on the diagnostic concordance between existing systems limits the potentials to ensure assay inter-operability for the further validation in routine clinical diagnosis. In several sub-Saharan African countries, Abbott and DaAn gene molecular systems have been widely used in response to COVID-19 molecular detection.[7,8] Thus, evaluating the diagnostic agreement would be of paramount importance in the routine COVID-19 diagnostic landscape in these settings, in an accurate detection of suspect cases for adequate clinical management, and in an optimal public health control measures through the identification and tracking of potential transmitters at community- level. We herein evaluated the diagnostic concordance of Abbott and DaAn gene rRT-PCR on both standard panels and clinical specimens.

Materials and Methods

Study design and population

Within the frame of the EDCTP PERFECT- Study (Performance Evaluation of COVID-19 Tests), a comparative study was conducted on 287 samples [273 nasopharyngeal clinical swabs and 14 quality control (CQ) panels] enrolled from February through May 2021 at the Chantal BIYA International Reference Center (CIRCB) for research on HIV/AIDS prevention and management. Study population was therefore made up of travelers, symptomatic patients, and contact cases; while panels for external quality control assessment were provided by the WHO country office and the African Society for Laboratory Medicine (ASLM) through a partnership with the Africa CDC.

Nasopharyngeal swabs were collected from consenting individuals and stored in tubes containing a Viral Transport Media (VTM) appropriate for COVID-19 molecular testing (Sansure Biotech Inc, https://www.appletonwoods.co.uk).

For each participant, the nasopharyngeal sample was then divided into two aliquots for a parallel laboratory processing on Abbott and DaAn gene rRT-PCR respectively. Socio-demographic and basic clinical data were collected using a standard datasheet for COVID-19 testing.

Description of the study site

CIRCB is a national reference laboratory for the molecular diagnosis of COVID- 19 under the Ministry of Public Health of Cameroon. To ensure reliability in COVID- 19 testing, CIRCB participates in external quality control programs for proficiency testing with the WHO and the ASLM. A total of 41,716 samples have been tested so far. CIRCB also has a sequencing platform dedicated to HIV genotyping and activities related to the COVID-19 genomic surveillance platform in Cameroon.

Laboratory procedures

Sample collection and processing

All clinical samples were collected from individuals tested for COVID-19 diagnosis in Yaoundé. Briefly, nasopharyngeal swabs were collected, by trained personnel, in a 1mL tube containing viral transport medium as per manufacturer’s instructions and under universal biosafety measures. After collection, the samples were stored between 4°C and 8°C for 2-5 hours, and referred for laboratory processing.

Nucleic acid extraction, amplification and detection with Abbott platform

For Abbott rRT-PCR assay for SARCoV- 2 detection a 1000μL aliquot of each inactivated sample (500μL nasopharyngeal swab + 500μL of DNase-RNase free water heated at 70°C for 10min) was loaded into Abbott m2000sp instrument, combined with the Abbott SARS-CoV-2 master mix containing an internal RNA control, primers, and probes targeting both an RNA dependent RNA polymerase (RdRp) gene, specific for SARS-CoV-2 as well as the conserved structural protein nucleocapsid (N) gene (www.fda.gov/media/136258/download).

Amplification was performed using thermocycler m2000rt after automated extraction and sample preparation using Abbott m2000sp instrument. At the end of amplification process, negative results were rendered as “target not detected” (no amplification was observed after 37 cycle numbers); meanwhile positive results were rendered as “target detected” with a given number representing the Cycle Number (CN) at which the detection phase was initiated. This CN value was inversely proportional to the viral load of the patient. According to the manufacturer, detection sensitivity for this assay is 100 copies/mL.

Nucleic acid extraction, amplification and detection with DaAn gene platform

For DaAn gene rRT-PCR assay, viral RNA was manually extracted from 140μL nasopharyngeal swab using the QIAamp Viral RNA Mini Kit (Qiagen Inc, Valencia, CA, USA) as per manufacturer’s instructions. Amplification was performed using the DaAn gene RT-PCR kit (www.daangene.com) on the Quant Studio 5 (Thermofisher). The protocol used probes targeting the open reading frame (ORF1ab) gene and the nucleocapsid (N) protein gene, with a lower limit of detection of 500 copies/mL and an amplification reaction of 45cycles. As per the manufacturer’s instructions, each sample with a Cycle Threshold (CT) value <40 was considered positive, while a CT value >40 or “undetermined” (no amplification after 45 cycles) was considered negative; according to national guidelines in Cameroon, we also evaluated the positivity of DaAn gene at a CT value<37; considering the threshold for SARS-CoV-2 transmissibility, we further evaluated the positivity of DaAn gene at a CT value<35.

Statistical analyses

Statistical analyses were performed using Graph-Pad software version 6; correlation analyses were done using the spearman’s correlation test; the concordance in diagnosis was evaluated following the Cohen’s kappa (k) value, and results were interpreted according to the criteria proposed by Landis & Koch: k=0.01-0.20 (poor concordance), k=0.21-0.40 (fair concordance), k=0.41-0.60 (moderate concordance), k=0.61-0.80 (strong concordance), and k=0.81-1.00 (almost perfect concordance). [15] The diagnostic concordance was then compared according to different CTvalues (<40 as per the manufacturer’s instructions; <37 and <35 as aforementioned) of DaAn gene assay. All pvalue< 0.05 were considered statistically significant, with 95% confidence interval.

Ethics

The study has obtained ethical approval from the Cameroon National Ethics Committee for Human Health Research (reference N°01/143/CNERSH/SP). Per the Helsinki’s declaration and the national regulations, informed consent was obtained from all participants; confidentiality was ensured through de-identification by the use of a unique identifier for each participant and the storage of data in a password-protected computer.

Results

Sociodemographic and clinical features of the study participants

Out of the 273 participants enrolled in this study 57.88% (158/273) were males and 42.12% (115/273) females; the median (IQR) age was 36 (26–46) years; and about 4% (10/273) of the study population reported a COVID-19 related symptom.

Positive and negative concordance between Abbott and DaAn gene assays according to the cycle thresholds for SARS-CoV-2 molecular diagnosis

According to the various CT thresholds in SARS-CoV-2 molecular diagnosis of the DaAn gene assay (<35, <37, <40 as per the manufacturer’s instructions), a declining concordance was observed between Abbott and DaAn gene for CTs values <35 (k=0.84), <37 (k=0.82), <40 (k= 0.78), with similar concordance at CT-values of 35 and 37 (Table 1).

Table 1.

Positive and negative concordance between Abbott and DaAn gene at CT/CN: 35, 37 and 40.

	Percentage (%), Kappa
Concordance (CT<35)
Positive concordance (67/93)	72.04
Negative concordance (175/194)	90.20 k: 0.84 (95% CI: 0.795-0.883)
Concordance (CT<37)
Positive concordance (72/108)	66.66
Negative concordance (165/179)	92.18 k: 0.82 (95% CI: 0.777-0.863)
Concordance CT<40
Positive concordance (73/122)	59.83
Negative concordance (152/165)	92.12 k: 0.78 (95% CI: 0.735-0.820)
Concordance (EQC)
Positive concordance (9/9)	100
Negative concordance (5/5)	100

Considering the overall concordance with each CT-value, the CT<37 was considered as the suitable threshold for positivity and has been used for subsequent analyses in this study.

Concordance between Abbott and DaAn gene protocols

The Abbott real-time SARS-CoV-2 assay showed a positivity rate of 29.96% [95%CI: 25.0–35.5%] (86/287) versus 37.63% [95%CI: 32.2-43.4%] (108/287) for the DaAn gene assay. The overall percentage agreement between these two assays was 82.57% (237/287), with k=0.82 (95% CI: 0.78-0.86), indicating an excellent concordance between the two diagnostic protocols. The positive and negative concordance were 66.67% (72/108) and 92.18% (165/179) respectively (Table 2).

Table 2.

General Concordance between Abbott and DaAn gene.

DaAn gene	Abbott			Kappa (k, 95% CI)
	Positive	Negative	Total
Positive	72	36	108	0.82 (0.78 – 0.86)
Negative	14	165	179
Total	86	201	287

Concordance between Abbott and DaAn gene according to the presence or absence of COVID-19 related symptoms

The positivity rate of the Abbott and DaAn gene protocols in symptomatic patients were respectively 80% and 100%; and respectively 26.23% and 33.84% asymptomatic patients (Figure 1). Looking at the confidence intervals, no significant difference was found between Abbott and DaAn gene protocols according to whether or not the patient presented with a Covid-19 related symptom. In our study population, the concordance between Abbott and DaAn gene in symptomatic and asymptomatic patients were k=0.80 (95% CI: 0.80-0.80) and k=0.82 (95% CI: 0.77-0.86) respectively.

Figure 1.

Positive and negative rates between Abbott and DaAn gene using in asymptomatic and symptomatic patients.

Positive concordance between Abbott and DaAn gene assays according to viral load estimates

Regarding the viral load estimates, the lower the CT/CN values, the higher the concordance between these two tests. Specifically, 100% positive agreement was found between the two tests and the mean gene when CT was <20, 93.10% when the mean CT was 88.88% between [20-30], 66.67% with a mean CT of 60.61% between [30-35], 48.27% with a mean CT of 29.17% between [35-37], and 12% with a mean CT of 10% between [37-40] (see details in Table 3).

Table 3.

Positive concordance between Abbott and DaAn gene assays according to viral concentration with (ORF1ab, N and mean) genes.

CT value	gene ORF1ab			gene N			Mean (ORF1ab, N)gene
	DaAn gene,	Abbott	Concordance	DaAn gene,	Abbott	Concordance	DaAn gene,	Abbott	Concordance
	(n)	(n)	(%)	(n)	(n)	(%)	(n)	(n)	(%)
CT <20	12	12	100	16	16	100	14	14	100
CT [20-30]	29	27	93.10	36	27	75	33	29	88.88
CT [30-35]	24	16	66.67	41	24	58.54	33	20	60.61
CT [35-37]	29	14	48.27	17	5	29.41	24	7	29.17
CT [37-40]	25	3	12	12	1	8.33	30	3	10

Correlation between Abbott CN and DaAn gene CT values

We obtained strong positive correlations between Abbott CN and all DaAn gene viral targets’ CTs; with ORF1ab gene, we found r=0.76, p<0.0001 and 95% Confidence Intervals (CI) [0.74–0.89]; with N gene, we found r=0.80, p<0.0001 and 95% CI [0.72–0.88], we combined both viral targets’ CT values under a unique number and we labelled as “Mean” and obtained a good correlation (r=0.81; p<0.0001 and 95% CI [0.77–0.90]; Figure 2). Additionally, we found a good correlation (r=0.96 p<0.001) with N gene using positive samples of EQC panels (Figure 3).

Figure 2.

Correlation between Abbott CN and the DaAn gene CT values according to gene of interest.

Figure 3.

Correlation between Abbott CN and the DaAn gene CT values according to gene of interest among EQC.

CT and CN variation constant using the curve equation from spearman correlation

Using the curve equation from spearman correlations, we obtained different constants in relation with Abbott CN and DaAn gene viral targets (N, ORF 1ab, Mean and EQC controls see Table 4 and Figure 4). These constants enabled the extrapolation of our data in order to estimate interchangeability of the results between the two protocols among positive swabs according to the gene of interest.

Table 4.

Variation constants obtained through the curve equation from spearman correlation.

	Equation	Variation Constant
Abbott vs. Mean (N & ORF 1) gene	Y = 0,9202*X + 8,977	8,977
Abbott vs. N gene	Y = 0,9568*X + 7,427	7,427
Abbott vs. ORF 1ab gene	Y = 0,9055*X + 10,27	10,27
Abbott vs. Mean (N & ORF 1) gene (EQC)	Y = 0,9097*X + 10,95	10,95
Abbott vs. N gene (EQC)	Y = 0,9889*X + 10,23	10,23
Abbott vs. ORF 1ab gene (EQC)	Y = 0,8300*X + 11,69	11,69

Figure 4.

Estimated CT and CN obtained through the curve equation from spearman correlation. Using the equation from the spearman correlation shown in Table 3, we determined DaAn gene values based on given Abbott values and vice versa. With Y = DaAn gene CT and X = Abbott CN; with Y=AX + B and X=(Y-B)/A. (4.1) represents corresponding DaAn gene CT values obtained through fixed Abbott CN; (4.2) represents corresponding Abbott CN values obtained through fixed DaAn gene CT.

DaAn gene values was determinated based on a given Abbott values and vice versa. The difference in CT and CN between the two tests was on average 6.75±0.3.

Discussion

In the frame of ongoing COVID-19 pandemic,[16] SARS-CoV-2 diagnosis is critical for the detection of infected individuals, contact-tracing and mitigating the viral transmission.[12] In this context, identifying the concordance between commonly used assays such as automated (Abbott) and manual (DaAn gene) rRT-PCR assays has a public health significance. In this study, participants were relatively young (median age of 36 years), reflecting the demography of most African countries. According to gender distribution, females were less represented than males, suggesting a possible male vulnerability to COVID-19.[17-19]

According to the clinical status, only few participants were symptomatic patients, which could be partly explained by the youthfulness of our study population, but this also underlines the important circulation of asymptomatic cases (silent spreader of the disease) in the general population as reported by other authors.[20,21]

We found an excellent concordance between the Abbott and DaAn gene assays using the Cohen’s kappa coefficient (k=0.82), suggesting a good diagnostic agreement between the two platforms. Overall, this study revealed that DaAn gene is more sensitive in detecting SARS-CoV-2 infection in symptomatic and asymptomatic patients. Such considerations as the method of extraction (manual or automated), as reported by several authors,[12,13] may help to explain these results. The slight discrepancy observed between the two tests could be related to the quality of the PCR (component, target) and the detection limit, which may influence the performance of one test over the other.[22,23]

Additionally, the RdRp gene targeted by Abbott is produced by the ORF1b which is more exposed than the N gene to host immunity and thus more susceptible to mutations.[8,24]

Taking into consideration the CT<35, as well as the locally adopted thresholds (CT<37) and the manufacturer’s one (CT<40), the concordance was similar at CTs <35, <37, reinforcing the accuracy of the local guidelines in COVID-19 diagnosis. Besides, our results show strong positive correlations between Abbott CN and all DaAn gene viral targets’ CTs (N gene, ORF 1ab and Mean) when considering only positive results. The CT value obtained with the Mean value and N gene had a better correlation with Abbott CN than ORF1ab, probably because assay performance varies between PCR targets as reported previously.[23] Also, several authors reported good stability and high production of N gene among infected cells,[25,26] which may also justify the better correlation of their CT values with Abbott CN as compared to that of ORF1ab gene.

Interestingly, we obtained an excellent correlation using the EQC samples panels (r=0.96) on N gene, suggesting a better stability and reliability of synthetic samples compared to biological samples throughout the extraction and amplification processes in COVID-19 testing. In view of that, quality control gives confidence in diagnosis interpretation. These results, globally, suggest the interchangeability of these two assays regarding positive nasopharyngeal swabs. Moreover, extrapolation of our data using variation constants obtained from correlation equations in this study, according to gene of interest, enlighten more the consideration of a possible algorithm for the interpretation of the results among positive specimens in our country; especially as these two assays are the most used locally. In perspective, assessing the cost-effectiveness of the two assays, with reference to a gold standard (digital droplet PCR, etc) would be of great relevance.

Conclusions

The excellent agreement (>80%) between the Abbott and DaAn gene rRTPCR platforms supports inter-operability between the two assays in the diagnosis of SARS-CoV-2 in tropical settings like Cameroon. The acceptable rate of discordance (<20%) often occurs at low-VL (threshold with limited risk of viral transmission), thus supporting the efficiency of these assays in the global surveillance and control of COVID-19 pandemic.