Evaluation of SARS-CoV-2 diagnostics and risk factors associated with SARS-CoV-2 infection in Zambia.

OBJECTIVES: To conduct a diagnostic validation study of SARS-CoV-2 diagnostic kits.
METHODS: We compared SARS-CoV-2 diagnostic test results from 3 RT-PCR assays used by the Zambian government between November 2020 and February 2021 (Panther Fusion assay, Da An Gene's 2019-nCoV RNA kit and Maccura's PCR Kit) with the Altona RealStar RT-PCR kit which served as the gold standard. We also evaluated results from rapid antigen testing and whether comorbidities were linked with increased odds of infection.
RESULTS: We recruited 244 participants, 61% (149/244) were positive by at least 1 PCR assay. Da An Gene, Maccura, and Panther Fusion assays had sensitivities of 0.0% (95% confidence interval [CI] 0%-41%), 27.1% (95% CI 15%-42%), and 76% (95% CI 65%-85%), respectively, but specificity was low (<85% for all 3 assays). HIV and TB were not associated with SARS-CoV-2, whereas female sex (OR 0.5 [0.3-0.9], p = 0.026) and chronic pulmonary disease (0.1 [0.0-0.8], p = 0.031) were associated with lower odds of SARS-CoV-2 infection. Of 44 samples, 84% sequenced were Beta variant.
CONCLUSIONS: The RT-PCR assays evaluated did not meet WHO recommended minimum sensitivity of 80%. Local diagnostic validation studies should be embedded within preparedness plans for future outbreaks to improve the public health response.

Introduction

Despite the rapid development and adoption of SARS-CoV-2 diagnostics, SARS-

CoV-2 diagnostics have been deployed to resource-limited settings without rigorous evaluation. Given the need to ensure tracking of the epidemic and the lack of resources, countries in resource-limited settings end up using any available diagnostics. Attempts to establish an effective testing and tracking system have led to indiscriminate use of any available nucleic acid tests (Twohig et al.), antigen and/or antibody-based tests (Arevalo-Rodriguez et al., 2020, Kobia and Gitaka, 2020) regardless of their diagnostic performance.

In Zambia, SARS-CoV-2 testing has relied on PCR assays and antigen tests that have been donated through the African CDC (Maccura & Da An Gene RT-PCR assays and Abbott and Roche rapid antigen tests). The Zambian government then procured the Panther PCR assay with the national testing programme, switching between these 3 assays in accordance with availability. An early outpatient study showed that referral centre testing would be an essential pillar of the diagnostic response to the pandemic in Zambia (Hines et al., 2021). During November 2020–February 2021, we implemented an observational clinical diagnostic evaluation study at 2 COVID-19 referral hospitals in Lusaka, Zambia to evaluate the performance of these 3 donated PCR assays compared with the Altona Diagnostics CE-IVD certified RealStar SARS-CoV-2 RT-PCR assay, which received FDA EUA on the April 22, 2020 (Freire-Paspuel et al., 2021) and was the back-bone of the German government's testing system early in the pandemic. It has been reviewed extensively and has proven to be robust, sensitive and versatile (van Kasteren et al., 2020, Visseaux et al., 2020).

Multiple variants of SARS-CoV-2 have been documented worldwide during this pandemic. Several new SARS-CoV-2 variants recently reported, Alpha, VOC B.1.1.7, Beta, VOC B.1.351, Gamma VOC B.1.1.28 and Delta VOC B.1.617.2. The Alpha, Beta, Gamma, and Delta variants were associated with high transmission, severe illness, and increased mortality (Bal et al., 2021, Challen et al., 2021, Davies et al., 2021, Gaymard et al., 2021, Twohig et al., 2021). More recently, the Omicron variant has shown increased potential for immune evasion in various studies (Wang et al., 2022, Zhang et al., 2021). This study took place during ‘Wave 2’ of the pandemic in Zambia, which coincided with the emergence and global spread of the Beta variant of SARS-CoV-2.

The primary objective of this study was to assess intertest agreement and other important operational characteristics of the diagnostic kits being used in Zambia. Secondary objectives included analysing whether comorbidities (such as tuberculosis or HIV) were associated with infection or poorer outcomes. Whole genome sequencing was used to determine variants that were circulating within our patient population at the time of the study.

Materials and Methods

Ethics statement

Informed written consent was obtained from all the participants. The study was approved by ERES Converge, Zambia (Ref No. 2020-JUL-07).

Study population and patient recruitment

The study was conducted at 2 COVID-19 referral hospitals in Lusaka, Zambia; the University Teaching Hospital (UTH), and Levy Mwanawasa University Teaching Hospital (LMUTH) between November 2020 and February 2021. All individuals aged >18 years, attending accident and emergency with suspected SARS-CoV-2 infection were eligible for the study. After obtaining informed consent, they were enrolled in the study and underwent a questionnaire to gather patient demographics, symptoms, underlying conditions and comorbidity, and recent travel history.

Specimen collection and laboratory analysis

A nasopharyngeal swab and blood sample were collected from each enrolled participant. One nasopharyngeal swab specimen was collected and placed in 3 ml of viral transport media for rapid antigen testing and parallel RT-PCR testing by the government laboratories using any 1 of 3 RT-PCR assays used by the national testing programme at the time: the Panther Fusion assay, Da An Gene's 2019-nCoV RNA kit, and Maccura's PCR Kit and by our research laboratory, using the RealStar SARS-CoV-2 RT-PCR kit (Altona Diagnostics GmbH, Germany).

Rapid antigen testing

Testing was done at the Zambian COVID-19 testing sites using either the Abbott Panbio COVID-19 Ag Rapid Test Device (Abbott Diagnostic GmbH, Jena, Germany) or the Roche SD Biosensor SARS-CoV-2 Rapid Antigen Test Nasal Test (Roche Diagnostics, Basel Switzerland), following manufacturer's guidelines.

RT-PCR testing

Altona SARS-CoV-2 Assay

RNA extraction for our RT-PCR testing was done using the QIAamp Viral RNA Mini extraction kit (QIAGEN GmbH, Hilden, Germany), following the manufacturer's instructions. RT-PCR set up was done with 10 μl of the RNA template in a 30 μl final reaction. The RealStar SARS-CoV-2 RT-PCR kit (altona Diagnostics GmbH, Germany), which targets the E and S genes of SARS-CoV-2, was used for RT-PCR testing on the Rotor-Gene 6000 cycler (QIAGEN GmbH, Hilden, Germany).

For the government laboratories, PCR testing was done using the available kit at the time and according to the manufacturer's instructions for the specific assay. RNA extraction was done using QIAamp Viral RNA Mini extraction kit (QIAGEN GmbH, Hilden, Germany) and the PCR was set up on a Light Cycler 480 or ABI7500 system. The 3 RT-PCR assays used were Aptima Panther Fusion SARS-CoV-2 assay (Hologic, Inc, San Diego, USA), which targets the ORF1ab gene; the Maccura SARS-CoV-2 assay (Maccura Biotechnology Co., Chengdu, P.R. China), which targets ORF1ab, E and N genes, and the Da An Gene SARS-C0V-2 assay (Daan Gene Co., Guangzhou, Guangdong, P.R. China), which targets the ORF1ab and N genes.

SARS-CoV-2 sequencing & analysis

SARS-CoV-2 genomes were sequenced in our laboratory by tiling PCR and Oxford Nanopore NGS sequencing methods. Bioinformatic analysis was done using the ARTIC pipeline, as described elsewhere (Manouana et al., 2021), and the lineages were obtained using Pangolin. In brief, tiling PCR was used to amplify 1200 bp fragments in 2 pools, covering the SARS-CoV-2 genome. Amplicons were purified using AMPure bead purification and barcoded using Rapid Barcoding kit (SQK-RBK004) from Oxford Nanopore and purified again before being combined into a full library and loaded onto the MinION for sequencing. Guppy version 3.6.0 was used for base-calling and demultiplexing all runs. The ARTIC Network bioinformatics protocol was used for all genome assembly and variant calling steps, and the lineages were obtained using Pangolin tool (O'Toole et al., 2021). All genomes were aligned with Wuhan-Hu-1 strain (NC_045512.2) using multiple alignment fast Fourier transform algorithm (Katoh et al., 2002), and the subsequent phylogenetic tree was constructed with the maximum likelihood method with 1000 bootstrap iterations using the general time-reversible (GTR) model with rate heterogeneity (GTR+G) in the IQ-TREE server (Trifinopoulos et al., 2016). The final dataset was displayed using the interactive tree of life (iTOL v6).

Anti-SARS-CoV-2 antibody screening

The presence of SARS-CoV-2 antibodies was determined using the Wantai SARS-CoV-2 total Ab ELISA kit (Wantai Biological, Beijing, China) according to the manufacturer's instructions. Absorbance reading were obtained using the BioTek EL800 microplate reader (BioTek, Winooski, USA) at 450 nm wavelength. Samples were considered to have anti-SARS-CoV-2 antibodies if the absorbance value was greater than 0.03.

Data analysis

Laboratory and clinical data were entered on the EpiInfo version 7.2.4.0 (CDC, USA) and exported as a.csv file. It was imported to RStudio version 1.2.5019 (R Core Team, 2019) and cleaned. All statistical analyses were done using packages and functions in RStudio version 1.2.5019 (R Core Team, 2019). Graphics were produced using the ggplot2 package and all confidence intervals (CIs) were reported at 95% level (Wickham, 2009). A multivariable logistic regression analysis to investigate comorbidities associated with SARS-CoV-2 infection (either Altona RealStar PCR positive or negative) was done using the R package stats (R Core Team, 2019). Sex and age were added as a fixed effect to the logistic regression and all the recorded comorbidities (hypertension, HIV, diabetes, tuberculosis, chronic obstructive pulmonary disease [COPD], asthma, obesity, renal disease, cardiac) and mortality outcome were used as explanatory variables.

Results

Recruitment and Cohort Descriptives

We recruited 244 patients with suspected COVID-19, attending either the University Teaching Hospital (UTH) or Levi Mwanawasa University Teaching Hospital (LMUTH), Lusaka, Zambia. Patients were recruited between November 24, 2020 and February 5, 2021, which coincided with the second wave of the COVID-19 pandemic in Zambia (Fig. 1 ). The COVID-19 isolation ward at LMUTH was established as the primary centre for COVID-19 treatment and care. Full-capacity was soon reached and so, as cases dramatically increased, UTH recruited 134 patients during the first 2 weeks of January, accounting for 55% of all recruits.

Fig. 1

Timeline showing number of patients recruited each week from each of the participating hospitals, along with the overall trajectory of the pandemic (cases/day) within Zambia nationally during the same period. UTH, University Teaching Hospital, LMUTH, Levi Mwanawasa University Teaching Hospital.

As both hospitals quickly reached capacity, home care was established, with patients triaged, and those with high oxygen saturation sent home to self-care with pulse oximeters. As such, only half of participants (48%; 115/244) were admitted to the hospital (Table 1 ).

Table 1

Characteristics of 244 participants attending hospital with suspected COVID-19.

	Comorbidity n= 89	No comorbidity n= 155	All participants (%) n= 244
Characteristic
Median Age in Years (IQR)	46 (36–61)	35 (28 – 43)	38 (30–50)
Sex (male)	46	83	129 (53%)
Admitted	62	53	115 (47)
Symptoms
Cough	72	121	193 (79)
Shortness of Breath	65	72	137 (56)
Sore Throat	37	71	108 (44)
Headache	20	57	77 (32)
Chest Pain	19	30	49 (20)
Diarrhoea	13	17	30 (12)
Nausea	15	15	29 (12)
Runny Nose	11	18	29 (12)
Loss of Taste	5	13	18 (7)
Median Temperature °C (IQR)	36.5 (36–37)	36.5 (36 – 37)	36.5 (36 – 37)
Days since onset of symptoms*
0–3 Days	27	62	89 (38)
4–7 Days	30	57	87 (37)
>= 8 Days	31	30	61 (26)
SARS-CoV-2 antibodies present#	33	58	91 (37)
Altona RT-PCR Positive	42	92	134 (55)
Median RT-PCR ct value (IQR)	27.5 (21.6 – 32.7)	28.1 (22.0 – 32.3)	27.8 (21.7 – 32.4)

data missing for 7 participants

antibodies detected with the Wantai SARS-CoV-2 total Ab ELISA kit. IQR, Interquartile range

IQR, interquartile range; RT-PCR, real-time PCR.

The median age of the study cohort was 38 years (IQR 30–50 years), and 53% (130/244) were male (Table 1). The most common symptom was persistent cough, affecting 79% (193/244) of participants, followed by shortness of breath (56%), sore throat (44%) and headache (32%). Loss of taste was relatively uncommon, being reported by just 7.4% (18/244) of participants (Table 1). The median temperature was 36.5 °C, with 21% (51/244) recorded as having fever. A total of 21% (48/244) of participants reported exposure to a known COVID-19 case and 74% (176/237) were recruited within 7 days of symptom onset (Table 1).

Diagnostic assay performance

Among the PCR assays used, the Altona RealStar RT-PCR assay had the highest yield, being positive in 55% (134/244). The yields of the 3 RT-PCR assays used by the government were highly variable, ranging from 3%–55%. In total, 61% (149/244) of participants had a positive PCR result from at least 1 RT-PCR assay. The rapid antigen test had a lower yield than the 2 leading PCR assays, being positive in only 33% (54/163) of participants (Table 2 ).

Table 2

Comparison between RT-PCR assays and rapid antigen test.

		Altona RealStar		Diagnostic Performance characteristics
		Negative	Positive	Sensitivity	Specificity	PPVa	NPVa
Da An Gene	Negative	31	7	0.0% (0–41%)	96.9% (84–100%)	0.0%	81.6% (43–46%)
	Positive	1	0
Maccura	Negative	26	35	27.1% (15–42%)	83.9% (66–95%)	67.3% (45–84%)	48.4% (43–54%)
	Positive	5	13
Panther Fusion®	Negative	37	19	76% (65–85%)	80.4% (66–91%)	82.7% (72–90%)	73.2% (64–81%)
	Positive	9	60
Ag RDT	Negative	59	48	44.8% (34–56%)	79.7% (69–88%)	73.1% (62–82%)	54% (49–60%)
	Positive	15	39
All gov't assays#	Negative	87	44	67% (56–75%)	79.1% (70–86%)	80% (71–87%)	66.4% (58–74%)
	Positive	23	90

Based on a prevalence of 55.1%

Positive case defined as positive on any of the RT-PCR assays used by the government or rapid antigen test. A positive result either on the rapid Ag or RT-PCR assays was notified as a confirmed COVID-19 case by the government.

Ag RDT, antigen detection rapid diagnostic test; gov't, government; NPV, negative predictive values; PPV, positive predictive values; RT-PCR, real-time PCR.

As the Altona RealStar assay had the highest yield, we used this as a proxy gold standard, against which to compare the other PCR assays and the antigen detection rapid diagnostic test (Ag RDT). Of the 3 assays used by the Zambian government, the Da An Gene assay performed extremely poorly, failing to detect a single positive case, resulting in a sensitivity of 0% (95% CI 0%–41%). The Maccura assay was also extremely insensitive, with a sensitivity of just 27% (95% CI 15%–42%). The Panther Fusion assay performed better with a sensitivity of 76% (95% CI 65%–85%). But the specificity of both the Panther Fusion and Maccura assays was low at 80% (95% CI 66%–91%) and 84% (95% CI 66%–95%), respectively (Table 2).

The Ag RDT test performed poorly, with a sensitivity of just 45% (95% CI 34%–56%) and specificity of just 80% (95% CI 69%–88%) (Table 2). Performance of the Ag RDT test varied depending on when patients presented themselves for testing after the date of onset of symptoms. The sensitivity and specificity were highest in participants presenting at the hospital within 3 days from onset of symptoms (53%; 95% CI 34%–72% and 84%; 95% CI 66%–95%, respectively) (Table 3 ). Test performance also improved when the Ag RDT test was evaluated among 56 participants with a high viral load (defined as an Altona RealStar RT-PCR cycle threshold [Ct] value less than 30); sensitivity was 67%; 95% CI 51%–79% and specificity was 100%; 95% CI 54%–100% (supplementary table).

Table 3

Sensitivity and Specificity of the Rapid Antigen test stratified by the number of days since symptoms onset.

		Altona RealStar		Diagnostic performance characteristics
Days since onset of symptoms⁎⁎	Ag RDT*	Negative	Positive	Sensitivity	Specificity
0–3 Days	Negative	26	14	53.3% (34%–72%)	83.9% (66%–95%)
	Positive	5	16
4–7 Days	Negative	17	20	42.8% (26%–61%)	68% (47%–85%)
	Positive	8	15
>= 8 Days	Negative	13	14	36.4% (17%–59%)	92.9% (66%–99.8%)
	Positive	1	8
All	Negative	60	48	44.8% (34%–56%)	79.7% (69%–88%)
	Positive	15	39

Rapid antigen test not done for 82 participants.

Number of days since the onset of symptoms unknown for 7 participants.

Ag RDT, antigen detection rapid diagnostic test.

The median Ct value of confirmed SARS-CoV-2 cases on the Altona RealStar assay was 27.8 (IQR 21.7–32.4). When we compared the mean Ct value of the missed government SARS-CoV-2 cases (false negatives) to the true positives, it was significantly higher (30 vs 24; p <0.01) and 66% of the missed cases had a Ct value greater than 30 (Fig. 2 ).

Fig. 2

A Plot of the Ct value (Altona assay) of confirmed SARS-CoV-2 cases showing the difference between false negatives (cases missed by the government RT-PCR assays) and true positives (participants positive on both the government RT-PCR assays and Altona RealStar assay). RT-PCR, real-time PCR.

Seroprevalence

Overall, SARS-CoV-2 antibodies were detected in 37% (91/244) of participants (Table 4 ). Seroprevalence did not differ significantly between RT-PCR positives (44%, 59/134) and RT-PCR negatives (29%, 32/110), and when stratifying by days since onset of symptoms, seroprevalence did not differ significantly by PCR status (Table 4).

Table 4

Seroprevalence by Wantai ELISA stratified by Altona RealStair RT-PCR result and days since onset of symptoms.

Days since onset of symptoms⁎⁎	Altona RealStar	Wantai ELISA seropositive
0–3 Days	Negative	27% (12/45)
	Positive	36% (16/44)
4–7 Days	Negative	29% (12/41)
	Positive	39% (18/46)
>= 8 Days	Negative	32% (6/19)
	Positive	60% (25/42)
All participants	Negative	29% (32/110)
	Positive	44% (59/134)

Number of days since the onset of symptoms unknown for 7 participants.

RT-PCR, real-time PCR.

SARS-CoV-2 detection & comorbidity

Hypertension, HIV, diabetes, tuberculosis (Tegally et al.) and COPD were the most common comorbidities and together accounted for 78.2% (68/87) of participants with comorbidities. In univariate binary logistic regression analysis, HIV, TB, and COPD were all associated with a reduced odds of being SARS-CoV-2 PCR positive, with ORs ranging from 0.1–0.3 (Table 5 ). In a multivariate regression model that included sex and age along with the highlighted comorbidity variables, being female (OR 0.5, 95% CI 0.3–0.9) and having COPD (OR 0.1, 95% CI 0.0–0.8) were associated with reduced odds of being SARS-CoV-2 positive (Table 5).

Table 5

Binary logistic regression analysis of various comorbidities as risk factors for being SARS-CoV-2 PCR positive.

	Altona RealStar				Univariate		Multivariate
	Negative		Positive		OR (95% CI)	p	OR (95% CI)	p
Median Age (IQR)	37 (29–48)		40 (30–52)		1.0 (1.0–1.0)	0.207	1.0 (1.0–1.0)	0.211
Sex (female)	58/109	53.2%	56/134	41.8%	0.6 (0.4–1.1)	0.077	0.5 (0.3–0.9)	0.026
Any Comorbidity	46/109	42.2%	41/134	30.6%	0.6 (0.36–1.0)	0.061
Hypertensive	17/109	15.6%	33/134	24.6%	1.8 (0.9–3.4)	0.086
HIV	19/109	17.4%	8/134	6.0%	0.3 (0.1–0.7)	0.007	0.4 (0.2–1.1)	0.086
Diabetes	7/109	6.4%	8/134	6.0%	0.9 (0.3–2.6)	0.884
Tuberculosis	10/109	9.2%	3/134	2.2%	0.2 (0.1–0.8)	0.027	0.4 (0.1–1.6)	0.189
CPD	8/109	7.3%	1/134	0.7%	0.1 (0.0–0.8)	0.028	0.1 (0.0–0.8)	0.031
Asthma	3/109	2.8%	2/134	1.5%	0.5 (0.1–3.3)	0.498
Obesity	1/109	0.9%	3/134	2.2%	2.5 (0.3–24)	0.436
Renal Disease	2/109	1.8%	1/134	0.7%	0.4 (0.0–4.5)	0.460
Cardiac	1/109	0.9%	2/134	1.5%	1.6 (0.1–18.3)	0.689
Outcome v Dieda	2/80	2.5%	1/85	1.2%	0.5 (0.0–5.2)	0.534

outcome data was only available for patients who were admitted (n=165)

CPD, chronic pulmonary disease.

SARS-CoV-2 Variants

Only samples that had a Ct value of below 30 and produced a complete consensus sequence on analysis that coverage of over 90% of the genome were submitted to GISAID and are presented here. More than 3/4 of the samples sequenced (84.1%; 37/44) were the Beta variant (B.1.351 Pangoline lineage), and 4 were B.1.306, with 1 sample each being B.1.1.7 and B.1.404. A total of 16 of the 44 samples sequenced were missed by all the 3 government PCR assays, confirming these as true positives. Phylogenetic analysis was undertaken on 44 samples from this study, and 185 Zambian sequences that have been published by other research groups, representing all SARS-CoV-2 sequence data available from samples sequenced in Zambia during the study period. (Fig. 3 )

Fig. 3

Phylogenetic Tree Maximum likelihood phylogenetic tree of currently available SARS-CoV-2 genomes from the Republic of Zambia collected during the study period Nov 2020 to Feb 2021. Coloured in accordance with SARS-CoV-2 variant type. Coloured id labels on nodes indicate samples sequenced by laboratory during the course of the study. All the full-length genomes retrieved from the GISAID (global database for influenza gene sequences) labelled as country of origin, GISAID ID. Branch lengths are drawn according to the number of nucleotide substitutions per site.

Discussion

The main aim of this study was to validate the diagnostics tests used by the Zambian government laboratories for the detection of SARS-CoV-2 infections against the Altona RealStar RT-PCR assay as gold standard. The overall proportion of RT-PCR positive SARS-CoV-2 cases, positive on any of the RT-PCR assays, was 61.1% (149/244); and the reference assay, the Altona RealStar SARS-CoV-2 RT-PCR kit, was the most sensitive. Of the 3 government RT-PCR assays used at the time, the Panther Fusion RT-PCR assay was most accurate (sensitivity 76% and specificity 80%), whereas the Da An Gene was least accurate (sensitivity was 0% and specificity 96.9%). Using the Altona assay as a gold standard, none of the 3 RT-PCR assays evaluated met the WHO recommended minimum sensitivity of 80% and specificity of 97% (WHO, 2020).

The analytical performance characteristics of these assays provided by the manufacturers does not correlate well with how the assays performed in a real-world clinical setting (Doust et al., 2021), consistent with their rapid development and deployment in the absence of clinical evaluations necessitated by the pandemic. Possible factors which might affect the operational performance in a real-world setting include variations in sample quality, transport and storage, and human factors such as how samples are collected and processed by both clinical and biomedical personnel (Doust et al., 2021, Fung et al., 2020). This study has illustrated the importance of local clinical validation and assay verification to characterise the performance of a diagnostic test in a specific clinical setting. For novel emerging pathogens, there is no endemic population within which novel diagnostics can be readily evaluated, and so pandemic preparedness planning should include skeleton protocols for the rapid validation of diagnostic assays.

Molecular diagnostic development is traditionally a slow and methodical process. When running 2 different tests on the same set of samples, there will always be some degree of discordance and so, careful work-up is required to elucidate the reasons for differences and to determine true positives. The SARS-CoV-2 pandemic has challenged this modus operandi and has showed that the rapid development and deployment of reliable molecular diagnostic assays is a central pillar of the pandemic response. At the time of study design and implementation, there was very little data available on the performance of the diagnostic assays being used, and the results of this study were reported in real time to the Zambian National Public Health Institute, which led to discontinuing the use of the Da An Gene assay that was being used in many African countries. At the time of writing, only 1 study had evaluated its use in Benin (Sander et al., 2021). The authors observed good analytical performance characteristics (using synthetic armoured transcripts) but poor clinical performance, which is consistent with the poor clinical performance observed in our study.

In the study herein, the median Ct value of the SARS-CoV-2 cases missed by the government assays was significantly higher than the median Ct value of the true positives (Fig. 2), indicating the poor performance might be due to low sensitivity. Conversely, a study from Ecuador reported a higher sensitivity of the Da An Gene assay (75%–100%) than that of the CDC 2019-nCoV CDC EUA assay which was considered as gold standard (Freire-Paspuel et al., 2021). This alternative gold standard has a higher LoD of 1000 copies/ml compared with 650 copies/ml for the Altona assay (Freire-Paspuel et al., 2021, Visseaux et al., 2020), and/or there could also be logistical/operational factors which contributed to the discrepancy between the 2 studies. With the rapid commercializing and scale up of manufacture, there could well have been quality control issues, which affected the performances of certain batches.

SARS-CoV-2 is evolving in both human and animal populations (Lauring and Hodcroft, 2021, Tegally et al., 2021) and when mutations occur in primer or probe sequences, this can impact assay performance (Artesi et al., 2020). Altona Diagnostics have not yet reported any mutations that they think might affect their assay, including for the recent Omicron variant (Diagnostics, 2021). The Altona RealStar assay targets both the E and S genes of the SARS-CoV-2 genome. The Aptima Panther Fusion assay targets ORF1ab; the Maccura assay targets ORF1ab, E, and N genes and the Dan An Gene assay targets the ORF1ab and N genes. We did not observe any probe failures with the Altona RealStar assay but mutations have the capacity to alter diagnostic assay performance, as has been widely documented for certain variants of concern (Valley-Omar et al., 2022, Wollschlager et al., 2021). This reinforces the need for assays which detect multiple targets and the broader need for genomic surveillance during pandemics with novel viral pathogens.

In evaluating diagnostics assays, the gold standard or reference test used must be accurate, reliable, efficient, highly sensitive and very robust to ensure the cases are correctly determined as either positives or negatives, and it should be appropriate for the population being tested (Doust et al., 2021). All study participants had 1 or more COVID-19 symptoms and were within 2 weeks of symptom onset, a period when the virus should typically be detectable by RT-PCR and antigen screening assays (He et al., 2020, Wolfel et al., 2020). Hence, the cohort of participants used was appropriate for evaluating SARS-CoV-2 diagnostic assays. The Altona Diagnostics RealStar SARS-CoV-2 RT-PCR assay has been extensively reviewed and found to be robust, versatile, and highly sensitive in detecting SARS-CoV-2 infections (van Kasteren et al., 2020, Visseaux et al., 2020). It can detect as low as 625 viral copies/mL compared with 1250 copies/mL LOD for most approved PCR assays (Visseaux et al., 2020). Moreover, the WHO recommends the use of a nucleic acid amplification test as the gold standard test to evaluate SARS-CoV-2 screening assays (WHO, 2020). Hence, the Altona RealStar assay was a credible reference test to use. We sequenced 16 of the samples that were positive only on the Altona assay but negative by the government assays, demonstrating that these were true positives.

The antigen tests evaluated in this study were not reliable in detecting SARS-CoV-2 infections in the general population as both the sensitivity (45%) and specificity (78%) were below the WHO recommended sensitivity of 80% and specificity of 97% (WHO, 2020). In other studies, the sensitivity of rapid antigen tests varies between 45% and 84.9% (Albert et al., 2021, Igloi et al., 2021, Lambert-Niclot et al., 2020, Linares et al., 2020, Osterman et al., 2021) and specificity is typically >99% (Albert et al., 2021, Igloi et al., 2021). There has been much debate about the use of less sensitive lateral flow antigen rapid tests compared to RT-PCR, with some arguing that many RT-PCR-positive cases might not be infectious, and that a less sensitive rapid Ag test is a better tool for identifying those who are at the highest risk of infecting others (Mina et al., 2021, Tom and Mina, 2020). The counter argument is that with a test from just 1 time point, you are unable to know whether the viral load might increase and so, RT-PCR is the only effective way to identify a sufficient number of infectious cases, to inform on isolation, and stop transmission.

The seroprevalence data from our study indicated that a significant minority of both PCR+ve and PCR-ves, had existing antibody to SARS-CoV-2, suggesting previous infection within wave 1, and/or possible cross-reactivity of the ELISA assay used with immunity to other circulating viruses. The overall seroprevalence of 37% among suspected COVID-19 cases is consistent with a community survey undertaken 6 months before the study, which reported seroprevalence of 9% in Lusaka district (Mulenga et al., 2021).

In our study, 78% (68/87) of the participants presented with hypertension, HIV, diabetes, Tuberculosis (Tegally et al.), and chronic pulmonary disease. Suspected cases who had HIV were not at increased risk of SARS-CoV-2 positivity, consistent with previous studies (Charre et al., 2020, Friedman et al., 2021, Inciarte et al., 2020). Our study was underpowered to evaluate whether HIV-infected cases had worse outcomes, but studies elsewhere have suggested HIV is not associated with worse outcomes (Cooper et al., 2020, Nagarakanti et al., 2021). Our observation that COPD was associated with a reduced risk of being SARS-CoV-2 positive (OR 0.1 95% CI 0.0–0.8) had a very wide 95% CI and was likely a sample size artefact. A comprehensive review found that COPD was associated with worse outcomes in COVID-19 patients (Leung et al., 2020).

Limitations of the study

The findings of our study are limited to symptomatic suspected SARS-CoV-2 cases and cannot be extrapolated to asymptomatic cases where diagnostic assay performance might vary. We could not reliably match the specific rapid antigen test used by the government laboratories to the results, and they ran out of Ag test kits during the course of the study, limiting the statistical power of the antigen test evaluation. The study was implemented during the exponential rise of cases during the second wave of the pandemic and changes in government advice/policy could have affected health seeking behaviour and clinical practice during the study.

Conclusions

The RT-PCR assays evaluated did not meet WHO recommended minimum sensitivity of 80%. This highlights the need for all governments to ensure that local plans for diagnostic validation are incorporated into pandemic preparedness planning. Molecular diagnostics have been pivotal in managing the SARS-CoV-2 pandemic and in Zambia and other countries; capacity should be maintained/developed to respond to future zoonoses and could also support much needed surveillance for ongoing endemic infectious disease threats such as antimicrobial resistance. The apparent negative association between female sex and COPD with SARS-CoV-2 had wide confidence limits and should be interpreted with caution.

Disclosures

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Funding Source

This work was funded by the European and Developing Countries Clinical Trials Partnership (EDCTP2) programme under the PANDORA-ID-NET Consortium (EDCTP Reg/Grant RIA2016E-1609). They had no role in the writing, study design; collection, analysis and interpretation of data, and decision to submit the article for publication.

Ethics statement

Informed written consent was obtained from all the participants. The study was approved by ERES Converge, Zambia (Ref No. 2020-JUL-07), in compliance with the laws of Zambia and the standards of Elsevier journals standards of ethics.