Evaluation of the performance of Panbio™ COVID-19 antigen rapid diagnostic test for the detection of SARS-CoV-2 in suspected patients in Ethiopia.

Objective: Reverse transcription-polymerase chain reaction is a gold standard diagnostic tool for coronavirus disease-2019. Limited coverage and long turnaround times are linked to the poor response to the pandemic in developing countries like Ethiopia. To overcome the challenges, rapid antigen diagnostic kits are recommended if their diagnostic performance is at an acceptable level. We explored the performance of the Panbio™ coronavirus disease-2019 antigen rapid diagnostic test in diagnosing the coronavirus disease-2019 infection.
Methods: A cross-sectional study was conducted on coronavirus disease-2019 suspected patients in Wollega University Referral Hospital, from 1 April to 30 May 2021. After obtaining consent/ assent, sociodemographic and pair of nasopharyngeal samples were collected from each and examined by Panbio antigen rapid diagnostic test and reverse transcription-polymerase chain reaction. Data were entered and analysed using SPSS version 24. Sensitivity, specificity, positive and negative predictive values, and kappa values were calculated.
Results: A total of 148 coronavirus disease-2019 suspected individuals (54.1% male) participated in the study. Of all, 73 (49.3%) were positive for severe acute respiratory syndrome corona virus by reverse transcription-polymerase chain reaction test. The sensitivity and specificity of Panbio were found 81% (95% confidence interval: 71%-91%) and 98.7% (95% confidence interval: 96%-100%), respectively. From 75 negative and 73 positive samples by reverse transcription-polymerase chain reaction, 1 (1.33%) and 14 (19.18%) were found false positive and negative by antigen rapid diagnostic test, respectively. Positive predictive value and negative predictive value of Panbio were 98.3% and 84.1%, respectively, and test agreement was substantial (kappa value = 0.80).
Conclusion: Panbio has fine performance in suspected patients. Further studies are needed to examine the accuracy of self-collecting and patient self-testing with healthcare workers, using antigen rapid diagnostic test against the reference standard.

Introduction

Coronavirus disease-2019 (COVID-19) is a pandemic acute respiratory disease that affects primarily the upper respiratory tract and follows the lower respiratory tract damage. The disease may remain an asymptomatic infection in many or mild flu-like illnesses and may lead to severe illnesses or a few deaths among patients with underlined conditions and obesity or elderly patients. It is caused by the severe acute respiratory syndrome corona virus (SARS-CoV-2) which was first detected in Wuhan, China in December 2019. SARS-CoV-2 is highly contagious and can spread without symptoms in a short time, resulting in a pandemic. The rapid spread of the virus is attributed to its silent and unforeseen transmission, for which infected patients can be asymptomatic or exhibit only flu-like signs in the early stage. Human-to-human transmission occurs primarily through respiratory droplets and to a lesser extent via aerosols. A person may be infected when within a metre of someone who has COVID-19 respiratory symptoms, such as coughing or sneezing, that result in his or her mucosae (mouth and nose) or conjunctiva (eyes) being exposed to potentially infective respiratory droplets.

Globally, the cases and deaths of COVID-19 have been increasing. As of 19 September 2021, nearly 228 million and over 4.6 million cumulative numbers of confirmed COVID-19 cases and deaths were reported, respectively, worldwide, whereas 332,961 confirmed cases and 5130 deaths were reported as a country in Ethiopia. Even though Ethiopia remains the highest burdened country with confirmed cases in the East Africa region, by accounting for 36.7% of the total cases in the region, the reported cases have still underestimated the overall burden, as merely a fraction of acute infections are diagnosed and reported. Serological surveys have suggested that the rate of prior exposure to SARS-CoV-2 exceeds about 10-fold or more than the rate of reported cases as reflected through seropositive. Undetected cases can cause an exceptional challenge to the containment of the virus and pose an awful threat to public health. To combat the COVID-19 pandemic, affordable and scaled-up laboratory services in diagnostic capacity and accuracy, result reporting rapidly and monitoring of the virus’s genetic changes are very important. The rapid identification of newly infected individuals and detection of new viral variants that may affect transmissibility, pathogenesis or severity of infection can be helpful to implement control and preventive public health measures as early as possible.

Currently, various COVID-19 diagnostic tools, including rapid diagnostic tests, have been introduced. They are designed to detect nucleic acids or antigens of SARS-CoV-2 and antibodies produced against the virus to determine if they were previously infected with SARS-CoV-2. Reverse transcription-polymerase chain reaction (RT-PCR) test that detects viral RNA from nasopharyngeal or oropharyngeal swabs or other upper respiratory tract samples is the gold standard test method for the detection of the SARS-CoV-2. However, in Africa, including Ethiopia, limited coverage due to the cost and long turnaround times of RT-PCR lags behind the COVID-19 pandemic response by far. The World Health Organization (WHO), as a result, has announced for low- and middle-income countries to avail 120 million antigen rapid diagnostic test (AgRDT) kits to those who lack the resources to implement national RT-PCR testing strategies. The AgRDT kits are useful and promising point-of-care alternatives because they are easy to run and offer results more rapidly (approximately 15–30 min) at a lower cost than doing highly sensitive nucleic acid amplification tests. Moreover, they help to expand SARS-CoV-2 test coverage that allows limiting the transmission by more rapidly identifying infectious persons for isolation. However, studies from early on cautioned against the use of AgRDT given lower sensitivity compared with RT-PCR. Therefore, the performance of the RDT kits needs to be evaluated and monitored periodically and during the introduction of new kits. Similarly, the RDT results have to be interpreted with vigilance, by recommending a confirmatory RT-PCR test following a negative AgRDT test, mostly in patients with high pretest probability. Most studies have previously evaluated AgRDTs for SARS-CoV-2 detection from nasopharyngeal or nasal swab specimens from symptomatic patients with moderate or severe symptoms. Nowadays, more studies that would evaluate the performance of AgRDTs to detect SARS-CoV-2 are needed to be carried out in the different settings of public health centres or in the community.

The Panbio™ is a WHO-recommended qualitative AgRDT for the detection of SARS-CoV-2 using specimens from nasopharyngeal swabs. The indicated nasopharyngeal swab versus nasopharyngeal RT-PCR sensitivity was 91.4% (94.1% for samples with cycle threshold (Ct) values ⩽ 33) and specificity was 99.8%. This study aimed to evaluate the performance of the Panbio AgRDT at a test site in symptomatic patients and close contacts, using the RT-PCR test as the gold standard.

Methods

Study setting and study design

A cross-sectional study was conducted in Wollega University Referral Hospital, East Wollega Zone Oromia region, Ethiopia from 1 April to 30 May 2021. The hospital has a COVID-19 diagnostic laboratory which is equipped with QuantStudio™ 5 Real-Time PCR System. The PCR was the WHO-recommended standard molecular method or RT-PCR. The laboratory has a national COVID-19 diagnosis laboratory certification standard from the Ethiopian Public Health Institute (EPHI) to undertake COVID-19 diagnosis.

Sample size estimation

The WHO predetermined minimum values of sensitivity (80%) and specificity (97%) of AgRDT were used to estimate the minimum sample size (n). The sample size was determined based on sample size estimation for testing the sensitivity (or specificity) of a single diagnostic test using the following formula

Po denotes the predetermined value of sensitivity or specificity where P1 is the value of sensitivity (or specificity) under the alternative hypothesis of the new diagnostic test. Based on using 95% confidence, 80% power to detect a difference of 10% (P1−Po) from the predetermined sensitivity or specificity at α = 0.05, β = 0.20, Zα/2 = 1.96 and Zβ = 0.84, the sample was estimated. Accordingly, the minimum sample size for the evaluation of Panbio AgRDT was 126 and 67, which were estimated from sensitivity and specificity, respectively. Finally, the sample size was increased to 148 based on the availability of kits.

Study participants and sample collection

The study participants were individuals who aged ⩾5 years and suspected for COVID-19 according to the national COVID-19 indicative symptoms criteria. The respondents were recruited following the COVID-19 suspect case definition purposively until the total sample size was obtained. After obtaining written informed consent/assent, sociodemographic data, including sex, age, and marital status, were collected by structured questionnaire. Then two separate nasopharyngeal swab samples were collected from each respondent at the same time by trained laboratory professionals at the Wollega University Referral Hospital. One of the swabs was tested by AgRDT at the collection site, where the other swab was preserved to Viral Transport Medium (VTM; Miraclean Technology Co., Ltd., China) tube for RT-PCR. The tools were pretested on 5% of sample size to evaluate and check its consistency at Nekemte public reference and research laboratory.

Laboratory tests

Laboratory testing was conducted by two independent trained senior medical laboratory personnel in the COVID-19 laboratory. One additional personnel was to read possible equivocal results in AgRDT. One swab sample was used for the detection of SARS-CoV-2 by AgRDT and the other swab was analysed by the reference method, RT-PCR.

AgRDT

One of the nasopharyngeal swabs was processed using a Panbio AgRDT kit (Abbott Diagnostic GmbH, Jena, Germany). The kit is a kind of membrane-based immuno-chromatography assay that detects the presence of the nucleocapsid (N) protein of SARS-CoV-2. A nasopharyngeal swab was inserted through the nostril and obtained the specimen by a gentle rub and roll movement about 3–4 times. The swab sample was inserted in about 300 μL of buffer-filled extraction tube based on the manufacturer’s protocol. After being mixed thoroughly, five drops of the mixture were allotted vertically into the specimen well (S) on the device. The results have been interpreted within 15 min following the manufacturer’s instructions. For a positive test result, the Panbio AgRDT forms a visible line that needs to be formed on the test line in the result window. A visible control line is needed to signify a test result is valid. Neither the test line nor the control lines are visible in the result window prior to the specimen dispensation on the device.

RT-PCR test

The VTM nasopharyngeal swab specimen was processed for RNA purification and detection by well-trained professionals. RNA extraction was undertaken using the QIAamp Viral RNA Mini Kit (Qiagen) according to the manufacturer’s instructions. A 200-μL swab was transferred into a 1.5 mL Eppendorf tube. Then, proteinase K (50 μL) and lysis buffer (200 μL) were added after brief centrifugation and incubation of the tube at 72°C for 10 min. Absolute alcohol (250 μL) was added and the mixture was entirely transferred to spin column. In the extraction procedure, positive and negative controls were included.

The RNA was detected by commercially available BGI RT-PCR assay (BGI Genomics Co. Ltd., Yantai, China) following the manufacturer’s instruction on a QuantStudio 5 Applied Biosystem RT-PCR Machine (S/No. 272521282, Thermo Fisher Scientific). The assay was developed for detecting specific single target gene, which is found on the ORF1ab region of SARS-CoV-2 genome. After PCR-Mix preparation following manufacturer’s guideline; 20 μL PCR-Mix (18.5 μL SARS-CoV-2 reaction mix and 1.5 μL SARS-CoV-2 enzyme mix) was filled into each well plate, and then added 10 μL of the RNA extract of no template (negative) control, patient sample, and positive control. The RT-PCR protocol was a step at 50°C for 20 min; a step at 95°C for 10 min; 40 steps at 95°C for 5 s and the last step at 60°C for 30 s. The FAM channel was set up for the detection of SARS-CoV-2 RNA and the VIC/HEX channel was set up for the detection of the human β-actin as a target gene for the internal control. After the positive and negative controls for RT-PCR detection of SARS-CoV-2 were evaluated using the nucleic acid amplification curve and Ct-values generated by the RT-PCR system, patient samples test results were assessed. The Ct-value in the FAM channel for a valid no template (negative) control should be ‘0’ and no sigmoidal amplification curve. The positive control should provide a sigmoidal amplification curve in both the FAM and VIC/HEX channels, and the Ct-values in the FAM and VIC/HEX channel should not be higher than 37 and 35, respectively. The patient sample is positive for SARS-CoV-2 if there is a sigmoidal amplification curve in the FAM channel, the Ct-value is not higher than 37; there may be a sigmoidal amplification curve in the VIC/HEX channel, and the Ct-value is not higher than 35. The specimen is negative for SARS-CoV-2 if there is no sigmoidal amplification curve in the FAM channel, there is a Ct-value of ‘0’ or ‘no data available’; there may be a sigmoidal amplification curve in the VIC/HEX channel, and the Ct-value is not higher than 35.

Statistical analysis

Statistical Package for the Social Sciences, version 24 (SPSS, Chicago, IL, USA) statistical software was used to determine the diagnostic accuracy of the index test in comparison with the reference method. The descriptive analysis was made to identify the frequency and percentage of the sociodemographic data. Two-by-two tables were used for the calculation of sensitivity, specificity and positive predictive value (PPV) and negative predictive value (NPV). The test agreement between AgRDT and RT-PCR for COVID-19 results was assessed using Cohen’s kappa statistics. The values of sensitivity and specificity are presented with their 95% confidence interval (95% CI), and the statistical significance was set at a p value of less than 0.05.

Ethical consideration

The ethical clearance was received from Wollega University Institutional Review Board (WU/RD/428/2013). Permission of study work was obtained from Wollega University Referral Hospital. Written informed consent or assent was obtained from each of the study participants and from their parent or guardian. Participants’ information sheet, which contains the objective of the study, inclusion/exclusion criteria, the required data and methods of data collection and informed consent/assent document were prepared in Afan Oromo, language of the region. The elements of participants’ information sheet were described to each of the study participants or parents in case of children below 18 years of age by trained local health personnel. Informed written consent was obtained from each participant and/or assent from children aged between 12 and 18 years. The participants’ laboratory test results were maintained confidentially for the duration of the study.

Result

Sociodemographic characteristics of study participants

Among a total of 148 COVID-19 suspected respondents, 80 (54.1%) and 68 (45.9%) were males and females, respectively. The participants’ age ranged from 5 to 80 years with a mean age of 43.0 (±4.1 SD) years. The majority, 89 (60.1%) of the study participants were married, followed by never married 41 (27.7%) of the study participants (Table 1).

Table 1.

Sociodemographic characteristics of COVID-19 suspected respondents.

Factors	Variables	n (%)
Sex	Male	80 (54.1)
	Female	68 (45.9)
Age (years)	5–14	11 (7.4)
	15–29	27 (18.2)
	30–39	35 (23.6)
	40–49	31 (20.9)
	50–59	27 (18.2)
	⩾60	17 (11.5)
Marital status	Non married	41 (27.7)
	Married	89 (60.1)
	Widowed/divorced	18 (12.2)

Prevalence of COVID-19

Among 148 participants, 74 (50.0%) were tested positive for COVID-19 by at least one of the diagnostic methods (rapid antigen test or RT-PCR). Each test separately detected SARS-CoV-2 in 15 (10.1%) participants. RT-PCR test results revealed that 74 (49.3%) participants were positive for COVID-19 and 60 (40.5%) participants were found positive for COVID-19 by Panbio AgRDT (Table 2).

Table 2.

Prevalence of COVID-19 by Panbio COVID-19 AgRDT and RT-PCR test.

Result	COVID-19 diagnostic methods		Combined result
	Panbio AgRDT	RT-PCR test
Total positive	60 (40.5%)	73 (49.3%)	74 (50.0%)
Total negative	88 (59.5%)	75 (50.7%)	74 (50.0%)

RT-PCR: reverse transcription-polymerase chain reaction.

Comparison and performance of Panbio AgRDT versus RT-PCR test

Among 73 positive results by RT-PCR, Panbio COVID-19 AgRDT detected 59 samples as positive and 14 samples as a false negative. In contrast, 1 sample was false positive by Panbio COVID-19 AgRDT from a total of 75 samples that were detected as negative by RT-PCR. The sensitivity, specificity, PPV and NPV of Panbio AgRDT for detecting SARS-CoV-2 were 81% (95% CI: 71%–91%), 98.7% (95% CI: 96%–100%), 98.3% and 84.1%, respectively. A test agreement between Panibo AgRDT and RT-PCR test to detect SARS-CoV-2 was found substantial with a kappa value of 0.80 (Table 3).

Table 3.

Test comparison and performance of Panbio COVID-19 AgRDT against RT-PCR.

Test comparison		RT-PCR		Total
		Positive	Negative
Panbio COVID-19	Positive	TP = 59	FP = 1	60
	Negative	FN = 14	TN = 74	88
Total		73	75	148
Diagnostic performance of Panbio COVID-19 against RT-PCR
Panbio COVID-19	Sensitivity	81% (95% CI: 71%–91%)
	Specificity	98.7% (95% CI: 96%–100%)
	PPV	98.3%
	NPV	84.1%
	Kappa value	0.80

TP: true positive; FN: false negative; TN: true negative; FP: false positive; PPV: positive predictive value; NPV: negative predictive value; RT-PCR: reverse transcription-polymerase chain reaction.

Discussion

To curb the spread of COVID-19, early detection of SARS-CoV-2, isolation of cases and timely clinical management of affected people is required. However, the challenges such as inadequate capacity, untrained laboratory personnel and inadequate funding for the gold standard testing (RT-PCR) for SARS-CoV-2 in resource-poor settings, such as Ethiopia, attract alternatively affordable and easier rapid diagnostic tests. Therefore, WHO has recommended that resource-limited countries use AgRDT for SARS-CoV-2 if the minimum sensitivity and specificity of AgRDT are 80% and 97%, respectively.

This study evaluated the Panbio AgRDT for SARS-CoV-2 in the COVID-19 test site Hospital of Wollega University, Western Oromia region, Ethiopia. During evaluation, the test sensitivity and specificity were 81% (95% CI: 71%–91%) and 98.7% (95% CI: 96%–100%), respectively. Similarly, based on a prevalence of 49.3%, the PPV and NPV of the AgRDT, respectively, were 98.3% and 84.1%, whereas the test agreement of tests was 0.8, which is a substantial test agreement. The sensitivity of this AgRDT was found lower than the sensitivity indicated by the manufacturer, but the specificity was comparable with the manufacturer. The sensitivity was concise with the minimum sensitivity limit of the WHO recommendation and with the study in Switzerland that reported the same sensitivity (81%) of oropharyngeal AgRDT and RT-PCR confirmed from nasopharyngeal swab samples. However, the sensitivity result of this evaluation is higher than the sensitivity result of the evaluation of the Panbio in symptomatic patients and close contacts in Spain (71.4%). The difference may be the geographical difference of study sites and the inclusion of an asymptomatic individual in Spain, regardless of the Ct-value of the real-time RT-PCR positive specimens. A previous study also supported this suggestion as shown that during employing Ct-values < 32 cycles cut-off for RT-PCR test positivity, the sensitivity of the Panbio was found above 95% for nasopharyngeal samples. However, 14 RT-PCR positive participants’ samples had been found negative by AgRDT. These antigen-negative and RT-PCR-positive specimens probably constitute non-infectious viral particles and a few may also constitute infectious viruses now no longer detected by the antigen test. Another study revealed that the false-negative results were associated with RT-PCR Ct-values. In regard to specificity, it is plausible and comparable with specificities observed in many countries worldwide, ranging from 96% to 100%.

The findings of this study are subjected to a few limitations. First, the study evaluated the Panbio AgRDT, and the results provided here cannot be generalized to different WHO-advocated SARS-CoV-2 antigen tests. Second, the Panbio AgRDT characteristic is probably different depending on whether an individual has previously examined positive. Third, the evaluation of self-sampling and patient self-testing of the AgRDT was not undertaken. That may result in more widespread and more frequent testing if operational errors are minimized through public health education. Finally, many factors would possibly restrict the number of viruses from a specimen and the incapability to detect viruses because a small quantity should not be interpreted to mean that a person is not infectious.

Conclusion

This study result reveals that Panbio AgRDT has fine performance in suspected patients. However, a negative AgRDT result must be considered presumptive and a confirmatory test might be required. Further studies are needed to examine the accuracy of self-sampling and potentially patient self-testing with a healthcare worker collecting swab samples, using SARS-CoV-2 AgRDT against the reference standard RT-PCR in different settings, patient status and during lower and higher pretest probabilities.

Click here for additional data file.

Supplemental material, sj-docx-1-smo-10.1177_20503121221110079 for Evaluation of the performance of Panbio™ COVID-19 antigen rapid diagnostic test for the detection of SARS-CoV-2 in suspected patients in Ethiopia by Dejene Dessalegn, Edosa Kifle Tola, Afework Tamiru and Biruk Zerfu in SAGE Open Medicine