Investigation of the diagnostic performance of the SARS-CoV-2 saliva antigen test: A meta-analysis.

BACKGROUND: The COVID-19 pandemic is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Rapid identification and isolation of patients with COVID-19 are critical strategies to contain COVID-19. The saliva antigen test has the advantages of noninvasiveness and decreased transmission risk to health-care professionals. This meta-analysis investigated the diagnostic accuracy of the saliva antigen test for SARS-CoV-2.
METHODS: We searched for relevant studies in PubMed, Embase, Cochrane Library, and Biomed Central. Studies evaluating the diagnostic accuracy of saliva antigen tests for SARS-CoV-2 were included. The data of the included studies were used to construct a 2 × 2 table on a per patient basis. The overall sensitivity and specificity of saliva antigen tests were determined using a bivariate random-effects model.
RESULTS: Nine studies enrolling 9842 patients were included. The meta-analysis generated a pooled sensitivity of 65.3% and a pooled specificity of 99.7%. A subgroup analysis of the studies performing the chemiluminescent enzyme immunoassay (CLEIA) for participants from airports and public health centers revealed a pooled sensitivity of 93.6%.
CONCLUSION: Our findings demonstrated that the saliva antigen test performed using CLEIA exhibited higher sensitivity for the detection of SARS-CoV-2. Therefore, the saliva antigen test performed using CLEIA might be an effective and noninvasive screening tool for SARS-CoV-2.

Introduction

Rapid transmission of severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) caused the COVID-19 pandemic. At least 50% of patients with COVID-19 contracted SARS-CoV-2 from asymptomatic individuals. To prevent the transmission of SARS-CoV-2, testing individuals with suspected COVID-19 and quarantining their contacts are major nonpharmaceutical interventions. Testing is an effective method to prevent the transmission of SARS-CoV-2, and the combination of testing and tracing is more effective than mass testing or self-isolation alone. Currently, reverse transcription–polymerase chain reaction (RT-PCR) performed using nasopharyngeal swab (NPS) samples is the standard diagnostic test for COVID-19. When the prevalence of COVID-19 increases in a community, large-scale testing programs are required to effectively contain COVID-19. A meta-analysis reported that the accuracy of saliva RT-PCR is similar to that of NPS RT-PCR in the ambulatory setting. Frequent testing and short turnaround time of testing are crucial to control the spread of SARS-CoV-2 within a community.

The collection of NPS specimens requires close contact between health-care staff and patients with suspected COVID-19. This procedure causes discomfort and increases the risks of disease transmission and bleeding, particularly in patients with bleeding disorders. Saliva sample collection is an economical and noninvasive procedure, and the use of saliva samples for testing has reduced disease transmission to health-care professionals. Moreover, saliva specimens can be self-collected, allowing for regular monitoring of SARS-CoV-2 viral load and large-scale screening. Saliva RT-PCR has high sensitivity and specificity for the detection of SARS-CoV-2. Saliva specimens can be self-collected in outpatient and community clinics. Antigen tests for SARS-CoV-2 have the advantages of low cost, short turnaround time, and prompt identification of patients with COVID-19 infection. Moreover, antigen tests are highly sensitive for the detection of SARS-CoV-2 within 7 days after symptom onset and are considered to be effective for the screening of community transmission. , The high sensitivity of the antigen test in symptomatic patients indicates its effectiveness for public health screening. With the reopening of borders, the saliva antigen test may enable the timely identification of travelers with COVID-19 infection at airports.

The diagnostic accuracy of saliva antigen tests for COVID-19 remains inconclusive. Therefore, this meta-analysis evaluated the accuracy of antigen tests for the detection of SARS-CoV-2 by using saliva specimens.

Methods

Literature search strategy

This study was conducted in accordance with the Preferred Reporting Items for a Systematic Review and Meta-Analysis of Diagnostic Test Accuracy Studies: The PRISMA-DTA Statement and is registered in PROSPERO (registration number: CRD42021276294).

We searched for relevant studies in PubMed, Embase, Cochrane Library, and Biomed Central. A literature search was conducted using the following search terms: (COVID-19 or severe acute respiratory syndrome coronavirus 2 or SARS-CoV-2) and (antigen test or SARS-COV-2 antigens or mass screening or community participation) and (RT-PCR or reverse transcriptase polymerase chain reaction or COVID-19 nucleic acid testing) and saliva. A combination of free-text and MeSH terms was used to identify relevant studies. No language restriction was applied to the literature search. Detailed search strategies are presented in supplementary material 1.

Inclusion and exclusion criteria

Studies that evaluated the diagnostic accuracy of saliva antigen tests for SARS-CoV-2 with reference standards in patients with suspected SARS-CoV-2 infection were included. However, we excluded review articles. Saliva specimens were collected from symptomatic or asymptomatic individuals. Studies employing RT-PCR as the reference standard were included. The literature search was conducted without time restrictions. We included studies that provided sufficient data to construct a 2 × 2 table on a per patient basis. We excluded preprint articles, case reports, case series, proposals, protocols, conference abstracts, and studies performing in-house tests. The last literature search was performed on January 2, 2022. One reviewer initially screened the titles and abstracts of potentially eligible studies identified using the search strategy. After the exclusion of irrelevant studies, two reviewers independently examined the full text of studies to select studies that met the inclusion criteria. Disagreements and disputes between the reviewers were resolved through joint discussions.

Quality assessment

The quality of the included studies was evaluated using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. According to QUADAS-2, saliva antigen tests for SARS-CoV-2 were defined as the index test and RT-PCR for SARS-CoV-2 as the reference standard. QUADAS-2 consists of four domains: patient selection, index test, reference standard, and flow and timing. Each domain includes questions that analyze the risk of bias. The quality of a diagnostic test is assessed in terms of the risk of bias and the applicability of a study. The tool evaluates the applicability of the included study to the research question for each domain. A study is considered to have high quality if each domain in the study exhibits a low risk of bias.

Statistical analysis

We extracted raw data for true positives, true negatives, false positives, and false negatives from each included study to construct 2 × 2 tables for recalculating pooled sensitivity, pooled specificity, and pooled diagnostic odds ratios (DORs). If 2 × 2 tables could not be extracted from the main text, we searched the supplementary material of the study for additional information. The sensitivity of a diagnostic test is defined as the proportion of those with the target disorder correctly identified as having the disorder, whereas the specificity of a diagnostic test is referred to the proportion of those without the target disorder correctly identified as not having the disorder.

We conducted a meta-analysis by using a bivariate random-effects model to summarize sensitivity and specificity on a per patient basis. We plotted the summary receiver operating characteristic (SROC) curve to determine the overall diagnostic performance of the index test. The closer the curve approaches the upper-left corner, the higher the overall performance is. The area under the curve (AUC) of an excellent test should be ≥ 0.97. An AUC of 0.93–0.96 is considered highly satisfactory, whereas an AUC of 0.75–0.92 is considered satisfactory. Possible sources of heterogeneity between the included studies were explored by performing a prespecified subgroup analysis including the patient population, technology used for the index test, and antigen assay cutoff value. Pooled sensitivity and specificity of the saliva antigen test were calculated with 95% confidence intervals (CIs). Furthermore, we calculated the Spearman correlation coefficient between the logit of sensitivity and the logit of 1 − specificity to investigate the threshold effect. A positive correlation indicated the presence of the threshold effect. All analyses were performed using MetaDiSc version 1.4 and MetaDTA software. , A p value of <0.05 was considered statistically significant.

Results

Meta-analysis

Nine studies including 9842 patients were retrieved for the meta-analysis.18, 19, 20, 21, 22, 23, 24, 25, 26 Fig. 1 depicts the literature search strategy, and Table 1 presents the detailed characteristics of the studies. Eight studies used a prospective study design, and five studies enrolled participants from hospitals, health institutes, or primary care centers. Four studies evaluated the diagnostic performance of antigen tests in patients with suspected COVID-19, and two studies mainly evaluated the performance of antigen tests in a screening setting (public testing sites and a hospital). Five studies performed antigen tests using chemiluminescent enzyme immunoassay (CLEIA) technology, and three studies used the lateral flow device (LFD). Four studies provided the cutoff values of antigen tests. Three studies indicated the cycle threshold (Ct) values of positive RT-PCR tests, and one study reported the cutoff value of Ct. The meta-analysis for saliva antigen tests generated a pooled sensitivity of 65.3% (95% CI: 37.7%–85.4%) and a pooled specificity of 99.7% (95% CI: 98.2%–99.9%; Fig. 2 ). The meta-analysis produced an I2 index of 97.2% for the pooled sensitivity and another I2 index of 98.3% for the pooled specificity, which indicated that high heterogeneity between studies. In addition, in the meta-analysis, a pooled DOR of 577.304 (95% CI: 102.951–3237.262) was calculated for saliva antigen tests; this value indicated the discriminative power of the index test. The AUC of the SROC for antigen tests was 0.98, indicating that saliva antigen tests may be suitable for the diagnosis of SARS-CoV-2 infection. Fig. 3 presents the sensitivities and specificities of saliva antigen tests for SARS-CoV-2 from the included studies. Supplementary Material 2 lists statistical data.

Figure 1

Flowchart of literature search.

Table 1

Characteristics of studies.

Study	Study design	Testing Site	Patient population	Participants (total/data extraction)	Age median (range)	Days post symptom onset median (range)	Saliva collection	Index test	Antigen assay cutoff (pg/mL)	Viral antigen detected	Reference standard	Ct value of positive RT-PCRmedian (range)
Igloi Z 2021	prospective	a large designatedtesting site	non-hospitalizedpatients	(789/789)	37 (18–79)	2 (0–41)(545 patients)	Zeesan Saliva RNACollection kit	SD BiosensorSARS-CoV-2 saliva antigen rapid test	NA	NA	RT-PCR (saliva)	25.5 (17.4–34.2)
Tanimoto Y 2021	prospective	a health institute	Suspected COVID-19 people	(116/116)	NA	NA	Salivettes®	Lumipulse® SARS-CoV-2 antigen kit,CLEIA	0.67	NA	RT-PCR (NPS)	NA
Audigé A 2021	prospective	NA	asymptomaticand symptomatic	(407/307)	36 (16–76)(SARS-CoV-2 positive patiets)	2 (1–15)(SARS-CoV-2 positive patiets)	VTM was added to the crude saliva.	Elecsys SARS-CoV-2 Antigen assay	Cutoff Index≧1	N antigen	RT-PCR (NPS)	24.18 (15.32–35.8)
Kobayashi R 2021	prospective	a publichealth centerand a hospital	Suspected COVID-19 patients	(5430/5386)	NA	NA	sterile tubes	Lumipulse® SARS-CoV-2 antigen kit,CLEIA	0.67	NA	RT-PCR (NPS)	NA
Yokota I 2021	prospective,consecutive	NA	hospitalised patients, close contacts identified at communityhealth centres and international arrivals at two airports	(2077/2056)	airport quarantine: 33.5 (22.6–47.4, IQR)inpatient: 69.8 (51.6–83.4, IQR)contact tracing (symptomatic): 42.2 (34.8–59.6, IQR)	NA	sterilised 15 mLpolystyrene sputumcollection tube	Lumipulse® SARS-CoV-2 antigen kit,CLEIA	0.19	SARS-CoV-2-N protein	RT-PCR (saliva)	NA
Asai N 2021	retrospective	a hospital and affiliated facilities	Suspected COVID-19 patients	(305/305)	NA	NA	NA	Lumipulse® SARS-CoV-2 antigen kit,CLEIA	0.67	NA	RT-PCR (saliva)	26.6 (15.5–36.2)
Masiá M 2021	prospective,consecutive	3 primary care centers and an emergency department	asymptomaticand symptomatic	(913/611)	40.6 (23–55.6, IQR)	3 (2–5, IQR)	100-mL sterile empty container	Panbio COVID-19 Ag Rapid Test Device	NA	nucleocapsidprotein	RT-PCR (NPS)	NA
Ishii T 2021	prospective	a hospital	asymptomaticand symptomatic	(597/136)	NA	NA	NA	Lumipulse® SARS-CoV-2 antigen kit,CLEIA	0.67	NA	RT-PCR (saliva)	NA
Sberna G 2021	prospective	an institute for infectious diseases	Suspected COVID-19 patients	(136/136)	NA	NA	NA	ESPLINE SARS-CoV-2	NA	NA	RT-PCR (saliva)	NA

CLEIA: chemiluminescent enzyme immunoassay; COVID-19 = coronavirus disease 2019; CT = cycle threshold; IQR: interquartile range; NA = not available; NPS: nasopharyngeal swab; RT-PCR = reverse transcription polymerase chain reaction; VTM: virus transport medium.

Figure 2

The summary estimate (square shape in blue) of HSROC curve presents the pooled sensitivity of 65.3% (95% CI: 37.7%–85.4%) and pooled specificity of 99.7% (95% CI: 98.2%–99.9%) for SARS-CoV-2 saliva antigen test. HSROC: hierarchical summary receiver operating characteristic.

Figure 3

Forest plots showing the sensitivities and specificities of SARS-CoV-2 saliva antigen tests with 95% CIs. The meta-analysis for saliva antigen tests generated a pooled sensitivity of 65.3% (95% CI: 37.7%–85.4%) and a pooled specificity of 99.7% (95% CI: 98.2%–99.9%). CIs: confidence intervals.

We used QUADAS-2 to evaluate the quality of the included studies in our meta-analysis. A study is considered to be of high quality when all domains for the study are judged to have a low risk of bias. Regarding patient selection, two studies enrolled patients randomly or consecutively; seven studies did not use a case–control study design, which might have led to an overestimation of the diagnostic accuracy. For the patient selection domain on the basis of the criteria of QUADAS-2, two studies were judged to have a low risk of bias. Regarding index tests, all the studies reported that index tests were interpreted without knowing the results of the reference standard. Thus, all the studies were judged to have a low risk of bias in the index test domain. Regarding the reference standard, six studies indicated that the reference standard likely correctly classified the target condition. Regarding the flow and timing domain, all the studies demonstrated that all patients received a reference standard. Four studies indicated that all patients were included in the analysis. Four articles were judged as having a low risk of bias in the flow and timing domain. With regard to applicability, the patient selection and index tests of the studies included in the meta-analysis matched our review title. Table 2 presents the quality of studies in the meta-analysis.

Table 2

Quality of individual studies.

Study	Risk of bias				Applicability concerns
	Patient selection	Index test	Reference standard	Flow and timing	Patient selection	Index test	Reference standard
Igloi Z 2021	U	L	U	L	L	L	U
Tanimoto Y 2021	U	L	L	L	L	L	L
Audigé A 2021	U	L	L	H	L	L	L
Kobayashi R 2021	U	L	L	H	L	L	L
Yokota I 2021	L	L	U	H	L	L	U
Asai N 2021	U	L	U	L	L	L	U
Masiá M 2021	L	L	L	H	L	L	L
Ishii T 2021	U	L	L	H	L	L	L
Sberna G 2021	U	L	L	L	L	L	L

H = high risk of bias; L = low risk of bias; U = unclear risk of bias.

Subgroup analysis

We performed subgroup analyses according to the patient population, technology used in the index test, and antigen assay cutoff value. Four studies including 5943 patients reported the accuracy of saliva antigen tests for patients with suspected COVID-19. The meta-analysis produced a pooled sensitivity of 67.7% (95% CI: 19.6%–94.7%) and a pooled specificity of 99.8% (95% CI: 97.6%–100%). The subgroup analysis of two studies that performed saliva antigen tests by using CLEIA in 7442 participants from airports and public health center generated a pooled sensitivity of 93.6% (95% CI: 77.8%–98.4%) and a pooled specificity of 99.3% (95% CI: 78.4%–1.00%). According to the detection technology used in the index test, we performed a subgroup analysis for the five studies that performed antigen tests using CLEIA in 7999 patients. This analysis generated a pooled sensitivity of 85.6% (95% CI: 69.2%–94%) and a pooled specificity of 98.9% (95% CI: 94.5%–99.8%). The subgroup analysis of the three studies that used the LFD demonstrated a pooled sensitivity of 27.4% (95% CI: 8.1%–61.9%) and a pooled specificity of 100% (95% CI: 93.8%–100%), respectively. This finding indicated that CLEIA exhibited higher sensitivity for the detection of COVID-19 than did the LFD. Four studies including 5943 participants reported the accuracy of the index test with a cutoff value of 0.67 pg/mL. The analysis generated a pooled sensitivity of 83.2% (95% CI: 58.6%–94.6%) and a pooled specificity of 99.2% (95% CI: 96.0%–99.9%), respectively. Table 3 presents the pooled estimates of the subgroup analyses.

Table 3

Subgroup analyses of the diagnostic accuracy of saliva antigen tests.

Subgroup	Number of studies	Number of patients	Sensitivity (%)(95% CI)	Specificity (%)(95% CI)
Suspected COVID-19 patients	4	5943	67.7 (19.6%–94.7%)	99.8 (97.6%–100%)
Studies that included participants from air port and public health center and assessed antigen tests using CLEIA	2	7442	93.6 (77.8%–98.4%)	99.3 (78.4%–1.00%)
Antigen tests using CLEIA for detecting COVID-19 patients	3	5807	85.4 (56%–96.4%)	99.5 (97.1%–99.9%)
Sample type (reference standard): nasopharyngeal	4	6420	64.5 (23.4%–91.5%)	99.9 (99.6%–100%)
Sample type (reference standard): saliva	5	3422	64.0 (31.4%–87.4%)	99.5 (90.8%–100%)
Antigen tests with CLEIA method	5	7999	85.6 (69.2%–94%)	98.9 (94.5%–99.8%)
Antigen tests with LFD method	3	1536	27.4 (8.1%–61.9%)	100 (93.8%–100%)
Index tests (CLEIA) with reference standard (nasopharyngeal)	2	5502	88.5 (45.2%–98.6%)	100 (98.7%–100%)
Index tests (CLEIA) with reference standard (saliva)	3	2497	82.1 (74.0%–88.0%)	96.1 (90.8%–98.4%)
Antigen assay cutoff (0.67 pg/mL)	4	5943	83.2 (58.6%–94.6%)	99.2 (96.0%–99.9%)

CI: confidence interval; CLEIA: chemiluminescent enzyme immunoassay; LFD: lateral flow device.

Assessment of the threshold effect

In the five studies that provided antigen cutoff values, we performed the threshold analysis to investigate the potential threshold effect. The Spearman's correlation coefficient was −0.1 (p = 0.9), indicating the absence of the threshold effect between these studies.

Meta-regression

To explore the potential sources of heterogeneity across the studies, we conducted a meta-regression analysis by using the following covariate: the specimen type of the reference standard (NPS or non-NPS). The results indicated that the specimen type of the reference standard did not affect the diagnostic performance of the index test (relative DOR = 17.82; 95% CI: 0.26–1211.47; p = 0.15).

Discussion

The findings of this study indicated that saliva antigen tests exhibited high specificity for the detection of SARS-CoV-2. The diagnostic performance of saliva RT-PCR for SARS-CoV-2 detection was similar to that of NPS RT-PCR in the ambulatory setting. Saliva RT-PCR is suggested as an alternative to NPS RT-PCR. To the best of our knowledge, this is the initial systematic review and meta-analysis examining the diagnostic performance of the saliva antigen test for SARS-CoV-2 and performing the pooled analysis of its sensitivity and specificity relative to RT-PCR. The results of our meta-analysis indicated the added clinical use of the antigen test for SARS-CoV-2. The availability of COVID-19 vaccines does not obviate the need for increased testing. Testing will remain crucial during vaccine rollout because of the limited supply of vaccines, the hesitancy and refusal of individuals to receive vaccination, and the emergence of more infectious SARS-CoV-2 variants. In addition, frequent antigen-based home testing and self-isolation can reduce SARS-CoV-2 infection and mortality. The antigen test is a reliable method for SARS-CoV-2 detection for symptomatic individuals in community transmission screening. Because saliva specimens can be self-collected, the saliva antigen test might increase the acceptability and popularity of the antigen test in community-based surveillance and screening.

The results of subgroup analysis in our meta-analysis indicated that saliva antigen tests performed using the CLEIA method exhibited higher sensitivity in detecting SARS-CoV-2. In another subgroup analysis of the studies including participants from airports and public health centers and examining the diagnostic performance of saliva antigen tests performed using the CLEIA, saliva antigen tests were determined to have higher sensitivity in detecting SARS-CoV-2 in a population. Saliva specimens are easy to handle and can be self-collected, thus enabling large-scale testing and preventing the viral exposure of health-care professionals. Based on the subgroup analyses, the sensitivity of saliva antigen tests with CLEIA technology is comparable to that of RT-PCR. Furthermore, saliva antigen tests are noninvasive and can be considered for self-testing. In addition, a combined approach of telemedicine and saliva antigen tests can be a practical strategy at airports. On arrival, saliva testing might be the most favorable method to reduce the number of passengers who have to quarantine after arrival. Our meta-analysis provided evidence for the high sensitivity of saliva antigen tests performed using the CLEIA in identifying infected passengers after arrival. To minimize the number of asymptomatic carriers, strategies including the mass screening of travelers at airports by using antigen tests, supervised quarantining, frequent retesting, and close follow-up of infected people are required to be implemented.

A study reported that breakthrough infections with SARS-CoV-2 might occur in fully vaccinated health-care professionals. Most health-care staff with breakthrough SARS-CoV-2 infections had mild symptoms or were asymptomatic. Although COVID-19 testing performance remains unclear in breakthrough infections in fully vaccinated individuals and asymptomatic individuals regardless of their vaccination status, the close contacts of persons with SARS-CoV-2 infection should undergo COVID-19 testing. Therefore, the saliva antigen test for SARS-CoV-2 remains necessary in postvaccination infection.

Children with SARS-CoV-2 infection have less severe symptoms when infected by SARS-CoV-2 compared with other respiratory viruses. The average viral load was lower in the pediatric population than in the adult population. Low viral load can affect the transmission of SARS-CoV-2 and the sensitivity of antigen tests in children. This can result in the lower sensitivity of antigen tests in the pediatric population. Correctly performing NPS collection in children is stressful and painful due to the poor cooperation of children. Collection of NPS specimens causes severe discomfort in children and requires close contact between health-care workers and patients, thus increasing the risk of contagion. Therefore, the saliva antigen test might be an ideal tool for the diagnosis of COVID-19 infection in the pediatric population.

Serial testing can compensate for the lower sensitivity of the antigen test. Serial testing is critical because a single antigen test might not be sufficient to identify asymptomatic children. Serial testing might identify children with infection because they subsequently develop high viral loads. Moreover, effective COVID-19 screening mainly depends on the frequency of testing and rapid turnaround time and is only slightly improved by test sensitivity. Furthermore, antigen testing is an accurate and convenient approach for individuals to screen for COVID-19 infection if performed two to three times each week.

Although the results of this meta-analysis indicated that the saliva antigen test performed using CLEIA exhibited high sensitivity in detecting SARS-CoV-2, this study has some limitations. The Ct threshold values reported in the included studies and the Ct values of individuals with SARS-CoV-2 infection were limited. The studies reporting consecutive or random patient recruitment were limited. No study in the meta-analysis provided information on SARS-CoV-2 variants. In addition, no study in the meta-analysis examined the accuracy of the saliva antigen test for SARS-CoV-2 in the pediatric population.

In conclusion, our major findings indicated that saliva antigen tests performed using CLEIA exhibited high sensitivity for the detection of SARS-CoV-2. Therefore, saliva antigen tests performed using CLEIA might be an effective tool for the screening of SARS-CoV-2 infection in passengers at airports based on the finding of our subgroup analyses. Additional studies should examine the accuracy of saliva antigen tests stratified by Ct values and evaluate the pediatric population to improve the applicability of saliva antigen tests for SARS-CoV-2.

Funding

This study was supported by a grant from (grant no.: 2022SKHAND008).

Institutional review board statement

The present study is a meta-analysis performed to examine effect sizes reported in the literature. Therefore, this study was exempt from the Institutional Review Board review.

Informed consent statement

Not applicable.

Data availability statement

All data generated or analyzed during this study are included in this published article (and its Supplementary Information files).

Declaration of competing interest

The authors declare no conflicts of interest.