Clinical evaluation of the SD Biosensor SARS-CoV-2 saliva antigen rapid test with symptomatic and asymptomatic, non-hospitalized patients.

BACKGROUND: Performance of the SD Biosensor saliva antigen rapid test was evaluated at a large designated testing site in non-hospitalized patients, with or without symptoms.
METHOD: All eligible people over 18 years of age presenting for a booked appointment at the designated SARS-CoV-2 testing site were approached for inclusion and enrolled following verbal informed consent. One nasopharyngeal swab was taken to carry out the default antigen rapid test from which the results were reported back to the patient and one saliva sample was self-taken according to verbal instruction on site. This was used for the saliva antigen rapid test, the RT-PCR and for virus culture. Sensitivity of the saliva antigen rapid test was analyzed in two ways: i, compared to saliva RT-PCR; and ii, compared to virus culture of the saliva samples. Study participants were also asked to fill in a short questionnaire stating age, sex, date of symptom onset. Recommended time of ≥30mins since last meal, drink or cigarette if applicable was also recorded. The study was carried out in February-March 2021 for 4 weeks.
RESULTS: We could include 789 people with complete records and results. Compared to saliva RT-PCR, overall sensitivity and specificity of the saliva antigen rapid test was 66.1% and 99.6% which increased to 88.6% with Ct ≤30 cutoff. Analysis by days post onset did not result in higher sensitivities because the large majority of people were in the very early phase of disease ie <3 days post onset. When breaking down the data for symptomatic and asymptomatic individuals, sensitivity ranged from 69.2% to 50% respectively, however the total number of RT-PCR positive asymptomatic participants was very low (n = 5). Importantly, almost all culture positive samples were detected by the rapid test.
CONCLUSION: Overall, the potential benefits of saliva antigen rapid test, could outweigh the lower sensitivity compared to nasopharyngeal antigen rapid test in a comprehensive testing strategy, especially for home/self-testing and in vulnerable populations like elderly, disabled or children where in intrusive testing is either not possible or causes unnecessary stress.

Introduction

In spite of having multiple vaccines available [1], the COVID-19 pandemic is far from over and testing remains an essential pillar of the pandemic response. In high income countries, by now sufficient diagnostic capacities have been established however access to testing can be hindered by remote locations, lack of information, financial obstacles etc. Besides access, willingness is another factor which can be increased by simple, comfortable and non-invasive testing options. To increase access and testing rates, self / home testing approaches are explored and being implemented worldwide. However there is considerable debate on the drawbacks mostly from the disease surveillance and reliability of the results point of view [2]. This is because the multiple components which cannot be controlled over time when a non-trained professional carries out a test. The first step to make self / home testing an acceptable option is the availability of good reliable tests which are CE marked and fully evaluated for this specific application; these are still scarcely available [3]. Furthermore currently besides a few [4-6], most antigen rapid detection tests (RDTs) have only been validated using nasopharyngeal swabs which arguably can best be executed by trained health-care professionals and are less suited for self-testing. Suitability of saliva/oral fluid has been explored as a sample type both for molecular and serological test and found both higher and lower sensitivity for saliva samples compared with nasopharyngeal swabs. However, meta-analyses of such studies suggest an overall similar or non-statistically significant lower sensitivity associated with the use of saliva samples [7]. Taking patient comfort and ease of sampling in account saliva is definitely well suited also for self-sampling.

Because of the above mentioned reasons, we evaluated the clinical performance of a RDT utilizing saliva as sample amongst both symptomatic and asymptomatic general population presenting at testing locations. In order to have reliable data of the performance, samples were collected and tested by trained professionals. We have found acceptable [8] performance among symptomatic individuals who were in the early onset ie <7 days of disease. Results from this study can encourage further evaluation of RDTs using saliva thereby laying down the road towards inclusion saliva RDT in self-testing approaches.

Materials and methods

Testing site, testing procedures, population and patient recruitment

The study was carried out at an XL testing location in Rotterdam—Rijnmond (Rotterdam is the second largest city in the Netherlands) which is by appointment only. Persons with any respiratory symptoms, or persons that have been contacts of confirmed cases regardless of symptoms are eligible for a free of charge SARS-CoV-2 test. Vulnerable persons (ie elderly or chronic condition) and priority groups (ie teachers, healthcare workers) are tested by reverse transcriptase polymerase chain reaction (RT-PCR), everyone else is tested by the SARS-CoV-2 Rapid Antigen Test (Distributed by Roche (SD Biosensor)) using a nasopharyngeal (NP) swab. In this study persons eligible for a rapid antigen test were enrolled. The SARS-CoV-2 Rapid Antigen Test-Standard Q COVID-19 Ag Saliva- Research use only (Lot number QCO9021001; expiry date 04-01-2023) was provided by SD Biosensor (http://www.sdbiosensor.com/xe/) but at time of writing this rapid antigen test was not available on the market.

At the entrance of the testing site all eligible people over 18 years of age were approached for inclusion and enrolled following verbal informed consent. Study participants were also asked to fill in a questionnaire stating age, sex, date of symptom onset. Recommended time of ≥30mins since last meal, drink or cigarette if applicable was also recorded. The study was carried out for 4 weeks (10 February-19 March) to reach an ideally total of 1000 inclusions. Study was terminated just prior to reaching the target at 816 inclusions for logistic reasons. From this, 789 had a PCR result due to a laboratory logistic problems.

Specimen collection, testing and culture procedures

Saliva was collected based on instructions of use by the test provider (nasal discharge and posterior pharyngeal spitting drooled into collection device) using the Zeesan Saliva RNA collection kit without preservation medium (http://www.zeesandx.com/coronavirus/1075.html). All saliva samples were tested for SARS-CoV-2 RNA using the cobas ® SARS-CoV-2 RT-PCR test on the COBAS6800 (Roche diagnostics). Genome copies/ml were calculated based on an in house established standard curve. All RT-PCR positive saliva samples were inoculated onto Vero cells following dilution with universal transport media (VTM) (HiViral; HiMedia Laboratories PVT, Ltd., http://www.himedialabs.com) to 6ml volume, filtration with 45μm bacterial filter and mixing with FBS: 1080ul saliva+VTM with 720ul FBS prior to freezing at -80°C for a maximum of two weeks. Samples were cultured for a maximum of 14 days or until cytopathic effect (CPE) was visible. The presence of SARS-CoV-2 was confirmed by immunofluorescence, using a rabbit polyclonal antibody targeting SARS CoV-2 nucleocapsid protein (Sino Biological Inc.).

Data analysis

Data from the RDTs, RT-PCR, virus culture and clinical questionnaire were merged using Microsoft Access (http://www.microsoft.com), and data analysis was performed using Microsoft excel and R software version 4.0.2 (https://www.r-project.org). Sensitivity and specificity of RDTs were calculated in relation to the saliva RT-PCR and virus culture results. As a comparator sensitivity of the nasopharyngeal RDT was also calculated however only to saliva RT-PCR results as no PCR could be carried out on nasopharyngeal samples. Negative and positive predictive values (NPV and PPV) were calculated using percentage PCR positivity figures as a good proxy for disease prevalence. Clopper-Pearson analysis was used to determine confidence intervals of proportions. Two sample t-test was used to define significance of difference between means.

Ethical clearance

The medical research ethics committee (MREC) of Erasmus Medical center decided the study was not subject to the Medical Research Involving Human Subjects Act (WMO) and did not require full review by an accredited MREC (protocol number MEC-2021-0083).

Results

In total 789 complete dataset were available for analysis. Results of both the saliva and the nasopharyngeal RDT and the virus culture was compared to RT-PCR results and categorized by RT- PCR Ct values as an indicator for viral load and detection limit, Table 1. Participants had a median age of 37 years, an equal proportion of males and female (male, 50.6%) and almost a quarter were smokers. Most people presenting for testing had symptoms (70.5%, 556/789) with recent onset of disease (median 2 days post onset). Of the symptomatic people who could provide the exact date of onset (545/556) vast majority was in the very early phase of disease (77.5% <3days) which is also seen in the proportion of low Ct values amongst the study population (70% Ct≤30). Of all participants, 7.9% tested positive by RT-PCR what was just below the current percentage positivity nationally (between 11.2%-8.6% during the course of the study) [9]. More positive samples were detected by the standard nasopharyngeal RDT than by saliva RDT (n = 52 vs. n = 44). All samples with Ct≤30 could be cultured.

Table 1

Characteristics of the study population.

Total N (RT-PCR and RDT results)	789
Age [median (min-max); N)	37 years (18–79 years)
Sex [%M, (n/N)]	50.6% (399/789)
Smoker [%Y, (n/N)]	20.8 (164/789)
Symptoms present [%Y, (n/N)]	70.5% (556/789)
Days from symptom onset [median (min-max); N]	2 days (0–41); 545
Days 0–3 [n/N (%)]	431/545 (77.5%)
Days 4–7 [n/N (%)]	91/545 (16.4%)
Days 8+ [n/N (%)]	23/545 (4.1%)
Positivity PCR SARS-2 E gene [%, (n/N)]	7.9% (62/789)
PCR Ct [median (min-max); N]	27.6 (17.4–35.1); 62
Ct > 30 [n, (%)]	18 29%
Ct ≤ 30 [n, (%)]	44, 70.1%
Ct ≤ 25 [n, (%)]	25, 40.3%
Positivity NP RDT [%, (n/N)]	6.6% (52/789)
NP RDT samples with positive PCR result [n]	49
PCR Ct SARS-2 E gene [median (min-max); N]	26.6 (17.4–34.2); 49
Ct > 30 [n, (%)]	9, 18.4%
Ct ≤ 30 [n, (%)]	40, 81.6%
Ct ≤ 25 [n, (%)]	22, 44.9%
Positivity saliva RDT [%, (n/N)]	5.6% (44/789)
Saliva RDT samples with positive PCR result [n]	41
PCR Ct [median (min-max); N]	25.5 (17.4–34.2); 41
Ct > 30 [n, (%)]	2, 4.9%
Ct ≤ 30 [n, (%)]	39, 95.1%
Ct ≤ 25 [n, (%)]	25, 60.9%
Positivity saliva virus culture [%, (n/N)]	48.4% (30/62)
Ct > 30 [n, (%)]	0, 0%
Ct ≤ 30 [n, (%)]	30, 100%
Ct ≤ 25 [n, (%)]	21, 70%

Data of all included people in the study were analyzed by basic demographics, smoking status, date of disease onset, RT-PCR Ct values and virus culture result.

RT-PCR, reverse transcription PCR; RDT, antigen rapid detection test; Min-max, minimum and maximum; M, male; Y, yes; n/N, amount of total sample size NP, nasopharyngeal swab; Ct, cycle threshold; E gene, envelope gene;

Sensitivity and specificity of the saliva RDT was analyzed in two ways: i, compared to saliva RT-PCR Table 2 and Fig 1; and ii, compared to virus culture Fig 1. Comparing to RT-PCR in saliva, overall sensitivity of the saliva RDT was 66.1%. When PCR Ct ≤30 (E gene copy/ml 2.17E+05) cutoff was used sensitivity increased to 88.6%. When breaking down the data for symptomatic and asymptomatic individuals, sensitivity ranged from 69.2% to 50% respectively, however the total number of RT-PCR positive asymptomatic participants was very low (n = 5). Analysis by days post onset did not result in higher sensitivities but this is due to the uneven representation of patients with various days post onset in the study population (majority <3 days). Specificity was comparable to previous findings using nasopharyngeal RDT compared to PCR (between 99.1% and 99.8%) [10].

Table 2

Sensitivity and specificity of both saliva and nasopharyngeal antigen RDT compared to RT-PCR.

	Compared to saliva RT-PCR
Saliva RDT	0–3 days post onset	0–7 days post onset	Overall	Symptomatic	Asymptomatic
Sensitivity% (CI95%), n	68.5%	65.0%	66.1%	69.2%	50.0%
	(54.5–80.5) (37)	(51.6–76.9) (39)	(52.9–77.6) (41)	(54.9–81.3) (36)	(18.7–81.3) (5)
≤Ct 30, Sensitivity% (CI95%), n	87.5%	88.1%	88.6%	88.2%	83.3%
	(73.2–95.8) (35)	(74.4–96.0) (37)	(75.4–96.2) (39)	(72.6–96.7) (30)	(35.9–99.6) (5)
Specificity% (CI95%), n	99.7%	99.6%	99.6%	99.8%	99.1%
	(98.8–99.9) (619)	(98.8–99.9) (703)	(98.8–99.9) (724)	(98.9–99.9) (503)	(96.8–9.9) (221)
NPV% (CI95%)	96.7%	96.3%	96.4%	96.7%	94.8%
	(95.1–97.7)	(94.8–97.3)	(95.0–97.4)	(95.2–97.8)	(90.7–97.1)
PPV% (CI95%)	95.9%	96.3%	94.6%	97.5%	86.0%
	(85.3–99.0)	(94.8–97.3)	(84.9–98.2)	(84.3–99.7)	(57.4–96.5)
	Compared to saliva RT-PCR
NP RDT	0–3 days post onset	0–7 days post onset	Overall	Symptomatic	Asymptomatic
Sensitivity% (CI95%), n	79.2%	78.7%	79.0%	82.7%	60.0%
	(65.0–89.5) (38)	(66.3–88.1) (48)	(66.8–88.3)(49)	(69.7–91.8) (43)	(26.2–87.9) (6)
≤Ct 30, Sensitivity% (CI95%), n	90.0%	90.5%	90.9%	91.2%	83.3%
	(76.3–97.2) (36)	(77.4–97.3) (38)	(78.3–97.5) (40)	(76.3–98.1) (31)	(35.9–99.6) (5)
Specificity% (CI95%), n	99.5%	99.6%	99.6%	99.6%	99.6%
	(98.6–99.9) (618)	(98.8–99.9) (703)	(98.8–99.9) (724)	(98.6–99.9) (502)	(97.5–99.9) (222)
NPV% (CI95%)	97.8%	97.7%	97.8%	98.1%	95.8%
	(96.2–98.7)	(96.3–98.6)	(96.4–98.6)	(96.7–99.0)	(91.4–98.0)
PPV% (CI95%)	94.7%	95.3%	95.5%	95.8%	93.6%
	(85.2–98.3)	(86.7–98.5)	(87.1–98.5)	(85.1–98.9)	(66.1–99.1)

Overall and stratified sensitivity and specificity with 95% confidence interval (CI95%) of the saliva and nasopharyngeal antigen RDT compared to RT PCR. Results were also analyzed by days post onset and symptom status. Negative and positive predictive values (NPV and PPV) were calculated using the mean 9.9% PCR positivity nationally as a proxy for prevalence [9] during the study period.

Fig 1

Cycle thresholds and genome copies of RT-PCR positive samples in relation to days since symptom onset, saliva and nasopharyngeal RDT positivity, and culture outcomes of participation with both symptomatic and asymptomatic patient (n = 62).

Data points shown on the left side of the dashed bar are from asymptomatic individuals. NP, nasopharyngeal swab; RDT, antigen rapid detection test; Ct, cycle threshold; E gene, envelope gene; Neg, negative; Pos, positive; RT-PCR, reverse transcription PCR.

As nasopharyngeal RDT was used for standard diagnostic in this population we have also calculated sensitivity and specificity of the nasopharyngeal RDT compared to RT-PCR in saliva as the only available material and compared the performance of the two RDTs. Overall sensitivity of saliva RDT was lower (66.1% vs. 79.0%). Similarly to saliva RDT, analysis by days post onset did not result in higher sensitivities but this is due to the uneven representation of patients with various days post onset in the study population (majority <3 days). Analysis by symptom status or by days post onset separately a similar difference was seen (symptoms present 69.2% vs. 82.7; <3 days 68.5% vs 79.2%). However sensitivity was similar with Ct ≤30 cutoff (88.6% vs. 90.9%). Generally sensitivity of nasopharyngeal RDT was lower but specificity was equally high and comparable to our previous study [10].

All PCR positive saliva samples were inoculated onto cell culture and approximately half (48.4%, 30/62) resulted in a positive CPE (mean Ct 24.2 of positive samples) Fig 1. Almost all samples with a positive culture result were detected either by nasopharyngeal RDT (90%, 27/30) or by saliva RDT (97%, 29/30). Only one sample was not detected by both RDTs and demonstrated a relatively high Ct-value (Ct 29). Culture negative samples (51.6%, 32/62) had higher Ct values (mean Ct 30.2 (p = 0.001). Results of the two RDTs were partially in concordance as 10/32 were not detected by neither tests. The n = 9 culture negative samples which were not detected by the saliva RDT were in the high Ct value range (median Ct 31.5). However there were n = 13 RDT positive but culture negative samples ranging from low to high Ct.

Discussion

There is very limited data available on the performance of saliva based RDTs [11, 12] and those that are available show below optimal results. The published studies report sensitivities of 63% and 24% compared to nasopharyngeal or nasal swab utilizing RDTs. In our study we have found that people with high viral load (Ct≤30 cutoff) could be detected with a saliva antigen RDT with relatively high sensitivity (88.6%). Furthermore, the majority of presumed infectious individuals (based on cell culture positivity in saliva) could be detected (96.7%). However there is still much to learn about viral kinetics in saliva and how it compares in patients with various disease severity, to other respiratory samples and the role of SARS-CoV-2 specific antibodies in saliva.

Currently nasopharyngeal swab remain the gold standard method for both PCR and RDT with room for exploration for saliva as evidence is still mostly lacking to fully understand the usability and reliability at various time points during disease and in different populations over time [7]. In this study, nasopharyngeal RDT was used as default diagnostic method and also as an additional reference method for the saliva RDT and demonstrated lower sensitivity and comparable specificity to our previous evaluation [10]. Reasons for this difference can be the changing population and also the comparator which was saliva RT-PCR. Using saliva RDT, despite lower general sensitivity compared to both saliva RT-PCR and nasopharyngeal RDT, based on RT-PCR Ct/viral load values almost all presumed infectious individuals could be detected, however this result could not be compared to nasopharyngeal swab virus culture in the current study. Saliva is a complex material shown to have comparable or slightly lower viral load to nasopharyngeal sample [13], it also contains SARS-2 specific antibodies [14] and different viral kinetics compared to nasopharynx which might explain the lower performance in the tested population. Furthermore, we have evaluated this test on non-hospitalized patients and a study found correlation of more severe disease with higher viral load in saliva [15].

Although our study is limited in the suboptimal number of positive samples, it provides promise for further exploration of saliva based RDTs to be used also for self/home testing. We did not investigate acceptance of saliva vs other invasive sample types but one could argue that collection of saliva is easier to perform and causes minimal discomfort and some studies showed high acceptability [16]. Limited studies on usability of RDTs showed relatively high sensitivity and low false negativity related to self-testing and these studies used either nasal or nasopharyngeal samples [17]. However comprehensive studies are still mostly lacking. Risk of aerosol generation could be higher in oral fluid sampling depending on exact sample type involving forced coughing. This risk for home /self–testing does not present an issue however for testing sites proper mitigation steps and set-up need to be thought of.

Availability of easy to use, comfortable RDTs might also increase willingness to test. Multiple studies showed that ease of access ie by central location of testing site or positive experience by minimizing discomfort and rapid availability of results all increased willingness [2, 18, 19]. Increased testing also helps with prompt identification and isolation of cases thereby stopping the spread. However the reliability of the test result depends also on disease prevalence in the population when the test is taken. At the time of our study the mean disease prevalence was 30/100.000 with a mean % PCR positivity of 9.9% resulting in a positive predictive value of the saliva RDT of 94.6%. By the time of the writing of the manuscript the lowest and the highest prevalence ranged between 2.9 (2.3% PCR positivity) and 58.4/100.00 (20.1% PCR positivity). If we apply these scenarios for the test performance, this would translate into 79.1%-97.6% PPV. Given the good specificity of the test the even under high prevalence the NPV still remains over 90% (99.2%– 92.1%) but there will still be individuals who obtain false negative results. The impact on transmission of false negative results should be considered, as individuals may demonstrate lower adherence to non-pharmaceutical measures believing that they tested negative. Furthermore with already high and still increasing vaccination rates continuous decline in prevalence could be expected and need to be taken into account for future testing policies.

Overall, the potential benefits of saliva RDT, could outweigh the lower sensitivity in a targeted and comprehensive testing strategy, especially for home/self-testing and in vulnerable populations like elderly, disabled or children where in intrusive testing is either not possible or causes unnecessary stress.

(XLSX)

Click here for additional data file.

17 Sep 2021

PONE-D-21-21141Clinical evaluation of the SD Biosensor saliva antigen rapid test with symptomatic and asymptomatic, non-hospitalized patients.PLOS ONE

Dear Dr. Iglói,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please re-write the results in detail.

Please submit your revised manuscript by Oct 31 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Etsuro Ito

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following financial disclosure:

The author(s) received no specific funding for this work.

NO - Include this sentence at the end of your statement: The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

At this time, please address the following queries:

a) Please clarify the sources of funding (financial or material support) for your study. List the grants or organizations that supported your study, including funding received from your institution.

b) State what role the funders took in the study. If the funders had no role in your study, please state: “The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”

c) If any authors received a salary from any of your funders, please state which authors and which funders.

d) If you did not receive any funding for this study, please state: “The authors received no specific funding for this work.”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

3. Thank you for stating the following in the Acknowledgments/ Funding Section of your manuscript:

The SARS-CoV-2 Rapid Antigen Test-Standard Q COVID-19 Ag Saliva- Research use only (Lot number QCO9021001; expiry date 04-01-2023) was provided by SD Biosensor

No other funding was received.

We note that you have provided additional information within the Acknowledgements Section that is not currently declared in your Funding Statement. Please note that funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

The author(s) received no specific funding for this work.

NO - Include this sentence at the end of your statement: The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

4. Please upload a new copy of Figure 1 as the format is not accepted. Please follow the link for more information: https://blogs.plos.org/plos/2019/06/looking-good-tips-for-creating-your-plos-figures-graphics/" https://blogs.plos.org/plos/2019/06/looking-good-tips-for-creating-your-plos-figures-graphics/

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The manuscript is clearly written and presents public health relevant findings. It will benefit from minor revision in relation to the data presentation and interpretation in the public health context.

Title: it would be helpful to specify in the title that this saliva test is for SARS-CoV2 antigen.

Abstract: clear and appropriate

Introduction: It would be valuable to introduce an applied epidemiology perspective here; such as

a) The manuscript discusses PPV, yet in public health, the NPV can be equally valuable. Especially in the context of self-tests being used by citizens prior to engaging in increased social interaction. Please elaborate on this perspective (useful reference: European Centre for Disease Prevention and Control. Considerations on the use of self-tests for COVID-19 in the EU/EEA – 17 March 2021. ECDC: Stockholm; 2021)

b) the dependency of population prevalence. In the manuscript, only the prevalence among the tested population is used, though it would be insightful to understand what was the population COVID19 prevalence at the time of this study;

c) Please describe what is known about access to testing in the described population of Rotterdam: how large is the estimated group that is unable to come to these XL-locations? This is relevant context in the discussion on the value of self tests.

Methods: Clearly written. It would be helpful to clarify why only 789 of the 816 included persons had a PCR result.

Results:

- Define 'recent onset' (P4-L89) (from table 1, we deduce this must be <8 days, but this could be made explicit)

- table 1:

Every time when "CT<=30" is printed in the table, this should be "CT<30"

The numbers of CT>=30 and CT<30 among the NP AgRDT group should add up, yet they don't: 9+39=48, yet 49 (of 52) should have a positive PCR. Please check the data

- Table 2: (page 7)

It is not clear why the N is larger than 556 in the columns of "0-3 days post onset" and "0-7 days post onset". It is my understanding that only 556 participants indicated an onset of symptoms.

The row

- Table 2 (page 8)

Here too: It is not clear why the N is larger than 556 in the columns of "0-3 days post onset" and "0-7 days post onset". It is my understanding that only 556 participants indicated an onset of symptoms.

Same on page 9

Discussion:

Clear focus chosen on the discussion topics. What is missing, is a perspective on public health consequences of the PPV and the NPV found in this study: it would not be inappropriate to indulge in a 'what if' scenario, exploring how the saliva test would perform if used widely in the given population of Rotterdam, given the estimated COVID19 prevalence at that time.

Lastly, it would be of interest to elaborate a bit on the risk of aerosolisation while taking the saliva samples, and discuss how that would compare to the other tests used in this study.

The numbers of CT>=30 and CT<30 among the saliva AgRDT group should add up, yet they don't: 2+38=40, yet 41 (of 44) should have a positive PCR. Please check the data

Reviewer #2: Review of “Clinical evaluation of the SD Biosensor saliva antigen rapid test with symptomatic andasymptomatic, non-hospitalized patients”. Short article with promising results. Methods section is very clear. The results could be better displayed as far as I am concerned. Also, the statistical comparison that is made to compare sensitivity estimates by time since symptom onset needs to be looked at more closely. Furthermore, I found it very encouraging that the saliva antigen test was able to detect 97% of the positive viral cultures. This is in my opinion a very convincing result to implement this easy-to-use test as a public health tool for in particular health care workers and for the general public for SARS-CoV-2 and potentially beyond for other infectious diseases as well for example influenza.

Major points:

I find the results section too brief. I would like to see clearer sentences where it is made explicit which two tests are being compared and show how the sensitivity was estimated. For example, "of the XX samples that tested positive with PCR saliva, XX tested positive with the saliva antigen test (XX/ XX = XX sensitivity)." I find it difficult at this point to determine from the text which tests were compared and how sensitivity was estimated.

To me the main finding of this study is that the saliva RDT detected all but one positive virus cultures. Assuming virus culture is the best indicator of infectiousness, the saliva RDT is in this study the best test to detect infectious persons, and would therefore be very suitable for at home use, and therefore be an excellent public health tool to stop transmission. I think this finding deserves a better location in the results and discussion section.

Line 100: Overall performance of saliva was inferior to these other two tests. But comparing one test to another will always result in an inferior test, right? There is always going to be one sample, which was tested positive in the bench mark test, that will be negative in the evaluated test? Also, inferior in finding what? RNA or samples with infectious potential? The saliva test found more of the samples that tested positive in viral culture than the nasopharyngeal test, right? In that sense, the saliva test is superior over the nasopharyngeal test finding samples that positive with positive culture.

Table 2: I am confused about the N-values. In total, there were 789 samples tested. Of these, 62 were PCR-positive, 52 with the NP Ag RDT, and 44 with the saliva Ag RDT. The sensitivity estimates can therefore never be estimated with the numbers I am seeing in this table. I would provide the actual numbers from which the sensitivity estimates were estimates in the parentheses instead of one N number, for example: 79.2% (XX/XX)

“Analysis by days post onset did not result in higher sensitivities”. How was this concluded? Looking at table 2, I see percentages for the group 0-3 days post onset and the group 0-7 post onset. For a proper comparison, one would have to make two different groups, for example comparing samples taken 0-3 days after symptom onset, and samples taken 4-7 days post symptom onset.

Minor points:

Line 47: People were either tested because of respiratory symptoms or because of being a contact of an individual with a confirmed SARS-CoV-2 infection as mentioned in line 47/48. Could you provide information on the number of samples that were from contact persons, perhaps in Table 1?

Line 62: This sentence requires rewriting.

Line 98: To me, this summary statistic is redundant and without meaning.

Line 99: Sensitivity increased to 88.6, but where is it increasing from?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Arnold Bosman

Reviewer #2: Yes: Tom Woudenberg

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

1 Nov 2021

POINT-TO-POINT RESPONSE TO THE REVIEWER

We thank the reviewers for their valuable comments, which we will address point-to-point below. The manuscript underwent significant revision thanks to the comments. The line numbers below correspond to the manuscript without track changes.

Reviewers comments:

Reviewer #1:

The manuscript is clearly written and presents public health relevant findings. It will benefit from minor revision in relation to the data presentation and interpretation in the public health context.

Title: it would be helpful to specify in the title that this saliva test is for SARS-CoV2 antigen.

Specified

Abstract: clear and appropriate

Introduction: It would be valuable to introduce an applied epidemiology perspective here; such as

a) The manuscript discusses PPV, yet in public health, the NPV can be equally valuable. Especially in the context of self-tests being used by citizens prior to engaging in increased social interaction. Please elaborate on this perspective (useful reference: European Centre for Disease Prevention and Control. Considerations on the use of self-tests for COVID-19 in the EU/EEA – 17 March 2021. ECDC: Stockholm; 2021)

Comment: addressed in detail in results (Table 2) and discussion. Also as a study was running for 4 weeks and during that period the amount of positive people ie % PCR positivity changed the mean value was used to calculate NPV and PPV. Furthermore in the discussion scenarios with lowest and highest % PCR positivity figures were explored. (Lines 221-231)

b) the dependency of population prevalence. In the manuscript, only the prevalence among the tested population is used, though it would be insightful to understand what was the population COVID19 prevalence at the time of this study;

Clarified in results section (Lines 128-129)

c) Please describe what is known about access to testing in the described population of Rotterdam: how large is the estimated group that is unable to come to these XL-locations? This is relevant context in the discussion on the value of self tests.

Comment: it is difficult to know precisely, we can only estimate from the number of people having low socio economic status (SES) in Rotterdam. Several studies have shown that marginalised people often represent hard to reach population. Furthermore, self-testing is not only to reach marginalized people but also to encourage more frequent testing in any population. Some of these discussion points are included in the discussion.

Methods: Clearly written. It would be helpful to clarify why only 789 of the 816 included persons had a PCR result.

Clarified in Lines 84-85

Results:

- Define 'recent onset' (P4-L89) (from table 1, we deduce this must be <8 days, but this could be made explicit)

Comment: in this particular case (now line 125) recent onset is clarified in the bracket following the statement ie 2 days. However point taken and statement clarified throughout the manuscript.

- table 1:

Every time when "CT<=30" is printed in the table, this should be "CT<30"

Corrected: Categories changed slightly: Ct ≤ 25, Ct ≤ 30 and Ct >30

The numbers of CT>=30 and CT<30 among the NP AgRDT group should add up, yet they don't: 9+39=48, yet 49 (of 52) should have a positive PCR. Please check the data

Correction and comment: the categories were changes as detailed in the above point. The reason for the difference in total RDT positives and the total which has ct value is due to specificity of the RDT ie false positives detected by RDT. For clarification an extra row was added in the table.

The numbers of CT>=30 and CT<30 among the saliva AgRDT group should add up, yet they don't: 2+38=40, yet 41 (of 44) should have a positive PCR. Please check the data

Correction and comment: see comment above.

- Table 2: (page 7)

It is not clear why the N is larger than 556 in the columns of "0-3 days post onset" and "0-7 days post onset". It is my understanding that only 556 participants indicated an onset of symptoms.

Corrected and comment: indeed confusingly the total population size was displayed not the concerning true positives/true negatives which are necessary for calculating the sensitivity/specificity.

The row

- Table 2 (page 8)

Here too: It is not clear why the N is larger than 556 in the columns of "0-3 days post onset" and "0-7 days post onset". It is my understanding that only 556 participants indicated an onset of symptoms.

Same on page 9

Clarified: However 556 people stated presence of symptoms only 545 provided date of onset. This was clarified in the text (Line 126)

Discussion:

Clear focus chosen on the discussion topics. What is missing, is a perspective on public health consequences of the PPV and the NPV found in this study: it would not be inappropriate to indulge in a 'what if' scenario, exploring how the saliva test would perform if used widely in the given population of Rotterdam, given the estimated COVID19 prevalence at that time.

Added: Lines 221-231

Lastly, it would be of interest to elaborate a bit on the risk of aerosolisation while taking the saliva samples, and discuss how that would compare to the other tests used in this study.

Edited: the above mentioned points were addressed as much as possible in the discussion (Lines 212-215).

Reviewer #2:

Review of “Clinical evaluation of the SD Biosensor saliva antigen rapid test with symptomatic and asymptomatic, non-hospitalized patients”. Short article with promising results. Methods section is very clear. The results could be better displayed as far as I am concerned. Also, the statistical comparison that is made to compare sensitivity estimates by time since symptom onset needs to be looked at more closely. Furthermore, I found it very encouraging that the saliva antigen test was able to detect 97% of the positive viral cultures. This is in my opinion a very convincing result to implement this easy-to-use test as a public health tool for in particular health care workers and for the general public for SARS-CoV-2 and potentially beyond for other infectious diseases as well for example influenza.

Major points:

I find the results section too brief. I would like to see clearer sentences where it is made explicit which two tests are being compared and show how the sensitivity was estimated. For example, "of the XX samples that tested positive with PCR saliva, XX tested positive with the saliva antigen test (XX/ XX = XX sensitivity)." I find it difficult at this point to determine from the text which tests were compared and how sensitivity was estimated.

Result section was rewritten and restructured and part of Table 2 removed as I believe that was confusing and redundant information.

To me the main finding of this study is that the saliva RDT detected all but one positive virus cultures. Assuming virus culture is the best indicator of infectiousness, the saliva RDT is in this study the best test to detect infectious persons, and would therefore be very suitable for at home use, and therefore be an excellent public health tool to stop transmission. I think this finding deserves a better location in the results and discussion section.

Comment: this was highlighted better both in results and discussion. In this study we did not culture from nasopharyngeal swabs however we did that in our previous studies and similar results were achieved meaning by Ct cut-off and days post onset ie early phase more infectious. In this current study days post onset did not result in increase due to the above detailed skewed dataset.

Line 100: Overall performance of saliva was inferior to these other two tests. But comparing one test to another will always result in an inferior test, right? There is always going to be one sample, which was tested positive in the bench mark test, that will be negative in the evaluated test? Also, inferior in finding what? RNA or samples with infectious potential? The saliva test found more of the samples that tested positive in viral culture than the nasopharyngeal test, right? In that sense, the saliva test is superior over the nasopharyngeal test finding samples that positive with positive culture.

Comment: Indeed most comparisons going to be inferior and in this case well expected as we compared a very sensitive molecular test to a serological test which never can be as sensitive as the molecular. It was inferior in performance to detect every RT-PCR or nasopharyngeal RDT positive person but the reviewer is right pointing out the additional value of detecting infectious individuals. Real value does come from detecting these people. Section was rephrased to reflect this (Lines 165-173 and 195-200).

Table 2: I am confused about the N-values. In total, there were 789 samples tested. Of these, 62 were PCR-positive, 52 with the NP Ag RDT, and 44 with the saliva Ag RDT. The sensitivity estimates can therefore never be estimated with the numbers I am seeing in this table. I would provide the actual numbers from which the sensitivity estimates were estimates in the parentheses instead of one N number, for example: 79.2% (XX/XX)

Correction and comment: Numbers were indeed confusing therefore table was edited to make it clearer.

“Analysis by days post onset did not result in higher sensitivities”. How was this concluded? Looking at table 2, I see percentages for the group 0-3 days post onset and the group 0-7 post onset. For a proper comparison, one would have to make two different groups, for example comparing samples taken 0-3 days after symptom onset, and samples taken 4-7 days post symptom onset.

Correction and comment: in previous studies big difference was usually seen between people who had symptom onset 0-3 days vs 0-7. In this population/study we did not see this due to the uneven split of the population who participated in the study and the comment refers to this fact. It was made clear both by the presented numbers in Table 2 (48 vs 38 people with symptom onset 0-7 days vs 0-3 days) and in the text (in abstract and Line 151-153)

Minor points:

Line 47: People were either tested because of respiratory symptoms or because of being a contact of an individual with a confirmed SARS-CoV-2 infection as mentioned in line 47/48. Could you provide information on the number of samples that were from contact persons, perhaps in Table 1?

Answer: this specifically was not recorded in the questionnaire, one can assume that many of the asymptomatic people were contacts.

Line 62: This sentence requires rewriting.

Comment: sentence was rephrased (Line 83-85).

Line 98: To me, this summary statistic is redundant and without meaning.

Comment: whole result section was rewritten and restructured.

Line 99: Sensitivity increased to 88.6, but where is it increasing from?

Comment: sentence was rephrased. Increase from 63%, the overall sensitivity.

________________________________________

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

19 Nov 2021

Clinical evaluation of the SD Biosensor SARS-CoV-2 saliva antigen rapid test with symptomatic and asymptomatic, non-hospitalized patients.

PONE-D-21-21141R1

Dear Dr. Iglói,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Etsuro Ito

Academic Editor

PLOS ONE

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The edited manuscript successfully addresses all points in the review. I would consider this manuscript appropriately suitable for publication.

Reviewer #2: Excellent revision. Results are very clear.

One final comment from my side, and I leave this up to the authors whether they'd like to change this. I'd change the first sentence of the discussion. I think it should be made in clear what the performance refers to. Is its poorer performance in detecting PCR+ve samples or positive cell cultures?

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Arnoldus Bosman

Reviewer #2: Yes: Tom Woudenberg

13 Dec 2021

PONE-D-21-21141R1

Clinical evaluation of the SD Biosensor SARS-CoV-2 saliva antigen rapid test with symptomatic and asymptomatic, non-hospitalized patients.

Dear Dr. Iglói:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Prof. Etsuro Ito

Academic Editor

PLOS ONE