Performance evaluation of a non-invasive one-step multiplex RT-qPCR assay for detection of SARS-CoV-2 direct from saliva.

Polymerase chain reaction (PCR) has proven to be the gold-standard for SARS-CoV-2 detection in clinical settings. The most common approaches rely on nasopharyngeal specimens obtained from swabs, followed by RNA extraction, reverse transcription and quantitative PCR. Although swab-based PCR is sensitive, swabbing is invasive and unpleasant to administer, reducing patient compliance for regular testing and resulting in an increased risk of improper sampling. To overcome these obstacles, we developed a non-invasive one-step RT-qPCR assay performed directly on saliva specimens. The University of Nottingham Asymptomatic Testing Service protocol simplifies sample collection and bypasses the need for RNA extraction, or additives, thus helping to encourage more regular testing and reducing processing time and costs. We have evaluated the assay against the performance criteria specified by the UK regulatory bodies and attained accreditation (BS EN ISO/IEC 17,025:2017) for SARS-CoV-2 diagnostic testing by the United Kingdom Accreditation Service. We observed a sensitivity of 1 viral copy per microlitre of saliva, and demonstrated a concordance of > 99.4% between our results and those of other accredited testing facilities. We concluded that saliva is a stable medium that allows for a highly precise, repeatable, and robust testing method.

Introduction

Since the emergence of Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) in 2019 in the Wuhan province China[1, 2], much attention has been given to the necessity of surveillance and molecular testing amongst populations worldwide to curb viral propagation. SARS-CoV-2 is the third known coronavirus capable of animal-to-human transmission, after SARS-CoV and MERS-CoV[2]. However, SARS-CoV-2 is a greater issue than its predecessors in terms of the number of global infections and infection-related mortality[2] and has fast become a global pandemic.

UK government statistics reported over 18.7 million confirmed COVID-19 cases since the emergence of the pandemic, resulting in over 194,000 deaths (https://coronavirus.data.gov.uk; accessed 23-May-2022). Globally, the World Health Organisation (WHO) statistics estimate over 522 million confirmed cases with over 6.2 million deaths (https://covid19.who.int; accessed 23-May-2022). Indeed, public health services have endured immense pressure amid rising hospital admissions, imparting considerable strain on an already overstretched medical staff. The majority of SARS-CoV-2 diagnostic testing is performed only when an infection is suspected, likely following the onset of characteristic symptoms e.g., fever, persistent cough, or the loss of taste or smell. However, there is now a greater appreciation for the impact of infections that are asymptomatic or have non-standard characteristics on subsequent transmission through communities. Indeed, meta-analysis shows that approximately 30% of infections remain asymptomatic[3, 4]. Evidence suggests that general population testing can reduce transmission, and is correlated to a decline in SARS-CoV-2-related mortality[5].

Rapid tests have been developed recently and have become widely used despite concerns around their accuracy; 34.1–88.1%[6]. The frontline and current gold-standard diagnostic tests for the detection of SARS-CoV-2 infection utilise nucleic acid amplification tests usually via RT-qPCR, most commonly from specimens obtained through nasopharyngeal swab (NPS) or oropharyngeal swab (OPS). Swab testing, although commonplace, is highly dependent on correct administration to yield accurate results; however, it is considered invasive and unpleasant by many, unsuitable for the very young or some individuals with learning disabilities, and the discomfort induced during an accurate swabbing procedure may potentially result in an unwillingness to test regularly[7]. Importantly, inappropriate nasopharyngeal swab sampling may associate with false-negative results, thus contributing to the spread of infection[8], and reducing confidence in the accuracy of the tests. Oral saliva self-provision as a means of specimen collection would benefit the aforementioned groups by being non-invasive, free of discomfort, and not requiring a forceful expulsion as with deep throat sputum. SARS-CoV-2 diagnostic tests utilising saliva therefore present an attractive alternative whilst being cheaper to process[9, 10].

Saliva is undoubtably a very heterogenous substance comprising electrolytes, proteins, lipids, amino acids, gingival crevicular fluid, antibodies, immune and epithelial cells, food matter, bacteria, and viruses[11, 12]. Nonetheless, the current literature contains a wealth of evidence in support of the use of saliva as a suitable medium for clinical diagnostics. To et al. report that salivary viral load is highest in the first week of infection, and the virus detectable in saliva up to 25 days after the onset of symptoms[13]. Furthermore, as angiotensin converting enzyme 2 (ACE-2), the entry receptor for SARS-CoV-2, is expressed within the oral mucosal epithelial cells[14, 15], and oral fluids have the potential to transmit SARS-CoV-2 infection[10], the concept that saliva is a key site for viral detection is supported. A study by Silva et al. has shown that the salivary viral load correlates to COVID-19 disease severity and may correlate with prognosis[10, 16]. Wyllie et al. report that saliva specimens provided a greater sensitivity for viral detection than swabs from hospital in-patients[17]. Furthermore, despite a slightly lower sensitivity reported, Caulley et al. find that 20% of infected persons returned a positive result by saliva but not via matched NPS[18]. A meta-analysis by Czumble et al. has found the sensitivity of saliva tests to be slightly lower yet comparable to those based on NPS (91% c.f. 98% respectively)[11]. This is a finding commonly reported in the literature, yet with notable benefits in terms of sample provision and user acceptability. However, the inconsistencies in the administration of the swabs and the specific PCR protocols used to diagnose the infection have to be considered for these comparisons.

The impact that the SARS-CoV-2 pandemic has had on the provision of teaching and the continuation of research in higher education institutions was devastating. At the University of Nottingham, we identified the need to establish a SARS-CoV-2 testing service to be offered to all university students and staff in order to monitor transmission within our community, hence allowing mitigating actions to reduce further transmission chains. To this end, we developed a simple, yet sensitive multiplexed RT-qPCR assay intended for the diagnostic detection of SARS-CoV-2 RNA in heat-inactivated saliva samples, specifically in asymptomatic individuals. The use of saliva bypasses the need for swab-based specimen provision and allows for effortless self-sampling, thus minimising the risk of improper administration and reducing the requirement for consumables in national shortage (such as swabs). Furthermore, our one-step assay eliminates the time-consuming and costly RNA purification steps and is based on the Centre for Disease Control and Prevention (CDC) approved Quantabio® UltraPlex 1-Step ToughMix™, an RT-qPCR reaction mix developed to be resistant to PCR inhibitors that can be present in crude biological samples such as saliva[19].

The detection of viral genome in the sample relies on three sets of primers and probes specific for as many gene targets (Table 1): The envelope (E) gene (from the Charité/Berlin method, as listed in the WHO-approved SARS-CoV-2 detection protocols), the nucleocapsid (N) gene (from the US-CDC’s nCOV_N2 assay), and the human RNase P gene (used as a control for saliva-sample integrity in the US-CDC_RNase P assay)[19-21].

Table 1

Primers and probes used in the assay, with respective sequences, modifications, and final qPCR concentrations.

Primer/probe	Amplicon size	Sequence/label/modifications	Concentration (nM)
CDC-nCOV_N2 Forward	67 bp	5’-TTACAAACATTGGCCGCAAA-3’	250
CDC-nCOV_N2 Reverse		5’-GCGCGACATTCCGAAGAA-3’	250
CDC-nCOV_N2 Probe		5’-FAM-ACAATTTGC/ZEN/CCCCAGCGCTTCAG-IBFQ-3’	62.5
Charité/Berlin E_sarbeco Forward	113 bp	5’-ACAGGTACGTTAATAGTTAATAGCGT-3’	200
Charité/Berlin E_sarbeco Reverse		5’-ATATTGCAGCAGTACGCACACA-3’	200
Charité/Berlin E_sarbeco Probe		5’-HEX-ACACTAGCC/ZEN/ATCCTTACTGCGCTTCG-IBFQ-3’	100
CDC-RNase P Forward	65 bp	5’-AGATTTGGACCTGCGAGCG-3’	62.5
CDC-RNase P Reverse		5’-GAGCGGCTGTCTCCACAAGT-3’	62.5
CDC-RNase P Probe		5’-ATTO647-TTCTGACCT/ZEN/GAAGGCTCTGCGCG-IBFQ-3’	62.5

In 2021, the UoNATS was assessed by the United Kingdom Accreditation Service (UKAS), and subsequently awarded accreditation as a SARS-CoV-2 testing provider (Testing Laboratory No. 22743). The following experimental data describe the methods used for the in-house validation of the UoNATS direct RT-qPCR diagnostic test, as well as a comparison with results generated from independent accredited laboratories. The validation work was performed with consideration to the Public Health England’s (PHE) published quality guidance on the UK Standards for Microbiology Investigations (UK SMI)[22], the Medicines and Healthcare Regulatory Agency (MHRA) target product profile (TPP) specifications for SARS-CoV-2 viral detection testing, and adhering to the applicable criteria defined within the European Standard for the competence of testing and calibration laboratories (BS EN ISO/IEC 17,025:2017). Whilst a parallel submitted manuscript (Tarantini et al.[23]) provides stepwise protocols for the UoNATS assay, this article defines the quality metrics used to validate and accredit the UoNATS SARS-CoV-2 detection method, as assessed by the following parameters: Linearity and range, analytical sensitivity (limit of detection, LOD), analytical specificity, diagnostic (clinical) sensitivity and specificity, positive and negative predictive value, repeatability and sample stability, accuracy and precision, multiplex validation, robustness and reproducibility.

Materials & methods

Ethical approval

The studies involving human participants were reviewed and approved by University of Nottingham Ethics Committee, reference number FMHS 96-0920. The patients/participants provided informed consent to participate in this study. All methods were performed in accordance with the relevant guidelines and regulations.

Specimen collection and processing

Detailed methods have been provided in a separate publication[23]. Clinical samples (required volumes between 0.1 and 1.9 mL) are inactivated prior to downstream processing; vials are heat-inactivated in a laboratory oven with the use of a temperature probe to ensure samples are held at 95 °C for 5 min, shown to inactivate SARS-CoV-2[24]. Samples are allowed to cool, then mixed thoroughly prior to PCR.

RT-qPCR

RT-qPCR is performed in a one-step protocol using primer and probe sequences as recommended by the US-CDC and the Charité/Berlin panel[20]; US-CDC-nCOV_N2:FAM, US-CDC_RNaseP:ATTO647, Charité-E:HEX (Integrated DNA Technologies). Quantabio UltraPlex 4 × Toughmix (VWR; Cat. 95,166) mastermix was used. Oligo sequences and final concentrations are given in (Table 1); reagents used in each 20 µL qPCR reaction were 5 µL of Ultraplex™ ToughMix (4x), 0.25 µL of primer and probe mix, 6.75 µL of nuclease-free water and 8 µL of heat-inactivated saliva. Nuclease-free water, and commercial SARS-CoV-2 standard of known concentration at 2 viral copies per microlitre (vc µL−1) are included in triplicate in every run as non-template and positive controls, respectively. All qPCR reactions were performed using 96-well low-profile PCR plates (Eppendorf TwinTec; VWR Cat. No. 732-0107) and optical plate sealers (Thermo-Fisher; Cat. No. AB-0558), on a Bio-Rad C1000 thermal cycler with CFX96 Touch™ optical unit, cycling conditions are described in (Table 2). Data analysis is performed using the Bio-Rad CFX Maestro software package. Positive samples were verified with a second confirmatory PCR. Positivity is determined where the CT for one or both SARS-CoV-2 targets is < 35; an inconclusive result is attributed to a sample with CT of > 35 but < 37; results of CT > 37 are deemed negative. As with all PCR-based assays, a positive result indicates the presence of viral nucleic acid in the specimen, however, cannot discern between current active or recent past infection.

Table 2

RT-qPCR Cycling conditions.

Step	Temperature (°C)	Duration (min:sec)	Cycles
Reverse Transcription	50	10:00	x1
RT Inactivation/Initial Denaturation	95	03:00	x1
Denaturation	95	00:03	x45
Annealing	55	00:30
Extension (Acquisition)	72	00:15

Acquisition of FAM: 510–530 nm; HEX: 560–580 nm; ATTO647 (Cy5 channel): 675–690 nm.

Linearity and range

To assess the linearity and dynamic range of the assay, a previously tested clinical SARS-CoV-2-positive sample was serially diluted in a pool of SARS-CoV-2-negative saliva derived from 30 individual negative samples and tested in triplicate. R2 values of > 0.95 are considered acceptable. The CT range generated spans 15–40 which is considered representative of high-to-low viral loads and comprises the vast majority of samples received at our facility.

Analytical sensitivity (limit of detection, LOD)

A commercial external run control (NATrol™ NATSARS(COV2)-ERC, ZeptoMetrix Corp.) consisting of whole chemically inactivated virions at 50 vc µL−1 was diluted from 1:3.125 (16 vc µL−1) to 1:400 (0.125 vc µL−1) in serial two-fold dilutions using a pool of combined negative saliva from 30 previously tested SARS-CoV-2-negative samples as the diluent. Replicate samples (n = 6, or n = 12 for lower concentrations) were tested. The experimental setup was performed in duplicate by separate operators using different pools of negative saliva diluent. The limit of detection is defined as the lowest concentration tested whereby 100% of replicates are detected by the assay.

Analytical specificity: cross reactivity of the UoNATS reaction mix

To assess potential cross-reactivity of the UoNATS assay with other commonly infecting pathogens, a commercial respiratory verification panel (NATRVP2.1-BIO; ZeptoMetrix, USA) was used as a control. The panel contains 21 inactivated microorganisms known to infect the respiratory system of humans, thus with the potential to be present in the specimens collected in the UoNATS workflow.

To monitor the on-going ability of the assay to detect prevalent variant strains of SARS-CoV-2, the primer probe sequences are periodically BLAST-aligned against sequences submitted to the EpiCov database, hosted by the Global Initiative for Sharing All Influenza Data (GISAID). Strains with 2 or more mutations in each of the assay target regions, or 1 or more mutations in the 3’ end of each target sequence are reported. The UK National External Quality Assurance Scheme (UK NEQAS) provides samples of known lineage, distributed monthly, from which we can verify detection of SARS-CoV-2 variants.

Diagnostic sensitivity and specificity

The diagnostic (clinical) sensitivity and specificity was assessed by testing both positive (n = 163) and negative (n = 250) saliva specimens and nasopharyngeal swab specimens submitted by the same donor preferably on the same day. Exclusion criteria were if swab and saliva had been provided > 5 days apart. The swab samples were tested by independent accredited testing facilities: Source Bioscience; Nottingham, UK (Laboratory No. 9571) & Nottingham University Hospitals NHS Trust (Laboratory No. 8848), saliva samples were tested in-house. Clinical concordance of the UoNATS results with those of external testing facilities were calculated as the percent agreement of the results. Concordance of 95% to 100% is considered acceptable.

Positive and negative predictive values (PPV and NPV)

To estimate the risk of false-positive or false-negative results, the diagnostic sensitivity and specificity were each used to calculate the PPV and NPV, respectively for the UoNATS test. PPV is calculated using the formula: PPV = true positives/(true-positives + true negatives) and NPV is calculated using the formula: NPV = true negatives/(false negatives + true negatives), as specified by Public Health England (PHE)[22]. The term ‘true’ herein shall refer to the results derived from independent testing facilities (Source Bioscience; Nottingham, UK & Nottingham University Hospitals NHS Trust) using a gold-standard (i.e., RT-qPCR) testing methodology for the reference method. The positive and negative predictive values were then estimated for several scenarios with low to high prevalence in the population (1, 5, 10, 25 and 50%) as specified by the MHRA TPP for laboratory-based SARS-CoV-2 viral detection tests (version 2.0), and for the estimated prevalence for November 2021 for the University of Nottingham students and staff population (0.61%) as determined by our own workflow. NPV and PPV were determined for specimens tested a single time as first-line screening, and after a subsequent second confirmatory test.

Sample stability, repeatability, reproducibility

To ascertain the stability of viral nucleic acid in saliva, three SARS-CoV-2-positive samples were aliquoted on the day of detection and aliquots (n = 10) stored at room temperature (18–22 °C), 4 °C, or − 80 °C. Aliquots kept at − 80 °C were re-usable and so represent a sample with multiple freeze/thaw cycles. Each set of aliquots was tested in triplicate every 2–3 days for a period of 20 days. Stability of viral RNA in the medium was inferred by the consistency of CT values throughout the time course.

The repeatability of the assay was determined by analysing the mean CT, standard deviation, and co-efficient of variation derived from the daily use of a standard of known concentration (2 × LOD) over 100 days; the co-efficient of variation should not exceed 5%.

To establish the reproducibility of results, 7 SARS-CoV-2-positive samples (including 1 known problematic specimen; sample 7) were tested in triplicate by each one of four laboratory operators using independently prepared reaction mixes and equipment. Tests were performed on the same day to exclude CT variations linked to storage conditions.

Accuracy and precision

An independently validated negative sample was spiked with an external positive control (NATrol™ NATSARS(COV2)-ERC, ZeptoMetrix Corp.) consisting of whole chemically inactivated virions at the limit of detection for the assay (determined as 1 vc µL−1). The sample was tested in 24 technical replicates by the same operator, standard deviations and co-efficient of variation was calculated; %CV should not exceed 10% for confidence in the accuracy and precision of the method.

Multiplexing validation

To assess if multiplexing the three primer probe sets resulted in any inhibition to the reactions which may affect the quantification and subsequent test result, dilutions of a previously tested SARS-CoV-2 positive clinical sample were produced and tested using; (1) CDC-N2 only; (2) Charité-E only; (3) UoNATS triplex N2/E/RP assay conditions. CT changes between mono/triplex of ≤ 1 were considered acceptable.

Robustness

The robustness evaluation of the assay was determined by quantifying the variation in results derived from the assay when under pressure from confounding variables; including, inaccurate pipetting volumes, and the uncontrollable presence of PCR inhibitory factors which may be present in the heterogenous saliva matrices. Co-efficient of variations of 0–5% were considered acceptable.

Results

To establish the accuracy of the assay over a range of viral concentrations, a strong positive clinical sample was serially diluted to yield a CT range of 15–40, encompassing the expected range for the vast majority of the samples tested in our workflow and intended to represent high-to-low viral load. PCR efficiencies for both N2 and E primer/probe assays were calculated at 107% when combined in the multiplexed assay, with R2 values all > 0.98 demonstrating accuracy and precision through this range (Fig. 1a).

Figure 1

Data in blue represents the CDC-N2 assay, and red data for Charité-E. Mean CT is plotted, error bars denote the standard deviation. (a) PCR efficiency and linear range derived from a serially diluted known positive clinical sample, spanning a range of high-to-low viral loads. The detection rates and LOD for the N2 and E assays are shown in panels (b) and (c) respectively. (d) The CT values of a positive sample tested over 20 days under various storage conditions. (e) Robustness results between operators and reagents. (f) Repeatability and reproducibility when sequentially testing a commercial standard at near-LOD daily for 100 days.

The limit of detection (LOD) of the assay was determined as the lowest viral concentration at which 100% detection is observed. A requirement for tests to detect viral loads of ≤ 1 vc µL−1 are specified by the relevant UK governing bodies (PHE & MHRA). The UoNATS assay can reliably detect 100% of replicates with a viral load equivalent to 1 viral copy per microlitre (vc µL−1) of saliva or above, for both gene targets. Figure 1b and c present the results from viral concentrations between 16 vc µL−1 and 0.25 vc µL−1. Each plot shows the combined results from 2 separate experiments.

Analytical specificity

To ensure the UoNATS assay does not detect other respiratory-infecting microorganisms which may be present in deposited samples, we tested the assay using a commercial respiratory verification panel consisting of 21 respiratory pathogens. There was no reactivity with either of the SARS-CoV-2 targets, N2 or E. A positive result was determined for both targets for SARS-CoV-2 (Supplementary Table S1).

Primer and probe sequences used in the UoNATS assay were BLAST-aligned against submitted SARS-CoV-2 sequences in the EpiCov (GISAID) database to ensure the assay will confidently detect variants of concern (Supplementary Table S2a & b). Those strains with 2 or more mutations in the assay regions, or 1 mutation if in the 3' end, were quantified. Omicron (B.1.1.529) and sub lineages, now > 99.3% of new sequences in Europe (EpiCov, GISAID; 17th May 2022) have a single C > T mismatch in the 5’ end of the Charité-E forward primer binding region, this has not been observed to affect assay performance. Only 1 of 576 strains analysed (0.17%) had a single mismatch in a 3’ assay region, from an isolate of the BA.2 sub lineage. No prevalent variants were identified with mutations in both N2, and E target regions simultaneously thus likely to affect viral detection in our assay.

The UoNATS assay correctly detected the wild-type, alpha, beta, gamma, and delta variants as part of the UK NEQAS quality assurance scheme (Supplementary Table S3), a small number of samples were confirmed as omicron via sequencing, validating our ability to detect currently circulating variants.

Diagnostic sensitivity & specificity

The clinical sensitivity and specificity of the method was assessed by comparing the results from SARS-CoV-2 nasopharyngeal swabs tested by independent accredited testing facilities (Source Bioscience, Nottingham, UK; and Nottingham University Hospitals NHS Trust) with UoNATS saliva results. The concordance was calculated by matching the results of 250 samples deemed negative by external testing facilities using our assay, all confirmed with exception of one false positive; and 163 samples tested positive by external testing facilities, also confirmed using our assay with one exception, being detected but falling beyond our internal CT cut-off of 37. We observed a 99.4% and 99.6% agreement with external positive and negative results, respectively. The corresponding Negative and Positive Predictive Values were 98.78% and 99.20%, respectively, satisfying the acceptance criteria (> 95%).

Positive (PPV) and negative (NPV) predictive values for various SARS-CoV-2 prevalence

To determine the number of false positives and false negatives which this assay could generate at fluctuating SARS-CoV-2 prevalence, a population of 10,000 was considered and the clinical sensitivity and specificity used. For a first round of tests, initial screening, the PPV was between 60 and 100% and the NPV was between 99 and 100%. When the number of positive samples detected are subjected to a second test, confirmatory test, the overall PPV is 99–100% and the NPV, 100%. Details of estimated predictive values based on populations with varying SARS-CoV-2 incidence are given in Table 3 (and described in Supplementary Table S4). In each case, the positive predictive value (PPV) was obtained by dividing the number of true positive cases into the total positive results. The negative predictive values (NPV) by dividing the number of true negative cases into the total negative results (see detailed methods).

Table 3

Positive and negative predictive value calculations after initial screening and confirmatory test of positive samples.

SARS-CoV-2Prevalence (%)	Positive predictive value (ppv)Initial screening (%)	Negative predictive value (NPV)Initial screening (%)	Positive predictive value (PPV)Confirmatory test (%)	Negative predictive value (NPV)Confirmatory test (%)
0.61	60	100	99.4	100
1	72	100	99.5	100
5	93	100	100	100
10	97	100	100	100
25	99	100	100	100
50	100	99	100	100

Sample stability, repeatability, and reproducibility

To further investigate the performance of the assay, we questioned the stability of viral nucleic acid in saliva over time in different storage conditions. SARS-CoV-2 RNA in clinical samples was stable within saliva for up to 20 days irrespective of storage temperature (RT: 18–22 °C, 4 °C, or − 80 °C), and was robust through at least 8 freeze/thaw cycles without detriment to the CT values obtained (Fig. 1d). Detailed statistics are given in Supplementary Table S5.

The mean CT values from daily testing of a commercial standard over a period of 100 days were collated and used to quantify the repeatability and reproducibility of the assay. This test reflects multiple batches of reagents, alternative users, PCR machines, and pipettes. Our assay demonstrated highly repeatable results, with co-efficient of variations of < 2.4% and < 2.1% for N2 and E, respectively (Fig. 1f).

We next tested 7 clinical positive samples using different PCR machines, reagents, and laboratory members to estimate reproducibility of the method and the effect of varying compositions of different saliva matrices on the assay performance. We observed a maximum co-efficient of variation of < 2.4%, with the exception of 1 sample (sample 7, 5.4% CV) which was highly viscous and difficult to process (Fig. 1e).

The descriptive statistics determined from multiple replicate tests of the same sample are presented in Supplementary Table S6. Commercially sourced SARS-CoV-2 positive run control was added into SARS-CoV-2 negative saliva to a concentration of 1 vc µL−1 corresponding to the limit of detection of the assay where variability would be most pronounced. A standard deviation of 0.9 and 1.0 and co-efficient of variation of 2.6% and 2.9% were observed across individual replicates (n = 24) for the N2 and E targets, respectively demonstrating precision of measurement.

Multiplex validation

To ascertain if multiplexing the primer/probe assays is detrimental to the PCR performance, serially diluted SARS-CoV-2 positive saliva was tested; the CT values for the individual N2, or E reactions were compared to those derived from the same samples using the multiplexed reaction mixture. The results showed a high concordance (see Supplementary Fig. 1 & Table S7) with mean CT differences between mono- and triplex of 0.25 for N2, and 0.17 for E.

Influence of matrix in the assay

To address the question of the heterogeneity of saliva on the accuracy of the assay; 28 samples were randomly selected from a pool of clinical samples previously confirmed as negative. Viral particles from commercial standards were added to each sample to a concentration of 1 vc µL−1 (LOD). Each sample was tested and the mean CT ± S.D., and coefficient of variation (%CV) were calculated. The CT values did not deviate markedly and %CV was < 3.5% suggesting that varying compositions of saliva do not inhibit the ability to detect SARS-CoV-2 precisely (Table 4).

Table 4

Quantifying variability from heterogeneous saliva matrices on the performance of the UoNATS assay; 28 different saliva samples were spiked with reference standard at the LOD of 1 vc µL−1 to measure the effect of uncontrollable inhibitory factors on the accuracy of results.

	1 × LOD (CDC-N2)	1 × LOD (Charité-E)
Number of samples	28	28
Mean CT	34.54	34.89
Std. Deviation	1.189	0.920
Coefficient of variation	3.442%	2.637%

Influence of pipetting volumes in the assay

To assess the effect of potential pipetting inaccuracies (due to sample viscosity) an external positive control was added to negative saliva to a concentration of 2 vc µL−1 (2 × LOD) and added in triplicate as 6, 7, 8, 9 and 10 µL sample volumes to 12 µL of master mix. The effects are summarised in Table 5 and demonstrates a low variability in CT values as a result of pipetting deviations of ± 2 µL from an 8 µL sample volume.

Table 5

Descriptive statistics showing the effect of controlled variation of volumes of saliva in a 20 µl reaction.

	6 μL	7 μL	8 μL	9 μL	10 μL
CDC-N2 (FAM)
Number of replicates	3	3	3	3	3
Mean CT	33.38	33.61	32.92	33.72	34.25
Std. Deviation	0.227	0.612	0.921	1.505	0.302
Coefficient of variation	0.680%	1.821%	2.796%	4.463%	0.882%
Charité-E (HEX)
Number of replicates	3	3	3	3	3
Mean CT	32.25	32.77	32.00	32.40	33.70
Std. Deviation	0.106	0.659	0.609	1.228	0.475
Coefficient of variation	0.329%	2.011%	1.902%	3.791%	1.408%

Problematic samples

Saliva specimens with a highly viscous and gel-like consistency are difficult to pipette accurately. A clinical specimen of this type was analysed to ascertain potential inaccuracies in the results which may be expected with such samples. Four operators tested the same sample in triplicate, Supplementary Table S8; Sample 7 summarises the results for each gene. Although this sample represents one of the worst examples of saliva to pipette, the coefficient of variation for N2 and E were lower than 5.4% when tested with the UoNATS assay.

Discussion

The SARS-CoV-2 pandemic created an unprecedented need for regular viral population-based testing. Of the solutions that have been utilised for this purpose, PCR-based diagnostic tests have constituted the frontline method of choice[6]. Here, we aimed to evaluate the efficacy of direct salivary PCR for the purpose of a routinely used asymptomatic-based testing regime and demonstrate a performance that meets the necessary acceptance standards for UKAS accreditation.

The majority of the currently available techniques, both antigen-based and nucleic acid-based, rely on nasopharyngeal swabs for specimen collection. Accurate diagnostic testing ultimately relies on accurate self-administration of swabs; many people find this unpleasant, and this increases the likelihood of ineffective sampling and potentially false-negative results[7, 8]. Alternative non-invasive methods of sampling will prove essential if frequent mass testing of individuals can be maintained both logistically and financially long-term throughout this pandemic. Using saliva overcomes the unwillingness of donors to administer unpleasant swabs, which may encourage more regular testing, and therefore detect cases which would otherwise be undiagnosed.

Some debate in the literature has arisen as to the sensitivity and accuracy of alternative (non-swab based) sample media. Saliva specimens are considered variable by some; however, others show that both viral stability and assay sensitivity using saliva can meet comparable specifications to that of correctly administered NPS[11]. We sought here to reinforce the idea that saliva can provide a reliable and highly sensitive medium for the detection of SARS-CoV-2 nucleic acid and should be utilised to a greater extent. Furthermore, we observed a surprising longevity of SARS-CoV-2 nucleic acid in saliva as a medium, across a range of storage temperatures, and after multiple freeze-thaws. We have demonstrated that the UoNATS assay is robust against what are inarguably highly heterogenous specimens and the potential inhibitory factors present within, with even the worst example we have encountered, yielding a co-efficient of variation (CV%) of just 5.4%.

Our work adds to the body of evidence which promotes the use of saliva more widely. Studies have shown saliva to offer an equal or greater sensitivity than NPS; Tan et al., find 69% of RT-qPCR based studies using saliva were more sensitive than swabs[25], a finding supported by Beyene[26] and Wyllie[17]. This is an important consideration when using NPS-based NAAT as the gold-standard reference method from which comparisons are made. Meta-analyses by Bastos[27], Czumble[11], and Butler-Laporte[28], all report at least a comparable performance of saliva against the gold standard[29]. It is important to note that although saliva is easy to provide there is variability in terms of volume, consistency, and composition and therefore there is likely some impact on this. However, as there will additionally be variability in self-administered NPS any sampling method has its limitations.

The technical validation of our in-house method was designed to address the required criteria for national UK testing standards, as defined by the guidance provided by UK regulatory and governing bodies; MHRA-TPP, Department for Health and Social Care (DHSC), and PHE. In 2021, the UoNATS service and testing protocol was assessed by UKAS for concordance with the requirements established by the International Standards Organisation for medical testing laboratories (ISO 17,025:2017) and was duly awarded the status of a nationally accredited SARS-CoV-2 testing laboratory.

When reviewing the clinical concordance of the UoNATS assay with that of independent validated test providers, we observed positive and negative results concordance of 99.4% and 99.6%, respectively. We have presented data to show that the assay is both highly repeatable and reproducible. As per the quality standard limits defined by MHRA/PHE guidance[22], the required sensitivity for nationally approved SARS-CoV-2 diagnostic tests in the UK is the ability to detect down to 1 vc µL−1 of specimen, we confidently detected this viral load with 100% occurrence. The dynamic range of the assay was quantified and demonstrates that the assay maintains accuracy and precision from high (CT 15) to low (CT 40) viral loads.

Some saliva-based methods published in the recent literature incorporate the addition of extraneous regents such as proteinase K, Tween-20 or Tris–Borate-EDTA (TBE) to optimise molecular detection[30, 31]. We have found these additives to be unnecessary when used in combination with the methods presented here. We have submitted our detailed methods and protocols in a separate publication[23] for any who wishes to accurately reproduce what we are doing.

Currently, the variants of concern are Omicron (B.1.1.529) and sub lineages, originating in South Africa. These are the dominant strains globally, representing between 84.4 and 99.9% of new sequences submitted in the last 30 days as of 17th May 2022 (EpiCov; GISAID). Despite a prevalent C > T mismatch in the Charité-E primer region, we observe no evidence of detriment to PCR performance as a result of these variants. In the UK, national restrictions such as mandatory face coverings, social distancing, and lockdowns have ended however the prevalence of SARS-CoV-2 has not diminished; it is likely that some sentinel testing will be beneficial for some time still to come, particularly to monitor the emergence of new variants, with the potential to avoid natural and induced immunity. PCR based detection systems should offer the ability to adapt and detect new variants in light of changing viral sequences.

The University of Nottingham’s SARS-CoV-2 testing service was established in August 2020 to offer regular asymptomatic PCR testing to all students and staff, free-of-charge, in a bid to curtail transmission chains. We have since extended this service to the young children of university affiliates. The service enabled the university to maintain a safe environment allowing a return to campus, and the continuation of uninterrupted research and face-to-face teaching. As of May 2022, we have processed in excess of 150,000 samples. Of these, there were 1900 SARS-CoV-2 positive cases in individuals who were classified as asymptomatic by clinical guidance at the time. Since these were not eligible for PCR testing through government and national health channels, this shows direct evidence for decreasing the likelihood of transmission through the community. Of course, such a testing regime incurs a substantial financial burden; however streamlined and cost-effective testing methodology has allowed long-term preservation of the service whilst maintaining highly sensitive and reproducible results.

Conclusion

The validation work we present here aimed to assess the performance of the UoNATS assay for the intended purpose as a SARS-CoV-2 nucleic acid detection test, as well as validating saliva as an appropriate specimen on which to determine infection status and make a COVID-19 diagnoses. The UoNATS assay can perform to the high specifications required for diagnostic SARS-CoV-2 RT-qPCR testing in the general population; meeting the performance criteria stipulated by UK governing bodies. Our assay is accurate and robust, with high sensitivity and specificity. Non-invasive saliva-based diagnostic tests have enormous potential to sustain surveillance of SARS-CoV-2.

Supplementary Information.