Single vs replicate Real-Time PCR SARS-CoV-2 testing: Lessons learned for effective pandemic management.

Coronavirus Disease 19 (COVID-19) caused by the SARS-CoV-2 virus remains a global pandemic having a serious impact on national economies and healthcare infrastructure. Accurate infection detection protocols are key to policy guidance and decision making. In this pilot study, we compared single versus replicate PCR testing for effective and accurate SARS-CoV-2 infection detection. One-Step Real-Time RT-PCR was employed for the detection of SARS-CoV-2 RNA isolated from individual nasopharyngeal swabs. A total of 10,014 swabs, sampled from the general public (hospital admissions, A&E, elective surgeries, cancer patients, care home residents and healthcare staff), were tested using standard replicate testing. Our analysis demonstrates that approximately 19% of SARS-CoV-2 infected individuals would have been reported as false negative if single sample Real-Time PCR testing was used. Therefore, two replicate tests can substantially decrease the risk of false negative reporting and reduce hospital and community infection rates. As the number of variants of concern increases, we believe that replicate testing is an essential consideration for effective SARS-CoV-2 infection detection and prevention of further outbreaks. A strategic approach limiting the number of missed infections is crucial in controlling the rise of new SARS-CoV-2 variants as well as the management of future pandemics.

Introduction

COVID-19 caused by the Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2) remains a global pandemic with 174 million infected and 5,268,849 deaths (09/12/2021, source: WHO COVID-19 dashboard). SARS-CoV-2 was identified as the causal agent of unusual pneumonia cases in the city of Wuhan, P.R. China, in December 2019—January 2020 [1]. The genetic sequence of SARS-CoV-2 was determined by Wu et al. on January 17th 2020 (NC_045512.2) [2]. Since then, at least 5,775 sequence variations have been reported and continue to rise [3]. Numerous potential coding targets for diagnostic use have been identified, including Open Reading Frame 1 ab (ORF1ab), S gene, ORF3a gene, E gene, M gene, ORF6a gene, ORF7a gene, ORF7b, ORF8 gene, N gene and ORF10 gene [4].

Currently, a range of genetic targets for diagnostic purposes have been utilised including the ORF1ab, N gene, S gene and E gene. The N gene sequence is conserved with other members of the corona virus family, therefore amplification of this region is utilised mainly as a guidance for re-testing with SARS virus sequence similarity of 90.25% (NC_004718.3) [5].

The specificity and accuracy of SARS-CoV-2 PCR diagnostic testing has been extensively considered in the last 19 months, as numerous perspectives in regard to the reporting of false negatives have been published [6-8]. Evidence of false negative reporting include cases of patients with COVID-19 specific symptoms and chest X-rays, a history of SARS-CoV-2 positive contacts but a negative nasopharyngeal swab PCR test result [9-12].

False negative reporting can significantly impact disease spread to close contacts and the community, especially for asymptomatic individuals [6]. Arevalo-Rodriguez et al. (2020) estimated false negative reporting between 2–29% [7]. This wide estimate is the product of a systematic review, which did not consider the sensitivity of PCR based SARS-CoV-2 detection protocols such as the number of replicates that should be carried out for nasopharyngeal swabs under investigation. In this pilot study, we have compared single versus replicate Real-Time PCR testing on a cohort of 10,014 nasopharyngeal swab samples.

Materials and methods

Nasopharyngeal swab collection

Nasopharyngeal swabs were tested at the North Kent Pathology Service (NKPS), Dartford and Gravesham NHS Trust. These were received form the Dartford and Gravesham NHS Trust, the Medway NHS Foundation Trust and the Oxleas NHS Foundation Trust (hospital admissions, A&E, elective surgeries, cancer patients, staff), as well as Dartford and Medway care home residents. Upon sampling each nasopharyngeal swab was placed in a sterile transport media tube and was kept chilled during transport and prior to viral RNA extraction. The nasopharyngeal swabs were processed within 24 hours from collection.

SARS-CoV-2 RNA extraction

Total RNA was extracted using HigherPurity™ Viral RNA extraction kit (Cat. No: AN0805-XL) from Canvax Biotech in accordance with the manufacturers protocol. Total RNA was eluted into 40μl of RNAse free water. Total RNA (duplicate testing) was tested immediately after extraction.

One-Step Real-Time RT-PCR SARS-CoV-2 detection

Two commercial quantitative One-Step Real-Time RT-PCR SARS-CoV-2 detection kits were used: (1) Vitassay SARS-CoV-2, Real-time PCR detection kit, Vitassay Healthcare, S.L.U. Huesca, Spain, (2) Primerdesign™ LTD, Coronavirus (COVID-19) Genesig® Real-Time PCR assay, Primerdesign Ltd, Chandlers Ford, Southampton, U.K. Both kits detect SARS-CoV-2 based on the SARS-CoV-2 Wuhan genome (NCBI Taxonomy ID:269049, REFSeq:NC_045512.2). One-Step Real-Time RT-PCR reactions were prepared in accordance with the manufacturers’ protocols. Internal controls were included in all RT-PCR reactions, in accordance with the manufacturers’ protocols in order to confirm the accuracy of each RT-PCR reaction. All quantitative PCR reactions (single and duplicate testing) were carried out on the Biorad CFX96 Real-Time PCR detection system and for both assay kits the total number of cycles was 45 (according to the manufacturers’ protocols).

The RT-PCR data is available as a supplementary document.

Ethical approval

Ethical approval is not applicable for this study. Patients were not specifically recruited for this study. This is a study of testing methodology only and the outcome of SARS-CoV-2 RT-PCR testing was analysed with no link to patient clinical information.

Results

In order to evaluate the effectiveness of a SARS-CoV-2 testing protocol that minimizes the frequency of false negative test results we employed a duplicate PCR reaction approach per tested nasopharyngeal swab. All RT-PCR reactions that resulted in a Ct value lower than 40 cycles and a clear amplification curve, were recorded as positive.

A total of 10,014 nasopharyngeal swabs were tested using 20,028 duplicate Real-Time PCR reactions. The tested nasopharyngeal swabs were collected between the months of May and July 2020 and the collection approach was not restricted to COVID-19 symptomatic or SARS-CoV-2 infected asymptomatic individuals. Therefore, our nasopharyngeal swab collection method accurately represents the public infection rates at the time of sampling. Primerdesign One-Step Real-Time RT-PCR assay was used for 18,672 tests and the Vitassay One-Step Real-Time RT-PCR assay was used for 1,356 tests. Out of 10,014 patients tested, 491 (2.45%) were reported positive with both assay kits (Fig 1A). Further analysis of the positive results showed that 302 or 61.5% of these were reported as duplicate positives (positive SARS-CoV-2 RNA detection on both replicate 1 and 2), 85 or 17.3% of the positive results were only detected on replicate 1 and 104 or 21.2% of the positive reported samples were only detected on replicate 2 (Fig 1A).

Fig 1

Graphic representation of the SARS-CoV-2 testing outcome of 10,014 nasopharyngeal swabs.

(A) Graphic representation of SARS-CoV-2 positive sample reporting relative to Real-Time PCR reaction outcome. 491 patients or 2.45% tested positive for the SARS-CoV-2 virus. While 61.5% of SARS-CoV-2 infections were detected in both Real-Time PCR repeat testing reactions almost 40% of SARS-CoV-2 infections were detected in only one of the two repeat Real-Time PCR testing reactions. (B) Total number of patient nasopharyngeal swab tests detected positive for SARS-CoV-2 on duplicate, replicate 1 or replicate 2 Real-Time PCR. (C) Graphic representation of the percentage of duplicate, replicate 1 only and replicate 2 only Real-Time PCR confirmed SARS-CoV-2 positive samples over all positive detected samples. (D) Graphic representation of the percentage of duplicate, replicate 1 only and replicate 2 only Real-Time PCR confirmed SARS-CoV-2 positive samples over all tested samples.

Out of the 10,014 patients, 678 (corresponding to 1,356 duplicate Real-Time RT-PCR reactions) were tested with the Vitassay One-Step Real-Time RT-PCR kit. Of these, 27 patients (3.98%) were reported as positive, with 23 (85% of all positive patients) detected as duplicate positives (detection was observed in both replicates). In addition, 1 patient was detected as positive on replicate 1 only and 3 patients were detected as positive on replicate 2 only. Therefore, 4 patients were detected positive on one replicate (15% of all positive patients).

The Primerdesign One-Step Real-Time RT-PCR assay was used to test 9,336 patients (corresponding to 18,672 duplicate Real-Time RT-PCR reactions). Of these, a total of 464 patients (4.97%) were detected as SARS-CoV-2 positive, with 279 patients (60% of all positive patients) detected as duplicate positive (detection on replicate 1 and replicate 2), 84 patients detected positive on replicate 1 only and 101 patients detected as positive on replicate 2 only. Overall, 185 patients were detected positive on one replicate (40% of all positive patients).

Fig 1B–1D summarizes the reporting profile for Real-Time PCR SARS-CoV-2 testing. Testing of 10,014 nasopharyngeal patient swabs has shown that 61.5% of SARS-CoV-2 positive patients were detected on double replicate positive PCR reactions, while 17.3% were detected positive on replicate 1 and 21.2% were detected positive on replicate 2. Overall, we can conclude that approximately 19% of patients were detected as SARS-CoV-2 positive by only one of the two PCR reaction replicates and these would be missed if single reaction testing was employed.

Discussion

In this study we examined whether replicate RT-PCR testing of SARS-CoV-2 RNA, isolated from nasopharyngeal swabs, increases the reliability of testing results’ reporting. We tested 10,014 nasopharyngeal swabs, which equates to 20,028 duplicate Real-Time PCR reactions and identified 491 SARS-CoV-2 positive patients. As mentioned above, RT-PCR reactions were recorded as positive when a Ct value lower than 40 cycles was detected and a clear amplification curve was confirmed. This applies to single and duplicate RT-PCR reactions.

Our findings demonstrate that a singlet testing policy would have failed to detect approximately 19% of SARS-CoV-2 positive patients, approximately 94 in 10,000 patients and a predictive ratio of 940/100,000 individuals. This level of false negatives, in correlation with an R0 number >1, can contribute to sustained and increasing SARS-CoV-2 infection rates within hospitals and the wider community.

False negative PCR outcomes can be influenced by substandard sampling and storage leading to inefficient viral RNA isolation and detection. In addition, a false negative PCR can be due to low viral copy number which can either reflect the end of the infection or, more worryingly, the early infection stage. We did not investigate these factors as they fell outside the scope of the present study, which instead focused on single vs duplicate testing and the rate of false negative outcomes. Our conclusion is in agreement with a systematic review conducted by Arevalo-Rodriguez et al. (2020) estimating a false negative reporting ranging between 2–29% [7] and shows that regardless of the cause for a false negative PCR, duplicate testing increases the accuracy of SARS-CoV-2 infection reporting and can minimise the chance of infection outbreaks. Moreover, our work complements the analysis carried out by Arevalo-Rodriguez et al. as to our knowledge, this is the first study that determines the proportion of PCR tested nasopharyngeal swabs that show repeatable results in replicate testing and considers whether duplicate positive or individual replicate positive testing can reliably detect the presence of SARS-CoV-2 viral RNA and consequently infection.

We acknowledge that duplicate testing may be prohibited due to its cost and practical considerations especially when the daily infection rate is high, but with several variants of concern arising, it is imperative to have a strategic methodology that limits the number of missed infections. Therefore, a duplicate testing protocol should be employed for accurate pandemic modelling management and prevention of infection outbrakes. This is crucial not only for this pandemic, which continues to change and develop but future pandemics as well.

One-Step Real-Time RT-PCR SARS-CoV-2 detection raw data.

The table summarizes the outcome of each of the 20,028 One-Step Real-Time RT-PCR reaction tests for the detection of SARS-CoV-2, carried out between May and July 2020.

(XLSX)

Click here for additional data file.

16 Mar 2022

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Ruslan Kalendar

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf  and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Reviewer #1:

In this report, the authors compare the accuracy estimates of two SARS-CoV-2 testing protocols, Vitassay One-Step Real-Time RT-PCR and Primerdesign One-Step Real-Time RT-PCR assay, specifically showing the need for replicate testing considering their accuracies. While the paper presents results that are scientifically sound and reiterate previously published accuracy estimates of SARS-CoV-2 tests, it does not address a sufficiently original research question to merit publication in PLoS One. Most diagnostic tests have non-zero error rates which means accuracy will be better with replicate testing. The paper doesn’t add novel value as to situations in the ongoing COVID-19 pandemic where replicate testing will be specifically necessary, eg. certain variants, populations, or viral loads.

Reviewer #2:

Interesting confirmatory paper in real field on the high risk of false negative results for SARS-CoV-2 RNA detection in clinical samples. Two different PCR assays were used on high number of clinical samples. The paper is short and well written.

Minor points

P5. Give the periods of inclusion.

P5. Please indicate the percentages in paragraphs 3 and 4.

P8. Update the reference 7:

Arevalo-Rodriguez I, Buitrago-Garcia D, Simancas-Racines D, Zambrano-Achig P, Del Campo R, Ciapponi A, Sued O, Martinez-García L, Rutjes AW, Low N, Bossuyt PM, Perez-Molina JA, Zamora J. False-negative results of initial RT-PCR assays for COVID-19: A systematic review. PLoS One. 2020 Dec 10;15(12):e0242958.

Reviewer #3:

Summary:

The authors us data from real-time PCR testing of 10,014 nasopharangeal swab specimens to compare the rate of detection of SARS-CoV-2 in single vs replicate testing scenarios. The authors make the argument that replicate testing provides a superior detection mechanism for ensuring accurate test results for pandemic control. Of the original 10,014 swabs, 9336 were tested twice using the Vitassay One-Step Real-Time RT-PCR assay, which uses 2 SARS-CoV-2 specific primer/probe set; and 678 swabs were tested using the Primerdesign LTD, Coronavirus (COVID-19) Genesig Real-Time PCR assay, which uses 1 SARS-CoV-2 specific primer/probe set. The authors report that, in aggregate, 2.45% of samples tested positive on either the first, second, or both replicates. Of these, 17.3% tested positive on only the second replicate, indicating that a non-replicate testing method would have missed a substantial fraction of positive cases.

Major concerns:

• More information is needed on sample selection within the Materials and Methods. Were these swabs collected from symptomatic, suspected positive cases, routine screening of individuals seeking medical care for all maladies, asymptomatic surveillance, or some other reason? This is important for understanding the relevance of the work to pandemic management.

• It is not clear from the methods section and the Ethics Statement that this work has undergone appropriate institutional ethics review. The manner in which patient samples were obtained, the sample identities anonymised, and sample identities tracked and recorded through the replicate testing process was not indicated. If this is a chart review, this should be stated and the appropriate research ethics approvals or exemptions cited. The authors should provide confirmation of ethical review or exemption in accordance with journal and institutional standards: "Any research involving personal information, whether identifiable or anonymised, must be approved via full or proportionate review by an appropriate Research Ethics Committee." Section 7.6.1: https://www.canterbury.ac.uk/asset-library/research/Governance-and-Ethics/code-of-conduct.pdf

• Two different testing platforms were used, with different rates of positivity and criteria for assigning a sample as ‘positive,’ yet the data were aggregated. Authors should explain why data aggregation was appropriate in this case, or they should separate the data according to testing platform.

• A major weakness in the interpretation of the results stems from the binary nature of the report of ‘positive’ or ‘negative.’ Depending on how a Cq value cutoff was applied, replicate tests falling near the Cq threshold may be separately categorized as positive or negative, despite experimental results that are almost identical. The authors should examine and/or discuss the degree to which replicate agreement is affected by the application of a Cq cutoff.

• Authors have indicated that data are freely available but have not indicated the location or means of obtaining the data.

Minor concerns:

• The duration and method of RNA storage between swab acquisition and testing, and between testing replicates should be described.

• It appears from the methods that replicate tests were performed on the same platform as the initial test, but this should be explicitly stated.

• In the end of the results section, the authors write, “Testing of 10,014 nasopharyngeal patient swabs has shown that 61.5% of SARS-CoV-2 positive patients were detected on double replicate positive PCR reactions, while approximately 19% of patients were detected as SARS-CoV-2 positive by only one out of the two PCR reaction replicates.” This statement appears to be made in error, since the two values do not add up to 100%. Authors should revisit the intent of this sentence and edit accordingly.

• The word ‘outbreaks’ is misspelled in the abstract.

• “corona virus” should be a single word in the Introduction.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Laurent Belec

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

2 May 2022

Dear Editor,

we thank you for the review process and the reviewers' comments.

A detailed "Response to reviewers" letter has been prepared and submitted for revision together with the revised manuscript with and without track changes.

We will be looking forward to your response.

Kind regards

Athina Mylona

Submitted filename: Response to reviewers.docx

Click here for additional data file.

30 May 2022

Single vs replicate Real-Time PCR SARS-CoV-2 testing: lessons learned for effective pandemic management.

PONE-D-21-40437R1

Dear Dr. Mylona,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Ruslan Kalendar

Academic Editor

PLOS ONE

6 Jul 2022

PONE-D-21-40437R1

Single vs replicate Real-Time PCR SARS-CoV-2 testing: lessons learned for effective pandemic management.

Dear Dr. Mylona:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Professor Ruslan Kalendar

Academic Editor

PLOS ONE