Rapid Development and Validation of a Novel Laboratory-Derived Test for the Detection of SARS-CoV-2.

OBJECTIVES: To increase testing capability for SARS-CoV-2 during a rapidly evolving public health emergency, we aimed to deploy a validated laboratory-developed real-time reverse transcription polymerase chain reaction (RT-PCR) diagnostic test for SARS-CoV-2 on an accelerated timeline and using reagent supply chains that were not constrained.
METHODS: A real-time RT-PCR assay that detects the structural envelope (E) gene of SARS-CoV-2 was developed and validated on the Roche cobas 6800 instrument platform with the omni Utility channel reagents, which performs automated nucleic acid extraction and purification, PCR amplification, and detection. In silico analysis was performed for both inclusivity of all SARS-CoV-2 variants and cross reactivity with other pathogenic organisms. Positive control material was used to determine the Limit of Detection (LOD) and patient samples (positive and negative) confirmed by another authorized assay were used for clinical validation. Experiments were carried out at the Christiana Care Health System's Molecular Diagnostics Laboratory (Newark, DE) between April 1 and April 4, 2020.
RESULTS: A real-time RT-PCR assay for SARS-Cov-2 was developed and validated in just two weeks. For all oligonucleotides, 100% homology to the available SARS-CoV-2 sequences was observed. Greater than 80% homology between one or more oligonucleotides was observed for SARS-Cov (Urbani strain) and Influenza A, however risk of cross reactivity was deemed to be low. The limit of detection (LOD) of the assay was 250 copies/mL. The assay identified 100% of positive patient samples (30/30) and 100% of negative patient samples (29/29 patient negatives and 1/1 saline). Up to 92 samples can be run on a single plate and analysis takes approximately 3.5 hours.
CONCLUSIONS: In this work, we demonstrate the development and validation of a single target laboratory-developed test for SARS-CoV-2 in two weeks. Key considerations for complementary supply chains enabled development on an accelerated timeline and an increase in testing capability.

Introduction

The outbreak of a novel coronavirus strain (SARS-CoV-2) first reported in December 2019 has escalated to a worldwide pandemic and rapidly evolving public health emergency. Genetic analysis shows that the SARS-CoV-2 strain belongs to the genus Betacoronavirus, which contains severe acute respiratory syndrome coronavirus (SARS-CoV) and Middle East respiratory syndrome coronavirus (MERS-CoV), both of which have caused severe respiratory disease in humans. Due to a lack of innate immunity to SARS-CoV-2 in humans, rapid diagnosis of SARS-CoV-2 infection is a critical step in developing effective containment protocols to curb the virus’s rapid spread across the globe.

Early diagnostic tests for SARS-CoV-2 developed in Germany, Hong Kong, and the USA have all used a qualitative real-time reverse transcription polymerase chain reaction (RT-PCR) approach to detect the presence of SARS-CoV-2 genomic RNA in patient samples. Each assay uses a different combination of genomic regions for detection (Germany: RdRp and E genes, Hong Kong: ORF1b and N genes, USA: two targets on the N gene) and requires the detection of two distinct regions on the virus genome to confirm infection. The rapid development and deployment of these assays has been key in limiting the outbreak to the extent it has been limited; yet, achieving widespread testing capability for a public health emergency of this size and scale has been hindered by assay throughput and the availability of reagents, supplies, and equipment. These initial assays require combinations of commercially-available reagent kits and specific equipment for testing.

During a public health emergency of this size and severity, establishing widespread and robust testing infrastructure is critical for monitoring community spread. Diagnostic companies have developed and marketed their own proprietary PCR-based assays as kits specifically designed for their respective in vitro diagnostic testing platforms; however, implementation is limited to testing facilities with access to these instruments and the availability of the corresponding kits. As the demand for testing increases, many labs experience supply chain issues and reagent shortages, which limits the scale-up of widespread testing. The supply of reagents required by these PCR-based protocols cannot keep up with rapidly increasing demand, ultimately impacting the ability to effectively diagnose and contain SARS-CoV-2. On March 31, 2020, due to the rapidly evolving pandemic and shortages of commercial in vitro diagnostic tests, the US Food and Drug Administration (FDA) determined that molecular-based laboratory developed tests are justified to protect the public health and issued guidelines for developing and validating these assays under Emergency Use Authorization (EUA). Under such guidance, assays should at least establish a limit of detection, include inclusivity and cross-reactivity analyses, as well as perform clinical assessment with a number of known patient-positive as well as non-reactive samples.

Laboratory-developed tests play an important role in helping healthcare systems care for their patients when commercial tests are difficult to obtain however developing such a test has historically required timelines on the order of months during previous viral outbreaks. In response to the first SARS outbreak in early 2003, the World Health Organization (WHO) international working group identified the novel coronavirus strain in April 2003, but early diagnostic tests developed were either unreliable or did not detect the virus until late in the illness’s course., It took nearly six months before a reliable assay could be established. During the Ebola outbreak of 2014-2015, the US Department of Defense obtained an EUA for their RT-PCR diagnostic in early August 2014, however, this was the only test to gain FDA authorization until BioFire Defense gained approval nearly three months later in late October 2014. Similarly, after the WHO declared Zika a public health emergency of international concern in February 2016, testing in the US was initially limited to the Centers for Disease Control and Prevention (CDC). Distribution of tests to authorized laboratories was controlled and managed by the CDC and the limitation in distribution of early CDC assays resulted in a backlog of nearly 1,000 samples in the Miami area at one point during the outbreak. The first commercial laboratory was issued an EUA in late April 2016 and to date, the only laboratory-developed test EUA was issued to Columbia University’s Center for Infection and Immunity in August 2017, a year and a half after the CDC test was authorized.

During the early phases of a viral outbreak, the ability to rapidly determine which patients are positive or negative for the virus is critical. It is important for treatment of the patient as well as for developing an understanding of the spread of the disease in the community. An inability to rapidly diagnose patients can also strain healthcare systems. For example, patients that present with symptoms, but for which a diagnosis must wait several days, are treated as if they may be positive, requiring the use of a significant amount of personal protective equipment by healthcare providers and requiring special isolation. If a patient tests negative, a significant amount of equipment and resources can be allocated to those in need.

The current SARS-CoV-2 situation also opens the opportunity to circumvent, or at least alleviate, certain bottlenecks in testing supplies with in-house laboratory-developed assays. Laboratories may need to strategize their choice of instruments, reagents, supplies, protocols, and sources for each, to quickly develop tests that are rigorous enough to gain regulatory acceptance, and that can be adopted for sustained use during an outbreak of unknown duration. Some issues to consider are speed to regulatory acceptance and risk of non-acceptance, the benefits and drawbacks of using less common targets and/or reagents, compatibility with commercial platforms, and the availability of custom and commercially available reagents. One possible approach is to apply laboratory-developed tests customized for SARS-CoV-2 to commercially available automated platforms that are already designed for in vitro diagnostics. Such an approach can increase throughput by relying on an automated platform that would require less staff resources and can test numerous samples at once, as well as by using reagents and supplies that are less limited by supply chain issues. This approach can also supplement the use of proprietary diagnostic tests to maximize instrument use time and decrease turnaround time. It also can help provide a rapid clinical diagnosis to patients.

Here, we report on the rapid development of a clinically validated laboratory-developed test for SARS-CoV-2 detection in just two weeks. The method relies on a commercially available platform for automated extraction, amplification, and detection, and has the benefit of being on a platform for which reagents were not limited by supply chain constraints. In the interest of accelerated development and the use of extraction and amplification kits that are not limiting during the outbreak, the assay relies on a single target in the E gene previously reported in the literature,, and other publicly available information. We present observations with this clinically validated assay, which was established in two weeks, with the goal of enabling other laboratories with similar instrument platforms to rapidly develop and validate an assay for SARS-CoV-2 detection. This method is described in a manner intended to facilitate broad adoption and the ability to secure an authorization to use this assay from the appropriate health authority.

Materials and Methods

Materials

The SARS-CoV-2 E-gene Forward Primer (5’-ACAGGTACGTTAATAGTTAATAGCmGT-3’) and the SARS-CoV-2 E-gene Reverse Primer (5’-ATATTGCAGCAGTACGCACAmCA-3’) were obtained from Integrated DNA Technologies (IDT; Coralville, Iowa; m = 2’-O-methyl base in penultimate to prevent primer dimer). The SARS-CoV-2 E-gene Probe (5´-Fam-ACACTAGCC/ZEN/ATCCTTACTGCGCTTCG-Iowa Black FQ-3’) was also purchased from IDT. The positive control was purchased from SeraCare (#0505-0126, AccuPlexTM SARS-CoV-2 Reference Material Kit). Other reagents for use on the Roche cobas 6800 instrument platform included: omni Utility Channel Reagent Kit (#06645348190), buffer negative control (#07002238190), MGP reagent (#06997546190), specimen diluent (#06997511190), lysis reagent (#06997538190), wash reagent (#06997503190), processing plate (#05534917001), amplification plate (#05534941001), pipette tips (#05534925001), liquid waste container (#07094388001), solid waste bag and containers (#07435967001, #07094361001, #08030073001 and #08387281001), and secondary tubes (#06438776001).

Clinical Samples

For clinical validation, 13 positive and 18 negative COVID-19 patient samples were donated by the Delaware Public Health Laboratory. Patient COVID-19 status was determined using the CDC assay. An additional 11 positive and 11 negative patient samples were donated to this effort by Christiana Care Health System. Patient COVID-19 status was determined using the Cepheid Xpert Xpress SARS-CoV-2 test (EUA granted March 20, 2020). All patient samples were deidentified prior to use and blinded to the laboratory technician carrying out the assay. Use of deidentified leftover patient samples was consistent with FDA Guidance.

Oligo, Controls, and Reagent Preparation

Oligo sequences for detection of the SARS-CoV-2 envelope (E) gene were obtained from published literature., Sequences were verified against the available RefSeq genome (NCBI Accession Number: NC_045512, accessed March 27, 2020).

Oligos were resuspended in molecular biology grade water to 1000µM and then further diluted and aliquoted into 100µM working stocks. Probes were diluted directly to 100µM.

Positive control standard (SeraCare) containing non-replicating SARS-CoV-2 RNA sequence segments encapsulated in a viral protein coat was used. Roche Master Mix 2 (MMx-R2) was prepared as previously described. Briefly, 6mL of MMx-R2 was transferred into a light-protected polypropylene tube. 84 µL each of the forward and reverse primers (100 µM working stock), 10.5 µL of the probe (100 µM working stock), plus 303 µL of molecular biology grade water were added to MMx-R2 to reach a final volume of 6.48 mL. Six mL of the prepared MMx-R2 was loaded into the reagent cassette according to the manufacturer’s protocol.

Sample Preparation

Positive control was diluted 1:20 in appropriate media (E-swab, Viral Transfer Media (VTM), or saline) to a final concentration of 250 genome copies / mL (cp/mL) and transferred to a secondary tube. Samples for limit of detection (LOD) determination were created by diluting positive control in appropriate media per the experimental design.

To run the samples on the instrument, 0.6 mL of sample material was transferred to a barcoded secondary tube with instrument parameters as previously described. An RNA Internal Control (RNA IC) was included and serves as both an extraction and amplification control. Nucleic acid from patient samples and an RNA IC is simultaneously extracted. Selective amplification of the RNA IC is achieved by the use of non-competitive sequence specific forward and reverse primers, which have no homology with the SARS-CoV-2 genome.

In silico Inclusivity and Cross-Reactivity Analysis

In silico inclusivity and cross-reactivity analyses were performed using NCBI BLAST (accessed March 31, 2020) to align the selected primer and probe sequences to sequences of interest. Default parameters were used for the alignments and scoring. To assess inclusivity of the assay, alignment was done against all published SARS-CoV-2 sequences available in the NCBI database (Tax ID 2697049). Potential cross-reactivity of the primer and probe sequences with the genomes of other respiratory flora and viral pathogens was assessed by aligning oligonucleotide sequences against 28 microorganisms and determining if significant primer homology existed (defined as greater than 80% homology to published sequence).

Results

Assay Description

The developed assay is a real-time RT-PCR test that uses a single primer and probe set to detect the structural envelope (E) gene of the SARS-CoV-2 virus in a clinical sample. The assay was developed and performed on the Roche cobas 6800 instrument platform with the omni Utility Channel that allows one to create and run custom laboratory-developed tests. After sample and control preparation, the instrument performs fully automated nucleic acid extraction and purification, PCR amplification, and detection, in approximately 3.5 hours. Up to 92 samples can be run on a single plate.

The software automatically calls whether a batch is valid or invalid and accordingly assigns a result to each sample. A batch is valid if no flags appear for the negative control or the positive control. Invalidation of results is performed by the software based on negative control failures, while invalidation of results due to positive control failures must be determined by the user. A valid batch may include both valid and invalid sample results. A positive result for the SARS-CoV-2 E-gene indicates that the target-specific nucleic acid has been detected while a negative result for the SARS-CoV-2 E-gene indicates that the target-specific nucleic acid has not been detected. An invalid result for a particular sample indicates that testing for that sample is invalid and should be repeated for that sample only.

In silico Inclusivity and Cross-Reactivity

Inclusivity analysis was performed by aligning the forward primer, reverse primer, and probe to the 401 published SARS-CoV-2 sequences deposited in the NCBI database as of March 31, 2020. For all oligonucleotides, 100% homology to the available SARS-CoV-2 sequences was observed (Table 1). In silico cross-reactivity analysis only showed significant primer homology (homology exceeding 80%) to SARS-coronavirus (Urbani strain), SARS-CoV-2, and Influenza A. Significant homology (100%) to the SARS-CoV genome was observed for the forward primer, reverse primer, and probe. Based on sequence analysis alone, cross-reactivity of this assay for SARS-CoV-2 with SARS-coronavirus (Urbani) cannot be ruled out. However, SARS-coronavirus (Urbani) has not circulated broadly in the human population since 2004 sand therefore cross-reactivity during the COVID-19 outbreak is statistically unlikely and not clinically-relevant at this time (the early phases of the SARS-CoV-2 outbreak). An alignment with 82% homology was found between the E-gene reverse primer and Influenza A, with one mismatch (18 of 19 base pairs). No significant alignment was found between Influenza A and the forward primer or probe. Despite the in silico prediction of potential cross-reactivity, experimental observations with seven Influenza A-positive patient samples and six Influenza A (H1N1)-positive patient samples revealed no cross-reactivity. In addition, the lack of homology between the forward primer and probe suggests that no amplification will occur even if the reverse primer were to bind to a portion of the Influenza A genome. Therefore, the risk that this assay detects Influenza A was deemed to be low. Experimental observations with other respiratory pathogens from clinical samples were consistent with these in silico observations.

Table 1

Results of in silico inclusivity and cross-reactivity analysis. Assay oligonucleotides were aligned to the listed microorganisms and any significant alignment (greater than 80% homology) was noted.

Organism	Taxonomy ID	Homology
SARS-CoV-2	2697049	E-gene Forward Primer – 100%E-gene Reverse Primer – 100%E-gene Detection Probe – 100%
Human coronavirus 229E	11137	No significant alignment was found
Human coronavirus OC43	31631	No significant alignment was found
Human coronavirus HKU1	290028	No significant alignment was found
Human coronavirus NL63	277944	No significant alignment was found
SARS-coronavirus (Urbani strain)	228330	E-gene Forward Primer – 100%E-gene Reverse Primer – 100%E-gene Detection Probe – 100%
MERS-coronavirus	1335626	No significant alignment was found
Adenovirus (C1 Ad. 71)	129875108098129951130310130308	No significant alignment was found
Human Metapneumovirus (hMPV)	162145	No significant alignment was found
Parainfluenza virus 1-4	127301979160112161979161	No significant alignment was found
Influenza A	11320	E-gene Reverse Primer – 82%
Influenza B	11520	No significant alignment was found
Enterovirus (EV68)	12059	No significant alignment was found
Respiratory syncytial virus	11250	No significant alignment was found
Rhinovirus	169066	No significant alignment was found
Chlamydia pneumoniae	83558	No significant alignment was found
Haemophilus influenzae	727	No significant alignment was found
Legionella pneumophila	446	No significant alignment was found
Mycobacterium tuberculosis	1773	No significant alignment was found
Streptococcus pneumoniae	1313	No significant alignment was found
Streptococcus pyogenes	1314	No significant alignment was found
Bordetella pertussis	520	No significant alignment was found
Mycoplasma pneumoniae	2104	No significant alignment was found
Pneumocystis jirovecii (PJP)	42068	No significant alignment was found
Candida albicans	5476	No significant alignment was found
Pseudomonas aeruginosa	136841	No significant alignment was found
Staphylococcus epidermis	1282	No significant alignment was found
Staphylococcus salivarius	1304	No significant alignment was found

Limit of Detection (LOD) Study

The intent of this clinical LOD study was to establish the LOD as quickly as possible as part of clinical assay validation (as would be required to use this test as part of a clinical diagnosis) and not to perform a detailed series of studies to quantitatively establish the true analytical limit. We selected a target of 250 cp/mL to test and designed the LOD study to document that the assay could detect this level. The experiments were performed to simultaneously determine the feasibility of the proposed assay and determine this LOD by spiking positive control material into three different sample matrices at levels spanning two orders of magnitude (5,000 cp/mL – 250 cp/mL). FDA guidance specifies that the LOD of the assay is the lowest detectable concentration of SARS-CoV-2 (cp/mL) at which greater than or equal to 95% of all replicates test positive, and this limit must be verified with at least 20 replicates at the claimed concentration. To fulfill this requirement, three concentrations of SARS-CoV-2 standard were evaluated across three sample matrices, including 20 replicates in VTM. All samples in the LOD experiment tested positive (Ct < 40 cycles) regardless of concentration or clinical matrix (Table 2). Based on this data, the assay’s LOD of 250 cp/mL was established.

Table 2

Results of LOD testing. All samples containing positive control material (SeraCare) tested positive for SARS-CoV-2 (Ct < 40 cycles). Four blank saline samples were run in parallel as non-reactive controls and 100% of those samples tested negative.

Matrix	RNA Concentration (cp/mL)	Number of Samples	Percent Positive Samples
Undiluted Control	5,000	3	100%
VTM + Control	1,000	3	100%
	250	20	100%
E-Swab + Control	1,000	3	100%
	250	7	100%
Saline + Control	1,000	3	100%
	250	7	100%
	0	4	0%

Clinical Evaluation

Clinical evaluation of the assay was performed with 30 patient-derived, positive samples and 30 non-reactive samples, as required by the FDA. Twenty-four confirmed-positive patient swabs were used to generate the 30 positive samples. Thirteen of these 24 samples were previously confirmed positive using the CDC’s assay and 11 of the 24 samples were confirmed positive using an alternate technology approved under an FDA EUA (Cepheid Xpert Xpress SARS-CoV-2 test, EUA on March 20, 2020). The remaining six samples required for clinical evaluation were created by diluting samples as show in Table 3. We also tested 29 previously confirmed non-reactive patient samples (18 samples tested by CDC assay and 11 by Cepheid assay) as well as one blank media sample. All confirmed-positive samples tested positive (Ct < 40) using the proposed assay, and all non-reactive samples were confirmed negative (Ct > 40) (Table 3).

Table 3

Results of the clinical evaluation. All samples previously confirmed to contain SARS-CoV-2 RNA tested positive using the proposed assay, and all negative samples tested negative using the proposed assay. All internal and negative controls were valid.

Test Status	Assay	Dilution	Number of Samples	Results
Positive	CDC	n/a	13	13/13 Positive (100%)
	Cepheid	n/a	11	11/11 Positive (100%)
	CDC	1:5	5	5/5 Positive (100%)
	CDC	1:1.375	1	1/1 Positive (100%)

Discussion

The experimental development and clinical validation of a single-target molecular diagnostic assay to detect the E-gene of SARS-CoV-2 was completed in two weeks including the validation with confirmed patient-positive SARS-CoV-2 samples. This two week window included a six calendar day delay waiting for custom primers and probes as well as an additional two day delay to troubleshoot a reagent that was not performing as expected. We believe that if supply chains had not been limiting and if reagents performed as expected, this assay could have been developed and validated in less than one week.

During the early phases of an outbreak, there is a significant demand for key reagents and supplies as testing capability expands globally, and these supply chain limitations can impact timelines. To circumvent limited availability of PCR extraction kits, this assay takes advantage of cobas reagents that are not specific to SARS-CoV-2 testing and which are not in limiting supply. Our experience was that to rapidly advance assay development, it was helpful to find multiple vendors and product listings to source critical reagents. While manufacturers of custom oligonucleotides, for example, can quickly ship primers and probes used by existing PCR tests established by the CDC, the production of alternate custom reagents was subject to some manufacturing delays. Our assay development effort benefited from sourcing primers, probes, controls, and other reagents from multiple vendors, as well as acquisition of different types of reagents from a given vendor (for example, lyophilized primers as well as ready to use primers). This approach saved time during method troubleshooting; however, it did increase the financial investment needed to launch the effort. However, during the early phases of this outbreak when healthcare systems are unable to keep up with demands, each day matters.

The usefulness of alternative sourcing strategies was observed when early test runs, using certain reagent lots from a vendor, were invalidated by low level amplification signals observed in negative control materials (39

It is important to consider failure modes early and prepare for rapid troubleshooting. While it is admittedly difficult, if not impossible, to foresee all possible issues, having a team consider failure modes early in the process helped prepare and develop mitigations. Even the smallest issues can impact the ability to validate an assay. For example, the reagent cassette used in this study must be vented with a pipette tip during loading of the prepared MMx-R2; however, the only tips available in the laboratory during early experiments were barrier tips which prevented air flow. The barrier tips made it difficult to pipette precise volumes into the reagent cassette during practice runs. This unforeseen issue was initially overcome by using a needle to break the barrier seal on the tip for practice runs by allowing venting of the cartridge and precise pipetting. Ultimately, the issue was addressed by acquiring non-barriered tips for this purpose.

Here, we presented the rapid development and clinical validation of a PCR-based assay for the detection of SARS-CoV-2 during the COVID-19 outbreak. We developed a laboratory-developed test using a widely available commercial instrument platform on an accelerated timeline. The assay was validated within two weeks of the launch of this study, with at least one week of that time dedicated to waiting for custom reagents and troubleshooting. This work suggests the possibility of rapidly validating a laboratory-developed test to support clinical diagnosis when used under appropriate authorization from a relevant health authority.

Public Health Implications

The outbreak of a novel coronavirus strain (SARS-CoV-2) first reported in December 2019 has escalated to a worldwide pandemic and rapidly evolving public health emergency. The ability to quickly develop testing capability is critical to establishing an understanding and response to such a threat. Public health laboratories have moved to rapidly develop polymerase chain reaction (PCR)-based tests and commercial diagnostics companies in the in vitro diagnostics business also develop proprietary kits. However, the demand for testing has far exceeded the supply of such tests (reagents and kits). Here we share an approach to rapidly validating a laboratory-developed test for SARS-CoV-2 on a widely available commercial instrument to increase local testing capability.