Understanding false positives and the detection of SARS-CoV-2 using the Cepheid Xpert Xpress SARS-CoV-2 and BD MAX SARS-CoV-2 assays.

Several real-time RT-PCR assays have received Emergency Use Authorization from the United States Food and Drug Administration. The BD MAX™ SARS-CoV-2 assay, run by the BD MAX™ system, is a qualitative test that detects the SARS-CoV-2 specific nucleocapsid phosphoprotein gene regions, N1 and N2. The human RNase P gene is used as the endogenous nucleic acid extraction control. The Cepheid Xpert® Xpress SARS-CoV-2 assay, run by the GeneXpert system, detects the pan-sarbecovirus E gene and the N2 region of the N gene. We evaluated the performance characteristics of the BD and Cepheid assays using matched patient samples. We also analyzed comparative Ct values for both assays using 183 positive samples tested at this facility. In addition, we mitigated reporting false positive results without relying on interpretive software. We found that both systems showed comparable sensitivity. We found an approximately 3.5% false positive rate from the BD MAX™ system results. Published by Elsevier Inc.

Introduction

The ongoing Coronavirus disease 2019 (COVID-19) pandemic caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) started in Wuhan City, Hubei Province, China and has been a novel experience for the entire world (CDC, 2020b). The Influenza pandemic of 1918 is the most recent comparable experience the world has endured. The unexpected nature of the current pandemic and the lack of experience in handling such a public health disaster are still causing chaos amongst the nations of the world. On January 30, 2020 the World Health Organization (WHO) declared COVID-19 a global health emergency (WHO, 2020). The vast number of asymptomatic carriers makes it extremely difficult to control the infection, contributing to the public health burden (Gandhi et al., 2020).

Detection of SARS-CoV-2 RNA is the gold standard of diagnosis. The United States Center for Disease Control and Prevention (CDC) developed initial real-time RT-PCR reagents, including primers N1 and N2 that target the virus nucleocapsid phosphoprotein (N) gene along with the RNase P gene for specific detection of SARS-CoV-2 and human nucleic acid, respectively. The detection of human nucleic acid serves as the internal control to ensure proper sampling (CDC, 2020a). This public health emergency caused the United States Food and Drug Administration (FDA) to issue policy guidelines, specific for Emergency Use Authorization (EUA), for in vitro diagnostics for detection and diagnosis of the virus that causes COVID-19. A number of manufacturers secured EUA status to produce testing kits for their respective laboratory real-time RT-PCR platforms suitable for Clinical Laboratory Improvement Amendments (CLIA) certified laboratories (FDA, 2020a). The Central Texas Veterans Health Care System (CTVHCS) microbiology division has been using the GeneXpert and BD MAX™ systems for other FDA cleared/approved molecular biology testing. Due to the limitation of testing kit availability, because of high demand, we tried to secure testing on both platforms to cater to our patient population. After the required verification procedures (Mitchell et al., 2020), we have been extensively using both platforms for the detection of COVID-19 in patient samples.

The BD MAX™ system uses multiplexed primers and probes targeting SARS-CoV-2 RNA from the N gene (N1 and N2 regions) and the human RNase P gene. The primer and probe sets are based on the CDC assay for specific detection of SARS‑CoV-2 which amplifies 2 unique regions of the N gene, N1 and N2. The assay can be performed using upper respiratory tract (URT) swab specimens in Viral Transport Media and Universal Transport Media (VTM/UTM), nasopharyngeal wash/aspirates, or nasal aspirates obtained from individuals suspected to have COVID-19 (Company, 2020). However, on July 7, 2020 the FDA issued a product advisory notification alerting clinical laboratory staff and health care providers of an increased risk of false positive results when BD SARS-CoV-2 Reagents for the BD MAX™ SARS-CoV-2 assay are used. Citing one study, the FDA found that approximately 3% of results were false positives (FDA, 2020b). The FDA requested laboratories to consider alternative authorized testing for presumptive positives from the BD MAX™ system.

Concurrently, the GeneXpert system has been widely used as a commercial testing platform. The Department of Veterans Affairs entered into a national contract with the company Cepheid for reagent supplies. The Cepheid Xpert® Xpress SARS-CoV-2 (Xpert®) test on the GeneXpert system detects the N2 region of the N gene and the envelope (E) gene (Cepheid, 2020).

In this quality assessment study, we evaluated the analytical and clinical performance characteristics of the Cepheid Xpert® Xpress SARS-CoV-2 and BD MAX™SARS-CoV-2 assays. We examined comparative cycle threshold (Ct) values from both systems with the 183 positive samples tested at this facility.

Materials and methods

BD MAX™ SARS-CoV-2 assay

This assay is an automated in vitro diagnostic test. The BD SARS-CoV-2 Reagents for the BD MAX™ system are used in a real-time RT-PCR test intended for the qualitative detection of nucleic acid from SARS-CoV-2 in nasopharyngeal, nasal, mid-turbinate, and oropharyngeal swab specimens; nasopharyngeal wash/aspirates or nasal aspirates obtained from individuals suspected to have COVID-19. The platform consists of the BD MAX™ instrument, a computer, and the preloaded software for running tests and viewing the results. The BD MAX™ ExK TNA-3 unitized reagent strip contains a combination of lytic and extraction reagents designed to perform cell lysis and total nucleic acid (TNA) extraction. Eluted TNA is transferred to SARS-CoV-2 primers and probes as well as the BD MAX™ TNA MMK master mix. The final rehydrated master mix is transferred to a PCR cartridge for real-time RT-PCR. The amplified cDNA targets are detected using hydrolysis (TaqMan®) probes, labeled at one end with a fluorescent reporter dye (fluorophore) and at the other end with a quencher moiety. Probes labeled with different fluorophores are used to detect the target analytes (N1, N2 and RNase P) in different optical channels of the BD MAX™ system.

Thermal cycling is performed at 58°C for 20 minutes for reverse transcription, followed by 95°C for 5 minutes, 45 cycles of 95°C for 5 seconds, and 58°C for 40 seconds. An internal control targeting the human RNase P gene is co-amplified, along with the N1 and N2 gene targets if present and serves as an endogenous control present in all properly collected patient samples. This control serves as both an extraction control and an internal amplification control. The BD MAX™ system has an interpretive software for interpretation of results with an established threshold for calling positives or negatives. The presence of N1 or N2 genes is reported as a positive result. Negative N1 and N2 results alongside positive RNase P results are interpreted as not detected. Unresolved N1 or N2 results with unresolved RNase P results are interpreted as unresolved (UNR) and require repeat testing. Positive results are indicative of the presence of SARS-CoV-2 RNA. Clinical correlation with patient history and other diagnostic information is necessary to determine patient infection status (Company, 2020).

Previously reported negative samples (known negative patients) by our in-house GeneXpert system and sample buffer tubes with RNase P positive control materials are used as negative controls. Previously characterized positive samples (known positive patients) and sample buffer tubes with N1 and/or N2 positive control materials are used as positive control options.

Determination method called for false-positive in BD MAX™

Irrespective of direct result interpretation by the BD MAX™ system, we always look for classical sigmoidal curves with 3 distinct typical RT-PCR phases to confirm the results. In Phase I there is little or no signal. Phase II shows exponential growth where the fluorescent signal is detected above the base line. Phase III shows starting inflection and plateaus (Bustin and Mueller, 2005). Our technicians were trained to understand typical cDNA application curves. A curve based positive decision was called if a typical amplification curve for N1 and/or N2 was present. A curve based negative decision was called if repeated atypical curves with proper amplification of the internal control was present. An atypical curve was followed by repeat testing.

Cepheid Xpert® Xpress SARS-CoV-2 assay

This system is also an automated in vitro diagnostic test for the qualitative detection of nucleic acids from SARS-CoV-2. The Cepheid Xpert® Xpress SARS-CoV-2 test is performed on the GeneXpert system that automates and integrates sample preparation, nucleic acid extraction, amplification, and detection of the target sequences using real-time RT-PCR assays. This system uses single-use disposable cartridges that hold the RT-PCR reagents and host the RT-PCR process. This test includes reagents for the detection of RNA from SARS-CoV-2 in nasopharyngeal, oropharyngeal, nasal, or mid-turbinate swab and/or nasal wash/aspirate specimens. A Sample Processing Control (SPC) and a Probe Check Control (PCC) are also included in the cartridge. The SPC is present to control for adequate processing of the sample and to monitor for the presence of potential inhibitors in the RT-PCR reaction. The SPC also ensures that the RT-PCR reaction conditions, temperature and time, are appropriate for the amplification reaction. It also ensures that the RT-PCR reagents are functional. The PCC verifies reagent rehydration, PCR tube filling, and confirms that all reaction components are present in the cartridge. Additionally, the PCC monitors for probe integrity and dye stability. The detection of N2 and E or solely N2 indicates positive results. The presence of only E indicates presumptive positive results while the presence of only SPC indicates negative results. The absence of all markers indicates an invalid result.

Sample preparation and selection for comparative study

The comparative performance of BD SARS-CoV-2 Reagents for the BD MAX™ system and the Cepheid Xpert® Xpress SARS-CoV-2 assay was retrospectively analyzed using 183 positive clinical samples reported in Tables 1 and 2 . The first batch of 98 randomly selected true positive samples were reported using the BD MAX™ platform before the FDA recommendation was released (Table 1). The second batch of 85 positive samples was initially tested using the BD MAX™ system and confirmed with the GeneXpert system after the FDA letter was released (Table 2). Samples were tested through our standard operating procedures based on the manufacturers’ recommendations. Clinical samples were collected by qualified personnel in accordance with the package insert of the collection device. Samples were handled as described in the package insert of the collection device. They were delivered to the lab under refrigeration to be tested immediately or stored at 2 to 8°C for up to 72 hours. If a delay in testing was expected, samples were stored at −70°C until use. All samples, including the 85 positive samples tested with BD MAX™ and retested with GeneXpert, were tested on the same day to avoid freeze thaw variability. Samples summarized in the Table 1 and Table 2 are before and after the FDA recommendation announcement, respectively. At the end of data collection, as shown in Table 2, 659 samples were processed by the BD MAX™ system.

Table 1

Comparative study of 54 randomly selected BD MAX™ positive samples and 44 randomly selected positive GeneXpert samples, a total of 98 samples, before the FDA released testing recommendations.

	BD MAX™ +	BD MAX™ -
GeneXpert+	93	2b
GeneXpert-	3c	75a

BD MAX™ negative results were compared with those of a reference lab.

Both samples were called negative by the reference lab, due to probable GeneXpert false positive.

Two of these samples were from previously known positive patients, the third sample is a true false positive from BD MAX™ as GeneXpert called the sample negative. Comparative Sensitivity of BD MAX™ is: . If known positives are considered, the absolute sensitivity would be: . Specificity of BD MAX™ excluding false positives is: . From April 21st to June 9th, the false positive rate was: .

75 BD MAX™ negative samples were compared with those from a reference lab (LabCorp) to study for specificity.

Table 2

Upon implementing FDA recommendations, out of 659 total tests, 85 samples were positive in BD MAX™ and subsequently compared in GeneXpert; 18 false positive samples from BD MAX™ were also tested with the alternative platform GeneXpert.

	BD MAX™ +	BD Max™ known false positivea
GeneXpert +	85	0
GeneXpert -	0	18a

Observed false positive samples were repeated in BD MAX™ and reconfirmed with GeneXpert, we experienced a 2.7% false positive rate from BD MAX™ while all samples were reported negative after repeat testing.

CTVHCS only used the CDC recommended method of specimen attainment and transport: to place nasopharyngeal (NP) swab specimen into 3ml of VTM. Specimen was briefly mixed by rapidly inverting collection tubes 5 times. Using the supplied 300 μL transfer pipette, the samples were transferred to the sample chamber of the Xpert® Xpress SARS-CoV-2 cartridge. As an alternative method, 750 μL from the VTM specimen were transferred, using a calibrated variable pipette, directly into the BD MAX™ TNA-3 Sample Buffer Tube. Both samples were loaded onto the respective instruments as per manufacturer specifications. This allowed for hands-off automated sample processing and real-time RT-PCR for the detection of viral RNA.

Statistics

Paired Ct values among the 2 systems were analyzed with a Wilcoxon Signed Rank test using GraphPad Prism software. Descriptive statistics of the comparative Ct value ranges were also analyzed with the same software. A P-value of less than 0.05 was considered significant.

Results

BD MAX™ generates 3.5% false positive results

After introducing BD MAX™ system testing on April 21, 2020 this laboratory ran 2597 tests through July 9, 2020. Of those tests 88 positive samples were determined to be false positives based on erroneous amplification curves, which necessitated repeated testing. Out of the 88 false positive samples, the system called N2 as positive 58 times, N1 as positive 18 times, and both markers as positive 12 times. Our repeated tests on the BD MAX™ system and the GeneXpert system confirmed those 88 samples were negative and subsequently reported them as negative results. The BD MAX™ system can generate single sample amplification plots that help ascertain proper cDNA amplification. Irrespective of direct result interpretation, we always look for classical sigmoidal curves. As such, Fig. 1 A shows representative false positive results from N2 amplification that can be identified by the slight drifting of the amplification curve off the baseline. However, as shown in Fig. 1C, when comparing the entire testing panel with 2 positive patient samples, the false blip of the N2 amplification is not noticeable, although the system called it positive. This lab always retests results using interpretive software that calls positive results with defective amplification curves. Fig. 1E shows repeated runs of a sample calling negative results with no remarkable drift of the amplification curve from the baseline. Fig. 1G shows the repeated testing of samples, including the one shown in Fig. 1E, run with a positive patient sample. In a similar manner, we show a false positive result generated from N1 amplification in Fig. 1B. This false positive interpretation was again due to the slight drift of the amplification curve above baseline noise level as shown with other true negative samples and 2 positive patient samples in Fig. 1D. When we retested samples to confirm false positive results, the curves did not drift above the baseline noise level as shown in Fig. 1F. Fig. 1H shows the same repeated negative sample with N1 false positive patient (Fig. 1B) results that are similar to the N2 false positive patient (Fig. 1A) results shown in 1G. In this manner, our close observation and individual sample examination have averted reporting false positive software interpretations before the FDA alert about this issue was even released.

Fig. 1.

Representative BD MAX™ technical false positive results from SARS-CoV-2 N2 and SARS-CoV-2 N1 amplification. (A) Zoomed out N2 false positive curve without the sigmoidal curve below a threshold of 200. B. Zoomed out N1 false amplification with a threshold below 250. (C and D) Samples of N2 and N1 false positives superimposed with 2 real positive controls in the same batch. (E and F) Zoomed out respective repeated samples of N2 and N1 of A and B with negative results showing no amplifications. (G and H) Samples of N2 and N1 repeat (negative) superimposed with a real positive control in the same batch.

Another example of a completed 12 run report is explained in Fig. 2 which shows a representative 12 patient sample set with one previously known positive sample serving as the positive control. Fig. 2A and Fig. 2B clearly show typical sigmoidal amplifications of the 3 positive samples with SARS-CoV-2 N1 and N2 respectively. In addition, Fig. 2C shows all samples having RNase P amplifications, justifying sample integrity and the endogenous nucleic acid extraction control. When we looked at the results run report generated by the BD MAX™ interpretive software system, 7 SARS-CoV-2 positives were shown. Out of these 7 samples, 4 were negative for SARS-CoV-2 N1 and positive for SARS-CoV-2 N2. Their individual amplifications are shown in Fig. 2D and Fig. 2E respectively. When the amplifications in Fig. 2E are compared to the quality of the real positive amplifications in Fig. 2A and Fig. 2B, it can be seen that Fig. 2E has no proper Phase II and III of a typical RT-PCR amplification curve. Fig. 2G shows a zoomed out single sample amplification curve from Fig. 2E, to clarify for the readership. The software called the sample positive as the curve had drifted from the baseline and crossed the threshold. However, the RNase P gene was smoothly amplified as shown in Fig. 2F indicating that this is not an error of sampling or faulty reagents. This type of technical false positive result could be due to errors in the thermocycling algorithm that caused the system to be unable to interpret baseline noise levels. Once we retested these samples in the same platform they were correctly called as negative. As we have examined the data shown in Tables 1, 2 and 3 , it is confirmed that we are correctly recognizing positive and false positive results.

Fig. 2.

Batch of 12 samples with 3 positive results and 4 false positive results in BD MAX™. (A) SARS-CoV-2 N1 amplification plot showing 3 positive curves. (B) Corresponding SARS-CoV-2 N2 amplification plot showing 3 positive curves. (C) SARS-CoV-2 RNase P amplification of 12 samples in the batch. (D) Zoomed out N1 amplification plot of 4 samples (N1 not detected) showing false positive results. (E) Zoomed out N2 amplification plot of 4 samples (N2 detected) showing false positive results. (F) Zoomed out corresponding SARS-CoV-2 RNase P amplification plot of 4 false positive samples showing false positive results. (G) Zoomed out single N2 false positive sample amplification.

Table 3

Evaluation with positive NP swab samples based on Ct value range.

Ct bracket	Valid results	% positive	N1 region for BD MAX™		N2 region comparison between BD MAX™ and GeneXpert			RNase P
			Agreement with expected results	Mean Ct value	Agreement with expected results	Mean Ct value (BD MAX™)	Mean Ct value (GeneXpert)	Mean Ct value
<25	60	100%	60/60	16.9 ± 4.6	60/60	16.6 ± 4.7	20.9 ± 4.3	21.3 ± 1.6
25–30	36	100%	36/36	28 ± 1.9	36/36	27.7 ± 1.5	31.6 ± 2.7	21.7 ± 1.5
30–34	59	100%	57b/58	32 ± 4.7	58/58	32.2 ± 1.1	35.7 ± 7	21.8 ± 1.5
>34	28a	92%	19c/30	35.5 ± 1.8	18d/30	36.4 ± 1.8	38.3 ± 8	22.5 ± 1.5

2 BD MAX™ samples were found to be false positives in this subset.

1 sample was negative for N1 detection but positive for N2 detection.

9 samples were positive for N2 detection but negative for N1 detection.

10 samples were positive for N1 detection but negative N2 detection.

BD MAX™ consistently shows significantly lower Ct values for N2 when compared with Cepheid Xpert® Xpress

We examined 98 positive samples between the 2 platforms (Table 1) and compared N2 Ct values as shown in Fig. 3 . Paired Ct values from the BD MAX™ system are significantly lower than those of the GeneXpert system (P < 0.0001) as shown through the Wilcoxon Signed Rank test. Four samples of the data set marked only N1 as positive and as a result Fig. 3 shows zero N2 Ct values for the BD MAX™ system. In addition, 2 samples were positive only on the BD MAX™ system, but the Xpert® test was negative showing zero for N2. Based on alternative testing (LabCorp Aptima SARS-CoV-2 assay), we found one was a false positive from the BD MAX™ system and the other a false positive from the GeneXpert system.

Fig. 3.

Comparative paired Ct values from N2 amplifications of positive samples (n = 98) between the BD MAX™ and Cepheid GeneXpert systems. BD MAX™ Ct counts are significantly lower than those of Cepheid GeneXpert (P < 0.0001).

Comparative evaluation

Even before the FDA issued a recommendation to consider an alternative platform to confirm positive BD MAX™ results, our lab noticed a number of false positive results and opened several troubleshooting cases with the BD technical service. Therefore, as a quality assessment effort, we compared representative positive results against each platform. We examined 98 positive clinical samples, shown in Fig. 3, using both platforms. The results are summarized in Table 1. Out of those tests, 45 were first reported positive by the GeneXpert system and were subsequently tested by the BD MAX™ system. Fifty-three tests were first reported positive in BD MAX™ and subsequently tested in GeneXpert. In addition, 75 samples originally reported negative in BD MAX™ were sent out to a reference lab (LabCorp Aptima SARS-CoV-2 assay) to be examined for negative agreement from an alternative platform, as shown in Table 1. The sensitivity of BD MAX™ compared with that of GeneXpert is 96.8% and the comparative specificity of BD MAX™ compared with the reference lab's data is 97.4%.

Once the FDA released a recommendation, issued on June 6, 2020, to consider an alternative platform to confirm BD MAX™ positive results (FDA, 2020b), we continuously retested positive BD MAX™ results on GeneXpert, as shown in Table 2. Eighty-five positive samples out of 659 clinical samples (excluding identified false positive results) were confirmed with GeneXpert showing 100% agreement. In addition, 18 of the 659 samples were recognized as false positives based on our criteria mentioned in the methods and were reconfirmed negative on GeneXpert. This rate of false positives, 2.7%, is again close to the rate of 3% that the FDA mentioned in their recommendation (FDA, 2020b).

In order to investigate Ct value range correlations among the 2 platforms, we evaluated the positive agreement of both platforms based on Ct value categories described by Reed et al. (Magleby et al., 2020), for high viral load (Ct < 25; n = 60), medium viral load (Ct = 25–30; n = 36), and low viral load (Ct > 30; n = 59). Another group (n = 28) of patients, reported positive with a Ct value count greater than 34, were compared in a separate group as a recent study suggested that patients with a Ct value count greater than 34 are not contagious (La Scola et al., 2020).

We found that samples with high, moderate, and low Ct values showed 100% agreement with expected results. In addition, positive samples with a Ct value count greater than 34 (n = 28), suggested to be from patients in noninfectious stages of the disease (La Scola et al., 2020), still showed a 92% positive agreement between the 2 platforms (Table 3). However, in the n = 28 positive group 9 samples were positive for N2 detection but negative for N1 detection and 10 samples were positive for N1 detection but negative for N2 detection in the BD MAX™ system. Of samples with Ct values less than 34, we only found one sample negative for N1 detection but positive for N2 detection in a low Ct value category. As previously shown by the data in Fig. 3, we found that in each category the BD MAX™ system has a lower mean Ct value than that of the GeneXpert system.

Discussion

This study found that both systems showed comparable sensitivity although the BD MAX™ SARS-CoV-2 assay had lower Ct values in all paired samples. In addition, we found the BD MAX™ system software called a rate of approximately 3.5% false positive interpretation. With basic knowledge of molecular biology and real-time RT-PCR our lab successfully averted reporting false positive results and clinical labs running the BD MAX™ system without molecular biology expertise would benefit from this study.

Real-time RT-PCR is the well-recognized high-throughput quantification of nucleic acids that has been beneficial in medical diagnostics. The present plethora of EUA COVID-19 diagnosis platforms are good examples of this technique. However, qualitative results from automated interpretation by high-throughput testing platforms are still technically difficult to achieve. This is primarily due to deficiencies of the threshold-based methodologies since the introduction of real-time PCR, as it easily ignores atypical amplification curves due to erroneous cDNA amplifications (Higuchi et al., 1993). Diagnoses of infections caused by SARS-CoV-2 are usually accomplished by performing real-time RT-PCR on URT specimens as an antibody response is not usually reported in the first 10 days postsymptoms (Haveri et al., 2020). Virus isolation in culture presents significant biosafety risks (Harcourt et al., 2020). Also, URT specimens may have detectable RNA during the presymptomatic phase (Pan et al., 2020). Therefore, specimens such as NP swabs in VTM/UTM have become the gold standard of sample acquisition for testing during the present pandemic (Lu et al., 2020).

Due to high demand in testing, the FDA issued a EUA for various commercial clinical real-time RT-PCR platforms (FDA, 2020a). Our hospital utilized 2 such existing platforms, the GeneXpert system and the BD MAX™ system. Previous multicenter evaluation on the Xpert® test reported that it has a 99.5% positive agreement and a 95.8% negative agreement (Loeffelholz et al., 2020). Unfortunately, reagent supply is a limitation for this platform. We are fortunate to have a second platform, the BD MAX™ system, with a considerably more flexible reagent supply conduit. This hospital reserves the GeneXpert system for emergency testing such as an admission necessitating an average turnaround time (TAT) of one hour. The rest of the testing, including employee screening, is done with the BD MAX™ system with an average TAT of 5 hours. BD MAX™ has been the main platform for COVID-19 testing at this institution.

After the initial recommended verification of the BD MAX™ platform (Mitchell et al., 2020), we consistently encountered false positive results and our retesting mostly mitigated the impact of false positives. This can be a challenging situation for clinical laboratories that lack molecular expertise. Our results explain how we handle such scenarios without jeopardizing patient safety. We believe this report can help others understand how to avert reporting “technical” false positive results when using the BD MAX™ platform for testing COVID-19 NP swabs. qPCR troubleshooting is beyond the scope of this article as this issue arises from a manufacturer's proprietary algorithm. However, we recommend the MEQE guidelines if our readership wants to understand the mechanism of false positive results (Bustin et al., 2009). Until the manufacturer troubleshoots this RT-PCR protocol, our lab will reconfirm new cases of COVID-19 positive samples with an alternative platform as per FDA recommendation. However, we exclude reconfirming previously known positive samples to save crucial resources. In addition, we kept repeating technical false positives in the BD MAX™ platform to verify that there is a real technical issue in the PCR thermocycling algorithm. If we found typical amplification curves in N1 and or N2 in repeatedly tested false positive samples, the results were reconfirmed again with an alternative platform (GeneXpert) although we have encountered this situation rarely.

In addition to troubleshooting and understanding false positive results, this quality improvement study shows that BD MAX™ positive results, excluding obvious false positives, have a strong correlation with comparable platforms. Positive samples with Ct counts less than 34 showed 100% agreement with GeneXpert results. Both of our platforms showed some inconsistencies when it comes to analyzing low viral load samples that generated a Ct value count greater than 34. As previously claimed (La Scola et al., 2020), if such samples are properly collected, there is a minimal risk of contagiousness. When compared with alternative platforms, both systems had few true false positives in the samples with Ct value counts greater than 34. In addition, when we retested 18 false positive samples in GeneXpert (Table 2), they all reported negative substantiating our interpretation criteria explained here.

We also found that RNase P is a sound quality control target. As shown in Table 3, properly collected samples have Ct values of approximately 22. In our limited observation of RNase P Ct values, Ct values greater than 25 generated variable poor-quality results. We did not have enough samples to analyze them statistically. Our observation of differential Ct values in each platform for the same sample could be due to different algorithms in the respective systems which may not reflect the sensitivity of the results. Further studies are needed to examine such a comparison. Overall, we believe this report will help other labs in troubleshooting their use of BD MAX™ instruments for the detection of SARS-CoV-2 in suspected COVID-19 patients.

In response to our inquiries, BD confirmed that most of these complaints concerned atypical curves with low endpoints. They found that all the runs showed a presence of background noise caused by signal drift in the Cy5 channel, with the N2 target causing an interruption in background correction. This caused the generation of a steady increase in fluorescence of the curves, resulting in false positive results. Contributing factors in every case included the instrument in combination with the User Defined Protocol (UDP) programming and product design. The root cause is under investigation and will be documented soon by the manufacturer. As we occasionally see N1 involving false positives, we requested BD to investigate that as well.

Small differences in amplification efficiency lead to quantitative errors. The frequency and magnitude of unconventional amplification curves are impossible to call correctly by an automated software driven fixed threshold approach. This resulted in false positive predictions by the BD MAX™ platform. Based on our observations, false positive results in SARS-CoV-2 testing can be detected by understanding the nature of cDNA amplification curves in the BD MAX™ system and by including known positive control patient samples in each batch of test runs. In addition, the impact of false positive results could be mitigated by requiring 2 independent positive tests to identify a sample as positive.

Conclusions

False positive results reported with the BD MAX™ SARS-CoV-2 assay could be mitigated by using amplification curve based decisions rather than having complete dependence on the BD MAX™ platform automated interpretive results. We have revalidated the BD MAX™ platform by examining it in contrast with an alternative platform, GeneXpert, to show that results from both platforms are comparable. With a basic understanding of cDNA amplification in RT-PCR, technical false positive results can be identified and averted easily.

Declaration of competing interest

The authors report no conflicts of interest relevant to this article.

Funding

This work was supported by the resources of the Central Texas Veterans Health Care System (Temple, TX) and the Central Texas Veterans Health Care System Research Service.

Author statement

Dhammika H. Navarathna: Conceptualization, Methodology, Software Data curation, Writing- Original draft preparation, Investigation and supervision Shawn Sharp.: Data curation, Writing- Original draft preparation. Validation Janell Lukey: Visualization, Investigation. Data curation Monica Arenas : Data curation.: Horace Villas: Data curation, Linda Wiley: Data curation, Ivy Englett: Data curation Ma Rowena San Juan: Data curation Chetan Jinadatha: Conceptualization Writing- Reviewing and Editing.