Distribution of SARS-CoV-2 PCR Cycle Threshold Values Provide Practical Insight Into Overall and Target-Specific Sensitivity Among Symptomatic Patients.

OBJECTIVES: We examined the distribution of reverse transcription polymerase chain reaction (RT-PCR) cycle threshold (CT) values obtained from symptomatic patients being evaluated for coronavirus disease 2019 (COVID-19) to determine the proportion of specimens containing a viral load near the assay limit of detection (LoD) to gain practical insight to the risk of false-negative results. We also examined the relationship between CT value and patient age to determine any age-dependent difference in viral load or test sensitivity.
METHODS: We collected CT values obtained from the cobas severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) assay corresponding to 1,213 combined nasopharyngeal-oropharyngeal specimens obtained from symptomatic individuals that were reported as positive or presumptive positive for SARS-CoV-2. CT values were stratified by SARS-CoV target and patient age group.
RESULTS: In total, 93.3% to 98.4% of specimens demonstrated CT values greater than 3× the assay LoD, at which point false-negative results would not be expected. The mean of CT values between age groups was statistically equivalent with the exception of patients in age group 80 to 89 years, which demonstrated slightly lower CTs.
CONCLUSIONS: Based on the distribution of observed CT values, including the small proportion of specimens with values near the assay LoD, there is a low risk of false-negative RT-PCR results in combined nasopharyngeal-oropharyngeal specimens obtained from symptomatic individuals. © American Society for Clinical Pathology, 2020. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Distribution of SARS-CoV-2 RT-PCR CT values suggests a low risk of false-negative results when testing symptomatic patients.

SARS-CoV-2 RT-PCR CT values are similar among different age groups, suggesting equivalent test performance irrespective of patient age.

The inclusion of a pan-Sarbecovirus target in a dual target SARS-CoV-2 RT-PCR assay increases the sensitivity by approximately 3%.

The diagnosis of coronavirus disease 2019 (COVID-19) in symptomatic patients is most frequently made based on the detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) RNA in respiratory specimens. The absolute sensitivity of reverse transcription polymerase chain reaction (RT-PCR) tests is difficult to ascertain due to lack of a true “gold standard” and is dependent on several factors, including specimen type, collection method, and the specific test used. Several studies have compared the performance of available RT-PCR tests using a split specimen approach and report 96% to 100% positive agreement (proxy for sensitivity) based on consensus test results.[1-4] Lower sensitivity of 75% to 90% has been reported for a rapid point-of-care (POC) molecular test when compared to laboratory-based RT-PCR assays.[2,5,6] This is likely due to a difference in limit of detection (LoD) between the laboratory-based tests (39-779 copies/mL) and the POC test (3,000-20,000 copies/mL).[2,4,6] Despite similar performance of laboratory-based tests, concern has been raised about the absolute sensitivity of RT-PCR based on studies using chest imaging and serial testing as the gold standard that suggested a sensitivity of only 71% to 83% for initial RT-PCR results.[7-9] Practically speaking, aside from preanalytical factors such as specimen quality, the sensitivity of a test is largely dependent on a combination of (1) LoD of the test and (2) the distribution of viral load in the population being tested. A population or specimen cohort containing a large proportion of viral loads near the test LoD will result in a lower observed clinical sensitivity than a cohort containing very few patients or specimens with a viral load near the test LoD. In this way, clinical sensitivity is at least partially independent of analytical sensitivity (ie, absolute LoD).

We conducted a review of cycle threshold (CT) values for 1,213 patients who tested “positive” or “presumptive positive” on the cobas SARS-CoV-2 assay (Roche) to determine the distribution of CT values obtained for SARS-CoV-2 specific target ORF1 and the pan-Sarbecovirus E-gene target. The primary aim of the study was to establish the distribution of observed CT values for each target in symptomatic patients, and to determine the proportion of patients with CT values near the LoD as an indicator of the likelihood of false-negative results due to low viral load. We also examined the correlation between patient age and CT value, and assessed the difference in CT value between ORF1 and E-gene targets as an indicator of relative sensitivity of each target for detection of SARS-CoV-2.

Materials and Methods

Specimen Collection and Prospective Testing

Wisconsin Diagnostic Laboratories offers separate and specific test orders for “symptomatic” and “asymptomatic” COVID-19 RT-PCR. Specimens included in this study were collected from patients at more than 30 hospitals and long-term care facilities in Southeast Wisconsin and Northeast Illinois who had a specific order for “symptomatic” COVID-19 RT-PCR. Combined nasopharyngeal and oropharyngeal swab specimens were collected in various viral transport medium including M4, M4-RT, M6 (Remel), UTM (Copan), and VTM (Hardy Diagnostics). All specimens were submitted to Wisconsin Diagnostic Laboratories, Milwaukee, WI, and were tested within 48 hours of collection using the cobas SARS-CoV-2 real-time RT-PCR assay run on the fully automated cobas 6800 system. All testing and interpretation were conducted in accordance with the cobas SARS-CoV-2 emergency use authorization (EUA) protocol (Doc Rev 3.0). A total of 1,213 specimens were reported as “positive” or “presumptive positive” based on the detection of SARS-CoV-2 ORF1 sequence (Target 1) and/or pan-Sarbecovirus E-gene sequence (Target 2) in accordance with the manufacturer instructions. CT values for Target 1 and Target 2 were recorded for each of the 1,213 specimens, in addition to patient age and gender.

Confirmation of Assay Limit of Detection

The manufacturer’s stated LoD based on dilution of a SARS-CoV-2 clinical isolate is 0.007 median tissue culture infectious dose (TCID50)/mL for Target 1 and 0.004 TCID50/mL for Target 2, which correspond to CT values of approximately 32.7 and 36.4, respectively. To confirm this LoD, we made 7 serial 10-fold dilutions of a clinical specimen with reported CT values of 22.23 (Target 1) and 22.57 (Target 2). Each dilution was run in quintet and CT values were recorded. The number of positive results for each target at each dilution were recorded, and CT values for each replicate and target were plotted against dilution factor to enable regression analysis and assessment of assay result linearity.

Statistical Analysis

Standard equations in Excel (Microsoft) were used to conduct regression analysis and to calculate mean, median, interquartile range, 95% confidence interval (CI), and standard deviation (SD) of the data sets. All P values were calculated using VassarStats website for statistical computation (http://vassarstats.net/).

Results

Limit of Detection

Seven serial 10-fold dilutions of a clinical specimens with initially reported CT values of 22.23 (Target 1) and 22.57 (Target 2) were tested in quintet to compare the LoD observed in our laboratory to the LoD reported by the manufacturer. Target 1 and Target 2 were detected in 100% of replicates tested at dilution of 10–1 to 10–5. This corresponded to average CT values of 35.9 for Target 1 and 37.2 for Target 2. The line of regression (R2 value) for CT values across this dilution range was 0.991 for Target 1 and 0.996 for Target 2, indicating a strong linear relationship between values across this range of viral concentrations . At a 10–6 dilution, Target 1 was detected in 60% of replicates and Target 2 was detected in 40% of replicates with average CT values 36.99 and 37.52, respectively. In addition, CT values for each target fell off the regression line, which is consistent with variable detection of SARS-CoV-2 at this low target concentration. Our findings are consistent with manufacturer data demonstrating higher average CT values for Target 2 compared with Target 1 despite a similar absolute LoD. However, we found absolute CT values corresponding to the assay LoD were 3.2 and 0.8 higher for Target 1 and 2, respectively, than reported by the manufacturer. These data are important for subsequent calculation of population distribution in relation to LoD and CT observed in our laboratory.

Standard curve for SARS-CoV-2 Targets 1 and 2. A series of 7 10-fold dilutions was created from a clinical specimen that initially tested positive for SARS-CoV-2. Five replicates of each dilution were tested in parallel. The initial specimen had reported cycle threshold (CT) values of 22.23 for Target 1 (A) and 22.57 for Target 2 (B). All replicates were detected at dilutions of 10–1 to 10–5 and fell onto a line with slope of –0.33 to –0.36 and R2 value greater than 0.99. At a 10–6 dilution only 60% of Target 1 replicates (C) and 40% of Target 2 replicates (D) were detected. These values did not fall onto the best-fit line of regression, indicating poor linearity. No replicates were detected at 10–7 dilution. y = –0.3328x + 7.4923; R2 = 0.9965.

Distribution of Target 1 and 2 CT Values Among Positive Specimens

In total, 1,178/1,213 (97.1%) specimens had a reportable CT value for both Target 1 and Target 2. The mean, median, and interquartile range (IQR) of CTs for Target 1 were all numerically lower than comparative values for Target 2 (mean, 25.26 vs 26.29; median, 25.02 vs 25.93; IQR, 20.35-20.99 vs 21.14-21.73). The difference in mean CT value between Target 1 and Target 2 was statistically significant (P < .001); however, the median and IQR demonstrated a high degree of similarity and overlap of CT values between the 2 targets . When stratified by age, the mean CT values for each age group were statistically equivalent to the entire data set for each target with exception of Target 1 CTs for patients age 80 to 89 years. The mean of CT values in this group were 1.77 CT lower than the mean of all Target 1 values (P = .028), suggesting a higher average viral load in specimens collected from this age group. Mean CT values were also noticeably lower for Target 2 in this age group, as well as Targets 1 and 2 in patients age 10 to 19 years, but failed to reach statistical significance of P < .05 (Table 2) .

Table 2

Cycle Threshold (CT) Values for 1,213 Specimens Reported as Positive or Presumptive for SARS-CoV-2

		Target 1 (ORF1)				Target 2 (E-Gene)
Group	No. of Specimens	Mean CT	Median CT	IQR	P Valuea	Mean CT	Median CT	IQR	P Valuea
All specimens	1,213 (100%)	24.57	25.02	20.35-29.99	—	26.29	25.93	21.14-31.73	—
Age <10 y	20 (1.6%)	25.95	24.02	21.59-30.82	.568	26.71	24.3	22.19-31.87	.764
Age 10-19 y	33 (2.7%)	22.78	21.87	19.09-27.00	.067	24.54	22.04	19.22-29.66	.114
Age 20-29 y	150 (12.3%)	24.59	24.76	20.17-29.31	.984	26.16	25.55	20.79-30.30	.818
Age 30-39 y	174 (14.3%)	24.74	24.98	20.65-30.00	.881	26.2	25.93	21.06-31.19	.872
Age 40-49 y	231 (19.0%)	24.84	25.02	20.61-30.05	.552	26.51	25.12	20.66-30.85	.624
Age 50-59 y	194 (16.0%)	24.88	25.66	20.02-29.82	.779	26.05	26.05	20.27-31.09	.631
Age 60-69 y	161 (13.3%)	24.54	25.92	20.33-30.03	.610	26.76	26.67	21.18-32.22	.362
Age 70-79 y	126 (10.4%)	25.05	25.62	21.07-30.34	.230	27.09	26.28	22.09-32.26	.170
Age 80-89 y	86 (7.1%)	22.80	22.42	19.29-26.67	.028	25.02	23.86	20.22-29.31	.068
Age >89 y	38 (3.1%)	23.81	24.62	19.59-28.96	.888	26.48	27.61	22.25-32.12	.849

IQR, interquartile range.

a P values are based on a comparison of mean CT values of each age group vs the mean of all CT values for Target 1 or Target 2. Bold value indicates significance, P < .05.

Comparison of cycle threshold (CT) by age. The mean (dots) and interquartile range (whiskers) of CT values obtained for Target 1 and Target 2 are plotted for patients in the indicated age ranges. The mean value of Target 1 CTs was significantly lower than the cumulative mean of all Target 1 CT values (*P = .028) for patients age 80-89 years, indicating potentially higher viral load in these specimens.

Specimens With a Single SARS-CoV Target Detected

Thirty-five (2.9%) specimens had a reportable CT value for only 1 of 2 targets. Among these, 33 (94.3%) were positive for pan-Sarbecovirus E gene (Target 2) and were reported as “presumptive positive” for SARS-CoV-2, while 2 (5.7%) were positive for SARS-CoV-2 specific ORF1 (Target 1) and were reported as “positive” for SARS-CoV-2. The average CT value of Target 2 in specimens negative for Target 1 was 37.64 (range, 34.56-41.05) indicating a low viral load. All 33 values were within the highest 10.9% of all Target 2 CTs reported, including 20/35 (57.4%) of all values higher than 37 and all 5 CT values higher than 40 reported. These data indicate increased sensitivity of the pan-Sarbecovirus E-gene target, resulting in a 2.8% increase in total test sensitivity when compared to the SARS-CoV-2 ORF1 target alone.

Estimated Risk of False-Negative Result Based on Distribution of CT Values

The LoD of an assay is frequently defined as the concentration of analyte at which 95% of tests are positive.[10,11] Based on our dilution studies, the CT value corresponding to the assay LoD is approximately 36 for Target 1 and 37 for Target 2. At concentration of 3× the LoD, 100% of specimens are expected to be detected. Therefore, the risk of false-negative result increases from 0% at 3× LoD (ie, 1.6 CT lower than LoD) to 95% at LoD, and progressively increases for specimens with viral load corresponding to CT values higher than the LoD. In our cohort of positive specimens, the Target 1 mean (24.57) and median (25.02) CT values for Target 1 are 11 to 12 CT below the LoD, which corresponds to a viral load in these specimens approximately 3.3 to 3.6 log viral copies/mL above the assay LoD. Only 19/1,180 (1.6%) reported CT values for Target 1 were higher than 34.5 (ie, <3× the assay LoD) and would be at risk of potential false-negative result due to a viral load near the assay LoD. Similarly, the Target 2 mean (26.29) and median (25.93) CT values are 10 to 11 CT below the assay LoD. A larger proportion of Target 2 CT values, 81/1,211 (6.7%), fell above 35.5 (ie, <3× the assay LoD) and would potentially be at risk of variable detection and less than 100% sensitivity based on low viral load . Of note, these specimens encompassed 28/33 (84.8%) of the specimens where Target 1 was not detected, supporting the increased sensitivity of the pan-Sarbecovirus E-gene target (Target 2) in this assay.

Distribution of cycle threshold (CT) values. The total number of specimens with indicated CT values for Target 1 and 2 are plotted. The estimated limit of detection for (A) Target 1 and (B) Target 2 are indicated by vertical dotted lines. Horizontal dotted lines encompass specimens with CT values less than 3× the LoD for which sensitivity of detection may be less than 100%. This included 19/1,180 (1.6%) reported CT values for Target 1 and 81/1,211 (6.7%) reported CT values for Target 2. Specimens with Target 1 or 2 reported as “not detected” are denoted as a CT value of “0.”

Discussion

The results of diagnostic tests play a central role in the identification and management of infected individuals. Therefore, it is important to have a thorough understanding of the performance characteristics of these methods to aid in interpretation of results. A key characteristic of any diagnostic test is the analytical LoD, which is commonly defined as the concentration of analyte that will be detected in 95% of replicate tests.[10,11] This LoD can have greater or lesser impact on the clinical sensitivity of a test (defined as the number of infected patients detected by a test/the number of total infected patients) depending on the distribution of analyte concentrations observed in a patient or specimen cohort. The closer the mean and median of a normally distributed sample set (“bell curve”) are to the test LoD, the larger the proportion of specimens that are near the test LoD will be. Specimens close to the LoD, especially those containing 3× the LoD of analyte and lower, are at the highest risk of not being detected and resulting in false-negative results. We sought to confirm the manufacturer-claimed LoD for the cobas SARS-CoV-2 EUA assay and estimate the risk of false-negative results in nasopharyngeal specimens collected from symptomatic patients based on the distribution of CT values reported.

Amplification efficiency resulting in variability in CT is observed even when using the same commercially available and FDA-cleared assay. Therefore, it was essential to establish the LoD observed using instrumentation and test reagents in our laboratory to control for this variability. The estimated LoD based on the greatest dilution in which 100% of replicates were detected and the CT value maintained an R2 value greater than 0.99 was 35.9 for Target 1 and 37.2 for Target 2. These values were 1 to 2 CT higher than claimed in the manufacturer’s product insert, which highlights the small interlaboratory variability frequently reported in multisite surveys using the same molecular assay.[12,13] While the estimated LoD based on detection of 100% of replicates was the same for Target 1 and 2, the average corresponding CT value was slightly lower for Target 1 at this dilution, resulting in a slightly lower R2 value for Target 1 across the linear range tested. This could be the result of differences in amplicon size, primer or probe binding efficiency, or other factors, but was consistent with target-specific CT data reported by the manufacturer indicating slightly lower CT values for Target 1 despite a similar LoD of 17 to 73 copies/mL for each target.[14]

Among 1,213 specimens reported as “positive” or “presumptive positive,” the median CT values for Target 1 and Target 2 were 25.02 and 25.93, respectively, indicating that the distribution of CT values observed in symptomatic patients is at least 10 CT above the LoD of the assay. A majority of Target 1 values (98.4%) and Target 2 values (93.3%) were more than 1.5 CT above the estimated LoD, corresponding to a viral concentration above 3× the LoD. Specimens at this viral concentration should be detected in 100% of replicates and would be unlikely to suffer a false-negative result. Specimens with viral load and corresponding CT values below 3× the LoD are expected to be at increased risk of generating a false-negative result due to variable detection of virus at these concentrations. The use of 2 independent targets appears to mitigate this risk. Of the 19 specimens with Target 1 CT values higher than 34.5 (ie, <3× the LoD), all were also positive for Target 2 with CT values of 25.94 to 37.76. In addition, 33 specimens reported as negative for Target 1 were positive for Target 2 with CT values of 34.56 to 41.58. Conversely, 2 specimens reported as negative for Target 2 were positive for Target 1 with CT values of 34.10 and 34.21. These data demonstrate that inclusion of a second target decreases the likelihood of false-negative result for specimens with low viral load and increases the final test sensitivity by nearly 3%. Specifically, inclusion of the pan-Sarbecovirus E-gene target (Target 2) results in the detection of SARS-CoV-2 in 33 (2.8%) more specimens than relying on the SARS-CoV-2 specific ORF1 (Target 1) alone. This is consistent with the higher regression analysis value for Target 2 CTs when including the lowest dilution, suggesting less variation of detection at lower viral concentrations and potentially higher sensitivity when compared to Target 1. However, 2/35 (5.7%) single target detections were Target 1 alone, which reinforces the role of both targets in increasing overall test sensitivity. Based on the distribution of CT values indicating 93% to 98% of specimens containing a viral concentration at least 3× the LoD, the use of 2 independent targets, and an analytical LoD of less than 100 viral copies/mL, we suggest that the risk of false-negative results in nasopharyngeal swab specimens collected for asymptomatic patients for the cobas SARS-CoV-2 assay is low (<5%).

In a subanalysis of the data, we reviewed the relationship of CT values to patient age. Surprisingly, the mean CT value in 7/10 age groups for Target 1 and 8/10 age groups for Target 2 were within +/– 1 CT of the mean of all specimens. This suggests a similar viral load in specimens obtained from patients across the age spectrum. The 2 exceptions were age groups 10 to 19 years and 80 to 89 years. In both groups the mean CT values were up to 2 CT lower, which equates to approximately 0.6 log10 higher viral copies/mL in specimens obtained from persons in these age groups. Importantly, while arithmetically lower, this difference only reached statistical significance for Target 1 CT values in age group 80 to 89 years. Combined, these data suggest very little variation in viral load present in specimens obtained from individuals in different age groups, and consequently no significant different in test sensitivity is expected based on subject age.

Our study does have limitations. First, in confirming the LoD of the cobas SARS-CoV-2 assay we tested only 5 replicates of each dilution. This only powered us to differentiate differences in sensitivity of 20% between dilutions; however, a 40% to 60% decrease in replicate positivity was observed between the lowest concentration with 100% detection and the next lowest dilution. We used the Target 1 and Target 2 CT values reported for the final dilution demonstrating 100% detection as the LoD for our analysis, which would provide a conservative underestimate of the assay LoD. This LoD was also supported by linear regression analysis of replicate CT values, which demonstrated linearity through the lowest dilution with 100% detection. Second, we did not test specimens with an alternative molecular test to identify additional potentially low concentration specimens that were not positive by the cobas test. Based on the similar published LoDs for other laboratory-based SARS-CoV-2 assays,[4,6] as well as assay comparison studies,[1-6] we would not expect this approach to identify a significant number of positive specimens. Finally, our study included only combined nasopharyngeal-oropharyngeal specimens obtained from symptomatic individuals. Differences in sensitivity have been reported among various upper respiratory specimens including nasopharyngeal, oropharyngeal, midturbinate, anterior nares, and saliva.[15-17] Therefore, our results cannot be extrapolated to other specimen types. Further, several publications have demonstrated increased detection rates in lower respiratory specimens,[18,19] including instances of symptomatic patients with negative nasopharyngeal but positive bronchioalveolar lavage results.[20] Similarly, poor sensitivity has been reported for specimens collected prior to or very early during symptom onset.[9] We did not conduct a chart review to establish the time from onset of symptoms to specimen collection; therefore, our data and conclusions may not be applicable to individuals with fewer than 2 days of symptoms or in asymptomatic populations. Because of these limitations, our data are not intended to suggest that a negative RT-PCR result using the Roche cobas test rules out SARS-CoV-2 infection. Test results should be interpreted in the context of clinical presentation and other laboratory values.

Conclusion

In conclusion, we present data suggesting a low risk of false-negative results when testing nasopharyngeal specimens obtained from symptomatic patients when using the using the cobas SARS-CoV-2 EUA assay. We also find similar CT values, indicative of similar viral load, in patients across the spectrum of age, which implies similar test sensitivity independent of patient age. Finally, the use of a 2-target approach to detect SARS-CoV-2, inclusive of a pan-Sarbecovirus target, may increase total test sensitivity by approximately 3% due to increased detection for specimens containing a low concentration of viral RNA.

Table 1

Confirmation of RT-PCR Limit of Detection

	Target 1 (ORF1)			Target 2 (E-Gene)
Dilution	Replicates Detected	Average CT Value	Standard Deviation	Replicates Detected	Average CT Value	Standard Deviation
10–1	4/4 (100%)	25.38	0.148	4/4 (100%)	25.41	0.135
10–2	5/5 (100%)	28.43	0.122	5/5 (100%)	28.44	0.155
10–3	5/5 (100%)	31.43	0.386	5/5 (100%)	31.74	0.439
10–4	5/5 (100%)	34.17	0.168	5/5 (100%)	34.86	0.178
10–5	5/5 (100%)	35.95	0.273	5/5 (100%)	37.16	0.574
10–6	3/5 (60%)	36.99	1.372	2/5 (40%)	37.52	1.060
10–7	0/5 (0%)	—	—	0/5 (0%)	—	—

CT, cycle threshold; RT-PCR, reverse transcription polymerase chain reaction.