Evaluation of the analytical performance and specificity of a SARS-CoV-2 transcription-mediated amplification assay.

The ongoing severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic requires fast and accurate high-throughput diagnostic tools. To evaluate the analytical performance of the Hologic Aptima transcription-mediated amplification (TMA) assay for detection of SARS-CoV-2 RNA from respiratory samples we analysed 103 clinical and proficiency panel samples pre-tested by real-time RT-PCR (Altona, RealStar) and found a positive percent agreement (sensitivity) of 95.7% and a negative percent agreement (specificity) of 100%. The limit of detection of the Aptima test was 150 copies/ml determined as 95% detection probability. To further assess the Aptima assay´s specificity we prospectively analysed 7545 clinical specimens from the upper and lower respiratory tract sent for the purpose of routine SARS-CoV-2 screening. SARS-CoV-2 RNA was detected in 16/7545 (0.2%) samples by the TMA assay and confirmed independently by the Xpert SARS-CoV-2 RT-PCR (Cepheid); in one case a previous discrepant result was confirmed as true SARS-CoV-2 infection in a subsequent sample from the same patient. Results from the Aptima SARS-CoV-2 TMA assay agreed well with RT-PCR and showed an excellent specificity in a large number of routine specimens despite the low prevalence at that time of the pandemic, indicating that this assay can be used even for screening purposes.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has spread as global pandemic (Dong et al., 2020; Hu et al., 2020; Zhu et al., 2020). SARS-CoV-2 infected persons occasionally present oligo- or asymptomatically and high viral loads can be shed before clinical signs occur, posing the risk of uncontrolled transmission (Arons et al., 2020; Hung et al., 2020; Lai et al., 2020; Rivett et al., 2020). Therefore, fast and reliable detection of SARS-CoV-2, using highly sensitive and specific nucleic acid amplification testing (NAAT) for a large sample number has become a major challenge for laboratories to provide timely diagnosis and prevent (nosocomial) transmission especially in healthcare settings. The Aptima SARS-CoV-2 transcription-mediated amplification (TMA) assay on the automated, random access Panther system (Hologic) detects viral RNA with a turn-around-time of 3.5 h and then up to 60 results per hour.

Here we aim to describe validation data upon test introduction and to prospectively evaluate the clinical performance, especially specificity, of the Aptima SARS-CoV-2 assay for respiratory samples obtained mainly for routine screening at a large University Hospital in Southern Germany during the first eight weeks after test introduction in summer 2020.

Test performance of the Aptima SARS-CoV-2 TMA (Hologic) assay was first evaluated by retrospective analysis of 94 respiratory specimens (n = 86 nasopharyngeal swabs in ESwab™ (Copan) or Sigma Virocult® (Medical Wire & Equipment) media and n = 8 tracheal aspirations) and 9 proficiency panel specimens (INSTAND) initially tested using real-time RT-PCR (RealStar SARS-CoV-2 RT-PCR Kit 1.0, Altona) amplifying conserved regions of the viral E-gene and S-gene. Retrospective specimens had been archived at −80 °C and were pre-selected according to Ct (cycle threshold) values determined by the RealStar assay: n = 25 with a Ct value < 25 (> 4 × 105 copies/mL), n = 24 with a Ct value of 25–32, n = 20 with a Ct value > 32 (< 2 × 103 copies/mL), and n = 25 negative samples. To semi-quantitatively calculate viral copy numbers corresponding to the Ct values of the E-gene RT-PCR we used a dilution series of a standard sample (107 copies/mL, Ch07469-1-CoV-2, INSTAND). RealStar RT-PCR had been done after extraction of nucleic acids (MagNA Pure 96, Roche) on a Light Cycler 480 II (Roche) according to manufacturer’s instructions.

The 95 % Limit of Detection (LoD) of the Aptima assay was determined by testing serial dilution samples of the AccuPlex™ SARS-CoV-2 verification panel (SeraCare) containing recombinant virus particles with SARS-CoV-2 genome sequences (dilution in log steps starting at a concentration of 1 × 105 copies/mL) and subsequent Probit regression analysis (SPSS Statistics 27, IBM).

For prospective assessment of assay specificity 7545 fresh respiratory specimens (7427 nasopharyngeal swabs, 96 tracheal aspirations, and 22 bronchoalveolar lavage specimens) sent for routine SARS-CoV-2 testing from patients at the University Hospital between July and September 2020 were analysed with the Aptima SARS-CoV-2 TMA assay. Positive results were confirmed independently by the Xpert SARS-CoV-2 test (Cepheid) that also determines Ct values. The majority of specimens were sent for the purpose of screening of patients without respiratory symptoms upon admission to hospital.

The Aptima SARS-CoV-2 TMA assay (Hologic) and the cartridge based Xpert Xpress SARS-CoV-2 (Cepheid) real-time RT-PCR assay were done according to manufacturers’ instructions on the Hologic Panther instrument or the Cepheid GeneXpert instrument, respectively.

This study was approved by the institutional ethics review board of the University Hospital Tuebingen (no. 795/2020).

Upon retrospective evaluation of test performance using archived specimens, 66/69 samples that had been tested positive for SARS-CoV-2 RNA by the RealStar (Altona) RT-PCR were concordantly positive in the Aptima assay, i.e. a positive percent agreement (PPA, sensitivity) of 95.7 %. The three samples discordantly not detected by the Aptima assay had relatively low viral loads with Ct values (E-gene RealStar RT-PCR) of 32, 31 and 36, respectively, corresponding to virus concentrations around or below 2000 copies/mL, i. e. likely close to the LoD of the RealStar RT-PCR assay.

All 25 samples tested negative by the RealStar assay were concordantly tested negative by the Aptima assay (Table 1 ), i.e. a negative percent agreement (NPA, specificity) of 100 %. Testing of nine samples of a human coronavirus (HCoV) proficiency panel (INSTAND) resulted in 100 % concordance, including negative results for three samples positive for HCoV-229E, -NL63 or -OC43, respectively, as well as positive results for the four provided SARS-CoV-2 positive samples.

Table 1

Comparison of test performance of the Aptima SARS-CoV-2 TMA with the RealStar SARS-CoV-2 RT-PCR assay.

	RealStar SARS-CoV-2
Aptima SARS-CoV-2	Positive	Negative	Kappa (κ)a (±95 % CI)b	PPAc	NPAd
Positive	66	0	0.921 (1.000−0.834)	95.7	100
Negative	3	25

Cohen’s Kappa (κ) defines the overall agreement with values categorized as follows: >0.90, almost perfect; 0.90−0.80, strong; 0.79−0.60, moderate; 0.59−0.40, weak; 0.39−0.21, minimal; 0.20−0, none.

±, upper/lower 95 % confidence interval (CI).

Positive percent agreement (PPA).

Negative percent agreement (NPA).

Analytical sensitivity determined by replicate testing of the AccuPlex SARS-CoV-2 verification panel serial dilution samples in the Aptima assay yielded a 95 % LoD of 150 (confidence interval 125–201) copies/mL (Table 2 ).

Table 2

Determination of the 95 % limit of detection (LoD) of the Aptima SARS-CoV-2 assay.

SARS-CoV-2 Copies / ml	Positive/total number tested (n)	Positive rate (%)
1×104	9/9	100 %
1×103	9/9	100 %
1×102	17/27	63.0%
1×101	1/27	3.7%
1×10°	0/27	0%

During prospective evaluation of the TMA assay’s clinical performance and specificity, SARS-CoV-2 RNA was detected by the Aptima assay in 16/7545 (0.2 %) examined samples, which were mostly sent for screening purposes upon hospital admission. 15/16 positive results were confirmed by re-testing the samples using the Cepheid Xpert SARS-CoV-2 test (Table 3 ). The only exception was a sample from patient C that tested discordantly positive in the Aptima assay, but could initially not be confirmed by the Xpert assay. However, on the next day a subsequent sample of patient C tested positive in both assays (Table 3). For 10/15 patients with detectable SARS-CoV-2 RNA this was the first positive SARS-CoV-2 result likely indicating an acute infection with SARS-CoV-2. For these individuals with previously unknown infections we found significantly higher median viral loads upon the Xpert SARS-CoV-2 E-gene RT-PCR (p = 0.0007, median Ct value 24.6, range 21.0–33.6) compared to the remaining five positive patients with already known SARS-CoV-2 infection (median Ct value 37.2, range 34.9–41.3).

Table 3

Confirmation of prospective samples tested positive by the Aptima SARS-CoV-2 assay using the Xpert Xpress SARS-CoV-2 assay (Cepheid).

Patient	Material	Aptima SARS-CoV-2	Xpert Xpress SARS-CoV-2 (Ct values E-gene/N2-gene)	Previous SARS-CoV-2 detection?
A	NPSa	Positive	Positive (41.3/39.4)	Yes
B	NPS	Positive	Positive (31.1/33.6)	No
C first sample	NPS	Positive	Negative	No
C second sample	NPS	Positive	Positive (33.6/36.8)	No (Aptima only)
D	NPS	Positive	Positive (29.3/31.7)	No
E	NPS	Positive	Positive (34.9/34.6)	Yes
F	NPS	Positive	Positive (27.4/30.1)	No
G	NPS	Positive	Positive (21.7/23.7)	Yes
H	NPS	Positive	Positive (21.2/23.6)	No
I	NPS	Positive	Positive (37.2/39.6)	Yes
J	NPS	Positive	Positive (21.0/23.1)	No
K	NPS	Positive	Positive (38.8/38.8)	Yes
L	BALb	Positive	Positive (35.4/35.6)	Yes
M	NPS	Positive	Positive (27.9/30.8)	No
N	NPS	Positive	Positive (19.5/22.3)	No
O	NPS	Positive	Positive (20.5/23.2)	No

Nasopharyngeal swab (NPS).

Bronchoalveolar lavage specimen (BAL).

The validation data of clinical and proficiency panel samples demonstrate a very good test concordance of the Aptima SARS-CoV-2 TMA assay (Hologic) and the RealStar SARS-CoV-2 RT-PCR (Altona), with a PPA of 95.7 % and NPA of 100 %. Similar agreement values have been described for the Aptima SARS-CoV-2 TMA test with the Panther Fusion SARS-CoV-2, the BioFire Defense COVID-19 or other RT-PCR assays (Cordes et al., 2020; Gorzalski et al., 2020; Kuo et al., 2021; Smith et al., 2020; Trémeaux et al., 2020). The three samples with discordant results (positive in the RealStar but negative in the Aptima assay) all had high Ct values corresponding to low viral loads: for two samples virus concentrations clearly below the LoD of the Aptima assay can be assumed, while the third sample which originated from a health-care worker without hospital admission had a viral load of approximately 3800 copies/mL (Real-Star E-Gene RT-PCR). Unfortunately, no data on the clinical course was available and no specimen was left for re-testing and final clarification if this sample was missed (i.e. false-negative) by the Aptima or rather false-positive in the RealStar assay. Decreasing RNA quality of archived samples as well as differences in the amount of processed input volume for different assays occasionally may lead to non-reproducible results in particular in samples with low viral loads and can be considered as general limitations of these kind of retrospective evaluations (Kuo et al., 2021; Mostafa et al., 2020).

With a LoD of 150 copies/mL the 95 % detection probability of the Aptima assay can be considered as sufficiently sensitive. This is consistent with data from others, who found a 100 % LoD of 62.5 copies/mL (synthetic RNA (Smith et al., 2020)) and a 95 % LoD of 288 copies/mL (AccuPlex SARS-CoV-2 Verification Panel (Cordes et al., 2020)).

Prospective analysis of the clinical performance in 7454 samples from both upper and lower respiratory tract sent over an eight-week period during summer 2020 revealed an excellent specificity of the Aptima SARS-CoV-2 assay. With one exemption all positive samples could be confirmed by the Xpert SARS-CoV-2 RT-PCR (Cepheid). For the only one “false positive” sample in the Aptima test, SARS-CoV-2 infection has been diagnosed by the confirmatory assay in a subsequent specimen of the same patient on the next day.

This excellent specificity is remarkable for two reasons: First, prevalence of SARS-CoV-2 infections in Germany during that time of the pandemic was very low. Second, according to the hospital’s test strategy many samples were sent for the purpose of screening of asymptomatic individuals. This is also resembled by the SARS-CoV-2 RNA positivity rate of 0.20 % in our cohort, which was even lower than the test positivity rate in Germany (0.81–1.16 %) during that time (Böttcher et al., 2020). A recently published comparable study analysed more than 19.000 samples and found a rather similar SARS-CoV-2 positivity rate of 0.47 % (Skittrall et al., 2021). SARS-CoV-2 RNA was detected in 43 samples for the first time (i.e. no previous result available) by the Aptima TMA assay, but only 29 out of these 43 (67 %) could by confirmed by Skittrall and colleagues upon re-testing with 1 or 2 other assays (Aptima TMA or an in-house RT-PCR). In contrast, 15/16 (94 %) positive results were confirmed in our study using the quite sensitive Xpert assay. We thus share the conclusion of Skittrall et al., that a confirmatory, ideally semi-quantitative test could be useful in certain settings.

The major advantages of the Aptima SARS-CoV-2 TMA assay are an excellent test performance, good turn-around-time and high automation including random access. In particular, the last point turned out to be a big plus for the daily laboratory work-flow during the pandemic, as it allowed continuous and independent processing of incoming samples, which is not possible for RT-PCR assays run in a batch format. One drawback of the Aptima assay however, is the lack of information on viral loads in the respiratory samples. Ct values determined by quantitative NAATs can be useful for evaluation if an acute, rather beginning or past infection has been detected, which is known from other infections (Baier et al., 2018; Banerjee et al., 2018) and has been demonstrated in our cohort by the significantly lower median Ct values for patients with first detection compared to known infections. Moreover, assessment of SARS-CoV-2 loads by Ct values becomes increasingly important for patient management in terms of lifting quarantine measures or assessing potential infectivity (Cevik et al., 2021; Perera et al., 2020; Robert-Koch-Institut, 30.11.2020; Wolfel et al., 2020). The problem of missing Ct values of course can be solved by subsequent re-testing positive samples using a SARS-CoV-2 real-time RT-PCR assay; however, this increases costs, hands-on and sample-to-answer time.

In summary, the Aptima SARS-CoV-2 TMA assay reliably detected SARS-CoV-2 infections and exhibited an excellent specificity in a large number of prospectively tested routine specimens despite low SARS-CoV-2 prevalence. Therefore, the assay can - if required- even be utilized for screening purposes of asymptomatic individuals, e.g. upon hospital admission.

CRediT authorship contribution statement

Markus Schneider: Data curation, Investigation, Formal analysis, Visualization, Writing - original draft. Thomas Iftner: Conceptualization, Funding acquisition, Writing - review & editing. Tina Ganzenmueller: Conceptualization, Formal analysis, Visualization, Supervision, Writing - review & editing.

Declaration of Competing Interest

The University Hospital Tuebingen (Thomas Iftner) received an unconditional research grant from Hologic for a longitudinal study on a comparison of HPV tests for cervical cancer screening in the past. The other authors report no conflict of interest. Hologic sponsored the first 250 tests of the evaluation study; otherwise the study was financed by institutional funds of the Virology Department of the University Hospital Tuebingen. Hologic did not have any influence on the scientific data interpretation or content of the submitted manuscript.