Performance Evaluation of the Aptima Assays in Comparison with the cobas 6800 Assays for the Detection of HIV-1, HBV, and HCV in Clinical Samples.

BACKGROUND: Accurate and consistent viral load (VL) quantitation of HIV type 1 (HIV-1), hepatitis B virus (HBV), and hepatitis C virus (HCV) is important for diagnosis and clinical monitoring. Assay results have to be concordant and compatible across laboratories. We evaluated the performance of three Aptima assays (Hologic, San Diego, CA, USA) and compared their VL values with corresponding cobas 6800 assay (Roche Diagnostics, Mannheim, Germany) results, using 840 clinical samples.
METHODS: The correlation between VL results obtained using the two assays was evaluated in terms of analytical sensitivity, precision/reproducibility, linearity, and cross-reactivity. Agreement rates were determined using kappa statistics. The overall agreement of VL values was examined using Passing-Bablok regression analysis.
RESULTS: All CVs were within 5%; the assays had good precision for detecting all three viruses. The linearity of quantitation assessed using three AccuSpan linearity panels (Seracare, Milford, MA, USA), was excellent for the Aptima assays. For HIV-1 and HCV, the results of both assays showed excellent agreement (κ=0.89 and 0.90, respectively) while for HBV, the results showed good agreement (κ=0.69). For analytical sensitivity, the VLs required for a 100% detection rate of HIV-1, HBV, and HCV were 20 copies/mL, 7.5 IU/mL, and 5.0 IU/mL, respectively. The results for HIV-1, HBV, and HCV obtained using both assays correlated strongly (R2=0.97, 0.93, and 0.95, respectively).
CONCLUSIONS: The cobas 6800 and Aptima assays, with fully automated and high-throughput molecular platforms for HIV-1, HBV, and HCV VL measurements, show good analytical performance and a strong correlation between results. The study results suggest that the assays can be used interchangeably for long-term monitoring of chronic infections.

INTRODUCTION

Viral load (VL) quantitation of HIV type 1, hepatitis B virus (HBV), and hepatitis C virus (HCV) is a part of the standard protocol for diagnosis, clinical monitoring, and treatment management [1-8]. In patients infected with HIV-1, the VL should be measured at diagnosis and at the initiation of antiretroviral therapy, and regularly between 6 and 12 months thereafter. Treatment success can be judged based on a VL remaining <50 copies/mL [9-11]. For HBV infections, several guidelines recommend monitoring the VL using highly sensitive nucleic acid amplification technologies every three months until the HBV DNA is undetectable, and every three to six months thereafter to detect persistent viremia and virological breakthrough [3, 7]. HCV RNA detection and VL quantification should be performed using a sensitive assay with a lower limit of detection (LOD) of 15 IU/mL. The endpoint of therapy is undetectable HCV RNA for 12–24 weeks after the end of treatment [5].

The assays used for monitoring VLs of HIV-1, HBV, or HCV must have a high sensitivity and accuracy. Infections with these viruses are chronic and require long-term treatment and monitoring, during which VL measurements may be performed at different laboratories or using different reagents and analyzers within the same laboratory. Providing results that are consistent, concordant, and interchangeable across laboratories and assays is necessary [11-13]. Various assays for the quantitation of HIV-1, HBV, and HCV in plasma or serum are commercially available, but reportedly have variable performance [9, 14]. Most of these assays use quantitative real-time PCR technology. The recently developed Aptima assay (Hologic, San Diego, CA, USA) involves three steps: target capture, target amplification via transcription-mediated amplification (TMA), and real-time detection of the amplicons using fluorescent probes; all these processes are automated and performed in a single tube on the Panther platform (Hologic). Previous comparative performance studies have examined quantitative real-time PCR technology-based assays against the Aptima assay [9, 11, 14-30]. However, few studies have compared the assays available for the fully automated, high-throughput molecular platforms, Roche cobas 6800 (Roche Diagnostics, Mannheim, Germany) and Panther [9, 11, 31, 32]. We evaluated the interchangeability of assay results for long-term monitoring of chronic infections to compare the performance of the Aptima and cobas 6800 assays, using samples sufficient for the determination of HIV-1, HBV, and HCV VL values. This is the first study to analyze more than 100 positive samples representing a sufficient analytical range for HIV-1, HBV, and HCV detection commonly used in clinical laboratories.

MATERIALS AND METHODS

Samples

Blood samples were collected from patients with HIV-1, HBV, or HCV infection. Plasma was obtained from these samples treated with EDTA to prevent coagulation. In total, 301, 300, and 239 plasma samples assayed using cobas 6800 between June and August 2020 were retested using the HIV-1, HBV, and HCV Aptima assays, respectively. The samples were stored at –70°C for up to four months before use in the Aptima assays. The research involving human samples complied with all the relevant national regulations, institutional policies, and the tenets of the Helsinki Declaration (as revised in 2013), and was approved by the Institutional Review Board of the Severance Hospital, Seoul, Korea (IRB No.: 1-2020-0019). The requirement for informed consent was waived due to the retrospective nature of this study, provided that all patient data were anonymized and that the study involved samples already available rather than prospectively collected.

Assay characteristics and evaluation

The characteristics of each assay are presented in Table 1. All the instruments and assays were used by trained operators in accordance with the manufacturers’ instructions.

Table 1

Characteristics of the Panther and cobas 6800 automated molecular platforms

		Panther*	Cobas 6800[†]
Principle		Transcription-mediated amplification	Real-time quantitative PCR
Assay type		Fully automated, random-access system	Fully automated, batched system
Targets	HIV-1	Pol, LTR regions	Gag gene and LTR region
	HBV	Pol, surface (S) gene	Pol, precore region
	HCV	5´ UTR	5´ UTR
Sample type	HIV-1	Plasma, DBS	Plasma, PSC
	HBV	Plasma, serum	Plasma, serum
	HCV	Plasma, serum	Plasma, serum
Sample volume		1200 μL (primary tube), 700 μL (secondary tube)	650 μL
Sample processing volume		500 μL	500 μL
Limit of detection	HIV-1	12 copies/mL	13.2 copies/mL
	HBV	4.8 IU/mL (plasma), 5.9 IU/mL (serum)	2.7 IU/mL (plasma), 2.4 IU/mL (serum)
	HCV	3.9 IU/mL (plasma), 3.4 IU/mL (serum)	8.5 IU/mL (plasma), 9.6 IU/mL (serum)
Linear assay range	HIV-1	30–1 × 106 copies/mL	20–1 × 107 copies/mL
	HBV	10–1 × 109 IU/mL	10–1 × 109 IU/mL
	HCV	10–1 × 108 IU/mL	15–1 × 108 IU/mL
Time to results from start		~ 210 minutes for five results	~ 180 minutes for 96 results
Result timeline		five results every 5 minutes	93-minute average time between plates

*The Panther platform was used for the Aptima assays; †The Cobas 6800 platform was used for the cobas 6800 assays.

Abbreviations: Pol, polymerase; LTR, long terminal repeat; HIV, Human immunodeficiency virus; HBV, hepatitis B virus; HCV, hepatitis C virus; 5´ UTR, 5´ untranslated region; DBS, dried blood spot; PSC, plasma separation card.

The imprecision was evaluated according to CLSI guideline EP15-A3, and the inter- and intra-assay precision was determined [33]. Low- and high-concentration samples of positive control materials provided by each manufacturer were used. The control materials were measured in two replicates twice daily for 10 days to assess inter-assay precision. For intra-assay precision assessment, 20 replicates of high- and low-concentration positive control materials were used.

The linearity was evaluated according to CLSI guideline EP06-A, using the AccuSpan HIV-1 RNA Linearity Panel (Seracare, Milford, MA, USA; 2410-0221/Batch #01383902) with a concentration range of 2.10–8.35 log copies/mL, the AccuSpan HBV DNA Linearity Panel (PHD802 [2410-0162]/Batch #10439611) with a concentration range of 1.74–8.40 log IU/mL, and the AccuSpan HCV RNA Linearity Panel (PHW805 [2410-0166]/Batch #10338244) with a concentration range of 1.26–7.34 log IU/mL [34]. Each panel item was measured twice, and the mean values obtained were compared with the expected values indicated in the panel datasheets. Linear-fit and second- and third-order polynomial regression analyses were performed. A regression equation was obtained using the observed and expected VL values as dependent and independent variables, respectively, from the best-fitting model.

The clinical samples from the patients with HIV-1, HBV, or HCV infection requiring VL monitoring were assayed with the routinely used cobas 6800 assays in parallel with the Aptima assays of the Panther platform. The assay results were categorized as “Target not detected” (TND), “below the lower limit of quantitation” (35]. For HCV genotyping, the Roche cobas HCV GT kit was used on the Roche cobas 4800 automated platform. HBV genotyping was not performed because HBV genotype C prevails predominantly among chronic carriers of the virus in Korea, irrespective of their clinical stages of liver disease and geographic origin.

To assess the analytical sensitivity of the Aptima assays, the clinical samples were diluted with virus-negative plasma to obtain target concentrations of 20, 30, and 40 copies/mL for HIV-1, and of 5, 7.5, 10, and 15 IU/mL for HBV and HCV, based on the cobas 6800 results. Twenty replicates of each dilution were assayed within the same run. The lowest concentrations yielding a 100% detection rate and 100% quantitation rate were compared with the LOD and LLOQ reported by the manufacturer.

The potential cross-reactivity with other viruses was assessed using two independent samples with detected HIV-1, HBV, HCV, cytomegalovirus, Epstein–Barr virus, BK virus, and herpes simplex virus, but not the virus to be measured by the assay.

Statistical analysis

All statistical analyses were performed using the Analyse-it software, version 4.65 (Analyse-it Software, Leeds, UK). The agreement rates between the two assays were determined using the kappa statistics with 95% confidence intervals (CIs) (with κ values of 0.001–0.2, 0.201–0.4, 0.401–0.6, 0.601–0.8, and 0.801–0.999 representing slight, fair, moderate, substantial, and excellent agreement, respectively). The overall correlation of VL values for samples quantitated with both assays was examined pairwise using Passing–Bablok regression analysis, which included the calculation of a correlation coefficient (R2), as well as Bland–Altman analysis, which included the calculation of the mean difference and 95% CI of the assay agreement. HCV values stratified according to the genotype were examined using ANOVA. P<0.05 was considered statistically significant.

RESULTS

Precision

The coefficients of variation (standard deviation as a percentage of the mean) for all assays are summarized in Table 2. All % CVs were within 5%, indicating good assay precision, for all three viruses. For the Aptima assays, the inter-assay % CV values were slightly higher than the corresponding intra-assay values.

Table 2

Intra- and inter-assay precision of the Aptima and cobas 6800 assays

	Aptima		cobas 6800
	Intra-assay	Inter-assay	Intra-assay	Inter-assay
Low-concentration positive control
HIV-1
Mean (log copies/mL)	2.91	2.91	2.38	2.35
SD (log copies/mL)	0.06	0.09	0.07	0.09
CV (%)	1.90	3.10	2.80	3.90
HBV
Mean (log IU/mL)	2.77	2.77	1.75	1.64
SD (log IU/mL)	0.04	0.06	0.08	0.07
CV (%)	1.35	2.21	4.71	4.32
HCV
Mean (log IU/mL)	2.32	2.32	2.93	2.96
SD (log IU/mL)	0.06	0.06	0.05	0.04
CV (%)	2.58	2.58	1.69	1.48
High-concentration positive control
HIV-1
Mean (log copies/mL)	5.20	5.20	5.51	5.51
SD (log copies/mL)	0.04	0.06	0.06	0.07
CV (%)	0.80	1.10	1.18	1.29
HBV
Mean (log IU/mL)	4.50	4.50	5.34	5.27
SD (log IU/mL)	0.04	0.05	0.08	0.11
CV (%)	0.87	1.11	1.43	2.11
HCV
Mean (log IU/mL)	5.41	5.41	7.01	7.03
SD (log IU/mL)	0.07	0.10	0.04	0.04
CV (%)	1.33	1.76	0.61	0.52

For the cobas 6800 assays, the % CV values for HIV-1 and HBV of the low-concentration positive control material were high when compared with those for HCV of low- and high-concentration positive control materials of the three viruses. For both low- and high-concentration positive control materials, the Aptima assays showed lower % CV values in the HIV-1 and HBV assays than did cobas 6800, whereas the cobas 6800 HCV assay yielded a lower % CV than the Aptima assays.

Linearity

The linearity of quantitation as assessed using the AccuSpan linearity panels was excellent for the Aptima assays (R2=0.995–0.996) (Fig. 1).

Fig. 1

Aptima results obtained with the AccuSpan Linearity Panels. The values are presented as log copies/mL for HIV-1 (A) and log IU/mL for HBV (B) and HCV (C) (N=2 per target concentration level).

Comparison of the Aptima and cobas 6800 assays using clinical samples

For HIV-1, the between-assay agreement was 94.68% (285/301) (κ=0.89; 95% CI, 0.83–0.94) (Table 3). Two samples were detected as quantitated (with 119 and 193 copies/mL, respectively) by the cobas 6800 assay and as TND by the Aptima assay. Three samples were detected as quantitated (VL=100, 110, and 133 copies/mL, respectively) by the cobas 6800 assay, whereas they were detected as

Table 3

Agreement between the results obtained using the Aptima and cobas 6800 assays in terms of detection and quantitation of HIV-1, HBV, and HCV in clinical plasma samples

	cobas 6800	Aptima
		TND	< LLOQ	Quantitated	Total
HIV-1	TND	89	11	0	100
	< LLOQ	0	0	0	0
	Quantitated	2	3	196	201
	Total	91	14	196	301
HBV	TND	71	28	1	100
	< LLOQ	0	0	0	0
	Quantitated	1	17	182	200
	Total	72	45	183	300
HCV	TND	98	1	2	101
	< LLOQ	1	3	2	6
	Quantitated	0	6	126	132
	Total	99	10	130	239

Abbreviations: TND, target not detected;

Fig. 2

Method comparison for HIV-1 (N=196), HBV (N=182), and HCV (N=126) VL assessments. (A) Passing–Bablok regression results for 196 quantitated HIV-1 clinical samples that were assayed with the Aptima and cobas 6800 assays. (B) Bland–Altman plot of the difference between the Aptima and cobas 6800 HIV results vs. the mean. The mean bias was –0.27 log copies/mL, with 95% CIs ranging from –0.84 to 0.29 log copies/mL. (C) Passing–Bablok regression for 182 quantitated HBV clinical samples that were assayed with both the Aptima and cobas 6800 assays. (D) Bland–Altman plot of the differences between the Aptima and cobas 6800 HBV results vs. the mean. The mean bias was –0.02 log copies/mL, with 95% CIs ranging from –0.75 to 0.71 log copies/mL. (E) Passing–Bablok regression results for 126 quantitated HCV clinical samples that were assayed with the Aptima and cobas 6800 assays. (F) Bland–Altman plot of the differences between the Aptima and cobas 6800 HCV results vs. the mean. The mean bias was –0.14 log copies/mL, with 95% CIs ranging from –1.00 to 0.72 log copies/mL.

Abbreviation: CI, confidence interval.

There were 49 patients with VL values of 50–200 copies/mL in at least one assay (Supplemental Data Table S1). The agreement rate for samples with low VL values (38%) was lower than that for the total samples (94.68%). The detection rates of anti-retroviral resistance mutations were similar among the VL groups, and no difference in VL values was observed between the detection (N=19) and non-detection groups (N=13) (for cobas 6800, the means±SDs for the detection and non-detection groups were 206.37±101.17 and 241.62±151.90 [P=0.44], respectively; the corresponding values in the Aptima assays were 116.79±169.58 and 110.54±50.37 [P=0.90], respectively). In the low VL value range, we observed no association between treatment resistance mutations and increasing VL values; the presence of mutations associated with antiretroviral drug resistance may have been due to repeatedly high VL values (Supplemental Data Table S2).

The HBV agreement rate between the assays was 84.33% (253/300) (κ=0.69; 95% CI, 0.63–0.76). One sample detected as quantitated by the cobas 6800 assay was detected as TND by the Aptima assay, and one sample detected as quantitated by the Aptima assay was detected as TND by the cobas 6800 assay. Both samples had low VL values (17 and 26 IU/mL). Seventeen samples quantitated by the cobas 6800 assay and detected as

The HCV agreement rate between the assays was 94.98% (227/239) (κ=0.90; 95% CI, 0.85–0.96). Four samples quantitated with the Aptima assay yielded VL values near the LLOQ value (19, 16, 16, and 10 IU/mL). The cobas 6800 assay yielded two samples as TND and two as

Analytical sensitivity

The overall detection rates of the Aptima assays for the three viruses were similar (Table 4). The VLs yielding a 100% detection rate of HIV-1, HBV, and HCV were 20 copies/mL, 7.5 IU/mL, and 5 IU/mL, respectively. Although the HIV-1 assay quantitated 100% of the replicates at 30 copies/mL (a value similar to the LLOQ reported by the manufacturer), the HBV and HCV assays quantitated 50% and 85% of the replicates at 10 IU/mL, respectively.

Table 4

Analytical sensitivity of the Aptima assays assessed using clinical samples at different target concentrations

	Concentration of target	Detected N (%)	Quantitated N (%)
HIV-1 (copies/mL)	20	20 (100)	11 (55)
	30	20 (100)	20 (100)
	40	20 (100)	20 (100)
HBV (IU/mL)	5	19 (95)	4 (20)
	7.5	20 (100)	3 (15)
	10	20 (100)	10 (50)
	15	20 (100)	14 (70)
HCV (IU/mL)	5	20 (100)	10 (50)
	7.5	20 (100)	13 (65)
	10	20 (100)	17 (85)
	15	20 (100)	19 (95)

Cross-reactivity

None of the viruses examined were detected by the Aptima assays. There was no evidence of cross-reactivity with other clinically important viruses.

DISCUSSION

The consistency of the assay results obtained from different laboratories and with different assays is paramount to the effective management of patients with chronic viral infections that require long-term treatment and monitoring [11-13]. The results produced by the cobas 6800 and Aptima assays for HIV-1 and HCV showed excellent agreement, and we found substantial concurrence between the assays for HBV, consistent with the results of previous studies [9, 11, 31, 32]. Overall, the results obtained with the cobas 6800 assays were slightly higher than those obtained with the Aptima assays, and the mean bias values for the three viruses were <–0.3 log.

Similarly, the results produced by the cobas 6800 and Aptima assays showed excellent agreement for the detection/non-detection of HIV-1 RNA in clinical samples (κ=0.89), and there was a good agreement between values obtained in samples quantitated by both assays (mean bias of –0.27 log). Five HIV-1 samples were detected as quantitated by the cobas 6800 assay and as TND or 11]. Blips are not associated with treatment failure and are generally considered an artifact caused by random assay variation rather than clinically significant elevations in viremia [11].

Current therapy guidelines for HIV-1 define virologic failure as a VL repeatedly exceeding 200 copies/mL [1, 6, 10]. The present cases were not associated with treatment failure or a need to change treatment. Patients with VL values of 50–200 copies/mL may continue with the same treatment with more frequent monitoring; there is no consensus on how to manage patients with VL values in this range [1, 6]. The risk of emerging treatment resistance is likely low [1, 6, 10]. In such cases, confirming that VL values remain above the lower LOD and assessing treatment adherence and possible drug–drug or drug–food interactions are recommended.

The results produced by the cobas 6800 and Aptima assays showed good agreement for the detection/non-detection of HBV DNA in clinical samples (κ=0.69) and the values obtained from samples quantitated by both assays (mean bias of –0.02 log). Samples detected as 32]. The VL values of the other samples were near the LLOQ (median VL of 16.10 IU/mL), and most were within the % CV of inter-assay precision.

Current guidelines for the monitoring of HCV-positive patients undergoing treatment recommend the use of a highly sensitive quantitative real-time HCV RNA assay with a LOD of 15 IU/mL and a LLOQ of 25 IU/mL; both the cobas 6800 and Aptima assays comply with these requirements [4, 5, 7]. Both assays had an excellent agreement for the detection/non-detection of HCV RNA in clinical samples (κ=0.90) and a good agreement of values in the samples quantitated by both assays (mean bias of –0.14 log). VL values of HCV genotype 3 have been associated with a high rate of under-reporting and high variance between assays [9, 14, 26]. The HCV genotypes detected were 1a (N=1), 1b (N=35), and 2 (N=44); there was no difference or significant variation in VL values between patients with genotypes 1b and 2 (Supplemental Data Figure S1).

In the quantitative comparison of the cobas 6800 and Aptima assays, several samples detected as

In the analytical sensitivity assessment, the lowest concentrations required for a 100% detection rate were 20 copies/mL, 7.5 IU/mL, and 5.0 IU/mL for HIV-1, HBV, and HCV, respectively, which are higher than the LODs reported by the manufacturer of the Aptima assays. Accurate assessment of the LOD requires precise dilutions close to the LOD concentration. For HBV and HCV, the quantitation rates were 50% and 85%, respectively, at the LLOQ level. The samples used for analytical sensitivity assessment were selected based on the cobas 6800 results, and upon comparison, the VL values of the Aptima assays were lower than those of cobas 6800, which we suspect to have influenced the low quantitation rate.

This study had some limitations. As the cobas 6800 system is routinely used in our laboratory, we collected the samples based on the assay results produced by this system, which may have affected the comparisons. The linearity and cross-reactivity were evaluated using the Aptima assays. HCV genotypes 3, 4, 5, and 6 were not detected. Future studies should compare the cobas 6800 and Aptima assays using samples assayed in parallel and assess assay sensitivity in HCV detection using samples with a wider range of genotypes than those detected in this study.

In summary, both the cobas 6800 and Aptima assays show good analytical performance for the determination of HIV-1, HBV, and HCV VL values. The results of this study, including the estimates of the correlation and agreement between the assay results obtained with these assays, suggest that the two assays can be used interchangeably for the long-term monitoring of chronic infections. Clinical laboratories should base their choice of assay system on factors such as the assay principle and type and the timeline for obtaining results.