Comparison of the Cobas 4800 HPV test and the Seeplex HPV4A ACE with the hybrid capture 2 test.

BACKGROUND: It is well-known that persistent cervical infections with high-risk human papillomavirus (HPV) are related to the development of high-grade cervical intraepithelial neoplasia and invasive cervical cancer and that infection with HPV 16 and HPV 18 accounts for approximately 70% of all cases of invasive cervical cancer.
METHODS: We performed 3 HPV molecular tests-the Cobas 4800 HPV test, the Seeplex HPV4A ACE, and the hybrid capture 2 (HC2) test-in 146 cervical swab samples to compare between these three tests.
RESULTS: There was a concordance rate of 82.8% between the results of the Cobas 4800 HPV and the HC2 test and a concordance rate of 84.9% between the results of the Seeplex HPV4A ACE and the HC2 test. Between the Cobas 4800 HPV test and the Seeplex HPV4A ACE, there was a concordance rate of 89.6% in the detection of high-risk HPV between the results and a concordance rate of 98.7% in the detection of HPV 16 or 18. When an abnormal Pap test was defined as ≥ low grade squamous intraepithelial lesion (LSIL), the sensitivity of the Cobas 4800 HPV test, the Seeplex HPV4A ACE and the HC2 test were 71.1%, 80.0%, and 88.9%, respectively, while their specificities were 76.4%, 74.5%, and 67.9%, respectively.
CONCLUSIONS: The results of this study suggest that the Cobas 4800 HPV test and the Seeplex HPV4A ACE may be as effective as the HC2 test in detecting HR HPV and that the concordance between the results of the Cobas 4800 HPV test and the Seeplex HDV4A ACE may be higher in the detection of HPV 16 and HPV18 than concerning high-risk HPV.

Introduction

Cervical cancer is one of the most common malignancies among women worldwide, particularly in developing countries 1, 2. It is believed that human papillomavirus (HPV) infection, which is detected in nearly 90% of all invasive cervical cancer tissues, is the most common risk factor for cervical cancer 2-4. More than 60 different HPV genotypes have been found in the human genital tract, being categorized as high, intermediate, and low-risk genotypes depending on their oncogenic potential 1, 5. High-risk HPV genotypes have been shown to increase the risk of precancerous cervical lesions and trigger the progression of these lesions to carcinomas 5. At least 18 high-risk HPV genotypes have been identified in humans, with common genotypes being HPV16 and HPV18 6. HPV16 is the most prevalent HPV genotype occurring in 68.1% of all carcinoma cases. This genotype has been reported to be the second most common genotype worldwide accounting for 15% of all cervical cancers 1. In women in their 30s, HPV infections are usually transient, but persistent high-risk HPV infection is a precursor of progression to cervical intraepithelial neoplasia (CIN) and finally cervical cancer 7, 8.

The Papanicolau (Pap) screening test has been successful in significantly reducing the incidence of cervical cancer and associated mortality by 75% 4. However, the Pap test has some limitations, such as inter-operator variably, low sensitivity and low reproducibility 2. Therefore, to overcome these limitations, HPV tests have been introduced as an adjuvant method. The hybrid capture 2 (HC2) test is a reliable signal-amplified hybridization assay which has been approved by the Food and Drug Administration (FDA) 1, 2, 9. However, HC2 can only determine whether an HPV infection is present or not; it cannot determine the genotype or distinguish between single and multiple infections 10, 11.

The Cobas 4800 HPV test and the Seeplex HPV4A ACE have widely been used at laboratories. The Cobas 4800 HPV test performs DNA extraction, PCR amplification and real-time detection in an automated fashion. It can rapidly distinguish HPV 16/18 from other types. Furthermore, it was approved for diagnostic use by the FDA in 2011 7. The Seeplex HPV4A ACE is multiplex PCR. It can also distinguish HPV16/18 and 16 other HR HPVs and 2 low-risk (LR) HPVs from other types.

Therefore, the aim of this study was to evaluate the HPV prevalence as assessed by Cobas 4800 HPV test, HC2 test and Seeplex HPV4A ACE test, respectively, for detecting HPV DNA using cervical swab specimens obtained from Korean women for cytological examination.

Materials and Methods

Samples Cervical swab specimens, which were obtained for screening test of cervical cancers, were collected from a total 146 women at Seoul St. Mary's Hospital between January and December of 2011. All specimens were collected, placed in a ThinPrep PreservCyt solution (Hologic Inc., Malborough, MA, USA) and a SurePath solution (BD, Franklin Lakes, NJ), and were then stored at -70°C until assayed for HPV tests. All specimens were classified into 4 groups according to their cytologic results: negative (n=62), ASCUS (n=39), LSIL (n=35) and HSIL (n=10).

HC2 test HPV DNA testing by the HC2 test was performed with an HC2 assay system according to the manufacturer's protocol (Qiagen Inc.,Valencia, CA, USA). The specimens were denatured at 65°C for 45 minutes and hybridized under high-stringency conditions with a mixture of RNA probes that detect 13 different oncogenic HPV types: 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, and 68. The resultant DNA-RNA hybrids were captured on the surface of the microtiter plate wells coated with anti-DNA-RNA hybrid antibody. The immobilized hybrids were then reacted with alkaline phosphatase-conjugated antihybrid monoclonal antibody. Light intensity was measured with a luminometer. The recommended positivity threshold of 1pg/mL was used as a cutoff, and all specimens with a relative light unit/control (RLU/CO) ratio of ≥1.0 were considered positive.

Cobas 4800 HPV test The Cobas 4800 HPV test features fully automated sample preparation combined with real-time PCR technology plus software that integrates the 2 components. The test is designed to extract, amplify and detect a broad spectrum of HR HPV genotypes as well as to co-amplify the human cellular globin gene. In this study, PCR amplification and detection occurred in a single tube, where probes with 4 different reporter dyes tracked different targets in the multiplex reaction (HPV-31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66 and 68); dyes 2 and 3 tracked HPVs 16 and 18, while dye 4 targeted β-globin to provide a control for cell adequacy, extraction and amplification.

Seeplex HPV4A ACE The Seeplex HPV4A ACE is designed for the detection of HPV 16 and/or HPV18 and screen for 16 HPV HR types (26, 31, 33, 35, 39, 45, 51, 52, 53, 56, 58, 59, 66, 68, 73 and 82) and HPV low-risk types 6 and 11. Residual cells in the ThinPrep PreservCyt and SurePath solution were centrifuged at 13,000 ×g for 15 minutes, and the cell pellets were resuspended in 100μL of phosphate-buffered saline. The resuspended cell pellets were added to a Qiamp DNA mini kit (Cat. No.51304, Qiagen Inc.). Total DNA was extracted according to the manufacturer's protocol (Seegen, Korea) and 8-methocypsorlaren (MOP), which prevents from contaminating DNA for amplification in an Applied Biosystem 9700 thermal cycler (Perkin -Elmer, Boston, MA, USA). Amplication was undertaken as follows. After a pre-heating step at 94°C for 15 minutes, 40 amplification cycles were carried out in the thermal cycler: denaturation at 94°C for 30 seconds annealing at 60°C for 90 seconds and extension at 72 °C for 90 seconds. Amplification was completed with a final extension step at 72°C for 10 minutes. The amplified PCR products were separated on the LabChip DX Seeplex assay system (Caliper, Hopkinton, MA, USA), an auto capillary electrophoresis system. The PCR products from each sample (20 μL) were transferred to a 96-well plate and placed in the LabChip DX instrument. The samples were loaded automatically on the Seeplex chip and processed sequentially according to the sample order. Anaysis was performed with the designated software (Seegen viewer) which presents each of the samples and identifies the fragments that yield a positive readout for the bands of interest in the presented results. A tabulated matching matrix provides a simple readout, pairing identified bands with the corresponding types of HPV. A text file is automatically generated and saved for each tape.

Statistical analysis

All statistical analyses were carried out using Analyse-it Method Evaluation Edition, version 2.22, software (Analyse-it software Ltd., Leeds, UK). The chi-square test was performed to compare the proportions of positive samples among the groups as classified according to the cytologic results. Concordance rates and kappa coefficient (κ) with 95% confidence intervals (CIs) were calculated to estimate the concordance between the results of different tests. The sensitivities, specificities, 95% CIs of the HC2 test, the Cobas 4800 and the Seeplex HPV4A ACE were calculated based on the results of cytological abnormalities. A p value of less than 0.05 was considered significant.

The study protocol was approved by the Institutional Review Board of the Catholic university of Korea, Seoul St. Mary's hospital (No. KC12SISI0623).

Results

Table 1 shows the positivity rates of each DNA test according to cytological results. All 3 tests revealed that positivity rates were higher for higher cytological grades than for lower grades (p < 0.0001) except in the cases of HPV 18 detected by the Cobas 4800 and the Seeplex HPV4A ACE. In all 146 samples, the overall positivity rates ranged from 36.99% to 41.10% in the Cobas 4800 and the Seeplex HPV4A ACE and 48.63% in the HC2 test. Table 2 shows the concordance rates between the HC2 test and the Cobas 4800 /the Seeplex HPV4A ACE. Regardless of HPV genotypes, the results of the 3 HPV detection tests agreed in 115 (78%) of all samples. Discrepant results between the 3 tests are shown in Table 2. Of the total samples, 8 (5%) were positive on the HC2 and Cobas 4800 but negative on the Seeplex HPV4A ACE. There were 3 cases which were positive only on the HC2, while 13 and 2 cases were positive only on the Seeplex HPV4A ACE and the Cobas 4800, respectively. The concordance rates between the results of the HC2 and of the 2 PCRs are shown in Table 3. The results of the Cobas 4800 agreed with those of the HC2 test in 82.88% of all samples (kappa coefficient, κ = 0.66), and the results of the Seeplex HPV4A ACE corresponded with those of the HC 2 test in 84.94% of the samples (κ = 0.7). The HC2 test produced positive results in 71 cases, which were greater than those of the Cobas 4800(54 positive samples) and the Seeplex HPV4A ACE (59 positive samples). Concordance rates between the result of the Cobas 4800 and the Seeplex HPV4A ACE are shown in Table 4. The results of the Cobas 4800 agreed with those of the Seeplex HPV4A ACE in 89.59% of all samples (kappa coefficient, κ = 0.78) regardless of genotypes. In cases which were infected with HPV 16, the Cobas 4800 and the Seeplex HPV4A ACE agreed in 7/9 samples (98.63%, κ = 0.87). In cases which were infected with HPV 18, the 2 PCR assays agreed in 3/3 samples (100%, κ=1.00). Table 5 shows the sensitivities and specificities of the 3 tests. When an abnormal Pap test was defined as ≥low grade squamous intraepithelial lesion(LSIL), the sensitivities of the HC 2 test, the Cobas 4800 and the Seeplex HPV4A ACE were 88.9% (95% CI = 83.4%-91.1%), 71.1% (95% CI = 65.2%-75.9%) and 80.0% (95% CI = 71.9%-87.3%), respectively, and their specificities are 67.9% (95% CI = 61.3%-69.8%), 76.4% (95% CI = 71.0%-77.8%) and 74.5% (95% CI = 64.3%-76.3), respectively.

Table 1

Positivity rates of the 3 HPV tests according to the cytological classification.

	Result of HPV test	Negative(n=62)	ASCUS(n=39)	LSIL(n=35)	HSIL(n=10)	ALL(n=146)	p-value
Hybrid capture 2	positive	14 (22.58%)	17(43.59%)	30(85.71%)	10(100.00%)	71(48.63%)	<0.0001
Cobas4800	positive	6(9.68%)	16(41.03%)	22(62.86%)	10(100.00%)	54(36.99%)	<0.0001
	16	0(0.00%)	2(5.13%)	4(11.43%)	3(30.00%)	9(6.16%)	0.0013
	18	1(1.61%)	0(0.00%)	0(0.00%)	1(10.00%)	2(1.37%)	0.0870
	other	5(8.06%)	14(35.90%)	18(51.43%)	6(60.00%)	43(29.45%)	<0.0001
Seegen	Positive	8(12.90%)	16(41.03%)	26(74.29%)	10(100.00%)	60(41.10%)	<0.0001
	16	0(0.00%)	1(2.56%)	3(8.57%)	3(30.00%)	7(4.79%)	0.0003
	18	1(1.61%)	0(0.00%)	0(0.00%)	1(10.00%)	2(1.37%)	0.0870
	other	7(11.29%)	15(38.46%)	23(65.71%)	6(60.00%)	51(34.93%)	<0.0001

Table 2

Concordance rates between the 3 tests.

HC2	Cobas4800	Seegen	n(%) (total=146)
negative	negative	Negative	68 (46)
negative	negative	positive	13 (9)
negative	positive	negative	2 (1)
positive	negative	negative	3 (2)
negative	positive	positive	3 (2)
positive	negative	positive	8 (5)
positive	positive	negative	2 (1)
positive	positive	positive	47 (32)

Table 3

Concordance rates between the results of the HC2 test and the other 2 tests.

		HC 2		ConcordanceRate (%)	KappaCoefficient	95% CI
		Positive (n=71)	Negative(n=75)
Cobas4800	Positive	50	4	82.88	0.66	0.54-0.77
	Negative	21	71
Seeplex	Positive	54	5	84.94	0.70	0.58-0.81
	Negative	17	70

Table 4

Concordance rates between the results of the Seeplex HPV4A ACE and the Cobas 4800 HPV test.

		Cobas 4800		ConcordanceRate (%)	KappaCoefficient	95% CI
		Positive (all)	negative
Seeplex	Positive	49	10	89.59	0.78	0.68-0.89
	Negative	5	80
		Positive for HPV16	Negative for HPV16
Seeplex	Positive	7	0	98.63	0.87	0.69 - 1.00
	Negative	2	137
		Positive for HPV18	Negative for HPV16
Seeplex	Positive	3	0	100.00	1.00	1.00-1.00
	Negative	0	143

Table 5

Sensitivities and specificities of HPV detection tests for high-risk genotypes in ≥LSIL.

Test	HPV genotype	Sensitivity(%)	95% CI (%)	Specificity(%)	95% CI (%)
HC 2	All HR	88.9	83.4-91.1	67.9	61.3-69.8
Cobas 4800	All HR	71.1	65.2-75.8	76.4	71.0-77.8
Seeplex	All HR	80.0	71.9-87.3	74.5	64.3-76.3

Discussion

The development of precancerous lesions is related to HPV genotypes and is being more related to HPV 16 and HPV 18 than to other types 1, 12. In our study, we compared the performances of a real-time PCR assay, multiplex PCR and the HC2 test for detecting HR HPVs. The positivity rates for HPV DNA increased according to the cervical cytological grade. In particular, the 3 tests detected HR HPVs from HSIL samples in 100% of cases. However, in our study, the overall HPV detection rates varied among different tests: the HC2 test produced positive higher HR HPV detection rate than the other 2 tests, regardless of the cytological classification. The concordance rates between the results of the HC2 test and the other 2 tests ranged from 82.8%-84.9 % with kappa coefficients of 0.66 to 0.70. In addition, discrepant results between the 3 tests for HPV DNA detection were seen in 31 (20%) of the 146 samples. Discrepant results between the Cobas 4800 and the Seeplex HPV4A ACE for HPV DNA detection were seen in 25 (16%) of the 146 samples, and discrepant results between these 2 tests for HR HPV detection were seen in 2 (1.37 %) of the146 samples. Previous studies have compared HR HPV detection rates between the HC2 test and the other 2 tests. Some investigators have reported that the Cobas 4800 performs comparably to the HC2 test, with kappa coefficients from 0.69 to 0.87 but that discrepant results between the HC2 and the Cobas 4800 occur in approximately 6% to 15% of all samples 7, 11, 13. Min et al. 14 have reported that the Seeplex HPV4A ACE also performs comparably to the HC2 test, with a kappa coefficient of 0.767 and a concordance rate of 81.7%. They also compare the Seeplex HPV4A ACE to another FDA-approved PCR method (Cervista) and stated that substantial agreement regarding HR HPV detection between the 2 tests was 89.5% (κ = 0.628). In our study, the reasons for discrepancies between the 3 tests may be explained as follows. The HC2 test was more likely to cross-react with certain noncarcinogenic HPV genotypes than the other 2 tests, and the sensitivities of the 3 tests were slightly different from one another 7, 12-14. The Cobas 4800 and the Seeplex HPV4A ACE have some advantages over other tests for HPV detection and genotyping. These 2 tests not only detect infections with HR HPV genotypes, but also allow for distinct partial HPV genotyping separately for HPV- 16 and HPV-18. In addition, there are higher concordance rates between with the results of the HC2 test and the other 2 tests.

We presented herein the results of carcinogenic HPV DNA detection between the 3 tests. Agreement between the 3 tests was high, especially between the results of the Cobas 4800 and the Seeplex HPV4A ACE. Our study has a limitation stemming from the small numbers of samples. Further studies with a larger sample size are needed to confirm our results.

In conclusion, the results of this study suggest that the Cobas 4800 and the Seeplex HPV4A ACE may be as effective as the HC2 test in detecting HR HPV and that there may be high concordance rates between the results of the Cobas 4800 and the Seeplex HPV4A ACE for detecting HPV 16 and HPV18.