Clinical relevance of the borderline results of the Hybrid Capture 2 High-Risk HPV DNA assay with cervical samples collected in Specimen Transport Medium.

Background The Hybrid Capture 2 (HC2) High-Risk HPV DNA assay serves as a triage test in the Slovenian national cervical cancer screening programme ZORA. To improve the limited analytical accuracy of HC2 test results near the cut-off value (1.0 relative light units/cut-off (RLU/CO)), we follow an internal protocol of repeating the test on all samples with borderline results within the 0.7-2.0 RLU/CO interval. The aim of the study was (i) to determine the clinical relevance of HC2 test results within three different "grey zones" for samples stored in Specimen Transport Medium (STM) and (ii) to determine whether the current algorithm of retesting "grey zone" STM specimens with the HC2 assay is clinically relevant. Patients and methods The study included 594 women between 20 and 65 years of age. All participating women were referred for colposcopy, and in cases of abnormal results, biopsy was performed. We assessed the distribution of HC2 test results and the corresponding proportion of cervical intraepithelial neoplasia grade 2 or worse (CIN2+) lesions in three different "grey zones" (1.0-2.5, 0.4-4.0 and 0.7-2.0 RLU/CO), retested specimens with results within a 0.4-4.0 RLU/CO interval and calculated the sensitivity and specificity for HC2 at different RLU/CO values. Results The proportion of specimens within 1.0-2.5, 0.4-4.0 and 0.7-2.0 RLU/CO intervals was 3.9%, 10.8% and 4.5%, respectively. The proportion of CIN2+ lesions within these "grey zones" was 2.5%, 5.6% and 1.2%, respectively. Retesting the samples did not detect any additional CIN2+ cases. Within the 1.0-2.5 RLU/CO interval, the sensitivity decreased from 93.8% to 91.4%, while the specificity increased from 63.3% to 67.5%; for the 0.4-4.0 RLU/CO interval, the sensitivity decreased from 95.1% to 89.5%, while the specificity increased from 56.8% to 69.4%; and for the 0.7-2.0 RLU/CO interval, the sensitivity remained nearly constant (94.4 vs. 93.2%), while the specificity increased from 60.6% to 66.4%. Conclusions Our results show that retesting STM samples within the "grey zones" is not necessary. Retesting samples in the negative "grey zone" does not increase sensitivity, and retesting in the positive "grey zone" is not followed by a less intensive management of women, since these women are recalled regardless of the results of the retest. Furthermore, the majority of samples retain the original HC2 results after retest, and the number of CIN2+ lesions among women with "grey zone" HC2 results is low.

Introduction

The Slovenian national cervical cancer screening programme ZORA uses the Hybrid Capture II (HC2) assay (Qiagen, Hilden, Germany) as a triage test that stratifies women with low-grade cervical changes into those with high and low risks for developing cervical cancer.1 The results of the HC2 test are presented as relative light units/ cut-off (RLU/CO) values, and the cut-off value for a positive result is 1.0 RLU/CO2. Many studies have confirmed the high reproducibility of the HC2 test results both within and between laboratories.3, 4, 5 Nevertheless, some studies have noted that analytical accuracy is significantly lower in the vicinity of the cut-off value.3, 4, 5, 6, 7, 8 Therefore, the manufacturer has published instructions for the further management of samples with borderline results that differ regarding which medium is used for sample collection. For PreservCyt specimens (Hologic Inc., Marlborough, United States), the manufacturer proposed the implementation of a borderline RLU/CO area called the “grey zone” in the range of 1.0–2.5 RLU/CO and recommended retesting the samples when results fall within this range.2 However, when storing the samples in the Specimen Transport Medium (STM) (Qiagen, Hilden, Germany), the manufacturer’s instructions are different, recommending retesting only the samples with suspected HPV infection and those with HC2 results near but below the 1.0 RLU/CO value. Retesting can be performed with the HC2 test or using another method.2

Several authors9, 10, 11, 12, 13, 14, 15, 16 have investigated the reproducibility and clinical significance of retesting PreservCyt samples with borderline HC2 results. However, we have not found a single study where the same problem was addressed for specimens collected in STM. A previous Slovenian study by Seme et al.6 evaluated the analytical accuracy of the results within 0.4–4.0 RLU/CO. Because these authors found poor reproducibility of HC2 within this range, they recommended that tests should be repeated by an alternative PCR-based method. At the Institute of Oncology Ljubljana, we adapted these instructions for our laboratory settings to repeat the test with the HC2 assay on specimens for which HC2 results fall within the 0.7–2.0 RLU/CO interval.

The aims of our study were (I) to determine the clinical relevance of the HC2 test results within three different “grey zones” for STM samples and (II) to determine whether the current algorithm for retesting “grey zone” STM specimens with the HC2 assay is clinically relevant.

Patients and methods

Study population

The study population included 596 women who participated in the Slovenian HPV self-sampling project L3-5512 from April 2014 to July 2016. The L3-5512 study protocol has been previously described.17 All women were referred to colposcopy where smears for high-risk HPV testing were obtained. The indications for colposcopy followed the Slovenian national guidelines, including high-grade cytology, HPV positive triage test after repeated low-grade cytology, positive HPV test for the surveillance of the women treated for high-grade intraepithelial lesion (HSIL) and a positive HPV test on self-sampling. Women with abnormal colposcopy underwent colposcopy-guided biopsy, followed by histological evaluation according to WHO recommendations.18 The reported high-grade histological outcomes within one year after colposcopy included cervical intraepithelial neoplasia grade 2 (CIN2), cervical intraepithelial neoplasia grade 3 (CIN3), squamous cell carcinoma, carcinoma with origin outside the cervix, cervical glandular intraepithelial neoplasia grade 2/adenocarcinoma in situ (CGIN2/AIS), vaginal intraepithelial neoplasia grade 3 (VAIN3) and vulvar intraepithelial neoplasia grade 3 (VIN3). All data were obtained from the Registry of the national screening programme ZORA.

The study was approved by the National Medical Ethics Committee at the Slovenian Ministry of Health (consents Nos. 155/03/13 and 136/04/14). All participating women provided written informed consent.

HPV testing

For the detection of high-risk HPV, we used the HC2 assay (Qiagen). The HC2 assay was performed according to the manufacturer’s instructions, and the results were reported as positive or negative using 1.0 RLU/CO as the cut-off value.2 Briefly, the test is a nucleic acid hybridization assay with signal amplification using microplate chemiluminescence for the qualitative detection of 13 high-risk types of HPV DNA in cervical and vaginal specimens.2 The results were also interpreted according to the “grey zone” ranges proposed by the manufacturer for PreservCyt (1–2.5 RLU/CO2), Seme et al. for STM (0.4–4.0 RLU/CO6) and the Department of Cytopathology Institute of Oncology Ljubljana for STM (0.7–2.0 RLU/CO). Residual samples were stored in the freezer (-30°C) after the denaturation step. If the results for a specimen were within the 0.4–4.0 RLU/CO range, then the HC2 assay was repeated. When the results of the retest differed from the original results, we reported the final result as inconclusive.

Statistics

The results are presented as ranges of RLU/CO values for the HPV test result; a proportion of HPV test results in a specific range for all HPV test results; a proportion of women with CIN2/3+ in a specific range for all women with CIN2/3+ in one year since colposcopy; and a risk for CIN2/3+ as a proportion of women with CIN2/3+ for all women in a specific range. Cohen’s kappa with a 95% confidence interval (CI) was calculated as a measure of agreement between the original and retested HPV test results as a binary variable at a 1.0 RLU/CO cut-off value. Sensitivities and specificities were calculated with a 95% CI at different RLU/CO cut-off values, and the ROC curve was plotted. All analyses were conducted with R v3.5.3.19

Results

Our final study group included 594 women after we excluded one woman who had undergone hysterectomy prior to colposcopy and another with missing data from colposcopy. Biopsy was performed in 352 women (59.3%). Out of 594 women, 291 (49.0%) were negative either on colposcopy (242) or on histology (49). A histologically confirmed low-grade intraepithelial lesion (LSIL) was diagnosed in 141 (23.7%) women and CIN2+ was diagnosed in 162 (27.3%) women. There were 48 CIN2, 102 CIN3, 1 VAIN3, 1 VIN3, 4 AIS and 6 squamous carcinomas.

HPV test results based on different definitions for the “grey zone” range

The number of HPV test results within and outside the three “grey zone” ranges are presented in Table 1. The proportion of samples located in 1.0–2.5 RLU/CO, 0.4–4.0 RLU/CO, and 0.7–2.0 RLU/CO was 3.9% (23/594), 10.8% (64/594), and 4.5% (27/594), respectively.

Table 1

Number of HPV test results according to “grey zones” proposed by the manufacturer (PreservCyt)I, Seme et al. (STM)II, and the Department of Cytopathology at Institute of Oncology Ljubljana (STM)Ill

RLU/CO value I	N and % women (Ntot = 594)
< 1.0	283 (47.6)
1.0–2.5	23 (3.9)
> 2.5	288 (48.5)

N = number; Ntot = total number

HPV test results for retested samples within the 0.4–4.0 RLU/CO range

All 64 samples with results between 0.4–4.0 RLU/ CO values were retested. In 8/64 (12.5%) retested samples, the results differed from the original results (Figure 1). Six samples for which the results changed from positive to negative originated from patients without CIN2+ lesions, while the seventh case was obtained from a patient with CIN3+ diagnosis. The sample for which the result changed from negative to positive was from a patient without a CIN2+ diagnosis. Kappa agreement between the results before and after retesting was 0.75 (95% CI: 0.59–0.91). Retesting samples with results within the 0.4–4.0 RLU/CO range did not detect any additional CIN2+ cases.

Figure 1

Changes in the RLU/CO values after retesting samples within the 0.4–4.0 RLU/CO range. The red arrow represents the samples with changes in the results from negative to positive, the blue arrows represent samples with changes in the results from positive to negative, and the black arrows represent samples that retained the original result.

Detection of CIN2+ and CIN3+ at different ranges of RLU/CO values

The distribution of women with a CIN2+/3+ diagnosis and the risk for CIN2+/3+ based on the RLU/CO values of their HPV test results are presented in Table 2. The majority of women with CIN2+ (85%) had RLU/CO values above 10, and 1.2%, 2.5% % and 5.6% of CIN2+ cases were found within RLU/CO intervals of 0.7–2.0, 1.0–2.5 and 0.4–4.0, respectively. The risk for CIN2+ in women within the 0.4–0.69 and 0.7–0.99 RLU/CO ranges was 5.6% and 8.3%, respectively; however, these results represent one woman per range.

Table 2

The distribution of women with a CIN2+/3+ diagnosis and the risk for CIN2+/3+ based on the RLU/CO values of their HPV test results

RLU/CO value	N and % women (Ntot = 594)	N and % CIN2+ (Ntot = 162)	Risk for CIN2+ (%)	N and % CIN3+ (Ntot = 114)	Risk for CIN3+ (%)
RLU/CO < 0.4	253 (42.6)	8 (4.9)	3.2	2 (1.8)	0.8
0.4 ≤ RLU/CO < 0.7	18 (3.0)	1 (0.6)	5.6	1 (0.9)	5.6
0.7 ≤ RLU/CO < 1.0	12 (2.0)	1 (0.6)	8.3	1 (0.9)	8.3
1.0 ≤ RLU/CO ≤ 2.0	15 (2.5)	1 (0.6)	6.7	1 (0.9)	6.7
2.0 < RLU/CO ≤ 2.5	8 (1.3)	3 (1.9)	37.5	2 (1.8)	25.0
2.5 < RLU/CO ≤ 4.0	11 (1.9)	3 (1.9)	27.3	2 (1.8)	18.2
4.0 < RLU/CO ≤ 10.0	26 (4.4)	7 (4.3)	26.9	6 (5.3)	23.1
10.0 < RLU/CO ≤ 100	112 (18.9)	48 (29.6)	42.9	33 (28.9)	29.5
100 < RLU/CO ≤ 1000	105 (17.7)	68 (42.0)	64.8	53 (46.5)	50.5
1000 < RLU/CO	34 (5.7)	22 (13.6)	64.7	13 (11.4)	38.2

CIN2+ = cervical intraepithelial neoplasia grade 2 or greater; CIN3+ = cervical intraepithelial neoplasia grade 3 or greater; N = number; Ntot = total number

Sensitivity and specificity of the HC2 test for CIN2+

Calculations of the sensitivity and specificity of HC2 for CIN2+ at various RLU/CO values are presented in Figure 2. At the threshold recommended by the manufacturer (RLU/CO = 1.0), the sensitivity was 93.8% (95% CI: 90.1–97.5%), and the specificity was 63.2% (95% CI: 58.6–67.6%). Increasing or decreasing the threshold in the vicinity of 1.0 RLU/ CO did not significantly improve one value without lowering the other. Increasing the cut-off value from 1.0 to 2.5 RLU/CO decreased the sensitivity from 93.8% to 91.4% (95% CI: 87.0–95.7%), while the specificity increased from 63.2% to 67.6% (95% CI: 63.0–72.0%). Within the interval of 0.7–2.0 RLU/ CO, the sensitivity remained nearly constant, with 94.4% (95% CI: 90.7–97.5%) at a cut-off value of 0.7 vs. 93.2% (95% CI: 89.5–96.9%) at a cut-off value of 2.0, while the specificity increased from 60.6% (95% CI: 56.0–65.3%) to 66.4% (95% CI: 61.8–70.8%). In the third “grey zone”, the sensitivity gradually decreased from 95.1% (95% CI: 91.4–98.1%) at 0.4 RLU/CO to 89.5% (95% CI: 84.6–93.8%) at 4.0 RLU/ CO, while the specificity gradually increased from 56.7% (95% CI: 51.9–61.3%) to 69.4% (95% CI: 65.0– 73.8%).

Figure 2

ROC curve demonstrating the sensitivity and specificity of the HC2 test for CIN2+ with marked RLU/CO cutoff values that represent the lower and upper borders of the “grey zone” ranges.

Discussion

Our results showed that relatively few HC2 test results fell within the “grey zone” ranges currently used by the Institute of Oncology Ljubljana and those proposed by the manufacturer for PreservCyt specimens (3.7% and 4.5%, respectively). The “grey zone” proposed by Seme et al.6 was broader and contained 10.8% of the HC2 results. The percentages of CIN2+ diagnoses detected within the above-mentioned “grey zones” were 1.2%, 2.5% and 5.6%, respectively. Retesting the samples within the broadest “grey zone” investigated did not detect additional CIN2+ cases. Calculations of sensitivity and specificity at different RLU/CO values showed that increasing or decreasing the cut-off value within the three “grey zones” did not significantly improve either of the variables.

We have not found any studies that examined the proportion of STM specimens with HC2 results located in the vicinity of a cut-off value. However, a few studies have reported on the number of HC2 results found in the “grey zone” interval proposed by the manufacturer (1.0–2.5 RLU/CO) for PreservCyt specimens.9, 10, 11 Muldrew et al.9, Rao et al.10 and Knoepp et al.11 found 3–5.2% of their specimens within this “grey zone”, which is similar to our result of 4.5%. This finding implies that the frequency of cases with HC2 results within this “grey zone” is similar, regardless of the medium used for specimen collection (STM or PreservCyt).

From a clinical point of view, the number of CIN2+ cases found within a “grey zone” is much more relevant than the number of HC2 equivocal results. Among all CIN2+ cases found in our whole study group, only 1.2% was within 0.7–2.0 RLU/ CO, 2.5% was within 1.0–2.5 RLU/CO and 5.6% was within 0.4–4.0 RLU/CO. Although several studies have evaluated the proportion of women with CIN2+ diagnoses among those with HC2 results near the cut-off point, it is difficult to compare our results to theirs. The results vary to a moderate extent, and some differences could be attributed to differences in the studied populations and to the definition of the equivocal results. Knoepp et al.12, for example, reported 8% of CIN2+ cases among the whole study population with equivocal HC2 results (1.0–2.5 RLU/CO) and 16.5% of CIN2+ cases of equivocal HC2 and ASC-US cytology. Origoni et al.7 found only 4.6% of CIN2+ cases in women with ASC-US cytology and HC2 between 1.0 and 10.0 RLU/CO, while LaMere et al.13 reported 6.8% CIN2+ among cases with low-grade cytology and 1.0–3.0 RLU/CO. Interestingly, Elkins et al.14 demonstrated that the clinical relevance of the HC2 test is age dependent. In two age groups, which together ranged from 15–49 years, these authors found an approximately equal percentage of CIN2+ diagnoses (6%) within the “grey zone” of 1.0–2.5 RLU/CO, while there were no CIN2+ cases in the group aged 50 years or more. Despite variations in their results, all authors concluded that women with equivocal HC2 results should be managed as unequivocal positive results.

Since the manufacturer recommends retesting PreservCyt specimens with results within the “grey zone”, several authors have already evaluated whether the retesting algorithm is effective.11, 12, 13, 14, 15, 16 Some authors have performed one retest15, 16, while others have reported two retests13, and some retested only those samples where the results were found within the “grey zone” after the first retest11 or below 1.0 RLU/CO.14 The range of the “grey zone” varied from 0.8 to 3.0 RLU/CO. Some authors have found that the majority of their specimens retained the original positive (87%–97%) HC2 result.11, 12, 13, 14 These results are comparable to our results for the STM specimens, which showed that 87.5% of specimens retained the original HC2 test result after retest, even though our “grey zone” was wider. Ramirez et al.15 and de Vries et al.16 obtained lower values for specimens that retained original HC2 diagnoses after retest (64% and 74%, respectively). Ramirez et al.15 was the only study besides ours where the “grey zone” extended below 1.0 RLU/CO, and these authors also found that two results changed from negative to positive. The low percentage of cases that retained positive HC2 results in the study of de Vries16 is due to the age of their study population of 50 years or older. All the above-mentioned authors concluded that retest is not necessary because few results change after retest or because retests might be less reliable. For example, Ramirez et al.15 mentioned that the amount of sample can influence the results, and viral loads may vary in successive tests.

The expected added value of retesting equivocal HC2 results is a potential increase in the accuracy of the triage test and a better identification of women who have low-risk for developing CIN2+ lesions since these individuals could be returned to screening or to a less intensive follow-up. However, an additional reason that speaks against retesting samples within the “grey zones” is the finding that sensitivity and specificity do not change much within these “grey zone” ranges. Calculating the sensitivity of the HC2 test at different RLU/CO values demonstrated that decreasing the cut-off for positivity to the lower border of the “grey zone” range would achieve a small increase in the sensitivity and therefore would not add a higher accuracy of the test. The sensitivity was 95.1% at a cut-off value of 0.4, 94.4% at the cut-off value of 0.7 and 93.8% at the cut-off value of 1.0 RLU/CO. Several authors have performed similar studies20, 21, 22, 23, 24, 25, 26 and a systematic review by Rebolj et al.27 concluded that the threshold could be increased to values between 2.0–10.0 RLU/CO without endangering the sensitivity level necessary for screening. This increase would avoid the problem of “grey zones”. Compared to our study, the majority of the reported studies have compared the relative values of sensitivity, since HPV-negative women did not undergo colposcopy examinations. Therefore, their calculations did not include the CIN2+ lesions with HC2 results in the negative part of the RLU/ CO range since such cases were missed. Thus, their values of sensitivity were higher.

The strength of our study is in reporting results within “grey zones” for STM specimens and investigating two “grey zones” that extended below 1.0 RLU/CO. The results within “grey zones” for STM samples have not been presented before, and most reports included “grey zones” above the proposed cut-off value. The limitation of our study is a small study population, comprising women invited to colposcopy, which showed a higher incidence of CIN2+ lesions. The risk for developing CIN2+ was higher within the investigated “grey zones” (7.4%, 17.4% and 14.1%, respectively) than among the general population since the prevalence of the disease was higher in women referred to colposcopy than in the population of women in our study. Therefore, our findings need to be tested on a larger population with the risk for CIN2+ comparable to that of the population where triage is recommended. An additional limitation of our study is the use of residual samples that were stored in the freezer (-30°C) after the denaturation step for HC2 retesting. This storage procedure could have caused sample degradation and might have influenced our results. These findings will be important for cervical cancer screening programmes that use the HC2 assay as a primary or triage test and collect cervical specimens in STM. The results will help specify the best protocol for handling STM specimens with results within the “grey zone”.

In conclusion, our results show that retesting STM samples within the “grey zones” is not necessary for several reasons. The majority of samples within the “grey zone” retain the original HC2 results after retest. The number of CIN2+ lesions among women with “grey zone” HC2 results is low. There is limited additional value of the retesting algorithm since sensitivity and specificity of HC2 for CIN2+ do not change much within the “grey zone”. Retesting samples with HC2 results in the negative range of the “grey zone” does not increase sensitivity, while retesting in the positive “grey zone” does not add to a less intensive management of women. Women with HC2 results above 1.0 RLU/CO but within the “grey zone” will be followed in the same manner, regardless of the outcome of the retest. Only women with two negative HC2 results will return to regular screening. Furthermore, according to Slovenian clinical guidelines, the management of women with discordant results between the original test and the repeated HC2 test does not allow women to be returned to screening or to a less intensive follow-up, since at least one additional test is needed before the decision could be made about further management.