Performance and Diagnostic Accuracy of Human Papillomavirus Testing on Self-Collected Urine and Vaginal Samples in a Referral Population.

PURPOSE: The study aimed to evaluate the diagnostic accuracy of polymerase chain reaction ‒based high-risk human papillomavirus (HPV) assays on self-collected vaginal and urine samples for detection of precancerous cervical lesions in referral population.
MATERIALS AND METHODS: Women referred for colposcopy following abnormal cytology, were included this study. A total of 314 matched urine, vaginal, and cervical samples were collected. All samples were tested for HPV DNA using the RealTime HR-S HPV and Anyplex II HPV 28 assays. Primary endpoints were sensitivity for cervical intraepithelial neoplasia (CIN) 2+/CIN3+ and specificity for <CIN2. Secondary endpoints were the relative accuracy of high-risk HPV (hrHPV) test results in vaginal and urine samples versus cervical samples.
RESULTS: The sensitivity of Realtime HR-S and Anyplex HPV assay was 93.13% (95% confidence interval [CI], 87.36 to 96.81) and 90.08% (95% CI, 83.63 to 94.61) for CIN2+ (n=130); specificity for <CIN2 was 32.69% (95% CI, 25.03 to 38.97) and 33.33% (95% CI, 26.26 to 41.00), respectively. Relative sensitivity of Realtime HR-S and Anyplex HPV tests for the detection of CIN2+ in vaginal versus cervical samples were 0.91 (95% CI, 0.90 to 1.03) and 0.87 (95% CI, 0.75 to 1.02), respectively; urine versus cervical comparisons were 0.79 (95% CI, 0.70 to 0.92) and 0.74 (95% CI, 0.61 to 0.89).
CONCLUSION: The detection performance for hrHPV and CIN2+ on self-collected vaginal samples was comparable to that of clinician-collected cervical samples. On the other hand, HPV tests using urine were inferior to those using clinician-collected cervical samples in terms of detecting hrHPV and CIN2+.

Introduction

Human papillomavirus (HPV) is well established as the main cause of cervical cancer [1]. HPV testing has very high sensitivity for detecting cervical precancerous lesions defined as high-grade squamous intraepithelial lesions (HSIL) [2,3].

However, a major obstacle to controlling cervical cancer is the lack of participation in screening programs. In developed countries, such as the United Kingdom and the United States, 20%–30% of women of screening age have not been screened within the past 5 years or have never been screened [4]. In countries without well-developed screening programs, the participation rate was low, and 50%–80% of women were not screened [5]. Recently, comparative modeling analysis suggests that both high HPV vaccination coverage and screening uptake will be necessary, particularly in countries with the highest burden, to achieve elimination of cervical cancer [6].

HPV test using non-invasive technique such as self-collected urine and vaginal sampling is feasible and may increase participation in screening programs [7,8]. An HPV test using self-collected vaginal samples showed a comparable clinical accuracy in detecting HSIL in a meta-analysis [9]. The IMPR-OVE study, a randomized controlled trial, suggested that HPV testing with a clinically-validated polymerase chain reaction (PCR)–based assay had similar accuracy between self-collected and clinician-collected samples in terms of detection of precursor lesions (HSIL) [10]. In addition, since the HPV test using urine seems to have good accuracy in detecting HPV infection, it could be an additional strategy for women who do not participate in regular screening programs [11].

The accuracy of self-collecting HPV has been shown by well-established evidence, but it still has some limitations. A meta-analysis showed that HPV testing had lower sensitivity when performed with self-collected samples than with clinician-collected samples [9]. Furthermore, only 8.8% of the women invited to the IMPROVE trial participated. This low participation rate might reflect reluctance to be included in a randomized trial [12]. In terms of the urine HPV test, there was substantial heterogeneity between the studies in terms of type of HPV test used, diverse methods, and disease outcomes. Moreover, the clinical sensitivity for detecting cervical cancer precursor lesions with urine-based HPV testing remains unknown [13].

The objective of this study was to determine the performance and diagnostic accuracy of PCR-based high-risk HPV assays on self-collected vaginal and urine samples for detection of precancerous cervical lesions in a large referral population and to compare the results with those using the same test from paired clinician-collected cervical samples collected at the same visit.

Materials and Methods

1. HPV URSELF study

A population-based study of women referred to colposcopy for abnormal cytology results was conducted at three medical centers in Korea between January 2018 and January 2020. Potential participants had to be between the ages of 20 and 60, not pregnant at the time of the study, and have had none of the following: previous treatment for cervical disease (including the loop electrosurgical excision procedure, cold knife conization, cryotherapy, and laser therapy), previous hysterectomy, prior chemotherapy, radiation treatment for cervical neoplasia or another concurrent cancer, and human immunodeficiency virus infection or acquired immune deficiency syndrome. Matched samples (clinician-collected cervical sample, self-collected vaginal, and urine samples) were collected from study participants within 1 week after their visit to the colposcopy center.

2. Sample collection and preparation

On the day each participant visited the colposcopy center, she was provided with a self-sampling kit consisting of a plastic brush (Flocked Swab, manufactured by Noble Biosciences, Inc., Hwaseong, Korea), PreservCyt Solution (ThinPrep, manufactured by Hologic, Marlborough, MA), a urine collection cup (BD Vacutainer, manufactured by BD Diagnostics, Franklin Lakes, NJ), and illustrated instructions. Participants were instructed to collect a vaginal sample by inserting the plastic brush one inch into the vagina, rotating the swab for 15 seconds, and then removing it. The brush was subsequently suspended in 5 mL of ThinPrep, PreservCyt Solution. Participants then underwent a pelvic exam during which the clinician-collected a cervical sample using a cervical brush (Cervical Brush, manufactured by Noble Biosciences, Inc.). This brush was also suspended in 5 mL of ThinPrep, PreservCyt Solution. On the morning of another day, participants were instructed to collect the initial flow of urine (first-void) samples (approximately 30 mL) with a urine collection cup. The clinician-collected cervical samples were used as a reference sample for HPV DNA detection. Cervical, vaginal, and urine samples were stored at 4°C and processed within 1 week [14].

3. HPV assay detail

DNA extraction was performed as previously described [15]. HPV genotyping was performed via two different methods, the RealTime HR-S HPV and Anyplex II HPV 28 assays, both of which were performed at the Korea University Guro Hospital. The procedures used for the two assays were performed as previously described [15].

4. Sample size

Based on previous studies on real-time PCR-based HPV testing using urine samples as in this study, we assumed the sensitivity for cervical intraepithelial neoplasia (CIN) 2+ was 90%, with a 95% confidence interval (CI) 81.2–98.8 [11,16]. The calculated minimum number of study participants with CIN2+ was 45. Based on previous literature, it was assumed that the prevalence of CIN2+ among women with abnormal cytology who were referred to colposcopic biopsy was 15% [17,18]. Therefore, the required sample size was 300 for this study.

5. Statistical analysis

The primary endpoint was the sensitivity and specificity of the assays to detect CIN2+ and CIN3+ in clinician-collected cervical samples and self-collected vaginal and urine samples. The relative accuracy of high-risk HPV (hrHPV) test results in vaginal and urine samples versus cervical samples was computed, and 95% CIs were calculated according to binomial distributions. McNemar’s test is a statistical test used to evaluate paired nominal data and can be used to compare the proportions of hrHPV positive results between self-collected vaginal/urine samples and clinician-collected cervical samples, while accounting for the correlation of multiple samples within subjects [19]. Anyplex II HPV 28 detects 19 hrHPV types (16, 18, 26, 31, 33, 35, 39, 45, 51, 52, 53, 56, 58, 59, 66, 68, 69, 73, and 82), while RealTime HR-S detects 14 hrHPV types (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68). Only the 14 HPV types detected by the latter assays were considered hrHPV types in this study. Confidence intervals were calculated, and the significance level was set at 0.05. The statistical analyses were performed using SPSS ver. 24.0 (IBM Corp., Armonk, NY) and MedCalc Software (MedCalc software, Mariakerke, Belgium).

Results

A flow diagram for this study is shown in Fig. 1. A total of 458 women agreed to provide clinician-collected cervical samples, self-collected vaginal samples, and first-void urine. Of these, 124 were excluded after enrollment because either a vaginal or urine sample was not collected, and 20 paired samples were further excluded due to invalid results or insufficient samples. Finally, matched samples (cervical, vaginal samples, and urine) were collected from 314 participants: 21 were diagnosed with CIN2, 104 were diagnosed with CIN3, and four were diagnosed with cervical cancer. The median age of the participants was 40±15.4 years.

Fig. 1

Study flow diagram. CIN, cervical intraepithelial neoplasia; HPV, human papillomavirus.

1. The agreement in HPV detection using self-collected vaginal/urine samples compared to clinician-collected cervical samples and results of McNemar’s test

Table 1 shows the agreement between self-collected vaginal or urine samples and clinician-collected cervical samples. The agreement between Realtime HR-S and Anyplex II HPV tests for vaginal versus cervical samples was 85.03% (95% CI, 80.60 to 88.79) and 82.17% (95% CI, 77.47 to 86.24), respectively. There was no significant difference in hrHPV detection between cervical and vaginal samples (RealTime HR-S, p=0.079; Anyplex II HPV, p=0.350). The agreement between HPV tests in urine compared with cervical samples was lower than vaginal samples (RealTime HR-S: 78.03%; 95% CI, 73.03 to 82.48; and Anyplex II: 74.84%; 95% CI, 69.66 to 79.55). In addition, there was a significant difference in hrHPV detection between cervical and urine specimens (RealTime HR-S, p < 0.001; Anyplex II HPV, p < 0.001).

Table 1

Overall agreement between clinician-collected cervical and self-collected vaginal/urine samples

Comparison of samples	Realtime HR-S HPV							Anyplex II HPV
	+/+	+/ −	−/+	−/ −	Agreement %	95% CI[a)]	McNemar p-value[a)]	+/+	+/ −	−/+	−/ −	Agreement %	95% CI	McNemar p-value[a)]
Cervical vs. vaginal	217	30	17	50	85.03	80.60–88.79	0.079	198	32	24	60	82.17	77.47–86.24	0.350
Cervical vs. urine	194	53	16	51	78.03	73.03–82.48	< 0.001	164	66	13	71	74.84	69.66–79.55	< 0.001

CI, confidence interval; HPV, human papillomavirus.

Two-tailed McNemar’s test.

Table 2 presents the concordance between urine and vaginal HPV tests compared to cervical HPV tests according to age group. There was no significant difference in hrHPV detection between cervical and vaginal samples between all age groups as shown in Table 1. However, urine HPV tests showed a significant difference compared to cervical HPV tests in women aged in their 20s and 30s, but not in those aged in their 40s and 50s.

Table 2

Agreement between clinician-collected cervical and self-collected vaginal/urine samples according to age groups

Age group (yr)	No.	Comparison of samples	Realtime HR-S HPV							Anyplex II HPV
			+/+	+/ −	−/+	−/ −	Agreement %	95% CI	McNemar p-value[a)]	+/+	+/ −	−/+	−/ −	Agreement %	95% CI	McNemar p-value[a)]
≥ 50	32	Cervical vs. vaginal	15	2	4	11	81.25	63.56–92.79	0.688	14	4	4	10	75.00	56.60–88.54	> 0.99
		Cervical vs. urine	15	2	3	12	84.38	67.21–94.73	> 0.99	14	4	2	12	81.25	63.56–92.79	0.688
40–49	65	Cervical vs. vaginal	45	10	5	5	76.92	64.81–86.47	0.302	46	9	5	5	78.46	66.51–87.70	0.424
		Cervical vs. urine	46	9	3	7	81.54	69.97–90.08	0.146	42	13	4	6	73.85	61.46–83.97	0.049
30–39	141	Cervical vs. vaginal	98	12	6	25	87.23	80.88–92.26	0.238	89	12	9	31	85.11	78.14–90.54	0.664
		Cervical vs. urine	85	25	8	23	76.60	68.73–83.31	0.005	73	28	6	34	75.89	67.97–82.69	< 0.001
20–29	76	Cervical vs. vaginal	59	6	2	9	89.47	80.31–95.35	0.289	49	7	6	14	82.90	72.53–90.57	> 0.99
		Cervical vs. urine	48	17	2	9	75.00	63.74–84.23	0.001	35	21	1	19	71.05	59.52–80.89	< 0.001

CI, confidence interval; HPV, human papillomavirus.

Two-tailed McNemar’s test.

Table 3 shows the agreement for HPV tests between vaginal/urine samples and cervical samples according to cytologic results. When comparing the cervical and vaginal HPV, there was a statistical difference in the patients diagnosed with ASC-H (atypical squamous cells of a high-grade squamous intraepithelial lesion cannot be ruled out) and HSIL by cytology. Urine HPV tests showed a significant difference from cervical HPV tests in all cytology results.

Table 3

Agreement between cervical and vaginal/urine samples according to cytology

Comparison samples	Cytology	No.	Realtime HR-S HPV							Anyplex II HPV
			+/+	+/ −	−/+	−/ −	Agreement %	95% CI	McNemar p-value[a)]	+/+	+/ −	−/+	−/ −	Agreement %	95% CI	McNemar p-value[a)]
Cervical vs. vaginal	All	314	217	30	17	50	85.03	80.60–88.79	0.079	198	32	24	60	82.17	77.47–86.24	0.350
	ASCUS/LSIL	244	159	23	17	45	83.61	78.35–88.02	0.430	144	21	23	56	81.97	76.56–86.58	0.880
	ASC-H/HSIL	70	58	7	0	5	90.00	80.48–95.88	0.016	54	11	1	4	82.86	71.97–90.82	< 0.001
Cervical vs. urine	All	314	194	53	16	51	78.03	73.03–82.48	< 0.001	164	66	13	71	74.84	69.66–79.55	< 0.001
	ASCUS/LSIL	244	143	39	14	48	78.28	72.57–83.29	0.001	117	48	13	66	75.00	69.08–80.30	< 0.001
	ASC-H/HSIL	70	51	14	2	3	77.14	65.55–86.33	0.002	47	18	0	5	74.29	62.44–83.99	< 0.001

ASC-H, atypical squamous cells of a high-grade squamous intraepithelial lesion cannot be ruled out; ASCUS, atypical squamous cells of undetermined significance; CI, confidence interval; HPV, human papillomavirus; HSIL, high-grade squamous intraepithelial lesion; LSIL, low-grade squamous intraepithelial lesion.

Two-tailed McNemar’s test.

2. Absolute sensitivity and specificity

As shown in Table 4 for the cervical samples, the sensitivity of Realtime HR-S and Anyplex II were 93.13% (95% CI, 87.36 to 96.81) and 90.08% (95% CI, 83.63 to 94.61), respectively. The specificity of the two HPV assays for detecting CIN2+ were 32.69% (Realtime HR-S: 95% CI, 25.03 to 38.97) and 33.33% (Anyplex II: 95% CI, 26.26 to 41.01) for detecting CIN2+.

Table 4

Clinical performance of HPV test to detect CIN2+ in clinician-collected cervical, self-collected vaginal and urine samples

	Realtime HR-S HPV		Anyplex II HPV
	Sensitivity (95% CI)	Specificity (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)
Cervical	93.13 (87.36–96.81)	32.69 (25.03–38.97)	90.08 (83.63–94.61)	33.33 (26.26–41.01)
Vaginal	84.73 (77.41–90.42)	26.79 (20.25–34.15)	78.63 (70.61–85.30)	29.17 (22.42–36.66)
Urine	73.28 (64.85–80.63)	32.14 (25.16–39.77)	66.41 (57.61–74.42)	46.43 (38.71–54.27)

CI, confidence interval; CIN, cervical intraepithelial neoplasia; HPV, human papillomavirus.

For the vaginal samples, the sensitivity of Realtime HR-S and Anyplex II was 84.73% (95% CI, 77.41 to 90.42) and 78.63% (95% CI, 70.61 to 85.30) for CIN2+, respectively. The specificity of Realtime HR-S and Anyplex II was 26.79% (95% CI, 20.25 to 34.15) and 29.17% (95% CI, 22.42 to 36.66), respectively. The sensitivities of the urine HPV test were slightly lower than those of cervical or vaginal HPV tests (Realtime HR-S: 73.28%; 95% CI, 64.85 to 80.63; Anyplex II: 66.41%; 95% CI, 57.61 to 74.42). The specificity of Realtime HR-S and Anyplex II was 32.14% (95% CI, 25.16 to 39.77) and 46.43% (95% CI, 38.71 to 54.27), respectively.

3. Relative sensitivity and specificity

The relative sensitivities and specificities for detecting CIN2+ in all samples are presented in Table 5. The sensitivity of HPV tests from vaginal samples was lower, but not significantly different compared to that of cervical samples (Realtime HR-S: 0.91; 95% CI, 0.80 to 1.04; Anyplex II: 0.87; 95% CI, 0.75 to 1.02). However, the relative sensitivity of HPV tests on urine specimens was significantly lower than cervical HPV tests (Realtime HR-S: 0.79; 95% CI, 0.70 to 0.92; Anyplex II: 0.74; 95% CI, 0.61 to 0.89). There was no significant difference in specificity for detecting CIN2+ between vaginal/urine samples compared with cervical samples.

Table 5

Relative sensitivity/specificity of HPV test to detect CIN2+ in self-collected vaginal and urine samples compared with clinician-collected cervical samples

Comparison of samples	Realtime HR-S HPV		Anyplex II HPV
	Relative sensitivity (95% CI)	Relative specificity (95% CI)	Relative sensitivity (95% CI)	Relative specificity (95% CI)
Cervical vs. vaginal	0.91 (0.80–1.04)	0.82 (0.52–1.37)	0.87 (0.75–1.02)	0.88 (0.55–1.38)
Cervical vs. urine	0.79 (0.70–0.92)	0.98 (0.65–1.59)	0.74 (0.61–0.89)	1.39 (0.94–2.07)

CI, confidence interval; CIN, cervical intraepithelial neoplasia; HPV, human papillomavirus.

Discussion

In this study, the paired sensitivity differences for detecting CIN2+ between cervical versus vaginal sampling were not significant, but the sensitivity of hrHPV tests on urine was significantly lower than that seen in cervical samples. In terms of concordance between samples, the agreement between hrHPV detection in self-collected vaginal and clinician-collected cervical samples was comparable, whereas there was a significant difference between self-collected urine and clinician-collected cervical samples with respect to the use of HPV tests to detect hrHPV infection.

Out results are in line with other studies that investigated the accuracy of HPV testing on self-collected vaginal samples. Previous studies including clinical trial and meta-analysis showed that HPV testing on self-collected vaginal samples had similar accuracy to clinician-collected cervical samples [9,10,20]. A 2014 meta-analysis showed that although the pooled sensitivity of HPV testing on self-samples was lower than HPV testing on a clinician-taken sample (ratio, 0.88 [95% CI, 0.85 to 0.91] for CIN2 or worse and 0.89 [0.83 to 0.96] for CIN3 or worse), PCR-based HPV tests generally showed similar sensitivity for both self-samples and clinician-based samples [9]. An updated meta-analysis also showed that when used with hrHPV assays based on PCR, testing on self-collected samples was similarly accurate as on clinician samples (pooled ratio of sensitivity for CIN2+, 0.99; 95% CI, 0.97 to 1.02) [20]. In a randomized, non-inferiority trial, HPV testing performed with a clinically-validated PCR-based assay had similar accuracy for self-collected and clinician-collected samples in terms of detection of CIN2+ or CIN3+ lesions [10].

This study showed that the hrHPV positivity and sensitivity for CIN2+ in urine was significantly lower than in cervical samples. Although many recent studies including meta-analyses that reported that PCR-based HPV testing on urine samples seemed to have good accuracy for detection of HPV [11,21-23], there was substantial heterogeneity between the studies. A 2014 meta-analysis of 14 studies reported a pooled sensitivity of 77% and specificity of 88% for urine detection of hrHPV [11]. Senkomago et al. demonstrated that the sensitivity of HPV tests in urine for CIN2+ detection was high (89.9%; 95% CI, 62.7 to 99.6) [24]. Cuzick et al. [13] showed the sensitivity from urine was slightly, but not significantly, lower (CIN3+, 91.4% [95% CI, 83.0 to 96.5]; p=0.300; CIN2+, 88.3% [95% CI, 81.9 to 93.0]; p=0.060). On the other hand, in a PaVDaG study, which is a population-based study, the relative sensitivity of hrHPV positivity for the detection of CIN2+ in urine versus cervical comparisons was significantly low (0.53; 95% CI, 0.42 to 0.67) [19]. Asciutto et al. showed that detection of hrHPV in urine samples had a sensitivity of 76.3% (95% CI, 67.9 to 84.2) for HSIL, which is similar to our results [25]. Therefore, evidence regarding clinical performance including sensitivity for detecting precancerous lesions with HPV testing on urine samples has been lacking until now [13].

There may be several reasons why the urine HPV test is less sensitive compared with cervical HPV tests in this study. First, since two HPV assays (Realtime HR-S and Anyplex II) were performed from one self-collected sample in the study, the amount of that sample might not have been sufficient. Moreover, because self-collected vaginal or urine samples did not contain enough exfoliated cervical cells for detection, this insufficient amount may have degraded the clinical performance of the self-collected vaginal samples [26]. Second, an uncontrolled urine sampling technique such as sampling at home has the risk of detecting HPV infections not correlated to the cervix and interacts negatively with test performance. Third, because the study did not use a chelating agent to collect cell-free DNA, this may be associated with the low concordance seen in the urine samples [27]. Previous studies have detected a substantial amount of non-cell-associated DNA, and a chelating agent can be used to avoid degradation of cell-free DNA [15]. Finally, there might have been DNA degradation in sample storage, due to urine collection without a preservative.

Additionally, the patient’s age may have influenced the outcome. There was no significant difference in hrHPV detection between cervical and urine samples in women aged 40–50. In a PaVDaG study, there was a 38% (24% to 57%) higher HPV detection rate in vaginal self-samples from women over 50 years compared with those ≤ 29 years [19]. Further studies are needed on the clinical performance of the urine HPV test according to age.

This study has several limitations. First, because a referral population was enrolled in this study, there may be limitations in evaluating the clinical performance of HPV testing on urine and vaginal samples. Previous studies have shown that a referral population is usually provides an efficient and accurate measure of sensitivity in a screening context, but its higher HPV positivity rate may make it is less reliable for assessing specificity [13]. Further studies are needed to validate the clinical performance of the Realtime HR-S and Anyplex II HPV tests with urine and vaginal sampling, especially for specificity in a screening population. Second, DNA extracts from participant urine and vaginal samples were divided for two kinds of HPV assays. Therefore, the reliability of the HPV test may deteriorate due to an insufficient amount of DNA. However, samples with incomprehensible results or invalid internal controls were excluded from analysis in this study. In addition, loss of urine during self-sampling at home and non-use of chelating agents and preservatives may be negative factors to lower the sensitivity of the urine HPV test.

Regardless of the limitations, the current study is the first Korean study to evaluate the clinical performance of PCR-based HPV tests using paired urine, vaginal, and cervical samples for detection of precancerous cervical lesions in high-risk women.

In conclusion, test performance to detect hrHPV and CIN2+ on self-collected vaginal samples was comparable with that of clinician-collected cervical samples. On the other hand, HPV tests using urine were inferior to those using clinician-collected cervical samples in terms of detecting HPV and CIN2+. HPV testing from self-collected vaginal samples may be useful for women who do not obtain cervical screening. Further research is needed to increase the sensitivity of urine HPV tests and optimize sampling methods.