Performance and acceptability of self-collected human papillomavirus testing among women living with HIV.

OBJECTIVE: To assess the validity, reliability, and acceptability of self-collected human papillomavirus (HPV) tests in women living with HIV (WLHIV) in the United States.
METHODS: WLHIV ≥30 years of age underwent self-collected (clinic and home) and clinician-collected HPV tests. Sensitivity and specificity analyses were performed using the clinician-collected HPV tests as the comparator. The unweighted kappa statistic was used to evaluate the validity and reliability of self-collected HPV testing, and the level of agreement between the clinician-collected mRNA test and a DNA test that was used for routine care. A 13-question survey was used to assess acceptability.
RESULTS: Among the 70 participants, the median age was 50 years, 75% had an undetectable HIV RNA, and 11% had a CD4 count of <200 cells/μl. Nearly 63% had at least one positive HPV test. The sensitivity and specificity of the self-collected HPV test were 84.6% (95% confidence interval (CI) 65.1-95.6%) and 62.9% (95% CI 44.9-78.5%), respectively (κ=0.5, 95% CI 0.2-0.7). The agreement between the two self-collected tests was good (κ=0.8, 95% CI 0.5-1.0). There was good agreement between clinician-collected mRNA tests and DNA tests (κ=0.8, 95% CI 0.7-1.0). Self-collection was highly acceptable.
CONCLUSIONS: Among WLHIV, self-collected HPV tests had good sensitivity and moderate specificity compared to clinician-collected HPV tests. The reliability between self-collected testing locations was good. Self-collected HPV testing had high acceptability.

Introduction

High-risk human papillomavirus (HPV) causes almost all cervical cancers (Walboomers et al., 1999). The American Cancer Society estimated that there were 13 170 new cervical cancer cases and 4250 cervical cancer deaths in the United States in 2019 (Siegel et al., 2019). Most HPV infections are asymptomatic and can be cleared, but if the infection persists and is left untreated, over time, it can lead to precancerous changes that may develop into cancer. One of the risk factors for persistence and progression of HPV-related disease is immunodeficiency (IARC Working Group on the Evaluation of Carcinogenic Risks to Humans, 2012). Women living with HIV/AIDS (WLHIV) have a four-fold higher risk of developing cervical cancer than the general population but are under-screened (Engels et al., 2006, Chaturvedi et al., 2009, Oster et al., 2009, Panel on Opportunistic Infections in Adults and Adolescents with HIV, 2009, Peprah et al., 2018). In addition, infection with multiple HPV types is seen more often among WLHIV compared to women without HIV (Levi et al., 2002, Moscicki et al., 2004, Menon et al., 2016), and faster HPV-related disease progression has been observed (Denslow et al., 2014).

HPV testing is a notable advancement in cervical cancer screening, given its ability to offer greater reassurance of low cancer risk compared to cytology alone. A randomized controlled trial (Ronco et al., 2010) comparing conventional liquid-based cytology versus HPV-based testing (either alone or combined with cytology) showed that HPV-based screening was more effective in preventing cervical cancer by detecting persistent high-grade lesions earlier. Additionally, HPV testing has a high sensitivity for detecting precancerous lesions (Bulkmans et al., 2007, Naucler et al., 2007, Clad et al., 2011). A negative HPV test provides greater reassurance of low cervical precancer risk compared with a negative Pap test (Cuzick et al., 2006, Gyllensten et al., 2012, Leinonen et al., 2012, Ogilvie et al., 2012, Malila et al., 2013, Ronco et al., 2014, Wright et al., 2015). Furthermore, compared to cytology, HPV testing is less subjective, more reproducible, and needs less training and expertise for users (Cuzick et al., 2006). The American Society for Colposcopy and Cervical Pathology (ASCCP) and the United States Public Service Task Force (USPSTF) support primary HPV screening as one of the first-line cervical cancer prevention strategies among women older than 25 years or 30 years, respectively (Huh et al., 2015; US Preventive Services Task Force et al., 2018).

Studies have shown that HPV self-collection increases cervical cancer screening participation in healthy women who do not routinely attend traditional cervical cancer screening programs, and the acceptability has been favorable (De Alba et al., 2008, Racey et al., 2013, Nelson et al., 2015, Winer et al., 2016). Therefore, self-collected samples potentially can increase the uptake of cervical cancer screening in WLHIV who are under-screened by offering screening at primary care sites, HIV specialty clinics, or non-clinical sites (e.g., home) that do not routinely perform pelvic examinations; however, there are very few studies to support this assertion. Therefore, the aim of this study was to assess the validity, reliability, and acceptability of HPV self-collected tests in a clinical setting and at home compared to conventional clinician-collected HPV tests in US WLHIV.

Methods

Study design and participants

WLHIV ≥30 years of age who were undergoing a Pap test with HPV DNA testing as part of routine care were eligible for inclusion. Women who were pregnant, had a hysterectomy, and had genital tract cancer were excluded. Recruitment took place at a large academic multidisciplinary clinic in Baltimore, Maryland. Institutional review board approval was obtained.

Procedures and data collection

Participants were instructed on how to perform an unsupervised vaginal HPV mRNA cytobrush collection (Aptima; Hologic, San Diego, CA, USA). At the time of the clinic visit, participants were instructed to insert a cytobrush into the vagina as far as possible, turn the brush five full rotations, and then place the brush into a vial of transport medium. Next, participants underwent a pelvic examination, during which clinicians collected a research cervical HPV mRNA cytobrush and a routine cervical sample for HPV and Pap co-test. The clinical laboratory used an assay that detected HPV DNA (Qiagen, Hilden, Germany), although both types of HPV test are commercially available. Participants were given a home collection kit that contained instructions on self-collection, one collection brush, storage containers, and a pre-addressed, postage-paid return cardboard envelope. Two weeks after the clinic visit, participants were reminded by a text message or phone call to self-collect at home and to mail the cytobrush back. Women were given a $20 gift card for their participation. Women with a positive result from any HPV test were phoned and advised to discuss the results with their clinicians for further guidance. AIDSinfo Guidelines for prevention and treatment of opportunistic infections in HIV-infected adults and adolescents (Panel on Opportunistic Infections in Adults and Adolescents with HIV, Panel on Opportunistic Infections in Adults and Adolescents with HIV) were used to identify women who required Colposcopy. Electronic medical records were abstracted to collect age, race, most recent HIV RNA concentration, CD4 T-cell count, antiretroviral therapy use, and smoking status. Due to the large proportion of women of Black race in the HIV clinic, race was classified to Black versus non-Black. In addition, we classified the most recent HIV RNA concentration into two groups: detectable (≥20 copies/ml) and undetectable (<20 copies/ml).

HPV DNA and mRNA assays

The clinician-collected (CC) cervical sample was tested for HPV DNA using a hybrid capture II (HC2) DNA nucleic acid hybridization assay that detected HPV DNA types 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, and 68. Positive samples had a relative light unit ≥1 pg/ml. Individual genotyping results were not available. Equivocal HPV DNA results were reclassified as positive (Knoepp et al., 2007). For research purposes, the self-collected (SC) and CC samples were tested for HPV using the Aptima transcription-mediated amplification RNA test that detected E6/E7 messenger RNA of the same HPV types as the HC2 assay plus type 66. Positive samples had signal-to-cutoff values of ≥0.5. Invalid HPV mRNA was not reclassified as there are no data available to inform the reclassification.

Survey

A 13-question survey was administered after self-collection at the clinic to assess the acceptability of self-collection. These questions assessed whether the patients (1) understood the instructions; (2) felt comfortable; (3) felt relaxed; (4) felt in control; (5) felt they were taking care of their health; (6) felt convenient; (7) felt anxious; (8) felt embarrassed; (9) felt it was painful; (10) worried that they might do the test wrong; (11) would use the cytobrush again; (12) would recommend the cytobrush to family and friends; and (13) would use the cytobrush in a clinic. A five-point Likert scale from strongly agree to strongly disagree was used. The survey also queried, “How easy or hard was it for you to collect the vaginal specimen using the brush?” with responses ranging from very easy, easy, and OK, to hard, or very hard.

Statistical analysis

To evaluate the ability of SC HPV mRNA tests to correctly identify HPV infection, sensitivity and specificity analyses with 95% confidence intervals (CI) were performed over 2 × 2 contingency tables using the CC HPV mRNA test as the comparator. The unweighted Cohen's kappa statistic was used to evaluate the level of agreement between SC and CC HPV mRNA tests. The McNemar test with Fisher's exact test was used to assess the significance of discordance. To examine the test–retest performance of the SC HPV mRNA tests, first (clinic) and repeated (home) test results were compared using the unweighted kappa statistic. The same method was used to compare the two CC HPV tests (mRNA test and DNA test). Kappa of 0.81–1.00 was interpreted as very good, 0.61–0.80 as good, 0.41–0.60 as moderate, 0.21–0.40 as fair, and <0.20 as poor (Altman, 1991). Based on a one-sided significance level of 0.05 and 80% power, 45 participants were needed to detect a kappa value >0.4, given that the expected kappa was 0.7. Participants with missing data or invalid test results were excluded from individual analyses. Statistics were performed using Stata version 14.1 (Stata Corp, College Station, TX, USA).

Results

Characteristics of participants

Seventy-five were women enrolled and four were excluded based on age or incomplete data. We also excluded one participant who had invalid results on CC HPV mRNA testing, leaving 70 evaluable participants. All of the 70 participants also underwent SC HPV mRNA tests at the clinic, with nine invalid samples (13%). Fifty-five participants performed home collection, with seven invalid samples (13%). Any HPV test positivity prevalence was 63% (95% CI 51–74%).

The median age of the participants was 50 years (interquartile range 41–56 years), and over 92% of participants were of Black race. Over half were current smokers (51%), the majority were taking antiretroviral therapy (94%), and 76% had an undetectable HIV RNA concentration. About 11% of the participants’ most recent CD4 T-cell count was <200 cells/μl (Table 1 ).

Table 1

Characteristics of the study population.

Characteristics	Total (N = 70)
Continuous	Range	Median (IQR)
Age (years)	30–66	50 (41–56)
Categorical	n	%
Race
Black	65	92.9
Non-Black	5	7.1
Smoking status
Current	36	51.4
Former	13	18.6
Never	21	30.0
On antiretroviral treatment
Yes	66	94.3
No	4	5.7
CD4 T cell count (cells/μl)
<200	8	11.4
≥200	62	88.6
HIV RNA
Detectable (>20 copies/ml)	17	24.3
Undetectable (<20 copies/ml)	53	75.7
Any HPV test positive	44	62.9

IQR, interquartile range; HPV, human papillomavirus.

Validity and reliability of HPV sample collection strategies

Using the CC HPV mRNA test as the comparator, the sensitivity of the SC HPV mRNA test was 84.6% (95% CI 65.1–95.6%) and specificity was 62.9% (95% CI 44.9%–78.5%) (Table 2 ). There was a difference in the proportion of positive versus negative results between these two HPV mRNA tests, with marginal significance under the McNemar test (p  = 0.049). The agreement between the CC HPV mRNA test and the SC HPV mRNA test was moderate, with a kappa value of 0.5 (95% CI 0.2–0.7). In addition, the reliability of the SC HPV mRNA tests (clinic versus home) was good, with a kappa value of 0.8 (95% CI 0.5–1.0). We performed a sensitivity analysis that included invalid test results as negative or positive. The reliability of SC HPV tests (clinic versus home) showed a good level of agreement if invalid results were included as negative results (κ  = 0.7, 95% CI 0.5–0.9), and a moderate level of agreement if invalid results were included as positive results (κ  = 0.5, 95% CI 0.3–0.7). The agreement between CC HPV mRNA tests and CC HPV DNA tests was good (κ  = 0.8, 95% CI 0.7–1.0).

Table 2

Validity and reliability of self-collected HPV testing.

Panel A: Validity of self-collected HPV mRNA tests
Clinic self-collected	Clinician-collected HPV mRNA tests			Kappa (95% CI)
	Positive	Negative	Total
Positive	22	13	35	0.5 (0.2–0.7)
Negative	4	22	26
Total	26	35	61
Sensitivity (95% CI), %	84.6 (65.1–95.6)
Specificity (95% CI), %	62.9 (45.0–78.5)
PPV (95% CI), %	62.9 (45.0–78.5)
NPV (95% CI), %	84.6 (65.1–95.6)

HPV, human papillomavirus; CI, confidence interval; PPV, positive predictive value; NPV, negative predictive value.

Self-collection acceptability survey

Sixty-one participants (87%) completed the survey. About 79% of women thought self-collection was easy. No participants described self-collection as hard or very hard. Almost all of the women (97%) agreed or strongly agreed that they understood the self-sampling instructions. Less than a third (27%) of participants felt anxious about self-sampling and felt it was painful (24%). Overall, 79.0% agreed that they would use the cytobrush again. An overwhelming majority (90%) agreed that self-sampling was convenient and comfortable. Additionally, 86% of participants would recommend the cytobrush to family and friends (Figure 1 ).

Fig. 1

Survey results.

Discussion

This study found that 63% of WLHIV had an HPV infection, which is similar to other studies among WLHIV (Kojic et al., 2011) and much higher than the rate in women without HIV (2.5–4.2%) (Sargent et al., 2008). HIV-induced immunodeficiency is thought to impede HPV clearance, resulting in the persistence of HPV infection, along with an increased risk of cervical cancer and precancer (Denny et al., 2012). Thus, developing a widely acceptable and accessible screening strategy and guidelines to optimize cervical cancer prevention among WLHIV is crucial.

Compared to the clinician-collected HPV test, it was found that self-collected tests had good sensitivity, moderate specificity, and moderate agreement. This finding is in agreement with one previous study that showed moderate concordance of self-collected vaginal samples compared with clinician-collected cervical samples (Cho et al., 2019). In the present study, the test–retest performance of the self-collected strategy was good and the overall performance of self-collected tests was acceptable, although it was still inferior to tests using clinician-collected cervical samples. Findings from several studies have also revealed similar concerns. A large population-based cervical cancer screening study in China showed self-collected HPV testing sensitivity and specificity for detecting precancerous lesions was 86.2% and 80.7%, respectively, whereas clinician-collected HPV testing sensitivity and specificity were 97% and 82.7%, respectively (Zhao et al., 2012). Results from a meta-analysis of 36 studies examining self-collected versus clinician-collected samples (Arbyn et al., 2014) also reported lower pooled sensitivity and specificity of HPV testing on self-collected than clinician-collected samples.

The present study had a higher than expected rate of invalid testing in the self-collected samples compared to clinician-collected samples. However, the level of agreement between the first (clinic) and repeated (home) test results of self-collection was good, despite the invalid test results. These invalid test results may have been due to insufficient sample material (Engesæter et al., 2016). It is possible that participants did not adhere to the instructions to insert the brush into the vagina deep enough or did not rotate the brush enough times, which could have translated to lower sample yield. Additionally, a cytobrush was used for collection, which may have been uncomfortable to use and may have led to hesitancy with sample collection. A higher number of invalid tests were not reflected in other self-collection studies that have been done on average-risk populations (Saville et al., 2018, Smith et al., 2018). However, these studies used HPV DNA tests and different collection tools. Also, compared to self-collected samples, clinician-collected samples likely had greater yield since clinicians are experienced with genital sample collection and performed it under speculum examination. Lower sample material could be a drawback with self-collected methods, but it is potentially associated with the type of collection tools or assay used. Future studies should investigate the impact of different collection devices and techniques on assay test performance similar to studies that have been performed for other female genital tract analyses (Dezzutti et al., 2011).

Another important finding of this study was the high acceptance of self-collected HPV testing among WLHIV. Women had positive attitudes toward this testing strategy and thought testing was easy. Also, they felt they were taking care of themselves and would use it again. This indicates that the self-collected HPV testing strategy may be able to increase cervical cancer screening, if the purpose of HPV testing is made clear to patients. For example, one of the largest randomized clinical trials in the US of home HPV testing reported only 12% of participants returned the home HPV test kit, and there was no difference in detection of precancerous lesions compared to usual care (Winer et al., 2019). The poor kit return in this trial might be explained by the patients’ lack of knowledge regarding the superior performance of HPV testing, the requirement to still attend the clinic for usual care, or the awareness that the results could not replace usual care screening methods (Moss et al., 2019).

The present study builds on this literature by describing HPV self-collection with a specific focus on an underserved urban population with a higher risk of cervical cancer. Moreover, unlike a large-scale study testing its hypothesis under ideal highly controlled conditions, this study examined the ‘real-world’ experience of HPV self-collection among WLHIV. An earlier study from our clinic found that WLHIV were less likely to come to gynecology visits as compared to primary HIV care visits, and one of several reasons was fear or discomfort associated with the pelvic examination (Tello et al., 2010). Given that data are insufficient to recommend pelvic examinations in the absence of symptoms (US Preventive Services Task Force et al., 2017) and the trend of making pelvic examinations contingent on medical history or symptoms (ACOG, 2018), the rationale for HPV self-collection at home or at non-gynecology clinic sites is strengthened. HPV self-collection could result in more WLHIV being screened for cervical cancer, which may justify its slightly inferior performance compared to clinician-collected sampling methods. Self-collected HPV testing could be a particularly effective strategy to reach underserved women, especially those living in health professional shortage areas (HPSAs) such as rural areas or those with barriers to attending medical appointments like lack of transportation or work/child care responsibilities.

This study has some limitations. First, the study population was relatively small, which limited the certainty of interpreting the results. However, the study was adequately powered and provided important evidence for future study of primary HPV cervical cancer screening strategies in WLHIV. Second, this study was limited by a higher than expected number of invalid results, although they did not appear to greatly impact the reliability of the tests. Last, the study population was predominately Black WLHIV enrolled at one urban site, which may not represent all WLHIV. However, Black patients accounted for 42% of the new HIV diagnoses in the US (CDC, 2019) and the population is a reflection of the nation's racial disparity.

In conclusion, the examination of self-collection highlights the specific needs of this population of US WLHIV and the opportunity to improve HPV screening in high-resource settings. Compared to the clinician-collected HPV test, the self-collected HPV test had good sensitivity and moderate specificity among WLHIV. Due to the possible inadequate HPV sample material resulting from self-collection, future studies should focus on how to improve self-collection techniques (including urine-based testing), since it is highly accepted by women and may improve cervical cancer screening in WLHIV who are at higher risk of cervical cancer.

Ethical approval

This study was approved by Johns Hopkins Institutional Review Boards.

Funding

This study was funded by grants from the (grant number 90042344).

Conflict of interest

Hologic Company donated HPV mRNA test kits for this study.