The Accuracy of Anal Swab-Based Tests to Detect High-Grade Anal Intraepithelial Neoplasia in HIV-Infected Patients: A Systematic Review and Meta-analysis.

BACKGROUND: The incidence of high-risk human papillomavirus (HR-HPV)-induced anal cancer is increasingly problematic among HIV-positive patients. Anal cancer is preceded by precursor lesions, anal intraepithelial neoplasia (AIN). AIN detection requires high-resolution anoscopy, a cumbersome and time-consuming procedure. We aggregated evidence on anal swab-based tests to detect AIN in HIV-positive patients.
METHODS: We searched MEDLINE and EMBASE for cross-sectional studies on AIN detection with anal cytology, HR-HPV DNA detection, HPV E6/E7 mRNA analysis, and P16INK4a and Ki-67 immunostaining. Summary estimates of sensitivity and specificity were calculated using bivariate logistic regression. Cytology was reported using the terms squamous intra-epithelial lesion (SIL) for AIN and high-grade SIL (HSIL) for high-grade AIN (HGAIN).
RESULTS: We included 22 studies. Using cytology with a cutoff of any SIL to detect HGAIN, we detected a sensitivity of 82% (95% CI, 74%-87%) and specificity of 45% (95% CI, 44%-66%); with the cutoff of HSIL, the sensitivity was 44% (95% CI, 45%-67%) and the specificity was 79% (95% CI, 69%-87%). The sensitivity of HPV DNA to detect HGAIN was 91% (95% CI, 82%-95%) and the specificity was 27% (95% CI, 21%-33%). For MSM, the positive predictive value (PPV) of cytology with a cutoff of any SIL was 36% (95% CI, 23%-50%) and the negative predictive value (NPV) was 87% (95% CI, 78%-93%), whereas cytology with a cutoff of HSIL had a PPV of 62% (95% CI, 50%-73%) and an NPV of 78% (95% CI, 65%-87%). The PPV of HR-HPV DNA detection was 37% (95% CI, 20%-57%) and the NPV was 87% (95% CI, 79%-93%).
CONCLUSIONS: Given its sensitivity, cytology with a cutoff of any SIL could be considered as a triaging method, whereas cytology with a cutoff of HSIL had better specificity and could be used for quality assurance. HR-HPV DNA detection had poor specificity and PPV, making it unsuitable for triage.

Human papillomavirus (HPV)–induced anal cancer is rare among the general population. However, there is a higher and increasing incidence among HIV-positive men who have sex with men (MSM) [1]. More than 90% of anal cancers are caused by high-risk (HR) HPV genotypes [2], of which HPV-16 is the most common [3]. In HIV-positive MSM, anal cancer is also frequently caused by other HPV types than HPV-16 [3].

Anal cancer is preceded by precursor lesions, called squamous intraepithelial lesions (SILs) or anal intraepithelial neoplasm (AIN). SIL and AIN are subdivided in stages according to malignant potent into low-grade (LSIL or LGAIN) and high-grade (HSIL or HGAIN). For cytological anal samples, the Bethesda grading system (designed for cervical precursor lesions) is used (Table 1) [4]. For histology, the LAST criteria recommended use of the terminology LSIL and HSIL, which may be further classified by AIN subcategorization. For clarity reasons, we further report cytology as LSIL and HSIL, and histology as LGAIN or HGAIN [5]. A systematic review by Machalek et al. found a 29.1% pooled prevalence of HGAIN in HIV-positive MSM [1]. They calculated a theoretical progression rate of HGAIN to anal cancer of about 1 in 600 per year in HIV-positive MSM, and roughly 1 in 4000 per year in HIV-negative MSM. Based on 2 retrospective studies, progression rates from HGAIN to anal cancer are estimated at between 0.16% and 2.8% after 2 years and between 0.6% and 5.6% after 5 years [6, 7]. Therefore, both the International Anal Neoplasia Society (IANS) and the European AIDS Clinical Society recommend routine AIN screening for at-risk populations. Most experts favor treatment if HGAIN is detected. The American CDC Sexually Transmitted Diseases treatment guideline, on the other hand, does not yet recommend screening [8-10].

Table 1. 

Terminology for Grading Anal Dysplasia for Cytology and Histology

	Bethesda, Used for Cytology [4]	LAST, Used for Histology [5]
No dysplasia	No dysplasia	No dysplasia
Atypia	ASC-US
	ASC-H
Mild dysplasia	LSIL	AIN1	LGAIN
Severe dysplasia	HSIL	AIN2	HGAIN
		AIN3
Squamous cell carcinoma	SCC	SCC

Abbreviations: AIN, anal intraepithelial neoplasia; ASC-H, atypical cells cannot exclude HSIL; ASC-US, atypical cells of unknown significance; HGAIN, high-grade anal intraepithelial neoplasia; HSIL, high-grade squamous intraepithelial lesion; LAST, Lower Anogenital Squamous Terminology; LGAIN, low-grade anal intraepithelial neoplasia; LSIL, low-grade squamous intraepithelial lesion; SCC, squamous cell carcinoma.

High-resolution anoscopy (HRA) is currently seen as the gold standard for detection of HGAIN. However, it is a cumbersome and time-consuming screening method [8]. In contrast to HRA, anal mucous cytological brush or swab samples are obtained more easily and can potentially be used as an alternative method to detect SIL lesions [11]. They can also be used as triaging tool to identify those that require HRA before the procedure [9, 12, 13] or as a quality measure for HRA after the procedure to identify possible HGAIN missed by HRA [14]. In our HRA screening setting in Amsterdam, we use anal cytology as a quality assurance marker, whereas in many US-based settings cytology is used as a triaging tool to indicate patients for HRA. With conventional cytology, samples are smeared onto a glass slide, and cells are stained for light microscopic visualization. With liquid-based cytology, samples are eluted in fixative, centrifuged, and collected on a small surface for visualization [11].

Preselection methods to indicate biopsies to exclude HGAIN are DNA detection of high-risk HPV (HR-HPV) and using biomarkers such as p16INK4A, Ki-67, and HPVE6/E7-mRNA. P16INK4A and Ki-67 expression are seen in transforming HPV infections, characterized by growing expression of HPV oncogenes and dysregulation of cells [15, 16]. E6 and E7 are oncoproteins that enable malignant transformation [15, 16].

Here, we performed a systematic literature review and meta-analysis to estimate the accuracy of anal cytology and HR-HPV DNA detection in anal swabs to screen for HGAIN in HIV-positive patients. We also evaluated the added value of biomarker analysis performed on swabs.

METHODS

This systematic review was registered in the PROSPERO systematic review registry (CRD42016051029) (Supplementary Table 1) [17]. We included single-gate cross-sectional studies on the accuracy of tests on anal swabs to diagnose HGAIN in HIV-positive patients, with histopathology results of HRA-guided anal biopsies or quadrant biopsies as the end point (using the LAST criteria) (Table 1) [5]. The time interval between the index test (cytology) and reference test (histopathology) had to be shorter than 6 months. We included studies describing 10 or more HIV-positive participants and at least 1 of the following index tests: anal cytology (conventional or liquid-based), HR-HPV DNA detection, HR-HPV E6/E7 mRNA analysis, P16INK4A and/or Ki-67 staining on cytology material, or possible other biomarkers. The comparison between the index test and reference standard had to be fully paired for the included studies. Single-gate studies (as defined by Cochrane) were excluded [18]. Longitudinal studies that performed the index test and reference standard at t = 0 were also included. In case of multiple publications of 1 data set, the most recently published source was included. We attempted to obtain missing data by aggregating data from other articles describing the same study population and through inquiries to the corresponding authors.

A medical information specialist (J.L.) performed a systematic search in OVID MEDLINE and OVID EMBASE from inception to May 15, 2018, to identify studies fulfilling the inclusion criteria (Supplementary table 2). Both MesH terms and text words were used, without language or other restrictions. The search consisted of 2 parts. In the first part, terms for HIV/HIV risk groups/sexual behavior or diagnostic accuracy (HIV risk broad) were combined with terms for anal cytology; in the second part, terms for HIV/HIV risk groups (HIV risk narrow) were combined with anal (pre-)cancer stages and cytology, anal swabs, or anal sampling. Records from I or II that did not mention HIV in the title, abstract, or MESH were separately analyzed in full text to check for inclusion of HIV-positive persons. We cross-checked the reference lists and the cited articles in the identified relevant papers and adapted the search in case of additional relevant studies. The bibliographic records retrieved were imported and de-duplicated in ENDNOTE.

Two authors (F.D.G.L. and J.D.V.) independently screened all identified records on title and abstract using the web application RAYYAN (rayyan.qcri.org) [19]. If the study potentially met all in- and exclusion criteria based on the abstract, the full text was downloaded for a definite eligibility check.

The primary outcomes were sensitivity and specificity of the index tests to predict HGAIN. Other outcomes were positive predictive value (PPV) and negative predictive value (NPV) of the index test for each subpopulation. For cytology, we used the following cutoff points: any SIL (containing atypical cells of unknown significance [ASC-US], LSIL, atypical cells cannot exclude HSIL [ASC-H], and HSIL) and HSIL (containing ASC-H and HSIL). For HR-HPV DNA detection and HR-HPV E6/E7 mRNA positivity, at least 1 of the oncogenic HPV types (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68) had to be detected. For biomarker p16INK4A and Ki-67 positivity, at least 1 stained cell for p16INK4A or Ki-67 had to be detected.

The QUADAS-2 checklist was used to assess the risk of bias in 4 domains: patient selection, index test, reference standard, and flow and timing [20]. QUADAS-2 also assesses the external validity of studies. The number of patients, population characteristics, study site, type of anal cytology test, types of HPV screened, and type of biomarker were extracted. Additionally, we extracted the numbers of true positives, false positives, false negatives, and true negatives defined at the considered thresholds, and eventually the sensitivity, specificity, positive predictive value, negative predictive value, and diagnostic odds ratio. When 2 index tests were compared, we also extracted the used measure of comparison with the P value.

We used the program R and the package Mada for the statistical analysis. We estimated the summary sensitivity and specificity of anal cytology and HR-HPV DNA detection by performing a meta-analysis using the bivariate model of Reitsma et al. [21]. Using the same method, we calculated the accuracy for subpopulations: MSM, men who have sex with women (MSW), women, and drug users. We included women with HIV overall instead of WSM only as the remaining population of WSW is known to be at extremely low risk for HIV. All tests were considered dichotomous variables. We only performed meta-analysis if more than 4 studies could be included in the analysis.

RESULTS

Study Description

We identified 707 articles and excluded 491 articles by title or abstract (Figure 1). After reading the full text, another 194 articles were excluded. Finally, 22 studies were included (Supplementary Table 3) [22-43]. Twenty-one studies described cytology [22–41, 43]. Seventeen described liquid-based cytology [22–31, 33, 35, 38–41, 43], 3 conventional cytology [32, 36, 37] and 1 both conventional and liquid-based cytology [34]. Eleven studies described HR-HPV DNA detection [23–25, 27, 30, 31, 33, 38–40, 43], 3 HPV E6/E7 mRNA analysis [38, 40, 44], 3 p16INK4a immunostaining [38, 39, 42], and 1 p16INK4a/Ki-67 dual staining [41]. All studies were cross-sectional and prospective, except for 1, which was retrospective [35].

Figure 1.

Flowchart of retrieval, selection, and exclusion of studies on the accuracy of anal swab–based tests for detection of high-grade anal intraepithelial neoplasia in HIV-infected patients. Abbreviations: HPV, human papillomavirus; HRA, high-resolution anoscopy.

Methodological Quality of Included Studies

A summary of the overall methodological quality is shown in Figure 2 and Supplementary Table 4. A more detailed summary of the methodological quality of each study can be found in Supplementary Table 5. Many studies scored “unclear” on the different domains because they did not provide sufficient information about the methodology. Overall, the concerns for applicability across studies were low.

Figure 2.

Methodological quality graph: authors’ judgments of each QUADAS-2 item, presented as percentage across all included studies on the accuracy of anal swab–based tests for detection of high-grade anal intraepithelial neoplasia in HIV-infected patients. Risk of bias left, concerns for applicability right. Abbreviations: HIGH, high risk of bias; LOW, low risk of bias; UNCLEAR, risk of bias is unclear.

Findings

The test characteristics for the detection of HSIL of each index test are shown in Supplementary Table 6. Table 2 shows the results of the meta-analysis. The prevalence of HGAIN in the total population of this review was 23%. The estimated SROC plot is found in Figure 3. The 2 × 2 tables and the syntax of the meta-analysis are found in Supplementary Table 7.

Table 2.

Results of Meta-analyses on the Accuracy of Anal Swab–Based Tests for Detection of High-Grade Anal Intraepithelial Neoplasia in HIV-Infected Patients

Population	Test	Threshold/HPV Types	Studies (No.; Prevalence HGAIN)	Summary Estimate of Sensitivity, %	(95% CI)	Summary Estimate of Specificity, %	(95% CI)	Summary Estimate of PPV, %	(95% CI)	Summary Estimate of NPV, %	(95% CI)
A, All HIV+ patients	Anal cytology	Any SIL	18 (4231; 23%)	82	(74–87)	45	(44–66)
		HSIL	17 (3578; 25%)	44	(45–67)	79	(69–87)
	HPV-DNA	HR-HPV	9 (2051; 23%)	91	(82–95)	27	(21–33)
B, HIV+ MSM	Anal cytology	Any SIL	12 (2825; 21%)	83	(72–90)	45	(31–60)	36	(23–50)	87	(78–93)
		HSIL	11 (2172; 23%)	30	(19–43)	94	(89–97)	62	(50–73)	78	(65–87)
	HPV-DNA	HR-HPV	8 (1855; 25%)	91	(80–96)	27	(21–35)	37	(20–57)	87	(79–93)

Abbreviations: CI, confidence interval; HGAIN, high-grade anal intraepithelial neoplasia; HIV+, HIV-positive; HPV, human papillomavirus; HR-HPV, high-risk HPV types; HSIL, high-grade squamous intraepithelial lesion; MSM, men who have sex with men; NPV, negative predictive value; PPV, positive predictive value; SIL, squamous intraepithelial lesion.

Figure 3.

Summary receiver operating characteristic (SROC) of anal cytology with test cutoff of any squamous intra-epithelial lesion (right), test cutoff of high-grade squamous intra-epithelial lesion (right), and human papillomavirus detection (under) for detection of high-grade anal intraepithelial neoplasia in HIV-infected patients. The dots represent the sensitivities and specificities found in the included studies. The line represents the estimated SROC curve. The dashed line around it represents the 95% confidence interval of the estimated SROC curve. The circle represents the estimated pooled sensitivity and specificity, and the ellipse represents its 95% confidence interval.

Eighteen studies used cytology with any SIL as threshold to detect HGAIN. The summary estimate of sensitivity in these studies was 82% (95% confidence interval [CI], 74%–87%), and the specificity was 45% (95% CI, 44%–66%). Using HSIL as the threshold (17 studies), the summary sensitivity was 44% (95% CI, 45%–67%), and the specificity was 79% (95% CI, 69%–87%).

The summary estimate of the sensitivity and specificity of HR-HPV DNA detection was, respectively, 91% (95% CI, 82%–95%) and 27% (95% CI, 21%–33%), based on 9 studies.

Limiting the number of genotyped HR-HPV types to 16 and 18 only resulted in decreased sensitivity and increased specificity. However, no study found that the proportion of correctly classified cases increased [23–25, 33, 39, 43, 45].

Five studies combined cytology with HR-HPV DNA detection. If either SIL or HR-HPV was detected, this was defined as a positive outcome. Using the combination of cytology and HR-HPV DNA detection to detect HGAIN resulted in sensitivities ranging from 70% to 100% and specificities ranging from 18% to 41%. Overall, the sensitivity increased and the specificity decreased when combining both tests compared with performing each test alone. However, none of the studies found a higher proportion of correctly classified cases when both tests were combined [23, 24].

Two studies investigated HPV E6/E7 mRNA analysis of high-risk HPV types to detect HGAIN and found a sensitivity of 70% (95% CI, 51%–89%) [38] and 71% (95% CI, 62%–80%) [40] and a specificity of 56% (95% CI, 46%–66%) [38] and 56% (95% CI, 46%–65%) [38, 40]. Salit et al. found a sensitivity and specificity of E6/E7 mRNA detection of HPV type 16 of, respectively, 36% (95% CI, 25%–47%) and 83% (95% CI, 77%–89%) [39]. Phanuphak et al. investigated the accuracy of HR-HPV E6/E7 mRNA and anal cytology combined, defining any SIL or E6/E7 mRNA detection as a positive outcome. They found a sensitivity of 83% (95% CI, 68%–98%) and a specificity of 52% (95% CI, 42%–62%). The proportion of correctly classified cases was not significantly higher compared with anal cytology alone [38].

In 3 studies evaluating p16INK4a immunostaining on anal cytology, the sensitivity ranged from 23% to 61%, and the specificity ranged from 44% to 77% [38, 39, 42]. Combining cytology with p16/Ki-67 did not increase accuracy [27].

The sensitivity and specificity of cytology and HR-HPV DNA detection estimated in HIV-positive MSM were similar to the overall estimates (Table 2). Using cytology with any SIL as the threshold to detect HGAIN, the summary estimate of the PPV and NPV was, respectively, 36% (95% CI, 23%–50%) and 87% (95% CI, 78%–93%), based on 12 studies; using HSIL as the threshold, the estimate of PPV and NPV was, respectively, 62% (95% CI, 50%–73%) and 78% (95% CI, 65%–87%), based on 11 studies. For HR-HPV DNA detection, we found a summary estimate of PPV and NPV of, respectively, 37% (95% CI, 20%–57%) and 87% (95% CI, 79%–93%).

DISCUSSION

There is an urgent need for accurate, noninvasive, and affordable tools for anal dysplasia screening, either as a triaging tool or as a quality assurance method for HRA. Anal cytology and other tests on either self-collected or clinician-collected swabs are less burdensome for the patient than HRA. Here, we report a systematic review on the accuracy of anal cytology and other tests on anal swabs for the detection of HGAIN.

To detect HGAIN with any SIL as the cutoff, anal cytology had an 82% summary estimate of sensitivity and a 45% summary estimate of specificity; with HSIL as the cutoff, the summary sensitivity and specificity were, respectively, 44% and 79%. HR-HPV DNA detection had a 91% summary sensitivity and 27% specificity. For HIV-positive MSM, the sensitivity and specificity of anal cytology and HR-HPV DNA detection were similar to the overall sensitivity and specificity. We found hardly any data on the accuracy of the index tests in MSW, women, or transgender persons. The prevalence of HPV is in these populations is different than in the MSM population [3]. Therefore, we expect other PPV and NPV values for these populations. Combining cytology with HR-HPV DNA detection or HPV E6/E7 mRNA detection increased sensitivity and decreased specificity, but we found no evidence that any combination increased the overall diagnostic accuracy.

The use of cytology with any SIL as a cutoff level results in good sensitivity and moderate specificity, so it could work well as a triage method. This would require incorporation into a screening pathway with repeat examinations at defined intervals in order to reduce missed cases. Although the sensitivity is of importance for triage methods, for a quality assurance tool, the specificity is of concern. We showed that the use of cytology with HSIL as a cutoff level results in the detection of HGAIN with good specificity, and thus it is ideal for quality assurance purposes. The moderate sensitivity could be accepted, because cytology is very patient friendly, yet the high specificity prevents false-positive results, which burden the patient with unnecessary additional procedures such as repeat HRA. HR-HPV DNA detection has an even higher sensitivity. However, >70% of HIV-positive MSM are infected with HR-HPV types, and a considerable portion do not have HGAIN disease, making DNA detection a less powerful method to prevent unnecessary HRA procedures [46]. Although the current evidence does not show a significant improvement when cytology is combined with either HR-HPV DNA detection or HPV E6/E7 mRNA detection, future biomarkers could likely further improve sensitivity.

In a comparable study, Cachay et al. included 11 articles on the accuracy of anal cytology and found similar results to ours: a sensitivity of 90% (95% CI, 76%–96%) and a specificity of 33% (95% CI, 20%–49%) for a test cutoff of any SIL and a sensitivity of 30% (95% CI, 19%–44%) and specificity of 93% (95% CI, 90%–95%) for a test cutoff of HSIL. Any differences between the earlier work and our results could be explained by the inclusion of 13 additional studies dating after 2012 and the exclusion of 6 studies that were not fully paired or did not distinguish patients by their HIV serostatus [47]. Two earlier less rigorous meta-analyses on the accuracy of anal cytology by Chiao et al. and the Ontario Medical Advisory Secretariat Ministry of Health and Long-Term Care reported comparable results [48, 49].

One of the strengths of this review is that we included the highest number of studies to date (n = 19) that evaluated the accuracy of cytology and HR-HPV DNA detection. Furthermore, we studied a wide range of additional neoplasia markers that can be performed on cytological material.

One of the limitations of the study is the crude estimation of the accuracy of HR-HPV DNA detection; there was large heterogeneity in the materials and methods used. Another limitation is that the reference standard was not always uniformly defined. There was heterogeneity concerning the colposcope or endoscope magnification, the use of biomarker p16INK4a to evaluate biopsies, and whether the anoscopist biopsied only suspected lesions or also random nonlesional regions. Furthermore, only 2 studies quantified the amount of experience of the HRA anoscopist [23, 38]. We have previously shown that the quality of HRA depends significantly on the training and experience of the anoscopist [14, 50]. In addition, we could include only a few studies that estimated the accuracy of individual biomarkers and the combination of cytology with HR-HPV DNA detection or biomarkers. Lastly, the largest part of the patient sample was MSM. We found few data about the accuracy of cytology in other HIV-positive subgroups.

Combining cytology with the biomarker HPV E6/E7 mRNA likely increases the accuracy of anal cytology. However, as this is based on a small number of studies, additional evidence on the accuracy of HPV E6/E7 and other biomarkers is needed. New tests like viral DNA methylation, human leukocyte antigen subtypes, markers of lymphoproliferative response, telomerase amplification, human papillomavirus–induced epigenetic effects, and Ki-67, p53, and pRb show promising results in the detection of cervical precursor lesions and could also work for HGAIN screening [51]. Combining cytology and other tests on anal swabs in a multistep algorithm could eventually reduce the number of patients requiring HRA. Dupin et al. combined HPV E6/E7 mRNA with p16INK4a/Ki-67 and HR-HPV DNA detection, and thus ruled out a large part of patients for HRA. They claimed to maintain an acceptable sensitivity [52]. Future studies should also focus on the cost-effectiveness of additional biomarkers, as these tests are costly and thus might not outweigh the benefit of relatively inexpensive tests such as digital anorectal examination and anal cytology.

In conclusion, cytology with any SIL as a threshold has decent sensitivity but moderate specificity, which makes it suitable for triaging purposes. When HSIL is chosen as the threshold, cytology has moderate sensitivity but good specificity, making it more appropriate for quality assurance purposes. HR-HPV DNA detection has a high sensitivity, but as the majority of HIV-positive MSM are positive for HR-HPV, the specificity and PPV are too low for triaging purposes. The diagnostic accuracy of anal cytology did not increase significantly when combined with any currently available biomarker, albeit only small numbers of patients have been evaluated with any of these combinations. The large majority of the patients in this review were MSM, and we have too little data to estimate diagnostic accuracies in other HIV-infected populations. Further research needs to be done to discover new, accurate biomarkers and the best combinations of cytology and additional biomarkers.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Click here for additional data file.