Accuracy of Smartphone Images of the Cervix After Acetic Acid Application for Diagnosing Cervical Intraepithelial Neoplasia Grade 2 or Greater in Women With Positive Cervical Screening: A Systematic Review and Meta-Analysis.

PURPOSE: Smartphones are used in cervical screening for visual inspection after acetic acid or Lugol's iodine (VIA/VILI) application to capture and share images to improve the sensitivity and interobserver variability of VIA/VILI. We undertook a systematic review and meta-analysis assessing the diagnostic accuracy of smartphone images of the cervix at the time of VIA/VILI (termed S-VIA) in the detection of precancerous lesions in women undergoing cervical screening.
METHODS: This systematic review was conducted in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Studies from January 1, 2010, to June 30, 2020, were assessed. MEDLINE/PubMed, Embase, CINAHL, Cochrane, and LILACS were searched. Cohort and cross-sectional studies were considered. S-VIA was compared with the reference standard of histopathology. We excluded studies where additional technology was added to the smartphone including artificial intelligence, enhanced visual assessment, and other algorithms to automatically diagnose precancerous lesions. The primary outcome was the accuracy of S-VIA for the diagnosis of cervical intraepithelial neoplasia grade 2 or greater (CIN 2+). Data were extracted, and we plotted the sensitivity, specificity, negative predictive value, and positive predictive value of S-VIA using forest plots. This study was prospectively registered with The International Prospective Register of Systematic Reviews:CRD42020204024.
RESULTS: Six thousand three studies were screened, 71 full texts assessed, and eight studies met criteria for inclusion, with six included in the final meta-analysis. The sensitivity of S-VIA for the diagnosis of CIN 2+ was 74.56% (95% CI, 70.16 to 78.95; I2 61.30%), specificity was 61.75% (95% CI, 56.35 to 67.15; I2 95.00%), negative predictive value was 93.71% (95% CI, 92.81 to 94.61; I2 0%), and positive predictive value was 26.97% (95% CI, 24.13 to 29.81; I2 61.3%).
CONCLUSION: Our results suggest that S-VIA has accuracy in the detection of CIN 2+ and may provide additional support to health care providers delivering care in low-resource settings.

INTRODUCTION

Cervical cancer contributes significantly to the burden of noncommunicable disease in low- and middle-income countries (LMICs),[1] and the WHO has recently put out the call to eliminate cervical cancer by 2030.[2] In addition to scaling up human papillomavirus vaccination, screening programs remain a critical component of addressing the fact that cervical cancer is the leading cause of death from cancer among women in 36 LMICs.[3] Screening and subsequent adequate treatment remains a significant challenge in LMICs, where physical and human resources, infrastructure, cost, technology, and acceptability to women are all barriers to effectiveness.[4]

CONTEXT

Key Objective

Undertake a systematic review and meta-analysis assessing the diagnostic accuracy of smartphone images of the cervix at the time of visual inspection after acetic acid or Lugol's iodine (termed S-VIA) in the detection of precancerous lesions in women undergoing cervical screening.

Knowledge Generated

The sensitivity for the diagnosis of cervical intraepithelial neoplasia grade 2 or greater was 74.56%, specificity 61.75%, negative predictive value 93.71%, and positive predictive value 26.97%. Further research may also provide the appropriate platform for emerging technologies in cervical cancer screening, including the use of artificial intelligence, enhanced visual assessment, and other algorithms to automatically diagnose precancerous or cancerous lesions.

Relevance

The diagnostic accuracy of S-VIA potentially opens up a wealth of human resources to women in low- and middle-income countries, whereby access to expert colposcopists is not limited by geographic location. Moreover, the burden of service provision in many low- and middle-income countries falls to midlevel health care workers, and S-VIA has the potential to provide and increase support and training of providers.

In LMICs where cervical cancer screening does exist, it often takes the form of visual inspection of the cervix following application of acetic acid or Lugol's iodine (VIA or VILI).[5] As acetic acid is cheaper and more readily available, VIA is more commonly used and so will be referred to here on in while VIA is of low cost and gives a point of care result (positive or negative) allowing for immediate treatment.[6,7] Although sensitivity for VIA is reported at 79% (95% CI, 73 to 85) and specificity reported to be 85% (95% CI, 81 to 89),[8] it can be hampered in its use by poor reproducibility and heterogenous interobserver variability.

Nearly everyone now has access to a working mobile phone, and the use of smartphones is becoming increasingly ubiquitous in health care, including as a diagnostic tool (eg, smartphone applications for triaging skin lesions[9]). The capacity for smartphones to take high-quality images that can be used for diagnostic purposes and rapidly transmit those images are both features that can be exploited for the benefit of patient care. This includes in the setting of cervical cancer screening, where smartphones have been used to identify potential cervical lesions at the time of VIA, through the process of sharing images with colleagues and experts who may be remote from the patient, and as a training tool to improve the sensitivity and interobserver variability of VIA, particularly for the midlevel health care workers who provide the majority of cervical screening in LMICs.[10-18] The process of smartphone image capture after VIA shall be referred to from here on as S-VIA.

The use of smartphone technology to improve cervical cancer screening in LMICs has great potential to expand the available resource base to deliver screening, but it remains that these innovations must be clinically accurate. We undertook a systematic review and meta-analysis assessing the diagnostic test accuracy of smartphone images of the cervix at the time of acetic acid or Lugol's iodine application in the detection of cervical intraepithelial neoplasia grade 2 or more severe (CIN 2+) in women undergoing cervical screening or assessment.

METHODS

Search Strategy and Selection Criteria

This systematic review and meta-analysis was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Studies published or written from January 1, 2010, to June 30, 2020, with no language restriction were assessed. MEDLINE/PubMed, Embase, CINAHL, Cochrane library, and LILACS were searched. Customized search strategies on the basis of the key words cervix, cervical intraepithelial neoplasia, smartphone, smart phone, mobile phone, and colposcopy were developed and can be found in the Data Supplement. Reference lists from included full texts were individually searched. An additional search of Google scholar (the first 100 results from a search of cervix and smartphone) was undertaken. This study was prospectively registered with The International Prospective Register of Systematic Reviews, registration number CRD42020204024.

The primary outcome of interest was the sensitivity and specificity of S-VIA for the diagnosis of CIN 2+. Prospective studies that compared smartphone-obtained images of the cervix with a reference standard for the diagnosis of CIN 2+ were considered. The ideal study for the assessment of diagnostic test performance is cross-sectional where the use of smartphone assessment of the cervix is performed on consecutively assessed patients is cross-classified with histology. Although less ideal, we also considered randomized controlled trials that use previously independently assessed tests. The population considered was women undergoing S-VIA in community health clinics or hospital-based settings, across all income-level countries. S-VIA is used both in primary cervical screening (where S-VIA is the sole screening test for precancerous lesions) and as an assessment tool after triage with cervical human papillomavirus or cytology testing. Therefore, we made a pragmatic decision to include all patients undergoing S-VIA. The index test of interest was smartphone cervical imaging after application of acetic acid and/or Lugol's iodine, and the target condition was CIN 2+ (defined as cervical intraepithelial neoplasia grade 2 or 3, adenocarcinoma in situ, or invasive malignancy). The reference standard to assess the diagnostic accuracy was histopathology. Exclusion criteria were case reports, study protocols without available data, and commentaries. We also excluded studies where additional equipment or technology was added to the smartphone, including the use of artificial intelligence (AI), enhanced visual assessment, and other algorithms to automatically diagnose precancerous or cancerous lesions. In addition, studies that did not report the numbers of true-positives, false-positives, false-negatives, and true-negatives relative to the use of smartphone image capture for the diagnosis of CIN 2+ were excluded. The secondary outcomes were to assess the accuracy of smartphone images with other routinely used methods of cervical assessment, describe the potential barriers to implementation of the technology, including quality of the images, and assess the patient acceptability of smartphone use in cervical screening.

Data Analysis

Covidence software program was used to manage citations identified in the search. Three authors (E.R.A., N.P., and M.P.S.) independently assessed the titles and abstracts of all identified studies after duplicates were removed using a prepiloted series of screening questions. Full-text articles were then reviewed by the same three authors. A list of the irrelevant records is available upon request. Any differences in screening or data extraction were discussed and if they could not be resolved by E.R.A./N.P./M.P.S., then a fourth author (K.M.S.) was involved.

Assessment of quality and risk of bias was completed independently in triplicate (E.R.A./N.P./M.P.S.) using the Quality Assessment of Diagnostic Accuracy Studies 2 tool.[19] The quality assessment of studies followed the risk of bias and applicability concerns assessment and was used for the preplanned sensitivity analysis. These were considered high quality if all four criteria were met (low risk of bias across all four domains) medium quality if two or three criteria were met (low risk of bias across two to three domains), and low quality if one or no criteria were met (low risk of bias in one or no domains).

The sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV) of smartphone image capture of the cervix along with the 95% CIs were plotted using forest plots. We assessed the studies' heterogeneity using the I2 statistic described by Higgins et al,[20] which measures the percentage of total variation that is due to heterogeneity rather than chance. If a statistically significant percentage of the total variation was found to be due to heterogeneity, then the combined proportion from the studies in the meta-analysis was estimated using a random-effects model in which each study was weighted equally. We estimated potential publication bias using funnel plots. Symmetry in a funnel plot suggests that publication bias is not present. The vertical line in the funnel plot indicates the fixed-effects summary estimate. The meta-analysis was performed using the inverse variance-weighted average method. The other lines in the plot represent the 95% CI for a given standard error assuming no heterogeneity among studies. We planned a sensitivity analysis for the diagnostic accuracy (sensitivity and specificity) of smartphone image capture for CIN 2+ using the high-quality studies only (low risk of bias across all four Quality Assessment of Diagnostic Accuracy Studies 2 domains). All statistical analyses were performed using Stata/MP v16.0 (College Station, TX).

Patient and Public Involvement

As this is a systematic review and meta-analysis, there is no patient or public involvement in this study.

RESULTS

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart is shown in Figure 1. The search strategy returned 6,003 studies after 1,310 duplicates were removed. Seventy-one full-text articles were assessed, with eight studies meeting the criteria for inclusion in the final analysis.

FIG 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart. VIA, visual inspection after acetic acid.

The eight studies included 687 participants and were conducted in Japan, India, and Madagascar. All studies compared the diagnostic accuracy of smartphone VIA to the reference standard of histopathology. Smartphone images in all studies were shared with and reviewed by an expert remote from the patient. Experts were defined by the study authors, and where descriptions were provided, included specialist and specialist in training gynecologists. One study[14] appeared to have data that overlapped with a paper published by the same group, which was confirmed on contacting the first author, and thus, this paper was removed from any further analysis. The characteristics of the included final seven studies (the overlapping study is not included as the characteristics related to the same patients) are shown in Table 1. It should be noted that the study by Sharma et al[12] reported zero true positives and zero false negatives and so, while meeting the inclusion for the systematic review, was not included in the meta-analysis.

TABLE 1

Characteristics of the Included Studies

The diagnostic accuracy of S-VIA for the primary outcome of interest. Meta-analysis for this included 6,172 cervical images assessed from 426 participants across six studies. It should be noted that in two studies, it was assumed on the basis of the methodology that each set of smartphone images underwent unbiased and independent review compared with other reviewers[14,15] and so, we considered each of these as separate data points in the meta-analysis. For the diagnosis of CIN 2+, the sensitivity was 74.56% (95% CI, 70.16 to 78.95; I2 61.30%) and the specificity was 61.75% (95% CI, 56.35 to 67.15; I2 61.75%). The NPV and PPV were 93.71% (95% CI, 92.81 to 94.61) and 26.97% (95% CI, 24.13 to 29.81), respectively. The forest plots for sensitivity and specificity are represented in Figures 2 and 3, respectively. When only high-quality studies were considered,[14,15] the outcomes were similar; sensitivity 73.14% (95% CI, 68.72 to 77.57; I2 57.1%), specificity 62.98% (95% CI, 57.46 to 68.51; I2 95.1%), NPV 93.57% (95% CI, 92.65 to 94.49), and PPV 24.78% (95% CI, 22.69 to 26.87).

FIG 2

Annotated forest plot of sensitivity of smartphone images of the cervix at the time of visual inspection after acetic acid or Lugol's iodine for cervical intraepithelial neoplasia grade 2 or greater (note multiple data points from repeated blinded review of images by different observers in the Ricard-Gauthier et al[15] and Tran et al[14] trials). Weights are from random-effects analysis. ES, effect size.

FIG 3

Annotated forest plot of specificity of smartphone images of the cervix at the time of visual inspection after acetic acid or Lugol's iodine for cervical intraepithelial neoplasia grade 2 or greater (note multiple data points from repeated blinded review of images by different observers in the Ricard-Gauthier et al[15] and Tran et al[14] trials). Weights are from random-effects analysis. ES, effect size.

Only one study reported the accuracy of colposcopy for the outcome of CIN 2+,[21] and three reported the accuracy of VIA.[12,15,16] Given the few studies and the small numbers, meta-analysis for this secondary outcome was not undertaken. Patient acceptability was not reported on in any included study. The quality of a small number of images was low or insufficient to use in some studies.[12,14,16,21,23] Other barriers to implementing S-VIA reported included difficulty supervising midlevel practitioners in the process, as well as supporting them to retain skills, and the short duration of training given to some providers before implementation.[12]

The risk of bias is represented in Figures 4 and 5.[24] Four studies had an unclear or high risk of bias in the domain of patient selection, because of lack of reporting about the process of patient selection and inclusion/exclusion criteria.[12,21-23] There was largely a low risk of bias in relation to the index test, although in two studies, there was an unclear risk as the process of reviewing the index test was not adequately described or the index test was interpreted with knowledge of a comparison test.[16,21] All included studies used the reference standard of histopathology, which was interpreted without knowledge of the index test. Studies that had a high risk of bias did so because the reference standard was not performed on all included patients.[12,21]

FIG 4

Distribution of risk of bias assessment in the included studies.

FIG 5

Domain-level judgments for each component of the Quality Assessment of Diagnostic Accuracy Studies 2 assessment.

Funnel plots for the primary outcomes (derived from the six studies representing 588 participants) are available in the Data Supplement. There is substantial bias present, which may be a consequence of the small sample sizes in the included studies, or the heterogeneity within the meta-analysis.

DISCUSSION

In this systematic review and meta-analysis, we report on the diagnostic accuracy of S-VIA for the outcome of CIN 2+, with a sensitivity of 74.56% and specificity of 61.75%. This finding is comparable to the seminal trial comparing VIA to cytology, whereby more than 10,000 women had concurrent testing with cytology and VIA, with a reported sensitivity of VIA of 76.7% and specificity of 64.1%.[25]

The strengths of this study include the strict inclusion criteria and broad search terms. The applicability of the results is strengthened by our decision to focus only on the accuracy of smartphone assessment of the cervix, meaning that any study using a digital camera in a similar process was excluded. This decision was made to avoid any issues with variances between image quality in the two modalities and to increase the applicability of the findings, given the extent of global mobile phone ownership.

There are limitations in this study. The results for the primary outcome are derived from a relatively small number of studies (and included participants) for which the heterogeneity is high, and for which apparent publication bias exists. The possible sources for this asymmetry include selection biases (publication bias or selective outcome reporting), poor methodologic quality leading to spuriously inflated effects in smaller studies, true heterogeneity, artifact, and chance. We also note the lack of reporting on degree of image magnification that may have been used as part of the intrinsic function of the smartphone devices and therefore cannot comment on how this may have affected any outcome. There was a pragmatic decision to include all women undergoing S-VIA; however, the variability in cervical screening testing before undergoing S-VIA may have led to an overestimation in sensitivity. Furthermore, we have in two studies assumed repetitive blinded review of smartphone images as separate data points in the meta-analysis and this may have influenced the primary outcome. All images were reviewed by specialist and specialist in training gynecologists; however, there is no description or assessment of colposcopic experience, which may be heterogenous, and may potentially affect the primary outcome.

Despite these limitations, the addition of a smartphone image capture at the time of standard VIA has several potential applications. Outside of trial settings, the poorer reported sensitivity and specificity in implementation of VIA screening programs may lead to both undertreatment and overtreatment.[26-28] This variation may be a consequence of the lack of quality control systems for VIA screening programs, issues with adequate initial and ongoing training in the application of VIA, or limitations of the test itself (ie, VIA has limited capacity to identify endocervical disease).[28] S-VIA may in part overcome some of these issues. VIA is largely provided by midlevel health care workers, and the addition of S-VIA to this scenario would allow for expert colposcopists to provide support from afar, which may both improve on the accuracy of the screening test and deliver midlevel health care providers with ongoing training and support to deliver high-quality care.[29] Moreover, if S-VIA had the potential to improve upon the sensitivity of VIA (and thereby decrease overtreatment), then there may be scope to reduce the considerable strain on resources in LMIC, which is particularly relevant when 16.8% of women (range, 11%-23.6%) presenting for screening have a positive result with VIA.[30]

The concept of digital image capture at the time of VIA is not novel; in a variety of contemporaneous studies, digital photography (sometimes called cervicography) has been found to have a sensitivity of 46%-97% and 92%-97% for the detection of cervical dysplasia.[31-33] That said, approaches to VIA with digital images have traditionally required additional resources (eg, additional technologies such as a pocket colposcope[34,35] or a digital camera and computer to upload images to[36]) beyond what is as easily or readily available as a smartphone. Moreover, the concept of subsequently transmitting images for review by experts remote from the patient has been successfully demonstrated in cervical cancer screening. Firnharber et al[37] showed that in women with HIV, cervical photography reviewed by an expert clinician remote from the patient improved upon the sensitivity of VIA alone (65%-75%).[37] Similarly, Liu et al[38] conducted a study over 2 years whereby digital colposcopic images were uploaded to an internet-based system and the rate of detection of high-grade cervical dysplasia was compared in the year before and after using the image sharing system. While detection rates increased with use of the access to remote experts, issues with delays in feedback and diagnoses were identified. S-VIA may be a solution to refining and streamlining this process.

Although the next step in the use of smartphones in cervical cancer screening is the use of deep learning and AI algorithms to allow instant results,[39] these are not yet approved for routine clinical application. The very clear current potential application of smartphone image capture of the cervix is the capacity to transmit the basic image of the cervix after application of acetic acid or Lugol's iodine without the addition of any AI to a colposcopic expert remote from the woman and the health care worker undertaking the procedure. The practicality of this process was demonstrated in a study in Madagascar where images captured with a smartphone were assessed by three expert colposcopists remote from the clinic setting and were found in 93% of cases to be appropriate-quality photos with diagnostic utility.[18] The taking of the images is not the critical step in diagnosis, but rather the sharing of them; what we have demonstrated in this meta-analysis is that image sharing rather than real-time assessment of the cervix does not worsen the diagnostic outcome for the woman.

While we undertook to assess diagnostic accuracy, large scale-up S-VIA has been done by Yeates et al,[29] who undertook S-VIA in more than 10,000 women. More than 99% of the images were reviewed by an off-site expert, and VIA-positive results improved with the addition of a smartphone in cervical screening. Although there was no correlation with histology to demonstrate the diagnostic accuracy of the S-VIA in this study, the demonstrable scale up of the program is notable. Although the ubiquity of phone ownership may make the implementation of S-VIA possible, any medical intervention must have acceptability to the women undergoing cervical screening, and the confidentiality aspects of the process must be considered. None of the included studies in this meta-analysis considered patient acceptability. Although the application was different, one study looking at the use of mobile phones to improve adherence with cervical cancer screening in South Africa found that 98% of women enrolled owned a phone and could potentially participate in a phone-based program; however, reasonable concerns regarding privacy were raised,[40] and it is clear this would need to be addressed in any plan to use smartphone cervical assessment, in particular if images are to be transmitted elsewhere.

In conclusion, this systematic review and meta-analysis affirms the diagnostic accuracy of S-VIA for the detection of CIN 2+. In addition, S-VIA appears practical and applicable. The prevention of cervical cancer in LMICs remains a critical global health priority, and the use and scale up of S-VIA may allow for the accurate detection of CIN 2+ alongside support and training of the health care providers delivering screening in LMICs.