Predictive value of 'Smartscopy' for the detection of preinvasive cervical lesions during the COVID-19 pandemic: a diagnostic study.

OBJECTIVE: To evaluate the performance of "Smartscopy" in diagnosing preinvasive cervical lesions among patients with abnormal cervical cancer screening results obtained during the coronavirus disease 2019 (COVID-19) pandemic.
METHODS: This diagnostic study enrolled non-pregnant women with abnormal cervical cancer screening results obtained at the colposcopy clinic at Srinagarind Hospital (Khon Kaen, Thailand) between September 2020 and March 2021. Two colposcopists independently evaluated the uterine cervix using a smartphone and colposcopy. Cervical biopsies and endocervical curettage were performed in accordance with standard procedures. The diagnostic performance of a smartphone in detecting low-grade squamous intraepithelial lesions or worse plus (LSIL+) and high-grade squamous intraepithelial lesions plus (HSIL+) was assessed.
RESULTS: In total, 247 patients were included. There was high agreement between the two colposcopists (κ=0.88; 95% confidence interval [CI], 0.82-0.93). The sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of the smartphone to detect LSIL+ were 96.6% (95% CI, 91.6-99.1), 12.9% (95% CI, 8.06-19.2), 46.2% (95% CI, 39.7-52.4), 83.3% (95% CI, 62.6-95.3), and 0.49% (95% CI, 0.43-0.55), respectively. The sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of smartscopy in diagnosing HSIL+ were 67.6% (95% CI, 55.2- 78.5), 85.4% (95% CI, 79.9-90.0), 60.5% (95% CI, 48.6-71.6), 88.9% (95% CI, 83.7-92.9), and 81.0% (95% CI, 0.75-0.85), respectively.
CONCLUSION: Smartscopy demonstrated a remarkable correlation with colposcopy and a high diagnostic performance value for the detection of preinvasive cervical lesions. Therefore, smartscopy may be an alternative tool for detecting abnormal cervical lesions in low to medium medical resource settings. Smartscopy may be applied in telemedicine during the COVID-19 pandemic.

Introduction

Cervical cancer is the fourth most common cancer among women and the main cause of death among women worldwide [1,2]. In low-and middle-income countries, cervical cancer remains prevalent and accounts for the majority (90%) of deaths [3]. Moreover, cervical cancer continues to be a major public health concern [4].

Cervical cytology and/or human papillomavirus (HPV) testing are screening methods for detecting cervical abnormalities-which, if treated in a timely manner-can reduce the incidence and mortality of cervical cancer [5]. In 2013, the World Health Organization (WHO) recommended cervical screening in low-resource settings using visual inspection with acetic acid (VIA) or HPV testing, and immediate treatment to eradicate disease and prevent loss to follow-up. However, a limitation of VIA screening is the variation in sensitivity (41–92%) caused by the inconsistent quality of personnel training and experience [6]. To eliminate cervical cancer, the WHO recommended HPV testing in 2018 as a primary screening tool, which is feasible for self-sampling [7].

Colposcopy was introduced in 1925. This method is performed after detecting abnormalities in cervical screening tests. Cervical evaluation of malignancy and preinvasive lesions for definitive treatment is an advantage of colposcopy. Moreover, it is a crucial tool for the visualization and treatment of high-grade intraepithelial cervical lesions [8]. The sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV) of colposcopy for detecting preinvasive and cervical cancer by experienced surgeons have been reported to be approximately 92%, 67%, 96%, and 52%, respectively [9]. Colposcopy yields high sensitivity in high-grade lesions but low sensitivity in low-grade lesions [10]. Due to the limited availability of colposcopy in low-resource settings and the coronavirus 2019 (COVID-19) pandemic, it may be the primary barrier to accessing health care and, thus, delaying treatment. Capturing images using a “Smartphone” is an alternative option in this context [11].

Currently, smartphones are being used in medical care. The development of high-resolution cameras and user-friendly interfaces in smartphones is widely applicable, has become more popular in telemedicine, and can be used instead of colposcopy [11-13]. This approach also helps maintain distancing and reduce pandemic exposure of medical professionals and patients [13]. One study was limited in evaluating the utility of smartphones due to small sample size [14]. Another study evaluated smartphones for telemedicine [15] or revealed only cervical intraepithelial neoplasia (CIN) 2+ detection [16]. The diagnostic performance of smartphones in detecting preinvasive cervical lesions remains limited [14-16]. However, we believe that recent advances in technology will improve the quality of results.

The present study aimed to evaluate the performance of “Smartscopy” in diagnosing preinvasive cervical lesions in patients with abnormal cervical cancer screening results during the COVID-19 pandemic.

Materials and methods

This study was conducted at the colposcopy outpatient clinic of the Srinagarind Hospital, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand. All eligible participants with abnormal cervical cytology and/or HPV DNA testing results between September 2020 and March 2021 were included. Collected data included baseline characteristics, initial cervical cytological results and/or HPV DNA testing, smartphone images, colposcopy findings, tissue collection procedures, and final histological results. Cervical cytological methods included conventional cytology (Papanicolaou) or liquid-based cytology using the Bethesda system 2014 [17].

1. Study population

Non-pregnant women ≥18 years of age with abnormal cervical cytology and/or abnormal HPV DNA test results, who required further evaluation with colposcopy, were recruited. Pregnant and hysterectomized women were excluded from this study.

2. Diagnostic criteria

1) Cervical photographs captured using the Smartphone

Photographs were captured using an iPhone 8 (Apple Inc, Cupertino, CA, USA) smartphone, which is equipped with a 12-megapixel camera, with an aperture size of F1.8. The flash mode (LED Truetone) is permanently activated. Maximum image size is 3,024×4,032 pixels. All photographs were captured at the same distance from the cervix (approximately 15 cm). A cervical biopsy was performed if abnormal lesions were detected on the photograph.

2) Reference standard

Colposcopy was the reference standard used in the present study. The colposcope (Leisegang 3ML; Leisegang Feinmechanik Optik GmbH, Berlin, Germany) was operated at magnifications of 7.5×, 15×, and 30×. Biopsy was performed if colposcopy detected a suspicious lesion. Endocervical curettage was performed in all participants. Colposcopic assessment was performed in accordance with the International Federation of Cervical Pathology and Colposcopy colposcopic terminology of the cervix in 2011 [18].

3. Test methods

All participants with abnormal cervical cytology and/or HPV DNA test results between September 2020 and March 2021 were enrolled. The test methodology used in this study was as follows. In the colposcopy clinic, after patients signed a consent form and were placed in the lithotomy position, a 4% acetic acid solution was applied to the cervix for 1 minute. The cervix was evaluated by two independent colposcopists using smartscopy and colposcopy. The iPhone 8 (Apple lnc) was used to evaluate cervical lesions. The first colposcopist (Dr. A.T.) examined cervical lesions using a smartphone camera, captured images, and identified lesions known as “SmartAreas”. Another colposcopist (Dr. II, A.A.) evaluated the cervix of the same patient with colposcopy and was blinded to any information from which to identify the lesion defined as “ColpoArea”. Next, Dr. II biopsied the cervical lesion using standard colposcopic biopsy based on both “SmartArea” and “ColpoArea”. To prevent false-negative results from colposcopy, endocervical curettage was performed on all participants for endocervical lesion evaluation, even though no lesions were detected using smartscopy or colposcopy. Drs. I and II were gynecological oncologists with experience in colposcopy.

The terms of interpretation were classified into five categories following the general assessment: normal colposcopic findings; abnormal colposcopic findings; suspicious for invasion; and miscellaneous findings [18]. The visibility of the squamocolumnar junction was reported to be completely visible, partially visible, or invisible. Smartscopy results were interpreted before obtaining the histological results, which were classified into three categories: normal, low-grade squamous intraepithelial lesion plus (LSIL+), and high-grade squamous intraepitherial lesion plus (HSIL+). Smartscopy results defined as LSIL+ were defined as LSIL, HSIL and cancer while HSIL+ was defined as HSIL and cancer. Finally, the correlation between the smartphone findings and the histological diagnosis was evaluated.

Histopathological examinations were performed by gynecological pathologists at the Srinagarind Hospital, Khon Kaen University. Histopathology results were classified as negative for intraepithelial lesion or malignancy, LSIL, HSIL, or invasive carcinoma based on the Lower Anogenital Squamous Terminology (i.e., LAST)/WHO recommendations for reporting histological LSIL including CIN 1 and HSIL including CIN 2 or CIN 3 [19].

4. Data analysis

The diagnostic performance of the smartphone for CIN was assessed. Sensitivity, specificity, PPV, NPV, and diagnostic accuracy were interpreted for the diagnosis of LSIL+ and HSIL+ using STATA, version 21 (StataCorp LLC, College Station, TX, USA). The kappa value (κ) was used to evaluate the agreement between the two gynecological oncologists.

5. Ethics approval

The present investigation was a diagnostic study. Information was identified only by medical record numbers, thus preserving participant anonymity. This study was approved by the Khon Kaen University Ethics Committee for Human Research (approval number HE621485). This study was registered in the Thai Clinical Trials Registry (TCTR20200709008) on September 7, 2020.

Results

A total of 274 participants were recruited for the present study (Fig. 1). The mean±standard deviation patient age was 42±11.4 years and mean age at first sex was 22±5.1 years. Most participants were non-smokers (97.8%) and had multiple partners (50.7%). The percentage of participants with normal cervical cytological findings was 14.5%. The proportion of atypical squamous cells of undetermined significance (i.e., ASCUS) was 11.3%, atypical squamous cells-cannot exclude HSIL was 7.7%, LSIL was 26.2%, HSIL was 22.1% and cervical cancer was 6 (26)%. Of 99 women who underwent HPV DNA testing, 22 (8.0%) had positive high-risk HPV type 16, six (2.1%) had positive high-risk HPV type 18, two (0.7%) had positive high-risk HPV type 16, 18, and 57 (21.0%) had positive high-risk HPV type non 16/18 (Table 1).

Fig. 1

Flow of participants through the study. LSIL, low-grade squamous intraepithelial lesion; HSIL, high-grade squamous intraepithelial lesion; ECC, endocervical curettage; NILM, negative for intraepithelial lesion or malignancy; SCCA, squamous cell carcinoma; AC, adenocarcinoma.

Table 1

Patient characteristic

Patient characteristic	Value
Age (yr)	42±11.4
Age first sex (yr)	22±5.1
Smoking status
Never smoke	268 (97.8)
Previous/currently smoke	6 (2.2)
History of multiple partner
Yes	139 (50.7)
No	135 (49.3)
Cervical cytology
NILM	40 (14.5)
ASC-US	31 (11.3)
ASC-H	21 (7.7)
LSIL	71 (26.0)
HSIL	60 (22.1)
AGC	41 (14.8)
AIS	4 (1.4)
Cancer	6 (2.2)
HPV testing status
Not done	177 (64.6)
Negative	10 (3.6)
Positive type 16	22 (8.0)
Positive type 18	6 (2.1)
Positive type 16 and 18	2 (0.7)
Positive others HPV type non 16 and 18	57 (21.0)
SCJ on smartphone/colposcopy
Visible	180 (66.0)
Partially visible	44 (16.0)
Not visible	50 (18.0)

Values are presented as mean±standard deviation or number (%). NILM, negative for intraepithelial lesion or malignancy; ASC-US, atypical squamous cells of undetermined significance; ASC-H, atypical squamous cells-cannot exclude HSIL; LSIL, low grade squamous intraepithelial lesion; HSlL, high grade squamous intraepithelial lesion; AGC, atypical glandular cells; AIS, adenocarcinoma in situ; HPV, human papillomavirus; SCJ, squamocolumnar junction.

Cervical images of LSIL and HSIL captured using the smartphone are presented in Fig. 2. The final histological findings were negative for intraepithelial lesions or malignancies (56.5%), LSIL (18.6%), HSIL (21.1%), and carcinoma of the cervix (3.5%) (Table 2).

Fig. 2

The uterine cervix image from smartphone. (A) LSIL; note the acetowhite epithelium at 2 o’clock. (B) HSIL; note the acetowhite epithelium at 6 o’clock.

Table 2

Histological finding

Histological finding	Value
NILM	155 (56.5)
LSIL	51 (18.6)
HSIL	58 (21.1)
Squamous cell carcinoma	6 (2.1)
Adenocarcinoma	4 (1.4)

Values are presented as number (%).

NILM, negative for intraepithelial lesion or malignancy; LSIL, low grade squamous intraepithelial lesion; HSIL, high grade squamous intraepithelial lesion.

LSIL+ was diagnosed using the smartphone in 250 patients, whereas 24 patients were classified as normal. The correlation between smartscopy and histological result “normal” and “normal” respectively was 20 patients, with histological finding result “LSIL+” being four patients. The correlation between smartscopy and histological findings result “LSIL+” and “normal” was found in 135 patients, and with histological finding “LSIL+” was found in 115 patients (Table 3).

Table 3

Smartphone result and histologic finding correlation for LSIL+ and HSIL+

	Histologic finding Normal	Histologic finding LSIL+	Histologic finding HSIL+	Total
LSIL+
Smartphone result normal	20	4		24
Smartphone result LSIL+	135	115		250
Total	155	119		274
HSIL+
Smartphone result normal	176		22	198
Smartphone result HSIL+	30		46	76
Total	206		68	274

LSIL, low grade squamous intraepithelial lesion; HSIL, high grade squamous intraepithelial lesion.

HSIL+ was diagnosed using the smartphone in 76 patients. A total of 198 patients had normal/LSIL. The correlation between smartscopy and histological findings “normal/LSIL” and “normal/LSIL” respectively was 176 patients, with histological finding “HSIL+” being 22 patients. The correlation between smartphone findings and histological findings “HSIL+” and “normal/LSIL” respectively was found in 30 patients, with histological finding “HSIL+” was found in 46 patients (Table 3).

The sensitivity, specificity, PPV, NPV, and accuracy of the smartscopy for detecting LSIL+ were 96.6% (95% confidence interval [CI], 91.6–99.1), 12.9% (95% CI, 8.1–19.2), 46.0% (95% CI, 39.7–52.4), 83.3% (95% CI, 62.6–95.3), and 49.2% (95% CI, 43.2–55.3), respectively. On the other hand, the sensitivity, specificity, PPV, NPV, and accuracy of smartscopy in the diagnosis of HSIL+ were 67.6% (95% CI, 55.2–78.5), 85.4% (95% CI, 79.9–90.0), 60.5% (95% CI, 48.6–71.6), 88.9% (95% CI, 83.7–92.9), and 81.0% (95% CI, 75.8–85.4), respectively (Table 4). Moreover, the diagnostic accuracy of smartscopy for the diagnosis of LSIL+ and HSIL+ was 49.2% (95% CI, 43.2–55.3) and 81.0% (95% CI, 75.8–85.4), respectively. The correlation between smartscopy and colposcopy with histology was statistically significant (κ=0.88; 95% CI, 0.82–0.93).

Table 4

Evaluate of Smartphone diagnosis for LSIL+ and HSIL+

Smartphone diagnosis	LSIL+	HSIL+
Sensitivity (%)	96.6 (91.6–99.1)	67.6 (55.2–78.5)
Specificity (%)	12.9 (8.1–19.2)	85.4 (79.9–90.0)
Positive predictive value (%)	46.0 (39.7–52.4)	60.5 (48.6–71.6)
Negative predictive value (%)	83.3 (62.6–95.3)	88.9 (83.7–92.9)
Accuracy (%)	49.2 (43.2–55.3)	81.0 (75.8–85.4)
Positive likelihood ratio (LR+)	1.11 (1.04–1.19)	4.65 (3.21–6.72)
Negative likelihood ratio (LR−)	0.26 (0.09–0.74)	0.38 (0.27–0.54)

Values are presented as number (95% CI).

Kappa value=88.3 (95% CI, 82.0–93.1).

LSIL, low grade squamous intraepithelial lesion; HSIL, high grade squamous intraepithelial lesion; LR, likelihood ratio; CI, confidence interval.

Discussion

Results of our study demonstrated that the sensitivity, specificity, PPV, NPV, and accuracy of smartscopy to detect LSIL+ were 96.6% (95% CI, 91.6–99.1), 12.9% (95% CI, 8.06–19.2), 46.0% (95% CI, 39.7–52.4), 83.3% (95% CI, 62.6–95.3), and 49.2% (95% CI, 43.2–55.3), respectively. This result suggests that smartscopy may be useful for screening patients with preinvasive cervical lesions that require colposcopy. This is because the smartscopy results demonstrated high sensitivity for detecting LSIL+ despite low specificity and PPV.

Moreover, the sensitivity, specificity, PPV, NPV, and accuracy of the smartphone in diagnosing HSIL were 67.6% (95% CI, 55.2–78.5), 85.4% (95% CI, 79.9–90.0), 60.5% (95% CI, 48.6–71.6), 88.9% (95% CI, 83.7–92.9), and 81% (95% CI, 75.8–85.4), respectively.

Tanaka et al. [14] reported that the sensitivity, specificity, PPV and NPV of the smartphone in the diagnosis of CIN1+ were 89%, 33%, 91%, and 30%, respectively, and the kappa value was 0.67. Our study yielded results similar to those by Tanaka et al. [14]; however, the specificity and PPV were lower in our study; moreover, the κ was 0.88 in our study and this correlation was stronger in our study. It is possible that the investigators in the study were gynecological oncologists and confirmed the accuracy of diagnosis with histological specimens even with normal findings, while only gynecologists were investigated in the study by Tanaka et al. [14] and did not confirm this in cases with normal imaging results. Our results demonstrated lower specificity and PPV, which may have been be due to the large sample size and the low prevalence of LSIL+.

Catarino et al. [15] used smartscopy as a telemedicine tool for screening cervical cancer in low-resource countries. They compared offsite and onsite clinicians to interpret cervical images using smartscopy to detect CIN. The results revealed that the sensitivity and specificity of both the onsite and offsite treatments were similar. The authors reported that the sensitivity and specificity were 66.7% and specificity was 82.3–85.7%.

Regarding the quality of the images from smartscopy, Gallay et al. [11] reported that smartscopy provided high-resolution images for VIA/Lugol’s iodine diagnosis in 93.3% of cases with moderate interobserver agreement (κ=0.45; 95% CI, 0.23–0.58). Therefore, cervical images captured by smartphone users are an alternative option to colposcopy in low- and middle-income settings.

Digital cervicography is an alternative method of cervical cancer screening [20,21]. It consists of a USB-pen camera and can connect to “Android” devices. In our study, the accuracy and sensitivity of smartscopy for detecting HSIL+ were 81% and 67.6%, respectively, which was the same as those of pap cytology screening (accuracy, 76%; sensitivity, 79.31%) and cervicography (accuracy, 92%; sensitivity, 72.41%) [22]. Our study found that the specificity of smartscopy for detecting HSIL+ was 60.5%, which was lower than that of digital cervicography (97%) [22]. It may be that the distance to capture an image from digital cervicography is shorter than that for smartscopy. However, digital cervicography is not available in all healthcare settings [23].

Regarding HSIL+ lesions, the study by Tran et al. [16] and our study found that the sensitivity of smartscopy to detect lesions was 67.6–71.3%. Therefore, it may not be suitable for diagnosing high-grade lesions of the cervix.

Our study had a large sample size, and we had the histological diagnosis in all cases from both colposcopy and smartscopy; moreover, no abnormal findings were detected using endocervical curettage, thus enabling correct assessment of disease. All cases were evaluated by two experienced gynecological oncologists (i.e., Drs. I and II) and performed throughout the study, which is the strength of the study.

However, owing to the low number of patients with high-grade cytology, the result of using smartscopy to screen HSIL+ with the sensitivity value was relatively low. Therefore, this tool is inappropriate for HSIL+ screening. Studies in this population should be conducted with larger sample sizes.

Smartphones capture a high-resolution image to detect abnormal cervical lesions. However, our results cannot be generalized to other populations due to variations in the prevalence of the disease, which affect PPV and NPV. Moreover, smartphones used for smartscopy may have variable costs and are generally expensive. Therefore, it may have drawbacks to its application.

The present study demonstrated the implications of using “Smartscopy“ for health care provider practice management as an alternative cervical cancer screening tool in low to medium medical resource settings and supported essential telemedicine services to maintain social distancing during the COVID-19 pandemic. Moreover, high-resolution images from the newly developed smartscope can improve study results.

Smartscopy demonstrated a remarkable correlation with colposcopy and a high predictive value for the detection of preinvasive cervical lesions. Therefore, the utility of smartscopy as an alternative tool for detecting abnormal cervical lesions in low-to-medium medical resource settings merits consideration. Smartscopy continues to play an important role in telemedicine during the COVID-19 pandemic.