The diagnostic performance of the Gynecologic Imaging Reporting and Data System (GI-RADS) in adnexal masses.

BACKGROUND: Adnexal masses, mostly benign, are common in the female genital system. However, adnexal masses are the leading cause of death among women with gynecologic cancer. Ultrasound is a common imaging method for diagnosing adnexal masses. Gynecologic Imaging Reporting and Data System (GI-RADS) is a useful diagnostic tool based on objective ultrasound features to diagnose the malignancy of the female genital system. Therefore, we conducted a meta-analysis to evaluate the ability of GI-RADS to differentiate adnexal masses.
METHODS: Published articles were searched in PubMed, Medline, and Embase from 1990 to February 2020. Pooled sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio, and area under the curve (AUC) were estimated via the extracted data from the selected studies.
RESULTS: Ten studies and 2,474 patients were included in this meta-analysis. The pooled sensitivity of selected studies was 0.95 [95% confidence intervals (CI): 0.94-0.97], and the pooled specificity was 0.86 (95% CI: 0.84-0.88). The pooled NLR and PLR were 0.06 (95% CI: 0.04-0.10), and 8.30 (95% CI: 4.93-13.97), respectively. Moreover, the pooled diagnostic odds ratio for GI-RADS was 174.59 (95% CI: 76.70-397.42), and the AUC was 0.9806.
CONCLUSIONS: This research indicates that GI-RADS might be a valuable tool to distinguish malignancies from adnexal masses. 2021 Annals of Translational Medicine. All rights reserved.

Introduction

Adnexal masses are common in the female genital system, and most of them are benign. Nevertheless, ovarian cancer is the third most common gynecologic malignancy (1.6%) and the leading cause of death among women with gynecologic cancer (1,2). Therefore, it is significant to find early and accurate diagnostic methods to recognize the malignant adnexal masses. Now, as recommended by the guidelines of ovarian cancer, differential diagnostic methods include ultrasound, Computerized Tomography (CT), Magnetic Resonance Imaging (MRI), and serum cancer antigen tests (3-5). Ultrasound examination is used in investigating pelvic tumors. However, because of the experiential discrepancy among inspectors, the results of evaluating adnexal masses are not objective enough. According to a multicentre clinical trial, the false positive rate of investigating ovarian cancer is 24% (6). Meanwhile, problems also exist in the interpretation of the information from ultrasound doctors to clinicians (6).

Similar to Breast Imaging Reporting and Data System (BI-RADS), designed by the American College of Radiology to reduce the influence of subjective judgment on ultrasonic examination results of breasts, Gynecologic Imaging Reporting and Data System (GI-RADS) was introduced by Amor. F to standardize the ultrasound report of the gynecologic system (7,8). This system included several ultrasonic morphological and clinical features and they were laterality, maximum diameter, echogenicity, wall thickness, cystic content, solid areas, septations, solid papillary projections, the presence of ascites, the doppler evaluation. The classification of GI-RADS includes GI-RADS 1 to GI-RADS 5, and a higher level indicates that the mass is more likely to be malignant. The kappa coefficients of GI-RADS categorization were 0.896 (95% CI: 0.775 to 1.000), according to a previous study (9). The diagnostic performance of GI-RADS in different studies is highly heterogeneous, and no clinical trial of large samples had investigated such discrepancy. Thus, this present study aimed to design a meta-analysis to evaluate and review the effect of the GI-RADS. Further, we also compared different ovarian cancer diagnostic methods with GI-RADS. We presented this article following the PRISMA reporting checklist (available at http://dx.doi.org/10.21037/atm-20-5170).

Methods

Search strategy

We searched the studies published from 1990 to February 2020 in PubMed, MEDLINE, and EMBASE, and the search terms were “Gynecologic Imaging Reporting and Data System” and “GI-RADS”. After reading the title and abstract of the studies, three reviewers (W Guo, Xiuhe Zou and HY Xu) decided whether the studies met the inclusion criteria independently. One experienced investigator (XL Ma) assessed the quality of the included studies and resolved the conflicts between three reviewers.

Inclusion and exclusion criteria

Inclusion criteria were that: (I) the patients had adnexal masses; (II) the adnexal mass had been verified by golden standards (biopsy or follow-up), and the minimal required follow-up time was 6 weeks; (III) the adnexal mass was evaluated by ultrasound and classified by the GI-RADS only; (IV) the article was written in English. The exclusion criteria were that: (I) studies only had abstracts or lacked sufficient data of the diagnostic effect; (II) the adnexal mass was evaluated by other imaging methods or methods combined with ultrasound; (III) the langue of articles was not English.

Data extraction

Two reviewers extracted these data: study characteristics, demographic characteristics, and outcomes from the selected articles. The details are as followings:

Study characteristics: author, year of the study, country, amount of the patients and masses, ultrasound equipment, probe frequency, the technology of the ultrasound, and the golden standard;

Demographic characteristics: age, the prevalence rate for adnexal malignancy, premenopausal rate;

Outcomes: specificity, sensitivity, positive predictive value, negative predictive value, and value of true positive (TP), false positive (FP), true negative (TN), false negative (FN) for the ability of GI-RADS to recognize the adnexal malignancy (10).

Statistical analysis

We used Meta-Disc version 1.4 statistical software to calculate the pooled specificity, sensitivity, positive likelihood ratios (PLR), negative likelihood ratios (NLR), and diagnostic odds ratio (DOR) with 95% confidence intervals (CIs). Also, the summary receiver operator characteristics (SROC) was constructed by this software to evaluate the relationship between specificity and sensitivity. We also used the inconsistency index (I2) to detect heterogeneity among the included studies and would use the random effect model in the meta-analysis if I2 was more than 50%. The Deeks’ funnel plot asymmetry test performed in Stata version 16.0 software (StatsCorp., College Station, TX) was used to value publication bias.

Quality assessment

To assess the included articles, we used the Quality Assessment of Diagnostic Accuracy Studies version 2 (QUADAS-2). The QUADAS-2 can evaluate the quality of studies from 14 items, four domains, and two aspects (11). According to the assessment results, these ten articles were reliable enough to conclude the ability of GI-RADS to differentiate adnexal masses. The QUADAS-2 was operated in Review Manager version 5.3 (Cochrane Informatics and Knowledge Management Department, Copenhagen, Denmark).

Results

Study selection

After removing the duplicates, we included 311 articles by the initial publication search and selected 77 studies for further filtration by reading the full text. Then, 67 of these publications failing to meet the inclusion criteria were excluded: 60 articles were not related to the subject; 4 articles did not use the GI-RADS to evaluate the adnexal masses; 3 articles did not use the ultrasound to detect the masses.

Additionally, though written by the same authors, two studies of different sample sizes were included.

Finally, we included a total of 10 articles in this study, and the selection flow chart was shown in .

Figure 1

Flow chart of researches selection.

Eligible study characteristics

Ten selected studies were published from 2009 to 2019, and the details of these studies were shown in . These studies were conducted in 7 different countries. The total number of patients was 2,474, and that of masses was 2,723. The mean age of all the patients was 42.75 (age range, 6–89), and the prevalence rate of malignant tumors ranged from 13.45% to 41.83%. The type of the ultrasound equipment in these studies was Voluson 730 (GE Healthcare, Milwaukee, WI, USA), GE E8 EXPERT (GE Medical Systems, Zipf, Austria), GE S8 (GE Medical Systems, Zipf, Austria), iU22 digital scanner (Philips Healthcare, Bothell, WA, USA), Voluson E8 (GE Medical Systems, Zipf, Austria), SonoScape S40 Exp/S40 Pro/S40/S35 (Sonoscape Medical Hong Kong Co., Ltd), DC-7 Mindary (DC-7 Mindary Medical Ltd, China), ACUSON X150 ultrasound system (Siemens Health care, Mountain View, CA, USA), ProSound Alpha 7 ultrasound (Hitachi Aloka Medical America, Inc. Germany). The golden standard of the differentiation of the gynecologic malignancy was biopsy and clinical symptoms during the follow-up. Histology diagnosis was used as the golden standard of the differentiation of the gynecologic malignancy in all included studies. What’s more, in Tan Z, Basha MAA, and Behnamfar F’studies, some of the patients had a spontaneous resolution in followed up by ultrasound study. Behnamfar F’s study defined the follow-up interval was 6 weeks, and the other two studies didn’t mention the specific follow-up time.

Table 1

Basic characteristics of included studies

Author	Country	Year	Patients	Mass	Age	Prevalence rate for malignancy	Premeno-pausal	Ultrasound equipment	Probe frequency	Technology	Golden standard
Amor F (8)	Chile	2009	171	183	39 [16–77]	13.45%	68.5%	Voluson 730	N/A	Color Doppler, pulsed Doppler	Histologic diagnosis
Amor F (12)	Spain, Chile	2011	372	432	44 [13–78]	26%	75%	Voluson 730	N/A	Color Doppler, pulsed Doppler	Histologic diagnosis
Diaz L (13)	Spain	2017	257	281	40.3 [18–87]	15.2%	77.8	ACUSON X150	Transvaginal probes & transrectal probes:4–9 MHz; abdominal probes: N/A	Color Doppler	Histologic diagnosis
Koneczny J (14)	Poland	2017	271	271	N/A	N/A	N/A	GE E8 EXPERT GE S8	Transvaginal probes: 5.0–9.0 MHz; abdominal probes: 3.5–5.0 MHz	Color Doppler, “power” Doppler	Histologic diagnosis
Zhang T (15)	China	2017	242	263	47.16 [19–73]	41.83%	52.47%	iU22 digital scanner	N/A	Color Doppler	Histologic diagnosis, Follow-up
Migda M (16)	Poland	2018	215	215	47.2 [13–89]	24.7%	N/A	Voluson E8	N/A	Color Doppler	Histologic diagnosis
Basha MAA (17)	Egypt	2019	308	325	45.33	39.1%	42.5%	SonoScape S40 Exp/S40 Pro/S40/S35	Transvaginal probes: 5.0–7.5 MHz; abdominal probes: 3.0–3.5 MHz	Color Doppler	Histologic diagnosis, Follow-up
Behnamfar F (18)	Iran	2019	197	197	37.51	N/A	81.4%	Voluson 730; DC-7 Mindary	N/A	Color Doppler, multi-frequency probe	Histologic diagnosis, Follow-up
Zheng H (19)	China	2019	325	400	40 [6–85]	N/A	74%	iU22 digital scanner; Voluson 730	N/A	Color Doppler, –	Histologic diagnosis
Khalaf LMR (20)	Egypt	2019	116	156	42 [10–82]	26.3%	68.6	ProSound Alpha 7 ultrasound	N/A	Color and spectral Doppler, –	Histologic diagnosis

Diagnostic value of differentiating malignancy

The pooled sensitivity of selected studies was 0.95 (95% CI: 0.94 to 0.97, I2=61.5%), and the pooled specificity was 0.86 (95% CI: 0.84 to 0.88, I2=94.3%). The pooled NLR and PLR were 0.06 (95% CI: 0.04 to 0.10, I2=59.3%), and 8.30 (95% CI: 4.93 to 13.97, I2=94.2%), respectively. The pooled DOR for GI-RADS was 174.59 (95% CI: 76.70 to 397.42, I2=74.6%). The SROC was shown in , and the overall area under the curve (AUC) was 0.9806 (SE, 0.0160).

Figure 2

Plots of sensitivity (A), specificity (B), negative likelihood ratio (C), positive likelihood ratio (D), diagnostic odds ratio (E) and receiver operating characteristic curve (F) of diagnosing adnexal mass. Positive LR, positive likelihood ratio; negative LR, negative likelihood ratio; AUC, area under the curve; CI, confidence intervals; diagnostic OR, diagnostic odds ratio; SROC, summary receiver operator characteristics.

Assessment of publication bias

The Deeks’ funnel plot asymmetry test was used to evaluate the publication bias in the included studies (). The results showed that the diagnostic OR of GI-RADS was not significantly biased in the differentiation of benign and malignant adnexal masses (P=0.6).

Figure 3

Risk of bias and applicability concerns: (A) graph; (B) summary.

The exact assessment results of the eligible studies were shown in , and the quality of these studies was sufficient for analysis.

Figure 4

Deeks’ funnel plot summarizes the publication bias in our meta-analysis of the diagnostic odds ratios.

Discussion

This meta-analysis included ten studies from 7 countries to evaluate the diagnostic value of the GI-RADS for adnexal masses. Notably, Amor F. conducted two studies with different patients in 2009 and 2011. The diagnostic accuracy of ultrasound depended on the operators’ experience, and the difference rate between experts and routines was 15% (95% CI: 0–29%) (8,21-23). Thus, introducing the GI-RADS can strengthen the consistency among the different examiners and improve the diagnostic ability to differentiate the malignancy from the adnexal masses.

Our research systematically analyzed the data of the diagnostic performance of the GI-RADS in these ten studies. The pooled sensitivity and specificity were 0.95 and 0.86, respectively. The pooled NLR was 0.06, and PLR was 8.30. PLR >10 and NLR <0.1 mean high diagnostic accuracy (24). Although the pooled NLR was 0.06 and PLR 8.30 was in our study, the PLR value failed to draw a convincing conclusion of the presence of the adnexal malignancy. Furthermore, the diagnostic accuracy of GI-RADS evaluated by the AUC was 0.9806, indicating a high diagnostic value (25). All of these results showed that the GI-RADS could differentiate gynecological adnexal malignancy from adnexal masses.

There were several limitations to this study. Firstly, since I2 of the pooled sensitivity and specificity were over 50%, we should consider the heterogeneity of data. However, lacking enough data in the included articles, we could not perform a sub-analysis. Besides, various types of ultrasound equipment used in different studies might account for heterogeneity among studies. Also, the age range of patients, the prevalence rate for adnexal malignancy, and menopausal proportions varied in these studies, which might increase the heterogeneity of the result. Finally, there were no factors in the included articles that allowed for subgroup analysis, which might hinder finding subgroups with higher diagnostic efficacy. Therefore, different ultrasound parameters are desired in subsequent studies to explore better diagnostic performance.

Although we excluded studies using other examine methods such as MRI and contrast-enhanced ultrasound (CEUS), their diagnostic power is worth further discussion. A previous study showed that when combining GI-RADS with 3D-CEUS, the diagnostic sensitivity, specificity, AUC, and kappa value were 94.2%, 95.5%, 0.969, and 0.872, respectively (26). Compared with using GI-RADS alone, the combination of GI-RADS and 3D-CEUS had higher specificity and lower sensitivity and smaller AUC according to this research, partly because the 3D-CEUS could not show real-time hemodynamic variance in the masses. Due to the limited number of studies included, large-scale and multicentre studies are required to compare these two methods.

Furthermore, the detectable rate of malignant adnexal masses detected by the ultrasound was low (8–20%), so MRI was introduced to a reporting data system named Ovarian Adnexal Reporting Data System Magnetic Resonance Imaging (O-RADS MRI). Based on a multicentre cohort study, the sensitivity, specificity, and AUC of O-RADS MRI were 0.93 (95% CI: 0.89–0.96), 0.91 (95% CI: 0.89–0.93), and 0.961 (95% CI: 0.948–0.971) (27-29). The diagnostic performance and accuracy of MRI were similar to that of GI-RADS.

In conclusion, GI-RADS is useful and reliable to improve the diagnostic ability of ultrasound to differentiate adnexal masses and malignant tumors. It is an effective method to settle the inconsistency among different ultrasound operators. With the development of the novel diagnostic tool, combining the GI-RADS with other examinations is a trend. However, further studies are required to confirm the results of our research.

The article’s supplementary files as