A preliminary study: The sequential use of the risk malignancy index and contrast-enhanced ultrasonography in differential diagnosis of adnexal masses.

The aim of this study was to explore the sequential use of risk malignancy index (RMI) combined with contrast-enhanced ultrasonography (CEUS) in identification diagnosis of adnexal masses.This study contained 2 steps: first, 151 patients were analyzed retrospectively with RMI 1, RMI 2, and RMI 3 indices; receiver operating characteristic (ROC) curves were plotted to analyze area under the curves (AUC), and then RMI cut-off value was obtained according to maximum Youden index (YI, Sensitivity + Specificity - 1) and calculating diagnostic sensitivity, specificity, positive/negative predictive value, and accuracy. Second, 151 cases were divided into 2 groups randomly (105 in study group and 46 in test group); in the study group, the lower cut-off value (LC), upper cut-off value (UC), CEUS cut-off value according to maximum YI, and then these cut-offs were validated in test group.There was no statistical significance in 3 RMI models (P = .35), and RMI1 model was established randomly for following study. When the RMI1 cut-off value was 149, the YI was maximal (0.53), and the sensitivity, specificity, positive/negative predictive value, and accuracy were 71.0%, 81.7%, 77.1%, 75.6%, and 76.2%, respectively. The LC was 15 (sensitivity was 98.0%), the UC was 3000 (specificity was 98.0%), and the CEUS cut-off value was 7 (maximal YI was 0.81). In the test group (46 cases), combining RMI1 LC (15) and UC (3000) with CEUS cut-off value (7), the sensitivity, specificity, positive/negative predictive value, and accuracy were up to 85.7%, 92.0%, 90.0%, 88.5%, and 89.1%, respectively.CEUS can help RMI to make a more effective differential diagnosis of the adnexal mass. Further validation by additional multicenter prospective trials is required.

Introduction

Malignant ovarian tumor is one of the most common gynecologic oncology in both developing and developed countries, which is a substantial burden to global public health. Although the mortality rates of ovarian cancer patients reduced significantly over the past decades, the overall incidence rates keep on increasing rapidly, even in countries that previously had low rates.[ Consequently, accurately characterizing adnexal masses enables patients with malignancy to be appropriately triaged for management. As the female ovary lies deep in the pelvis and has complex histological components, almost 80% ovarian cancer is diagnosed in advanced or late stages.[ Moreover, the 5-year survival rate is only 20% in advanced cancer, which is much lower than that of 70.0%∼90.0% in early stage.[ Surgical treatment is required for adnexal mass patients with sonographic malignant tendency. Earlier observational studies[ have confirmed that recognizing cancer means that treatment is not delayed and appropriate staging can be carried out in specialized surgical centers. Therefore, accurate characterization of adnexal masses is pivotal to avoid unnecessary anxiety and costs in case of benign pathology.

To distinguish benign and malignant adnexal masses, serum CA125 and conventional transvaginal ultrasonography have been frequently used; however, the specificity is very low when transvaginal ultrasonography or CA125 is used independently.[ Meanwhile, various prediction models have been developed to be used for the characterization of adnexal masses. These include simple rules based on the ultrasonic morphological appearance, the risk malignancy index (RMI) scoring system, International Ovarian Tumor Analysis logistic regression model 2, and so on. In addition, earlier meta-analysis has demonstrated that subjective assessment in the hands of experienced ultrasound examiners yielded best results (with a pooled sensitivity of 0.93 and specificity of 0.89),[ but it is constraint in the clinical application because of the disadvantage of subjectivity; the advantage of these prediction models over subjective assessment is their objectivity and simplicity.

The RMI is a simple scoring product based on ultrasound scores (U), the menstrual state (M), and the absolute serum CA125 level (U/mL). RMI1 was originally developed in 1990.[ Some researchers later modified scoring method (the allocation of the U and M scores) and developed RMI2 and RMI3,[ accordingly. Although increasing and compelling evidence supports simple rules, logistic regression model 2, and subjective assessment, RMI is advocated by many national guidelines in the classification of adnexal masses and is still routinely used in clinical practice to triage patients. The RMI has been a standardized index to clinically evaluate adnexal mass and has been validated in many European countries.[

Contrast-enhanced ultrasonography (CEUS) is a reliable differential diagnostic imaging technique widely applied in the adnexal mass patients. It has many advantages, including low requirements for kidney function, low risk of allergy, no radioactivity, and high resolution.[ As we know, ultrasonography with color Doppler and sonographic contrast agents have been developed to improve the preoperative assessment in adnexal masses; unlike the color Doppler ultrasound, which is specialized in grading masses with large vascular networks, CEUS is better at visualizing tumor microvascularity and previous studies have indicated that microvessel density is invaluable in evaluating adnexal tumors.[

In this paper, we explore the possibility of using ovarian CEUS as a sequential clinical application in women with RMI values not associated with either the LC or the UC for discriminating between benign and malignant adnexal masses (Fig. 1).

Figure 1

Strategy using RMI and CEUS sequentially for differential diagnosis of adnexal masses. CEUS = contrast-enhanced ultrasonography. LC = lower cut-off value, M = menstrual state, RMI = risk malignancy index, U = ultrasound score, UC = upper cut-off value.

Materials and methods

Patients

This study was undertaken at West China Second University Hospital Sichuan University in January 2015 to March 2017. This retrospective study was approved by the hospital ethics committee and all the participants signed the informed consent form of CEUS. The clinical data included results of transabdominal or transvaginal ultrasound examination, menstrual state, and serum CA125 value; this makes it possible to retrospectively calculate RMI values. Basic information collected also included age and detailed pathologic diagnosis. Finally, 151 women with ultrasonographically and clinically verified adnexal lesions and who had been referred for surgery were enrolled for this analysis. Cases who had allergies to contrast agents, received chemotherapy or radiation before surgery, and had severe heart or lung disease were excluded.

Ultrasonography and CEUS

All cases were scanned by an experienced radiologist specializing in ultrasonography and CEUS to ensure reproducibility, and the enrolled cases underwent ultrasonography and CEUS with EPIQ7 (PHILIPS Company; Philips Medical Systems Nederland B.V., Veenpluis, The Netherlands), C10-3v probe and C5-1 probe) ultrasound diagnostic instrument. All patients were examined using the same B-mode and CEUS device settings.

Conventional ultrasound characteristics of interest were the presence of a multilocular cystic lesion; solid areas; bilateral lesions; ascites; intra-abdominal metastases. Simultaneously, color Doppler and resistance index were used to realize the degree of vascularization of the pathologic lesion.

CEUS began with administration of the contrast agent (Sonovue, Bracco, Italy) as a bolus of 4.8 cc in the elbow vein over approximately 5 seconds. The examined area covered not only the whole lesion, or, if the entire lesion was too large, the selected cross-section of its solid part, but also part of normal myometrium, or if hysterectomy had been performed, the sectional natural pelvic muscles. The entire examination, which lasted 120 seconds, was recorded on the hard disk and analyzed offline by an experienced ultrasonographer. The following parameters were analyzed: enhancement time, enhancement level, and enhancement morphology. The reference object for evaluation was the normal myometrium or pelvic normal muscle. This analysis was blinded with regard to clinical history or mass histology.

RMI calculations

On the basis of the data obtained, the RMI 1, RMI 2, and RMI 3 were calculated for all patients:

RMI1 = U × M × CA125, the ultrasound score (U) was based on the 5 features (the presence of a multilocular cystic lesion; solid areas; bilateral lesions; ascites; intra-abdominal metastases), each of which scores 1 point, a total score of 0 yielded U = 0, a total score of 1 yielded U = 1, and a total score of 2 to 5 yielded U = 3. The menstrual state (M) separately represents premenopausal state (M = 1) and postmenopausal state (M = 3).

RMI2 = U × M × CA125, the U expressed as 1 (if a total score ≤1) and 4 (if a total score of 2–5), the postmenopausal state expressed as 4 (compared with RMI 1).

RMI3 = U × M × CA125, the U expressed as 3 if a total score of 2 to 5 (compared with RMI 2), the M scored the same as RMI 1.

Postmenopausal status was defined as more than 1 year of amenorrhea or age greater than 50 years in women who have performed hysterectomy. All other participants were considered premenopausal.

The serum CA125 level was used directly in the calculation.

Analysis

The objective of this study was to develop a sequential strategy, which employed the RMI and CEUS so as to increase the diagnostic accuracy of RMI for the discrimination of adnexal masses. The following scheme was adopted for this purpose: all included cases (151) were randomly divided into 2 groups: the study group (105 cases) and the test group (46 cases). In the study group, we first defined the LC when the diagnostic sensitivity was 98%, namely, patients with RMI score below LC was approximately considered to have a negative (nonmalignant) outcome. We also defined the UC when the diagnostic specificity was 98%; patients with RMI score beyond UC were almost considered to have a positive (malignant) outcome correspondingly. If RMI value was between LC and UC, the CEUS cut-off value was used to determine the test outcome. The specific method was a semi-quantitative method and as follows: enhancement time (late, synchronous, early enhancement was scored 1, 2, 3, respectively), enhancement level (no or low, equal, high enhancement was scored 1, 2, 3, respectively), and enhancement morphology (homogeneous and inhomogeneous enhancement was scored 1, 2, respectively) were included in the evaluation, the CEUS scoring results were multiplied, and the CEUS cut-off value was defined according to the maximized YI (sensitivity + specificity − 1). Therefore, the test outcome was positive when CEUS score was beyond the CEUS cut-off value and negative otherwise. Subsequently, in the test group, the combination of LC, UC, and CEUS cut-off value was validated.

Statistical analysis was performed using SPSS 21.0 for Windows (SPSS Inc.). The mean and standard deviation (x ± s) was employed for continuous variables, while the rate (%) was employed for categorical data. Independent t test or analysis of variance (ANOVA) with F-test was used to compare means or medians of continuous variables, and the Chi-square test was used to compare proportions. Receiver operating characteristic (ROC) curves were constructed and the difference in AUC was analyzed by Kruskal–Wallis test. There was a statistically significant difference when P < .05. The maximum YI was used for the diagnostic cut-off value of RMI and CEUS. The sensitivity, specificity, positive/negative predictive value, and accuracy of RMI model, in addition to the sequential use of RMI and CEUS model, were calculated.

Results

Patient characteristics

Of 151 patients, 69 (45.7%) had malignant disease and 82 (54.3%) had benign pathology (including 34 benign lesion cases and 48 benign tumor cases). The average age of patients with malignant tumor was 46.58 ± 9.80 years, and in those with benign pathology, it was 37.11 ± 12.20 years. The pathological classification of all the cases is detailed in Table 1. Before menopause, the percentage of total benign cases was 64.81%, and that of total malignant cases was 35.19%, while more malignant masses appeared (72.09%) after menopause.

Table 1

Pathological type of 151 adnexal mass cases.

The distribution of 69 malignant and 82 benign cases by age, ultrasound score, menstrual status, and CA125 level are summarized in Table 2. In univariate analysis, the average age of malignant patients was greater than that of the benign, and a significant linear trend for malignancy was found by increasing ultrasound score. Before menopause, more benign masses appeared, and more malignant lesions emerged after menopause. The mean CA125 value was significantly higher in malignant patients when compared with cases suffered from benign tumors.

Table 2

Distribution of age, ultrasound score, menstrual status, and serum CA125 levels in 151 women.

RMI

One hundred fifty-one cases were scored with RMI 1, RMI 2, and RMI 3, and the scores were expressed as scatter plots. At the same time, 151 cases were scored with 3 RMIs according to the pathological classification, respectively. It showed that the RMI score of adnexal benign cases was lower, which was higher in the malignant (Fig. 2).

Figure 2

(A) Scatter plot of score results with 3 RMIs in 151 adnexal mass cases. (B) Scatter plot of score results with 3 RMIs in 34 adnexal benign lesion cases. (C) Scatter plot of score results with 3 RMIs in 48 adnexal benign tumor cases. (D) Scatter plot of score results with 3 RMIs in 69 adnexal malignant tumor cases. RMI = risk malignancy index.

The ROC curve was drawn according to 3 RMI scores of 151 cases (Fig. 3). The AUCs were estimated as 0.82 [95% confidence interval (95% CI), 0.75–0.88], 0.81 (95% CI, 0.74–0.88), and 0.80 (95% CI, 0.73–0.87), respectively. The difference of the 3 AUCs was not statistically significant (P = .35, Kruskal–Wallis test), which suggested that there was no obvious difference of the 3 RMIs for the differential diagnosis of adnexal masses. At a RMI1 cut-off value 150, the YI was maximal (0.53), and the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were 71.0%, 81.7%, 77.1%, 75.6%, and 76.2%, respectively.

Figure 3

Receiver operating characteristic (ROC) curves of risk malignancy index 1 (RMI1), risk malignancy index 2 (RMI2), and risk malignancy index 3 (RMI3) for 151 cases.

LC, UC, and CEUS cut-off

The RMI1 score was calculated for the 105 women in case group, and the sensitivity and specificity for different cut-off values are given in Table 3. At a RMI1 LC 15, sensitivity and specificity were achieved as 98.0% and 39.0%, respectively, predicating that 98% of malignant adnexal mass cases had a RMI1 score more than 15. In the subgroup with a RMI1 ≤ 15 (n = 20 cases), only 1 case (with a left ovarian carcinoid) was improperly classified as benign pathology. A RMI1 UC 3000 gave a sensitivity and specificity of 17.0% and 98.0%, respectively, meaning that 98% of benign adnexal mass cases had a RMI1 score less than 3000. In the subgroup with a RMI1 ≥ 3000 (n = 9 cases), just 1 case (with a left ovarian fibrous follicular tumor) was incorrectly divided into malignant pathology.

Table 3

Sensitivity and specificity of the different cut-off values for the RMI1 of 105 cases.

The ROC curve was drawn based on the scores of CEUS for 105 women in the case group (Fig. 4). Calculating the YI at different cut-off values, we obtained the maximal YI (0.805) and CEUS cut-off value (7), with a sensitivity and specificity of 88.0% and 93.0%, respectively.

Figure 4

Receiver operating characteristic (ROC) curves of contrast-enhanced ultrasonography (CEUS) for 105 cases.

Test

In the test group, we applied the sequential strategy (first using RMI cut-off values 15 and 3000 and then using CEUS cut-off value 7 in those with a RMI1 score between 15 and 3000) on 46 patients independently (Table 4). Application of the RMI1 would directly discover 8 women with a low probability (RMI1 ≤ 15) and 4 women with a high probability (RMI1 ≥ 3000) for adnexal malignant pathology. Among patients with RMI1 ≤ 15, a 39-year-old woman with CA125 of 10 U/mL and RMI1 of 10 was found with a borderline serous papillary fibroma. In the RMI1 “gray zone” (15 < RMI1 < 3000), CEUS was used for the other 34 undefined women. The subgroup with a CEUS cut-off value ≥7 (n = 16 cases) included 14 cases with malignant tumors and 2 cases with benign pathology (with 1 case of fallopian pyogenic inflammation and another of benign Brenner tumor). The other 18 women in the RMI1 “gray zone” had CEUS cut-off value < 7; of these women, 16 were found to have benign tumors and 2 had borderline tumors at histology. Analysis of the 46 patients provided an overall sensitivity, specificity, positive/negative predictive value, and accuracy of 85.7%, 92.0%, 90.0%, 88.5%, and 89.1%, respectively. Combinations of RMI with CEUS showed improvements over RMI alone.

Table 4

Using RMI1 and CEUS sequentially for differential diagnosis in 46 cases test group.

Discussion

About 10% of women undergo exploratory surgery for evaluation of ovarian masses during their lifetime.[ Prompt identification of ovarian malignancies and referral to a gynecology oncologist can enhance the patient survival rates, but a single method that was able to accurately predict ovarian malignancy is still unavailable.[ The aim of this study is to ascertain whether CEUS could add to the performance of the RMI in distinguishing benign from malignant adnexal masses. In the test group of 46 patients, where cases with a RMI1 less than or equal to 15 (n = 8) are classified as having a benign lesion, those with a RMI1 greater than or equal to 3000 (n = 4) are classified as having a malignant lesion, and those with a RMI1 between 15 and 3000 (“gray-zone,” n = 34) are classified according to CEUS cut-off value (7); an overall sensitivity of 85.7% and a specificity of 92.0% is achieved by this sequential approach. As opposed to the traditional RMI1 cut-off value 149, this method would result in an increasing sensitivity from 71.0% to 85.7%.

Currently, the RMI is advocated as the discriminatory tool. In this study, there is no statistical significance in the performance characteristics of 3 RMIs, which is in agreement with the results from other studies.[ Only Morgante et al[ have found that RMI2 was more reliable in discriminating benign and malignant ovarian disease than RMI1.

The key group for CEUS to refine the RMI are those with a RMI1 of 15 to 3000, as the performance of RMI1 falls off within the range. Differentiating between benign and malignant tumor is crucial to the selection of appropriate treatment, as the type of surgery depends on the degree of malignant. The real-time CEUS can reveal the microvascular component of the adnexal masses and improve the reflectivity of blood flow. Wu et al[ carried out a meta-analysis and concluded that the corresponding sensitivity, specificity, and AUC of CEUS appeared to be 0.89, 0.91, and 0.96, respectively. Furthermore, Qiao et al[ adopted a meta-analysis with 7 high-quality clinical trials (contained a combined total of 375 ovarian cancer patients) and revealed that CEUS was associated with the following performance measures in differential diagnosis of ovarian tumors: pooled Sen was 0.96, the summary Spec was 0.91, and the area under the summary ROC curve was 0.98.

To our knowledge, CEUS is a new technology for evaluation of microvessels and tissue perfusion. In its present form, ovarian CEUS analyzes critical and sensitive tissue blood perfusion information. This analysis is based on the fact that angiogenesis is a key step for ovarian tumor cell proliferation, which results in physical changes in the reflected ultrasound images. Some studies have suggested that quantitative parameters of CEUS (time intensity curve) were able to discriminate begin or malignant adnexal masses. Fleischer et al[ and Testa et al[ calculated peak intensity, the AUC, time to peak, sharpness, and half wash-out time, and concluded that the values for peak intensity and AUC in malignant tumors were significantly higher than those in borderline tumors and benign tumors, while Sconfienza et al[ indicated that malignant adnexal lesions were characterized by significantly shorter time to peak than benign lesions. Meanwhile, in other 2 studies, Orden et al[ and Marret et al[ both showed that the AUC value was the highest in the invasively malignant tumors, but the time to peak value was inconsistent between them. On the contrary, the qualitative analysis of CEUS, including enhancement time, enhancement level, and enhancement morphology, was proved to be very practical and less time-consuming by earlier studies.[ Moreover, the imaging information would not be disturbed by any movement of the probe or the patients.[ Therefore, in this study, a semi-quantitative method, on the basis of qualitative analysis of CEUS, is first used and proved to be effective for determining the nature of ovarian masses.

The data used in this work are obtained from a clinical study that is intended for examining the accuracy of CEUS alone, without consideration of RMI, which is collected and analyzed retrospectively for this study. It could be considered suboptimal. However, the archived 2D and CEUS images are all acquired by a sonographer with 10 years’ experience, and the range of RMI scores for these patients is consistent with other RMI validation studies.[

This is the first study to explore the use of CEUS as a sequential second-line test to supplement the RMI for the discrimination of adnexal masses. Van Trappen et al[ analyzed 123 patients who were managed sequentially, first using RMI cut-off values of ≤25 and >1000 and then using specialist ultrasound and magnetic resonance imaging in those with a RMI between 25 and 1000 provided an overall sensitivity of 94% and a specificity of 90%. The main advantage of the current method (applying CEUS sequentially with RMI1 cut-off values between 15 and 3000) is that it is needless for patients to undergo an additional expertise-dependent test; furthermore, magnetic resonance imaging, an unconventional second-line test, is costly and time-consuming. Vaes et al[ carried out a study that assessed the value of ovarian Histo-Scanning in combination with RMI in improving triage for women with adnexal masses, and resulted in a sensitivity and specificity of 88% and 95%, respectively. Although the result of Vaes et al[ is similar with this study, the analytical method of Histo-Scanning is only based on structural changes rather than blood perfusion changes.

There are some limitations of this study. First, this is a retrospective study and moderate sample size is included in the test group. Second, some investigators have obtained that there are other prediction models more sensitive than RMI. For instance, Wynants et al[ investigated RMI, logistic regression model 2, Assessment of Different NEoplasias in the adnexa risk model, and the Simple Rules risk score methods used for a total of 2763 patients with adnexal masses, and finally concluded that Assessment of Different NEoplasias in the adnexa risk model and Simple Rules risk score are clinically more useful than RMI to select patients with adnexal masses for specialized oncology care. Consequently, our findings require further validation by additional multicenter prospective trials; at the same time, future research may contain other prediction models to compare with this strategy.

Conclusion

CEUS can help RMI to make a more effective differential diagnosis of the adnexal mass. Further validation by additional multicenter prospective trials is required.

Author contributions

Data curation: Li Qiu, Fan Yang.

Formal analysis: Li Qiu, Hong Luo.

Funding acquisition: Hong Luo, Fan Yang.

Investigation: Li Qiu, Fan Yang.

Methodology: Fan Yang, Hong Luo.

Project administration: Hong Luo.

Software: Fan Yang.

Validation: Li Qiu, Fan Yang.

Visualization: Hong Luo.

Writing – original draft: Li Qiu.

Writing – review & editing: Fan Yang, Hong Luo.