Diagnostic usefulness of the Risk of Ovarian Malignancy Algorithm using the electrochemiluminescence immunoassay for HE4 and the chemiluminescence microparticle immunoassay for CA125.

The present study aimed to investigate the usefulness of the Risk of Ovarian Malignancy Algorithm (ROMA) in the preoperative stratification of patients with ovarian tumors using a novel combination of laboratory tests. The study group (n=619) consisted of 354 premenopausal and 265 postmenopausal patients. The levels of carbohydrate antigen 125 (CA125) and human epididymis protein 4 (HE4) were determined, and ROMA calculations were performed for each pre- and postmenopausal patient. HE4 levels were determined using an electrochemiluminescence immunoassay, while CA125 levels were determined by a chemiluminescence microparticle immunoassay. A contingency table was applied to calculate the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). Receiver operating characteristic curves were also constructed, and areas under the curves (AUCs) were compared between the marker determinations and ROMA algorithms. In terms of distinguishing between ovarian cancer and benign disease, the sensitivity of ROMA was 88.3%, specificity was 88.2%, PPV was 75.3% and NPV was 94.9% among all patients. The respective parameters were 71.1, 90.1, 48.2 and 91.1% in premenopausal patients and 93.6, 82.9, 86.6 and 91.6% in postmenopausal patients. The AUC value for the ROMA algorithm was 0.926 for the ovarian cancer vs. benign groups in all patients, 0.813 in premenopausal patients and 0.939 in postmenopausal patients. The respective AUC values were 0.911, 0.879 and 0.934 for CA125; and 0.879, 0.783 and 0.889 for HE4. In this combination, the ROMA algorithm is characterized by an extremely high sensitivity of prediction of ovarian cancer in women with pelvic masses, and may constitute a precise tool with which to support the qualification of patients to appropriate surgical procedures. The ROMA may be useful in diagnosing ovarian endometrial changes in young patients.

Introduction

According to statistical data from 2012, 238,719 new cases of ovarian cancer are diagnosed each year, while the mortality rate exceeds 150,000 patients per year (1). Ovarian cancer is responsible for 4% of all morbidities in females; however, it is associated with the highest mortality rate of all gynecological morbidities in developed countries (1,2). Due to poor clinical manifestations in the early stages of the disease, it is usually diagnosed at a late stage in which prognosis becomes extremely unfavorable. Of the various prognostic factors that are associated with ovarian cancer, clinical stage remains the most important (2). In recent years, it has been demonstrated that the hospital at which the patient is diagnosed and treated is of great importance; a correct and early diagnosis, appropriate preparation for a major surgical procedure, and the procedure itself being performed by an experienced gynecological surgeon who operates on a sufficiently large number of ovarian tumors each year are key factors (3). Therefore, there is a requirement for diagnostic tools that can allow the primary care physicians, including family doctors and gynecologists in outpatient practices, to perform simple and rapid preliminary diagnostic examinations on patients with suspected ovarian cancer so as to refer these patients to appropriate referral centers.

At present, several algorithms are available for the stratification of risk in patients with ovarian tumors. The most popular of these include the Risk of Malignancy Index (RMI) (4–6), OVA1 (7–9), LR2 (a logistic regression model) (10) and Risk of Ovarian Malignancy Algorithm (ROMA) (11–15). As RMI and LR2 require expertise in gynecological ultrasonography, they cannot be used by family physicians or gynecologists with poor knowledge of ultrasonographic diagnostic techniques. OVA1, which was recently approved by the Food and Drug Administration (FDA), is relatively expensive as it requires as many as five laboratory assays (10). ROMA appears to be the most simple diagnostic tool, while also being appropriately sensitive and specific. ROMA requires only the determination of carbohydrate antigen 125 (CA125) and human epididymis protein 4 (HE4) marker levels in the serum and knowledge of the patient's hormonal status in order to calculate the percentage chance of a diagnosis of ovarian cancer in a patient with an adnexal tumor (12). Levels of CA125 and of the recently introduced HE4 may be assessed using various laboratory assays (11–15). Currently, four main combinations of the most popular laboratory assays are used in calculating ROMA values: Chemiluminescent microparticle immunoassays (CMIAs) for CA125 and for HE4; CMIA for CA125 and EIA for HE4; CanAg EIA for CA125 and EIA for HE4; and finally, electrochemiluminescence immunoassays (ECLIAs) for CA125 and for HE4.

The objective of the current study was to assess the diagnostic usefulness of ROMA in patients with ovarian cancer as compared with benign ovarian lesions and other gynecological malignancies, making use of a novel combination of known laboratory assays: CMIA for determination of CA125 levels and ECLIA for determination of HE4 levels. In addition, the usefulness of ROMA in patients subjected to prophylactic adnexectomy due to BRCA1 gene mutation carrier status was assessed.

Materials and methods

Patients

The study was conducted on 619 patients undergoing surgical treatment in the Department of Gynecological Surgery and Gynecological Oncology of Adults and Adolescents, Pomeranian Medical University (Szczecin, Poland) between November 2012 and December 2014. Of these, 354 women were premenopausal and the remaining 265 were postmenopausal. Patients who qualified for the study were those presenting at the clinic due to pathological lesions (tumors or cysts) within the adnexa, suspected ovarian cancer, other gynecological malignancies (except endometrial cancer) or BRCA1 gene mutation. After signing the informed consent form, all patients were subjected to blood collection procedures on the day of their hospital admission. Determinations of HE4 and CA125 marker levels were performed on the same day at the hospital's central laboratory. Due to use of laboratory assays that had not been previously tested for usefulness in the calculation of ROMA values, ROMA values were not calculated preoperatively and the respective results were not taken into account in everyday clinical practice. Additional diagnostic value was provided preoperatively only by the separate results of CA125 and HE4 assessments. Following histopathological assessment, patients were assigned into individual groups and subgroups, and ROMA values were calculated for each group and subgroup on the basis of standard mathematical formulas.

The inclusion criteria were as follows: i) ≥18 years of age; ii) presence of an ovarian cyst, ovarian tumor, tumor of the pelvis minor, ascites or persistent elevated CA125 levels; iii) consent to participate in the study; iv) availability of the final histopathological result. The following exclusion criteria were also applied: i) Renal diseases; ii) lung diseases; iii) creatinine levels of >1.3 mg/dl; iv) no consent to participate in the study.

Based on the inclusion and exclusion criteria, as well as on the result of histopathological examination, the patients were divided into the following groups: i) Ovarian cancer; ii) benign gynecological disorders of the adnexa; iii) other gynecological malignancies (excluding endometrial cancers) and metastases into ovaries; iv) epithelial ovarian tumors of borderline malignancy; v) BRCA1 mutation carriers. Detailed information regarding the division of the study population into groups and subgroups is presented in Table I.

Table I.

Patient characteristics.

Parameter	Ovarian cancers	Benign diseases	Other cancers	Borderline tumors	BRCA1 mutations
Total, n	162	342	33	23	59
Age, years
Mean	59.7	40.7	57.5	46.3	47.6
Range	24–90	18–89	27–87	19–83	34–64
Hormonal status, n
Premenopausal	38	256	10	11	39
Postmenopausal	124	86	23	12	20
Ovarian cancer histopathology, n (%)
Serous	132 (81.5)	–	–	16 (69.6)	–
Mucinous	9 (5.6)	–	–	6 (26.1)	–
Clear cell	8 (4.9)	–	–	0 (0.0)	–
Endometrioid	13 (8.0)	–	–	1 (4.3)	–
Ovarian cancer FIGO stage, n (%)
I and II	54 (33.3)	–	–	22 (95.7)	–
III and IV	108 (66.7)	–	–	1 (4.3)	–
Ovarian cancer grade, n (%)
1	34 (21.0)	–	–	–	–
2	54 (33.3)	–	–	–	–
3	74 (45.7)	–	–	–	–
Benign tumor histopathology, n (%)
Endometriosis	–	121 (35.4)	–	–	–
Teratoma	–	44 (12.9)	–	–	–
Serous cystadenoma	–	22 (6.4)	–	–	–
Mucinous cystadenoma	–	25 (7.3)	–	–	–
Cystadenofibroma	–	21 (6.1)	–	–	–
Follicular cysts	–	36 (10.5)	–	–	–
Paraovarian cysts	–	26 (7.6)	–	–	–
Hemorrhagic cysts	–	30 (8.8)	–	–	–
Inflammatory tumors	–	13 (3.8)	–	–	–
Cirrhosis	–	4 (1.2)	–	–	–

BRCA1, breast cancer 1; -, not applicable; FIGO, International Federation of Gynecology and Obstetrics.

Comparative analysis of groups and appropriate subgroups was conducted with regard to the serum HE4 and CA125 levels as well as the calculated ROMA values. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were calculated for each marker, with cut-off values of 35 U/ml for CA125, and 70 or 140 pmol/l (for premenopausal or postmenopausal patients, respectively) for HE4, as previously determined (11,12). Cut-off points for ROMA values were determined by the DeLong method (16). The diagnostic usefulness of each marker was assessed by means of the area under curve the receiver operating characteristic curve (ROC-AUC).

Laboratory methods

The HE4 serum levels of the marker were determined using the Roche Elecsys® assay (Roche Diagnostics, Basel, Switzerland) on a Cobas e601 apparatus. This is a one-step sandwich ECLIA for quantitative determination of HE4. The detection range for HE4 was 15.0–1,500 pmol/l; in case of values >1,500 pmol/l, the samples were diluted in a 1:20 ratio using Elecsys Diluent.

The serum CA125 levels were determined using the ARCHITECT CA125 II assay on an ARCHITECT 2200SR System (Abbott Diagnostics, Abbott Park, IL, USA). This is a two-step immunoassay to determine the presence of CA125 antigen using CMIA technology. CA125 and HE4 assays were conducted according to the manufacturer's instructions, with appropriate controls testing within the normal ranges (http://www.fda.gov/ohrms/dockets/ac/08/briefing/2008-4403b1-03%20ARCHITECT%20CA125II%20Package%20I nsert%20.pdf).

The predictive index (PI) of ROMA was calculated separately for premenopausal and postmenopausal patients using the following formulas: Premenopausal PI = −12.0 + 2.38 × ln[HE4] + 0.0626 × ln[CA125]; Postmenopausal PI = −12.0 + 1.04 × ln[HE4] + 0.732 × ln[CA125]; where ln is the natural logarithm. The CA125 levels wer in IU/ml and the HE4 levels were in pmol/l. Final ROMA values were calculated by inserting the obtained PI into the following formula: % ROMA = exp(PI) / [1-exp(PI)] × 100.

Statistical analysis

Descriptive characteristics of the examined population of patients were prepared, including the minimum, maximum, mean and median values. Additionally, the scatter diagrams of the empirical values of markers were plotted for individual study groups. The mean/median values in individual groups and subgroups were compared using the non-parametric Mann-Whitney U test.

A contingency table was used in the assessment of the diagnostic usefulness of CA125 and HE4 assays and ROMA values, and subsequent calculation of the following parameters: Sensitivity = TP/(TP+FN); specificity = TN/(FP+TN); PPV = TP/(TP+FP); and NPV = TN/(FN+TN); where TP is the number of true positives, FN is the number of false negatives, TN is the number of true negatives and FP is the number of false positives.

Diagnostic performance was assessed using receiver operating characteristic (ROC) curves based on continuous variables. HE4, CA125, and ROMA represented diagnostic variables acting as stimulants which increase the probability of ovarian cancer proportionally to their rising value. The area under the curve (AUC), standard error, and confidence interval values for AUC were calculated according to the non-parametric method of DeLong (16). This method was used to compare AUCs considering the fact that measurements of HE4, CA125 and ROMA were performed for the same objects (groups of patients). The level of significance was set as P<0.05.

Results

Serum levels and comparisons of biomarkers (HE4 and CA125) and ROMA algorithm in different gynecological conditions

Table II lists the median values as well as the appropriate ranges for HE4, CA125 and ROMA values in ovarian cancer patients (with consideration given to the histopathological type, stage and differentiation of cancer), epithelial borderline tumors, patients with other gynecological cancers and patients with metastatic ovarian tumors, as well as in patients with benign ovarian lesions according to different histopathological diagnoses. Hormonal status of patients was taken into consideration when presenting the results. When analyzing all patients (regardless of their menopausal status), the median HE4, CA125 and ROMA values in ovarian cancer patients (HE4, 333.35 pmol/l; CA125, 380.35 U/ml; ROMA, 90.15%) were found to be significantly higher than in patients with benign ovarian lesions and BRCA1 mutation (HE4, 46.8 pmol/l, P<0.001; CA125, 20.6 U/ml, P<0.001; ROMA, 6.9%, P<0.001), patients with other gynecological cancers/ovarian metastases (HE4, 71.8 pmol/l, P<0.001; CA125, 66.2 U/ml, P<0.001; ROMA, 21.7%, P<0.001) and patients with borderline ovarian tumors (HE4, 63.5 pmol/l, P<0.001; CA125, 39.25 U/ml, P<0.001; ROMA, 12.9%, P<0.001). All comparisons in postmenopausal patients were similar to the above, with HE4, CA125 and ROMA values being significantly higher in ovarian cancer patients than in patients with benign gynecological lesions, epithelial borderline tumors, other cancers and metastatic tumors. By contrast, the behavior of serum levels of HE4 and CA125 and ROMA values in premenopausal patients is somewhat different: When comparing the ovarian cancer group with the benign lesion group, the marker levels and ROMA values remain significantly higher in the former; however, when comparing the ovarian cancer group with the group of other gynecological cancers or metastatic tumors, significant differences can be observed only with regard to CA125 (P=0.0153) and ROMA (P=0.0496). Additionally, in cases of borderline tumors, CA125 and ROMA values perform better as biomarkers than HE4 (P=0.0016, P=0.0154 and P=0.0508, respectively).

Table II.

Serum CA125, HE4 and ROMA levels according to histology, FIGO stage and tumor grade.

	Premenopausal						Postmenopausal
	CA125 (U/ml)		HE4 (pmol/l)		ROMA (%)		CA125 (U/ml)		HE4 (pmol/l)		ROMA (%)
	Median	Range	Median	Range	Median	Range	Median	Range	Median	Range	Median	Range
Ovarian cancer (all)	265.4	14.0–4,638.8	89.5	15.0–1,500.0	32.3	3.2–99.7	416.9	9.0–5,887.0	470.3	15.0–4,940.0	95.2	4.5–100.0
Serous	352.4	14.0–4,638.8	99.7	15.0–1,500.0	35.2	2.0–99.7	425.0	9.0–5,887.0	555.9	18.0–4,940.0	96.3	4.5–100.0
Mucinous	27.0	26.4–37.9	51.9	45.4–75.6	8.4	6.2–18.0	87.2	11.3–600.0	76.4	15.0–538.0	52.9	7.8–95.8
Clear cell	146.3	95.8–196.8	75.8	49.4–102.1	21.1	8.1–34.1	787.5	96.4–1,725.5	276.9	76.0–849.9	94.2	44.2–98.2
Endometrioid	500.0	500.0	64.5	64.5	21.0	21.0	448.8	41.5–2,996.9	390.0	46.1–1,235.0	94.8	26.7–99.3
FIGO stage
I and II	95.8	14.0–1,252.0	64.5	15.0–211.6	18.0	2.5–75.1	174.9	9.0–2,347.0	108.8	15.0–1,235.0	56.8	4.5–99.3
III and IV	543.5	64.3–4,638.8	343.3	20.7–1,500.0	90.7	9.0–99.7	540.0	18.0–5,887.0	639.0	62.8–4,940.0	96.9	25.0–100.0
Grade
1	69.7	25.3–459.7	47.7	15.0–414.3	7.2	2.5–93.9	66.5	9.0–245.0	86.4	18.0–283.5	47.1	4.5–85.9
2	389.1	14.0–2,096.0	94.1	20.7–1,500.0	44.5	8.4–99.7	415.9	41.5–5,659.0	385.3	35.0–3,608.0	94.6	26.7–99.0
3	350.8	54.3–4,638.8	211.6	20.4–1,500.0	75.1	2.0–99.7	559.1	11.2–5,109.8	639.0	30.0–4,001.0	97.0	12.0–100
Borderline tumors	38.9	5.4–206.3	51.0	39.2–89.0	8.0	4.0–25.3	79.9	6.7–421.2	103.8	40.1–274.2	49.0	5.0–88.0
Other cancers and metastatic ovarian tumors	42.4	8.2–1,090.7	50.4	32.9–356.6	7.4	2.6–91.0	93.7	11.1–4,855.4	81.6	15.0–580.1	43.1	8.0–94.9
Benign diseases (all)	22.7	2.4–2,257.0	44.2	15.0–269.8	5.8	2.0–99.3	15.6	2.3–1,255.0	56.4	15.0–282.6	12.9	1.1–88.8
Endometriosis	46.8	8.8–377.0	45.2	17.8–86.7	6.4	1.8–24.1	18.9	6.7–82.1	47.2	27.5–73.0	11.4	1.8–35.3
Teratoma tumors	15.9	7.7–51.9	44.4	26.3–69.1	5.8	1.7–14.7	15.3	6.3–18.2	49.1	45.7–72.4	11.2	6.7–17.9
Cystadenoma tumors	22.3	8.6–125.9	41.4	15.0–101.6	5.2	2.2–32.8	19.2	2.3–272.0	56.9	15.0–282.6	14.9	1.1–86.4
Follicular cysts	4.6	9.1–88.0	43.5	24.2–84.7	5.5	1.6–22.1	11.7	3.2–79.8	65.7	41.5–206.5	13.1	5.3–65.7
Paraovarian cysts	15.3	4.1–111.4	48.2	36.4–78.0	7.1	3.7–18.5	9.3	4.9–502.7	52.0	40.5–186.8	8.7	4.8–87.0
Hemorrhagic cysts	14.6	5.5–274.8	44.2	24.8–88.6	5.7	1.7–23.3	–	–	–	–	–	–
Inflammatory tumors	101.4	2.4–1,670.6	54.3	39.0–269.8	9.3	3.8–83.1	–	–	–	–	–	–
Cirrhosis	2,257.0	2,257.0	109.2	109.2	41.3	41.3	254.9	161.3–1,256.0	167.8	115.8–199.2	83.4	75.7–88.8
BRCA1 mutations	15.5	3.0–148.1	34.2	15.0–109.1	3.2	2.4–33.6	15.2	3.0–36.3	51.3	15.0–83.9	10.6	3.9–26.8

-, no diagnoses made; CA125, carbohydrate antigen 125; HE4, human epididymis protein 4; ROMA, Risk of Ovarian Malignancy Algorithm; FIGO, International Federation of Gynecology and Obstetrics; BRCA1, breast cancer 1.

The comparison of the group of patients with benign ovarian lesions and patients with borderline ovarian tumors also appears to be clinically significant. In all study patients, regardless of their menopausal status, the values of HE4, CA125 and ROMA were observed to be significantly higher in borderline tumors (P<0.001, P=0.0152, and P<0.001 for HE4, CA125, and ROMA, respectively). In postmenopausal women, HE4, CA125 and ROMA values were significantly higher in patients with borderline tumors than in patients with benign tumors (P=0.0058, P=0.0394 and P=0.0316, respectively). In younger, premenopausal women, the situation is different, with only HE4 and ROMA being significantly higher in the group of borderline tumor patients than in patients with benign lesions (P=0.0169 and P=0.0441 for HE4 and ROMA, respectively; P=0.1212 for CA125). The data obtained regarding CA125, HE4 and ROMA values are graphically presented in Figs. 1 and 2.

Figure 1.

Range and median values of HE4, CA125 and ROMA results in studied groups: (A) All, (B) premenopausal and (C) postmenopausal patients. HE4, human epididymis protein 4; CA125, carbohydrate antigen 125; ROMA, Risk of Ovarian Malignancy Algorithm; BRCA1, breast cancer 1.

Figure 2.

HE4 and CA125 for preoperative detection of epithelial ovarian cancer in patients with a pelvic mass. Scatter plots show HE4 and CA125 levels in all patients, postmenopausal patients and premenopausal patients. HE4, human epididymis protein 4; CA125, carbohydrate antigen 125.

Assessment of the patterns of HE4, CA125, and ROMA values within the ovarian cancer group indicated that the analyzed marker levels and the ROMA values were significantly lower in mucinous tumors compared with serous or endometrial tumors (P=0.0026, P=0.0004 and P=0.0024 for ROMA, HE4 and CA125, respectively). No significant differences were observed between mucinous and clear cell carcinomas with regard to HE4 and ROMA values; however, significantly higher CA125 levels were observed in clear cell carcinomas compared with mucinous tumors (P=0.0183). When restricting the analysis to the postmenopausal group, similar significantly lower concentrations of HE4 and CA125 and ROMA values were observed in the subgroup of mucinous tumors as compared with serous tumors (P=0.0121, P=0.0281 and P=0.0093, respectively) and endometrial tumors (P=0.0495, P=0.035 and P=0.0277, respectively). No comparisons were made in the postmenopausal group due to the insufficient number of cases, with respective values presented in Table II. In the entire study population as well as in the groups of pre- and postmenopausal patients, significantly higher concentrations of markers and ROMA values were observed in patients with advanced ovarian cancer [International Federation of Gynecology and Obstetrics (FIGO) stages III and IV] as compared with less advanced ovarian cancer (FIGO stages I and II) (P<0.0001 for ROMA, HE4 and CA125).

Cancer differentiation grade (G) also affected the values of the tested markers. Analysis of all cases (regardless of hormonal status) revealed similar results as those for postmenopausal patients: HE4, CA125, and ROMA values were significantly higher in G3 patients as compared with G1 patients and with G2 patients (P<0.0001); for G1 vs. G2 groups, statistical significance was observed only for HE4 and ROMA values (P<0.0001). In premenopausal women, significant differences were observed only between the highly differentiated (G1) and poorly differentiated (G3) tumors, with all analyzed parameters being higher in the G3 group (P=0.0132, P=0.0154 and P=0.0030 for ROMA, HE4 and CA125, respectively). When comparing G2 and G3 tumors, there were no significant differences in any parameters between the two groups. Analysis of HE4, CA125 and ROMA values between G1 and G2 tumors revealed statistically significant differences only for serum HE4 levels and ROMA values (P=0.0075 and P=0.0408 for ROMA and HE4, respectively).

Table II shows the median values of HE4, CA125 and ROMA in individual subgroups of benign ovarian lesions. In all patients, as well as in premenopausal patients only, the median CA125 levels were higher in the endometrial cyst subgroup as compared to the subgroups with teratomas, cystadenomas, follicular cysts, paraovarian cysts or hemorrhagic cysts. There were no significant differences between both groups with regard to HE4 and ROMA values. Significantly higher values of CA125, HE4 and ROMA were observed in patients with endometrial tumors as compared with patients carrying BRCA1 gene mutations (P=0.0437, P=0.0336 and P<0.0001 for ROMA, HE4 and CA125, respectively). Higher values of all tested parameters were also found in patients with inflammatory ovarian tumors as compared with patients with endometrial cysts (P=0.0032, P=0.0033 and P=0.0039 for ROMA, HE4 and CA125, respectively). No significant differences were observed in the marker levels or the ROMA value between the compared subgroups of postmenopausal women. In all analyzed BRCA1 mutation carriers, the mean marker levels and ROMA values were not significantly different from those observed in the remaining patients in the group of benign ovarian lesions. In younger, premenopausal patients, statistically significant differences between BRCA1 mutation carriers and the remaining patients were observed in each subgroup: HE4 and ROMA values were significantly higher in patients with endometriosis, teratomas, follicular cysts, hemorrhagic cysts, paraovarian cysts, and inflammatory tumors than in BRCA1 mutation carriers (P<0.0001, P=0.0035, P=0.0348, P=0.0005, P=0.0036 and P=0.0021 for HE4, and P<0.0001, P=0.0027, P=0.0433, P=0.0005, P=0.0023 and P=0.0033 for ROMA, in endometriosis, teratomas, follicular cysts, inflammatory tumors, hemorrhagic cysts and paraovarian cysts, respectively). Serum levels of CA125 were significantly higher in premenopausal women with endometriosis, follicular cysts and inflammatory tumors than in BRCA1 mutation carriers (P<0.0001, P<0.0001 and P=0.0489 for CA125 in endometriosis, follicular cysts and inflammatory tumors, respectively). Latent ovarian cancer was diagnosed in histopathological material collected during prophylactic surgery in three BRCA1 mutation carriers. ROMA values measured in those patients were 19 and 3% for two premenopausal women and 25% in one postmenopausal patient.

ROC-AUC analysis of ROMA, HE4 and CA125

Table III shows ROC-AUC values for the examined markers and ROMA values with consideration of hormonal status and comparison of results for different statuses. All parameters evaluated in a separate manner meet the criteria of useful diagnostic tests. No statistical superiority of CA125 was observed in any comparison in Table III, while ROMA value was found to be significantly statistically superior to CA125 in three comparisons: Patients with advanced ovarian cancers vs. benign ovarian lesions, P=0.0071 (all) and P=0.0235 (postmenopausal); and postmenopausal ovarian cancer patients vs. other gynecological cancers and metastatic ovarian tumors, P=0.016. The diagnostic value of ROMA was also, in many cases, better than that of HE4 when analyzed separately; superiority of ROMA (P=0.0012) and CA125 (P=0.0236) over HE4 is significant, particularly in patients with early-stage ovarian cancer vs. benign disease.

Table III.

Values and comparisons of ROC-AUC for ROMA, CA125 and HE4 in the studied groups.

A, All ovarian cancers vs. benign ovarian diseases and BRCA1 mutation patients
		Comparison of ROC-AUC, P-value
Menopausal status	Tumor marker ROC-AUC (95% CI)	ROMA vs. CA125	ROMA vs. HE4	CA125 vs. HE4
All		0.3692	0.0004	0.1285
ROMA	0.926 (0.894–0.957)
HE4	0.879 (0.838–0.921)
CA125	0.911 (0.881–0.941)
Premenopausal		0.1619	0.2064	0.072
ROMA	0.813 (0.712–0.916)
HE4	0.783 (0.673–0.892)
CA125	0.879 (0.822–0.938)
Postmenopausal		0.6677	0.0014	0.0453
ROMA	0.939 (0.907–0.971)
HE4	0.889 (0.843–0.935)
CA125	0.934 (0.901–0.968)
B, Advanced ovarian cancers vs. benign ovarian diseases and BRCA1 mutation patients
		Comparison of ROC-AUC, P-value
Menopausal status	Tumor marker ROC-AUC (95% CI)	ROMA vs. CA125	ROMA vs. HE4	CA125 vs. HE4
All		0.0071	0.1249	0.3607
ROMA	0.992 (0.984–0.999)
HE4	0.977 (0.957–0.996)
CA125	0.965 (0.946–0.9840
Premenopausal		0.2646	0.2850	0.3506
ROMA	0.982 (0.955–1.000)
HE4	0.926 (0.821–1.000)
CA125	0.977 (0.954–1.000)
Postmenopausal		0.0235	0.0406	0.5872
ROMA	0.989 (0.981–0.999)
HE4	0.975 (0.958–0.991)
CA125	0.971 (0.949–0.992)
C, Non-advanced ovarian cancers vs. benign ovarian diseases and BRCA1 mutation patients
		Comparison of ROC-AUC, P-value
Menopausal status	Tumor marker ROC-AUC (95% CI)	ROMA vs. CA125	ROMA vs. HE4	CA125 vs. HE4
All		0.7657	0.0012	0.0236
ROMA	0.803 (0.721–0.883)
HE4	0.690 (0.588–0.793)
CA125	0.816 (0.746–0.884)
Premenopausal		0.1201	0.0355	0.1004
ROMA	0.660 (0.484–0.836)
HE4	0.650 (0.709–0.896)
CA125	0.803 (0.709–0.896)
Postmenopausal		0.3741	0.0048	0.0123
ROMA	0.819 (0.723–0.914)
HE4	0.678 (0.546–0.809)
CA125	0.848 (0.761–0.935)
D, Ovarian cancers vs. other malignant neoplasms and metastatic ovarian tumors
		Comparison of ROC-AUC, P-value
Menopausal status	Tumor marker ROC-AUC (95% CI)	ROMA vs. CA125	ROMA vs. HE4	CA125 vs. HE4
All		0.3116	0.851	0.4518
ROMA	0.765 (0.688–0.840)
HE4	0.767 (0.630–0.839)
CA125	0.733 (0.630–0.835)
Premenopausal		0.4531	0.2582	0.3000
ROMA	0.700 (0.518–0.882)
HE4	0.673 (0.495–0.849)
CA125	0.753 (0.559–0.945)
Postmenopausal		0.016	0.7613	0.1253
ROMA	0.801 (0.719–0.884)
HE4	0.807 (0.731–0.884)
CA125	0.716 (0.591–0.841)
E, Ovarian cancers vs. borderline tumors
		Comparison of ROC-AUC, P-value
Menopausal status	Tumor marker ROC-AUC (95% CI)	ROMA vs. CA125	ROMA vs. HE4	CA125 vs. HE4
All		0.941	0.0058	0.1419
ROMA	0.835 (0.762–0.907)
HE4	0.784 (0.713–0.855)
CA125	0.836 (0.767–0.909)
Premenopausal		0.2816	0.1057	0.1007
ROMA	0.743 (0.602–0.883)
HE4	0.696 (0.551–0.842)
CA125	0.815 (0.689–0.942)
Postmenopausal		0.942	0.1494	0.5279
ROMA	0.827 (0.730–0.924)
HE4	0.794 (0.707–0.882)
CA125	0.823 (0.715–0.932)

ROC-AUC, area under the receiver operating characteristic curve; ROMA, Risk of Ovarian Malignancy Algorithm; CA125, carbohydrate antigen 125; HE4, human epididymis protein 4; BRCA1, breast cancer 1; CI, confidence interval.

Sensitivity, specificity, PPV and NPV of ROMA, HE4 and CA125 in the analyzed groups of patients

Table IV presents the values of sensitivity, specificity, PPV and NPV for CA125, HE4, and ROMA in different inter-group comparisons. Based on the DeLong method we calculated new cut-off values for ROMA algorithm based on new combination of CA125 and HE4 kits: 14.1% for premenopausal and 25% for postmenopausal women. The sensitivities of CA125 and ROMA were comparable for all patients in the present study at 88.4 vs. 88.3%; however, the sensitivity of ROMA measurement was lower than that of CA125 in premenopausal women (71.1 vs. 84.2%) and higher than that of HE4 in postmenopausal women (93.6 vs. 73.0%). In terms of specificity, ROMA and HE4 were markedly superior to CA125 in virtually all comparisons. In the majority of comparisons, the PPV and NPV were also more favorable for ROMA and HE4 values than for CA125. CA125 was only slightly superior in differential diagnostics of ovarian cancer and hemorrhagic, follicular and paraovarian cysts, as well as in differential diagnostics of borderline tumors and benign ovarian lesions.

Table IV.

Sensitivity, specificity, PPV and NPV of ROMA, HE4 and CA125 between compared groups.

	Sensitivity (%)			Specificity (%)			PPV (%)			NPV (%)
Groups compared	All	PM	M	All	PM	M	All	PM	M	All	PM	M
OC vs. benign group
ROMA	88.3	71.1	93.6	88.2	90.1	82.9	75.3	48.2	86.6	94.9	96.1	91.6
HE4	–	65.8	73.0	–	93.8	94.3	–	58.1	93.9	–	95.4	74.4
CA125	88.4	84.2	89.7	70.2	67.0	79.1	54.9	25.0	83.7	93.6	97.0	86.5
OC vs. endometriosis
ROMA	88.3	71.1	93.5	95.0	95.5	88.9	95.9	84.4	99.2	85.8	90.7	50.0
HE4	–	65.8	73.0	–	99.1	100.0	–	96.2	100.0	–	89.4	20.9
CA125	88.4	84.2	89.7	41.3	38.7	77.8	67.1	32.0	98.3	72.5	87.7	35.0
OC vs. teratoma tumors
ROMA	88.3	71.1	93.5	97.7	97.4	100.0	99.3	96.4	100.0	69.4	77.1	42.8
HE4	–	65.8	73.0	–	100.0	100.0	–	100.0	100.0	–	74.1	15.0
CA125	88.4	84.2	89.7	90.9	89.5	100.0	97.3	88.9	100.0	67.8	85.0	31.6
OC vs. cystadenoma tumors
ROMA	88.3	71.1	93.6	80.6	88.0	76.2	91.6	90	92.1	73.9	66.7	80.0
HE4	–	65.8	73.0	–	92.0	92.9	–	92.6	96.8	–	63.9	53.4
CA125	88.4	84.2	89.7	68.7	76.0	64.3	87.4	84.2	88.3	70.8	76.0	67.5
OC vs. follicular, hemorrhagic and paraovarian cysts
ROMA	88.3	71.1	93.6	85.9	84.9	88.5	91.7	72.9	97.5	80.6	83.6	74.2
HE4	–	65.8	73.0	–	86.1	92.3	–	73.5	97.9	–	81.2	41.4
CA125	88.4	84.2	89.7	90.2	84.9	88.5	94.2	76.2	97.4	81.4	90.3	63.9
A-OC vs. benign group
ROMA	99.1	94.4	100.0	88.2	90.1	82.9	68.9	36.9	82.9	99.7	99.6	100.0
HE4	–	88.9	89.9	–	93.8	94.3	–	47.1	93.0	–	99.3	91.7
CA125	95.3	100.0	94.4	70.2	67.0	79.1	46.2	15.8	79.3	98.3	100.0	94.3
NA-OC vs. benign group
ROMA	70.4	52.6	80.0	88.2	90.1	82.9	44.8	25.6	60.9	95.7	96.7	92.6
HE4	–	47.4	31.4	–	93.4	94.3	–	33.3	64.7	–	96.5	80.5
CA125	77.8	73.7	80.0	70.2	67.0	79.1	26.1	12.7	56.0	95.9	97.5	92.2
OC vs. other malignant neoplasms
ROMA	88.3	–	–	57.6	–	–	91.1	–	–	50.0	–	–
HE4	–	–	–	–	–	–	–	–	–	–	–	–
CA125	88.4	–	–	42.2	–	–	88.4	–		42.2	–	–
OC vs. borderline ovarian tumors
ROMA	88.3	71.1	93.6	59.1	72.7	45.5	94.1	90.0	95.1	40.6	42.1	38.5
HE4	–	65.8	73.0	–	72.7	58.3	–	89.3	94.9	–	38.1	17.1
CA125	88.4	84.2	89.7	45.5	40.0	50.0	92.4	84.2	94.9	34.5	40.0	31.6
Borderline ovarian tumors vs. benign group
ROMA	11.8	9.9	17.1	59.1	72.7	45.5	83.9	90.6	75.0	3.6	2.9	5.4
HE4	–	6.2	5.7	–	72.7	58.3	–	85.7	54.6	–	2.9	6.6
CA125	29.8	32.3	20.1	45.5	40.0	50.0	90.8	94.1	78.6	3.5	2.0	6.7

PPV, positive predictive value; NPV, negative predictive value; ROMA, Risk of Ovarian Malignancy Algorithm; HE4, human epididymis protein 4; CA125, carbohydrate antigen 125; OC, ovarian cancer; A-OC, advanced ovarian cancer; NA-OC, non-advanced ovarian cancer; PM, premenopausal; M, menopausal; -, data not available.

Discussion

Numerous studies have demonstrated that combined analysis of CA125 and HE4 allows for significant improvements of the sensitivity and specificity of prediction of pathological lesions within the ovaries (11,12,17–23). For Moore et al (11), this became the basis for the development of a new diagnostic algorithm based on the levels of these two serum proteins. As reported by the authors, a logistic regression model, developed from the examination of >500 patients, was characterized by sensitivities and specificities of 76.5 and 74.8% in premenopausal patients, and 92.3 and 74.7% in postmenopausal patients, respectively (12). Their first study, presenting the diagnostic importance and basic principles of ROMA, made use of the EIA assay (Fujirebio Diagnostics, Inc., Malvern, PA, USA) to determine HA4 and the ARCHITECT CA125 II assay (Abbott Diagnostics, Abbott Park, IL, USA) for determination of CA125. Finally, the FDA approved the algorithm for the prediction of pathological lesions within the ovaries on the basis of the study conducted by Moore et al (12) in 2011. Subsequently, the era of examinations conducted with the use of this algorithm began (17–24). The diagnostic usefulness of the algorithm in conjunction with various diagnostic tests was examined, with the four pairs of assays listed in the introduction making their way into routine clinical practice. The usefulness of the algorithm was also assessed in relation to different control groups (healthy individuals and patients with various pathological lesions within the adnexa) (24–30). The algorithm was rapidly established in the context of diagnosis of pathological adnexal lesions due to its simplicity, relatively low price and wide availability.

The present study, comprising a sample of >600 patients, demonstrated that our novel combination of laboratory assays may be routinely used in clinical practice, as the obtained values of sensitivity, specificity, PPV and NPV for ROMA, and the results of the ROC-AUC assessment, are very similar or superior to those obtained by other researchers who conducted studies using the widely available combinations of HE4 and CA125 assays. The current study was conducted in patients with ovarian cancer and other adnexal pathologies, benign or malignant, which are commonly included in differential diagnostics of ovarian cancer. Deliberately excluded from the study were patients with adnexitis and liver cirrhosis, who pose significant difficulties in differential diagnostics due to the extremely high CA125 levels and the accompanying ascites.

When comparing the group of ovarian cancers with all benign diseases included in the study, in postmenopausal women, ROMA was characterized by the highest sensitivity (93.6%), specificity (82.9%), PPV (86.6%) and NPV (91.6%) as compared with CA125 and HE4 analyzed separately. In premenopausal women, ROMA was characterized by the highest specificity (90.1%) and PPV (48.2%), but the selectivity (71.1%) and NPV (96.1%) of the algorithm were marginally inferior to those of CA125 (84.2 and 97%, respectively). The highest ROC-AUCs were observed for ROMA for the entire study population (0.926) and in postmenopausal patients (0.939). In premenopausal patients, better results were obtained for CA125 (0.874) than for ROMA (0.813). The differences, however, were not statistically significant.

The literature on this topic contains varying reports on the usefulness of ROMA in the differential diagnosis of adnexal pathologies in groups of patients similar to those analyzed in the present study. The majority of results are consistent with those obtained in the current study (12–15,25–30). In one of the earlier studies, highly similar results were obtained by Moore et al (11), who examined samples from 476 patients (including 89 cases of ovarian cancer); however, the sensitivity, specificity, PPV and NPV results in the current study were somewhat better. It must be considered that these authors did not compare the ROMA results with CA125 and HE4 levels as analyzed separately. Lenhardt et al (27) assessed the diagnostic efficacy of ROMA by comparing two standard combinations of assays (CA125 II with HE4 ARCHITECT; and CA125 II ARCHITECT with HE4 EIA). Analyzing the ROC-AUCs, the authors obtained results that were nearly identical to those obtained in the present study: 0.831 (HE4 and CA125 II ARCHITECT) and 0.820 (CA125 II ARCHITECT and HE4 EIA) for premenopausal patients; and 0.939 and 0.932, respectively, for postmenopausal patients.

In their ROMA calculations, Anton et al (24) used a combination of CA125 Elecsys and HE4 EIA assays, with different cut-off points: One set equal to these proposed to Moore et al (12) and another resulting from their own statistical analyses. ROC-AUC values of 0.791 and 0.840 were obtained for premenopausal and postmenopausal women, respectively; these results were inferior to those obtained in the current study. The sensitivity and specificity values using the standard cut-off points were 77.8 and 69%, respectively, for premenopausal women, and 72.2 and 81%, respectively, for postmenopausal women. Using their own optimum cut-off points (24), the authors arrived at different results, particularly improving the specificity parameter. However, in the conclusions, the authors highlighted that similar results had been obtained for ROMA and CA125 and HE4 levels, as analyzed separately.

Using the combination of CA125 CanAg and HE4 EIA assays, Montagnana et al (31) were able to obtain excellent results for ROMA, particularly in postmenopausal women, with ROC-AUC values (0.77 in premenopausal patients and 0.92 for postmenopausal patients) that were nearly identical to those obtained in the current study.

However, it must be considered that even the ‘negative’ studies never questioned the fact that the algorithm in question met all the criteria for a diagnostic test; said studies challenged only its superiority to other diagnostic methods (10,16,32). The usefulness and widespread use of the test, shown not only by the ROC-AUC values or the sensitivity and specificity parameters, but also by the availability, objectivity and cost, favor ROMA-based assessments in the majority of cases.

Endometriosis is a common ovarian pathology observed in young premenopausal women. In many cases, patients are childless women in whom any surgical intervention should consist of the best fertility-sparing procedure, characterized by the lowest possible degree of invasiveness so as to reduce the risk of adhesions. To date, it has been fairly common for high CA125 values in endometriosis patients (sometimes even close to those observed in ovarian cancer patients) to lead to an excessively aggressive surgical treatment, particularly at centers with less clinical experience. Inclusion of HE4 and, later, ROMA, into preoperative diagnosis (12) significantly increased the accuracy of preoperative diagnoses of endometriosis. The results of the current study confirm the excellent specificity of ROMA, which allows correct diagnoses to be made with >90% accuracy. By comparison, the specificity of CA125 in diagnosing endometrial lesions determined in the current study was only 38.7%. Additionally, as revealed by the comparative analysis between the study groups, of all the study parameters, only CA125 was statistically significantly elevated in patients with endometriosis compared with patients with other benign adnexal pathologies; this was responsible for the high number of false positive results. HE4 and ROMA performed much better in this comparative analysis. The usefulness of HE4 and ROMA in preoperative diagnosis of endometriosis was also confirmed in other studies (14,33,34).

Borderline tumors are a significant challenge for gynecologists and gynecological oncologists. It is known that, in premenopausal women, these tumors may be successfully treated in a sparing fashion, particularly at earlier clinical stages. However, para-aortic lymphadenectomy should be performed in patients with suspected ovarian cancer; this burdensome procedure is unnecessary in cases of borderline tumors. Therefore, preoperative suspicion of borderline tumor is a highly important issue; similar to ovarian cancers, surgery for these tumors should be performed at high volume hospitals, where a large number of patients with ovarian cancer are operated, such as teaching hospitals. In the current study, the sensitivity, specificity, PPV and NPV were assessed with regard to the comparison of cancers vs. borderline tumors, and borderline tumors vs. benign, non-neoplastic lesions within the ovaries. The superiority of ROMA over CA125 and HE4 was demonstrated in the differential diagnosis of borderline tumors vs. cancers. The results of the use of ROMA as well as CA125 and HE4 levels in the differential diagnosis of endothelial tumors of borderline malignancy and non-neoplastic ovarian cysts were most favorable for separately determined CA125; however, good results were obtained for none of the examined parameters with the exception of PPV. A comparative analysis of the groups demonstrated that borderline tumors were characterized by significantly lower levels of CA125 and values of ROMA than ovarian cancers, and also that significantly higher serum levels of CA125 and HE4 were observed as compared to benign ovarian pathologies. The median ROMA values for borderline tumors in the current study are similar to those obtained by Partheen et al (35), equal to 8.0% (4–25.3%) in the present study and 14.4% (3.6–44.6%) in the reference study. Analogously, median results for postmenopausal women were 44.8% (5–88%) and 34.2% (9.5–83.7%), respectively. Unfortunately, none of the analyzed parameters was ideal in this pathology. Interesting results on the use of ROMA in patients with borderline ovarian tumors were presented by Gizzo et al (36), who showed that knowledge of the ROMA value reduced the risk of under-diagnosis (36).

BRCA1 mutation carriers undergo screening tests until the decision to perform prophylactic adnexectomy is made. In some cases, occult ovarian cancers are detected following histopathological assessment of ovaries and fallopian tubes resected during surgery (37–39). The reported incidence of diagnosing occult ovarian cancer varies in the literature from 1.9% (36) to 16% (39). To date, no reports have been published regarding the usefulness of HE4 or ROMA in preoperative diagnosis aimed at possible detection of occult ovarian cancers in this group of patients. In the group of 59 BRCA1 mutation carriers in the present study, occult cancer was detected in 3 patients. Of these, 2 patients had elevated ROMA values and each of them presented with elevated levels of HE4 or CA125. It appears that CA125, HE4 and ROMA values should be determined in female patients subjected to prophylactic surgeries; in cases with elevated values, this would be associated with intraoperative examination. In addition, particular attention should be paid to the fact that HE4 and ROMA values are significantly lower in the group of premenopausal BRCA1 mutation carriers, and therefore even marginal increases in these values should trigger oncological alertness. This, however, requires further investigation.

In conclusion, the novel combination of laboratory tests (ECLIA for determination of HE4 levels and CMIA for determination of CA125 levels) for use in calculating ROMA values and stratifying ovarian tumor patients into groups of high or low risk of ovarian cancer, meets the criteria of a very good diagnostic test. Similarly to other assays associated with ROMA calculations, the highest diagnostic precision was demonstrated in postmenopausal women. The algorithm is also useful in diagnosing ovarian endometriosis changes in younger women. Further studies are required to assess the usefulness of ROMA in the group of BRCA1 mutation carriers.