A radiologic-laparoscopic model to predict suboptimal (or complete and optimal) debulking surgery in advanced ovarian cancer: a pilot study.

Introduction: Medical models assist clinicians in making diagnostic and prognostic decisions in complex situations. In advanced ovarian cancer, medical models could help prevent unnecessary exploratory surgery. We designed two models to predict suboptimal or complete and optimal cytoreductive surgery in patients with advanced ovarian cancer.
Methods: We collected clinical, pathological, surgical, and residual tumor data from 110 patients with advanced ovarian cancer. Computed tomographic and laparoscopic data from these patients were used to determine peritoneal cancer index (PCI) and lesion size score. These data were then used to construct two-by-two contingency tables and our two predictive models. Each model included three risk score levels; the R4 model also included operative PCI, while the R3 model did not. Finally, we used the original patient data to validate the models (narrow validation).
Results: Our models predicted suboptimal or complete and optimal cytoreductive surgery with a sensitivity of 83% (R4 model) and 69% (R3 model). Our results also showed that PCI>20 was a major risk factor for unresectability.
Conclusion: Our medical models successfully predicted suboptimal or complete and optimal cytoreductive surgery in 110 patients with advanced ovarian cancer. Our models are easy to construct, based on readily available laboratory test data, simple to use clinically, and could reduce unnecessary exploratory surgery in this patient group.

Introduction

A clinical model is a tool that quantifies the individual contribution of several factors (clinical, analytical, radiological, etc.) when evaluating the diagnosis or prognosis of a specific patient. Its purpose is to solve complex or uncertain decisions in different scenarios, to give precise individual prognoses, and to save costs without increasing risks for the patient. Its elaboration is mathematical, based on the statistical association between certain factors and a result. The model calculates the probabilities (predictive values) of obtaining a concrete result or a certain degree of efficacy for a medical intervention.1,2

All models are constructed retrospectively on a wide range of already available clinical information. Later, these models must be validated in different environments to verify their accuracy and generalize their usefulness. This is done by analyzing the calibration between the expected results based on the model and the observed results, and their discrimination among opposite-sign results. This proof of reproducibility and the confirmation of the likelihood of changing clinical decisions that imply improved clinical results constitute the model’s maximum degree of evidence.3

A determining factor for survival in advanced ovarian cancer (AOC) is removal of the entire tumor burden to achieve either complete removal of the tumor upon visual inspection (complete cytoreductive surgery, CCS) or a residual tumor of <1 cm (optimal cytoreductive surgery, OCS). Only these two surgical outcomes improve survival.4 But in the most advanced cases of disease, the likelihood of obtaining CCS or OCS versus suboptimal cytoreductive surgery (SCS) remains uncertain before or even during the surgery. Specific imaging tests and laparoscopic exploration are fundamental when evaluating possibilities for OCS. When the probability of CCS is not high, it is necessary to rely on neoadjuvant treatments that decrease the tumor burden to allow for surgery.4

Accordingly, a SCS could be considered a surgical failure in the attempt to improve survival and, therefore, unnecessary high risk surgery. The objective of this study was to build a model that combines and enhances the information from imaging tests and laparoscopy to determine the probability of SCS versus CCS or OCS for each patient. This may lead to a more appropriate choice of therapeutic strategy. We also validated the model by analyzing the same data used for its construction (narrow validation), as a first step before prospectively addressing its reproducibility (broad validation) in subsequent studies with new samples of patients.

Methods

Patients

110 consecutive patients diagnosed by CT-scan with AOC were treated at the Multidisciplinary Unit of Abdominal Pelvic Oncology Surgery (MUAPOS) of the University General Hospital of Castellon, Spain from January 2013 to December 2016. We excluded patients meeting the radiologic-laparoscopic criteria for unresectability (RLCU)5 defined in preoperative studies (Table 1).

Table 1

Radiologic-laparoscopic criteria for unresectability (RLCU)

CT-scan	Lung metastasisHepatic metastasis in 3 o more hepatic segmentsSevere hepatic pedicle involvementProgression after NACT
Diagnostic laparoscopy	Diffuse serous small bowel disease

Abbreviations: CT, computed tomography; NACT, neoadjuvant chemotherapy.

Patients older than 80 years and ECOG greater than 1 were also excluded. The same surgical team performed preoperative laparoscopy and debulking surgery. In all patients a A CT-scan was performed. The same radiologist evaluated the radiological PCI. Kappa concordance index between radiological and laparoscopic PCI was 51%. Data describing patients’ clinical and pathological characteristics, surgical procedures, and residual disease at surgery were collected prospectively and analyzed retrospectively.

Written informed consent was obtained from all patients, and the Ethics Committee of the University General Hospital of Castellon approved the study which were guided by international and national ethical requirements concerning biomedical research. In addition to this, the study was conducted in compliance with the Declaration of Helsinki.

Predictive factors

Peritoneal cancer index (PCI) is used to quantitatively assess cancer distribution in the peritoneum based on calculating the sizes of lesions in 13 abdominopelvic regions, as described before.6 The sizes of the lesions are then converted to scores of 0–3: a lesion size score (LSS) of 0 defines no visible tumor burden in the peritoneum, while an LLS of 1, 2, or 3 describes lesions with a maximum diameter of 0.5, 5.0, and >5 cm or lesion confluence, respectively. PCI is calculated by adding the LSS for all regions, giving a maximum PCI of 39 (13×3).7

PCI was determined in all patients in this study by preoperative thoraco-abdominal computed tomography (CT) (80 patients) and/or laparoscopy (49 patients). To quantify the radiological PCI, we chose the largest tumor implant in the assessed region and assigned a score of 0–3. The sum of the scores for each region was then used to calculate the radiological PCI. PCI was calculated before and during surgery, and was categorized into three ordinal levels: 1–10, 11–20, and >20. All specimens were collected and labeled relative to PCI areas. Intestinal obstruction was defined clinically when the following signs were present: abdominal distention, with nausea or vomiting and absence of evacuation; and radiologically, with distention of the small or large bowel.

CCS was defined as no residual macroscopic tumor, OCS as a residual tumor <1 cm in diameter, and suboptimal cytoreductive surgery (SCS) as a residual tumor >1 cm in diameter. Postoperative complications were described according to the Clavien-Dindo classification,8 and grade IIIb–IV complications were considered major complications. Patient follow-up began with the diagnosis. First-line adjuvant chemotherapy involved all patients receiving 6–8 cycles of intravenous carboplatin and placitaxel. After primary adjuvant chemotherapy, we evaluated patients every 3–6 months. Relapse and response to first-line chemotherapy were defined according to the guidelines for response evaluation criteria in solid tumors.9

Statistical analysis and model development design

In this study, we use a method for developing a new model for predicting SCS or CCS and OCS in AOC based on Spiegelhalter and Knill-Jones’ method.10 The core of Spiegelhalter and Knill-Jones’ method expresses the “weight of evidence” because it combines the available patient information with previous experience, while time prediction scores are presented in a form (standardized weights) that is less mathematical and more clinically relevant than conventional logistic regression analysis.11 Two-by-two contingency tables were constructed with the possible predictive factors derived from staging tests and the result variable (type of cytoreduction) in binary form: in rows, PCI>20 vs PCI≤20 from radiological-laparoscopic preoperative reports and surgical findings, or presence vs absence of partial bowel obstruction, ascites, and pleural effusion; in columns, suboptimal vs complete and optimal debulking. Factors with p<0.10 based on the Chi-square or Fisher’s exact test were selected to construct the model. Subsequently, from the same contingency tables, we calculated positive- and negative-probabilistic weights10,11 and their difference, setting the value (points) to 0 in the absence of a factor or with PCI ≤20 (Table 2).

Table 2

(A) Obtaining scores according to the most significant risk factors

Predictive Factors	Positive Weight (a): Factor present	Negative Weight (b): Factor absent	p-value	Points***
CT PCI*	1	0	0.09	1
Laparoscopic PCI*	1	0	0.06	1
Operative PCI*	1	−1	0.0007	2
CT or clinical partial bowel obstruction**	2	0	0.03	2

Notes: *Cut-off: PCI>20 vs PCI≤20. ** Presence vs Absence. ***Points: Difference between positive-probabilistic weight (a) and negative-probabilistic weight (b). This difference is the difference in natural logarithms for the positive- and negative-likelihood ratios for suboptimal cytoreduction: [log (sensitivity/1−specificity)] − [log (1−sensitivity/specificity)], setting the value to 0 in the absence of the risk factor.

Based on the points for each factor, two suboptimal cytoreductive risk models were constructed, depending on inclusion or exclusion of the operative PCI. Low risk, intermediate risk, or high risk was assigned for each patient depending on the total points (Table 3).

Table 3:

Suboptimal cytoreductive risk models

Risk Score 4 Factors (Model R4)*	Points
1 (Low)	0–2
2 (Intermediate)	3–4
3 (High)	5–6

Note: *(With operative peritoneal cancer index)

Model performance

Validation (accuracy) of the model is analyzed through its power of calibration and discrimination. Calibration refers to the degree of coincidence between the expected suboptimal cytoreduction by the model and the really observed. Discrimination refers to the probability that the model correctly distinguishes two opposite-sign results, and is represented by the AUC.

The calibration and discrimination power of the model was validated using the same data from the series (narrow validation). Calibration was analyzed graphically, comparing the observed SCS percentages and the levels of risk predicted by the model. Discrimination was analyzed using receiver operating characteristic curves from the points assigned by the model and the observed presence of SCS. Univariate logistic regression was used to calculate how much the risk of suboptimal debulking increases when adding a supplementary point in the total score, where the odds ratio represents the multiplicative constant.

Statistical analysis was performed using MedCalc 15 for Windows (MedCalc Software, Ostend, Belgium).

Results

A total of 110 patients with suspected AOC were treated at the MUAPOS at the University General Hospital of Castellon, Spain from January 2013 to December 2016. Among these, 80 patients where eligible for primary debulking surgery and were included in this study. None of the patients included in this study met our RLCU criteria. We excluded 30 (27%) patients who met our RLCU criteria; these patients received neoadjuvant chemotherapy followed by interval debulking surgery or second-line chemotherapy.

Patients’ clinicopathological and surgical characteristics of are summarized in Table 4. Most patients presented with serous (55%) and Federation of Gynecology and Obstetrics stage IIIC (71%) epithelial ovarian cancer.

Table 4

Patients’ clinicopathologic and surgical characteristics

	CCS and OCS (n=69)	SCS (n=11)	TOTAL (n=80)
Age (years±SD)	60±11	58±10	60±11
FIGO stage, n (%)
IIIC	53 (27%)	4 (36%)	57 (71%)
IV	16 (23%)	7 (64%)	23 (29%)
CT-PCI	10±6	15±7	11±7
Categorized CT-PCI, n (%)
1–10	44 (64%)	5 (46%)	49 (61%)
10–20	21 (30%)	3 (27%)	24 (30%)
> 20	4 (6%)	3 (27%)	7 (9%)
CT Ascites, n (%)	18 (26%)	4 (36%)	22 (28%)
Clinical-CT Partial Bowel Obstruction, n (%)	3 (4%)	3 (27%)	6 (8%)
CT Pleural Effusion, n (%)	10 (14%)	2 (18%)	12 (15%)
Laparoscopic PCI, n (%)
1–10	20 (49%)	0	20 (41%)
10–20	18 (44%)	5 (62%)	23 (47%)
>20	3 (7%)	3 (38%)	6 (12%)
Operative PCI, n±SD	12±8	23±10	14±9
Categorized Operative PCI, n (%)
1–10	32 (46%)	2 (18%)	34 (43%)
10–20	24 (35%)	1 (9%)	25 (31%)
>20	13 (19%)	8 (73%)	21 (26%)
Visceral Resections per patient, n±SD	3±3	4±4	3±3
Analyzed Lymph Nodes, n±SD	26±14	29±18	26±15
Lymph Node Ratio, n±SD	0.25±0.29	0.33±0.38	0.26±0.30
All Postoperative Complications, n (%)	38 (55%)	7 (64%)	45 (56%)
Postoperative 90-day Mortality, n (%)	2 (3%)	1 (9%)	3 (3.7%)

Abbreviations: CCS, complete cytoreductive surgery; OCS, optimal cytoreductive surgery; SCS, suboptimal cytoreductive surgery; SD, standard deviation; FIGO, Federation of Gynecology and Obstetrics; CT, computed tomography; PCI, peritoneal cancer index.

The aim of surgery was to achieve maximum tumor debulking. At laparotomy, CCS and OCS were achieved in 64 (80%) and 5 (6%) patients, respectively, while SCS was achieved in the remaining 11 (14%) patients. The surgical procedures performed included abdominal and pelvic peritonectomy in 54 (67%) patients, rectosigmoidectomy in 35 (43%), and large bowel resection in 40 (50%). Upper abdominal surgery (UAS) was required in 57 (71%) patients, including diaphragmatic peritonectomy in 40 (50%), distal pancreatectomy in 8 (10%), splenectomy in 23 (29%), and liver resection in 9 (11%).

Pelvic and aortic lymphadenectomy were performed in 59 patients (74%), with positive lymph nodes in the pelvic, paraaortic, or both systems in 36 (61%) patients. Clavien-Dindo Grade IIIb–IV complications were found in 25 (31%) patients, with a higher incidence in patients with a PCI>10 (p<0.001). The main postoperative complications were of intestinal origin (12%), including intestinal and pancreatic leakage and colorectal anastomotic fistula. The 90-day postoperative mortality was 3.7% (3 patients).

Calibration of the two models was made using the same series data (narrow validation), which revealed different rates of SCS at different risk levels (Figures 1 and 2). Both models showed good SCS prediction at level 3 risk, 80% and 100% with model R4 and R3, respectively. Discrimination of the two models showed good performance (R4 area under the curve (AUC) =83% and R3 AUC=69%) when detecting SCS (Figures 3 and 4).

Figure 1

Calibration R4 model.

Figure 2

Calibration R3 model.

Figure 3

Discrimination R4 model.

Figure 4

Discrimination R3 model.

The predictive values derived from each model are shown in Table 5. The risk of SCS is determined by the positive predictive value. Given that the predictive values for the same sensitivity and specificity depend on the prevalence of SCS that can be achieved, Table 5 includes different levels of prevalence with the corresponding predictive values derived from the sensitivity and specificity of the model. The multiplicative probability constant for the risk of SCS calculated by logistic regression is shown in Table 6.

Table 5

Predictive values derived from each model

A) R4 Model (Sensitivity = 82%; Specificity = 75%)
Actual prevalence of SCS	Positive predictive value for SCS	Positive predictive value for CCS+OCS(Negative predictive value for SCS)
10%	27%	97%
20%	45%	94%
30%	58%	91%
40%	69%	86%
50%	77%	81%

Abbreviations: SCS, suboptimal cytoreductive surgery; CCS, complete cytoreductive surgery; OCS, optimal cytoreductive surgery.

Table 6

Logistic regression for R4 and R3 model scores and SCS

	Odds ratio	95% confidence interval	p-value
Model R3	8	2–32	0.003
Model R4	10	3–33	0.0003

The multiplicative constant for the probability of SCS produced by an added point in the total score is shown in Table 6. For R3 and R4 models, each increment of 1 unit in the total score multiplies the predicted risk of SCS by 8 and 10, respectively.

Discussion

Surgery remains the cornerstone of treatment for AOC, but it is not applicable to all patients. Neoadjuvant chemotherapy remains the best option for patients who are not surgical candidates12 .The residual tumor after surgery remains the most important prognostic factor in AOC, and the decision between primary debulking surgery and interval debulking surgery after neoadjuvant chemotherapy remains the key point in surgical treatment in AOC.13,14 In the MUAPOS guide to managing AOC, the preoperative RLCU criteria define which patients receive neoadjuvant chemotherapy.5 Therefore, in patients who are candidates for primary surgery, preoperatively determining which surgery will be optimal or suboptimal could be invaluable.

To select patients suitable for complete surgery and to compare patient outcomes, surgeons need scoring systems with an accurate description of the disease. The FIGO staging system is inadequate for detailed assessment of the extent of the peritoneal spread of carcinomatosis. Therefore, detailed clinical scoring systems are needed pre- and intraoperatively, and they are also necessary to standardize the procedure and compare the results among different surgeons.

Considering a patient’s signs, symptoms, and test results, comparing them with past experience, and arriving at a reasoned decision defines clinical medicine. Scoring systems are usually used in medicine to assist physicians with complex diagnostic or therapeutic decisions. Individually, CT-PCI>20 offers an excellent false positive rate for SCS (Specificity 94%) but its Sensitivity for SCS is low (27%), and its discrimination power is intermediate (AUC 69%). Logically, when adding other predictive factors of significant weight in the model, the final result improves. In an isolated way, CT is more useful to identify non-resectability than to predict SCS.6

Our dynamic model predicts either SCS or CCS and OCS depending on the prevalence of the SCS of the surgical team, and it changes with time depending on incorporating new therapeutic approaches and technical innovations. Combining sensitivity and specificity of the model with the actual prevalence of cytoreduction in a certain scenario (Bayes’ theorem), we obtained the predictive values. Positive predictive value for SCS is a simple probability but it is the one that must aid to determine the decision to be taken according to its magnitude. When handling binary variables, the sensitivity for SCS corresponds to the specificity for CCS+OCS, and vice versa. Therefore, the negative predictive value of SCS corresponds to the positive predictive value for CCS+OCS. In summary, positive predictive values for SCS and CCS+OCS represent the predicted final diagnosis by a model in different scenarios of achieved cytoreduction. These values based on our data are shown in Table 5. This means that in scenarios with low SCS prevalence, the model is more reliable for predicting CCS and OCS than SCS, whereas with higher SCS rates, the model will better predict SCS. Therefore, with low SCS prevalence, a low score ensures CCS and OCS, while a high score does not ensure SCS. Conversely, with high SCS prevalence, a high score predicts a high risk of SCS while a low score leaves uncertainty regarding the possibility of achieving CCS and OCS. The prevalence of better or worse cytoreduction depends on how advanced the disease is and physicians’ surgical skill. Therefore, the degree of cytoreduction may differ from one working group to another, and we cannot expect that the same model will work equally in all environments. This is also why models must be validated in each environment to prove their usefulness beforehand. Only if a model works well in a specific environment, will it reach its maximum degree of evidence and have universal impact.3 The multiplicative constant for the probability of SCS produced by an added point in the total score is shown in Table 6 and proves that the group of selected predictive factors is essential for predicting the final analyzed outcome.

Evaluating PCI in preoperative CT scans has been investigated in several studies showing that PCI was useful when planning surgery.15 However, these results are more variable when predicting outcome. Each of the published series reported different positive predictive values for SCS depending on the rate of OCS of the surgical team.16–18 In our data, CTPCI was useful when determining PCI>20 and/or intestinal obstruction in preoperative studies, as both were important prognostic factors for a CCS and OCS rate of 86%.

Several scores have been described to determine the extent of disease and to predict resectability including Alleti, PCI, Eisenkop, and Fagotti models.7,19–21 The most widely-used scores are the Fagotti model and PCI scores. The Fagotti model describes a laparoscopic model based on a score of 0–12 for progressive disease. The authors reported an overall OCS rate of 67% with an accuracy of 69% when the score was >10 with a 34% unnecessarily-explored rate. External validation of this score was performed by Brun et al who reported an OCS rate of 69% with a decreased accuracy of 60% in model results. Recently, Petrillo et al updated the Fagotti score with an increase in their OCS rate to 80% by introducing UAS. Once again, the key point is the relationship between surgeon-related factors and the surgical outcome.22,23

Our data demonstrated that laparoscopic PCI>20 is a major risk factor for unresectability. Nevertheless, as do other authors, we believe that laparoscopy alone lacks in describing several anatomical regions of the abdomen. Lesions in the retrohepatic area, suprahepatic veins, retroperitoneal space and/or infiltration of the liver pedicle may be underestimated while the presence of these findings is a major determining factor for suboptimal cytoreduction. In the specific areas, CT could play a crucial role to avoid unnecessary laparotomy and too for neoadjuvant chemotherapy.6,24

Chéreau et al25 demonstrated a strong correlation between Fagotti score and PCI at laparotomy when describing tumor distribution in the abdominal cavity and compared the different models when predicting resectability in AOC. The authors reported an AUC rate of 56% and 51% for PCI and Fagotti score, respectively. Combining CT and laparoscopy, our R4 and R3 models produced AUC rates of 83% and 69%, respectively, when discriminating SCS from CCS and OCS.

The strengths of our study lie in the investigation, design strategy, and performance of our two dynamic clinical-radiological-laparoscopic models to predict suboptimal (or complete and optimal) debulking surgery in a homogeneous cohort of patients with primary epithelial ovarian cancer treated in a university hospital with a surgical team experienced in AOC. We propose an easily-constructed model based on the diagnostic tests usually used in these patients. The dynamic nature of the models means that they can be applied in different institutions with different OCS or SCS rates, and they can be changed with changes in the SCS prevalence of the surgical team. The proper integration of the R3 and R4 models with recently developed molecular26 and clinical-radiological models,27 including age, performance status, and comorbidities may help improve the decision-making process in the future. Suidan et al27 use several radiological criteria, in addition to other clinical criteria, to predict SCS in a more complicated model. This differs greatly from the calculation of a radiological PCI, although the substrate can be considered similar. However, we believe that some of these criteria are more related with unresectability than with suboptimal surgery.

The limitations of our study are the retrospective component, small number of patients, and the absence of a broad validation of the models. However, because of the important impact of SCS in prognosis in AOC treatment, our models could be of clinical interest to reduce the number of unnecessary laparotomic explorations. Broad validation of the models in a large number of patients remains necessary, which may be addressed in the near future.

Conclusion

Our findings emphasize that the proper application of our R3-R4 models in primary debulking surgery requires maximal surgical effort including UAS techniques and well-prepared multidisciplinary surgical teams. These two models can predict CCS+OCS and SCS, depending on the SCS prevalence of the surgical team.

Table 2 (B) Final Scores by presence or absence of risk factors
Predictive Factors	Points
CT PCI≤20CT PCI>20	01
Laparoscopic PCI≤20Laparoscopic PCI>20	01
Operative PCI≤20Operative PCI>20	02
CT or clinical absence of partial bowel obstructionCT or clinical presence of partial bowel obstruction	02

Abbreviations: CT, computed tomography; PCI, peritoneal cancer index.

Risk Score 3 factors (Model R3)*	Points
1 (Low)	0–1
2 (Intermediate)	2–3
3 (High)	4

Note: *(Without operative peritoneal cancer index)

B) R3 Model (Sensitivity = 45%; Specificity = 91%)
Actual prevalence of SCS	Positive predictive value for SCS	Positive predictive value for CCS+OCS(Negative predictive value for SCS)
10%	36%	94%
20%	56%	87%
30%	68%	79%
40%	77%	71%
50%	83%	62%

Abbreviations: SCS, suboptimal cytoreductive surgery; CCS, complete cytoreductive surgery; OCS, optimal cytoreductive surgery.