Development and multicentre validation of a prognostic model to predict resectability of pancreatic head malignancy.

BACKGROUND: At the time of planned pancreatoduodenectomy patients frequently undergo exploratory laparotomy without resection, leading to delayed systemic therapy. This study aimed to develop and validate a prognostic model for the preoperative prediction of resectability of pancreatic head tumours.
METHODS: This was a retrospective study of patients undergoing attempted resection for confirmed malignant tumours of the pancreatic head in a university hospital in Hannover, Germany. The prognostic value of patient and tumour characteristics was investigated in a multivariable logistic regression model. External validation was performed using data from two other centres.
RESULTS: Some 109 patients were included in the development cohort, with 51 and 175 patients in the two validation cohorts. Eighty patients (73·4 per cent) in the development cohort underwent resection, and 37 (73 per cent) and 141 (80·6 per cent) in the validation cohorts. The main reasons for performing no resection in the development cohort were: local invasion of vasculature or arterial abutment (15 patients, 52 per cent), and liver (12, 41 per cent), peritoneal (8, 28 per cent) and aortocaval lymph node (6, 21 per cent) metastases. The final model contained the following variables: time to surgery (odds ratio (OR) 0·99, 95 per cent c.i. 0·98 to 0·99), carbohydrate antigen 19-9 concentration (OR 0·99, 0·99 to 0·99), jaundice (OR 4·45, 1·21 to 16·36) and back pain (OR 0·02, 0·00 to 0·22), with an area under the receiver operating characteristic (ROC) curve (AUROC) of 0·918 in the development cohort. AUROC values were 0·813 and 0·761 in the validation cohorts. The positive predictive value of the final model for prediction of resectability was 98·0 per cent in the development cohort, and 91·7 and 94·7 per cent in the two external validation cohorts. [Corrections added on 18 July 2018, after first online publication: The figures for OR of the variables time to surgery and CA19-9 in the abstract and in Table 3 and Table 4 were amended from 1·00 to 0·99].
CONCLUSION: For preoperative prediction of the likelihood of resectability of pancreatic head tumours, this validated model is a valuable addition to CT findings.

Introduction

The management of pancreatic head cancers, including pancreatic adenocarcinoma, distal cholangiocarcinoma and ampullary cancer, remains a medical challenge. Pancreatoduodenectomy is the mainstay of treatment with curative intent, but tumours are frequently diagnosed at an advanced or metastatic stage1 2. Timely patient referral to specialized centres and surgery has a significant impact on the achievable outcome for affected patients3.

At presentation, pancreatic head tumours are usually diagnosed and staged with CT4. The negative predictive value of CT in assessing the likelihood of tumour resectability is high, with rates varying from 89 to 100 per cent5. The positive predictive value, however, is low, with reported rates of 45–79 per cent5. Diagnostic laparoscopy may detect occult metastases in 15–51 per cent of patients6, although this rate is decreasing with increasing precision of CT7. Many patients undergo exploratory surgery without removal of the tumour. They are exposed to surgical risk and delayed appropriate systemic therapy8 9.

To date, validated prognostic models enabling reliable preoperative prediction of resectability are lacking. An externally validated model could serve as a readily applicable additional tool to be used in the preoperative staging process. This could help identify patients who may benefit from further assessment of resectability, for instance by PET–CT, endoscopic ultrasonography or laparoscopy, rather than immediate exposure to surgical risks at laparotomy, including complications and potentially unnecessary surgery. The aim of the present study was to develop and validate such a multivariable prognostic model for preoperative assessment of the probability of resectability using preoperative clinical variables.

Methods

This was a retrospective study designed at Hannover Medical School, a university hospital in Hannover, Germany. Details of patients undergoing attempted resection of a pancreatic head tumour (pancreatic adenocarcinoma, cholangiocarcinoma or ampullary adenocarcinoma) with curative intent between 1 January 2000 and 31 December 2012 were retrieved from a prospectively developed institutional database. All tumours were considered potentially resectable on CT and the tumour type was confirmed histologically to distinguish between the different tumour entities. Patients with available preoperative carbohydrate antigen (CA) 19‐9 values were eligible. Patients undergoing emergency surgery or resection of recurrent tumours were excluded. Patients did not receive neoadjuvant chemotherapy. Staging with laparoscopy was not performed.

Data from the University Medical Centre Schleswig‐Holstein in Kiel, Germany, and Queen Elizabeth Hospital, a university hospital in Birmingham, UK, were used for external model validation. Similar inclusion and exclusion criteria were applied to these cohorts. Patients in Kiel were eligible if they had been operated on between 1 January 2009 and 31 December 2015; patients in Birmingham were eligible if they had been operated on between 1 June 2015 and 31 December 2016.

This study was carried out according to the requirements of the TRIPOD statement10. The institutional review board of Hannover Medical School was consulted; neither informed consent nor approval of an ethics committee was considered necessary (reference number 2979‐2015).

Study endpoints and hypothesis

The primary study endpoint was defined as resectability determined at operation. Criteria for the determination of unresectability included: local invasion of vasculature or arterial abutment, liver metastasis, peritoneal seeding and aortocaval lymph node metastasis. The null hypothesis was that it was not possible to develop and externally validate a prognostic model for preoperative prediction of resectability with clinical variables and an area under the receiver operating characteristic (ROC) curve (AUROC) greater than 0·700.

Definition of relevant predictors

The team in Hannover defined potential predictors for resectability before model development. The variables comprised in the final model were defined, as outlined below.

Time to surgery (days) was defined as the time between first contact to a local hospital or a physician with symptoms of the present disease that led to the diagnosis of pancreatic head malignancy and finally performed surgery.

CA19‐9 (kunits/l) was defined as the most recent CA19‐9 value obtained, no more than 28 days before surgery.

Jaundice was defined as the presence of jaundice reported at any time before surgery.

Back pain was defined as the presence of new‐onset back pain reported at any time before surgery.

Resectability was defined as completion of pancreatic resection with curative intent.

These definitions were used throughout in both the development and validation cohorts. Data on predictors were collected retrospectively from patient charts by investigators who were familiar with the predictor definitions.

Prognostic model design

The training cohort from Hannover was used to agree prognostic factors for resectability that are commonly known before surgery. These factors were used as candidate variables for prognostic model design after univariable statistical evaluation using logistic regression analysis to identify their influence on resectability. All variables with an α value of 0·200 or less were chosen as candidates for multivariable regression modelling and investigated for variable correlations in principal component analysis. Correlations greater than 0·500 between two variables were used to choose only one of those variables for inclusion into multivariable regression analyses, in order to avoid multicollinearity.

Variable interactions were suspected per definition when changes greater than 25 per cent in the magnitude of variable coefficients were detected while comparing the regression models in a stepwise, backwards elimination process of the least significant variables until only significant variables (P < 0·050) remained in the preliminary multivariable model. After exclusion of relevant variable interactions, all previously unconsidered variables were added one at a time to the preliminary model and checked for significance (P < 0·050). These additional variables were included in the final model only if they demonstrated a significant contribution to the outcome at an α level of 5 per cent. This approach to modelling was based on the purposeful selection of co‐variables in multivariable logistic regression, as described by Hosmer and colleagues11. The logit of the final prognostic model was transformed to express the risk of resectability in percentage terms. The Hosmer–Lemeshow test was used to determine the model calibration of regression models. Non‐significant test results indicate good model fit.

Model accuracy, adequacy and correctness were assessed with ROC curve analysis and determination of AUROC12. The cut‐off value for prediction of resectability was set to a predicted probability of at least 90 per cent, with the goal of achieving a positive predictive value of 90 per cent or more with high specificity.

Statistical analysis

Sample size calculations for the development cohort revealed that at least 80 patients needed to be included for an expected AUROC of 0·700 and a null hypothesis value of 0·500 to reach a power of 80 per cent at an α level of 5 per cent. For the minimum sample size of validation cohorts, the AUROC of the development cohort was used and a null hypothesis value of 0·700. With a power of 80 per cent at an α level of 5 per cent, a minimum sample size of 38 was calculated.

For internal model validation, bootstrapping was applied with 100 bootstraps randomized retrospectively. External model validation deployed the logit function of the final prognostic model developed with the Hannover cohort to calculate putative risks of resectability in the Kiel and Birmingham cohorts. The putative risks of resectability were examined with ROC curve analysis and the respective AUROC values were determined to test the study hypothesis. The final prognostic model was also tested after exclusion of patients with ampullary carcinoma and cholangiocarcinoma to assess the robustness of its predictive capability for different tumour entities.

All tests described above, including Wald tests, Hosmer–Lemeshow tests, effect likelihood ratio tests and additional Kaplan–Meier survival analyses with log rank tests, were applied as appropriate. For all statistical tests, P < 0·050 was considered significant, except where otherwise indicated. JMP® Pro statistical software versions 11.2.0 and 13.0.0 (SAS Institute, Cary, North Carolina, USA) and MedCalc® version 16.8 (MedCalc Software, Ostend, Belgium) were used to perform statistical analyses.

Results

The flow of patients through the study, as well as the study design and decision criteria employed for model development and validation are outlined in Fig. 1.

Figure 1

Flow of patients through the study and key decision criteria. AUROC, area under the receiver operating characteristic curve

Model development data set from Hannover

Some 109 patients were included, of whom 29 (26·6 per cent) were considered to have unresectable disease based on intraoperative findings. The main findings were: local invasion of vasculature or arterial abutment (15 patients, 52 per cent), liver metastasis (12, 41 per cent), peritoneal seeding (8, 28 per cent) and aortocaval lymph node metastasis (6, 21 per cent). These patients underwent exploratory laparotomy with or without palliative bypass procedures. Eighty patients (73·4 per cent) were treated with pancreatic head resection including pancreatoduodenectomy and total pancreatectomy. Tumour characteristics of the development cohort are summarized in Table 1.

Table 1

Tumour characteristics in the model development cohort from Hannover

Postoperative variables	Total (n = 109)
Death at end of observation time	100 (91·7)
Pancreatic adenocarcinoma	78 (71·6)
Ampullary adenocarcinoma	11 (10·1)
Cholangiocarcinoma	18 (16·5)
Cholangiocarcinoma or pancreatic adenocarcinoma*	2 (1·8)
Resectable	n = 80
pT status†
pT1	5 (6)
Pancreatic adenocarcinoma	2
Ampullary adenocarcinoma	2
Cholangiocarcinoma	1
pT2	12 (15)
Pancreatic adenocarcinoma	4
Ampullary adenocarcinoma	4
Cholangiocarcinoma	4
pT3	58 (73)
Pancreatic adenocarcinoma	46
Ampullary adenocarcinoma	3
Cholangiocarcinoma	9
pT4	5 (6)
Pancreatic adenocarcinoma	0
Ampullary adenocarcinoma	2
Cholangiocarcinoma	3
pN status†
pN0	32 (40)
Pancreatic adenocarcinoma	20
Ampullary adenocarcinoma	6
Cholangiocarcinoma	6
pN1	48 (60)
Pancreatic adenocarcinoma	32
Ampullary adenocarcinoma	5
Cholangiocarcinoma	11
Grade†
1	2 (3)
1–2	2 (3)
2	55 (69)
2–3	4 (5)
3	17 (21)

Values in parentheses are percentages.

Both were unresectable tumours where the pathological examination of the biopsy could not be assigned clearly to any histological group.

No pTNM status or grade was available for unresectable tumours owing to lack of a completely resected tumour.

Descriptive statistics for the preoperative variables investigated for their influence on resectability in the development cohort are shown in Table 2.

Table 2

Descriptive statistics for model development and validation cohorts

Preoperative variables	Resectable	Unresectable	Missing
Development cohort from Hannover	n = 80	n = 29
Age at operation (years)*	68 (46–88)	65 (41–80)	0 (0)
Time to surgery (days)*	30 (7–285)	42 (5–735)	9 (8·3)
CA19‐9 (kunits/l)*	100 (1–5592)	320 (1–11 487)	0 (0)
Weight loss (kg)*	0 (0–25)	0 (0–24)	5 (4·6)
Height (cm)*	173 (144–194)	170 (155–190)	0 (0)
Weight (kg)*	75 (50–125)	67 (47–113)	0 (0)
BMI (kg/m2)*	25·3 (18·1–39·5)	23·8 (17·9–34·9)	0 (0)
Length of survival (months)*	23 (0–115)	8 (0–36)	0 (0)
Sex ratio (M : F)	51 : 29	13 : 16	0 (0)
Jaundice	64 (80)	13 of 28 (46)	1 (0·9)
Back pain	2 (3)	8 of 28 (29)	1 (0·9)
Upper abdominal pain	36 (45)	20 of 28 (71)	1 (0·9)
Weight loss	40 (50)	13 of 28 (46)	1 (0·9)
Preoperative biliary stent placement	52 (65)	13 (45)	0 (0)
Insulin dependency	11 (14)	5 (17)	0 (0)
National validation cohort from Kiel	n = 37	n = 14
Time to surgery (days)*	24 (10–196)	60 (9–365)	0 (0)
CA19‐9 (kunits/l)*	125 (1–4484)	563 (74–10 000)	0 (0)
Jaundice	27 (73)	4 (29)	0 (0)
Back pain	3 (8)	4 (29)	0 (0)
International validation cohort from Birmingham	n = 141	n = 34
Time to surgery (days)*	57 (8–625)	95 (36–539)	0 (0)
CA19‐9 (kunits/l)*	103 (2–55 074)	478 (6–16 128)	0 (0)
Jaundice	124 (87·9)	22 (65)	0 (0)
Back pain	5 (3·5)	6 (18)	0 (0)

Values in parentheses are percentages unless indicated otherwise;

values are median (range).

CA, carbohydrate antigen.

External model validation data sets from Kiel and Birmingham

Some 51 patients from Kiel and 175 from Birmingham were included. In these cohorts, 14 (27 per cent) and 34 (19·4 per cent) patients respectively were considered to have unresectable disease at operation. Descriptive statistics for the preoperative variables used for model validation are summarized in Table 2.

Prognostic model development

Results of univariable logistic regression analyses are summarized in Table 3. The pairs of variables, weight and BMI, weight and male sex, and jaundice and preoperative biliary stenting, had a correlation of 0·847, 0·548 and 0·530 respectively. Thus, the variables BMI, male sex and jaundice were selected for prognostic model development to avoid multicollinearity in regression while retaining the greatest possible number of variables. This resulted in a total of eight variables to be included in the stepwise multivariable logistic regression analysis.

Table 3

Univariable logistic regression analysis for the model development cohort

	Odds ratio*	P	Included in multivariable analysis
Continuous variables
Age at surgery (years)	1·03 (0·99, 1·08)	0·169	Yes
Time to operation (days)	0·99 (0·98, 0·99)	0·011	Yes
CA19‐9 (kunits/l)	0·99 (0·99, 0·99)	< 0·001	Yes
Weight loss (kg)	0·99 (0·93, 1·07)	0·845	No (P > 0·200)
Height (cm)	1·03 (0·98, 1·08)	0·248	No (P > 0·200)
Weight (kg)	1·04 (1·00, 1·07)	0·024	No (correlation with BMI and male sex > 0·500)
BMI (kg/m2)	1·14 (1·00, 1·29)	0·034	Yes
Binary variables
Male sex	2·16 (0·91, 5·13)	0·078	Yes
Jaundice	4·62 (1·83, 11·62)	0·001	Yes
Back pain	0·06 (0·01, 0·33)	< 0·001	Yes
Upper abdominal pain	0·33 (0·13, 0·83)	0·015	Yes
Weight loss	1·15 (0·49, 2·73)	0·745	No (P > 0·200)
Preoperative biliary stent placement	2·29 (0·97, 5·51)	0·060	No (correlation with jaundice > 0·500)
Insulin dependency	0·77 (0·24, 2·43)	0·654	No (P > 0·200)

Values in parentheses are 95 per cent confidence intervals.

CA, carbohydrate antigen.

The stepwise multivariable regression analysis in the model development cohort from Hannover revealed the final prognostic model as:

The logit of the final model indicated an AUROC of 0·918 (Fig. 2a).

Figure 2

Receiver operating characteristic (ROC) curve analysis of the final prognostic model for prediction of resectability in a the development cohort from Hannover (area under the ROC curve (AUROC) = 0·918), b the Kiel validation cohort (AUROC = 0·813) and c the Birmingham validation cohort (AUROC = 0·761)

Prediction of the percentage probability of resectability was determined as follows:

The specificity was 96·2 per cent and the positive predictive value 98·0 per cent. The Hosmer–Lemeshow test revealed a non‐significant result (P = 0·984). Results of the final multivariable regression model are shown in Table 4.

Table 4

Multivariable regression analysis of the final prognostic model for prediction of resectability in the Hannover development cohort

	Odds ratio	P
Time to surgery (days)	0·99 (0·98, 0·99)	0·040
Jaundice	4·45 (1·21, 16·36)	0·024
Back pain	0·02 (0·00, 0·22)	< 0·001
CA19‐9 (kunits/l)	0·99 (0·99, 0·99)	< 0·001

Values in parentheses are 95 per cent confidence intervals. CA, carbohydrate antigen.

Patients with resectable tumours had a significantly better median (range) postoperative survival than those with unresectable tumours (23 (0–115) versus 8 (0–36) months; P < 0·001).

Prognostic model validation

Internal model validation revealed AUROC values for the prediction of resectability with the final model between 0·828 and 0·947 (median 0·923, mean 0·917). Multicentre external model validation revealed AUROC values for prediction of resectability with the final model of 0·813 in the Kiel (Fig. 2b) and 0·761 in the Birmingham (Fig. 2c) cohort, indicating that the model developed in the Hannover cohort could successfully be validated internally and externally. Specificity and positive predictive value of the final prognostic model for the prediction of resectability in the Kiel cohort were assessed as 85·7 and 91·7 per cent respectively, and in the Birmingham cohort as 88·2 and 94·7 per cent. In these external validation cohorts, the P value for the Hosmer–Lemeshow test was 0·420 and 0·307 respectively. Analysis of the model's robustness in different tumour entities revealed AUROC values of 0·935 for the reduced Hannover cohort (70 patients), and 0·837 and 0·731 for the reduced validation cohort from Kiel (45) and Birmingham (136).

Deployment of the developed prognostic model in clinical practice

Using the above‐mentioned risk score, it is possible to convert the logit (y) of the developed model for the prediction of resectability into an individual percentage probability of resectability. A web‐based calculator for the determination of preoperative resectability probability is available at https://www.uhb.nhs.uk/prediction-of-resectability-calculator.htm.

For immediate transfer to surgery, 150 patients (46·2 per cent) from Hannover, Kiel and Birmingham were detected, whereas 148 patients (45·5 per cent) had a medium and 27 (8·3 per cent) had only a low chance of resectability (Fig. 3) (owing to missing values, it was not possible to calculate the chance of resectability for ten patients from Hannover). Fig. 3 shows that the prognostic model was less accurate in the prediction of a low likelihood of resectability (less than 10 per cent) compared with a high chance of resectability (90 per cent or more). A clinical decision tree on how to use the prognostic model in clinical practice is shown in Fig. 4.

Figure 3

Predicted versus observed chance of resectability in 325 evaluable patients from Hannover, Kiel and Birmingham. For a 90 per cent or greater prediction of resectability, the positive predictive value (PPV) was 95·3 per cent and the negative predictive value (NPV) 38·3 per cent. The proposed model gains maximum utility when combined with CT, which has a low PPV and a high NPV for prediction of resectablity5

Figure 4

Recommended clinical use of the developed model

Discussion

This study has provided a simple, multicentre, validated prognostic model using routinely measured data to predict the likelihood of resectability in patients with tumours of the head of the pancreas that are deemed ‘resectable’ based on CT findings. CT is known to have a high negative predictive value and a low positive predictive value for the prediction of resectability5. This is not surprising as radiological evaluation without additional clinical data is often misleading. Parameters included in the tested model were time to surgery, CA19‐9 concentration, jaundice and back pain. Based on the results of this study, clinical recommendations could be derived on patients who should be operated on without further delay and those who might benefit from further preoperative assessment with less invasive methods. If the predicted probability of resectability is less than 90 per cent, additional investigations to assess resectability are recommended.

In the past, patients with biliary and duodenal obstruction would frequently undergo bypass surgery if the tumour was considered unresectable. As non‐operative interventions to treat obstruction, such as stenting, have become available, unnecessary explorative laparotomies should be avoided in patients with unresectable tumours. Early systemic therapy as induction of palliative treatment would benefit these patients.

Even in recent cohorts where preoperative resectability was evaluated with multidetector CT, unresectability rates of 22 per cent at laparotomy have been reported13. Newer approaches to the evaluation of resectability include the use of laparoscopic ultrasound imaging combined with endoscopic ultrasonography (EUS)14, but these techniques should not be used routinely in patients with potentially resectable pancreatic or periampullary cancer as determined by CT15. EUS is more sensitive, but slightly less specific, in detecting vascular invasion and nodal staging than CT, as shown in a recent meta‐analysis16. For vascular invasion and nodal staging, the pooled sensitivity of EUS was 86 and 58 per cent respectively, compared with 58 and 24 per cent for CT, whereas the pooled specificity of EUS was 93 and 85 per cent, versus 95 and 88 per cent for CT16. Diagnostic laparoscopy with ultrasound imaging is an alternative to detect occult liver metastasis, vascular involvement and peritoneal metastasis, and, as a consequence, prevent unnecessary open surgery, in about 33 per cent of patients compared with standard imaging alone17. The authors of the present study advocate laparoscopy when the predicted probability of resectability is less than 90 per cent.

Timely patient referral to specialists is critical in the management of pancreatic cancer18. The present study clearly reinforces the importance of early referral to specialists and early operation to reduce the likelihood of unresectable disease18 19. The observation that delays to surgery relate to increased rates of unresectability has been reported by other groups18 20, 21. Jaundiced patients who undergo early surgery without preoperative biliary drainage have a significant reduction in complications22. The majority of patients scheduled for resection of periampullary tumours first undergo biliary drainage, even in centres with an active programme of offering surgery without preoperative biliary drainage19. Facilitating early surgery is achievable, but more must be done to ensure a wider availability given the clear benefits of avoiding biliary drainage and greater likelihood of potentially curative resection.

This study has demonstrated that patients with back pain frequently have unresectable disease. This is in line with similar reports23, whereas the presence of jaundice indicated resectable disease in the present study. This latter finding might be explained by the fact that some symptoms seem to be more threatening from the patient's point of view, leading to earlier consultation of professional medical help and thus earlier diagnosis of the underlying disease. Moreover, early jaundice is a favourable sign as it may reflect a tumour located closer to the ampulla and further from the superior mesenteric vein and superior mesenteric artery.

Limitations of the present study include its retrospective design. In addition, radiological data for patients in the development cohort were somewhat dated, and CT has undergone improvements in recent years. Nevertheless, the model was successfully validated in two current cohorts. The clinical definition of back pain is very loose and may be recorded or interpreted with large variations dependent on the mentality of affected patients and the sensitivity of documenting doctors. A small minority of pancreatic cancers secrete low or very low amounts of CA19‐9 and, as this is a sialylated Lewis blood group antigen, there are also some patients who do not produce this antigen at all, leading to false‐negative results24. In patients who do not generate CA19‐9 at all, the proposed model should be applied with great caution as CA19‐9 is a crucial variable in the model.

The model, these limitations, and all available diagnostic findings, as well as the patient's individual needs and circumstances, should be taken into account when making the final decision to proceed to surgery or not. When there is any doubt about resectability, even after further investigation, exploratory surgery remains mandatory.

Disclosure

The authors declare no conflict of interest.