Risk prediction platform for pancreatic fistula after pancreatoduodenectomy using artificial intelligence.

BACKGROUND: Despite advancements in operative technique and improvements in postoperative managements, postoperative pancreatic fistula (POPF) is a life-threatening complication following pancreatoduodenectomy (PD). There are some reports to predict POPF preoperatively or intraoperatively, but the accuracy of those is questionable. Artificial intelligence (AI) technology is being actively used in the medical field, but few studies have reported applying it to outcomes after PD. AIM: To develop a risk prediction platform for POPF using an AI model.
METHODS: Medical records were reviewed from 1769 patients at Samsung Medical Center who underwent PD from 2007 to 2016. A total of 38 variables were inserted into AI-driven algorithms. The algorithms tested to make the risk prediction platform were random forest (RF) and a neural network (NN) with or without recursive feature elimination (RFE). The median imputation method was used for missing values. The area under the curve (AUC) was calculated to examine the discriminative power of algorithm for POPF prediction.
RESULTS: The number of POPFs was 221 (12.5%) according to the International Study Group of Pancreatic Fistula definition 2016. After median imputation, AUCs using 38 variables were 0.68 ± 0.02 with RF and 0.71 ± 0.02 with NN. The maximal AUC using NN with RFE was 0.74. Sixteen risk factors for POPF were identified by AI algorithm: Pancreatic duct diameter, body mass index, preoperative serum albumin, lipase level, amount of intraoperative fluid infusion, age, platelet count, extrapancreatic location of tumor, combined venous resection, co-existing pancreatitis, neoadjuvant radiotherapy, American Society of Anesthesiologists' score, sex, soft texture of the pancreas, underlying heart disease, and preoperative endoscopic biliary decompression. We developed a web-based POPF prediction platform, and this application is freely available at http://popfrisk.smchbp.org.
CONCLUSION: This study is the first to predict POPF with multiple risk factors using AI. This platform is reliable (AUC 0.74), so it could be used to select patients who need especially intense therapy and to preoperatively establish an effective treatment strategy. ©The Author(s) 2020. Published by Baishideng Publishing Group Inc. All rights reserved.

Core tip: Postoperative pancreatic fistula (POPF) is a life-threatening complication following pancreatoduodenectomy. This is a retrospective study to develop a risk prediction platform for POPF using an Artificial intelligence (AI) model. Compared with established POPF risk prediction methods, this machine learning algorithms better predict the POPF risk correctly (AUC 0.74). This AI-driven platform can identify patients who need especially intense therapy and aid in the establishment of an effective treatment strategy.

INTRODUCTION

Despite advancements in surgical technique and operative management, postoperative pancreatic fistula (POPF) is still widely considered to be the greatest contributor to major morbidity and mortality after pancreatoduodenectomy (PD), with an incidence of 10%-30%[1-5]. Furthermore, it frequently delays the timely delivery of adjuvant therapies, and reduces overall patient survival[6]. The indications of PD have been widening, and the procedure is offered to an increasing number of elderly patients with multiple comorbidities[7,8], prompting the need to accurately define which patients are fit for PD and could tolerate a potentially life-threatening POPF.

Recently, the management of POPF has undergone a paradigm shift from a standardized and uniform approach that could not reflect an individual's characteristics to a proactive mitigation strategy. The new strategy uses various predictive systems to enable early prediction and prevention and optimize individual treatment decisions[6,9,10]. Previous predictive systems[6,9,10] might reflect POPF incidence and had the merit of simplicity, but their predictive accuracy is somewhat questionable[11,12]. Therefore, to more accurately predict POPF, further research is needed.

Machine-learning (ML) is an artificial intelligence (AI) technology that has been adopted in many areas of modern society, including medical science. In ML, computational models composed of multiple processing layers learn various data representations with multiple levels of abstraction[13]. ML is currently being used in not only surgery[5,14,15] but also other areas, such as, pharmacogenomics, image classification, and medical decision support systems[16-20]. Therefore, for this study we aimed to develop a new risk prediction platform for POPF after PD using ML algorithms. If so, we expected that a patient’s predicted POPF risk could direct their clinical management and prevent or mitigate untoward outcomes.

MATERIALS AND METHODS

Patient selection

Under institutional review board approval (No. SMC 2017-01-017), we retrospectively collected clinicopathological variables for 1846 patients who underwent PD to treat various periampullary tumors at Samsung Medical Center (Seoul, Republic of Korea) between January 2007 and December 2016. Among them, we excluded 77 (4.2%) patients who had metastasis from sites other than the primary tumor origin, had direct invasion from the primary tumor into adjacent organs, underwent surgery for a recurrence, or lacked medical information about POPF. We analyzed the remaining 1769 patients (1079 men and 690 women).

Definition and Selection of pre- and intraoperative input variables

Data on preoperative, intraoperative, and postoperative outcomes were collected and maintained on a web-based database (MDB, Seoul, Korea). We originally analyzed 38 preoperative and intraoperative variables that could be associated with POPF. Preoperative laboratory data, such as serum C-reactive protein, amylase, lipase, and carbohydrate antigen (CA) 19-9 level just before the operation was entered to algorithms. Co-existing pancreatitis was defined as classic feature of pancreatitis on preoperative Computed Tomography (CT) scan or intraoperative findings. Underlying heart diseases included hypertension on medication, coronary or valvular heart disease, or various arrhythmic diseases. The location of tumors and pancreatic duct (p-duct) diameter were determined or measured by preoperative CT scan. Total intraoperative fluid infusion consisted of total amount of intravenous crystalloid, colloid, volume expanders, or blood transfusion. The pancreatic texture was determined as soft or hard by the surgeon during the operation.

Among the continuous variables, there was some level of missing data. Median imputation[19], which is a common approach for dealing with missing values in ML algorithms, was used. None of the categorical variables had missing values. We used the one-hot encoding technique to encode the categorical variables when only one of the categories was assigned. When a categorical variable had three exclusive choices, then we transformed the categorical variable into three individual binary variables. When the choices within a categorical variable are not exclusive (e.g., “A”, “B”, “both A and B”, and “none”), binary encoding was used by splitting each choice into separate columns and converting to binary codes[21,22]. As a result, 44 encoded variables were input in the ML models (Table 1).

Table 1

Clinicopathologic variables included in the machine learning algorithms

Variables	Values	Variables	Values
Age (yr)	67.7 ± 10.1	Preoperative ERBD/ENBD (n, %)	470 (26.6)
Sex (male/female)	1079: 690	Preoperative PTBD (n, %)	254 (14.4)
Body mass index (kg/m2)	22.5 ± 3.8	Preoperative ERPD (n, %)	19 (1.1)
Heart disease including hypertension (n, %)	735 (41.5)	Neoadjuvant therapy (RT/chemotherapy/CCRT/No)	2: 6: 16: 1745
Diabetes mellitus (n, %)	472 (26.7)	Operative time (min)	443.2 ± 90.1
Pulmonary disease (n, %)	153 (8.6)	†Intraoperative fluid infusion (mL)	3129 ± 3495
Liver disease (n, %)	99 (5.6)	Intraoperative transfusion (n, %)	171 (9.7)
Cerebrovascular disease (n, %)	73 (4.1)	Estimated blood loss (ml)	962.4 ± 665.1
Chronic kidney disease (n, %)	17 (1.0)	Soft pancreas (n, %)	750 (43.2)
ASA score (1-4)	372:1267: 128: 2	Pancreatic duct diameter (mm)	4.2 ± 2.8
White blood cell count (x10³/μL)	6.6 ± 2.3	Type of surgery (PPPD/PRPD/PD)	1254: 244: 271
Hemoglobin (g/dL)	10.9 ± 1.5	Combined organ resection (n, %)	67 (3.8)
Platelet count (x10³/μL)	280.2 ± 52.7	Combined vascular resection(n, %)	188 (10.6)
Albumin (d/dL)	3.5 ± 0.4	C- reactive protein (mg/dL)	1.9 ± 10.9
Total bilirubin level (mg/dL)	3.5 ± 4.3	CA 19-9 (U/mL)	1786.5 ± 7141.5
Combined portal vein resection (n, %)	175 (9.9)	P-duct stent (Internal/external/none)	1051: 134: 584
Amylase (U/L)	94.5 ± 5586.9	Co-existing pancreatitis (n, %)	370 (20.9)
Lipase (U/L)	160.5 ± 277.0	‡Location of tumor (Pancreas/others)	856: 913
Anastomotic methods (1) (Duct-to-mucosa/Dunkin)	1756: 13	Anastomotic methods (2) (P-J/P-G/Others)	1761: 8: 0

Total fluid infusion consisted of total amount of intravenous crystalloid, colloid, transfusion. 377 ampulla of Vater cancers, 446 bile duct cancers, 90 duodenal cancers. Pancreatic cystic tumors and neuroendocrine tumors belonged to pancreas. RBD: Endoscopic retrograde biliary drainage; ENBD: Endoscopic nasogastric biliary drainage; PTBD: Percutaneous transhepatic biliary drainage; ERPD: Endoscopic retrograde pancreatic drainage; RT: Radiotherapy; CCRT: Concomitant chemo- and radiotherapy; PP- or PRPD: Pylorus-preserving or pylorus-resecting pancreaticoduodenectomy; CA: Carbohydrate antigen; P-J: Pancreaticojejunostomy; P-G: Pancreaticogastrostomy.

Surgical techniques and perioperative management

In cases of cholangitis or jaundice, preoperative endoscopic or percutaneous biliary drainage was performed. After the introduction of a definition of borderline resectable pancreatic cancer, 24 (1.4%) patients received neoadjuvant treatment using various regimens. All surgical procedures were performed by experienced 5 pancreatic surgeons at Samsung Medical Center who underwent more than 50 PDs annually. To create pancreatic anastomosis, 1761 (99.5%) patients underwent pancreati-cojejunostomy (PJ) and 8 (0.5%) patients underwent pancreaticogastrostomy. Pancreatoenteric anastomosis with stents was conducted in 1185 patients (70.0%) (Table 1). At the end of each surgical procedure, two or three drains were placed adjacent to the PJ anastomosis and on the right side of the superior mesenteric arterial resection margin. Serum and drain fluid amylase levels were routinely measured on postoperative days 1-3 and 5, 6, or 7, if the drains were maintained. In this study, we used ‘the definition of POPF in the 2016 update of the International Study Group (ISGPS) definition and grading of POPF’[23]. As a result, the grade A fistula has been removed from the POPF classification in this study. Drains adjacent to the PJ anastomosis were removed if no evidence of a leak was found in an abdominal CT scan on postoperative days 5-7. Patients who experienced POPF received proper management, including conservative, interventional, or surgical treatment, depending on each patient’s clinical condition.

Machine learning algorithms as artificial intelligence

Two ML algorithms, random forest (RF) and neural network (NN), were used to predict POPF. RF method is a kind of ensemble learning algorithm that builds multiple decision trees expecting better performance by taking mode or mean of individual trees[24]. An NN is a ML algorithm that emulates the synaptic structure of the brain[13]. It contains singular or multiple hidden layers, between the input and output layers[13]. Recursive feature elimination (RFE), which is a feature selection method that removes the weakest features until the maximum area under the curve (AUC) is reached[25,26], was used to identify the subset of features used in the final NN model. We tuned hyperparameters of NN (such as number of hidden layers, number of nodes, learning rates, batch size, dropout rate, and so on) to maximize the performance by grid search algorithm on each RFE step[27].

These AI-driven POPF prediction algorithms were developed using MATLAB Release 2017 band Python software with Tensoflow library.

Data analysis and statistical methods

The characteristics of the study population were described for each dataset, including the mean and standard deviation for each variable. For the development of the ML algorithms, the total dataset was split into a training set and a test set. The training set was used to derive the POPF prediction algorithms, and the test set was used to evaluate the derived algorithms. In order to evaluate our ML approaches, we used a stratified 5-fold cross-validation test. This randomly divides all the data into 5 partitions (folds) keeping each one with similar positive and negative data distribution. Then, we train a model with four of the partitions and test the model with the remaining fold. By changing the folds for training and testing, this process is performed 5 times. Also, the whole cross-validation was repeated 10 times by random split of the dataset, evaluating the performance of the models at the end. These processes ensure the generalized performance of a model by preventing overfitting to the samples. Because the outputs from the ML-driven POPF prediction algorithms are probabilistic estimates of risk, the performance for the test data was evaluated using AUC. The clinical meaning of the AI-driven risk factors for POPF was identified by sliding window approach[28]. All statistical and mechanical analyses assessing algorithm performance were done by Cho K and Cho BH from Medical AI Research Center, Samsung Medical Center using GraphPad Prism version 5.00 for Windows (GraphPad Software, SD, CA) and MATLAB Release 2017b (MathWorks, Inc., Natick, MA) software.

RESULTS

Clinicopathological characteristics and outcomes

Table 1 provides the clinicopathologic details of the 1769 patients. Among them, grade B or C POPF occurred in 221 (12.5%) patients according to the ISGPF 2016 definition, and 130 (7.3%) patients had an American Society of Anesthesiologists’ (ASA) score ≥ 3. The mean value of body mass index (BMI) was 22.5 kg/m2, and the mean albumin level was 4.0 g/dl. The mean operating time was 443.2 min. The mean total fluid input and estimated blood loss during the operation were 3129.5 and 962.4 mL, respectively. A soft pancreas was observed in 750 (42.4%) patients. The mean diameter of the pancreatic duct was 4.2 mm. The most common tumor location was the pancreas, which occurred in 568 (32.1%) patients. Presumed pancreatitis was observed in 370 (20.9%) patients (Table 1). 30-d postoperative mortality was observed in 23 (1.3%) patients.

Development of machine learning models using random forest and neural network

Table 2 summarizes the results from each algorithm using three different configurations of the dataset. Firstly, the data with complete values for the 38 original variables were input into the two ML algorithms. The average AUCs over the 5-fold cross validation with 10 repetition were 0.67 with the RF and 0.74 with the NN, respectively. When complete data for 34 original variables (without serum C-reactive protein, amylase, lipase, and CA 19-9 level) were input into the algorithms, the 5-fold average AUCs were 0.67 with the RF and 0.72 with the NN. For the configuration of missing values treatment, we input data from 1769 patients into the ML algorithms. Those 5-fold average AUCs increased to 0.68 with the RF and 0.71 with the NN. All those AUCs are summarized in Table 2.

Table 2

Prediction performance of the various dataset for postoperative pancreatic fistula

Dataset and algorithm	Number of original variables	Number of encoded variables	Number of samples	Area under curve
Random forest with complete cases	38	44	889	0.670.02
Neural network with complete cases				0.740.02
Random forest with complete variables	34	40	1769	0.670.01
Neural network with complete variables				0.720.02
Random forest with missing data treatment	38	44	1769	0.680.02
Neural network with missing data treatment				0.710.02

Machine learning models using neural network with recursive feature elimination

Using all 1769 data samples after missing data treatment, we could further improve the AUC from 0.71 to 0.74 using NN with RFE method (Table 2). Sixteen risk factors for POPF were identified using NN with RFE method: Pancreatic duct diameter, BMI, preoperative serum albumin, lipase level, amount of intraoperative fluid infusion, age, platelet count, extrapancreatic location of tumor, combined venous resection, co-existing pancreatitis, neoadjuvant radiotherapy, ASA score, sex, soft texture of the pancreas, underlying heart disease, and preoperative endoscopic biliary decompression. (Figure 1). The post hoc analysis revealed a nonlinear relationship by showing the response of NN model to each input variable at every RFE step. Ten discrete points cover the observed range of variation for each corresponding variable. We found several patterns of NN output response, in which the predicted POPF risk seemed to have a positive, negative, or biphasic relationship with each variable. The contribution profiles of the top 16 variables are shown in Supplement Figure 1. Based on these multiple and complex relationships among the risk factors for POPF after PD, we made a network connections illustration to improve understanding (Figure 2).

Figure 1

Performance of the neural network models optimized within each recursive feature elimination step. 1: Pancreatic duct diameter; 2: Body mass index; 3: Serum albumin; 4: Amount of intraoperative fluid infusion; 5: Age; 6: Platelet count; 7: Extrapancreatic location of tumor; 8: Combined venous resection; 9: Co-existing pancreatitis; 10: Serum lipase; 11: Neoadjuvant radiotherapy; 12: ASA score; 13: Sex; 14: Soft texture of pancreas; 15: Underlying heart disease; 16: Preoperative endoscopic biliary decompression; 17: Hemoglobin; 18: Serum total bilirubin; 19: Operative time; 20: Intraoperative transfusion; 21: Neoadjuvant chemotherapy; 22: Anastomotic methods (1); 23: Serum amylase; 24: Anastomotic methods (2-1); 25: Pancreatic duct stent (1); 26: White blood cell count; 27: Type of surgery (1); 28: Serum carbohydrate antigen 19-9; 29: Serum C- reactive protein; 30 Estimated blood loss; 31: Combined vascular resection; 32: Pancreatic duct stent (2); 33: Preoperative percutaneous biliary drainage; 34: Underlying cerebrovascular disease; 35: Combined organ resection; 36: Type of surgery (2); 37: Type of surgery (3); 38: Anastomotic methods (2-2); 39: Underlying liver disease; 40: Underlying chronic kidney disease; 41: Underlying pulmonary disease; 42: Underlying cerebrovascular disease; 43: Diabetes mellitus; 44: Preoperative endoscopic pancreatic drainage; ASA: American Society of Anesthesiologists; AUC: Area under the curve.

Figure 2

Illustration of artificial intelligence algorithm for 16 risk factors affecting postoperative pancreatic fistula. PV-SMV: Portal vein-superior mesenteric vein; ASA: American Society of Anesthesiologists; ERBD: Endoscopic retrograde biliary drainage; ENBD: Endoscopic nasobiliary drainage; POPF: Postoperative pancreatic fistula.

Establishment of risk prediction platform for postoperative pancreatic fistula using artificial intelligence

NN algorithm using RFE that had the best performance across the metrics of discrimination, calibration, and overall performance was integrated into an interactive interface. We designed our clinical decision tool to collect values entered by a clinician, feed those values into the pre-trained algorithm, retrieve the result, and output that result to the clinician in real time. This POPF prediction platform is available as an open-access, web-based application programmed to be accessible and adaptable for use on desktops, tablets, and smartphones. It is freely available at https://popfrisk.smchbp.org/.

DISCUSSION

POPF is a serious inherent risk of a pancreatic resection. The best option for managing POPF is undoubtedly prevention using a preoperative and intraoperative POPF risk assessment that guides response measures postoperatively[6,9,10]. Theoretically, ML could offer an opportunity to improve the accuracy of risk assessment by exploiting the complex interactions among risk factors that affect POPF. To the best of our knowledge, this study presents the first ML algorithm for predicting POPF using multiple pre- and intraoperative variables derived from a large, single-institutional dataset. The maximum AUC of this model was considerable: 0.74 with NN with RFE method (Figure 1). Ultimately, a patient’s predicted POPF risk could direct their clinical management and prevent or mitigate untoward outcomes.

AI in the form of ML discovers intricate structures in large datasets by using a backpropagation algorithm to indicate how a machine should change the internal parameters it uses to compute the representation in each layer based on the representation in the previous layer[13]. ML can identify latent variables that are unlikely to be observed but might be inferred from other variables[18,19,24]. For example, NN with RFE algorithm found many pre- and perioperative predictors for POPF that we used in our final modeling (Figure 1). Previously developed risk assessment models[6,9,10] implicitly assume that the risk factors are related to POPF in a linear fashion. Those models could thus oversimplify complex, nonlinear relationships among many risk factors. Even if it might be cumbersome to calculate the risk of POPF using 16 variables in actual clinical care, our AI-driven risk platform better incorporates multiple risk factors and can account for more nuanced relationships between the risk factors and POPF (Figure 2). An example of nonlinearity is shown in Supplement Figure 1, and note that each variable has a variable effect.

This ML algorithms found 16 risk factors for POPF (Figure 1). These risk factors can be categorized into 3 groups: The technically demanding group, intraoperative volume status–related group, and poor general condition group (Figure 2). The risk factors in the technically demanding group (soft pancreas[2,5,6,9], small pancreatic duct[6,9,29], extrapancreatic lesion[6], absence of preoperative pancreatitis or low lipase level[30], absence of preoperative endoscopic biliary decompression, absence of neoadjuvant radiotherapy, and high BMI[7]) indicate potential difficulty in reconstructing the pancreatic-enteric anastomosis, which could cause POPF. Patients with pancreatic cancer, chronic pancreatitis, or neoadjuvant treatment have increased pancreatic fibrosis and a lower incidence of POPF than other PD patients[30,31]. Also, it is well-known that preoperative endoscopic biliary drainage is frequently associated with procedure-related pancreatitis[32]. As a result, those procedures might reduce the risk of POPF. The risk factors in the intraoperative volume status–related group (large intraoperative fluid administration, concomitant portal vein-superior mesenteric vein resection, and low platelet count) could cause ischemia and poor healing of the pancreatic-enteric anastomosis, which is compounded by tissue edema from aggressive volume replacement in a rebound fashion[6,33]. The resultant swelling of the anastomosis can cause duct occlusion or suture disruption. The risk factors in the poor general condition group (old age, underlying heart disease, low preoperative serum albumin level, and low ASA score) could be related to poor nutritional status, which is considered to correlate with a high risk of POPF[34,35].

Notably, a high probability of POPF in patients characterized by only 1 or 2 classical fistula risk factors could not be determined. In this study, we found 16 risk factors by using AI algorithms (Figure 1), but controversy remains about the true risk factors for POPF. The varying results from different studies could be influenced by study design, the composition of the patient populations, or statistical methods. For example, there is still debate about whether intraoperative volume status, such as intraoperative blood loss, transfusion, or the amount of fluid administration, are risk factors for POPF[6,9,33,36]. Some reports suggest intraoperative volume status as an independent risk factor for POPF[6,33] because of pancreatic parenchymal and intestinal edema from aggressive volume replacement, but other studies have denied its adverse effect because estimation of blood loss during surgery is unreliable and inaccurate[3,9]. We think this discrepancy about the prognostic value of different risk factors for POPF could be a fundamental interpretation error caused by the assumption of linearity and an attempt to simplify what isn't actually simple. Therefore, ML algorithms such as those used in the study will be an important tool for POPF risk assessments.

Other recently proposed risk prediction models[6,9,10] have the advantage of being easily performed because they use only 3–6 variables. However, previously unknown risk factors for POPF are still being newly identified. For example, preoperative sarcopenia, an age-related decrease in muscle mass, has been identified as a risk factor for POPF[37-39]. Existing models cannot reflect new factors for POPF as they emerge but must be re-analyzed and developed from scratch. To make matters worse, as the number of potential risk factors increases, the complexity of the conventional models can cause over-fitting, yielding implausible results. However, we addressed that possibility by using active and appropriate choices in pre-training, hyper-parameter selection, and regularization in our AI-driven algorithms[19]. Because AI is scalable, there is no need to develop a new model; even if many new variables affecting POPF are introduced, it is possible to just continue adding them to the original model. As the amount of pancreatectomy data continues to grow, the creation and deployment of learning systems as accessible tools could significantly enhance the prognosis and management of POPF. New learning algorithms and architectures that are currently being developed, such as convolution[40] or recurrent[41] NNs, will accelerate this progress.

This AI-driven risk prediction platform for POPF could assist the drive toward personalized medicine by better tailoring risk management to individual patients. For example, after a risk evaluation, high-risk patients could be selected for a multiple-drain strategy and postoperative prophylactic octreotide use. In this way, we expect our platform to help select patients who need more intense therapy and establish effective (and cost-effective) treatment strategies for POPF. Various mitigation strategies have been proposed to reduce the occurrence and morbidity of POPF, including technical variations, such as, pancreaticogastrostomy reconstruction[2,42], dunking/invaginating anastomosis[1,43,44], absorbable mesh patches[45,46], and the use of intraperitoneal drains[29], anastomotic stents[47], and prophylactic somatostatin analogues[4,48,49]. As a part of those efforts, we have an ongoing trial of this risk score wherein we are applying a somatostatin analogue during postoperative days 0–3 in high-risk patients. Future prospective studies could stratify treatments based on the outcome of this platform and provide comprehensive treatment algorithms.

This study has several limitations. First, Co-existing pancreatitis, which is bound to be subjective, was defined as classic feature of pancreatitis on preoperative CT scan or intraoperative findings. Also, the input data used to develop risk prediction platform was pre- and intraoperative variables. In practice, it is important for both pre- and intraoperative variables to enter the algorithms in order to improve the predictability of POPF, but clinically, it may be helpful for only preoperative variables to enter the algorithms. Therefore, we will sooner or later conduct modeling for only preoperative variables using multicenter data. Second, the NN’s output response to changing each input variable partially revealed the variables’ nonlinear relationships to POPF risk. Therefore, the pattern of the output response should not be understood as a direct relationship between an input variable and POPF risk. Nevertheless, this process could help inform further explorations of diverse predictive risk factors and the future development of new risk prediction approaches and algorithms. Finally, the study, though a large institution, was not only conducted on patients in single center but also had the disadvantage of not performing external validation. As a result, the follow-up study will be conducted by performing an external validation on patients in multiple institutions.

In conclusion, ML algorithms are promising tools for the prediction of POPF that can be integrated into clinically useful decision tools. Compared with established POPF risk prediction methods, our ML algorithms better predict the POPF risk correctly. After external validation, this new platform could be used to select patients who need more intense therapy and to preoperatively establish an effective treatment strategy.

ARTICLE HIGHLIGHTS

Research background

Despite advancements in operative technique and improvements in postoperative managements, postoperative pancreatic fistula (POPF) is a life-threatening complication following pancreatoduodenectomy (PD). Artificial intelligence (AI) technology is being actively used in the medical field, but few studies have reported applying it to outcomes after PD.

Research motivation

There are some reports to predict POPF preoperatively or intraoperatively, but the accuracy of those is questionable. Compared with established POPF risk prediction methods, we expect that our ML algorithms can better predict the POPF risk correctly.

Research objectives

This study aimed to develop a risk prediction platform for POPF with single center dataset using an AI model.

Research methods

A total of 38 variables from 1769 patients who underwent PD from 2007 to 2016 at Samsung Medical Center were inserted into AI-driven algorithms. The algorithms tested to make the risk prediction platform were random forest (RF) and a neural network (NN) with or without recursive feature elimination (RFE). These algorithms can better incorporate multiple risk factors and account for more nuanced relationships between the risk factors and POPF. The median imputation method was used for missing values.

Research results

The number of POPFs was 221 (12.5%) according to the International Study Group of Pancreatic Fistula definition 2016. The maximal AUC using NN with RFE was 0.74. Sixteen risk factors for POPF were identified by AI algorithm: Pancreatic duct diameter, body mass index, preoperative serum albumin, lipase level, amount of intraoperative fluid infusion, age, platelet count, extrapancreatic location of tumor, combined venous resection, co-existing pancreatitis, neoadjuvant radiotherapy, American Society of Anesthesiologists’ score, sex, soft texture of the pancreas, underlying heart disease, and preoperative endoscopic biliary decompression. We developed a web-based POPF prediction platform available at https://popfrisk.smchbp.org.

Research conclusions

This study is the first to predict POPF with multiple risk factors using AI. This platform is reliable (AUC 0.74), so it could be used to select patients who need especially intense therapy and to preoperatively establish an effective treatment strategy.

Research perspectives

This study developed a risk prediction platform for POPF with single center dataset using an AI model. The follow-up study will be conducted by performing an external validation on patients in multiple institutions. Also, future prospective studies could stratify treatments based on the outcome of this platform and provide comprehensive treatment algorithms.

ACKNOWLEDGEMENTS

We thank the study research nurse, Hyemin Kim, for her tremendous work in the acquisition of data, and we thank Sang Eun Lee for dedicated support with medical illustration at Samsung Medical Information & Media Services at Samsung Medical Center.