Development and external validation of models to predict acute respiratory distress syndrome related to severe acute pancreatitis.

BACKGROUND: Acute respiratory distress syndrome (ARDS) is a major cause of death in patients with severe acute pancreatitis (SAP). Although a series of prediction models have been developed for early identification of such patients, the majority are complicated or lack validation. A simpler and more credible model is required for clinical practice. AIM: To develop and validate a predictive model for SAP related ARDS.
METHODS: Patients diagnosed with AP from four hospitals located at different regions of China were retrospectively grouped into derivation and validation cohorts. Statistically significant variables were identified using the least absolute shrinkage and selection operator regression method. Predictive models with nomograms were further built using multiple logistic regression analysis with these picked predictors. The discriminatory power of new models was compared with some common models. The performance of calibration ability and clinical utility of the predictive models were evaluated.
RESULTS: Out of 597 patients with AP, 139 were diagnosed with SAP (80 in derivation cohort and 59 in validation cohort) and 99 with ARDS (62 in derivation cohort and 37 in validation cohort). Four identical variables were identified as independent risk factors for both SAP and ARDS: heart rate [odds ratio (OR) = 1.05; 95%CI: 1.04-1.07; P < 0.001; OR = 1.05, 95%CI: 1.03-1.07, P < 0.001], respiratory rate (OR = 1.08, 95%CI: 1.0-1.17, P = 0.047; OR = 1.10, 95%CI: 1.02-1.19, P = 0.014), serum calcium concentration (OR = 0.26, 95%CI: 0.09-0.73, P = 0.011; OR = 0.17, 95%CI: 0.06-0.48, P = 0.001) and blood urea nitrogen (OR = 1.15, 95%CI: 1.09-1.23, P < 0.001; OR = 1.12, 95%CI: 1.05-1.19, P < 0.001). The area under receiver operating characteristic curve was 0.879 (95%CI: 0.830-0.928) and 0.898 (95%CI: 0.848-0.949) for SAP prediction in derivation and validation cohorts, respectively. This value was 0.892 (95%CI: 0.843-0.941) and 0.833 (95%CI: 0.754-0.912) for ARDS prediction, respectively. The discriminatory power of our models was improved compared with that of other widely used models and the calibration ability and clinical utility of the prediction models performed adequately.
CONCLUSION: The present study constructed and validated a simple and accurate predictive model for SAP-related ARDS in patients with AP. ©The Author(s) 2022. Published by Baishideng Publishing Group Inc. All rights reserved.

Core Tip: Severe acute pancreatitis (SAP)-related acute respiratory distress syndrome (ARDS) affect the mortality of patients with AP. Early identification of patients at high risk for SAP and ARDS can aid clinicians to adopt interventions to stop disease progression. However, current predictive models are either too complicated due to various parameters or unreliable due to lack of validation. This study developed new models to predict SAP and ARDS using only four routine clinical items within 24 h of admission. New models were externally validated and performed as well as or with a higher efficiency than other models.

INTRODUCTION

Acute pancreatitis (AP) is a common event incurring pain, socioeconomic loss, and even death. The majority of the patients who present with mild organ injury and self-limited course are diagnosed with mild acute pancreatitis (MAP) or moderately severe acute pancreatitis (MSAP)[1,2]. However, it is estimated that approximately 20% of patients are critically ill and develop SAP, leading to consistent organ failure and significant mortality[2,3]. Our previous studies indicated that the lung are the most commonly affected organs in SAP[4,5], and acute respiratory distress syndrome (ARDS) is recognized as an important cause of respiratory failure, with a high mortality rate[6-8]. It is reported that 4%-15% of AP patients are complicated with ARDS[9], while this proportion might be as high as one third in SAP[10]. However, to date, the therapeutic options for SAP and ARDS are limited. Therefore, it is necessary to identify patients at risk and adopt interventions to prevent MAP or MSAP from progressing to SAP and ARDS. The protective effect of early intervention for patients with predicted SAP or patients at risk of ARDS has been confirmed by numerous clinical trials and meta-analyses, although the inclusion criteria for patients have varied according to different studies[11-15].

A plethora of models have been published to predict the risk of SAP in AP patients, including acute physiology and chronic health evaluation II (APACHE-II) score, Ranson criteria, computed tomography severity index (CTSI), and bedside index for severity in acute pancreatitis (BISAP)[16,17]. Lung injury prediction score (LIPS) and other models have also been used to evaluate the risk of ARDS in patients with non-AP[18-20]. However, to date the models used to predict ARDS in AP are scarce. Furthermore, the majority of the SAP predictive models are hard to use in practice due to various parameters, complicated calculation and dependence on radiological assessment. The majority of the models also lack internal or external validation, which reduces their reliability in other cohorts[16]. Therefore, a new concise model may be more practical in the emergency department in order to identify SAP and ARDS in the early course of AP. This model should involve limited available clinical data and should not rely on radiological examinations.

The objective of the present study was to develop and validate models to predict SAP and ARDS in patients with AP based on multicenter retrospective cohorts. The comparison of novel models with quick sequential organ failure assessment (qSOFA), systemic inflammatory response syndrome (SIRS) and BISAP is essential to display the power of different models with a low number of variables. These models usually contain three, four and five items, respectively[21,22].

MATERIALS AND METHODS

Study design and participants

This was a multicenter retrospective study. Sample size was calculated with PASS 11.0. The proportion of SAP was set to 20%, and the incidence of ARDS in SAP and non-SAP patients was set to 1/3 and 4%, respectively. Considering that the dropout rate was low in hospitals, we set it to 5%. In the end, with α = 0.01 and β = 0.10, a total of 211 participants were needed. Patients diagnosed with AP between 1 January 2017 and 31 December 2019 were recruited from different regions of China (Peking Union Medical College Hospital and The Sixth Hospital of Beijing at Northern China, The Fourth Affiliated Hospital of Harbin Medical University at Northeastern China, West China Longquan Hospital Sichuan University at Southwestern China). The patients were categorized into the derivation cohort in order to develop a clinical predictive model. The independent external validation cohort consisted of patients diagnosed with AP between 1 January 2020 and 31 May 2021 at the Peking Union Medical College Hospital. AP was diagnosed if at least two of the three following criteria were met: (1) abdominal pain consistent with AP; (2) serum lipase or amylase levels that were more than three times the upper limit of the normal range; and (3) characteristic radiological findings of AP on computed tomography (CT) and magnetic resonance imaging or ultrasonography[23]. SAP was identified by the presence of persistent organ failure for > 48 h[23]. ARDS was diagnosed based on Berlin definition[24]. Patients aged < 18 years who lacked the necessary information provided by the Atlanta Classification or relevant etiology information were excluded.

Clinical variables

The following demographic and laboratory data were collected from the electronic medical record system within 24 h of admission: age, sex, temperature, heart rate (HR), respiratory rate (RR), systolic blood pressure, Glasgow coma score (GCS), white blood counts, hematocrit, platelet, serum electrolyte concentration (K, Na and Ca), creatine, blood urea nitrogen (BUN) and glucose. Other clinical information, such as, admission date, local complication, length of hospital stay, length of intensive care unit stay, mortality and ventilator use, was also collected. Weekend admission corresponded to admission on Saturday or Sunday and local complication included acute peripancreatitc fluid collection, acute necrosis collection, pseudocyst and walled-off necrosis. The ventilator included invasive or noninvasive mechanical ventilation. qSOFA, SIRS and BISAP scores were calculated based on the aforementioned data. All data were collected and checked by two or more authors independently. Missing items were added following review of the clinical records. The data that could not be completed were removed and the complete-case dataset was finally analyzed.

Statistical analysis

Categorical variables were expressed as frequencies and percentages and compared using χ or Fisher’s exact tests. Normally distributed continuous variables were described as the mean ± SD and compared using a two-sided Student’s t test. Non-normally distributed continuous variables were expressed as the median with the interquartile range and compared using the Mann-Whitney U test. Continuous variables were analyzed in their original forms to preserve information[25]. The least absolute shrinkage and selection operator (LASSO) regression method was used to select predictors in the derivation cohort. The predictive models were further built using multiple logistic regression analysis. The nomogram was formulated based on multivariate logistic regression analysis. Receiver operating characteristic (ROC) curve and the area under ROC curve (AUC) were used to evaluate the discriminative power of the predictive model, which referred to the ability of the model to differentiate between the subjects that did or did not experience the outcome event[25]. The calibration curves were plotted to measure the predictive accuracy of the model, which reflected the agreement between predictions from the model and observed outcomes. A well-calibrated model indicated that the prediction was lying on or around the 45° line of the calibration plot[25]. Hosmer-Lemeshow (H-L) goodness of fit test was used to quantify the calibration curve. The P value was determined by the H-L test. P > 0.05 suggested an optimal consistency between model prediction and the criteria required for standard diagnosis. Decision curve analysis (DCA) was used to assess the clinical utility of the model, which indicated he relationship between a model-predicted probability threshold and the relative value of net benefit[25].

Statistical analysis was performed using R 4.0.3[26] and MedCalc 15.8 software. A two-side P < 0.05 was considered to indicate a statistically significant difference. The nomogram and calibration curve were plotted using rms package and DCA was plotted using rmda package. ROC was plotted with MedCalc 15.8.

RESULTS

Baseline characteristics

Between January 1 2017 and May 31 2021, 628 patients with AP were recruited from four hospitals and reviewed. The exclusion criteria included the following: age < 18 years (2 patients), lack of Atlanta Classification or etiology (26 patients) and incomplete data (3 patients). Following screening, 407 and 190 patients were involved in the derivation and validation cohorts, respectively. The number of participants in each cohort met the requirement of sample size. The detailed demographic and clinical information were described in Tables 1 and 2.

Table 1

Characteristics of non-severe acute pancreatitis and severe acute pancreatitis patients in derivation and validation cohort

Characteristic	Derivation cohort		P value	Validation cohort		P value
	non-SAP (n = 327)	SAP (n = 80)		non-SAP (n = 131)	SAP (n = 59)
Age (yr)	49.0 (36.0; 63.5)	46.0 (34.8; 62.2)	0.424	40.0 (32.0; 58.5)	44.0 (38.0; 53.0)	0.371
Female (%)	131 (40.1)	27 (33.8)	0.363	63 (48.1)	14 (23.7)	0.003
Weekend admission (%)	65 (19.9)	26 (32.5)	0.023	37 (28.2)	13 (22.0)	0.471
Etiology (%)			0.021			0.003
Biliary	113 (34.6)	31 (38.8)		54 (41.2)	11 (18.6)
Hypertriglyceridemia	113 (34.6)	36 (45.0)		44 (33.6)	21 (35.6)
Alcoholic	27 (8.26)	7 (8.75)		5 (3.82)	8 (13.6)
Others	74 (22.6)	6 (7.50)		28 (21.4)	19 (32.2)
Peritoneal irritation (%)	74 (22.6)	31 (38.8)	0.005	27 (20.6)	32 (54.2)	< 0.001
GCS < 15 (%)	2 (0.61)	5 (6.25)	0.004	3 (2.29)	7 (11.9)	0.011
Tem (℃)	36.6 (36.4; 37.0)	37.2 (36.5; 38.2)	< 0.001	37.0 (36.8; 37.4)	38.0 (37.5; 38.4)	< 0.001
SBP (KPa)	17.2 (15.9; 18.7)	16.4 (14.9; 18.8)	0.119	16.3 (15.2; 18.4)	16.8 (14.7; 19.5)	0.949
HR (bpm)	82.0 (76.0; 94.5)	118 (99.5; 132)	< 0.001	89.0 (79.0; 102)	110 (100; 130)	< 0.001
RR (bpm)	20.0 (18.0; 20.0)	22.0 (20.0; 28.2)	< 0.001	20.0 (18.0; 21.0)	23.0 (20.5; 29.5)	< 0.001
WBC (× 109/L)	11.6 (8.74; 15.2)	14.2 (10.2; 18.7)	0.001	12.4 (8.70; 15.6)	16.4 (12.2; 21.1)	< 0.001
HCT (%)	40.8 (37.5; 44.8)	43.6 (36.9; 48.4)	0.022	41.3 (37.8; 45.0)	42.9 (36.5; 49.0)	0.218
PLT (×109/L)	221 (178; 276)	192 (156; 274)	0.03	232 (196; 278)	202 (166; 264)	0.025
K (mmol/L)	3.90 (3.60; 4.25)	3.90 (3.50; 4.30)	0.817	3.90 (3.50; 4.10)	4.00 (3.50; 4.55)	0.074
Na (mmol/L)	137 (134; 140)	137 (132; 140)	0.909	137 (133; 140)	137 (133; 140)	0.664
Ca (mmol/L)	2.26 (2.13; 2.34)	1.96 (1.74; 2.16)	< 0.001	2.22 (2.08; 2.32)	1.93 (1.77; 2.10)	< 0.001
Cr (μmol/L)	67.0 (55.0; 80.0)	90.0 (70.8; 142)	< 0.001	61.0 (52.0; 77.0)	90.0 (62.5; 227)	< 0.001
BUN (mmol/L)	5.00 (3.88; 6.34)	7.06 (5.26; 10.4)	< 0.001	4.66 (3.32; 5.84)	6.67 (4.91; 14.2)	< 0.001
Glu (mmol/L)	8.00 (6.50; 12.9)	11.0 (7.45; 16.3)	< 0.001	8.70 (6.80; 11.8)	11.3 (8.05; 17.2)	0.004
Local complication (%)	104 (31.8)	73 (91.2)	< 0.001	66 (50.4)	55 (93.2)	< 0.001
ICU admission (%)	12 (3.67)	37 (46.2)	< 0.001	12 (9.16)	38 (64.4)	< 0.001
ICU stays (d)	0.00 (0.00; 0.00)	0.00 (0.00; 10.0)	< 0.001	0.00 (0.00; 0.00)	5.00 (0.00; 9.00)	< 0.001
Hospital stays (d)	10.0 (7.00; 14.0)	17.5 (8.75; 25.0)	< 0.001	9.00 (4.00; 15.0)	23.0 (15.0; 30.0)	< 0.001
Mortality (%)	0 (0.00)	3 (3.75)	0.007	1 (0.76)	4 (6.78)	0.033
qSOFA	0.00 (0.00; 0.00)	1.00 (0.00; 1.00)	< 0.001	0.00 (0.00; 1.00)	1.00 (0.00; 1.00)	< 0.001
SIRS	1.00 (0.00; 2.00)	3.00 (2.00; 3.00)	< 0.001	1.00 (0.00; 2.00)	3.00 (2.00; 3.00)	< 0.001
BISAP	1.00 (0.00; 1.00)	2.00 (2.00; 3.00)	< 0.001	1.00 (0.00; 2.00)	2.00 (2.00; 3.00)	< 0.001

SAP: Severe acute pancreatitis; GCS: Glasgow coma score; Tem: Temperature; SBP: Systolic blood pressure; HR: Heart rate; RR: Respiratory rate; WBC: White blood cell; HCT: Hematocrit; PLT: Platelet; Cr: Creatine; BUN: Blood urea nitrogen; Glu: Glucose; ICU: Intensive care unit; qSOFA: Quick sequential organ failure assessment; SIRS: Systemic inflammatory response syndrome; BISAP: Bedside index for severity in acute pancreatitis.

Table 2

Characteristics of non-acute respiratory distress syndrome and acute respiratory distress syndrome patients in derivation and validation cohorts

Characteristic	Derivation cohort		P value	Validation cohort		P value
	non-ARDS (n = 345)	ARDS (n = 62)		non-ARDS (n = 153)	ARDS (n = 37)
Age (yr)	49.0 (36.0; 64.0)	42.0 (33.2; 59.5)	0.062	40.0 (33.0; 57.0)	48.0 (41.0; 57.0)	0.082
Female (%)	138 (40.0)	20 (32.3)	0.312	67 (43.8)	10 (27.0)	0.093
Weekend admission (%)	73 (21.2)	18 (29.0)	0.228	39 (25.5)	11 (29.7)	0.751
Etiology (%)			0.024			0.187
Biliary	122 (35.4)	22 (35.5)		56 (36.6)	9 (24.3)
Hypertriglyceridemia	119 (34.5)	30 (48.4)		53 (34.6)	12 (32.4)
Alcoholic	28 (8.12)	6 (9.68)		8 (5.23)	5 (13.5)
Others	76 (22.0)	4 (6.45)		36 (23.5)	11 (29.7)
Peritoneal irritation (%)	80 (23.2)	25 (40.3)	0.007	39 (25.5)	20 (54.1)	0.002
GCS < 15 (%)	4 (1.16)	3 (4.84)	0.075	4 (2.61)	6 (16.2)	0.004
Tem (℃)	36.6 (36.4; 37.0)	37.5 (36.5; 38.2)	< 0.001	37.0 (36.8; 37.5)	38.0 (37.8; 38.5)	< 0.001
SBP (mmHg)	129 (119; 140)	123 (110; 141)	0.113	123 (116; 140)	119 (106; 147)	0.192
HR (bpm)	82.0 (76.0; 96.0)	120 (103; 134)	< 0.001	90.0 (80.0; 110)	115 (100; 140)	< 0.001
RR (bpm)	20.0 (18.0; 20.0)	23.0 (20.0; 30.0)	< 0.001	20.0 (18.0; 22.0)	24.0 (20.0; 29.0)	< 0.001
WBC (× 109/L)	11.9 (8.76; 15.3)	13.9 (9.91; 19.5)	0.006	12.5 (9.07; 16.3)	17.6 (13.6; 22.8)	< 0.001
HCT (%)	40.8 (37.5; 44.8)	45.2 (38.3; 48.7)	0.002	41.3 (37.4; 45.2)	43.9 (38.0; 49.4)	0.094
PLT (× 109/L)	218 (176; 275)	207 (167; 274)	0.26	228 (192; 278)	208 (168; 261)	0.112
K (mmol/L)	3.90 (3.60; 4.30)	3.90 (3.50; 4.20)	0.441	3.90 (3.50; 4.20)	3.90 (3.50; 4.40)	0.429
Na (mmol/L)	137 (134; 140)	137 (132; 140)	0.95	137 (133; 140)	139 (132; 144)	0.038
Ca (mmol/L)	2.25 (2.12; 2.34)	1.88 (1.65; 2.12)	< 0.001	2.16 (2.01; 2.29)	1.96 (1.77; 2.22)	0.002
Cr (μmol/L)	67.0 (55.0; 81.0)	86.5 (72.0; 134)	< 0.001	62.0 (53.0; 79.0)	96.0 (65.0; 230)	< 0.001
BUN (mmol/L)	5.03 (3.90; 6.44)	7.28 (5.25; 10.3)	< 0.001	4.80 (3.37; 6.48)	6.97 (5.37; 14.4)	< 0.001
Glu (mmol/L)	8.10 (6.50; 13.1)	11.0 (7.67; 16.1)	< 0.001	8.80 (6.90; 13.6)	11.3 (8.10; 16.8)	0.032
Local complication (%)	115 (33.3)	62 (100)	< 0.001	88 (57.5)	33 (89.2)	0.001
Ventilator (%)	3 (0.87)	4 (6.45)	0.012	0 (0.00)	23 (62.2)	< 0.001
ICU admission (%)	16 (4.64)	33 (53.2)	< 0.001	21 (13.7)	29 (78.4)	< 0.001
ICU stays (d)	0.00 (0.00; 0.00)	2.50 (0.00; 11.8)	< 0.001	0.00 (0.00; 0.00)	7.00 (1.00; 11.0)	< 0.001
Hospital stays (d)	10.0 (7.00; 15.0)	19.0 (11.0; 25.0)	< 0.001	10.0 (4.00; 19.0)	23.0 (15.0; 30.0)	< 0.001
Mortality (%)	1 (0.29)	2 (3.23)	0.062	1 (0.65)	4 (10.8)	0.005

ARDS: Acute respiratory distress syndrome; GCS: Glasgow coma score; Tem: Temperature; SBP: Systolic blood pressure; HR: Heart rate; RR: Respiratory rate; WBC: White blood cell; HCT: Hematocrit; PLT: Platelet; Cr: Creatine; BUN: Blood urea nitrogen; Glu: Glucose; ICU: Intensive care unit; qSOFA: Quick sequential organ failure assessment; SIRS: Systemic inflammatory response syndrome; BISAP: Bedside index for severity in acute pancreatitis.

Predictors and model construction

Four variables (HR, RR, Ca and BUN) were extracted as the predictors of SAP through LASSO regression. Supplementary Figure 1 and Supplementary Table 1 show this process in more detail. Multivariable logistic regression revealed that all four variables were independent predictors (Table 3). The probability (PA) of SAP could be calculated according to the following formula: PA = 1/{1 + exp [- (-6.42 + 0.05 × HR + 0.08 × RR - 1.30 × Ca + 0.14 × BUN)]}. Analysis of ARDS obtained similar results (Table 3). The following formula was used for ARDS: PA = 1/{1 + exp [- (-5.46 + 0.05 × HR + 0.10 × RR - 1.78 × Ca + 0.11 × BUN)]}. Two nomogram plots were displayed using prediction models (Figure 1).

Table 3

Multivariable logistic regression analysis for severe acute pancreatitis and acute respiratory distress syndrome prediction in derivation cohort

Variable	SAP			ARDS
	β	OR (95% CI)	P value	β	OR (95% CI)	P value
Intercept	-6.42	0.00 (0.00-0.05)	< 0.001	-5.46	0.00 (0.00-0.13)	0.002
HR	0.05	1.05 (1.04-1.07)	< 0.001	0.05	1.05 (1.03-1.07)	< 0.001
RR	0.08	1.08 (1.00-1.17)	0.047	0.10	1.10 (1.02-1.19)	0.014
Ca	-1.30	0.26 (0.09-0.73)	0.011	-1.78	0.17 (0.06-0.48)	0.001
BUN	0.14	1.15 (1.09-1.23)	< 0.001	0.11	1.12 (1.05-1.19)	< 0.001

SAP: Severe acute pancreatitis; ARDS: Acute respiratory distress syndrome; OR: Odds ratio; HR: Heart rate; RR: Respiratory rate; Ca: Serum calcium concentration; BUN: Blood urea nitrogen.

Figure 1

Nomograms of new predictive models. A: Nomogram of severe acute pancreatitis predictive model; B: Nomogram of acute respiratory distress syndrome prediction model. SAP: Severe acute pancreatitis; ARDS: Acute respiratory distress syndrome; HR: Heart rate; RR: Respiratory rate; Ca: Serum calcium concentration; BUN: Blood urea nitrogen.

Model performance

The new model indicated a great power of discrimination for SAP. Following 1000 interactions of bootstrapping to minimize the risk of overfitting to the original models, the AUC in the derivation cohort was estimated to 0.879 (95%CI: 0.830-0.928), which was significantly superior to that of SIRS (AUC = 0.808, 95%CI: 0.757-0.859, P = 0.002) and qSOFA (AUC = 0.730, 95%CI: 0.672-0.789, P < 0.001) and not inferior to that of the BISAP score (AUC = 0.888, 95%CI: 0.847-0.929, P = 0.6629) (Figure 2A, Table 4). In addition, the model indicated an optimal behavior in the validation cohort (AUC = 0.898, 95%CI: 0.848-0.949) (Figure 2B). The AUC of the new model in derivation was 0.892 (95%CI: 0.843-0.941) for ARDS prediction, which was superior to SIRS (AUC = 0.815, 95%CI: 0.766-0.864, P = 0.001) and qSOFA (AUC = 0.742, 95%CI: 0.678-0.807, P < 0.001) and not inferior to BISAP (AUC = 0.871, 95%CI: 0.827-0.916, P = 0.344) (Figure 2C, Table 4). Despite the assessment of the model in the validation cohort, its performance was moderate (AUC = 0.833. 95%CI: 0.754-0.912) (Figure 2D). When the cut-off value was set as PA > 25% for SAP prediction, the novel model suggested an optimal performance in the combined dataset (sensitivity 0.78, specificity 0.88) (Table 4). The best cut-off value was PA > 18% for ARDS prediction, with a sensitivity of 0.78 and a specificity of 0.85 (Table 4).

Figure 2

Receiver operating characteristic curves of different predictive models in derivation and validation cohort. A: Receiver operating characteristic (ROC) curves of severe acute pancreatitis (SAP) predictive models in derivation cohort; B: ROC curves of SAP predictive models in validation cohort; C: ROC curves of acute respiratory distress syndrome (ARDS) predictive models in derivation cohort; D: ROC curves of ARDS predictive models in validation cohort. BISAP: Bedside index for severity in acute pancreatitis; qSOFA: Quick sequential organ failure assessment; SIRS: Systemic inflammatory response syndrome.

Table 4

Sensitivity, specificity, positive predictive value, negative predictive value for of the predictive models in combined cohort

SAP						ARDS
PA	SE	SP	Youden index	PPV	NPV	PA	SE	SP	Youden index	PPV	NPV
> 10%	0.91	0.63	0.54	0.43	0.96	> 5%	0.94	0.48	0.42	0.26	0.98
> 20%	0.82	0.83	0.65	0.59	0.94	> 10%	0.85	0.71	0.56	0.37	0.96
> 25%	0.78	0.88	0.66	0.66	0.93	> 18%	0.78	0.85	0.63	0.51	0.95
> 40%	0.71	0.93	0.64	0.75	0.91	> 30%	0.67	0.93	0.59	0.65	0.93
> 60%	0.51	0.97	0.48	0.84	0.87	> 50%	0.46	0.96	0.43	0.71	0.90
> 80%	0.32	0.99	0.30	0.88	0.83	> 70%	0.29	0.98	0.27	0.71	0.87
> 95%	0.14	1.00	0.14	0.91	0.79	> 85%	0.12	1.00	0.12	0.86	0.85

PA: Probability; SE: Sensitivity; SP: Specificity; PPV: Positive predictive value; NPV: Negative predictive value.

Graphical assessment indicated a strong agreement between prediction and observation in both new models (Figure 3). The H-L test indicated that the difference between prediction and observation was not significant both in the derivation (χ= 12.675, P = 0.124) and validation cohorts (χ= 5.852, P = 0.664) with regard to SAP prediction. The model for ARDS prediction revealed improved performance with regard to the calibration in the derivation (χ= 3.753, P = 0.879) and validation cohorts (χ= 2.933, P = 0.939).

Figure 3

Calibration curves of new predictive models. A: Calibration plot of severe acute pancreatitis (SAP) predictive model in derivation cohort; B: Calibration plot of SAP predictive model in validation cohort; C: Calibration plot of acute respiratory distress syndrome (ARDS) predictive model in derivation cohort; D: Calibration plot of ARDS predictive model in validation cohort.

DCA indicated that if the threshold PA was < 80%, using the new model to recognize and manage SAP had a positive net benefit compared with either the treat-all or treat-none (Figure 4A). In case the threshold probability was set to < 70%, the prediction and intervention for ARDS also produced net benefit (Figure 4B).

Figure 4

Decision curve analysis of new predictive models. A: Decision curve analysis of severe acute pancreatitis predictive model in derivation and validation cohort; B: Decision curve analysis of acute respiratory distress syndrome predictive model in derivation and validation cohort.

DISCUSSION

In the current study, novel prediction models were established for SAP and ARDS in patients with AP. The models were also externally validated and exhibited remarkable discriminative power and high degree of consistency with the observation both in the derivation and external validation cohorts. These models suggested that patients with AP who manifested a higher heart rate, respiratory rate, blood urea nitrogen concentrations and lower serum calcium concentrations at admission exhibited a higher risk of developing SAP and ARDS.

AP is a major cause of acute abdomen. Patients with AP usually present with multiple organ dysfunction syndrome (MODS). Although organ dysfunction is mild and transient (< 48 h), approximately 20% of patients will proceed to consistent organ failure (> 48 h), leading to SAP and a high risk of mortality[27]. AP primarily affects the respiratory system and to a lesser extent the renal and cardiovascular systems[23]. ARDS is the critical event, which is noted during lung injury in AP[28]. Cyclooxygenase-2 inhibitors and enteral nutrition have been shown to prevent SAP, shorten the length of hospital stay and reduce infectious complications and mortality in patients with predicted SAP[11,12,29]. Administration of antiplatelet therapy, withdrawal of prehospital amiodarone treatment and administration of nebulized heparin may decrease the incidence of ARDS, inhibit the progression of lung injury and accelerate the recovery of patients at risk of developing ARDS[13-15,30]. Therefore, early identification of patients at risk of developing SAP and ARDS is clinically significant for improving the prognosis of AP.

Various models have been developed to predict SAP and organ dysfunction for AP management[16]. Although the majority of the models were deficient due to some limitations, such as small sample size, single center studies and lack of internal or external validation, several SAP models have been widely used and validated in different cohorts. These validations were performed using APACHE-II score, Ranson criteria, CTSI and BISAP[31-34]. BISAP contains only five variables and is simpler than APACHE-II score (18 items) and the Ranson criteria (11 items). However, their predictive power is equal[35]. In addition, ultrasound is preferred to CT as an efficient and nonradioactive examination used in the emergency department to initially evaluate potential development of AP. Therefore, pleural effusion and CT presentation could not be evaluated in this case to gain the BISAP and CTSI scores. Although LIPS is a popular model to predict ARDS for patients at risk, its calculation is considerably complicated[18]. In addition, its original developing cohort involved only a small part of patients with AP. LIPS had not been previously validated in patients with AP.

Therefore, a simple model with a low number of parameters and without radiology findings would be more practical. The novel predictive model reported in the current study involved only four parameters for both SAP and ARDS prediction and all these variables were routinely tested. The discriminatory power of the novel model was not inferior to that of BISAP. To the best of our knowledge, the prediction of SAP or ARDS for patients with AP has not been previously assessed by models that were as simple and accurate as this reported in the current study.

Using LASSO regression, calcium was identified as a predictor for both SAP and ARDS. Multiple logistic regression analysis indicated that serum calcium concentration was also the independent predictor. The models of the present study were easier to use than the APACHE-II, Ranson, CTSI and BISAP models and demonstrated improved efficacy than the other two simple models, suggesting their potential clinical significance. ARDS was a non-negligible manifestation of MODS in patients with AP. The majority of the models have mainly focused on the severity classification or mortality prediction of ARDS, whereas the identification of ARDS at an early stage is still challenging. Although LIPS was widely used to predict ARDS, the calculation of the LIPS score was complicated for patients admitted to the emergency department, since certain parameters may be unavailable[18]. The pathogenesis of ARDS involves the activation of signaling pathways, which include various cytokines and inflammatory mediators. Certain molecules, such as interleukin (IL)-6, IL-8, protein C, angiopoietin-2 and miRNAs and specific imaging examinations (X-ray and lung ultrasound) were also identified as predictors of ARDS in single or combined forms[8,36-38]. The data indicated that these new predictors seemed promising. However, the molecules and the examinations identified were not part of the routine clinical practice, which limited their clinical utility. Fei et al[10] used an artificial neural network algorithm to predict ARDS following SAP. The model by Fei et al[10] indicated high accuracy. However, the variable pancreatic necrosis rate was hard to assess when CT was not used and was not evident in the early course of AP.

The pathogenesis of SAP and ARDS involves a series of acute inflammatory reactions[2,39]. SIRS is widely adopted to assess the severity of diseases associated with acute inflammation. Both HR and RR are used in the SIRS model. Therefore, it is reasonable that both HR and RR were identified as predictors of SAP and ARDS. BUN has been shown to reflect volume depletion, renal function, the quality of resuscitates and even the ischemic injury of the pancreas during AP[40]. It has been reported that BUN can independently predict both SAP and the mortality of SAP[40-44]. Therefore, BUN was also involved in other predictive models of SAP, such as GCS, Ranson criteria and BISAP, in addition to our new model. The levels of BUN have not been used as a direct predictor of ARDS. However, this marker can be used as a predictor of pathogenesis in association with other risk factors, such as pancreatitis[39]. Calcium concentration has been closely associated with AP[45]. Hypocalcemia was common in the cohort of the present study and in other AP cohorts; notably in critically ill patients[46]. Elevated cytosolic calcium of pancreatic acinar cells causes premature trypsinogen activation, vacuolization and acinar cell death, which play critical roles in the pathogenesis of AP[47]. However, during the development of certain models for the prediction of SAP, serum calcium was excluded for a variety of factors[48]. Calcium is also involved in the pathogenesis of ARDS as a signaling molecule, leading to paracellular hyperpermeability through endothelial junction-cytoskeleton dissociation[49]. The current model suggested that calcium was an independent predictor of SAP and ARDS in AP, indicating the potential of developing novel drugs for the treatment of AP[50]. To the best of our knowledge, the model of the present study was the simplest used to predict SAP and ARDS within 24 h of AP admission. It is also the first model that involved serum calcium concentration to predict ARDS in AP.

The present study had several limitations. Firstly, organ dysfunction occurred mainly in the first week of AP, whereas accurate onset time was not available in the present study. Therefore, certain patients with SAP or ARDS may have been missed. Moreover, the exact onset time of ARDS was not recorded, so the new model only predicted the risk of ARDS during the whole admission (7-15 d) using the scores gained within 24 h of admission. It might not be appropriate, and the new model could not tell the clinicians when they should prepare for the possible onset of ARDS. Further studies are needed to verify the value of new models on a dynamic timescale. Secondly, the derivation cohort comprised tertiary and secondary hospitals from different regions of China. However, model validation was performed in a tertiary teaching hospital. Although the result of validation was also encouraging, it is hard to ignore that the incidence of SAP and ARDS in the validation cohort was considerably higher than that of the derivation cohort, which could reduce the generalizability in primary or secondary class hospitals, where MAP and MSAP exhibited high proportions. Moreover, certain laboratory examination technologies were different among four hospitals, which increased the systemic error of the data. It must be mentioned that mechanical ventilation will attenuate systemic inflammation of ARDS and the effect varies with patterns[51]. It is unavoidable that new scores to predict ARDS will also be affected. Unfortunately, detailed information of mechanical ventilation was not collected in this study, and further research is needed to investigate the influence of different ventilatory patterns on new models. Thirdly, selection bias was inevitable in a retrospective study. Furthermore, any missing value was deleted to obtain a complete-case dataset for analysis rather than imputating missing data with statistical methods (e.g., multi-imputation), which were not recommended. However, in the present study, only three individuals were removed due to missing data, accounting for a tiny part of the cohort. It was considered that complete data analysis would not affect the overall conclusion. Finally, other common predictive models, such as APACHE-II score, Ranson criteria, CTSI and LIPS were not evaluated due to lack of essential parameters. Therefore, direct comparison among different models was unavailable.

CONCLUSION

Novel models were developed containing only four items to predict SAP and ARDS in patients with AP, which were as accurate as BISAP but simpler. Serum calcium was identified as an important predictor, indicating a potential new strategy for management of AP. Further prospective studies are required to reveal whether early intervention based on novel prediction models could reduce the incidence of SAP and ARDS and finally improve the outcome of patients with AP.

ARTICLE HIGHLIGHTS

Research background

Acute respiratory distress syndrome (ARDS) is a major cause of death in patients with severe acute pancreatitis (SAP), but simple and credible predictive models are absent.

Research motivation

Use of models to predict and identify early patients with SAP and SAP-related ARDS, so that clinicians can manage these patients early to decrease mortality during admission.

Research objectives

To develop and verify new models to predict SAP and SAP-related ARDS.

Research methods

Clinical data from four centers were retrospectively collected. Items selected with least absolute shrinkage and selection operator regression method were involved in multiple logistic regression to develop the final model in development cohort. New models were than verified in validation cohort and assessed with C-index, calibration curve and decision-curve analysis.

Research results

New models could predict SAP and SAP-related ARDS with four easily available items, and performed well.

Research conclusions

We developed and verified simple and reliable models to predict SAP and SAP-related ARDS.

Research perspectives

To verify new models in a larger size of sample, and to investigate the questions raised by reviewer.