Development and External Validation of Safe Discharge Criteria After Radical Gastrectomy.

AIM: Enhanced recovery after surgery (ERAS) gradually shortens the length of stay but increases the rate of unplanned readmission after discharge. Currently, objective discharge criteria for patients after radical gastrectomy is lacking. This study aimed to construct and validate a nomogram for estimation of the possibility of safe discharge on the fifth-day post radical gastrectomy.
METHODS: We enrolled 496 consecutive patients undergoing radical gastrectomy as the development cohort. After the fifth day of surgery, patients were assigned to the postoperative complication group and no postoperative complication group. Multivariate logistic regression analyses were performed for both groups. Then, we constructed the risk prediction model of postoperative severe complications (PSCs) and applied it to evaluate whether the patient could be discharged safely. The external validation cohort comprised 245 patients, whom we used to evaluate the capability of our model to predict the risk of PSCs. The primary measure was the negative predictive rate (NPR) and the area under the curve (AUC).
RESULTS: Through multivariate analysis, gender, maximum body temperature on the 4th postoperative day (POD4), oral intake and ambulatory duration on POD4, the proportion of neutrophils (≥75% or <75%) and pain score (≥4 or <4) on POD5, and defecation with 5 days after the procedure (yes or no) were identified as independent predictors for PSCs. Upon incorporation of these variables, the nomogram demonstrated a good NPR of 0.957 and 0.916 and AUC of 0.918 and 0.719 in the two cohorts, respectively. With a nomogram score of 110, patients were stratified into low and high risk of PSCs.
CONCLUSION: The nomogram demonstrated good predictive potential for low-risk patients. It could serve as an objective safe discharge approach for patients after the fifth day of radical gastrectomy.

Introduction

Compelling evidence shows that the protocols of Enhanced Recovery After Surgery (ERAS) reduce the potential risk of nosocomial infection, length of hospital stay, and health-care costs, and improve the quality of life of patients after surgery.1–4 At the same time, with the implementation of ERAS, the rate of unplanned readmission after discharge (URAD) increases significantly.1,5 Also, studies have revealed the discharge criteria as an independent risk factor (IRF) for URAD.6

The discharge criteria after radical gastrectomy (DCRG) is mainly based on the clinical judgment of the surgeon, in terms of the postoperative pain, exhaustion and defecation, oral intake, ambulatory duration after the procedure, etc. In the context of ERAS implementation, lack of quantitative discharge criteria and evidence-based medical evidence explains why the rate of URAD cannot be reduced effectively.

With our clinical experience, radical gastrectomy requires an inpatient stay for 4~6 days with the implementation of ERAS. Herein, we selected the fifth-day post radical gastrectomy during which we evaluated whether the patient was eligible for discharge from the hospital. First, the risk factors of complications after the fifth day of radical gastrectomy were analyzed in the retrospective cohort, and the risk prediction model was constructed as an objective evaluation criterion to judge whether or not the patients were eligible for discharge on the fifth day post-surgery. We finally validated, the predictive efficacy of the prediction model using subsequent observation cohort.

Materials and Methods

Data Collection and Processing

This study was performed in a large teaching hospital in China. All patients provided written informed consent for review of their medical records, in accordance with the Declaration of Helsinki. The Ethics Committee of Xijing Hospital approved all protocols.

Study was achieved in two phases. In the first stage, we constructed a predictive model of the risk of severe complications after the fifth day of radical gastrectomy using the retrospective cohort. Patients who underwent radical gastrectomy between March 2016 and March 2017 were searched as the development set (DS) from the electronic medical record system of the hospital. Exclusion criteria for patients included: history of gastric surgery, emergency surgery, failure to perform surgery or patient refusal to undertake the operation, ASA grade>III, combined with other malignant tumors or palliative surgery due to peritoneal dissemination or distant metastases, length of postoperative hospital stay<5 days, complications with Clavien-Dindo (CD) grade≧III within 5 days after the procedure, failure to meet the design requirements of complete data collection and refusal to sign informed consent. In the second stage, we validated the model using another observational cohort. Patients who met the inclusion criteria from September 2018 to March 2019 were enrolled as the validation set (VS). Figure 1 shows a flow chart of the research design.

Figure 1

Study flow chart.

The protocols of ERAS were implemented in all cases. Discharge criteria were based on traditional clinical experience, including no obvious postoperative complications, stable vital signs, no parenteral nutrition, autonomous activity, using oral pain medication to manage postoperative pain, and so on.

The clinical data, intraoperative conditions, pathology, postoperative conditions on the 4th day of surgery (POD4) and POD5, and postoperative complications were recorded between the 5th and 30th-day post-surgery.

We defined safe discharge as no postoperative severe complications (CD≥III) after discharge. Using our clinical experience and literature review on ERAS protocols in radical gastrectomy for gastric cancer, we, on the 5th day after the operation, judged whether the patient could be discharged safely. As such, the safe discharge criteria after the fifth day of radical gastrectomy were based on the prediction of the risk of serious complications considered safe to allow discharge.

Serious postoperative complications between the 5th and 30th-day post-surgery were regarded as positive events, whereas no serious complications were considered as negative events. To reduce the unplanned readmission rate after discharge, the primary measure of accuracy was the negative predictive rate (NPR) and the area under the curve (AUC), and secondary measure was specificity and sensitivity.

This is an observational cohort study, data collection was performed by a clinical fellow.

Statistical Analysis

All statistical data were analyzed via SPSS 22.0 (SPSS Inc., Chicago, IL, USA), R software (version 3.5.1, ), and EmpowerStats (, X&Y Solutions, Inc., Boston, MA). The Mann–Whitney U-test was applied to analyze the continuous variables, expressed as median ± SD. Besides, categorical data were cross-tabulated, and differences in proportions were tested using the chi-squared test or Fisher’s exact test. All tests were two-sided, and p-values less than 0.05 denoted statistical significance.

Univariate and multivariate logistic regression analyses were performed to reveal the independent risk factors associated with PSCs; thereafter, we recorded the odds ratios (OR) and their 95% confidence intervals (CI). According to the results of multivariate analysis, a prediction model was constructed using R software and then subjected to external validation. Calibration curves were regenerated to assess the predictive accuracy of the model. Parameters, including AUC, the optimal cutoff point of the model, the sensitivity, specificity, positive prediction rate (PPR), NPR, and theirs 95% CI were used for performance evaluation of the model. Finally, we applied the decision curve analysis (DCA) to evaluate the clinical net benefit of the model.

Results

Patient Characteristics

After screening, 496 patients in the development set and 245 patients in the validation set who had relatively complete follow-up details were analyzed. Postoperative severe complications occurred in 118 patients (23.8%) in DS, and 46 patients (18.5%) in VS. The clinical details of the patients are outlined in Table 1.

Table 1

Comparison of Demographics of the Development and Validation Sets

Variables	Development Set (N=496)	Validation Set (N=245)
Age, years, mean±SD	58.4±10.7	58.6±17.7
Gender, female, %	132 (26.6%)	53 (21.6%)
BMI, kg/m2, mean±SD	22.6±3.1	23.4±3.3
Smoke, n, %	300 (60.5%)	116 (47.3%)
Diabetes, n, %	46 (9.3%)	18 (7.3%)
Hypertension, n, %	88 (17.7%)	41 (16.7%)
COPD, cases, %	56 (11.3%)	1 (0.4%)
NRS2002≧3, n, %	234 (47.2%)	77 (31.4%)
ASA grade, n, %
I	10 (2.0%)	2 (0.8%)
II	424 (85.5%)	22 (90.6%)
III	62 (12.5%)	21 (8.6%)
T stage, n, %
T1	118 (23.8%)	59 (24.1%)
T2	58 (11.7%)	34 (13.9%)
T3	96 (19.4%)	73 (29.8%)
T4	224 (45.2%)	79 (32.2%)
N stage, n, %
N0	188 (37.9%)	102 (41.6%)
N1	84 (16.9%)	43 (17.5%)
N2	94 (19.0%)	51 (20.8%)
N3	130 (26.2%)	49 (20.0%)
Type of gastrectomy, n, %
Proximal gastrectomy	8 (1.6%)	30 (12.2%)
Distal gastrectomy	298 (60.1%)	105 (42.9%)
Total gastrectomy	190 (38.3%)	110 (44.9%)
Multivisceral resection, n, %	34 (6.9%)	6 (2.4%)
Operation time, min, mean±SD	214.1±59.5	204.4±56.4
IBL, mL, mean±SD	153.2±130.1	149.7±169.7
PSCs, n, %	118(23.8%)	46 (18.5%)
Pulmonary infection	52 (10.5%)	21 (8.6%)
Abdominal infection	38 (7.7%)	12 (4.9%)
Paralytic ileus	30 (6.1%)	8 (3.3%)
Fistula	30 (6.1%)	12 (4.9%)
Mechanical ileus	12 (2.4%)	5 (2.0%)
Wound dehiscence	10 (2.0%)	2 (0.8%)
Chylous fistula	6 (1.2%)	0
Gastroparesis	2 (0.4%)	3 (1.2%)
Haemorrhage	12 (2.4%)	7 (2.9%)
Urinary tract infection	2 (0.4%)	0
Respiratory failure	4 (0.8%)	3 (1.2%)
Pulmonary embolism	2 (0.4%)	2 (0.8%)
Atelectasis	10 (2.0%)	6 (2.4%)
Death	8 (1.6%)	2 (0.8%)
PHS, day, mean±SD	7.5±5.5	6.9±4.5

Abbreviations: SD, standard deviation; BMI, body mass index; COPD, chronic obstructive pulmonary disease; NRS, nutritional risk screening; ASA, American Society of Anesthesiologists; IBL, Intraoperative blood loss; PSCs, postoperative severe complications; PHS, postoperative hospital stay.

Independent Factors Influencing Safe Discharge

All patients were subjected to 30 days of postoperative follow-up. Univariate analysis demonstrated that age, gender, type of gastrectomy, maximum body temperature, oral intake and ambulatory duration on POD4, heart rate, leucocyte count, the proportion of neutrophils and pain score on POD5, and defecation with 5 days after procedure were associated with PSCs (p<0.1) (Table 2). Multivariate analysis revealed that gender, maximum body temperature on the 4th postoperative day (POD), oral intake and ambulatory duration on POD4, the proportion of neutrophils and pain score on POD5, and defecation with 5 days after the procedure were independent factors associated with safe discharge (p<0.05). These factors influenced safe discharge in gastric cancer patients after radical gastrectomy.

Table 2

Univariate and Multivariate Analysis of Postoperative Severe Complications for the Development Set

Variables	Univariate Analysis		Multivariate Analysis
	OR (95% CI)	p	OR (95% CI)	p
Age	1.040 (1.018–1.062)	0.018	1.025 (0.990–1.061)	0.158
Gender		0.077		0.016
Female	1		1
Male	1.567 (0.950–2.585)		2.652 (1.196–5.885)
Type of gastrectomy		0.075		0.545
Total	1		1
Distal	0.662 (0.434–1.008)		1.494 (0.721–3.095)
Proximal	0.840 (0.164–4.289)		0.832 (0.028–24.360)
Ambulatory	0.551 (0.501–0.600)	0.000	0.579 (0.466–0.718)	0.000
Intake	0.403 (0.346–0.459)	0.000	0.999 (0.998–0.999)	0.000
Temperature	1.015 (1.012–1.017)	0.000	3.413 (2.083–5.591)	0.000
HR		0.000		0.967
<100bpm	1		1
≧100bpm	5.236 (2.086–13.142)		0.971 (0.243–3.884)
Leucocyte		0.033		0.285
<12*10^9/L	1		1
≧12*10^9/L	1.881 (1.044–3.388)		1.638 (0.663–4.048)
Neutrophils		0.000		0.002
<75%	1		1
≧75%	3.213 (2.097–4.924)		2.701 (1.423–5.124)
Defecation		0.000		0.000
No	1		1
Yes	0.409 (0.267–0.627)		0.274 (0.140–0.534)
VAS		0.000		0.000
<4	1		1
≧4	8.346 (3.130–22.255)		5.033 (1.311–19.332)

Abbreviations: Ambulatory, ambulatory duration on POD4; Intake, oral intake on POD4; Temperature, maximum body temperature on POD4; HR, heart rate on POD5; Leucocyte, leucocyte count on POD5; Neutrophils, the proportion of neutrophils on POD5; Defecation, defecation with 5 days after surgery; VAS, pain score on POD5.

Construction of the Nomogram

Considering the results of multivariate analysis, we applied the seven significant independent factors to construct the prognostic nomogram to predict safe discharge post radical gastrectomy (Figure 2). Among the factors, ambulatory duration, oral intake, and maximum body temperature on POD4 were most strongly associated with the prediction of PSCs.

Figure 2

Nomogram for predicting safe discharge five days after radical gastrectomy. To apply this nomogram, each variable axis represented an individual risk factor, and the line drawn upwards was applied to determine the points of each variable. Then, the total points were calculated to obtain the probability of postoperative severe complications (PSCs) 5 days after radical gastrectomy. Total points less than 110 implied that the patient had a low risk of PSCs, and was safely discharged.

Validation of the Nomogram

Our prognostic nomogram had score range of 0–260, and at best threshold of 110 had the NPR of 0.957 (95% CI: 0.928–0.976), AUC of 0.918 (95% CI: 0.891–0.941), sensitivity of 0.881 (95% CI: 0.809–0.934), specificity of 0.815 (95% CI: 0.772–0.853). The quality assessment and the ROC curve of the nomogram are depicted in Table 3 and Figure 3, respectively. The calibration curves for PSCs demonstrated between the nomogram-predicted probability and the actual observations concurred (Figure 4). For the DCA, our nomogram showed a superior net benefit across a wider scale of threshold probabilities for the prediction of PSCs (Figure 5).

Table 3

Quality Assessment of Prediction Model for Postoperative Severe Complications Occurring After 5 Days After Radical Gastrectomy

Evaluation Index (95% CI)	Development Set	Validation Set
Actual prevalence	0.238 (0.201–0.278)	0.188 (0.141–0.242)
Predicted prevalence	0.351 (0.309–0.395)	0.514 (0.450–0.578)
Area under ROC curve(AUC)	0.918 (0.891–0.941)	0.719 (0.645–0.794)
Sensitivity	0.881 (0.809–0.934)	0.783 (0.636–0.891)
Specificity	0.815 (0.772–0.853)	0.548 (0.476–0.618)
Diagnostic accuracy	0.831 (0.795–0.863)	0.592 (0.527–0.654)
Positive predictive rate	0.598 (0.521–0.671)	0.286 (0.209–0.373)
Negative predictive rate	0.957 (0.928–0.976)	0.916 (0.851–0.959)

Figure 3

Receiver operating characteristic curve of prediction model for postoperative severe complications occurring after five days after radical gastrectomy.

Figure 4

Calibration curves for the nomogram. The nomogram predicted postoperative severe complications occurring after 5 days after radical gastrectomy ((A) For development set; (B) For validation set). The actual probability of PSCs was plotted on the y-axis, and x-axis showed the nomogram-predicted probability.

Figure 5

Decision curve analysis of gastric cancer patients after radical gastrectomy. Decision curve analysis was used to estimate the clinical net benefit of the nomogram ((A) For development set; (B) For validation set). On decision curve analysis, solid lines represent model predictions, and the red line represents the assumed all patients would have postoperative severe complications(PSCs), and horizontal lines assumed no patients would have PSCs.

In the validation set, the nomogram exhibited a high accuracy of prognosis prediction (NPR=0.916, 95% CI: 0.851–0.959) (Table 3). The calibration curves between nomogram predictions and actual observations in low-risk patients were consistent (Figure 3). DCA demonstrated that the nomogram model improved clinical benefits in low-risk patients (Figure 5).

Discussion

Herein, we constructed a nomogram that integrated the independent risk factors for the prediction of the postoperative severe complications of gastric cancer patients after the fifth day of radical gastrectomy. Independent risk factors included gender, the maximum body temperature, oral intake and ambulatory duration on POD4, the proportion of neutrophils (≥75% or <75%) and pain score (≥4 or <4) on POD5, and defecation within 5 days post the procedure (yes or no). The best threshold of the nomogram was 110; however, at a lower score, the probability of non-postoperative severe complications 5 days after surgery was 95.7% when the patient’s score was lower than 110. According to the nomogram, whether a patient’s score exceeding 110 on the fifth day after surgery can be applied as an objective safe discharge criterion after radical gastrectomy.

In recent years, ERAS protocols, including reduced hospital stay and early discharge, have become part of routine perioperative management. The mechanisms for evading postoperative severe complications and unplanned readmission after discharge have become an important clinical issue. Schneider et al7 found that the length of stay decreased continuously in colorectal surgery in the past 20 years, but the readmission rate increased, which demonstrates the need for safe discharge criteria. Elsewhere, Merkow et al8 revealed that postoperative readmission was associated with new complications after discharge, rather than complications during hospitalization. Postoperative severe complications after discharge are closely related to unplanned readmission,8 therefore, the resultant risk of PSCs should be regarded as the standard. Notably, a low risk of complications after discharge would mean that risk of unplanned secondary admission after discharge is also low. As such, the discharge of the patient is safe under this risk probability. A retrospective study on discharge criteria of day surgery by Abdullah et al9 also proposed the importance of the balanced relationship between surgical efficiency and patient postoperative safety, and the establishment of strict discharge criteria.

Previous reports indicate that discharge criteria include factors, such as gastrointestinal function recovery, oral intake, ambulatory duration, pain, body temperature, etc., which concur with independent risk factors in our study. A meta-study by Fiore et al11 on discharge criteria for colorectal cancer surgery revealed that most of the discharge criteria encompassed the recovery of intestinal function, amount of food tolerance, time of autonomous activity, and postoperative pain control. Studies have reported similar drawbacks in the discharge criteria of gastrectomy, among them, more subjectivity and lack of objective criteria. For instance, Stellato et al12 in their study on gastric bypass surgery found that the discharge criteria included an appropriate oral volume, tolerance of oral drug pain, adequate understanding of the risk of surgery, and excelling in written tests. Besides, the criteria for discharge defined by Geubbels et al13 included adequate pain control, no fever, no postoperative nausea and vomiting, fluid diet tolerance, activity, and willingness to leave the hospital. In a study on radical gastrectomy by Liu et al,14 the discharge criteria included recovery of intestinal function after the operation, more than 70% of the original intake and no intravenous fluids, no pain or manageable pain via oral pain medication, and normal daily activities. In previous studies in our hospital, we reported the following discharge criteria: Normal postoperative body temperature, postoperative oral pain medication, postoperative routine activities, no subjective discomfort, normal oral intake, no parenteral nutrition, recovery of intestinal function, Karnofsky’s score≧80 points and the patient’s willingness to go home.14 However, these discharge criteria as mentioned above cannot predict the probability of PSCs after discharge.

Literature reports and clinical experience demonstrate that gastrointestinal function is recovered on the 5th day after radical gastrectomy;9,10 therefore, we constructed the risk prediction model of PSCs and applied it to judge whether patients could safely be discharged. Particularly, we evaluated the possibility of discarding low-risk patients and keeping the high-risk patients for further assessment. Through multivariate logistic regression analysis, we found seven independent risk factors of PSCs after the fifth day of radical gastrectomy, which we applied to construct a model. In the development set, the results of the calibration curve and DCA demonstrated satisfactory model performance in all patients. The model also efficient in low-risk patients, that is, the validation set. Furthermore, we revealed that men and the proportion of neutrophils were independent risk factors of postoperative severe complications after the 5th day of surgery. Sex, as an IRF, may be associated with larger tumors and later stages,15 and higher smoking rates in men. Neutrophils reflect the levels of inflammation, whereby more than 75% show the state of inflammation. Subsequently, discharge criteria based on the prediction of PSCs after the fifth day of radical gastrectomy were constructed successfully. Our primary goal with this model was to identify patients who were eligible for safe discharge on the fifth day of radical gastrectomy, that is, patients with low risk of postoperative severe complications. This required a high negative prediction rate of the model. Both internal and external validation demonstrated that the accuracy of prediction of low-risk cases exceeded 90%, affirming its extremely high clinical application value. Although the prediction accuracy of the model for high-risk cases was low in external validation, this patient group can significantly reduce the UARD rate because they do not meet the discharge criteria according to the model.

In the present study, external validation revealed that the AUC and NPR of the prediction model were inferior to the internal validation results. We suggested the following two aspects as the possible explanation. Firstly, the large period between the development set and validation set potentially impacted the predictive effectiveness of the model. A prediction model should follow two basic principles, (i) principle of inertia16,17 which assume that the relationship between the forecast index and the forecast target remains the same, and (ii) the principle of action attenuation16,18 which assume that the effect of the occurrence of an event on various related systems gradually decreases over time. In this study, the gap between development set and validation set was more than 2 years, during which there were advances in surgical techniques, stricter enforcement on protocols of ERAS, and other non-medical-related factors. Also, the relationship between predictive indicators and predictive targets based on the development set may vary over time, weakening the predictive model reliability in the validation cohort. Secondly, data errors may be associated with an increased risk of predictive complications. In the predictive model, the three indexes, including ambulatory duration, oral intake, and maximum body temperature on POD4 had the greatest effect on the predicted results. In the actual data collection process, the estimation of total intake and daily activity time has the risk of underestimation. As a result, the model tends to increase the risk of complications prediction, that is, it increases the proportion of patients predicted as high risk. This subsequently lowers the reliability of the model prediction.

There are some limitations in the research design and experimental methods. First, being is a single-center study, multi-center external validation is required in the future. Secondly, some of the clinical index data collection are subjective, which may be erroneous due to manual collection. Moreover, the time span of DS and VS cases is large, which lowers the reliability of the prediction model. Thirdly, this predictive model is only applicable to patients who do not have PSCs within 5 days after radical gastrectomy; however, it is not applicable in the prediction of risk of complications within 5 days after surgery. Fourth, our predictive model is based on the assumption that the risk of postoperative severe complications is consistent in both hospitalized and home patients after the fifth day. Notably, under the same conditions, the risk of leaving medical care is slightly higher, which is why we have emphasized more on the negative prediction rate than the positive prediction rate. Patients at high risk of discharge are kept in the hospital as much as possible to ensure their safety.

Conclusion

This study has successfully constructed a nomogram for the prediction of low-risk patients eligible for discharge on the fifth day after radical gastrectomy. Validated using independent data, the likelihood of the patients to have postoperative severe complications after the fifth day after surgery was more than 91.6% at nomogram score less than 110. This scoring standard, to our knowledge, is the first objective criteria to judge on the possibility of safe discharge of patients on the fifth day after radical gastrectomy.