Development and validation of a novel nomogram for predicting systemic inflammatory response syndrome's occurrence in patients undertaking flexible ureteroscopy.

Background: The occurrence of systemic inflammatory response syndrome (SIRS) is an early alert for sepsis after flexible ureteroscopy (fURS). Once sepsis occurs, it often leads to severe or fatal consequences. We aimed to identify SIRS patients preoperatively by developing and validating a feasible prognostic nomogram model based on retrospective cohort analysis.
Methods: A total of 311 patients who underwent fURS in Dongguan Kanghua Hospital (Dongguan, China) between 2016 and 2020 were included and randomly divided into a primary cohort (n=219) and validation cohort (n=92). Single factor regression analysis was used to identify the primary cohort's meaningful characters between SIRS and non-SIRS groups. Factors of the primary cohort were then identified by least absolute shrinkage and selection operator (LASSO) regression analysis, and a nomogram was built to execute the subsequent analysis using these factors. Finally, we analyzed and drew the calibration curve, receiver operating characteristic (ROC) curve, and decision curve analysis (DCA) curve to validate the prognostic value of the nomogram in calibration and discrimination.
Results: Review of the single regression analysis of characters in the primary cohort showed gender, stone burden, diabetes, neutrophil (N), lymphocyte (L), neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), lymphocyte-to-monocytes ratio (LMR), urine-WBC, nitrite (Nit), urine culture, and surgery time as significant factors between the SIRS and non-SIRS groups (P<0.05). The LASSO regression analysis suggested NLR, PLR, and urine culture were substantial factors in predicting SIRS postoperatively, lambda.min and lambda.1se (standard error, SE) were 0.01491 and 0.0796. A nomogram built with the three factors showed good calibration and discrimination, with the Brier values 0.064 and 0.034 and the area under curve (AUC) values 0.897 (95% CI: 0.837-0.957) and 0.976 (95% CI: 0.947-1.000) in the primary and validation cohort, respectively. DCA demonstrated the nomogram was clinically useful, and the predict probability of SIRS's occurrence was very close to the actual rate as the risk threshold increased by higher than 60% in clinical impact curve analysis. Conclusions: NLR, PLR, and urine culture were significantly related to the occurrence of SIRS's after fURS. The nomogram with these three factors showed excellent calibration, discrimination, and clinical usefulness. 2022 Translational Andrology and Urology. All rights reserved.

Introduction

With the rapid development of intracorporeal lithotripsy technology and the continuous improvement of supporting equipment, ureteroscopic surgery has increasingly highlighted its minimally invasive and efficient characteristics, becoming one of the main methods for treating upper urinary calculi (1,2).

Uro-sepsis is caused by infection that occurs in the urinary tract and has an incidence between 0.1–4.3% after ureteroscopy, which may be fatal in some cases (3,4). Systemic inflammatory response syndrome (SIRS) is an early hyper-inflammatory phase of sepsis characterized by fever and hyper-metabolism, which can lead to multiple organ dysfunction and septic shock (5,6). While some prediction models or prognostic factors have been reported for SIRS and sepsis after flexible ureteroscopy (fURS), a standard has not been reached (7-10).

This study analyzed the preoperative data of SIRS in patients after fURS to find the prognostic factors of SIRS and achieve the purpose of early detection, early prevention, and early treatment. Our study aimed to identify SIRS patients postoperatively by developing and validating a feasible prognostic model based on a retrospective cohort analysis. We present the following article in accordance with the TRIPOD reporting checklist (available at https://tau.amegroups.com/article/view/10.21037/tau-22-34/rc).

Methods

Source of data and participants

All procedures performed in this study involving human participants were in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Ethics Committee of Dongguan Kanghua Hospital (Dongguan, China) (No. 2021020). Individual consent for this retrospective analysis was waived. A total of 311 patients diagnosed with upper urinary stones who underwent fURS at Dongguan Kanghua Hospital (Dongguan, China) from January 2016 to December 2020 were included and the population was randomly divided into two cohort: a primary cohort (n=219) and validation cohort (n=92), with no missing data in this study. All patients underwent a preoperative CT scan, X-ray scan, blood test, regular urine test, and urine culture, and the variables of each test are shown in . Stone burden was calculated in mm2 according to the formula Σ(0.785 × length(max) × width(max)) (11), and surgery time was recorded during the procedure. Patients who were suffering SIRS or severe infection before surgery were excluded.

Table 1

Descriptive statistics for study variables (n=311)

Study variable	Descriptive statistics
Sample size (N)	311
Gender, n (%)
Male	101 (32.5)
Female	210 (67.5)
Age, median (IQR)	47.0 (38.0, 56.0)
BMI, median (IQR), kg/m2	24.6 (22.1, 27.3)
Diabetes, n (%)	28 (9.0)
Hypertension, n (%)	43 (13.8)
Stone location, n (%)
Ureter	75 (24.1)
Kidney and combined	236 (75.9)
Unilateral or Bilateral, n (%)
Unilateral	260 (83.6)
Bilateral	51 (16.4)
Stones in Inferior calyx, n (%)
No	204 (65.6)
Yes	107 (34.4)
Hydronephrosis, n (%)
None or mild	229 (73.6)
Moderate or severe	82 (26.4)
Stone density, Hu, median (IQR)	718.0 (507.0, 1007.0)
Stone Burden, mm2, median (IQR)	160.0 (96.0, 245.0)
WBC, ×109/L, median (IQR)	7.0 (5.6, 8.1)
N, ×109/L, median (IQR)	4.2 (3.3, 5.3)
M, ×109/L, median (IQR)	0.4 (0.3, 0.5)
L, ×109/L, median (IQR)	1.8 (1.4, 2.3)
PLT, ×109/L, median (IQR)	245.0 (201.0, 280.0)
NLR, median (IQR)	2.3 (1.7, 3.1)
PLR, median (IQR)	134.4 (105.1, 173.8)
LMR, median (IQR)	4.7 (3.4, 5.9)
Creatinine, μmol/L, median (IQR)	77.0 (63.0, 93.0)
BUN, mmol/L, median (IQR)	4.9 (3.9, 6.2)
Urine WBC, /μL, median (IQR)	100.0 (37.0, 173.0)
Urine RBC, /μL, median (IQR)	150.0 (60.0, 818.0)
Nit, n (%)
Negative	265 (85.2)
Positive	46 (14.8)
Urine culture, n (%)
Negative	274 (88.1)
Positive	37 (11.9)
Surgery time, minutes, median (IQR)	65.0 (50.0, 95.0)
SIRS(+), n (%)	28(9.0)

N, neutrophil; L lymphocyte; PLT, platelets; NLR, neutrophil-to-lymphocyte ratio; PLR, platelet-to-lymphocyte ratio; LMR, lymphocyte-to-monocytes ratio; BUN, blood urea nitrogen; Nit, nitrite; SIRS, systemic inflammatory response syndrome.

fURS procedure

The operations were performed by a qualified and well-experienced urologist, and all surgeries were performed after stenting (Cook Medical, Bloomington, USA), which stayed in the ureter for two weeks (or longer) to dilate the operating tract. All patients were given antibiotics 30 minutes just before surgery, and general anesthesia in the lithotomy position was applied. A Karl Storz Flex-Xc digital flexible ureteroscope (Karl Storz, Tuttlingen, Germany) and 65 w Holmium laser (Miracle Laser, Wuhan, China) were used for lithotripsy. The 12/14 Fr ureteral access sheath (UAS) (Cook Medical, Bloomington, USA) was placed just beneath the pelvic-ureter junction (PUJ) with guide-wire guidance to maintain low pressure during surgery, while the irrigate pressure was set at 50 mL/min. A 4.7 Fr double J stent (Cook Medical, Bloomington, USA) was placed after lithotripsy and removed between two or four weeks later in all cases.

Outcome

The presence of SIRS was identified according to the criteria of the International Sepsis Definitions Conference in 2001 (two or more items as follows) (12): (I) Temperature >38 °C or <36 °C; (II) heart rate >90 beats per minute; (III) respiratory rate >20 breaths per minute or PaCO2 (arterial carbon dioxide tension) <32 mmHg; (IV) WBC count >12,000 or <4,000 cells/µL.

Statistical analysis

The primary statistical analysis was performed to identify risk factors using STATA software (version 16.0), and the results were compared using the Chi-squared test or Fisher's exact test. Mann-Whitney U test and t-test were used to compare non-parametric and continuous variables, and the subsequent analysis was performed in R software (version 4.1.2), as described in the results section. LASSO regression analysis was used to reduce data dimension and to select significant prognostic factors. The nomogram was built with the results of multivariable logistic regression analysis using the selected factors from LASSO regression analysis, and validation of the model’s performance was quantified with calibration, discrimination, and clinical usefulness.

The accuracy of calibration and discrimination was assessed by Spiegelhalter Z-test, unreliability test, and the receiver operating characteristic (ROC) curve. The nomogram’s clinical usefulness was evaluated by DCA through calculating net benefits at different threshold probabilities. P<0.05 with two-tailed were considered significant statistically.

Results

Population and clinical characteristics

We enrolled 311 patients diagnosed with upper urinary stones treated with fURS into the study, and all procedures were performed successfully. shows the clinical demographics of patients and reveals their median age was 47.0 (range, 38 to 56), and there were 101 males and 210 females. There were 28 (9.0%) patients with diabetes and 42 (13.8%) were hypertensive. Most stones (236, 75.9%) were located in the kidney and combined position (kidney and ureter), while 107 (34.4%) patients had stones located in the inferior calyx. Bilateral procedures were performed in 51 cases, and the median surgery time of all cases was 65 (range, 50 to 95) minutes. The total occurrence of SIRS postoperatively was 28 (9.0%). Blood and urine samples were collected and recorded preoperatively, and 37 (11.9%) patients showed a positive urine culture.

The population was randomly divided into a primary cohort (n=219) and validation cohort (n=92), for the purpose of statistical analysis. shows the comparison between two cohorts, with the SIRS rate 9.1% [20] and 8.7% [8], respectively. Gender, diabetes, stone burden, neutrophil (N), lymphocyte (L), neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), urine-WBC, nitrite (Nit), urine culture, and surgery time were significantly correlated with the SIRS group in the primary cohort.

Table 2

Characteristics of patients in the primary cohort and validation cohort

Characteristic	Primary cohort (n=219)				Validation cohort(n=92)
	SIRS(–)	SIRS(+)	P value		SIRS(–)	SIRS(+)	P value
N (%)	199 (90.9)	20 (9.1)			84 (91.3)	8 (8.7)
Gender, n (%)			<0.01				0.2
Male	62 (31.2)	15 (75.0)			20 (23.8)	4 (50.0)
Female	137 (68.8)	5 (25.0)			64 (76.2)	4 (50.0)
Age, median (IQR)	46.0 (37.0, 58.0)	53.0 (46.0, 56.5)	0.07		45.5 (39.0, 53.5)	48.5 (36.0, 54.5)	0.98
BMI, kg/m2, median (IQR)	24.2 (22.1, 26.8)	25.5 (21.0, 28.2)	0.66		25.5 (22.0, 27.2)	25.9 (23.6, 29.3)	0.38
Diabetes, n (%)	17 (8.5)	6 (30.0)	<0.01		5 (6.0)	0	1
Hypertension, n (%)	30 (15.1)	4 (20.0)	0.52		8 (9.5)	1 (12.5)	0.58
Stone location, n (%)			0.17				0.68
Ureter	52 (26.1)	2 (10.0)			20 (23.8)	1 (12.5)
Kidney and combined	147 (73.9)	18 (90.0)			64 (76.2)	7 (87.5)
Unilateral or Bilateral, n (%)			0.18				0.67
Unilateral	172 (86.4)	15 (75.0)			67 (79.8)	6 (75.0)
Bilateral	27 (13.6)	5 (25.0)			17 (20.2)	2 (25.0)
Stones in inferior calyx, n (%)			0.63				0.05
No	126 (63.3)	14 (70.0)			61 (72.6)	3 (37.5)
Yes	73 (36.7)	6 (30.0)			23 (27.4)	5 (62.5)
Hydronephrosis			1.00				0.39
None or mild	144 (72.4)	15 (75.0)			65 (77.4)	5 (62.5)
Moderate or severe	55 (27.6)	5 (25.0)			19 (22.6)	3 (37.5)
Stone density, Hu, median (IQR)	718.0 (504.0, 991.0)	838.0 (521.5, 1273.0)	0.23		690.5 (517.0, 1012.5)	688.5 (482.5, 937.0)	0.7
Stone Burden, mm2, median (IQR)	150.0 (90.0, 227.0)	200.7 (131.5, 315.0)	0.04		171.1 (89.0, 292.0)	153.0 (112.5, 255.0)	0.97
WBC, ×109/L, median (IQR)	7.0 (5.6, 8.1)	7.6 (5.9, 9.7)	0.35		6.6 (5.5, 7.9)	8.5 (6.5, 11.1)	0.02
N, ×109/L, median (IQR)	4.1 (3.2, 5.3)	6.1 (4.2, 7.1)	<0.01		4.0 (3.3, 5.0)	6.0 (5.1, 9.2)	<0.01
M, ×109/L, median (IQR)	0.4 (0.3, 0.5)	0.4 (0.3, 0.5)	0.72		0.4 (0.3, 0.5)	0.7 (0.3, 1.1)	0.17
L, ×109/L, median (IQR)	1.9 (1.5, 2.3)	1.2 (1.0, 1.9)	<0.01		1.8 (1.4, 2.3)	1.0 (0.9, 1.4)	<0.01
PLT, ×109/L, median (IQR)	247.0 (204.0, 285.0)	259.0 (201.0, 307.0)	0.49		227.5 (196.0, 266.0)	245.5 (216.5, 266.0)	0.6
NLR, median (IQR)	2.2 (1.7, 2.9)	4.4 (3.2, 5.9)	<0.01		2.2 (1.7, 3.0)	7.2 (4.5, 9.4)	<0.01
PLR, median (IQR)	130.6 (104.0, 167.6)	200.0 (128.6, 246.0)	<0.01		129.1 (99.2, 158.6)	194.2 (178.0, 250.1)	<0.01
LMR, median (IQR)	4.9 (3.6, 6.1)	2.8 (2.5, 4.7)	<0.01		4.6 (3.8, 5.4)	1.5 (1.1, 3.1)	<0.01
Creatinine, μmol/L, median (IQR)	77.0 (63.0, 93.0)	78.5 (58.5, 98.0)	0.98		77.5 (62.0, 92.5)	76.5 (65.5, 104.5)	0.74
BUN, mmol/L, median (IQR)	4.8 (3.9, 6.2)	5.0 (3.7, 7.3)	0.84		5.0 (4.2, 6.0)	3.6 (3.2, 5.5)	0.09
Urine WBC, /μL, median (IQR)	100.0 (45.9, 150.0)	500.3 (133.5, 921.5)	<0.01		74.4 (28.9, 150.0)	168.5 (150.0, 370.0)	0.04
Urine RBC, /μL, median (IQR)	150.0 (74.6, 937.0)	161.8 (73.5, 742.5)	0.95		150.0 (50.0, 813.0)	150.0 (33.0, 253.5)	0.45
Nit, n (%)			<0.01				0.01
Negative	177 (88.9)	9 (45.0)			75 (89.3)	4 (50.0)
Positive	22 (11.1)	11 (55.0)			9 (10.7)	4 (50.0)
Urine culture, n (%)			<0.01				0.03
Negative	182 (91.5)	9 (45.0)			78 (92.9)	5 (62.5)
Positive	17 (8.5)	11 (55.0)			6 (7.1)	3 (37.5)
Surgery time, minutes, median (IQR)	60.0 (45.0, 90.0)	95.0 (76.0, 115.0)	<0.01		67.5 (50.0, 90.0)	80.0 (76.5, 100.0)	0.22

SIRS, systemic inflammatory response syndrome; N, neutrophil; L lymphocyte; PLT, platelets; NLR, neutrophil-to-lymphocyte ratio; PLR, platelet-to-lymphocyte ratio; LMR, lymphocyte-to-monocytes ratio; BUN, blood urea nitrogen; Nit, nitrite.

Nomogram development

To select characters, the LASSO regression technique [“glmnet” package (13) in R software] was used in the primary cohort. The LASSO regression analysis selected variables based on non-zero coefficients from the whole of 26 variables collected from patients. The lambda.min was 0.0149 and selected 11 variables, while the lambda.1se was 0.0796 and selected three variables (). In this study, the prognostic factors were chosen by the lambda.1se criteria, and the three predictors included NLR, PLR, and urine culture. A multivariable logistic regression model was built with the three factors, and the model was developed as a nomogram as presented in using “rms” package (14) in R software. The risk score of SIRS was calculated by summarizing the scores of all selected factors.

Figure 1

LASSO regression analysis for factors selection. (A) Selection of the tuning parameter (λ) by using 10-fold cross-validation via minimum and 1-SE criteria in the LASSO regression analysis. (B) Non-zero coefficients selection by using the 10-fold cross-validation, while the optimal λ (1-SE criteria, the right line) selected three non-zero coefficients (NLR, PLR, and urine culture). NLR, neutrophil-to-lymphocyte ratio; PLR, platelet-to-lymphocyte ratio.

Figure 2

Presentation of the prognostic nomogram model. The nomogram was constructed with three factors (NLR, PLR, and urine culture) selected by LASSO regression analysis in the primary cohort. NLR, neutrophil-to-lymphocyte ratio; PLR, platelet-to-lymphocyte ratio.

Validation of the nomogram

To validate the performance of the nomogram in predicting the probability of SIRS after fURS, we drew the calibration and discrimination curves by using “rms” (14) and “pROC” (15) packages in R software (). The Brier scores were 0.064 and 0.034, with P values of 0.671 and 0.749, respectively, in the primary and validation cohort, indicating good accuracy of calibration. The ROC curves presented the AUC values of 0.897 (95% CI: 0.837–0.957) and 0.976 (95% CI: 0.947–1.000), respectively, revealing excellent discrimination of the nomogram.

Figure 3

Calibration and discrimination curves of the prognostic nomogram model. Calibration curves of the primary cohort (A) and the validation cohort (B). The Y-axis scale represents the actual value probability of SIRS and the X-axis scale represents the predicted value generated using the nomogram model. The dotted grey line represents the nomogram's prediction performance while the solid black line represents an ideal model. Discrimination curve: ROC curves showed AUC values of 0.897 and 0.976, respectively, in the primary cohort (C) and validation cohort (D). ROC, receiver operating characteristic; AUC, area under ROC curve; SIRS, systemic inflammatory response syndrome.

Clinical usefulness assessment

The DCA curve presented in was constructed using the “rmda” package (16) in R software. DCA showed a significant range threshold probability, indicating the nomogram had an excellent net benefit to identify SIRS patients after fURS, and that patients would benefit significantly by using it. Additionally, the predicted probability of SIRS’s occurrence was very close to the actual rate as the risk threshold increased by higher than 60% in clinical impact curve analysis.

Figure 4

The DCA curves (A) and the clinical impact curve (B) of the nomogram model. (A) The Y-axis scale represents net benefit. The red and green lines represent the performance of the nomogram model in the primary and validation cohort, respectively. (B) By applying the nomogram model in predicting probabilities for 1,000 patients, the red line represents the number of high-risk patients divided according to the nomogram model with different probabilities and the dotted blue curve is the actual number of patients with SIRS in different probabilities. DCA, decision curve analysis; SIRS, systemic inflammatory response syndrome.

Discussion

In this study, we found there was no significant difference in WBC count, stone location, stone density, hydronephrosis and surgery time between SIRS and non-SIRS groups. Interestingly, although diabetes, gender, and stone burden were considered significant, the LASSO regression model excluded these characters by the tuning parameters through the cohort analysis. Therefore, it is difficult to predict a SIRS case using regular blood tests or imaging findings. We further calculated blood inflammation markers such as NLR, PLR, and LMR (17), and urine inflammation markers to explore potential correlation with the occurrence of SIRS’s. After LASSO regression analysis was performed, NLR, PLR, and urine culture were selected as predictors for building the nomogram.

Several studies have demonstrated the value of NLR and PLR in the diagnosis, treatment, and prognosis assessment of many diseases, such as malignant tumours, cardiovascular diseases, sepsis, and even coronavirus (COVID-19) disease (18-21). As the body’s inflammation-immune-related markers, NLR and PLR play essential roles in the occurrence and development of inflammation, and in the early stage of sepsis many mature neutrophils are rapidly activated and the number increases rapidly. The lymphocyte count reflects the patient’s immune status, and usually with the progression of inflammation, the patient’s immune system function declines and many T cells are inhibited or even apoptosed. Simultaneously, the release of various anti-inflammatory cytokines into the blood will lead to immunosuppression and the decrease of T lymphocyte proliferation activity. Platelets (PLT) participate in the amplification process of sepsis inflammation, induce neutrophils to secrete many inflammatory factors, and aggravate the inflammatory response (22). A meta-analysis (23) involving 14 studies and comprising 11,564 cases, demonstrated that the initial NLR value might be a promising candidate as a prognostic biomarker of sepsis. Similarly, in this study, the preoperative NLR and PLR levels may reflect the reserve capacity of the inflammation-immune response in patients, which means, as the NLR and PLR increase, the inflammation-immune capacity may decline simultaneously. Thus, these measures can predict the occurrence of postoperative SIRS, but advanced studies are needed to confirm this.

A positive urine culture indicates the presence of pathogenic bacteria in the urine or stones (7). At the same time, lithotripsy is accompanied by the perfusion of saline, bacteria, and endotoxins in the urine or stones may flow into the blood leading to postoperative SIRS and sepsis, resulting in increased surgery times. Therefore, those with positive urine culture before surgery should be carefully monitored, and sensitive antibiotics for anti-infection treatment based on the results of drug sensitivity or clinical experience should be considered (7,9,24). Similarly, our study revealed that urine culture was one of the main predictors for SIRS.

It is more important to prevent sepsis or SIRS than deal with it without preparation. To this end, we developed a prognostic nomogram based on preoperative predictors. The Brier scores were <0.25 (0.064 and 0.034) and P>0.05 (0.671 and 0.749), indicating good calibration accuracy (25,26), and the validation results showed excellent AUC (0.897 and 0.976), respectively, in the primary and validation cohort, representing good discrimination (27). Moreover, the DCA curve was drawn and showed a significant range threshold probability, suggesting excellent clinical usefulness and indicating patients would obtain a considerable net benefit by using the nomogram. Finally, the clinical impact curve showed patients with a risk score which increases higher than 60% will benefit from the nomogram model. Accordingly, we recommend a reconsideration of performing RIRS surgery while the risk score is higher than 60%, or taking adjuvant treatment before or after the surgery, to prevent SIRS.

There are several limitations to this study. Firstly, this was a retrospective analysis of a single medical institution, and selection bias was prone to occur, which is likely to cause some deviations in the results. It needs to be further confirmed by multi-center clinical trials. Secondly, the LASSO method we used in selecting predictors is a data- based algorithm, although it showed excellent performance in this study, we may take some clinical variables in to construct a more reasonable and accurate nomogram based on practice experience while multi-center data added in.

In summary, we established a novel and feasible predictive nomogram with NLR, PLR, and urine culture to predict the occurrence of SIRS after fURS and validated it in internal and external cohorts. Although the nomogram was mainly constructed with preoperative laboratory findings, it showed excellent performance in discriminatory, calibration, and clinical usefulness. We believe that patients will obtain promising benefits from the use of this nomogram, and clinicians should incorporate it into surgery decision making and pretreatment preoperatively. In future research more specific variables during surgery and the outcomes of pretreating these high-risk patients should be observed.

The article’s supplementary files as