Derivation and external validation of a model to predict 2-year mortality risk of patients with advanced schistosomiasis after discharge.

To date, no risk prediction tools have been developed to identify high mortality risk of patients with advanced schistosomiasis within 2 years after discharge. We aim to derive and validate a risk prediction model to be applied in clinical practice. The risk prediction model was derived from 1487 patients from Jingzhou and externally validated by 723 patients of Huangshi, two prefecture-level cities in Hubei province, China (from September 2014 to January 2015, with follow-up to January 2017). The baseline variables were collected. The mean age [SD] was 62.89 [10.38] years for the derivation cohort and 62.95 [12.22] years for the external validation cohort. The females accounted for 36.3% and 43.7% of the derivation and validation cohorts, respectively. 8.27% patients (123/1487) in the derivation cohort and 7.75% patients (56/723) in the external validation cohort died within 2 years after discharge. We constructed 4 models based on the 7 selected variables: age, clinical classification, serum direct bilirubin (DBil), aspartate aminotransferase (AST), alkaline phosphatase (ALP), hepatitis B surface antigen (HBsAg), alpha fetoprotein (AFP) at admission. In the external validation cohort, the multivariate model including 7 variables had a C statistic of 0.717 (95% CI, 0.646-0.788) and improved integrated discrimination improvement (IDI) value and net reclassification improvement (NRI) value compared to the other reduced models. Therefore, a multivariate model was developed to predict the 2-year mortality risk for patients with advanced schistosomiasis after discharge. It could also help guide follow-up, aid prognostic assessment and inform resource allocation.

Evidence before this study

We searched the PubMed database according to the terms [(“prediction” OR “risk prediction” OR “prediction model” OR “predictive” OR “predictive modeling”) AND (“advanced schistosomiasis”)] among English-language articles before July 13th, 2019. We identified one study using three data mining models to predict the prognosis of patients with advanced schistosomiasis and another study using D-dimer to predict the occurrence of ascites. None of these studies have attempted to predict the 2-year mortality risk of patients with advanced schistosomiasis after discharge based on commonly used clinical indicators at admission, or discussed the performance between different mortality risk prediction models. We hypothesized that applying some commonly used clinical indicators at admission into a risk prediction model can predict 2-year mortality of patients with advanced schistosomiasis and improve the long term outcome of them.

Added value of this study

Our study constructed and compared 4 different models based on some commonly used clinical parameters to predict the 2-year mortality risk of patients with advanced schistosomiasis after discharge. We validated our models in internal and external cohorts by IDI values, NRI values, C statistics and ROC curves. A 7-variable model showed the best performance in the internal and external validation cohorts. Such risk prediction model could help guide follow-up, aid prognostic assessment and inform resource allocation since the disease carries a high risk of complications.

Implications of all the available evidence

To solve the problem of mortality risk prediction model shortages in patients with advanced schistosomiasis, we built a prediction model using age, clinical classification and some routine laboratory test indices at admission to predict the 2-year mortality risk after discharge. The model had shown good predictive performance.

Alt-text: Unlabelled Box

Introduction

Schistosomiasis is caused by infection with worms of the trematode genus Schistosoma, which is a neglected tropical disease affecting up to 250 million people in 76 countries [1]. The number of disability-adjusted life years (DALYs) lost caused by this disease was estimated up to 3.5 million in 2015 [2,3]. In China, schistosomiasis is caused by Schistosoma japonicum. The disease has long captured the attention of the Chinese authorities who have undertaken remarkably successful control programs to substantially reduced the schistosomiasis disease burden over the past 70 years [4].

Advanced schistosomiasis is a chronic disease caused by repeated infection of Schistosomiasis japonicum, which is associated with portal hypertension, splenomegaly, ascites, and gastro-esophageal variceal bleeding, or with granulomatous disease of the colon or severe growth retardation [5]. It has been classified into four clinical sub-types based on patients' major symptoms, namely ascites, splenomegaly, colonic tumoroid proliferation, and dwarfism. Over the past 70 years, significant achievements on schistosomiasis control have been attained through ongoing national control programs in China. At present, dwarfism and colonic tumoroid proliferation are rarely found [6,7]. However, ascites and splenomegaly are still common, typically in foci of high transmission intensity [5]. Based on the national schistosomiasis report issued by the Centers for Disease Control and Prevention in China in 2017, there were 37,601 cases of S. japonicum infection and 29,407 patients suffered from advanced schistosomiasis [8]. These cases were registered and managed independently, since the disease results in high levels of mortality and disability as well as poor quality of life.

Because of the serious consequences of advanced schistosomiasis, a more evidence-based management approach has been advocated by clinical experts [9]. The current consensus among the experts is that patients should be followed for 1 year at least to determine whether their symptoms such as ascites or hemorrhage of upper digestive tract should be controlled and whether they need further treatment [9,10].

However, many patients with advanced schistosomiasis were not sufficiently followed up and were unable to adequately access medical care when their condition exacerbated, which resulted in inadequate opportunities to timely intervene and improve long-term prognosis. On the other hand, not all patients with advanced schistosomiasis progress to death in 2 years after discharge. The follow up of all advanced schistosomiasis patients may lead to unnecessary waste of medical resources. Thus, identifying and screening patients with advanced schistosomiasis of high 2-year mortality risk after discharge can improve the long term prognosis for the patients, while preventing the waste of medical resources. In addition to allowing clinicians to screen for patients with high 2-year mortality, the analysis also has the potential to clarify risk factors for morbidity in acute schistosomiasis patients, including co-morbidities. Furthermore, none of previous prognosis prediction studies on advanced schistosomiasis has attempted to predict the 2-year mortality risk after discharge based on commonly used clinical indicators at admission, or discussed the performance of different mortality risk prediction models [11,12].

In this study, we used population-based demographic, laboratory and clinical data from Jingzhou and Huangshi, two prefecture-level cities in Hubei province, China, to derive and validate mortality risk prediction models for advanced schistosomiasis. Our aim was to develop a practical risk prediction model that could identify patients with advanced schistosomiasis at high 2-year mortality risk after discharge.

Methods

We constructed multivariate prediction models that followed the TRIPOD statement [13] (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis). All procedures performed in this study involving human participants were in accordance with the 1964 Helsinki declaration and its later amendments. Patients' confidentiality was protected by ensuring that the data were addressed in anonymous mode with personal information appropriately de-identified. The study was approved by the Ethics Committee of Tongji Medical College, Huazhong University of Science and Technology. It met the definition of “minimal risk” and a waiver of informed consent was granted. Since this is a retrospective research involving no interventions, the waiver of consent will not adversely affect any rights and welfare of the subjects.

Study population and outcome

We formed a derivation and a validation cohort using a previously constructed database of patients with advanced schistosomiasis from Hubei Province, China. The database was a population-based repository that included blood biochemical measurements from >4000 cases at admission [11]. Patients were diagnosed according to the Diagnostic Criteria for Schistosomiasis (WS261–2006) issued by China's Ministry of Health which covered: (1) the patient who had ever lived in S. japonicum endemic region and has contact history with S. japonicum; (2) involvement of ascites, splenomegaly, portal hypertension and gastro-esophageal variceal bleeding, or with granulomatous lesion of the colon and rectum or severe growth retardation; (3) anti-S. japonicum antibody could be detected using at least one of the following tests: ELISA, IHA, DDI, COPT or DIFA; and (4) the result of stool examination or rectal biopsy was positive.

The discharge time window for the patients was from September 2014 to January 2015. These patients were followed to January 2017. All patients were advised to receive regular follow-ups. Patients were generally followed up every 3 months in the first 2 years and annually in the following years. Patients who did not visit our hospital as scheduled were telephoned for follow-ups to obtain the treatment information and vital status. The primary outcome was 2-year all-cause death after discharge. The study period was defined as the time from the discharge to the date of the last follow-up or death. The all-cause death events which occurred after January 2017 were considered nonevents for our study.

There are several standard treatment plans based on clinical classification, specific symptoms and disease severity. Splenectomy was provided to the splenomegaly type patients if hypersplenism symptom existed. Symptomatic treatment and liver protection measures were applied for ascites type patients. The praziquantel (PZD) treatment was administrated to patients after a stable period of 6 months during which they were in a good general condition (e.g. no ascites or hemorrhage symptoms).

Inclusion and exclusion criteria in derivation and validation cohorts

The patients with advanced schistosomiasis from Jingzhou served as the model derivation cohort. Approximately 50% of them were randomly selected as the internal validation cohort. The external validation cohort consisted of patients from Huangshi over the same time period.

The patients with advanced schistosomiasis in the derivation cohort were from Jingzhou that covers 9 counties. The inclusion criteria were as follows: (1) diagnosed as advanced schistosomiasis; (2) patients with a long-term repeated history of infected water exposure or a definitive treatment history; (3) patients hospitalized to receive surgical or medical treatment. We excluded the patients with incomplete information, including lack of survival outcome, information on clinical characteristics, or laboratory test data. The number of patients with dwarfism and colonic tumoroid proliferation was too small to have any impact on the final analysis and were excluded in the analyses. The inclusion and exclusion criteria in external validation were the same as the derivation cohort.

Using the data from the derivation cohort, we sought to identify risk predictors of 2-year mortality after discharge through univariate and multivariate Cox proportional hazards models. The performance of the models was evaluated in the internal and external cohorts.

Candidate predictor variables

We chose 16 baseline candidate variables based on clinical practice and literature review. Candidate variables should be easily obtained in clinical settings and thus served as convenient predictors to be translated into future studies [14]. These variables included: age, gender, nourishment status, history of splenectomy, other diseases (cardiovascular diseases, digestive diseases and others), clinical classification, course of disease, frequency of ascites, serum total bilirubin (TBil), direct bilirubin (DBil), alanine aminotransferase (ALT), aspartate aminotransferase (AST), alkaline phosphatase (ALP), albumin (ALB), Hepatitis B surface antigen (HBsAg) and alpha fetoprotein (AFP).

Patients received routine laboratory tests at admission. Serum samples were collected and clotted at room temperature, then centrifuged at 3500 r/min for 10 min, which could be used to estimate the levels of serum biomarkers. The baseline information, including age, gender, history of splenectomy, other diseases, course of disease and frequencies of ascites were extracted using standard sociodemographic and epidemiological questionnaires. The nourishment status of these patients was evaluated by measuring the triceps skinfold (TSF). Normal adult TSF values were defined as 12.5 mm in males and 16.5 mm in females. Well nourishment status referred to the measured value >90% of the normal adult TSF value. General nourishment status referred to the measured value equivalent to 80% ~ 90% of the normal adult TSF value. Poor nourishment status referred to the measured value <80% of the normal adult TSF value. Clinical classification was also identified according to WS261–2006.

Model derivation

Firstly, restricted cubic splines (RCS) were applied to detect the possible nonlinear dependency of the relationship between 6 continuous serum biochemical variables (TBil, DBil, ALT, AST, ALP and ALB) and the mortality risk, using 4 knots at the 5th, 25th, 75th, and 95th percentiles of corresponding variable (Table S1, Fig. S1). There were potential threshold associations between serum albumin (ALB) and the 2-year mortality risk of advanced schistosomiasis patients after discharge (P value for nonlinear<0.001). We have categorized the serum ALB to ≤45 g/L and >45 g/L groups based on the RCS curve. The other 5 serum biochemical indices (TBil, DBil, ALT, AST and ALP) were categorized to normal and abnormal groups based on the medical reference value, respectively.

Fig. S1

Nonlinear associations between the selected 6 serum indices from univariate analysis and 2-year mortality risk were tested using Restricted cubic splines (RCS) curves. a.TBil and 2-year mortality risk;  b. DBil and 2-year mortality risk; c. ALT and 2-year mortality risk; d. AST and 2-year mortality risk; e. ALP and 2-year mortality risk; f. ALB and 2-year mortality risk.

The descriptive statistics was used to analyze the baseline information in derivation, internal validation and external validation cohorts. Univariate and multivariable Cox proportional hazards models were then used for variables selection. Each variable was first screened in the univariate model. The potential interactions between selected co-variates after univariate analysis were also examined by multiple collinearity diagnosis using variance inflation factors (VIF) test before multivariate analysis. The existence of multi-collinearity between co-variates were determined by the VIF value (VIF value > 5) and tolerances (tolerance < 0.2). Multivariate Cox proportional hazards model was applied for all the statistically significant variables in univariate analysis, using backwards elimination. Variables with statistically significant differences in the multivariate Cox proportional hazard model (log-rank test, P < .05) were chosen to build the mortality risk prediction model. The results were presented as hazard ratios (HRs) and 95% confidence intervals (CIs).

The selected variables from the derivation cohort were included as potential covariates in multivariate logistic regression models. We then fit a series of reduced models by removing 2–3 variables at a time according to the specific clinical significance and compared the simplified models with the full multivariable model, using C statistic (equivalent to AUC value), net reclassification improvement (NRI) and integrated discrimination improvement (IDI). Reduced models were composed of variables more easily obtained during hospitalization. To further support the clinical use of this model, we developed a simple scale for measuring patients' mortality risk in form of a risk index. The performance of the risk index was determined by its accuracy in predicting patient's death outcome.

Prediction model performance

The regression coefficients from the logistic regression models in the derivation sample were fixed. We applied the fitted model to the internal and external validation cohorts. The Akaike Information Criteria (AIC) were applied to compare the overall goodness-of-fit of these models.

The discrimination abilities of the models were compared by IDI values, C statistics and Reciever operating characteristic (ROC) curves [15,16].

The IDI value reflects the percent improvement in discrimination attained by the full multivariate model over the reduced model with fewer variables.a = (mean predicted probability in patients with death outcome – mean predicted probability in patients without death outcome) in the full multivariate model. b = (mean predicted probability in patients with death outcome – mean predicted probability in patients without death outcome) in the reduced model with fewer variables.

Model calibration was assessed by the calibration slope, the calibration intercept and locally weighted scatterplot smoothing (LOESS) plots of observed predicted probabilities of the outcome. We applied the NRI values to compare the ability of models to reclassify patients into low, moderate or high-risk categories. We divided the risk thresholds for the outcome into three groups which were <10.0%, 10.0–30.0%, and 30.0% or higher according to expert opinion. Based on the predicted probabilities of the two models, the movement between risk categories was evaluated by NRI value. We defined the net reclassification improvement as the movement of events into a higher-risk category and nonevents into a lower-risk category. Similarly, we also regarded the worsening of classification as movement of nonevents into a higher-risk category and events into a lower-risk category. The model with better performance can predict higher event rates for those with the events.

The NRI value was calculated by the sum of the percent improvement for events and nonevents，using the full multivariate model versus the reduced model with fewer variables [17].a = proportion in higher-risk category based on predicted values from the full multivariate model rather than the reduced model with fewer variables. b = proportion in lower-risk category based on predicted values from the full multivariate model rather than the reduced model with fewer variables.

We also applied the continuous NRI values to compare the reclassification capacities of various models across a continuous range of risk thresholds. P value <.05 was considered statistically significant using 2-sided testing. Statistical analyses were performed using SPSS version 22.0 software (SPSS Inc., Chicago, IL, USA) and R version 3.5.2 software (The R Foundation for Statistical Computing, Vienna, Austria. http://www.r-project.org).

The R packages “PredictABEL”, “nricens”, “survival”, “survminer”, “cmprsk”, “Hmisc”, “rms”, “ggplot2”, “lattice”, “Formula”and “grid” were applied in data analyses. ROC analyses were performed using MedCalc software version 15.8 (MedCalc Software, Ostend, Belgium) by DeLong function.

Results

Characteristics of the cohorts

A total of 1487 advanced schistosomiasis patients from Jingzhou were included in the derivation cohort, 700 of which were randomly selected as the internal validation cohort. 723 patients from Huangshi were included in the external validation cohort (Fig. 1). The mean age [SD] was 62.89 [10.38] years for the derivation cohort and 62.95 [12.22] years for the external validation cohort. The females accounted for 36.3% and 43.7% of the derivation and validation cohorts, respectively. General nourishment status accounted for 70.07% and 97.79% of the derivation and validation cohorts, respectively. 35.84% (533/1487) of the patients in the derivation cohort and 50.76% (367/723) of the patients in the external validation cohort underwent splenectomy. 47.88% (712/1487) patients had no comorbidities, followed by 22.06% (328/1487) patients with digestive diseases, 11.23% (167/1487) patients with cardiovascular diseases, 18.83% (280/1487) patients with comorbidities of other systems in the derivation cohort. The corresponding proportions were 48.55%, 6.64%, 23.37% and 21.44% in the external validation cohort. Most patients (n = 1311, 88.16%) were categorized as ascites type in the derivation cohort. In the external validation cohort, the proportion of ascites type patients was 82.99%.

Fig. 1

Formation of the Derivation, Internal Validation, and External Validation Cohorts.

50.50% (751/1487) patients' course of disease was less than or equal to 4 years in the derivation cohort. The corresponding proportion was 32.23% in the external validation cohort. 65.23% (970/1487) patients' frequency of ascites was more than or equal to 5 times. The corresponding proportion was 23.24% in the external validation cohort (Table 1). 8.27% patients (123/1487) in the derivation cohort and 7.75% patients (56/723) in the external validation cohort died within 2 years after discharge.

Table 1

Baseline characteristics of the derivation, internal validation, and external validation cohorts.

		Cohort, No. (%) of patientsa
		Derivation (n = 1487)	Internal validation (n = 700)	External validation (n = 723)
Demographics and clinical characteristics
Age, mean (SD), years		62.89 (10.38)	62.58 (10.13)	62.95 (12.22)
Women		539 (36.25)	258 (26.86)	316 (43.71)
Nourishment status	Well	351 (23.60)	150 (21.43)	4 (0.55)
	General	1042 (70.07)	516 (73.71)	707 (97.79)
	Poor	94 (6.32)	34 (4.86)	12 (1.66)
History of splenectomy Yes		533 (35.84)	244 (34.86)	367 (50.76)
Other disease	None	712 (47.88)	349 (49.86)	351 (48.55)
	Cardiovascular	167 (11.23)	79 (11.29)	48 (6.64)
	Digestive	328 (22.06)	147 (21.00)	169 (23.37)
	Other	280 (18.83)	125 (17.86)	155 (21.44)
Clinical classification	Splenomegaly	176 (11.84)	81 (11.57)	123 (17.01)
	Ascites	1311(88.16)	619 (88.43)	600 (82.99)
Course of disease	≤4 years	751 (50.50)	357 (51.00)	233 (32.23)
	>4 years	736 (49.50)	343 (49.00)	490 (67.77)
Frequencies of ascites	<5 times	517 (34.77)	251 (35.86)	168 (23.24)
	≥5 times	970 (65.23)	449 (64.14)	555 (76.76)
Laboratory data
TBil (μmol/L)	Normal	1053 (70.81)	500 (71.43)	426 (58.92)
	Abnormal	434 (29.19)	200 (28.57)	297 (41.08)
DBil (μmol/L)	Normal	1112 (74.78)	527 (75.29)	542 (74.97)
	Abnormal	375 (25.21)	173 (24.71)	181 (25.03)
ALT (u/L)	Normal	1160 (78.01)	557 (79.57)	589 (81.47)
	Abnormal	327 (21.99)	143 (20.43)	134 (18.53)
AST (u/L)	Normal	900 (60.52)	439 (62.71)	502 (69.43)
	Abnormal	587 (39.48)	261 (37.29)	221 (30.57)
ALP (u/L)	Normal	1262 (84.87)	603 (86.14)	530 (73.31)
	Abnormal	225 (15.13)	97 (13.86)	193 (26.69)
ALB (g/L)	≤45	1194 (80.30)	571 (81.57)	584 (80.77)
	>45	293 (19.70)	129 (18.43)	139 (19.23)
HBsAg	Negative	1219 (81.98)	572 (81.71)	542 (74.97)
	Positive	268 (18.02)	128 (18.29)	181 (25.03)
AFP	Negative	1452 (97.65)	683 (97.57)	720 (99.59)
	Positive	35 (2.35)	17 (2.43)	3 (0.41)

Predict 2-year mortality risk in the derivation cohort

Seven variables were selected by univariate and multivariate Cox proportional hazards model analysis using backwards elimination. Older age, splenomegaly clinical classification, abnormal serum DBil, AST, ALP, positive HBsAg and AFP (Table 2) were associated with a higher 2-year mortality risk in the derivation cohort. There was no potential interactions between co-variates based on the variance inflation factor (all VIF values < 5) and tolerance(all tolerances > 0.2) (Table S2).

Table 2

Univariate and multivariate Cox proportional hazards regression analysis for variables selection.

Variables		Overall survival
		Univariate analysis			Multivariate analysis
		HR	95%CI	P	HR	95%CI	P
Age (years)		1.062	1.042–1.083	0.000	1.068	1.046–1.089	0.000
Gender	Males/Females	0.881	0.606–1.282	0.510			NI
Nourishment status	Well/General/Poor	1.066	0.756–1.504	0.715			NI
History of splenectomy	None/Yes	0.653	0.438–0.973	0.036			NS
Other disease	None/Cardiovascular/Digestive/Other	1.033	0.942–1.134	0.486			NI
Clinical classification	Splenomegaly/Ascites	1.778	1.130–2.798	0.013	1.783	1.120–2.840	0.015
Course of disease	≤4/>4	0.856	0.600–1.220	0.389			NI
Frequencies of ascites (times)	<5/≥5	0.837	0.582–1.202	0.334			NI
TBil	Normal/Abnormal	3.056	2.143–4.358	0.000			NS
DBil	Normal/Abnormal	3.674	2.577–5.237	0.000	2.319	1.580–3.404	0.000
AST	Normal/Abnormal	2.458	1.711–3.531	0.000	1.695	1.135–2.529	0.001
ALP	Normal/Abnormal	2.753	1.883–4.025	0.000	1.662	1.101–2.508	0.016
ALB	≤45/>45 g/L	0.312	0.158–0.616	0.001			NS
HBsAg	Negative/Positive	1.750	1.178–2.591	0.006	1.900	1.254–2.876	0.002
AFP	Negative/Positive	3.653	1.853–7.201	0.000	2.072	1.014–4.234	0.046

HR, hazard ratio; CI, confidence interval; NS, not significant; NI, not included.

We constructed a full multivariate model and three reduced models according to specific clinical significance. We took age and clinical classification as general demographic information. Abnormal serum DBil, AST and ALP were considered as indexes of liver function deterioration. Positive HBsAg and AFP were regarded as indexes reflecting coinfection with HBV and schistosomiasis japonica and potential risk of evolving to primary hepatic carcinoma. Therefore, model 1 included age, clinical classification, serum DBil, AST, ALP, HBsAg and AFP; model 2 included age, clinical classification, serum DBil, AST, ALP; model 3 included age, clinical classification, HBsAg and AFP; model 4 included serum DBil, AST, ALP, HBsAg and AFP.

The 7-variable model (model 1, Table 3) had the highest C statistic (0.785; 95% CI, 0.744–0.825) and lowest Akaike Information Criterion (AIC = 739.16). Discrimination capacities (based on the C statistic, ROC curves and IDI) were lower in the reduced models with fewer variables (Fig. 2). The lowest C statistic was observed in model 3 (0.713; 95% CI, 0.666–0.760) and model 4 (0.713; 95% CI, 0.663–0.763), which was significantly lower than model 1 (both P < 0.001).

Table 3

Death events predictors of advanced schistosomiasis and performance of models in the derivation cohort.a

	Models, Odds Ratio(95% CI)
	Model 1. Age, Clinical classification, DBil, AST, ALP, HBsAg, AFP	Model 2. Age, Clinical classification, DBil, AST, ALP	Model 3. Age, Clinical classification, HBsAg, AFP	Model 4. DBil, AST, ALP, HBsAg, AFP
Predictors
Age, per year increase	1.075 (1.052–1.099)	1.067 (1.045–1.091)	1.077 (1.054–1.101)
Clinical classification
Ascites	1 [Reference]	1 [Reference]	1 [Reference]
Splenomegaly	1.863 (1.097–3.165)	1.793 (1.063–3.025)	2.100 (1.266–3.482)
DBil
Normal	1 [Reference]	1 [Reference]		1 [Reference]
Abnormal	2.580 (1.703–3.907)	2.809 (1.870–4.220)		2.967 (1.990–4.422)
AST
Normal	1 [Reference]	1 [Reference]		1 [Reference]
Abnormal	1.875 (1.221–2.880)	1.930 (1.263–2.949)		1.630 (1.076–2.470)
ALP
Normal	1 [Reference]	1 [Reference]		1 [Reference]
Abnormal	1.832 (1.149–2.920)	1.826 (1.155–2.888)		1.959 (1.250–3.070)
HBsAg
Negative	1 [Reference]		1 [Reference]	1 [Reference]
Positive	2.169 (1.359–3.463)		2.361 (1.506–3.702)	1.640 (1.050–2.563)
AFP
Negative	1 [Reference]		1 [Reference]	1 [Reference]
Positive	2.246 (1.044–5.366)		3.741 (1.634–8.564)	1.939 (0.823–4.571)
Model Performance Measures
AIC	739.16	749.85	789.38	783.29
C statistic	0.785 (0.744–0.825)	0.768 (0.726–0.809)	0.713 (0.666–0.760)	0.713 (0.663–0.763)
Difference	1 [Reference]b	0.161 (−0.003–0.035)	0.071 (0.0292–0.113)	0.0704 (0.0355–0.105)
P value		0.096	<0.001	<0.001
IDI (95%CI), %		0.0158 (0.0043–0.0272)	0.0533 (0.0344–0.0722)	0.041 (0.0232–0.0588)
P value		0.007	0.00	<0.001

Of the 1487 patients in the derivation cohort, 123 patients had death events.

Models 2 through 4 were each compared with model 1. For model comparisons, Integrated Discrimination Improvement (IDI) values and differences in a C statistic > 0 indicate better performance for the full model 1 than for the reduced models.

Fig. 2

Reclassification Performances of Model 1–4 in the Different Cohorts.

Model 1 included age, clinical classification, DBil, AST, ALP, HBsAg and AFP.

Model 2 included age, clinical classification, DBil, AST, ALP.

Model 3 included age, clinical classification, HBsAg and AFP.

Model 4 included DBil, AST, ALP, HBsAg and AFP.

a. ROC curves of the derivation cohort and AUC values (Model 1, 0.785; Model 2, 0.768; Model 3, 0.713; Model 4, 0.713).

b. ROC curves of the internal validation cohort and AUC values (Model 1, 0.763; Model 2, 0.745; Model 3, 0.719; Model 4, 0.669).

c. ROC curves of the external validation cohort and AUC values (Model 1, 0.717; Model 2, 0.717; Model 3, 0.609; Model 4, 0.684).

Predict 2-year mortality risk in the internal validation cohort

In the internal validation cohort, the 7-variable model also had a higher C statistic (0.763; 95% CI, 0.696–0.83) than the reduced models with fewer variables. The ROC curves, IDI, continuous NRI and categorical NRI favored the 7-variable model compared with the reduced models. For example, significantly improved classification into risk categories was observed (Table 3, Fig. 2) in the 7-variable model 1 compared to model 2 (Continuous NRI, 28.25%; 95% CI, 2.73–53.80%; P = .03), model 3 (Continuous NRI, 56.70%; 95% CI, 30.0–83.40%; P < 0.001. Categorical NRI, 18.54%; 95% CI, 4.34–32.56%; P = .01) and model 4 (Continuous NRI, 52.70%; 95% CI, 25.80–79.60%; P < 0.001. Categorical NRI, 23.13%; 95% CI, 9.20–37.10%; P = .0011). The IDI index also obtained improvement in the model 1 (all P value < 0.05) compared to model 2–4.

The risk index (Fig. 3) was developed from the 7-variable model. Fig. 4A demonstrated observed versus predicted risks based on the risk index.

Fig. 3

Risk Index for Patients with Advanced Schistosomiasis after discharge.

a. Point values for each variable (A patient's total risk score can be obtained by summing points assigned to values of each variable, which can determine his/her corresponding predicted 2-year mortality risk).

b. Predicted risk of 2-year mortality after discharge.

Fig. 4

Predicted vs. Observed Probability of 2-year Mortality Risk based on the Risk Index in the Internal and External validation Cohorts.

Internal validation cohort (n = 700) and external validation cohort (n = 723).

Error bars indicate 95% confidence intervals.

The risk scores observed in both cohorts ranged from a minimum value of 1 to maximum value of 9.

Predict 2-year mortality risk in the external validation cohort

In the external validation cohort, the 7-variable model 1 (0.717; 95% CI, 0.646–0.788) and 5-variable model 2 (0.717; 95% CI, 0.647–0.787) also had a higher C statistic than model 3 and model 4. The ROC curves, IDI, continuous NRI and categorical NRI favored the 7-variable model compared with the reduced models. For example, significantly improved classification into risk categories was observed (Table 3, Fig. 2) in the 7-variable model 1 compared to model 2 (Categorical NRI, 1.35%; 95% CI, 0.38–2.32%; P = .0064), model 3 (Continuous NRI, 53.90%; 95% CI, 28.0–79.80%; P < 0.001. Categorical NRI, 17.32%; 95% CI, 0.23–34.42%; P = .047) and model 4 (Continuous NRI, 27.40%; 95% CI, 0.22–54.60%; P = .0482. Categorical NRI, 11.46%; 95% CI, 1.68–21.25%; P = .022). The IDI value also obtained improvement in model 1 (all P value < 0.05) compared to model 2–4. (Table 4)

Table 4

Predictive Performance of Models for death events in Internal and External Validation Cohorts.a

	Models, Measures of Predictive Performance
	Model 1. Age, Clinical classification, DBil, AST, ALP, HBsAg, AFP		Model 2. Age, Clinical classification, DBil, AST, ALP		Model 3. Age, Clinical classification, HBsAg, AFP		Model 4. DBil, AST, ALP, HBsAg, AFP
	Internal validation	External validation	Internal validation	External validation	Internal validation	External validation	Internal validation	External validation
Calibration intercept	0.233	−0.396	−0.312	−0.83	0.027	−0.81	−2.32	−1.11
P value	0.98	0.25	0.39	0.14	0.53	0.067	0.03	0.062
Calibration slope	0.962	1.07	1.05	1.13	0.996	1.11	1.29	1.15
P value	0.97	0.13	0.23	0.059	0.46	0.022	0.013	0.022
C statistic (95%CI)	0.763 (0.696–0.83)	0.717 (0.646–0.788)	0.745 (0.677–0.812)	0.717 (0.647–0.787)	0.719 (0.650–0.788)	0.609 (0.538–0.680)	0.669 (0.586–0.752)	0.684 (0.611–0.757)
Difference in C statistic (95%CI)	1[Reference]b		0.0184 (−0.015–0.05)	0.0005 (−0.0086–0.0095)	0.0437 (−0.003–0.0905)	0.108 (0.031–0.185)	0.094 (0.032–0.156)	0.0331 (−0.0072–0.0734)
P value			0.282	0.920	0.068	0.0062	0.003	0.107
IDI (95%CI), %			0.0256 (0.0044–0.0467)	0.0021 (0–0.0042)	0.0513 (0.027–0.076)	0.0393 (0.019–0.059)	0.064 (0.029–0.098)	0.0128 (0.0034–0.0222)
P value			0.018	0.048	<0.001	<0.001	<0.001	0.0076
NRI (95%CI), %
Continuous			0.2825 (0.0273–0.538)	0.0842 (−0.1408–0.3091)	0.567 (0.300–0.834)	0.539 (0.28–0.798)	0.527 (0.258–0.796)	0.274 (0.0022–0.546)
P value			0.03	0.46	<0.001	<0.001	<0.001	0.0482
Categoricalc			0.1153 (−0.0071–0.2277)	0.0135 (0.0038–0.0232)	0.1854 (0.0434–0.3256)	0.1732 (0.0023–0.3442)	0.2313 (0.092–0.371)	0.1146 (0.0168–0.2125)
P value			0.0649	0.0064	0.01	0.047	0.0011	0.022
Events, No. (%)d			6 (10.9)	≤5 (≤8.93)	11 (20.0)	≤5 (≤8.93)	14 (25.45)	≤5 (≤8.93)
Nonevents, No. (%)			≤5 (≤0.78)	9 (1.35)	≤5 (≤0.78)	56 (8.40)	≤5 (≤0.78)	≤5 (≤0.75)
Overall, No. (%)			NR	NR	NR	NR	NR	NR

Abbreviations: IDI, integrated discrimination improvement; NRI, net reclassification improvement; NR, not reportable (cells associated with 5 or fewer events are empty because of institute for Clinical Evaluation Science policy);

Of the 700 patients in the internal validation cohort, 55 had death events, and 56 patients had death events of the 723 patients in the external cohort.

Models 2 through 4 were each compared with model 1. For model comparisons differences in the C statistics, IDI, and NRI, values >0 indicate better performance for model 1 than the reduced models.

Risk categories include patients with <10%, 10% ~30%, 30% or higher risk of death.

Net reclassification improvement events refer to occurrence of death, and NRI nonevents refer to no occurrence of death.

Similarly, Fig. 4B demonstrated observed versus predicted risks based on the risk index.

Discussion

The AUC values of model 3–4 were significantly less than model 1 in derivation cohort. The AUC value of model 1 was not significantly higher than model 2 in derivation cohort. However, significantly improved classification abilities were observed in model 1 compared to model 2 in internal and external validation cohort. The IDI value, which represented the improvement of model's overall performance, had also shown improvement in model 1 (all P values < 0.05) when compared to model 2–4 in internal and external cohorts. Furthermore, the selected 7 variables can be obtained readily in routine clinical practice with low economic cost. Therefore, we recommended model 1 with age, clinical classification, baseline serum DBil, AST, ALP, HBsAg and AFP at admission as the optimal 2-year mortality risk prediction model for patients with advanced schistosomiasis after discharge.

The predictor variables incorporated in this model and the risk index suggest a new plausible tool for clinicians during the discharge planning process to screen advanced schistosomiasis patients at high mortality risk. Such risk stratification tool can help guide follow-up, aid prognostic evaluation and inform resource allocation, since advanced schistosomiasis disease carries a high risk of complications and leads to high resource requirements to provide medical care. The predictor variables in the risk models have been studied before. Similar to previous studies, the highest schistosomiasis-related mortality was observed in older age groups [18]. This can be explained by the chronic nature of the disease, evolving into severe clinical forms, as well as more frequent chronic co-morbidities in the elderly, may possibly increase the mortality risk [19]. With regard to clinical classification, splenomegaly type patients have complex disease conditions and many complications after splenectomy, especially combined with hepatic encephalopathy and rebleeding, such as esophageal varices bleeding, which may lead to higher mortality risk than ascites type patients [20,21].

Previous studies also suggested that HBV infection exacerbates liver damage due to schistosomiasis, for coinfection with HBV and S. japonicum is associated with accelerated deterioration in hepatic function, which can lead to more severe fibrosis and inflammatory activity in the liver [22]. In the advanced stage of schistosomiasis, the fibrosis and serious fibro-obstructive pathology lead to portal hypertension, ascites, hepatosplenomegaly and eventually, fatal hematemesis.

Furthermore, a portion of hepatitis B patients will develop chronic carriers, including cirrhosis and primary hepatic carcinoma, one of the most serious diseases threatening human health due to its high mortality. Similarly, the alpha-fetoprotein (AFP) has been also developed as a biomarker for the early diagnosis of primary liver cancer [23]. The HBsAg and AFP indices can be integrated into the mortality prediction model to express the combined risk of primary hepatic carcinoma.

Bilirubin has previously been proven to be a marker of liver injury and is incorporated in several prognostic scoring models, such as the model of end-stage liver disease (MELD) and the Child-Pugh (CP) [24]. In previous study, abnormal DBil was suggested to be independently associated with an increased risk of increased fibrosis indices [25]. Glucuronyl conjugation of bilirubin and biliary excretion of DBil are distinctly impaired in advanced cirrhosis [26], which was closely associated with poor prognosis of advanced schistosomiasis. Therefore, the abnormal serum DBil level can be used as a good prognostic marker for advanced schistosomiasis patients accompanied with decompensated liver cirrhosis, an apparent life-threatening event. There is a high risk of developing further fatal complications of hepatic cirrhosis, such as spontaneous bacterial peritonitis and hepatorenal syndrome (HRS) which are the most common decompensating events in patients with advanced schistosomiasis [27].

Elevated AST activity is a surrogate marker for cirrhosis because of reduced plasma clearance of AST secondary to impaired function of sinusoidal cells [28]. The aspartate aminotransferases (AST) to platelet ratio index (APRI) score is used to predict the degree of liver fibrosis directly [29]. Given that AST and PLT are frequently part of routine tests, and that APRI is easy to compute, the APRI index is a highly cost effective and readily available diagnostic tool [30]. As we all know, hepatic fibrosis caused by schistosome infection can be fatal. Of the routinely tested markers significantly associated with fibrosis, AST level was found to be a reliable and sensitive marker for differentiating significant hepatic fibrosis in patients with advanced schistosomiasis japonica in previous studies, including cases co-infected with HBV [31].

ALP is a hydrolytic enzyme that dephosphorylates and transphosphorylates molecules including nucleotides (adenosine triphosphate, adenosine biphosphate), pathogen-associated molecule patterns and danger-associated molecule patterns [32]. The serum ALP level increases under some pathological conditions including liver injury, hepatocellular carcinoma (HCC), cholangiocarcinoma and biliary cirrhosis [33]. The level of ALP was reported as an independent prognostic biomarker for patients with HCC [34] and as a variable integrated into Chinese University Prognostic Index (CUPI) system to predict survival outcome [35]. Some studies suggested that low AAPR, an index combined by ALB and ALP, may reflect a poor nutrition status, inactive immune reaction, exhausted liver function, all of which may lead to poor survival outcomes [[36], [37], [38]]. Moreover, ALP is an easy-to-get laboratory test variable for assessing liver function in clinical practice [37]. This work has integrated serum DBil, AST and ALP variables into multivariate risk models to reveal the deterioration of liver function. The above-mentioned 7 variables can be readily obtained and therefore integrated into mortality risk prediction models to estimate 2-year individualized mortality risk of patients with advanced schistosomiasis after discharge.

The predictive performance of 2-year mortality risk prediction models needs to be validated internally and externally. Both internal (C-statistic, 0.763; 95% CI, 0.696–0.830) and external validation results (C statistic of model 1, 0.717; 95% CI, 0.646–0.788) indicated that the models were likely not overfit and performed well. We identified the mortality risk models as useful clinical tool for decision making particularly when the C-statistic is higher than 0.70 [39]. Furthermore, the NRI of the full model compared with simpler models demonstrates that the model 1 could improve the accuracy of decision making compared with model 2–4 [15,16].

Strengths of this study include that the model was developed in a large, population-representative cohort. Since the Chinese government has embarked an effort to manage these advanced schistosomiasis cases independently and those registered in the Advanced Schistosomiasis Cases Management System receive a RMB 5000 subsidy yearly per capita for therapy, including anthelmintic therapy, anti-fibrosis treatment, diuresis, and hormonotherapy, the patients lost to follow-up were rare [40]. The predictor variables in these risk models consist of easily accessible patients' demographics and laboratory test information, which could allow them to be implemented into clinical practice. The model was validated in a distinct, external cohort and had shown good predictive performance.

Our study also has several limitations. First, candidate variables were identified from secondary analysis of data, which does not include all potential risk factors, such as blood pressure, family nursing support and details of surgical or medical treatments. However, this study has included the most common and representative variables which were reported in other related literatures. Second, the participants did not have serum markers of liver fibrosis measured, such as Hyaluronic acid (HA), Laminin (LN), Collagen IV (CIV) and Procollagen III (PCIII), which may influence the performance of the models [41,42]. Likewise, the pathology evidence of advanced schistosomiasis can be directly obtained from ultrasound examination of the liver and spleen [43]. Unfortunately, we failed to collect the complete information about the patients' ultrasonography examination results. However, models without serum markers of liver fibrosis and ultrasonography examination results of the liver and spleen also performed well, suggesting that the accurate risk prediction of the survival outcome could still be achieved without these indices. Third, models were derived and validated in cohorts from Hubei Province. The generalizability to patients of other regions needs further research.

This study may have implications for clinicians, patients and policy makers. Low rates of follow-up were demonstrated in these patients possibly because rural doctors lack the awareness of screening, or lack of health resources in these isolated rural villages. Hence, providing continuity of care between the hospital and community was inadequate in common clinical practice. The risk prediction model provides an accurate but simple method that can stratify patients into clinically meaningful risk groups at a time of hospital discharge and facilitate a further management in the local community and family.

In conclusion, we have identified the clinical predictors for 2-year mortality risk of patients with advanced schistosomiasis after discharge. We have also developed a prediction model that enables clinicians to estimate the 2-year mortality risk based on these variables. The prediction model has a good predictive performance through internal and external validation. The application of the model may materially improve the patients' long-term outcome, provided it is followed by adequate treatment and follow-up.

The following are the supplementary data related to this article.

Table S1

Table S2

Funding sources

This work was financially supported by the National Key R&D Program of China [grant number 2017YFC1310000]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Z.Z. had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Authors' contributions

Z.Z. and X. L. lead the study. G.L. performed the data analysis, implemented the methodology; X.S., W.S., J.M. collected the data; G.L. prepared the original draft; B.L., Y.Y. and S.W. helped to perfect the figures. S.H., L.L. and Z.Z. reviewed and edited the final manuscript.

Declaration of Competing Interest

The authors declare that they have no competing interests.