Development and Validation of a Nomogram for 90-day Outcome in Patients with Hepatitis B Virus-related Acute-on-chronic Liver Failure.

Background and Aims: It is challenging to predict the 90-day outcomes of patients infected with hepatitis B virus-related acute-on-chronic liver failure (HBV-ACLF) via prevailing predictive models. This study aimed to develop an innovative model to enhance the analytical efficacy of 90-day mortality in HBV-ACLF.
Methods: In this study, 149 HBV-ACLF patients were evaluated by constructing a death risk prediction nomogram. Bootstrap resampling and an independent validation cohort comprising 31 patients from June 2019 to February 2020 were assessed for model confirmation.
Results: The nomogram was constructed by entering and identifying five factors (age, total bilirubin, prothrombin activity (PTA), lymphocyte (L)%, and monocyte (M)%. Healthy refinement was achieved from the nomogram analysis, where the area under the receiver operating characteristic curve was 0.864 for the training cohort and 0.874 was achieved for the validation cohort. There was admirable concordance between the predicted and true results in the equilibrium curve. The decision curve assessment revealed the useful clinical application of the nomogram. Conclusions: We constructed an innovative nomogram and validated it for the prediction of 90-day HBV-ACLF patient outcomes. This model might help develop optimized treatment protocol recommendations for HBV-ACLF patients.

Introduction

Over the last few decades, liver problems and manifestations have become the leading cause of death and sickness worldwide. Conferring to the Global Burden of Disease project, over 2 million deaths due to of major hepatic manifestoes, like hepatitis and liver cirrhosis, were recorded in 2010, accounting for almost 4% of deaths across the globe.1

Although scientists have successfully developed vaccines and antiviral drugs, liver-related diseases remain a substantial burden globally, due to sedentary lifestyles and poor nutrition habits. HBV-ACLF is a deadly disease, causing severe complications and often multiple-organ failure, with a mortality rate of 50–90%.2,3 In Asia, especially in China and India, HBV has posed life-threatening consequences to public health, causing chronic liver manifestations.4

Liver transplantation is the main treatment method for patients with HBV-ACLF.5 It is a sedulous manifesto, affecting the patient’s survival before and even after transplantation. However, the transplantation approach might realistically be the only successful viable therapy. Indeed, the probability of being ‘viable’ is far greater for transplantation than for sympathetic treatment; nonetheless, the procedure is relatively rare and restricted because of lack of donors as well as due to the careful selection of recipients that is necessitated. Currently, the individuals with HBV-ACLF prerequisite to liver transplant are assessed by the end-stage liver disease (MELD) score,6 although there are significant differences in definitions and diagnostic criteria between Eastern and Western patients,7 and the MELD score cannot accurately assess the prognosis of HBV-ACLF patients. It is, thus, critical to establish a more convenient and accurate model to predict the prognosis of HBV-ACLF patients.

In all models, the nomogram calculates the ratio through the graph of the statistical model, optimizes the prediction accuracy of the patient, and is more convenient and accurate than the traditional scoring system.8 To increase the accuracy and usability of prediction, we have established a nomogram model for this disease. In addition, the accuracy of the new model to predict 90-day mortality rate was compared with the MELD score.

Methods

Study design

This retrospective study included 445 individuals diagnosed with HBV-ACLF in Beijing Youan Hospital, Capital Medical University, from June 2014 and December 2018, and a training cohort (n=149) was constructed. Applying exclusion and inclusion criteria, 31 patients were included in the independent validation cohort from June 2019 to February 2020 at the Beijing Youan Hospital, Capital Medical University. The Ethical Committee of the Beijing Youan Hospital, Capital Medical University approved this study. All the protocols and procedures were in accordance with the laws of the Declaration of Helsinki.

The inclusion criteria for the patients were in accordance with the Asian Pacific Association for HBV-ACLF,9 which was detailed as follows: (a) age not lower than 16 years; (b) hepatitis B virus surface antigen (HBsAg) positivity for at least 6 months; (c) serum bilirubin ≥5 mg/dL (≥85 µmol/L) with a sudden exacerbation of liver disease; (d) international normalized ratio (INR) >1.5; and (e) hepatic encephalopathy (HE) and/or ascites within 28 days. The exclusion standards were as follows: (a) pregnancy or lactation; (b) co-infection with human immunodeficiency virus; (c) severe comorbidities, e.g., previous renal failure, carcinoma, cardiac dysfunctions, etc; (d) other hepatic disorders, like hepatitis A, C or E, autoimmune hepatitis, alcohol consumption, or hereditary liver diseases; or (e) infections that required hospitalization.

In total, 33 parameters were collected retrospectively and considered as potential risk factors. These parameters included age, sex, level of serum creatinine, serum urea nitrogen (BUN), aspartate transaminase (AST), aspartate alanine transaminase (ALT), albumin, total bilirubin (Tbil), serum sodium, serum potassium, and ammonia, prothrombin activity (PTA), INR, counts of white blood cells (WBCs), red blood cells (RBCs), and platelets (PLTs), lymphocytes (Ls), neutrophils (Ns), and monocytes (Ms), concentration of hemoglobin (HGB), time begin, hepatitis B virus surface antigen (HBsAg), HBV-DNA, and complications, such as: hepatorenal syndrome, ascites, infection, pleural effusion, cirrhosis, and HE. Among 180 patients, 34 (19%) took entecavir, and the rest took lamivudine and tenofovir antiviral therapy. The output (viability or deaths) of the individuals having HBV-ACLF was recorded. The MELD equation was statistically made applicable to analyze the score of severity: 9.57×ln(creatinine, mg/dL)+3.78×ln(bilirubin, mg/dL)+11.2×ln(INR)+6.43, in which the minimal values were forced to 1.0 for calculation purposes.10

Least-absolute shrinkage and selection operator (LASSO) regression

LASSO logistic regression has wide applications for high dimensional data and is utilized for the prediction of most sensitive characteristics in a training cohort. LASSO is also a regression analysis method that automatically deletes unnecessary covariates by adjusting the penalty coefficient. To reduce overfitting, LASSO calculates the set of collinearity variables in the regression, which are subsequently placed in a set of more relevant predictor variables.11 LASSO uses continuous shrinking to minimize the regression coefficients and reduce overfitting. To develop the model, we performed LASSO regression analysis to determine the links between clinical and laboratory features of such individuals. All candidate variables were entered into the LASSO regression analysis and the assumption of proportional hazards was confirmed. In accordance with the LASSO regression analysis, the LASSO model was constructed and a confidence interval (CI) of 95% as well as odds ratio (OR) was calculated.

Prediction model development and evaluation

The nomogram construction was made with independent predictors. To analyze the nomogram, the area under the receiver operating characteristic curve (AUROC) was calculated and its calibration was proceeded to evaluate the good-fit with the Hosmer-Lemeshow test.

Prediction model internal and external validation

To validate internally, bootstrap analysis was performed, in which samples were replenished randomly from the original data having the same sample size as found in the primary cohort and 1,000 repetitions were made, where the validation cohort was applied for the prediction model. Finally, AUROC was deliberated and a calibration curve was plotted.

Clinical application

The advantage of the prediction model for various possibilities was examined based on the decision curve to assess the nomogram’s applications. As a cutoff value, the maximum Youden’s index was curtained to analyze the sensitivity, specificity, accuracy, constructive predictive value, and destructive predictive value of the prediction model in the training and validation cohorts.

Statistical analysis

SPSS statistical software (version 25; IBM Corp., Armonk, NY, USA) and R 4.0.2 (http://www.r-project.org) were used for statistical analysis. LASSO regression findings and ORs having 95% probability were used, and then a nomogram was developed to estimate correlation. Its enactment was then evaluated by calibration and discrimination. The discriminative performance was demonstrated by the AUROC, which ranged from 0.5 (no discrimination) to 1 (perfect discrimination). The Hosmer-Lemeshow test assessed the calibration ability of predictive models.

Results

Baseline characteristics

In total, 149 patients were included in the training cohort, while 31 were included in the validation cohort. The features of the patients in the training and validation cohorts are specified in Table 1. There was no statistically significant difference between these two cohorts, namely the clinical physiognomies recorded were similar in both.

Table 1

Baseline characteristics of the study cohort

Characteristics	Training	Validation	p-value
No of patients	149	31
Age in years	45.14±12.4	47.97±10.28	<0.0001
Sex
Female	16	3	1
Male	133	28	0.501
HBeAg
Yes	81	20	0.582
No	68	11	0.829
Ascites
Yes	86	13	0.981
No	63	18	0.119
Infection
Yes	76	6	1
No	73	25	0.176
Electrolytes
Yes	13	2	1
No	136	29	0.316
HE
Yes	14	0	0.000
No	135	31	0.281
HRS
Yes	5	0	0.000
No	144	31	0.346
Hydrothorax
Yes	5	2	1
No	143	29	0.391
Entecavir
Yes	28	6	1
No	121	25	0.425
Cirrhosis
Yes	121	24	0.962
No	28	7	0.2
Time (days)	20 (10.0, 30.0)	15 (10.0, 30.0)	0.218
lgHBV DNA	4.96 (3.3, 6.3)	4.27 (3.2, 5.7)	0.643
HBsAg	3,100 (414.4, 6,505.5)	250.0 (154.1, 5,442.0)	0.071
ALT (U/L)	307.0 (121.7, 561.0)	331.0 (202.8, 486.0)	0.639
AST (U/L)	218.0 (157.8, 507.1)	231.0 (140.0, 512.5)	0.995
Tbil (μmol/L)	336.8 (249.9, 471.7)	283.7 (212.3, 342.5)	0.018
ALB (g/L)	31.04±4.03	30.38±3.18	0.121
BUN	4.04 (3.2, 6.2)	4.00 (3.4, 5.3)	0.890
Cr	65.0 (55.6, 78.6)	71.0 (62.0, 85.0)	0.205
NH3	85.13±41.89	63.53±35.44	0.837
TG	0.83 (0.55, 1.36)	0.83 (0.54, 1.41)	0.753
CHO	2.21 (1.8, 2.8)	1.96 (1.7, 2.4)	0.182
WBC	6.54 (4.9, 8.9)	5.45 (4.6, 7.9)	0.181
L%	19.2 (14.3, 26.4)	20 (14.8, 28.1)	0.414
M%	9.31±3.8	10.89±3.64	0.692
N%	67.3 (60.6, 76.7)	61 (5.7, 70.3)	0.006
PTA	35.63±9.53	36.84±7.0	0.269
INR	1.97 (1.7, 2.3)	1.79 (1.6, 2.0)	0.035
RBC	3.89±0.86	3.85±0.79	0.747
HGB	126 (109.0, 139.5)	131 (113.0, 142.0)	0.637
PLT	102 (66.5, 129.0)	88 (60.4, 146.0)	0.789

ALT, alanine transaminase; AST, aspartate transaminase; BUN, urea nitrogen; CHO, cholesterol ; HBsAg, hepatitis B virus surface antigen; HBV, hepatitis B virus; HE, hepatic encephalopathy; HGB, hemoglobin; HRS, hepatorenal syndrome; INR, international normalized ratio; L, lymphocyte; M, monocyte; N, neutrophil; NH3, blood ammonia; PLT, platelet; PTA, prothrombin activity; RBC, red blood cell; TBIL, total bilirubin; TG , triglyceride; WBC, white blood cell.

Using LASSO analysis to screen out the predictors of 90-day mortality

Figure 1A and B show the results of the 33 variables involved in the LASSO regression and the consistent coefficients for diverse values of the drawback parameters in the training cohort. All 33 variables remained in the model. Figure 1A shows that five variables (age, Tbil, PTA, L, and M) remained in the model for the longest time, while the other variables quickly approached zero (Fig. 1B).

Fig. 1

Results of the 33 variables in the LASSO regression.

(A) Features’ selection using LASSO regularization. LASSO coefficient profiles (y-axis) of the 33 features. The lower x-axis indicates the log (λ). The top x-axis indicates the average number of predictors. (B) Identification of the optimal penalization coefficient (λ) in the LASSO model performed via 3-fold cross-validation based on minimum criteria. LASSO, Least-absolute shrinkage and selection operator.

Table 2 shows the β-coefficient, OR and 95% CI of the LASSO regression equation in the training cohort. The results showed that M, Tbil and age were positively correlated with the HBV-ACLF 90-day mortality, and that PTA and L were negatively correlated with the HBV-ACLF 90-day mortality.

Table 2

Multivariable predictors of mortality for the training cohort

Variable	β-coefficient	OR (95% CI)	p-value
Tbil	0.006	1.006 (1.003, 1.009)	0.000
PTA	−0.107	0.899 (0.850, 0.950)	0.000
L%	−0.116	0.890 (0.837, 0.947)	0.000
M%	0.199	1.220 (1.066, 1.395)	0.004
Age	0.048	1.049 (1.013, 1.087)	0.008

Construction of prognostic nomogram

Grounded in the LASSO regression analysis outputs in the training cohort, a nomogram was constructed to predict the 90-day mortality rate (Fig. 2). A higher score calculated based on the sum of the assigned points of each predictor in the nomogram corresponded to a higher probability of death.

Fig. 2

Construction of prediction nomogram in the training cohort.

The value of each variable was given a score on the point scale axis. A total score could be easily calculated by adding every single score and projecting the total score to the lower total point scale. As such, the probability of death was able to be estimated. The nomogram is composed of age, L, M, PTA, and Tbil. L, lymphocyte; M, monocyte; PTA, prothrombin activity; Tbil, total bilirubin.

In the data of 1,000 bootstraps, the calibration curve of the nomogram indicated a good fit (Fig. 3A, B), and its Brier score was 0.1898. The x-axis signified the estimated viability derived from the nomogram, and the y-axis characterized the real survival. The decision curve showed that use of the nomogram for prediction provided more benefit than use of the MELD score in the training and invalidation cohorts (Fig. 3C, D).

Fig. 3

Calibration and decision curve analysis of the nomogram.

The calibration curve and decision curve in the training cohort (A, C) and validation cohort 1 (B, D). (A, B) The x-axis indicates the estimated viability derived from the nomogram, and the y-axis characterizes the real survival. (C, D) The y-axis measures the net benefit. The red line represents the MELD score. The blue line represents the nomogram. The grey line represents all patients. The net benefit was calculated by subtracting the proportion of all patients who are false positive from the proportion, which is a true positive.

Performance of model and clinical usefulness in the training cohort and the validation cohort

The predictive power for 90-day mortality of HBV-ACLF between the LASSO regression and MELD score was compared (Fig. 4) in the training and validation cohorts. The performance of the LASSO regression in the training cohort was high, with an AUROC of 0.864 (0.789–0.915); the MELD score had an AUROC of 0.723, which was significantly lower (p=0.0008) than in the training and the validation cohorts (Table 3). The calibration curve showed good agreement between prediction and observation (Fig. 3A, B) and provided superior net benefit in the decision curve analysis(DCA)(Fig. 3C, D).

Fig. 4

ROC curves in the training cohort (P1) and validation cohort (P2).

The ROC curve is used to distinguish the validation (P2) and MELD score, and the training (P1) and MELD score, respectively. MELD, model for end-stage liver disease.

Table 3

Predictive value of mortality for the training and validation cohorts

Model	AUROC	95% CI	p-value	Youden’s index	Sensitivity, %	Specificity, %
P1	0.864	0.798–0.915	0.0008	0.5847	87.50	70.97
P2	0.874	0.705–0.965	0.0962	0.6429	64.29	100.0
MELD	0.723	0.644–0.793		0.4382	75.0	68.82

P1, training; P2, validation. MELD, model for end-stage liver disease.

In the validation cohort, the Youden’s index was 0.6429, sensitivity was 64.29%, and specificity was 100%. While in the training cohort, the Youden’s index was 0.5847, sensitivity was 87.5%, and specificity was 70.97%. The Youden’s index, sensitivity and specificity of the training and validation cohorts are listed in Table 3.

Discussion

HBV-ACLF is the most common cases among the hepatic manifesto in the Asia Pacific region.12 ACLF is broadly defined by the Asian Pacific association for the study of the liver (APASL) ACLF research consortium (AARC) and the European Association for the Study of Liver-Chronic Liver Failure (EASL-CLIF). Being the pioneer in the CANONIC study, ACLF was defined as “an acute corrosion of pre-existing chronic hepatic disorders, mostly correlated with precipitating event having increased mortality at 12 weeks because of multisystem organs failure”.13 As reported, ACLF is a manifestation of jaundice (≥85 mmol/L) and coagulopathy (INR ≥1.5 or prothrombin activity <40%). The complications occur during a 28-day period, perusant to and/or HE in those individuals who are declared/undeclared with chronic hepatic disorders or cirrhosis; they are also correlated with high 28-day mortality, where homogeneity is a major difference between the two declarations of ACLF. The CLIF declaration primarily focuses on evaluating mortality at 90-days, while the APASL demonstration starts evaluation at 28 days;14 in this study, the CLIF definition was considered for the assessments, like APASL.

Due to the rapid progress of individuals with hepatic failure, accurate analysis of the 90-day mortality of HBV-ACLF patients is dignifying for therapeutic strategies. Although the MELD score has been verified for its utility in assessment for the allocation of donor livers, it is still not an ideal clinical evaluation model for HBV-ACLF patients.15 To disclose this problem, Li et al.16 anticipated an extrapolative model that was specially constructed for patients with HBV-ACLF. This was declared by EASL-CLIF, but individuals meeting the APASL criterion had not been enrolled and thus a novel prognostic nomogram model was constructed.

The current study focused on the construction and validation of a nomogram for patients with HBV-ACLF. We used 33 candidate variables in the training cohort, reduced them to five predictor variables using the LASSO regression method, and established a nomogram. LASSO regression is suitable for analyzing clinical factors and avoiding overfitting. It has been demonstrated that the nomogram is more accurate than traditional models in predicting patient prognosis.17 Our research established a new LASSO model, including age, Tbil, L, M and PTA as a prognostic factor of 90-day mortality. The variables screened by LASSO regression were used to establish a nomogram to predict the prognosis of ACLF patients at 90 days (Fig. 2). Each subtype of these variables was scored on a score-scale. By summarizing the points allocated to every variable, a straight line could be constructed to demonstrate the estimated survival probability time point for each patient. The nomogram scaling could then be utilized to predict survival more accurately. To determine the prediction accuracy of the nomogram, the calibration curve and decision curve (Fig. 3) were built.

A nomogram is a model of prediction schematically subliming numerical clinical events. Here, a nomogram model was constructed, comprised of age, L, M, PTA, and Tbil, that might affirm the prognosticate mortality for individuals with ACLF. MELD scoring is not sufficient to disclose a systemic inflammatory response; although, it can predict the severity of hepatic manifestations. ACLF includes the presentation of an inflammatory response, where concentrations of Ls and Ms are cost-effective markers of inflammation, and both can be applied to the analysis of non-alcoholic fatty liver manifestations, cirrhosis and transplantation outcomes.18 The mortality of individuals with ACLF might result from high levels of Ls and Ms, and both have been shown to be associated with different interleukins (interleukin [IL]6, IL-7, IL-8, IL-12, and IL-17) and tumor necrosis factor (TNF)-alpha.

Additionally, our study revealed that our constructed nomogram design was more significant in predicting 90-day mortality than the MELD score. Our designed nomogram showed excellent discrimination and calibration in primary and validation cohorts. As demonstrated previously,19 the mortality in individuals with age of the ACLF patient was curtained. Furthermore, overall increasing age among the general population may be a reason for a worse prognosis. As shown in the calibration chart (Fig. 3), the 90-day prognostic nomogram we have established is valid, reliable and generalizable. According to the nomogram and MELD AUROC scores, it is suggested that the nomogram can provide better accuracy, and the prediction of 90-day mortality is higher than with the MELD score.

In addition, our research has some advantages, such as using LASSO regression to analyze clinical factors, which is more accurate than previous traditional models. Using the new model with HBV-ACLF patients, we were able to develop a risk stratification that has potential for more accuracy and usability than the traditional scoring system. However, there are some limitations to this study. First, this analysis was based on data from a single institution and patients with liver cirrhosis accounted for 80% of the total population. Whether our nomogram applies to patients with chronic hepatitis needs further verification. Second, future research is needed to further confirm the reliability of the nomogram. Third, the predicted mortality of HBV-ACLF previously reported includes HBeAg serum status,20 intrahepatic damage caused by HBV outbreak,21 and organ failure, especially renal failure; since our sample size was relatively limited, the selected factors failed to include these aspects. It is critical to expand the sample size and perform multi-center verification to further improve the accuracy. In addition, our model includes age, bilirubin, PTA, Ls and Ms. This is different from the previously established scoring system, especially considering the Ls and Ms in our model. This may indicate that systemic inflammation is involved in the pathogenesis of ACLF, but Ls and Ms are non-specific inflammatory markers.18 In future research, we need to further expand the research data to verify it. The AARC scoring system, with a wide scoring range, is superior to the MELD score for predicting short-term mortality. It can be used for early intervention and improve the prognosis of ACLF patients through APASL standards. In our future prospective studies, we will make further comparisons with the AARC score.22,23

In conclusion, the nomogram prediction model based on target parameters may play a key role in clinical application for 90-day prognosis prediction for patients with HBV-ACLF.