A Prediction Model Based on Blood Biomarker for Mortality Risk in Patients with Acute Venous Thromboembolism.

Background: Most studies to date have focused on predicting the risk of venous thromboembolism (VTE), but prediction models about mortality risk in VTE are rarely reported. We sought to develop and validate a multivariable model to predict the all-cause mortality risk in patients with acute VTE in emergency settings.
Methods: A total of 700 patients were included from Qilu Hospital of Shandong University and were randomly assigned into training set (n=490) and validation set (n=210) in an 7:3 ratio. Multivariate logistics regression analysis was performed to identify independent variables and develop a prediction model, which was validated internally using bootstrap method. The discrimination, calibration and clinical utility were evaluated by receiver operating characteristic curve (ROC) analysis, Hosmer-Lemeshow (HL) test, Kaplan-meier (KM) analysis and decision curve analysis (DCA).
Results: There were 52 patients (10.6%) dying and 437 (89.4%) surviving in training set. Age (odds ratio [OR]: 4.158, 95% confidence interval [CI]: 2.426-7.127), pulmonary embolism (OR: 1.779, 95% CI: 1.124-2.814), platelet count (OR: 0.507, 95% CI: 0.310-0.830), D-dimer (OR: 1.826, 95% CI: 1.133-2.942) and platelet/lymphocyte ratio (OR: 2.166, 95% CI: 1.259-3.727) were independent risk variables associated with all-cause mortality. The model had good predictive capability with an AUC of 0.746 (95% CI: 0.668,0.825), a sensitivity of 0.769 (95% CI: 0.607,0.889), a specificity of 0.672 (95% CI: 0.634,0.707). The validation model had an AUC of 0.739 (95% CI: 0.685,0.793), a sensitivity of 0.690 (95% CI: 0.580,0.787), a specificity of 0.693 (95% CI: 0.655,0.729). The model is well calibrated and the HL test showed a good fit (χ2=5.291, p=0.726, Nagelkerke R2=0.137). KM analysis and DCA showed a good clinical utility of the nomogram.
Conclusion: This study identified independent variables affecting all-cause mortality in patients with acute VTE, and developed a prediction model and provided a nomogram with good prediction capability and clinical utility.

Introduction

Venous thromboembolism (VTE), which includes lower extremity deep vein thrombosis (DVT) and pulmonary embolism (PE), is the third-leading cause of accidental in-hospital death.1,2 VTE can also cause post-thrombotic syndrome (PTS), which can lead to varicose veins in the lower limbs, limb edema and chronic skin ulcers.3,4 A number of risk variables associated with the onset of VTE have been identified;1,5 nevertheless, it is impossible to evaluate the prognosis of VTE based on these risk variables alone. The application of physical examination, hematological tests and imaging examination is still recommended. Furthermore, the assessment of death and PTS risk when managing patients with acute/subacute VTE in the emergency setting is a real and critical issue in clinical practice. Most studies to date have focused on predicting the risk of VTE in specific populations.6–14 However, studies of risk variables associated with all-cause mortality and PTS in VTE patients are relatively uncommon,11,15–18 and multivariable clinical prediction models in emergency setting are also rarely reported.

The D-dimer is one of the most commonly used blood markers for the diagnosis of VTE, and has high sensitivity and low specificity.19 There is growing evidence that the immune/inflammatory response plays an important role in the development of vascular disease.20–23 Representative indicators include the neutrophil/lymphocyte ratio (NLR) and the platelet/lymphocyte ratio (PLR), which have been shown to predict the onset of VTE and to be strongly associated with mortality.24–28 The systemic immune/inflammation index (SII) has been found to be associated with survival in patients with cancer.29,30 However, two recent studies have identified SII as an independent risk variable for predicting acute PE and acute VTE.31,32 We sought to investigate the association between these indicators and prognosis and PTS, and included them in the predictive model to improve their clinical utility.

In this study, we retrospectively analyzed clinical data of VTE patients with an interval time from onset to admission to our hospital of less than 30 days in the last 10 years. All-cause mortality events during the follow-up period were recorded, and PTS was evaluated using the Villalta score. The independent risk variables were identified, and a multivariable prediction model was derived and validated internally using the bootstrap method. A nomogram was also produced to provide a potential guide for clinical practice. The discrimination, calibration and clinical utility were evaluated in training set and validation set.

Patients and Methods

Study Population

This was a single-center retrospective study, and all medical records were obtained from the electronic medical record system of Qilu Hospital of Shandong University. Patients enrolled in the study were those admitted to our hospital between January 1, 2011, and December 31, 2020. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013) and was approved by the Ethics Committee of Qilu Hospital of Shandong University. The need for written informed consent was waived because this was a retrospective study based on previous medical record data and did not involve human specimens. The personal identifiers of all patients were removed. All the data were anonymized and maintained with confidentiality. The inclusion criteria were as follows: (1) definite diagnosis of lower extremity DVT on ultrasound and PE on pulmonary computed tomography angiogram; (2) interval time from onset to admission less than 30 days; (3) first onset of VTE; and (4) laboratory testing of blood samples within 24 hours of admission. Exclusion criteria were: (1) interval time from onset to admission exceeding 30 days; (2) patients with recurrent VTE; (3) in-hospital onset of VTE; and (4) patients with malignancy.

We collected the clinical data of all enrolled patients, including age, sex, smoking history, alcohol history, surgery history within 3 months, interval time from onset to admission, days of in-hospital stay, follow-up duration, blood test results, thrombus location, and comorbidities of hypertension, diabetes, coronary artery disease, neurological disease, fracture, immune rheumatic disease, and lower limb vascular disease. Blood biomarkers were obtained within 24 hours of admission, including white blood cell count, neutrophil count, lymphocyte count, platelet count, D-dimer, NLR, PLR, and SII. NLR was defined as neutrophil count (109/L)/lymphocyte count (109/L). PLR was defined as platelet count (109/L)/lymphocyte count (109/L). SII was defined as platelet count (109/L) × neutrophil count (109/L)/lymphocyte count (109/L).

The follow-up period ended on 31 December 2021, and we obtained follow-up results by telephone or via outpatient interviews. All-cause mortality was recorded by telephone follow-up. If the patient was alive, an outpatient interview was assigned to evaluate the PTS. PTS scores for deceased patients were filled by the random forest method. PTS was evaluated by using the Villalta score, where a score of 0–4 is defined as no PTS, ≥5 as PTS, 5–9 as mild PTS, 10–14 as moderate PTS, and ≥15 or combined with ulceration as severe PTS.33 The calculation of the Villalta score was carried out by two vascular surgeons. If there was disagreement, a third vascular surgeon made the final judgement.

Statistical Analysis

Statistical analysis was performed using R version 4.0.3. Continuous variables were expressed as medians (25th, 75th percentiles) and analyzed using an independent Student’s t-test or Mann–Whitney test, as appropriate. Categorical variables were expressed as number of cases and percentages (%), which were analyzed with the χ2 test or Fisher’s exact test. Blood biomarkers, such as white blood cell count, neutrophil count, lymphocyte count, platelet count, D-dimer, NLR, PLR and SII, were transformed into dichotomous variables for further analysis, based on the median value.

Univariate logistics regression analysis was performed to identify variables associated with mortality, and variables with a p-value<0.1 were analyzed by multivariate analysis. A stepwise method was used to identify independent risk variables, with the intensity of association indicated by odds ratio (OR). The 95% confidence interval (CI) was also listed. The final prediction model retained risk variables with p<0.05. During the development of model, internal validation and optimization was conducted using a bootstrap method (1000 times). A weighted score from 0 to 100 was assigned to each variable included in the final model based on the regression coefficients from the multivariate analysis. A nomogram was then produced to provide guidance for clinical practice.

ROC analysis was used to assess the discrimination of the prediction model in training set and validation set, deriving area under the curve (AUC), sensitivity, specificity. Actual and predicted incidence rates were compared using calibration plots, and calibration of the model was reflected by a fit of the calibration curve to the diagonal slope. The logistic model was also evaluated by Hosmer-Lemeshow test (HL test) to identify the match between the expected mortality events based on the model and the observed mortality events.

The total point of the nomogram for probability of mortality for each patient was calculated. Kaplan-meier analysis (Log rank test) of the survival probability based on the median value of total points was conducted. The clinical value of the model was assessed by decision curve analysis (DCA) with the threshold probability as the horizontal coordinate and the net benefit (NB) as the vertical coordinate. The blue horizontal line indicates that all patients are dead and the NB is zero, while the green diagonal line indicates that all patients survive and the inverse slope of the NB is negative; the red solid line represents the prediction model.

This study was conducted and reported in line with the Transparent Reporting of a multivariate prediction model for Individual Prediction or Diagnosis (TRIPOD) guidelines.34 And a TRIPOD checklist was also provided.

Results

There were 700 VTE patients between January 1, 2010 and December 31, 2020 enrolled in the study based on inclusion and exclusion criteria, with 313 males and 387 females at a median age of 60 years (49, 68). A total of 364 patients (52.00%) were diagnosed with DVT, while 336 (48.00%) with DVT and PE. For all 700 patients, anticoagulation was continued for at least 3 months, with 314 patients (44.85%) on rivaroxaban and 386 patients (55.15%) on warfarin. Rivaroxaban was administered at a dose of 15 mg bid for 3 weeks, followed by 20 mg qd for at least 3 months. Warfarin was administered dynamically according to the prothrombin time-international normalized ratio (PT-INR), which was required to be maintained between 2.0 to 3.0. All patients were advised to use compression stockings for at least 6 months. The mean follow-up period was 53 months (32, 81), with 84 patients (12%) dying and 616 (88%) surviving. The shortest follow-up period was 12 months and the longest follow-up period was 124 months. According to Villalta score, 416 patients (59.5%) scored 0, while 246 (35.1%) scored 1–4, 24 (3.4%) had mild PTS (scoring 5–9), 6 (0.9%) had moderate PTS (scoring 10–14) and 8 (1.1%) had severe PTS (>15). Detailed clinical baseline characteristics are presented in Table 1.

Table 1

Clinical Baseline Characteristics in Training Set and Validation Set

	Total (n=700)	Training Set (n=489)	Validation Set (n=211)	P value
Gender				0.376
Male	313 (45%)	224 (46%)	89 (42%)
Female	387 (55%)	265 (54%)	122 (58%)
Age (years)	60 (49, 68)	60 (49, 68)	57 (48, 66)	0.890
Interval time* (days)	7 (3, 10)	7 (3, 10)	7 (4, 10)	0.950
Hospitalization duration (days)	14 (10, 15)	14 (10, 15)	14 (11, 15)	0.440
Smoke	198 (28%)	143 (29%)	55 (26%)	0.392
Alcohol	181 (26%)	130 (27%)	51 (24%)	0.503
Comorbidity
Hypertension	223 (32%)	172 (35%)	51 (24%)	0.004
Diabetes mellitus	58 (8%)	40 (8%)	18 (9%)	0.877
Cerebral disease	67 (10%)	54 (11%)	13 (6%)	0.044
Coronary heart disease	121 (17%)	85 (17%)	36 (17%)	0.918
Pulmonary disease	48 (7%)	35 (7%)	13 (6%)	0.632
Fracture	59 (8%)	42 (9%)	17 (9%)	0.816
Immune rheumatic disease	42 (6%)	29 (6%)	13 (6%)	0.906
Peripheral vascular disease	65 (9%)	49 (10%)	16 (8%)	0.308
Surgery within last three months	175 (25%)	113 (23%)	62 (29%)	0.078
Venous thromboembolism				0.384
DVT	364(52%)	249 (51%)	115 (55%)
DVT+PE	336 (48%)	240 (49%)	96 (45%)
Thrombus location in limbs				0.578
Ilio-femoral vein	427 (61%)	295 (60%)	132 (63%)
Popliteal vein and calf vein only	273 (39%)	294 (40%)	79 (37%)
Main blood examination
White cell count (109/L)	7.15 (5.77, 9.15)	7.26 (5.71, 9.18)	6.95 (5.87, 9.11)	0.826
Neutrophil count (109/L)	5.09 (3.76, 6.82)	5.11 (3.71, 6.82)	5.08 (3.81, 6.82)	0.776
Lymphocyte count (109/L)	1.42 (1.13, 1.79)	1.40 (1.11, 1.80)	1.45 (1.16, 1.76)	0.978
Platelet count (109/L)	218 (174, 272)	216 (173, 273)	224 (174, 269)	0.974
D-Dimer (ug/mL)	3.01 (1.09, 6.05)	2.99 (1.08, 5.56)	3.25 (1.12, 6.53)	0.160
NLR	3.50 (2.37, 5.16)	3.54 (2.37, 5.27)	3.46 (2.37, 5.09)	0.448
PLR	150.48 (112.48, 205.61)	150 (111.42, 210.63)	151.53 (114.01, 196.15)	0.138
SII	738.96 (484.92, 1175.05)	753.37 (486.73, 1183.64)	705.50 (476.59, 1145.04)	0.293

Notes: Data are presented as counts with percentages n(%) or median with interquartile range (median [Q1, Q3]). Interval time*: days from VTE onset to admission to hospital.

Abbreviations: DVT, deep venous thrombosis; PE, pulmonary embolism; NLR, neutrophil/lymphocyte ratio; PLR, platelet/lymphocyte ratio; ROC, receiver operating characteristic curve.

All patients were randomly assigned into training set (n=490) and validation set (n=210) in an 7:3 ratio. There were 52 patients (10.6%) dying and 437 (89.4%) surviving in training set, and detailed clinical baseline characteristics are presented in Table 2. Age, PE, platelet count, D-dimer and PLR were identified as independent risk variables associated with all-cause mortality, shown on a forest plot (Figure 1A). These variables were used to develop a multivariable prediction model and a nomogram was produced. (Figure 1B) ROC analysis showed that the model had good predictive capability with an AUC of 0.746 (95% CI:0.668,0.825), a sensitivity of 0.769 (95% CI:0.607,0.889), a specificity of 0.672 (95% CI:0.634,0.707). (Figure 2A) The model in validation set had an AUC of 0.739 (95% CI:0.685,0.793), a sensitivity of 0.690 (95% CI:0.580,0.787), a specificity of 0.693 (95% CI:0.655,0.729). (Figure 2B) The calibration plots showed that the predicted and actual survival curves were close, with no significant differences (Figure 2C and D), indicating a good overall consistency of the model. HL test showed a good fit (χ2=5.291, p=0.726, Nagelkerke R2=0.137).

Table 2

Clinical Baseline Characteristics and Univariate Analyses Between Survivor and Death in Training Set

	Total (n=490)	Survivor (n=437)	Death (n=52)	P value
Gender				0.521
Male	224 (46%)	198 (45%)	26 (50%)
Female	265 (54%)	239 (55%)	26 (50%)
Age (years)	60 (49, 68)	60 (47, 67)	70 (61, 77)	<0.001
Interval time* (days)	7 (3, 10)	7 (3, 10)	7 (4, 19)	0.116
Hospitalization duration (days)	14 (10, 15)	14 (10, 15)	12 (9, 15)	0.254
Smoke	143 (29%)	129 (30%)	14 (27%)	0.697
Alcohol	130 (27%)	117 (27%)	13 (25%)	0.784
Comorbidity
Hypertension	172 (35%)	148 (34%)	24 (46%)	0.079
Diabetes mellitus	40 (8%)	30 (7%)	10 (19%)	0.002
Cerebral disease	54 (11%)	39 (9%)	15 (29%)	<0.001
Coronary heart disease	85 (17%)	69 (16%)	16 (31%)	0.007
Pulmonary disease	35 (7%)	27 (6%)	8 (15%)	0.015
Fracture	42 (9%)	35 (8%)	7 (13%)	0.185
Immune rheumatic disease	29 (6%)	27 (6%)	2 (4%)	0.501
Peripheral vascular disease	49 (10%)	46 (11%)	3 (6%)	0.280
Surgery within last three months	113 (23%)	104 (24%)	9 (17%)	0.294
Venous thromboembolism				0.189
DVT	249 (51%)	227 (52%)	22 (42%)
DVT+PE	240 (49%)	210 (48%)	30 (58%)
Thrombus location in limbs				0.312
Ilio-femoral vein	295 (60%)	267 (61%)	28 (54%)
Popliteal vein and calf vein only	194 (40%)	170 (39%)	24 (46%)
Main blood examination
White cell count (109/L)	7.26 (5.71, 9.18)	7.20 (5.67, 9.14)	7.43 (6.37, 10.04)	0.102
Neutrophil count (109/L)	5.11 (3.71, 6.82)	5.05 (3.69, 6.82)	5.60 (4.09, 7.21)	0.091
Lymphocyte count (109/L)	1.40 (1.11, 1.80)	1.43 (1.12, 1.81)	1.23 (1.01, 1.60)	0.095
Platelet count (109/L)	216 (173, 273)	216 (174, 273)	215 (167, 281)	0.871
D-Dimer (ug/mL)	2.99 (1.08, 5.56)	2.93 (1.07, 5.98)	3.02 (1.11, 5.20)	0.155
NLR	3.54 (2.37, 5.27)	3.47 (2.35, 5.07)	4.76 (2.53, 7.31)	0.040
PLR	150 (111.42, 210.63)	150 (110.30, 207.27)	152 (126.39, 225.01)	0.093
SII	753.37 (486.73, 1183.64)	730.35 (477.56, 1165.42)	923.75 (569.24, 1524.29)	0.081

Notes: Data are presented as counts with percentages n(%) or median with interquartile range (median [Q1, Q3]). Interval time*: days from VTE onset to admission to hospital. DVT, deep venous thrombosis; PE, pulmonary embolism; NLR, neutrophil/lymphocyte ratio; PLR, platelet/lymphocyte ratio; ROC, receiver operating characteristic curve.

Figure 1

(A) A forest plot of independent risk variables with OR for all-cause mortality in patients with acute VTE. (B) The nomogram of the prediction model for mortality risk.

Figure 2

(A) The ROC curve of the prediction model in training set. (B) The ROC curve of the prediction model in validation set. (C) The calibration plot in training set. (D) The calibration plot in validation set.

The total point for probability of survival in each patient was calculated, the median value of which was 200. Patients with scores ≥200 were assigned to the high-risk group, and those with scores <200 were assigned to the low-risk group. Kaplan-meier analysis showed that the survival probability of high-risk and low-risk groups was significantly different in both the training set and validation set, indicating that the prediction model was clinically useful.(Figure 3A and B) The DCA showed that the model curve is above the horizontal and dashed lines, indicating that the model provides sound clinical guidance.(Figure 3C and D) The DCA showed a better net benefit of our model for prediction of mortality risk within threshold probability ranged from 0% to 19% in training set, and 0% to 13% in validation set.

Figure 3

Kaplan-meier analysis of survival probability based on the median value of the total point in the nomogram showed a significant difference between the ≥200 and <200 groups (Log rank p<0.001) in training set (A) and validation set (B). The DCA showed a better net benefit of the prediction model for mortality risk within threshold probability ranged from 0% to 19% in training set (C), and 0% to 13% in validation set (D).

Discussion

Most previous studies in this area have focused on the prediction of VTE risk in specific populations.7–11,35 For example, McCurdy et al identified the incidence and risk variables for VTE in 2161 patients with inflammatory bowel disease (IBD) after discharge and developed a prediction model to assess VTE risk.35 The final model included age >45 years, multiple admissions, intensive care unit admissions, length of admissions >7 days and central venous catheters. The model was able to differentiate between discharges associated with and without VTE (optimism corrected statistic, 0.70; 95% CI: 0.58–0.77). The nomogram in this report also provides a useful and easy calculation of VTE risk after discharge in patients with IBD. However, emergency physicians are interested in assessing the death risk of patients with acute VTE in the emergency setting.

Only a few studies on prediction models of prognosis in VTE patients are available. For example, Huang et al identified serum albumin, tumor stage and history of surgery within 30 days as independent risk variables for 6-month survival in patients with malignancy combined with VTE who underwent inferior vena cava filter placement.11 The prediction model had an AUC of 0.815. However, the authors did not provide further parameters or calibration plots for this model, and neither internal nor external validation was conducted. A nomogram was also not available. As there is little relevant literature focusing on this topic, we developed a predictive model based on the baseline characteristics of VTE patients who required an emergency hospital admission. In comparison, our prediction model for VTE prognosis has an AUC of 0.746 with specific parameters indicating a good predictive capability. The model has also been calibrated and internally validated, and a nomogram is also available.

There are several notable differences in this study compared to previous studies on VTE prediction models. Firstly, we enrolled patients with VTE less than 30 days from onset to hospital admission, which means that patients were in the acute or subacute phase. The result therefore provides important clinical guidance for the management of patients with acute VTE. Secondly, the prediction model includes all-cause VTE, rather than VTE in a specific population, which has excellent broad applicability for clinical practice. Third, our prediction model was able to evaluate all-cause mortality in patients with VTE at the time of admission. Fourth, the immune/inflammation indicators used in the model have been widely verified to be associated with the occurrence and prognosis of VTE. However, these indicators have rarely been included in the development and validation of prediction models. In the present study, we included these indicators in the prediction model in addition to the common clinical variables. The above guarantees both the innovation of this study and the clinical utility of the prediction models.

New immune/inflammatory indicators including NLR, PLR, and SII have been reported in several studies to be closely related to the diagnosis and prognosis of vascular diseases such as VTE and ASO.36,37 In this study, we identified D-dimer, PLR, advanced age and pulmonary embolism as independent risk variables for mortality, while platelet count was protective variables. These variables are readily available in clinical practice, and therefore a nomogram consisting of these variables is clinically convenient and practical. The total points calculated from the nomogram can predict the mortality probability of VTE patients, and can therefore be applied to stratify the risk of VTE with different clinical intervention strategies. For a patient with acute VTE admitted to the emergency clinic, the emergency physician can assess the risk of death using the prediction nomogram. If the patient is identified as being at high risk of death, additional brain/cardiac/peripheral vascular examinations and screening for occult disease, as well as primary and secondary prevention, can be performed in addition to treatment for VTE.

There are also some limitations to this study. Firstly, the patient screening criterion have excluded patients with malignancy and these patients may have had a different prognosis than the present cohort. Secondly, the prediction model cannot provide guidance for patients with VTE in the chronic stage (>30 days). Thirdly, external validation and calibration using an independent cohort dataset is required for this model before it can be implemented into clinical practice.

Conclusion

This study identified independent risk variables for all-cause mortality risk in patients with acute VTE, and developed a clinical prediction model and provided a nomogram with good prediction capability and clinical utility, which may facilitate risk stratification and individualized treatment strategies. Prospective cohort studies are still required to validate and calibrate this model before clinical application.