Validation of the IMPROVE-DD risk assessment model for venous thromboembolism among hospitalized patients with COVID-19.

BACKGROUND: Antithrombotic guidance statements for hospitalized patients with coronavirus disease 2019 (COVID-19) suggest a universal thromboprophylactic strategy with potential to escalate doses in high-risk patients. To date, no clear approach exists to discriminate patients at high risk for venous thromboembolism (VTE).
OBJECTIVES: The objective of this study is to externally validate the IMPROVE-DD risk assessment model (RAM) for VTE in a large cohort of hospitalized patients with COVID-19 within a multihospital health system.
METHODS: This retrospective cohort study evaluated the IMPROVE-DD RAM on adult inpatients with COVID-19 hospitalized between March 1, 2020, and April 27, 2020. Diagnosis of VTE was defined by new acute deep venous thrombosis or pulmonary embolism by Radiology Department imaging or point-of-care ultrasound. The receiver operating characteristic (ROC) curve was plotted and area under the curve (AUC) calculated. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated using standard methods.
RESULTS: A total of 9407 patients were included, with a VTE prevalence of 2.9%. The VTE rate was 0.4% for IMPROVE-DD score 0-1 (low risk), 1.3% for score 2-3 (moderate risk), and 5.3% for score ≥ 4 (high risk). Approximately 45% of the total population scored high VTE risk, while 21% scored low VTE risk. IMPROVE-DD discrimination of low versus medium/high risk showed sensitivity of 0.971, specificity of 0.218, PPV of 0.036, and NPV of 0.996. ROC AUC was 0.702.
CONCLUSIONS: The IMPROVE-DD VTE RAM demonstrated very good discrimination to identify hospitalized patients with COVID-19 as low, moderate, and high VTE risk in this large external validation study with potential to individualize thromboprophylactic strategies.

No clear approach exists to discriminate high‐risk hospitalized patients with coronavirus disease 2019 (COVID‐19) for venous thromboembolism (VTE).

Our study included a large hospitalized COVID‐19 population within a multihospital health system.

The IMPROVE‐DD VTE risk score classified approximately 45% of the population into high VTE risk.

The IMPROVE‐DD score has the potential to individualize strategies to prevent VTE in this population.

INTRODUCTION

The coronavirus disease 2019 (COVID‐19) pandemic has been associated with elevated rates of thrombotic events. The majority of events represent venous thromboembolism (VTE) and include classic macrovessel disease such as deep vein thrombosis (DVT) or pulmonary embolism (PE), as well as microvessel disease and in situ fatal thrombosis. , Although initial reports suggested VTE rates of 46% or higher in hospitalized patients – especially those with critical illness – subsequent larger US studies have shown much lower VTE rates of 1.7% to 3.6%. , , Antithrombotic guidance statements on hospitalized patients with COVID‐19 suggest a universal thromboprophylactic strategy with potential to escalate doses in high‐risk groups, though identifying and discriminating these groups remains a challenge.

The International Medical Prevention Registry on Venous Thromboembolism and D‐Dimer (IMPROVE‐DD) risk assessment model (RAM) expands upon a well‐validated RAM – the IMPROVE VTE RAM ‐ and incorporates a novel biomarker, an elevated d‐dimer (Dd). , Elevated Dd appears to be highly predictive of increased thrombotic risk and poor outcomes in patients hospitalized with COVID‐19. Our aim was to externally validate the IMPROVE‐DD RAM for VTE in a large cohort of hospitalized patients with COVID‐19.

METHODS

This retrospective cohort study included adult patients aged ≥ 18 years old with COVID‐19 diagnosed by polymerase chain reaction and hospitalized in 1 of 13 acute care hospitals across an integrated health care network in the New York metropolitan region between March 1, 2020, and April 27, 2020. Patients were excluded if they were on the obstetrics service; death or discharge had not been reached by April 30, 2020; length of stay was <8 hours; key variables were missing; or a VTE event had occurred within the first 8 hours after presentation. The study was performed with institutional review board approval and waiver of informed consent. Data were obtained from the enterprise inpatient electronic health record (Sunrise Clinical Manager, Allscripts, Chicago, IL). All data and outcomes were tracked until April 30, 2020.

VTE was defined as new acute DVT or PE events diagnosed by imaging performed by the Department of Radiology or by point‐of‐care lower extremity ultrasound and verified by manual review of the radiology reports by two attending radiologists in a routine care setting where there was no policy for screening. We collected IMPROVE‐DD variables, patient demographics, comorbidities, and treatment. We defined history of VTE and cancer (including cancer in remission as well as current cancer undergoing anticancer treatment) from International Classification of Diseases, Tenth Revision (ICD‐10) codes. Immobility was scored for all hospitalized patients. A patient was considered to have been admitted to the intensive care unit (ICU) if care included vasopressors, ventilation, or admission to a named ICU. Mechanical ventilation was designated a surrogate for lower limb paralysis. Maximum Dd was the highest Dd throughout hospitalization for patients without VTE, or the highest Dd before VTE for patients with VTE, using a cutoff of ≥ 2 times the upper limit of local laboratory normal. Thrombophilia status was unknown. We identified major comorbidities by ICD‐10 coding: coronary artery disease, chronic kidney disease or end‐stage renal disease, peripheral arterial disease or peripheral vascular disease, and cerebrovascular disease. Thromboprophylaxis was determined to be none, treatment dose, or prophylaxis dose based on the highest dose before the first diagnosed VTE or highest anticoagulant dose before discharge (deceased or alive) for patients without VTE.

The receiver operating characteristic (ROC) curve was plotted and area under the curve (AUC) calculated. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated using standard methods. The observed prevalence of VTE was used in the calculation of PPV and NPV. We also conducted a sensitivity analysis that included for none, prophylactic, and treatment dose anticoagulation in the logistic regression model. We then plotted the ROC curve for the model using IMPROVE‐DD VTE RAM alone and for the model using IMPROVE‐DD VTE RAM adjusted for anticoagulant dose. The areas under these correlated ROC curves were then compared using the nonparametric approach. All analyses were performed with SAS version 9.4 (SAS institute, Cary, NC, USA).

RESULTS

A total of 11 265 patients were considered. After 1858 exclusions, 9407 patients met study criteria. Overall prevalence of VTE was 2.9%. Patient characteristics and distribution of model VTE risk factors are listed in Table 1. The VTE rate by IMPROVE ‐DD score was 0.4% for a score of 0‐1 (low risk), 1.3% for a score of 2‐3 (moderate VTE risk), and 5.3% for a score of ≥ 4 (high VTE risk) (Table 2). Approximately 45% of the total population was identified as high VTE risk, while approximately 21% was identified as low risk. Across the three groups of low, moderate, and high VTE risk as identified by the model, no anticoagulants were given in 15.9%, 11.9%, and 6.8% of patients, respectively; prophylactic‐dose anticoagulants were given in 81.3%, 77.7%, and 61.2% of patients, respectively; and treatment‐dose anticoagulants given in 2.9%, 10.4%, and 32.0% of patients, respectively. The AUC of the ROC was 0.702. A sensitivity analysis that adjusted for none, prophylactic, and treatment dose anticoagulation found that the AUC of the ROC for the model changed to 0.715 (P < .042), a difference that is not clinically meaningful. Discriminating low versus medium/high VTE risk showed a sensitivity of 0.971, specificity of 0.218, PPV of 0.036, and NPV of 0.996.

TABLE 1

Patient characteristics and VTE risk factors

Patient characteristics	N (%)
All	9407 (100)
IMPROVE‐DD variables (points)
Prior episode of VTE (3)	247 (2.6)
Thrombophilia (2) a	NA
Paralysis of the lower extremity during hospitalization (2) b	1557 (16.6)
Cancer (2)	727 (7.7)
Max d‐dimer > 2× ULN (2) c	4364 (46.4)
Immobilization for at least 7 days (1)	9407 (100)
ICU stay (1)	2203 (23.4)
Age > 60 (1)	5785 (61.5%)
Patient characteristics
BMI
Unknown	2029 (21.6)
≤35	6154 (65.4
>35	1224 (13.0)
Male	5580 (59.3)
Age
18‐59	3407 (36.2)
60‐75	3365 (35.8)
75+	2635 (28.0)
Comorbidity
Coronary artery disease	1200 (12.8)
Heart failure	773 (8.2)
Peripheral artery/vascular disease	379 (4.0)
Cerebrovascular disease	551 (5.9)
ESRD or CKD	783 (8.3)
Treatment
Hospital anticoagulation
None	979 (10.4)
Prophylaxis dose	6675 (71.0)
Treatment dose	1753 (18.6)
Home or hospital antiplatelet
None	4531 (48.2)
Present	2804 (29.8%)
NA	2072 (22.0%)

Abbreviations: BMI, body mass index; CKD, chronic kidney disease; ESRD, end‐stage renal disease; ICU, intensive care unit; ULN, upper limit of normal; VTE, venous thromboembolism.

Unknown for all patients.

On vent used as surrogate.

d‐dimer not done on 2566 subjects.

TABLE 2

Observed VTE events of IMPROVE‐DD VTE RAM in Hospitalized Patients COVID‐19 Based on Score Thresholds

Patient characteristics	VTE, n (%)	No VTE, n (%)
IMPROVE‐DD
0‐1, low risk	8 (0.4)	1988 (99.6)
2‐3, moderate risk	40 (1.3)	3093 (98.7)
4‐12, high risk	226 (5.3)	4052 (94.7)
Total	274 (2.9)	9133 (97.1)

Abbreviations: COVID‐19, coronavirus disease 2019; RAM, risk assessment model; VTE, venous thromboembolism.

DISCUSSION

Predicting VTE risk in hospitalized patients with COVID‐19 remains an important and difficult clinical issue, as the majority of thromboembolic events in this population are venous in origin. The IMPROVE‐DD VTE RAM, a modification of the well‐validated IMPROVE VTE RAM in hospitalized medically ill patients, showed very good model discrimination and excellent NPV in predicting VTE risk in a large cohort of hospitalized patients with COVID‐19. When compared to expected and observed VTE rates from previous derivation and external validation efforts of the original IMPROVE VTE RAM, the model also showed excellent calibration across the three cutoffs of low, moderate, and high VTE risk. , To our knowledge, this is the first dedicated external validation study using a VTE RAM in hospitalized patients with COVID‐19.

International guidelines are moving toward an individualized, risk‐adapted approach to thromboprophylaxis in hospitalized medically ill patients, which would include the subset of hospitalized patients with COVID‐19. Pre–COVID‐era studies in medically ill patients, including those with pneumonia and sepsis, have suggested a much lower percentage (10% to ~ 25%) of high‐VTE‐risk patients compared to our study’s findings. Although nearly 80% of hospitalized patients with COVID‐19 in our study were at moderate or high VTE risk, a sizable proportion (~21%) were at a low VTE risk of ~ 0.4%; such patients may be subject to potential harms in a universal anticoagulant thromboprophylaxis policy. Conversely, the 45% of hospitalized patients with COVID‐19 deemed at high VTE risk by our study (score of ≥ 4) may be considered for escalated dose thromboprophylaxis or therapeutic anticoagulation (as is being studied in current randomized trials) or considered for extended postdischarge thromboprohylaxis as suggested by current guidelines – including the most recent Scientific and Standardization Guidance Statement from the International Society on Thrombosis and Haemostasis. ,

The model’s AUC of 70%, sensitivity of 97%, and specificity of 22% are in line with previous validation studies of routinely used medical VTE RAMs (eg, Khorana and IMPROVE VTE scores). , Previous efforts using VTE risk scores in hospitalized patients with COVID‐19 were underpowered, used surrogate outcomes, or included ad hoc risk factors that were not included within the original derivation study. , In our study, nearly half of patients had elevated Dd, approximately 25% had an ICU stay, and >60% were above age 60. This distribution of risk factors follows previous studies predicting VTE and poor outcomes in hospitalized patients with COVID‐19, especially with elevated Dd. Study limitations included the inability to capture thrombophilia as a risk factor (although this would apply to a very small percentage of patients and not expected to alter model characteristics appreciably), as well as the reliance on surrogates to capture immobility (hospital stay) and lower limb paralysis (mechanical ventilation). Another study limitation is that maximal (and not initial) d‐dimers were used, which may affect the utility of the RAM when used during hospital admission.

In conclusion, in this large external validation study the IMPROVE‐DD VTE RAM demonstrated very good discrimination to identify hospitalized patients with COVID‐19 as low, moderate, or high VTE risk. Approximately half the patients were at high VTE risk, while 21% were low VTE risk. These results have potential to individualize VTE thromboprophylactic strategies in this complex hospitalized medically ill population.

AUTHOR CONTRIBUTIONS

ACS contributed to the study design. NK and ML contributed to data acquisition. All authors contributed to data analysis and interpretation.

RELATIONSHIP DISCLOSURE

SLC received funding support from the Association of University Radiologists GE Radiology Research Academy Fellowship (GERRAF) and Siemens Healthineers and was a consultant for Infervision 2019. ACS is a consultant for Boehringer Ingelheim, Janssen, Bayer, and Portola. All other authors declare no conflicts of interest.