Risk prediction of recurrent venous thrombosis; where are we now and what can we add?

BACKGROUND: Several models are available to predict recurrent venous thrombosis (VT) in patients with unprovoked first events.
OBJECTIVES: To validate these prediction models externally.
METHODS: Within the MEGA follow-up study (n = 3750), we externally validated the Vienna and DASH score. These models were validated (a) by using the original study's criteria for patients with unprovoked VT and (b) by using our own criteria for unprovoked VT. In addition, absolute recurrence risks based on individual VT risk factors were calculated.
RESULTS: The recurrence rate was 5.2 (95% CI, 4.6-5.9) per 100 patient-years in those who had a first unprovoked VT according to our definition. For the Vienna model it was 3.4 per 100 patient-years and for DASH 3.8 per 100 patient-years. The C-statistic was 0.62 for Vienna and 0.65 for DASH. The C-statistic declined to 0.58 for both Vienna and DASH when we used our own definition of "unprovoked VT." Within the provoked group a strong gradient in risk was found dependent on the presence of traditional risk factors or biomarkers in a patient.
CONCLUSIONS: The ability to distinguish patients' recurrence risks is lower than proposed in the original prediction model studies and dependent on the definition that is used for an unprovoked first event. Furthermore, our results suggest that a more-refined risk estimation is possible, also in patients with a provoked first event, who are currently all classified as low risk.

Prediction models for recurrent VT need external validation to establish clinical usefulness.

Vienna and DASH models were externally validated in the MEGA follow‐up study.

These models were moderately able to distinguish in recurrence risks.

The definition of “unprovoked” first events strongly influenced the predictive performances.

INTRODUCTION

Patients with a first venous thrombosis (VT; the composite of deep vein thrombosis [DVT] and pulmonary embolism) have a high risk of a recurrent event.1, 2 Anticoagulant therapy is effective for treating a first event as well as for preventing a recurrence. Trials have shown that 3 months of anticoagulant therapy is the minimum duration of initial treatment.3 However, it is uncertain whether anticoagulant treatment should be continued beyond this period to prevent recurrences. This uncertainty arises from the associated risk of this treatment, that is severe bleeding.4, 5 Both recurrent VT and major bleeding are serious events, with case‐fatality rates ranging between 4% and 10%.6, 7

The decision on treatment duration has strong lifelong implications, as the cumulative thrombosis and bleeding risks will become high over a person's lifetime. Nevertheless, the propensity to develop recurrent VT or bleeding differs strongly between individuals, depending on their genetic makeup and environmental circumstances.8 Attempts have been made to quantify these risks at an individual level as a basis for predicting the risk. The three best known prediction models for VT, that is, the Men continue and HERDOO2 rule, the Vienna prediction model, and the DASH score, are exclusively aimed at patients with an unprovoked first event (about 30%‐50% of the total VT population, depending on the definition of unprovoked).9, 10, 11 External validation studies showed less discriminative ability than the original studies.12, 13, 14 In the HERDOO2 study, authors confirmed that women with a first unprovoked VT and none or one of the HERDOO2 criteria could safely discontinue anticoagulants after completing short‐term treatment.15 Nevertheless, the majority of their population could not be classified in this low‐risk group (n = 2125; 76%).

The current guidelines advise to classify all patients in only two groups, with either high or low risk of VT recurrence.16, 17 This classification is based on one determinant, that is, whether the index event was provoked by a transient risk factor or whether it was unprovoked. Roughly speaking, the provoked/low‐risk group is advised to stop anticoagulant treatment and the unprovoked/high‐risk group is advised to continue if the associated bleeding risk is expected to be low to moderate.16 One problem with this approach is that the definition of (un)provoked is unclear and varies between centers and over time.18

We aimed to validate the DASH and Vienna prediction models for recurrent VT externally within the MEGA follow‐up study. Since in MEGA HemosIL D‐dimer was performed rather than VIDAS D‐dimer,19 and symptoms of posttraumatic stress were self‐reported in MEGA, we did not validate HERDOO2. Furthermore, we determined the effect of using different definitions for “unprovoked.” Last, we estimated the absolute risk of recurrence based on individual combinations of characteristics currently regarded to have the strongest predictive value.

METHODS

Patient population

Between March 1999 and August 2004, 4956 patients aged 18 to 70 years with an objectively diagnosed first DVT of the leg (n = 2887, of which n = 349 had distal DVT) or PE (n = 2069) were included in a population‐based case‐control study (MEGA study). All patients filled in an extensive questionnaire on putative risk factors for VT and blood samples were collected. For logistic reasons, blood sampling was performed for participants included up to June 2002. Details of the MEGA study have been described previously.20

These cases were further followed for recurrence (MEGA follow‐up study).21 For this, 225 of the 4956 patients did not consent. Between 2007 and 2009 the vital status of all patients was acquired from the central Dutch population register and for the patients who died, a cause of death (ICD‐10‐CM) was obtained from the national register of death certificates at the Central Bureau of Statistics. Short‐answer forms concerning recurrent VT were sent by mail between June 2008 and July 2009 and supplemented by telephone interviews. Furthermore, all patients were asked to complete a second questionnaire on the presence of risk factors for VT after their first event. This study was approved by the Medical Ethics Committee of the Leiden University Medical Center, and all participants gave written informed consent.

Recurrent venous thrombosis

Questionnaires concerning recurrent venous thrombosis were sent by mail to all survivors and consenting individuals between June 2008 and July 2009, and supplemented by telephone interviews. Additional information was acquired from the regional anticoagulation clinics and from hospitals. Deaths resulting from recurrent venous thrombosis were counted as fatal recurrent events. On the basis of hospital discharge letters, the information from the anticoagulation clinic, questionnaires filled in by the patients, and causes of death, possible recurrences were classified into certain and uncertain recurrences, following a decision rule.21 Patients with uncertain recurrent events were censored from this event onward. Censoring uncertain events decreases non‐differential misclassification, although excluding uncertain recurrent events could have resulted in a lower‐than‐expected recurrent VT rate, which may have affected the performance of the model to some extent.

Blood sampling and laboratory analyses

Approximately 3 months after discontinuation of oral anticoagulant therapy, patients were invited for collection of a blood sample. When they were still on anticoagulant therapy 1 year after their event, blood was drawn during treatment. Blood sampling was requested until June 2002, which means blood samples are available for roughly 50% of the study population. After 2002 patients were sent buccal swabs to collect DNA. Blood samples were drawn into vacuum tubes containing 0.1‐volume 0.106‐mol/L trisodium citrate and centrifuged for 10 min at 4°C, after which plasma was aliquoted, frozen, and stored at −80°C. D‐dimer was assayed using the D‐dimer HemosIL assay (Instrumentation Laboratory, Warrington, UK). The HemosIL D‐Dimer HS is an automated latex‐enhanced immunoassay performed on the ACL TOP 700CTS (Instrumentation Laboratory).

Statistical analyses

Follow‐up started at time of discontinuation of anticoagulant treatment similar to the development studies of the three prediction models. Patients diagnosed with cancer within 5 years before VT or patients with missing data on cancer were excluded, also following the original studies. All other patients with VT were included regardless of the presence or absence of risk factors. All analyses (see later discussion) did, however, make a sharp distinction between provoked and unprovoked first events as Vienna and DASH only included unprovoked events. The end of follow‐up was defined as the date of a recurrent event and, in the absence of a recurrence, the date of filling in the short questionnaire.21 If a patient did not fill in the questionnaire he or she was censored at the last date we knew the patient was recurrence‐free. This could be either the last visit to the anticoagulant clinic, the date of death or emigration, or the last moment the patient was known to be recurrence‐free from information of the MEGA case‐control study. Duration of follow‐up was calculated by starting follow‐up at the date of discontinuation of anticoagulant therapy.

(A) External validation of currently available models

We validated the Vienna model and the DASH score. Predictor variables included were D‐dimer (Vienna, DASH), age (DASH), sex (Vienna, DASH), site of index event (Vienna), and hormone therapy (DASH). Since the performance of a prediction model depends on the patient population included, and the two models used different definitions of unprovoked VT, we aimed to validate the models in two ways: (a) by using the original study's criteria for unprovoked events and (b) by using our own criteria for unprovoked VT, which were: VT without surgery, trauma, plaster cast, pregnancy or immobilization in the first 3 months before the event, prolonged travel in the first 2 months before the event, and hormone use (oral contraceptives or hormone replacement therapy) at the time of the event. For the selection of patients with an unprovoked first event according to various definitions we were dependent on variables that were available in MEGA; that is, we did not know whether patients had positive lupus anticoagulants, positive anticardiolipin antibodies, or positive antiphospholipid antibodies (exclusion criteria for Vienna and DASH, respectively). Furthermore, blood samples were available for 51% of the population as we ceased collecting halfway during the study. This means that we could select only half of the population according to the Vienna definition of unprovoked venous thrombosis since natural anticoagulant deficiencies (antithrombin, protein C, and protein S) were exclusion criteria according to this definition. For each risk assessment model, observed vs. predicted risks were plotted in a table. We assessed the discriminative performance of the three models by means of Harrell's C‐statistic. Since D‐dimer levels are a predictor variable in both models, the external validation analyses were run on 51% of the MEGA subjects who fulfilled the various criteria of unprovoked VT in the main analysis. In sensitivity analyses, all missing information on D‐dimer levels was imputed 10 times and results were pooled according to Rubin's rules.

(B) Risk stratification

To study whether further refined risk estimation is possible within the two risk groups that are currently distinguished, we estimated recurrence rates in six groups of patients, according to some well‐described risk factors for recurrence, that is: (a) women who underwent surgery within 3 months before their thrombotic event and with low levels of D‐dimer (<500 ng/mL) after discontinuation of anticoagulant treatment; (b) men who had a provoked event unrelated to surgery and high levels of D‐dimer (≥500 ng/mL); (c) women with an unprovoked event who had low levels of D‐dimer; (d) men with an unprovoked event with high D‐dimer levels; (e) men who had a provoked event unrelated to surgery and high levels of D‐dimer and high factor VIII:Ag (>200 IU/dL), and (f) men who had an unprovoked event and high levels of D‐dimer and high factor VIII:Ag. No formal power calculation was made as in all of these analyses >100 recurrent events were noted, making the statistical analyses robust. Please note that results on risk stratification should be seen as illustrative for the principle.

RESULTS

A total of 715 patients were excluded because their follow‐up ended before or at the moment of discontinuation of anticoagulant treatment. An additional 266 patients were excluded because of a cancer diagnosis or missing data on cancer, leading to a total of 3750 patients (Figure 1). Their main characteristics are provided in Table 1, overall and categorized by model. As can be observed, the age at time of first VT, percentage of women, and high D‐dimer levels differed strongly according to the definitions applied.

Figure 1

Flowchart of the MEGA follow‐up study. Of analyzed patients, there were 336 (9%) lost to follow‐up. They were analyzed until the moment they were lost to follow‐up. *Of these patients there were 52 who had a recurrent event before blood sampling

Table 1

Clinical characteristics of patients included in the MEGA follow‐up study

	MEGA follow‐upb	Viennac	DASHd
Total	3750 (100)	3750 (100)	3750 (100)
Missing for provoked/unprovoked	76 (2)	931 (25)	78 (2)
Provoked	2592 (69)	2022 (54)	1514 (40)
Unprovoked	1082 (29)	797 (21)	2158 (58)
Men	786 (73)	583 (73)	1018 (47)
Age at first event (years)	54 (19‐70)	53 (18‐70)	49 (18‐70)
Hormone therapya (% women)	0 (0)	0 (0)	754 (66)
High D‐dimer levela	209 (19)	277 (35)	343 (16)
Type of event
DVT	625 (58)	443 (56)	1265 (59)
PE	355 (33)	263 (33)	685 (32)
DVT + PE	102 (9)	91 (11)	209 (9)

Note.Continuous variables are shown as mean (range) and categorical variables as number (%).

DVT, deep vein thrombosis; NA, not applicable; PE, pulmonary embolism.

A total of 41 were missing for hormone therapy and 1834 missingd for D‐dimer level.

Our definition of unprovoked venous thrombosis.

Vienna definition (positive lupus anticoagulant not included in definition since this was not measured in MEGA).

DASH definition (positive antiphospholipid antibodies not included in definition since these were not measured in MEGA).

During a median follow‐up of 5.7 (interquartile range 3.2‐7.4) years, 600 certain recurrent events occurred, for an incidence rate of 3.1 (95% CI, 2.9‐3.4) per 100 patient‐years (Table 2). The recurrence rate was 5.2 (95% CI, 4.6‐5.9) per 100 patient‐years in those who had a first unprovoked VT according to our definition, while it was 3.4 per 100 patient‐years for the Vienna model and 3.8 per 100 patient‐years for the DASH prediction rule.

Table 2

Risk of recurrent venous thrombosis in the MEGA follow‐up study

	N	Observation years	Recurrent events	Recurrence rate per 100 patient‐years (95% CI)
Categories
All patients in MEGA follow‐up	3750	19201	600	3.1 (2.9‐3.4)
With provoked first VTa	2592	13727	324	2.4 (2.1‐2.6)
With unprovoked VTa	1082	5180	269	5.2 (4.6‐5.9)
Unprovoked VT according to Vienna definitionb	797	4576	156	3.4 (2.9‐4.0)
With unprovoked first VTa	523	3000	109	3.6 (3.0‐4.4)
Unprovoked VT according to DASH definitionc	2158	10900	411	3.8 (3.4‐4.2)
With unprovoked first VTa	1082	5180	269	5.2 (4.6‐5.9)

Our definition of unprovoked venous thrombosis (VT).

Vienna definition (positive lupus anticoagulant not included in definition since this was not measured in MEGA).

DASH definition (positive antiphospholipid antibodies not included in definition since these were not measured in MEGA).

In Table 3, we show the results of the external validation of the two prediction models. The C‐statistic was 0.62 for Vienna and 0.66 for DASH. We also observed that the C‐statistic declined to 0.61 for Vienna and 0.56 for DASH when we used our definition of unprovoked VT. Results were similar in a sensitivity analysis where we included patients with missing data on cancer at time of VT and where we imputed data of patients who had missing D‐dimer levels. As can be observed from Table 4, the observed risks in MEGA more or less followed the predicted risks in Vienna and DASH, although absolute risks were lower in MEGA as compared with the development studies.13, 14

Table 3

External validation of three prediction models

Main analysis	Harrell's C‐statistic (95% CI)
	Development set	External validation
		MEGA FU studya	MEGA FU studyb
Vienna model	0.65	0.62 (0.57‐0.67)	0.61 (0.56‐0.67)
DASH score	0.71	0.66 (0.63‐0.68)	0.56 (0.51‐0.61)

Definition of unprovoked venous thrombosis in development datasets.

Our definition of unprovoked venous thrombosis.

Patients with missing D‐dimer levels imputed.

Table 4

Observed vs. predicted risks in the MEGA study vs. the development studiesa

	Predicted	Observedb	Observedc
Cumulative recurrence risk at 12 months (95% CI)
Vienna model
Quintile
1	3.0 (1.3‐3.7)	1.6 (0.4‐6.3)	1.1 (0.2‐7.9)
2	4.3 (3.7‐4.9)	5.0 (2.2‐10.9)	5.0 (1.9‐12.8)
3	5.4 (4.9‐6.0)	3.4 (1.3‐8.8)	5.2 (2.0‐13.3)
4	6.6 (6.0‐7.5)	5.1 (2.3‐11.0)	4.9 (1.9‐12.5)
5	9.5 (7.5‐18.5)	7.0 (3.6‐13.5)	6.7 (2.8‐15.1)
Cumulative recurrence risk at 24 months (95% CI)
DASH score
≤−1	2.6 (0.3‐4.9)	2.3 (1.1‐4.8)	NA
0	5.4 (3.1‐9.3)	7.6 (4.4‐13.0)	6.7 (2.6‐17.0)
1	8.7 (6.3‐12.0)	6.6 (4.8‐9.1)	6.1 (3.2‐11.3)
2	12.8 (9.9‐16.4)	8.2 (5.5‐12.1)	8.9 (4.8‐14.9)
3	20.5 (16.4‐25.5)	16.6 (13.6‐20.1)	10.5 (6.4‐17.2)
4	33.6 (23.3‐46.8)	19.4 (14.7‐25.4)	10.8 (2.8‐36.9)

NA denotes not available.

For the Vienna prediction model, numbers were obtained from the article of Marcucci et al.,14 since predicted risks per quintile were not available in the development study; for DASH, numbers were obtained from the validation study of DASH.15

Definition of unprovoked venous thrombosis in development datasets.

Our definition of unprovoked venous thrombosis.

Table 5 shows that male patients with a VT provoked by something other than surgery and a high D‐dimer level had an absolute recurrence risk of 6.8% (95% 5.0‐9.3) per year, which risk was virtually similar to the absolute risk of men who had unprovoked VT and a high D‐dimer level. In contrast, women with unprovoked first VT and a low D‐dimer had an absolute risk of 2.3% (95% CI, 1.4‐3.8) per year to develop recurrence, which is about the same risk as that of patients who had a first provoked event. Since we have previously shown in the MEGA follow‐up study that factor VIII improved prediction of recurrence, we added (in a post hoc analysis) high factor VIII:Ag (>200 IU/dL) to the group of men with first VT who had no surgery and a high D‐dimer and to the group of men with unprovoked VT and a high D‐dimer, which led to incidence rates for recurrence of 6.2% per year (95% CI 2.9‐12.9) and 6.2% per year (95% CI, 3.9‐9.7), respectively.

Table 5

Stratification of risk of recurrent venous thrombosis in patients with a provoked or unprovoked first venous thrombosis

	N	Recurrences	Observation years	Recurrence rate per 100 person‐years (95% CI)
Provoked in MEGAa	1349	138	8104	1.7 (1.4‐2.0)
Low‐risk group
Women, surgery, low D‐dimer	121	8	768	1.0 (0.5‐2.6)
High‐risk group
Men, no surgery, high D‐dimer	73	21	352	6.0 (3.9‐9.2)
Men, no surgery, high D‐dimer, high factor VIII	25	7	114	6.2 (2.9‐12.9)
Unprovoked in MEGAa	539	115	3089	3.7 (3.1‐4.5)
Low‐risk group
Women, low D‐dimer	86	14	539	2.6 (1.5‐4.4)
High‐risk group
Men, high D‐dimer	153	43	816	5.3 (3.9‐7.1)
Men, high D‐dimer, high factor VIII	58	19	308	6.2 (3.9‐9.7)

Patients with (un)provoked first venous thrombosis in MEGA, excluding patients with missing for surgery or D‐dimer.

DISCUSSION

This validation study of two previously established prediction models for recurrent VT shows that their ability to distinguish recurrence risks was lower than in the original studies.8, 9, 10 This is not surprising since validation studies usually show more conservative results as they were not optimized for the data from which they were originally derived. Nevertheless, although the C‐statistics were lower, they all confirmed the robustness of the prediction models, despite differences in patient characteristics in the cohorts and the application of slightly different rules in MEGA than in the original studies. Our results are comparable with previous studies that also found lower C‐statistics, with a C‐statistic of 0.39 when the Vienna prediction rule was validated in elderly patients (n = 156),12 a C‐statistic of 0.63 in the original Vienna cohort with extended follow‐up (n = 156), 22 a C‐statistic of 0.63 in another cohort study (n = 904) that externally validated the Vienna prediction rule,14 and a C‐statistic of 0.65 in a cohort study (n = 827) that externally validated the original DASH cohort.14

The effect of using different definitions for unprovoked events by the two models is illustrated by the differing characteristics of the patients selected according to the definitions applied by these models. Furthermore, the C‐statistics clearly dropped when we used our definition of unprovoked VT. This implies that definitions of what is and what is not an unprovoked event should be standardized and universally applied.18 Alternatively, and preferable in routine patient care, it could be considered to develop and use models that apply to all patients with a first event, without determining whether it was unprovoked or not.

Our results also showed that the absolute risk of recurrence varied 6‐fold (from 1.0/100 person‐years to 6.2/100 person‐years) when we stratified patients with a provoked first event according to presence of risk factors for recurrence. This indicates that there is much to gain from current guidelines that advise classifying all patients in only two groups: either high (unprovoked) or low (provoked) risk, where continuing treatment is recommended in those with unprovoked events in whom the bleeding risk is deemed to be low.18

One limitation is that our findings are only in part generalizable to DASH and Vienna since the follow‐up time between MEGA‐follow‐up and these studies differed. Although a longer follow‐up usually leads to lower yield of correct prediction, the Vienna model showed that risk of recurrent VT can be predicted from multiple random time points up to 5 years of follow‐up,22 so a negative effect of longer follow‐up is expected to be limited.

Another potential limitation is that D‐dimer levels were obtained with a different D‐dimer assay in the MEGA cohort and this might, in part, have contributed to the different D‐dimer level distributions observed. In addition, blood sampling was done 3 months after discontinuation, which is fundamentally different from the original models (Vienna and DASH), where blood is drawn 1 month after discontinuation. This may have led to underestimation of risks, as some studies have suggested a rebound phenomenon in which several markers of coagulation, including D‐dimer levels, increase shortly after anticoagulant treatment is withheld and because the risk of recurrence is highest shortly after anticoagulation withdrawal.23, 24, 25 Of note, 52 patients had a recurrent event before D‐dimer testing and were not included in the analyses as this could lead to immortal time bias. A further limitation is that we could not make patients completely comparable to the original populations of Vienna and DASH as we did not have information on antiphospholipid antibodies. The withdrawal of anticoagulants after VT was not standardized but left at the discretion of the treating physician. The implications of this varying duration are probably small, as the duration was mainly based on whether or not the first event was a DVT or PE, which distinction is not clearly related to recurrence risk.3 All these limitations, however, are expected to apply in current clinical practice, where definitions, treatment time, and follow‐up of patients after VT will inevitably differ. We did not include a validation of the HERDOO2 score as for the D‐dimer measurement in HERDOO2 a Vidas D‐dimer was drawn on anticoagulants with a 250 μg/L cutoff, while in MEGA we have done a HemosIL D‐dimer at 500 μg/L cutoff, off anticoagulants. It has been shown in a previous study that the HemosILD‐dimer assays should not be used in the HERDOO2 rule because of poor concordance with the Vidas D‐dimer assay leading to unacceptable misclassification of women at high and low risk of recurrent venous thrombosis.19 Furthermore, the results on risk stratification should be seen as illustrative only and not confirmative, as for this aim, one would need to build a new risk model, which is beyond the scope of the present study. Our validation study has focused strongly on the C‐statistic. It should be mentioned that the assumption that a model only works well if the C‐statistic is high is flawed since the C‐statistic only describes how well models can rank order high‐risk patients and low‐risk patients, but is not a function of the actual predicted probabilities, as we also explain in an accompanying Forum article.26 Nevertheless, much of risk prediction was determined by the definition of “unprovoked,” which suggests that there is much to gain when universal definitions of unprovoked events are applied or when a definition is not applied at all (i.e. all patients with VT events are considered to be at risk of recurrence).

In summary, this validation study showed that predicting the risk of unprovoked recurrent VT is possible to some extent with the currently available models, but that their predictive performance is lower than in the original studies. The predictive ability is strongly influenced by the definition of unprovoked VT. Furthermore, risks of recurrence clearly vary in patients with provoked events according to presence of risk factors for recurrence. This implies that the current policy of classifying recurrence risk on the basis of whether the first event was (un)provoked is too crude and should be reconsidered. In an accompanying Forum paper,26 we present solutions as to how a more‐refined risk estimation is possible at an individual level in patients at risk of recurrent venous thrombosis.

CONFLICT OF INTERESTS

The authors state that they have no conflict of interest.

AUTHOR CONTRIBUTIONS

J. F. Timp performed the statistical analyses and wrote the initial version of the manuscript. W. M. Lijfering: revised the manuscript. S. le Cessie assisted with and performed some of the statistical analyses and revised the manuscript. F. R. Rosendaal was responsible for the MEGA study concept and revision of the manuscript. S. C. Cannegieter designed the analyses and revised the manuscript.