Development of a clinical prediction model for an international normalised ratio ≥ 4·5 in hospitalised patients using vitamin K antagonists.

Vitamin K antagonists (VKAs) used for the prevention and treatment of thromboembolic disease, increase the risk of bleeding complications. We developed and validated a model to predict the risk of an international normalised ratio (INR) ≥ 4·5 during a hospital stay. Adult patients admitted to a tertiary hospital and treated with VKAs between 2006 and 2010 were analysed. Bleeding risk was operationalised as an INR value ≥4·5. Multivariable logistic regression analysis was used to assess the association between potential predictors and an INR ≥ 4·5 and validated in an independent cohort of patients from the same hospital between 2011 and 2014. We identified 8996 admissions of patients treated with VKAs, of which 1507 (17%) involved an INR ≥ 4·5. The final model included the following predictors: gender, age, concomitant medication and several biochemical parameters. Temporal validation showed a c statistic of 0·71. We developed and validated a clinical prediction model for an INR ≥ 4·5 in VKA-treated patients admitted to our hospital. The model includes factors that are collected during routine care and are extractable from electronic patient records, enabling easy use of this model to predict an increased bleeding risk in clinical practice.

Vitamin K antagonists (VKAs) are frequently used medications in the prevention and treatment of thromboembolic disease (Wysowski et al, 2007; Ansell et al, 2008). However, the benefit of their use is partially offset by the increased risk of bleeding complications. The reported overall risk of major bleeding complications is 1·4–2·1 per 100‐person years of VKA treatment (Palareti et al, 1996; Schulman et al, 2008; Roskell et al, 2012).

The risk of bleeding is related to the international normalised ratio (INR) and is influenced by many factors, such as dietary intake of vitamin K, concomitant medication, comorbidities and genetic factors (Schulman et al, 2008). Although numerous risk factors have been linked to a higher bleeding risk, it is difficult for physicians to assess the risk of bleeding by VKAs for an individual patient. Prediction models could help physicians to predict VKA‐associated bleeding complications and make more accurate assessments, which may lead to adjustments in therapy or closer monitoring.

Prediction models for bleeding complications or a supratherapeutic INR in patients on VKA therapy can be found in the literature (Beyth et al, 1998; Kuijer et al, 1999; Gage et al, 2006; Shireman et al, 2006; Ruiz‐Gimenez et al, 2008; Pisters et al, 2010; Fang et al, 2011; Hippisley‐Cox & Coupland, 2014; Focks et al, 2016). Given that bleeding itself is often not registered electronically, supratherapeutic INR can be used as a substitute as this is a proven risk factor for bleeding complications (Palareti et al, 1996; Hylek et al, 2003; Amouyel et al, 2009). Generally, these prediction models include comorbidities, such as hypertension (Gage et al, 2006; Pisters et al, 2010; Fang et al, 2011; Hippisley‐Cox & Coupland, 2014), history of stroke (Beyth et al, 1998; Gage et al, 2006; Pisters et al, 2010), prior bleeding (Beyth et al, 1998; Gage et al, 2006; Shireman et al, 2006; Pisters et al, 2010; Fang et al, 2011), malignancy (Kuijer et al, 1999; Gage et al, 2006; Ruiz‐Gimenez et al, 2008), genetic polymorphism (Gage et al, 2006) or fall risk (Gage et al, 2006). These are mainly risk factors that are not easily extractable from electronic medical records (EMRs). Therefore, the available prediction models cannot be implemented as electronic clinical decision support rules (‘clinical rules’). Yet, the application of such rules would greatly assist physicians to identify those patients for whom the risk of bleeding is high.

Furthermore, most models focus on patients with a specific indication, such as atrial fibrillation (AF) (Gage et al, 2006; Shireman et al, 2006; Pisters et al, 2010; Fang et al, 2011; Focks et al, 2016) or venous thromboembolism (VTE) (Kuijer et al, 1999; Ruiz‐Gimenez et al, 2008), and are derived from cohorts of patients in ambulatory care (Beyth et al, 1998; Ruiz‐Gimenez et al, 2008; Fang et al, 2011), or ambulatory and hospitalised patients together (Kuijer et al, 1999; Gage et al, 2006; Pisters et al, 2010).

Factors associated with the occurrence of bleeding in non‐selected patient populations during a hospital stay might be different from those during ambulatory care. Existing prediction models for bleeding events in patients using VKAs are not applicable for clinical rules and do not concern the general hospitalised population.

Therefore, we aimed to develop a model predicting the risk of an INR ≥ 4·5 during hospital stay, for adult patients who are treated with VKAs, based on risk factors that are electronically collected during routine care.

Methods

Study design

This study was designed as a cohort study. Data were prospectively recorded and retrospectively analysed. The medical ethics committee granted permission for this study.

Study setting

The study was conducted in the Erasmus University Medical Centre (Erasmus MC), a 1320‐bed University Medical Centre based in Rotterdam, the Netherlands.

Study population

Patients aged 18 years and older who were admitted to the Erasmus MC between January 2006 and December 2010 and treated with VKAs were included in the study. The VKAs used were acenocoumarol (B01AA07) and phenprocoumon (B01AA04), and are the most commonly used VKAs in the Netherlands. Exclusion criteria were the following: (i) patients with an admission to the intensive care unit (ICU), (ii) patients without an INR measurement during their treatment with a VKA, (iii) patients with an INR ≥ 4·5 as reason for hospitalization, which is defined as the occurrence of an INR ≥ 4·5 within 12 h after hospitalization.

Patients were considered at risk in the period between start of the prescription of the VKA until the end of the hospital prescription, plus a wash‐out period. The wash‐out period of a maximum of 5 times the elimination half‐life was set to 5 days for acenocoumarol and 14 days for phenprocoumon (Palareti et al, 1996).

Data collection

The hospital information system was used for data collection (Table SI). Patient data were coded according to Dutch privacy guidelines. Bleeding risk was operationalised as INR ≥ 4·5. Data were collected from start of VKA treatment until the first occurrence of an INR ≥ 4·5 or until the end of exposure to VKAs, discharge, in‐hospital death, or until the end of the study (31 December 2010 for the development cohort and 31 December 2014 for the validation cohort), whichever came first.

Candidate predictors

The following candidate predictors were included in the analysis: gender, age, occurrence of an INR ≥ 4·5 during a previous admission (yes/no), type of VKA (acenocoumarol/phenprocoumon), concomitant use of known interacting drugs and type of ward (medical/surgical). The cardiology wards, internal medicine wards, oncology wards and psychiatry wards were classified as ‘medical’, and the surgical wards, ear‐nose‐throat and eye surgery wards were classified as ‘surgical’. We also considered the most recently (with a maximum of 7 days) measured laboratory value in the week before start of VKA therapy of alanine aminotransferase (ALAT), aspartate aminotransferase (ASAT), gamma‐glutamyl transferase (GGT), lactate dehydrogenase (LDH), albumin, estimated glomerular filtration rate (e‐GFR) calculated with the modification of diet in renal disease (MDRD) formula (Levey et al, 1999), haemoglobin (Hb), creatinine, thyroid stimulation hormone (TSH), triiodothyronine (T3), thyroxin (T4), C‐reactive protein (CRP), platelet count (Plt) and leucocyte count (Leu).

We defined concomitant use of known interacting drugs as an active prescription at the same time the VKA was prescribed, or when the interacting drug was stopped before start of VKA but within the wash‐out period of a maximum of 5 times the elimination half‐life of the interacting drug (Palareti et al, 1996). The following drugs were considered as interacting drugs that increase the effect of VKAs the most; miconazole, cotrimoxazole, fluconazole, voriconazole and amiodarone. Rifampicin, carbamazepine, phenytoin, colestyramin and anti‐thyroid drugs were considered to decrease the effect of VKAs the most (De Federatie van Nederlandse Trombosediensten).

Statistical analysis

All data were analysed using R (The R Foundation for Statistical Computing, Vienna, Austria). Missing values of candidate predictors were filled in with multiple imputation (MI). Each missing value was imputed ten times. Imputed values were drawn from the predictive distribution in an imputation model that included all candidate predictors and the outcome (INR ≥ 4·5). MI resulted in ten complete datasets, which were analysed with standard data methods. The results were combined to produce overall estimates and standard errors that reflect missing data uncertainty (Van Buuren et al, 2006). Univariable and multivariable logistic regression analysis was used to assess the association of candidate predictors with the risk of an INR ≥ 4·5. As some patients were included multiple times for the recreation of our model we used random effect modelling (Harrell, 2001). For the continuous predictor age, a linear relationship with outcome was found to be a good approximation after assessment of nonlinearity using restricted cubic splines (Harrell, 2001). Age was included as piecewise linear with two pieces, up to 60 years and above 60 years, for a better description of the shape of the association with an INR ≥ 4·5. Some laboratory values (LDH and CRP) were log transformed for the same reason. Odds ratios for continuous variables were given for the 75th percentile versus 25th percentile of the variable. Using a backward elimination strategy with P < 0·15, the strongest prognostic factors were included in the final model (Steyerberg, 2008).

Internal validation

Despite the large cohort (N = 8996), the number of events were limited (N = 1507). Therefore we used bootstrap resampling to adjust for possible over‐fitting and optimistic performance of the model. One hundred bootstrap samples were drawn with replacement; a prognostic model was developed in each sample; and the performance was evaluated in the bootstrap sample and in the original sample. The average calibration slope of the bootstrap procedure was used to shrink the regression coefficients in the final model. The resulting final model was applied in an Excel risk calculator. The discriminative ability of the model was assessed with the concordance statistic (c‐statistic). Calibration was assessed with the calibration intercept and slope.

External validation

In order to validate the clinical prediction model, it was applied to a separate cohort of patients who were treated with VKAs and admitted to medical or surgical wards between 2011 and 2014 in the Erasmus MC. These patients were enrolled according to the same criteria as the patients in the development cohort.

Results

Cohort description

The study included 8996 admissions of 6073 individual patients treated with VKAs (Table 1). The median length of stay per admission was 6 days (interquartile range 3–11). The median age was 72 (interquartile range 62–82) years and 41% of patients were female. Acenocoumarol was prescribed more often (in 89% of admissions) than phenprocoumon (in 11% of admissions). We identified 1507 admissions (17%) with an INR ≥ 4·5 for 1112 individual patients.

Table 1

Baseline characteristics of the patients included in the development and validation cohorts, number of patients (%) unless otherwise stated

Characteristic	Development cohort (2006–2010) (n = 8996)	Validation cohort (2011–2014) (n = 9018)
Male gender	5310 (59·0)	5420 (60·1)
Age, yearsa	72 (62·0–82·0)	69·0 (58·0–77·0)
INR ≥ 4·5 during a previous admission	868 (9·6)	813 (9·0)
VKA type, acenocoumarol	7978 (88·7)	8192 (90·8)
Ward type, medical ward	5497 (59·8)	5112 (56·7)
Use of concomitant medication
Miconazole	153 (1·7)	82 (0·9)
Cotrimoxazole	337 (3·7)	214 (2·4)
Fluconazole	119 (1·3)	52 (0·6)
Voriconazole	7 (0·1)	27 (0·3)
Amiodarone	724 (8·0)	720 (8·0)
Rifampicin	53 (0·6)	58 (0·6)
Carbamazepine	73 (0·8)	49 (0·5)
Phenytoin	89 (1·0)	54 (0·6)
Colestyramin	17 (0·2)	89 (1·0)
Anti‐thyroid drugs	110 (1·2)	75 (0·8)
Laboratory parameters
ALAT (u/l)a	25·0 (16·0–44·0)	25·0 (17·0–43·0)
ASAT (u/l)a	30·0 (22·0–44·0)	31·0 (23·0–46·0)
GGT (u/l)a	61·0 (33·0–131·0)	66·0 (32·0–144·3)
LDH (u/l)a	442·5 (357·0–589·8)	251·0 (198·0–328·0)
Albumin (g/l)a	36·0 (31·0–41·0)	37·0 (32·0–42·0)
e‐GFR (ml/min/1·73 m2)a	70·0 (49·0–90·0)	68·0 (47·0–89·0)
Hb (g/l)a	116 (100–134)	116 (102–134)
TSH (mu/l)a	1·4 (0·7–2·8)	1·7 (1·0–3·0)
T3 (nmol/l)a	1·4 (1·0–1·8)	1·5 (1·4–1·9)
T4 (nmol/l)a	104·5 (83·5–132·0)	96·5 (79·5–123·5)
CRP (mg/l)a	30·0 (8·0–82·0)	23·0 (5·4–65·0)
Plt (×109/l)a	229·0 (175·0–300·8)	216·5 (165·0–288·0)
Leu (×109/l)a	8·4 (6·5–11·1)	8·7 (6·7–11·5)

ALAT, alanine amino transferase; ASAT, aspartate amino transferase; CRP, C‐reactive protein; e‐GFR, estimated glomerular filtration rate, calculated with the modification of diet in renal disease formula (Levey et al, 1999); GGT, gamma‐glutamyl transferase; Hb, haemoglobin; INR, international normalised ratio; LDH, lactate dehydrogenase; Leu, leucocyte count; Plt, platelet count; T3, triiodothyronine; T4, thyroxin; TSH, thyroid stimulation hormone; VKA, vitamin K agonist.

Results are presented as median (interquartile range).

Prediction model

After multivariate analysis, the following variables were identified as predictors: gender, age, ALAT, albumin, e‐GFR, and the natural logarithm (Ln) of both LDH and CRP. The strongest predictors for an INR ≥ 4·5 during hospitalization were concomitant use of miconazole, cotrimoxazole, fluconazole, voriconazole, amiodarone or anti‐thyroid drugs. The odds ratio (OR) and 95% confidence intervals (CI) are shown in Table 2. The predicted risk of an INR ≥ 4·5 during hospital stay was calculated using the formula detailed in Table 3. TSH, T3 and T4 There were too many missing values for TSH, T3 and T4 and these were excluded from the analysis. The variables for which laboratory values were missing are listed in Table SII

Table 2

Associations between predictors and bleeding complications

Characteristic	Coding	Odds ratio (95% confidence interval)
		Univariable	Multivariable
Gender	Female versus male	1·29 (1·13–1·48)	1·19 (1·04–1·36)
Age, years	>60 vs. ≤60	1·72 (1·50–1·97)	1·38 (1·20–1·59)
INR ≥ 4·5 during a previous admission	INR ≥ 4·5 vs. INR < 4·5	1·39 (1·15–1·67)	–
VKA type	Phenprocoumon versus acenocoumarol	0·98 (0·79–1·21)	–
Ward type	Surgical ward versus medical ward	1·06 (0·93–1·21)	–
Concomitant medication
Miconazole	Miconazole versus no miconazole	2·70 (1·82–4·00)	1·85 (1·24–2·78)
Cotrimoxazole	Cotrimoxazole versus no cotrimoxazole	2·41 (1·81–3·19)	2·20 (1·63–2·98)
Fluconazole	Fluconazole versus no fluconazole	3·55 (2·32–5·44)	2·68 (1·68–4·29)
Voriconazole	Voriconazole versus no voriconazole	17·51 (2·55–120·41)	9·36 (1·53–57·46)
Amiodarone	Amiodarone versus no amiodarone	2·23 (1·81–2·75)	2·28 (1·82–2·87)
Rifampicin	Rifampicin versus no rifampicin	2·06 (1·01–4·20)	–
Carbamazepine	Carbamazepine versus no carbamazepine	0·89 (0·42–1·90)	–
Phenytoin	Phenytoin versus no phenytoin	1·66 (0·92–2·99)	–
Colestyramin	Colestyramin versus no colestyramin	3·36 (1·07–10·60)	–
Anti‐thyroid drugs	Anti‐thyroid drugs versus no anti‐thyroid drugs	2·09 (1·25–3·50)	1·80 (1·08–3·00)
Laboratory parameters
ALAT (u/l)		0·98 (0·92–1·05)	0·93 (0·87–0·98)
ASAT (u/l)		1·05 (0·99–1·11)	–
GGT (u/l)		1·35 (1·14–1·59)	–
LDH (u/l)		1·48 (1·29–1·69)	1·34 (1·20–1·49)
Albumin (g/l)		0·52 (0·44–0·61)	0·66 (0·55–0·78)
e‐GFR (ml/min/1·73 m2)		0·69 (0·63–0·76)	0·68 (0·58–0·80)
Hb (g/l)		0·47 (0·40–0·54)	–
CRP (mg/l)		2·46 (2·08–2·91)	1·62 (1·31–2·00)
Plt (×109/l)		0·94 (0·82–1·07)	–
Leu (×109/l)		1·47 (1·32–1·64)	–

ALAT, alanine amino transferase; ASAT, aspartate amino transferase; CRP, C‐reactive protein; e‐GFR, estimated glomerular filtration rate, calculated with the modification of diet in renal disease formula (Levey et al, 1999); GGT, gamma‐glutamyl transferase; Hb, haemoglobin; INR, international normalised ratio; LDH, lactate dehydrogenase; Leu, leucocyte count; Plt, platelet count; VKA, vitamin K agonist.

Table 3

Prediction model

Steps	Formula
1. Calculate lp “all variables”a	=0·016 × Ageb + 0·176 × Genderc−0·003 × ALATd + 0·580 × log(LDH)e−0·042 × Albuminf−0·009 × e‐GFRg + 0·206 × log(CRP)h + 0·617 × Miconazolei + 0·789 × Cotrimoxazolej + 0·987 × Fluconazolek + 2·237 × Voriconazolel + 0·826 × Amiodaronem + 0·587 × Anti‐thyroid drugsn
2. Calculate the lp with the intercept	=−4·282 + lp
3. Calculate the prediction of an INR ≥ 4·5	=[1/(1 + exp (−lp))] × 100%

ALAT, alanine amino transferase; CRP, C‐reactive protein; e‐GFR, estimated glomerular filtration rate, calculated with the modification of diet in renal disease formula (Levey et al, 1999); INR, international normalised ratio; LDH, lactate dehydrogenase.

lp refers to the linear predictor in a logistic regression model.

Age (=0, for age ≤60 years; =age (in years) – 60, for age >60 years).

Gender (female = 1, male = 0).

ALAT (alanine amino transferase) in u/l.

LDH (lactate dehydrogenase) in u/l.

Albumin in g/l.

e‐GFR (estimated glomerular filtration rate) in ml/min/1·73 m2.

CRP (c‐reactive protein) in mg/l.

Concomitant use of miconazole (yes = 1, no = 0).

Concomitant use of cotrimoxazole (yes = 1, no = 0).

Concomitant use fluconazole (yes = 1, no = 0).

Concomitant use of voriconazole (yes = 1, no = 0).

Concomitant use of amiodarone (yes = 1, no = 0).

Concomitant use of anti‐thyroid drugs (yes = 1, no = 0).

Bootstrapping resulted in a shrinkage factor of 0·95. The c‐statistic was 0·72 before and 0·71 after shrinkage, which shows our initial model had only minor optimism.

We identified 1,227 admissions (14%) with an INR ≥ 4·5 for 1052 individual patients in the validation cohort. External, temporal validation resulted in a c‐statistic of 0·71, which shows that the prediction model is applicable to patients that were hospitalised in a different time period than that of our development cohort. The calibration plots represent the agreement between the predicted and observed INR values ≥ 4·5 (Fig 1). The calibration‐in‐the‐large was 0·34 and the calibration slope was 1·06. After correction for the calibration‐in‐the‐large, the calibration‐in‐the‐large was 0 and the calibration slope was 1·06.

Figure 1

Validation plots for the prediction model of an international normalised ratio ≥ 4·5. (A) the calibration‐in‐the‐large (a|b) was 0·34 and the calibration slope (slope b) was 1·06 before correction; (B) after correction for the calibration‐in‐the‐large, the calibration‐in‐the‐large was 0 and the calibration slope was 1·06. The distribution of predicted risks is shown at the bottom of the graphs. Triangles indicate the observed proportions by quintiles of predicted risks.

Figure 2 is a score chart that is based on the formula detailed in Table 3. The score chart can be used to obtain approximate predictions for individual patients. For example, according to Fig 2, for an 80‐year‐old woman, admitted to a medical ward with an ALAT of 23 u/l, LDH of 370 u/l, albumin of 40 g/l, e‐GFR of 30 ml/min/1·73 m2, and a CRP of 80 mg/l and treated with phenprocoumon, fluconazole and amiodarone, the risk of an INR ≥ 4·5 during her hospital stay would be 10·8%.

Figure 2

Screenshot of the spreadsheet with calculations for an individual patient using the prediction model. aAge (=0, for age ≤60 years; =age (in years) – 60, for age >60 years). ALAT alanine amino transferase; CRP: C‐reactive protein; e‐GFR estimated glomerular filtration rate, calculated with the modification of diet in renal disease formula (Levey et al, 1999); INR, international normalised ratio; LDH, lactate dehydrogenase.

Discussion

We developed and validated a clinical prediction model for the risk of an INR ≥ 4·5 in patients admitted to medical or surgical wards who are treated with VKAs. The prediction model can help physicians to identify patients at the lower spectrum of thromboembolic risk and those for whom the risk of bleeding during VKA therapy is high. Using the prediction model may also help when counselling and informing patients about their potential risk for haemorrhage while on anticoagulants, and in identifying patients who might benefit from more careful management of anticoagulation.

To our knowledge, this is the first study to develop such a clinical prediction model.

The strongest predictors for an INR ≥ 4·5 during hospitalization were concomitant use of voriconazole, fluconazole, amiodarone, cotrimoxazole or miconazole. These drugs inhibit the metabolism of VKAs by inhibiting the liver enzyme CYP2C9 (Harder & Thurmann, 1996; Cadiou et al, 2008), and thus, an increased risk of an INR ≥ 4·5 is expected. Concomitant use of rifampicin, carbamazepine, phenytoin or colestyramin showed no association of the occurrence of an INR ≥ 4·5. These medications induce the metabolism of VKAs by inducing the liver enzyme CYP2C9, and thus, a decreased risk of an INR ≥ 4·5 was expected. We expected to find a similar effect of anti‐thyroid drugs, which also induce the metabolism of VKAs leading to a decrease of an INR ≥ 4·5. However, our study showed an increased risk of an INR ≥ 4·5 when anti‐thyroid drugs were used concomitantly. We have no explanation for this.

In our study we found that women had a 1·2‐fold (95% CI, 1·1–1·4) higher risk of an INR ≥ 4·5 than men. Our results are in line with prior studies that found an increased frequency of bleeding among women treated with VKAs. Cosma Rochat et al (2009) found that hospitalised women receiving vitamin K antagonists had a 4‐fold increased risk of bleeding compared with men. A possible explanation for the higher bleeding risk in women may be a systematic sex difference in the coagulation and fibrinolytic cascades (Reynolds et al, 2007; Cosma Rochat et al, 2009).

Furthermore, advanced age was associated with an increased risk of an INR ≥ 4·5, a finding that is consistent with previous studies (Kuijer et al, 1999; Torn et al, 2005; Gage et al, 2006; Pisters et al, 2010). The predictors ‘type of ward’ and ‘type of VKA’ showed no relationship with INR ≥ 4·5 in this study. Gadisseur et al (2002) found that the short acting acenocoumarol is associated with more variability in INR, but this did not lead to a higher risk of overanticoagulation compared to phenprocoumon in our study.

The risk profile and metabolism of warfarin, which is the main VKA used in other countries, is generally similar to that of acenocoumarol and phenprocoumon (Ufer, 2005; Beinema et al, 2008). These VKAs differ in elimination half‐life and response to polymorphisms in the gene coding for the metabolizing enzyme CYP2C9. Acenocoumarol has the shortest half‐life (08:00–14:00 h) and greatest response to polymorphisms. Phenprocoumon has the longest elimination half‐life (12:00–20:00 h) and lowest response. The half‐life of warfarin ranges from 20 to 60 h, with a mean of about 40 h (Ufer, 2005; Beinema et al, 2008).

Several other models included an INR ≥ 4·5 during a previous hospital admission in the final model (Beyth et al, 1998; Kuijer et al, 1999; Gage et al, 2006; Pisters et al, 2010; Fang et al, 2011), but this was not confirmed in our study. A reason for this may be that two subsequent hospitalizations are totally different (i.e. type of ward, concomitant medication, reason for hospitalization) and too far apart, with the result that both hospitalisations cannot be compared to each other.

Elevated liver enzymes (ALAT, ASAT, GGT and LDH) may indicate inflammation or damage to cells in the liver. The observed association in this study of an increased LDH with an increased risk of an INR ≥ 4·5 could be the result of a deteriorating capacity of the liver to produce clotting factors or to metabolise VKAs properly. The same association was expected between ALAT, ASAT, GGT and INR ≥ 4·5. However, patients with an elevated ALAT level had a lower risk of an INR ≥ 4·5 and ASAT and GGT showed no relationship with INR ≥ 4·5 in this study. As shown in Table 2, the observed ALAT and ASAT levels in our population were not very high. This may be the reason that our findings are contrary to what we expected.

Higher concentrations of albumin were predictive for a decreased risk of an INR ≥ 4·5. VKAs bind to albumin in plasma and only unbound drugs have a pharmacological effect. Another possible explanation is that lower concentrations of albumin represent a deteriorating condition of the patient resulting in a reduced intake of vitamin K.

Patients with a high e‐GFR have a 0·7‐fold (95% CI, 0·6–0·8) lower risk of an INR ≥ 4·5 than patients with a low renal function. However, the VKA elimination does not depend on renal function so a causal link cannot be established. VKAs are mainly metabolised by liver enzymes to inactive metabolites that are excreted in the urine. The positive effect of a good renal function may be the result of a better condition of the patient in general.

Our results also showed that high CRP levels were predictive for an increased risk of an INR ≥ 4·5. CRP has a positive association with infections and inflammations, which might affect coagulation. A potential mechanism for the higher risk of an INR ≥ 4·5 during infections and inflammations is the increased catabolism of vitamin K dependent clotting factors and inhibition of VKA metabolism (Timothy et al, 2015).

Most models that have been developed by others use binary values for age groups, liver and renal disease (Kuijer et al, 1999; Beyth et al, 2002; Gage et al, 2006; Pisters et al, 2010; Fang et al, 2011). Our final prediction model consists of predictors with continuous values for age and for laboratory values. This makes it difficult to compare our model to other models. The predictors, age (Beyth et al, 1998; Kuijer et al, 1999; Gage et al, 2006; Ruiz‐Gimenez et al, 2008; Pisters et al, 2010; Fang et al, 2011) and renal function (Beyth et al, 1998; Kuijer et al, 1999; Gage et al, 2006; Ruiz‐Gimenez et al, 2008; Pisters et al, 2010; Fang et al, 2011) seem to be present in most models. Our model includes several concomitant medications that are easily extractable from the EMR.

Limitations

The first limitation is that we used a surrogate marker for an increased risk of bleeding. We would have preferred to predict bleeding itself, but that complication was not registered in an easily extractable way in the EMR. An INR ≥ 4·5 is an adequate marker because 4·5 is the level at which the risk of bleeding increases sharply (Palareti et al, 1996; Hylek et al, 2003; Amouyel et al, 2009). Second, although we had many candidate predictors, several potentially significant predictors were not available. For example, information on the indication for VKA treatment or on the target INR was lacking. Patients with a mechanical heart valve, for example, have a higher target INR (2·5–3·5) than patients with atrial fibrillation, where the target INR ranges from 2·0 to 3·0 (Kirchhof et al, 2016). Patients with a higher target INR are therefore more susceptible to reach a supratherapeutic INR (Meschengieser et al, 1997). We couldn't include comorbidities, because they were not extractable from the EMR. Furthermore, this study has included all adult patients admitted to the hospital, except those admitted to the ICU. This might introduce selection bias because patients who have been transferred to the ICU represent a special group of patients with increased disease severity that is not represented in this study. Finally, the study was performed in one university hospital, which may limit generalizability.

Strengths

Notwithstanding these limitations, the strong point of our study is that we included all adult patients admitted to medical or surgical wards of the hospital. Another strength is that we validated our model, which showed that the prediction model is applicable to patients that were hospitalised in a different time period than that of our development cohort. Furthermore, the predictors in the model are extracted from the EMR, which makes it possible to develop an electronic prediction rule, enabling doctors to easily assess the individual risks of an INR ≥ 4·5.

Implications

This study shows that it is possible to develop an electronic prediction rule for an INR ≥ 4·5 in hospitalised patients using VKAs. The prediction model can help physicians to identify patients at the lower spectrum of thromboembolic risk and those for whom the risk of bleeding during VKA therapy is high. Using the prediction model may also help when counselling and informing patients about their potential risk for haemorrhage while on anticoagulants, and in identifying those patients who might benefit from more careful management of anticoagulation. Alternatively, these patients can also be switched to direct oral anticoagulants (DOACs), which cause less major bleeding, such as intracranial haemorrhages, compared to VKAs (Adam et al, 2012).

The methodology for developing an electronic prediction rule for VKAs used in our study may also be applied to other anticoagulants, such as the DOACs. Future studies are necessary to further improve the prediction model by including patients admitted to the ICU, and by incorporating time in the therapeutic range (TTR) which is associated with the effectiveness and safety of VKA therapy (Lin et al, 2017).

Furthermore, information about the indication of the VKA, the duration of use of VKAs before admission and comorbidities can be included to the prediction model to investigate whether it leads to a more accurate prediction model. Ideally, a prospective intervention study should be performed after implementation of the electronic prediction rule, to investigate whether the use of such a rule leads to a decrease in the number of admissions during which an INR ≥ 4·5 occurs and whether this results in less bleeding complications.

Conclusions

We developed a clinical prediction rule with a c‐statistic of 0·71 for an INR ≥ 4·5 in patients admitted to medical or surgical wards who are treated with VKAs. The model includes several risk factors, including concomitant medication, which are easily extractable from electronic patient records. This enables the creation of a clinical decision support rule, based on the prediction model identified in this study.

Contributors

ARD and ADL wrote the manuscript; all other co‐authors commented on previous versions of the manuscript and agreed with the final content. ARD and JSB coordinated the data collection. YV performed the analysis. PMLAvdB en MJHAK designed the study. JD and ADL participated in the study design. All authors read and approved the final manuscript.

Funding

None.

Table SI. Data collection.

Table SII. The variables for which laboratory values were missing.

Click here for additional data file.