Clinical Prediction Model for Time in Therapeutic Range While on Warfarin in Newly Diagnosed Atrial Fibrillation.

BACKGROUND: Though warfarin has historically been the primary oral anticoagulant for stroke prevention in newly diagnosed atrial fibrillation (AF), several new direct oral anticoagulants may be preferred when anticoagulation control with warfarin is expected to be poor. This study developed a prediction model for time in therapeutic range (TTR) among newly diagnosed AF patients on newly initiated warfarin as a tool to assist decision making between warfarin and direct oral anticoagulants. METHODS AND
RESULTS: This electronic medical record-based, retrospective study included newly diagnosed, nonvalvular AF patients with no recent warfarin exposure receiving primary care services through a large healthcare system in rural Pennsylvania. TTR was estimated as the percentage of time international normalized ratio measurements were between 2.0 and 3.0 during the first year following warfarin initiation. Candidate predictors of TTR were chosen from data elements collected during usual clinical care. A TTR prediction model was developed and temporally validated and its predictive performance was compared with the SAMe-TT2R2 score (sex, age, medical history, treatment, tobacco, race) using R2 and c-statistics. A total of 7877 newly diagnosed AF patients met study inclusion criteria. Median (interquartile range) TTR within the first year of starting warfarin was 51% (32, 67). Of 85 candidate predictors evaluated, 15 were included in the final validated model with an R2 of 15.4%. The proposed model showed better predictive performance than the SAMe-TT2R2 score (R2=3.0%).
CONCLUSIONS: The proposed prediction model may assist decision making on the proper mode of oral anticoagulant among newly diagnosed AF patients. However, predicting TTR on warfarin remains challenging.

Clinical Perspective

What Is New?

This study developed and validated a new clinical prediction model for estimating time in therapeutic range (TTR) while on warfarin among patients with newly diagnosed atrial fibrillation.

The proposed model contains 15 predictors of TTR that demonstrated stronger predictive performance than a previously published model designed with the same intent.

The model allows estimation of TTR on a per‐patient basis, which may facilitate decision making between warfarin and alternative anticoagulants among warfarin‐naïve patients with a new atrial fibrillation diagnosis.

What Are the Clinical Implications?

The decision to initiate warfarin versus an alternative anticoagulant can be challenging given the various benefits, risks, and costs associated with these therapies.

The efficacy and safety of warfarin are highly dependent on TTR, so an argument can be made that warfarin should be preferred when TTR is expected to be high.

The proposed prediction model enables individual‐level estimation of TTR: high estimated TTRs from the model provide support for warfarin therapy, while low estimated TTRs suggest alternatives should be considered.

Atrial fibrillation (AF) is a growing public health and clinical problem, as an aging population, prolonged survival of patients with cardiac conditions predisposing to AF, enhanced detection of more sporadic forms of AF, and other factors have collectively served to expand the number of AF diagnoses.1 AF portends a 5‐fold increased risk of thromboembolic events; thus, any new AF diagnosis requires assessment of thromboembolic risk and in many cases, proper oral anticoagulation (OAC) to mitigate excess risk.2, 3, 4, 5, 6 Though warfarin and other vitamin K antagonists (VKA) have been the primary OACs for several decades, the recent introduction of 4 non–vitamin K antagonists, direct OACs (DOACs), has expanded the therapeutic options for thromboprophylaxis in AF. Meta‐analyses have reported small improvements in efficacy and safety with DOACs compared with warfarin; however, these modest improvements must be considered with the higher cost of DOACs, which have created uncertainty about their overall cost‐effectiveness.7, 8, 9, 10 Other concerns limiting more widespread use of DOACs include an inability to monitor their anticoagulation effects, short half‐lives that increase thrombosis risk when doses are missed, and lack of experience with reversal agents that may be needed in urgent settings.11 Alternatively, warfarin is an inexpensive, long‐used, well‐researched, and effective therapeutic, associated with a 64% reduction in ischemic stroke in nonvalvular AF compared with placebo.12, 13 However, warfarin possesses multiple untoward attributes complicating its use, including a narrow therapeutic range, numerous drug and dietary interactions, and a metabolism strongly dependent on multiple genetic polymorphisms that make warfarin's ultimate pharmacologic effects difficult to predict.9, 12, 13, 14

The safety and efficacy of warfarin is highly dependent on the quality of anticoagulation control, often measured by the percent of time in therapeutic range (TTR), with TTR demonstrating strong inverse associations with both ischemic stroke and bleeding events.15, 16, 17, 18, 19, 20 In light of available evidence, warfarin might be the preferred initial choice of OAC provided a high TTR can be expected, while DOACs might alternatively be preferred with a low expected TTR. Though multiple studies have evaluated predictors of TTR, only a single study has combined the effects of multiple TTR predictors into a prediction tool designed to assist decision making between warfarin and DOACs (the SAMe‐TT2R2 [sex, age, medical history, treatment, tobacco, race] score).9, 21 The SAMe‐TT2R2 score was derived from a relatively small randomized clinical trial population, considered a limited number of candidate predictors, and demonstrated less than ideal predictive performance at external validation.16, 21, 22, 23 Accordingly, the current study sought to create an improved clinical prediction model for estimating TTR among newly diagnosed AF patients for whom warfarin was initiated for anticoagulation control. The proposed prediction model was compared with the SAMe‐TT2R2 score with regard to predictive performance while simultaneously providing an additional external validation of the SAMe‐TT2R2 score. Finally, numerical results are provided in a format allowing clinical application of the model.

Methods

This retrospective study incorporates the patient population and electronic medical record (EMR) data warehouse of the Geisinger Health System (Geisinger). The study was approved by the Geisinger Institutional Review Board who granted a waiver of patient consent. The study was restricted to patients receiving primary care and other healthcare services through Geisinger for at least a 2‐year period in order to better rule out pre‐existing AF before entering the Geisinger EMR system, identify warfarin use before a new AF diagnosis, and provide a detailed account of baseline characteristics at the time of newly diagnosed AF (see below).

Study Population

The intent of the study inclusion criteria was to identify a group of warfarin‐naïve patients with newly diagnosed AF subsequently prescribed warfarin but who would have been equally eligible for a DOAC under contemporary guidelines.3 Newly diagnosed AF occurring between 2003 and 2014 was defined by observing the appropriate International Classification of Diseases—9th Revision codes (427.3, 427.31, 427.32) at either 1 inpatient or 2 separate outpatient encounters following a minimum 2‐year period where no EMR documentation of these codes was found.24, 25, 26, 27, 28 Study patients also had no EMR documentation of warfarin use in the 2+ years before the AF diagnosis and were subsequently prescribed warfarin within 90 days of diagnosis. In accord with recent guidelines, patients with a documented history of rheumatic mitral stenosis, valve replacement, or mitral valve repair were excluded because of valvular AF.3 Study patients were required to have at least 4 international normalized ratio (INR) measurements within the first year following warfarin initiation in order to provide a reasonably valid estimate of TTR. The 1‐year postwarfarin initiation time period for TTR determination was chosen as a reasonable interval for judging anticoagulation quality with newly initiated warfarin. The TTR percentage was calculated as the percentage of days where the estimated INR was between 2.0 and 3.0 inclusive using linear interpolation to estimate unmeasured INRs between consecutive measurements.29 INRs were interpolated only when time intervals between successive measurements were 60 days or less, consistent with prior studies.21, 30, 31, 32, 33, 34, 35, 36

Candidate TTR Predictors

A set of 85 candidate predictors of TTR was identified through data elements gathered during usual clinical care and stored within patient EMRs. The general strategy for candidate predictor selection was to assemble a large and diverse set of potential predictors from various domains that might assist in predicting TTR on warfarin. These variable domains included demographics, vital signs, medical history including diagnoses and procedures, medications, and laboratory tests (Table 1). All study variables were evaluated with reference to a baseline date, the date of the first warfarin prescription following the AF diagnosis. All historical diagnoses and procedures clinically documented at any encounter at the baseline date or earlier were considered present at baseline. Diagnoses and procedures are defined by International Classification of Diseases—9th Revision and/or Current Procedural Terminology codes. Vital signs and laboratory values assigned to a patient at baseline were determined in a hierarchical manner with (1) an outpatient value measured on the baseline date given highest priority; followed by (2) the outpatient value measured prior, but closest to, the baseline date; and then (3) the outpatient value measured following, but closest to, the baseline date up to 90 days following baseline. Inpatient values were considered in the same temporal fashion when no outpatient value was available. All laboratory tests available on >75% of study patients were considered as candidate predictors. Missing data for vital signs and laboratory tests are not missing at random (missing data imply better health); thus, usual imputation strategies are of questionable validity. Accordingly, a conservative imputation approach was taken whereby missing values were imputed via random selection from the empirical distribution. The random selection was repeated multiple times to evaluate the sensitivity of effect estimates to this approach. Medications at baseline were those ordered or affirmed on medication reconciliation lists up to 1 year before or 90 days following baseline with the exception of aspirin and other platelet aggregation inhibitors such as clopidogrel. As use of these medications must be re‐evaluated and possibly discontinued when starting warfarin, only new orders documented after the warfarin initiation date defined users of these medication subclasses at baseline.

Table 1

Baseline Characteristics of Newly Diagnosed AF Patients Started on Warfarin Within 90 Days of Diagnosis

	All Patients (n=7877)	Development Set (n=5173)	Validation Set (n=2704)
Demographics and vital signs
Age, y	74 (66, 81)	74 (65, 81)	75 (66, 82)
Male, %	55	55	54
White, %	99	99	99
Smoking status
Current smoker, %	11	11	10
Former smoker, %	36	32	44
Never smoker, %	53	57	46
Systolic blood pressure, mm Hg	132 (118, 148)	132 (118, 148)	134 (120, 150)
Diastolic blood pressure, mm Hg	78 (68, 84)	76 (68, 84)	78 (70, 84)
Heart rate, bpm	78 (68, 88)	78 (68, 88)	80 (70, 88)
Body mass index, kg/m2	30 (26, 35)	30 (26, 35)	30 (26, 36)
CHA2DS2–VASc score	3.5±1.8	3.3±1.8	3.8±1.8
SAMe‐TT2R2 score	1.6±1.1	1.5±1.1	1.7±1.1
Medical history
Alcohol problem, %	2	2	3
Anemia, %	25	21	32
Anxiety, %	13	12	16
Arrhythmia (non‐AF), %	17	16	20
Cancer, %	20	18	23
Cardiomyopathy (non‐HF), %	6	6	8
CBVD (nonstroke/TIA), %	13	12	15
Coagulation defect, %	1	1	1
Conduction disorder, %	6	5	8
Congenital heart disease, %	3	3	3
Coronary bypass surgery, %	9	9	10
Coronary artery disease, %	34	33	37
Dementia, %	2	1	2
Depression, %	14	12	19
Diabetes mellitus, %	29	27	34
Gastrointestinal bleeding, %	6	5	8
Gout, %	6	5	8
Heart failure, %	25	24	27
Hyperlipidemia, %	57	52	67
Hypertension, %	69	65	76
ICD, %	2	1	2
Kidney disease, %	15	11	24
Liver disease, %	3	2	4
Lung disease, %	20	19	22
Memory loss, %	1	1	2
Myocardial infarction, %	13	12	16
Pacemaker, %	5	4	5
PCI, %	8	7	10
Peripheral artery disease, %	10	9	12
Pulmonary embolism, %	3	3	3
Sleep apnea, %	8	6	13
Stroke—hemorrhagic, %	1	<1	1
Stroke—ischemic, %	8	6	13
Tachycardia, %	4	4	6
Thrombocytopenia, %	3	3	3
Transient ischemic attack, %	5	4	6
Valve disease, %	20	19	21
Venous thromboembolism, %	3	3	4
Medications
ACE inhibitor/ARB, %	62	60	66
Antiadrenergic antihypertensive, %	11	11	12
Any antiarrhythmic, %	35	32	41
β‐Blocker, %	82	80	86
Calcium channel blocker, %	46	42	55
Digoxin, %	28	31	22
Statins, %	61	55	71
Diuretic, %	69	66	73
Platelet aggregation inhibitor, %	6	5	7
Aspirin, %	24	20	32
Laboratory tests
Alanine aminotransferase, IU/L	22 (16, 31)	22 (16, 31)	20 (15, 30)
Albumin, g/dL	4.0 (3.7, 4.3)	4.0 (3.6, 4.3)	4.1 (3.7, 4.3)
Alkaline phosphatase, U/L	77 (62, 96)	77 (62, 96)	75 (61, 94)
Aspartate aminotransferase, U/L	25 (20, 31)	25 (20, 32)	24 (20, 31)
Bilirubin, mg/dL	0.5 (0.4, 0.7)	0.5 (0.4, 0.7)	0.5 (0.3, 0.7)
Blood urea nitrogen, mg/dL	19 (15, 25)	19 (15, 25)	20 (15, 26)
Calcium, mg/dL	9.3 (9.0, 9.6)	9.3 (9.0, 9.6)	9.3 (8.9, 9.6)
Carbon dioxide, mEq/L	28 (26, 30)	28 (26, 30)	28 (26, 30)
Chloride, mmol/L	102 (100, 104)	102 (100, 104)	102 (99, 104)
Cholesterol, mg/dL	166 (139, 195)	168 (142, 196)	162 (135, 191)
Glomerular filtration rate, mL/min	60 (52, 60)	60 (54, 60)	60 (49, 60)
Glucose, mg/dL	106 (93, 133)	105 (93, 131)	107 (93, 136)
HDL cholesterol, mg/dL	47 (38, 58)	47 (39, 59)	47 (38, 58)
Hematocrit, %	39 (36, 43)	40 (36, 43)	39 (35, 42)
Hemoglobin, g/dL	13.2 (11.8, 14.5)	13.3 (12.0, 14.6)	13.0 (11.5, 14.3)
LDL cholesterol, mg/dL	88 (68, 112)	89 (69, 113)	86 (64, 110)
Lymphocyte, % of total WBC	21 (15, 28)	20 (14, 27)	22 (16, 28)
MCHC, g/dL	33.7 (32.8, 34.3)	33.9 (33.2, 34.4)	33.2 (32.2, 34.0)
MCH, pg	30.5 (29.3, 31.7)	30.7 (29.5, 31.9)	30.1 (28.9, 31.4)
Mean corpuscular volume, fL	90.8 (87.6, 94.0)	90.8 (87.6, 94.0)	90.8 (87.7, 94.1)
Mean platelet volume, fL	9.5 (8.3, 10.5)	9.0 (8.0, 10.1)	10.2 (9.4, 11.0)
Neutrophil, % of total WBC	66 (59, 74)	67 (60, 74)	65 (58, 73)
Platelet count, ×103/mcL	228 (186, 281)	231 (189, 285)	223 (181, 272)
Potassium, mEq/L	4.3 (4.0, 4.6)	4.3 (4.0, 4.6)	4.2 (4.0, 4.5)
Protein, g/dL	6.9 (6.5, 7.2)	6.9 (6.5, 7.3)	6.8 (6.4, 7.2)
Red blood cell count, ×106/mcL	4.4 (3.9, 4.7)	4.4 (3.9, 4.8)	4.3 (3.8, 4.7)
Red blood cell distribution width, %	14.1 (13.3, 15.4)	14.1 (13.3, 15.4)	14.1 (13.3, 15.3)
Sodium, mmol/L	139 (137, 141)	140 (137, 141)	139 (137, 141)
White blood cell count, ×103/mcL	7.5 (6.1, 9.2)	7.5 (6.1, 9.2)	7.4 (6.1, 9.3)

CHA2DS2–VASc=congestive heart failure, hypertension, age (≥75), diabetes mellitus, stroke/TIA/TE, vascular disease, age (65–74), sex. SAMe‐TT2R2=sex, age, medical history, treatment, tobacco, race. ACE indicates angiotensin‐converting enzyme; AF, atrial fibrillation; ARB, angiotensin receptor blocker; bpm, beats per minute; CBVD, cerebrovascular disease; HDL, high‐density lipoprotein; ICD, implanted cardioverter defibrillator; LDL, low‐density lipoprotein; MCH, mean corpuscular hemoglobin; MCHC, mean corpuscular hemoglobin concentration; PCI, percutaneous coronary intervention; SAMe‐TT2R2=sex, age, medical history, treatment, tobacco, race; TE, Thromboembolism; TIA, transient ischemic attack; WBC, white blood cells.

Analytic Strategy

The primary analysis focuses on TTR within the first year of warfarin initiation as a continuous variable. Though multiple thresholds for poor and optimal TTR have been suggested, there remains no consensus and accordingly various TTR thresholds were evaluated as secondary end points.9, 12, 19, 23, 31 All continuous variables were categorized into 6 groups split at the 10th, 25th, 50th, 75th, and 90th percentiles of the empirical distributions in order to accommodate possible nonlinear and nonmonotonic associations between continuous covariates and TTR, identify possible threshold effects, and circumvent the potential adverse impact of extreme outliers on the magnitude of regression coefficients. The 6 ordinal groups are henceforth referred to as very low (<10th percentile), low (10–25), low‐normal (25–50), high‐normal (50–75), high (75–90), and very high (>90).

The patient cohort was temporally split in an ≈2:1 ratio at May 1, 2011, into development and validation sets; an initial prediction model was built using the development set and tested on the validation set.37, 38 Linear regression modeling was used and a forward stepwise variable selection algorithm applied in order to identify the strongest independent predictors of continuous TTR.38, 39 All models generated during the variable selection process were reviewed closely for collinearity and adjustments made as needed. As all variables were placed on categorical scales (binary or 6‐group variables as described above), the criterion for variable inclusion/exclusion in the development set was an absolute 3% difference in TTR across a variable's levels as opposed to the usual P‐value threshold criterion. For the 6‐group variables, if adjusted regression model coefficients differed by ≥3% across any pair of levels, the variable was kept in the development set model. Once the final development set model was confirmed, a model with the same parameterization was applied to the validation set. Development model variables were considered validated if the 3% difference criterion was also observed in the independent validation set. In essence, the development set is used to hypothesize potential data patterns and any observed pattern is considered validated if also observed in a second, independent data set. This modeling tactic serves as a safeguard against model overfitting by requiring effects observed in the development set to persist in an independent group of patients. The final reported model then considers the entire patient cohort and is restricted to the validated variables. For all models, the R 2 statistic is reported, quantifying the amount of variation in TTR explained by the collective set of included variables.

Secondary Analyses

Several secondary analyses were performed. The large sample size permitted fitting a full model containing all 85 candidate covariates in a 220‐degree of freedom model, which allows quantifying via R 2, the maximum predictive capacity of the entire collection of candidate predictors. The full model is less informative for evaluating effects of individual predictors because of extensive collinearity; thus, regression coefficients are not reported. Next, several TTR cut points indicative of possible poor or optimal TTR were identified and logistic regression models fit to evaluate the impact of the final model predictors with respect to these cut points. C‐statistics are reported for all logistic regression models. All predictive performance metrics were repeated after including the SAMe‐TT2R2 score as the sole covariate in the various models. All statistical analyses were conducted with the SAS statistical software, version 9.4.

This study was an investigator‐initiated study funded by Daiichi Sankyo, who made minor suggestions to the original study proposal but otherwise had no role in study design, execution, or article preparation.

Results

Out of 20 183 new AF diagnoses between 2003 and 2014, 10 652 (53%) were taking warfarin within 90 days of diagnosis, while 1267 (6%) were taking a DOAC. Among warfarin users, 7877 (74%) met the definition for nonvalvular AF, had no documentation of warfarin use before diagnosis, and had at least 4 INR measurements within 1 year following warfarin initiation. The median (interquartile range) TTR within the first year following warfarin initiation was 51% (32, 67), and beyond the first year was 61% (44, 74). The distribution of first‐year TTRs is shown in Figure. The linear regression model containing all 85 candidate predictors (the full model) had R 2=19.5%.

Figure 1

Distribution of percent time in therapeutic range on warfarin among newly diagnosed atrial fibrillation patients within the first year of starting warfarin.

Development Set

The development set contained 5173 patients with newly diagnosed AF before May 1, 2011, and had a median first‐year TTR of 52% (33, 67). After applying the forward stepwise variable selection algorithm, 32 of the original 85 candidate predictors met development set model inclusion criteria (Table 2). The 10 strongest predictors of TTR in descending rank order according to F statistic magnitude were as follows: (1) any antiarrhythmic; (2) aspirin; (3) valve disease; (4) lung disease; (5) myocardial infarction; (6) anemia; (7) gastrointestinal bleeding; (8) albumin; (9) venous thromboembolism; and (10) stroke—hemorrhagic. The R 2 for this model was 18.3%.

Table 2

Linear Regression Models Predicting Estimated Percentage of Time in Therapeutic Range Among Newly Diagnosed AF Patients Prescribed Warfarin

Variable	Regression Coefficients: Development Set	Regression Coefficients: Validation Set	Regression Coefficients: Total Cohort
Intercept	87.5	63.4	69.5
Age, y
≤56	−5.6 (−8.6, −2.5)	−5.1 (−9.3, −0.9)	−6.5 (−8.9, −4.2)
57 to 65	−0.6 (−3.3, 2.2)	−3.5 (−7.1, 0.1)	−2.3 (−4.4, −0.2)
66 to 73	−0.6 (−3.1, 1.9)	−1.2 (−4.4, 2.0)	−1.2 (−3.1, 0.7)
74 to 80	−1.0 (−3.4, 1.5)	+0.4 (−2.6, 3.5)	−0.6 (−2.5, 1.3)
81 to 85	−1.7 (−4.4, 0.9)	+0.7 (−2.6, 3.9)	−0.8 (−2.8, 1.2)
>85	0	0	0
	P<0.001	P=0.020	P<0.001
Nonwhite	−4.5 (−11.3, 2.3) P=0.194	+0.1 (−9.4, 9.6) P=0.978	Not in model
Systolic blood pressure, mm Hg
≤100	−3.4 (−5.7, −1.1)	−6.6 (−9.8, −3.4)	−5.5 (−7.4, −3.7)
>100 to 118	−1.4 (−3.4, 0.5)	+1.1 (−2.0, 4.2)	−0.6 (−2.2, 1.0)
>118 to 132	0	0	0
>132 to 148	−1.2 (−2.9, 0.5)	−0.6 (−3.1, 1.9)	−1.3 (−2.7, 0.2)
>148 to 164	−0.2 (−2.2, 1.8)	−0.1 (−3.0, 2.8)	−0.6 (−2.2, 1.0)
>164	−3.8 (−6.2, −1.4)	−3.4 (−6.6, −0.2)	−4.2 (−6.0, −2.3)
	P=0.006	P<0.001	P<0.001
Heart rate, bpm
≤60	−0.4 (−2.7, 1.9)	+0.1 (−3.6, 3.8)	Not in model
>60 to 68	0	0
>68 to 78	−1.5 (−3.5, 0.4)	+0.4 (−2.6, 3.3)
>78 to 88	−2.0 (−3.9, −0.1)	0.0 (−2.9, 2.9)
>88 to 96	−1.3 (−3.8, 1.1)	−1.2 (−4.7, 2.3)
>96	−4.0 (−6.7, −1.4)	−1.6 (−5.2, 2.0)
	P=0.049	P=0.824
Body mass index, kg/m2
≤23	−3.7 (−6.2, −1.1)	−3.7 (−7.3, −0.2)	−3.7 (−5.8, −1.7)
>23 to 26	−3.3 (−5.6, −1.0)	−5.8 (−9.1, −2.5)	−4.6 (−6.5, −2.7)
>26 to 30	−0.7 (−2.7, 1.4)	−2.8 (−5.6, 0.0)	−1.6 (−3.2, 0.0)
>30 to 35	−0.5 (−2.5, 1.5)	−2.5 (−5.3, 0.4)	−1.2 (−2.8, 0.4)
>35 to 41	0	0	0
>41	−0.9 (−3.5, 1.7)	−1.5 (−5.0, 2.0)	−1.2 (−3.3, 0.9)
	P=0.007	P=0.032	P<0.001
Alcohol problem	−3.6 (−8.0, 0.8) P=0.108	−6.8 (−12.3, −1.4) P=0.014	−5.9 (−9.3, −2.5) P<0.001
Anemia	−3.7 (−5.4, −2.1) P<0.001	−5.6 (−7.7, −3.5) P<0.001	−5.0 (−6.2, −3.7) P<0.001
Dementia	−6.4 (−12.5, −0.4) P=0.036	−1.3 (−7.3, 4.7) P=0.675	Not in model
Gastrointestinal bleed	−4.5 (−7.4, −1.6) P=0.002	−2.0 (−5.3, 1.3) P=0.241	Not in model
No liver disease	−4.5 (−8.9, −0.2) P=0.042	+1.9 (−2.7, 6.4) P=0.426	Not in model
Lung disease	−3.8 (−5.4, −2.2) P<0.001	−3.3 (−5.5, −1.0) P=0.004	−4.0 (−5.3, −2.7) P<0.001
Memory loss	−4.5 (−11.2, 2.2) P=0.190	+1.1 (−5.1, 7.3) P=0.728	Not in model
Myocardial infarction	−4.5 (−6.5, −2.6) P<0.001	−0.8 (−3.3, 1.7) P=0.516	Not in model
Stroke hemorrhagic	−13.2 (−23.3, −3.1) P=0.010	−8.4 (−16.1, −0.6) P=0.034	−9.2 (−15.2, −3.1) P=0.003
Thrombocytopenia	−4.1 (−7.9, −0.4) P=0.031	−7.6 (−12.5, −2.7) P=0.002	−5.4 (−8.4, −2.4) P<0.001
Valve disease	−3.9 (−5.5, −2.3) P<0.001	+1.8 (−0.5, 4.1) P=0.123	Not in model
Venous thromboembolism	−4.9 (−8.5, −1.3) P=0.007	−5.3 (−9.7, −0.9) P=0.019	−5.4 (−8.2, −2.6) P<0.001
Any antiarrhythmic	−5.3 (−6.7, −3.8) P<0.001	−3.8 (−5.7, −1.8) P<0.001	−5.0 (−6.2, −3.9) P<0.001
Aspirin	−4.6 (−6.2, −3.0) P<0.001	−5.3 (−7.3, −3.3) P<0.001	−5.2 (−6.5, −4.0) P<0.001
Albumin, g/dL
≤3.2	−6.1 (−8.4, −3.9)	−6.8 (−10.3, −3.2)	−8.2 (−10.2, −6.1)
>3.2 to 3.6	−5.3 (−7.4, −3.2)	−2.1 (−5.1, 1.0)	−5.8 (−7.7, −3.9)
>3.6 to 4.0	−2.5 (−4.3, −0.8)	−0.8 (−3.2, 1.6)	−2.8 (−4.4, −1.1)
>4.0 to 4.3	0	0	0
>4.3 to 4.6	0.0 (−2.1, 2.0)	+0.6 (−2.1, 3.3)	−0.4 (−2.1, 1.2)
>4.6	−0.9 (−3.2, 1.4)	+0.4 (−3.5, 4.3)	−1.0 (−3.1, 1.2)
	P<0.001	P=0.005	P<0.001
Aspartate aminotransferase, U/L
≤16	−3.4 (−5.8, −1.0)	+0.1 (−3.3, 3.5)	Not in model
>16 to 20	−1.7 (−3.7, 0.3)	−0.5 (−3.3, 2.3)
>20 to 25	−0.4 (−2.2, 1.4)	+0.3 (−2.2, 2.9)
>25 to 32	0	0
>32 to 42	−1.6 (−3.7, 0.4)	+0.9 (−2.1, 3.9)
>42	−1.8 (−4.0, 0.5)	−3.9 (−7.4, −0.4)
	P=0.060	P=0.180
Blood urea nitrogen, mg/dL
≤12	−3.2 (−6.1, −0.3)	−0.7 (−4.9, 3.5)	Not in model
>12 to 15	−2.8 (−5.3, −0.3)	−0.6 (−4.2, 3.0)
>15 to 19	−0.3 (−2.5, 1.8)	−0.3 (−3.3, 2.8)
>19 to 25	−0.7 (−2.7, 1.4)	+0.9 (−2.1, 3.9)
>25 to 35	0	0
>35	−0.4 (−3.2, 2.3)	−2.1 (−5.8, 1.7)
	P=0.059	P=0.701
Carbon dioxide, mEq/L
≤24	−4.3 (−7.0, −1.6)	+0.5 (−3.7, 4.7)	Not in model
>24 to 26	−2.0 (−4.6, 0.5)	+2.5 (−1.5, 6.5)
>26 to 28	−2.3 (−4.8, 0.1)	+2.4 (−1.5, 6.3)
>28 to 30	−0.7 (−3.2, 1.7)	+3.3 (−0.6, 7.3)
>30 to 32	−1.3 (−3.9, 1.4)	+ 2.2 (−2.1, 6.5)
>32	0	0
	P=0.013	P=0.398
Glomerular filtration rate, mL/min
≤38	−3.8 (−6.7, −1.0)	−0.5 (−4.0, 3.1)	Not in model
>38 to 52	0	0
>52 to <60	−0.7 (−3.3, 1.8)	−1.9 (−5.2, 1.5)
≥60	−3.0 (−5.0, −0.9)	+0.9 (−1.8, 3.7)
	P=0.003	P=0.279
Glucose, mg/dL
≤85	−3.4 (−6.0, −0.9)	+0.2 (−3.4, 3.7)	Not in model
>85 to 93	−0.9 (−3.2, 1.5)	+2.3 (−1.0, 5.6)
>93 to 106	−1.3 (−3.3, 0.7)	+3.0 (0.2, 5.8)
>106 to 132	−0.7 (−2.8, 1.3)	−0.5 (−3.4, 2.3)
>132 to 173	0	0
>173	−3.3 (−5.9, −0.8)	−0.6 (−4.0, 2.8)
	P=0.032	P=0.048
Hematocrit, %
≤32	−4.5 (−7.8, −1.3)	−1.8 (−7.0, 3.3)	Not in model
>32 to 36	−2.6 (−4.9, −0.2)	−0.4 (−4.0, 3.2)
>36 to 39	−1.9 (−3.8, 0.1)	−0.5 (−3.4, 2.5)
>39 to 43	0	0
>43 to 46	−0.5 (−2.6, 1.6)	0.0 (−3.2, 3.2)
>46	−1.9 (−4.8, 0.9)	+0.9 (−3.5, 5.4)
	P=0.086	P=0.979
Neutrophil, %
≤52	−1.6 (−4.1, 1.0)	−0.7 (−4.0, 2.6)	−1.0 (−3.0, 1.0)
>52 to 59	0	0	0
>59 to 66	−2.4 (−4.5, −0.4)	−1.2 (−4.0, 1.5)	−1.8 (−3.5, −0.2)
>66 to 74	−2.6 (−4.6, −0.6)	−1.4 (−4.2, 1.4)	−2.4 (−4.0, −0.8)
>74 to 81	−2.2 (−4.6, 0.1)	−2.4 (−5.7, 0.9)	−2.9 (−4.8, −1.0)
>81	−4.4 (−7.0, −1.8)	−3.5 (−7.5, 0.6)	−4.9 (−7.0, −2.8)
	P=0.027	P=0.578	P<0.001
Potassium, mEq/L
≤3.7	−3.6 (−6.4, −0.8)	−0.8 (−4.9, 3.4)	Not in model
>3.7 to 4.0	−0.4 (−3.0, 2.1)	+0.9 (−3.0, 4.7)
>4.0 to 4.3	−0.5 (−3.0, 1.9)	+1.1 (−2.5, 4.8)
>4.3 to 4.6	−1.2 (−3.7, 1.3)	+2.8 (−1.0, 6.5)
>4.6 to 4.9	−0.4 (−3.2, 2.3)	+2.2 (−1.9, 6.3)
>4.9	0	0
	P=0.069	P=0.292
RBC count, ×106/mcL
≤3.5	−7.0 (−10.9, −3.0)	−7.1 (−13.4, −0.8)	−9.7 (−12.1, −7.2)
>3.5 to 3.9	−4.4 (−7.8, −1.0)	−4.0 (−9.3, 1.3)	−5.4 (−7.6, −3.3)
>3.9 to 4.4	−2.2 (−5.1, 0.8)	−4.9 (−9.6, −0.2)	−3.3 (−5.2, −1.3)
>4.4 to 4.8	−3.9 (−6.8, −1.0)	+0.2 (−4.3, 4.7)	−2.2 (−4.2, −0.2)
>4.8 to 5.1	−2.3 (−5.0, 0.4)	+0.2 (−3.9, 4.3)	−1.0 (−3.0, 1.1)
>5.1	0	0	0
	P=0.004	P=0.008	P<0.001
Red blood cell distribution width, %
≤12.8	−0.2 (−2.6, 2.2)	+0.5 (−3.0, 3.9)	−0.1 (−2.1, 1.9)
>12.8 to 13.3	0	0	0
>13.3 to 14.1	−0.9 (−2.9, 1.1)	−2.7 (−5.5, 0.2)	−1.8 (−3.4, −0.2)
>14.1 to 15.4	−1.7 (−3.7, 0.4)	−1.7 (−4.6, 1.2)	−2.3 (−4.0, −0.7)
>15.4 to 16.9	−3.7 (−6.1, −1.4)	−3.5 (−6.9, −0.1)	−4.8 (−6.7, −2.8)
>16.9	−6.0 (−8.7, −3.3)	−3.9 (−7.9, 0.1)	−6.8 (−9.0, −4.6)
	P<0.001	P=0.115	P<0.001
Sodium, mmol/L
≤135	−3.5 (−6.5, −0.5)	−1.3 (−6.0, 3.3)	Not in model
>135 to 137	−2.5 (−5.3, 0.4)	−1.4 (−5.8, 3.0)
>137 to 139	−2.6 (−5.3, 0.1)	−2.0 (−6.2, 2.2)
>139 to 141	−1.3 (−3.9, 1.3)	−0.4 (−4.5, 3.8)
>141 to 143	−1.2 (−3.9, 1.6)	−3.3 (−7.8, 1.2)
>143	0	0
	P=0.127	P=0.434
White blood cell count, ×103/mcL
≤5.1	−0.3 (−2.9, 2.2)	−0.8 (−4.3, 2.8)	Not in model
>5.1 to 6.1	0	0
>6.1 to 7.5	−1.2 (−3.2, 0.8)	−0.5 (−3.3, 2.4)
>7.5 to 9.2	−1.2 (−3.2, 0.8)	−1.3 (−4.3, 1.6)
>9.2 to 11.3	−2.9 (−5.3, −0.6)	−1.9 (−5.1, 1.3)
>11.3	−3.1 (−5.7, −0.5)	−1.0 (−4.8, 2.8)
	P=0.084	P=0.876

AF indicates atrial fibrillation; bpm, beats per minute; RBC, red blood cells.

Validation Set

The validation set contained 2704 patients with newly diagnosed AF on or after May 1, 2011, and had a median first‐year TTR of 51% (32, 67). When applying the development set model to the validation set, the R 2 statistic was 19.4%, but 17 of the development set variables did not meet the specified validation criteria (Table 2). The 17 predictors failing to validate were removed, and the remaining 15 predictors were fit in a final model using the entire cohort with development and validation sets recombined. The predictive strength of the TTR predictors in the final model in descending rank order according to F statistic magnitude were as follows: (1) any antiarrhythmic; (2) aspirin; (3) anemia; (4) lung disease; (5) albumin; (6) red blood cell count; (7) venous thromboembolism; (8) thrombocytopenia; (9) alcohol problem; (10) red blood cell distribution width; (11) systolic blood pressure; (12) stroke—hemorrhagic; (13) age; (14) body mass index; and (15) neutrophil percentage (Table 2). All variables included in the final model had P values <0.05, with the highest P value being 0.003 (P values for categorized continuous variables were for differences across all levels). The R 2 statistic for the final model was 15.4%. A user‐friendly format for applying the final model is provided in Table 3. In Table 3, the final model is parameterized so all regression coefficients are negative (≤ −3.0%). Thus, estimated per‐individual TTRs are calculated by subtracting coefficients for the applicable model elements from the overall model intercept (69.5%). As reported, estimated TTR will be <60% if 4 or more poor TTR factors are present, and <50% if 7 or more poor TTR factors are present. When different random imputations for missing data were applied, regression coefficients typically differed by <0.3.

Table 3

Tool for Calculating Expected Time in Therapeutic Range on Warfarin

Continuous Variables	Very Low	Low	Low‐Normal	High‐Normal	High	Very High
Age, y	≤56	57 to 65	66 to 73	74 to 80	81 to 85	>85
	−6.5a	···	···	···	···	···
Systolic blood pressure, mm Hg	≤100	>100 to 118	>118 to 132	>132 to 148	>148 to 164	>164
	−5.5a	···	···	···	···	−4.2a
Body mass index, kg/m2	≤23	>23 to 26	>26 to 30	>30 to 35	>35 to 41	>41
	−3.7a	−4.6a	···	···	···	···
Albumin, g/dL	≤3.2	>3.2 to 3.6	>3.6 to 4.0	>4.0 to 4.3	>4.3 to 4.6	>4.6
	−8.2a	−5.8a	···	···	···	···
Neutrophil, %	≤52	>52 to 59	>59 to 66	>66 to 74	>74 to 81	>81
	···	···	···	···	···	−4.9a
Red blood cell count, ×106/mcL	≤3.5	>3.5 to 3.9	>3.9 to 4.4	>4.4 to 4.8	>4.8 to 5.1	>5.1
	−9.7a	−5.4a	−3.3a	···	···	···
Red blood cell distribution width, %	≤12.8	>12.8 to 13.3	>13.3 to 14.1	>14.1 to 15.4	>15.4 to 16.9	>16.9
	···	···	···	···	−4.8a	−6.8a
Binary variables	Yes	No
Alcohol problem	−5.9a	···
Anemia	−5.0a	···
Lung disease	−4.0a	···
Stroke hemorrhagic	−9.2a	···
Thrombocytopenia	−5.4a	···
Venous thromboembolism	−5.4a	···
Any antiarrhythmic	−5.0a	···
Aspirin	−5.2a	···

Intercept: 69.5%. TTR indicates time in therapeutic range.

Estimated TTR is calculated by subtracting appropriate model elements in “*” from the intercept term.

Comparison to SAMe‐TT2R2 Score

The various models reported above showed modest discrimination for various TTR thresholds (Table 4). C‐statistics were near or above 0.70, and were generally higher as TTR percent thresholds were lowered. When including the SAMe‐TT2R2 score as the sole variable in the various regression models, R 2 statistics ranged from 2.2% to 3.5% (versus 15.4–19.5% in the newly developed models), and c‐statistics were consistently below 0.60 (Table 4).

Table 4

C‐Statistics for Discrimination From Logistic Regression Models for Multiple TTR Cut Points: Geisinger Model Versus SAMe‐TT2R2 Score

TTR Cut Point	Geisinger Model				SAMe‐TT2R2 Score
	Full Model	Development Set	Validation Set	Final Model	Full Model	Development Set	Validation Set	Final Model
TTR 30%	0.753	0.747	0.739	0.714	0.589	0.599	0.567	0.589
TTR 40%	0.729	0.720	0.725	0.697	0.579	0.586	0.566	0.579
TTR 50%	0.723	0.708	0.726	0.690	0.579	0.584	0.565	0.579
TTR 60%	0.714	0.695	0.728	0.679	0.585	0.585	0.583	0.585
TTR 70%	0.718	0.695	0.739	0.679	0.583	0.580	0.589	0.583

SAMe‐TT2R2 indicates sex, age, medical history, treatment, tobacco, race; TTR, time in therapeutic range.

Discussion

The current study developed and validated a clinical prediction model for estimating TTR within the first year following warfarin initiation among previously warfarin‐naïve patients with a new diagnosis of AF. The study was able to consider a more extensive array of candidate predictors than previous studies, including several historical diagnoses and laboratory tests collected during usual clinical care, many of which were found to be associated with TTR. The proposed prediction model demonstrated stronger predictive performance in a validation cohort than a competing model designed with the same intent, and may prove valuable in differentiating those likely to achieve an adequate TTR on warfarin from those who may be more properly anticoagulated with a DOAC. Despite the improved predictive performance, the proposed prediction model only explained a modest amount of variation in TTR.

The efficacy and safety of warfarin is highly dependent on the quality of anticoagulation achieved as measured by the TTR, so among newly diagnosed, warfarin‐naïve AF patients, it may be clinically valuable to identify those individuals likely to have poor TTRs on warfarin so alternative therapies such as DOACs can be applied. Multiple studies have examined predictors of poor TTR, and our study shows many consistencies with, yet extends, previous findings.9, 21, 32, 35, 40, 41, 42, 43, 44 In particular, our study affirms younger age, lower body mass, lung disease, application of a rhythm control treatment strategy, kidney dysfunction, and alcohol problems as predictors of poor TTR.9, 21, 32, 35, 40, 41, 42, 43, 44 Our study advances prior work by considering an extended list of historical diagnoses and several inexpensively attained, commonly measured laboratory tests that serve as markers of potentially salient physiologic features such as kidney function, liver function, inflammation, nutritional status, metabolic derangements, coagulation propensity, red blood cell production and function, volume status, and frailty. Notably, 4 of the final 15 validated predictors were laboratory tests—albumin, neutrophil percentage, red blood cell count, and red blood cell distribution width and, as more detailed descriptors of the ailments they characterize (eg, kidney function), laboratory tests often outpredicted their dichotomous surrogates (eg, history of kidney disease). The limited predictive power provided by the collection of model elements reinforces that anticoagulation control with warfarin is inherently difficult to predict, as even our full model containing all 85 candidate predictors explained just 19.5% of the total variation in first‐year TTR, while our final model explained 15.4% of the variation. Likewise, another TTR prediction model (the SAMe‐TT2R2 score) explained just 10% of TTR variation in its development set while explaining about 3% of TTR variation in the external validation described here.21 Notably, several predictors found associated with poor TTR in the SAMe‐TT2R2 score and other prior studies were not confirmed in our study, including female sex, nonwhite race, and recent smoking status.21, 32, 35, 43, 44, 45

The proposed TTR prediction model allows estimating per‐individual TTR percentage within the first year following newly initiated warfarin among patients with a new AF diagnosis as a means to assist decision making regarding proper OAC selection (warfarin versus DOAC). The study inclusion criteria were designed to identify a set of AF patients for whom the warfarin‐or‐DOAC question was most applicable (no prior warfarin use, nonvalvular AF). Unfortunately, there is no universally accepted definition of nonvalvular AF that could be applied, and indeed, the 4 major DOAC trials applied slightly different definitions in their respective trials.3, 46, 47 A previously developed prediction model, the SAMe‐TT2R2 score, was developed with the same intent, though it did not focus specifically on new AF diagnoses.21, 48 The external validation assessment of the SAMe‐TT2R2 score performed here did not suggest a strong predictive performance in our patient cohort, with the score explaining about 3% of TTR variation, and c‐statistics for discrimination uniformly below 0.60 for multiple TTR cut points. The poor validation may be related to the different patient populations (new AF versus not) and/or study settings (observational study versus clinical trial).

Quantitative application of the prediction model proposed here produces an estimated TTR (from 0% to 100%) for any individual patient. Incorporation of the resulting numerical value into clinical decision making requires careful clinical judgment, as no uniform thresholds for poor or optimal TTR have reached consensus. Stated optimal TTRs have typically been in the >60% to 70% range, whereas poor TTRs are more difficult to pinpoint but may serve as the more salient decision point in the clinical environment.5, 12, 49 TTRs as low as 40% to 50% have been suggested as possible “poor” thresholds, as some studies have shown at least some benefit with warfarin compared with no OAC above this threshold, but not below, with improving rates of efficacy and safety as TTR rises.9, 19, 31 Prediction model estimates below these values likely warrant more serious consideration of a DOAC. Of note, our study population consists of warfarin‐naïve patients at the time of AF diagnosis, and TTRs are known to increase over time among new warfarin users as experience grows and dosing is optimized.32, 35, 43, 50 Indeed, the median TTR during the first year following warfarin initiation (51%) was noticeably lower than the median TTR restricted to INR measurements taken after the first year (61%). The 1‐year threshold was chosen as a reasonable time interval to judge the quality of anticoagulation control with newly started warfarin. Notably, our overall median TTR (55%) observed in a real‐world clinical environment is very consistent with previous observational studies but noticeably lower than average TTRs observed in the major DOAC trials.9, 30, 31, 35 Nonetheless, in situations where expected TTRs are systematically higher or lower than observed in the current study, a model recalibration (via an intercept adjustment) may be needed in order to provide the most valid estimate of TTR.

Some limitations of the current study should be noted. The current study, though designed to capture a real‐world cohort of newly diagnosed AF patients, was performed within a single healthcare system serving a predominantly white population, so the proposed prediction model should be externally validated at other institutions. Though the proposed decision tool was designed to be applied in a situation of clinical equipoise between warfarin and DOACs, we acknowledge that certain patient characteristics may favor one form of OAC over another despite lack of acknowledgment in published guidelines. Furthermore, any predictor of poor TTR related to warfarin compliance (eg, history of an alcohol problem) could also be related to DOAC compliance. Thus, it must be stressed that prediction model output should only be 1 part of the decision making process in OAC selection. EMR‐derived variables are prone to misclassification and measurement error, but our study likely reflects the extent of data quality and completeness prevalent in real clinical practice where the proposed model is most applicable. Missing vital sign and laboratory data are not missing at random—missing implies healthier—thus, usual imputation techniques were not applied.37, 38 The imputation method was intentionally chosen to be a conservative approach, more likely to attenuate effect estimates. Other data analysis tactics could have been used; however, we emphasize that the final prediction model only endorses data patterns independently observed in both the development and validation sets, which lend credibility to their veracity. Finally, our study could not consider those patients who were OAC‐eligible but not prescribed warfarin, or those warfarin users with fewer than 4 INR measurements for whom an appropriate TTR could not be calculated.

In conclusion, our proposed prediction model consists of several validated predictors of first‐year TTR on warfarin that can be measured easily and inexpensively in the clinical setting, enabling individual‐level estimation of the expected TTR among patients with newly diagnosed AF. Together with other relevant clinical factors, application of the proposed model may assist decision making regarding the proper mode of OAC in this growing patient population. However, TTR prediction remains challenging, and future studies should attempt to find additional predictors that explain a greater proportion of TTR variance.

Sources of Funding

Funding for this study was provided by Daiichi Sankyo.

Disclosures

Williams receives research funding from Merck, Roche, Biosense Webster, Gilead, Boehringer Ingelheim, Edwards Lifesciences, and Janssen. Evans, Honushefsky, and Berger have no disclosures to report.