Prediction Score for Anticoagulation Control Quality Among Older Adults.

BACKGROUND: Time in the therapeutic range (TTR) is associated with the effectiveness and safety of vitamin K antagonist (VKA) therapy. To optimize prescribing of VKA, we aimed to develop and validate a prediction model for TTR in older adults taking VKA for nonvalvular atrial fibrillation and venous thromboembolism. METHODS AND
RESULTS: The study cohort comprised patients aged ≥65 years who were taking VKA for atrial fibrillation or venous thromboembolism and who were identified in the 2 US electronic health record databases linked with Medicare claims data from 2007 through 2014. With the predictors identified from a systematic review and clinical knowledge, we built a prediction model for TTR, using one electronic health record system as the training set and the other as the validation set. We compared the performance of the new models to that of a published prediction score for TTR, SAMe-TT2R2. Based on 1663 patients in the training set and 1181 in the validation set, our optimized score included 42 variables and the simplified model included 7 variables, abbreviated as PROSPER (Pneumonia, Renal dysfunction, Oozing blood [prior bleeding], Staying in hospital ≥7 days, Pain medication use, no Enhanced [structured] anticoagulation services, Rx for antibiotics). The PROSPER score outperformed SAMe-TT2R2 when predicting both TTR ≥70% (area under the receiver operating characteristic curve 0.67 versus 0.55) and the thromboembolic and bleeding outcomes (area under the receiver operating characteristic curve 0.62 versus 0.52).
CONCLUSIONS: Our geriatric TTR score can be used as a clinical decision aid to select appropriate candidates to receive VKA therapy and as a research tool to address confounding and treatment effect heterogeneity by anticoagulation quality.

Clinical Perspective

What Is New?

In patients aged ≥65 years, our prediction model for anticoagulation control quality outperformed the published score, SAMe‐TT2R2.

Time in the therapeutic range was ≥70% (area under receiver operating characteristic curve 0.71 versus 0.57, a significant difference.).

What Are the Clinical Implications?

Our prediction score for anticoagulation quality can help clinicians select the appropriate older adult candidates to receive vitamin K antagonist therapy and can provide researchers with a tool to adjust for confounding and to investigate treatment effect heterogeneity due to predicted anticoagulation quality.

Introduction

Vitamin K antagonist (VKA; eg, warfarin) therapy is an effective anticoagulation option for stroke prevention in patients with nonvalvular atrial fibrillation (AF) and for treatment and secondary prevention of venous thromboembolism (VTE; including deep vein thrombosis and pulmonary embolism).1, 2, 3 The safety and effectiveness of VKAs, however, depends on regular international normalized ratio (INR) monitoring and anticoagulation control quality, often measured by the time in therapeutic range (TTR), for which INR 2.0 to 3.0 is the standard therapeutic range for AF and VTE.4, 5, 6 Patients on VKA with poor anticoagulation quality (ie, low TTR) have been shown to have a higher risk of thromboembolic and bleeding complications and thus a worse risk–benefit ratio.4, 7, 8

Although clinical trials have shown that direct‐acting oral anticoagulants (DOACs) are therapeutically advantageous over or at least noninferior to VKAs,9, 10, 11 clinical equipoise still exists when patients are likely to have good anticoagulation control based on pretreatment characteristics.7, 12 This choice is particularly difficult to make in older adults because DOACs have been associated with a higher risk of major gastrointestinal bleeding than VKAs in the older population.13, 14, 15 Moreover, chronic kidney disease is highly prevalent in older adults,16 which makes lack of routine monitoring tests for DOACs a challenge rather than an advantage because some DOACs are substantially renally excreted (eg, 80% for dabigatran). Consequently, it is critical to understand how patient characteristics are associated with anticoagulation quality so we can identify the ideal candidates for VKA therapy.

In the existing literature, there is only 1 published prediction score for anticoagulation quality: the SAMe‐TT2R2 score.17 It did not consider some clinically important predictors for TTR (eg, polypharmacy, hospitalizations, antibiotic use)18, 19, 20, 21, 22 and was found to have suboptimal performance in external validation populations (area under receiver operating characteristic curve [AUC] for relevant clinical end points <0.6).23, 24, 25 In addition, although the majority of oral anticoagulant users are older adults,3, 18 SAMe‐TT2R2 was developed with 52.7% of the population aged <70 years. Because comorbidity profiles vary substantially by age, the generalizability and applicability of SAMe‐TT2R2 in the older population is unclear.

We aimed to develop and validate a new prediction model for TTR, particularly in patients aged ≥65 years taking VKA for nonvalvular AF or VTE. Because prior studies found that the TTR predictors identified in AF patients were similar to those in VTE patients,18 for clinical simplicity we developed 1 score for both indications but validated the performance in patients with nonvalvular AF and VTE separately.

Methods

Data Source

We linked electronic health record (EHR) data from 2 large US academic provider networks with Medicare claims data. The first network consists of 1 tertiary hospital, 2 community hospitals, 17 primary care centers, and 1 anticoagulation clinic that manages VKA‐related care for all patients within the network. The second network includes 1 tertiary hospital, 1 community hospital, 16 primary care centers, and an anticoagulation clinic. Patients in network 1 were used as the training set for the prediction model derivation, and those in network 2 were used as the validation set. The EHR database contains information on patient demographics, diagnosis and procedure codes, medications, lifestyle factors, laboratory data, and various clinical notes. Both inpatient and outpatient EHR data were used in this study. The Medicare claims data contain information on demographics, inpatient and outpatient diagnosis and procedure codes, and dispensed medications.26 This study was approved by Partners HealthCare Institutional Review Board (IRB).

Study Population

In the linked Medicare claims–EHR data, we identified all patients aged ≥65 years with nonvalvular AF or VTE initiating a VKA from January 1, 2007, to December 31, 2014, with no use of any oral anticoagulants (VKAs or DOACs) in the prior 90 days (new user design27). The VKA initiation date was the index (cohort entry) date. The study cohort was required to have at least 180 days of continuous enrollment in Medicare inpatient, outpatient, and prescription benefits with at least 1 EHR encounter with date of service after January 1, 2007, and before the index date. To ensure our ability to assess the primary outcome reliably, patients were required to have at least 5 INR values recorded in the system. To assess whether this requirement would select an unrepresentative cohort, we compared the distributions of the combined comorbidity score28 in those with versus without at least 5 INRs. We computed standardized differences between proportions of each combined comorbidity score category in those with versus without 5 INRs. A standardized difference of <0.1 was used to indicate an acceptable discrepancy.29

Outcome Definition of the Anticoagulation Control

We calculated TTR using Rosendaal's method,30 which assigns an INR value to each day by linear interpolation of successive observed INR values with gaps <56 days. After interpolation, we computed the proportions of time that fell within the therapeutic range (INR 2.0–3.0). We ascertained TTR starting the 29th day after the index date until the earliest of the following: 12 months after the index date, lack of INRs with a gap ≤56 days, death, discontinuation of VKA, or study end (December 31, 2014). We did not assess TTR for the first month because variability of INR values in the first month generally reflects expected fluctuations in INRs during the titration phase of VKA therapy. VKA discontinuation was defined based on an algorithm validated in a prior study in which high agreement with actual VKA use was demonstrated by chart review (κ=0.84).31

Candidate Predictors and Building of the TTR Prediction Model

We conducted a systematic review to identify original articles reporting predictors of anticoagulation control quality (assessed by TTR or INR variability) in users of VKAs, after multivariate adjustment. Figure 1 summarizes the search terms and the selection process. The significant predictors reported by the selected articles, along with variables deemed clinically important to predict anticoagulation quality, were used as the candidate predictors to build our prediction model. Based on these variables, we built a model predicting continuous TTR by Lasso regression with 5‐fold cross‐validation, using the data in the training set.32 We referred to the predicted TTR derived from this model as the geriatric TTR score. To build a simplified model for clinical use, we excluded biophysiologic variables requiring additional testing and used Lasso regression with a Bayesian information criterion, which tends to generate a more parsimonious model than do other criteria.33 The points of the scoring system were the nearest integer proportional to the unstandardized coefficient in this simplified model. All predictors were assessed in the 180 days before (and including) the initiation of a VKA, with the exception of receiving structured anticoagulation management service, which was assessed until 28 days after VKA initiation (immediately before the start of the follow‐up).

Figure 1

Systematic review on significant predictors for anticoagulation quality (TTR). AF indicates atrial fibrillation; INR, international normalized ratio; TTR, time in therapeutic range; VKA, vitamin K antagonist; VTE, venous thromboembolism.

Performance of the Geriatric TTR Score Versus SAMe‐TT2R2

We calculated a coefficient‐based and simplified version of the SAMe‐TT2R2 score for each patient (see Table S1 for details).17 Model performance was compared (1) between the geriatric TTR score and the coefficient‐based SAMe‐TT2R2 score and (2) between the simplified point system of the geriatric TTR score and the simple SAMe‐TT2R2 score. The validation set was subdivided into AF and VTE populations. We computed the AUC when predicting TTR >70%, a cutoff to indicate good anticoagulation quality in the literature.34, 35 We then computed the AUC when predicting incidence of a composite clinical outcome of stroke, systemic arterial embolism, VTE, and major bleeding (see detailed definitions in Table S2) that occurred between the 29th and 365th days following the index date (ie, the same ascertainment period as TTR). We also evaluated thromboembolic and bleeding events separately. In addition, we computed Hosmer–Lemeshow goodness‐of‐fit statistics to assess calibration of the models. The hypothesis testing for AUC comparison was done with methods proposed by DeLong et al.36

Missing Data

The information on smoking and body mass index was recorded in the study EHRs as both structured data and text in the clinical notes. To reduce missing data, we used natural language processing37 to extract information on these 2 variables from the clinical notes; this approach reduced the proportion of patients missing smoking data from 54.4% to 7.8% and of those missing body mass index from 38.5% to 32.2% (see Data S1 for details). For those still missing smoking and body mass index information after natural language processing and with other variables with missing data, we used the missing indicator method in the analysis.

Statistical Analyses

First, we tested the sensitivity of our results to the length of the baseline assessment period (365 instead of 180 days) and the definition of the new initiator of VKA (no use of VKA in the 180 days instead of 90 days before the index date). We calculated the Spearman correlation coefficient between the new scores based on revised strategies and the original score to quantify discrepancies. Second, to evaluate whether our results were sensitive to outliers or skewed the distribution of TTR, we repeated our analyses after (1) Box–Cox transformation of TTR38 and (2) exclusion of those with extreme outcomes (TTR=0) from the analysis. The statistical analyses were conducted with SAS 9.4 (SAS Institute Inc).

Results

Systematic Review

From a total of 3692 studies, we selected 16 articles (Figure 1 summarizes the search and selection process). Among them, 11 articles investigated patients taking VKA for nonvalvular AF,17, 18, 20, 22, 39, 40, 41, 42, 43, 44, 45 4 for mixed indications19, 21, 46, 47 and 2 for VTE.18, 48 Based on these selected articles, we identified 8 positive and 42 negative predictors (Table 1). To enrich the candidate predictor pool, we added 23 variables based on clinical knowledge (see Table S3 for the full list of candidate predictors and their definitions).

Table 1

Significant Predictors for Anticoagulation Control Quality From a Systematic Review

First Author	Study Period	Sample Size	Region	Mean Age, y	Female, %	Indication	Significant Positive Predictors	Significant Negative Predictors
Boulanger39	1998–2003	13709	USA	67	43	NVAF	Male sex	CHF, DM, residing in Northeast vs Midwest or West
Apostolakis17	1995–2001	1061	North America	69	41	NVAF	Use of β blocker, verapamil vs amiodarone, age >50 y, male sex	Ethnic minority, smoking (within 2 y), comorbidities defined as >2 of the following: hypertension, DM, coronary artery disease/myocardial infarction, PVD, CHF, previous stroke, pulmonary disease, and hepatic or renal disease
Tomita40	2011–2012	163	Japan	74.4	38	NVAF	Male sex	CHF
Dlott20	2007–2008	138319	USA	74	49	NVAF	Male sex, age 55–84.9 vs ≥85 y	Length of INR testing period, age <55 vs ≥85 y, physicians with lower case load, lower median income range, geographic region in the United States
Kim22	2006–2008	62156	USA	72.2–73.4	…	NVAF	…	CHF, AST >80 U/L, Alkaline phosphatase >150 U/L, sodium <140 mEq/L use of metolazone, and hospitalization for CHF
Kose41	2011–2013	55	Japan	67.8	25	NVAF	…	CHF
Macedo18	2000–2013	140078	UK	73.5	44	NVAF	Lipid‐lowering drugs, older age	Low/normal BMI (<25), smoking, having acute respiratory infections, chronic lung disease (COPD, asthma), DM, epilepsy; use of pain medications, and number of hospitalizations
Nelson42	2006–2010	9433	USA	72.6	…	NVAF	Male sex, age >75 y, hypertension	CHF and DM
Pignatelli43	2008–2013	553	Italy	72.9	40	NVAF	Use of ACEI/ARB	DM
White44	2009–2013	290	USA	70–72	44	NVAF	Older age, male sex	…
Yong45	2003–2012	184161	USA	…	100	NVAF	…	Nonwhite race
Kooistra48	2007–2011	3825	Multination	…	46	VTE	…	Underweight, active cancer at baseline, secondary VTE, INR <2.0 at stop of bridging therapy
Macedo18	2000–2013	70371	UK	65.4	52	VTE	Older age, male sex	Lower BMI, smoking, cancer, chronic use of pain medication, chronic lung disease (COPD or asthma), dementia, DM, epilepsy
Rose46	2006–2008	124619	USA	…	3 (VA population)	Mixed	For TTR in the first 6 mo: hyperlipidemia, older age; for TTR after the first 6 mo: hyperlipidemia, hypertension, older age, male sex	For TTR in the first 6 mo: race, living in a poor area, driving distance (weak association), cancer, DM, CKD, CAD, alcohol abuse, bipolar disorder, substance abuse, dementia, polypharmacy, number of hospitalizations; for TTR after the first 6 mo: nonwhite race, living in a poor area, duration on VKA, cancer, liver disease, epilepsy, CKD, DM, chronic lung disease, CAD, PVD, and heart failure, alcohol abuse, dementia, substance abuse, major depression, and bipolar disorder, number of concomitant medication use, number of hospitalizations
Efird21	2007–2008	1763	USA	65.7–70.7	1.6 (VA population)	Mixed	…	Chronic liver disease, increased levels of AST and creatinine, lower levels of albumin
Nilsson47	1996–2012	2068	Denmark	Female (49.4); male (55.2)	34	Mixed	Male sex
Paradise19	2007–2008	28216	USA	…	3 (VA population)	Mixed	…	Bipolar, depression, psychotic disorders

ACEI indicates angiotensin‐converting enzyme inhibitor; ARB, angiotensin II receptor blocker; AST, aspartate aminotransferase; BMI, body mass index; CAD, coronary artery disease; CHF, congestive heart failure; CKD, chronic kidney disease; COPD, chronic obstructive pulmonary disease; DM, diabetes mellitus; INR, international normalized ratio; NVAF, nonvalvular atrial fibrillation; PVD, peripheral vascular disease; TTR, time in therapeutic range; VA, Veterans Administration; VKA, vitamin K antagonist; VTE, venous thromboembolism.

Patient Characteristics of the Study Population

Among 14250 VKA new initiators with at least 180 days of Medicare enrollment and 1 EHR encounter in the study system, we selected the study cohort with at least 5 INRs recorded in our database, including 1663 patients in the training set, 694 in the AF validation set, and 487 in the VTE validation set. The distribution of combined comorbidity score was similar in patients with versus without 5 INRs with a mean standardized difference of 0.02 between proportions of all combined comorbidity score categories in those included versus excluded (Figure S1). The mean TTR was 0.47 to 0.56 in our training and validation sets (Table 2 and Figure S2). We observed modest differences in some predictors in AF versus VTE validation populations (eg, higher prevalence of cancer and more use of antibiotics in the VTE; Table 2).

Table 2

Patient Characteristics of the Study Populations

Continuous variables	Training Set (n=1663), mean (SD)	AF Validation Set (n=694), mean (SD)	VTE Validation Set (n=487), mean (SD)
Age, y	77.0 (7.5)	76.5 (7.2)	75.6 (7.1)
Anticoagulation management service participation, %a	0.41 (0.49)	0.55 (0.50)	0.48 (0.50)
TTRb, INR 2–3	0.56 (0.25)	0.53 (0.27)	0.47 (0.28)
Percentage of time below range	0.32 (0.26)	0.38 (0.30)	0.43 (0.32)
Percentage of time above range	0.12 (0.14)	0.09 (0.12)	0.09 (0.13)
Mean follow‐up time to assess INR, d	179.3 (121.7)	174.5 (131.0)	136.3 (117.9)

ACEI indicates angiotensin‐converting enzyme inhibitor; AF, atrial fibrillation; ALP, alkaline phosphatase; BMI, body mass index; CHF, congestive heart failure; eGFR, estimated glomerular filtration rate; INR, international normalized ratio; PSA, prostate‐specific antigen; TTR, time in therapeutic range; VKA, vitamin K antagonist; VTE, venous thromboembolism.

Percentage of patients participating in a dedicated anticoagulation management service.

See the distribution of TTR in training and validation sets in Figure S2.

Based on ZIP codes of residence.

Including all the major upper and lower gastrointestinal, and other extracranial bleeding events.

In the 180 days before VKA initiation.

At the time of VKA initiation.

Prediction Models for TTR

From 50 predictors identified in the systematic review and 23 additional variables, we built the new geriatric TTR score with 42 predictors through lasso regression (R 2=0.19; Table 3). The simplified model included a total of 7 variables (R 2=0.14). We summarized these variables using the acronym PROSPER (Pneumonia, Renal dysfunction, Oozing blood [prior bleeding], Staying in hospital ≥7 days, use of Pain medications, lack of Enhanced [dedicated and structured] anticoagulation care, Rx for antibiotics; see Table 4 and Table S3 for detailed definitions). There was no significant difference between the AUCs of PROSPER versus the full geriatric TTR model predicting TTR >70% in the validation set (AUC 0.678 versus 0.680, P=0.86 for difference). The 2 most influential predictors of TTR were lack of participation in a dedicated anticoagulation management service (assigned 4 points) and renal dysfunction (assigned 2 points). The rest of the variables were assigned 1 point each.

Table 3

The Geriatric TTR Prediction Model for Anticoagulation Control Qualitya

Predictor	Coefficient (SE)b
Intercept	0.583 (0.030)
AF vs VTE	0.010 (0.013)
Dedicated anticoagulation management service: yes vs no	0.105 (0.014)
Sex, female vs male	−0.016 (0.014)
Black race	−0.046 (0.036)
Nonblack, nonwhite race	0.036 (0.026)
White	Ref
Limited English proficiency	−0.033 (0.021)
Income: below medianc	−0.009 (0.014)
Income: missingc	0.018 (0.047)
Income: median or higherc	Ref
Living 10–20 miles from facilityd	0.016 (0.018)
Living 5–10 miles from facilityd	0.026 (0.016)
Living >20 miles from facilityd	−0.041 (0.017)
Living <5 miles from facilityd	Ref
BMI 25–29.9	0.059 (0.020)
BMI 30–34.9	0.027 (0.021)
BMI 35–39.9	0.065 (0.028)
BMI <18.5	−0.057 (0.050)
BMI ≥40	0.049 (0.029)
BMI missing	0.035 (0.019)
BMI 18.5–24.9	Ref
CHF	−0.019 (0.015)
Epilepsy	−0.016 (0.027)
Cancer	−0.026 (0.012)
Renal dysfunction	−0.043 (0.015)
Prior bleedinge	−0.021 (0.015)
Pneumonia	−0.016 (0.016)
Drug abuse	−0.073 (0.056)
Chronic liver disease	−0.025 (0.021)
Psychosis	−0.022 (0.024)
Hyperlipidemia	0.025 (0.014)
No. of regular medications, 5–9	−0.029 (0.014)
No. of regular medications, ≥10	−0.036 (0.020)
No. of regular medications, <5	Ref
Hospitalization d ≥7 d: yes vs no	−0.001 (0.019)
No. of hospitalizations ≥2: yes vs no	−0.001 (0.018)
Albumin 2.5–3.49 g/dL	−0.014 (0.017)
Albumin <2.5 g/dL	−0.075 (0.035)
Albumin missing	0.020 (0.038)
Albumin ≥3.5 g/dL	Ref
ALP >150 U/L	−0.068 (0.029)
ALP missing	−0.014 (0.037)
ALP ≤150 U/L	Ref
Sodium ≤130 mmol/L	0.017 (0.056)
Sodium missing	−0.078 (0.027)
Sodium >130 mmol/L	Ref
eGFR 15–29.9 mL/min/1.73m2	−0.038 (0.036)
eGFR 30–59.9 mL/min/1.73m2	−0.003 (0.015)
eGFR <15 mL/min/1.73m2	−0.070 (0.029)
eGFR missing	−0.017 (0.023)
eGFR ≥60 mL/min/1.73m2	Ref
Peripheral vascular disease	−0.016 (0.017)
Use of β blocker	0.028 (0.013)
Use of ACEI	0.018 (0.012)
Use of metolazone	−0.075 (0.089)
Use of opioids	−0.013 (0.016)
Use of statins	−0.027 (0.014)
Use of acetaminophen	−0.024 (0.016)
Use of antibiotics	−0.021 (0.013)
Use of antiplatelet agents	−0.027 (0.015)
Use of oral steroids	−0.012 (0.017)
Influenza vaccine	0.020 (0.012)
PSA test	0.037 (0.018)
Mammography	0.061 (0.022)
Pap smear	−0.048 (0.037)
Falls	−0.021 (0.021)
Fractures	−0.013 (0.021)
Parkinson disease	0.097 (0.043)

ACEI indicates angiotensin‐converting enzyme inhibitors; AF, atrial fibrillation; ALP, alkaline phosphatase; BMI, body mass index; CHF, congestive heart failure; eGFR, estimated glomerular filtration rate; PSA, prostate‐specific antigen; TTR, time in therapeutic range; VTE, venous thromboembolism.

Quantified by international normalized ratio (INR) time in therapeutic range (TTR), see the distribution of TTR in training and validation sets in Figure S2.

Unstandardized coefficients based on a lasso regression model including all the variables listed in this table.

Based on the mean income level of the ZIP code the patients resided in.

Average distance based on ZIP codes from the nearest facility in the network.

Including all the major upper and lower gastrointestinal, and other extracranial bleeding events.

Table 4

Simplified Geriatric Prediction Score for Anticoagulation Control Qualitya: PROSPER

Predictor	Coefficient (SE)b	Point
Intercept	0.719 (0.012)	···
Pneumonia	−0.030 (0.015)	1
Renal dysfunctionc	−0.068 (0.013)	2
Oozing blood (bleeding history)	−0.026 (0.015)	1
Staying in hospital ≥7 d	−0.029 (0.015)	1
Pain medications	−0.037 (0.013)	1
No Enhanced anticoagulation cared	−0.122 (0.012)	4
Rx for antibiotics	−0.030 (0.013)	1

All variables should be assessed in the 6 mo before initiating a VKA, except for no enhanced anticoagulation care, which was assessed at the time of initiation. PROSPER indicates Pneumonia, Renal dysfunction, Oozing blood [prior bleeding], Staying in hospital ≥7 days, use of Pain medications, lack of Enhanced [dedicated and structured] anticoagulation care, Rx for antibiotics; VKA, vitamin K antagonist.

Quantified by international normalized ratio time in therapeutic range (TTR).

Unstandardized coefficients based on a model selected based on a Bayesian information criterion.

Renal dysfunction was defined as having records for acute kidney injury, chronic kidney disease, or end‐stage kidney disease in the prior 180 days.

Lack of participation (no access or plan) in a dedicated anticoagulation management service when initiating a VKA.

Comparison of Performance: SAMe‐TT2R2 Versus Geriatric TTR Score

In the training set, the AUC for the geriatric TTR score predicting TTR >70% (AUC=0.71) was substantially larger than that for coefficient‐based SAMe‐TT2R2 (AUC=0.57, P<0.001 for difference); the AUC for the geriatric TTR score predicting the primary clinical outcome (AUC=0.65) was significantly larger than that for SAMe‐TT2R2 (AUC=0.53, P<0.001 for difference). The results were similar in the validation set (Figure 2). This pattern was consistent when the validation set was subdivided into AF and VTE validation sets (Table 5). We also found similar findings when the composite clinical outcome was subdivided into thromboembolic versus bleeding outcomes (Table S4). The Hosmer–Lemeshow goodness‐of‐fit test for predicting TTR >70% confirmed good calibration for the both the full geriatric model and coefficient‐based SAMe‐TT2R2 in the training and validation sets (Table S5).

Figure 2

Comparison of AUC: new geriatric score superior to SAMe‐TT. A, Predicting TTR >70%. B, Predicting clinical outcomes. *Composite outcomes of incident stroke, systemic embolism, VTE, and major bleeding events. AF indicates atrial fibrillation; AUC, area under the receiver operating characteristic curve; INR, international normalized ratio; TTR, time in therapeutic range; VTE, venous thromboembolism.

Table 5

Comparison of Model Performance of Original SAMe‐TT2R2 and Geriatric TTR Score

	Optimized Prediction Models			Simplified Prediction Models
	SAMe‐TT2R2 a AUC (95% CI)	Geriatric TTR Score AUC (95% CI)	P for Difference	SAMe‐TT2R2 b AUC (95% CI)	PROSPERc AUC (95% CI)	P for Difference
Prediction TTR >70%, training set	0.57 (0.54–0.59)	0.71 (0.68–0.73)	<0.001	0.55 (0.52–0.58)	0.67 (0.64–0.69)	<0.0001
Prediction TTR >70%, AF validation set	0.57 (0.52–0.61)	0.66 (0.62–0.70)	0.0011	0.58 (0.53–0.62)	0.67 (0.62–0.71)	0.0016
Prediction TTR >70%, VTE validation set	0.57 (0.51–0.63)	0.74 (0.69–0.79)	<0.001	0.59 (0.54–0.65)	0.71 (0.66–0.77)	0.0003
Prediction clinical outcomes,d training set	0.53 (0.49–0.56)	0.65 (0.62–0.69)	<0.001	0.52 (0.49–0.56)	0.62 (0.58–0.66)	<0.0001
Prediction clinical outcomes,d AF validation set	0.60 (0.54–0.66)	0.74 (0.69–0.79)	<0.001	0.60 (0.55–0.66)	0.73 (0.68–0.77)	<0.0001
Prediction clinical outcomes,d VTE validation set	0.57 (0.51–0.63)	0.67 (0.61–0.72)	0.01	0.59 (0.53–0.65)	0.65 (0.60–0.71)	0.098

AF indicates atrial fibrillation; AUC, area under receiver operating characteristic curve; CI, confidence interval; PROSPER, Pneumonia, Renal dysfunction, Oozing blood [prior bleeding], Staying in hospital ≥7 days, use of Pain medications, lack of Enhanced [dedicated and structured] anticoagulation care, Rx for antibiotics; TTR, time in therapeutic range; VTE, venous thromboembolism.

Based on original coefficients.

Simple scoring system of SAMe‐TT2R2.

Simplified geriatric TTR scoring system, see details in Table 4.

Composite outcomes of incident stroke, systemic embolism, VTE, and major bleeding events.

Comparison of Performance: SAMe‐TT2R2 Simple Scoring System Versus PROSPER

In the training set, the AUC for PROSPER predicting TTR >70% (AUC=0.67) was substantially larger than that for SAMe‐TT2R2 (AUC=0.55, P<0.001 for difference); the AUC for PROSPER predicting the primary clinical outcome (AUC=0.62) was significantly larger than that for SAMe‐TT2R2 (AUC=0.52, P<0.001 for difference). A similar pattern was observed in the AF and VTE validation sets when predicting both types of outcomes (Table 5). Patients stratified by PROSPER had a clear decreasing trend of mean TTR, ranging from 0.71 to 0.30, in both the training and validation sets (Table 6).

Table 6

Mean TTR by Simplified New Geriatric Score

Simplified Geriatric Score (PROSPER)a	Training Set (n=1663)		Validation Set (n=1033)
	n (%)	Mean TTR (SD)	n (%)	Mean TTR (SD)
0	154 (9.3)	0.71 (0.17)	106 (10.3)	0.70 (0.18)
1	148 (8.9)	0.67 (0.20)	99 (9.6)	0.63 (0.19)
2	118 (7.1)	0.67 (0.18)	121 (11.7)	0.61 (0.21)
3	79 (4.8)	0.64 (0.18)	78 (7.6)	0.58 (0.23)
4	225 (13.5)	0.59 (0.25)	112 (10.8)	0.56 (0.24)
5	217 (13.0)	0.55 (0.25)	102 (9.9)	0.49 (0.31)
6	202 (12.1)	0.55 (0.25)	115 (11.1)	0.49 (0.28)
7	162 (9.7)	0.52 (0.25)	77 (7.5)	0.34 (0.30)
8	128 (7.7)	0.45 (0.28)	74 (7.2)	0.32 (0.26)
9	104 (6.3)	0.43 (0.27)	58 (5.6)	0.31 (0.28)
10	91 (5.5)	0.41 (0.26)	58 (5.6)	0.43 (0.31)
11	35 (2.1)	0.35 (0.25)	33 (3.2)	0.30 (0.23)

PROSPER indicates Pneumonia, Renal dysfunction, Oozing blood [prior bleeding], Staying in hospital ≥7 days, use of Pain medications, lack of Enhanced [dedicated and structured] anticoagulation care, Rx for antibiotics; TTR, time in therapeutic range.

See details in Table 4.

Sensitivity Analyses

After changing the length of baseline assessment period from 180 to 365 days, the revised prediction score was highly correlated with the original one (Spearman coefficient=0.89). After defining new initiation of VKA as no use in the 180 days rather than 90 days before the index date, the revised prediction score was highly correlated with the original one (Spearman coefficient=0.99). The performance of these revised models was similar to that of the original model (Table S6). After Box–Cox transformation, the distribution of TTR became more symmetric (Fisher‐Pearson skewness coefficient49 reduced by 46%), resulting in a prediction score highly correlated with the predicted value generated by the original model (Spearman coefficient=0.99). Similar patterns were found when excluding those with TTR 0 (Table S6).

Discussion

We developed and validated a new prediction score in the older adult population. Our geriatric TTR score included 42 predictors, and the simplified clinical scoring system, PROSPER, had 7 variables. The geriatric TTR score and PROSPER outperformed the corresponding coefficient‐based and simple version of SAMe‐TT2R2, available for the past 4 years, when predicting TTR ≥70% and thromboembolic and bleeding outcomes for those aged ≥65 years. The performance of PROSPER was not significantly worse than that of the full model in the validation set.

Physicians can use geriatric TTR scores to identify patients with good predicted TTR (>70%) as good candidates for VKA therapy for nonvalvular AF or VTE; otherwise, a DOAC may be preferred unless contraindicated. It is feasible to develop an automated program in an EHR system for computing the predicted TTR based on the full model as a clinical decision support tool; otherwise, PROSPER can be readily calculated without an aid. Our findings suggest that a PROSPER score >2 is predictive of having poor TTR; therefore, initiating a VKA may not be ideal. This cut point is associated with reasonable specificity (75%) for TTR >70% and sensitivity (85%; Table 7) for TTR <50% (another cut point suggested in the literature to indicate poor anticoagulation quality34). Alternatively, the categorization of PROSPER as 0 to 2, 3 to 6, and ≥7 approximately subdivided the population into tertiles that correlated well with TTR. These 3 categories may be used to indicate low, moderate, and high risk of having poor TTR (Table 8). Our work highlights the importance of a structured approach to warfarin management; lack of a dedicated anticoagulation management service was found to be the strongest predictor of poor TTR. This finding is in line with several prior studies in which structured anticoagulation care was shown to improve TTR and to reduce risk of complications.50, 51, 52, 53 In the current era when DOACs are available, unstructured warfarin management is a particularly unattractive treatment option. If DOAC treatment is not possible and a patient has a PROSPER score >2, providers should encourage patient participation in a dedicated anticoagulation management service or some equivalently well‐organized warfarin treatment setting (eg, a practice with a nurse dedicated to managing warfarin). Renal dysfunction, defined as the presence of acute kidney injury, chronic kidney disease, or end‐stage kidney disease in the prior 180 days, was also found to be an important predictor of poor TTR. The anticoagulation decision is particularly difficult in AF patients with renal dysfunction, for whom there is uncertainty as to the net benefit of warfarin or DOACs with poor renal function. One approach can be to favor use of a DOAC that is less renally excreted (eg, apixaban) with necessary dose adjustment.

Table 7

Sensitivity and Specificity in the Validation Set (AF and VTE)

Cutoff of Simplified Geriatric Score (PROSPER)a	TTR >70%		TTR <50%
	Sensitivity (%)	Specificity (%)	Sensitivity (%)	Specificity (%)
0	19.5	93.4	97.7	15.9
1	31.4	84.7	92.6	28.7
2	47.1	74.6	85.2	43.5
3	56.0	67.6	79.4	52.3
4	68.9	57.6	70.5	64.6
5	79.5	48.0	59.2	73.4
6	89.1	36.2	46.6	83.6
7	91.8	26.9	35.5	88.4
8	93.5	17.6	22.7	91.5
9	95.2	10.4	13.0	94.2
10	99.7	4.3	5.8	98.7
11	100.0	0	0	100.0

AF indicates atrial fibrillation; PROSPER, Pneumonia, Renal dysfunction, Oozing blood [prior bleeding], Staying in hospital ≥7 days, use of Pain medications, lack of Enhanced [dedicated and structured] anticoagulation care, Rx for antibiotics; TTR, time in therapeutic range; VTE, venous thromboembolism.

Defining patients with scores lower than or equal to this cutoff as having TTR >70% and those with scores higher than the cut point as having TTR <50%.

Table 8

TTR by the Simplified Geriatric Score Categories

Simplified Geriatric Score (PROSPER)	Training Set		Validation Set (AF and VTE)
	n (%)	Mean (SD)	n (%)	Mean (SD)
0–2	420 (25.3)	0.69 (0.18)	326 (31.6)	0.64 (0.20)
3–6	723 (43.5)	0.57 (0.25)	407 (39.4)	0.52 (0.27)
≥7	520 (31.3)	0.45 (0.27)	300 (29.0)	0.34 (0.28)
Total	1663	0.56 (0.25)	1033	0.51 (0.27)

AF indicates atrial fibrillation; PROSPER, Pneumonia, Renal dysfunction, Oozing blood [prior bleeding], Staying in hospital ≥7 days, use of Pain medications, lack of Enhanced [dedicated and structured] anticoagulation care, Rx for antibiotics; TTR, time in therapeutic range; VTE, venous thromboembolism.

Our score can also be helpful in a research context. First, researchers can evaluate the potential treatment‐effect heterogeneity by levels of predicted anticoagulation quality based on the full model with a computer program. This assessment can provide direct evidence for choosing the ideal candidates for VKA versus DOACs based on the pretreatment characteristics predictive of anticoagulation quality. Next, our score can be used as a proxy adjustment tool for confounding by anticoagulation control quality. This adjustment is otherwise difficult because calculating TTR requires intensive INR recording in the study databases, which are often incomplete or nonexistent. Because some researchers do not have information on biophysiologic variables, we also presented an alternative model excluding these variables that requires additional testing (Table S7). This alternative model had performance similar to the geriatric TTR score (data not shown).

We have demonstrated that the performance of the new geriatric TTR score was clearly superior to that of SAMe‐TT2R2 in the older adult population. The authors of SAMe‐TT2R2 demonstrated good discrimination performance only when predicting those with TTR <5th percentile but not for those with TTR <25th percentile (AUC=0.58).17 However, the latter is closer to the clinical relevant cut points (eg, TTR >70% usually composes about 30% to 40% of the population17, 18, 34). Because the majority of VKA users are older adults3, 18 who are also more vulnerable to developing bleeding complications,54 we developed an alternative score dedicated to patients aged ≥65 years. We built a prediction model for both nonvalvular AF and VTE indications because prior studies found that AF and VTE patients share many risk factors for TTR18 and because including interaction terms with the VKA indications did not materially improve the model in our analysis. We validated the performance of our model in AF and VTE populations separately and found consistent results.

There are some important limitations. We used linked claims EHR data for higher data quality: The claims data provided comprehensive data across care settings, and EHRs provided necessary clinical information to ascertain TTR and important predictors. However, requiring overlap of the 2 databases reduces our sample size substantially and limits our ability to investigate each clinical outcome individually.55 Besides, for biophysiologic variables (eg, body mass index, albumin levels), we had 29% to 34% people with missing data in the relevant period. We handled it by the missing indicator method because not having certain tests done could, by itself, be informative of the general health state, and this approach allows use of these scores even if some variables are not available. As a sensitivity analysis, the scores not including these variables with missing data were highly correlated with the original one and had similar model performance (data not shown). In addition, some of the possible determinants of TTR are not available in our data sets, such as diet information and genetic profiles associated with warfarin pharmacokinetics, which may limit our model performance. Consequently, our prediction scores should be used as an aid, not as the only ground for decision‐making. Next, we chose to build one prediction model for both nonvalvular AF and VTE indications because prior studies found that AF and VTE patients have many common risk factors for poor anticoagulation qualilty.18 We validated the performance of this score in AF and VTE populations separately and found consistent results. Nonetheless, we acknowledge the alternative approach to build separate models for different indications, which may increase specificity at the cost of simplicity and applicability. Last, the predictors identified in our study should not be interpreted as having causal effect on anticoagulation quality because they could be merely the markers or proxies of the real causal factors.

In conclusion, we developed and validated a prediction score for anticoagulation control quality quantified by TTR in the older adult population. It outperformed the published score, SAMe‐TT2R2, in patients aged ≥65 years when predicting TTR as well as thromboembolic and bleeding events. The full model of the geriatric TTR score can be used as an embedded algorithm within an EHR or for a research study. The simplified scoring system, PROSPER, had comparable performance and can be used in daily practice to help choose the best candidates to receive VKA therapy.

Sources of Funding

Lin received a stipend from the Pharmacoepidemiology program in the Department of Epidemiology, Harvard T.H. Chan School of Public Health and Department of Medicine, Brigham and Women's Hospital, Harvard Medical School. Singer was supported by the Eliot B. and Edith C. Shoolman Fund of the Massachusetts General Hospital (Boston, MA). Drs. Lin and Schneeweiss were supported by PCORI grant 282364.5077585.0007 (ARCH/SCILHS).

Disclosures

Oertel has occasionally participated on Advisory Boards for Roche Diagnostics and Alere. Dr. Schneeweiss is consultant to WHISCON, LLC and to Aetion, Inc., a software manufacturer of which he also owns equity. He is principal investigator of investigator‐initiated grants to the Brigham and Women’s Hospital from Bayer, Genentech, and Boehringer Ingelheim unrelated to the topic of this study. The remaining authors have no disclosures to report.

Data S1. Natural language processing to improve missing data.

Table S1. SAMe‐TT2R2 Original Model

Table S2. Definitions of Clinical Outcomes

Table S3. Definitions of Predictors for Anticoagulation Control Quality

Table S4. Sensitivity Analysis: Similar Performance With Composite Versus Specific Outcomes

Table S5. Hosmer–Lemeshow Goodness‐of‐Fit Table of Geriatric TTR Model in the Validation Set. TTR indicates time in therapeutic range.

Table S6. Sensitivity Analysis: Similar Performance With Different Analysis Strategies

Table S7. New Geriatric Prediction Model for Anticoagulation Control Quality* Based on Only Variables Available in an Insurance Claims Database

Figure S1. Similar distribution of combined comorbidity* score in those with vs without 5 INRs in the study EHR system.

Figure S2. The distribution of TTR in the training & validation sets.

Click here for additional data file.