Development and Evaluation of Novel Electronic Medical Record Tools For Avoiding Bleeding After Percutaneous Coronary Intervention.

Background Bleeding remains the most common complication of percutaneous coronary intervention. Guidelines recommend assessing bleeding risk before percutaneous coronary intervention to target use of bleeding avoidance strategies and mitigate bleeding events. Cedars-Sinai Medical Center undertook an initiative to integrate these recommendations into the electronic medical record. Methods and Results The intervention included a voluntary clinical decision alert to assess bleeding risk before percutaneous coronary intervention, a bleeding risk calculator tool based on the NCDR (National Cardiovascular Data Registry) risk prediction model and, when indicated, a second alert to consider 4 bleeding avoidance strategies. We tested for changes in the use of bleeding avoidance strategies and bleeding event rates by comparing procedures performed before versus after implementation of the electronic medical record-based intervention and with versus without use of the bleeding risk calculator tool. Use of radial access increased (47.6% versus 64.8%; P<0.001) and glycoprotein IIb/IIIa inhibitors decreased (12.8% versus 3.17%; P<0.001) from before to after implementation, though risk-adjusted bleeding event rates were stable (odds ratio, 0.82; P=0.164), even for high-risk procedures. Use versus nonuse of the bleeding risk calculator tool was associated with increased radial access and reductions in glycoprotein IIb/IIIa inhibitors, but no change in bleeding events. Conclusions Integrating guideline recommendations into the electronic medical record to promote assessments of bleeding risk and use of bleeding avoidance strategies was feasible and associated with changes in clinical practice. Future work is needed to ensure that bleeding avoidance strategies are not overused among lower-risk patients, and that, for high-risk patients, the potential benefits of elective percutaneous coronary intervention are carefully weighed against the risk of bleeding.

Clinical Perspective

What Is New?

Electronic medical record integration of a bleeding risk calculator tool for patients undergoing percutaneous coronary intervention is feasible and associated with an increase in the use of certain bleeding avoidance strategies, including increased use of radial access and decreased use of glycoprotein IIb/IIIa inhibitors.

What Are the Clinical Implications?

Electronic medical record–based risk stratification tools may be an effective way of providing data at the point of care and informing clinical decision making.

Introduction

Over 600 000 percutaneous coronary interventions (PCIs) are performed annually in the United States for the treatment of coronary artery disease.1 Given the invasive nature of PCI and concurrent need for anticoagulation, bleeding is the most common complication, occurring in more than 1 in 20 patients.2 Bleeding events effectively increase morbidity and mortality risk, offsetting the potential benefits of PCI.3, 4 Multiple factors affect bleeding rates, including age, renal function, anemia, and type and dose of antiplatelet and anticoagulant use, among others.5, 6, 7 To identify at‐risk patients, bleeding risk prediction models have been developed to add prognostic ability beyond clinical assessment alone.8 Accordingly, the American College of Cardiology and American Heart Association guidelines recommend that providers formally assess bleeding risk before PCI9 to guide the use of established bleeding avoidance strategies,10, 11, 12, 13, 14, 15 especially in high‐risk patients.

Despite these guidelines, use of bleeding risk scores is not common, and an understanding of how best to implement bleeding risk models in clinical practice is limited.16 Barriers to guideline adherence are known to be multifactorial, including lack of awareness, attitudes toward guidelines, agreement with specific recommendations, and external challenges to implementing recommendations. With respect to scoring bleeding risk, the complexity of calculating a risk score and the time required to do so are among the most important barriers.17, 18 For instance, the most commonly cited bleeding risk model includes 10 preprocedure variables needed to stratify patients into low‐, intermediate‐, and high‐risk categories.19

Fortunately, expansion of electronic medical record (EMR) systems provides the opportunity to offer tools that enable the rapid calculation of otherwise cumbersome risk models. Thus, EMR‐based tools could facilitate implementation of the American College of Cardiology and American Heart Association guideline recommendation for managing bleeding risk in particular. Our institution, having observed that post‐PCI bleeding event rates were above the national average, thus established a quality improvement (QI) initiative with 3 basic components: (1) An established risk prediction model was used to create a bleeding risk calculator in the EMR; (2) clinical decision alerts were created to notify physicians before PCI if patients were high risk; and (3) a system was created to monitor changes in care processes and outcomes related to PCI and bleeding events.

In the current analysis, we sought to examine the feasibility and effectiveness of this QI initiative. We specifically assessed adherence to the new EMR‐based tool and also performed pre‐post analyses that included 2 outcomes: (1) use of bleeding avoidance strategies and (2) bleeding event rates. Given that attention has traditionally been focused on high‐risk patients, who represent a minority of those undergoing PCI, we stratified analyses of care and outcomes by level of risk for bleeding.

Methods

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Setting and Baseline Conditions

At our institution, Cedars‐Sinai Medical Center (Los Angeles, CA), between 1800 and 2000 PCIs are performed annually, approximately half of which are outpatient procedures. PCIs are performed by both faculty and private practice cardiologists. Our institution participates in and reports all PCIs to the CathPCI Registry, the NCDR (National Cardiovascular Data Registry) for PCI, which is designed to ensure evidence‐based care, improve patient outcomes, and lower costs.20 At baseline, the bleeding event rate at Cedars‐Sinai Medical Center in the 4 quarters preceding this study was 5.8%.

The medical center has a single Epic‐based EMR (Verona, WI) utilized by all providers in the inpatient setting since 2012. The EMR is fully integrated, including for use with documentation, laboratory results, and order entry. Epic supports customizable decision support tools and is the most widely used EMR system in the United States.

QI Initiative

In response to the institutional focus on lowering bleeding events, the Division of Cardiology created a PCI Taskforce Committee, consisting of both faculty and private practice cardiologists. This committee selected the NCDR CathPCI Registry risk prediction model to assess periprocedural bleeding risk for patients undergoing PCI, given the model's relative ease of completion, past retrospective validation in a large national cohort, and widespread acceptance by practitioners.19 The model estimates bleeding risk based on 10 readily available preprocedural clinical variables, stratifying patients as at low (<1% risk of bleeding), intermediate (1–3%), or high risk (>3%) for bleeding (Table 1).19

Table 1

National Cardiovascular Database Registry Bleeding Risk Score

Variable	Points
ST‐segment–elevation myocardial infarction	10
Non‐ST‐segment–elevation myocardial infarction/unstable angina	3
Cardiogenic shock	8
Female sex	6
Previous history of congestive heart failure	5
No previous PCI	4
Current New York Heart Association class IV symptoms	4
Peripheral vascular disease	2
Age 66 to 75 y	2
Age 76 to 85 y	5
Age >85 y	8
Estimated glomerular filtration rate	1 point per 10 units <90

Scoring notes: Assigned points are summed for each patient. A score of ≤7 is low risk, 8 to 17 intermediate risk, and ≥18 high risk for bleeding after percutaneous coronary intervention. Reprinted from Mehta et al19 with permission. Copyright ©2009, Wolters Kluwer Health, Inc. PCI indicates percutaneous coronary intervention.

The PCI Taskforce also felt that recommendations on use of bleeding avoidance strategies for high‐ and intermediate‐risk patients should be integrated into the QI initiative. Committee members reviewed current literature and selected 4 bleeding avoidance strategies to be recommended for patients at intermediate or high bleeding risk: (1) radial access; (2) use of femoral closure device, if femoral access was selected; (3) avoidance of IIb/IIIa inhibitors; and (4) use of bivalirudin as a procedural anticoagulant.10, 11, 12, 13, 14

Working in conjunction with the Division of Cardiology, the Cedars‐Sinai Enterprise Information Systems Division developed an EMR‐based solution consisting of: (1) an alert to assess bleeding risk; (2) an optional bleeding risk calculator (BRC) tool to calculate bleeding risk based on the NCDR risk prediction model; and (3) a clinical decision support (CDS) alert that fires based on the result of the BRC tool.19 To avoid burdening other members of the care team, these EMR functions are only triggered for cardiology practitioners (physicians, fellows, nurse practitioners, and physicians assistants).

At Cedars‐Sinai Medical Center, an order for cardiac catheterization must be placed in the EMR before the procedure. Given that this must occur before all PCIs, placing this order was used as the trigger for the BRC tool, which prompted users to input the required data to calculate a bleeding risk score based on the NCDR risk prediction model. The risk score could be calculated by any member of the cardiology team involved in a patient's care, with the result provided to the attending interventional cardiologist. The alert was not a “hard stop,” allowing providers to skip risk stratification.

The resultant score from the BRC tool subsequently posted to the patient's chart in the EMR (in the title bar and nursing flow sheet). For patients found to have a high‐ or intermediate‐risk score, the CDS alert prompted the user to consider use of the bleeding avoidance strategies (Figure S1). If a user did not use the BRC tool to generate a score, no CDS alert fired for that patient.

Protocol Testing an Implementation

The BRC tool and CDS functions were developed in a test environment in which they were exposed to multiple clinical scenarios. While initially intended as a fully automated system, requiring no user input after an order for PCI was placed, testing identified factors preventing complete automation. Specifically, the lack of a unified EMR across outpatient private practices resulted in missing information or data located in free text rather than discrete data fields. Furthermore, patients presenting emergently lacked basic information, including age. As such, the BRC tool was transitioned to a computer‐assisted calculator, with the provider entering in the required clinical variables that the tool could not extract directly from the EMR.

Prototypes of the EMR‐based BRC tool and CDS alert were presented during 3 successive weekly catheterization conferences before launch. Interventional cardiologists and fellows attend this nonmandatory conference. These presentations reviewed reasons for implementation, how to use the BRC tool to obtain a risk score, and the CDS recommendations for bleeding avoidance strategy use. Changes to the BRC tool were made based on feedback from these presentations. Specifically, the tool was moved to a sidebar on the screen to allow physicians to move through the chart while the BRC tool remained open. The alert appeared until the score was either completed or cancelled by the user. The CDS appeared only for moderate‐ and high‐risk patients and disappeared when the user acknowledged the score.

The BRC tool and CDS alert were launched on February 9, 2016 with a system‐wide upgrade including multiple other changes to the EMR. The use of the BRC tool and bleeding rates were tracked weekly by the Division of Cardiology's Director of Quality and Physician Outreach. Providers with low performance in BRC tool use or high bleeding event rates were notified by e‐mail or in‐person meeting. Both use rates of the BRC tool and bleeding event rates were reported quarterly to the Quality Council.

Evaluation Methods

To evaluate the intervention, we performed 3 prespecified analyses: (1) We examined use of the BRC tool after implementation, including overall and by month; (2) we compared bleeding avoidance strategy use and bleeding event rates before and after implementation of the EMR‐based intervention; and (3) we compared bleeding avoidance strategy use and bleeding event rates after implementation between patients for whom the BRC tool was versus was not used. In the latter 2 analyses, we conducted secondary analyses that stratified patients by risk of bleeding.

Eligible procedures and population

The unit of analysis was an individual PCI procedure and the subsequent 30 days. Eligibility criteria included any percutaneous coronary angioplasty, with or without stent placement, performed by either a faculty or private practice physician. Exclusion criteria included: patient aged <18 years at time of PCI; hemoglobin <8 or >16 g/dL before PCI; unplanned placement on extracorporeal membrane oxygenation before or during PCI; coronary artery bypass grafting during the same hospitalization or within 30 days of PCI; or death on the same day as the PCI. We did not exclude repeat PCIs in the same patient.

Data Sources

We obtained clinical data, including the use of bleeding avoidance strategies, occurrence of bleeding events, and covariates, from internal forms completed as part of Cedars‐Sinai's routine reporting to the NCDR CathPCI Registry (obtained for all PCIs, performed at Cedars‐Sinai Medical Center). We ascertained whether clinicians used the BRC tool before PCI by accessing automated time stamps in the EMR.

Outcome measures

We examined 3 outcomes: (1) use of the BRC tool (yes or no); (2) use of any of 4 specific bleeding avoidance strategies (yes or no); and (3) the occurrence of a bleeding event (yes or no). Use of bleeding avoidance strategies and bleeding event rates were coprimary outcomes.

We defined BRC tool use as PCIs for which EMR time stamps indicated that the tool was completed before the beginning of the PCI procedure. Nonuse included PCIs without a time stamp for the BRC tool or with a time stamp for the procedure that preceded the time stamp for the BRC tool. We used these data to calculate the proportion of PCIs involving use of the BRC tool overall and by study month.

We defined use of 4 widely accepted bleeding avoidance strategies as any of the following: (1) PCI performed by radial access (including if the operator started with radial and later switched to femoral access); (2) use of a femoral closure device, if femoral access was used; (3) avoidance of IIb/IIIa inhibitors; and (4) use of bivalirudin as a procedural anticoagulant.10, 11, 12, 13, 14

Bleeding events were based on 2014 NCDR definitions and included ≥1 of the following: (1) hemoglobin drop of >3 g/dL, relative to preprocedure hemoglobin values; (2) cardiac tamponade; (3) frank internal or external bleeding; (4) red blood cell transfusion; and (5) intracranial hemorrhage. All events occurred within 72 hours of the PCI, except that transfusions could occur up to 30 days later.

Independent variables

For the before versus after analyses, the independent variable was whether the PCI was performed before (September 30, 2014 to February 9, 2016) or after implementation of the intervention (February 9, 2016 to September 30, 2017), irrespective of whether the tool was used in the implementation period. For analyses examining associations between adherence to the tool and care processes and outcomes, the independent variable was use versus nonuse of the BRC tool after implementation (February 9, 2016 to September 30, 2017).

Covariates

Bleeding risk was calculated in the study for all study‐eligible PCIs whether or not he BRC was used before PCI. The analysis used the NCDR risk prediction model to retrospectively calculate bleeding risk (low, intermediate, or high) for each eligible PCI. If ≥1 data elements were missing, we classified the bleeding risk as nonscorable because of missing data.

To explore other possible covariates, we extracted additional clinical variables for each PCI, including the date, patient age, sex, setting (inpatient versus outpatient), preprocedure creatinine, preprocedure hemoglobin and smoking status, as well as the presence of hypertension, dyslipidemia, family history of premature coronary artery disease, previous myocardial infarction, previous heart failure, previous valve surgery, previous PCI, previous coronary artery bypass grafting, if on dialysis, known cerebrovascular disease, peripheral arterial disease, chronic lung disease, and diabetes mellitus. Definitions were those used in the NCDR CathPCI Registry.21 This study was approved by the Cedar‐Sinai institutional review board.

Statistical Analysis

Data are presented as frequency (percentage, %) for categorical variables and mean (±SD) for continuous variables. Correlated data from the same subject were taken into account in all analyses. Patient characteristics were compared for eligible PCIs before versus after implementation of the BRC, as well as for PCIs in the implementation period in which the BRC was used versus not used, with logistic regression models accommodating correlated data using generalized estimating equations, assuming exchangeable correlation between observations on the same patient. Logistic regression models considering correlated data were further used in univariate and multivariable analyses of bleeding events. We performed 2 sets of multivariable analyses of bleeding events with predictor variables, before versus after implementation and use versus nonuse of the BRC tool after implementation. Multivariable analyses were carried out by entering covariates into the model and using a backward variable selection method with an alpha level of removal of 0.1 while a predictor variable was forced into the model. All analyses were done using SAS software (version 9.4; SAS Institute Inc., Cary, NC) with 2‐sided tests and a significant level of 0.05.

Results

Procedures and Population

Among 5277 PCIs during the study period, 4867 met eligibility criteria, including 1965 before and 2902 after implementation of the intervention (Figure 1). During the study period, 494 patients had >1 PCI, including 116 with at least 1 PCI before and 1 after implementation.

Figure 1

Eligibility of PCI procedures and strata of bleeding risk before and after implementation of the quality improvement initiative and use vs nonuse of a bleeding risk calculator (BRC) tool for procedures performed after implementation. CABG indicates coronary artery bypass grafting; ECMO, extracorporeal membrane oxygenation; Hb, hemoglobin; PCI, percutaneous coronary intervention.

Patients who underwent PCI before versus after implementation were similar in regard to bleeding risk (see Table 2), age (69.7±12.3 versus 69.8±12.1 years; P=0.91), male sex (73.5% versus 74.4%; P=0.56), preprocedure creatinine (1.4±1.7 versus 1.5±1.7 mg/dL; P=0.17), preprocedure hemoglobin (12.9±1.9 versus 12.9±1.9 g/dL; P=0.889), and other clinical covariates.

Table 2

Comparison of PCI Procedure and Patient Characteristics Between Procedures Performed Before Versus After Implementation of the Quality Improvement Initiative and Between Procedures Performed After Implementation for Which the BRC Tool Was Used Versus Not Used

	Before vs After Implementation			BRC Tool Use vs Nonuse
Variable	Before Implementation (n=1965)	After Implementation (n=2902)	P Value	Implementation Period, BRC Not Used (n=1622)	Implementation Period, BRC Used (n=1280)	P Value
Bleeding risk category, n (%)a			0.832			<0.001
High	574 (30.1)	817 (29.2)		504 (32.4)	313 (25.3)
Intermediate	982 (51.4)	1448 (51.8)		775 (49.7)	673 (54.3)
Low	355 (18.6)	532 (19.0)		279 (17.9)	253 (20.4)
Age at PCI; mean years (±SD)	69.7 (12.3)	69.8 (12.1)	0.909	70.15 (12.3)	69.26 (11.9)	0.046
Sex; n male (%)	1445 (73.5)	2158 (74.4)	0.560	1197 (73.8)	961 (75.1)	0.437
Preprocedure creatinine, mean (±SD)	1.56 (1.7)	1.49 (1.7)	0.172	1.49 (1.65)	1.49 (1.65)	0.946
Preprocedure hemoglobin, mean (±SD)	12.9 (1.9)	12.9 (1.9)	0.889	12.93 (1.88)	12.79 (1.92)	0.056
Outpatient PCI, n (%)	899 (45.8)	1412 (48.7)	0.057	761 (46.9)	651 (50.9)	0.037
Smoking, n (%)	245 (12.5)	313 (10.8)	0.102	166 (10.2)	147 (11.5)	0.272
Hypertension, n (%)	1612 (82.0)	2520 (86.9)	<0.001	1400 (86.4)	1120 (87.5)	0.365
Dyslipidemia, n (%)	1573 (80.1)	2477 (85.4)	<0.001	1360 (83.9)	1117 (87.3)	0.010
Family history of premature CAD, n (%)	263 (13.4)	286 (9.9)	<0.001	158 (9.75)	128 (10.0)	0.819
Previous MI, n (%)	537 (27.3)	704 (24.3)	0.024	378 (23.3)	326 (25.5)	0.189
Previous heart failure, n (%)	363 (18.5)	570 (19.7)	0.334	335 (20.7)	235 (18.4)	0.118
Previous valve surgery procedure, n (%)	87 (4.4)	164 (5.7)	0.072	108 (6.66)	56 (4.38)	0.008
Previous PCI, n (%)	870 (44.3)	1301 (44.9)	0.700	737 (45.5)	564 (44.1)	0.451
Previous CABG, n (%)	320 (16.3)	438 (15.1)	0.311	240 (14.8)	198 (15.5)	0.621
Currently on dialysis, n (%)	132 (6.72)	174 (6.0)	0.380	93 (5.74)	81 (6.33)	0.512
Cerebrovascular disease, n (%)	242 (12.3)	429 (14.8)	0.022	227 (14.0)	202 (15.8)	0.182
Peripheral arterial disease, n (%)	262 (13.3)	383 (13.2)	0.906	206 (12.7)	177 (13.8)	0.385
Chronic lung disease, n (%)	147 (7.48)	233 (8.03)	0.507	117 (7.22)	116 (9.06)	0.071
Diabetes mellitus, n (%)	725 (36.9)	1213 (41.8)	0.002	670 (41.3)	543 (42.4)	0.556

BRC indicates bleeding risk calculator; CABG, coronary artery bypass grafting; CAD, coronary artery disease; MI, myocardial infarction; PCI, percutaneous coronary intervention.

Comparison of low‐, intermediate‐, and high‐risk groups.

Uptake of the BRC Tool

Overall the BRC tool was used in 1280 PCIs (44.1%; Figure S2). The median rate of use of the BRC tool was 37.5% during the first 6‐month period after implementation, 55.0% in the second 6‐month period, and 40.9% thereafter.

Among PCIs in which the BRC tool was used versus not used, bleeding risk (as calculated by the research team) was actually lower (low risk, 20.4% versus 17.9%; intermediate risk, 54.3% versus 49.7%; high risk, 25.3% versus 32.4%; overall P value<0.001). In addition, more procedures were performed on an outpatient basis (50.9% versus 46.9%; P=0.019), patients had higher rates of dyslipidemia (87.3% versus 83.9%; P=0.010), and a history of previous valve surgery was less common (4.38% versus 6.66%; P=0.008; Table 2).

Before Versus After Implementation

Near universal use of at least 1 bleeding avoidance strategy occurred both before and after implementation (99.7% versus 99.9%; P=0.143). Following introduction of the BRC tool and CDS alert, the use of individual bleeding avoidance strategies changed on univariate analyses, with an increase in radial access (47.6% versus 64.8%; P<0.001) and a decrease in glycoprotein IIb/IIIa inhibitor use (12.8% versus 3.17%; P<0.001). There was no difference in use of femoral closure devices among femoral procedures (95.3% versus 94.5%; P=0.408). Use of bivalirudin declined (1.48% versus 0.59%; P=0.004; Figure 2). Use of radial access and avoidance of IIb/IIIa inhibitors increased across all bleeding risk categories (Figure 3).

Figure 2

Comparison of unadjusted rates of bleeding avoidance strategy use (left) and bleeding events (right) between procedures performed before vs after implementation of the quality improvement initiative (top) and between procedures performed after implementation for which the bleeding risk calculator (BRC) tool was used vs not used (bottom). *Closure device use rate calculated only for cases with femoral access.

Figure 3

Comparison of unadjusted rates of bleeding avoidance strategy use between procedures performed before vs after implementation of the quality improvement initiative, stratified by category of bleeding risk.

The unadjusted bleeding event rate was 5.5% before implementation and 4.4% after (P=0.08). Stratification by bleeding risk did not change these findings (Figure 2). In multivariable analysis after adjusting for bleeding risk, preprocedure hemoglobin, smoking status, dyslipidemia, and previous CABG, bleeding event rates were similar before and after implementation (odds ratio, 0.82; 95% CI, 0.63–1.08; P=0.164; Table 3). Comparing the types of qualifying bleeding events before versus after implementation, the percentage of PCIs followed by a drop in hemoglobin declined (3.3% versus 2.3%; P=0.05) whereas the percentage associated with frank bleeds within 72 hours rose (1.5% versus 2.4%; P=0.03; Figure S3).

Table 3

Comparison of Adjusted Bleeding Event Rate Between Procedures Performed Before Versus After Implementation of the Quality Improvement Initiative

Variable	Multivariable Model
	Odds Ratio (95% CI)	P Value
Before implementation vs after implementation
After implementation	0.82 (0.63–1.08)	0.164
Before implementation	1 (Reference)
Bleeding risk level
High	8.21 (3.91–17.25)	<0.001
Intermediate	3.00 (1.45–6.22)	0.003
Low	1 (Reference)
Preprocedure hemoglobin	0.82 (0.75–0.90)	<0.001
Smoking	1.54 (1.03–2.29)	0.036
Dyslipidemia	0.53 (0.39–0.71)	<0.001
Previous CABG	0.68 (0.45–1.03)	0.068

A total of 4579 observations were used in the multivariable model. CABG indicates coronary artery bypass grafting.

Use of BRC Tool Versus Nonuse After Implementation

When physicians used the BRC tool before PCI versus did not use it, they were more likely to use radial access (67.7% versus 62.5%; P=0.006); however, there is no difference in the use of closure devices (94.0% versus 94.9%; P=0.509), avoidance of IIb/IIIa inhibitors (97.3% versus 96.5%; P=0.207), and use bivalirudin (0.7% versus 0.5%; P=0.399) on univariate analyses (Figure 2). Stratified by bleeding risk, use of bleeding avoidance strategies was similar when the BRC was versus was not used (Figure S4).

In univariate analyses, the bleeding event rate was significantly lower for PCIs involving use of the BRC tool versus nonuse (3.5% versus 5.1%; P=0.040; Figure 4). Stratified by bleeding risk, use of the BRC tool was only associated with a reduction in the unadjusted bleeding event rate among intermediate‐risk PCIs (low risk, 0.4% versus 1.2%; P=0.281; intermediate risk, 3.5% versus 1.8%; P=0.041; high risk, 10.5% versus 9.3%; P=0.560; Figure 2). In multivariable analyses after adjusting for bleeding risk, preprocedure hemoglobin, smoking status, and previous coronary artery bypass grafting, the bleeding event rate was similar when physicians used versus did not use the BRC tool (odds ratio, 0.71; 95% CI, 0.48–1.04; P=0.079; Table 4).

Figure 4

Comparison of unadjusted bleeding event rates between procedures performed before vs after implementation of the quality improvement initiative (left) and between procedures performed after implementation for which the bleeding risk calculator (BRC) tool was used vs not used (right), stratified by bleeding risk.

Table 4

Comparison of Adjusted Bleeding Event Rates Between Procedures Performed After Implementation of the Quality Improvement Initiative, With Use Versus Nonuse of Bleeding Risk Calculator Tool

Variable	Multivariable Model
	Odds Ratio (95% CI)	P Value
BRC tool use (after implementation PCIs only)
BRC tool used	0.71 (0.48–1.04)	0.079
BRC tool not used	1 (Reference)
Bleeding risk level
High	9.14 (3.19–26.16)	<0.001
Intermediate	3.17 (1.13–8.90)	0.029
Low	1 (Reference)
Preprocedure hemoglobin	0.79 (0.70–0.90)	<0.001
Smoking	1.94 (1.14–3.31)	0.015
Previous CABG	0.50 (0.26–0.94)	0.030

A total of 2713 observations were used in the multivariable model. BRC indicates bleeding risk calculator; CABG, coronary artery bypass grafting; PCIs, percutaneous coronary interventions.

Discussion

This evaluation of an EMR‐based bleeding risk calculator tool and associated clinical decision support alert designed to reduce bleeding after PCI had 3 major findings. First, physicians placing orders for PCI—the majority of whom were cardiology fellows—were willing to score bleeding risk and did so more often when the risk of bleeding was low to moderate. Second, despite near universal use of ≥1 strategies for avoiding bleeding at baseline, implementation of the intervention was temporally associated with increases in the use of radial access and avoidance of glycoprotein IIb/IIIa inhibitors, based on unadjusted analyses. Third, although clinicians used the BRC tool before many PCIs and strategies for preventing bleeding changed, there were no significant differences in bleeding rates either from before to after implementation or between PCIs where the tool was versus was not used, based on adjusted analyses.

Our study demonstrated that integration of a BRC tool and clinical decision support alert into the EMR is feasible and can be adapted into clinical workflows. Physicians’ use of the BRC tool to score bleeding risk before PCI in our study ranged from 38% and 55%, consistent with EMR tool adoption rates in other published literature.22, 23 Reasons for the overall low uptake of the bleeding risk tool and other CDS tools in general have been previously described and include concerns over the omission of possibly beneficial therapies, inundation with perceived unnecessary information, and the belief that experience trumps evidence in clinical decision making.24 Notably, physicians used the BRC tool more often for outpatient procedures and procedures that involved a low‐to‐moderate risk of bleeding, but less often for inpatient procedures and procedures that involved a high risk of bleeding. Inpatient and high‐risk procedures are more likely to be more emergent (ie, for ST‐segment–elevation myocardial infarction); therefore, physicians may not have had adequate time to complete the risk score. Alternatively, physicians may have skipped using the tool when they already knew that bleeding risk was high or if they felt that use would not alter their clinical management (eg, if they already planned to use ≥1 bleeding avoidance strategies).

Another factor likely to have reduced use of the BRC tool was the data entry burden for physicians placing orders for PCI. Previous studies have identified physicians’ perceptions of impedance to clinical workflow as a major barrier to their use of clinical decision support tools.22, 25, 26 A meta‐analysis conducted for the Agency for Healthcare Research and Quality found moderate evidence that avoiding the need for clinician data entry when using decision support tools was associated with greater effectiveness.27 Although automation can reduce the burden of data entry, past research has demonstrated that as few as 16% of computerized scoring systems can be completely automated, often because data are incomplete or encoded as free text rather than as discrete variables, as with our BRC tool.28 Of note, a previous study of the ePRISM (Patient Risk Information Services Manager) system demonstrated a reduction in bleeding events following the program's introduction.29 In that multicenter study of non‐ST‐segment–elevation myocardial infarction PCIs, use of ePRISM, an electronic platform for patient consent and risk stratification external to, but integrated with, the electronic health record was not voluntary, and obtaining bleeding risk scores was not necessarily contingent on data entry from the physician. The lack of automation and inclusion of emergent PCIs in our study represent importance design differences, which may have contributed to the difference in outcomes.

Despite the lack of automation, implementation of the BRC tool systemwide was associated with changes in care processes, specifically increases in the use of radial access and decreases in the use of IIb/IIIa inhibitors, with use versus no use of the BRC tool associated with higher radial access use. Increases in these 2 bleeding avoidance strategies occurred despite very high rates of use at baseline at Cedars‐Sinai, particularly compared with other hospitals nationally.12, 30 Our findings are consistent with a recent systematic review and meta‐analysis of clinical decision support tools, which found good evidence that such tools can improve performance on process measures like ordering a correct test.31 Characteristics of clinical decision support that have been associated with greater improvements in performance on process measures include the presence of the tool at the time of clinical decision making and whether the tool offers specific recommendations—both key attributes of our intervention.32, 33

We observed changes in care processes across all risk groups, including PCIs involving a low risk of bleeding, which did not trigger alerts recommending use of the 4 bleeding avoidance strategies. Moreover, increases in use of bleeding avoidance strategies were found for procedures in which the BRC tool was not utilized. This suggests that the prompt to complete the BRC tool might have reminded physicians of the risk of bleeding and availability of bleeding avoidance strategies, even if physicians did not complete the BRC tool or tailor the use of bleeding avoidance strategies to the actual risk of bleeding. Other researchers have described a “risk‐treatment paradox” in which patients at low risk of bleeding after PCI received more interventions to reduce bleeding risk than high‐risk patients did. Unnecessary use of bleeding avoidance strategies increases costs and may contribute to low‐value care.10 However, factors other than the QI initiative may explain the trends we observed in the use of bleeding avoidance strategies, including nation‐wide increases radial access and declines in the use of IIb/IIIa inhibitors.30, 34, 35 Furthermore, the overall low‐rate bivalirudin use may be related to both cost considerations as well as recent reports questioning its efficacy.14, 36

In addition to the differences in use of bleeding avoidance strategies, we saw a numerical decline in the bleeding event rate of borderline significance. There may be 2 types of explanations. First, earlier literature has found scant evidence of associations between use of EMR‐based tools and clinical outcomes, despite demonstrated improvements in processes of care, because outcome measures are subject to far greater random statistical variation, among other factors.31, 37, 38 Second, using bleeding avoidance strategies is only moderately effective at reducing the risk of bleeding, particularly among high‐risk patients. At least 70% of variation in bleeding rates persists after controlling for bleeding avoidance strategy use.2 Consistent with this, our data demonstrate that in unadjusted analysis, intermediate‐risk patients benefited the most from risk stratification. Persistence of bleeding risk suggests that future research and QI initiatives should focus not only on which bleeding avoidance strategies to pursue, but also whether the risk‐benefit ratio favors performing a PCI in the first place. Furthermore, risk stratification and subsequent bleeding avoidance strategy use may be most beneficial in intermediate‐risk rather than high‐risk patients.

There are several limitations to our study, including its focus on practice patterns at only 1 institution. However, the inclusion of private physicians as well as faculty enhances generalizability. The study used an observational before‐after design, without a contemporaneous control group. Although we examined the intervention under real‐world conditions, we cannot exclude the potential role of secular trends in our dependent variables, particularly use of bleeding avoidance strategies. Our comparisons between PCIs for which the BRC tool was used versus not used were subject to confounding by indication; however, the results of these analyses were similar to those of the before‐after analyses. Furthermore, the data on bleeding avoidance strategies evolved during the study, specifically with softening data for the use of bivalirudin, which may have skewed the results of both the use of bleeding avoidance strategies and bleeding event rates. Finally, although the NCDR bleeding risk model is considered more accurate than other risk models, it fails to account for other known bleeding risk factors such as platelet count, liver disease, or use of oral anticoagulants.

In conclusion, this study demonstrates that integrating guideline recommendations into the EMR to promote assessments of bleeding risk and use of bleeding avoidance strategies was feasible and associated with changes in clinical practice. Future work is needed to ensure that bleeding avoidance strategies are not overused among lower‐risk patients, and that, for patients at high risk of bleeding, the potential benefits of elective PCI are carefully weighed against the risks.

Sources of Funding

Dr Ebinger was provided funding by the National Institutes of Health (5 T32HL116273 05). The work was supported by the Cedars‐Sinai Eigler‐Whiting‐Mann Clinical Scholars/UCLA CTSI—Training Program in Translational Science (TPTS).

Disclosures

None.

Figure S1. Integration of electronic medical‐record–based bleeding risk calculator into clinical workflow.

Figure S2. Run chart of bleeding risk calculator use during after implementation period. Median line represents the median use of the bleeding risk tool.

Figure S3. Rates of qualifying bleeding events by implementation period.

Figure S4. Comparison of unadjusted rates of bleeding avoidance strategy use between procedures performed after implementation of the quality improvement initiative for which the bleeding risk calculator (BRC) tool was used vs not used, stratified by category of bleeding risk.

Click here for additional data file.