Electronic pulmonary embolism clinical decision support and effect on yield of computerized tomographic pulmonary angiography: ePE-A pragmatic prospective cohort study.

OBJECTIVE: Multiple professional societies recommend pre-test probability (PTP) assessment prior to imaging in the evaluation of patients with suspected pulmonary embolism (PE), however, PTP testing remains uncommon, with imaging occurring frequently and rates of confirmed PE remaining low. The goal of this study was to assess the impact of a clinical decision support tool embedded into the electronic health record to improve the diagnostic yield of computerized tomography pulmonary angiography (CTPA) in suspected patients with PE in the emergency department (ED).
METHODS: Between July 24, 2014 and December 31, 2016, 4 hospitals from a healthcare system embedded an optional electronic clinical decision support system to assist in the diagnosis of pulmonary embolism (ePE). This system employs the Pulmonary Embolism Rule-out Criteria (PERC) and revised Geneva Score (RGS) in series prior to CT imaging. We compared the diagnostic yield of CTPA) among patients for whom the physician opted to use ePE versus the diagnostic yield of CTPA when ePE was not used.
RESULTS: During the 2.5-year study period, 37,288 adult patients were eligible and included for study evaluation. Of eligible patients, 1949 of 37,288 (5.2%) were enrolled by activation of the tool. A total of 16,526 CTPAs were performed system-wide. When ePE was not engaged, CTPA was positive for PE in 1556 of 15,546 scans for a positive yield of 10.0%. When ePE was used, CTPA identified PE in 211 of 980 scans (21.5% yield) (P < 0.001).
CONCLUSIONS: ePE significantly increased the diagnostic yield of CTPA without missing 30-day clinically overt PE.

INTRODUCTION

Background

Acute pulmonary embolism (PE) is diagnosed in ∼340,000 patients each year in the United States. However, PE is suspected much more frequently, and only ∼10% of all suspected cases of acute PE lead to the diagnosis. , , , Although PE is a diagnosis that is frequently considered among patients presenting to the emergency department (ED), the diagnostic yield (or rate of imaging positive for PE) of pulmonary vasculature imaging is low. Multiple professional societies, including the American College of Chest Physicians, European Society of Cardiology, and the American College of Emergency Physicians, recommend formal assessment of pre‐test probability (PTP) for PE followed by laboratory testing or imaging based on PTP risk estimate for patients suspected of PE. , , This strategic recommendation reduces the number of patients who will undergo imaging tests by ruling out the disease on the basis of PTP and laboratory results in selected patients. This has the ultimate effect of reducing net radiation exposure in this cohort of patients, reducing direct and indirect costs, and reducing false‐positive results in a low PTP cohort.

Importance

The two‐ and three‐tiered Wells Criteria, the Revised Geneva Score (RGS), the Pulmonary Embolism Rule Out Criteria (PERC), and the YEARS score are all tools to assess PTP among patients suspected of having PE. , , , Despite a high level of evidence supporting PTP application toward patients suspected of PE, reliable physician implementation of PTP testing is poor. To encourage, support, and optimize PTP assessment among patients suspected of PE, investigators have embedded PTP assessment in the ED workflow using the electronic health record (EHR) to present clinical decision support (CDS) related to PE diagnostic strategies to the ED physician. , , , , , , , , , This effort has been shown to reduce nonadherence to PTP assessment, decrease advanced imaging use, and improve the diagnostic yield of computerized tomography pulmonary angiography (CTPA). , , , , This effort may have been limited by the requirement for subjective components in these scores including, by the history of derivation and validation of the scores occurring at academic medical centers, and by concern regarding generalizability of the results to community practice settings.

Goals of this investigation

The goal of this study was to assess the impact of a CDS tool embedded into the EHR to improve the diagnostic yield of CTPA in suspected patients with PE in the ED.

METHODS

Study design and setting

In this prospective cohort study, we report the results of an electronic CDS (eCDS) embedded in the clinical workflow of an integrated healthcare system. After developing this system, we used the ED's desktop computers to implement the PERC and RGS in series to assess PTP for PE. We compare the diagnostic yield of CTPA among patients for whom the treating physician opted to use eCDS with that of treating physicians that did not.

This study was conducted in an integrated community hospital system within 4 EDs with a combined annual ED census of >165,000 patient visits. In this experiment, ePE was integrated into the shared EHR at 3 suburban/rural hospitals and 1 tertiary‐referral level 1 trauma center. We compared patients in which ePE was used with those in which ePE was not used. Data were collected from July 24, 2014 to December 31, 2016.

Intervention

Our novel desktop application, electronic pulmonary embolism (ePE), was engineered to implement a consistent, clinical diagnostic algorithm among patients presenting to the ED who were suspected of having pulmonary embolism. The ePE tool was built by our hospital system through a collaboration of emergency physicians, internal medicine physicians with clinical focus on thrombosis, and informatics experts and was based on professional society guidelines (Figure 1). When launched from the ED physician workstation desktop, ePE prompted the emergency physician to collect data germane to calculating the PERC, then if indicated, the Revised Geneva Score (RGS). In low‐ and intermediate‐probability patients, as determined by an RGS score of <11, a D‐dimer was recommended. This was followed by a recommended CTPA in patients with an age‐adjusted positive D‐dimer (age >50 × 10 ng/mL). D‐dimer results are returned to the physician and are highlighted as either abnormal or normal depending on age‐adjusted cut‐offs prompting physicians to order advanced imaging studies when appropriate. CTPA was also recommended among patients with a high probability for PE defined as an RGS >11. In patients with an estimated glomerular filtration rate (eGFR) <30 mg/dL or contrast allergy, a ventilation‐perfusion (V/Q) scan is recommended in lieu of a CTPA. Where possible, data for the calculation of PERC and RGS are auto‐populated from extant data in our longitudinal EHR, which includes objective components such as age, heart rate, oxygen saturation, and past medical history. The subjective and physical examination components in these risk scores required physician input. The presenting vital signs recorded in the EHR prior to tool use were used to populate the tool. The treating physicians could manually edit vital signs or objective components if warranted.

FIGURE 1

Electronic pulmonary embolism (ePE) application example

Pre‐test probability testing for pulmonary embolism (PE) is uncommon, with low diagnosis rates. This prospective study of an embedded electronic pulmonary embolism PE (ePE) clinical decision support tool in 1949 patients with suspected PE had a higher positive diagnostic rate of 21.5%, compared with 10% positive diagnosis in 35,339 patients without the tool. The ePE tool had an OR of 2.7, without missing 30 day PEs.

ePE uses a standards‐based workflow management toolset that implements the Object Management Group's (OMG's) Business Process Modeling and Notation standard, version 2 (BPMN 2.0). This technology supports the development of executable clinical workflows designed to support multistep protocols. The application was deployed using custom data access services to extract relevant data from Intermountain Healthcare legacy EHR. A Java‐based, application environment and .Net client provided the underlying software infrastructure to deliver the application to the bedside.

The training and implementation process for physicians who would use ePE was as follows: all emergency medicine physicians at the 4 intervention hospitals were formally educated to the tool's existence, understood its functionality, as well as its optional use status through presentations and demonstrations at departmental meetings, email tip sheets, and on‐shift tutorials when requested.

To heighten physician awareness of ePE, an alert to the patient tracking system (PTS) board prompted the emergency physician to launch ePE if the health‐unit coordinator entered an order for a D‐dimer, CTPA, or ventilation perfusion (VQ) scan. The alert was sent to a protocol column on the PTS board initially as a green “PE,” but changed to yellow, red, and intermittent flashing with each 10‐minute interval that the tool was not launched. The only way to dismiss the alert was to activate the tool; however, it was possible for physicians to ignore the alert. The physician was prompted to cancel the D‐dimer if the patient was PERC‐negative or cancel the CTPA if the ePE tool did not recommend the test. If the patient had an RGS >11, and a D‐dimer was ordered initially, the physician was prompted to cancel the D‐dimer order and instead place an order for CTPA. Physicians could alternatively access the tool at any point by clicking on an ePE icon that was in place on every ED computer desktop to launch the application.

Selection of participants

All patients age >17 years with suspected PE, as determined by the treating emergency physician through the order for a D‐dimer, CTPA, V/Q scan, or activation of the tool, were included for evaluation. Demographics of included patients are listed in Table 1. Patients were excluded if they were age ≤17 years, pregnant at the time of ED evaluation, or prisoners.

TABLE 1

Demographics and use by hospital

Variable	All % (n)	No ePE n = 35,339 % (n)	ePE n = 1,949 % (n)
Female	61% (22878)	61% (21666)	62% (1212)
Age (y) (Mean ± SD)	52 ± 18	52 ± 18	51 ± 18
Cancer	12% (4615)	12% (4376)	12% (239)
Obesity	23% (8732)	23% (8258)	24% (474)
Hypercoagulability	4% (1656)	4% (1559)	5% (97)
Prior VTE	13% (4688)	12% (4346)	18% (342)
Hormone replacement therapy	4% (1670)	4% (1569)	5% (101)
Congestive heart failure	13% (4841)	13% (4619)	11% (222)
Diabetes	21% (7799)	21% (7393)	21% (406)
Current tobacco use a	20% (7330)	20% (6908)	22% (422)
Bed rest	0% (0)	0% (0)	0% (0)
Surgery (in past month)	2% (577)	2% (557)	1% (20)
Central venous catheter	3% (946)	3% (925)	1% (21)
Infection	24% (9083)	24% (8626)	23% (457)
PICC line	2% (833)	2% (811)	1% (22)
Sepsis	10% (3795)	10% (3638)	8% (157)
LOS (day) (Mean ± SD)	2.6 ± 17.4	2.7 ± 17.9	1.3 ± 1.6

Abbreviations: ePE, electronic pulmonary embolism; LOS, length of stay; PICC, peripherally inserted central catheter; VTE, venous thromboembolism.

Missing data in 6677 of “No ePE” and 2 of “ePE.”

Analysis

All patients for whom the treating physician opted to use the tool were enrolled and their data were prospectively collected. Data from the ePE tool were stored on a secured research server within the institutional firewall. It was decided a priori that all patients for whom the tool was initiated would have all their data, including outcomes, collected in an intention to treat model regardless of whether the tool was run to completion. Patients with suspected PE that did not have ePE used or available had their data collected retrospectively from the hospitals electronic data warehouse (EDW) using standard query techniques by using ICD‐9/10, previously validated natural language processing tool for VTE outcomes, and CPT codes. A 30‐day follow up was performed using validated methods to determine missed PE. For the primary outcome, incidence of PE on CTPA, we assumed an increase of ∼7%. We planned to enroll a minimum of 650 patients with a CTPA to demonstrate a significant difference with a 5% margin of error and 95% confidence interval (CI). All tests were computed using Student's t‐test, and corresponding methods were used to create the confidence intervals on the means and differences of means. Logistic regression was performed in the subset of patients with CTPA (Table 3). Both sample sizes were large enough that the results will be approximately correct even in the cases where the data indicates a non‐normal population. The study was performed according to STROBE guidelines. The Intermountain institutional review board approved the study with a waiver of informed consent (IRB 1024529).

TABLE 3

CTPA yield logistic regression model

Variable	Odds ratio (95% CI)	P‐value
Intercept	0.03 ( 0.02, 0.04)	<0.001
Year		–
2015 versus 2014	0.88 ( 0.75, 1.02)	0.087
2016 versus 2014	0.94 ( 0.81, 1.10)	0.427
Tool used	2.83 ( 2.33, 3.42)	<0.001
Sex	–	–
Male versus female	1.19 ( 1.06, 1.34)	0.002
Age (y)	1.01 ( 1.00, 1.01)	0.002
Cancer	0.85 ( 0.67, 1.07)	0.167
Obesity	1.72 ( 1.50, 1.96)	<0.001
Hypercoagulability	1.13 ( 0.91, 1.40)	0.274
Prior VTE	13.84 (12.34, 15.52)	<0.001
Hormone replacement therapy	1.38 ( 1.04, 1.82)	0.024
Congestive heart failure	0.85 ( 0.72, 1.00)	0.045
Diabetes	0.72 ( 0.63, 0.84)	<0.001
Surgery (in past month)	0.67 ( 0.43, 1.00)	0.059
Infection	0.92 ( 0.76, 1.10)	0.370
Sepsis	0.59 ( 0.41, 0.82)	0.003
LOS (day)	1.00 ( 1.00, 1.00)	0.390

Abbreviations: CTPA, computerized tomography pulmonary angiography; LOS, length of stay; PICC, peripherally inserted central catheter; VTE, venous thromboembolism.

Note: Year is not significant to CTPA yield (full/reduced test, P = 0.211).

RESULTS

During the 2.5‐year study period, 37,288 patients were included for evaluation at 4 hospitals; 22,878 (61%) patients were female, with a mean age of 52 years (SD = 18). ePE was available for use in all patients with suspected PE at the study hospitals; however, it was optional and used in 1949/37,288 (5.2%) of possible ED encounters. Mean age of the cohort for whom the tool was used was 51 years (SD = 18), 62% were female. In the ePE cohort, 227 (11.6%) patients were diagnosed with a PE; 16 (7%) by V/Q scan and the remaining 211 (93%) by CTPA. System‐wide, there were 16,526 ED encounters in which a CTPA was performed. The CTPA diagnostic yield in the study group when ePE was engaged was 211/980 (21.5%) compared with 1556/15,546 (10.0%) in the non‐tool use group (Table 2). In 89 instances, the ED physician disagreed with the recommendation for a D‐dimer prior to CTPA; 35 (39%) of those were diagnosed with PE. Among patients for which the ePE tool recommended no further work‐up or imaging the 30‐day rate of PE was 0/1142 (0%; 95% CI = 0–0.004).

TABLE 2

No ePE versus ePE usage CTPA results

Variable	All % (n)	No ePE use n = 35,339 % (n)	ePE use n = 1949 % (n)
Sex
Male	39% (14,409)	39% (13,672)	38% (737)
Female	61% (22,878)	61% (21,666)	62% (1212)
Other	0% (1)	0% (1)	0% (0)
Age (y)
(Mean ± SD)	52 ± 18	52 ± 18	51 ± 18
D‐dimer	77% (28,691)	76% (26,969)	88% (1722)
CTPA	44% (16,526)	44% (15,546)	50% (980)
PE	5% (1767)	4% (1556)	11% (211)
CTPA yield	10.7%	10.0%	21.5%

Abbreviations: CTPA, computerized tomography pulmonary angiography; ePE, electronic pulmonary embolism; PE, pulmonary embolism.

An order for a D‐dimer occurred in 1722/1949 (88%) of patients when ePE was activated and in 26,969/35,339 (76%) in the non‐tool use group. ePE use was strongly associated with a positive CTPA, OR = 2.68 (95% CI = 2.21–3.23 P < 0.001).

LIMITATIONS

Given that ePE was used in only 5.2% of possible ED encounters, selection bias cannot be excluded. If physicians chose to use the tool in higher risk patients, this could account for the higher incidence of PE with ePE use. It should be noted, however, that regardless of physician‐initiated usage of ePE, all physicians were visually notified of tool availability on ordering any testing for PE.

We cannot rule out that in some instances where the ePE tool was not used, physicians were ordering D‐dimer for reasons other than concern for PE (such as disseminated intravascular coagulation, deep vein thrombosis, etc). We believe that the overall rate of D‐dimer ordering for alternative etiologies in the ED is low and unlikely to influence our conclusions.

Although patients in our non‐tool use cohort may have had PE suspected with no testing done (ie, PERC negative or clinician gestalt) and would not have been captured in our analysis, this would not affect our results. Given our primary evaluation was CTPA use/yield and not any testing was performed, patients that did not have CTPA testing performed because they were ruled out by the above‐mentioned approaches would not have influenced the CTPA yield rate.

We cannot refute the possibility that we did not capture PE diagnosed within 30 days if a patient presented to a hospital outside our system.

DISCUSSION

The use of an electronic pulmonary embolism CDS tool appears to be associated with reduced CTPA use and improved CTPA diagnostic yield. , , , , , , , Our results are consistent with these observations. ePE appears safe given that we did not observe any 30‐day PE events among patients for which ePE recommended no further assessment. No PE was diagnosed when a physician rejected the ePE recommendation and performed CTPA in the setting of a negative PERC or negative D‐dimer. However, in patients that were PERC‐positive and D‐dimer was recommended based on an RGS <11, but deferred by the physician, there was a high rate of PE diagnosis (39%). Reasons for disagreement were collected during tool use, and in 88% of cases, the D‐dimer was bypassed due to “recent surgery and D‐dimer unlikely to be negative.” Presumably, had D‐dimers been used, these patients would likely have had positive test cutoffs resulting in CTPAs. This observation suggests that physicians can intuit high risk for PE based on clinical presentation with good success.

A recent study by Buchanan et al. reported that 25.5% of patients found to be PERC‐negative underwent advanced imaging resulting in a PE diagnosis rate of 0.4%. Yan et al. reported that among patients suspected of PE, the diagnostic yield was found to be 4.2% when the physician overrode the CDS recommendation compared with 11.2% when the clinician did not. Consistent with our study, a significant percentage of patients underwent CTPA testing without prior pretest probability assessment and further risk stratification. It is unclear if physicians used other clinical support tools not embedded in the EMR to make such decisions, although given the low diagnostic yield among patients in the non‐tool use cohort, we believe that this is unlikely.

To our knowledge, there have been no studies evaluating an eCDS implemented in the ED in a large community health system using PERC in series with the RGS. Current published literature overwhelmingly uses Wells Criteria with few additionally incorporating PERC. These studies have demonstrated improved CTPA yield rates of 9.8%–16.5% with pre‐ and post‐rule implementation rates ranging from 1.1 to 5.3 times the baseline rate. , , , , , , , , Our results demonstrated a CTPA‐positive yield rate of 21.5% compared with 10.0% resulting in a >2‐fold improvement that aligns with prior studies. Our study also used age‐adjusted D‐dimer that likely influenced our reported diagnostic yield for CTPA.

Our ePE tool is compliant with the Protecting Access to Medicare Act of 2014 (PAMA) and appears to be the first published ED pulmonary embolism eCDS meeting these criteria. The 2014 PAMA regulations changed how Medicare reimburses under the Clinical Laboratory Fee Schedule (CLFS). These regulations, starting January 1, 2022, require physicians to consult appropriate use criteria through a qualified CDS Mechanism (qCDSM) before ordering advanced imaging. PAMA compliance will affect all EDs in the near future, and having an embedded eCDS will ensure appropriate compliance and reimbursement. We believe that ePE will both improve diagnostic yield of CTPA and assure PAMA compliance. We propose that implementation of this eCDS in our EHR system‐wide would improve the diagnostic yield for CTPA among patients suspected of PE without increasing the rate of missed diagnoses. Further studies are needed to compare eCDS formats across different healthcare systems as well as focus on optimal eCDS implementation to improve eCDS use.

In our integrated 4‐hospital healthcare system, ePE tool use was associated with a significantly higher diagnostic yield of PE on CTPA compared with a combination of non‐ePE usage without missed 30‐day clinically overt PE.

AUTHOR CONTRIBUTIONS

JRB, CK, SMS, SCW, CGE, PH, and TLA were responsible for manuscript drafts. AMB, JRJ, and JFL were responsible for statistical analysis. All authors assisted in manuscript editing. CK was responsible for journal submission and final responsibility.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.