Implementation of pharmacogenomic clinical decision support for health systems: a cost-utility analysis.

We constructed a cost-effectiveness model to assess the clinical and economic value of a CDS alert program that provides pharmacogenomic (PGx) testing results, compared to no alert program in acute coronary syndrome (ACS) and atrial fibrillation (AF), from a health system perspective. We defaulted that 20% of 500,000 health-system members between the ages of 55 and 65 received PGx testing for CYP2C19 (ACS-clopidogrel) and CYP2C9, CYP4F2 and VKORC1 (AF-warfarin) annually. Clinical events, costs, and quality-adjusted life years (QALYs) were calculated over 20 years with an annual discount rate of 3%. In total, 3169 alerts would be fired. The CDS alert program would help avoid 16 major clinical events and 6 deaths for ACS; and 2 clinical events and 0.9 deaths for AF. The incremental cost-effectiveness ratio was $39,477/QALY. A PGx-CDS alert program was cost-effective, under a willingness-to-pay threshold of $100,000/QALY gained, compared to no alert program.

INTRODUCTION

Pharmacogenomics (PGx) offers significant potential to improve drug outcomes.(1,2) The Clinical Pharmacogenomics Implementation Consortium (CPIC) has published 25 guidelines for 20 pharmacogenes and 61 drugs.(3) The prevalence of variants and the life-long relevant of germline biomarkers have motivated clinician-researchers to implement preemptive genotyping programs.(4–6) However, there are a few barriers resulting in low incorporate of PGx testing into routine clinical practice.(7–9) First, germline genomic testing will frequently be performed well before a decision needs to be made, often compromising the availability of that information when needed.(10) In addition, incorporating PGx information into current workflows is challenging.(11) Finally, most clinicians lack the training to readily interpret genomic test results.(11–13)

Clinical decision support (CDS) alerts, embedded in electronic health records (EHRs), promise to be a viable solution to these challenges.(14,15) CDS programs help provide clinical knowledge and patient-level information to aid decision-making at the point of care. For example, CDS programs can prompt with reminders for screening procedures and fire alerts to draw attention to important and relevant medical history. Ideally, the CDS program can reduce clinicians’ mental workload, smooth clinical workflow and improve patients’ health outcomes.(16,17)

However, CDS has not been universally adopted, especially in the context of PGx testing. A potential concern is around uncertain value of such a CDS program and economic burden to health systems. The effectiveness of CDS tools in guiding clinical management using genetic information remains largely inconsistent.(18,19) Additionally, because the CDS program involves advanced technologies and requires sufficient informatics equipment and the accompanying workforce,(17) the financial considerations are of importance to health systems.(20),(21) Therefore, we aimed to assess clinical utility and economic value of developing and implementing a CDS alert program in the context of PGx testing, using cost-utility analysis from a health system perspective compared to no alert program.

METHODS

Model structure

We developed a cost-effectiveness model for a hypothetical cohort of 500 000 health-system members to compare a CDS alert program to no alert program (Figure 1). We based our model in the disease areas of acute coronary syndrome (ACS) and atrial fibrillation (AF) in which the value of PGx testing has been most widely studied.(22–30) We adopted an annual, cross-sectional approach. We did not follow the hypothetical cohort of patients over time, but rather, looked at a sequential cross-sectional average, testing a certain proportion of patients each year within each strategy. We reasoned that this cross-sectional approach would reflect real-world implementation of PGx testing, as membership in a health system is dynamic and therefore any health system-wide decision would necessarily be implemented repeatedly to ensure newly eligible members have same opportunity to benefit from the decision. Because our model estimated the value of a CDS alert program, and not PGx testing, in both strategies the same proportion of people aged between 55 and 65 would receive pre-emptive PGx testing each year.

Figure 1.

Model Schematics.

Decision trees were displayed for antiplatelet selection based on PGx testing of CYP2C19 (A), and warfarin dosing based on PGx testing of CYP2C9, VKORC1, and CYP4F2 (B). PGx testing: pharmacogenomic testing; CDS: clinical decision support. A: PGx-CDS alerts for ACS and clopidogrel. B: PGx-CDS alerts for AF and warfarin.

A proportion of individuals in each strategy who underwent PGx testing were identified as a pharmacogene carrier for ACS, based on their race/ethnicity status. Carriers who were later diagnosed with ACS were at risk of inappropriately receiving clopidogrel. With the CDS alert program, an alert was fired to notify the provider of the carrier status, and suggest an alternative prescription for ticagrelor. Patients would gain benefit if a provider followed the alert’s suggestion. The pathway is the same for AF except that patients would gain benefit if dosing of warfarin is adjusted based on PGx information. We applied an annual 3% discount rate to the investment.(31) The model was built in R version 3.6.3.

Population

The hypothetical cohort consisted of 500,000 individuals between the ages of 18 and 100 years. The age and race/ethnicity (European, African, and Asian) distribution followed that of the US general population in 2020.(32)

Key Assumptions

We assumed that the CDS alert program was in place at the beginning of the study, and PGx testing results were able to be embedded into CDS alert program with no delay. Based on experts’ opinion, we assumed the CDS alert program lasted for 20 years. In addition, we defaulted that each year, 20% of individuals in a health system aged between 55 and 65 would receive preemptive PGx testing to reflect a plausible, non-selective preemptive PGx screening strategy. This uptake rate was assumed to be constant over 20 years. The probability of undergoing PGx testing over 20 years for each individual was capped at 100%. Moreover, we assumed that providers might still look for PGx results even in the absence of an alert program, reasoning that they might have received sufficient education about PGx testing or had prior experience with PGx testing.

Input parameters

All input parameters are listed in Table 1.

Table 1.

Model Input Parameters

Parameters	Base value	Range	Distribution	Sources
Probabilities
Population characteristics
% of individuals by age	Example: 0.167 for 18-year-old	NA	NA	(32)
Proportion of White	0.8028	NA	NA	(32)
Proportion of African American	0.1369	NA	NA	(32)
Proportion of Asian	0.0603	NA	NA	(32)
PGx testing
Intermediate or poor metabolizer, in White	0.3818	NA	NA	(33)
Intermediate or poor metabolizer, in African American	0.3840	NA	NA	(33)
Intermediate or poor metabolizer, in Asian	0.5394	NA	NA	(33)
Eligibility to benefit from PGx testing for warfarin	0.67	0.40, 0.90	Beta	(39)
Incident prescription
Annual probability of initiating clopidogrel therapy for ACS	Example: Age 18–24: 0.0003% Age 55–59: 0.1160%	NA	NA	IBM MarketScan database analysis (42)
Annual probability of initiating warfarin therapy for AF	Example: Age 18–24: 0.0005%; Age 55–59: 0.0333%	NA	NA	IBM MarketScan database analysis (42)
Provider behavior
Probability of adjusting treatment with an alert	0.25	0.20–0.50	Beta	Rapid Review. (44–50)
Probability of adjusting treatment without an alert	0.10	0–0.14	Beta	Assumption
Relative risk
Relative risk of incidence prescription
Relative risk of initiating clopidogrel therapy for ACS	1	0.50, 1.50	Log-normal	Assumption
Relative risk of initiating warfarin therapy for AF	1	0.50, 1.50	Log-normal	Assumption
Costs
Cost payoffs
Cost payoff of PGx testing for clopidogrel per intermediate or poor metabolizer, $	7 043	5 000–10 000	Normal	(51)
Cost payoff of PGx testing for warfarin per patient tested, $	−165	−365, 35	Normal	(52)
Costs of developing PGx-CDS alerts
Number of hours needed to develop alerting system	200	50, 500	Log-normal	(53)
Hourly wage for health informatician, $	100	50, 150	Log-normal	(53)
Proportion of one-time start-up cost as annual maintenance cost	0.20	0.10, 0.30	Beta	(53)
QALYs
QALY payoffs
QALY of PGx testing for clopidogrel, per intermediate or poor metabolizer	0.179	0.10, 0.25	Beta	(51)
QALY of PGx testing for warfarin per patient tested	0.008	0.005–0.011	Beta	(52)
Clinical events
Clinical event payoffs – PGx testing for CYP2C19, per intermediate or poor metabolizer, compared to no PGx testing				(51)
Non-fatal myocardial infarction	−0.029	NA	NA	(51)
Stent thrombosis	−0.015	NA	NA	(51)
Coronary artery bypass graft revascularization	−0.0021	NA	NA	(51)
Percutaneous coronary intervention revascularization	−0.0175	NA	NA	(51)
Cardiovascular death	−0.0232	NA	NA	(51)
Coronary artery bypass graft -related bleeding	0.0004	NA	NA	(51)
Non-fatal extracranial bleeding	0.0011	NA	NA	(51)
Non-fatal intracranial bleeding	0.0007	NA	NA	(51)
Clinical event payoffs – PGx testing for CYP2C9, CYP4F2, VKORC1, per patient tested, compared to no PGx testing
Bleeding	−0.007	NA	NA	(52)
Clotting	−.002	NA	NA	(52)
Cardiovascular death	−0.004	NA	NA	(52)
Other parameters
PGx testing pattern
Age for eligibility to receive PGx testing	55–65	NA	NA	Assumption
Annual probability to receive PGx testing	0.20	NA	NA	Assumption

NA: not applicable; PGx testing: pharmacogenomic testing; ACS: acute coronary syndrome; AF: atrial fibrillation; QALYs: quality-adjusted life years.

Pharmacogenes and risk of clinical events

The cytochrome P450 2C19 (CYP2C19) genotype guides antiplatelet selection for patients with ACS.(33) Patients who carry one or two loss-of-function alleles, are intermediate and poor metabolizers, respectively. They are at high risk of clinical events if receiving clopidogrel, such as thrombosis.(33) Ticagrelor is considered an alternative.(33)

Cytochrome P450 2C9 (CYP2C9), cytochrome P450 4F2 precursor (CYP4F2) and vitamin K epoxide reductase complex subunit 1 (VKORC1) are used to guide warfarin dosing for patients with AF.(34) Evidence from randomized clinical trials in which PGx-guided dosing was compared to clinical dosing algorithms suggests that information about these genes, regardless of phenotype, can aid dosing management and reduce time to achieving the maintenance dose.(35–41) Therefore, we did not specify the phenotype from which patients could benefit from PGx testing. Rather, we assumed a proportion of individuals who received PGx testing for CYP4F2, CYP2C9 and VKROC1 would benefit from PGx testing if the test results suggested a dose different from that suggested by a clinical algorithm.(39)

Risk of diseases

We estimated lifetime risk of an incident prescription by age group from 18 to 100 years, using the IBM MarketScan® Research Databases 2015–2019, consisting of the Commercial Claims and Encounters Database and Medicare Supplemental and Coordination of Benefits Database.(42) To reflect cross-sectional cohort modeling, we analyzed two annual probabilities of initiating clopidogrel for ACS, and initiating warfarin for AF.

Details of the analysis can be found in Appendix A. Briefly, we estimated the proportion of individuals who initiated a clopidogrel for ACS or warfarin therapy for AF among all adults in a given calendar year. Enrollees were required to have a 12-month continuous enrollment prior to an incident prescription of clopidogrel or warfarin and must have a diagnosis record of ACS for an incident clopidogrel or AF for an incident warfarin, in either inpatient or outpatient claims, within three months prior to or after the incident prescription.

Providers’ behavior

We acknowledged the presence of alert fatigue in clinical practice,(43) and incorporated the probability of alert fatigue into the model using estimates from the literature. Despite the presence of variation, the alert override rates were high, based on literature.(44–50) Thus, we defaulted that 25% of the time, a provider would follow the prescription recommendation in the alert, and 10% of the time, a provider would follow the prescription recommendation even without an alert.

PGx outcomes

Because the same proportion of individuals in either strategy received PGx testing, the difference between the two strategies was rooted in whether the CDS alerting program aided delivery of the PGx test results. Our goal was to compare the outcomes of PGx testing with and without a CDS alert program. Thus, we turned to published cost-effectiveness studies to identify clinical and economic value of PGx testing compared to no PGx testing, in ACS and AF.

We performed a systematic literature review. Details can be found in Appendix B. Briefly, we applied the following criteria to select cost-effectiveness models to inform our model, including (1) a lifetime horizon, (2) US population, (3) reported incremental costs, quality-adjusted life years (QALYs) and clinical events comparing PGx testing to no testing. Specifically for AF, we prioritized cost-effectiveness studies that were based on evidence synthesis from randomized trials.(35–41)

Only one study met the inclusion criteria, for ACS and AF, respectively.(51,52) The first article assessed the clinical and economic utility by comparing PGx testing to no PGx testing in a US patient population with ACS, from a payer perspective.(51) The second article assessed the clinical and economic utility in a US patient population with AF who needed warfarin, from a payer perspective.(52) These two studies were served as the main source of the outcomes (clinical events, QALY outcomes, and cost outcomes) comparing PGx to no PGx testing, in the following sections.

Clinical events

Clinical events for ACS included major non-fatal clinical events, bleeding events and cardiovascular death. Major non-fatal clinical events consisted of non-fatal myocardial infarction (MI), stent thrombosis, coronary artery bypass grafting (CABG) revascularization, and percutaneous coronary intervention (PCI) revascularization. Bleeding events consisted of nonfatal intracranial bleeding, nonfatal extracranial bleeding, and CABG bleeding. Risk changes of clinical events in ACS were lifetime risk changes due to PGx testing, per carrier.(51)

Clinical events for AF included major clinical events of bleeding and clotting, and cardiovascular death. The first year following warfarin initiation was the most relevant time period for any clinical event, and therefore, we adopted one-year risk of clinical events comparing PGx testing to no PGx testing, per patient tested.(52)

QALY outcome of PGx testing vs no PGx testing.

QALY outcomes reflected lifetime QALYs gained due to PGx testing compared to no PGx testing, for ACS and AF, per carrier and per patient tested, respectively.(51,52)

Cost outcome of PGx testing vs no PGx testing

Cost outcomes included prescription drug costs and the costs associated with the occurrence of each clinical event.(51,52) As the same proportion of individuals in our two strategies (i.e., PGx-CDS alert program and no alert program) underwent PGx testing, the cost of PGx testing was cancelled out. Thus, we subtracted the cost of PGx testing from the cost of each strategy.

Costs of developing and maintaining a CDS alert program

We incorporated a one-time start-up cost to reflect the financial burden of CDS alert infrastructure establishment, obtained from our previous empirical work.(53) We also incorporated an annual maintenance cost of the alert system in years 2 through 20, estimated as 20% of establishment costs.(53)

We adjusted all costs to 2021 US dollars by applying CPI, as the medical components of CPI was not fully applicable to the costs of developing and maintaining a CDS alert program.(54)

Outcomes

We first calculated implementation outcomes: the number of alerts fired, and medical, health informatics, and total cost per alert fired over the 20-year life of the alert program.

Clinical outcomes are the number of clinical events averted or induced by the CDS alert program, and the number of alerts needed to fire per clinical event averted or induced. For ACS, we focused on major non-fatal clinical events, bleeding events and cardiovascular death. For AF, we focused on major clinical events of bleeding and clotting, and cardiovascular death.

Finally, we estimated economic outcomes - the incremental costs and QALYs, and the incremental cost to incremental effectiveness ratio (ICER) of the CDS alert program compared to no alert program. We compared the estimated ICER to WTP thresholds of $50 000/QALY, $100 000/QALY and $150 000/QALY.(31)

Sensitivity Analyses

To examine the robustness of the economic value to input parameters, we performed a one-way sensitivity analysis (OWSA) on all parameters. We further performed a probabilistic sensitivity analysis (PSA) by varying all parameters using plausible ranges in 5 000 Monte Carlo simulations.(55)

Scenario Analyses

We identified three plausible scenarios (high, medium, and low PGx testing). In the high-testing scenario, all individuals aged between 45 and 75 years would undergo PGx testing at the beginning of the alerting program. In the medium-testing scenario, individuals aged between 55 and 65 years would have 30% chance of undergoing PGx testing every year. In the low-testing scenario, individuals aged between 55 and 65 years would have 1% chance of undergoing PGx testing every year.

RESULTS

Base-case results

Implementation outcomes

The model predicted that 3 169 PGx-CDS alerts would fire, including 1 721 alerts for clopidogrel for patients with ACS, and 1 448 for warfarin for patients with AF, over 20 years. This corresponded to 0.003 alerts per person in the PGx-CDS alert program. On average, the total cost was $420/alert fired, consisting of a medical cost of $395/alert fired, and an informatics cost $24/alert fired. The PGx-CDS alert program costs the health system just under $3 per person, over 20 years.

Clinical outcomes

On average, 105 alerts were needed to fire for clopidogrel use for ACS to avert one major non-fatal clinical event, 287 alerts were needed to avert one cardiovascular death, and 3 019 alerts had to fire prompt one additional bleeding event.

The CDS alert program helped reduce the number of major non-fatal clinical events by 16.32 and the number of cardiovascular deaths by 5.99. However, it also resulted in additional 0.57 bleeding events.

Similarly, 739 and 1 664 alerts would be needed to fire for warfarin use for AF to avert one clinical event and one death, respectively. In addition, the CDS alert program decreased the number of clinical events and deaths by 1.96 and 0.87, respectively. (Table 2, Table S1)

Table 2.

Base-case Clinical Events[a]

Clinical events related to clopidogrel use for ACS patients	Number of clinical events averted or induced due to PGx testing		Effect of the CDS alert program, compared to no CDS alert program.	Number of alerts needed to fire, per clinical event averted or induced[d]
	PGx testing with a CDS alert program, compared to no PGx testing.	PGx testing without a CDS alert program, compared to no PGx testing.
Major non-fatal clinical events[b]	−27.19	−10.88	−16.32	105
Cardiovascular death	−9.98	−3.99	−5.99	287
Bleeding	0.95	0.38	0.57	3 019
Clinical Event related to warfarin use for AF patients	Number of clinical events averted or induced due to PGx testing		Effect of the CDS alert program, compared to no CDS alert program.	Number of alerts needed to fire, per clinical event averted or induced[e]
	PGx testing with a CDS alert program, compared to no PGx testing.	PGx testing without a CDS alert program, compared to no PGx testing.
Clinical events[c]	−3.26	−1.3	−1.96	739
Cardiovascular death	−1.45	−0.58	−0.87	1 664

In the base-case, there were 500 000 individuals in the health plans. Every year, 20% of individuals who were aged between 55–65 would receive PGx testing. The CDS alert program lasted for 20 years.

Major non-fatal clinical events included non-fatal myocardial infarction (MI), stent thrombosis, coronary artery bypass grafting (CABG) revascularization, and percutaneous coronary intervention (PCI) revascularization.

Clinical events included bleeding and clotting.

Number of alerts needed to fire, per clinical event averted or induced were calculated using the number of alerts fired for ACS (1 721 in the base-case) divided by the number of clinical events averted or induced due to the CDS alert program, compared to no CDS alert program. For example, in total, 1 721 alerts for clopidogrel-ACS were fired and there were 16.32 major non-fatal clinical events averted due to the CDS alert program: 1 721/16.32=105 alerts needed to fire per major non-fatal clinical event.

Number of alerts needed to fire, per clinical event averted or induced were calculated using the number of alerts fired for AF (1 448 in the base-case) divided by the number of clinical events averted or induced due to the CDS alert program, compared to no CDS alert program. For example, in total, 1 448 alerts for warfarin-AF were fired and there were 1.96 clinical events averted due to the CDS alert program: 1 448/1.96=739 alerts needed to fire per clinical event.

PGx testing: pharmacogenomic testing; CDS: clinical decision support; ACS: acute coronary syndrome; AF: atrial fibrillation.

Economic outcomes

The incremental cost was $1 330 375, and the incremental QALYs gained were 33.7 comparing a CDS program to no CDS program. The ICER was estimated as $39 477 per QALY gained. (Table 3)

Table 3.

Base-case Cost-utility Analysis Results[a]

	PGx testing with a CDS alert program, compared to no PGx testing.	PGx testing without a CDS alert program, compared to no PGx testing	Incremental effects of the CDS alert program, compared to no CDS alert program	Number of alerts needed to fire, per QALY gained[b]
Costs, $	2 165 760.9	835 386.3	1 330 374.6	NA
QALYs gained	56.1	22.4	33.7	94
ICER[c], $ per QALY gained	39 477

In the base-case, there were 500 000 individuals in the health plans. Every year, 20% of individuals who were aged between 55–65 would receive PGx testing. The CDS alert program lasted for 20 years.

Number of alerts needed to fire, per QALY gained were calculated using the number of alerts fired for both ACS and AF (3 169 in the base-case) divided by the number of QALYs gained due to the CDS alert program, compared to no CDS alert program: 3 169/33.7=94 alerts needed to fire per QALY gained.

ICER was calculated using incremental costs due to CDS alert program divided by the incremental QALYs gained due to CDS alert program.

NA: not applicable; PGx testing: pharmacogenomic testing; CDS: clinical decision support; QALY: quality-adjusted life years; ICER: incremental costs and effectiveness ratio.

Sensitivity analyses

Five parameters that were most influential on the ICER were the QALYs and costs of PGx testing for ACS compared to no PGx testing, number of hours needed to develop the CDS system, the probability of providers’ change treatment with an alert, and the hourly wage for health informaticians to develop the CDS system. (Figure 2). The probabilities that the PGx-CDS was cost-effective were 71.8%%, 98.3%, and 99.5% under $50 000/QALY, $100 000/QALY and $150 000/QALY willingness to pay (WTP) thresholds, respectively. (Figure 3)

Figure 2.

One-way probabilistic sensitivity analysis (OWSA).

Parameters that were most influential to the base-case cost-utility analysis were listed. Values of parameters were based on ranges (Table 1). PGx testing: pharmacogenomic testing; CDS: clinical decision support; QALY: quality-adjusted life year; ACS: acute coronary syndrome; AF: atrial fibrillation.

Figure 3.

Cost-effectiveness acceptability curve (CEAC).

We performed a probabilistic sensitivity analysis by varying all parameters using plausible ranges (Table 1) and by conducting 5 000 Monte Carlo simulations. Probabilities were displayed that PGx-CDS alerts or no alerts was cost-effective among 5 000 Monte Carlo simulations under the willingness-to-pay thresholds ranging from $40 000 per QALY gained to $250 000 per QALY gained. PGx testing: pharmacogenomic testing; CDS: clinical decision support; QALY: quality-adjusted life year.

Scenario analyses

A total 6 670 alerts, would be fired in the high testing scenario. The estimated ICER was $38 095 per QALY gained. In a medium testing scenario, a total 3 485 alerts fired, resulting in an ICER of $39 196 per QALY gained. In the low testing scenario, only 228 alerts were fired and the ICER was $71 874 per QALY gained. (Table 4, Table S2–S3)

Table 4.

Cost-utility Results in Scenario Analyses

High-testing scenario[a]
	PGx testing with a CDS alert program, compared to no PGx testing.	PGx testing without a CDS alert program, compared to no PGx testing	Incremental effects of the CDS alert program, compared to no CDS alert program	Number of alerts needed to fire, per QALY gained[d]
Costs, $	4 710 305.4	1 853 204.1	2 857 101.3	NA
QALYs gained	125	50	75	90
ICER, $ per QALY gained	38 094.7
Medium-testing scenario[b]
	PGx testing with a CDS alert program, compared to no PGx testing.	PGx testing without a CDS alert program, compared to no PGx testing	Incremental effects of the CDS alert program, compared to no CDS alert program	Number of alerts needed to fire, per QALY gained[e]
Costs, $	2 398 243.7	928 379.4	1 469 864.3	NA
QALYs gained	62.4	24.9	37.5	93
ICER, $ per QALY gained	39 196.4
Low-testing scenario[c]
	PGx testing with a CDS alert program, compared to no PGx testing.	PGx testing without a CDS alert program, compared to no PGx testing	Incremental effects of the CDS alert program, compared to no CDS alert program	Number of alerts needed to fire, per QALY gained[f]
Costs, $	223 987.2	58 676.8	165 310.4	NA
QALYs gained	3.9	1.6	2.3	99
ICER, $ per QALY gained	71 874.1

In a high-testing scenario, all individuals aged between 45 and 75 would receive PGx testing, every year, over the 20 years of the CDS alert program. The probability of a given individual receiving PGx testing was capped at 100%.

In a medium-testing scenario, 30% of individuals aged between 55 and 65 would receiving PGx testing, every year, over the 20 years of the CDS alert program. The probability of a given individual receiving PGx testing was capped at 100%.

In a low-testing scenario, 10% of individuals aged between 55 and 65 would receive PGx testing, every year, over the 20 years of the CDS alert program. The probability of a given individual receiving PGx testing was capped at 100%.

In a high-testing scenario, over the 20 years of the CDS alert program, in total, 6 760 alerts were fired. The number of alerts needed to fire per QALY gained was calculated by the number of alerts fired divided by QALYs gained: 6 760/75=90 alerts needed to fire per QALY gained.

In a medium-testing scenario, over the 20 years of the CDS alert program, in total, 3 485 alerts were fired. The number of alerts needed to fire per QALY gained was calculated by the number of alerts fired divided by QALYs gained: 3 485/37.5=93 alerts needed to fire per QALY gained.

In a low-testing scenario, over the 20 years of the CDS alert program, in total, 228 alerts were fired. The number of alerts needed to fire per QALY gained was calculated by the number of alerts fired divided by QALYs gained: 228/2.3=99 alerts needed to fire per QALY gained.

NA: not applicable; PGx testing: pharmacogenomic testing; CDS: clinical decision support; QALY: quality-adjusted life year; ICER: incremental cost and effectiveness ratio.

DISCUSSION

Our study is the first to provide a structured and scientific approach to answer three key questions – “What are the implementation outcomes, clinical impacts, and the economic value of a CDS alert program in the context of PGx compared to no alert program?”. We found that 3 169 alerts would be fired with a PGx-CDS alert program, and each alert would cost on average $420. Alerts would help reduce clinical events and deaths for both ACS and AF. The estimated ICER of $39 477 per QALY gained was below the WTP threshold of $100 000 per QALY gained, suggesting that a CDS alert program was cost-effective compared to no alert program. The value of the CDS alert program was most sensitive to the cost and benefits of PGx testing, costs of developing and maintaining a PGx-CDS alert program and providers’ behavior in following the alerted prescribing recommendation. A PGx-CDS alert program was cost-effective at 98% of the time based on a WTP threshold of $100 000/QALY, given PGx testing was performed 20% per year in a population aged between 55 and 65 years, for 20 years.

Our study has a few implications. First, the results that a PGx-CDS alert program has clinical utility for patients in improving health outcomes emphasizes the importance establishing CDS infrastructure in delivering PGx information and guide prescribing.(18) However, the clinical utility of such a program first relies on the value of PGx testing and whether information is utilized in clinical practice. This demonstrates the power of CDS infrastructure in distributing crucial information and supporting clinical decision-making.(56,57) The interplay of PGx testing and a CDS alert program to guide prescribing suggests a wholistic approach in clinical practice to improve health outcomes.

Second, our results that a PGx-CDS alert program is cost-effective suggest that CDS investment provides good value for money, which addresses a common economic concern in adopting CDS alert programs in health systems.(20,21) However, establishing a CDS alert program is not cost-saving. The incremental cost consists of costs of using ticagrelor for ACS, a more expensive drug than clopidogrel, which will increase financial burden to payers and patients, and the financial investment in CDS.(17) To promote adoption of a PGx-CDS alert program, decision-makers should consider budget impacts and cost implications for payers and patients, along with the value information of a PGx-CDS alerts.(20,21)

Third, our result highlights the impact of scale of PGx testing on the cost and value of a PGx-CDS alert program.(58) In our scenario analyses, as the PGx testing rate increases, the cost of developing and implementing the CDS alert program per alert fired decreases significantly, from $339 per alert to $11 per alert (Table S2) and the value of a PGx-CDS alert program increases too, from $71 874 per QALY gained to $38 095 per QALY gained (Table 3). Although the PGx-CDS alert program remains cost-effective even in a low-testing scenario, the scale of PGx testing is a key factor in determining the value of the CDS alert program. Decision makers should incorporate the current uptake of PGx testing within their system first, and deliberate the possibility of expanding PGx testing for members to best exert the power of a CDS alert program.

Fourth, our modeling approach has implications for designing the scope of a CDS alert program. More than 100 pharmacogenes have the highest level of clinical evidence in corresponding disease areas, and are considered actionable.(3) A recent study found that many drugs with actionable pharmacogenes were commonly dispensed in practice.(59) This evidence suggests that incorporating a broad list of genes, drugs, and diseases when designing a PGx-CDS alert program should be considered. In addition, because of the decreasing costs of PGx testing, the marginal cost of testing an additional gene is reducing, and thus a comprehensive PGx-CDS program can potentially bring economies of scale and influence the system-level practice. Although our model only included clopidogrel-ACS and warfarin-AF for which there was a largest amount of data in support of PGx testing, it may serve as a prototype that allows for adding multiple genes, drugs and diseases in the future, which potentially increases the value of a PGx-CDS alert. Especially in the context of panel testing and even exome sequencing, preemptive PGx testing will become more comprehensive and have the potential to further increase the value of PGx-CDS alert program.

However, although a CDS alert program is promising and capable of delivering a broad range of PGx test information, value of developing a CDS alert program varies by costs and clinical benefits of PGx testing in different diseases. With the modeling approach, presenting tradeoff between costs and effectiveness helps rationalize investments in CDS alerts. Future studies should explore the cutoff for value of PGx testing to realize good value for money spent on developing a CDS alert program.

Lastly, our study findings can be particularly relevant for Learning Health Systems (LHSs), in which science, informatics, incentives and culture are aligned and new knowledge is integrated into delivery.(60) The wholistic approach where testing and informatics are integrated in advancing precision medicine encourages different functions in a LHS to collaborate together, and promotes efforts in learning their own patients’ genetic information, providers’ behavior, and PGx testing patterns. The learning will, in return, help guide their own decision-making in developing a PGx-CDS alert program in LHSs and eventually make the workflow in LHSs more efficient and cohesive.(61)

Our study has a few strengths. We based our cost evaluations on our prior work that examined real-world cost estimates of developing and implementing CDS alerts for PGx testing.(53) Additionally, we conducted database analyses using the IBM MarketScan ® Research Databases 2015–2019,(42) to generate real-world estimates of incident prescription use of clopidogrel for ACS, and warfarin for AF. Particularly, the real-world estimates of incident warfarin during 2015 to 2019 reflect the decreased use of warfarin, due to introduction of direct-acting oral anticoagulants (DOACs). Moreover, we incorporated alert fatigue to mimic the real-world acceptance rate of CDS alerts, based on estimates from the literature.(44–50) We performed a systematic literature review to identify outcomes of PGx testing compared to no PGx testing that were most aligned with our study setting.

Our study also has a few limitations. The idea of PGx-CDS alerts is simplified. We did not focus on factors such as visual design, and timing and frequency of alerts, which may affect the usability of alerts.(17) However, the incorporation of alert fatigue should overall account for the impact of these factors. Moreover, we only used alerts to guide prescribing based on PGx results. However, a CDS program virtually can be configured with other types of supports that help deliver PGx results and guide prescribing. Examples are data presentation features that display relevant PGx test results, order facilitators that provide recommended drugs and doses based on the PGx test results, and a reference guidance feature that presents PGx test guidance.(62) Incorporating these features may increase or decrease the value of a PGx-CDS program. Future work should examine the clinical and economic utility of types of CDS in PGx testing. Additionally, we assumed that PGx test results were embedded into CDS alerts with no delay, and thus, we did not account for waiting time for obtaining PGx results. Furthermore, clinical benefits for patients prescribed with warfarin for AF were based on population-level average estimates. Although we believed this would be the best approach based on current evidence from randomized controlled trials, it is likely that heterogeneity exists, which we did not address in our model. Moreover, we acknowledged that the defaulted 20% individuals who would receive PGx testing every year was a crude and optimistic assumption. Thus, we performed scenario analyses where the proportion of patients who received PGx testing varied from 1% to 100% and found that even with 10% of individuals receiving PGx testing every year, the ICER of $71 874.1 per QALY gained was still below the WTP threshold of $100 000 per QALY gained. However, we encouraged health systems used their own estimates to assess the ICER. Lastly, we modeled the incident prescription of clopidogrel and warfarin, and therefore did not consider alerts for refills. In addition, clopidogrel or warfarin were modeled separately, and thus the same patient would not trigger multiple alerts for multiple drugs. Incorporation of alerts fired for refills and the possibility that the same patient may require multiple drugs would likely change the implementation outcomes. Future work may enrich the model by accounting for these complex set-ups and examine the change in the outcomes.

Our model demonstrates a PGx-CDS alert program helps reduce clinical events and is cost-effective, compared to no alert program, for patients with ACS and AF. Future studies should explore the cutoff for value of PGx testing to realize good value for money spent on a CDS alert program.

Data Availability Statement

All data used in the model are publicly available and available by directly contacting the authors, as well as being included in the manuscript.