Identifying Outcome Measures for Coronary Artery Disease Value-Based Contracting Using the Delphi Method.

INTRODUCTION: Value-based contracts (VBCs) that link drug payments to disease-related performance metrics aim to increase the value and lower the cost of medications by aligning incentives and sharing risk between payers and pharmaceutical manufacturers. This study sought to identify outcome measures that are meaningful to key stakeholders to inform VBCs for coronary artery disease (CAD) medications.
METHODS: We administered a modified Delphi survey to gather expert opinion from a diverse panel of patients (n = 9), cardiologists (n = 4), primary care physicians (n = 5), payers (n = 2), pharmacy benefits managers (n = 3), and pharmaceutical company representatives (n = 2). A list of 16 CAD-associated clinical indicators was generated from the literature and expert consultation. Delphi participants rated the importance of each outcome on a five-point Likert scale, and selected the three most meaningful outcomes. We defined consensus as ≥ 75% agreement on the importance of an outcome (Likert scores 4 or 5 or selection of an outcome as most meaningful).
RESULTS: Eleven of 13 outcomes reached consensus for importance on the Likert scale. "Preventing heart attacks" was selected as the most meaningful outcome (80%) while "preventing death" ranked second (76%).
CONCLUSIONS: Our study results verify the utility of a widely used clinical CAD outcome measure, myocardial infarction events, for the purpose of pharmaceutical value-based contracting.

Introduction

P2Y12 inhibitors are a mainstay of therapy for patients with coronary artery disease (CAD) and have been shown to reduce recurrent major adverse cardiac events including heart attacks and cardiovascular death [1]. Newer P2Y12 inhibitors prasugrel and ticagrelor have demonstrated improved clinical outcomes compared to clopidogrel but are associated with higher costs and lower adherence [2]. In response to the rising costs of pharmaceuticals, there has been an increased interest in the development of value-based contracts for P2Y12 inhibitors that aim to increase the value and lower the price of medications by aligning incentives and sharing risk between payers and manufacturers [3]. In contrast to traditional volume-based pharmaceutical payment models, value-based contracts link drug payments to real-world, disease-related performance metrics [3]. Because there are numerous potential clinical outcomes in CAD, uncertainty remains regarding which CAD outcomes are most meaningful to all stakeholders—including patients and providers—and should be included in value-based contracts.

There have been three publicly disclosed value-based contracts for antiplatelet medications in the United States to date [4]. Two of the three base drug pricing on reduced hospitalizations due to cardiovascular events, while the third ensures shared coverage of treatment costs if the rate of heart attacks in patients taking the antiplatelet exceeds a certain threshold [5-7]. It is unclear how these outcome measures were chosen, and whether they are meaningful to all stakeholders affected by value-based contracts. While the Delphi method has been used in previous studies to reach provider expert consensus on clinical CAD indicators [8], to our knowledge, this method has not been used to identify meaningful CAD outcomes among all relevant stakeholders for the specific purpose of value-based contract development.

We therefore conducted a modified Delphi study incorporating multiple stakeholder perspectives, including those of patients, cardiologists, primary care physicians (PCPs), payers, pharmacy benefits manager (PBM) representatives, and pharmaceutical company representatives, to identify the most meaningful CAD outcomes to inform value-based contracts for antiplatelet medications.

Methods

We administered a modified Delphi survey using well-established methods to assess the value of various CAD indicators to diverse stakeholders [9, 10]. The Delphi technique employs anonymous, iterative questionnaires to reach expert consensus, allowing all participants to weigh in equally without dominance of the discussion by a small subset of individuals [9]. Between survey rounds, group responses are aggregated and incorporated into the next round’s questionnaire, where participants are given the option to reconsider and revise their previous selections. This process continues until consensus is reached. Unlike a traditional Delphi in which the first round consists of an open-ended solicitation of ideas [10], this modified Delphi utilized a list of CAD indicators established in advance of study onset, as discussed below. This study was deemed exempt by the University of Pittsburgh Institutional Review Board. All procedures performed were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki Declaration and its later amendments.

Outcome Selection

We performed an extensive literature search to identify the most pertinent CAD indicators [1, 8, 11–15]. A resultant list of 16 outcome measures was generated to span both traditional clinical endpoints examined in CAD trials for antiplatelet medications (e.g., myocardial infarction, stroke, death, major bleeding) [1, 8, 11, 13, 14] as well as patient-reported outcomes related to symptom burden (e.g., quality of life, function, shortness of breath, chest pain, fatigue) [8, 12, 15]. A pilot survey that included these 16 indicators unanimously achieved face validity without revision by a multidisciplinary, internal workgroup of cardiologists, PCPs, pharmacists, and clinical care managers.

Delphi Participants

Participants were recruited to represent multiple value-based contract stakeholder viewpoints with a deliberate majority stake given to patients and providers, whose perspectives are often underrepresented in standard value-based contracts between manufacturers and payers [16]. We assembled a diverse patient stakeholder group with regard to duration of time since CAD diagnosis; history of cardiovascular procedures (heart surgery, placement of a heart stent, neither procedure, or both procedures); and health insurance products (commercial, Medicaid, or Medicare). All patient stakeholders were taking an antiplatelet medication and were participants of the UPMC Health Plan Pharmacy Care Management Program, an outreach initiative designed to improve treatment adherence and decrease unplanned care among CAD patients. A care manager recruited patients by phone to inform them of the research study, respond to questions or concerns, and obtain verbal consent for participation.

Non-patient stakeholders were recruited by e-mail, including cardiologists and PCPs with expertise in CAD from two large, distinct healthcare organizations, and representatives from the health insurance, PBM, and pharmaceutical manufacturer industries. We invited PCPs to participate on the panel in addition to cardiologists, since approximately 50% of ambulatory care visits for cardiovascular disease in the U.S. occur in PCP offices [17]. Given that PCPs manage a substantial proportion of patients with cardiovascular disease, we wanted to capture their perspective—in addition to cardiologists—regarding the real-world outcomes of CAD patients. All non-patient participants were identified as having extensive CAD experience by either affiliate account representatives (pharmaceutical company and PBM representative stakeholders), clinical departmental leaders (cardiologist and PCP stakeholders), or by an internal workgroup (payer stakeholders). The provider stakeholder group was made up of experienced CAD clinicians and included department leaders, health services researchers, and quality improvement champions. Pharmaceutical company representative stakeholders managed antiplatelet medication accounts, pharmacy benefits manager stakeholders oversaw contracting for antiplatelet medications, and payer stakeholders were involved in clinical and pharmacy operations for patients with CAD. The targeted size of our Delphi panel was based on general guidelines found in the literature [9, 10].

Patients were paid $20 upon conclusion of Round 1, and $30 upon conclusion of Round 2. Other participants were not financially compensated.

Survey Administration

Patients opted to complete either an online questionnaire through SurveyMonkey® or a paper questionnaire via US mail (Supplemental Figure S1). Non-patient participants completed the survey online. Unique identifiers were assigned to all participants to ensure anonymity. The study team sent biweekly reminder e-mails and made follow-up calls as needed to encourage participants to complete the survey within a 2-week timeframe. All data were collected between January and February 2018.

Round 1

A brief survey introduction stated the study’s purpose and described the Delphi method. Participants were asked to indicate which stakeholder group they represented, and patients were subsequently asked whether they have a heart stent and/or have undergone heart surgery (including open heart surgery). Participants were then prompted to rate each CAD outcome using a five-point Likert scale, with 1 denoting an outcome as “not important” and 5 denoting an outcome as “very important” in terms of its value in reflecting CAD disease status. A plain language definition was provided for each outcome. Patients who reported prior stent placement (with or without heart surgery) were asked to rate all 16 outcomes, including “preventing stent blockage” and “preventing need for a repeat heart procedure” (Supplemental Figure S1). Patients who reported prior heart surgery but no stent were asked to rate 15 outcomes, in which “preventing stent blockage” was omitted. All other participants were asked to rate the remaining 14 outcomes. In anticipation that participants might rate all indicators as “important” or “very important,” we then requested that stakeholders choose the top three most meaningful outcomes. An optional, open-ended response field was provided to gather additional outcome suggestions.

Round 2

Stakeholders who completed Round 1 were mailed or e-mailed the Round 2 survey, which included all Round 1 outcomes sans the three lowest rated. As in previous Delphi studies [9], the group’s aggregate Round 1 response rating of each indicator was displayed alongside the individual’s Round 1 rating. For example, if a participant rated a given outcome as “very important” in Round 1 and the collective mean for that outcome was “4.25,” the outcome in Round 2 would be accompanied by the text: “Group average from Round 1 was 4.25; Your response was 5.” Stakeholders were then given the option to adjust their prior responses based on this additional information.

Similarly, for the follow-up question prompting participants to choose the three most meaningful indicators, individual and aggregate group responses from Round 1 were shown beside each item in Round 2. For example, beside the third most popular chosen indicator, text might read, “This option was the third most popular from the group from Round 1; You also chose this answer.”

Consensus and Statistical Analyses

Based on literature [18], consensus was defined as ≥ 75% agreement by stakeholders on the importance of an indicator (Likert scores 4 and 5), or selection of an indicator as most meaningful by ≥ 75% of stakeholders. A two-sample test of proportions was performed to identify statistically significant differences in outcome rankings between patient and non-patient stakeholders.

Results

Thirty (30) individuals, including ten (33.3%) patients, ten (33.3%) providers, four (13.3%) payers, four (13.3%) PBM representatives, and two (6.7%) pharmaceutical company representatives expressed interest in participating in the study and were sent the first-round questionnaire. Of these, 27 (90%) completed the Round 1 survey, and 25 of those 27 (93%) completed Round 2. The final Delphi panel consisted of nine (36%) patients, nine (36%) providers, two (8%) payers, three (12%) PBM representatives, and two (8%) pharmaceutical company representatives. Provider experts included five PCPs and four cardiologists, each with at least 5 years of clinical CAD experience. Of the nine patient participants, four indicated that they had a previous heart stent procedure but not a previous heart surgery, one had a previous heart surgery but not a heart stent procedure, three had both a stent procedure and heart surgery, and one had neither procedure. None of the panelists submitted additional indicators to include. “Reducing depression symptoms,” “preventing missed days of work,” and “reducing medication complications like minor bleeding” were ranked the lowest in importance in Round 1 and were removed from the Round 2 survey.

After two Delphi rounds, 11 of 13 outcomes were rated as “important” or “very important” (4 or 5 on the Likert scale) by over 75% of panelists (Table 1). Two of these outcomes (“preventing heart attacks” and “preventing need for a repeat heart procedure”) received a score of 4 or 5 by all stakeholders in Round 2 (100% agreement; Table 1). When asked to select only the top three most meaningful outcomes, 80% of participants chose “preventing heart attacks” and 76% chose “preventing death,” both reaching consensus after the second round (Table 2, last column). Agreement increased between Round 1 and Round 2 for these top two measures, corresponding to decreased selection of several other measures, most notably “reducing medication complications like major bleeding” and “preventing need for a repeat heart procedure” (Table 2).

Table 1

Importance of CAD outcome measures

Outcome measures	Not important/slightly important (1–2)		Moderately important (3)		Important/very important (4–5)
	Round 1	Round 2	Round 1	Round 2	Round 1	Round 2
Preventing heart attacksa	0 (0%)	0 (0%)	0 (0%)	0 (0%)	27 (100%)	25 (100%)
Preventing need for a repeat heart procedurea,b	0 (0%)	0 (0%)	1 (3.8%)	0 (0%)	25 (96.2%)	24 (100%)
Preventing deatha	0 (0%)	0 (0%)	2 (7.4%)	1 (4.0%)	25 (92.6%)	24 (96.0%)
Preventing stroke or mini-strokea	1 (3.7%)	1 (4.0%)	1 (3.7%)	0 (0%)	25 (92.6%)	24 (96.0%)
Preventing heart failure from heart diseasea	1 (3.7%)	0 (0%)	1 (3.7%)	2 (8.0%)	25 (92.6%)	23 (92.0%)
Reducing medication complications like major bleedinga	1 (3.7%)	1 (4.0%)	2 (7.4%)	1 (4.0%)	24 (88.9%)	23 (92.0%)
Improving quality of lifea	1 (3.7%)	0 (0%)	1 (3.7%)	2 (8.3%)	25 (92.6%)	22 (91.7%)
Improving functiona	1 (3.7%)	0 (0%)	1 (3.7%)	2 (8.3%)	25 (92.6%)	22 (91.7%)
Reducing shortness of breatha	0 (0%)	0 (0%)	3 (11.1%)	2 (8.3%)	24 (88.9%)	22 (91.7%)
Preventing stent blockagea,b	2 (8.0%)	2 (8.7%)	1 (4.0%)	0 (0%)	22 (88.0%)	21 (91.3%)
Reducing chest paina	0 (0%)	0 (0%)	3 (11.1%)	3 (12.0%)	24 (88.9%)	22 (88.0%)
Preventing heart medication switches because of failure of treatment, side effects, or other issues with medications	2 (7.4%)	1 (4.0%)	7 (25.9%)	8 (32.0%)	18 (66.7%)	16 (64.0%)
Reducing fatigue	2 (7.4%)	2 (8.3%)	6 (22.2%)	7 (29.2%)	19 (70.4%)	15 (62.5%)

Participants responded to prompt: “When a medication is started for heart disease/coronary artery disease (CAD), how important are the following outcomes”

aConsensus (greater than or equal to 75% agreement) was reached after Round 2

bPatient-reported prior heart procedures: 4 stents; 1 heart surgery; 3 both stent and surgery; 1 neither stent nor surgery

Table 2

Most meaningful CAD outcome measures

Outcome measures	Patients, n (%)		Providers, n (%)		Payers, n (%)		PBMs, n (%)		Pharma reps, n (%)		Total, N (%)
	R1 (n = 9)	R2 (n = 9)	R1 (n = 10)	R2 (n = 9)	R1 (n = 3)	R2 (n = 2)	R1 (n = 3)	R2 (n = 3)	R1 (n = 2)	R2 (n = 2)	R1 (N = 27)	R2 (N = 25)
Preventing heart attacksa	6 (66.7)	6 (66.7)	6 (60.0)	7 (77.8)	2 (66.7)	2 (100)	3 (100)	3 (100)	2 (100)	2 (100)	19 (70.4)	20 (80.0)
Preventing deatha	4 (44.4)	6 (66.7)	8 (80.0)	8 (88.9)	1 (33.3)	0 (0)	3 (100)	3 (100)	2 (100)	2 (100)	18 (66.7)	19 (76.0)
Improving quality of life	4 (44.4)	4 (44.4)	7 (70.0)	6 (66.7)	1 (33.3)	1 (50.0)	0 (0)	0 (0)	0 (0)	0 (0)	12 (44.4)	11 (44.0)
Preventing stroke or mini-stroke	5 (55.6)	3 (33.3)	1 (10.0)	2 (22.2)	1 (33.3)	1 (50.0)	3 (100)	3 (100)	2 (100)	1 (50.0)	12 (44.4)	10 (40.0)
Preventing need for a repeat heart procedureb	3 (37.5)	2 (25.0)	2 (20.0)	1 (11.1)	1 (33.3)	1 (50.0)	0 (0)	0 (0)	0 (0)	0 (0)	6 (23.1)	4 (16.7)
Preventing heart failure from heart disease	2 (22.2)	1 (11.1)	2 (20.0)	1 (11.1)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	1 (50.0)	4 (14.8)	3 (12.0)
Improving function	0 (0)	0 (0)	2 (20.0)	1 (11.1)	1 (33.3)	1 (50.0)	0 (0)	0 (0)	0 (0)	0 (0)	3 (11.1)	2 (8.0)
Reducing chest painc	1 (11.1)	2 (22.2)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	1 (3.7)	2 (8.0)
Reducing shortness of breathc	1 (11.1)	2 (22.2)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	1 (3.7)	2 (8.0)
Reducing fatigue	1 (11.1)	1 (11.1)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	1 (3.7)	1 (4.0)
Preventing heart medication switches because of failure of treatment, side effects, or other issues with medications	0 (0)	0 (0)	0 (0)	1 (11.1)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	1 (4.0)
Reducing medication complications like major bleeding	0 (0)	0 (0)	2 (20.0)	0 (0)	1 (33.3)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	3 (11.1)	0 (0)
Preventing stent blockageb	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)

Participants responded to prompt: “If we had to limit the number of heart disease/CAD measures to 3 that are most meaningful, which ones would you choose to keep (please select only 3)?”

aConsensus (greater than or equal to 75% agreement) was reached after Round 2

bPatient-reported prior heart procedures: 4 stents; 1 heart surgery; 3 both stent and surgery; 1 neither stent nor surgery

cStatistically significant difference found between patient and non-patient stakeholder groups; p = 0.049 for each of two outcomes noted

A two-sample test of proportions identified statistically significant differences between patient and non-patient stakeholders in the ranking of two outcomes as most meaningful. Specifically, “reducing chest pain” and “reducing shortness of breath” were each selected as a most meaningful outcome by 22.2% of patients but by 0% of non-patient stakeholders in Round 2 (p = 0.049, Table 2).

Discussion

To our knowledge, this is the first Delphi-based study performed among a diverse panel of expert stakeholders to identify meaningful CAD outcome measures for the specific purpose of informing value-based pharmaceutical contract development. After two survey rounds, participants reached consensus in ranking “preventing heart attacks” and “preventing death” as the two most meaningful outcomes.

For value-based contracting to evolve and deliver on the promise of improved care at lower costs, there needs to be greater process transparency, including in the selection of meaningful disease outcomes. Previous Delphi studies have been conducted to reach expert consensus on clinical CAD indicators, including an initiative by the nonprofit International Consortium for Health Outcomes Measurement (ICHOM) that sought to define a consensus standard set of measures for tracking, comparing, and improving the outcomes of CAD care [8]. A working group of 17 CAD experts and patients identified a core set of 13 CAD outcomes that they recommend should be clinically monitored in a standardized manner, which included occurrence of major adverse cardiac events such as myocardial infarction, heart failure, stroke, renal failure, and death, as well as patient-reported outcomes such as quality of life, shortness of breath, chest pain, depression, and functional status [8]. While monitoring this comprehensive list of indicators is beneficial in a clinical setting, measuring 13 outcomes would be onerous and impractical in the context of value-based contracting between payers and pharmaceutical manufacturers. Notably, the ICHOM Delphi panel did not include key value-based contract stakeholders (i.e., PBM representatives, pharmaceutical company representatives, or payers), and thus does not address the question of which measures are considered most meaningful for value-based contract development. Nevertheless, the ICHOM core outcome set substantially informed the development of our Delphi questionnaire.

Heart attack, stroke, and cardiovascular death, as well as major and minor bleeding events, are endpoints typically used in modern clinical trials for antiplatelet therapy [13, 14]. While it is unclear how previous value-based contracts for antiplatelet medications selected their outcomes, their foci on the incidence of heart attacks and hospitalizations due to cardiovascular events suggest the perceived value of these indicators in assessing drug effectiveness in a manner that is feasible to collect through administrative claims data [5-7]. Our finding that “preventing heart attacks” ranks as the most meaningful outcome among our diverse stakeholder group verifies and supports the utility of this measure in future value-based contracts for CAD medications.

While it may seem commonsensical that patients would find these clinical outcomes to be of high importance, it has been shown that what patients value can differ from conventional clinical endpoints [12]. Performing a Delphi-based assessment that incorporates multiple stakeholder opinions can shed light on such differences. Interestingly, a greater proportion of patients in our study chose “reducing chest pain” and “reducing shortness of breath” as most meaningful outcomes compared to non-patient panelists. While these indicators were selected as most meaningful by only a small minority of patients, this divergence highlights the possibility that what patients may perceive as desired or expected disease outcomes may be overlooked by non-patient stakeholders when defining how the value of medications is evaluated. While the incorporation of patient-reported outcomes in value-based pharmaceutical contracts may present unique challenges to payers and manufacturers, it is important that decision-makers take into consideration the inclusion of such measures as indicators of drug value when they are identified as meaningful.

Our study has several limitations. First, because all patient panelists were recruited from our health plan’s internal Pharmacy Care Management Program, selection bias may have been introduced. Since this program outreaches to patients who have experienced myocardial infarction, coronary artery bypass graft surgery, and/or automatic implantable cardioverter-defibrillator placement and who have been referred for physical or behavioral health intervention, this cohort may have poorer health than the general CAD population. Conversely, patients who participate in the program and who replied to our questionnaires may be more engaged than others. Therefore, the directional effect of this potential selection bias is unclear. Nonetheless, we made a purposeful effort to include a diverse patient group by recruiting patients with different cardiovascular procedure histories, years living with CAD, and insurance products. Another limitation was that all provider participants are located in southwestern Pennsylvania, and therefore their input may not be representative of a broader provider population. Although geographic diversity was not represented among providers, both cardiologists and PCPs across two health systems were included on the panel to capture potential differences in opinion between generalists and specialists with expertise in CAD care. While it was beyond the scope of this study, the conduct of a multi-center, multi-regional Delphi might illuminate potential differences in opinion related to varied local and professional cultural perspectives. In addition, not all individuals who originally expressed interest in participating responded to both survey rounds, and consequently a subset of viewpoints were potentially not represented in the final results. Notably, our response rates of 90% in Round 1 and 93% in Round 2 are consistent with what is typically reported for Delphi studies [9], and participation was sufficient to achieve our study goals. Lastly, while our stakeholder panel was similarly sized if not larger than those described in the literature [9], it was comprised of a relatively small number of non-provider, non-patient representatives. Including a larger sample of individuals in each stakeholder group may have resulted in a different outcome, however, we purposefully over-sampled patients and providers proportional to other stakeholder groups because patient and provider viewpoints are often underrepresented in typical value-based contracts between payers and manufacturers [16].

Conclusions

Our finding that “preventing heart attacks” ranks as the most meaningful outcome among our diverse stakeholder panel verifies and supports the utility of this widely used clinical measure in future value-based contracts for CAD medications.

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 226 kb)