Risk, Results, and Costs: Optimizing Clinical Trial Efficiency through Prognostic Enrichment.

When designing and conducting a clinical trial, investigators use “enrichment” strategies to efficiently address clinical questions. Defined by the U.S. Food and Drug Administration as “the prospective use of any patient characteristic to select a study population in which detection of a drug effect (if one is in fact present) is more likely than it would be in an unselected population” (1), enrichment can be practical, predictive, and/or prognostic (2). Practical enrichment attempts to reduce “noise” by selecting a homogeneous study population, excluding patients who are likely to discontinue the intervention and including those who can be reliably assessed for trial endpoints. Predictive enrichment seeks to identify and include patients who are more likely to benefit from the intervention for mechanistic reasons. Finally, prognostic enrichment, the strategy proposed and examined in this issue of the Journal by Scott and colleagues (pp. 726–736), aims to reduce sample size for event-driven trials by including patients at a high risk for events of interest (3).

In an event-driven trial, statistical power depends on the effect size of the intervention and the number of events that occur in the control group. Thus, given an assumed effect size (represented by a relative risk or hazard ratio), a specific number of events needs to be observed for a trial to have adequate statistical power. If events driving the primary endpoint are uncommon or the event times are long event-driven trials, it can require exceptionally large numbers of subjects and long follow-up periods, increasing costs and reducing feasibility. With this challenge in mind, the Food and Drug Administration recently suggested prognostic enrichment and provided a number of examples in a guidance document on clinical trials, at the same time acknowledging that “whether these strategies are useful as enrichment tools is not yet established” (1).

Though prognostic enrichment has a long history in clinical trials (4), important work is required to identify the best prognostic enrichment strategy for a given condition. Some prognostic risk scores may perform better than others, and the threshold used to define “high-risk” must be calibrated. Choose too high a threshold and the study population may be exceptionally difficult to enroll. One cannot enroll only the top percentile in risk, for example, without excluding 99% of those with the condition of interest. Choose too low a standard and the “high-risk” group may not be sufficiently high risk. Scott and colleagues (3) therefore compared three previously published risk scores to identify patients with pulmonary arterial hypertension (PAH) who are most likely to experience a clinical worsening event, and they then simulated sample size and treatment time reductions that would result from using these scores to enrich the study population during a PAH treatment trial. Specifically, the authors used data collected during three recent PAH trials—all of which used time to clinical worsening as the primary endpoint—to arrive at their suggested PAH study population through three steps:

They compared the predictive value of three published risk scores (COMPERA [Comparative, Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension], French [French pulmonary hypertension registry score], and REVEAL 2.0 [the U.S. Registry to Evaluate Early and Long-Term PAH Disease Management]) using receiver operator characteristics curves.

They constructed risk groups (e.g., high, intermediate, etc.) for each of the risk scores using a survival tree approach, examining a range of cut points, and selecting cut points that produced the largest differences in the predictive probabilities of events.

They estimated potential cost savings that would result from using three different enrichment strategies (high risk only, a 50/50 mix of high risk and non–high risk, and a 50/50 mix of high and intermediate risk). The estimated costs accounted for trial sample size, the time each subject would receive the study treatment, and screening costs.

Scott and colleagues conclude that the greatest cost savings (reducing the total trial cost by 40%) would be achieved by enrolling a 50/50 mix of patients at intermediate risk/high risk, with risk defined by the REVEAL 2.0 score (5). These results are likely to benefit not only investigators conducting future PAH trials who could employ this specific prognostic enrichment strategy but also those designing event-driven clinical trials in other areas, which could follow the approach outlined by Scott and colleagues to identify a cost-saving enrichment strategy for other conditions. When doing so, however, investigators should keep several points in mind.

First, Scott and coworkers found that cost savings were achieved through a combination of a reduced sample size and shorter treatment times because patients who have events faster will individually be in the trial for less time. These cost savings were partially offset by a substantially higher number of screen failures that occur when enrollment is restricted to patients at a high and intermediate risk. This increases screening costs and, perhaps more importantly, the amount of time (and/or additional study sites) needed to recruit eligible patients. Investigators should therefore carefully consider their own cost paradigm and enrollment rates to verify anticipated cost savings. In addition, because Scott and colleagues examined multiple risk scores, multiple cut points, and multiple enrichment strategies, the anticipated cost savings they report are likely overestimated to an unknown degree.

Second, investigators should note important differences between prognostic and predictive enrichment. The goals of prognostic enrichment (to increase the event rate) and predictive enrichment (to increase the intervention’s effect size) are often complementary, with effect size either not varying with risk or increasing among those at higher risk, as has been noted for some critical care interventions, including corticosteroids for severe coronavirus disease (COVID-19) (6). This relationship, however, is not guaranteed. It is possible, as noted by Scott and colleagues, that an intervention’s mechanism of action might provide greater benefit to patients at a lower risk (7), in which case what is gained by increasing the event rate may be lost through a lower treatment effect in the selected population. In addition, the results (regarding either efficacy or safety) of a trial conducted in a prognostically enriched population may not translate to the patients at a lower risk; the benefit/risk tradeoff might be substantially different. If, for example, the percentage of patients who experience a severe side effect is fixed at 0.5% and the drug’s benefit leads to a relative risk of 50%, then the risk/benefit tradeoff will be more favorable in a high-risk population, in which event rates might drop from 10% to 5%, than in a low-risk group, in which event rates might drop from 0.2% to 0.1%.

The current COVID-19 pandemic, during which new, rapidly developed, and high-quality evidence is desperately needed to inform medical decision making, has highlighted for the broader public what clinical trialists have known for decades: a multitude of difficult decisions must be made when designing and conducting a clinical trial, with numerous important tradeoffs being considered. Investigators who design PAH trials, and likely those studying many other conditions, are now better informed about the potential benefits of prognostic enrichment thanks to the work presented by Scott and colleagues (3). We hope and expect their study to not only inform the design of PAH trials but to also prompt additional research that will inform and advance clinical trial design in the future.