BACKGROUND: Payers frequently rely on budget impact model (BIM) results to help determine drug coverage policy and its effect on their bottom line. It is unclear whether BIMs typically overestimate or underestimate real-world budget impact. OBJECTIVE: We examined how different modeling assumptions influenced the results of 6 BIMs from the Institute for Clinical and Economic Review (ICER). STUDY DESIGN: Retrospective analysis of pharmaceutical sales data. METHODS: From ICER reports issued before 2016, we collected estimates of 3 BIM outputs: aggregate therapy cost (ie, cost to treat the patient population with a particular therapy), therapy uptake, and price. We compared these against real-world estimates that we generated using drug sales data. We considered 2 classes of BIM estimates: those forecasting future uptake of new agents, which assumed "unmanaged uptake," and those describing the contemporaneous market state (ie, estimates of current, managed uptake and budget impact for compounds already on the market). RESULTS: Differences between ICER's estimates and our own were largest for forecasted studies. Here, ICER's uptake estimates exceeded real-world estimates by factors ranging from 7.4 (sacubitril/valsartan) to 54 (hepatitis C treatments). The "unmanaged uptake" assumption (removed from ICER's approach in 2017) yields large deviations between BIM estimates and real-world consumption. Nevertheless, in some cases, ICER's BIMs that relied on current market estimates also deviated substantially from real-world sales data. CONCLUSIONS: This study highlights challenges with forecasting budget impact. In particular, assumptions about uptake and data source selection can greatly influence the accuracy of results.
BACKGROUND: Payers frequently rely on budget impact model (BIM) results to help determine drug coverage policy and its effect on their bottom line. It is unclear whether BIMs typically overestimate or underestimate real-world budget impact. OBJECTIVE: We examined how different modeling assumptions influenced the results of 6 BIMs from the Institute for Clinical and Economic Review (ICER). STUDY DESIGN: Retrospective analysis of pharmaceutical sales data. METHODS: From ICER reports issued before 2016, we collected estimates of 3 BIM outputs: aggregate therapy cost (ie, cost to treat the patient population with a particular therapy), therapy uptake, and price. We compared these against real-world estimates that we generated using drug sales data. We considered 2 classes of BIM estimates: those forecasting future uptake of new agents, which assumed "unmanaged uptake," and those describing the contemporaneous market state (ie, estimates of current, managed uptake and budget impact for compounds already on the market). RESULTS: Differences between ICER's estimates and our own were largest for forecasted studies. Here, ICER's uptake estimates exceeded real-world estimates by factors ranging from 7.4 (sacubitril/valsartan) to 54 (hepatitis C treatments). The "unmanaged uptake" assumption (removed from ICER's approach in 2017) yields large deviations between BIM estimates and real-world consumption. Nevertheless, in some cases, ICER's BIMs that relied on current market estimates also deviated substantially from real-world sales data. CONCLUSIONS: This study highlights challenges with forecasting budget impact. In particular, assumptions about uptake and data source selection can greatly influence the accuracy of results.