Decision-makers' preferences for approving new medicines in Wales: a discrete-choice experiment with assessment of external validity.

BACKGROUND: Few studies to date have explored the stated preferences of national decision makers for health technology adoption criteria, and none of these have compared stated decision-making behaviours against actual behaviours. Assessment of the external validity of stated preference studies, such as discrete-choice experiments (DCEs), remains an under-researched area.
OBJECTIVES: The primary aim was to explore the preferences of All Wales Medicines Strategy Group (AWMSG) appraisal committee and appraisal sub-committee (the New Medicines Group) members ('appraisal committees') for specific new medicines adoption criteria. Secondary aims were to explore the external validity of respondents' stated preferences and the impact of question choice options upon preference structures in DCEs.
METHODS: A DCE was conducted to estimate appraisal committees members' preferences for incremental cost effectiveness, quality-adjusted life-years (QALYs) gained, annual number of patients expected to be treated, the impact of the disease on patients before treatment, and the assessment of uncertainty in the economic evidence submitted for new medicines compared with current UK NHS treatment. Respondents evaluated 28 pairs of hypothetical new medicines, making a primary forced choice between each pair and a more flexible secondary choice, which permitted either, neither or both new medicines to be chosen. The performance of the resultant models was compared against previous AWMSG decisions.
RESULTS: Forty-one out of a total of 80 past and present members of AWMSG appraisal committees completed the DCE. The incremental cost effectiveness of new medicines, and the QALY gains they provide, significantly (p < 0.0001) influence recommendations. Committee members were willing to accept higher incremental cost-effectiveness ratios and lower QALY gains for medicines that treat disease impacting primarily upon survival rather than quality of life, and where uncertainty in the cost-effectiveness estimates has been thoroughly explored. The number of patients to be treated by the new medicine did not exert a significant influence upon recommendations. The use of a flexible-choice question format revealed a different preference structure to the forced-choice format, but the performance of the two models was similar. Aggregate decisions of the AWMSG were well predicted by both models, but their sensitivity (64 %, 68 %) and specificity (55 %, 64 %) were limited.
CONCLUSIONS: A willingness to trade the cost effectiveness and QALY gains against other factors indicates that economic efficiency and QALY maximisation are not the only considerations of committee members when making recommendations on the use of medicines in Wales. On average, appraisal committee members' stated preferences appear consistent with their actual decision-making behaviours, providing support for the external validity of our DCEs. However, as health technology assessment involves complex decision-making processes, and each individual recommendation may be influenced to varying degrees by a multitude of different considerations, the ability of our models to predict individual medicine recommendations is more limited.