A Note on the Validity and Reliability of Multi-Criteria Decision Analysis for the Benefit-Risk Assessment of Medicines.

The comparative evaluation of benefits and risks is one of the most important tasks during the development, market authorization and post-approval pharmacovigilance of medicinal products. Multi-criteria decision analysis (MCDA) has been recommended to support decision making in the benefit-risk assessment (BRA) of medicines. This paper identifies challenges associated with bias or variability that practitioners may encounter in this field and presents solutions to overcome them. The inclusion of overlapping or preference-complementary criteria, which are frequent violations to the assumptions of this model, should be avoided. For each criterion, a value function translates the original outcomes into preference-related scores. Applying non-linear value functions to criteria defined as the risk of suffering a certain event during the study introduces specific risk behaviours in this prescriptive, rather than descriptive, model and is therefore a questionable practice. MCDA uses weights to compare the importance of the model criteria with each other; during their elicitation a frequent situation where (generally favourable) mild effects are directly traded off against low probabilities of suffering (generally unfavourable) severe effects during the study is known to lead to biased and variable weights and ought to be prevented. The way the outcomes are framed during the elicitation process, positively versus negatively for instance, may also lead to differences in the preference weights, warranting an appropriate justification during each implementation. Finally, extending the weighted-sum MCDA model into a fully inferential tool through a probabilistic sensitivity analysis is desirable. However, this task is troublesome and should not ignore that clinical trial endpoints generally are positively correlated.