Quasi-experimental longitudinal designs to evaluate drug benefit policy changes with low policy compliance.

A causal relation between drug benefit policy change and the increase in adverse outcomes can be tested by comparing the experience of a group of patients affected by the policy vs. the (counterfactual) experience of the same patients if the policy had not been implemented. Because counterfactual experiences cannot be observed, it must be assumed that the counterfactual is correctly described by extrapolating from the same population's previous experience. The null hypothesis of no policy effect can be empirically tested using quasi-experimental longitudinal designs with repeated measures. If compliance to a policy is low, results may be biased towards the null, but a subgroup analysis of compliers may be biased by nonignorable treatment selection. Using the example of reference drug pricing in British Columbia we discuss assumptions for causal interpretations of such analyses, and provide supplementary analyses to assess and improve the validity of findings. Results from nonrandomized comparisons of subgroups defined by their compliance to a policy change should generally be interpreted cautiously, and several biases should be explored.