Commentary: Propensity score methods, causal inference, and hazard ratios.

Adin-Cristian Andrei, PhD

In studies that compare overall survival via hazard ratios, a causal interpretation of the results is potentially problematic due to the risk of covariate imbalance after the first failure time.

See Article page 195.

Kainuma and colleagues examine the impact of multiple coronary percutaneous interventions (PCIs) on overall survival and clinical outcomes in patients requiring surgical correction of chronic ischemic mitral regurgitation. The observational nature of the study serves as justification for adopting an approach to statistical analysis that relies on inverse probability of treatment weighting (IPTW). Weighting involves the propensity score, defined as the conditional probability of having undergone at least 2 PCIs. Cox models and resulting hazard ratios are used to justify the findings.

Two aspects deserve particular attention. The first one is the authors' declared intent is to evaluate the “average treatment effect” of multiple PCIs on outcomes, thus casting the results in causal inference terms. The title itself affirms the causal interpretation by stating that multiple PCIs worsen outcomes. The second aspect of attention concerns the manner in which covariate balance following IPTW is evaluated.

Causal interpretation of hazard ratios is not without complications and still a topic of debate in the biostatistical literature.3, 4, 5 As Hernán pointed out, a causal interpretation for hazard ratios as an average treatment effect is not a simple translation of the concept from the linear regression to Cox regression. In randomized studies, Aalen and colleagues demonstrate that the standard Cox model hazard ratio is not a causal treatment effect when heterogeneity remains unmodeled. The root cause is the potential covariate imbalance among control and treatment patients who are still at-risk after the first failure time. Baseline balance conferred by randomization in the original groups might thus be lost. Martinussen and colleagues expand on these matters and introduce modified hazard ratios that support a causal interpretation. Therefore, the standard hazard ratios derived by Kainuma and colleagues do not support a causal interpretation as average treatment effect for PCI intervention but maintain correct interpretation in associative terms: multiple PCIs do not worsen outcomes, they are associated with worse outcomes.

A second aspect deserving attention is the adjudication of acceptable covariate balance after propensity score matching or IPTW. Kainuma and colleagues use standardized mean differences (SMDs) and, following general practical guidance, consider absolute value SMDs <0.25 as adequate balance. The choice of 0.25 or any other threshold remains a subjective call. An alternative developed by Andrei and McCarthy involves an omnibus approach for overall covariate balance assessment. Instead of evaluating each covariate separately, this approach is to evaluate overall covariate imbalance in one sweep. They show that although no covariate exceeds an SMD threshold, substantial overall imbalance is still possible. Conversely, there are cases in which the SMD threshold is exceeded by multiple covariates, yet an overall imbalance is not detectable via this omnibus approach. It is therefore necessary to adopt SMD thresholds tailored to the subject matter at hand and to account for any residual imbalance via regression models.

In conclusion, Kainuma and colleagues present exciting new findings and an opportunity to discuss some subtle aspects concerning the causal interpretation of PSM and IPTW analyses.