Comment on: Suissa and Azoulay. Metformin and the risk of cancer: time-related biases in observational studies. Diabetes Care 2012;35:2665-2673.

In a recent article in Diabetes Care, Suissa and Azoulay (1) concluded that the impressive results of the metformin-associated reduced cancer risk were due to many researchers failing to adjust for immortal time bias and not using time-dependent analysis of drug exposure. However, this conclusion is not justified since it remains controversial whether immortal time would introduce substantial bias.

We used statins and their effect on cardiovascular disease (CVD) to illustrate how different analyses could yield different results in pharmacoepidemiological studies. In a time-fixed Cox model, statin use was associated with a hazard ratio (HR) of 0.66 (95% CI 0.50–0.88) for CVD, an effect size similar to that in randomized trials despite 48% of the total follow-up time in the statin users being immortal time (i.e., without drug exposure). Herein, immortal time had introduced two sources of bias: 1) the nonexposure to statins misclassified as “exposed” that tended to inflate HR and 2) the nil risk of CVD in statin users during the immortal time periods misclassified as being at the same risk of nonusers that deflated the HR. Since these two sources of bias tended to neutralize each other, the HR of 0.66 was close to the real effect demonstrated in randomized studies (2).

On the other hand, if we used the time-dependent statin exposure analysis as proposed by Suissa (3), we obtained an HR of 1.74 (95% CI 1.30–2.31). If we further applied the immortal time-correcting formula suggested by Suissa (3), the HR was 1.47 (1.12–1.96) (2), i.e., increased CVD risk with statin use. Nevertheless, if the same immortal time periods of the statin users were added to the follow-up periods of the nonstatin users, the HR was decreased to 0.23 (0.14–0.36) (2).

Based on the recommendation of Rothman and Suissa (4), we excluded the immortal time periods among metformin users and reestimated the HR of metformin use for cancer using published data (5). By reestimating covariables at the time of initiation of metformin treatment during follow-up, we obtained a multivariable HR of 0.57 (95% CI 0.37–0.86) for cancer risk with metformin use. The additive interaction between nonuse of metformin and HDL cholesterol <1.0 mmol/L remained significant (multivariable attributable proportion due to interaction (AP): 0.48 (0.11–0.84), P < 0.05. Using this as the gold standard, we noted that time-dependent Cox model analysis yielded a multivariable HR of 0.97 (0.63–1.50) while time-fixed Cox model analysis yielded a multivariable HR of 0.40 (0.26–0.60). In other words, the HR was underestimated by 30% using time-fixed Cox model analysis and overestimated by 70% using the time-dependent Cox model.

In conclusion, our data show that immortal time bias, especially the proposed time-dependent drug exposure analysis, remains controversial. Using time-dependent drug exposure analysis to judge the scientific merits of pharmacoepidemiological studies of drug effects will only lead to more confusion rather than clarity with negative impacts on clinical practice and research. Here, we call for validation of the method using a drug with a known effect before reporting drug effects on cancer in diabetes.