Impact of nonrandom selection mechanisms on the causal effect estimation for two-sample Mendelian randomization methods.

Nonrandom selection in one-sample Mendelian Randomization (MR) results in biased estimates and inflated type I error rates only when the selection effects are sufficiently large. In two-sample MR, the different selection mechanisms in two samples may more seriously affect the causal effect estimation. Firstly, we propose sufficient conditions for causal effect invariance under different selection mechanisms using two-sample MR methods. In the simulation study, we consider 49 possible selection mechanisms in two-sample MR, which depend on genetic variants (G), exposures (X), outcomes (Y) and their combination. We further compare eight pleiotropy-robust methods under different selection mechanisms. Results of simulation reveal that nonrandom selection in sample II has a larger influence on biases and type I error rates than those in sample I. Furthermore, selections depending on X+Y, G+Y, or G+X+Y in sample II lead to larger biases than other selection mechanisms. Notably, when selection depends on Y, bias of causal estimation for non-zero causal effect is larger than that for null causal effect. Especially, the mode based estimate has the largest standard errors among the eight methods. In the absence of pleiotropy, selections depending on Y or G in sample II show nearly unbiased causal effect estimations when the casual effect is null. In the scenarios of balanced pleiotropy, all eight MR methods, especially MR-Egger, demonstrate large biases because the nonrandom selections result in the violation of the Instrument Strength Independent of Direct Effect (InSIDE) assumption. When directional pleiotropy exists, nonrandom selections have a severe impact on the eight MR methods. Application demonstrates that the nonrandom selection in sample II (coronary heart disease patients) can magnify the causal effect estimation of obesity on HbA1c levels. In conclusion, nonrandom selection in two-sample MR exacerbates the bias of causal effect estimation for pleiotropy-robust MR methods.

1 Introduction

Mendelian randomization (MR) uses genetic variants as instrumental variables (IV) to obtain an unbiased causal effect estimation in the presence of unmeasured confounding [1]. MR analysis assumes that the genetic variants satisfy instrumental variable assumptions including IV Relevance (the IV must be robustly associated with the exposure), IV Independence (the IV must be independent of unmeasured confounders), and Exclusion restriction (the IV must not have a direct effect on the outcome that is not mediated by the exposure) [2-4]. Two-sample MR leverages the summary-level genetic associations of exposure and outcome from two non-overlapping datasets to estimate causal effect. These summary-level genetic associations can be obtained from published literature provided by consortia of genome-wide association studies (GWAS), or directly from individual-level participant data [5, 6].

Nonrandom selection in one-sample MR [7] can result in biased estimates and inflated type I error rates. The magnitude of bias can differ according to the strength of instruments, the complexity of exposure-instrument association, and the nature of exposure effects. When selection depends on instrumental variables (G), conducting the analysis on the selected sample does not lead to biased estimates because, as shown in Fig 1, G is not a collider (or a descendant of a collider) and thus does not induce a new open path between G and exposure (X) or G and outcome (Y). In contrast, because X is a collider in the path G→X←U→Y, when selection depends on X, conditioning on selection opens up a new path between G and U, which violates the IV Independence assumption. Similarly, when there is a causal effect of X on Y, Y is also a descendant of collider X, and selection that depends on Y will also open this path. Thus selection depending on Y will induce bias only for the non-null causal effect of X on Y [8]. When there is a null causal effect of X on Y, Y is no longer a descendant of collider X, and nonrandom selection based on Y does not induce any bias; hence there is no type I error inflation. Therefore, when selection depends on X, Y, or X and Y, estimates of the causal effect are biased [7].

Fig 1

Illustrative diagram of Mendelian randomization. G, j-th genetic variant, with effect ϕ on confounders U, a direct effect α on the exposure X and a direct effect γ on the outcome Y.

For two-sample MR analyses, we assume that genetic association statistics with the exposure (e.g. beta-coefficient and standard error (SE) ) are obtained from sample I and the genetic association statistics with the outcome (e.g. beta-coefficient and SE ) are obtained from sample II. Whether sample I or sample II is a random sample of the target population requires more attention. Two-sample MR analyses are more prone to be affected by nonrandom selection than one-sample MR analyses, in that two heterogeneous samples can be selected each under a different mechanism that differs from the source population [7]. Such sampling mechanisms include willingness to participate and survival to the participation date [1, 7]. This could lead to the violation of MR assumption. Recent IV-based estimators in MR studies include consensus methods, regression-based methods, likelihood-based methods, outlier-robust methods [9-16], which focus on relaxing the Exclusion restriction. Pleiotropy is characterized by genetic variants associated with multiple phenotypic variables that is common in MR studies and can lead to the violation of the Exclusion restriction. The extent to which these pleiotropy-robust methods can affect causal effect estimations under different selection mechanisms remains unclear.

In this study, we first propose sufficient conditions for causal effect invariance under different selection mechanisms using two-sample MR methods. We then consider 49 possible selection mechanisms in two-sample MR, which depend on G, X, Y and their combination, respectively. In the simulation, we compare eight pleiotropy-robust methods under different selection mechanisms. Finally, we use an application to explore the extent to which nonrandom selection influences the estimation of the causal effect of obesity on HbA1c levels.

2 Materials and methods

2.1 Modeling assumptions and summary level data

Let G = {G1,G2,…,G) denote J genetic variants that are mutually independent, and X, Y and U be the exposure, outcome and unmeasured confounder, respectively. We assume the model of Fig 1 is:

Each of the valid genetic variants must satisfy the three IV assumptions (Relevance, Independence and Exclusion restriction), as well as linearity, homogeneity, monotonicity and non-overlap assumptions [11, 17].

Linear and homogeneity assumption

Estimating the causal effect of X on Y in the full study population requires linearity of the IV-X, IV-Y and X-Y relationships. There is no effect heterogeneity in the X-Y relationships. Linearity for the IV-X association is necessary for point estimates but not for testing the null hypothesis.

Monotonicity assumption

Monotonicity in the context of MR means that increasing the number of effect alleles for an individual can only increase the exposure.

No sample overlap

Two-sample MR requires two non-overlapping samples to estimate causal effects. MR analyses using IV-X and IV-Y associations in the same sample or in partially overlapping samples may be prone to weak instrument bias towards the X-Y estimate that would be obtained using conventional methods. A simulation suggested that bias due to sample overlap is a linear function of the proportion of overlap between samples [17]. More details of modeling assumptions are shown in S1 Text.

2.2 Sufficient conditions for causal effect invariance

Our objective is to calculate the causal effect of X on Y using two-sample MR methods at the population-level distribution (i.e., Wald ratio method). The genetic association statistics with exposure (, j = 1,2,…,J) and outcome (, j = 1,2,…,J) can be obtained from sample I and sample II, respectively. Sample I and sample II are random samples from population I and population II, respectively. S1 and S2 are binary variables indicating whether a participant is selected or unselected in sample I and sample II, respectively. Restricting the analysis to the selected sample implies conditioning on S1 or S2 equal to one, which is represented by a box around S1 or S2. Due to the preferential selection, the estimation is . The natural question to ask is under what conditions the causal effect can be recovered by sample I and II with preferential selection, that is, , and the extent to which selection may affect the causal effect estimate.

Based on the classical instrumental variable assumptions [2-4], we propose the following theorem to explore the sufficient conditions for causal effect invariance in two-sample MR based on the Wald ratio method.

Theorem 1 The sufficient conditions for causal effect invariance under different selection mechanisms from two populations are:

for each valid instrumental variable G, S1⊥G or S1⊥X|G in population I and S2⊥G or S2⊥Y|G in population II, respectively, or

G⊥Y|S2 and G⊥Y for each valid instrumental variable in population II.

Theorem 1 provides sufficient conditions for causal effect invariance under different selection mechanisms using two-sample MR methods. We also provide Directed Acyclic Graphs (DAGs) that satisfied condition (a-b) in Theorem 1 (Figs 2 and 3). Three scenarios including no nonrandom selection mechanism, selection depending on unmeasured confounders and selection depending on genetic variants (Fig 2), satisfy the condition (a) in sample I or sample II. Two scenarios including selection depending on the outcome or genetic variants (Fig 3) in sample II satisfy the condition (b). The proof of this theorem is provided in S2 Text. In the case of multiple independent instrumental variables with selection (e.g., G→S and G→S), the selection will result in a spurious association between G and G. Inverse-variance weighting (IVW) with generalized least squares can reduce this bias [18].

Fig 2

Possible causal diagrams for condition (a) of Theorem 1.

Fig 3

Possible causal diagrams for condition (b) of Theorem 1.

When the exposure and outcome are binary, traditional MR methods can be used to determine whether there is a causal effect, but cannot estimate causal effect accurately [11]. In this case, the Wald ratio can be expressed as . In other words, beta-coefficients in the linear regression are replaced by log(OR)-coefficients in the logistic regression. We also provide sufficient conditions for the invariance of causal relationship using two-sample MR methods on the OR scale in Theorem 2 (S2 Text). Because the non-collapsibility [19], S1⊥G and S2⊥G in condition (a) are replaced by S1⊥G|X and S2⊥G|Y, respectively. In comparison with Theorem 1, OR can avoid the influence of outcome-dependent selection bias, especially in case-control study designs [20]. The difference for the DAGs satisfying Theorem 2 is that selection depending on the unmeasured confounder no longer satisfies the condition (a) in either sample I or sample II. Instead, selection depending on exposure in sample I or outcome in sample II satisfies condition (a) in Theorem 2. Theorem 2 and its proof as well as DAGs satisfying conditions (a-b), are provided in S2 Text.

2.3 Two-sample MR methods

Numerous pleiotropy-robust methods have been proposed in recent years. The third core assumption of MR would be violated if pleiotropy exists, that is, the pathway between the IVs and the outcome may not be via the exposure (X). There are two types of pleiotropy: horizontal and vertical pleiotropy. Vertical pleiotropy is that a single nucleotide polymorphism (SNP) influences one trait, which in turn influences another. Horizontal pleiotropy occurs when SNPs influence two traits through independent pathways [21]. For example, if there are selections depending on X, a noncausal pathway between G and Y via U (G→X←U→Y) will be unlocked. This directly results in the violation of the MR assumption and further make the instrumental variables invalid. Nowadays, many premiere MR studies feature new instrument-based estimators that do not, strictly speaking, require that all proposed instruments are valid instruments. We wonder whether these methods are robust when non-random selections exist.

We consider eight methods that can be classified into four main types: consensus methods, regression-based methods, likelihood-based methods and outlier-robust methods. The consensus methods take their causal estimate as a summary measure of the distribution of the ratio estimates (), including two methods: the weighted median method [10] and the mode-based estimate (MBE) method [14]. The regression-based methods regress the genetic associations with outcome against the genetic associations with exposure using a variety of regression methods, including IVW, MR-Egger [9] and MR-robust method [12]. The likelihood-based methods include the contamination mixture method [15] and MR-Robust Adjusted Profile Score (RAPS) [16]. We also study methods that remove outliers and then estimate the causal effect of exposure on the outcome, such as MR-Lasso method [22]. The details of the eight methods are provided in S3 Text.

2.4 Selection mechanisms in two-sample MR

We consider seven possible selection mechanisms which depend on G, X, Y and their combinations in two samples, respectively. A total of 49 nonrandom selection mechanisms are considered. The DAGs depicted in Fig 4 show the causal relationships among the variables in sample I and sample II of the MR analysis under different selection mechanisms. Fig 4A-G correspond to selection depending on X, Y, G, X+Y, G+X, G+Y, and G+X+Y, respectively. Note that we consider the selection mechanisms depending on Y in the GWAS analysis which aims to investigate the genetic association with X, and vice versa. For example, nonrandom selection depends on X in sample II unlocking the path G→X←U→Y thus misestimating the relationship of G-Y. We also consider the selection mechanism depending on G because nonrandom selection is based on another phenotype that the genetic variants also affect, that is pleiotropy, but not non random selection using observed genotyping data.

Fig 4

Direct acyclic graphs of two-sample Mendelian randomization analysis under seven different selection mechanisms in Sample I (S1) and Sample II (S2), respectively.

A-G corresponding to selection depending on X, Y, G, X+Y, G+X, G+X+Y, respectively.

2.5 Simulation settings

In order to compare the performances of the above eight methods, we generate the following datasets as shown in Fig 4. For the i-th individual, we have where e, e and e are allowed to take values of –2, –1, –0.5, 0, 0.5, 1 and 2. The genetic variants are modelled as SNPs with a minor allele frequency of 30%, and take on values of 0, 1 or 2. The error terms ε, ε and ε follow independent normal distributions with a mean 0 and unit variance. The selection S follows a binomial distribution with selection probability depending on the exposure, outcome, and genetic variants.

G~Binomial(2,0.3) j∈(1,⋯,J),

ε, ε and ε~N(0,1)

S~Binomial(1,π) where ,

We consider the following four scenarios:

No pleiotropy: The Instrument Strength Independent of Direct Effect assumption (InSIDE) satisfied–valid IVs with no direct effect on the outcome (γ = 0) and the unmeasured confounder (ϕ = 0).

Balanced pleiotropy, InSIDE satisfied: Invalid IVs (G) with direct effects on the outcome generated from a normal distribution centered at zero, i.e. γ~N(0,0.15), and genetic effects on the confounder are zero (ϕ = 0).

Directional pleiotropy, InSIDE satisfied: Invalid IVs (G) with direct effects on the outcome generated from a normal distribution centered away from zero, i.e. γ~N(0.1,0.15), and genetic effects on the confounder are zero (ϕ = 0).

Directional pleiotropy, InSIDE violated: Invalid IVs (G) with direct effects on the outcome generated from a normal distribution centered away from zero, i.e., γ~N(0.1,0.15), and indirect effects on the outcome via the unmeasured confounder, i.e., ϕ~U(0,0.1).

The causal effect of exposure on the outcome is either taken as null (θ = 0) or positive (θ = 0.2). Genetic associations with exposure α are drawn from a uniform distribution. Parameters are chosen such that the total proportion of variance explained in the exposure by direct effects of the genetic variants is approximately 10%. We simulate data on J = 50 and 100 genetic variants, and the proportion of invalid instrumental variables is 30% and 70%. We firstly generate two populations with 1,000,000 individuals, respectively. For each selection mechanism, 10,000 individuals are selected from above two populations. We generate 1,000 simulated datasets for each scenario.

In each scenario, we consider the following seven selection mechanisms (Fig 4A-G) with e denoting the selection effect in sample I and sample II, respectively.

The selection S depends on exposure (X), i.e. e = e, e = e = 0;

The selection S depends on outcome (Y), i.e. e = e, e = e = 0;

The selection S depends on genetic variants (G), i.e. e = e, e = e = 0;

The selection S depends on exposure (X) and outcome (Y), i.e. e = 0, e = e = e;

The selection S depends on exposure (X) and genetic variants (G), i.e. e = e = e, e = 0;

The selection S depends on genetic variants (G) and outcome (Y), i.e. e = e = e, e = 0;

The selection S depends on exposure (X), outcome (Y) and genetic variants (G), i.e. e = e = e = e.

A total of 49 nonrandom selection mechanisms are considered. For each scenario, we assess the performances of eight pleiotropy-robust methods based on biases, SEs, type I error rates and powers. The nominal level is set to 0.05.

2.6 Application example

Coronary heart disease (CHD) is the leading cause of death and disability and its prevalence is increasing worldwide [23]. Its modifiable risk factors, including obesity and HbA1c play important roles in CHD prevention [24-26]. Obesity, typically defined based on body mass index (BMI), as well as waist-to-hip ratio (WHR), is a leading cause of CHD in the population. WHR adjusted for BMI (WHRadjBMI) is a surrogate measure of abdominal adiposity and has been correlated with direct imaging assessments of abdominal fat. Emdin et al. found that a genetic predisposition to higher WHR adjusted for BMI is associated with an increased risk of CHD [26]. A MR study using UK Biobank revealed that HbA1c caused CHD [25]. A Network MR analysis inferred that a higher BMI conferred an increased risk of CHD, which was partially mediated by HbA1c [24]. We aim to explore whether the causal estimation of obesity on HbA1c are different in patients with CHD and the general population. The realistic causal diagram is shown in Fig A in S4 Text. Fig A1 in S4 Text shows the DAG for sample I. Figs A2 and A3 in S4 Text are the DAGs for sample II in general population and CHD patients respectively.

We use GWAS summary data on BMI [27], WHR and WHRadjBMI [28] for European descent from the Genetic Investigation of ANthropometric Traits (GIANT) by GWAS meta-analyses of 339,224 and 224,459 individuals, respectively. GIANT is an international collaboration that seeks to identify genetic loci that modulate human body size and shape by performing meta-analysis of GWAS data and other large-scale genetic datasets. We choose the SNPs with significant association with obesity (p<5×10−8), minor allele frequency (MAF) more than 5% and satisfying Hardy–Weinberg equilibrium (p>0.05). We prune the variants by linkage disequilibrium (LD) (r2>0.001).

We retrieve the individual data for HbA1c from the UK Biobank with a sample of 487,314 Europeans. The UK Biobank [29] is a prospective cohort study with rich genetic, physical and health data collected from more than 500,000 individuals (age range 40–69 years) across the United Kingdom in 2006–2010. To examine the bias of the causal effect estimation under a nonrandom selection mechanism, we also use a selected sample enriched for CHD patients with 26,765 individuals as the selected population. The HbA1c levels is measured by HPLC analysis on a Bio-Rad VARIANT II Turbo and is natural log-transformed to approximate normal distributions. CHD is defined by ICD-10 I20–I25.9 and self-reported as 1066. We performed GWAS analysis in both the general and CHD populations. Results are available for BMI, WHR and WHRadjBMI-associated leading SNPs for HbA1c. GWAS summary data for application can be found in S1 Data.

3 Results

3.1 Results of simulation

When the causal effect is zero (θ = 0), Fig 5 shows the tendency of estimations under different selection mechanisms while varying across the selection effects of X, Y or G (e, e, or e) in scenario 1. Each row represents one of the seven different selection mechanisms in sample I, and the columns represent seven different selection mechanisms in sample II. The first row of Fig 5 illustrates the simulation results when the selection mechanism depends on X in sample I and all selection mechanisms in sample II, respectively. The biases of all methods are negative and increase as the selection effect increases when the selection depends on G, X+Y, G+X, G+Y, G+X+Y in sample II. Among these eight methods, MR-Egger shows less biases than other methods, especially when selection depends on G+X, G+Y and G+X+Y. On the contrary, selections depending on Y and G in sample II show nearly unbiased causal effect estimations. When the selections depend on Y, G, X+Y, or G+Y in sample I, the biases of estimations show a similar tendency as that when selection depends on X. However, selections depending on G+X and G+X+Y in sample I show different results. When the selections depend on Y or G in sample II, all eight models also show unbiased causal effect estimations. When the selection depends on X, X+Y and G+X in sample II, the biases firstly increase then decrease with the selection effect increasing. In addition, the biases when selection depends on G+Y and G+X+Y firstly increase and then reduce to zero, and finally rise in the opposite direction with the selection effect increasing. In summary, the biases of selections depending on X+Y, G+Y and G+X+Y in sample II are larger than those of other selection mechanisms. When the selection mechanism in sample II is fixed, the different selection mechanisms in sample I show similar trends.

Fig 5

Simulation results for causal estimations of eight Pleiotropy-robust MR Methods varying across selection effect from -2 to 2 under different selection mechanisms with Null causal effect in scenario 1 (50 genetic variants).

Sel I and II represent selection in sample I and sample II, respectively.

Fig 6 shows the tendency of the SEs under different selection mechanisms. In general, the SEs of the MBE model are larger than those of other models. The SEs of selection depending on G+Y and G+X+Y in sample II are larger than those of other selection mechanisms regardless of the selection mechanism in sample I. Fig 7 displays the tendency of type I error rates under different selection mechanisms and simulation situations. Consistent with Fig 5, the type I error rates of selections depending on Y and G in sample II are close to 0.05 regardless of the selection mechanism in sample I. Furthermore, the type I error inflation can be observed under other selection mechanisms due to the biased causal effect estimations of X on Y.

Fig 6

Simulation results for standard error of eight Pleiotropy-robust MR Methods varying across selection effect from -2 to 2 under different selection mechanisms with Null causal effect in scenario 1 (50 genetic variants).

Fig 7

Simulation results for type I error rates of eight Pleiotropy-robust MR Methods varying across selection effect from -2 to 2 under different selection mechanisms with Null causal effect in scenario 1 (50 genetic variants).

When the causal effect is positive (θ = 0.2), Figs 8 and 9 display a similar tendency of estimations and SEs with Figs 5 and 6. Note that when the selection depends on Y in Sample I, the biases are larger than those in the case of null causal effect. Fig 10 shows the tendency of the power under different selection mechanisms. The eight methods cannot effectively reject the null hypothesis due to nonrandom selection. For SE, MBE has the worst performance among the eight methods.

Fig 8

Simulation results for causal estimation of eight Pleiotropy-robust MR Methods varying across selection effect from -2 to 2 under different selection mechanisms with Positive causal effect in scenario 1 (50 genetic variants).

Fig 9

Simulation results for standard error of eight Pleiotropy-robust MR Methods varying across selection effect from -2 to 2 under different selection mechanisms with Positive causal effect in scenario 1 (50 genetic variants).

Fig 10

Simulation results about statistic power of eight Pleiotropy-robust MR Methods varying across selection effect from -2 to 2 under different selection mechanisms with Positive causal effect in scenario 1 (50 genetic variants).

We further investigate the impact of different proportions of invalid IVs (30% and 70%), different numbers of total IVs (50 and 100 variants) and different pleiotropy scenarios (scenarios 1–4, described in section 3.1) on biases, SEs, type I error rates and statistical power. The results and details are displayed in S5–S8 Texts. When pleiotropy exists, the eight MR methods show large biases and the type I error inflation. Even in the scenarios of balanced pleiotropy, all the eight MR methods especially MR-Egger demonstrate large negative bias regardless of the null and positive causal effect. We have provided four spreadsheets (S1–S4 Tables) as supplementary materials giving the data points for all the Figs.

3.2 Results of application example

After the quality control process described in section 2.6, 33, 24 and 67 independent loci associated with BMI, WHR and WHRadjBMI, respectively, are included in our study. These SNPs can explain 0.41%, 0.14% and 0.15% (F statistics >>10) of the variance of the three exposures, respectively. We use the same SNPs in the general population and the selected population, for the latter population we select individuals who are CHD patients. We then retrieve the GWAS summary results on HbA1c from the UK Biobank. We consider the causal effects of BMI, WHR, and WHRadjBMI on HbA1c levels in the general population and patients with CHD, respectively. All analyses in our study are implemented by R package TwoSampleMR.

The results are shown in Fig 11. In the general population, there are strong evidences for positive causal associations of BMI, WHR and WHRadjBMI on HbA1c levels. This means that a high BMI and WHR can improve the HbA1c levels. And the eight MR methods demonstrate consistent results. The effect estimates are magnified in the patients with CHD. This verifies that the nonrandom selection in sample II bias the effect estimation.

Fig 11

Results of MR analysis of obesity on the risk of HbA1c levels.

Three columns from left to right represent MR results of body mass index (BMI), waist-to-hip ratio (WHR) and WHR adjusted for BMI (WHRadjBMI) on HbA1c levels, respectively. The red and blue nodes represent MR analysis in general population and coronary heart disease (CHD) patients, respectively.

4 Discussion

The goal of this study is to explore the influence of nonrandom selection mechanisms on causal effect estimation in two-sample MR methods. Our simulation results indicate that nonrandom selection mechanisms will lead to substantial bias in the MR estimation and inflated type I error rates. When all the instrumental variables are valid, the different selection mechanism in sample II are found to have larger influence on estimation than sample I. Selections depending on the combination of Y and other variables (G or X) in sample II lead to larger biases of estimation than other selection mechanisms. The type I error inflation can be observed under 49 different selection mechanisms. Notably when the causal effect is positive, selection depending on Y leads to a larger bias than the case of a null causal effect. None of the eight methods can effectively reject the null hypothesis due to selection bias. In particular, the MBE has the worst performance as its large SE. When pleiotropy exists, eight MR methods perform poorly. Even in the scenarios of balanced pleiotropy, all eight MR methods especially MR-Egger demonstrate large negative bias regardless of the null and positive causal effect.

In sample I, the nonrandom selection depending on Y has less impact on the relationship between G and X. On the contrary, the relationship between G and Y is largely biased the nonrandom selection depending on X in sample II, regardless of null or positive causal effect. This is because X is a collider in the pathway G→X←U→Y and S is the descendant node of collider X. Conditioning on S also unlocks the pathway G→X←U→Y and violates the assumption of IV Independence and Exclusion restriction. To some extent, all the eight MR methods can minimize the impact of violating the assumption of Exclusion restriction. However, the IV Independence assumption is difficult to test and relax because of the unmeasured confounder U.

Selections depending on the combination of Y and other variables (G or X) in sample II lead to larger biases of estimation than other selection mechanisms. Figs 5 and 8 show that the selections depending on X+Y, G+Y and G+X+Y in sample II lead to large biases of estimation. Selection depending on G and Y simultaneously, induces a spurious association between G and Y due to conditioning on collider S, that is, a horizontal pleiotropy. Selection depending on X and Y simultaneously, not only induces a spurious association between X and Y, but also unlocks the pathway G→X←U→Y [8]. Selection depending on G, X and Y simultaneously combines the above two cases. Hartwig et al. [30] also have found that the causal structure in the second sample had a larger influence on causal effect estimation. Their work aimed to assess whether different covariable-adjusted summary associations in two-sample MR could distort causal effect estimation. They found that using covariable-adjusted summary associations may bias the MR analyses. Particularly, the presence of an unmeasured confounder between the covariate and outcome in the second sample would render the covariate a collider. This type of collider bias is called the analytical colliding bias [31]. Their work is similar to our work’s conclusion in that S is also a collider and restricting the analysis to the selected population will lead to another type of collider bias sampling colliding bias [31]. Both types of collider bias will distort the true relationship between the common causes of collider.

For all eight pleiotropy-robust methods, performance is poor under nonrandom selection mechanisms, even when the extra pleiotropy exists. When the nonrandom selection depends on multiple independent genetic variants (G), spurious associations among these genetic variants would be induced. This may disturb the selection of valid IVs or make the valid IVs invalid. The proportion of invalid IVs caused by nonrandom selection is difficult to measure, which influences the performance of pleiotropy-robust methods to different extents. For example, IVW requires that all the IVs are valid, weighted median allows 50% IVs are invalid, weighted MBE allows 50%-100% IVs are invalid and MR-Egger allows 100% IVs are invalid. In addition, unlocking the pathway G→X←U→Y violates the InSIDE assumption, which is necessary for MR-Egger [14].

We provide a short review of the relevant selection mechanisms in GWASs and examples corresponding to selection mechanisms in published two-sample MR studies, as well as existing methods to correct for selection in these situations (Table 1). Because the two samples used in MR analysis are both from GWAS analysis, we only list possible selection mechanisms in one GWAS sample, including three cases: selection depending on genetic variants, phenotype (exposure X or outcome Y in MR analysis) and both. In our application, we restrict analysis to CHD patients to reveal the significant influence of nonrandom selection on the causal effect estimation of obesity on HbA1c levels. Several MR studies have found that obesity and HbA1c levels play important roles in CHD prevention [24-26]. HbA1c is the outcome of interest and S is a binary variable indicating whether a participant is a CHD patient in sample II (Fig A in S4 Text). In other words, we restrict the analysis to CHD patients, that is, conditioning S = 1, which depends on the outcome and exposure. In this situation, this selection mechanism can magnify the causal effect estimation of obesity on HbA1c levels.

Table 1

Possible selections in published analysis and existing methods to correct for selection bias.

DAGs	Relevant selection mechanisms in GWASs	Examples corresponding to selection mechanisms in published two-sample MR studies	Selection-corrected methods
G_j→X/Y↓S	Survival bias, that is, survival until study inclusion [32]; When considering a secondary disease outcome, such as disease progression [33]; Case-control study [8]; Differential loss to follow up in cohort studies [34]; Participant dropout [7]; Case-only study[7].	Identifying risk factors for brain tumors using a case-control design [37]; A case-only study is used to explore the causal effect of BMI on breast cancer survival for ER-positive breast cancer [38]; Lp(a) was not associated with future cardiovascular mortality in the population of individuals who already established CHD [39]; Genetic associations of alcohol-related variants with esophageal cancer have been considered separately in non-drinkers, moderate drinkers and heavy drinkers [40].	Consistent estimators of the causal relative risk and odds ratio if a priori knowledge is available regarding either the population disease prevalence or the population distribution of the IV [43]; Causal effect estimation under a quasi-empirical likelihood framework [44]; Inverse probability weighting method [7,33,45,46]; Inverse probability-of-censoring weighted estimation [34]; Bayesian approach [47,48]; Heckman’s Two-Step Method [48,49].
G_j→X/Y↓S	A misalignment in time zero [35]; Non-random selection based on another phenotype that the genetic variants also affect.	A mendelian randomization study revealed that low blood HDL-C is a potential causal risk factor for impaired cognition during aging in non-Hispanic whites of European ancestry [41]; A Mendelian randomization study of the effects of late-life cholesterol levels on dementia risk could be biased if earlier cholesterol levels affect living long enough to be at risk of dementia [35].	Inverse probability weighting method [7,33,45,46]; Bayesian approach [47,48]; Heckman’s Two-Step Method [48,49].
G_j→X/Y↘↙S	Restrict population by specific features [36].	If a genotype and maternal folate status affect embryo failure, the selection bias will occur in the population with embryo failure. [36,42]	Inverse probability weighting method [7,33,45,46]

In conclusion, nonrandom selection mechanisms in two-sample MR exacerbate the estimation bias for pleiotropy-robust MR methods. The biases tend to be exaggerated in the presence of pleiotropy.

Modeling assumptions.

(PDF)

Click here for additional data file.

Proof of Theorems.

(PDF)

Click here for additional data file.

Eight pleiotropy-robust methods in simulation.

(PDF)

Click here for additional data file.

DAG for application.

(PDF)

Click here for additional data file.

Simulation results of eight Pleiotropy-robust MR Methods with 100 valid genetic variants in scenario 1.

(PDF)

Click here for additional data file.

Simulation results of eight Pleiotropy-robust MR Methods in scenario 2 (Balanced pleiotropy, InSIDE satisfied).

(PDF)

Click here for additional data file.

Simulation results of eight Pleiotropy-robust MR Methods in scenario 3 (Directional pleiotropy, InSIDE satisfied).

(PDF)

Click here for additional data file.

Simulation results of eight Pleiotropy-robust MR Methods in scenario 4 (Directional pleiotropy, InSIDE violated).

(PDF)

Click here for additional data file.

Code to implement the method and reproduce all simulations and analyses.

(PDF)

Click here for additional data file.

Quantification of data shown in Figs 5–10 and A-F in S5 Text.

(XLSX)

Click here for additional data file.

Quantification of data shown in Figs A-X in S6 Text.

(XLSX)

Click here for additional data file.

Quantification of data shown in Figs A-X in S7 Text.

(XLSX)

Click here for additional data file.

Quantification of data shown in Figs A-X in S8 Text.

(XLSX)

Click here for additional data file.

GWAS summary data for application.

(XLSX)

Click here for additional data file.

28 Oct 2021

Dear Dr Xue,

Thank you very much for submitting your Research Article entitled 'Impact of Nonrandom Selection Mechanisms on the Causal Effect Estimation for Two-sample Mendelian Randomization Methods' to PLOS Genetics.

We apologise for the lengthy wait for a decision, we had considerable difficulties recruiting reviewers.  There is one reviewer report attached and an editor comment that identify some concerns that we ask you address in a revised manuscript.

In addition we ask that you:

1) Provide a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript.

2) Upload a Striking Image with a corresponding caption to accompany your manuscript if one is available (either a new image or an existing one from within your manuscript). If this image is judged to be suitable, it may be featured on our website. Images should ideally be high resolution, eye-catching, single panel square images. For examples, please browse our archive. If your image is from someone other than yourself, please ensure that the artist has read and agreed to the terms and conditions of the Creative Commons Attribution License. Note: we cannot publish copyrighted images.

We hope to receive your revised manuscript within the next 30 days. If you anticipate any delay in its return, we would ask you to let us know the expected resubmission date by email to plosgenetics@plos.org.

If present, accompanying reviewer attachments should be included with this email; please notify the journal office if any appear to be missing. They will also be available for download from the link below. You can use this link to log into the system when you are ready to submit a revised version, having first consulted our Submission Checklist.

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Please be aware that our data availability policy requires that all numerical data underlying graphs or summary statistics are included with the submission, and you will need to provide this upon resubmission if not already present. In addition, we do not permit the inclusion of phrases such as "data not shown" or "unpublished results" in manuscripts. All points should be backed up by data provided with the submission.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

PLOS has incorporated Similarity Check, powered by iThenticate, into its journal-wide submission system in order to screen submitted content for originality before publication. Each PLOS journal undertakes screening on a proportion of submitted articles. You will be contacted if needed following the screening process.

To resubmit, you will need to go to the link below and 'Revise Submission' in the 'Submissions Needing Revision' folder.

[LINK]

Please let us know if you have any questions while making these revisions.

Yours sincerely,

Caroline Relton

Associate Editor

PLOS Genetics

David Balding

Section Editor: Methods

PLOS Genetics

Editor comment:

Although the standard of scientific English is generally very good, particularly in Section 3.2 there were sentences that were confusing to me, such as "... there was an evidence that it more likely to be a morning person (chronotype) were associated with a lower odds of smoking cessation. Therefore, the population enriched for smokers is a selected population depends on chronotype."  As a result, I find it hard to understand the significance of this application: are you claiming that there have been published errors?  Or just the potential for error due to selection?  This application was also not mentioned in the Discussion, making its impact unclear.  Can you point to any possible or likely errors in published analyses due to the selection effects that you describe?

Reviewer #1:

Please see attachment.

**********

Have all data underlying the figures and results presented in the manuscript been provided?

Large-scale datasets should be made available via a public repository as described in the PLOS Genetics data availability policy, and numerical data that underlies graphs or summary statistics should be provided in spreadsheet form as supporting information.

Reviewer #1: No: Please provide a spreadsheet as supplemental material giving the data points for all the figures.

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Submitted filename: Yu MR selection.docx

Click here for additional data file.

24 Nov 2021

Submitted filename: Point by point responses to editor and reviewers.docx

Click here for additional data file.

20 Dec 2021

Dear Dr Xue,

Thank you very much for submitting your Research Article entitled 'Impact of Nonrandom Selection Mechanisms on the Causal Effect Estimation for Two-sample Mendelian Randomization Methods' to PLOS Genetics.

The manuscript was evaluated at the editorial level and by the independent peer reviewer. Although we appreciate the improvements in the revised manuscript, we felt that insufficient care and attention to detail was made in preparing the revision.  The reviewer has identified a number of issues that remain to be addressed.  We ask you to address these in a further revised manuscript.  The required changes should not take very much effort, but we ask you to take care to check all aspects of the manuscript for clarity and accuracy, to create what we hope will be essentially a final version.

Please also:

1) Provide a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript.

2) Upload a Striking Image with a corresponding caption to accompany your manuscript if one is available (either a new image or an existing one from within your manuscript). If this image is judged to be suitable, it may be featured on our website. Images should ideally be high resolution, eye-catching, single panel square images. For examples, please browse our archive. If your image is from someone other than yourself, please ensure that the artist has read and agreed to the terms and conditions of the Creative Commons Attribution License. Note: we cannot publish copyrighted images.

We hope to receive your revised manuscript within the next 30 days. If you anticipate any delay in its return, we would ask you to let us know the expected resubmission date by email to plosgenetics@plos.org.

If present, accompanying reviewer attachments should be included with this email; please notify the journal office if any appear to be missing. They will also be available for download from the link below. You can use this link to log into the system when you are ready to submit a revised version, having first consulted our Submission Checklist.

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Please be aware that our data availability policy requires that all numerical data underlying graphs or summary statistics are included with the submission, and you will need to provide this upon resubmission if not already present. In addition, we do not permit the inclusion of phrases such as "data not shown" or "unpublished results" in manuscripts. All points should be backed up by data provided with the submission.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

PLOS has incorporated Similarity Check, powered by iThenticate, into its journal-wide submission system in order to screen submitted content for originality before publication. Each PLOS journal undertakes screening on a proportion of submitted articles. You will be contacted if needed following the screening process.

To resubmit, you will need to go to the link below and 'Revise Submission' in the 'Submissions Needing Revision' folder.

[LINK]

Please let us know if you have any questions while making these revisions.

Yours sincerely,

Caroline Relton, PhD

Associate Editor

PLOS Genetics

David Balding

Section Editor: Methods

PLOS Genetics

Reviewer's Responses to Questions

Reviewer #1: The authors have responded to my comments and improved the paper. I have some further comments arising.

1. Table 1. You mention collider-corrected regression as a method to correct for selection bias, but it is not.

2. Table 1. it is not clear how population stratification and dynastic effects lead to selection bias. They are well known as confounders of genetic associations, but selection effects are not so clear; please explain.

3. Table 1 row 1 col 3. ref 48 does not use inverse probability weighting.

4. The author summary could be improved, it is not really at a lay level, X Y and G are undefined, etc.

5. Page 5 line 92. "(pleiotropy) always leads to the violation..." is incorrect, it can do, but not always.

6. Equation 1 and Figure 1. Why are gamma and phi both shown by grey dotted lines in the figure, but only gamma appears in the equation?

7. Page 8 line 157. "Existence of causal relationship" looks wrong, I think you mean something like invariance of causal relationship.

8. Page 8 line 174. IVW is undefined at this point. Not clear what bias you are referring to.

9. Page 11 line 232. "Ratio" should be "proportion".

10. Page 12 lines 250-254. The discussion of circadian rhythm relating to chemotherapy is distracting from the main question of chronotype as a cause of breast cancer. Not clear that this is necessary.

11. Page 15 line 326. "To avoid the influence of winner's curse" is poor wording - it is still possible among these selected SNPs. The point in the earlier review was that winner's curse effects would be different between two analyses using different numbers of SNPs. This clause could be deleted without detriment.

12. The smoking example is unconvincing. You claim that smoking is a descendant of the collider at chronotype. But there is no evidence that chronotype causes smoking. It may be smoking that causes chronotype, or their association could be due to confounding. Furthermore there may simply be an effect modification such that the chronotype-breast cancer association is different for smokers. The attenuated associations in the smokers are consistent with selection bias, but are not evidence for it. Finally the association reported by Gibson et al was with smoking cessation, not smoking itself.

**********

Have all data underlying the figures and results presented in the manuscript been provided?

Large-scale datasets should be made available via a public repository as described in the PLOS Genetics data availability policy, and numerical data that underlies graphs or summary statistics should be provided in spreadsheet form as supporting information.

Reviewer #1: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

12 Jan 2022

Submitted filename: Point_by_point_responses_to_the_reviewer-20220112.docx

Click here for additional data file.

16 Feb 2022

Dear Dr Xue,

We are pleased to inform you that your manuscript entitled "Impact of Nonrandom Selection Mechanisms on the Causal Effect Estimation for Two-sample Mendelian Randomization Methods" has been editorially accepted for publication in PLOS Genetics. Congratulations!

Before your submission can be formally accepted and sent to production you will need to complete our formatting changes, which you will receive in a follow up email. Please be aware that it may take several days for you to receive this email; during this time no action is required by you. Please note: the accept date on your published article will reflect the date of this provisional acceptance, but your manuscript will not be scheduled for publication until the required changes have been made.

Once your paper is formally accepted, an uncorrected proof of your manuscript will be published online ahead of the final version, unless you’ve already opted out via the online submission form. If, for any reason, you do not want an earlier version of your manuscript published online or are unsure if you have already indicated as such, please let the journal staff know immediately at plosgenetics@plos.org.

In the meantime, please log into Editorial Manager at https://www.editorialmanager.com/pgenetics/, click the "Update My Information" link at the top of the page, and update your user information to ensure an efficient production and billing process. Note that PLOS requires an ORCID iD for all corresponding authors. Therefore, please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field.  This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager.

If you have a press-related query, or would like to know about making your underlying data available (as you will be aware, this is required for publication), please see the end of this email. If your institution or institutions have a press office, please notify them about your upcoming article at this point, to enable them to help maximise its impact. Inform journal staff as soon as possible if you are preparing a press release for your article and need a publication date.

Thank you again for supporting open-access publishing; we are looking forward to publishing your work in PLOS Genetics!

Yours sincerely,

Caroline Relton, PhD

Associate Editor

PLOS Genetics

David Balding

Section Editor: Methods

PLOS Genetics

www.plosgenetics.org

Twitter: @PLOSGenetics

----------------------------------------------------

Data Deposition

If you have submitted a Research Article or Front Matter that has associated data that are not suitable for deposition in a subject-specific public repository (such as GenBank or ArrayExpress), one way to make that data available is to deposit it in the Dryad Digital Repository. As you may recall, we ask all authors to agree to make data available; this is one way to achieve that. A full list of recommended repositories can be found on our website.

The following link will take you to the Dryad record for your article, so you won't have to re‐enter its bibliographic information, and can upload your files directly:

http://datadryad.org/submit?journalID=pgenetics&manu=PGENETICS-D-21-00846R2

More information about depositing data in Dryad is available at http://www.datadryad.org/depositing. If you experience any difficulties in submitting your data, please contact help@datadryad.org for support.

Additionally, please be aware that our data availability policy requires that all numerical data underlying display items are included with the submission, and you will need to provide this before we can formally accept your manuscript, if not already present.

----------------------------------------------------

Press Queries

If you or your institution will be preparing press materials for this manuscript, or if you need to know your paper's publication date for media purposes, please inform the journal staff as soon as possible so that your submission can be scheduled accordingly. Your manuscript will remain under a strict press embargo until the publication date and time. This means an early version of your manuscript will not be published ahead of your final version. PLOS Genetics may also choose to issue a press release for your article. If there's anything the journal should know or you'd like more information, please get in touch via plosgenetics@plos.org.

4 Mar 2022

PGENETICS-D-21-00846R2

Impact of Nonrandom Selection Mechanisms on the Causal Effect Estimation for Two-sample Mendelian Randomization Methods

Dear Dr Xue,

We are pleased to inform you that your manuscript entitled "Impact of Nonrandom Selection Mechanisms on the Causal Effect Estimation for Two-sample Mendelian Randomization Methods" has been formally accepted for publication in PLOS Genetics! Your manuscript is now with our production department and you will be notified of the publication date in due course.

The corresponding author will soon be receiving a typeset proof for review, to ensure errors have not been introduced during production. Please review the PDF proof of your manuscript carefully, as this is the last chance to correct any errors. Please note that major changes, or those which affect the scientific understanding of the work, will likely cause delays to the publication date of your manuscript.

Soon after your final files are uploaded, unless you have opted out or your manuscript is a front-matter piece, the early version of your manuscript will be published online. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.

Thank you again for supporting PLOS Genetics and open-access publishing. We are looking forward to publishing your work!

With kind regards,

Livia Horvath

PLOS Genetics

On behalf of:

The PLOS Genetics Team

Carlyle House, Carlyle Road, Cambridge CB4 3DN | United Kingdom

plosgenetics@plos.org | +44 (0) 1223-442823

plosgenetics.org | Twitter: @PLOSGenetics