Mixed model segregation analysis of LDL-C concentration with genotype-covariate interaction.

Mixed model complex segregation analyses have in the past ignored the possibility of genotype-covariate interaction. Only in the nonmixed model with polygenic heritability equal to zero have segregation analyses been performed that allowed for genotype specific regression of the phenotype on covariates. We present an extension of Hasstedt's [1982] mixed model likelihood approximation which does allow for genotype-covariate interaction in the mixed model. Following description of this approximation, we validate the likelihood calculation by a Monte Carlo procedure based on the actual pedigree and missing data structure used in a complex segregation analysis of low density plus very low density lipoprotein cholesterol (LDL-C + VLDL-C) in baboons. The observed averages of the bootstrap parameter estimates adequately recover the generating values, which included parameters specifying genotype-covariate interaction. We then applied both a traditional complex segregation analysis and an analysis with genotype-covariate interaction to test for the presence of a major locus affecting LDL-C levels in baboons. The model including genotype-covariate interaction was significantly different from the model without interactions, and strongly supported the hypothesis that there is a segregating Mendelian locus as opposed to a random environmental factor. This major locus accounts for approximately 46% of the variance in LDL-C levels, as compared to 40% explained by a locus with no genotype-covariate interaction.