Robustness of the unified model to shared environmental effects in the analysis of dichotomous traits.

Simulation studies were conducted to assess to what extent the conclusions of segregation analysis, performed under the unified model, can be affected by the presence of unmeasured environmental factors shared by family members. Dichotomous data were generated on six-member nuclear families under two variants of the mixed model, incorporating environmental effects shared by all family members. When the generating model includes a polygenic component and a shared environmental effect, there is false detection of a major gene, especially when the joint likelihood of parents' and offspring's phenotypes is computed. The proportion of false conclusions increases as the shared environmental effect increases. On the other hand, the presence of a shared environmental effect in addition to a major gene component does not alter the detection of the major gene nor the transmission probability estimates, which are close to the expected Mendelian values. The rejection of the Mendelian transmission hypothesis, as observed in familial analyses of affected disorders, might be the result of mechanisms other than those considered here, such as more complex sources of environmental resemblance or a possible genetic heterogeneity.