Detection of genetic heterogeneity among pedigrees through complex segregation analysis: an application to hypercholesterolemia.

Several methods for investigating genetic heterogeneity for extreme levels of a quantitative trait with hypothesized multiple genetic etiologies require a priori stratification of families and/or identification of distinct phenotypes among affected individuals. We present a statistical approach for detecting genetic heterogeneity that does not rely on either a priori stratification or discrete disease phenotypes. Complex segregation analysis was applied to total serum cholesterol measurements in 709 relatives of 98 healthy index cases selected from 3,666 school children surveyed for lipid levels in Rochester, Minnesota. Thirty-three of the index cases and 109 relatives had hypercholesterolemia (cholesterol levels greater than the 95th percentile for their age and sex). Through application of the mixed genetic model and then estimation of conditional probabilities for having the mutant allele at the major locus, genetic heterogeneity for hypercholesterolemia was indicated. In three of 70 pedigrees with one or more hypercholesterolemics, there is strong evidence for segregation at a major locus. In the remaining pedigrees, only polygene variation and/or environmental variation are associated with cholesterol variability. Grandparents in the three pedigrees that were segregating at the major locus had the highest rates of death due to coronary heart disease. This study establishes that the mixed model has the potential to identify pedigrees with different genetic etiologies for variability in quantitative traits.