Hypothesis testing under mixture models: application to genetic linkage analysis.

In this paper we propose a new class of statistics to test a simple hypothesis against a family of alternatives characterized by a mixture model. Unlike the likelihood ratio statistic, whose large sample distribution is still unknown in this situation, these new statistics have a simple asymptotic distribution to which to refer under the null hypothesis. Simulation results suggest that it has adequate power in detecting the alternatives. Its application to genetic linkage analysis in the presence of the genetic heterogeneity that motivated this work is emphasized.