Selection for environmental variation: a statistical analysis and power calculations to detect response.

Data from uterine capacity in rabbits (litter size) were analyzed to determine whether the environmental variance was partly genetically determined. The fit of a classical homogeneous variance mixed linear (HOM) model and that of a genetically structured heterogeneous variance mixed linear (HET) model were compared. Various methods to assess the quality of fit favor the HET model. The posterior mean (95% posterior interval) of the additive genetic variance affecting the environmental variance was 0.16 (0.10; 0.25) and the corresponding number for the coefficient of correlation between genes affecting mean and variance was -0.74 (-0.90;-0.52). It is argued that stronger support for the HET model than that derived from statistical analysis of data would be provided by a successful selection experiment designed to modify the environmental variance. A simple selection criterion is suggested (average squared deviation from the mean of repeated records within individuals) and its predicted response and variance under the HET model are derived. This is used to determine the appropriate size and length of a selection experiment designed to change the environmental variance. Results from the analytical expressions are compared with those obtained using simulation. There is good agreement provided selection intensity is not intense.