Better estimates of genetic covariance matrices by "bending" using penalized maximum likelihood.

Obtaining accurate estimates of the genetic covariance matrix Sigma(G) for multivariate data is a fundamental task in quantitative genetics and important for both evolutionary biologists and plant or animal breeders. Classical methods for estimating Sigma(G) are well known to suffer from substantial sampling errors; importantly, its leading eigenvalues are systematically overestimated. This article proposes a framework that exploits information in the phenotypic covariance matrix Sigma(P) in a new way to obtain more accurate estimates of Sigma(G). The approach focuses on the "canonical heritabilities" (the eigenvalues of Sigma(P)(-1)Sigma(G)), which may be estimated with more precision than those of Sigma(G) because Sigma(P) is estimated more accurately. Our method uses penalized maximum likelihood and shrinkage to reduce bias in estimates of the canonical heritabilities. This in turn can be exploited to get substantial reductions in bias for estimates of the eigenvalues of Sigma(G) and a reduction in sampling errors for estimates of Sigma(G). Simulations show that improvements are greatest when sample sizes are small and the canonical heritabilities are closely spaced. An application to data from beef cattle demonstrates the efficacy this approach and the effect on estimates of heritabilities and correlations. Penalized estimation is recommended for multivariate analyses involving more than a few traits or problems with limited data.