Testing differentially expressed genes in dose-response studies and with ordinal phenotypes.

When testing for differentially expressed genes between more than two groups, the groups are often defined by dose levels in dose-response experiments or ordinal phenotypes, such as disease stages. We discuss the potential of a new approach that uses the levels' ordering without making any structural assumptions, such as monotonicity, by testing for zero variance components in a mixed models framework. Since the mixed effects model approach borrows strength across doses/levels, the test proposed can also be applied when the number of dose levels/phenotypes is large and/or the number of subjects per group is small. We illustrate the new test in simulation studies and on several publicly available datasets and compare it to alternative testing procedures. All tests considered are implemented in R and are publicly available. The new approach offers a very fast and powerful way to test for differentially expressed genes between ordered groups without making restrictive assumptions with respect to the true relationship between factor levels and response.