Kathleen F Kerr1. 1. Department of Biostatistics, Box 357232, University of Washington, Seattle, WA 98195, USA. katiek@u.washington.edu
Abstract
MOTIVATION: Permutation testing is very popular for analyzing microarray data to identify differentially expressed (DE) genes; estimating false discovery rates (FDRs) is a very popular way to address the inherent multiple testing problem. However, combining these approaches may be problematic when sample sizes are unequal. RESULTS: With unbalanced data, permutation tests may not be suitable because they do not test the hypothesis of interest. In addition, permutation tests can be biased. Using biased P-values to estimate the FDR can produce unacceptable bias in those estimates. Results also show that the approach of pooling permutation null distributions across genes can produce invalid P-values, since even non-DE genes can have different permutation null distributions. We encourage researchers to use statistics that have been shown to reliably discriminate DE genes, but caution that associated P-values may be either invalid, or a less-effective metric for discriminating DE genes.
MOTIVATION: Permutation testing is very popular for analyzing microarray data to identify differentially expressed (DE) genes; estimating false discovery rates (FDRs) is a very popular way to address the inherent multiple testing problem. However, combining these approaches may be problematic when sample sizes are unequal. RESULTS: With unbalanced data, permutation tests may not be suitable because they do not test the hypothesis of interest. In addition, permutation tests can be biased. Using biased P-values to estimate the FDR can produce unacceptable bias in those estimates. Results also show that the approach of pooling permutation null distributions across genes can produce invalid P-values, since even non-DE genes can have different permutation null distributions. We encourage researchers to use statistics that have been shown to reliably discriminate DE genes, but caution that associated P-values may be either invalid, or a less-effective metric for discriminating DE genes.
Authors: Grier P Page; Jode W Edwards; Gary L Gadbury; Prashanth Yelisetti; Jelai Wang; Prinal Trivedi; David B Allison Journal: BMC Bioinformatics Date: 2006-02-22 Impact factor: 3.169
Authors: Elena Flowers; Annesa Flentje; Jon Levine; Adam Olshen; Marilyn Hammer; Steven Paul; Yvette Conley; Christine Miaskowski; Kord M Kober Journal: Biol Res Nurs Date: 2019-01-31 Impact factor: 2.522
Authors: Ashok Sharma; Shoshana M Bartell; Clifton A Baile; Bo Chen; Robert H Podolsky; Richard A McIndoe; Jin-Xiong She Journal: PLoS One Date: 2010-08-16 Impact factor: 3.240