Methodological and Reporting Errors in Meta-Analytic Reviews Make Other Meta-Analysts Angry: A Commentary on Ferguson (2015).

Although Ferguson's (2015, this issue) meta-analysis addresses an important topic, we have serious concerns about how it was conducted. Because there was only one coder, we have no confidence in the reliability or validity of the coded variables. Two independent raters should have coded the studies. Ferguson synthesized partial correlations as if they were zero-order correlations, which can increase or decrease (sometimes substantially) the variance of the partial correlation. Moreover, he partialled different numbers of variables from different effects, partialled different variables from different studies, and did not report what was partialled from each study. Ferguson used an idiosyncratic "tandem procedure" for detecting publication bias. He also "corrected" his results for publication bias, even though there is no such thing as a "correction" for publication bias. Thus, we believe that Ferguson's meta-analysis is fatally flawed and should not have been accepted for publication in Perspective on Psychological Science (or any other journal).