A review of multiple hypothesis testing in otolaryngology literature.

OBJECTIVES/HYPOTHESIS: Multiple hypothesis testing (or multiple testing) refers to testing more than one hypothesis within a single analysis, and can inflate the type I error rate (false positives) within a study. The aim of this review was to quantify multiple testing in recent large clinical studies in the otolaryngology literature and to discuss strategies to address this potential problem. DATA SOURCES: Original clinical research articles with >100 subjects published in 2012 in the four general otolaryngology journals with the highest Journal Citation Reports 5-year impact factors. REVIEW
METHODS: Articles were reviewed to determine whether the authors tested greater than five hypotheses in at least one family of inferences. For the articles meeting this criterion for multiple testing, type I error rates were calculated, and statistical correction was applied to the reported results.
RESULTS: Of the 195 original clinical research articles reviewed, 72% met the criterion for multiple testing. Within these studies, there was a mean 41% chance of a type I error and, on average, 18% of significant results were likely to be false positives. After the Bonferroni correction was applied, only 57% of significant results reported within the articles remained significant.
CONCLUSIONS: Multiple testing is common in recent large clinical studies in otolaryngology and deserves closer attention from researchers, reviewers, and editors. Strategies for adjusting for multiple testing are discussed.