OBJECTIVES: To illustrate how multiple hypotheses testing can produce associations with no clinical plausibility. STUDY DESIGN AND SETTING: We conducted a study of all 10,674,945 residents of Ontario aged between 18 and 100 years in 2000. Residents were randomly assigned to equally sized derivation and validation cohorts and classified according to their astrological sign. Using the derivation cohort, we searched through 223 of the most common diagnoses for hospitalization until we identified two for which subjects born under one astrological sign had a significantly higher probability of hospitalization compared to subjects born under the remaining signs combined (P<0.05). RESULTS: We tested these 24 associations in the independent validation cohort. Residents born under Leo had a higher probability of gastrointestinal hemorrhage (P=0.0447), while Sagittarians had a higher probability of humerus fracture (P=0.0123) compared to all other signs combined. After adjusting the significance level to account for multiple comparisons, none of the identified associations remained significant in either the derivation or validation cohort. CONCLUSIONS: Our analyses illustrate how the testing of multiple, non-prespecified hypotheses increases the likelihood of detecting implausible associations. Our findings have important implications for the analysis and interpretation of clinical studies.
RCT Entities:
OBJECTIVES: To illustrate how multiple hypotheses testing can produce associations with no clinical plausibility. STUDY DESIGN AND SETTING: We conducted a study of all 10,674,945 residents of Ontario aged between 18 and 100 years in 2000. Residents were randomly assigned to equally sized derivation and validation cohorts and classified according to their astrological sign. Using the derivation cohort, we searched through 223 of the most common diagnoses for hospitalization until we identified two for which subjects born under one astrological sign had a significantly higher probability of hospitalization compared to subjects born under the remaining signs combined (P<0.05). RESULTS: We tested these 24 associations in the independent validation cohort. Residents born under Leo had a higher probability of gastrointestinal hemorrhage (P=0.0447), while Sagittarians had a higher probability of humerus fracture (P=0.0123) compared to all other signs combined. After adjusting the significance level to account for multiple comparisons, none of the identified associations remained significant in either the derivation or validation cohort. CONCLUSIONS: Our analyses illustrate how the testing of multiple, non-prespecified hypotheses increases the likelihood of detecting implausible associations. Our findings have important implications for the analysis and interpretation of clinical studies.
Authors: Marcella Nunez-Smith; Nanlesta Pilgrim; Matthew Wynia; Mayur M Desai; Beth A Jones; Cedric Bright; Harlan M Krumholz; Elizabeth H Bradley Journal: J Gen Intern Med Date: 2009-09-01 Impact factor: 5.128
Authors: Marc L Berger; Bradley C Martin; Don Husereau; Karen Worley; J Daniel Allen; Winnie Yang; Nicole C Quon; C Daniel Mullins; Kristijan H Kahler; William Crown Journal: Value Health Date: 2014-03 Impact factor: 5.725
Authors: Jonathan R Weinstein; Juliane Schulze; Richard V Lee; Hunter Phillips; Dannielle Zierath; Patricia Tanzi; Dean Shibata; Kevin C Cain; Kyra J Becker Journal: Neuroreport Date: 2014-05-28 Impact factor: 1.837
Authors: Aaron R Caldwell; Andrew D Vigotsky; Matthew S Tenan; Rémi Radel; David T Mellor; Andreas Kreutzer; Ian M Lahart; John P Mills; Matthieu P Boisgontier Journal: Sports Med Date: 2020-03 Impact factor: 11.136