Claims of sex differences: an empirical assessment in genetic associations.

CONTEXT: Many studies try to probe for differences in risks between men and women, and this is a major challenge in the expanding literature of associations between genetic variants and common diseases or traits.
OBJECTIVE: To evaluate whether prominently claimed sex differences for genetic effects have sufficient internal and external validity. DATA SOURCES: We searched PubMed through July 6, 2007, for genetic association studies claiming sex-related differences in the articles' titles. Titles and abstracts and, if necessary, the full text of the article were assessed for eligibility. STUDY SELECTION: Two hundred fifteen articles were retrieved by the search. We considered eligible all retrieved association studies that claimed different genetic effects across sexes of 1 or more gene variants for any human disease or phenotype. We considered both biallelic and multiallelic markers (including haplotypes) and both binary and continuous phenotypes and traits. We excluded non-English-language studies; studies evaluating only 1 sex; studies in which sex was treated only as an independent predictor of disease; studies that did not address any association of the investigated genetic variant with a disease or trait; studies not involving humans; and studies in which the authors did not claim any sex difference. DATA EXTRACTION: Two evaluators independently extracted data with a third evaluator arbitrating their discrepancies. Data evaluation included whether analyses were stated to have been specified a priori; whether sex effects were evaluated in the whole study or subgroups thereof; and whether the claims were appropriately documented, insufficiently documented, or spurious. For appropriately and insufficiently documented claims we performed the calculations for gene-sex interaction whenever raw data were available. Finally, we compared the sex-difference claims with the best internal validity against the results of other studies addressing the same interaction.
RESULTS: We appraised 432 sex-difference claims in 77 eligible articles. Authors stated that sex comparisons were decided a priori for 286 claims (66.2%), while the entire sample size was used in 210 (48.6%) claims. Appropriate documentation of gene-sex interaction was recorded in 55 claims (12.7%); documentation was insufficient for 303 claims and spurious for the other 74. Data for reanalysis of claims were available for 188 comparisons. Of these, 83 (44.1%) were nominally statistically significant at a P = .05 threshold, and more than half of them (n = 44) had modest P values between .01 and .05. Of 60 claims with seemingly the best internal validity, only 1 was consistently replicated in at least 2 other studies.
CONCLUSION: In this sample of highly prominent claims of sex-related differences in genetic associations, most claims were insufficiently documented or spurious, and claims with documented good internal and external validity were uncommon.