Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis.

Important design aspects were decreasingly reported in NSAID trials over the years, whereas the quality of statistical analysis improved. In half of the trials, the effect variables in the methods and results sections were not the same, and the interpretation of the erythrocyte sedimentation rate in the reports seemed to depend on whether a significant difference was found. Statistically significant results appeared in 93 reports (47%). In 73 trials they favored only the new drug, and in 8 only the active control. All 39 trials with a significant difference in side effects favored the new drug. Choice of dose, multiple comparisons, wrong calculation, subgroup and within-groups analyses, wrong sampling units (in 63% of trials for effect variables, in 23% for side effects), change in measurement scale before analysis, baseline difference, and selective reporting of significant results were some of the verified or possible causes for the large proportion of results that favored the new drug. Doubtful or invalid statements were found in 76% of the conclusions or abstracts. Bias consistently favored the new drug in 81 trials, and the control in only one trial. It is not obvious how a reliable meta-analysis could be done in these trials.