K Afshar1, S Jafari, A Seth, J K Lee, A E MacNeily. 1. Department of Urologic Sciences and School of Population and Public Health (KA, SJ), University of British Columbia, Vancouver, British Columbia, Canada.
Abstract
PURPOSE: We provide a systematic assessment of the quality of research methodology, statistical analysis and reporting in 2 recent pediatric supplements of The Journal of Urology. MATERIALS AND METHODS: All original clinical publications in 2 Pediatric Supplements of The Journal of Urology (2005 and 2007) were identified for formal review. We collected data on variables indicating the quality of methodology and statistical analysis. Two independent reviewers with formal training in clinical epidemiology reviewed each article. RESULTS: Of the 103 published articles 92 met study inclusion criteria. Common study designs included a retrospective cohort in 68% of articles and a prospective cohort in 17%. Demographic statistics were incomplete in many articles, that is in 24% no measure of central tendency and in 18% no measures of dispersion were provided. In 40% of articles the statistical methods were not clearly described. Of all studies eligible for univariate or multivariate analysis only 52% and 23%, respectively, provided the appropriate analysis. Deficiencies in reporting and interpreting p values, ORs/RRs and CIs were noted in 53%, 83% and 78% of studies, respectively. Limitations of studies, such as biases and confounders, were only acknowledged in a third of articles. Comparison of articles published in 2005 vs 2007 revealed a significant improvement in reporting the results of multivariate analysis (p = 0.0297). CONCLUSIONS: Methodological and statistical shortcomings were common in our sampling of the pediatric urology literature. This may result in misleading conclusions. Pediatric urologists are encouraged to involve colleagues with formal training in research design from the outset of their studies.
PURPOSE: We provide a systematic assessment of the quality of research methodology, statistical analysis and reporting in 2 recent pediatric supplements of The Journal of Urology. MATERIALS AND METHODS: All original clinical publications in 2 Pediatric Supplements of The Journal of Urology (2005 and 2007) were identified for formal review. We collected data on variables indicating the quality of methodology and statistical analysis. Two independent reviewers with formal training in clinical epidemiology reviewed each article. RESULTS: Of the 103 published articles 92 met study inclusion criteria. Common study designs included a retrospective cohort in 68% of articles and a prospective cohort in 17%. Demographic statistics were incomplete in many articles, that is in 24% no measure of central tendency and in 18% no measures of dispersion were provided. In 40% of articles the statistical methods were not clearly described. Of all studies eligible for univariate or multivariate analysis only 52% and 23%, respectively, provided the appropriate analysis. Deficiencies in reporting and interpreting p values, ORs/RRs and CIs were noted in 53%, 83% and 78% of studies, respectively. Limitations of studies, such as biases and confounders, were only acknowledged in a third of articles. Comparison of articles published in 2005 vs 2007 revealed a significant improvement in reporting the results of multivariate analysis (p = 0.0297). CONCLUSIONS: Methodological and statistical shortcomings were common in our sampling of the pediatric urology literature. This may result in misleading conclusions. Pediatric urologists are encouraged to involve colleagues with formal training in research design from the outset of their studies.