J H Park1, M Eisenhut2, H J van der Vliet3, J I Shin4. 1. College of Medicine, Yonsei University, Severance Hospital, Seoul, Korea. 2. Luton & Dunstable University Hospital NHS Foundation Trust, Luton, UK. 3. Department of Medical Oncology, VU University Medical Center, Amsterdam, The Netherlands. 4. Department of Pediatrics, Institute of Kidney Disease Research, Yonsei University College of Medicine, Severance Children's Hospital, Seoul, Korea.
Abstract
BACKGROUND: Various errors in the design, conduct, and analysis of medical and public health research studies can produce false results and waste valuable resources. While systematic reviews and meta-analyses are arguably considered the most dependable source of evidence-based medicine, increasing numbers of studies are indicating that, on the contrary to the public's belief, many of these investigations are redundant, erroneous, and even biased. METHODS: Ninety-four meta-analyses on microRNA polymorphism and risk of cancer were extracted from Pubmed database on August 2016. Two investigators independently extracted data (i.e. number of studies, ethnicity, number of cases/controls, bias, etc.) from each meta-analysis. PROSPERO registration status and reference status were also recorded. RESULTS: Among the 217 microRNA gene-variant cancer associations reported by 94 published meta-analyses, 37% had overlapping results and were extracted from the exact identical case-control studies. However, not one meta-analysis was registered into PROSPERO. Thirty-one percent of the overlapping associations referenced a previous meta-analysis investigating the same association; although only 36% of these overlapping associations referenced earlier meta-analysis that had the same overlapping results. Seventy-four percent of these references were limited to mere citations. Twenty-six percent of the overlapping associations from 16 meta-analyses showed discordant results, and of these, 87% of the genotype comparisons were found significant, contrary to the initial reports of being non-significant. However, no association was noteworthy in regards to false positive rate probability calculations at a given prior probability of 0.001 and 0.000001 and statistical power to detect an odds ratio (OR) of 1.1 and 1.5. CONCLUSIONS: Genetic association meta-analyses were by far more redundant, erroneous, and lacking references than initially expected. Careful search of similar studies, attention to small details, and inclination to reference previous works are needed. This paper proposes potential solutions for these problems in hopes of standardizing research efforts and in improving the quality of medical research.
BACKGROUND: Various errors in the design, conduct, and analysis of medical and public health research studies can produce false results and waste valuable resources. While systematic reviews and meta-analyses are arguably considered the most dependable source of evidence-based medicine, increasing numbers of studies are indicating that, on the contrary to the public's belief, many of these investigations are redundant, erroneous, and even biased. METHODS: Ninety-four meta-analyses on microRNA polymorphism and risk of cancer were extracted from Pubmed database on August 2016. Two investigators independently extracted data (i.e. number of studies, ethnicity, number of cases/controls, bias, etc.) from each meta-analysis. PROSPERO registration status and reference status were also recorded. RESULTS: Among the 217 microRNA gene-variant cancer associations reported by 94 published meta-analyses, 37% had overlapping results and were extracted from the exact identical case-control studies. However, not one meta-analysis was registered into PROSPERO. Thirty-one percent of the overlapping associations referenced a previous meta-analysis investigating the same association; although only 36% of these overlapping associations referenced earlier meta-analysis that had the same overlapping results. Seventy-four percent of these references were limited to mere citations. Twenty-six percent of the overlapping associations from 16 meta-analyses showed discordant results, and of these, 87% of the genotype comparisons were found significant, contrary to the initial reports of being non-significant. However, no association was noteworthy in regards to false positive rate probability calculations at a given prior probability of 0.001 and 0.000001 and statistical power to detect an odds ratio (OR) of 1.1 and 1.5. CONCLUSIONS: Genetic association meta-analyses were by far more redundant, erroneous, and lacking references than initially expected. Careful search of similar studies, attention to small details, and inclination to reference previous works are needed. This paper proposes potential solutions for these problems in hopes of standardizing research efforts and in improving the quality of medical research.
Authors: Armen Yuri Gasparyan; Marlen Yessirkepov; Akmaral Duisenova; Vladimir I Trukhachev; Elena I Kostyukova; George D Kitas Journal: J Korean Med Sci Date: 2018-04-18 Impact factor: 2.153
Authors: Jordi Martorell-Marugan; Daniel Toro-Dominguez; Marta E Alarcon-Riquelme; Pedro Carmona-Saez Journal: BMC Bioinformatics Date: 2017-12-16 Impact factor: 3.169
Authors: Gwang Hun Jeong; Keum Hwa Lee; Jong Yeob Kim; Michael Eisenhut; Andreas Kronbichler; Hans J van der Vliet; Jae Il Shin; Gabriele Gamerith Journal: J Clin Med Date: 2020-01-23 Impact factor: 4.241