Marko Zdravkovic1, Joana Berger-Estilita2, Bogdan Zdravkovic1, David Berger3. 1. Department of Anaesthesiology, Intensive Care and Pain Management, University Medical Centre Maribor, Maribor, Slovenia. 2. Department of Anaesthesiology and Pain Medicine, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland. 3. Department of Intensive Care Medicine, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland.
Abstract
BACKGROUND: A debate about the scientific quality of COVID-19 themed research has emerged. We explored whether the quality of evidence of COVID-19 publications is lower when compared to nonCOVID-19 publications in the three highest ranked scientific medical journals. METHODS: We searched the PubMed Database from March 12 to April 12, 2020 and identified 559 publications in the New England Journal of Medicine, the Journal of the American Medical Association, and The Lancet which were divided into COVID-19 (cases, n = 204) and nonCOVID-19 (controls, n = 355) associated content. After exclusion of secondary, unauthored, response letters and non-matching article types, 155 COVID-19 publications (including 13 original articles) and 130 nonCOVID-19 publications (including 52 original articles) were included in the comparative analysis. The hierarchical level of evidence was determined for each publication included and compared between cases and controls as the main outcome. A quantitative scoring of quality was carried out for the subgroup of original articles. The numbers of authors and citation rates were also compared between groups. RESULTS: The 130 nonCOVID-19 publications were associated with higher levels of evidence on the level of evidence pyramid, with a strong association measure (Cramer's V: 0.452, P <0.001). The 155 COVID-19 publications were 186-fold more likely to be of lower evidence (95% confidence interval [CI] for odds ratio, 7.0-47; P <0.001). The quantitative quality score (maximum possible score, 28) was significantly different in favor of nonCOVID-19 (mean difference, 11.1; 95% CI, 8.5-13.7; P <0.001). There was a significant difference in the early citation rate of the original articles that favored the COVID-19 original articles (median [interquartile range], 45 [30-244] vs. 2 [1-4] citations; P <0.001). CONCLUSIONS: We conclude that the quality of COVID-19 publications in the three highest ranked scientific medical journals is below the quality average of these journals. These findings need to be verified at a later stage of the pandemic.
BACKGROUND: A debate about the scientific quality of COVID-19 themed research has emerged. We explored whether the quality of evidence of COVID-19 publications is lower when compared to nonCOVID-19 publications in the three highest ranked scientific medical journals. METHODS: We searched the PubMed Database from March 12 to April 12, 2020 and identified 559 publications in the New England Journal of Medicine, the Journal of the American Medical Association, and The Lancet which were divided into COVID-19 (cases, n = 204) and nonCOVID-19 (controls, n = 355) associated content. After exclusion of secondary, unauthored, response letters and non-matching article types, 155 COVID-19 publications (including 13 original articles) and 130 nonCOVID-19 publications (including 52 original articles) were included in the comparative analysis. The hierarchical level of evidence was determined for each publication included and compared between cases and controls as the main outcome. A quantitative scoring of quality was carried out for the subgroup of original articles. The numbers of authors and citation rates were also compared between groups. RESULTS: The 130 nonCOVID-19 publications were associated with higher levels of evidence on the level of evidence pyramid, with a strong association measure (Cramer's V: 0.452, P <0.001). The 155 COVID-19 publications were 186-fold more likely to be of lower evidence (95% confidence interval [CI] for odds ratio, 7.0-47; P <0.001). The quantitative quality score (maximum possible score, 28) was significantly different in favor of nonCOVID-19 (mean difference, 11.1; 95% CI, 8.5-13.7; P <0.001). There was a significant difference in the early citation rate of the original articles that favored the COVID-19 original articles (median [interquartile range], 45 [30-244] vs. 2 [1-4] citations; P <0.001). CONCLUSIONS: We conclude that the quality of COVID-19 publications in the three highest ranked scientific medical journals is below the quality average of these journals. These findings need to be verified at a later stage of the pandemic.
Authors: Qun Li; Xuhua Guan; Peng Wu; Xiaoye Wang; Lei Zhou; Yeqing Tong; Ruiqi Ren; Kathy S M Leung; Eric H Y Lau; Jessica Y Wong; Xuesen Xing; Nijuan Xiang; Yang Wu; Chao Li; Qi Chen; Dan Li; Tian Liu; Jing Zhao; Man Liu; Wenxiao Tu; Chuding Chen; Lianmei Jin; Rui Yang; Qi Wang; Suhua Zhou; Rui Wang; Hui Liu; Yinbo Luo; Yuan Liu; Ge Shao; Huan Li; Zhongfa Tao; Yang Yang; Zhiqiang Deng; Boxi Liu; Zhitao Ma; Yanping Zhang; Guoqing Shi; Tommy T Y Lam; Joseph T Wu; George F Gao; Benjamin J Cowling; Bo Yang; Gabriel M Leung; Zijian Feng Journal: N Engl J Med Date: 2020-01-29 Impact factor: 176.079
Authors: Alfred H J Kim; Jeffrey A Sparks; Jean W Liew; Michael S Putman; Francis Berenbaum; Alí Duarte-García; Elizabeth R Graef; Peter Korsten; Sebastian E Sattui; Emily Sirotich; Manuel F Ugarte-Gil; Kate Webb; Rebecca Grainger Journal: Ann Intern Med Date: 2020-03-30 Impact factor: 25.391
Authors: Temet M McMichael; Dustin W Currie; Shauna Clark; Sargis Pogosjans; Meagan Kay; Noah G Schwartz; James Lewis; Atar Baer; Vance Kawakami; Margaret D Lukoff; Jessica Ferro; Claire Brostrom-Smith; Thomas D Rea; Michael R Sayre; Francis X Riedo; Denny Russell; Brian Hiatt; Patricia Montgomery; Agam K Rao; Eric J Chow; Farrell Tobolowsky; Michael J Hughes; Ana C Bardossy; Lisa P Oakley; Jesica R Jacobs; Nimalie D Stone; Sujan C Reddy; John A Jernigan; Margaret A Honein; Thomas A Clark; Jeffrey S Duchin Journal: N Engl J Med Date: 2020-03-27 Impact factor: 91.245
Authors: Charles Reynard; Daniel Darbyshire; Gabrielle Prager; Anisa Jabeen Nasir Jafar; Mina Naguib; Govind Oliver; Patricia van den Berg; Richard Body; Harriet Ambroziak; Simon Carley Journal: BMJ Simul Technol Enhanc Learn Date: 2021-06-08
Authors: Jelena Jacimovic; Aleksandar Jakovljevic; Venkateshbabu Nagendrababu; Henry Fergus Duncan; Paul M H Dummer Journal: Clin Oral Investig Date: 2021-04-06 Impact factor: 3.606