Sara Fernandes-Taylor1, Jenny K Hyun, Rachelle N Reeder, Alex Hs Harris. 1. Center for Health Care Evaluation, VA Palo Alto Health Care System and Stanford University School of Medicine, 795 Willow Road (MPD-152), Menlo Park, CA 94025, USA. sarataylorrohe@gmail.com.
Abstract
BACKGROUND: To assist educators and researchers in improving the quality of medical research, we surveyed the editors and statistical reviewers of high-impact medical journals to ascertain the most frequent and critical statistical errors in submitted manuscripts. FINDINGS: The Editors-in-Chief and statistical reviewers of the 38 medical journals with the highest impact factor in the 2007 Science Journal Citation Report and the 2007 Social Science Journal Citation Report were invited to complete an online survey about the statistical and design problems they most frequently found in manuscripts. Content analysis of the responses identified major issues. Editors and statistical reviewers (n = 25) from 20 journals responded. Respondents described problems that we classified into two, broad themes: A. statistical and sampling issues and B. inadequate reporting clarity or completeness. Problems included in the first theme were (1) inappropriate or incomplete analysis, including violations of model assumptions and analysis errors, (2) uninformed use of propensity scores, (3) failing to account for clustering in data analysis, (4) improperly addressing missing data, and (5) power/sample size concerns. Issues subsumed under the second theme were (1) Inadequate description of the methods and analysis and (2) Misstatement of results, including undue emphasis on p-values and incorrect inferences and interpretations. CONCLUSIONS: The scientific quality of submitted manuscripts would increase if researchers addressed these common design, analytical, and reporting issues. Improving the application and presentation of quantitative methods in scholarly manuscripts is essential to advancing medical research.
BACKGROUND: To assist educators and researchers in improving the quality of medical research, we surveyed the editors and statistical reviewers of high-impact medical journals to ascertain the most frequent and critical statistical errors in submitted manuscripts. FINDINGS: The Editors-in-Chief and statistical reviewers of the 38 medical journals with the highest impact factor in the 2007 Science Journal Citation Report and the 2007 Social Science Journal Citation Report were invited to complete an online survey about the statistical and design problems they most frequently found in manuscripts. Content analysis of the responses identified major issues. Editors and statistical reviewers (n = 25) from 20 journals responded. Respondents described problems that we classified into two, broad themes: A. statistical and sampling issues and B. inadequate reporting clarity or completeness. Problems included in the first theme were (1) inappropriate or incomplete analysis, including violations of model assumptions and analysis errors, (2) uninformed use of propensity scores, (3) failing to account for clustering in data analysis, (4) improperly addressing missing data, and (5) power/sample size concerns. Issues subsumed under the second theme were (1) Inadequate description of the methods and analysis and (2) Misstatement of results, including undue emphasis on p-values and incorrect inferences and interpretations. CONCLUSIONS: The scientific quality of submitted manuscripts would increase if researchers addressed these common design, analytical, and reporting issues. Improving the application and presentation of quantitative methods in scholarly manuscripts is essential to advancing medical research.
Authors: David Atkins; Dana Best; Peter A Briss; Martin Eccles; Yngve Falck-Ytter; Signe Flottorp; Gordon H Guyatt; Robin T Harbour; Margaret C Haugh; David Henry; Suzanne Hill; Roman Jaeschke; Gillian Leng; Alessandro Liberati; Nicola Magrini; James Mason; Philippa Middleton; Jacek Mrukowicz; Dianne O'Connell; Andrew D Oxman; Bob Phillips; Holger J Schünemann; Tessa Tan-Torres Edejer; Helena Varonen; Gunn E Vist; John W Williams; Stephanie Zaza Journal: BMJ Date: 2004-06-19
Authors: M Cases; L I Furlong; J Albanell; R B Altman; R Bellazzi; S Boyer; A Brand; A J Brookes; S Brunak; T W Clark; J Gea; P Ghazal; N Graf; R Guigó; T E Klein; N López-Bigas; V Maojo; B Mons; M Musen; J L Oliveira; A Rowe; P Ruch; A Shabo; E H Shortliffe; A Valencia; J van der Lei; M A Mayer; F Sanz Journal: J Intern Med Date: 2013-07-15 Impact factor: 8.989
Authors: A Carmona-Bayonas; P Jimenez-Fonseca; A Fernández-Somoano; F Álvarez-Manceñido; E Castañón; A Custodio; F A de la Peña; R M Payo; L P Valiente Journal: Clin Transl Oncol Date: 2017-12-07 Impact factor: 3.405
Authors: Robert A Oster; Christopher J Lindsell; Leah J Welty; Madhu Mazumdar; Sally W Thurston; Mohammad H Rahbar; Rickey E Carter; Bradley H Pollock; Andrew J Cucchiara; Elizabeth J Kopras; Borko D Jovanovic; Felicity T Enders Journal: Clin Transl Sci Date: 2014-09-12 Impact factor: 4.689
Authors: Jamie A Dean; Kee H Wong; Liam C Welsh; Ann-Britt Jones; Ulrike Schick; Kate L Newbold; Shreerang A Bhide; Kevin J Harrington; Christopher M Nutting; Sarah L Gulliford Journal: Radiother Oncol Date: 2016-05-27 Impact factor: 6.280