Editorial research and the publication process in biomedicine and health: Report from the Esteve Foundation Discussion Group, December 2012.

Despite the fact that there are more than twenty thousand biomedical journals in the world, research into the work of editors and publication process in biomedical and health care journals is rare. In December 2012, the Esteve Foundation, a non-profit scientific institution that fosters progress in pharmacotherapy by means of scientific communication and discussion organized a discussion group of 7 editors and/or experts in peer review biomedical publishing. They presented findings of past editorial research, discussed the lack of competitive funding schemes and specialized journals for dissemination of editorial research, and reported on the great diversity of misconduct and conflict of interest policies, as well as adherence to reporting guidelines. Furthermore, they reported on the reluctance of editors to investigate allegations of misconduct or increase the level of data sharing in health research. In the end, they concluded that if editors are to remain gatekeepers of scientific knowledge they should reaffirm their focus on the integrity of the scientific record and completeness of the data they publish. Additionally, more research should be undertaken to understand why many journals are not adhering to editorial standards, and what obstacles editors face when engaging in editorial research.

Introduction

Journal editors have various tasks and responsibilities: from defining submission, peer review and disseminating processes, selecting articles to be published, contributing editorial pieces, dealing with scientific misconduct, providing leadership and editorial team management, and ultimately raising standards in methodological rigor, scientific reporting and public knowledge. They also face numerous challenges, particularly in ensuring the integrity of the published records, such as publication misconduct. Some of these issues have been addressed by the policies and guidelines of editorial organizations such as the World Association of Medical Editors (WAME), International Committee of Medical Journal Editors (ICMJE), Committee on Publication Ethics (COPE) and Council of Science Editors (CSE). Yet, despite an estimated number of more than twenty thousand journals (and thus many more editors) in the world, research into the work of editors and the publication process in biomedical and health care research is quite rare.

On the 12th and 13th December 2012 in Barcelona, an invitation-only Esteve Foundation Discussion Group was organized by the Esteve Foundation, a non-profit scientific institution whose primary goal is to foster progress in pharmacotherapy by means of scientific communication and discussion. The goal of the discussion group was to examine current editorial research in biomedicine and make recommendations for its future.

Plenary presentations

The two day discussion started with a presentation by Ana Marušić, editor in chief of the Journal of Global Health, vice president of the European Association of Science Editors (EASE) and former president of WAME and CSE. She elaborated on terms indexed by the National Library of Medicine (NLM) through the Medical Subject Headings (MeSH) thesaurus. These terms present concepts in medicine, which are then used to find evidence for research and practical questions in biomedicine (1,2). Mapping of key concepts in editorial research, such as journal, impact factor, publishing, peer review, and authorship, is rather incomplete and often only a recent addition to the MeSH. Most of these concepts are categorized within information sciences, but some do not match their current use in health research. A good example is the definition of authorship. Whereas MeSH defines the concept of authorship as: “The profession of writing. Also the identity of the writer as the creator of a literary production.”; the current ICMJE guidelines specifically address four qualifying criteria related to research, writing, approval of the manuscript and accountability for the work (3). Prof. Marušić also pointed out that several general and specialty journals publish articles on editorial research (often called ‘peer review’ or ‘publication research’) but, to date, no journal fully specializes in this research field. Likewise, the funding opportunities for editorial research are scarce.

Annette Flanagin, executive managing editor of JAMA (Journal of the American Medical Association) and coordinator of the International Congress on Peer Review and Biomedical Publication, reported on some of the groundbreaking research presented at the past six Peer Review Congresses and described its origins. Drummond Rennie, JAMA editor and director of the Peer Review Congresses, has initiated and led the effort to put peer review and other methods of assessment in biomedical research under the same level of scrutiny that journals and editors demand of science itself. Since 1989, the quadrennial Peer Review Congresses have encouraged and provided a forum for research into all aspects of the biomedical research dissemination enterprise, including research into management and reporting of bias, transparency of research reporting, authorship and contributor-ship, peer review, editorial evaluation, and dissemination of biomedical information (4).

Xavier Bosch, consultant internist at the Hospital Clinic of Barcelona and researcher in peer review, reflected on the scale of scientific misconduct in biomedicine, means of dealing with it and the journals’ role in promoting research integrity. In a survey of 41 journals which published retractions or corrections due to scientific misconduct from 1992 to 1999 only 14% addressed scientific misconduct in their instructions to authors (5). Of 50 high-profile biomedical journals surveyed in 2006, only 7 (14%) had written and publicly available misconduct policies (6). In 2009, a survey of 197 editors of randomly chosen journals showed that 94 (48%) journals had a written policy, of which 53 (56%) were policies of the publishers (7). Additionally, of 399 top-ranked journals, 140 (35%) had explicit definition of misconduct, 143 (36%) used policies of the publishers, and another 124 (31%) adopted them from policy-producing bodies. A total of 112 (28%) journals used plagiarism-checking services (8).

Erik von Elm, senior epidemiologist at the Institute of Social and Preventive Medicine in Lausanne, Switzerland, co-director of Cochrane Switzerland, and Academic Editor of PLoS ONE presented evidence on journals’ endorsement of recommendations aimed at improving publication practice. Several surveys of paediatric, haematology and urology journals showed that about 60% of journals direct their submitting authors to the Recommendations for the Conduct, Reporting, Editing and Publication of Scholarly Work in Medical Journals (formerly, Uniform Requirements), published by the International Committee of Medical Journal Editors (ICMJE) (3). About 35% required that trials reported in submitted manuscripts had been previously registered. 70% of journals had conflict of interest policies but only 24% recommend use of any reporting guidelines (9–13). These numbers demonstrate that more efforts are needed to improve endorsement of such recommendations. In turn, more accurate and complete reporting of primary studies would enable their inclusion in systematic reviews and, thus, potentially contribute to improvements in health care.

Peter Gøtzsche, professor of Clinical Research Design and Analysis, and Director of The Nordic Cochrane Centre in Copenhagen, Denmark, reflected on the accountability, transparency and honesty in the medical sciences. He called for a revolution in scientific research and reporting, and emphasised that too many published papers serve no one but the authors or sponsors, with a large number of papers being biased or superfluous and therefore unusable for furthering of public knowledge. He also stressed that anonymised raw patient-level data, alongside research protocols and amendments, needs to be made publicly available for independent research and scrutiny, and this practice enforced by legislations (14).

David L. Schriger, professor of emergency medicine at the University of California, Los Angeles, USA, and deputy editor of Annals of Emergency Medicine, discussed current peer review practices. While peer reviewers are typically given specific instructions on the format of their review, they are often not told the main purpose of their task. Most likely, editors also lack a clear view on whether reviewers are being asked to sort the true from the non-true, or if their primary job is to ensure that the paper contains sufficient information to allow readers to decide whether they agree with the authors’ conclusion, a state Ziman termed consensibility (15). The current focus on methodological rigor may make reviewers believe that their primary task is to determine validity but this can be a dangerous practice, e.g. in the case of Helicobacter discovery, which would later be awarded the Nobel Prize in Medicine, editors of Lancet were unable to find reviewers who believed the study findings (16). An argument can therefore be made that the main focus of peer review is to ensure that the work is complete and clear so that the scientific community can judge its validity. Similarly, in keeping with the focus on methodological rigor, most reporting guidelines emphasize the reporting of methods, giving less attention to the means of reporting results. Measures such as enhanced access to datasets, use of improved graphics and liberal use of electronic supplementary data tables and figures, can provide readers with a more comprehensive view of published research findings.

Harvey Marcovitch, former chairman of COPE, associate editor of the British Medical Journal, and a former Board Director of CSE, reported on retractions of publications and pressures faced by editors. From 2000 to 2010 there has been an increase in the number of retracted publications in biomedicine (17). Research on 2047 retracted articles indexed by PubMed showed that most (67%) were retracted due to misconduct (18). There is also evidence from individual cases that editors ignore suspicions of fraud and fail to properly investigate matters, even after receiving support by organizations such as COPE. As membership in COPE is optional, legislative or other measures are needed to deal with and prevent scientific misconduct.

Specific discussion and recommendations

Following each of the plenary lectures, many questions were raised, discussed and some solutions for the existing problems proposed. In the final session and through input provided one week after the meeting, several key topics and recommendations emerged.

Guidelines for editors

There is considerable heterogeneity in funding, experience and tradition of large and small journals, as well as those in different countries and fields. This requires broader editorial guidelines that would cover the whole spectrum of editorial endeavour, and include sections on means to obtain indexation of journals in bibliographic and citation databases, as well as methods of screening for duplicate or plagiarized research. As many editors of academic and scholarly journals are researchers themselves and may publish in journals they edit (19), clear guidelines and editorial processes need to be established and made transparent to tackle this specific conflict of interest. Likewise, existing guidelines for editors need to be made more visible and their effectiveness appraised.

Editorial research

Depending on the available resources, contract agreements, publisher dependence, and level of interest, editors differ in the amount of time they dedicate to editorial research. On a larger scale, many aspects of editorial work thus remain unknown, including more precise estimates of key editorial practices such as rejection rates, proportion of manuscripts sent for review and changes that occur following peer review. In order to encourage more editors and independent researchers to become involved in editorial research, and especially in multi-journal collaborations, it is critical that publishers and funders recognize the importance of this research field and establish funding schemes that would both sustain and increase editorial research competitiveness, as well as its quality. Likewise, additions and re-definitions of key concepts of editorial research in MeSH are needed to increase the field’s visibility.

Need for policy change

Initiatives of individual journals and editorial groups alone are probably insufficient to bring about necessary changes in reporting, availability and accessibility of scientific research and data. Legislative actions, similar to those of the Research Councils’ Policy on Open Access and Supporting Guidance in the UK (20), and the proposed Fair Access to Science and Technology Research Act in the USA (21), would not only secure open accessibility of all research involving humans, but also establish, promote and monitor quality standards of scientific reporting. Ethical and institutional boards need to play a more proactive role in monitoring research they approve by requiring project updates or milestone reports. Study protocols of all types of research should be made available either through institutional boards or publicly accessible registries. These databases should allow for submission of raw data, within a fixed timeframe following the studies completion, and in that way prevent instances of non-publication or partial publication and unavailability of data from publicly or industry funded human research. Educational courses on research methodology, statistics and scientific reporting, complemented by courses on scientific integrity, should be mandatory in graduate curriculums. And lastly, a part of a research grant should be reserved exclusively for publication purposes, depositing raw data in an easily analyzable format, and possibly for adherence to reporting guidelines.

Collaborating to improve peer review

Knowledge, experience and practice from editorial research in biomedicine and health research should be disseminated to other fields and vice versa. To reduce wasteful use of resources, peer review reports of manuscripts submitted but not accepted for publication could be made available through journal websites or specific databases to help and enable editors of other journals to make better informed decisions. This might also prevent instances in which authors submit unfinished manuscripts and abuse a journal’s peer review process as a means of assistance with writing (or polishing) their papers.

Letters to the editor and post publication peer review

A crucial component of scholarly publishing, and science in general, is debate. However, many journals impose stringent word limits and short submission periods for letters to the editors in response to published articles (22). To provide opportunities for debate, electronic means for commenting and reviewing published articles have been introduced by some journals (e.g. BMJ or Pediatrics), but smaller journals may not have the resources to do so. For online post-publication peer review to be useful, effective models of its moderation and linkage to literature databases need to be developed.

Conclusion

Editors today should reaffirm their focus on the integrity and completeness of the data they publish. Several initiatives and instruments exist to help them with this endeavour, but additional efforts are needed to make them known, in particular to editors of regional and newly established journals. Initiatives of individual journals, e.g. publication of submitted manuscripts alongside peer review comments and final versions, demonstrate that innovative procedures are feasible, at least at small scale. However, without changes in policy, either state or inter-journal, these initiatives remain only perks of individual journals. More research is needed to better understand critical questions such as why many journals are reluctant to use reporting guidelines or checklists, or to react adamantly to improperly published papers (23). Furthermore, better identification of actors and stakeholders of editorial research is needed, including identification of obstacles editors face when engaging in editorial research, and incentives that are or should be in place in order for them to do so. In the fast changing world of biomedical publishing, editorial research could play an essential role in identifying the changes that are needed and viable if editors are to remain gatekeepers of scientific knowledge.