Researcher's Perceptions on Publishing "Negative" Results and Open Access.

Scientific advance is based on reproducibility, corroboration, and availability of research results. However, large numbers of experimental results that contradict previous work do not get published and many research results are not freely available as they are hidden behind paywalls. As part of COST Action "DARTER", a network of researchers in the field of RNA therapeutics, we have performed a small survey among our members and their colleagues to assess their opinion on the subject of publishing contradictory or ambiguous results and their attitude to open access (OA) publishing. Our survey indicates that, although researchers highly value publication of "negative" results, they often do not publish their own, citing lack of time and the perception that those results may not be as highly cited. OA, on the other hand, seems to be widely accepted, but in many cases not actively sought by researchers due to higher costs associated with it.

Introduction

We live in the era of “publish or perish” and the publication of experimental negative results is disappearing from most disciplines and countries: it has been described that the percentage of studies, where the initial hypothesis was verified, grew by 22% between 1990 and 2007, with papers claiming to have produced positive results rising to 85% of published studies [1]. This shows that the publication of negative findings is very limited at present. However, science is based on testing hypotheses and working to prove them should be considered equally important regardless of whether they are wrong or right. Results that do not confirm a research hypothesis or that contradict or do not reproduce previously published data may still provide highly valuable information for the comprehension and evaluation of scientific paradigms. Similarly, inconclusive results are often due to limitations in the technical approaches used in the study and their publication could encourage other groups to use better methodologies to study the same or similar problems.

Considering the nature of scientific research, the high percentage of publications reporting positive results is clearly not a true reflection of research results and, in some measure, it is linked to scientific misconduct and lack of reproducibility of results in science [1,2]. This issue arises, for example, when experiments are repeated several times until a positive result is achieved or when positive findings, which are obtained by using an insufficient number of replicates, are published as genuine research results.

Many reasons could be at the origin of this bias toward the preferential publication of positive results: one of them is the pressure to publish high-impact data, particularly among academics who are led to believe that their work needs to be groundbreaking to get published [3]. This misconception is strengthened during the review process, when authors are induced to prioritize novelty and interest over data accuracy [2]. In 2012, the European Commission recommended all EU member states to put public-funded research results in the public domain to improve science and strengthen Europe's knowledge-based economy. To encourage this further, all projects receiving funding from the Horizon 2020 programme (the largest financial instrument funding research in the EU during the period 2014–2020) were required to make sure that any peer-reviewed journal article they publish was openly accessible, free of charge. The possibility of using an “open access” option allows scientists to freely access the content of a research output and, therefore, it significantly increases the visibility of a study. Journals usually allow the choice between two open access (OA) plans: a “green” option where the publication is archived in an OA repository for a limited period before becoming freely accessible, and a “gold” option where the content of the research output is made immediately available [4]. Usually, the publication under the latter “golden” option requires the payment of charges that are often high and sometimes prohibitive for small research groups.

As members of a large international network of mainly European researchers, we wanted to gauge our colleagues' opinion on two interesting topics of debate: the publication of “negative” results and the choice of publishing under OA. To gather this information, we conducted the survey described next.

Materials and Methods

A document that could be easily shared by email was prepared by using google surveys. Through it, we gathered information about the respondents' profile (career stage, field of research, average number of publications), and we asked several questions about their experience with publishing negative results and OA science (see Supplementary Data S1 for a copy of the survey and full summary of results here: https://tinyurl.com/s2md4h8).

Participants

The survey was emailed to all members of COST Action CA17103 “Delivery of Antisense RNA ThERapeutics (DARTER)” (www.antisenserna.eu) with a request to complete and distribute it to their colleagues. We received 96 responses from 18 different European countries (Table 1). This included senior researchers (more than 7 years after completing a PhD, 57/96), young PhD researchers (26/96), and predoctoral researchers (13/96). Their research fields were mainly biology/biomedicine disciplines (72/96). The remaining researchers (24/96) belonged to wide academic backgrounds: chemistry, clinical medicine, chemical biology, computer science and rehabilitation, drug delivery, engineering, gene and cell therapy, genetics, public health, toxicology, and veterinary.

Table 1.

Countries and Number of Participants to the “Open Access and Negative Results Survey” Conducted by COST DARTER Action

Country	n
Austria	1
Croacia	3
Cyprus	2
France	13
Germany	1
Greece	1
Italy	11
Latvia	2
Netherlands	6
Poland	2
Portugal	7
Romania	1
Serbia	5
Slovenia	1
Spain	28
Sweden	1
Turkey	4
United Kingdom	7
Total	96

Results

The majority of our respondents (85%) said that they were able to decide or had a direct input when deciding how and where to publish their work, and most of them (68%) affirmed that they publish one or two articles per year.

Publishing negative results

On the subject of the value of negative results, our survey revealed that 79% of respondents considered that finding negative or unexpected results is not an undesirable thing and 82% of them think that they should be shared among laboratories (Fig. 1A, B). Nevertheless, only 14% of them had tried to publish negative results and, interestingly, most of them had managed to publish their work (Fig. 1C).

FIG. 1.

Researchers' perceptions on publishing negative results. Summary of the main findings of our survey on the subject of negative results. (A) Only 4% of responders think that finding a different result from what is already published is a negative thing. (B) The majority of responders considers the sharing of negative results with the scientific community to be valuable. (C) Eighty-six percent of respondents have never published “negative results”. (D) The majority of respondents would like to publish negative results. (E) Lack of time (53%) and perception of being less cited (26%), are indicated as the main reasons not to try to publish negative results. Results of the full survey are available at https://tinyurl.com/s2md4h8

Although most respondents were willing to publish their contradictory results (59% would like to publish them and 41% would at least consider it), the majority of them did not publish these results, arguing that this is too time consuming (53%) or worrying that their work may be less cited (26%) (Fig. 1D, E).

Publishing research results under OA

Although most respondents (80%) had already published in OA journals, 35 out of 96 respondents considered choosing an OA journal for their publication a major requirement. The remaining admitted that they had published in OA by chance (49/96), or because they had specific funding for this option (13/96) (Fig. 2A).

FIG. 2.

Researchers' opinions on OA. Summary of the main findings of our survey on the subject of publishing in OA. (A) Eighty percent of respondents have previously published in OA, but most of them did not actively seek publishing OA when they did. (B) Impact factor and popularity in the field are the main reasons to choose a journal. (C) High publishing costs of OA options is the main hurdle to researchers. Results of the full survey are available at https://tinyurl.com/s2md4h8 OA, open access.

According to the results obtained in our survey, the two main considerations when choosing a target journal were the journal's impact factor (86/96) and the perceived popularity of such a journal in the particular field of the respondents (65/96). Cost (25/96), quality feedback (23/96), ease of publication (21/96), and turnaround time (15/96) were considered to be less critical (Fig. 2B). It is interesting to note that those not interested in publishing on OA journals claimed the high article processing charges (APCs) as their main reason not to do it (Fig. 2C).

Discussion

Our survey was distributed among a wide representation of European researchers and its results allow us to better understand the general feeling of our scientific community regarding the publication of negative results and the use of OA, two important initiatives to promote a more productive and global science. It is widely accepted by the scientific community that OA contributes to more efficient science and innovation in public and private sectors by making accessible scientific publications and research data. In addition, it could be argued that every researcher and/or citizen should have access to the scientific results that have been funded mostly by public sources. Accordingly, most researchers in our survey would be happy to make their work accessible, but APC costs seem to be their main reason not to do it. If this is the case, funding authorities may have a role in either negotiating better APC rates with publishers or considering increasing specific OA funding. Some European initiatives are being launched on this purpose, such as the Horizon 2020 programme, which demands to ensure OA to all scientific publications related to this program (https://ec.europa.eu/programmes/horizon2020/en).

Scientific knowledge is advancing very fast and the ways of doing science are changing with it. Alternative options to share scientific results are growing, such as public repository sites like Arxiv (https://arxiv.org/) or Biorxiv (https://www.biorxiv.org) as well as private companies hosting websites where to share publications and other datasets such as Research Gate or Academia.edu. Nevertheless, there is still much work left to promote and extend their use, as a large percentage of researchers do not to use any (Fig. 2).

Regarding the sharing of negative and inconclusive results, we have found that, although researchers think that negative results are valuable for the scientific field, they do not publish them because they consider it detrimental to their individual careers [5] or because it is perceived that the whole process of research output preparation is still too time consuming for the final value of the product. There are many initiatives that can make the publication of such results more attractive: well-regarded journals have published editorials praising the publication of negative results, replication studies, and null results [6,7]; new journals have been specifically created to publish negative results (such as Negative Results Scientific Journal, Journal of Negative Results in Biomedicine, the Journal of Negative Results—Ecology and Evolutionary Biology, and the Psychology Journal of Articles in Support of the Null Hypothesis). To encourage this further, some international organizations, such as the European College of Neuropsychopharmacology (ECNP), the Human Brain Mapping, or the Berlin Institute of Health Research, offer awards for the best negative and replication studies [7,8]. Outside of traditional academic publishing, a suggestion to make an OA repository for all research findings has also been made, but this initiative has had very limited success so far. Again, funding bodies may play a pivotal role to incentivize this effort by modifying the indicators currently used to evaluate research projects and the researcher's profiles: the currently used metrics are becoming an objective themselves; this makes some results preferable to others, and it makes scientific misconduct much more profitable. A system based on prioritizing well-planned and thorough research, other than number of publications, impact factors, and the researcher's H factor, would de-incentivize publishing practices that make negative results unattractive to researchers.

A change in the way funding bodies approach this problem is in their own interest: after all, the current situation increases the likelihood of scientists repeating work that has already been proven wrong, and encourages scientific misconduct, which is not only ethically wrong but also very expensive for those funding research.

Therefore, options to encourage the publication of negative results are available and scientists should abandon the misconception that publishing negative results is too much time consuming to be valuable and be more proactive in exploiting such resources. A change in mindset is likely important to achieve such shift in attitude. When scientists were asked “How would you make negative results more attractive for publication?” some concepts were recurring in comments such as “Share them with other groups working in the same field and try to collaborate” or “Validating together with other laboratories to make a stronger point” suggesting that there is the willingness to include negative findings in publications but after sharing and pulling them with others from different laboratories. Hence, these negative or inconclusive results would also be substantially validated by collaborations between multiple laboratories and that would make such results more valuable, visible, and, in turn, impactful in the field.

In this regard, our survey may be biased, as the researchers belonging to our COST action may be naturally keener to share their work as it is one of the main objectives of COST actions and possibly those that replied to the survey have a particular interest in the subject and we acknowledge this. We consider that these results are a starting point for our future activities: our network will continue until October 2022 and we have set up a “negative results” working group. We already allocate time to discuss negative results in our workshops and we will be looking for ways to incentivise this further among our members. We anticipate some difficulties, but we think we are on the verge of a shift in the Open Science policies that will probably have a knock-on effect on other practices. We are looking forward to contributing to this paradigm change.