Contradicting/negative results in clinical research: Why (do we get these)? Why not (get these published)? Where (to publish)?

Obtaining negative or contradicting results, whereas conducting a study has always been overlooked as inadequacies on the part of the researcher. Many-a-times, negative results are arrived at even after conducting the study with great care and effort. This cannot be considered, a flaw, always. Contradicting results may be arrived at because of various reasons and should be considered and published in order to arrive at a definitive result. Various journals are available which publish these contradicting results exclusively. It is the duty of the researcher to know in detail the cause and effect of these, considering the pros and cons. This article gives a bird's eye view of when, where and how to manage negative or contradicting results of a study.

INTRODUCTION

“Once you replace negative thoughts with positive ones, you’ll start having positive results” a famous quote by an American musician, Willie Nelson. Furthermore, Francis Bacon in The Advancement of Learning states “It is human nature for the affirmative or active to effect more than the negative or privative”. Hence a few times hitting, or presence, countervails oft-times failing or absence”. It is common to find evaluation of same kind of hypothesis by many researchers. The effect obtained in each of these studies may differ and the hypothesis may be accepted in some while rejected in other studies. Although adequately powered and other determining and confounding factors have been controlled, a study may show a result that contradicts the already known observations/facts. If such negative/contradicting studies are not published, the ascertained effect size remains falsified.

WHY DO WE GET CONTRADICTING RESULTS FOR THE SAME STUDY?

Before a study is initiated, the scientist sets up a hypothesis in mind. Many a times, the scientist knows what he/she wants and therefore, this will influence the study results. Hence, it is prudent to confirm the effect of any intervention by repeating it in different laboratories/conditions by various scientists. When done so, the subsequent studies may either contradict or may show reduced or stronger effect size than the earlier ones. A study published in the journal nature[1] indicates that only 11% of the findings from preclinical studies carried out on anti-cancer drugs under controlled laboratory conditions were reproducible. This may either be attributed to genuine entities such as regression to the mean/declining effect or may be, pitfalls due to researchers’ tendency to stick to their hypothesis and publishing only those results pertaining to, while omitting the others. This declining effect has been reported to exist in all fields of medicine, especially cardiovascular and psychiatry.[23] Ioannidis[4] studied 49 trials that were cited more than 1000 times and published in journals with high impact factor and found 16% producing contradictory results, more common in nonrandomized than in randomized studies. Contradictory results are rampantly seen in genomic studies. A comprehensive analysis by Hirschhorn et al.[5] on 166 genome-wide association studies has shown that only six of them were reproducible.

WHY ARE THESE NOT PUBLISHED AND SO WHAT?

The researcher may feel that a study with a negative result may not have good impact in their respective fields or does not help them in getting funds or promotion in their career and hence may not submit their work to any journal. A study carried out in United States[6] has confirmed that the academic competition and pressure increases the risk of scientists’ bias in not publishing negative studies. The journals on the other hand may not be willing to publish a study with negative or contradicting results when compared to the earlier ones as they might not have a positive impact. Decoursey[7] argues that though there are journals that are intended to publish negative results, the journal editors usually ask explanation for the contradicting effect that was arrived as compared to previous one, which is obviously difficult to analyze and hence, it becomes difficult to publish such studies. Although, the International Committee of Medical Journal Editors (ICMJE)[8] clearly recommends that it is not obligatory for the journal editors to be under control of the comments of the peer reviewer, sometimes, the journal editor may fall prey to such process, recommending rejection of contradicting studies.[9] Not publishing the results of a study done on participants is unethical and can also be considered, a scientific misconduct. Easterbrook et al.[10] in a retrospective study from Institutional Ethics Committee records reported that the results of observational studies were less commonly published than randomized trials. Among the randomized clinical trials, nearly one-third of the results ultimately do not get published. In 1990, the proportion of papers published with positive results was found to be around 70% while in 2007, it had risen to almost 86% across different disciplines and it was almost 90% in biomedical field on the same year.[11] Further, the same study also illustrates that corresponding authors from Asian countries reported more positive results than from United States or Europe. Even in the published clinical trials, not all the results get mentioned. Of course, this is one of the main reasons for creating a central clinical trial registry platform and making it mandatory that the journals that follow ICMJE should intend to publish a clinical study only when they are registered in any such database. The results mentioned in the article should be complete and compatible with the outcome that was intended to be measured as shown in the registry. Due to the non-availability of results from these studies, a publication bias may emerge. This was rightly described by Rosenthal as “Fail safe File-drawer” problem.[12] Although the absence of these study results may affect a narrative review, more significant impact is observed in either a systematic review or a meta-analysis. Meta-analysis includes analysis of analyses and involves pooling of various studies that evaluated the same hypothesis. In such analysis, when studies of negative results are not included, the final pooled estimate will be distorted. Although tools such as “Egger's regression analysis”, “Funnel Plot”, “Fail-safe N”, are available, none of them are shown to be perfect.[1314] Furthermore, it is important to include these studies in health technology assessment, where the effect and the cost of an intervention are compared with others, because, this influences the decision-making bodies such as National Institute of Health and Clinical Excellence and National Screening Committee.[15] Considering the limitation of budget allocated for conducting research studies, the availability of such negative results will defer another from repeating the same study.

WHERE TO PUBLISH THE RESULTS OF THESE STUDIES?

In view of the importance of publishing studies with contradicting results, ICMJE has recommended journal editors to consider publishing such studies if they are genuine and scientifically valid. Recently, a few journals have emerged whose scope is to publish only those studies with contradicting results/negative results. These are Journal of Negative Results in Biomedicine,[16] Journal of Negative Results,[17] Journal of Contradicting Results in Science,[18] The all Results Journal,[19] Journal of Negative Pharmaceutical Results,[20] International Journal of Negative and Null Results,[21] Journal of Errology.[22] Apart from these journals dedicated for studies with contradicting results such as PLoS One,[23] F1000 Research[24] and Scientific Reports[25] in their publication criteria states that a study reporting negative results will be considered if their insight is useful. All of these are open access and charge for publishing the article, although F1000 research had waived these charges recently. In January 2013, as a joint venture between British Medical Journal, Center for Evidence-based Medicine in Oxford, Cochrane collaboration, Sense about Science, Dartmouth Institute, PLoS, James Lind Alliance and Bad science, an initiative called All Trials was launched.[26] This campaign urges registration and reporting of results of all the clinical trials even when it is negative or null so that bad treatment decisions or missed opportunities for good medicine or unnecessary repetition of trials will be minimized. Another initiative is from European Commission under the Seventh Framework Program called “OPEN” with an objective “To overcome failure to publish negative findings”.[27] This group first attempts to identify and explore the current extent of publication bias by conducting a systematic review[28] and secondly will find their impact on health technology assessments and ways to address this issue. Schooler[29] suggests on developing an open-access repository, where a researcher lets his hypothesis and methodology to be stored before starting the study and intends to publish the results irrespective of the outcome.

To conclude, both the researcher and the journal editor should feel that studies with negative results are equally important to be published providing opportunities for more transparent disclosure of results that may impact the public health.