Are we engaged in reproducible research?

Sir,

Over last two centuries, we have witnessed the impact of significant advancement of science and medicine on the betterment of the human health. However, we often get to know stories in media on scientific misconduct or cases of lack of reproducibility which is generally translated into the concerns that the “science is fractured.” Like any other human endeavors, science is not flawless, and it is believed to be a field which is subjected to the most rigorous scientific scrutiny and policing compared to other field of activities.[1] Moreover, it could be the only field where errors are systematically analyzed, debated/criticized, and often in time corrective measures are taken. In the recent issue of the Journal of Ayurveda and Integrative Medicine, the editor has rightly pointed out that to do good publications a good research is needed and also discussed several critical factors to achieve the same.[2] We in this letter wish to continue on the discussion on why we need to focus on reproducible research and how the stakeholders need to reprogram themselves to yield a robust, and reproducible research outcome.

Drug development process is highly dependent on the literature available on the subject with a main focus on new targets and biology. In this scenario, the academia and industry are expected to work hand in hand to generate robust, reliable, and trustworthy research data which can be used mutually by these major pillars of the drug discovery and development. This scientific research is expected to be based on two basic characteristics namely; reproducibility - the ability to recompute results, and replicability - the ability to replicate experiments with consistency.[3] With an ultimate objective to convert these “discoveries into the recoveries” entire scientific community is expected to work in a complimentary fashion. However, the ground reality is not as is expected. Recently, Amgen Inc. a California-based biotechnology company did run an in-house research program in Hematology and Oncology Department to confirm and reproduce the published findings in the oncology area. They selected 53 published studies which were considered as “landmark” as they reported innovative approaches to target cancers and repurposing of existing therapeutics. With their shocking observation they found that only 6 (11%) of those could be confirmed, even keeping in mind the limitations of preclinical research in oncology.[4] A similar experience was noticed by massive effort run by Bayer pharmaceuticals, where they could match up only 14 studies out of 67 target validation projects in oncology, women's health, and cardiovascular medicines. This evidence pushes us to ponder over the possible reasons for this alarming situation.

Among the various factors including incentive and reward system in Universities and Research Institutions, expectation to publish consistently novel ideas and research, undue emphasis by publishing houses and scientific fraternity on positive only results, publishing in high impact journals, performance-based promotion etc., need a careful review followed by systematic changes. Scientific journal editors, reviewers, and grant Review Committees tend to often read the perfect scientific story which is simple, clear, and complete, thus in a way tempting the investigators to provide selective datasets for publication and no encouragement to provide negative findings. There are a growing number of the biomedical students pursuing their postdoctoral research for several years, and the long time to get first independent grants funded are among detrimental factors in boosting the morale of the young generation. Even the pharmaceutical companies are not totally transparent on disclosing the “not so encouraging” research facts from their preclinical and clinical development programs. Overall, the scientific behavior of both the authors and institutions (academia, industry) has changed over the decades and need to be corrected through a systematic approach.

In order to improve these situation ample opportunities and encouragement to present negative results must be created, e.g., by organizing the conferences focused on negative findings thus boosting the researchers to share their findings and create healthy scientific discussion to dissects the reasons to learn, improve, and modify approaches prospectively. Journals must come up with the special sections to report the negative findings of their work, a healthy dialogue among the key players, e.g. physicians, basic, applied scientist, and the feedback from the patients need to be considered while designing the research plan, more importance can be given to teaching, mentoring activities as against only consistently publishing into high impacted journals which otherwise weakens the contribution of the great teachers, mentors, and gurus! Other approaches to enhance protocol sharing, collaborative multi-centric research, called as team science must be boosted. More consortiums to support multi-centric studies both preclinical and clinical, standardized protocol use, open exchange of datasets, and results are needed to overcome discrepancies in methods and inference of scientific experimental studies.

In an attempt to address this issue National Institute of Health in 2014,[5] issued the guidelines for reporting the preclinical research in consultation and backing from over 30 basic/preclinical science journals in which the agency funded research is published. The PLOS one and science exchange had started an initiative through which the scientist can pay and get their work validated with the help of an independent lab. Nature had also included an 18 point checklist for authors to ensure that all the related technical and statistical information which is critical for reproducibility of an experiment is provided and or which may lead to bias in publication (http://www.nature.com/ncomms/authors/submit.html#Stat-guidelines) Also perspectives on psychological science had introduced a section on replications (http://www.psychologicalscience.org/index.php/replication). Other good measures such as strict financial disclosure norms, improved transparency, and more data disclosures would be necessary.

In conclusion, it would be wise not to put so much pressure on the young scientist to publish frequently but to publish little but good research. In the process, the mindset needs a constructive change among the stakeholders such as senior scientific leaders, advocacy groups, government agencies, university leaderships, teachers, research organizations, and pharmaceutical companies to promote ethical and transparent research practices which will yield a culture of reproducible research for the betterment of human race.