Scientific integrity resource guide: Efforts by federal agencies, foundations, nonprofit organizations, professional societies, and academia in the United States.

Scientific integrity is at the forefront of the scientific research enterprise. This paper provides an overview of key existing efforts on scientific integrity by federal agencies, foundations, nonprofit organizations, professional societies, and academia from 1989 to April 2016. It serves as a resource for the scientific community on scientific integrity work and helps to identify areas in which more action is needed. Overall, there is tremendous activity in this area and there are clear linkages among the efforts of the five sectors. All the same, scientific integrity needs to remain visible in the scientific community and evolve along with new research paradigms. High priority in instilling these values falls upon all stakeholders.

American Association for the Advancement of Science

Association of American Medical Colleges

American Chemical Society

American Sociological Association

Berkeley Initiative for Transparency in the Social Sciences

Center for Advanced Study in the Behavioral Sciences

Center for Clinical Research Ethics

Centers for Disease Control and Prevention

Code of Federal Regulations

Committee on Publication Ethics

Center for Open Science

Council of Science Editors

Departmental Regulation

US Environmental Protection Agency

Federation of American Societies for Experimental Biology

US Food and Drug Administration

Federal Register

US Department of Health and Human Services

Health and Medicine Division of NAS (formerly known as IOM)

NIH Institutes and Centers

Institute of Clinical and Translational Sciences

Institute of Medicine (former name of HMD)

Laura and John Arnold Foundation

National Academy of Sciences

National Institutes of Health

National Science Foundation

NIH Office of Extramural Research

Office of Research Integrity

Center for Open Science's Open Science Framework

Office of Scientific Integrity

Office of Science and Technology Policy

Public Employees for Environmental Responsibility

Public Health Service

Responsible Conduct of Research

US Agency for International Development

US Department of Agriculture

Introduction

Scientific integrity is at the forefront of the scientific research enterprise, more so than ever in recent years owing to the apparent increase in misconduct and misuse of research data. This paper provides a resource for the scientific community on the extensive work being done on scientific integrity and helps to identify areas in which more attention is needed. This work provides an overview of key existing efforts by federal agencies, foundations, nonprofit organizations, professional societies, and academia on their scientific integrity efforts from 1989 to April 2016, as described on their websites. A description of the linkages among the efforts is provided. A literature search in PubMed was not conducted for this paper. It is designed as a living document that will be updated online to include existing efforts that have not been captured and new work in the future.

Key work on scientific integrity

Federal agencies

Key work by federal agencies is included in Table 1.

Table 1.

Scientific integrity work by federal agencies.

Federal Agency	Type of Scientific Integrity Developed	Year Developed	Key Target Group	Website
OSTP	Published the federal policy on research misconduct in the Federal Register as a final, government-wide policy	2000	Federal agencies	https://www.federalregister.gov/articles/2000/12/06/00-30852/executive-office-of-the-president-federal-policy-on-research-misconduct-preamble-for-research
OSTP	Provided guidance to federal agencies to respond to the 2009 Presidential Memorandum on Scientific Integrity	2010	Federal agencies	https://www.whitehouse.gov/sites/default/files/microsites/ostp/scientific-integrity-memo-12172010.pdf
OSTP	Issued a scientific integrity report card for individual federal agencies on their progress	2013	Federal agencies	http://www.peer.org/assets/docs/SI_Report_Card_Comparison_Chart%20-%20Sorted%20by%20Score.pdf
NSF	Published the Research Misconduct regulation in the CFR	2002	NSF employees and grant awardees	https://www.gpo.gov/fdsys/granule/CFR-2012-title45-vol3/CFR-2012-title45-vol3-part689/content-detail.html
NSF	Released the Responsible Conduct of Research, requiring institutions to provide appropriate training/oversight in the responsible and ethical conduct of research to undergraduate students, graduate students, and postdoctoral fellows	2009	Research institutions applying for NSF grants	https://www.gpo.gov/fdsys/pkg/FR-2009-08-20/html/E9-19930.htm
NSF and NAS	Jointly published the third edition of On Being a Scientist: A Guide to Responsible Conduct in Research	2009	Graduate students and beginning researchers but applies to all scientists	http://www.nap.edu/catalog/12192/on-being-a-scientist-a-guide-to-responsible-conduct-in
NSF	Released draft Scientific Integrity Policy	2011	Civil service employees; visiting scientists, engineers, and educators; those working atNSF under the Intergovernmental Personnel Act; and political appointees	http://www.nsf.gov/bfa/dias/policy/si/sipolicy.pdf
NSF	Supported research grants on scientific integrity	1990s–present	Undergraduate and graduate students	http://digitalworlds.ufl.edu/projects/gap/
HHS	Policies and Principles for Assuring Scientific Integrity	2011	11 HHS operating divisions	http://www.hhs.gov/sites/default/files/open/pres-actions/scientifc-integrity-principles-12-19-11.pdf
HHS ORI	Commission on Research Integrity report, “Integrity and Misconduct in Research”	1995	Federal agency scientists and professional societies	http://ori.hhs.gov/historical-background
HHS ORI	Development of biennial Research Conferences on Research Integrity	2000	Research community	https://ori.hhs.gov/rri_conference
HHS ORI	Research on Research Integrity Program	2001	Research community	http://ori.hhs.gov/rri-program
HHS ORI	Rapid Response for Technical Assistance program	2001	Universities and institutions	http://ori.hhs.gov/rapid-response-technical-assistance
HHS ORI	RCR Resource Development Program	2002	Research community	https://ori.hhs.gov/rcr-resource-development-program
HHS ORI	RCR Program for Academic Societies, a collaboration with the Association of American Medical Colleges	2002	Academic societies and their members	https://ori.hhs.gov/program-academic-societies
HHS ORI	New regulation on PHS Policies on Research Misconduct (42 CFR Part 93)	2005	10 PHS offices and agencies	https://ori.hhs.gov/sites/default/files/42_cfr_parts_50_and_93_2005.pdf
HHS ORI	ORI Introduction to the Responsible Conduct of Research	2003, updated in 2007	Research community	http://ori.hhs.gov/ori-introduction-responsible-conduct-research
HHS ORI	RCR Program for Graduate Schools, a collaboration with the Council of Graduate Schools	2007	Graduate schools and students	http://cgsnet.org/scholarly-integrity-and-responsible-conduct-research-rcr?tabid=123
HHS ORI	RCR Program for Postdoctoral Students	2007	Graduate students and postdoctoral fellows	http://ori.hhs.gov/historical-background
HHS ORI	“The Lab: Avoiding Research Misconduct”	2011	Graduate students (movie available in English, Spanish, and Chinese)	http://ori.hhs.gov/thelab
HHS ORI	Posts case studies on research misconduct on website	Ongoing	Research community	https://ori.hhs.gov/case_summary
HHS NIH	NIH OER oversees the policy for the NIH peer review process using a two-tiered system	Ongoing	Peer reviewers	http://grants.nih.gov/grants/guide/notice-files/NOT-OD-11-120.html
HHS NIH	NIH Policies and Procedures for Promoting Scientific Integrity	2012	NIH employees and grantees	https://ombudsman.nih.gov/ScientificIntegritynov2012.pdf
HHS NIH	PubMed Commons	2013	Authors	http://www.ncbi.nlm.nih.gov/pubmedcommons/
HHS NIH	Joint workshop with Nature Publishing Group and AAAS on reproducibility and rigor of research findings	2014	Research community	http://www.nih.gov/research-training/rigor-reproducibility/principles-guidelines-reporting-preclinical-research
HHS NIH	Launched a new website to help guide reviewers through the peer review process	2015	Peer reviewers	https://grants.nih.gov/grants/policy/review.htm
HHS NIH	Statement on Integrity in Peer Review	2015	Research community	http://grants.nih.gov/grants/guide/notice-files/NOT-OD-15-106.html#sthash.XJY7n6c6.dpuf
HHS NIH	New guidelines on Enhancing Reproducibility Through Rigor and Transparency in Grant Applications	2015	NIH employees and grantees	http://grants.nih.gov/grants/guide/notice-files/NOT-OD-15-103.html
HHS FDA	Established the Office of Scientific Integrity	2009	FDA employees	http://www.fda.gov/AboutFDA/CentersOffices/OC/OfficeofScientificandMedicalPrograms/ucm197861.htm
HHS FDA	Issued 11 key principles of scientific integrity	2012	FDA employees	http://www.fda.gov/AboutFDA/ReportsManualsForms/StaffManualGuides/ucm289975.htm
HHS CDC	Guidance on Scientific Integrity	2016	CDC employees	https://www.cdc.gov/od/science/docs/cdcsiguide_042516.pdf
EPA	Principles of Scientific Integrity	1999	EPA employees	http://www.epa.gov/osa/epas-principles-scientific-integrity-fact-sheet
EPA	Scientific Integrity Policy	2012	EPA employees	http://www2.epa.gov/sites/production/files/2014-02/documents/scientific_integrity_policy_2012.pdf
EPA	Community of Practice for Statistics	2014	EPA employees	http://www.ncbi.nlm.nih.gov/pubmed/25795653
EPA	Peer Review Handbook, fourth edition	2015	EPA employees	http://www2.epa.gov/sites/production/files/2015-09/documents/final_epa_peer_review_handbook-_4th_ed_091415_dummy_link.pdf
USDA	Departmental Regulation 1074-001 on Scientific Integrity	2013	USDA employees and contractors	http://www.ocio.usda.gov/sites/default/files/docs/2012/DR%201074-001_0.pdf
USDA	Scientific Integrity Policy Handbook	2013	USDA employees and contractors	http://www.usda.gov/documents/usda-scientific-integrity-policy-handbook.pdf
USDA	Scientific Integrity Allegations Summary Report	Annually, beginning in 2013	USDA employees and contractors	http://www.usda.gov/documents/usda-updated-scientific-integrity-summary-report.pdf
USAID	Scientific Integrity Policy	2012	USAID staff and implementing partners	https://www.usaid.gov/sites/default/files/documents/15396/integrity.pdf
USAID	Scientific Research Policy	2014	USAID staff and implementing partners	https://www.usaid.gov/sites/default/files/documents/15396/USAID%20Scientific%20Research%20Policy%2012-3-14.pdf

CDC, Centers for Disease Control and Prevention; CFR, Code of Federal Regulations; EPA, US Environmental Protection Agency; FDA, US Food and Drug Administration; HHS, US Department of Health and Human Services; NAS, National Academy of Sciences; NIH, National Institutes of Health; NSF, National Science Foundation; OER, Office of Extramural Research; ORI, Office of Research Integrity; OSTP, Office of Science and Technology Policy; PHS, Public Health Service; RCR, Responsible Conduct Research; USDA, US Department of Agriculture; USAID,

In 2009, President Barack Obama issued a Presidential Memorandum on Scientific Integrity for the heads of executive branch departments and agencies to ensure “the highest level of integrity in all aspects of… scientific and technological processes” (White House Office of the Press Secretary, 2009). The memorandum articulated on “six principles central to the preservation and promotion of scientific integrity” (Holdren, 2010).

Office of Science and Technology Policy (OSTP)

OSTP issued an implementation guide for federal agencies to respond to the directive in 2010. The implementation guide provides guidance on four areas: foundations of scientific integrity in government, public communications, use of federal advisory committees, and professional development of government scientists and engineers (Holdren, 2010).

Individual federal agencies then began work to develop their own set of scientific integrity policies. In 2013, Public Employees for Environmental Responsibility (PEER) issued a scientific integrity report card for individual federal agencies. The report card provided a snapshot of progress that had been made across federal agencies in drafting and adopting scientific integrity policies, and PEER describes the report's aims as follows (PEER, 2013a):

This report card is meant to provide a basis for comparing the agencies' progress in shielding science from improper political manipulation through their scientific integrity policies… Policies were scored on three main areas of interest to PEER's work: (a) scientific misconduct regards whether the agency has prohibited political manipulation of science, has in place a defined process for investigating allegations of such misconduct, as well as protections for complainants and sanctions for those who have engaged in misconduct; (b) public communications of science regards generally how free agency scientists are to speak to peers and the public regarding their work; and (c) transparency of policy decision-making regards to what extent the public, external scientists, or legal challenger to a policy might become aware of any political manipulation of science.

Fifteen federal agencies had scientific integrity policies in place, and their report cards were compiled in a comparison chart (PEER, 2013b).

Even before the 2009 Presidential Memorandum, OSTP had been working on issues of scientific integrity. In 2000, the OSTP National Science and Technology Council published the federal policy on research misconduct in the Federal Register (FR), 65 FR 76260, as the final, government-wide policy that addressed research misconduct. The policy's purpose was to establish (1) uniformity among the federal agencies' definitions of research misconduct and (2) consistency in federal agencies' processes for responding to allegations of research misconduct. The OSTP policy covers both intramural research as well as extramural research (Executive Office of the President, 2000). Federal agencies have codified these definitions in their agency's regulations, the Code of Federal Regulations (CFR).

National Science Foundation (NSF)

NSF is an independent federal agency with an annual budget of $7.3 billion (for fiscal year 2015). NSF funds approximately 24% of all federally supported basic research conducted by America's colleges and universities (NSF, 2016).

In 2002, in response to OSTP 65 FR 76260, NSF published its regulation on Research Misconduct (45 CFR 689). NSF defines research misconduct as “…fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting research results. Fabrication means making up results and recording or reporting them. Falsification means manipulating research materials, equipment, or processes or changing or omitting data or results such that the research is not accurately represented in the research record. Plagiarism means the appropriation of another person's ideas, processes, results, or words without giving appropriate credit” (US Government Publishing Office, 2012).

In 2009, NSF published a paper on “Responsible Conduct of Research” in response to the 2007 America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science Act. The act requires “each institution that applies for financial assistance from NSF for science and engineering research or education describe in its grant proposal a plan to provide appropriate training and oversight in the responsible and ethical conduct of research to undergraduate students, graduate students, and postdoctoral researchers participating in the proposed research project” (NSF, 2009b). Since 2010, NSF has required that an institution's authorized organizational representative must certify that the institution has a plan to implement the training at the time of proposal submission (NSF, 2009b).

In 2009, NSF and the National Academy of Sciences (NAS) jointly funded the third edition of On Being a Scientist: A Guide to Responsible Conduct in Research (for more detail on the guide, see NAS, under Nonprofit Organizations) (NAS, 2009a).

In accordance with the 2009 Presidential Memorandum on Scientific Integrity, NSF released a draft Scientific Integrity Policy in 2011 for public comments, which states that “NSF's policy applies to civil service employees; visiting scientists, engineers, and educators; those working at NSF under the Intergovernmental Personnel Act; and political appointees” (NSF, 2011).

NSF has also supported research grants on scientific integrity, especially in the 1990s, when a large number of grants were funded in this area. More recently, NSF funded grants including “The Nature of Ethical Decision-Making in Research” at Western Michigan University (NSF, 2009a) and “Gaming Against Plagiarism” funded at the University of Florida (Digital Worlds Institute, 2016). “Gaming Against Plagiarism” includes an educational video game for students that provides an understanding and appreciation of the issues involved in ethical conduct related to intellectual property. In 2015, NSF began funding a new project at Purdue University titled “Collaborative Research: Foundations of Social and Ethical Responsibility Among Undergraduate Engineering Students: Comparing Across Time, Institutions, and Interventions” (NSF, 2014).

US Department of Health and Human Services (HHS)

HHS is a cabinet-level department of the US federal government and aims to protect the health of Americans (HHS, 2016a). HHS has 11 operating divisions, including eight Public Health Service (PHS) agencies and three human services agencies (HHS, 2016b). PHS is composed of the following offices and agencies: Office of Public Health and Science, National Institutes of Health (NIH), Centers for Disease Control and Prevention (CDC), US Food and Drug Administration (FDA), Substance Abuse and Mental Health Services Administration, Health Resources and Services Administration, Agency for Healthcare Research and Quality, Agency for Toxic Substances and Disease Registry, Indian Health Service, and Office of Regional Health Administrators. HHS issued Policies and Principles for Assuring Scientific Integrity in 2011 in response to the 2009 Presidential Memorandum (HHS, 2016c). Within HHS, several of the agencies also developed their own policies on scientific integrity, and efforts by the Office of Research Integrity (ORI), NIH, FDA, and CDC are described below.

Office of Research Integrity (ORI)

ORI oversees and directs PHS research integrity activities on behalf of the HHS Secretary, with the exception of the regulatory research integrity activities of FDA. ORI was established as an independent entity within HHS as part of the NIH Revitalization Act of 1993.

In its work on research integrity, ORI is focused on the research community and supports activities within professional societies. The NIH Revitalization Act of 1993 also “mandated that a Commission on Research Integrity be created to review the system for protecting against research misconduct. The Commission delivered its report, Integrity and Misconduct in Research, to the Secretary of HHS in November 1995. The Commission… made 33 recommendations including the development of a regulation on the protection of whistleblowers in research misconduct cases and the extension of the misconduct in science assurance to required institution to establish educational programs on the responsible conduct of research (RCR)” (ORI, 2016a).The Commission on Research Integrity recommended that “Professional societies [should] adopt a code of ethics in research… [and] should consider initiating activities that will further promote the ethical conduct of research” (AAAS, 2000).

In 2000, ORI held the first of several biennial Research Conferences on Research Integrity (ORI, 2016b) “to expand the knowledge base and develop a research community focused on the responsible conduct of research, research integrity, and research misconduct” (ORI, 2016a). In 2001, ORI began the Research on Research Integrity Program (HHS ORI, 2016f) and the Rapid Response for Technical Assistance Program “to provide technical assistance to any institution that is responding to an allegation of research misconduct” and “to facilitate high-quality and well-documented investigations and help resolve research misconduct cases promptly” (HHS ORI, 2016d).

In 2002, ORI developed the RCR Resource Development Program (HHS ORI, 2016e) and the RCR Program for Academic Societies (HHS ORI, 2016c) based on an ORI-commissioned report from the Health and Medicine Division (HMD) of NAS (formerly known as the Institute of Medicine [IOM]) (see NAS, under Nonprofit Organizations). According to ORI, “The former program was designed to facilitate the development of materials for teaching the responsible conduct of research by the research community for use in the research community. The latter program, a collaboration with the Association of American Medical Colleges (AAMC), supported activities within academic societies designed to promote the responsible conduct of research among their members. The first RCR Expo was held in 2003 to call attention to the new RCR materials” (ORI, 2016a). ORI also published the ORI Introduction to the Responsible Conduct of Research, last updated in 2007 (Steneck, 2007), and created the RCR Program for Graduate Schools in collaboration with the Council of Graduate Schools to institutionalize RCR education in graduate training (Council of Graduate Schools, 2016).

A new regulation, PHS Policies on Research Misconduct (42 CFR Part 93), became effective in 2005 (HHS, 2005). Based on this regulation, ORI developed a training program for institutional Research Integrity Officers in 2005, which led to the production of a 2006 orientation video and 2007 boot camps. Also in 2007, ORI began the RCR Program for Postdocs and developed a laboratory management training program in collaboration with the Laboratory Management Institute at the University of California–Davis (ORI, 2016a).

In 2011, ORI developed an interactive video on research misconduct titled “The Lab: Avoiding Research Misconduct,” which includes role playing by participants as they make decisions about research integrity that can lead to consequences for the character or others in the lab. The RCR simulation includes topics such as avoiding research misconduct, mentorship responsibilities, data handling, responsible authorship, and questionable research practices. “The Lab” is available in English, Spanish, and Chinese (HHS ORI, 2016b).

ORI investigates research misconduct and confirms multiple cases a year. ORI posts case studies of confirmed research misconduct in which administrative actions were enforced, and only case studies that currently have an administrative action imposed are posted. ORI does not post the names of individuals whose administrative action periods have expired (HHS ORI, 2016a).

National Institutes of Health (NIH)

NIH is the nation's medical research agency and is the largest single funder of biomedical research (NIH, 2016c). In November 2012, NIH developed the NIH Policies and Procedures for Promoting Scientific Integrity (NIH Office of the Director, 2012). The agency describes its efforts in this area as follows (NIH Office of the Director, 2012):

The NIH Office of Extramural Research (OER) develops and oversees the implementation of policy for the NIH peer review process. This two-tiered system involves initial peer review for scientific and technical merit and subsequent review by advisory councils or boards… in the 27 institutes and centers of NIH (ICs) that are considering applications for funding… Scientific integrity is the cornerstone of the NIH peer review process and is exemplified in its core values: (1) expert assessment; (2) transparency; (3) impartiality; (4) fairness; (5) confidentiality; (6) integrity; and (7) efficiency… To help ensure scientific integrity in the initial peer review process, OER has developed a policy (NIH, 2011) for managing conflict of interest, the appearance of conflict of interest, prejudice, bias, or predisposition.

In December 2013, NIH launched an online forum called PubMed Commons to enable authors to share opinions and information about scientific publications in PubMed (http://www.ncbi.nlm.nih.gov/pubmedcommons/). To be eligible to submit comments on PubMed Commons, an individual must have at least one publication in PubMed. According to PubMed Commons (2015), “About half the comments are on clinical or health-related publications. Members have been using PubMed Commons to: update and expand the public record, for instance by pointing to new data, relevant publications, or alternative interpretations; note corrections and retractions to publications; post discussion and critique, either directly or via links to blog posts and other platforms; provide links to datasets, code, or publicly accessible versions of publications; call attention to issues affecting reproducibility, such as cell line misidentification.”

In 2014, NIH held a joint workshop on reproducibility and rigor of research findings with Nature Publishing Group and Science, the journal of the American Association for the Advancement of Science (AAAS). Journal editors representing over 30 basic/preclinical science journals, representatives from funding agencies, and scientific leaders attended. The goal was to identify “the common opportunities in the scientific publishing arena to enhance rigor and further support research that is reproducible, robust, and transparent” (NIH, 2016b). Attendees reached consensus on a set of principles to facilitate these goals, including the following (NIH, 2016b):

(1) rigorous statistical analysis; (2) transparency in reporting; (3) data and material sharing; (4) consideration of refutations; (5) consider establishing best practice guidelines for: image-based data (image screening for manipulation, i.e., Western blots), description of biological material with enough information to uniquely identify the reagents (for example, unique accession number in repository), in particular for: antibodies: also report source, characteristics, dilutions and how they were validated; cell lines: also report source, authentication and mycoplasma contamination status; and animals: also report source, species, strain, sex, age, husbandry, inbred and strain characteristics of transgenic animals.

Since then, several journals have endorsed these principles.

NIH (2015b) states that it is “…fully committed to maintaining public trust in the NIH research enterprise. Attempts to influence the outcome of the peer review process through inappropriate or unethical means result in needless expenditure of government funds and resources, and erode public trust in science.” In 2015, NIH launched a new website to help guide reviewers through the peer review process (https://grants.nih.gov/grants/policy/review.htm). The site provides step-by-step instructions, policy notices, guidelines, videos, and critique templates (NIH OER, 2016a). The NIH also released its Statement on Integrity in Peer Review in 2015 (NIH, 2015b), which included, “the responsibilities of all participants, including individuals named on applications and officials of applicant organizations, in maintaining the integrity of the NIH peer review process and potential consequences for any unlawful or unethical attempt to influence the outcome of NIH peer review” (NIH, 2016a).

NIH also published new guidelines on Enhancing Reproducibility Through Rigor and Transparency in Grant Applications with the goal “to enhance reproducibility of research findings” (NIH, 2015a). The mandatory guidelines went into effect for new grant applications starting in January 2016. The revised application instructions “focus on four areas deemed important for enhancing rigor and transparency: the scientific premise of the proposed research, rigorous experimental design for robust and unbiased results, consideration of relevant biological variables, and authentication of key biological and/or chemical resources” (NIH OER, 2016b). NIH developed a training module for intramural postdoctoral fellows on these guidelines, which emphasized good experimental design (NIH OER, 2016b).

US Food and Drug Administration (FDA)

FDA is responsible for protecting the public health by assuring the safety, efficacy, and security of human and veterinary drugs, biological products, medical devices, the nation's food supply, cosmetics, and products that emit radiation (FDA, 2016b). In 2009, FDA established the Office of Scientific Integrity (OSI). OSI works to “ensure that FDA's policies and procedures are current and applied across the Agency, and resolve scientific disputes that may arise internally or externally, among other functions” (FDA, 2016a). In response to the 2009 Presidential Memorandum, FDA issued an agency-wide policy in 2012 that presents 11 key principles of scientific integrity (FDA, 2012).

Centers for Disease Control and Prevention (CDC)

CDC is the nation's health protection agency and works to protect Americans from health, safety, and security threats, both foreign and domestic (CDC, 2016a). The CDC Office of the Associate Director for Science “provides service and support to the CDC scientists” and is “home to the Office of Science Quality, which is responsible for advancing the quality of CDC's science and championing the translation of research through the development of science policies and best practices (e.g., authorship, scientific clearance, peer review, and extramural research policies); and the Office of Scientific Integrity, which ensures that CDC science and research activities comply with various federal laws, regulations, and policies…,” among other functions (CDC, 2016b). The Office of the Associate Director for Science released the latest version of CDC's Guidance on Scientific Integrity in April 2016 (CDC Office of the Associate Director for Science, 2016).

US Environmental Protection Agency (EPA)

The mission of EPA is to protect human health and the environment (EPA, 2016a). EPA believes that “scientific integrity results from adherence to professional values and practices, when conducting and applying the results of science and scholarship. It ensures: objectivity; clarity; reproducibility; and utility. Scientific integrity is important because it provides insulation from: bias; fabrication; falsification; plagiarism; outside interference; censorship; and inadequate procedural and information security” (EPA, 2016b).

EPA's Principles of Scientific Integrity were published in 1999 and laid out basic rules for ethical behavior by all agency employees in conducting scientific research, interpreting and presenting results, and using scientific information and data (EPA, 2016c).

In response to the 2009 Presidential Memorandum, EPA released a draft policy on scientific integrity for public comment. EPA (2014) stated that

All of the public comments were considered and, in combination with discussions with other Federal agencies, contributed to an improved final policy which was released in February 2012. The EPA Scientific Integrity Policy builds upon EPA's significant earlier scientific integrity efforts, focusing on the: (1) promotion of a culture of scientific integrity throughout the EPA; (2) release of scientific information to the public; (3) consistent use of peer review and federal advisory committees; and (4) professional development of government scientists.

EPA's scientific integrity official is the agency's focal point on scientific integrity and serves as the agency's expert on such matters. EPA's Scientific Integrity Committee implements the policy, and this committee consists of deputy scientific integrity officials representing each of the agency's program offices and regions (EPA, 2016b).

In 2014, EPA's recognition of the need for a greater statistical presence in science led to the creation of a Community of Practice for Statistics. It has the goal of strengthening statistics, considering study design, and ensuring that these same factors are evaluated during the review and approval of study protocols. Three working groups developed process and guidance documents to add value during the design phase for three types of research approaches: experimental, observational, and modeling (George et al., 2015).

In 1998, EPA developed the Peer Review Handbook to “provide guidance to EPA staff and managers who are planning and conducting peer reviews. It is intended to improve the internal management of EPA peer review by providing recommended procedures and approaches for EPA staff and managers” (EPA Science and Technology Policy Council, 2015a). The handbook was revised in 2000 (second edition), 2006 (third edition), and 2009 (with an addendum on the “Appearance of a Lack of Impartiality in External Peer Review”). The fourth edition in 2015 incorporated agency organizational changes, the 2009 addendum, and additional processes put in place after 2006 (EPA Science and Technology Policy Council, 2015b).

US Department of Agriculture (USDA)

USDA provides leadership on food, agriculture, natural resources, rural development, and nutrition. In 2013, USDA issued a Departmental Regulation (DR) on Scientific Integrity in response to the 2010 OSTP implementation guide. The DR provides guidance to all employees and contractors on “the proper use of scientific findings and the principles of conducting scientific activities” (USDA Office of the Chief Scientist, 2013). USDA then released its Scientific Integrity Policy Handbook in 2013 (USDA, 2013). USDA has agency scientific integrity officers in place at each of its agencies to ensure that policies are aligned with the USDA policy (USDA Office of the Chief Scientist, 2016) and issues a Scientific Integrity Allegations Summary Report on an annual basis (USDA, 2015).

US Agency for International Development (USAID)

The USAID is the lead US government agency that works to end extreme global poverty and enable resilient, democratic societies to realize their potential (USAID, 2016). In March 2012, USAID released its Scientific Integrity Policy (USAID, 2012), which states the following:

This document articulates the principles the Agency will follow to ensure the integrity of its scientific and scholarly activities, including how they are supported and carried out, and research findings are used and disseminated. These principles pertain to five specific aspects of the Agency's activities: (1) protecting the scientific process from misconduct and from inappropriate influence; (2) promoting access to scientific and technical information; (3) maintaining a highly skilled technical and scientific staff; (4) using federal advisory committees ethically and transparently; and (5) ensuring quality, methodological rigor, and ethical standards in all USAID-funded research activities. Future development of specific, implementation-focused policies and updates to existing policies in each of these areas will use the framework provided by this scientific integrity policy.

USAID released its Scientific Research Policy in December 2014, which consists of “operational policies of particular relevance to the design and management of research activities,” including quality standards for research plans and reports and open data. The policy also covers scientific peer review and publication (USAID, 2014).

Foundations

Table 2 presents details of scientific integrity by foundations.

Table 2.

Scientific integrity work by foundations.

Foundation	Type of Scientific Integrity Grant	Year Developed	Key Target Group	Website
Laura and John Arnold Foundation	$66,859,986 in grant dollars for research integrity projects (includes the grants below)	2011–2015	Organizations and academia conducting research integrity projects	http://www.arnoldfoundation.org/grants/
Laura and John Arnold Foundation	8 grants COS totaling $10,052,621	2013–2018	COS	http://www.arnoldfoundation.org/grants/
Laura and John Arnold Foundation	$100,000 grant to AAAS	2014–2016	AAAS	http://www.arnoldfoundation.org/grants/
Laura and John Arnold Foundation	$300,000 grant to The Center for Scientific Integrity	2015–2017	Center for Scientific Integrity	http://retractionwatch.com/2015/08/03/new-300000-grant-marks-the-fifth-anniversary-of-retraction-watch/
John Templeton Foundation	$2,109,856 grant to COS	2014–2015	COS	http://www.arnoldfoundation.org/grants/
John Templeton Foundation	$80,000 grant to the BITSS	2015	BITSS	http://www.prweb.com/releases/2015/05/prweb12727015.htm
John D. and Catherine T. MacArthur Foundation	$400,000 grant to the Center for Scientific Integrity	2014	Center for Scientific Integrity	http://retractionwatch.com/2014/12/15/retraction-watch-growing-thanks-400000-grant-macarthur-foundation/
Alfred P. Sloan Foundation	Digital Information Technology program	2011	Researchers, technologists, engineers, academic publishers, and university administrators	http://www.sloan.org/fileadmin/media/files/annual_reports/2013-Annual-Report.pdf
Alfred P. Sloan Foundation	$168,600 grant to COS	2014	COS	https://cos.io/about_sponsors/

AAAS, American Association for the Advancement of Science; BITSS, Berkeley Initiative for Transparency in the Social Sciences; COS, Center for Open Science.

Laura and John Arnold Foundation (LJAF)

Research integrity is one main area of focus for the LJAF. According to the foundation, “LJAF's Research Integrity initiative aims to improve the reliability and validity of scientific evidence across fields that inform governmental policy, philanthropic endeavors, and individual decision making” (LJAF, 2016b). LJAF supports organizations that are committed to improving the openness, transparency, and quality of research. Between 2011 and 2015, the foundation provided over $85 million in grants for research integrity projects. Between 2013 and 2018, the foundation is funding eight grants to the Center for Open Science (COS), totaling over $10 million (see COS, under Nonprofit Organizations). LJAF provided a $100,000 grant, between 2014 and 2016, to AAAS (see AAAS, under Professional Societies) to foster open, reliable, and rigorous scientific research by sponsoring three workshops on publication standards (LJAF, 2016a). LJAF has awarded another $300,000 grant to the Center for Scientific Integrity to continue work from 2015 to 2017 on the Center's Retraction Watch database and support other projects (see Center for Scientific Integrity, under Nonprofit Organizations) (Retraction Watch, 2015).

John Templeton Foundation

The John Templeton Foundation is a philanthropic organization whose vision includes a commitment to rigorous scientific research and related scholarship. The foundation provided over $2 million to COS for activities in 2014–2016 to increase scientific openness and integrity and to expand the features and connectivity of the Open Science Framework (OSF) (see COS, under Nonprofit Organizations) (Templeton Foundation, 2016).

In 2015, the foundation provided an $80,000 grant to the Berkeley Initiative for Transparency in the Social Sciences (BITSS) to establish the Leamer–Rosenthal Prizes for Open Social Science (see BITTS, under Academia) (PRWeb, 2015).

John D. and Catherine T. MacArthur Foundation

The John D. and Catherine T. MacArthur Foundation is one of the nation's largest independent foundations (MacArthur Foundation, 2016). In 2014, the MacArthur Foundation awarded the Center for Scientific Integrity a $400,000 grant over 2 years to help the Center operate and expand Retraction Watch. The goal of the grant is to create a comprehensive and freely available database of retractions, something that did not previously exist. This was a gap that deprived scholarly publishing of a critical mechanism for self-correction (see the Center for Scientific Integrity, under Nonprofit Organizations) (Retraction Watch, 2014).

Alfred P. Sloan Foundation

The Alfred P. Sloan Foundation is a private philanthropic organization that provides grants on various subjects, including digital information technology (Sloan Foundation, 2016). The program, launched in 2011, has the following aims (Sloan Foundation, 2013):

[T]o leverage developments in digital information technology to empower scientists, and enable new forms of data intensive research. The program works directly with researchers, technologists, engineers, academic publishers, and university administrators to speed the development and adoption of tools, standards, norms, and practices that will enable researchers to better communicate with one another and more effectively work with large datasets. The program also seeks to support and encourage the effective use of new datasets, data repositories, data dissemination, and computational techniques across the Sloan Foundations' other grant making programs.

In 2014, the Sloan Foundation awarded $168,600 grant to COS to connect the OSF with tools created by other open-source and open-science service providers. The funds support data management planning, preregistration of research designs, data archiving, data analysis, and journal management (see COS, under Nonprofit Organizations) (COS, 2016c).

Nonprofit organizations

Key work by nonprofit organizations is included in Table 3.

Table 3.

Scientific integrity work by nonprofit organizations.

Nonprofit Organization	Type of Scientific Integrity Developed	Year Developed	Key Target Group	Website
NAS	Report (“The Role and Activities of Scientific Societiesin Promoting Research Integrity”) recommending the development of activities/materials to improve integrity of research	1989	Scientific organizations representing the research community and scientific journals	http://www.aaas.org/sites/default/files/content_files/The%20Role%20and%20Activities%20of%20Scientific%20Societies%20in%20Promoting%20Research%20Integrity.pdf
NAS	Good Science and Responsible Scientists	1992	Scientific community	http://www.aaas.org/sites/default/files/content_files/The%20Role%20and%20Activities%20of%20Scientific%20Societies%20in%20Promoting%20Research%20Integrity.pdf
NAS	Responsible Science: Ensuring the Integrity of the Research Process, Volume I	1992	Research community	http://www.nap.edu/openbook.php?record_id=1864&page=R1
NAS	Integrity in Scientific Research: Creating an Environment That Promotes Responsible Conduct	2002	Research community	http://iom.nationalacademies.org/Reports/2002/Integrity-in-Scientific-Research-Creating-an-Environment-That-Promotes-Responsible-Conduct.aspx
NAS	On Being a Scientist: A Guide to Responsible Conduct in Research	2009	Research community	http://www.nap.edu/catalog/12192/on-being-a-scientist-a-guide-to-responsible-conduct-in
NAS	Workshop: “Conflict of Interest and Medical Innovation: Ensuring Integrity While Facilitating Innovation in Medical Research”	2014	Research community	http://www.nap.edu/catalog/18723/conflict-of-interest-and-medical-innovation-ensuring-integrity-while-facilitating
NAS	Workshop: “Does the Public Trust Science? Trust and Confidence at the Intersections of the Life Sciences and Society”	2015	Research community	http://nas-sites.org/publicinterfaces/roundtable/events/trust/,http://www.nap.edu/catalog/21798/trust-and-confidence-at-the-interfaces-of-the-life-sciences-and-society
NAS	Colloquia: “Reproducibility of Research: Issues and Proposed Remedies”	2017	Research community	http://www.nasonline.org/programs/sackler-colloquia/upcoming-colloquia/
COS	Open Science Framework	2014 (ongoing)	Research community	http://openscienceframework.org/
COS	COS communities: Publishing Initiatives, Metascience, Infrastructure, and Interest Groups	2014 (ongoing)	Research community	https://cos.io/communities/
COS	Workshop: “Creating Standards for Reproducible Research”	2014	Research community	https://bitssblog.wordpress.com/2014/11/06/creating-standards-for-reproducible-research-overview-of-cos-meeting/
Center for Scientific Integrity	Retraction Watch	2010	Research community	http://retractionwatch.com/

COS, Center for Open Science; NAS, National Academy of Sciences.

National Academy of Sciences (NAS)

NAS is a private, nonprofit organization of the country's leading researchers. Through its National Academies of Sciences, Engineering, and Medicine, the NAS provides objective, science-based advice on critical issues affecting the nation (NAS, 2016c). The NAS IOM issued a report in 1989, which recommended that “scientific organizations representing the research community should develop educational and training activities and materials to improve the integrity of research [and that] scientific journals should develop policies to promote responsible authorship practices, including procedures for responding to allegations or indications of misconduct in published research or reports submitted for publication” (AAAS, 2000).

The NAS Committee on Science, Engineering, and Public Policy convened a panel on scientific responsibility and the conduct of research and published Responsible Science: Ensuring the Integrity of the Research Process in 1992 (NAS, 1992). Also in 1992, AAAS issued a report titled Good Science and Responsible Scientists, which examined misconduct in science and the response of the scientific community. The report noted that “scientific societies serve as custodians of their disciplines' distinct knowledge, traditions, and professional norms. The… standards of proper research practices adopted by a scientific society embody the collective conscience of the discipline and are an expression of its ethical responsibilities” (AAAS, 2000). Among the report's findings was that scientific societies “play a major role in influencing the moral tone and ethical climate in which research is conducted” (AAAS, 2000).

In 2002, HHS ORI commissioned NAS to form a Committee on Assessing Integrity in Research Environments, which published its report Integrity in Scientific Research: Creating an Environment That Promotes Responsible Conduct. The report focused on “the research environment and attempts to define and describe those elements that allow and encourage unique individuals, regardless of their role in the research organization or their backgrounds on entry, to act with integrity” (NAS, 2002).

Several overarching conclusions emerged, including: (1) attention to issues of integrity in scientific research is very important to the public, scientists, the institutions in which they work, and the scientific enterprise itself; (2) no established measures for assessing integrity in the research environment exist; (3) promulgation of and adherence to policies and procedures are necessary, but they are not sufficient means to ensure the responsible conduct of research; (4) there is a lack of evidence to definitively support any one way to approach the problem of promoting and evaluating research integrity; (5) education in the responsible conduct of research is critical, but if it is not done appropriately and in a creative way, it is likely to be of only modest help and may be ineffective; and (6) institutional self-assessment is one promising approach to assessing and continually improving integrity in research.

As a result of the report, HHS ORI established two key programs in 2002, the RCR Resource Development Program and the RCR Program for Academic Societies (NAS, 2002) (see HHS ORI, under Federal Agencies).

Recognizing that the foundation of the scientific research enterprise is built on trust, both NAS and NSF funded a study that was the basis for the third edition of On Being a Scientist: A Guide to Responsible Conduct in Research in 2009, which was originally published in 1989 (NAS, 2009a). The guide (NAS, 2009a) aims to ensure that

…the scientific community devotes itself to exemplifying and transmitting the values associated with ethical scientific conduct… On Being a Scientist is designed to supplement the informal lessons in ethics provided by research supervisors and mentors… and …describes the ethical foundations of scientific practices and some of the personal and professional issues that researchers encounter in their work… The book is aimed primarily at graduate students and beginning researchers, but its lessons apply to all scientists at all stages of their scientific careers. A continuing feature of this Guide is the inclusion of a number of hypothetical scenarios offering guidance in thinking about and discussing these scenarios.

Also in 2009, NAS published the report Ensuring the Integrity, Accessibility, and Stewardship of Research Data in the Digital Age, which included three principles: (1) researchers are responsible for ensuring the integrity of their data, (2) data from published papers should be publicly accessible, and (3) data should be properly archived. The report offered 11 recommendations urging scientists, institutions, journals, and other players to develop standards and provide proper training (Kaiser, 2009; NAS, 2009b).

The 2014 HMD (formerly known as IOM) workshop on “Conflict of Interest and Medical Innovation: Ensuring Integrity While Facilitating Innovation in Medical Research” addressed the scientific integrity issues that arise with collaborative efforts among different stakeholders. According to Beachy et al. (2014), “While the potential benefits of collaboration are significant, the fact that the relationships among development partners are often financial means that it is vital to ensure trust by identifying, disclosing, and managing any potential sources of conflict that could create bias in the research being performed together.”

The National Research Council Roundtable on Public Interfaces of the Life Sciences hosted a workshop in 2015 titled “Does the Public Trust Science? Trust and Confidence at the Intersections of the Life Sciences and Society.” The event aimed to explore “the nuanced nature of trust in science,” including “what the elements of trust are, and how trust is built, maintained, or lost” (NAS, 2015a). The workshop summary was published in 2015 (NAS, 2015b).

NAS also developed the Arthur M. Sackler Colloquia, which addresses scientific topics of broad and current interest that cut across the boundaries of traditional disciplines (NAS, 2016a). The colloquia will present a conference on “Reproducibility of Research: Issues and Proposed Remedies” in Washington, DC, in 2017 (NAS, 2016b).

Center for Open Science (COS)

COS is a nonprofit technology startup founded in 2013 with the mission to “…increase openness, integrity, and reproducibility of scientific research” (COS, 2016a). COS pursues this mission by building communities around open science practices, supporting metascience research, and developing and maintaining free, open-source software tools. COS's efforts aim to inform best practices and serve as platforms to demonstrate reproducible research methods (COS, 2016a).

Although it is a relatively new organization, COS has become a major player in advancing scientific integrity and has secured funding from three major foundations (see LJAF, Templeton Foundation, and Sloan Foundation, under Foundations).

COS has a 2-year $2.1 million grant from the Templeton Foundation, which began in 2014, to build the infrastructure for the OSF. This is a web application that connects and supports research workflow, enabling scientists to increase the efficiency and effectiveness of their research (OSF, 2016). The organization describes the OSF as “a scholarly commons to connect the entire research cycle. Researchers can use the OSF to collaborate, document, archive, share, and register research projects, materials, and data. The grant allows COS to build communities around open science values. Scientific integrity can be improved with strategies that make the fundamental but abstract accuracy motive—getting it right—competitive with the more tangible and concrete incentive—getting it published.” Three major activities of the OSF are as follows: “(1) building the OSF to provide features that provide value to the scientist's existing workflow and enables or automates good practices, (2) building community, training and outreach to facilitate use of the OSF, and (3) connecting a variety of tools (e.g., data repositories, data visualization tools, analytic tools) through the OSF to support the entire research lifecycle and facilitate documentation and archival of research materials and data” (Templeton Foundation, 2016).

COS held a workshop in 2014 that was “aimed at creating standards to promote transparent and reproducible research in the social-behavioral sciences” (Christensen, 2016). Representatives came together from across the scientific disciplines, including funders, publishers, editors, and researchers from over 40 leading institutions (Christensen, 2016).

COS has received seven grants from LJAF (2016a), with the following goals: (1) to encourage preregistration of scientific studies by offering monetary awards to selected researchers who publish studies that have been preregistered on the OSF, (2) to support the creation of a new database of clinical trials, and (3) to set up four COS communities (Publishing Initiatives, Metascience, Infrastructure, and Interest Groups).

Projects within the Publishing Initiatives community include the following (COS, 2016b):

(1) COS Communities Badges to Acknowledge Open Practices: Badges incentivize open research by rewarding authors for three distinct practices: open data, open materials, and preregistration. As a visual reward, badges appear directly on publications… (2) Registered Reports offer journals an alternative structure to the current publishing format to promote transparency and reproducibility in scientific research. In this model, peer review occurs twice. Each study procedure and analysis plan are evaluated prior to data collection for in-principle acceptance; if accepted, the final manuscript is essentially guaranteed publication regardless of the reported outcome, with a second peer review to ensure the accepted methodology was conducted. This supports the publication of negative or neutral results instead of only positive results into the scientific literature.

Projects within the Metascience community include the following (COS, 2016b):

(1) Reproducibility Project–Psychology: The project is a collaborative community effort to replicate published psychology experiments from three journals. Replication teams follow a standard protocol to maximize consistency and quality across replications, and the accumulated … workflow are to be open for critical review on the OSF. (2) Reproducibility Project–Cancer Biology: The project is an initiative to conduct direct replications of 50 high-impact cancer biology studies. The project anticipates learning more about predictors of reproducibility…. (3) Collaborative Replications and Education Project: The project facilitates student research training through conducting replications. The community-led team composed a list of studies that could be replicated as part of research methods courses… Replication teams are encouraged to submit their results to an information commons for aggregation for potential publication…(4) Crowdsourcing a Dataset: …a method of data analysis in which multiple independent analysts investigate the same research question on the same data set in whatever manner they consider to be best. This first crowdsourcing project establishes a protocol for independent simultaneous analysis of a single dataset by multiple teams, and resolution of the variation in analytic strategies and effect estimates among them.

Projects within the Infrastructure community include open-source developers and the OSF.

Projects within the Interest Groups community include the following (COS, 2016b):

(1) Ambassadors: COS trains ambassadors to act as the local authority on COS, the OSF, and open science practices in their community…. (2) Open Science Collaboration is a network of researchers, professionals, citizen scientists, and others with an interest in open science, metascience, and good scientific practices. The goal is to promote open collaboration of scientific ideas(COS, 2016b).

Center for Scientific Integrity

The Center for Scientific Integrity's mission is “to promote transparency and integrity in science and scientific publishing, and to disseminate best practices and increase efficiency in science” (Retraction Watch, 2016). A major product of the Center is a database of retractions generated by the work of Retraction Watch (http://retractionwatch.com/), the first publicly available database to compile retracted scientific publications (Retraction Watch, 2016). Retraction Watch is funded mainly by a $400,000 grant from LJAF (see LJAF, under Foundations) (Retraction Watch, 2014).

Professional societies

Key work by professional societies is included in Table 4.

Table 4.

Scientific integrity work by professional societies.

Professional Societies	Type of Scientific Integrity Developed	Year Developed	Key Target Group	Website
COPE	Maintains database of research/publication misconduct cases discussed at the COPE Forum	1997	Research Community	http://publicationethics.org/cases
COPE	eLearning course	Ongoing development	COPE members who are editors and publishers	http://publicationethics.org/resources/e-learning
COPE	Code of Conduct and Best Practice Guidelines for Journal Editors	2011	COPE members	http://publicationethics.org/files/Code_of_conduct_for_journal_editors_Mar11.pdf
CSE	“Promoting Integrity in Scientific Journal Publications”	2006–present (ongoing)	Scientific editors	http://www.councilscienceeditors.org/resource-library/editorial-policies/white-paper-on-publication-ethics/
AAAS	5 VHS videos on ethical scientific research and a Discussion and Resource Guide	1996	Research community	http://www.aaas.org/page/integrity-scientific-research-video-series
AAAS	The Role andActivities of Scientific Societies in Promoting Research Integrity	2000	Scientific societies	http://www.aaas.org/sites/default/files/content_files/The%20Role%20and%20Activities%20of%20Scientific%20Societies%20in%20Promoting%20Research%20Integrity.pdf
AAAS	Joint workshop with Nature Publishing Group and NIH on reproducibility and rigor of research findings	2014	Research community	http://www.nih.gov/research-training/rigor-reproducibility/principles-guidelines-reporting-preclinical-research
FASEB	Set of recommendations on enhancing research reproducibility	2016	Research community and journalists	https://www.faseb.org/Portals/2/PDFs/opa/2016/FASEB_Enhancing%20Research%20Reproducibility.pdf
ASA	Statement on Maintaining Integrity of U.S. Presidential Appointments	2004	President of the United States	http://www.asanet.org/about-asa/how-asa-operates/council-statements/integrity-presidential-appointment-scientists
ASA	Statement on Academic Independence and Scientific Integrity	2006	Scientific community	http://www.asanet.org/about-asa/how-asa-operates/council-statements/academic-independence-and-scientific-integrity
ACS	Position Statement on Scientific Integrity in Public Policy	2014–2017	Research community	http://www.acs.org/content/acs/en/policy/publicpolicies/promote/scientificintegrity.html
ACS	“Catching Errors: Peer Review and Retractions in Publishing” webinar	2015	ACS members	http://www.acs.org/content/acs/en/acs-webinars/popular-chemistry/catching-errors.html

AAAS, American Association for the Advancement of Science; ACS, American Chemical Society; COPE, Committee on Publication Ethics; CSE, Council of Science Editors; FASEB, Federation of American Societies for Experimental Biology; NIH, National Institutes of Health; ASA, American Sociological Association .

Committee on Publication Ethics (COPE)

COPE was initiated by medical journal editors in the United Kingdom in 1997 but has expanded to over 10,000 editors of academic journals and persons interested in publication ethics worldwide. COPE “…provides advice to editors and publishers on all aspects of publication ethics and, in particular, how to handle cases of research and publication misconduct. It also provides a forum for its members to discuss individual cases. COPE does not investigate individual cases but encourages editors to ensure that cases are investigated by the appropriate authorities (usually a research institution or employer)” (COPE, 2016a). COPE maintains a database of all cases that have been discussed at the COPE Forum since 1997 as well as podcasts of the discussion, including the outcome of the cases and the advice given. The database now contains over 500 cases (COPE, 2016b).

COPE has produced an eLearning course that provides the tools and knowledge to enhance one's skills in publication ethics. The course has 11 modules, including 10 that are currently available to members: selective reporting; reviewer misconduct; redundant publication; plagiarism; introduction to publication ethics; falsification; fabrication; corrections, retractions and expressions of concern; conflict of interest; and authorship (COPE, 2016d).

COPE also funds research on publication ethics (COPE, 2016e), organizes annual seminars globally (COPE, 2016f), and has created an audit tool for members to measure compliance with its Code of Conduct and Best Practices Guidelines for Journal Editors, which all members are expected to follow (COPE, 2016c).

Council of Science Editors (CSE)

CSE is an international membership organization for editorial professionals publishing in the sciences (CSE, 2016). CSE works with other professional organizations to shape the scientific journal environment so that the integrity of publications is upheld.

Since 2006, CSE's Editorial Policy Committee has maintained a white paper titled “Promoting Integrity in Scientific Journal Publications”(CSE, 2012b). The most recent update of the white paper in 2012 includes “information on citation manipulation, publication planning by study sponsors, and ethical conduct of sponsors; reorganization of the section on reporting suspect manuscripts; updated information on international models for responding to research misconduct; and more recent examples of corrections, retractions, and expressions of concern” (CSE, 2012a). The white paper is a living document that is continuously updated through feedback received from members.

American Association for the Advancement of Science (AAAS)

AAAS is an international nonprofit organization dedicated to advancing science for the benefit of all people. It is the world's largest general scientific society and publishes the journal, Science (AAAS, 2016a).

One of the highest priorities for AAAS in the late 1990s and early 2000s focused on the role of scientific societies in promoting research integrity. In 2000, AAAS published a report summarizing proceedings from an April 2000 conference on “The Role and Activities of Scientific Societies in Promoting Research Integrity,” which was cosponsored by AAAS and ORI. The report reviewed some of the recent history of the perceived roles and activities of scientific societies in promoting ethical conduct, discusses codes of ethics and support activities, and concluded with some findings and recommendations for research and action related to the societies' roles in promoting research integrity (AAAS, 2000).

At the 31st Annual AAAS Forum on Science and Technology Policy in 2006, a panel titled “Protecting the Integrity of Science” explored the subtle dilemmas of science ethics and integrity, with presentations on “Attacks on Peer Review,” “Political Uses of Science,” and “Protecting the Integrity of Science: Scientific Misconduct” (AAAS, 2006).

AAAS was awarded a $100,000 grant from LJAF (which runs from 2014 to 2016) to “foster open, reliable, and rigorous scientific research by sponsoring [three] workshops on publication standards” (LJAF, 2016a) (see NIH, under Federal Agencies; and LJAF, under Foundations).

AAAS has developed a video series to “help improve the ability of scientists, post-doctoral fellows, undergraduate and graduate students, administrators, and technicians to develop informed and well-reasoned responses to ethical issues that arise in scientific research” (AAAS, 2016b). The series includes (AAAS, 2016b):

…five ‘trigger’ videos, short dramatizations aimed at provoking discussion on a series of ethical issues confronting various participants in the research process, and a Discussion and Resource Guide. Among the topics addressed in the videos are: role and responsibilities of mentors and lab chiefs; determination of authorship; allocation of credit; impact of legal rules on conduct of science; data retention, selection, sharing, and reporting; pressures in the research environment; sloppiness in research; scientific misconduct and institutional responses; whistle blowing; peer review; animals in research; intellectual property; commingling of private and public funds for research; privileged information; and responsibilities of collaborators.

Federation of American Societies for Experimental Biology (FASEB)

FASEB is the nation's largest coalition of biomedical researchers, representing 30 scientific societies and over 125,000 researchers from around the world (FASEB, 2016b). FASEB believes that maintaining public trust in medical research and preventing the introduction of bias is absolutely critical, and the organization supports efforts to preserve the integrity of science (FASEB, 2016c).

In January 2016, FASEB published a set of recommendations on enhancing research reproducibility, which resulted from FASEB's 2015 Science Policy Symposium and subsequent roundtable discussions. Participants agreed that “three general factors impede the ability to reproduce experimental results: lack of uniform definitions to describe the problem, insufficient reporting of key experimental details, and gaps in scientific training” (FASEB, 2016a). The recommendations included definitions for terms used in describing research, improvements in reporting of research materials and methods, and focus on “two key tools critical to basic research: mouse models and antibodies,” and advocacy for “robust training of researchers in rigorous experimental design” (FASEB, 2016a).

American Sociological Association (ASA)

ASA, founded in 1905, is a nonprofit membership association dedicated to advancing sociology as a scientific discipline and profession serving the public good. In 2004, ASA issued a statement on maintaining the integrity of U.S. Presidential Appointments of scientists, strongly urging the President of the United States to “consider scientific expertise as the primary basis for soliciting and nominating or appointing advisors to scientific, technological, and health-related posts or governmental advisory committees” (ASA, 2004). ASA believes that scientific expertise should play the dominant role (implicitly or explicitly) in the President's decision about whom to select for these positions. This will ensure that “scientific expertise is the primary consideration in such appointments, as a means to help protect the influence of America's critical scientific, technological, and health science enterprises” (ASA, 2004).

In 2006, ASA issued a statement on academic independence and scientific integrity. The statement affirmed ASA's “ongoing support for the protection of academic independence and the integrity of scientific research through the open movement of faculty and students between universities irrespective of nationality or political views. Similarly, ASA also strongly endorses the principle of scientific worthiness as the primary basis for assessing articles for scholarly publication, service on editorial boards of scholarly journals, and participation in scholarly conferences, not excluding persons on the basis of nationality or political views” (ASA, 2006).

American Chemical Society (ACS)

ACS is a scientific society that supports scientific inquiry in the field of chemistry (ACS, 2016a). ACS published a Position Statement on Scientific Integrity in Public Policy for 2014–2017. Within this statement, ACS “encourages scientific integrity policies that help the federal government obtain and integrate scientific assessments into policy development and implementation” (ACS, 2016c). The statement includes sections on recommendations for government, scientific processes and procedures, data quality use and review, and scientific access and advice (ACS, 2016c).

ACS's 2015 webinar titled “Catching Errors: Peer Review and Retractions in Publishing” included speakers from Retraction Watch, COPE, and the Center for Clinical Research Ethics (CCRE). The speakers discussed the efforts that are being made to combat issues in peer review as well as what could be changed to improve the review process (ACS, 2016b).

Academia

Key work by academia is included in Table 5.

Table 5.

Scientific integrity work by academia.

Academia	Type of Scientific Integrity Developed	Year Developed	Key Target Group	Website
ICTS	Course on the responsible conduct of research	2007	Students at Washington University in St. Louis and members from ICTS partner institutions	http://ethicsresearchcore.org/education/rcr/
CCRE	Professional Integrity program	2012	Investigators who have engaged in wrongdoing or unprofessional behavior	http://icts.wustl.edu/icts-researchers/education
CCRE	Library of Research Ethics Case Studies	Ongoing	Research community	http://ethicsresearchcore.org/education/case-studies/
CASBS	Group on Best Practices in Science	2013	Research community	https://bps.stanford.edu/?page_id=1977
CASBS	Best Practices in Science conference	2015	Scientists actively conducting research on scientific integrity	http://bps.stanford.edu/?page_id=4587
BITSS	Leamer-Rosenthal Prizes for Open Social Science	2015	Emerging researchers and leaders in education	https://bitssblog.wordpress.com/prizes/

BITSS, Berkeley Initiative for Transparency in the Social Sciences; CASBS, Center for Advanced Study in the Behavioral Sciences; CCRE, Center for Clinical Research Ethics; ICTS, Institute of Clinical and Translational Sciences.

Institute of Clinical And Translational Sciences (ICTS)

ICTS at Washington University in St. Louis was established in 2007 through funding from the NIH Clinical and Translational Science Award program, Washington University, and BJC HealthCare (ICTS, 2016e). According to its website, “ICTS and the Office of the Vice Chancellor for Research at Washington University jointly sponsor a course on RCR. Some of the topics covered in the course include: Conflicts of Interest; Research Misconduct; Mentor/Mentee Responsibilities and Relationships; Collaborative Research; Responsible Authorship and Publication; The Scientist as a Responsible Member of Society; and Data Management, Ownership, and Sharing. The course is designed to satisfy both the NIH and the NSF RCR training requirements” (ICTS, 2016f).

Center for Clinical Research Ethics (CCRE)

CCRE is a partnership between ICTS at Washington University in St. Louis and the Center for Health Care Ethics at Saint Louis University. CCRE “exists to provide education in the areas of clinical research ethics and RCR, to conduct research on ethics and integrity in research, and to provide consultations to investigators on ethical issues in clinical and translational research or the design of studies on ethical issues” (ICTS, 2016a).

CCRE sponsored a Professional Integrity program, which began in 2012. This is the first remediation education program that provides “intensive professional development education for investigators who have engaged in wrongdoing or unprofessional behavior” (ICTS, 2016b).

CCRE has also developed a Library of Research Ethics Case Studies (ICTS, 2016d). In addition, CCRE faculty have ongoing research and scholarly projects in a variety of areas, including the understanding and preventing wrongdoing in research and environmental factors predictive of misbehavior. CCRE faculty researchers have received grants from HHS ORI and NIH, among other agencies (ICTS, 2016c).

Center for Advanced Study in the Behavioral Sciences (CASBS)

CASBS at Stanford University was founded in 1954 (CASBS, 2016a). Since 2013, CASBS has hosted a group of scholars called the Group on Best Practices in Science. The group is a collaboration coordinated by Stanford University, Rutgers University, University of California–Davis, and many other institutions. The collaboration “represents one attempt to increase the validity and credibility of scientific research. It (1) documents research on how scientific practices may become compromised and (2) provides a platform to spur discussion about such issues” (CASBS, 2016b). The group is currently engaged in the following efforts (CASBS, 2016c):

(1) writing a memorandum for the White House about how scientific practice can become compromised and how the federal government can help improve scientific practices; (2) writing a grant proposal for the NSF; (3) developing archival studies assessing: the evidentiary value and prevalence of p-hacking, the association of sample size with journal impact factor, changes in sample size over time, the rate at which hypotheses are clearly stated and operationalized, and how well suggested reforms have succeeded at increasing the credibility and validity of research findings; and (4) developing questionnaire studies on: exploratory interviews to design questionnaires assessing scientists' beliefs about the prevalence of integrity-impairing practices in their discipline, etc.

CASBS hosted a “Best Practices in Science” conference in June 2015. The conference had two primary purposes: (1) to share ideas among those actively conducting research on scientific integrity with one another and the wider scholarly community and (2) to set the stage for the participants to generate articles on scientific integrity to be published in a special issue of a journal or edited book (CASBS, 2015).

Berkeley Initiative for Transparency in the Social Sciences (BITSS)

BITSS is “an international network of researchers and institutions committed to improving the standards of openness and integrity in economics, political science, psychology, and related disciplines” (BITSS, 2016c). BITSS further describes its efforts as follows (BITSS, 2016c):

BITSS is a program of the Center for Effective Global Action at the University of California, Berkeley. Central to BITSS efforts is the identification of useful tools and strategies for increasing transparency and reproducibility in research, including the use of study registries, pre-analysis plans, version control, data sharing platforms, disclosure standards, and replications. BITSS seeks to achieve its objectives by identifying practical resources and tools for increasing openness and reproducibility, in close partnership with data scientists and empirical researchers; and promoting the dissemination and adoption of effective approaches through training courses, grant competitions, and coordination with funders, government agencies, professional societies, and academic journals.

Furthermore, BITSS (2016a) states that “Today, researchers are not explicitly rewarded for disclosing their data collection and analysis methods, registering detailed pre-analysis plans, or making data and other research materials available to the public… In order to promote… transparent research, and to offer recognition and visibility to scholars practicing open social science, BITSS has launched two prizes named for pioneers who laid the foundation for transparency…,” and these efforts are supported by the Templeton Foundation (see Templeton Foundation, under Foundations). Beginning in 2015, the Leamer–Rosenthal Prizes for Open Social Science offered prizes for emerging researchers and leaders in education. Prizes for Emerging Researchers focus on “early-career researchers—junior faculty, postdoctoral researchers or graduate students—who adopt transparent research practices or pioneer new methods to increase the rigor of research” (BITSS, 2015). The Leaders in Education Prize “awards the work of professors who incorporate instruction in transparent practices in social science research into their curricula” (BITSS, 2015). The total available prize money is $60,000 for emerging researchers (in $10,000–$15,000 increments) and $20,000 for leaders in education (in $10,000 increments) (BITSS, 2016b).

Linkages of activities across entities working on scientific integrity

Table 6 illustrates where the five different sectors are focusing on scientific integrity issues and helps to identify areas in which more attention is needed.

Table 6

Current scientific integrity-related initiatives for the five sectors.

Type of Scientific Integrity Work	Federal Agencies	Foundations	Nonprofit Organizations	Professional Societies	Academia
Scientific integrity policies	13			1
Research on scientific integrity	2	1			2
Research misconduct	5	2	1	1	1
Training modules	7			2	2
Guidance handbooks	3		3
Workshops	2		4	1
Peer review	4
Reproducibility	2			1
Publication	1	1		3
Open science/transparency		4	2
Monetary prizes		1			1

Numerals represent the current number of activities on scientific integrity by a sector. The darker shaded spaces represent areas in which a sector is not currently engaged and helps to identify where more attention may be needed.

Federal agencies have focused on developing their individual scientific integrity policies in response to the 2009 Presidential Memorandum with guidance from the OSTP implementation guide. Within their policies and work, the agencies have concentrated on different aspects of scientific integrity, depending on their unique mission. Several agencies, including the USDA, CDC, EPA, and FDA, have created an office of scientific integrity that oversees the implementation of their scientific integrity policies. Some have created officials within these departments, including deputy scientific integrity officials at EPA and scientific integrity officers at USDA. NSF has formal CFR regulations on what defines research misconduct, whereas HHS ORI publishes cases of research misconduct and USDA issues an annual allegations summary report. NIH has focused on creating policies for integrity in the peer review process and establishing guidelines on enhancing reproducibility through rigor and transparency in grant applications. Peer review is also a focus for EPA, which developed the Peer Review Handbook. In addition, EPA has a program of work to address the need for a greater statistical presence in scientific research. HHS ORI and NSF have developed training for scientists on responsible conduct in research, such as HHS ORI's interactive video on research misconduct.

The development of training modules and educational tools is also an important aspect of addressing scientific integrity for other organizations and institutions. NAS has published reports on integrity in scientific research and developed a guide on responsible conduct in research to train scientists. COPE has an eLearning course that provides extensive training for journal editors and publishers on publication ethics. AAAS has a video series for scientists and students on ethical issues in scientific research. ICTS teaches a course on the responsible conduct of research, and CCRE produces a course that is designed to be a remediation course for those who have engaged in misconduct. Similar to HHS ORI and USDA, CCRE also has its Library of Research Ethics Case Studies that can be used as an educational tool.

Three successful efforts to create online forums for discussion of scientific integrity include NIH PubMed Commons, the Center for Scientific Integrity's Retraction Watch, and the OSF created by COS. PubMed Commons and Retraction Watch are public forums for discussion on the integrity of publications and the research they present, whereas the OSF is a scholarly commons to connect the entire research cycle and provides the tools to share protocols, raw data, and analysis. This commitment to open science is possible in part through the generosity of foundations cited in this paper, who are funding grants to nonprofit organizations and academia to build the necessary infrastructure and communities needed to drive open science. These funds also support the development of tools needed to effectively work within large data sets.

Like NIH, other groups have focused on issues of reproducibility. COS held a workshop on this topic that included 40 leading institutions. Recently, FASEB published a set of recommendations on enhancing research reproducibility. BITSS focuses on the identification of tools and strategies for increasing transparency and reproducibility. NAS will examine the issues and remedies for reproducibility of research at a future colloquium that is already scheduled.

Beyond NIH and EPA, peer review and publication ethics have been addressed by professional societies including COPE, CSE, and AAAS. ACS held a webinar for members that addressed issues of peer review and retractions.

In academia, CASBS hosts the Group on Best Practices in Science for researchers to convene and attempt to increase the validity and credibility of scientific integrity. This includes documenting research on how scientific practices may become compromised, which will be outlined in a memorandum to the White House. BITTS has created two monetary prizes that are awarded to researchers and educators who exemplify aspects of scientific integrity.

Overall, there is tremendous activity in the area of scientific integrity and there are clear linkages among the efforts of the federal agencies, foundations, nonprofit organizations, professional societies, and academia. Yet there continues to be a need to address the issue of scientific integrity despite the work that has already been done. The better the pressures that lead to misconduct are understood, the more effective efforts will be to instill scientific integrity in researchers. Scientific integrity needs to remain visible in the scientific community and evolve along with new research paradigms. All stakeholders must place high priority in instilling these values.

A comprehensive analysis of the details of the individual federal agency scientific integrity policies as well as the work done by foundations, nonprofit organizations, professional societies, and academia was not conducted for this publication. A second manuscript will be forthcoming with the goal to synthesize the policies of these federal agencies and organizations into a set of principles or best practices for scientific integrity.