Engaging actively with issues in the responsible conduct of science: lessons from international efforts are relevant for undergraduate education in the United States.

Numerous studies are demonstrating that engaging undergraduate students in original research can improve their achievement in the science, technology, engineering, and mathematics (STEM) fields and increase the likelihood that some of them will decide to pursue careers in these disciplines. Associated with this increased prominence of research in the undergraduate curriculum are greater expectations from funders, colleges, and universities that faculty mentors will help those students, along with their graduate students and postdoctoral fellows, develop an understanding and sense of personal and collective obligation for responsible conduct of science (RCS). This Feature describes an ongoing National Research Council (NRC) project and a recent report about educating faculty members in culturally diverse settings (Middle East/North Africa and Asia) to employ active-learning strategies to engage their students and colleagues deeply in issues related to RCS. The NRC report describes the first phase of this project, which took place in Aqaba and Amman, Jordan, in September 2012 and April 2013, respectively. Here we highlight the findings from that report and our subsequent experience with a similar interactive institute in Kuala Lumpur, Malaysia. Our work provides insights and perspectives for faculty members in the United States as they engage undergraduate and graduate students, as well as postdoctoral fellows, to help them better understand the intricacies of and connections among various components of RCS. Further, our experiences can provide insights for those who may wish to establish "train-the-trainer" programs at their home institutions.

BACKGROUND

An emerging body of scholarly literature (many of the papers published or summarized in this journal [e.g., Lopatto, 2004, 2007; Balster ; Prunuske ; Kabacoff ], publications from the Council on Undergraduate Research and others) is demonstrating the efficacy of research experiences for undergraduates in science. This strategy is effective in a variety of institutional settings (e.g., Kardash, 2000; Hathaway et al., 2002; Bauer and Bennett, 2003; Seymour [especially Table 1]; Hunter ; Lopatto, 2007, 2010; Russell ; Cejda and Hensel, 2009; Karukstis and Hensel, 2010; Shaffer ; Hensel, 2012) and in other disciplines as well (e.g., Sales and Folkman, 2000; Behling, 2009; Klos ). Engaging students early in undergraduate research has demonstrated benefits, especially in encouraging academic advancement and success for underrepresented students (multiple papers cited in Russell ; Boyd and Wesemann, 2009; National Research Council [NRC], 2011). Given the positive impact that undergraduate research has on students, there are several recommendations to increase significantly the number of science undergraduates who have had a research experience by the time they graduate, preferably starting in the early years (Boyer Commission on Educating Undergraduates in the Research University, 1998; NRC, 2003; American Society for Biochemistry and Molecular Biology, 2008).

As more undergraduates engage in scientific research, we also should consider how to provide them with training to conduct research responsibly. Engaging students in discussions about responsible conduct of science (RCS) and helping them make RCS principles integral to the fabric of their research will help them maintain the integrity of the research process, increasing the likelihood that their work will be undertaken with expectations of appropriate scientific conduct and of their collecting and reporting data accurately (e.g., National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2009).

Actively involving students in thinking about and practicing RCS throughout their research experiences has the added benefit of being in compliance with both the letter and the spirit of regulations that are mandated by funding agencies and by individual colleges and universities. For example, if you are currently the recipient of a research grant from the National Science Foundation (NSF), whether to conduct your own research or to train postsecondary students in research, you are required to provide opportunities for your students to become “trained” on issues related to the responsible and ethical conduct of science (e.g., Box 1). The National Institutes of Health (NIH) has similar requirements for all training grants, fellowship awards, research education grants, and so on. NIH defines RCS as “the practice of scientific investigation with integrity. It involves the awareness and application of established professional norms and ethical principles in the performance of all activities related to scientific research” (NIH, 2009). The same NIH bulletin suggests that the following topics be considered as part of any training program in RCS:

conflict of interest—personal, professional, and financial;

policies regarding human subjects, live vertebrate animal subjects in research, and safe laboratory practices;

mentor/mentee responsibilities and relationships;

collaborative research including collaborations with industry;

peer review;

data acquisition and laboratory tools; management, sharing and ownership;

research misconduct and policies for handling misconduct;

responsible authorship and publication;

the scientist as a responsible member of society, contemporary ethical issues in biomedical research, and the environmental and societal impacts of scientific research. (NIH, 2009)

Statutory Requirement: “The Director shall require that each institution that applies for financial assistance from the Foundation for science and engineering research or education describe in its grant proposal a plan to provide appropriate training and oversight in the responsible and ethical conduct of research to undergraduate students, graduate students, and postdoctoral researchers participating in the proposed research project.”

(Section 7009 of the America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science [COMPETES] Act. [http://www.nsf.gov/bfa/dias/policy/rcr.jsp].

It should be noted that terminology differs greatly in this realm, with “responsible conduct of science,” “responsible conduct of research,” “research integrity,” “scientific integrity,” and “research ethics” often being used interchangeably in different bodies of literature. For this Feature, we have adopted “responsible conduct of science” to refer to all of the aforementioned components of appropriate behavior of practicing and future scientists. This term is also consistent with terminology that has evolved in several reports from the National Academies (reviewed in NRC, 2013).

There is considerable variability in how students in undergraduate research programs may be introduced to aspects of RCS. While some programs may extend face-to-face mentoring on RCS to undergraduates, many students and professional scientists meet the requirements for training through online tutorials or other sessions that involve passive learning, using this strategy to complete and “check off” this requirement for receiving and using federal funds. At the other end of the continuum, Junge report that undergraduate research students at Emory University spend some 15 contact hours during the orientation program participating in activities such as role-playing and discussions of case studies on a number of RCS issues. Students then spend time during weekly small-group meetings discussing these issues and asking how important RCS questions such as authorship are addressed for their research projects.

Despite inclusion of aspects of RCS in training for doing research, students often self-report that they have not gained much from the training they receive about these issues. For example, Lopatto (2003) noted that when asked to rate their undergraduate summer research experiences, 1135 undergraduate students from 41 colleges and universities rated training to “learn ethical conduct” the lowest of all of the potential benefits of their work out of 20 categories surveyed. This rating was consistent for respondents overall, including students whose research was funded by the Howard Hughes Medical Institute, students who decided to pursue graduate education in science, and those who elected not to do so as a result of their research experiences (see Table 1).

Table 1.

Mean responses to 20 gains from the undergraduate research experiencea

Item	Overall means	Means of HHMI-funded respondents	Respondents who changed to graduate education in science	Respondents who changed away from graduate education in science
Understanding of the research process	4.13	4.20	4.13	4.14
Readiness for more demanding research	4.03	4.07	4.18	3.29
Understanding how scientists work on real problems	4.00	4.10	4.20	3.92
Learning lab techniques	4.00	4.21	4.28	4.00
Tolerance for obstacles	3.99	4.10	4.18	3.67
Learning to work independently	3.85	3.97	4.38	3.56
Skill in the interpretation of results	3.83	3.91	4.33	3.65
Ability to analyze data	3.82	3.89	4.22	3.44
Understanding how knowledge is constructed	3.79	3.91	4.05	3.38
Becoming part of the learning community	3.78	3.90	4.35	3.56
Ability to integrate theory and practice	3.78	3.85	4.13	3.58
Understanding primary literature	3.68	3.83	3.87	3.69
Assertions require supporting evidence	3.67	3.79	4.08	3.65
Understanding science	3.63	3.76	4.03	3.69
Understanding how scientists think	3.62	3.71	3.95	3.27
Self-confidence	3.50	3.59	4.03	3.23
Clarification of a career path	3.42	3.42	3.98	3.76
Skill in oral presentation	3.42	3.49	3.81	3.19
Skill in science writing	3.32	3.38	3.75	3.00
Learning ethical conduct	3.15	3.27	3.25	3.02

aResponses were on a scale of 1 (no gain) to 5 (very large gain). Reconstructed from Table 5 of Lopatto (2004, p. 273).

In a later study with a different cohort of students, Lopatto (2007, p. 301) reported that students who were provided with seminars and other experiences on ethics as part of their research experience demonstrated greater learning gains in that area. However, these students also rated their “instruction and discussion in ethics” as the second lowest of the components they experienced, with only “seminars on safety in the laboratory” rated lower.

In response to the growing number of undergraduate researchers at the University of Wisconsin–Madison, one group designed an Entering Research (ER) course that helps students navigate their first research experience. The course meets 1 h each week and uses active learning to introduce topics such as research skills, communication, and professional development to participating undergraduate researchers. Part of the course engages students in discussions about RCS, particularly wrongdoing in science. Students self-reported that the sessions on science ethics were helpful and that after the course they could more easily identify scientific wrongdoing. Although the ER course included only one RCS topic, the approach of using active learning to deliver that topic was effective (Balster ).

Under the aegis of the NRC's Board on Life Sciences, the U.S. Department of State has supported a series of workshops (NRC, 2009, 2012b) and institutes (NRC, 2013) to develop “train-the-trainer” models for educating faculty in other nations to engage colleagues and students in issues associated with “dual-use” research specifically and RCS more broadly. These components of the institutes in Jordan and Malaysia are described below and in detail in the NRC report (2013; see also Figure 1). These institutes could serve as templates for methods to engage students and faculty colleagues in the United States with these concepts and issues, particularly because these workshops have used best practices in pedagogy to deliver RCS content. These approaches could be adapted by individual faculty mentors on a broader institutional scale for similar train-the-trainer programs.

Figure 1.

Cover images of the NRC report and stand-alone English/Arabic summary from the education institute held in Aqaba, Jordan.

These RCS institutes have been modeled on the National Academies Summer Institutes for Undergraduate Education (SI), which, since 2004, have engaged some 800 faculty and university academic officials from more than 200 U.S. colleges and universities in weeklong convenings at which they learn about research on human learning, practice active-learning pedagogies, develop authentic assessments that are tied to learning goals and objectives, and consider ways to create an inclusive learning environment (Wood and Gentile, 2003; Wood and Handelsman, 2004; Pfund ; Labov and Young, 2013). The U.S.-based NRC committee members and staff who organized and participated as instructors and facilitators for the RCS content of the institutes, working with other committee members and staff who have been associated with the SIs, integrated active learning into many sessions, including authorship, plagiarism, data fabrication and falsification, and dual-use issues. Although most of the RCS content experts had not attended an SI, they immediately saw the relevance of the SI model for engaging students and colleagues in their own disciplines and in discussions of RCS in research settings in the United States.

Accordingly, in this Feature, we provide a brief overview of these education institutes; the committee's perspectives for broadening their reach to other parts of the world, as well as within the United States; and some preliminary ideas for research to measure the effectiveness of such approaches in helping students learn about and embrace those concepts that fall under the umbrella of RCS. Additional details can be found in NRC (2013) and the separate executive summary (available in English and Arabic at http://dels.nas.edu/Materials/Special-Products/MENA?bname).

OVERVIEW OF AND INSIGHTS FROM THE JORDAN INSTITUTE

The first RCS institute was held in Aqaba, Jordan, in September 2012 for scientists from the Middle East/North Africa region. Planning for the Jordan institute was informed by collaboration with colleagues from the Great Library of Alexandria, Egypt. Invited participants to the Jordan institute included individuals and teams of faculty and graduate students from Algeria, Egypt, Jordan, Libya, and Yemen. The outcomes and lessons learned from this institute (detailed in NRC, 2013, and summarized below) informed the organization of a second institute that was held in August 2013 in Kuala Lumpur, Malaysia, for faculty and students from that nation, India, and Pakistan (organized in cooperation with the Academy of Sciences Malaysia).

The institutes each lasted 5.5 d. Participants were engaged in the institutes’ activities during the mornings and afternoons (∼50 h total). These events offered a combination of sessions that focused on various components of the content of RCS, including dual-use research of concern, as well as on best practices in pedagogy, using full integration of active-learning techniques in all sessions. Sessions that focused on pedagogy stressed the evidence base for how people learn and the implications of that research for teaching (NRC, 1999, 2000), including aligning assessments (both formative and summative) with learning goals and objectives through the process of backward design (e.g., Wiggins and McTighe, 2005; Handelsman et al., 2006; Dirks ) and techniques for engaging students in active learning (Mazur, 1997; Sadler, 1989; Handelsman et al., 2006; Stiggins ).

Sessions that dealt with RCS at the Jordan institute focused on three themes illustrated by a series of case studies.

Development of Professionalism in Science

Discussion centered on the roles of government regulations and institutional policies as they relate to research misconduct and more generally about being a science professional. The case selected addressed a number of topics clearly related to professionalism, such as data selection, financial disclosure, ethical issues related to research involving children, and interactions with the press.

Conducting Research Responsibly

The topics in this session included the use and protection of human subjects; use, care, and treatment of laboratory animals; conflict of interest; and data collection and management. These issues were introduced through a case study that focused on the appropriateness of introducing viruses in the field and one on the U.S. Public Health Service syphilis studies in Guatemala. The virus case concerns a hypothetical proposal to test a live vaccine on a population of chimpanzees living on an island. The syphilis case focuses on ethical standards of research with human subjects and the harm that research on infectious diseases may cause when research participants’ interests are made secondary to scientific goals.

Being Part of the Responsible Scientific Community

Discussions emphasized collaborative research, authorship and publication, and peer review using case studies. The first case focused on the “Darsee affair,” a medical researcher at Harvard University who was discovered to have fabricated and falsified data in several publications (additional details in NRC, 2013, p. 55). Thus, this case considered the roles and responsibilities of coauthors of scientific publications, as well as who is entitled to be a coauthor of a peer-reviewed paper, along with the responsibilities of coauthors when violations of the code of ethics, such as data fabrication or falsification, are uncovered. Additional discussion under this theme considered mentor and trainee responsibilities; research with dual-use potential; and biosafety concerns in research, through examination of the controversy concerning whether scientific journals should have published papers in their entirety (or at all) about research that reveals how the H5N1 avian influenza virus could be modified to efficiently transmit among mammals.

Committee members with expertise in RCS “content” and active-learning pedagogies worked in teams to organize each of the sessions. Thus, sessions focusing on active learning and assessment made specific connections with the content topics that had been or would be considered. For example, when assessment was discussed, the group engaged in discussions about how to write high-quality clicker questions around RCS. Content-focused sessions used an array of active-learning techniques throughout. Some of these techniques included the use of iClickers, think–pair–share, role playing, jigsaw discussions, concept mapping, poster sessions, and whole-group discussions.

As in the SIs, during most afternoons, participants worked in small groups with trained facilitators (committee members and others with appropriate expertise). The goal of these sessions was to develop teachable units, to generate an array of teaching and learning activities and assessments based on the aforementioned themes that participants could implement at their own institutions. Each group of participants presented these units to the rest of the workshop presentations were followed by constructive evaluation by peers and the committee members. Part of the final morning of the institute was devoted to discussions by participants from the various nations about how they might use what they had learned to catalyze change in their higher education systems.

Results from a survey that was distributed to Jordan institute participants several weeks after the event showed that all aspects of the institute were rated good to excellent (Figures 2 and 3). Participants indicated that the top two reasons for attending the institute were to “become more involved with future efforts to improve education about the responsible conduct of research in my country” and “to discover tools, resources and best practices for incorporating evidence-based teaching techniques into my courses.” However, the feedback from this survey also told the committee that 1) the scope of topics and issues around RCS was too great, since many participants were unfamiliar with the concepts; 2) many participants were also unfamiliar with active-learning pedagogies; and 3) there were issues stemming from the fact that some participants spoke English as a second language.

Figure 2.

Participants’ ratings of the institute's sessions, delivery of workshop material, and group sessions. Participants’ ratings of different aspects of the institute, as defined: (A) Quality of sessions about the responsible conduct of science. (B) Quality of sessions about the scientific basis for the use of active-learning techniques. (C) Inclusion of information and perspectives from a diverse range of views. (D) Amount of time devoted to discussions during plenary sessions. (E) Balance of time spent in whole-group and team break-out sessions. (F) Helpfulness of your break-out group's facilitators. From data compiled by the committee. Source: NRC (2013, Figure 6-4, p. 76).

Figure 3.

Participants’ ratings of the institute's goals, instructional materials, and relevance to their careers. Participants’ ratings of different aspects of the institute, as defined: (A) Clarity of institute's goals and objectives. (B) Relevance of topics that were presented in relation to the stated goals of the institute. (C) Usefulness of resources provided by the organizers and presenters (e.g., background resources posted in a Dropbox and briefing book). (D) Value of the institute as a learning or professional development experience. (E) Relevance to you and your work of the issues presented. (F) Time to meet and interact with other participants. From data compiled by the committee. Source: NRC (2013, Figure 6-5, p. 77).

The committee also developed and administered pre- and postinstitute assessments (which participants completed while in Jordan) to try to measure learning gains in both content knowledge and understanding of active-learning techniques. As detailed in Chapter 6 of the report on the institute (NRC, 2013), the results were ambiguous in part because of language limitations and the ways in which participants understood various technical terms.

ADJUSTMENTS FOR THE MALAYSIA INSTITUTE

Feedback from the pre- and postassessments and the postinstitute survey, coupled with committee members’ own reflections about what worked well and what did not work during the Jordan institute, led to some restructuring for the Malaysia institute. The committee maintained the three major themes from the Jordan institute, but made changes as detailed in the following sections.

Content

The committee and staff decided to present fewer cases. In Malaysia we based our discussions not on the RCS cases, but on the issues raised by the Korean stem cell case (based on the 2004–2005 infractions committed in the process of creating human embryonic stem cells) and the case of H5N1 discussed above. The goals for this change were to 1) reduce the breadth of knowledge required concerning multiple cases, 2) showcase that irresponsible practices are interrelated and can have international repercussions, and 3) reinforce the pedagogical approach to help learners develop conceptual frameworks and transfer knowledge and expertise from one subject domain to another (e.g., NRC, 2000).

Added to the Malaysia institute was a movie night, when all participants, facilitators, and committee members watched the film Contagion one evening after dinner (popcorn was provided). The following morning's session was devoted to a discussion of the scientific merits and problems of the film. In addition, a role-playing exercise was held in which groups of participants assumed the roles of representatives of four different organizations represented in the film (World Health Organization, Centers for Disease Control, the media, and the public); each group advised the other groups about the needs of the sectors they represented and how other sectors could have been more helpful.

Pedagogy

The committee increased its efforts to integrate and model active-learning pedagogies in the sessions that focused on aspects of RCS.

Implications

The potential for these kinds of activities to help scientists internationally is clear. The participants at both the Jordan and Malaysia institutes stressed the importance of becoming more aware of international standards and expectations for issues of RCS, so they could more readily become collaborators in the international scientific community that adheres to established norms for authorship, plagiarism, and dealing with human and animal subjects, for example. They also recognized the importance of being able to share ideas with others and in working together to try to influence changes to policies that impacted both science and education in their countries.

However, as noted earlier, the members of the organizing committee who are experts in the content of these institutes, and who must think about complying every day with government rules and regulations for training young scientists in RCS, also immediately recognized the value that an active-learning approach would have if implemented in U.S. institutions. The committee members commented frequently that few RCS programs in the United States, especially those aimed at undergraduates, utilize the kind of integrated approaches that had been developed for these international audiences.

We envision several ways in which these active approaches to teaching and learning RCS might be adapted to undergraduate education:

Issues related to RCS could be used in both introductory and upper-level courses to provide a conceptual framework for scientific topics to be discussed in class. A class session might begin with consideration of some RCS issues for which there are not clear answers. Conversations that begin with issues that are contentious or encourage a variety of responses can be opened by projecting a question with responses that students could select using clickers or other response systems. The array of responses could prompt questions such as “What biology/science do we need to know to begin to address these questions in more informed ways?” Or “How have these issues led to the development of the stringent procedures we have adopted for our research space?”

Alternatively, asking students for examples of what they have seen or consider to represent wrongdoing can lead to an informative discussion. This will help to unpack misunderstandings because it will identify what individuals consider to be wrong. Others might not share the same view. This is more open than the suggestion above.

Participants at the institutes we ran spent concentrated time together that is unlikely to be found in many undergraduate classes. However, honors or upper-level seminars could allow students to engage more deeply with these issues during the seminar itself, with case studies built around the experimental work that is the focus of the seminar.

Campus-wide or department-wide book assignments could focus on aspects of RCS with guest scholars, videos, and other activities to supplement discussion of the book selected. Issues of research integrity are relevant to intellectual standards across all disciplines.

Based on the committee's experience from these two institutes, bringing together RCS content experts with those whose scholarly work focuses on pedagogy can be a winning combination to develop teaching/learning modules, courses, seminars, and other activities that integrate content and pedagogy from the outset (e.g., NRC, 2012a).

Faculty might develop hypothetical cases focused on the behavior of undergraduates in the lab and/or any recent incidents in the local or national news, two strategies that help generate immediacy for that audience.

Research has demonstrated the efficacy of both active learning and early research experiences as possible strategies to spark students’ interest in STEM subjects and have identified the converse, namely, uninspiring traditional teaching methods, as reasons why students leave science (Seymour and Hewitt, 1997; Handelsman ; NRC, 2012b). Experience from the SIs and other professional development opportunities for faculty in the United States (described in Hilborn, 2013) indicate that faculty who receive sufficient pedagogical training using active learning prefer this strategy rather than using more passive forms of pedagogy, even though it may be difficult to maintain (e.g., Ebert-May ). One can anticipate that both strategies (early research opportunities, active learning) will become increasingly common on U.S. campuses, generating both the need and the means for including RCS training in the undergraduate curriculum. However, an increasing number of students in science at U.S. universities are from other nations, and these students may have had very different prior educational experiences. Research is needed to determine whether active learning has the same impact in helping both foreign students who are studying in the U.S. and foreign students who remain in their home countries, but who also must be prepared in RCS to enter the international science community. Both U.S. and international students need opportunities to understand these concepts deeply, and to retain and internalize them, to be successful citizens and contributors to the scientific enterprise.