Incorporating Scientific Publishing into an Undergraduate Neuroscience Course: A Case Study Using IMPULSE.

The journal IMPULSE offers undergraduates worldwide the opportunity to publish research and serve as peer reviewers for the submissions of others. Undergraduate faculty have recognized the journal's value in engaging students working in their labs in the publication process. However, integration of scientific publication into an undergraduate laboratory classroom setting has been lacking. We report here on a course at Ursinus College where 20 students taking Molecular Neurobiology were required to submit manuscripts to IMPULSE. The syllabus allowed for the laboratory research to coincide with the background research and writing of the manuscript. Students completed their projects on the impact of drugs on the Daphnia magna nervous system while producing manuscripts ready for submission by week 7 of the course. Findings from a survey completed by the students and perceptions of the faculty member teaching the course indicated that students spent much more time writing, were more focused on completing the assays, completed the assays with larger data sets, were more engaged in learning the scientific concepts and were more thorough with their revisions of the paper knowing that it might be published. Further, the professor found she was more thorough in critiquing students' papers knowing they would be externally reviewed. Incorporating journal submission into the course stimulated an in depth writing experience and allowed for a deeper exploration of the topic than students would have experienced otherwise. This case study provides evidence that IMPULSE can be successfully used as a means of incorporating scientific publication into an undergraduate laboratory science course. This approach to teaching undergraduate neuroscience allows for a larger number of students to have hands-on research and scientific publishing experience than would be possible with the current model of a few students in a faculty member's laboratory. This report illustrates that IMPULSE can be incorporated as an integral part of an academic curriculum with positive outcomes on student engagement and performance.

Since the Boyer Commission Report (Boyer Commission, 1998) first appeared there has been steady pressure on undergraduate educational institutions to improve the level of student engagement in learning, particularly in the STEM (science, technology, engineering, and math) disciplines. Before that, as Katkin (2003) wrote, the research institutions’ responses were “for the most part slow, scattered, and largely at the margins.” Primarily Undergraduate Institutions, so-called PUIs, have done a better job of connecting student learning to practical experience, but all institutions, from community colleges to research I universities, are struggling with the economic limitations of trying to provide hands-on experiences in the sciences.

An important part of research training is teaching how to write, submit, and revise an article. Most graduate programs include courses both on scientific writing and article review as part of student training; undergraduates also need these opportunities. In recent years there has been an explosion in undergraduate journals, as documented by Tatalovic (2008), reflecting the growing interest by students and their mentors to create those opportunities. There has also been a surge in articles on how to teach scientific writing (Goldbort, 2006; Reynolds, et al., 2009), and specifically using peer reviewing (Woodget, 2003; Prichard, 2005; Blair et al., 2007; Hartberg et al., 2008), indicating the pressure to provide that training as well. However, the integration of these two types of training, how to do research and how to publish and write, and the impact that writing and publishing might have on the research experience, has not, we believe, been addressed. Earlier work commented on this lacuna in the literature (Lopatto, 2004), although there are many articles on how research itself promotes understanding (e.g., Healey, 2005; Baxter Magolda, 2009). The current study examines that specific question, looking particularly at the students’ experience of primary research when publishing is a stated goal.

In 2003, a group of undergraduates decided to start an online, international journal, IMPULSE, expressly for undergraduates (Jones et al., 2003). The intent was to help students bypass often stagnant, science-classroom learning and encourage them to engage in the full scientific process, from project to publication, as well as participating in the review process (Jones et al., 2006; Jones et al., 2009). While the last eight years have seen a slow recognition of this opportunity, as seen by references to IMPULSE in literature on students publishing (Ruszkiewicz et al., 2006; Traywick, 2010) it has been mostly individual students who see the possibilities and join the review team or submit manuscripts. However, a few faculty have begun to envision ways of integrating publication into their curriculum. In one case, the possibility of publication was a reward for the best paper produced in a class. An instructor from Lake Forest College chose a paper from others in the class and invited that student to submit his manuscript for publication (Paul, 2006). There has not, however, been an examination of how the publishing opportunity and subsequent experience affects the research learning.

The current report outlines a novel means of involving all of the students in a class in authentic, primary research that leads to writing and submitting a manuscript for potential publication. The protocol described allowed the students not only to draft a manuscript, but also to submit it for review and receive the referees’ comments, and revise the manuscript for further consideration. To our knowledge this is the first example of an undergraduate course providing the experience for all students enrolled in the class of taking a project from initial conception, through experimental design and research, to completion of a manuscript for submission, review, revision, and publication.

COURSE OVERVIEW

Course syllabus

In Spring 2009, one of us (RK) taught an undergraduate Molecular Neurobiology course at Ursinus College with submission of a manuscript to the journal, IMPULSE, incorporated into the syllabus. There were 20 students in the course, 16 women and four men, and all were juniors or seniors. Students were majors in Biology, Neuroscience, or Biochemistry and Molecular Biology. The prerequisites for the course included Ecology and Evolution, Cell Biology, and Genetics. The timeline for this trial, shown in Figure 1, involved planning a syllabus so that students were multitasking aspects of the research, such as designing the experiments, while also preparing the manuscript by doing the background literature research. The laboratory section of Molecular Neurobiology met once per week for three contact hours. In addition, the lecture section met three times per week for three contact hours. Students had access to the laboratory for additional work time when the building was open and when there were no other laboratory courses meeting in the lab space. There were approximately 100 hours per week available when students could have access to the laboratory outside of the scheduled meeting time for the course.

Figure 1

Syllabus timetable for molecular neurobiology. The course was designed to accommodate experimental design and execution along with manuscript preparation, submission, and revision.

In the laboratory section of the course, students were asked to develop an experiment examining the effect of commonly used stimulants and/or depressants, alone and in combination, on the crustacean Daphnia magna (Ward’s Natural Science, Rochester, NY, USA). The five groups of students chose the following combinations of chemicals for their studies: 1) ethanol and guarana, 2) ethanol and melatonin, 3) antihistamine and curcumin, 4) ethanol and acetaminophen, and 5) ethanol and aspartame. Students ran preliminary trials to determine experimental conditions and then collected data over several weeks (Figure 1).

Discussions about scientific writing were incorporated throughout the course so that students had a clear understanding of the formats of different sections of a scientific paper (Figure 1). Students working together in a group determined whether they would work on different sections of a paper and then bring them together as a cohesive whole, or whether they would all contribute to writing each section. Students were instructed that they could choose either option as long as the language of the paper was consistent and students participated for equal amounts of time. As part of the revision process, students participated in a peer-review workshop (Figure 1). This workshop involved each group reading the paper from another group out loud and writing brief comments in the margin of the paper. Students then elaborated on their suggestions for improving the paper to the other group. Students were directed to consider clarity, organization, and how well students explained background information, their findings, and the meaning of their findings. Following revisions, the instructor reviewed the paper again before it was submitted to IMPULSE. The instructor factored in a one month turn-around time on the review process, which is the typical review time for manuscripts submitted to the journal.

Course grading

In the Molecular Neurobiology course, 25% of the grade was based on the laboratory section and 75% of the grade was based on the lecture section of the course. The paper students submitted to IMPULSE was worth 10% of their final grade in the course and a second paper that students wrote individually on a different laboratory experiment was worth 15% of their final grade. For the 100 points possible for the paper, the instructor reserved 75 points to be earned based on the paper that was originally submitted to IMPULSE. These points were all assigned for the document submitted during Week 7 of the course (Figure 1). The remaining 25 points were assigned depending on the quality of the students’ responses to reviewers’ comments. Thus, only 2.5% of their final course grade depended in any way on the response from IMPULSE to the manuscript. These points were all assigned for the document submitted during Week 13 of the course (Figure 1). The instructor planned that if reviews were not received from the journal in time for students to respond by the end of the semester, she would critique the papers and grade students on their revisions based on her comments. Students were not required to continue working on the manuscripts past the end of the semester, but were encouraged to do so if needed. The instructor included in the syllabus that if a manuscript was not accepted for publication that it would not impact their grade and that as long as they completed the assignments described in the syllabus, their project would be complete at the end of the semester.

COURSE OUTCOMES

Course Evaluation

After the course concluded, and some students graduated from Ursinus College, assessment data was collected and analyzed. Students were contacted via e-mail or Facebook and a questionnaire was distributed. Of the 20 students enrolled in the course in 2009, 15 students responded to the questionnaire prior to publication of the findings in 2011. This research was approved for Exemption by the IRB at Ursinus College. The survey assessed the impact of the course on writing and engagement in understanding scientific concepts.

For a quantitative assessment of student engagement in laboratory experiments and scientific writing, students completed a Likert-scale questionnaire (Likert, 1932; Table 1). Students responded to several questions addressing the time, effort, and thought that went into their experiments and writing when they were completing a paper that would be submitted to a peer-reviewed journal compared to writing a paper that would be submitted only to their instructor. On the numerical scale used in the questionnaire, a score of “5” indicated that students spent much more time, effort, or thought, a score of “3” indicated that students spent the same time, effort, or thought, and a score of “1” indicated that students spent much less time, effort, or thought.

Table 1

Quantitative assessment of student engagement in experiments and writing for a peer-reviewed journal.

Questions		Mean ± SEM
Compared to papers of similar length you have written for other biology courses, choose a number for each of the questions below. “3” indicates that your response is the same compared to other biology courses, “1” indicates that your response is much less compared to other biology courses, and “5” indicates that your response is much more than for other biology courses. n=15
1.	How much time did you spend doing background reading for the paper?	4.5 ± 0.2
2.	How much effort went into experimental planning for the paper?	4.7 ± 0.1
3.	How much effort went into completing the experiments for the paper?	4.6 ± 0.1
4.	How much thought did you put into presenting your findings for the paper?	4.7 ± 0.1
5.	How much thought did you put into understanding the meanings of your findings?	4.5 ± 0.2

Students were also asked to compare the amount of time they spent writing and revising their papers for submission to a peer-reviewed journal compared to when they wrote papers and solely submitted them to their instructor. Students responded to the following questions: 1) “In Biology courses you have taken previously, estimate how many hours you spent writing a paper of similar length that you turned in to your professor (the total time should include any preliminary versions you completed before turning in a version to your professor)”, 2) “Estimate how many hours you spent writing the IMPULSE paper prior to submitting it to IMPULSE (the total time should include any preliminary versions completed before turning in a version to IMPULSE)”, 3) “In Biology courses you have taken previously, estimate how many hours you spent revising a paper of similar length to incorporate comments from your professor”, and 4) “Estimate how many hours you spent revising the IMPULSE paper prior to resubmission.”

For a qualitative analysis, students were directed to assess how writing a paper for a peer-reviewed journal affected their engagement in a scientific study. For this topic, students provided written responses to a question, “How was your experience designing and carrying out an experiment informed by knowing that you would submit your study to a journal?” and to respond to a request for information, “Explain how your level of engagement in understanding the effect of stimulants and/or depressants on the nervous system was altered as a result of knowing that you would submit a paper to a journal.”

Undergraduate Authors

As a result of the publication process described in this article, 20 undergraduate students became published authors on five peer-reviewed manuscripts. This number represents 100% of the students enrolled in the Molecular Neurobiology course. In the process of designing experiments, collecting data, and writing scientific papers describing their findings, students explored how several common chemicals affect nervous system activity, including ethanol (Kaas et al., 2009; Leatherman et al., 2009; Schleidt et al., 2009; Bleaken et al., 2010), and combinations of ethanol with over-the-counter drugs (Kaas et al., 2009; Bleaken et al., 2010) and chemicals that are frequently found in beverages, such as aspartame, an artificial sweetener, and guarana, a common component of energy drinks (Leatherman et al., 2009; Schleidt et al., 2009). Students also examined whether curcumin, which is found in curry, affects nervous system function in D. magnus (Vaidya et al., 2009).

For all aspects of experimental design, completion of experiments, analysis of experimental findings, background reading, and presentation of findings that were assessed, students spent more time when the paper was submitted to a peer-reviewed journal than when it was only submitted to their instructor (Table 1).

Students spent significantly more time on writing both their first submitted version and on their revisions when they submitted their papers to a peer-reviewed journal. The difference between time spent on papers for submission to a peer-reviewed journal or solely to an instructor was significantly different as shown by results from a Student t test, * indicates p<0.05 and ** indicates p<0.01 (Figure 2).

Figure 2

Quantitative analysis of time spent writing a scientific paper for a peer-reviewed journal. Results from a student questionnaire are shown. White bars indicate time spent on a paper submitted solely to the professor. Gray bars indicate time spent on a paper submitted to a peer-reviewed journal. The pair of bars labeled “First submission” represents the time students spent preparing a paper for submission. The pair of bars labeled “Revision” represents the time students spent revising a paper after receiving comments. Fifteen students completed the questionnaire. Error bars represent standard error of the mean. Asterisks indicate significant difference (Student’s t test; * indicates p<0.05 and ** indicates p< 0.01).

For the qualitative analysis, 15 of the 20 students in the course wrote detailed responses and many students indicated that writing a paper that was submitted to a peer-reviewed journal played a positive role in their experience with the laboratory exercise. Some students included a neutral response and one student included a negative response on the survey.

For the question, “How was your experience designing and carrying out an experiment informed by knowing that you would submit your study to a journal?” 14 of the 15 students who responded referred to positive aspects of their experience and one student indicated both a positive and a negative experience. Some students indicated that they were more thoughtful about their experimental design. One wrote, “Knowing that the experiment had the potential of being published motivated our group to really examine our research question and significance.” Another wrote, “My experience designing and carrying out this experiment was heavily influenced by knowing that I would be submitting to a journal. The experiment was carefully planned, researched, not only for background information but also for methods that would be useful in performing the experiment.” Students indicated that they were more diligent in carrying out the experiment. A student wrote, “I took the time to conduct more trials and include additional samples so that my group’s work would be more convincing.” A second student wrote, “Knowing that we were going to submit to a journal made everyone in the group take the project more seriously than most other projects. For example, all the members in the group put in extra hours outside of class to make sure we were keeping up with looming deadlines.” One student reflected on a positive outcome of the experience, “I can say that I spent more time on this paper than most others before,” but also perceived a negative outcome, “I noticed more than ever before that every group had one or two people who took the lead and most of the responsibility.”

When asked to address their engagement in examining the scientific concepts in the study, students responded with both positive and neutral comments to the request, “Explain how your level of engagement in understanding the effect of stimulants and/or depressants on the nervous system was altered as a result of knowing that you would submit a paper to a journal.” Of the 15 students who responded, 13 indicated that submitting a paper to a peer-reviewed journal increased their engagement in exploring the topic of study. Some students indicated that they analyzed the topic of study more carefully. A student wrote, “I did a lot more background research and therefore I had a better understanding of the effect of stimulants and depressants on the nervous system. I was also very interested in the topic because I was using my knowledge to explain something that was happening in an organism and something that I could actually see hands on.” The same student added, “Because I was very engaged in the subject material I learned a lot more from it and retained most of the information. When I have written other papers for other classes, many times I have read background information and understood it, but never really retained most of the information. The opposite happened when I wrote this paper.” Another student wrote, “Knowing that we would be submitting a research paper to IMPULSE made me take greater care in my research and preparation. I read more background information, not only on the effects of stimulants and depressants, but also on the organism we used, the stimulant and depressant we chose, and the clinical data on the affects of stimulants and depressants when taken together.” Two students did not indicate an increase in their level of engagement in studying the scientific material. One wrote, “I did not think that part was different for me personally. I always try to write the best paper possible, within an appropriate time frame.” Another student described that they examined nervous system function, but did not indicate that their experience was enhanced by writing a paper for submission to a peer-reviewed journal.

INVOLVING IMPULSE IN THE COURSE

IMPULSE review teams

While the instructor factored in a one month turn-around time on the review process, in fact, receiving five manuscripts at once was a first for the journal and required a change in the reviewing procedures to assure that the reviews took place in the usual timeframe. Typically, manuscripts arrive sporadically and the customary review process allows for all reviewers to participate. Thus, when a manuscript is submitted, and there are no others being reviewed, then the entire IMPULSE Review Board participates and as many as 80 reviewers from undergraduate institutions around the world contribute to the review. However, when multiple submissions occur simultaneously, the reviewers are divided up and assigned a single manuscript to review. The individual reviewers receive the manuscript from their assigned Associate Editor who, in turn, compiles the reviews from their team of approximately 10–15 Reviewers, with the help of a Faculty Advisor. Each of the Associate Editors then sends their compilation to the Executive Editor, who compiles these into a single, final review with the help of the Editor-in-Chief and a Faculty Advisor. It is this final review that is sent to the corresponding author.

At the time of the Ursinus submissions there were five Associate Editors and three Faculty Advisors. Each submission was sent to a different team of reviewers for simultaneous review to keep the process on track. As the manuscripts were being reviewed, one of the Faculty Advisors also contacted the Faculty Mentor (RK; in this case also the last author on each of the manuscripts) to confirm that she was aware of the submissions. In all cases when IMPULSE receives a submission there must be a Faculty Mentor identified by the student submitting the work, and IMPULSE always contacts this individual to assure that there has been some degree of faculty input on the manuscript production and also to verify that the student has permission to submit the manuscript from the listed institution.

It should be noted that no one at the journal was aware that this trial was underway and that the instructor counted on receiving the comments back within the month. While it would be useful if others contemplating following this model would notify the Editor-in-Chief in advance, the review process at IMPULSE has now been adjusted to accommodate this pattern of submission. Also, the current expansion of the journal’s Review Board and number of Reviewer Training Sites (a location where a Faculty Advisor oversees the reviewing process of a group of students, teaching them about reviewing and the publishing process) will help manage future increases in submission numbers. There are now four additional Associate Editors (nine total), as well as several new Reviewer Training Sites and Faculty Advisors (now seven) since the Ursinus project. There are also other Reviewer Training Sites in the process of forming, and this continued growth will allow for a smoother response to multiple, similar projects in the future.

IMPULSE reviews to students

Following the first review cycle, the compiled reviewers’ comments for each manuscript were sent back to the five corresponding authors. Since the course syllabus required the students to resubmit the manuscript within one week, all of the manuscripts were returned for reconsideration very promptly. The resubmissions coincided with finals week for most of the student review team. As a result of this timing, four of the five manuscripts received secondary reviews within a month, and all final corrections were made and the final versions of the articles were resubmitted for posting to the website within the third month (Kaas et al., 2009; Leatherman et al., 2009; Schleidt et al., 2009; Vaidya et al., 2009). There was one manuscript that contained more than minor writing and formatting problems and thus required a more substantial second revision, and this manuscript was accepted in the fall and finalized for posting in the 2010 issue (Bleaken et al., 2010).

Undergraduate engagement in scientific review

A large number of undergraduates participated in the review of the manuscripts and, as a result, the 80 undergraduates on the IMPULSE Review Board gained experience reviewing scientific literature, while the five undergraduate Associate Editors gained experience compiling reviews. The Associate Editors are tasked with several responsibilities: maintaining regular contact with their team of individual Reviewers (10–15); sending out the manuscripts to the Reviewers; making sure to get back the reviews in a timely manner; and compiling all the individual reviews into a single, cogent review to be sent to the Executive Editor. This experience provided these students with additional opportunity to hone skills of communication, organization, analysis, writing, and leadership.

DISCUSSION

Impact on students

This course has been taught three times previously by the instructor (RK) without the incorporation of submission of manuscripts to IMPULSE. Results of the survey of students indicated that they benefitted from carrying out an experiment the results of which would be submitted to a peer-reviewed journal. In addition to committing more time to writing and revising the paper, they also spent more time on designing and carrying out their experiments. Most students were more thoroughly engaged in the process of learning about a scientific concept, answering questions through experimentation, and writing about their findings. While all students reflected on positive outcomes in their planning and carrying out experiments, one student reflected that leaders emerged in groups who took more responsibility for pursuing the research projects. While this is often true in group work and could have been enhanced due to the increased importance students placed on the outcome of the project, only one member of one group found that this difference in engagement was significant enough to comment on in the survey. However, this possible outcome should certainly be considered by the professor teaching any laboratory course with group work so that all students have a productive experience. A small number of students indicated that their engagement in learning about scientific concepts was not enhanced by this exercise. One student indicated that they already were thoroughly engaged and another wrote that they learned the concepts, but did not indicate that their learning was enhanced. As almost 87% of the students taking the course indicated that they were more engaged in learning the scientific concepts, the exercise was successful in increasing how well most students invested themselves in exploring a scientific concept.

In addition to a survey of students, the instructor (RK) made observations about differences in student engagement in experiments and in the writing process compared to previous years. She indicated that the students spent much more time conferring with her about background information on the neuronal pathways affected by the chemicals they were testing than had students in previous courses. They asked insightful questions to clarify their understanding of complex processes and the instructor had the perception that students tried harder to incorporate an in-depth understanding of neuronal processes in the background section of papers compared to previous times she taught the course. Since the only difference was that students now had the possibility of publication, her interpretation is that it was this option that improved the caliber of the students’ scholarship. They were also more focused on completing the assays well and included a larger data set than students in previous courses had when the paper was not being submitted for external review for potential publication.

The instructor noted that students were more enthusiastic about spending additional time completing their experiments beyond the scheduled laboratory class. Further, rather than merely receiving a grade along with their instructor’s comments and being finished with the paper, the students had to respond to the IMPULSE reviewers’ comments. Thus, they spent time rewriting sections of the manuscript and providing more thorough revisions of the work than in previous courses. A bonus for the students is that each one ended up with a published paper to add to their résumé as they prepared their post-baccalaureate program applications.

A further advantage not typically experienced by undergraduates was the opportunity to respond to peer review comments. This form of collaboration with peer scientists is a necessary but under-emphasized skill for all future science professionals.

The course assessment showed that incorporating paper submission to a peer-reviewed journal was effective in increasing student engagement in writing and in studying scientific concepts. Quantitative and qualitative responses indicated that students increased the time, effort, and thought that they spent on developing and carrying out a scientific study and on writing about their findings. Specifically, students spent more time reading backgrounds information, writing a paper for initial submission to a peer-reviewed journal and revising the paper after receiving reviewers’ comments. Students indicated that they increased the effort that went into designing and completing their scientific studies. They also wrote that they were more careful in their analysis of the scientific topic of study and more engaged in relating their findings to scientific literature. More thought went into analyzing their findings and to presenting their findings in a manner suitable for journal publication. Taken together, the quantitative and qualitative assessment indicated that students were more thorough in their scientific studies and in their writing when they knew the resulting manuscript would be vetted by external reviewers with the possibility of publication.

In the future, it could be useful to build in some flexibility for the timing of the second laboratory exercise in the course that is unrelated to the paper topic. That way, if students need more time to determine appropriate conditions for their experiments, they could complete their data collection with some of the time that would have been spent on the second project. The course can also be made more efficient if students are allowed to design their experiments using a predetermined set of reagents. That way the instructor can purchase the reagents in advance and experiments will not be affected by shipping delays.

Impact on Instructor

The professor herself found that she was more thorough in critiquing students’ papers; rather than setting expectations for how an undergraduate should write, she considered how a published author should write. Specifically, she found that she commented on more aspects of the papers that could be improved so that students could complete more thorough revisions. In addition to focusing on major changes that could improve a paper, she also included comments on small details that would be expected in a scientific publication. For example, she gave more thorough explanations of how sentences would be worded in a scientific publication. She noted that while this consideration increased the time she spent reviewing student work, the effort resulted in feedback for her students that supported their writing of more polished papers and development as scientists/scientific writers. She considers the experience valuable and plans to continue the exercise with new experiments each time she teaches the course. By including the process of manuscript writing, submission, and revision into the curriculum of the course, she was able to provide the students with a full-scale model of the research experience, from experimental design and planning, through manuscript preparation and publishing.

Impact on Journal

Ordinarily, a single paper resubmitted with all reviewer concerns addressed would have been posted shortly thereafter. However, receiving five resubmissions at the end of the IMPULSE reviewers’ own semesters taxed the system, and the first article was not posted until August; the journal has modified its review process as a result of this positive development, and encourages other neuroscience faculty to consider incorporating manuscript submission to IMPULSE as part of their courses.

Impact on enhancing science education

Although institutions of higher education are under pressure to provide undergraduate students with active hands-on learning in the STEM fields, the ability to provide these experiences to large numbers of students is limited by economic resources. The current report illustrates one way to involve a larger number of students in both laboratory neuroscience and scientific publishing in an economically feasible manner; the exercise of submitting articles to the undergraduate Neuroscience journal IMPULSE was a successful learning experience. The specific impact of peer review on the scientific learning experience has not been addressed in this work, but future studies expanding on the work of others in this area (Woodget, 2003; Prichard, 2005; Blair et al., 2007; Hartberg et al., 2008) using the IMPULSE experience are currently being planned by the student editorial team. This report illustrates that IMPULSE can be incorporated as an integral part of an academic curriculum allowing for larger student involvement with hands-on neuroscience research and scientific publishing.