Scaling Up: Adapting a Phage-Hunting Course to Increase Participation of First-Year Students in Research.

Authentic research experiences are valuable components of effective undergraduate education. Research experiences during the first years of college are especially critical to increase persistence in science, technology, engineering, and mathematics fields. The Science Education Alliance Phage Hunters Advancing Genomics and Evolutionary Science (SEA-PHAGES) model provides a high-impact research experience to first-year students but is usually available to a limited number of students, and its implementation is costly in faculty time and laboratory space. To offer a research experience to all students taking introductory biology at Gonzaga University (n = 350/yr), we modified the traditional two-semester SEA-PHAGES course by streamlining the first-semester Phage Discovery lab and integrating the second SEA-PHAGES semester into other courses in the biology curriculum. Because most students in the introductory course are not biology majors, the Phage Discovery semester may be their only encounter with research. To discover whether students benefit from the first semester alone, we assessed the effects of the one-semester Phage Discovery course on students' understanding of course content. Specifically, students showed improvement in knowledge of bacteriophages, lab math skills, and understanding experimental design and interpretation. They also reported learning gains and benefits comparable with other course-based research experiences. Responses to open-ended questions suggest that students experienced this course as a true undergraduate research experience.

INTRODUCTION

It is well established that inquiry-based laboratory courses and authentic research experiences are valuable components of effective undergraduate science education. The National Science Foundation, the National Academies of Sciences, and the American Association for the Advancement of Science (AAAS) emphasize the importance of these practices in science curricula (National Research Council, 2003; AAAS, 2011; Connelly and Sharp, 2011). For example, the authors of the AAAS Vision and Change report emphasize “learning science means learning to do science” and advocate research-like experiences for all students (AAAS, 2011). Typically, undergraduate research is conducted by advanced students who are science majors. These students are already invested in, and committed to, a science degree and typically participate in research via the apprenticeship, or internship, model. While the internship model—research conducted one on one with a faculty mentor—can be a particularly deep and transformative learning experience (e.g., Lopatto, 2010a,b), it is inefficient in terms of faculty time and is generally available only to a select few students. Course-based research experiences (CREs) address the challenge of making research experiences available to the majority of students (Gammie and Erdeniz, 2004; Goyette and DeLuca, 2007; Brame ; Temple ; Auchincloss ; Shaffer ). Yet because these research opportunities are often only available to students in upper-division courses, most students do not have the opportunity to experience discovery-based research as freshmen.

Research experiences during the first 2 yr of college, however, are key to increasing the number of science, technology, engineering, and mathematics (STEM) graduates in the United States (President’s Council of Advisors on Science and Technology, 2012). There is growing recognition that a research experience in the first year not only provides educational gains but also positively influences persistence in science, especially among underrepresented minorities (Nagda ; Summers and Hrabowski, 2006; Weaver ; Wink and Weaver, 2008; Jones ; Graham ; Bangera and Brownell, 2014). A number of programs have been implemented recently with the goal of providing first-year undergraduates with research experiences (Summers and Hrabowski, 2006; Feng ; Temple ; Brownell ). On a broad level, the cross-institutional and Howard Hughes Medical Institute (HHMI)-supported Science Education Alliance Phage Hunters Advancing Genomics and Evolutionary Science (SEA-PHAGES) project has been particularly successful at engaging freshmen in research (Hanauer ; Caruso ; Hatfull ; Harrison ; Jordan ).

Phage Hunters is an inquiry-based research and education platform for students in high school or the beginning years of college (Hatfull, 2015). Since 2008, thousands of undergraduate students have participated in SEA-PHAGES courses at 95 institutions (Jordan ; Hatfull, 2015). In a typical SEA-PHAGES course, first- or second-year students discover, isolate, and characterize novel bacteriophages from soil samples during the fall—the Phage Discovery semester. The genomic DNA of one of the phages is sequenced during winter break, and the students annotate the new genome and conduct comparative genomics with it during the Spring semester. The novel genome is then submitted to GenBank to be available for the scientific community to study. The broad implementation of the SEA-PHAGES course across many institutions shows that beginning students with no prior training in lab skills or content knowledge can become engaged in authentic research, which then positively affects the students’ education (Jordan ). Specifically, assessment data suggest that the learning gains of students in the SEA-PHAGES course are at least equivalent to, if not greater than, those of students participating in an internship-type research experience during the summer (Jordan ).

The typical implementation of a SEA-PHAGES course is resource intensive in terms of faculty time and required laboratory space. The course meets twice each week (4–6 h total) and thus requires more time from faculty than a typical lab course and occupies a lab classroom for twice the amount of time each week. In addition, in accordance with the SEA-PHAGES guidelines, it is suggested that two faculty members be present for each lab period. Because of the meeting frequency and faculty commitment involved, it is difficult to provide the SEA-PHAGES experience to more than a small fraction of biology students at an institution each year. Most schools that teach a SEA-PHAGES course offer it to only a subset of students—typically one or two lab sections per year (D. Russell, personal communication). Depending on the institution, the students are selected in a variety of ways: by application, on a first-come, first-served basis, or because they are honors or at-risk students. For many institutions, the current SEA-PHAGES course is either optional within the biology curriculum or is an alternative for a standard introductory laboratory course (Jordan ).

Furthermore, because the traditional SEA-PHAGES course is time intensive for students in requiring two lab sessions per week for an entire year, it may not be attractive to students who are indecisive about a major in science and are unwilling to invest such a large amount of time in something with which they are unfamiliar and of which they are unsure. The gains observed among the select students who have participated in the SEA-PHAGES course (Jordan ), however, are the kinds of learning gains we want all biology students to experience. Unfortunately, it may not be practical or possible to scale up the typical SEA-PHAGES course at many institutions to allow all introductory students to participate. Introductory biology is usually the largest undergraduate course that biology departments teach. Too few faculty members are available to teach lab sections twice each week, and there is typically not enough lab space available for these classes to meet that frequently.

Thus, to engage all introductory biology students at Gonzaga University (350/yr) in a transformative research-based lab course during their freshman year, we modified the typical 1-yr SEA-PHAGES lab course so that all biology students could participate in at least the first semester of Phage Discovery. We describe how we brought the SEA-PHAGES model to scale and review the challenges of scaling up the SEA-PHAGES course and our solutions to those challenges. We then use a mixed-methods approach to assess the value and outcomes of participation in this new one-semester lab course. Our hypothesis is that student participation in the single-semester Phage Discovery course alone will lead to gains in student learning and will show benefits comparable with other CREs. Furthermore, we expect that these students will experience the course as an “authentic” research experience or course-based research experience as defined in the literature (Auchincloss ).

METHODS

SEA-PHAGES Course Modification and Description

To offer a SEA-PHAGES research experience to all students taking introductory biology at Gonzaga University, we modified the standard SEA-PHAGES course in several ways in accordance with available space and resources.

The yearlong course was split into two separate courses. The first semester—phage discovery and characterization—became the lab course for introductory biology (Information Flow in Biological Systems, BIOL 105), which serves ∼350 students each year. The second semester—genome annotation and analysis—became the lab course for Genetics (BIOL 207), taken by ∼125 students each year. This modification is significant, because nearly two-thirds of the students who participate in the first course do not proceed to the second course, either because their academic major does not require Genetics (e.g., nursing, human physiology, psychology, computer science, and others) or they switch majors before taking Genetics. Biology and biochemistry majors (and other students with premedicine interests) experience the genome annotation portion of the SEA-PHAGES course when they take Genetics, typically in their sophomore year. This implementation of the second semester is similar to the SEA-PHAGES approach, with a mix of bioinformatics and molecular biology bench work. We added a third course (Advanced Phage Research lab) so that interested students could pursue phage research in a more independent setting after they complete the Genetics course. Recent iterations of this course involved five to seven students per semester in a project to functionally characterize transcriptional promoters in phage genomes that have been predicted by bioinformatics. The split of the standard 1-yr SEA-PHAGES course into two separate courses, with an additional third phage-research course, fully integrates the SEA-PHAGES material into the biology curriculum. This allows all students in the introductory biology class (primarily non–biology majors) to participate in the first course, the Phage Discovery semester (Figure 1).

Figure 1.

Conversion of a powerful SEA-PHAGES model to a scaled-up model that reaches more students. In the Gonzaga University model, the thrill of discovery is available for all introductory biology students in the Phage Discovery lab. A fraction of those students, biology and biochemistry majors, experience a computational approach to research in the Phage Genomics module embedded in the required Genetics lab. Students who wish to extend their research experience participate in independent work with molecular methods or bioinformatic analysis in the Advanced Phage Research lab.

The Phage Discovery semester, which typically meets twice each week (4–6 h) in the SEA-PHAGES course, was streamlined to fit into a standard semester lab course that meets once each week for 3 h. To reduce the time needed to isolate and purify phages, we eliminated SEA-PHAGES experiments aimed at generating a large-volume, high-titer lysate from a preliminary “medium-titer lysate.” Instead, the students used the preliminary lysate directly to isolate DNA and to prepare transmission electron microscope grids for imaging at the nearby Franceschi Microscopy and Imaging Center at Washington State University (see Table 1 for the weekly class schedule and Supplemental Table S1 for a comparison of Gonzaga course activities with a typical SEA-PHAGES course). To make efficient use of class time, we asked students to be prepared for each week’s lab by completing readings and assignments, viewing videos demonstrating new lab techniques, and taking an online prelab quiz.

Table 1.

Weekly schedule of Gonzaga’s one-semester Phage Discovery lab course

Week	Activity
1	Collect soil, inoculate enrichment culture
2	Plate enrichment supernatant in dilution series
	Achieve these activities in weeks 3–7:
3	Test putative plaques
4	Plaque-purify at least three rounds
5	Determine titers of plaque lysates
6	Flood “web” plate to produce medium titer lysate
7	Titer medium titer lysate
8	Mount samples for transmission electron microscopy
9	DNA prep
10	DNA prep
11	Restriction digest
12	Agarose gel electrophoresis
13	Restriction fragment gel analysis
14	Student poster presentations
15	Final exam and assessment

To give students opportunities to repeat failed experiments or do additional rounds of purification or DNA isolation, open lab times were offered on some evenings and most weekends. The open labs were staffed by undergraduate teaching assistants (TAs). Most of the open lab periods were during the first half of the semester, when repeated culturing of phage with bacteria was necessary. Most sessions were 2–4 h in length, with students finishing their experiments in 1–2 h. Later in the semester, fewer students needed time outside of class to repeat specialized protocols such as extracting DNA or running a gel, and the number of open lab periods was reduced. These later sessions required a more experienced TA or instructor to be present.

To scale up the Phage Discovery course to 12 sections of 16 students each semester, we made several changes to reduce costs and to help the course run efficiently. To reduce consumption of supplies, we had students work in pairs to isolate and characterize a phage, rather than working individually. The need to purchase large incubator spaces (for up to 20 plates per group) was eliminated by incubating most cultures and plates at room temperature. Most significantly, the course had a full-time lab coordinator who managed the preparation of media, reagents, and supplies for the labs; trained and managed undergraduate TAs; managed content delivery through Blackboard (the course-management system); and administered assessment surveys and tests. The laboratory coordinator also taught several of the lab sections.

To facilitate implementation of the modified Phage Discovery lab, the laboratory coordinator participated in the HHMI SEA-PHAGES training workshop. Additional faculty instructors were trained to teach the course by participating in a summer workshop at Gonzaga University that replicated the course experience: each faculty member worked six afternoons across 6 wk to isolate and characterize a novel phage just as the students do in the Phage Discovery semester. In subsequent years, newly hired instructors participated in a 1-h introductory workshop and learned to teach the course from the teaching materials and knowledgeable peers teaching the other lab sections.

Although the modifications above reduced the amount of time that students spend doing research compared with the typical SEA-PHAGES two-semester course, the research goals and the learning objectives remain similar. The research aims are to find and purify a bacteriophage from an environmental sample, obtain morphological information about the phage with transmission electron microscopy, and examine the phage’s DNA by restriction fragment size analysis. In addition to the original SEA-PHAGES curriculum, students perform tests for temperature-dependent growth and the ability to superinfect lysogens carrying known prophages. (The lysogen infection test can indicate whether phages are related to each other. Phages that are related to the prophage in the lysogen are usually unable to form plaques on the lysogen, whereas unrelated phages are usually able to infect the lysogen and form plaques.) These characterizations allow students to compare the similarities and differences of their newly discovered phages. Students name the phage they discover and submit data to the Actinobacteriophage Database at www.phagesdb.org, a database that contains records of all bacteriophages discovered in the SEA-PHAGES program. Students prepare their phage lysate for frozen archival storage and understand that this phage may be studied further in the Genetics lab and its DNA may be sequenced.

The learning objectives for this course are organized under three categories: the nature of science, the nature of laboratory work, and the nature of viruses. Class assignments and assessments are aligned with the learning objectives (listed in Table 2). Students learn about the nature of scientific investigation by evaluating their experimental results, drawing conclusions based on evidence, and making decisions about what experiments to perform next. Students also learn about laboratory work, including analytical and microbiology lab skills; how to keep a notebook; and how to communicate results effectively. The students are introduced to the components of scientific communication by making a poster about their phage and presenting it to the class; this is in addition to the important routine communication between lab partners. Students learn basic bacteriophage biology to understand their experiments, and additional readings are assigned to introduce students to the broader world of viruses and research. Teaching materials, assignments, and assessments are available by request.

Table 2.

Learning objectives for Gonzaga’s one-semester Phage Discovery lab course

Outcomes	Assessments
Nature of science	Formative
Students should be able to:	Discussion with faculty, TAs, and class
Predict outcomes of experiments.	Prelab quizzes
Interpret results, including evaluation of positive and negative controls.	Notebook checks
Design experiment to address a question.	Summative
	Final notebook
	Final exam
	Poster
Nature of lab work	Formative
Students should be able to:	Pre-lab quizzes
Perform unit conversions.	Notebook checks
Make serial dilutions.	Midterm practical exam
Calculate titers.	Summative
Perform scale conversions.	Final notebook
Make standard curves.	Final practical exam
Demonstrate aseptic technique.
Use pipettors correctly.
Nature of viruses	Formative
Students should be able to:	Prelab assignments
Describe life cycles of bacteriophages.	Q-and-A sessions
Describe the scope of virus diversity.	Class discussions
Describe the role viruses play in biology.	Summative
Describe the relevance of virus diversity to research.	Final exam

The modification and scale-up of the Phage Discovery lab coincided with a major revision of the biology curriculum. The previous introductory biology course, Diversity of Life (BIOL 101) was replaced with Information Flow in Biological Systems (BIOL 105) at the same time that the lab was replaced by the Phage Discovery lab. In the semester before the switch, we piloted the Phage Discovery lab in two of seven BIOL 101 lab sections (32 students, randomly assigned) and in a nonmajors biology course with 13 students. These students were not assessed with the instruments described below but were instead invited to take a short postcourse survey. We proceeded with the curriculum-wide implementation because the Phage Discovery students more frequently reported, compared with their traditional lab counterparts, that the course helped them learn how scientists think and learn what scientists do, and were more likely to recommend the lab to other students (Supplemental Figure S1).

Assessment Instruments

We used a mixed-methods approach (Creswell and Clark, 2007) to collect both quantitative and qualitative data about students’ experiences in a number of areas, including learning of content, perceived learning gains, and perceptions of the research experience overall. Gonzaga University’s Institutional Review Board for Human Subjects Research approved all aspects of this study (Gonzaga IRB protocol no. 1404BIOAND and 1602STAANDBIO).

Learning of Content: Pre/postcourse Test.

To objectively measure gains in knowledge and scientific skills, we devised a short, multiple-choice test (see Supplemental Material). Three questions addressed basic knowledge of bacteriophages (bacteriophage definition, plaque formation, life cycle); three questions assessed lab math skills (metric unit conversion, concentrations, dilutions); and four questions assessed understanding of experimental logic (controls and interpretation of results). In the first week of the course, students took the test on Blackboard. The test results were not communicated to the students, and the test itself was not discussed in class. After completing the written final exam in the last week of the semester, students took the same multiple-choice test using classroom computers. Students were awarded a small number of extra-credit points for completing the test. Test results were obtained in four semesters: Spring 2013, Spring 2014, Fall 2014, and Spring 2015. The answers from students who took both tests were collected to produce a paired data set of 528 students. Pre/posttest scores were compared with a two-tailed, paired t test. Each question was analyzed using McNemar’s test.

Quantitative Responses: The Classroom Undergraduate Research Experience (CURE) Survey.

We used the SEA-CURE (Classroom Undergraduate Research Experience) survey, a variation of the CURE survey developed for the SEA-PHAGES program, to collect students’ self-reported learning gains (Jordan ). Students took the precourse survey at the beginning of the course and the postcourse survey at the end. To measure learning gains, we had students report what level of gain resulted from their research experience (no or very small gain, small gain, moderate gain, large gain, very large gain) in response to a series of statements (e.g., Clarification of career path, Skill in the interpretation of results; see Figure 3 later in this article for a complete list). We collected data from students from 2012 to 2015 (n = 906) and compared those results with those of students who participated in two-semester SEA-PHAGES courses across the country (2011–2015, n = 3037), students in other CRE courses across the country (2015, n = 8960, CURE survey), and students who took the SURE III (Summer Undergraduate Research Experience) survey in 2014.

Figure 3.

Learning gains in 21 areas in the SEA-CURE survey, self-reported by students. The learning gain items shown are the same as gains in the CURE survey and the SURE III survey. Students were asked to rate their self-perceived gains in each area on a scale of 1 through 5 (1: no or very small gain; 2: small gain; 3: moderate gain; 4: large gain; 5: very large gain). Mean ratings are shown for students in the one-semester Gonzaga Phage Discovery course (red triangles, n≤ 906, 2012–2015), all students in the SEA-PHAGES program excluding Gonzaga students (green squares, n≤ 3037, 2011–2015), students in other CRE courses (yellow circles, n≤ 8960, 2015), and summer research students who took the SURE III survey (blue diamonds, n≤ 3041, 2014). Vertical lines indicate two SEs above and below the mean for the SURE survey. SEs are too small to display for the other categories.

Qualitative Responses: The Lab Exit Survey of Student Experiences.

To gather students’ experiences in the Phage Discovery course, we designed a survey with three open-ended prompts:

Write a brief description of your favorite “aha” moment in BIOL 105 lab. This would be a time when something suddenly made sense or became more interesting or when you suddenly realized what you were doing (right or wrong).

Give a specific example of how this lab has impacted your understanding of science.

Give a specific example (or a few examples) of how this lab has influenced your specific educational or career goals.

The survey was administered in the Fall of 2013 and Spring of 2014 on the last day of class. Students entered their answers on classroom computers and were awarded a small number of extra-credit points for entering responses. All student identifiers were removed from the responses before analysis. Responses from Fall 2013 (n = 129) and Spring 2014 (n = 161) were analyzed separately and then combined once no significant differences were found between the two cohorts.

Using previously established guidelines (Saldana, 2012), we independently identified dominant response themes. On the basis of common features, we reconciled the response themes into four categories: Career and interest, Thinking and working like a scientist, Skill development, and Self-confidence. The category Thinking and working like a scientist was further split into five subcategories: ownership, iteration, metacognition and connections, mentorship and relationships, and science as a process. These themes are consistent with those found in undergraduates’ writing about their experiences doing apprentice-style independent research (Seymour ).

The student responses from each semester were read and coded independently by two researchers, who identified which category or subcategory best described each response. Responses scored differently by each coder (30% of responses) were discussed, and a consensus was reached. Responses to each of the three prompts were analyzed individually.

Participants.

The Phage Discovery lab is the required laboratory component of the introductory lecture course Information Flow in Biological Systems (BIOL 105), typically taken by students in their first year of college. In the academic years 2012–2013, 2013–2014, and 2014–2015, 10–12 lab sections were offered each semester, with a capacity of 16 students per section. Thirty-five percent of our students were biology majors, 38% nursing or human physiology majors, 9% biochemistry majors, and 18% other majors. Seventy percent of students were in their first year of college. The fraction of females in the class was relatively high (70%), largely due to the largely due to the high frequency of females among nursing students at this university.

RESULTS

Discovery of Phages

The research performed by students in the Phage Discovery lab has contributed positively to the overall SEA-PHAGES crowd-sourcing project. The efforts of 1072 Gonzaga students resulted in the isolation of 538 phages. The phages were characterized by transmission electron microscopy and DNA restriction fragment length analysis and archived as frozen lysates for future study. In 2013–2014 and 2014–2015, information for 364 phages was entered into the Actinobacteriophage Database at www.phagesdb.org for use by other scientists. Many of the phages discovered in the course have been studied further by the students in the Genetics lab. In 2014–2015, 65 phages were partially sequenced and assigned to phage clusters, eight phages were sequenced at the whole-genome level, and two genomes were annotated for submission to GenBank.

Learning of Content: Pre/postcourse Test

To test whether students gained knowledge during the course, we administered a multiple-choice pre/posttest aligned with the learning objectives, with questions related to knowledge of bacteriophage biology, lab math, and experimental logic. A set of pre–post results was obtained from 528 students across four semesters. The students exhibited significantly higher scores at the end of the course (Figure 2A; p = 10−105, two-tailed paired t test) and showed improvement on each question of the test (Figure 2B; p values ranged from 10−3 to 10−70, McNemar’s test). Students in the lab course scored significantly higher on the posttest than students enrolled in the lecture but not in the lab portion of the course (unpublished data). These findings demonstrate that students improved their basic knowledge of bacteriophages, lab math skills, and understanding of experimental design and interpretation after taking the one-semester phage course.

Figure 2.

Results of knowledge test given before and after the course. A 10-question multiple-choice test was given at the start and finish of four semesters. Precourse and postcourse answers from 528 students were compared. (A) Distribution of precourse and postcourse scores. Score represents the number of correct answers. The mean scores were 4.5 (pre) and 6.9 (post) (p = 10−105, two-tailed, paired t test). (B) Percentage of students who answered each question correctly. McNemar’s chi-squared test shows that student success on each question improved significantly (question 1 (Q1): p = 10−7; Q2: p = 10−12; Q3: p = 10−23; Q4: p = 10−3; Q5: p = 10−9; Q6: p = 10−5; Q7: p = 10−27; Q8: p = 10−36; Q9: p = 10−12; Q10: p = 10−70).

Quantitative Responses: The CURE Survey

Results from the SEA-CURE survey for six semesters (2012–2015) show that the one-semester phage course at Gonzaga compared favorably with national averages for other CREs (CURE survey, 2015) and summer undergraduate research programs (SURE III survey, 2014; Figure 3). Most of the self-perceived gains reported by students in the one-semester course were similar to, or higher than, those in CRE courses across the country, with the exception of Ability to read and understand primary literature and Skill in how to give an effective oral presentation. Students in the one-semester course rated gains in a number of items similar to, or higher than, those who participated in summer undergraduate research experiences. These items are related to scientific thinking: Understanding science, Understanding how scientists think, Ability to analyze data and other information, Skill in interpreting results, and Understanding scientific assertions require evidence. Across all the items in the survey, students in the one-semester phage course reported similar or lower gains than those who completed two-semester SEA-PHAGES courses.

To account for the variance among students in the one-semester course, we conducted further analyses on the learning gains items using three variables—major, gender, and instructor (novice vs. veteran). These analyses indicated no differences in benefits or gains on any of the 21 items found in Figure 3. In other words, students reported similar gains despite their major, gender, or instructor (unpublished data).

Qualitative Responses: The Lab Exit Survey of Student Experiences

The Fall 2013 and Spring 2014 exit survey responses demonstrated experiential learning gains in four categories: Career and interest, Thinking and working like a scientist, Skill development, and Self-confidence. The category Career and interest included statements by students that referred to preparing them for their career, increasing their interest in their major or career, or informing their decision to pursue (or not pursue) biology research. The category Skill development included gains in basic content knowledge and basic lab skills, such as using a micropipette. Comments coded as Self-confidence made a reference to gaining some personal confidence in themselves or their ability. And finally, the category Thinking and working like a scientist incorporated gains in problem solving, conceptualizing scientific problems, and understanding how scientific knowledge is formed. The range of traits associated with scientific thinking required differentiating responses into the following subcategories: ownership, iteration, metacognition, mentorship, and scientific process (Auchincloss ).

The first prompt was designed to determine which element(s) of the class contributed to their learning. When asked to describe a favorite “aha moment” from the course, the majority of student responses (67%) were in the category of Thinking and working like a scientist (Table 3). Responses in the subcategories of iteration and scientific process documented that students faced and addressed obstacles, and learned how to work on “real-world” scientific problems. Examples of responses coded in these categories are

Table 3.

Proportions of open-ended survey responses by categorya

Category	Prompt 1: “Aha” moment	Prompt 2: Example of understanding science	Prompt 3: How this lab influenced goals
Career and interest	0.35% (1)	5% (15)	65% (177)
Thinking and working like a scientist	67% (193)	44% (124)	13% (36)
Ownership	32% (61)	9% (11)	17% (6)
Iteration	16% (31)	8% (10)	8% (3)
Metacognition	41% (79)	30% (37)	53% (19)
Mentorship	9% (17)	2% (3)	8% (3)
Science as a process	3% (5)	51% (63)	14% (5)
Skill development	28% (80)	47% (134)	14% (37)
Self-confidence	4% (12)	4% (11)	8% (22)
n	286	284	272

aOn the basis of common features, we categorized responses into four themes. The category Career and interest included student responses that referred to preparing them for their career, increasing their interest in their major or career, or informing their decision to continue pursuing research. Responses in the category Thinking and working like a scientist involved problem solving, conceptualizing scientific problems, and understanding how scientific knowledge is formed. The category Skill development included responses that addressed basic content knowledge and lab skills, such as using a micropipette. And finally, responses coded as Self-confidence made a reference to gaining some confidence in themselves or in their ability in the lab. The first prompt (“Write a brief description of your favorite ’aha’ moment in BIOL 105 lab”) was designed to determine which element(s) of the class contributed to student learning and the second prompt (“Give a specific example of how this lab has impacted your understanding of science”) to assess whether students view science as a series of facts or as a process; the third prompt asked students directly how the experience influenced their educational or career goals. Student responses to these open-ended questions suggest that students experienced this course in ways that are consistent with a research experience.

“It has been a lab where one mistake forces you to do everything over. You cannot just skip a step. For example, when I contaminated our lysate we were forced to remake it. It made me realize that science isn’t always easy or goes my way, but with time and patience results will be seen and redoing steps is rewarding at the end of the day.” (coded as iteration)

“This lab impacted my understanding of science in teaching me to better use lab equipment and to also think on my own of how to solve problems we may face in a lab environment.” (coded as scientific process)

The next most common theme for the first prompt was Skill development, identified by approximately one-quarter (28%) of the students (Table 3). In general, these students mentioned improvement in doing a particular technique. For example,

“My ah-ha moment came when I finally figured out how to perform aseptic technique quickly and effectively.”

The second prompt asked students to describe how the lab impacted their understanding of science. The question was designed to assess how students view science: Is science a series of facts to memorize or is it a process of how knowledge is tested and constructed? The majority of student responses (91%) fell into two categories: Thinking and working like a scientist and Skill development (Table 3). When the category Thinking and working like a scientist was subdivided into its component parts, we identified more specifically whether students thought science was a process or whether they addressed some other aspect of being a scientist. Slightly more than half of these responses (51%) indicated that students identified Science as a process (Table 3). The following is an example of a response coded in this category:

“This is my first experience with any sort of research-based lab work, and it helped me understand the practical aspect to lab science. The open-ended question of our objective helped me see scientific questions the way real researchers do: full of possibility and intrigue, with a motivation to find an answer, whatever that answer may be (expected or not). The first time I got contamination on a plate helped me realize that each of my actions directly impacted my study, and showed me how easily the process of science can be interfered [with] and how careful you have to be.”

The third prompt asked students how the experience influenced their educational or career goals. Because this prompt pointed student answers in a specific direction, it is not surprising that a majority (65%) of student responses fell under the category of Career and interest (Table 3), such as this response:

“This lab has sparked my curiosity about lab research as a possible future career. I really enjoyed discovering new things I had previously never been exposed to. It gave me a sense of accomplishment after all our hard work paid off and we got results and information about our phage. This is something I could see myself doing later in life.”

Smaller proportions of responses showed learning gains in the categories Thinking and working like a scientist (13%) and Skill development (14%), especially those viewed as relating to the students’ career or major choices. Interestingly, several students (15% or 40/272 responses) reported learning that they do not want to pursue a research career, even though they found the lab experience interesting. For example,

“I’m fairly certain I don’t want to necessarily enter a science field, but performing a lab has given me a new appreciation for scientists who regularly do this work and the amount of patience one has to have!”

Although not specifically requested by any of the three open-ended questions, students mentioned gains in confidence, as noted in other free-response surveys of undergraduate researchers (Laursen ). These responses were present at low frequencies for all three prompts in our survey (4%, 4%, and 8%, respectively). Responses coded as self-confidence include

“The first week of the lab was super scary and intimidating. I had never been in a lab environment like this one in high school. At about the third week in lab, it started to become fun. I understood the basic concept of what we were doing and specifically grasped the concept of why we were doing the dilution series. It changed from this scary, unknown thing to a lab course that I could see myself completing successfully.”

“I tend to be kind of intimidated by situations where I am going in not knowing exactly what to do or what is going to happen, but I think this lab has really helped me learn to think like a researcher and not be afraid of things like designing experiments, getting (and interpreting) unexpected data, trying things again/altering procedures that aren’t working, etc., and I honestly think that will probably be one of the most valuable things that I EVER get out of a science class here at Gonzaga University.”

DISCUSSION

The key features of the modification and scale-up of the SEA-PHAGES course were 1) streamlining the Phage Discovery lab to meet once per week instead of twice so that it could become the introductory biology laboratory course; 2) integrating the second SEA-PHAGES semester into the standard Genetics lab course; and 3) hiring a lab manager, peer TAs, and prep assistants to facilitate running multiple sections.

Students who participated in the modified first-semester SEA-PHAGES course benefited from the research experience, even though they had not engaged in the genomic research and publication activities of a second-semester SEA-PHAGES course. Students in the course demonstrated an increase in knowledge of phages, laboratory and math skills, and the logic of experimentation (Figure 2). The students reported research-related learning gains that were higher than those in other CRE courses and compared favorably with students who participated in undergraduate summer research experiences (Figure 3). Finally, student responses to open-ended questions suggest that they experienced this course in ways that are consistent with a research experience (Table 3).

On the pre/postcourse content test, the largest learning gains occurred in three of the four questions on experimental logic: questions 7, 8, and 10 (Figure 2). Questions 7–9 tested for the ability to identify or interpret positive and negative controls in a non–phage experiment, and question 10 tested the ability to interpret DNA fragment sizes after gel electrophoresis. Students had the most difficulty with these questions on the precourse test, so many of them had the opportunity to improve their understanding in these areas over the course of the semester before they took the postcourse test.

Students in the one-semester course reported gains on the CURE survey that are comparable in a number of ways with those of students who experienced research in other CRE courses or in summer undergraduate research programs (Figure 3). For nearly every research-related item on the CURE survey, students in the one-semester course reported similar or greater gains than students in other CREs across the country. The two exceptions were gains in ability to read primary literature and skill in oral presentation, consistent with the curriculum of the one-semester course, which includes little exposure to primary literature and only one short oral presentation at the end of the term. CURE survey items related to scientific thinking were rated by students in the one-semester course with similar or greater gains than those who experienced summer undergraduate research. Two of these items, Ability to analyze data and Understanding scientific assertions require evidence, had ratings that were nearly identical between the Phage Discovery students and those who completed the SEA-PHAGES course. Together, these data clearly indicate that the primarily first-year college students in the one-semester Phage Discovery course experience many of the same benefits that other students do when they participate in research.

The CURE survey also shows that students in the one-semester course consistently rated research-related items lower than students who completed both semesters of the SEA-PHAGES course (Figure 3). It is not surprising to us that SEA-PHAGES students reported higher self-perceived gains. They spent more time in the laboratory (two semesters at 4–6 h/wk) than our Phage Discovery students did (one semester at 3 h/wk), and they participated in additional genomics research that produced a scientific product (annotated genome) that was ready to share with the scientific community. The CURE survey results suggest that students benefit more from both semesters of a SEA-PHAGES course than our students do in the one-semester Phage Discovery course. This highlights the importance of incorporating the Genomics semester of the SEA-PHAGES course into the curriculum for biology and biochemistry majors. Nevertheless, this result should not diminish the conclusion that students who take the Phage Discovery course actually do experience the benefits that come from engaging in research.

Several recent reports examine the advantages of authentic research experiences for undergraduates and provide frameworks to define a research experience (Laursen ; Brownell ; Auchincloss ; Linn ). In general, the reports agree that research experiences provide: engagement in real science problems (broadly relevant work); problems with no known solution or with unknown outcomes (discovery); interaction with peers and other scientists (collaboration); experienced mentors modeling problem solving (iteration, collaboration); tools, skills, and practices that are current and real (iteration, scientific practices); and high potential for presentation and/or publication (broadly relevant work, scientific practices). Student responses to open-ended survey prompts identified all of these components in the one-semester Phage Discovery course, suggesting that students were experiencing the course as a research experience.

Auchincloss use the term “discovery” to describe gaining knowledge from an investigation in which the outcome is unknown by all involved; the students experienced this sense of discovery in our one-semester Phage Discovery course. The characteristics of each isolated phage were unknown to everyone involved. Most commonly, survey responses expressing a sense of ownership reflected the students’ awareness of their discovery:

“My favorite moment was when we discovered our phage. After weeks of disappointment to finally see that we had gotten our own phage was so cool! I also loved seeing our rapid growth at 37 degrees C. It was the first time I started to grasp the enormity that is mycobacteriophage in our world.”

As this response suggests, students also learned to appreciate their Phage Discovery projects as part of a bigger research endeavor, even without participating in the analysis and publication of the phage genome that occurs in the second semester of a SEA-PHAGES course. Students began to recognize that their work in the lab course was broadly relevant when they entered their phage discovery information into the Actinobacteriophage Database.

An unexpected advantage to assessing student learning gains with an open-ended survey was insight into how working as a scientist gave students a sense of engagement as a scientist. The unrestricted nature of the prompts allowed students to more fully explain the learning gains recorded in the SEA-CURE survey. For example, students were motivated by the engagement in the community of practice (Lave and Wenger, 1991) established in the Phage Discovery course; they recognized the meaning of their work:

“It has helped me feel more engaged in science. Chemistry labs are just repeating experiments that have already been done where this lab makes me feel as though I am doing something worthwhile.”

“Through this lab I realized how much I love research and how fun it can be to discover something, even as small as a phage. Furthermore, I enjoyed how I was able to contribute to PhagesDB, I feel like I was actually doing something beneficial.”

Other important aspects of authentic research are, in the words of the CURE survey, Becoming part of a learning community and Understanding how scientists work on real problems. Responses to our qualitative survey support the learning gains observed for these CURE items:

“My favorite, and possibly largest ah-ha moment had to be in lab just last week, when we finally had all our phage info and I was comparing it to the other info of phages on phagesdb. I first started by comparing the DNA sequencing and found two clusters that had similarly sequenced DNA to my phage. Then I decided to look at the phage morphology and the TEM images and was shocked when all the phages in cluster C, who was one that had very similar DNA, looked just like my phage!! I then discovered how Cluster C is the phages with myroviridae tails, meaning they are short and don’t bend, and was so excited when everything just fit into place and my phage found its group to fit in with.”

“It’s very interesting the way phage might be able to be used in medicine instead of antibiotics.”

Other elements of a research experience are incorporated into the one-semester Phage Discovery course as well. Students used scientific practices, including making and presenting a scientific poster as part of scientific communication. Students collaborated with their lab partners to accomplish their research objectives, and clear communication with instructors and TAs was required throughout the semester. Students exchanged data with other students to help answer their research questions about the novelty of their discovered phages. Finally, iteration—tolerating failure or repetition (Laursen )—often occurred when, after interpreting their results, students realized they needed to repeat one (or many) steps before moving forward.

Being immersed in science research for a semester gives students insight into what the process of science actually is and allows them to truly think and work like a scientist, which allows them to make informed choices about their major and career goals. Specifically, multiweek immersion gives students time to fully experience the iterative aspect of science and ensures they receive enough instructional time to achieve the intended benefits (Shaffer ). The iteration component was often a time for students to make gains in content learning as well as in understanding and appreciating the scientific process.

Although this streamlined Phage Discovery course provided students with a valuable research experience, we do not think that this course would be as effective if the subsequent genomics component was not also present in the curriculum. The scientific discoveries in the phage-hunters program, and subsequent contributions to the scientific literature, come from the analysis of the genomes of newly discovered phages. The authentic nature of the research in the Phage Discovery course would disappear if students in the second SEA-PHAGES semester did not analyze and disseminate annotated genomes of phages discovered in the first semester. At Gonzaga, this takes place in the Genetics lab course. It is this later work that gives the first-semester research its authenticity and significance.

We encourage those who are interested in adopting the SEA-PHAGES course, or this modification of the course, to contact HHMI to learn about becoming a member of the Science Education Alliance. HHMI trains new cohorts of alliance members each year to implement the course.

In summary, we have modified the first semester of the SEA-PHAGES course and implemented it as a Phage Discovery course taken by all introductory biology students. We found that, despite the reduced number of hours in the lab and the lack of a second SEA-PHAGES semester for some students, students who took the one-semester Phage Discovery course exhibited characteristics of students who have engaged in an authentic research experience. We conclude that this course is beneficial for all students and particularly for those who may never take another research-based lab course.