Facilitating long-term changes in student approaches to learning science.

Undergraduates entering science curricula differ greatly in individual starting points and learning needs. The fast pace, high enrollment, and high stakes of introductory science courses, however, limit students' opportunities to self-assess and modify learning strategies. The University of Washington's Biology Fellows Program (BFP) intervenes through a 20-session, premajors course that introduces students to the rigor expected of bioscience majors and assists their development as science learners. This study uses quantitative and qualitative approaches to assess whether the 2007-2009 BFP achieved its desired short- and long-term impacts on student learning. Adjusting for differences in students' high school grade point average and Scholastic Aptitude Test scores, we found that participation in the BFP was associated with higher grades in two subsequent gateway biology courses, across multiple quarters and instructors. Two to 4 yr after participating in the program, students attributed changes in how they approached learning science to BFP participation. They reported having learned to "think like a scientist" and to value active-learning strategies and learning communities. In addition, they reported having developed a sense of belonging in bioscience communities. The achievement of long-term impacts for a short-term instructional investment suggests a practical means to prepare diverse students for the rigors of science curricula.

INTRODUCTION

Empowering students to learn as scientists is a key goal of efforts to transform undergraduate biology education (American Association for the Advancement of Science [AAAS], 2011). This goal encompasses both engaging students in the process of science (Handelsman ; AAAS, 2011) and assisting their development of deep, versatile approaches to learning science (Tomanek and Montplaisir, 2004; McGuire, 2006). Accordingly, there is increasing recognition of the importance of investigating the impact of curricular and programmatic interventions on student development over the long term (Asai, 2011).

While incoming student preparation is highly heterogeneous, introductory science courses typically do not address the diverse starting points and learning needs of individual learners (Tanner, 2011). In particular, the fast pace, high enrollment, and high stakes of introductory science courses limit opportunities for students to self-assess and modify their strategies for learning science. Such limitations can be particularly significant for educationally disadvantaged students, who may lack adequate guidance on how to study and who face large achievement gaps in introductory science courses (Haak ), which serve as gateways to the majors in science, technology, engineering, and mathematics (STEM) fields.

While underrepresented minority (URM) freshmen are as interested in pursuing STEM degrees as their white and Asian-American peers (Seymour and Hewitt, 1997; Hurtado ), URM retention rates in STEM fields are disproportionately low (Hurtado ). Many factors contribute to these disparate retention rates, including differences in prior preparation, academic adjustment during the transition to college, self-efficacy, engagement in learning communities, sense of belonging, and performance in introductory science courses (Hurtado , 2007; Chemers ; Espinosa, 2011; National Academy of Sciences [NAS], 2011; Ong ).

Interventions that are designed to improve student performance and narrow achievement gaps have taken multiple forms, focusing on either instructor pedagogy (Deslauriers ; Haak ) or student preparation (Maton ; Matsui ; Barlow and Villarejo, 2004; Dirks and Cunningham, 2006; McGuire, 2006; Wischusen and Wischusen, 2007). While there is a clear need to improve course design and instructional practices (Handelsman ; AAAS, 2011), intervening with educationally disadvantaged students before their entry into high-stakes gateway courses has the potential both to narrow achievement gaps and to contribute to students’ academic adjustment and sense of belonging.

The Biology Fellows Program (BFP) at the University of Washington (UW) was founded with the goal of increasing the success of diverse undergraduates in the biosciences. As the UW's introductory biology series is a gateway to all of the bioscience majors, one of the BFP's first aims was to improve student performance in this critical series of three courses (BIOL 180, 200, and 220). To this end, the key feature of the program was a course that strongly emphasized the development of science process skills, including graphing, data analysis, experimental design, and scientific communication. The program also provided supplementary instruction to students enrolled in the introductory biology courses. Previously, Dirks and Cunningham (2006) reported on the qualities and performance of the 2003–2004 Biology Fellows, showing that they earned higher grades, on average, than their non-BFP peers in BIOL 180, 200, and 220.

Since the previous study, we have revised the BFP, expanding its emphasis from developing science process skills to developing holistic strategies for learning science. The revised program focuses on three interconnected strategies: 1) unpacking “thinking and writing like a scientist”; 2) testing and practicing a variety of metacognitive learning strategies; and 3) building supportive bioscience learning communities. Selection of strategies to address these goals was based on prior program experience, faculty input, and published literature that advocates teaching core competencies that are critical for deep learning in the discipline (Handelsman ; Donovan and Bransford, 2005; AAAS, 2011). Specifically, we incorporated a deliberate practice framework to boost students’ problem-solving abilities (Deslauriers ), focused on skills and topics that faculty identified as especially challenging for students, and provided frequent opportunities for students to use active, metacognitive learning strategies and peer-learning communities (Donovan and Bransford, 2005; Schraw ). In essence, by embedding opportunities to reiteratively and collaboratively practice strategies for learning alongside relevant content in a premajors course, the 2007–2009 BFP aimed to have measurable impact on how students, as individual learners, approached learning science.

While it is relatively straightforward to assess an intervention's impact on students’ grade performances, such measurements leave open the question of how the intervention may be facilitating change (Matsui ; Dirks and Cunningham, 2006). A mixed-methods approach with complementary quantitative and qualitative assessments provides a means to address this question and to capture more fully the richness of individuals’ learning. Accordingly, this study used a design framework with quantitative and qualitative assessment methods and external evaluators (Piore, 2006) to determine whether impact of the current version of the BFP was significant and to identify effective practices.

METHODS

BFP Design and Implementation

The BFP intervened through a premajors course (BIOL 106) that students completed before beginning the introductory biology series. This series consists of three 10-wk courses (BIOL 180, 200, and 220). Successful completion of the first course is a requirement for all UW bioscience majors, and completion of all three courses is required for the biology major. Because of introductory chemistry prerequisites, students typically did not enroll in BIOL 180 until Spring quarter of their first year or Fall quarter of their second year. This enrollment pattern provided opportunities for the BFP to recruit students already on campus and to intervene before they entered BIOL 180.

In keeping with the BFP's overall goal of increasing the success of diverse undergraduates in the biosciences, the BFP worked with partners across the UW campus to recruit participants. For example, program directors participated in UW Office of Minority Affairs and Diversity student orientation events and communicated regularly with UW Educational Opportunity Program (EOP) learning centers and advisers who referred students to the program. In addition, students who indicated “pre-science” interests upon admission to the UW were emailed information about the BFP.

For the 2007–2009 cohorts, BIOL 106 met for 2.5 h per wk, for a total of 20 sessions over two academic quarters. Course learning goals were based in part on previous program experience (Dirks and Cunningham, 2006) and in part on faculty and student responses to two surveys. A group of faculty who were currently or had recently taught in the introductory biology courses, were asked to describe the most difficult concepts for students to learn in their courses, the most important skills for students to have before entering their courses, and the attributes of their most successful students. A cohort of newly recruited BFP participants was asked to describe what they anticipated would be the most challenging aspects of the introductory biology courses. For the 2007–2009 BFP, the program directors categorized, tallied, and used the survey responses to prioritize BIOL 106 learning goals and design new assignments to meet each goal (Table 1).

Table 1.

BFP course design and assessment questions

Course goal	Key assignments (number of dedicated sessions or “R” for short reiterative coverage)	Focus group question
Understanding hypotheses	Use two research publications (4): interpret data figures, propose hypotheses, evaluate with peers, and revise hypotheses	Do Biology Fellows understand what constitutes “thinking like a scientist?”
	Use research seminar (2): propose initial hypotheses about the topic, listen to seminar, and revise and refine hypotheses
Learning scientific writing	Discuss principles of scientific writing (1)
	Study examples from two research publications (1)
	Listen to research seminar, write a short summary of content, evaluate and discuss examples from peers (2)
	Practice writing concise and precise answers (R)
Improving exam performance	Listen to BIOL 106 lectures on selected introductory biology topics (2); read and study text, and evaluate current study and exam-taking strategies (R); take two timed BIOL 106 exams (2); reevaluate, test, and revise strategies (R)	Do Biology Fellows value active-learning strategies and learning communities?
Improving metacognition	Learn about research studies on learning strategies (R)
	Complete in-class assignments practicing study strategies, rate performance, and identify areas for improvement (R)
	Maintain portfolio of course materials and assignments (R); evaluate performance midcourse (1); discuss with instructor (20-min individual appointment); evaluate portfolio at end of course (1)
Encouraging peer support	Interact with different groups each class session (R)
	Participate in discussions comparing learning challenges (R)
	Work cooperatively in class on small-group assignments (R)
Encouraging proactive use of academic resources	Listen to and discuss advice from a panel of more advanced students and academic advisers (1)	Do Biology Fellows develop a sense of belonging in bioscience communities?
	Meet individually with academic adviser (1)
	Meet individually with instructor (1)
	Prepare a job resume and inquiry letter (1)
	Maintain email contact with BFP peers, instructors, and academic advisers to learn about support and science opportunities (R)

Whereas the BFP's original implementation emphasized the development of science process skills through a series of hierarchical experimental design assignments (Coil ), assignments in the revised BIOL 106 unpacked complex skills into composite parts and challenged students to apply their knowledge reflectively and reiteratively, following the educational approach of deliberate practice (Deslauriers ). For example, for three separate case studies that increased in sophistication, students were asked to: 1) propose original hypotheses to explain published observations; 2) self-evaluate and peer-evaluate hypotheses using the same set of instructor-provided questions; and 3) revise to improve their hypotheses. By breaking hypothesis formulation and evaluation into smaller steps, these case studies also integrated self-assessment into thinking, writing, and learning like a scientist. In addition, BIOL 106 instructors explicitly stressed the need for the students to gain skills and confidence in evaluating hypotheses and scientific writing.

To further assist premajors in developing expert-like strategies for learning science, instructors modeled a variety of learning strategies in the classroom and asked students to apply and evaluate these strategies for their own use. In addition, BIOL 106 lectures, assignments, and exams were explicitly connected to authentic content from the introductory biology courses, a strategy inspired by the Lousiana State University's Biology Intensive Orientation for Students (BIOS) Program of Wischusen and Wischusen (2007). We used examples of foundational content (e.g., cell division, Mendelian genetics, and natural selection) and concepts that instructors of the introductory biology series identified as especially challenging for students to learn (e.g., Hardy-Weinberg equilibrium, phylogenetic relationships, and transcription). For example, lectures introduced active-learning techniques in the context of the biology topics. Students were then asked to study the material, complete an exam, analyze their own performance, review research-validated learning strategies, and describe how they would change their own approaches to studying and taking exams in light of their experience. All course materials were designed specifically for BIOL 106, but the level was similar to that of a typical introductory biology course. However, unlike the large-enrollment introductory biology courses, BIOL 106 provided students with frequent, reflective opportunities to apply different learning strategies to solve a variety of problems and to practice their writing skills.

Drawing inspiration from the University of California–Berkeley's Biology Scholars Program (Matsui ), the BFP strongly emphasized building supportive learning communities with peers, advisers, and faculty. In BIOL 106, students were assigned to new groups for collaborative activities each week, with the explicit goals of empowering them to extend their network of potential study partners, to “normalize” their experiences so that they could strive for improvement together, and to understand the responsibilities of individual learners within an effective learning community. The BFP instructors also shared their own challenges as learners to set the tone that it is acceptable to not know yet and that learning well takes time and hard work. Following BIOL 106, the Biology Fellows were encouraged, but not required, to enroll in the introductory biology courses and to participate in supplementary instruction or tutoring together to maintain peer interactions. They were also encouraged to participate in undergraduate research.

Statistical Analysis of Student Background and Introductory Biology Performance

For each student enrolled in BIOL 180, 200, or 220 between Spring quarter 2007 and Summer quarter 2010 (14 quarters total), we extracted the following information from the UW student database: 1) high school grade point average (GPA; on a 4.0 scale); 2) Scholastic Aptitude Test (SAT) Math and Verbal/Critical Reading scores (on standard 800 scales); 3) EOP status, UW's category for URM, economically disadvantaged, and first-generation college students (yes or no); 4) participation in 2007–2009 BFP (yes or no); 5) year and quarter when the student took each of the three courses; and 6) final course grades (on a 4.0 scale).

Information about all of the categories was not available for some students. Missing data in statistical analysis can sometimes be problematic, particularly if they are associated with the outcome of interest. In our study, however, the distribution of grades in the group of students with missing information was approximately the same as the distribution of grades in the group of students with no missing information. Therefore, we proceeded with analyzing the performance of students with complete data.

To examine the relationship between participation in the BFP and performance in the introductory biology courses, we used nonhierarchical linear regression analysis, including all of the explanatory variables listed above. Our model was the following, for each of the three courses (BIOL 180, 200, and 220) separately:

According to this model and our coding of the EOP and BFP variables, the intercept (β0) represents the average grade of a student who did not participate in the BFP; was non-EOP; had high school GPA, SAT Math score, and SAT Verbal score of 0; and took the course in Fall quarter 2007, the earliest period we considered. Each of the remaining coefficients (β1 to βγ) that we estimated can be interpreted as the average change in a student's grade associated with a unit change in the corresponding variable, with everything else held constant. βγ represents the coefficients for a group of 14 binary variables, one for each of the quarter/year combinations in which a section of the course was taught. This group of coefficients accounts for the variability of overall student grades from section to section.

To assess whether there was a difference in the strength of the association between BFP participation and grade performance for EOP and non-EOP students, we included an interaction term BFP*EOP. Thus, the interaction coefficient β3 is the average change in the grade associated with participating in the BFP and having EOP status. A positive β3 would suggest that the BFP benefits EOP students more than non-EOP students. A negative β3 would suggest that the BFP benefits EOP students less than non-EOP students.

Residuals from the fitted regression model did not show any patterns, and there were no influential outliers. The only slight divergence from the normality assumptions inherent in the model was that the residuals had a longer left tail than expected. To alleviate concerns that too many parameters may be fitted (20) for the number of students in the smallest category (BIOL 180: 44; BIOL 200: 47; BIOL 220: 35), the regression was run without the “SECTION” variable. The estimates did not change by a large amount, and the statistically significant findings remained so.

Retrospective Focus Group Study of Biology Fellows

The BFP directors asked two research scientists in the UW Office of Educational Assessment (OEA) to organize and conduct in February 2011 a focus group study of BFP students’ experiences. The purpose of the study was to gather information from BFP participants retrospectively—2 to 4 yr after their participation in the program—about their learning experiences in and beyond the program and the relationship between those experiences and program goals. Following a brief discussion of BFP design with the program directors, the OEA scientists independently designed and facilitated the student focus groups and analyzed the results.

The study methods included a short survey and 90-min focus group conversations composed of 11 open-ended questions developed by OEA and BFP program directors. One OEA researcher asked the questions and the other took notes electronically, capturing close to verbatim what the students said. Once focus groups were completed, students’ survey responses were systematically hand-tallied.

Students’ responses to open-ended questions were analyzed using a constant comparison method (Merriam, 2001), an inductive process that allows themes to emerge based on the students’ own comments. Open-ended responses of the 2007/2008 and 2009 groups, as well as those of EOP and non-EOP students, were analyzed separately in order to note differences, and responses were then compared. Researchers used tables to code responses, identifying themes, noting the frequency with which themes were mentioned and addressed, and tracking the extent of agreement about ideas raised within and across groups. Although generalizability is not a focus of qualitative research, the focus group study of BFP students’ experience included methods that strengthen external validity, such as the use of standard sampling procedures, of multiple cases to study the same questions, of predetermined questions that repeated from group to group, of specific procedures for coding qualitative data, and of using two researchers who “checked” each other's analyses and interpretations (Merriam, 2001).

RESULTS AND DISCUSSION

Introductory Biology Achievement Gaps and Challenges

UW assigns EOP status to students upon entry into the university. The EOP category includes URM, economically disadvantaged, and first-generation college students. Achievement gaps between EOP and non-EOP students in the UW's introductory biology courses (BIOL 180, 200, and 220) are large (Haak ) and have persisted for more than a decade. For Spring quarter 2007 to Summer quarter 2010, which is the relevant time period for this study of the BFP, the EOP achievement gaps for BIOL 180, 200, and 220 were 0.54 ± 0.16, 0.36 ± 0.22, and 0.34 ± 0.13 grade points (on a 4.0 scale), respectively. These data suggest that an investment in EOP student training should begin before entry into the first introductory biology course.

To define challenges that might be faced by premajors, we asked faculty who taught in the introductory biology series to identify the top three skills that they wanted students to develop before taking their courses. The group of 18 respondents included, but was not exclusive to, the faculty who taught the 2007–2009 BFP participants. These faculty instructors ranked the following skills as most important: 1) evaluating one's own understanding (74% of the responses); 2) thinking scientifically (35%); and 3) communicating effectively (30%). In contrast, beginning Biology Fellows (n = 63), who had not yet enrolled in BIOL 180, anticipated challenges in the introductory biology courses to be: 1) the need for general study skills (56%); 2) a heavy course workload (22%); and 3) lack of previous preparation for biology (16%). Furthermore, when asked to describe challenges that they anticipated in developing scientific skills, 59% of the students did not demonstrate an understanding of what constitutes a scientific skill. This “expectation gap” between faculty and student perspectives of what it takes to be a successful student in the introductory courses is common to diverse institutions (McGuire, 2006; Coil ) and suggests that beginning students would benefit from a more realistic understanding of the skills required to think, write, and learn like scientists.

A challenge for students enrolled in UW's three-quarter introductory biology series is that the courses are most often team-taught by different instructors and with a variety of instructional styles. Between Spring quarter 2007 and Summer quarter 2010, 12, 9, and 11 different instructors taught in the UW's BIOL 180, 200, and 220 courses, respectively. Students in the 2007–2009 BFP had four to seven different instructors as they progressed through the series. In addition, average grades varied over the 14 quarters of the study interval and for each course (2.73 ± 0.14 for BIOL 180, 2.80 ± 0.10 for BIOL 200, and 2.85 ± 0.16 for BIOL 220). Variability in instruction suggests a need for students and programs alike to develop versatile approaches that can be applied in a variety of learning contexts.

Taken together, these characteristics of beginning courses and students demand the development of deep and flexible learning strategies. However, many students, especially educationally disadvantaged students, lack adequate guidance on how to study and sufficient time to make thoughtful adjustments. Thus, strategies that help individual students develop not only as scientifically skilled, but also adaptive learners of science, should be of particular benefit to beginning students.

Recruitment of BFP Participants and Student Starting Points

To determine whether the 2007–2009 BFP was successful in meeting its goal to recruit diverse cohorts of students, we compared the Biology Fellows (n = 139) with the overall population of BIOL 180 students. Fifty-one percent of the Biology Fellows were EOP, compared with 16% of BIOL 180 students. Furthermore, 36% of the Biology Fellows were URMs, compared with 9% of BIOL 180 students. Thus, the 2007–2009 Biology Fellows were more diverse than the overall population of BIOL 180 students, and more diverse than participants of earlier versions of the BFP (Dirks and Cunningham, 2006).

The 2007–2009 BFP accepted nearly all of the students who applied. Thus, it is possible that those who applied differed in starting motivation or preparation from the comparison group of non–Biology Fellows who took introductory biology. Accordingly, we examined whether the Biology Fellows and non–Biology Fellows had comparable starting points, specifically high school GPAs and SAT scores. In addition, because of the historically large achievement gaps in introductory biology, we separately examined EOP and non-EOP populations within these groups. While the differences between EOP BFP and EOP non-BFP high school GPAs, non-EOP BFP and non-EOP–non-BFP high school GPAs, and EOP BFP and EOP non-BFP SAT Verbal scores were statistically significant, the differences were small: 0.04 grade points, 0.04 grade points, and 22 points, respectively. The other differences were not statistically significant (Table 2). While we cannot measure differences in motivation, we conclude that the starting points of the 2007–2009 BFP and non-BFP students were comparable.

Table 2.

Comparison of BFP and non-BFP high school GPAs and SAT scores

	Mean score ± SD (n)
	EOP BFP	EOP non-BFP	Non-EOP BFP	Non-EOP Non-BFP
High school GPA	3.73 ± 0.252 (149)	3.69 ± 0.289 (1320)	3.82 ± 0.173 (147)	3.78 ± 0.214 (8072)
SAT Math	587.45 ± 83.037 (145)	580.75 ± 75.568 (1324)	649.93 ± 75.253 (139)	649.37 ± 75.018 (7899)
SAT Verbal	564.69 ± 84.188 (145)	542.76 ± 84.417 (1324)	598.52 ± 79.242 (142)	602.56 ± 88.401 (7899)

Student Performance in Introductory Biology Courses

The UW's introductory biology series is a gateway to bioscience majors. The series provides a foundation for upper-level course work, and grades earned in these courses contribute to competitiveness for STEM majors, scholarships, internships, and other opportunities. As one of the BFP's aims is to improve student performance in these critical courses, we performed a linear regression analysis to examine the relationship between multiple potentially explanatory variables and performance in the introductory biology courses.

For BIOL 180, 200, and 220, the regression analysis indicated a statistically significant association between student grades and high school GPA, SAT Math score, and SAT Verbal score (Table 3). The association between student grades and EOP status was significant only for BIOL 180. Using regression analysis to examine the effect of BFP participation as an isolated variable, participation in the BFP was associated with statistically significant average increases in grades for the first and second introductory biology courses (BIOL 180 and 200): 0.29 and 0.39 grade points, respectively (Table 3). The performance gains in the third course (BIOL 220) were not statistically significant. In addition, the interaction effects between BFP participation and EOP status were not statistically significant.

Table 3.

Regression analysis: regression model coefficients

	Regression parameter estimates ± SE
Regression model coefficient or summary statistic	BIOL 180	BIOL 200	BIOL 220
BFP participation (β1)	0.29* ± 0.105	0.39* ± 0.107	0.11 ± 0.098
EOP status (β2)	−0.09* ± 0.036	−0.07 ± 0.041	−0.03 ± 0.039
BFP*EOP (β3)	0.10 ± 0.145	−0.01 ± 0.150	−0.06 ± 0.142
High school GPA (β4)a	0.99* ± 0.052	0.85* ± 0.060	0.72* ± 0.058
SAT Math score (β5)b	0.002* ± 0.00017	0.002* ± 0.00019	0.002* ± 0.00020
SAT Verbal score (β6)b	0.003* ± 0.00015	0.002* ± 0.00017	0.002* ± 0.00016
Residual SE	0.72	0.71	0.61
Sample size	3766	2872	2294
Multiple R2	0.36	0.23	0.25

*Statistically significant, p < 0.05.

a4.0 scale; b800 scale.

A key question for a premajors program is whether its participants will apply the strategies they develop in the program to later course work. The association of BFP participation with increased performance in two subsequent introductory biology courses (BIOL 180 and 200) suggests that, for the BFP, the answer is yes. In addition, these results confirm the continued effectiveness of the BFP. The lack of statistical significance for the third course (BIOL 220) may be explained by one or more alternative hypotheses: 1) the population of students enrolled in BIOL 220, which is not required for all bioscience majors, may be inherently different from those enrolled in BIOL 180 and 200; 2) unidentified differences in course instruction may impact the effectiveness of, or students’ need for, the BFP; or 3) variability in when students take BIOL 220 may reduce the benefits of BFP participation, including belonging to a cohort of Biology Fellows. Our current study does not distinguish among these or other hypotheses. Nevertheless, higher grades in the first two introductory biology courses suggest that students applied the BFP's strategies with versatility, since these courses were taught by different instructors over multiple quarters.

With large achievement gaps between EOP and non-EOP students in the introductory biology courses, one might anticipate strong associations in the regression analysis between EOP status and student grades. EOP students, as a group, however, have lower high school GPAs, SAT Math scores, and SAT Verbal scores (Table 2), all of which are variables in the regression model. This helps to explain the relatively small contribution of EOP status, which was statistically significant only for BIOL 180 (Table 3). Similarly, while the lack of interaction effects between EOP status and BFP participation might suggest that EOP students did not benefit from the BFP any more than their non-EOP peers, as a group EOP students stand to gain more from even quantitatively similar grade increases, as these increases allow educationally disadvantaged students to progress through the introductory biology series and remain competitive for bioscience majors.

Focus Group Study of BFP's Long-Term Effects on Student Learning

While quantitative assessment demonstrates that participation in the BFP is associated with increased performance in introductory biology courses, qualitative assessment has the potential to reveal long-term changes in learning strategies and insights into how the program may be facilitating these changes. Accordingly, we conducted focus group discussions of Biology Fellows 2 to 4 yr after their participation in the BFP.

All students in the 2007, 2008, and 2009 BFP cohorts were invited to participate. Twenty-seven students agreed to take part and were organized into three focus groups—nine from the 2007 and 2008 BFP cohorts in one focus group and 18 from the 2009 cohort in two other focus groups. The study's 27 students included 13 (48%) EOP students; this reflected the overall representation of EOP students (51%) in the BFP.

In the focus group discussions, Biology Fellows attributed long-term changes in how they approached learning science to their participation in the BFP. To investigate the nature of these changes and how program approaches and activities influenced student learning, we report on whether Biology Fellows: 1) understand what constitutes “thinking like a scientist”; 2) value active-learning strategies and learning communities; and 3) develop a sense of belonging in bioscience communities.

Evaluation of Impact on Thinking Like a Scientist.

“Thinking like a scientist” is a core disciplinary competency (AAAS, 2011). Beginning students, however, typically emphasize the findings of science rather than its process, and their study strategies do not embrace the depth of learning in the discipline (Tomanek and Montplaisir, 2004, McGuire, 2006). Accordingly, we probed the level of sophistication with which Biology Fellows defined “thinking like a scientist” and whether their understanding of this topic influenced their learning strategies.

When asked to define what it meant to “think like a scientist,” students in each focus group built a composite definition from many contributions, rather than creating a single definition that all participants agreed on. The three groups identified nine attributes of scientific thinking, and there was broad agreement in each group about the attributes that the group members identified. Six of those nine ideas came up in more than one group: 1) having curiosity; 2) thinking creatively, in terms of discovery and conducting experiments; 3) thinking critically, including challenging data through a careful thought process; 4) keeping an open mind and seeking and adapting to new information; 5) being a problem solver; and 6) communicating effectively. In addition to these six ideas that crossed the three focus groups, there was considerable agreement among students within individual focus groups about three additional attributes: 7) designing good experiments and interpreting them accurately; 8) identifying the most important information on a given topic; and 9) using scientific thinking in everyday life.

When Biology Fellows discussed, “What skills, if any, did you learn in the Biology Fellows Program that you have used in your science classes and how have you used them? Have you used these skills in any of your non-science courses or outside class?,” three major themes emerged: 1) how to be concise and precise, which they identified as a key element of biology exams and scientific thinking; 2) how to study actively; and 3) how to read scientific texts. Importantly, they spoke of using these skills in a variety of settings beyond the science classroom, including in their jobs, in their interpersonal relationships, and in their nonscience classes. For example:

“[The BFP instructor] made us do the whole hypothesis thing every single week. I carried that with me. I say it all the time: ‘my hypothesis for that is this.’ It is my thinking process now—I think this is how this works and this is how I would test it. I do that for everything. I say that to my boyfriend sometimes. When I'm reading, too, sometimes [I notice] they say what works but they don't say why.”

Furthermore, Biology Fellows identified learning how to think like a scientist as one of the most important things they learned in the BFP:

“Just how to break a question down, how to break a problem down and really analyze it like a biologist would do—to systematically understand it piece by piece. I learned that from [the program].”

We conclude that Biology Fellows' perspectives on “thinking like a scientist” are sophisticated and more closely resemble those of experts than those of beginning students. They recognize the complexity of scientific thinking and integrate a complex set of skills and attitudes into their understanding of the process of science. Furthermore, they value its relevance to success in their courses and lives, identifying scientific thinking as a key learning strategy. While the focus group study cannot distinguish fully between the contributions of the BFP and subsequent biology course work, the Biology Fellows themselves attribute growth in scientific thinking and related skills to their participation in the BFP.

Evaluation of Impact on Active-Learning Strategies and Engagement in Learning Communities.

Successful learners in the sciences employ active, metacognitive learning strategies and engage in learning communities (Donovan and Branford, 2005; Schraw ). Undergraduate students entering STEM disciplines, however, typically undervalue and underutilize these approaches (Tomanek and Montplaisir, 2004; McGuire, 2006). Given that one of the primary approaches of the BFP is to model and provide opportunities to practice these behaviors, we sought to understand whether participation in the BFP actually led students to value and apply these strategies in their studies. When asked to identify what they had learned from the BFP that was the most important to them, Biology Fellows emphasized five themes: 1) value of study groups; 2) how to study actively; 3) how to think like a scientist; 4) information about resources and opportunities at the UW; and 5) value of paying attention to the graphs, tables, and charts in the textbook as they read.

By far the strongest theme of the five, mentioned by 14 of the 27 participants, was that students had learned the value of study groups. Many students mentioned that they had never studied in groups in high school, and two students noted that it is intimidating to try to form study groups in a large class. The BFP forced students to work together, however, and, in working in groups, students learned the value of that practice. Several students spoke of the importance of study groups to them after the introductory biology series:

“It's something I've carried with me all throughout my college career. Science is really complex, and there's always a chance that you missed something. But working with people, they help you cover those holes. There was a lot of group work in the BFP, and you'd hear a lot of opinions from all sides, which stretches your way of how you view something.”

A second strong theme in students’ responses was that the BFP taught them to study actively, rather than just memorizing:

“[The BFP instructor] stressed active study instead of passive study. Don't highlight and underline—those things don't work. Talk to people, draw pictures, do active things like that. It was nice to have someone reveal that to you, to know that it isn't just you who zones out while reading. And it was helpful that he pointed us in some definite directions for how to study better.”

Students often connected active studying to two other aspects of their learning. The first was identifying what the student did not know, the skill that 74% of the surveyed faculty who taught in the UW's introductory biology series identified as the most important skill for students to develop before taking their courses. The second was that it related to “thinking like a scientist.” While the students did not explicitly connect examining figures, studying actively, and thinking scientifically, analyzing figures is also a key active-reading strategy for science courses and an important component of “thinking like a scientist” (Felder, 1993). Taken together, these connections suggest that Biology Fellows are incorporating these themes into the more integrated learning strategies of practicing scientists:

“We got the sense that the best way was to study to understand it from our own interests as scientists. And that's what I do now in my other classes—study as a scientist, to learn because of my interests and to put that information into the things I already know. It was a message that that's important, and you can keep it in mind. It inspires you.”

When discussing, “What skills, if any, did you learn in the Biology Fellows Program that you have used in your science classes and how have you used them? Have you used these skills in any of your non-science courses or outside class?,” there was wide agreement across the three focus groups that learning to study actively, rather than passively, was a major skill they had learned from the BFP. Students’ responses included a number of specific examples, including doing problems, talking to others, and reviewing one's own performance on exams after the exams are returned. Four of the 11 students who spoke of active studying mentioned an emphasis on driving their studying with their own hypotheses. One of those four students noted that how one defines a hypothesis in college differs from how a hypothesis is defined in high school, and many students in that focus group agreed with that characterization.

Taken together, these results suggest that the BFP is influencing how Biology Fellows approach learning science. We conclude that they are not only deepening their understanding of scientific thinking, metacognitive learning strategies, and the value of learning communities, but that they are also connecting them into an integrated model of a scientist, and thus assisting their learning in the discipline. Importantly, they are continuing to integrate and apply these approaches to their studies years after their participation in the BFP.

Evaluation of Impact on a Sense of Belonging in Bioscience Communities.

Intellectual and social integration is important for all students, and a sense of belonging in the discipline is of particular significance for URM students (NAS, 2011). Accordingly, we sought to understand whether the BFP had influenced Biology Fellows’ sense of belonging. Students in all three focus groups strongly felt that the BFP created a sense of community for them that lasted beyond the program. They identified numerous benefits of continuing as members of that community, including feeling connected, supporting one another academically and socially during shared struggles, exchanging information about resources and opportunities, and readily forming effective study groups for a variety of courses.

Students in all three groups said that the sense of community they experienced was intentionally fostered and modeled by program leaders. Students also spoke of how the collaborative nature of BIOL 106 activities led to a sense of community and how persistent communication with the program organizer extended the community beyond BIOL 106. In addition, several students characterized the kind of community created by the BFP as distinctly different from what they experienced in their science courses in general, as the following interaction illustrates:

Student A: “I think everything in the sciences is considered so competitive, and relationships get broken because of that. [The BFP instructor] encouraged us to be so connected, so that all of us could be better than what we were alone. We learned to see people as someone we could learn from, rather than as someone we needed to compete with.”

Student B: “It's because classes are curved. I think that made a difference [in the BFP]. We didn't go there and sit and take notes. You're at a table and everyone helps you out.”

To further investigate how the BFP facilitated engagement in bioscience communities, we asked, “Have your interactions with others been influenced in any way by your BFP experience?” The same themes emerged across all three focus groups. They reported that the BFP helped them talk to faculty, speak with academic advisers early, and understand that peers were important to their learning. Another interaction that students said had changed because of the BFP was their willingness to ask questions in class and during office hours, although they found doing so intimidating.

From the students’ responses, we conclude that the BFP is having a long-term impact on Biology Fellows’ sense of belonging in the biosciences and on their interactions within that community. Furthermore, modeling and practicing related behaviors in a supportive environment, rather than simply describing them, appear to be key programmatic approaches.

Implications for the BFP as a Practical Model

With large introductory science courses often unable to address the heterogeneous starting points of individual learners (Tanner, 2011), external strategies that target the needs of individuals can complement efforts to transform the gateway courses themselves. For example, the University of California–Berkeley's Biology Scholars Program's building of supportive academic communities, the UW's BFP's development of science process skills, and the Louisiana State University's Biology Intensive Orientation for Students’ preview of course content and study skills were all associated with higher student grades in subsequent science courses (Matsui ; Dirks and Cunningham, 2006; Wischusen and Wischusen, 2007). Given the different programmatic approaches and the refinement of the BFP model to emphasize developing effective science learning strategies, the current study addresses the question of how the BFP facilitates student learning and which BFP practices are important for effecting change. Here we discuss implications for implementing the BFP as a practical model for other universities.

First, why intervene before students enroll in their first introductory biology course? Introductory biology courses have several characteristics that make them among the most challenging learning experiences for students. The courses are usually large. At the UW, the first introductory course has up to 700 students per quarter. The courses are often team-taught, requiring students to adjust to a variety of instructional styles. In addition, they are fast paced. The UW's 10-wk introductory biology course has four lectures and one laboratory session per week and three or four high-stakes exams. The high student-to-faculty ratios allow little time for individualized instruction and faculty guidance for those students who are unprepared for university-level science courses. Furthermore, poor performance in entry-level courses may be difficult for students to overcome, either discouraging or preventing them from pursuing STEM majors (Seymour and Hewitt, 1997).

The BFP's premajors course (BIOL 106) presents multiple strengths to prepare students for these challenges: 1) students can develop realistic expectations of the rigor of the gateway courses and practice new learning strategies without risking high-stakes penalties; 2) without the need to rapidly cover content, the course can focus explicitly on strategies for learning and communicating science; and 3) implementing a premajors course can be easier than modifying the gateway courses, given their complex framework of preexisting goals and dependencies. For example, a recent survey of more than 400 science chairs found that while 46% of the chairs felt that gateway courses harmed diversity by driving away potentially successful students, 57% felt no need to modify these courses to retain more STEM students (Bayer Corporation, 2011).

Second, how does this premajors course influence long-term development of individual students? In this mixed-methods study, the 2007–2009 Biology Fellows reported positive impacts on how they learned and practiced science at 2 to 4 yr after participating in the program. They learned to understand what constitutes “thinking like a scientist,” to value active-learning strategies and learning communities, and to develop a sense of belonging in bioscience communities. Recognizing and owning these attributes are key qualities of successful STEM students. Furthermore, deep learning in the discipline plays a central role in undergraduate development and immersion in the major (Beyer ). Facilitating growth in these areas is generally regarded as difficult to accomplish in the classroom, yet fostering such growth is critical for enhancing student learning and success in the discipline (Donovan and Bransford, 2005).

In addition, when participants were asked to identify strengths of the BFP, six themes emerged from their responses. The students strongly noted that one of the program's major strengths was the instructors’ emphasis on teaching and modeling what it meant to be part of a scientific learning community. Additionally, students identified as strengths the information about resources and opportunities provided by the BFP; the study methods they learned in the BFP, including learning to work in study groups and to “think like a scientist”; the preparation and support provided by the BFP for success in the introductory biology series and other science courses; their new sense of what they could do; and the friends and sense of community provided by the BFP. Taken together, these findings suggest that the BFP's student-centered focus on modeling and practicing learning strategies within a supportive classroom environment is an early investment that pays both short- and long-term dividends.

Importantly, while the BFP's long-term impact on students’ approaches to learning science is meaningful for all students, the consequences are especially significant for educationally disadvantaged students. Self-efficacy and sense of identity as a scientist are critical factors contributing to academic performance and retention in STEM (Chemers ; Ong ). Furthermore, academic adjustment and a sense of belonging during the first year of college are linked (Hurtado ). Thus, adoption of learning strategies effective for university-level STEM courses has the potential to promote not only success in the classroom but also a sense of belonging in STEM. Accordingly, the BFP continues to invest in effective recruitment partnerships to increase EOP participation, while simultaneously increasing program size. BIOL 106 is currently taught to 60–90 students as a 10-wk course that meets twice per week. If offered twice per year, it has the capacity to improve the learning of more than half of all EOP students who enter introductory biology at the UW each year. Furthermore, as the BFP strategies were not designed for a single course or instructor in mind, they would likely benefit beginning students in other science disciplines and at other institutions as well.

Prominent education and policy recommendations call upon scientist educators and mentors to grow and diversify talent in STEM fields (NAS, 2010, 2011) and to place earlier and stronger emphasis on the process of science (AAAS, 2011). Narrowing the achievement gap and facilitating changes in student approaches to learning science are long-term impacts for the BFP's relatively short-term investment. Based on our findings here, the BFP provides a practical model for addressing these needs by connecting premajors’ development of skills and effective learning strategies to authentic content in a way that amplifies the learning of all students.