Collectively Improving Our Teaching: Attempting Biology Department-wide Professional Development in Scientific Teaching.

Many efforts to improve science teaching in higher education focus on a few faculty members at an institution at a time, with limited published evidence on attempts to engage faculty across entire departments. We created a long-term, department-wide collaborative professional development program, Biology Faculty Explorations in Scientific Teaching (Biology FEST). Across 3 years of Biology FEST, 89% of the department's faculty completed a weeklong scientific teaching institute, and 83% of eligible instructors participated in additional semester-long follow-up programs. A semester after institute completion, the majority of Biology FEST alumni reported adding active learning to their courses. These instructor self-reports were corroborated by audio analysis of classroom noise and surveys of students in biology courses on the frequency of active-learning techniques used in classes taught by Biology FEST alumni and nonalumni. Three years after Biology FEST launched, faculty participants overwhelmingly reported that their teaching was positively affected. Unexpectedly, most respondents also believed that they had improved relationships with departmental colleagues and felt a greater sense of belonging to the department. Overall, our results indicate that biology department-wide collaborative efforts to develop scientific teaching skills can indeed attract large numbers of faculty, spark widespread change in teaching practices, and improve departmental relations.

INTRODUCTION

In recent years, there have been many calls to reform the way science is taught to focus on student-centered learning, use assessment to generate evidence to improve and enhance teaching, and enable all students to be successful (American Association for the Advancement of Science, 2009; President’s Council of Advisors on Science and Technology, 2012; Singer ). Decades of studies testing various active-learning techniques have shown that these approaches consistently promote student learning when compared with traditional classroom lecturing (Freeman ). In response, many faculty have sought training in evidence-based teaching techniques. Because many universities lack internal training programs in scientific teaching for their faculty, instructors interested in improving their teaching instead must turn to external programs that offer pedagogical training to current and aspiring science, technology, engineering, and mathematics (STEM) faculty, including the Summer Institutes on Scientific Teaching sponsored by the Howard Hughes Medical Institute (HHMI; www.summerinstitutes.org; Wood and Gentile, 2003; Pfund ), the Faculty Institutes for Reforming Science Teaching (Ebert-May ), the Science Teaching Fellows Program and Biology Scholars Program sponsored by the American Society for Microbiology, On the Cutting Edge Workshops and Resources for Early Career Geoscience faculty (Macdonald ), the Workshop for New Physics and Astronomy Faculty (Henderson, 2008), and the summer institutes run by the Center for the Integration of Research, Teaching and Learning (Pfund ). Although these programs offer benefits to participating faculty (Derting ; Pfund ), they typically train only a few self-selected faculty members from an institution at a time and thus are necessarily limited in their reach and extent of impact within a department. Therefore, there is a need for models of training programs in scientific teaching that can sustainably reach a large proportion of faculty within a single department.

Introduction to Change Strategies

To convince a large number of faculty within a single department to embark on pedagogical change, it may be necessary to deploy several strategies in concert. Researchers who study change theory have found five core change strategies that are commonly used in STEM education reform efforts: disseminating curriculum and pedagogy, enacting policy, developing reflective teachers, developing shared vision, and using local science education expertise (Henderson ). They also report that using one of these strategies alone may be less helpful than employing a combination of these strategies (Borrego and Henderson, 2014). However, what implementation of these change strategies might look like in the context of department-wide transformation is still only minimally understood.

Disseminating Curriculum and Pedagogy

One strategy, disseminating curriculum and pedagogy, can help faculty who are interested in techniques they can implement in their classes immediately. However, the change literature also strongly suggests that mere dissemination is not sufficient for widespread, long-lasting change (Henderson ). A successful department-wide transformation effort probably needs to go beyond simply providing materials on innovative teaching methods.

Enacting Policy

Another strategy, enacting policy, involves influencing faculty behavior through changing elements of their environment, such as adding requirements or altering reward structures (Henderson ). Some advocates of scientific teaching argue that widespread pedagogical transformation will come only when departments and institutions require instructors to demonstrate adherence to scientific teaching practices for tenure and promotion (Wieman, 2007; Bradforth ). However, studies in many contexts both inside and outside education have shown that if the people affected by a policy change do not agree with or understand the need for that change, they can resist and thereby limit the impact of the new policies (Burke, 2002). Therefore, in addition to enacting policy changes, a successful department-wide transformation effort probably needs to cultivate widespread faculty agreement about why promoting evidence-based teaching is desirable.

Developing Reflective Teachers

A third change strategy, developing reflective teachers, emphasizes empowering faculty to make instructional choices based on reflection and consideration of assessment evidence. It has two primary manifestations: encouraging faculty to take a scholarly approach to their teaching and forming faculty learning communities in which faculty support one another (Lave and Wenger, 1991; Borrego and Henderson, 2014). When instructors take a scholarly approach to their teaching by collecting assessment evidence about what their students think, they may be dissatisfied with how much or what their students are learning, driving them to change (Grunwald and Peterson, 2003). When instructors participate in learning communities, also called “communities of practice,” with other instructors, they can talk, share examples of curricular activities, observe one another’s teaching, discuss common problems, reflect on their own practice, and push one another toward different teaching practices (Lave and Wenger, 1991; Gormally ; Corbo ; McCourt ). Decades of science education research supports the value of instructors being members of a professional community rather than isolated practitioners (Loucks-Horsley ). A successful department-wide transformation effort probably needs to provide this sort of community, one in which faculty can talk to one another about their efforts to improve their teaching.

Developing Shared Vision

In a fourth change strategy, developing shared vision, change comes from the collective action of groups within a system (Borrego and Henderson, 2014). To develop a shared vision, there must be informal communities of practice in which individuals share discoveries about their teaching to increase the knowledge of the whole organization (Lave and Wenger, 1991; Borrego and Henderson, 2014). Currently, many science faculty have professional identities that are focused around research, not teaching, which may inhibit them from talking with one another about improving their teaching and therefore from forming these communities of practice (Brownell and Tanner, 2012). A successful department-wide transformation effort probably needs to provide a department-sanctioned forum for faculty to discuss teaching, encouraging them to integrate teaching into their professional identities and to form communities of practice that may facilitate the development of shared vision.

Using Local Science Education Expertise

Yet another change strategy, using local science education expertise, involves individuals within a department who have expertise in both that department’s discipline and STEM pedagogy. These individuals usually demonstrate an understanding of the beliefs of the individuals involved and of the nature of the university as a complex system, and they can guide long-term interventions that last a semester or more. Both of these traits are characteristics that are present in the most successful teaching reform efforts (Henderson ). Examples of such individuals include a growing number of science faculty with education specialties (SFES), who are STEM pedagogy experts housed within science departments, and the Partnership for Undergraduate Life Sciences Education fellows, who are undergraduate life sciences faculty who advocate for pedagogical change (Bush ; Aguirre ). Research on SFES shows that both SFES and other faculty report that SFES have a strong impact on undergraduate teaching, specifically influencing the teaching practices of their fellow faculty (Bush ; Andrews ). However, SFES also report that, when hiring SFES, departments often overlook their potential to be involved in departmental reform, suggesting that there is a need for models that make use of their knowledge (Bush ). Another model that has been used to change teaching is to hire science education specialists, postdocs with both knowledge about teaching and a PhD in the discipline, to work one-on-one with faculty in a department to transform courses (Wieman ). One such effort, the Science Education Initiatives at the University of Colorado and the University of British Columbia, has altered the teaching of more than half of all instructors across four departments (Wieman ). However, at many institutions, hiring the necessary number of postdocs may be cost prohibitive. A successful department-wide transformation effort probably needs to leverage the expertise of individuals already in the department to help change the way other instructors teach, creating a long-lasting intervention that is sensitive to the needs and concerns of the other faculty.

Questions Arising from Our Department-wide Professional Development Program

In 2013, our Department of Biology developed a department-wide training program in scientific teaching, the Biology Faculty Explorations of Scientific Teaching (Biology FEST), funded by an HHMI Undergraduate Science Education grant and grounded in the five change strategies discussed earlier. There were many questions about how well the program would be received and what effects it would have. While the pedagogical training and support programs were designed to have the capacity to serve all instructional faculty in biology, how many instructors would choose to participate? To what extent would faculty participants report changes in their teaching, and would their self-reports be independently corroborated with other evidence such as reports from students and classroom noise analysis? How would Biology FEST impact faculty research and the departmental community? Finally, what were the primary motivations for our faculty to participate in professional development?

Here, we share evidence, insights, and answers to these questions, exploring Biology FEST as one example of a change strategy–based model for department-wide science teaching reform. After describing the multiple activities that made up Biology FEST, we present data on faculty participation and provide evidence from both faculty and students about how faculty have integrated active-learning practices into their teaching. Next, we examine the long-term impact of Biology FEST on faculty. We report the positive effect this program has had on the departmental community. Finally, we explore the motivations underlying faculty participation in these training activities, which largely did not center on compensation-related reasons.

METHODS

In this section, we describe the context and activities of Biology FEST and how we investigated its effectiveness. We first describe the faculty and department. We then describe the Biology FEST program, which consisted of workshops, scientific teaching institutes, and follow-up programs, and the leadership and policies that supported its success. We also tie specific elements of our program to the five change strategies discussed earlier. Finally, we describe the recruitment, assessment, and analyses.

Description of Faculty and Department

Biology FEST was launched with the goal of creating a voluntary program with the capacity to engage all of the biology instructional faculty in more than 100 hours of professional development in scientific teaching. Our Department of Biology has 39 tenure-track or tenured faculty and roughly 23 long-term lecturers, all of whom will be referred to as “instructors” or “faculty” here. Our institution is housed in a large urban comprehensive university with a diverse student body that is 37% first-generation college students and 45% underrepresented minorities (San Francisco State University, 2017). Our department is the second-largest department on campus, serving roughly 4600 students (∼20% of the total student body) each semester, who are enrolled in majors and nonmajors biology courses.

Overview of Biology FEST Professional Development Program

The overall intention of Biology FEST was to enhance faculty pedagogical expertise, support iterative change in biology teaching, engage faculty in regular discussions of student assessment evidence, and build infrastructure to support comprehensive teaching reform in the Department of Biology. Our ultimate aim was to support instructors in educating students as they grow into scientifically literate and curious leaders in society regardless of their eventual career path.

As opposed to focusing on disseminating a particular curriculum, we aspired to prepare biology instructors pedagogically by introducing them to scientific teaching, a framework that uses our skills as scientists to collect assessment evidence on the impact of our own teaching on student learning and to use this evidence to drive innovation and change in our practice (Handelsman ). The specific scientific framework that we implemented included four major themes (Figure 1A). The first theme was “equity and diversity,” which focused on developing strategies to build an inclusive learning environment where all students can excel. The second theme was “active learning,” which focused on implementing student-centered activities in the classroom. The third theme was “collecting classroom evidence,” which was tied closely with the fourth theme, “assessment,” which involved gathering and evaluating data on student learning in the classroom. The framework was constructed as a feedback loop, as all of these themes influence one another: evidence gathered on student learning drives classroom instruction and innovation.

FIGURE 1.

Overview of Biology FEST professional development activities. (A) Framework for scientific teaching describing iterative changes in the realms of active learning, assessment, and equity and diversity based on classroom evidence. (B) Timeline and descriptions of Biology FEST professional development activities, including workshops, scientific teaching institutes, and follow-up programs conducted in academic years 2013–2014 and 2014–2015.

Biology FEST involved several strategies to introduce instructors to scientific teaching while building a professional learning community to support collaboration and further teaching innovation (Figure 1B). We planned for at least 3 years of activities, knowing that long-term interventions have more opportunity to create long-lasting impact (Henderson ). Biology FEST had three main components: FEST workshops, the Biology FEST scientific teaching institute, and follow-up programs.

Disseminating Pedagogy: Biology FEST Workshops

The four Biology FEST workshops, implemented at the beginning of the effort in Spring 2013, were an opportunity for faculty to try out Biology FEST activities with limited commitment to the program. These 2-hour luncheon workshops disseminated pedagogy by introducing faculty to a range of teaching strategies on active learning, crafting classroom questions, effective use of clickers, and promoting student engagement. The workshops previewed the next component of Biology FEST, the scientific teaching institutes.

Developing Reflective Teachers: Biology FEST Scientific Teaching Institutes

The next component of Biology FEST was the 5-day Biology FEST institute, which provided further dissemination of pedagogy and launched professional learning communities. The format and curriculum of the institute was previously developed through a National Science Foundation (NSF)-funded program for community college instructors, and it was first offered to Biology FEST faculty in Summer 2013 and subsequently repeated four times over the course of 2 years (Figure 1B). It was taught with the principles of scientific teaching, so that instructors could experience firsthand what students might experience in a classroom taught with active learning, equity and diversity, and assessment in mind. Therefore, each day of our Biology FEST institute consisted of a variety of activities, including small-group discussions, think–pair–shares, clicker questions, jigsaws, poster presentations, and time for instructors to reflect on and translate the ideas explored each day to their own teaching contexts. We sought to model what faculty could do to support learning using simple resources, such as index cards, colored paper, name tents, and clickers, that were freely available to the faculty in our teaching resource center. Written reflections were collected in the last 15 minutes of each day of the institute to provide formative feedback to the Biology FEST leadership team. A representative participant agenda with learning goals for each day from the Winter 2014 institute is included as Supplemental Material 1. The institutes brought together faculty from across the department, both tenure-track and non–tenure track, teaching across all curricular areas in both majors and nonmajors courses. By allowing them time to discuss teaching strategies and challenges and successes in the classroom, the institutes allowed instructors to practice reflecting on their teaching and created professional learning communities.

Developing Shared Vision: Biology FEST Follow-Up Programs

To further support instructors in their implementation of scientific teaching, we created a number of semester-long follow-up programs, all of which focused on bringing together instructors to share their personal insights about teaching (Figure 1B). In the first set of follow-up programs, the Teaching Squares or Classroom Partnerships implemented in Fall 2013 and Spring 2014, faculty met in small groups with fellow instructors to discuss their teaching. In the Teaching Squares program, groups of four instructors met monthly, observed one another’s teaching, and had an opportunity to reflect on their own teaching practices by viewing video-recordings of themselves teaching. A different subset of faculty had the opportunity to participate in another follow-up program, Classroom Partnerships, which provided funds to support a teaching partner, either a postdoctoral fellow or a graduate student, in the classroom. These partnerships provided intellectual support for faculty to assist them in translating scientific teaching approaches and helped them to collect evidence about student learning in their classrooms.

In the second set of follow-up programs, implemented in Fall of 2014 and Spring of 2015, faculty were encouraged both to become part of a professional learning community and to develop a more scholarly approach to their teaching through collecting evidence of classroom learning and engagement (Figure 1B). One follow-up program, Changing Minds, was developed in response to faculty requests for more support in assessment. It convened all participating faculty every other week to collaboratively work on assessment, with the goal of collecting and analyzing four or more sets of pre- and post­assessment evidence of student learning in their courses by the end of each semester. In the second follow-up program, Talk Matters, faculty, in collaboration with postdoctoral fellows, collected direct evidence of teaching innovations by audio-recording all class sessions in one or more of their own courses. Transcripts and audio levels of the recordings were analyzed, and the results are being disseminated in conferences and peer-reviewed publications, with all participating faculty as coauthors (Owens ). By the end of the Biology FEST effort, faculty who had gone through both a scientific teaching institute and a follow-up program had spent nearly 100 hours discussing teaching with their fellow faculty, which provided the necessary time and space to communally share and reflect on their teaching and to allow a shared vision to emerge.

Using Local Science Education Expertise: Biology FEST Leadership

Biology FEST took advantage of local science education expertise in the form of its two coleaders, an SFES and the department’s associate chair, who was responsible for overseeing the curriculum. Both were long-standing members of the department, were trained as researchers in the discipline, and had deep connections with other faculty members. Therefore, Biology FEST activities took into account the beliefs of the individuals involved and the nature of opportunities and constraints present in our university and department with regard to teaching (Henderson ). In addition, a postdoctoral fellow was hired to provide program coordination and support.

Enacting Policy: Support from Departmental and Campus Leadership

The project had the visible backing of departmental and university leaders, who reiterated that biology education research would be considered equivalent to basic biology research when considering promotion and tenure policy. The associate chair of the Department of Biology was the co–principal investigator of the HHMI grant, and the chair of the department attended a scientific teaching institute and follow-up program meetings. In addition, the department chair, dean, provost, and university president all contributed strong letters of support for the grant. In those letters, they made clear that, per the tenure and promotion criteria in our department, “scholarly grants and peer-reviewed publications in biology education would be held in as high esteem in the tenure and promotion process as those same types of activities in basic science research.” Faculty also knew that the department’s resident SFES had recently achieved tenure based on biology education research alone. However, no other criteria for tenure and promotion for tenure-track and tenured faculty and for retention of lecturers were altered, and no instructors were required to participate in Biology FEST, as we felt we would achieve more grassroots support with a voluntary approach. Although Biology FEST did not focus on enacting policy as its primary change strategy, the changes made did make it clear to us that our scholarly efforts in our teaching would be valued.

Recruitment, Assessment, and Analyses

We took a mixed-methods research approach, which included quantitative and qualitative approaches. We received approval through the San Francisco State University (SFSU) Human and Research Protections Protocol #E14-197.

Instructor Recruitment

All instructional faculty and instructional support staff in the SFSU Department of Biology were invited to participate in Biology FEST activities. It is important to note that we consciously chose to make no distinctions during Biology FEST between tenured/tenure-track and non–tenure track faculty. As adjunct faculty in general often feel isolated from their departmental communities and in need of professional development opportunities, we sought to include non–tenure track faculty as equal peers (Eagan ).

Participating instructors, regardless of their position within the university, were provided the same monetary stipend for attending a Biology FEST institute. In addition, instructors who chose to participate in a follow-up program were provided release time for future credit toward one course, and subsequent participation in follow-up programs resulted in a choice of either summer salary or additional release time from teaching one course in the future.

Biology FEST Final Reflection Analysis

To learn what changes the faculty reported implementing in their classrooms, at the end of the semester following institute participation, we asked faculty to reflect on the extent to which the institute affected their teaching. Faculty were asked to write a final reflection that was 1200–1800 words in length by responding to what extent they agreed or disagreed with the following statement: “I have changed the way I teach as a result of my institute experiences.” Forty-six final reflections were analyzed, representing 86.8% of Department of Biology participants. Final reflections were analyzed with a grounded theory approach. They were assigned random pseudonyms and then read in their entirety for emergent themes. Quotes were input to a spreadsheet that contained quotes from all faculty as well as the emergent themes. A separate spreadsheet tracked whether individual faculty mentioned each of the themes. Emergent themes were then calculated as percentages of faculty who mentioned that theme out of the 46 total faculty reflections.

Resource Center Usage

To give an estimate of the number of instructors in the department who had implemented active learning, we tracked the number of instructors visiting the resource center, which is open to anyone in the Department of Biology and gives out materials for free that are often used in active learning, such as index cards, colored cardstock for name tents, markers, and clickers. The resource center documents each visit by completing a loan slip that details contact information and resources borrowed. All available slips, starting from the inception of the resource center in 2005, were categorized based on whether the individual was a Department of Biology instructor. Visits per individual for each academic year were then calculated, with academic year defined as August of the first year to July of the following year.

Faculty/Student Perceptions of Biology Classrooms Survey

As another way of quantifying the number of instructors in the department who used active learning, we conducted surveys of faculty and student perceptions of biology classrooms. These surveys were conducted in the last third of the Spring semester in the 2014–2017 semesters, which were, respectively, at the end of the first through fourth academic years after the first Biology FEST institute. Unfortunately, baseline data from before the effort are unavailable.

The purpose of the Faculty Perceptions of Biology Classrooms survey was to determine the extent to which instructors in the department perceived themselves using active-learning techniques. All instructors teaching a course that semester, whether they had attended a Biology FEST institute or not, were invited to take the Faculty Perceptions survey. The version of the survey given in 2014 is included as Supplemental Material 2; the surveys given in subsequent years were the same, except for updates to the listings of courses. Briefly, as a subset of the survey, instructors were asked closed-ended questions about how often various pedagogical techniques such as “traditional lecture” and “small-group discussion or think–pair–share” were used in their courses on a per course basis. Respondents could indicate how frequently each technique was used. These questions were taken from a previous study (Macdonald ). For the purposes of this paper, responses were collapsed into the following categories: “weekly or more,” “less than weekly,” and “never.” The remainder of responses were “don’t know,” which were not included in reported data, as they made up a very small proportion of the responses (five responses or fewer for each category). Instructors received a gift card for each course for which they completed the survey.

To complement the Faculty Perceptions survey, we also created a corresponding Student Perceptions of Biology Classrooms survey that asked students the same questions about their classes. After obtaining instructor permission, we contacted students and invited them to complete the survey for each biology course in which they were enrolled. Students were entered in a raffle for prizes once for each biology course in which they were enrolled and for which they filled out the survey. The Student Perceptions survey asked the students the same questions about the frequency of usage of various active-learning techniques in their classrooms as the Faculty Perception survey did, allowing us to compare faculty and student responses. The version of the survey given in 2014 is included as Supplemental Material 3; again, the surveys given in subsequent years were the same, except for updates to the listings of courses and faculty. Student responses were removed if they were incomplete or if the same individual completed multiple responses for the same course. Student responses were also excluded from analysis if the instructors either did not complete the survey or did not give permission to contact their students. The Student Perceptions survey was sent out in the last 2 weeks of the semester, which meant that that the courses were still fresh in students’ minds and that they had experienced enough of the courses to answer the questions on the survey.

Bonferroni-corrected statistical significance was calculated by chi-square analysis in Excel (Microsoft) and JMP, version 10.0 (SAS Institute). Effect sizes were calculated with Cramér’s V in Excel. Approval for this portion of the study was conferred by SFSU Human and Research Protections Protocols #E14-197, X13-29, and X16-28.

Investigation of Classroom Noise Using Decibel Analysis for Research in Teaching

To obtain an objective estimate of the amount of active learning in the classes of Biology FEST alumni, we used the Decibel Analysis for Research in Teaching (DART) tool, which is a machine-learning-based tool that can quickly analyze classroom audio recordings with near 90% accuracy (Owens ). In Spring 2015, all Biology FEST alumni teaching a course that semester were invited to participate in a follow-up program in which they would audio-record their courses. Participating faculty were given audio recorders (Sony, model ICD-PX333) and asked to record every class session of at least one of the courses they were teaching that semester. They were instructed to place the audio recorders at the front of the classroom (e.g., on a lectern) with the microphone pointing in the general direction of students. For this analysis, we included only nonlaboratory biology courses for which at least 30% of class sessions were recorded. Recordings were trimmed by hand at the beginning and end to exclude noise associated with student arrival and departure. Trimmed class-session recordings were processed with the DART algorithm, and DART data were analyzed with Excel.

Approval for this portion of the study was conferred by SFSU Human and Research Protections Protocols #E14-141a-d. Courses presented in this analysis were part of the aggregated data shown in Owens but have been disaggregated for publication here.

Motivations Survey

To obtain the faculty’s perceptions of the long-term effects of Biology FEST participation and their motivations for participating in Biology FEST, we created a motivations survey asking about these topics. The survey is included as Supplemental Material 4. In the Fall of 2015, all faculty who had attended an institute and taught at SFSU were asked via email to participate in this survey.

To learn their perceptions of the effects of Biology FEST, we used closed-ended questions probing various possible outcomes in the survey. The full text of the questions is included in Supplemental Material 4. Briefly, the questions were of two types. In the first type, participants were asked to select a response to complete the following sentence: “As a result of my participation in HHMI Biology FEST, I feel that my (attribute) has ___.” The answer choices were “increased,” “not changed,” and “decreased.” For chi-square analysis, because of the paucity of “decreased” responses, “not changed” and “decreased” responses were collapsed into a single category called “decreased/not changed.” In the second type of question, participants were asked to select a response to complete the following sentence: “As a result of my participation in HHMI Biology FEST, I feel that my (attribute) has been ___ affected.” Because of a paucity of “negatively” and “strongly negatively” responses, some responses categories were collapsed, so that the analyzed categories were “strongly positively/positively,” “strongly negatively/negatively,” and “not applicable.” Statistical significance was calculated by chi-square analysis, and effect sizes were calculated with Cramér’s V in Excel.

To learn their motivations, we asked faculty to answer the following question: “What were the primary reasons you INITIALLY decided to participate in HHMI Biology FEST?” Faculty who had indicated that they had participated in any follow-up program were also asked, “What are the primary reasons you have CONTINUED to participate in HHMI Biology FEST?” Motivations were analyzed using grounded theory for emergent themes.

RESULTS

Our analysis has yielded eight key findings concerning the effects of Biology FEST, a professional development effort grounded in five change strategies. First, we will show instructor participation rates. Next, we will provide evidence from a variety of sources—including instructor self-report, resource center usage, student report, and classroom noise analysis—supporting that Biology FEST participants used active-learning techniques in their teaching. Finally, we will discuss instructors’ perceptions of the long-term effects of the Biology FEST scientific teaching institute and follow-up programs on their professional lives and their motivations for participating in Biology FEST.

Nearly 90% of SFSU Biology Faculty Have Participated in Biology FEST

Although Biology FEST was designed to accommodate the entire department, it was not clear how many instructors would choose to spend extended periods of time engaged in professional development in scientific teaching. We found that faculty participation in the Biology FEST institutes and follow-up programs was strong. Out of the total of 39 tenured and tenure-track faculty, 90% (35 of 39) participated in a 5-day Biology FEST institute (Table 1). In addition, 87% of lecturers (20 of 23) participated, giving an overall departmental participation rate of 89% (55 of 62 instructors). Similarly, there was strong participation in follow-up programs, especially given the significant time commitment required for these activities. For follow-up programs offered in 2013–2014 and 2014–2015, we had a total of 84 and 81% of eligible instructors involved, respectively (Table 1). The high faculty participation rate exceeded what is often predicted in the literature on faculty resistance to innovative teaching (Brownell and Tanner, 2012; Tagg, 2012).

TABLE 1.

Biology faculty participation in Biology FEST and follow-up program activities, 2013–2015

Biology faculty	Total (n)	Biology FEST scientific teaching institute participation % (participants/eligible total)	Follow-up program 2013–2014 participation % (participants/eligible total)	Follow-up program 2014–2015 participation % (participants/eligible total)
Tenured/tenure-track faculty	39	90 (35/39)	85 (22/26a)	89 (25/28b)
Lecturer faculty	23	87 (20/23)	82 (14/17c)	70 (14/20)
Total faculty	62	89 (55/62)	84 (36/43)	81 (39/48)

aSeven tenured/tenure-track faculty participated in later institutes and thus were not eligible to participate in follow-up Programs in 2013–2014. Two were administrators and thus do not teach (35 − 9 = 26 eligible).

bFour tenured/tenure-track faculty were on sabbatical, and three were administrators who were not teaching (35 − 7 = 28).

cThree lecturers participated in later institutes and thus were not eligible to participate in follow-up programs in 2013–2014 (20 − 3 = 17).

One Semester after Completing the Biology FEST Institute, Participants Reported Changes in Their Teaching

Analysis of postassessment evidence collected from faculty one semester after participating in the 5-day Biology FEST institute suggests that faculty implemented changes in their classroom instruction. Representative quotes from faculty on the impact of the institute on their teaching are provided in Table 2. These changes could be classified broadly as adding active-learning strategies, adjusting assessment strategies, implementing strategies to embrace equity and diversity, and modifying lesson plan strategies. In particular, a striking 80% of the faculty reported adding active-learning strategies in their classrooms (Table 2). One example of this category comes from Casey (all names used here are pseudonyms):

TABLE 2.

Sample quotes from Biology FEST institute final reflections show that faculty implemented scientific teaching principles in their classes in a variety of waysa

Theme	Quotes
Active learning (80%)	Before this last year, I did more lecturing that did not include activities or points where the students could discuss or problem solve together. This semester I tried to incorporate different strategies to help them DO things in class to learn concepts in a different way.—Juanita
	This past fall semester I challenged myself by trying one student-centered activity every class session. At first I thought this would be difficult, but once the rhythm of the class got going doing simple think–pair–shares seemed routine.—Liana
	This time, however, I devised several hands-on activities to explicitly form links from concepts that at times seem abstract to students … to the profoundly concrete molecules that these concepts describe.—Todd
	I completely changed the seminar from the primarily lecture-based format it has had for the past two years to one that was really more workshop oriented.—Hugh
Assessment (60%)	I have changed the way I run my class meetings to routinely include evidence-gathering activities about student learning.—Frida
	With [this person’s] help, we did all types of assessment including developing survey monkeys to poll the students both pre- and post[experience].—Casey
	For the first time I used cartoons to identify and correct misconceptions and I also used student work to correct misconceptions.—Angela
	I started doing more short written responses and made much more of an effort to incorporate those responses into the subsequent classes.—Stefan
Equity and diversity (53%)	I decided to change my approaches and learned to use real life events that are relevant to the learning subject and student life to connect with everyone in the class.—Ernie
	I have always had a significant component of discussion structured around open-ended questions or problems, but I have been more conscious of ensur[ing] that we hear from all students, especially those that appear more reluctant to participate.—Gavin
	I introduced mandatory nametags (with spaces for written reflections for each class meeting and a “top ten” list at the end) in order to learn more students’ names and to increase participation.—Angela
	I spoke explicitly in class about the value and importance of everyone’s contribution. I replaced unstructured group discussions (which tend to further marginalize those starting at even a slight disadvantage), with highly structured, goal-oriented activities, in which I assigned roles such as recorder, timekeeper and a moderator to help ensure everyone had a voice.—Quintin
Lesson-planning strategy (40%)	I have started to write lesson plans to have a clearer idea of how to mix lectures and activities and, if I have time, to write some notes after each class.—Fausto
	In redesigning this course, I started with developing goals for each week[‘]s worth of lectures.—Casey
	This semester I worked on using the 5-E model; specifically I tried to add more engagement to the beginning of a section and to, in some cases, move the explain portion of the module after the explore portion.—Becky
	I used a “backwards design,” which, surprisingly, I had never done.—Hans

aQuotes taken from Biology faculty final reflections submitted after completion of Biology FEST scientific teaching institute in Summer 2013 or Winter 2014 and one semester of teaching implementation. Quotes are categorized into themes; percentage reflects the number of faculty who mentioned said theme in their reflections. Names are pseudonyms.

For the first time in the large course I have used both TPS (Think–pair–share) questions and clicker questions and these have been successful in engaging the students and having them discuss the topic with each other. I have also used this as a tool for having the students work on their critical thinking skills.

The next most common category was making adjustments to their assessment strategies, reported by 60% of the faculty. Hans wrote,

I generated a formative pre-assessment for each topic, and had students complete these either at the end of the class period BEFORE that topic is presented in class or on [our learning management system] during the days before class. The goal of this assessment was to identify student misconceptions regarding the topic so that I could tailor my instruction to address those misconceptions… Each day I asked them to compose a short reflection evaluation statement about what they found most interesting or compelling about the day’s class.

Another important goal of Biology FEST was to inspire faculty to examine equity and diversity in the classroom. Indeed, 53% of the participants reported making changes in the classroom to address equity and diversity issues. The following is a representative quote by Vince:

To attempt to rectify any feeling of alienation in my class, I [assured] my students that the qualities of a successful scientist were unrelated to race, ethnicity, gender, but were directly a function of desire and scholarship. I ensured that my visual presentations and oral stories included reference to colleagues of varying gender, color, age, and ethnicity.

Finally, 40% of the participants also mentioned making modifications to their lesson plans. An example of this is the following, said by Magdalen:

In [class], I reduced number of topics covered and focused on fundamentals. I also gave more time for students to sink their teeth into these areas, especially by giving more time to work on their independent projects.

Multiple Types of Evidence Suggest Biology FEST Faculty Use Active-Learning Techniques in Their Teaching

Because previous research has found that instructors who undergo professional development may overstate the extent to which they have “reformed” their courses (Ebert-May ), we sought ways to corroborate Biology FEST alumni’s self-reports of changes in teaching practice. The most common claim that instructors made with regard to pedagogical change was that they added active learning to their classes (Table 2). To verify this assertion, a strategy that one could use could be to directly observe a large number of classes taught by Biology FEST alumni and compare their use of active-learning techniques before and after participating in an institute. However, with 62 instructors, we did not have the capacity or resources to observe so many classes, nor were we able to collect baseline data of instructors before they participated in Biology FEST. Therefore, we relied on three indirect measures of the implementation of active-learning practices in the department: 1) usage of the resource center, 2) results of a survey of biology students and instructors that asked about teaching techniques in their classes, and 3) analysis of classroom noise levels in a large number of biology classes.

Instructors Increased Their Use of the Resource Center

Analysis of the number of Department of Biology instructors using the resource center shows a sharp and sustained increase in usage in the academic years after the first scientific teaching institute (Figure 2). The resource center is open to anyone in the Department of Biology and gives out materials that are often used in active learning, such as index cards, colored cardstock for name tents, markers, and clickers. The use of these materials in active learning is modeled during the institute, when it is also advertised how the resource center can help support teaching. When instructors pick up materials, they must fill out a checkout slip with their name, date, and requested materials. We counted the number of distinct Department of Biology instructors who checked out materials in each academic year since the resource center was established in 2005. In the years before the first Biology FEST institute, usage had stabilized at around 20 distinct Department of Biology instructors per year (Figure 2). After the first institute, usage increased to more than 50 distinct Department of Biology instructors per year, an increase that has been sustained for 3 years (Figure 2). This figure may be an underestimate for several reasons. First, some instructors send a teaching assistant or other proxy to check out materials, which would mean that the instructor’s name would not be recorded. Second, some instructors prefer to use their own materials. Third, many forms of active learning do not involve any physical materials. Fourth, in more recent years, the department’s own stockroom, a supply warehouse that does not charge instructors and has traditionally only stocked supplies related to basic science research, began stocking the most popular materials used in teaching, so instructors had alternate sources. While resource center usage is an indirect measure of classroom active learning, the rise in usage suggests a sustained increase in the number of instructors using these teaching techniques.

FIGURE 2.

Resource center usage increased sharply after the first scientific teaching institute and persisted multiple years later. The number of individual biology faculty who requested materials from the resource center for each academic year (August–July). Instructors were only counted once per academic year, regardless of how many times they requested materials. Arrow marks timing of the first Biology FEST scientific teaching institute in Summer 2013.

Students’ Perceptions of Use of Active-Learning Techniques

In my 4th year at SF State, I have seen a huge improvement in the equity climate in my science learning … using this semester as an example, most of my biology classes operate in similar ways that feel more equal than semesters before. In 3 of my 5 classes, we use name cards with reflections. We use clickers. And we perform a lot of think–pair–shares. All 5 instructors actively try to connect our learning to outside world examples (via clinical scenarios, jokes, cartoons, articles, etc.), and instructors are really trying to encourage students not to be afraid of the material, to try endless number of ways to learn the material (post videos, animations, practice problems, practice exams, etc.), and to understand concepts, not memorize details.—Biology undergraduate student, Fall 2015

Results from a department-wide survey of biology students also suggest robust use of active-learning techniques by Biology FEST alumni. In paired surveys of student and faculty perceptions of college biology classrooms conducted at the end of the first through fourth academic years after the first Biology FEST institute, we surveyed both Biology FEST–participating and nonparticipating faculty and their students on the frequency of usage of various pedagogical techniques, including traditional lecture, small-group discussion or think–pair–share, and in-class exercises. These questions were taken from a previous study (Macdonald ). Faculty response rates in 2014, 2015, 2016, and 2017 were 67, 72, 73, and 80%, respectively. Response rates were slightly lower for instructors who had not participated in Biology FEST than for instructors who had; in 2014, the only year with high numbers of non–Biology FEST participants, the response rates were 75% for Biology FEST participants and 56% for nonparticipants. As part of the Faculty Perceptions survey, faculty were asked whether their students could be contacted to complete the Student Perceptions survey. Nearly all participating instructors agreed, with agreement rates of 97% in 2014, 100% in 2015, 90% in 2016, and 96% in 2017. For the students contacted, student response rates in 2014, 2015, 2016, and 2017 were 30, 35, 26, and 34%, respectively.

Analysis of the Faculty and Student Perceptions surveys revealed that the student responses corroborate instructor reports of the frequency of the use of active-learning strategies. For both Biology FEST participants and nonparticipants, the percentage of students reporting usage of various teaching techniques echoed that of faculty, with both reporting the majority of courses using traditional lecture at least weekly and roughly half of courses using small-group discussion or think–pair–shares at least weekly (Figure 3, A and B).

FIGURE 3.

Students corroborated faculty pedagogical reports of the use of scientific teaching. (A) Faculty responses from survey from end of first academic year after the first institute (Spring 2014). Non-FEST, faculty who did not attend an institute; FEST, faculty who did attend an institute. (B) Student responses from survey from the end of the first academic year after the first institute. FEST, students of faculty who had attended an institute; non-FEST, students of faculty who had not attended an institute. **, p = 0.002; ***, p < 0.0001 for χ2 analysis.

Results from the first year of the survey suggest that Biology FEST participants used active-learning techniques more frequently than nonparticipants did. Students taking courses from Biology FEST alumni (n = 412) reported significantly more use of the active-learning strategies “small-group discussion” and “in-class exercises” than students taking courses from non–Biology FEST instructors (p < 0.0001, n = 113; Figure 3B). The effect sizes for these results, as measured by Cramér’s V, are 0.36 and 0.32, respectively, which can be interpreted as a moderate association between FEST status and frequency of learning strategy usage (Cohen, 1977). This finding echoes instructor self-reports, as captured through their postassessments (Table 2), that they had added active learning to their classes in the semester after a Biology FEST institute. Faculty who participated in Biology FEST (n = 29) also reported using active-learning techniques at higher frequencies than non–Biology FEST participants (n = 6; Figure 3A), although these differences were only statistically significant for in-class exercises (p = 0.002; effect size as measured by Cramér’s V of 0.59, a strong association; Cohen, 1977). Neither students nor faculty reported differences between FEST and non-FEST instructors in the use of traditional lecture, which was commonly used in both groups. In later years of the survey, because so many faculty had attended a Biology FEST institute, the number of responses from students taking courses taught by non–Biology FEST instructors was too low (n = 33 for 2015, n = 26 for 2016, n = 22 for 2017) for similar comparisons between FEST and non–FEST students and faculty.

Student reports of active learning in the courses of Biology FEST alumni persist across 3 years. We compared Student Perceptions survey responses from 2014 to 2017 from students taking courses from any Biology FEST alumni. In these years, the frequency of any of the teaching techniques analyzed (traditional lecture, small-group discussion, in-class exercises) did not differ between years, revealing that the overall use of active-learning strategies implemented by Biology FEST faculty was sustained for at least 3 years (Figure 4). In comparison, previous research on faculty who adopt innovative teaching techniques found that as many as one-third of faculty who try research-based instructional strategies such as peer instruction or cooperative group problem solving discontinue these practices in a short amount of time (Henderson ). In summary, these results reveal that the overall use of active-learning strategies implemented by Biology FEST faculty has not declined.

FIGURE 4.

Student reports of the use of active learning persists across 4 years. Student responses from any faculty who had taken an institute in the first through fourth years (Spring semesters 2014–2017) after the first scientific teaching institute.

Classroom Noise Analysis Indicates Widespread Usage of Active Learning

To obtain an objective estimate of the amount of active learning occurring in the classrooms of Biology FEST participants, audio recordings of their class sessions were analyzed with the DART tool (Owens ). In Spring 2015, we invited all Biology FEST alumni who were teaching a course that semester to be part of a follow-up program in which they audio-recorded each class session of at least one of their courses. Of the 42 instructors invited, 31 participated by recording at least one class session of one course, for a participation rate of 74%. These instructors recorded a total of 42 courses, as many instructors taught multiple courses. However, in this analysis, we only included nonlaboratory biology courses for which at least 30% of class sessions were recorded. Therefore, seven courses were excluded for having low numbers of recordings and three courses were excluded for being labs, yielding 32 analyzed courses for an inclusion rate of 32/42 = 76%.

DART analysis showed that the majority of Biology FEST participants regularly use active learning in their courses. DART estimates with almost 90% accuracy the amount of time spent in a class session in each of three modes: single voice, with one person speaking; multiple voices, with many people speaking simultaneously; and no voice, when the classroom is quiet (Owens ). Single voice is usually associated with traditional lecturing, while multiple voice and no voice are typically indications of activities associated with active learning such as small-group or pair discussions (multiple voice) or silent writing (no voice; Owens ). To determine whether the courses taught by Biology FEST participants contained any active learning, we calculated the percentage of class sessions within each course that included any multiple or no voice (i.e., were <100% single voice). Although only 22% of courses (7/32) had active learning in every class session, 81% of courses (26/32) had active learning in at least half of their class sessions (Figure 5). As DART is a conservative estimate of the amount of active learning, these results suggest that most Biology FEST participants incorporated active learning into most of their class sessions (Owens ).

FIGURE 5.

Classroom noise analysis shows that most Biology FEST participants use active learning in at least half their class sessions. Proportion of courses taught by Biology FEST follow-up program participants in Spring 2015 (n = 32) where all class sessions have some multiple or no voice (<100% single voice) (left) and where at least half of all class sessions have some multiple or no voice (right).

The Majority of Faculty Reported Positive Outcomes from Participating in Biology FEST

To investigate the long-term effects of Biology FEST on not only the instructors’ teaching but also on other aspects of their professional lives, we surveyed all instructor participants on how Biology FEST has affected their teaching, research, and feelings of departmental community 3 years after beginning the Biology FEST effort. Of the 62 participants contacted, 50 responded, giving a response rate of 81%.

Instructors Reported Positive Effects on Teaching

As expected, instructors who participated in Biology FEST generally reported positive effects on their teaching (Figure 6A). The vast majority of participants selected “strongly positively” or “positively” when asked how their teaching or their confidence in their teaching had been affected (96 and 84%, respectively; Figure 6A). Only one instructor (2%) thought his/her teaching had been negatively or strongly negatively affected. A slightly higher percentage, 6%, thought their confidence in teaching had been negatively or strongly negatively affected, but this percentage is still low given that the program challenged some traditional methods of teaching. Importantly, 84% of respondents said that their willingness to reflect on and make changes in their teaching and their willingness to take risks with their teaching had increased (Figure 6A).

FIGURE 6.

The vast majority of participants reported positive effects on their teaching and no negative effects on their research. (A) Faculty views on how participation in Biology FEST affected various aspects of their teaching. (B) Faculty views on how participation in Biology FEST affected their research.

Instructors Did Not Report Negative Effects on Research

Our results also suggest that faculty did not believe that participation in Biology FEST negatively affected their research programs. At our institution, tenured and tenure-track faculty are expected to spend a significant amount of time conducting research and obtaining external grant funding. Because previous studies have suggested that interactive teaching methods can require more preparation time than traditional lecture (Miller ; Hanson and Moser, 2003; Pundak and Rozner, 2008), we had hypothesized that participation in Biology FEST might negatively affect instructor research productivity. However, when participants were asked how participation in Biology FEST had affected their research, only 6% (three people) thought that Biology FEST had negatively or strongly negatively affected their research (Figure 6B). The majority of instructors (63%) selected “not applicable.” One reason for this response may be that the respondents include lecturers who do not have research programs. Unfortunately, we did not ask respondents about their job categories (i.e., lecturer vs. tenured or tenure-track) and thus cannot correlate their responses with their research responsibilities. More strikingly, 31% of respondents believed that their research had been affected “positively” or “strongly positively,” even though our professional development did not address research at all (Figure 6B).

Instructors Reported Positive Effects on Departmental Community, Particularly Those Participating in Follow-Up Programs

Because theories of change suggest that professional learning communities where groups of instructors grapple with similar teaching challenges are key for promoting ongoing pedagogical growth (Lave and Wenger, 1991; Loucks-Horsley ), we wanted to know the extent to which participants felt that Biology FEST improved their relationships with their peers and the department. The majority of instructors felt that participation in Biology FEST increased their interactions with departmental colleagues around teaching (80%; Figure 7). Moreover, their sense of belonging in their department (76%) and relationships with departmental colleagues (84%) were affected positively or strongly positively. This effect was significantly stronger for respondents who had participated in a follow-up program compared with those who just participated in an institute (by χ2 analysis, p < 0.001 and Cramér’s V of 0.47 for interactions with colleagues; p = 0.010 and Cramér’s V of 0.43 for sense of belonging; p = 0.001 and Cramér’s V of 0.52 for relationships with colleagues; Figure 7). No such differences between instructors who had participated in a follow-up program and ones who had only attended an institute were seen in response to questions asking about teaching or research (unpublished data). Although it is likely that instructors seeking a stronger sense of community disproportionately chose to participate in follow-up programs, these results suggest that implementing longer-term professional development programs could successfully strengthen departmental community.

FIGURE 7.

Participants in follow-up programs reported more strongly positive effects on departmental community than people who only attended an institute. Respondents were disaggregated by whether they only attended an institute (Institute Only) or also attended a follow-up program (Follow Up Program). *, p = 0.01; **, p = 0.001 for χ2 analysis.

Faculty Were Motivated to Participate in Biology FEST by Teaching- and Community-Related Goals

Given that achieving department-wide professional development necessarily involves incentivizing large numbers of instructors to participate, we were curious about the reasons participants would provide to explain why they would spend a week or more (in the case of follow-up program participants, significantly more) of their time participating in Biology FEST. Therefore, as part of the survey discussed earlier, we asked instructors to give two reasons for why they initially participated. In addition, we asked individuals who had done a follow-up program why they had continued to participate in Biology FEST. We hypothesized that a primary reason driving participation might have been formal compensation, in the form of stipends, summer salary, and release time.

Contrary to our hypothesis, the majority of instructors did not cite formal compensation as one of their top two reasons for either initially participating in Biology FEST or for participating in a follow-up program (Figure 8). Instead, the reasons most people mentioned could be categorized into one of two categories: teaching-related and community-related. For example, the responses “I had had no formal teaching training and was interested in improving my teaching skills” and “I wanted to interact more effectively with students in my large lecture course” were categorized as teaching-related; and the responses “I wanted to join colleagues in talking about our teaching challenges and learn together how to creatively come up with solutions” and “I had heard good things about the HHMI [FEST] from others in the department” were categorized as community-related. For initial participation, 92% of instructors cited teaching-related reasons and 44% cited community-related reasons (Figure 8). For continuing participation, 75% of instructors cited teaching-related reasons and 67% cited community-related reasons. In both cases, compensation-related reasons were cited by only a minority of instructors, 16% for initial participation and 22% for continuing participation (Figure 8). These results suggest that, although formal compensation may be important for some participants, especially for those joining higher-commitment, longer-term programs, it was not crucial for the majority of instructors.

FIGURE 8.

Biology FEST participants reported mostly non–compensation related reasons for participating. Percentage of respondents who gave various types of reasons for either their initial participation in Biology FEST (all respondents, n = 50) or their continuing participation in Biology FEST (only respondents who participated in a follow-up program, n = 36).

DISCUSSION

Our department’s experience with Biology FEST demonstrates that it is possible to engage nearly 90% of the instructors in one department, including adjunct, tenure-track, and tenured faculty, in intensive professional development in scientific teaching and use measures beyond self-report to estimate their use of active-learning techniques. In addition, our results demonstrate that these efforts have benefited the faculty in multiple ways, including ways that were unexpected, such as building a greater sense of collegiality and community. They also suggest that our faculty’s participation was motivated primarily by reasons related to teaching and community, not formal compensation. Overall, our results indicate that department-wide efforts to collaboratively develop scientific teaching skills that are grounded in change theories can attract large numbers of faculty, spark change in teaching practices, and improve departmental relationships, arguing for department-wide involvement as a strategy for change. Below, we present our professional development effort within the context of formal theories of change, discuss some caveats and limitations to this work, and describe implications for future professional development efforts in our department and at other institutions.

Linking Biology FEST Outcomes to Theories of Change

Biology FEST drew upon several theories of change in concert to engage participants and encourage them to use more elements of scientific teaching. Two of the change strategies, disseminating pedagogy and developing more reflective teachers, served to attract and provide benefits to faculty in our program. Disseminating pedagogy was necessary for initially drawing in the vast majority of participants who were interested in improving their teaching skills, as nearly everyone cited teaching-related reasons for attending an initial institute (Figure 8). This dissemination also gave faculty examples of teaching strategies that they could implement almost immediately in the semester following an institute (Table 2). In contrast, developing more reflective teachers through creating faculty learning communities and developing shared vision drew in participants who sought out community, which was particularly important for sustaining engagement with Biology FEST through our follow-up programs. While less than half of participants gave community-related reasons for participating in an institute, two-thirds of participants cited community-related reasons for participating in follow-up programs (Figure 8). Follow-up program participants were also more likely to feel more strongly connected to their department and colleagues (Figure 7). Our results, which are in concord with other research on the benefits of faculty learning communities to instructors and the students they teach, indicate that faculty valued and sought out the community that we formed around teaching (Cox, 2001).

In contrast to change strategies that directly attracted participants to Biology FEST, other change strategies served to set the stage for the success of the program. Enacting policy, which required eliciting the clear support of departmental and university officials, ensured that participants knew that they would not be explicitly or implicitly penalized for engaging in our program. In fact, they knew that extensive participation would be rewarded through authorship of papers that could count for promotion, tenure, or retention. The use of local science education experience within the department allowed Biology FEST to be grounded in disciplinary relevance and identity, which research has shown is important to the success of change efforts (Loucks-Horsley ; Henderson ). The fact that the leaders of Biology FEST were embedded in the department and had substantial expertise in both biology and science education gave those leaders legitimacy. It meant that participants knew the leaders as colleagues and trusted that they had an understanding of issues in the discipline. It was also less costly than an approach that required hiring many experts to implement change, which made this effort more feasible for our institution.

Moving Away from the Faculty Deficit Model

The results from Biology FEST question one of the implicit assumptions behind many reform efforts, what we call the faculty deficit model. A student deficit model posits that the difficulties students face in classes arise from deficiencies within individual students, such as a lack of ability or drive, rather than problems in the education system or society as a whole (Campbell, 1996; Ford and Grantham, 2003; Valencia, 2012). Similarly, a faculty deficit model assumes that the reasons that instructors do not adopt new teaching practices lie solely within those individual instructors, such as a lack of motivation around teaching or resistance to teaching differently (Brownell and Tanner, 2012; Tagg, 2012). Our results suggest that the educational reform community could benefit from higher expectations of faculty interest in teaching and being ambassadors of their disciplines to their students. One possible source of low expectations of faculty comes from research suggesting that faculty tend to overestimate how student-centered their teaching is (Ebert-May ). However, in our study, the concordance between the Student and Faculty Perceptions survey results suggests that faculty can be both honest and relatively accurate in estimating how often they use research-based teaching techniques. More importantly, there seems to be a focus on faculty resistance as a major barrier to the widespread implementation of scientific teaching (Wieman, 2007; Brownell and Tanner, 2012; Tagg, 2012). However, we found that nearly all faculty in our department, whether non–tenure track, tenure-track, or tenured, were quite eager to learn how to improve their teaching and to form a community centered around teaching. Finally, many people, including us initially, have hypothesized that faculty would require monetary stipends to participate in extensive professional development, greatly increasing the cost of such efforts. We did not collect data specifically on the stipend as a significant motive, and we recognize that it might have been a factor influencing faculty participation. Nevertheless, the faculty did not cite the monetary stipend as a major motivator. Other researchers studying faculty motivation for participation in professional development efforts have also found that their faculty do not cite formal compensation as a driving factor (McCourt ). Overall, our study highlights the importance to future professional development efforts of not making assumptions about the ability and motivations of faculty participants.

Caveats and Limitations

Although our results suggest that Biology FEST helped faculty change their teaching, we cannot definitely state that it did so. Like many other reform efforts, we were unable to collect baseline data on how instructors taught before attending an institute. Such is the nature of successfully engaging faculty without making them feel studied before they begin. In particular, we did not systemically collect data on classroom noise levels or on faculty and student perceptions of courses before implementing Biology FEST. Therefore, it is possible that the faculty who chose to participate in Biology FEST had already implemented active learning in their classrooms. However, the sheer number of faculty participating in the Biology FEST effort makes this possibility unlikely. While some instructors had been exposed to ideas about scientific teaching through attending teaching workshops or coteaching with the department’s SFES, the majority of the 62 participants had not, as reflected in their self-reports.

Another limitation of our work concerns the quality of the changes participants in Biology FEST made in their teaching. We have limited data on the frequency of scientific teaching strategy use. The student and faculty perceptions surveys only asked about techniques relating to active learning, not those relating to equity and diversity or assessment practices. In addition, the surveys did not ask whether teaching techniques were used more frequently than “in nearly every class.” Given that differences in achievement have been found between high-structure and moderate-structure courses, both of which use active-learning techniques in every class session, it may have been informative to have more fine-grained frequency data (Connell ). More fundamentally, for this study, we did not have human observers watch participants’ classrooms or directly collect common measures of student learning, which means that we could not tell whether an instructor’s implementation of an evidence-based teaching technique was superficial or deep. This is a concern, because studies have found that instructors often alter research-based teaching techniques when they use them, limiting the effectiveness of those techniques (Vickrey ; Dancy ). Nevertheless, the bulk of the research into active learning indicates that even minimal use of active learning increases learning compared with lecture alone (Ruhl ; Freeman ). Furthermore, to reach as many students as possible, it may be more fruitful to convince the majority of instructors to implement some form of active learning rather than persuade a handful of faculty to completely overhaul their courses (Fairweather, 2008). In this respect, it is promising that our most objective measure, DART, confirms that the majority of our instructors spend at least some time not engaged in traditional lecturing in most of their class sessions (Figure 5).

Additional limitations concern our survey of instructor motivations. The survey was conducted 3 years after the effort began, allowing the participants to give us their retrospective views. However, by that point, instructors may not have accurately remembered why they initially participated in Biology FEST. Also, even though the survey was anonymous, some participants may not have wanted to admit that their motivations were related to formal compensation. However, even if the survey slightly underestimated the percentage of instructors who were primarily motivated by compensation, it would still be true that most instructors were motivated by other factors.

Further Questions for Reform Efforts in Scientific Teaching

Questions about Evaluating the Effectiveness of Change Efforts.

Our Biology FEST efforts have opened up further questions about department-wide professional development more generally. One is how to evaluate its effectiveness on changing faculty teaching. To see whether Biology FEST altered teaching, we surveyed students about the frequency of active learning in their classes and analyzed recordings of classroom audio with DART. While these are good first steps for figuring out how faculty are changing their teaching, having more detailed observational data of what activities are being used in instructors’ classrooms would provide a more complete picture. While DART is the most feasible way of getting a broad characterization of how much active learning occurs in the courses of dozens of instructors teaching thousands of hours of class sessions, selective deployment of observational survey tools such as the Reformed Teaching Observation Protocol or Classroom Observation Protocol for Undergraduate STEM could be used to obtain a more nuanced view of what activities instructors are employing in their teaching (Sawada ; Smith ; Owens ).

In further work, it would also be important to measure the effects of faculty professional development on students. While it is difficult to see how learning gains could be compared across all the myriad courses offered by an entire department, one could measure the development of affective traits in students, such as motivation, self-efficacy, sense of belonging, and science identity, which can have a large impact on those students’ success and willingness to stay in science (Trujillo and Tanner, 2014). It is also important to evaluate the long-term impacts on students. While it is still too early to detect whether Biology FEST has altered our department’s 4- and 6-year graduation rates, previous research has suggested that active-learning approaches can disproportionately increase course performance for students from underserved groups, providing hope that our efforts will increase retention and graduation rates (Eddy and Hogan, 2014).

Questions about Biology FEST as a Model for Other Institutions.

Another big question that Biology FEST has opened is whether it can be a model for similar approaches at another institution. None of the elements of Biology FEST were specific to our department or institution. However, it is possible that underlying features of the department made its members more receptive to Biology FEST’s methods. For example, the department had previously hired an SFES who was then awarded tenure, showing that the leadership of the department already acknowledged the value of education research and scientific teaching. In addition, a subset of the faculty were involved in NSF-funded K–12 teacher training efforts or in a previous collaborative-teaching reform effort centered around implementing a new university requirement for writing in the major. It is likely that a different department that is embracing different values or that is encountering different challenges might find another method more effective, which is important because any effort to change faculty and their teaching needs to be grounded in the instructors’ goals and concerns (Aguirre ).

In addition, we also acknowledge the role of grant funding for our program. While we initially believed that the stipends provided to faculty would be a major motivator for participation, our data do not support that belief (Figure 8). However, we believe that the funding from our grant was essential for creating leadership capacity. First, it gave the principal investigators time to devote to the program through course releases and summer salary. Second, it provided for postdoctoral fellows, who were essential, because they created capacity for program development, coordination, and research. Our grant also helped fund the resource center, although the establishment of the center dated from a previous external grant. Any department that wished to conduct department-wide teaching reform would have to find some way to translate that desire into tangible support for the people leading and organizing those reform efforts, both senior personnel and trainees like postdocs who could emerge as the next generation of reform-minded faculty, including those who occupy SFES positions. Determining what factors influence how institutions or departments approach pedagogical reform and how widespread and sustained that reform can be is crucial for ensuring that more institutions adopt such reforms.

IN CONCLUSION

The ultimate goal of our Department of Biology, similar to that of many biology departments nationwide, is to develop students into leaders who are curious and scientifically literate. For too long, we have focused on accomplishing this goal by expanding access to outside enrichment programs, which by definition can only reach a subset of the undergraduate population. In contrast, all students take classes. Persuading faculty department-wide to implement scientific teaching in their courses would allow us to reach all of our students throughout their undergraduate careers and maximize their chances for effective learning. Here, we present Biology FEST as a potential model that incorporates an assortment of change strategies, including disseminating pedagogy, developing reflective teachers, developing a shared vision, enacting policy, and using local science education expertise to transform teaching across a whole department at an academic institution.

The Biology FEST effort itself is currently drawing to a close, but the department is not finished with professional development. There is still much room for faculty to build on the changes that they have initiated through, for example, more consistently including active-learning techniques in their classes, increasing use of assessment, and incorporating class material that is culturally relevant to our diverse student body. Biology FEST was an excellent start, but we know that we are just beginning to improve our faculty’s instruction and our students’ educational experiences.

Click here for additional data file.