Supports: A Key Factor in Faculty Implementation of Evidence-Based Teaching.

Evidence-based teaching (EBT), such as active learning and formative assessment, benefits student learning but is not present in many college science classrooms. The choices faculty make about how to teach their science courses are influenced by their personal beliefs and motivations, as well as their departmental structures and institutional cultures. With data from 584 science, technology, engineering, and mathematics (STEM) faculty trained in EBT, we compare which of the following factors most relate to faculty's use of EBT: 1) faculty's personal motivations (e.g., teaching value, confidence, beliefs about intelligence); and 2) their experiences with their institutional teaching environments (e.g., departmental support, student enthusiasm). Faculty's perceived supports in their teaching environments (e.g., having supportive colleagues, being able to access curricular resources) were by far most predictive of their use of EBT. Faculty's personal motivations had little to no relationship when supports were included in these models. The effects were robust, even when controlling for faculty gender, minority status, and teaching experience. Much of the literature has focused on perceived barriers to EBT implementation (e.g., lack of time, constrained teaching space). The current data indicate that a focus on building supports for faculty may have the greatest impact on increasing the presence of EBT in college STEM courses.

INTRODUCTION

Evidence-based teaching (EBT) is linked with positive student outcomes for college students, such as increased grades and engagement (Freeman ; Cavanagh , 2018). For example, in Freeman ) meta-analysis of EBT studies in college classrooms, researchers found that students’ scores on exams and concept inventories improved and the likelihood of students’ course failure decreased. Cavanagh , 2018) found science, technology, engineering, and mathematics (STEM) students’ exposure and commitment to EBT predicted their self-regulated learning strategies and final course grades. Purposefully encompassing a broad set of practices, EBT refers to teaching approaches that are often constructivist in nature, encouraging students to be active participants in building upon and reflecting on their learning. However, evidence-based practices such as active learning, formative assessment, and inclusive teaching are not typical for most college courses. This is particularly so in large introductory classes at research-focused institutions where faculty reliance upon methods that are not widely supported by empirical research in psychology and education, such as uninterrupted lectures, are still commonplace (National Research Council, 2003, 2012; Pfund ; President’s Council of Advisors on Science and Technology, 2012; Wieman, 2017).

Teaching approaches are shaped by faculty’s personal beliefs and motivations (Trigwell, 2012; Reeve and Su, 2014; Richardson ; Lund and Stains, 2015) as well as the characteristics of their teaching environments, such as departmental structure and relationships with colleagues (e.g., Henderson and Dancy, 2007; Ajzen, 2011; Bradforth ; Brownell and Tanner, 2015; Brew and Mantai, 2017). To increase the presence of EBT in college classrooms, it is essential to continually consider what influences faculty’s educational practices. In the present study, the data come from a sample of more than 500 STEM faculty from across the United States who have been trained in EBT practices. This research addresses two questions: 1) What role do faculty’s own motivations and confidence play in their implementation of EBT? 2) What role do faculty’s perceptions of the learning environment play in the amount of EBT they report using in the classroom?

Faculty Motivation and Confidence

Instructors’ teaching practices and interactions with their students are influenced by their motivations and previous teaching experiences (Guskey, 1988; Trigwell, 2012; Richardson ). Here, we focus on instructors’ value toward teaching and research, their anxiety toward teaching, their sense of confidence, and their growth mindset (i.e., their belief that achievement is related to effort rather than talent). Each of these factors relates to teaching practices, and we hypothesize they will relate to instructors’ use of EBT.

Growth Mindset.

Faculty vary in their belief that intelligence is malleable. For some, intelligence is related to effort rather than a fixed, or innate, ability. Those who believe intelligence is malleable are said to have a “growth mindset” and those who believe intelligence is stable are said to hold a “fixed mindset,” according to Dweck’s work on mindset theory (Dweck, 2016). Variation in this belief can influence faculty’s expectations and interactions with students (Masters, 2014; Dweck, 2016). For example, faculty who believe achievement is related to effort may communicate that students are capable of learning material through effort and varied study strategies rather than communicating that students’ achievements are related to factors beyond their control.

Confidence.

Instructors’ confidence, or self-efficacy, is also related to their classroom behavior and instructional quality (e.g., Guskey, 1988; Holzberger ). For example, instructors with greater confidence tend to engage in more discursive, open-ended classroom practices that are often reflective of EBT.

Teaching Anxiety.

Conversely, being anxious about teaching is related to rigidity in teaching, exhaustion with teaching, and a reluctance to interact with students in a dynamic manner (e.g., Frenzel, 2014).

Value.

Value often relates to greater persistence and effort (Postareff and Lindblom-Ylänne, 2011; Reeve and Su, 2014; Richardson ). Faculty’s value for teaching and research can motivate them to persist even in difficult circumstances. Because implementing EBT can require changing one’s course activities and curriculum multiple times based on students’ feedback, value for teaching may be a necessary component for what drives faculty to use EBT.

We draw on and expand this previous work by testing the relationship of these variables to instructors’ implementation of EBT. Understanding how to prepare and encourage greater use of these practices based on faculty’s motivation and experience is crucial, as faculty make curricular decisions based on such factors. For example, in a study of faculty aiming to implement EBT in their courses through case-study activities, faculty often relied on their own anecdotal experiences rather than the evidence from published sources (Andrews and Lemons, 2015).

Characteristics of Teaching Environment Related to EBT

Faculty and instructors also work within a departmental system that involves structural and interpersonal relationships, including administrative procedures, peer influences, and student feedback. For example, in addition to an individual instructor’s own unique characteristics or personal style, many educational decisions are shaped by the culture and resources within the departments where they teach. Professional development work on institutional change emphasizes the complexity of these educational choices, which often involve multiple levels of a department with potentially varying needs and priorities (Pundak and Rozner, 2008; Rogers, 2010; Henderson ). To assess instructors’ perceptions of their teaching environments, we examine the factors they view as influencing their college science classrooms: specifically, their perception of challenges and supports to using EBT.

Previous research has highlighted the challenges in implementing evidence-based practices. These challenges, which we refer to as barriers, include factors such as time constraints for revising curriculum, requirements for class content, student resistance to active learning, and departmental emphasis on research and traditional teaching (Michael, 2007; Walczyk ; Ebert-May ; Hora, 2012; Brownell and Tanner, 2015). This previous work provides instructor perspectives of the expectations and experiences of restructuring their teaching to incorporate EBT, which can inform faculty development and institutional administration aiming to bring EBT into a school. We hypothesize that these barriers can decrease the amount of EBT in the classroom, but the strength of the relationship between barriers and EBT in the classroom has not been well assessed.

There are also enabling characteristics, or supports, within the system that aid faculty to more easily implement EBT. These include: teaching incentives in the tenure structure; the purposeful development of local and regional teaching communities; and institution funding for building flexible classrooms (Bradforth ; Wieman, 2017). There is less systematic study of faculty’s perceived supports, that is, factors that faculty say encourage the use of EBT, and their effect on classroom practices. Here, we examine the effects of instructors’ perceived barriers and supports on their reported use of evidence-based practices in conjunction with individual-level characteristics, such as teacher motivation and confidence.

Study Context

Using a sample of STEM instructors from across the United States trained in EBT practices, our current work investigates potential influences on instructors’ teaching. What role do instructors’ own characteristics, motivations, and confidence play in their use of EBT, and do instructors’ perceptions of supports and barriers account for the amount of EBT they report using in the classroom? This research specifically addresses the characteristics of individual faculty members that matter for implementing EBT, and whether these individual characteristics matter less when in a highly supported department or institutional context.

METHODS

Participants and Procedure

Participants were attendees of the Summer Institutes on Scientific Teaching (SI) from 2004 to 2014. This training is a weeklong regional program that teaches STEM faculty EBT approaches using hands-on, group-facilitated activities focusing on active learning, formative assessment, inclusive teaching, and backwards design (Handelsman ; Pfund ). The SIs have been running since 2004 and have trained more than 1800 instructors from more than 350 institutions. Ongoing evaluation has been a part of this training and has established an impact of this training on teacher practice (Pfund ; Moss-Racusin ; Aragón , 2018). Many participants return to the SIs to facilitate future trainings, and many have gone on to run smaller-scale versions of the SI at their own institutions. We recognize that focusing on SI participants as our sample does give us a self-selected sample of faculty who have opted into an EBT training and may already be more interested in EBT than the general pool of STEM faculty. While this does attenuate the generalizability of our findings to faculty who are at least familiar or interested in EBT, selecting SI participants for the present research study does afford us a way to establish a shared baseline knowledge of EBT to examine the use of practices that are anticipated as outcomes of this training.

As part of a larger evaluation project, an online survey was sent to roughly 1100 faculty and instructors (hereafter referred to as “faculty”). Of the 728 who completed the survey, 584 faculty were included in this work, as these are the faculty who taught college courses and attended the SI in the past 5 years. Sample size varies by analysis due to small fluctuations in instructors’ completion of measures. These sample sizes are noted by analysis. Faculty were 60.8% female and 16.8% minority (83.2% Caucasian, 4.3% African American, 4% Asian, 3% Hispanic, 1.2% South Asian, 0.2% Native Hawaiian/Pacific Islander, 0.2% American Indian/Alaska Native, 0.2% Middle Eastern, 3% other). They had taught between 1 and 43 years with an average of 3.98 years (SD = 2.53). This work was considered exempt through the authors’ institutional review board (Yale IRB Protocol #1411014955).

Materials

Motivational Variables.

Faculty motivation was measured by concepts reflected in major motivational theories such as Ajzen’s work on planned behaviors (Ajzen, 2011), expectancy-value theory (Wigfield and Eccles, 2000; Eccles and Wigfield, 2002), and mindset theory (Dweck, 2016), and builds on prior research on the negative impacts of anxiety on teaching (Trigwell, 2012). These motivational measures include the five variables described here. 1) Faculty’s value of teaching and their 2) value of research, each of which has four items asking about the perceived importance of teaching and research from a variety of perspectives (faculty’s value, department, academic peers, and scientific field). Previous work shows that research and teaching goals tend to be seen as competing for time and focus, so we include both types of value here to see their relationship with EBT (Brew, 2010; Anderson ; Finkelstein, 2013). 3) Faculty’s growth mindset, which represents a belief that one’s intelligence is related to effort as opposed to luck or talent. This three-item measure comes from a previously validated scale (Dweck ) asking the degree to which faculty hold a fixed mindset. We reverse coded these items to represent faculty’s growth mindset (i.e., by measuring their disagreement with fixed mindset). 4) Faculty’s teaching anxiety also came from a previously validated scale from which we selected a subset of items most appropriate for the classroom contexts of our study (see the Supplemental Material for full measures). Items were kept as close to the original version as possible but adapted to ask about student interactions in the classroom. For example, the original scale has the item “I find myself worrying that I won’t know what to say in social situations.” We adapted it to read “I find myself worrying that I won’t know what to say in social situations with my students.” 5) Faculty’s general sense of confidence measures how successful they believe they are at achieving their goals. This measure used a subset of five items from a previously validated survey (Schwarzer and Jerusalem, 1995). See Table 1 for references and further descriptions of these measures. The Supplemental Material contains our full measures. These measures of value and confidence are quite broad and are not intended to provide detailed testing of the subcomponents of expectancy-value theory, for example. Rather, they are examined at a broader level to explore this line of research and minimize overly ambitious surveys with heavy time demand on our participants.

TABLE 1.

Description of measures

Measure	Example	Scale	Citation
Value of teaching (4 items)	How important is it to [blank] that you teach science well? [Blank filled with: you, your department, your academic peers, the scientific field]	Sliding scale from not at all important (0) to very important (100)	Eccles, 1995; Wigfield and Eccles, 2000; Eccles and Wigfield, 2002
Value of research (4 items)	How important is it to [blank] that you conduct scientific research? [Blank filled with: you, your department, your academic peers, the scientific field]	Sliding scale from not at all important (0) to very important (100)	Eccles, 1995; Wigfield and Eccles, 2000; Eccles and Wigfield, 2002
Growth mindset (3 items)	Intelligence is something about you that you can’t change very much.	Likert scale from strongly disagree (1) to strongly agree (6); reversed coded	Dweck et al., 1995; Dweck, 2016
Teaching anxiety (9 items)	I get tense when speaking in front of my class.	Likert scale from strongly disagree (1) to strongly agree (5)	Mattick and Clarke, 1998
Confidence (5 items)	I will be able to achieve most of the goals that I have set for myself.	Likert scale from strongly disagree (1) to strongly agree (5)	Pajares, 1996; Bandura, 1997
Perception of supports (30 items)	I get support from my department.	Binary item: implemented (1), did not implement (0)	Henderson and Dancy, 2007; Walczyk et al., 2007; Herreid and Schiller, 2013
Perception of barriers (30 items)	I do not have enough time during class for the activities.	Binary item: implemented (1), did not implement (0)	Henderson and Dancy, 2007; Walczyk et al., 2007; Herreid and Schiller, 2013
Implementation of EBT (19 items)	Using exercises that generate group discussion	Binary item: implemented (1), did not implement (0)	Handelsman et al., 2007; Couch et al., 2015

Supports and Barriers.

Measures of the perceptions of supports and barriers were developed through three main sources: 1) open-ended exit-survey data from 249 previous SI alumni who were asked to identify challenges and supports they believed they would encounter when implementing EBT; 2) twice-yearly structured conversations with SI leaders at an in-person meeting at which supports and barriers themes from their recent SI attendees were communicated; and 3) a general review of existing literature on supports and barriers to EBT. This iterative process resulted in the creation of 30 supports and 30 barriers that are also reflective of those commonly described in the literature reviewed (e.g., Michael, 2007; Walczyk ; Hora, 2012). In addition, there were general themes identified among the items, including academic receptivity (perceived support for EBT from faculty’s departments and colleagues), logistic considerations (perceived practical ease or challenge of implementing EBT in courses), student receptivity (faculty’s perceptions of their students’ reception of EBT), and personal teaching preference (perception of the alignment between preferred teaching techniques and EBT). We note that, while these themes were able to be categorized, they represent only broad areas that describe common supports and barriers. Therefore, within the present data set, these four themes only serve as an organizer—they are not intended to create four subscales, because items within each theme are not necessarily continuous. Specifically, if a participant selected one support, he or she may not necessarily choose others within that theme. There may be strong relationships in the presence of support items across categories, as is suggested by our previous work (Bathgate ). Separating and analyzing the data by these general themes may artificially separate the data and lead to over- or underemphasizing the relationship of a particular theme with EBT in the classroom. As such, a sum score was used for each of these variables, wherein increases in supports/barriers items selected result in a higher supports/barriers “score.” The method of summing across items has been used previously in science education research (e.g., Finson ; Sha ).

Implementation.

Our implementation measure consisted of 19 binary items measuring core practices associated with the EBT (active learning, assessments, inclusive teaching practices) presented at the SI training. These practices reflect previous work associated with the SI curriculum (Handelsman ; Couch ). Because the SI training does not teach specific prescriptive practices that faculty must use, but instead provides an approach toward teaching through which faculty could apply any number of activities, some of the items for our implementation measures, as well as supports and barriers items, are purposefully broad (e.g., implementation item: Using exercises that lead students to draw their own conclusions). In our measure, faculty responded as to whether or not they used each practice in their courses (binary response). Examples include “setting and communicating learning goals for each class,” “using exercises that generate group discussion,” and “implementing formative assessments while learning is occurring that inform students’ progress toward a desired outcome.” See the Supplemental Material for full measure.

Background Variables.

We included three background variables in each of our models: faculty teaching experience, gender, and minority status. The inclusion of these variables in the model tested whether implementation is predicted by other factors that could explain differences in faculty’s implementation of EBT beyond our immediate hypotheses. Faculty’s teaching experience was measured by the number of years they reported teaching. Gender was measured as male or female for those who provided this information (85.6%). Participants were given the option to not respond (14.2%) or select “other” if they preferred (0.2%). For the current purposes, we used a binary female–male gender item, because we lacked sufficient sample power to explore those who selected “other” in this sample. Minority status was measured by faculty’s selection of ethnicity grouped into non-Caucasian (minority) and Caucasian. The connection between minority status and EBT was tested, because implementing such practices may go against the norm of a department, raising concerns of stereotype threat for minorities.

All measures here are self-reported, as most of these variables rely on faculty’s subjective assessment of their own perspectives (e.g., assessing their own motivations).

Analysis

We begin by providing descriptive scale information related to our measures, including exploratory factor analysis summaries and Cronbach’s alpha. For exploring our primary research questions, an initial series of three multiple linear regressions (models 1–3) were run with faculty’s reported implementation of EBT as the outcome variable. The first regression tests the relationship of faculty’s individual characteristics with implementation, the second tests the relationship of perceived supports and barriers and implementation, and the final regression combines both individual characteristics and perceptions of supports and barriers together in the same model predicting implementation. We also recognized that two of the supports and barriers categories, personal teaching preferences and student reactions, may be more strongly related to the motivational measures assessed here (e.g., valuing teaching). While faculty reported these topics as supports and barriers to EBT, we wanted to carefully analyze the potential overlapping relationships among our variables that may affect interpretation. To confirm that the items within personal teaching preferences and student receptivity were not attenuating the effects of motivation, we include a subsequent set of analyses that removed these personal teaching preferences and student receptivity items from the supports and barriers variables. We again ran the same regressions described in models 1–3, but with this reduced set of supports and barriers variables. These regression models are referred to as models 4–6. Control variables of faculty gender, minority status, and teaching experience were included in all six regression models but were not significant predictors of implementation in these models with one exception. The number of years faculty taught had a small relationship with implementation for model 1 only (β = 0.08, p = 0.053). Standardized scores (z-score) were used in all regression analyses. All regressions were run using SPSS v. 22.0.

RESULTS

Descriptive Scale Information

Descriptive information for our measures, including sample sizes, means, standard deviations, exploratory factor analysis results, and Cronbach’s alpha is provided in Table 2. Among the variables in Table 3, correlations do not exceed commonly accepted statistical standards for multicollinearity (all r < 0.5; also our variance inflation factor statistics in our regression models do not approach metrics near diagnoses for multicollinearity in such models, as they are all below 2.0). While statistical correlations are not a concern, we wanted to test to whether our teaching preferences and student receptivity items are too theoretically close to our motivational variables. As such, we do systematically test the relationship of supports and barriers to EBT with these items removed.

TABLE 2.

Descriptive information for our measures including exploratory factor analysis results and Cronbach’s alpha where appropriatea

Measure	N	M	SD	EFA results	α
Growth mindset	508	4.18	1.26	One-factor; 0.94–0.95	0.94
Confidence	511	4.06	0.57	One-factor; loadings 0.75–0.79	0.82
Teaching anxiety	505	2.1	0.74	Initial two-factor solution with three double-loading items; fits a one-factor solution when constrained with loadings 0.58–0.84	0.88
Teaching value	513	82.27	13.39	One-factor; loadings 0.62–0.87	0.70
Research value	497	75.51	24.09	One-factor; loadings 0.79–0.92	0.88
Perceived supports	510	15.08	6.80	n/a	n/a
Perceived barriers	494	6.34	3.50	n/a	n/a
Background variables
Years teaching	569	3.98	2.53	n/a	n/a
Gender	500	60.8% female
Ethnicity	494	16.8% minority

aRegressions vary by response rates, dependent upon how many people responded to each measure.

TABLE 3.

Descriptive statistics and correlations among variables

	Measure	N	M	SD	V2	V3	V4	V5	V6	V7	V8
V1	Self-efficacy	511	4.1	0.6	0.13**	0.03	0.04	−0.27***	0.21***	−0.14***	0.17**
V2	Teaching value	513	82.3	13.4		−0.17***	0.02	−0.02	0.19***	−0.13**	0.03
V3	Research value	497	75.5	24.1			<0.01	0.05	−0.10*	−0.10*	−0.03
V4	Growth mindset	508	4.2	1.3				−0.05	0.17***	−0.10*	−0.13**
V5	Teaching anxiety	505	2.1	0.7					−0.13**	0.21***	−0.12**
V6	Perceived supports	510	15.1	6.8						0.01	0.49***
V7	Perceived barriers	494	6.3	3.5							−0.21***
V8	Implementation	512	4.4	0.7

*p ≤ 0.05.

**p ≤ 0.01.

***p ≤ 0.001.

Regression Results

Table 4 shows the regression results demonstrating the novel contribution that supports have in predicting faculty implementation of EBT practices. The overall pattern of results indicates that supports, as perceived by faculty, are the largest predictor of EBT implementation. When supports are considered, faculty’s personal motivations and confidence, as measured in this sample of faculty, show little to no relationship with their reported implementations. This pattern was robust even when controlling for gender, minority status, and teaching experience. We now walk through the three models that demonstrate these results.

TABLE 4.

Multiple linear regression models testing each set of variables with reported implementation of EBT as the dependent variablea

	Model 1	Model 2	Model 3	Model 4	Model 5
Variable	Motivation variables	Supports and barriers variables	All variables	Adjusted supports and barriers variables	All variables with adjusted supports and barriers
	β	β	β	β	β
Growth mindset	0.13*		n.s.		0.10*
Confidence	0.12***		n.s.		0.09*
Teaching anxiety	−0.10*		n.s.		n.s.
Teaching value	n.s.		−0.11*		n.s.
Research value	n.s.		n.s		n.s.
Perceived supports		0.51***	0.51**	0.36***	0.35***
Perceived barriers		n.s.	n.s.	n.s.	n.s.
Adjusted R2	4%	26%	27%	26%	13%

aIn model 5, the perceived barriers (without personal and student items) variable was trending toward significance at a p value of 0.08. Years teaching, gender, and minority status were included in each model. These results are not shown, as they were consistently not significantly related with implementation, with one exception: The number of years faculty taught was related to slightly more reported use of EBT practices (β = 0.08, p = 0.053).

*p ≤ 0.05.

**p ≤ 0.01.

***p ≤ 0.001.

Model 1 shows how individual characteristics relate to faculty’s reported EBT through a multiple linear regression analysis. Faculty implemented more EBT practices when they believed their efforts were related to achievement (growth mindset) and when they had greater confidence, whereas being anxious about teaching was related to using less EBT in the classroom. Notably, there was no relationship between the faculty’s value for teaching or research and their level of implementation. Faculty could highly value their research and still be able to bring evidence-based practices into their classes. Similarly, faculty who see less value in placing their time into teaching are still able to implement evidence-based practices.

Model 2 shows the relationship of faculty’s perception of the teaching environment (perceived supports and barriers) to implementation. Perceiving more supports was strongly related to EBT in the classroom. When faculty felt encouraged by colleagues in their departments and knew where to seek curricular resources, for example, they implemented more EBT. Conversely, the barriers faculty perceived, such as time constraints on teaching efforts, had no relationship with implementation. Perceiving more challenges to EBT was not related to these faculty reporting using less EBT.

Model 3 combines all variables we tested. It includes both the individual characteristic variables from model 1 with the perceived supports and barriers variables from model 2 in the same regression model. In this full model, perceived supports continued to have the strongest relationship to EBT implementation. In fact, faculty’s motivations had little relationship with implementation once supports and barriers were included. All of the individual characteristics that were significant contributors in the first analysis (model 1) become nonsignificant when supports and barriers are included in the second analysis. There is a remarkably strong relationship between supports and implementation, emphasizing the value of helping faculty identify and develop supports for EBT at their local institutions.

Also in model 3, faculty’s teaching value had a small and unexpectedly negative relationship with implementation, wherein faculty implemented slightly less when they valued teaching more. The data here do not explain this pattern. Any interpretation is speculative, but this slight negative pattern could be indicative of faculty who highly value teaching taking a different approach to adapting their teaching, such as taking small steps to integrate EBT with more time for reflection and feedback. Further work is needed to explain this connection.

The adjusted R2 change for each model is presented at the bottom of Table 4 and indicates how much the variance in reported implementation is explained by the variables in each model. We used the adjusted R2, as it accounts for the different number of variables across regression models. Perceived supports and barriers explain 27% of the variance in reported EBT, whereas individual characteristics account for only 4% of teaching variation. Faculty reported implementing more EBT when they perceived more supports, even when considering their motivation and confidence and accounting for the potential relationships of gender, minority status, and teaching experience. According to our results, a faculty member who is anxious about teaching and has low self-confidence can still use a high amount of evidence-based practices when he or she feels supported. These results suggest that, when considering how to increase EBT in college education, generating and promoting supports can have a large impact, regardless of variation in individual motivations, at least with faculty who are interested in engaging with EBT concepts, as in this sample.

We next reran regression models 2 and 3 from Table 4 but removed the personal teaching preferences and student receptivity items from the supports and barriers variables. Although faculty and SI leaders clearly described these personal teaching preferences and student responses as supports (or barriers) to EBT, we recognize that items within these areas may strongly relate to faculty’s motivations measured here. For example, personal teaching preferences items asked about enjoyment and comfort with EBT, which may be closely theoretically related to our motivation variables and potentially suppress their effect. Therefore, we tested whether motivational variables and supports and barriers showed a similar relationship with EBT once the personal teaching preferences and student receptivity items were removed.

Table 4 shows the results of these two models. Model 4 shows the relationship of the adjusted perceived supports and barriers variables with reported implementation. The perceived supports variable continues to have the largest relationship with implementation, although the standardized beta is slightly reduced with fewer items. The barriers variable continues to have no relationship with reported implementation in this sample. Model 5 includes the motivation and value variables with the adjusted supports and barriers variables (i.e., personal and student support/barrier items removed). Model 5 shows growth mindset and confidence still have a significant relationship with reported EBT when supports and barriers are included in the model, although their effect is somewhat attenuated. The R2 is substantially lower in model 5 compared with model 3, suggesting that the removed teaching preferences and student receptivity items have a unique contribution toward explaining the variance in reported implementation. The clear and major result across all of our models is that perceived supports had the strongest relationship to implementing EBT, even when faculty’s motivations and confidence were considered.

DISCUSSION

Our data suggest that focusing on building supports could have greater impact on promoting evidence-based practices in the classroom. The variable perceived supports, when included with a range of personal and departmental factors, showed the strongest relationship with reported EBT implementation in our sample: the more perceived supports faculty reported, the more implementation they indicated, even when barriers and motivational variables were included in the regressions.

In our sample, instructors’ motivations and confidence were less consistently related to implementation than the supports variable—and always to a lesser degree. Of these variables, instructors’ growth mindset and confidence were most related to implementing EBT, and each maintained a small positive relationship with reported EBT when academic and logistic supports were included in the regression (model 5).

Our data show that even those instructors who were anxious about teaching had nevertheless adopted EBT, particularly when they felt supported. This finding speaks to administrators and faculty developers who work with instructors who may feel unprepared or highly challenged by using evidence-based practices by helping such instructors identify and build supports. For example, finding a community of peers or developing strategies to provide necessary content for a course may be beneficial to overcoming these challenges, although our data do not directly test the relationship of particular supports to these outcomes.

The slight negative relationship of teaching value with reported implementation is worth noting again here. While the present data set cannot definitively reconcile this unexpected relationship, it may be that faculty who value teaching may go about implementing EBT in an unexpected way. For example, perhaps faculty with high teaching value already have invested a great deal of effort into their teaching and may not as readily change what they see as high-quality instruction. Or perhaps these faculty integrate specific EBT practices more completely before including many practices. Further work is needed to explain this connection and to test these ideas.

Limitations

There are limitations to our data that include the following: 1) self-selected sample of SI participants, 2) single time-point data that limit our knowledge of directionality, 3) variation in the frequency and quality of implementation, 4) distinguishing among the impacts across the themes of supports and barriers, and 5) unmeasured variables that could further impact amount of implementation. Each of these is outlined below.

Self-Selected Sample.

Our data represent hundreds of faculty from across the United States, a uniquely large sample of college instructors trained in EBT, and an empirically strong pattern in the factors that influence their teaching emerges from the data analysis. Still, these results are based on individuals largely from research-intensive STEM departments who self-­selected into a training program on EBT. There may also be differences in norms or resources around EBT across disciplines within the sciences (e.g., biology, engineering). The generalizability of this pattern should therefore be the focus of future studies.

Directionality.

Our data come from a single time-point measurement, which leaves open the direction of causality between supports and implementation. We can speak to the strength of the relationship between supports and barriers to implementation, but we cannot conclude that supports unequivocally caused more implementation. It could be that faculty who implement more EBT perceive more supports. We assert that there is likely a positive feedback cycle in which, when faculty feel supported, they implement more, which generates more supports. This hypothesis needs empirical testing, and until we test and observe causality through multiple time points, language of causality should be avoided.

Frequency and Quality of Implementation.

There is likely variation in the frequency with which faculty use specific EBT practices, as well as the quality with which these practices are implemented. Here, we have assessed whether or not faculty report using each practice in a nominal way—responding positively to each practice is assumed to indicate higher use of EBT. There could be faculty in our sample who use a few practices very often. How often faculty use each practice and the efficacy with which this practice is conducted in the classroom likely have important effects characterizing their practice. More nuanced data collection describing broader patterns in faculty’s use of EBT in STEM classrooms can be found elsewhere, such as Stains .

Support and Barrier Themes.

There are four general themes in our supports and barriers items (academic receptivity, logistic considerations, student receptivity, and personal teaching preferences). While it would be useful to know how each of these themes individually relates to implementation, there are clear reasons that prevent this analysis. First, the themes are intended to be descriptive but are not indicative of an underlying “scale.” There is not necessarily a continuous factor for each theme; that is, if a participant selected one support, they may not necessarily choose another within that theme. As such, to treat them as subscales would not be appropriate in this case. Second, the items are not intended to be mutually exclusive across themes. An item related to logistic considerations may also relate to student receptivity, for example. If we firmly classify items into categories, we may be artificially distancing themes and items that are empirically related. If we then interpret these findings, we could risk misattributing relationships between themes and reported implementation. As such, we err on the side of caution to avoid erroneous interpretation of our data and potential ineffective applications to practice. Exploring whether broader themes are able to be empirically distinguished and whether they differentially relate to implementation may point toward practical applications and should be the focus of future work. This would involve further revision and validation of the items and themes measured here.

Additional Variables.

There also remains variance in EBT implementation unaccounted for by our variables. Our design did not fully incorporate all aspects of motivational theories, such as expectancy-value theory’s focus on cost and the separation of attainment from utility value. There are also other contextual features that vary, such as the institutional focus on research and enrollment size. This work serves as a direction for future research to pursue the generalizability, directionality, and additional variables that can explain variance in faculty’s incorporation of EBT in STEM classrooms and assist in producing greater impact.

Implications

The current analysis presents supports across a range of topics, and we suggest faculty and administrators work to identify which supports may be most appropriate for their given circumstances. More generally, our study corroborates other efforts suggesting that faculty developers and department administrators work to identify or develop supports that are targeted to what faculty need in order to incorporate EBT in their classrooms and across a department or institution (Walczyk ; Hora, 2012; Bradforth ; Wieman, 2017). For example, connecting faculty to EBT resources, establishing a local or regional community of instructors interested in EBT, and working with faculty to promote positive student reactions to EBT, among other supports, would likely be associated with increases in faculty use of evidence-based practices. Part of this process would most effectively include stakeholders across various levels within a given department or institution. Developing shared vision, sharing curriculum, enacting policy changes, and establishing reflective teaching practices around EBT would likely be most effective for long-term change, according to Henderson analytic review of institutional change in STEM education.

CONCLUSION

College science faculty should consider focusing on identifying and using supports to help them implement EBT practices instead of trying to navigate around challenges and barriers. These supports include ideas such as establishing connections to local teaching experts and building a clear departmental value toward incorporating evidence-based practices into teaching. Faculty training and development initiatives must also move toward identifying and generating more supports for faculty in order to implement EBT practices on a wider scale. Such action could promote meeting the long-standing goal of making college science more active and inclusive of all learners.

Click here for additional data file.