From Gatekeeping to Engagement: A Multicontextual, Mixed Method Study of Student Academic Engagement in Introductory STEM Courses.

The lack of academic engagement in introductory science courses is considered by some to be a primary reason why students switch out of science majors. This study employed a sequential, explanatory mixed methods approach to provide a richer understanding of the relationship between student engagement and introductory science instruction. Quantitative survey data were drawn from 2,873 students within 73 introductory science, technology, engineering, and mathematics (STEM) courses across 15 colleges and universities, and qualitative data were collected from 41 student focus groups at eight of these institutions. The findings indicate that students tended to be more engaged in courses where the instructor consistently signaled an openness to student questions and recognizes her/his role in helping students succeed. Likewise, students who reported feeling comfortable asking questions in class, seeking out tutoring, attending supplemental instruction sessions, and collaborating with other students in the course were also more likely to be engaged. Instructional implications for improving students' levels of academic engagement are discussed.

Introduction

In discussing why students leave science, Daryl Chubin, director of the American Association for the Advancement of Sciences’ Center for Advancing Science & Engineering Capacity, stated: “The culture of science says, ‘Not everybody is good enough to cut it, and we’re going to make it hard for them, and the cream will rise to the top’” (Epstein 2006). Nowhere is this more evident than in introductory “gatekeeper” courses, those initial introductory college math and science courses which either explicitly or implicitly function to eliminate all but the ‘top tier’ students and champion the concept that “scientists are born, not made” (Tobias 1990, p. 11). It is within the first 2 years of taking these courses that the majority of attrition in the sciences occurs in college (Chang et al. 2008; Seymour and Hewitt 1997). In fact, Seymour and Hewitt (1997) identify STEM students’ performance in introductory courses as one of the key indicators as to whether they switch out of their intended STEM majors during college.

Scholars have linked these high attrition rates to several factors, including a reliance on large lecture-based courses and lack of engaging pedagogy. Although it may represent an efficient method for presenting a tremendous amount of content to a large audience, lecture tends to encourage one-way, passive, superficial learning (Biggs 1999; Bransford et al. 2000; Moore et al. 1996; Seymour and Hewitt 1997). Additionally, lectures tend to promote memorization over conceptual understanding (Bligh 2000; Booth 2001; Knight and Wood 2005; Novak et al. 1999). Indeed, college-level introductory science and mathematics courses have been shown to focus too much on the acquisition of content knowledge through memorization and too little on the development of meta-cognitive skills related to critical thinking and scientific literacy (Handelsman et al. 2004; Hurd 1997; Williams et al. 2004). Instead, Glasson and Lalik (1993) argue that students need to reflect on, relate to, and examine concepts as they are presented as part of an active, constructive process. According to Thalheimer (2003), unless a student cognitively processes a question and participates in answering it (even if mentally), learning rarely takes place. This higher level of engagement is unlikely to occur in traditional lecture classes where the constant stream of information leaves students scrambling to take accurate notes with little time to process questions and concepts (Beswick and Ramsden 1987).

The critics of introductory “gatekeeper” courses in STEM have argued that traditional lecture-based courses need to be transformed from pedantic, instructor-led classrooms to dynamic student-centered learning arenas (National Research Council [NRC], 2003; Wood 2003). The movement espousing this transformation has gained widespread support from national funders including: the National Science Foundation, the National Science Resources Center, the National Science Teachers Association, the National Institutes of Health (NIH), and many professional scientific societies. The call for more active learning in introductory science courses has led to a burgeoning body of research known as the scholarship of teaching (Boyer 1990), conducted by the research scientists teaching these courses, that investigates student outcomes related to these strategies. Scholars, however, have not given as much attention to the climate of introductory courses that students encounter during their first year of college, particularly in the STEM context. Given that introductory STEM courses may serve as a critical barrier to students’ progress toward their STEM degree aspirations, an examination of the relationship between students’ experiences in these courses and their resultant self-reported engagement is warranted, as a substantial proportion of intended science majors decide to leave STEM fields by the end of the first year of college (Chang et al. 2008).

Using a sequential, explanatory mixed methods approach, this study draws from quantitative survey data from 73 introductory STEM courses across 15 colleges and universities in combination with qualitative data collected from focus groups with students enrolled in these courses at eight of these institutions to examine the learning strategies and pedagogical practices that best relate to students’ self-reported academic engagement. With this research design, we sought not only to examine the predictive power of specific learning strategies and classroom contexts that relate to STEM students’ engagement in introductory courses but also to further support and enrich these findings through students’ narrative experiences of being enrolled in these courses.

The Importance of Academic Engagement

In the early 1980s, Astin (1984, 1985) and Pace (1984), and later Kuh et al. (1991) pioneered the early work in student engagement theory. Despite their use of various terms, “their views were based on the simple, but powerful, premise that students learn from what they do in college” (Pike and Kuh 2005, p. 186). While this basic premise still holds true today, academic engagement continues to be defined by scholars in a number of nuanced ways. Fredricks et al. (2004) note that academic engagement represents a multifaceted construct that includes three dimensions: behavioral, emotional, and cognitive. Behavioral engagement encompasses student involvement in academic tasks and includes measures of effort, asking questions, and paying attention in class (Birch and Ladd 1997). Emotional engagement relates to students’ feelings of boredom, anxiety, and excitement in the classroom (Connell and Wellborn 1991; Skinner and Belmont 1993), and cognitive engagement is conceptualized as students’ investment in learning with measures relating to individuals’ commitment to working hard and exceeding expectations (Connell and Wellborn 1991; Greene and Miller 1996). In conceptualizing this study, we define engagement as mainly behavioral and draw from several studies that have examined this form of engagement. Svanum and Bigatti (2009) measured course engagement as attendance in lectures and review sessions whereas Larose et al. (1998) conceptualized engagement simply as collaboration with other students. Handelsman et al. (2005) operationalized course engagement as interaction with course content both inside and outside the classroom.

In this study, we also draw from theories of motivation and learning to offer insight into the psychological traits that foster greater behavioral academic engagement among students. Hofer (2002) maintains that extrinsically motivated students focus more on earning grades and receiving recognition from their teachers and professors rather than emphasizing concept mastery. Thus, a drive for higher grades, admission into a particular program, or recognition from faculty, such as a letter of recommendation, may motivate students to become more engaged in the classroom. By contrast, Ryan and Deci (2000) argue that intrinsically motivated students have a greater likelihood of having quality educational experiences in the classroom due to their interest and enjoyment in learning for the sake of learning. These students are likely to become more engaged when instructors provide them greater autonomy and choice regarding learning exploration (McCombs 1991). Thus, students who lack excitement for course material or go unchallenged by the rigor of the academic work may demonstrate lower levels of engagement in the classroom.

In addition to motivation, other student factors also influence engagement. Students who lack confidence in their ability to succeed may also lack willingness to engage in their courses (Rocca 2010). Similarly, feelings of intimidation and inadequacy may prohibit students from participating in class, particularly when the intimidation is tied to a lack of understanding of course content (Fassinger 1995; Weaver and Qi 2005). These psychological factors may well promote or prohibit students’ academic engagement in introductory STEM courses.

Faculty behaviors that set the classroom context also contribute to students’ academic engagement. Many scholars, including Chickering and Gamson (1987), Ewell and Jones (1996), and Tinto (1993, 2000), have documented the strong association between faculty-student interactions and increased student learning and engagement. Students often arrive at college not knowing whom or how to ask for help, and faculty shape the climate and classroom activities that orient students in ways that affect their likelihood of success. When it comes to providing access to institutional resources and socializing students in the learning process, for example, faculty signal accessibility cues that indicate to students their openness to questions and availability for support (Wilson et al. 1974). Crombie et al. (2003) also found that students who sensed that their instructors cared about them demonstrated increased levels of engagement in their courses, which underscores the importance of faculty behavior. Mutual respect in the classroom that values student contributions and encourages participation and questions provides for a more inviting classroom climate that promotes greater engagement (Crombie et al. 2003; Dallimore et al. 2004). When instructors validate and affirm students’ responses to questions, they also increase students’ willingness to participate and engage in class (West and Pearson 1994). Thus, while academic engagement has been conceptualized as an attribute of student behavior, it is also dependent on the classroom context, particularly the interchange between the instructor and students.

To summarize, there is a long and well established link between academic engagement, performance, and persistence (Astin 1993; Hurtado and Carter 1997; Pascarella and Terenzini 2005; Stage and Hossler 2000), which is particularly relevant to STEM undergraduate education (Seymour and Hewitt 1997). Academically engaged students have an increased likelihood of persisting beyond the first year of college (Nelson Laird et al. 2008), and students who participate more in class and seek out faculty during office hours also tend to earn significantly higher end-of-course grades (Handelsman et al. 2005). Overall, being more academically engaged improves students’ motivation, critical thinking skills, personal character, and academic abilities (Gellin 2003; Pike 1999, 2000; Pike and Killian 2001; Rocca 2010).

Active Learning Pedagogies

Another important consideration for this study are active learning based teaching strategies (NRC 2003; Wood 2003), which are generally understood as techniques that promote student engagement in the learning process (Prince 2004). Chickering and Gamson (1987) argue that active learning techniques require students to write, think, and talk about their learning while simultaneously applying it to their own lives. Instructors who encourage active learning amongst their students have an opportunity to capitalize on students’ intrinsic motivation, which can lead to greater academic engagement (Tagg 2003).

According to Tagg (2003), by incorporating collaborative activities and real-world applications into teaching, instructors have the capacity to encourage greater comprehension and learning, as students engage with one another to better master a concept and are able to take conceptual knowledge and apply it to real-world problems. At the undergraduate level, this may include a number of in- and out-of-class activities, such as team projects and peer reviews that promote a deeper level of understanding (Chickering and Gamson 1987). Similarly, Rogers and Freiberg (1994) argue for a person-centered approach to teaching whereby faculty facilitate knowledge acquisition through a variety of pedagogical methods rather than simply distributing knowledge through lecture. It is this active process of learning that, according to Handelsman et al. (2004), helps students to develop the habits of mind that drive science.

Although critics often note that STEM faculty are reluctant to embrace active learning practices in introductory courses, the use of these strategies appears to be growing. Some examples of more widely used strategies include “bookending” the lecture with questions that tightly focus student discussion, incorporating student response systems, requiring group projects and presentations, implementing peer-led team learning, and introducing problem-based learning or case studies (Allen and Tanner 2005). A number of researchers have tested these active learning strategies, but the impact on course satisfaction and grades tends to be mixed.

One active learning strategy that has been implemented and tested in numerous settings is the use of student response systems or “clickers.” Caldwell (2007) contends that the use of clickers may increase attendance and engagement while decreasing course attrition. Clickers have the potential to increase students’ understanding of course content, as they provide immediate signals to the instructor that students have not fully grasped the concept from a lecture or demonstration; however, instructors must also have an adaptive response with immediate feedback once they become aware of students’ difficulty in grasping concepts (Caldwell 2007; Crossgrove and Curran 2008). These devices have also been shown to increase students’ attentiveness and alertness in class (Nagy-Shadman and Desrochers 2008).

Another active learning approach includes collaborative and cooperative learning, which researchers have examined in the context of introductory science courses. Such strategies include, but are not limited to, Just in Time Teaching, Peer-Led Team Learning, workshops, inquiry labs, and problem-based learning (Smith et al. 2005). Collaborative and cooperative learning strategies require students to work together, but Prince (2004) distinguishes the two based on the method of assessment; in cooperative learning, students are graded individually while collaborative learning techniques call for a group-based grade. Some studies have found that adding cooperative problem solving and student participation activities to class results in increased learning gains and higher levels of conceptual understanding (Knight and Woods 2005; Tagg 2003).

Web-based pedagogy in introductory courses is another way to engage students by combining in-class lectures with out-of-class web-enhanced activities. It has become popular over the past decade with the introduction of programs such as WebCT, Blackboard, and eClassroom. McFarlin (2008) found that students who took a course that offers 50% of the course content via traditional methods and 50% via the web (“hybrid” courses) reported appreciating the self-paced nature of web-based technology, which enhanced autonomy as advocated by McCombs (1991). Students in hybrid courses also reported that they enjoyed using web-based technology for accessing course materials and interacting with their peers and instructors (Smith et al. 2005). Thus, such practices have the potential to increase student engagement.

Unfortunately, the research regarding active learning pedagogy is somewhat limited. Most of the studies have been conducted in one or two classrooms at one institution by the instructors responsible for implementing these strategies while teaching the course. This research design is problematic, given the difficulty in generalizing findings from one setting and possible inherent bias on the part of the researcher who is testing a method in his/her own course. Additionally, a majority of the research has been quantitative in nature with data collected via surveys and course evaluations. The present study addresses these limitations through the use of a mixed-methods approach across multiple classrooms and institutions. We have collected both quantitative and qualitative data at multiple institutions in hopes of reporting more generalizable findings enhanced by nuanced explorations of engagement in introductory STEM classrooms.

Methodology

To address student engagement in introductory STEM classrooms, we utilized a sequential explanatory mixed methods design, which consisted of collecting, analyzing, and integrating both quantitative and qualitative data during the research process (Creswell 2005; Teddlie and Tashakkori 2009). This approach was taken as neither quantitative nor qualitative methods were solely sufficient to capture the breadth and depth of student experiences across multiple introductory STEM classrooms (Johnson et al. 2007). When properly combined, quantitative and qualitative methods can complement each other and provide a more complete picture of the research problem (Creswell and Plano Clark 2011; Green et al. 1989; Johnson and Turner 2003; Tashakkori and Teddlie 1998). The quantitative data were collected first, preliminarily analyzed, and used to inform the selection of institutional sites for qualitative data collection. The results of the quantitative and qualitative phases were then more fully integrated in the results and discussion sections.

In integrating the quantitative and qualitative data, we aimed to satisfy the mixed methods evaluation criteria defined by Creswell and Plano Clark (2011) wherein they describe “results or interpretation sections in which the authors explicitly brought together the two databases” (p. 269). Additionally, we were guided by Bryman et al.’s (2008) criteria for evaluating mixed methods studies, which further emphasized integration or mixing of findings and the importance of providing a rationale for utilizing mixed methods. In this study, the quantitative data and results provided a general picture of students’ engagement nested in 73 classrooms and 15 institutions, whereas the qualitative data refined and explained those statistical results by exploring the participants’ views regarding their introductory classroom experience in more depth. (See Fig. 1 for a diagram of the mixed methods sequential explanatory design procedures employed in this study.) The use of this figure itself, utilizing the standardized mixed methods research notational system, satisfies one of the nine general characteristics of quality in mixed methods research (Tashakkori and Teddlie 2010). This mixed method design not only enhances explanation but also provides an opportunity for cross-validation of findings across techniques and across multiple institutions, which enhances generalizability of the findings while also maintaining an emphasis on contextual differences.

Fig. 1

Visual model of mixed-methods design procedures (adapted from Ivankova et al. 2006)

Quantitative Design

Sample

The quantitative data for this study come from three surveys conducted during the spring of 2010: a pre- and post-survey of students in introductory STEM courses and a one-time survey of faculty teaching these introductory science, technology, engineering, and mathematics (STEM) courses. In the fall of 2010, we identified 15 campuses to participate in a study of introductory STEM courses funded by the NIH through the American Recovery and Reinvestment Act (ARRA) of 2009. These campuses varied by institutional control, size, selectivity, minority-serving status [i.e., Historically Black Colleges and Universities (HBCUs) and Hispanic-Serving Institutions (HSIs)], geographic region, and Carnegie classification.

Within each campus, we worked with administrators to identify introductory STEM courses. A total of 81 separate courses composed the classroom sample, and the number of courses selected within each institution ranged from one to 18, with the average institution having five courses. These introductory courses spanned several disciplines and included such courses as calculus, pre-calculus, cell biology, introduction to design, introduction to computer science, introduction to physics, and general chemistry, among others. After identifying introductory STEM courses within each institution, we emailed all students enrolled in these selected courses with an invitation to participate in an online survey.

This initial survey, administered at the beginning of the academic term, requested information on students’ pre-college preparation, pre-college experiences, background characteristics, and educational and career plans. Two weeks prior to the end of the academic term, we invited all students in these introductory courses to complete a follow-up survey, which post-tested career and educational aspirations and inquired about students’ experiences inside and outside the context of their introductory course, their perceptions of faculty members’ pedagogical style, and scientific habits of mind. We piloted versions of these two student surveys during the 2006–2007 academic year, and many of the survey items were developed from Conley’s (2005) work on the development of scientific dispositions and collegiate habits of mind. Copies of the survey instrument are available upon request. Students received a $10 gift card for completing each survey. In total, 3,205 students completed both surveys, which provided for a longitudinal response rate of 42.1%.

In addition to the student surveys, we conducted a survey of all faculty teaching these 81 introductory STEM courses. The faculty survey requested information regarding faculty members’ pedagogical styles in their introductory courses, their expectations of students, their perceptions of students’ preparation, and issues of faculty workload. Faculty members received a $100 gift card for completing the survey, and the response rate for the faculty survey was 90.1%.

After deleting cases with missing data on the outcome and demographic characteristics and deleting students in courses where faculty did not complete a survey, the final analytic sample for this study includes 2,873 students in 73 introductory STEM courses across 15 colleges and universities. The institutional sample for the quantitative portion of the study included a balanced mix between public and private institutions; research, masters, and minority-serving institutions, including three HBCUs and two HSIs. A majority of the student sample identified as White (52%), and 61% of students were women. Nearly half (42%) of students aspired to earn a medical degree, and 21% of students wanted to earn a Ph.D. or an Ed.D. Approximately 75% of students reported majoring in a STEM discipline. Table 1 provides descriptive statistics for all variables included in the analysis.

Table 1

Descriptive statistics of variables included in the HLM analysis

	Mean	SD	Min.	Max.
Dependent variable
Academic engagement	0.00	1.00	−1.60	2.62
Course-level variables
Faculty course goal: Use technology to effectively engage students	3.16	0.88	1.00	4.00
Faculty agreement: In my classroom, there is no such thing as a question that is too elementary	3.52	0.82	1.00	4.00
Faculty agreement: I feel it is primarily up to students whether they succeed in this course	3.00	0.72	1.00	4.00
Faculty agreement: There is not enough time available to give every student individualized attention	3.05	1.00	1.00	4.00
Tenure status: Tenured (reference group: not on tenure track)	0.42	0.50	0.00	1.00
Tenure status: Not tenured, on tenure track (reference group: not on tenure track)	0.13	0.34	0.00	1.00
Student-level variables
Sex: Female	0.61	0.48	0.00	1.00
Race: White (reference group: non-White)	0.52	0.50	0.00	1.00
SAT composite score (×100)	12.58	1.55	4.00	16.00
HS biology grade	3.73	0.51	1.00	4.00
HS chemistry grade	3.65	0.57	0.00	4.00
Earned college math credits in high school	0.21	0.41	0.00	1.00
Freshman (reference group: non freshman)	0.60	0.49	0.00	1.00
Degree aspiration: Medical doctorate (reference group: all other aspirations)	0.42	0.49	0.00	1.00
Frequency: Asked a teacher for advice or help outside of class in HS	1.94	0.66	1.00	3.00
Self-rating: Communication skills	3.69	0.90	1.00	5.00
Self-rating: Initiative-taking	3.66	0.88	1.00	5.00
Self-rating: Ability to know when and whom to ask for help	3.63	0.89	1.00	5.00
Agreement: Faculty gave students written feedback on their performance or progress in the course	2.59	0.98	1.00	4.00
Self-rating: Competitiveness	2.62	0.93	1.00	5.00
Frequency: Sought tutoring from a campus office or program	2.26	1.41	1.00	5.00
Frequency: Sought a professional (off-campus) tutor	1.71	1.17	1.00	5.00
Frequency: Attended supplemental instruction sessions	2.58	2.15	1.00	13.00
Frequency: Felt excited about learning new concepts	3.13	1.04	1.00	5.00
Frequency: Felt collaboration among other students in this course	2.95	1.12	1.00	5.00
Proportion of class time devoted to lecture	5.84	1.16	1.00	7.00
Proportion of class time devoted to class discussion	2.48	1.56	1.00	7.00
Proportion of class time devoted to group work	1.95	1.32	1.00	7.00
Agreement: Felt my hard work was reflected in my grades	2.68	0.84	1.00	4.00
Agreement: Felt comfortable asking questions in class	2.71	0.79	1.00	4.00
Agreement: I was motivated to try hard on course assignments and exams	3.07	0.74	1.00	4.00
Agreement: I received feedback that helped me learn and improve	2.62	0.77	1.00	4.00

Variables

The dependent variable in the quantitative analyses represents the extent to which students reported being academically engaged in their introductory STEM course. Eight items compose the factor of behavioral academic engagement: frequency with which students asked questions in class, discussed course grades or assignments with the instructor, attended professor’s office hours, participated in class discussions, tutored other students in their introductory STEM course, reviewed class material before it was covered, attended review or help sessions to enhance understanding of course content, and studied with students from their introductory STEM course. In creating this construct, we relied on the work of Handelsman et al. (2004) and Fredricks et al. (2004), with the items reflecting the behaviors of students who strive for excellence in learning science and are the activities of successful students. Principal axis factoring suggested a single-factor solution for these eight items, and Table 2 provides the factor loadings for an academic engagement construct, which had a Cronbach’s α of 0.75.

Table 2

Factor loadings for academic engagement

	Cronbach’s α	Factor loading
Academic engagement	0.75
Asked questions in class		0.73
Discussed grades or assignments with the instructor		0.71
Attended my professor’s office hours		0.69
Participated in class discussions		0.59
Tutored other students in this class		0.57
Reviewed class material before it was covered		0.53
Attended review or help sessions to enhance understanding of the content of the course		0.46
Studied with other students from this course		0.38

Note. The scale for each the first seven items ranged from 1 = never to 5 = very often; the scale for “studied with other students from this course” ranged from 1 = 0 h per week to 13 = more than 10 h per week

The initial exploratory factor analysis included items related to emotional engagement, such as feeling excited about learning new concepts, and these items loaded separately on a factor that appeared to represent emotional engagement. We detected no cross-loading of items between the behavioral and emotional engagement factors and decided to proceed with statistical model that used course experiences, background characteristics, emotional engagement measures, and classroom contexts to predict variation in students’ behavioral academic engagement.

To predict variation across students in their reported level of academic engagement in their introductory STEM course, we included several student-level predictors that address students’ demographics, prior academic achievement (Rocca 2010), intrinsic and extrinsic motivation (Hofer 2002; Ryan and Deci 2000), and connections to and perceptions of institutional agents, or faculty (Stanton-Salazar 2010). The analyses examined the predictive power of aspiring to a medical degree (compared to any other degree), which served as a proxy for respondents’ extrinsic motivation, as we hypothesized that pre-med students may demonstrate greater levels of engagement due to the competitive nature of medical school admissions. Measures of intrinsic motivation included students’ self-rated competitiveness, resourcefulness, excitement for the course, and motivation to try hard on exams.

Regarding students’ connections to institutional agents, we examined how students’ connections with teachers in high school and their likelihood to seek tutoring or supplemental instruction (SI) related to their academic engagement in the course. Three items measured students’ reports of the pedagogical methods used in the course: proportion of time devoted to lecture, class discussion, and group work. Classroom climate was represented by students’ perception that their hard work was reflected in their grades (fairness), their sense of collaboration among their peers, and their level of comfort asking questions in class.

To account for variation across classrooms, we included several course-level variables collected from the faculty survey. The level-2 model includes faculty perceptions of the climate they perpetuate in their classroom, their goals for undergraduate education, and dummy variables representing whether faculty have earned tenure or are on the tenure track (compared to their colleagues who are not on the tenure track). Because of the limited number of institutions in our sample, we did not include any predictors at level 3, the institutional model. Table 4 in the Appendix includes all of the variables and their coding schemes.

Analyses

Before using multivariate statistical techniques to analyze the data, we first weighted the data to adjust for non-response bias and accounted for cases with missing data. To adjust for non-response bias, we weighted the data by race, gender, and class standing, all of which were collected for the population of students enrolled in our selected introductory STEM courses within each institution. We then used logistic regression to determine the probability of responding to both surveys for each student originally enrolled in the introductory course. The weight given to each student was the inverse of the probability of responding to both surveys. Thus, a student with a 33% probability of responding to both surveys received a weight of 3. We then normalized the weight to avoid inflated t-statistics.

To account for missing data, we relied on the expectation maximization (EM) algorithm for all ordinal and continuous student-level variables excluding the dependent variable. For student respondents with missing data on dichotomous variables or the dependent variable, we used listwise deletion of cases. The vast majority of variables had fewer than 5% of cases with missing data, and composite SAT score had the highest proportion of missing data at 12.1%. The EM algorithm relies on maximum likelihood techniques to impute missing values when variables have a small proportion of missing data (Allison 2002; McLachlan and Krishnan 1997), and thus this method provides for a more robust process of handling missing data than mean replacement or listwise deletion (McLachlan and Krishnan 1997). A drawback to the EM algorithm is that it provides a single imputation for missing values, and more recent research suggests missing values may be a source of variation in and of itself, and averaging over that variation requires more sophisticated methods, such as multiple imputation (Sinharay et al. 2001).

After accounting for missing data and weighting the data for non-response bias, we began descriptive and multivariate analyses of the data. The quantitative data represent a three-stage nested design, as students are clustered within classrooms which are clustered within institutions. Given the clustered nature of the data and the continuous outcome variable, we used a three-level hierarchical linear model (HLM). HLM represents the most appropriate statistical technique when analyzing multi-level, clustered data, as this method accounts for the homogeneity of errors with groups (i.e., classrooms and institutions); additionally, using HLM helps researchers to avoid making a Type I statistical error by erroneously concluding the significance of a parameter (Raudenbush and Bryk 2002). HLM avoids violating the assumption of independence of observations by partitioning variance to each respective level (i.e., student, classroom, and institution). Furthermore, HLM provides for a more efficient estimation of cross-level effects (i.e., slopes as outcomes) where analysts can examine how variables at one level affect the relationship between an independent predictor and the dependent variable at another level (Raudenbush and Bryk 2002).

To justify the use of HLM, the outcome variable must differ significantly across groups. For this study, we determined the extent of variation in the students’ self-reported academic engagement across classrooms and institutions by running a fully unconditional model (Raudenbush and Bryk 2002). The fully unconditional model produced estimates of the intra-class correlation (ICC), or the extent to which academic engagement significant varied across classrooms and across institutions. These estimates suggested that 3.1 and 4.1% of the variance in academic engagement was attributable to differences across classrooms and institutions, respectively. In other words, classrooms and institutions appear to have a marginal effect on students’ academic engagement, and the vast majority of variance we see in academic engagement can be attributed to differences between students. Although these ICCs indicate that nearly all (92.8%) of the variation in academic engagement occurs at the student level, given our interest in examining pedagogical issues of classroom climate and appropriately accounting for the three-stage nested design, we proceeded with an HLM analysis using a three-level model. Similar to other research with a limited number of observations at level 3 (e.g., Marks 2000), our level-3 model only includes the random intercept, as having just 15 institutions at level 3 limits our ability to make robust statistical inferences from institutional predictors; however, we believe it is important to account for the clustering of students and classrooms within institutions given potential similarities of courses and students within each institution (Raudenbush and Bryk 2002).

Qualitative Design

In the second, qualitative phase, we purposefully selected institutions based upon their high rates of participation in our quantitative survey and the resultant evidence of higher amounts of classroom innovation occurring on each campus. In sum, 41 focus groups were conducted over a five-month time span, from October 2010 to February 2011, with 241 student participants from eight universities across the United States: one HSI, one HBCU, and six predominantly White institutions (PWI). Of these institutions, four are publically funded, and four are privately funded. We include a chart summarizing key characteristics of each institution in Table 5 in the Appendix, where we report the full-time enrollment, funding, predominant racial designation, Carnegie Classification, region, SAT selectivity measure, and annual research dollars. Yet when reporting student quotes we include only the institutional descriptors of geographic location, public versus private, and Carnegie classification to provide the reader with an identifier based on context.

The student sample included 14% African Americans, 54% Whites, 8% Latino/as, 21% Asian Americans, and 3% Native American; 62% were women; 42% were freshmen, 33% sophomores, 18% juniors, and 1% seniors; 30% of the student sample described themselves as pre-med, while another 4% described themselves as pre-dental.

The student focus groups consisted of students either currently enrolled in introductory STEM courses or students who had completed these courses and participated in our quantitative data collection in spring 2010. We asked students to describe their experiences in introductory STEM courses through a series of nine main questions and corresponding probes, centering around student motivation, course structure, learning, instruction, and assessment, allowing their responses to dictate the order with which we asked the questions.

Focus group interviews, ranging from 60 to 90 min, were conducted with two to ten participants per session, averaging five focus groups per campus. We utilized a semi-structured interview technique that allowed us to respond “to the situation at hand, to the emerging worldview of the respondent, and to new ideas on the topic” (Merriam 1998, p. 74). Maxwell (2005) suggests that this technique increases the “internal validity and contextual understanding and is particularly useful in revealing the processes that led to specific outcomes” (p. 80). Prior to the interviews, participants were asked to complete a brief biographical questionnaire, which gathered data on a range of relevant background characteristics (e.g., demographic information, educational attainment, and research experience). All interviews were digitally recorded, transcribed verbatim by a professional transcription company, checked for accuracy, and loaded into NVivo8 qualitative software.

In order to develop the coding architecture utilized in NVivo, each transcript was open coded by examining the raw data and identifying salient themes supported by the text. This constant comparative approach followed an inductive process of narrowing from particular (text segments) to larger themes while allowing the researcher to attempt “to ‘saturate’ the categories—to look for instances that represent the category and to continue looking until the new information does not provide further insight into the category” (Creswell 2005, pp. 150–151). Our team of six researchers each read transcripts from two institutions, gathering and comparing themes across focus groups and institutions, which also enabled analytical triangulation (Patton 2002). Once we determined that we had reached saturation in generating themes, we developed several iterations of coding schemes, wherein codes were created, expanded, defined, and refined. These categories/themes in the raw data were then labeled as “nodes.” Six researchers thematically coded three randomly selected sections of text, and inter-coder reliability ratings consistently ranged between 80% and 85% (Miles and Huberman 1994). Following inter-coder reliability exercises, the coding was re-validated and we were able to add new codes and sub-codes where necessary. Once the coding structure was finalized, we utilized 22 primary nodes, 114 secondary nodes, and 14 tertiary nodes in NVivo8. The data selected were stored under the node and the link to the full record was maintained. After these bins of relevant data were created, we re-read the data repeatedly in order to solidify our understanding and see connections amongst the categories. Queries were run linking participant attributes with coding references.

Findings

Academic Engagement in STEM Classrooms

Quantitative results provide general information about the relationship between student learning strategies, faculty attitudes and characteristics, pedagogical techniques, and student level engagement in introductory STEM courses and are enhanced by qualitative findings that provide more detail about student perspectives about their own and faculty behavior. The findings are integrated, or “mixed,” throughout the results section and discussed in terms of direction (i.e., more vs. less engaged). Table 3 provides a full accounting of the HLM results analyzing students’ academic engagement. Among the student-level variables, the findings suggest that White and students of color do not significantly differ in their reported level of academic engagement in introductory STEM courses. Likewise, we detect no significant differences across gender. Only one of the four pre-college preparation variables remained significant to the final model: high school chemistry grade. Students who reported earning higher grades in high school chemistry also reported themselves to be significantly more academically engaged in their introductory college STEM courses; however, high school biology grade, SAT composite score, and earning college math credits did not significantly predict students’ academic engagement in introductory STEM courses. Freshmen reported significantly higher levels of academic engagement in introductory courses than students who had been in college for a longer period of time, which suggests that students who wait to fulfill STEM course requirements are likely to be less engaged.

Table 3

HLM results predicting students’ academic engagement

	Coef.	SE	Sig.
Course-level variables
Faculty course goal: Use technology to effectively engage students	0.03	0.02
Faculty agreement: In my classroom, there is no such thing as a question that is too elementary	0.05	0.02	*
Faculty agreement: I feel it is primarily up to students whether they succeed in this course	−0.04	0.02	*
Faculty agreement: There is not enough time available to give every student individualized attention	−0.03	0.01	*
Tenure status: Tenured (reference group: not on tenure track)	0.09	0.04	*
Tenure status: Not tenured, on tenure track (reference group: not on tenure track)	0.10	0.06
Student characteristics and pre-college variables
Sex: Female	−0.03	0.02
Race: White (reference group: non-White)	0.02	0.03
SAT composite score	0.01	0.01
HS biology grade	−0.04	0.03
HS chemistry grade	0.05	0.02	*
Earned college math credits in high school	0.05	0.03
Freshman (reference group: non freshman)	0.11	0.03	***
Degree aspiration: Medical doctorate (reference group: all other aspirations)	0.05	0.02	*
Frequency: Asked a teacher for advice or help outside of class in HS	0.13	0.02	***
Students’ self-rated abilities
Communication skills	0.05	0.02	*
Initiative-taking	0.05	0.02	*
Ability to know when and whom to ask for help	0.06	0.02	***
Competitiveness	0.04	0.01	**
Course-related experiences
Agreement: Faculty gave students written feedback on their performance or progress in the course	0.05	0.02	***
Frequency: Sought tutoring from a campus office or program	0.19	0.01	***
Frequency: Sought a professional (off-campus) tutor	0.23	0.01	***
Frequency: Attended supplemental instruction sessions	0.05	0.01	***
Emotional academic engagement
Frequency: Felt excited about learning new concepts	0.08	0.01	***
Frequency: Felt collaboration among other students in this course	0.09	0.01	*
Agreement: Felt my hard work was reflected in my grades	−0.05	0.02	*
Agreement: I was motivated to try hard on course assignments and exams	0.07	0.02	***
Faculty pedagogy
Proportion of class time devoted to lecture	−0.04	0.01	***
Proportion of class time devoted to class discussion	0.02	0.01	*
Proportion of class time devoted to group work	0.04	0.01	**
Classroom climate
Agreement: Felt comfortable asking questions in class	0.22	0.02	***
Faculty agreement: In my classroom, there is no such thing as a question that is too elementary	0.06	0.03	*
Faculty agreement: There is not enough time available to give every student individualized attention	−0.06	0.02	*
Agreement: I received feedback that helped me learn and improve	0.07	0.02	***
Model statistics
Level-1 variance explained	0.56
Level-2 variance explained	0.96
Overall variance explained	0.55

* p < 0.05; ** p < 0.01; *** p < 0.001

Although background characteristics and pre-college preparation largely lacked significance in predicting students’ engagement in introductory STEM courses, the quantitative findings suggest that students who felt excited about learning new concepts tended to report significantly higher levels of engagement. Initially, we examined students’ interest in taking introductory courses, and taking the course for personal interest predicted higher levels of engagement; however, when we added students’ sense of excitement about learning new concepts to the model, this variable accounted for the predictive power of the personal interest variable. This finding suggests that a genuine interest in learning, rather than simply striving to make a certain grade in the course, is significantly related to students’ engagement in their studies, which supports work by Ryan and Deci (2000) regarding intrinsic motivation and engagement. Our qualitative data enriches this point, as expressed by Marie:

I’m realizing for the first time, maybe, that with these classes, like, I want my knowledge to be furthered. I obviously want to get an A, but like, in my biology class he [the professor] was saying ‘You don’t necessarily have to read this chapter for the test, there’s some material you don’t necessarily need to know for the test, but you can if you want,’ and I found myself like, ‘I want to read this chapter’, you know? Like, I want to know, just to further my knowledge, to know the material better, and I think that – I dunno, just measuring success is like, furthering knowledge and enriching that, I guess. (Southwestern Private Research University)

Marie appears to be highly engaged in her introductory courses, as she is willing to go above and beyond course requirements to enrich her own knowledge and learn for learning’s sake. Her statement, however, was somewhat unique in that few others in the focus groups articulated engagement in the same way.

The Pre-Med Phenomenon – Career Motivation for Engagement

When asked why they enrolled in introductory STEM courses, most students simply said that they needed to fulfill a requirement for their major. We found that pre-med students, however, were an exception. The quantitative findings indicate that those students who came to college with aspirations for medical school tended to report being significantly more academically engaged than their peers with other educational goals. Given the competitive nature of medical school admissions, students with plans for medical school likely recognize the need to do as well as possible in their science courses, which may explain these students’ increased levels of academic engagement.

The pre-med students who participated in our focus groups had a keen awareness of the importance of introductory STEM courses and described needing to excel in order to form a strong science knowledge base to succeed in upper level courses, be prepared for their planned scientific careers, and do well on the MCAT. Maggie, a pre-med student, discussed her approach to introductory STEM courses:Maggie’s statement is indicative of the approach to learning and high level of engagement with the course and class content evidenced by the majority of pre-meds in the focus groups, and such a perspective underscores these students’ motivation to learn and master course content. While their learning is often motivated by a desire to be prepared for future steps along the STEM pipeline, that does not mean that grades are unimportant or that competition does not exist. In fact, the HLM results indicate that in these introductory courses, students who thought of themselves as more competitive than their peers tended to report significantly higher levels of academic engagement, even after controlling for respondents’ educational aspirations.

All my math and science classes I’m currently taking are a requirement for my major, and they fit nicely in this semester. However, you know, it’s not just you know, oh, I wanna get in this class and get an A, it’s also like you know, if I later on choose to take the MCAT you wanna know what you’re doing on there, and learn it well. (Southwestern Public Research University)

Although qualitatively we did not find substantial data regarding students’ internal sense of competition, the interaction of engagement and competition was evidenced in other ways. Students who were particularly engaged in their courses, who were striving to earn an A, most often described feeling a sense of competition in their classes, either between themselves and pre-meds or amongst pre-meds, as evidenced by student interaction. As Marian reported, it becomes “survival of the fittest.”This attitude extended not only to more weighty opportunities like internships but also into more fundamental daily interactions, even to something as simple as answering one another’s questions. Many students described either being unwilling to help their fellow students or encountering students unwilling to help others for fear of boosting other students’ grades, and being edged out of their A.

When I know that someone is pre-med, I mean, I can be friends with the person, but there’s always gonna be, if I know…that there is an internship for a hospital, I’m not gonna tell the person next to me that. I’m gonna get it, I’m gonna find out, I am, yes, sometimes I am very selfish when it comes to that. Because it’s – survival of the fittest. (Southwestern Public Research University)

In my [STEM] classes I, I just would be like, ‘Oh can we, like can you like explain this to me?’ and they’re kinda more like, ‘Well I kinda need to study. Like I don’t have time,’ or…I remember right before an exam I was in the hallway asking. I asked someone a question like, ‘Do you know about…something about mitosis.’ And they’re like, ‘Oh sorry I don’t know.’ Which I knew they knew because they were studying mitosis right on their sheet. (Rose, Midwestern Public Research University)

It is clear that these students are engaged, working hard, but also trying to outdo one another. This type of competition not only affects the climate inside the courses but also occasionally extends into interactions outside the classroom where students who do not view themselves as particularly competitive feel uncomfortable interacting with their more competitive peers, which affect attempts at collaborative learning, as Celina explained.Celina’s conception of other students “showing how much they know” or Rose’s notion of students just looking out for themselves speaks to the ways in which a competitive climate in these courses can potentially limit a fundamental mechanism of student engagement: genuine collaboration.

I don’t really, really like to do study groups because I feel like it’s just them trying to compete and them trying to show me how much they know versus how much I know, and I know some people do like study groups. It does help them, but because they’re competitive, and naturally I’m not really a competitive person so I usually study by myself. (Southwestern Private Research University)

Collaboration with Peers and Use of Interventions

Quantitatively, collaboration among peers positively predicted students’ level of engagement in the course. This finding suggests that more active student learning with peers provides a context that enhances students’ connection and interest in the class (Tagg 2003). Likewise, many students in the focus groups described the favorable influence of a collaborative environment in their introductory courses, which often encouraged them to engage more fully. As suggested by Stewart below, students were often more inclined to get involved in SI and/or tutoring sessions if other students joined them:

I know in math, I usually try to find people around me, like, in pre-Calculus I had a small group of people that sat around me in class. And you know, if I didn’t get something, it was easier to turn to them and see if any of them understood it. And if they did, then I could ask them for help, and it was a lot easier than trying to ask the professor, and then you know — you don’t wanna study alone for something you don’t get, so it’s nice to have a small group of people to go with you to like the Q lab or SI sessions and things. So that way, you know, if one person doesn’t get it then the other people can help him. If the [SI Leader] is not being as helpful as they could be, then there are other people to help back that up. (Southwestern Public Research University)

In considering use of SI and/or tutoring sessions, the quantitative findings suggest that students who spend more time with tutors, either on-campus or off-campus in private settings, tend to also have significantly higher levels of academic engagement than peers who seek out tutoring less frequently. Similarly, students who attended more SI sessions reported being significantly more academically engaged in their introductory courses. We know from our qualitative data that some institutions in our sample required these sessions of students whereas other campuses made SI optional. The quantitative findings suggest that, regardless of why students attend these sessions, more frequent attendance contributed to higher levels of engagement within the course. Our qualitative data provide a more nuanced view of how students’ use tutoring and/or SI sessions, suggesting that students often sought this additional instruction when they felt that their classroom-based instruction was insufficient for them to properly learn the material.

I absolutely didn’t understand anything the professors were telling me. So, I finally broke down and went to the tutors, the second time I had to retake Chemistry II, and it helped so much. The way they were explaining things, they actually got the models and showed me how the molecules break, and came together, or they would test me. They wouldn’t give me the answer, they’d make me work for the answer. I did so much better in that class. He [the instructor] just talked, and tried to explain it like, as if he was teaching in class again, so. I chose to go to tutors before I’d ever go to the professors. (Cadence, Northeastern Private Master’s College)

Cadence was not able to learn from her professor, so she took an additional step and ‘broke down’ by going to the tutoring center. Through the use of tutors who utilized active learning strategies like hands on models and problem based learning, she was able to better understand concepts and more fully engage with the content of her courses. Cadence’s issue with her professor is representative of sentiments expressed by many other students who struggled in lecture-based, teacher-centered courses.

Faculty Pedagogy and Engagement

Similarly, our quantitative findings suggest that students who described the course as predominantly lecture-based tended to report significantly less engagement in the course. These sentiments were echoed throughout the focus groups, as many students described the mind-numbing lectures that they regularly sat through, often feeling unengaged and unenthusiastic about the course. Talia described it well:

In a lot of my biology courses, the professor just sort of talks at me, and I’m like – I don’t feel, like, as engaged or I feel like, in those courses there is a lot more memorization only, which is why I don’t get as much out of them because I’m very hands-on. Like, if I’m doing something in the class, I can grasp that I’m understanding it, but if the professor is just talking at me, writing stuff on the board, expecting me to write it down, like that’s doing nothing, and then I’m completely disengaged in class. (Southeastern Private Master’s College)

By contrast, quantitative analyses revealed that sensing a greater proportion of time devoted to class discussion or group work seemed to enhance students’ frequency of engaging with course content and connecting with faculty inside and outside the classroom. Not only did the focus groups confirm this finding but they also provided more details about the innovative pedagogical techniques that students found to be most engaging. We present exemplars representing the three innovations students discussed most often: group work, clickers, and web-based pedagogy.

For my class now he gives us like the first five minutes of class, like he’ll assign us groups and then say just discuss the homework…just getting in groups, sometimes explaining it as a student to another student who’s just read the same chapter as you or something, who maybe understands it a little better can be a lot, can be more helpful than even talking to a professor who’s not sure what you’re confused about or something like that. So I think group work in class can really be, can really be effective. (Bernadette, Southeastern Private Master’s College)

They use clickers like, throughout their lecture. So, as you’re like, following along and taking notes, like, they’ll ask a question, it’s good to get some interaction, I think. And that kinda keeps you like, waiting…it keeps you kind of into it when they get to a question…they’re not just sitting up there teaching the whole time, they’ll ask, like in the middle of a lecture, he might ask you like five questions in a row about what he’s lectured on so far, so you can kind of apply what you’ve learned, if it’s chemistry or if it’s biology you can just kind of see, if you remember what you just covered in the last like, 20 min. So I think that’s really helpful for me. (Ashley, Northeastern Private Master’s College)

In biology we used this website called mastering biology and it has like videos and questions and quizzes online and different like videos of the cells. Then you can have videos of it being used in real life with the organisms themselves so it gives you like different perspectives. It was really interesting. (Holly, Western Private Master’s College)

Students described these pedagogical techniques as “helpful,” providing “interaction,” and representing “different perspectives,” – all words indicating the positive effects of these active learning strategies on students’ engagement. Yet while the majority of students preferred these active teaching approaches and recognized their benefits, a small number of students in the focus groups, primarily those who were pre-med, indicated that they prefer STEM courses to be mostly lecture, since there is so much information to be covered. Even when lecture was the primary vehicle for conveying course content, the professor’s attitude, knowledge base, enthusiasm for the subject, and ability to explain the content clearly were all important characteristics that influenced students’ level of engagement. These qualitative findings suggest that the lecture itself may not be the problem, but the style of lecture combined with the professor’s own engagement profoundly impact students’ engagement. Here Harlow described the importance of professors’ enthusiasm for the subject matter they are teaching.

Well, I think there are some professors, too, who get real excited during lectures. They’re very excited and just passionate about the subject. When that happens, I feel it’s easier for you to become more excited about it and enjoy the class and come to class and be like, ‘Oh, this is fun. This is great.’ This professor enjoys what they’re talking about, so it’s easier to learn from them. It’s not just dull and boring. Those are usually the professors that have group activities or other things that are built in that allow for a more well-rounded way to learn than just lecture. (Western Private Master’s College)

Accessibility Cues and Engagement

While professors’ teaching strategies, attitudes, and enthusiasm were influential factors of classroom climate that shaped students’ engagement in introductory classrooms, the quantitative results also indicate that students who felt comfortable asking questions in class reported being significantly more engaged in their introductory STEM courses. This finding not only points to the effect of students’ self-confidence on engagement, as we controlled for self-rated communication ability and initiative-taking as well as prior behaviors in asking for help, but also underscores the overall climate students perceive in class. That is, students are likely to feel more comfortable asking questions in class when professors encourage questions and participation. In fact, we found that the positive benefits on academic engagement of feeling comfortable asking questions in class significantly varied across classrooms. Tests for cross-level interaction effects revealed that being in a classroom with a faculty member who felt there is no such thing as a question that is too elementary enhanced the positive benefits of students feeling comfortable asking questions (see Table 3). By contrast, being in a classroom where faculty felt there was not enough time to give every student individualized attention mitigated the positive relationship between feeling comfortable asking questions and academic engagement. Thus, although feeling comfortable asking questions may promote engagement, that confidence can be discouraged if faculty are dismissive or do not take the time to affirm a student’s inquiry.

Students in the focus groups also discussed the importance of professors providing a supportive environment in STEM introductory classrooms. Although some students said they felt uncomfortable asking questions in class, others stated that their professors created an environment that was friendly and playful, which made it easier to ask questions. This is highlighted by Carter’s words:

On the other hand, I have my [biology] class and it’s like 200 kids, and I don’t have a problem asking a question in that class because he’s just – he’s cool about it, you know…he has the attitude that there’s not a stupid question and he’s really neat about it. It’s like he’s heard them all, you know, and he’ll make jokes about stuff and things like that and he really has a really playful attitude. (Southwestern Private Research University)

The connection between academic engagement and students’ perception of classroom climate is further underscored by results of the course-level variables in the level-2 model of the quantitative analysis. For example, significantly higher levels of academic engagement were reported in classrooms where faculty agreed with the statement that “there is no such thing as a question that is too elementary” (Table 3). When faculty are open to student questions in this way, they may be communicating a broader set of explicit or implicit messages that signal to students that the classroom represents a safe environment, which in turn may encourage students to become more engaged in and curious about course content. Even in a situation where the professor initially seems intimidating, students may become more engaged after receiving a small signal from the professor, as illustrated by Bella’s words:

My Chemistry teacher, at first he looked scary cause he was almost like kind of yelling. But I guess that’s how he is. But there was this one time like, everybody was scared to answer this one question. And then he was like, ‘Why?’ And then one student was like, ‘Cause I don’t understand,’ and then he was like, ‘There you go. There you go. Just ask me what it is,’ and then he gave her like five points on her exam. So, after that we were all like, ‘Hey Professor, how do you do this?’…he was like, really open. (Southwestern Public Research University)

Beyond encouraging students to ask questions, the classroom climate may be further enhanced when professors provide immediate feedback and make themselves accessible to students. Findings across classrooms show that students who received feedback that helped them to learn and improve reported being significantly more engaged in their introductory STEM courses (see Table 3). Students also discussed the value of such immediate feedback as they described the value of clickers and other student response systems based on the same principles.

By contrast, findings indicate that students had significantly lower levels of academic engagement in classrooms where faculty reported a lack of time to provide them with individualized attention or where faculty agreed that it is primarily up to students to be successful in their introductory courses (Table 3). Together these classroom-level findings point to how accessibility cues from faculty can affect engagement; thus, if students perceive faculty to be uncaring, unengaged, or unavailable to help them succeed in learning, they may disengage from the course. This finding is further highlighted by Tilly’s words:When professors signal that they are available, students are more likely to ask questions, attend office hours, and participate in out-of-class review sessions with the professor. Our findings highlight the importance of the professor in creating an engaging classroom environment in order to enhance student learning.

There are some situations where you can reach out to a professor, but there are some where you can’t. You can suggest to get a study session going for a certain course, but if that professor doesn’t have enough time or doesn’t have time available to meet up with everyone, then it’s like a wasted attempt. And after a while, once you keep attempting and attempting to get those study sessions going and that professor is continuously unavailable, it kinda like replays in your head. Okay, well, I might as well just not even try with that professor. Just do it on my own as I’ve been doing. (Southeastern Public Master’s College)

Empowered and Resourceful Students

While many students expect faculty to provide support, encouragement, and space for mastering course content, students must still play an active role in their own learning. Those students who embrace their own agency and actively seek out professors are typically the most engaged. Indeed, considering students’ self-ratings and educational aspirations, the quantitative results indicate that students who conceived of themselves as more resourceful at the beginning of the academic term, as indicated by higher self-ratings on communication skills, initiative-taking, and ability to know when and whom to ask for help, reported being significantly more academically engaged in their introductory STEM courses. Likewise, respondents who reported more frequently asking a teacher for advice or help outside of class in high school tended to have higher levels of academic engagement in college introductory STEM courses. Given that the outcome for the quantitative portion of the study included items related to asking questions in class, it is not surprising that students with better communication skills, initiative taking, and an understanding of how to be resourceful with instructors significantly and positively relate to students’ reported level of academic engagement in their introductory STEM course.

Through our qualitative data, we are able to describe the numerous ways in which engaged students enacted this resourcefulness in terms of interacting with faculty members. Some described interactions as simple and straightforward as asking a question in class, but even this act requires students to initiate the interaction and as Marshall described, many students are reluctant to do this:Other students took a different approach and sought professors outside of class, utilizing their resourcefulness to actively engage in their own learning. There were three typical ways in which students sought this additional communication with their professors: after class, during office hours, and through email. In these ways, many students were able to actively engage with their professors and courses. Through these interactions, students increased their understanding of course content, built relationships with their professors, and got the help that they needed by being resourceful and learning how to acquire their professors’ support.

I don’t usually ask questions just to ask questions, the professors will ask, ‘Does everyone understand? Does anyone have any questions?’ and then at that point I’m like, ‘I do not know what he’s talking about at all’, so I’ll ask a question. I think a lot of different people do that but some people just don’t really. I think sometimes professors like want you to ask questions…because I think it helps them understand like instead of just going through and lecturing all day. (Southwestern Private Research University)

Well, I think there’s just like different ways of teaching that topic. Just sometimes if they can go over – and a lot of times, you can ask, I can go and talk to my professor after class and he would explain it a different way and make more sense. So, I think just being able to approach your professors really is helpful. (Finnian, Western Private Master’s College)

I’d have to say probably office hours, like sitting and talking to the professor does help out, because they know the stuff. And they can probably have a different way of explaining it than what they did in class. (Blaze, Western Public Research University)

So, I try to e-mail ‘em, and most of ‘em are pretty good at getting back e-mails, and sometimes they’ll work around your schedule too. ‘Cuz like the teachers that I had, they always wanted you to know if you need any extra help, they’re there…If you really want to get help, they will help you. (Sky, Southwestern Public Research University)

Conclusion

With increased interest in STEM among entering students, the U.S. is at a critical crossroads in terms of its opportunity to improve the production of science degrees at four-year colleges and universities. Indeed, students’ intentions to major in biomedical and behavioral sciences have nearly doubled in the last 15 years, and engineering fields have regained student interest (Pryor et al. 2007); additionally, there has been increasingly more focus on STEM education from a national perspective. These positive developments may be offset by the substantial proportion of intended science majors who leave STEM fields by the end of the first-year of college (Chang et al. 2008) often because of their experiences in introductory STEM courses. Even if we significantly raised the level of student preparation in high school science, it may not necessarily improve STEM degree completion unless we also address engagement in college introductory courses. Such academic engagement has as much to do with the engagement behaviors and attitudes of faculty who teach these courses as it does with motivated, resourceful, and engaged students themselves.

We have considered academic engagement mainly from a behavioral angle, combining elements from several studies examining behavioral aspects of engagement. Additionally, we have considered the psychological traits of an engaged student, while taking into account the associations between instructor characteristics and engagement. It is this comprehensive conceptualization of academic engagement that sets our study apart from current literature. In fact, some scholars suggest that the term engagement should be reserved specifically for work such as this, where multiple components are considered as part of the dynamic between faculty and students (Fredricks et al. 2004; Wentzel and Wigfield 2009). In examining students’ engagement across multiple classrooms and institutions utilizing both quantitative and qualitative data, we were able to learn a great deal more about engagement in introductory courses. The measure of academic engagement we employed emphasizes striving for excellence, as articulated in Essential Learning Outcomes for personal responsibility associated with learning (AAC&U 2002).

We found that students’ behavior, emotions, and cognition were important factors in predicting their academic engagement in introductory STEM courses. Students who were genuinely interested in course content, learning for the sake of learning, intrinsically motivated, and resourceful tended to be the most engaged. Supporting the findings of Birch and Ladd (1997), these students possess strong communication skills, take initiative, and have the ability to know when/whom to ask for help. Academically engaged students not only ask questions in class, they also interact with their professors after class, during office hours, and via email. This is in line with current literature on student-faculty interaction, where increased student-faculty interactions are positively related to student engagement (Tinto 2000; Umbach and Wawrzynski 2005). Further, engaged students utilize all available resources, including SI (whether it is required or not), tutors, and review sessions, and they regularly collaborate with their peers, whether in formal or informal study groups, or working homework problems. These findings echo those of Light (2001) and Horowitz (2010) which also found that more successful and engaged students studied in structured groups with their peers, as a result of formal and/or informal course activities.

We also found that in introductory classrooms, pre-med students were more engaged than their peers, as pre-meds were motivated by a range of unique factors, most commonly a strong desire to master content in order to build a firm foundation for upper level courses or for future careers, as well as in preparation for the MCAT. Given the potential selection bias among pre-med students, who typically have stronger academic preparation than their peers not intending to pursue a medical degree, future research should use advanced statistical techniques, such as propensity score matching, to determine whether pre-meds’ significantly greater level of academic engagement remains after accounting for potential biases in students’ self-selection into a pre-med tract. Our findings seem to combine conflicting conclusions from the literature regarding the motivational factors driving pre-med students. While some research supports the explanation that pre-med students are highly motivated by the desire to master course content and by intellectual stimulation (Horowitz 2010; Lovecchio and Dundes 2002), other research highlights pre-med students’ exclusive focus on high grades and performance on the MCAT as key motivating factors (Kortemeyer 2007). Whatever their motivation, we found that pre-med students’ behavior can undermine collaboration by increasing other students’ sense of competition, which in turn can negatively affect all students’ learning in introductory courses. Faculty can address the negative effects that emerge from a highly competitive climate by employing practices that move away from gatekeeping strategies where few are expected to do well and instead move toward practices that ensure more students can succeed.

Indeed, professors play an important role in sustaining engagement in science by signaling accessibility cues through a unique set of behaviors and attitudes, which can have a tremendous effect on student learning and engagement. This is well supported in the literature, where faculty practices such as active learning and higher-order cognitive activities create environments that positively relate to student engagement (Chickering and Gamson 1987; Ewell and Jones 1996; Kumbach and Wawrzynski 2005; Tinto 1993, 2000). Additionally, we found that students tend to be more engaged when professors create an atmosphere where inquiry is valued and no question is too elementary. Confirming studies by Light (2001) and Horowitz (2010), we found that a professor’s demeanor and attitude signal implicit and explicit messages that influence whether or not students feel engaged in class, and those professors who utilized humor, exhibited care, or showed a real passion for their subject matter were more likely to be viewed by students as the most engaging.

It is clear that professors’ behaviors are just as important as those of their students in determining engagement. We found that when professors utilized active learning pedagogies, employed teaching strategies that offer immediate feedback, ensured that more students have grasped the material, or made the most of new technologies for illustrating concepts, students became more engaged in courses. Professors can also enhance engagement by encouraging student collaboration through group projects and assignments that require teamwork. These findings support previous research that found that adding cooperative learning strategies increase educational gains, conceptual understanding, and engagement (Knight and Wood 2005; Kumbach and Wawrzynski 2005; Tagg 2003; Zeilik and Morris 2004). By contrast, consistent with previous findings (Biggs 1999; Bransford et al. 2000; Moore et al. 1996; Seymour and Hewitt 1997), students were less engaged in teacher-centered, lecture-based classes, where they must figure things out on their own and faculty do not check for student understanding except via exams or other high stakes evaluation.

Composite Representations

Based on these mixed methods findings, we have gained tremendous insight into the traits of “Gatekeeper professors,” “Engaged STEM professors,” and “Engaged STEM students.” We present these idealized composite representations as narratives combining many of the qualities of students and professors that significantly predict students’ level of academic engagement in the introductory STEM classroom. Sewn together from the words of students in the focus groups, and while realizing that most professors and students are complex combinations of positive and negative traits, these composites provide a rich context regarding what has been, what is possible, and what we should be avoiding or striving to achieve in STEM education, while also providing a more holistic depiction of our findings.

The “Gatekeeper” Professor

The term “gatekeeper” has been used to describe introductory STEM courses, but perhaps this analogy is better suited for describing the approach undertaken by some introductory STEM course instructors. Gatekeeper professors lecture straight from a PowerPoint while students hang on their every word. It is nearly impossible to write down everything the gatekeeper says, let alone process the information, so students make the personal decision to listen, in hopes of understanding, or to take notes, in hopes of making sense of it later. A brave student may ask the gatekeeper to slow down, but most students know that the gatekeeper does not want to slow down or repeat what has already been said. Students may have a better chance of processing the information in lecture if they had an outline to follow, but the gatekeeper does not believe in posting the PowerPoint lecture online because this may discourage students from coming to class. Students will not ask questions in class because the gatekeeper has already trained them that lectures are for “listening,” not for interacting.

Additionally, gatekeeper professors disregard individual learning styles because they are so focused on conveying the abundance of information that must be passed on to students who are worthy of passing through the gates. Their expectation is that students can and should understand the content at a sophisticated level. The gatekeeper may deflect students from the major by continuously making content feel intimidating and difficult to learn. Students who learn through visual presentation or hands-on methods often struggle with the gatekeeper’s style of lecture. Students may wonder if all STEM courses will lack the engagement and real-world application that they need to learn the material. Others get discouraged because they spend so much out-of-class time trying to understand the concepts that the gatekeeper failed to explain clearly during lecture. Beyond lecture, the gatekeeper is inaccessible through office hours and e-mail, making it difficult for students to get additional help. Clearly there is a disconnect between the gatekeeper’s style of teaching and the learning style of students, leaving most feeling disengaged from their introductory STEM courses.

The “Engaging” Professor

The “gatekeeper” is a well known image that has had a lasting impact in higher education classrooms despite calls for more engaging pedagogy in the STEM disciplines. The image of an “engaging” professor, however, has begun to emerge from more innovative institutions, and students have taken notice. The engaging professor uses strategies that encourage active learning, cooperation among students, and student-faculty contact (Chickering and Gamson 1987; Tagg 2003). A collaborative learning environment is fostered by the engaging professor, both in- and out-of-class. After engaging professors explain a concept—for example, the way blood flows through the heart—they will ask students to get into groups and explain the concept to each other. Walking around the room allows the engaging professor to gauge the general level of understanding while students personally evaluate their own ability to explain the way blood flows through the heart. Out-of-class group projects also foster a collaborative spirit amongst students while encouraging students to process the material beyond the lecture. Through course websites, the engaging professor provides students with a one-stop shop for downloading PowerPoint presentations to be used in lecture, podcasts from previous lectures, video clips that can enhance understanding of difficult topics, and discussion groups that encourage additional collaboration amongst students. The night before a quiz, students utilize the web-based discussion groups to ask each other questions while the engaged professor continues to facilitate discussions from the comfort of home.

The engaging professor also facilitates student excitement in the classroom through humor, enthusiasm, and practical application. The excitement and passion for the subject is contagious, and students begin to have fun and learn in an environment that fosters interest in STEM disciplines. Studying disease suddenly becomes more enjoyable because the dense content is applied to a real-world problem that students can relate to, such as cancer or HIV. Beyond real-world applications, the engaging professor uses physical objects, including Legos, puppets, play guns, and balloons, to model the concepts in class. This strategy helps students to understand subjects such as physics, microbiology, and organic chemistry because the abstract concept is now connected to a three dimensional demonstration. Beyond the classroom, the engaging professor is highly accessible to students and encourages them to participate in additional learning opportunities provided by the university. The engaging professor responds to e-mails, encourages students to stop by for office hours, and provides additional assistance at SI workshops. There is no limit to the things the engaging professor will do to get students motivated in their STEM major and excited about the possibilities of pursuing a STEM career.

The “Engaged” Student

In introductory STEM courses, an engaged student is often, but not always, pre-med and looking to bolster her knowledge base for the MCAT or more advanced science courses, as well as to fulfill major requirements. This student is very focused on getting high grades while also mastering the material. The engaged student chooses professors strategically, looking for engaging and thorough teachers by utilizing Rate My Professor or through advice from upper classmen. Armed with strong pre-college preparation, this student enters these courses with eagerness and a strong interest in science, which is evidenced by commonly seeking out professors after class, attending office hours, and emailing them regularly.

The engaged student is incredibly resourceful, taking advantage of multiple opportunities to enhance learning. Regular attendance is commonplace at class-related SI or recitations. This student has strong study skills and often meets with peers in self-initiated study groups where they re-teach one another content and prepare for tests together. Tutors and tutoring centers provide additional opportunities for the engaged student to work with more advanced students or graduate students to clarify understanding. This student takes advantage of any available opportunities to collaborate with professors and build relationships with them by participating in undergraduate research or by becoming an SI instructor or TA. Assuming professors are somewhat engaging, the engaged student’s experience in introductory STEM courses typically furthers interest in the topic and dedication to science.

Implications for Research and Practice

Keeping these composites in mind, we turn to the ways in which we can work towards transforming introductory STEM instruction from gatekeeping to engaging. Most STEM faculty prize the “engaged student” who goes the extra mile to learn material, demonstrates intellectual curiosity, and strives for excellence in class. Their own attitudes and behaviors, however, are critical in producing more of these students and in the process of transitioning their role from “gatekeepers” to “engaged faculty.” Although multiple authors have stated that few faculty are familiar with the science education literature and unaware of the data and analyses that demonstrate the effectiveness of active learning techniques (Handelsman et al. 2004; Labov 2004), the burgeoning body of literature indicating the importance of active learning techniques can no longer be ignored (see Caldwell 2007; Crossgrove and Curran 2008; Knight and Wood 2005; Preszler et al. 2007; Prince 2004; Smith et al. 2005; Zeilik and Morris 2004). If the goal is to increase STEM degree attainment, STEM faculty must begin to take responsibility for engaging students in the classroom. While we recognize that faculty face considerable demands besides teaching and that it takes substantial time to develop an active learning approach, more and more introductory STEM instructors are embracing the call to enact change in their classrooms, as this approach is truly vital to student success, as shown in this study. The scope and depth of re-shaping classrooms into engaging spaces through curriculum redesign and adoption of innovative teaching techniques can certainly be enhanced through institutional support that encourages and sustains those efforts.

In addition to supporting faculty in their desire to revamp their introductory STEM classroom, institutions need to simultaneously take proactive steps to train students how to be more like the “engaged student.” These strategies can involve tailoring orientation activities to students’ potential majors, assigning upper classmen as mentors, reinforcing available resources repeatedly throughout the semester, perhaps through STEM study habit seminars, or increasing mandatory student collaboration both in and out of class. One highly effective way to increase student collaboration and engagement is to change the landscape of large lecture classes, as many of the institutions in this study have done, by adding smaller, required class sessions, whether called “workshops,” “SI,” or “recitation.” Additionally, as students may come to college with study habits poorly adaptable to college-level work, faculty need to be prepared to help students learn how to learn or at the very least make learning more engaging by utilizing more innovative pedagogy and making themselves available and accessible to their students. It is critical to recognize that these innovative pedagogies need not be cost prohibitive. For example, several instructors in our sample whose institutions could not afford clickers instead utilized paper response systems, single pieces of paper, with A, B, C, D in each corner, folded into four blocks, thus allowing the professor to query their students and keep them engaged all for the cost of printing one page per student.

As educators adopt well-established active learning pedagogies and search for new ways to engage students and enhance learning environments, institutional researchers can also play a key role by helping to identify gatekeeper courses that curtail students’ aspirations in the sciences, as well as those that increase student engagement. This involves not only assessing student behaviors but also faculty behaviors across classrooms and sections, and communicating findings across faculty and departments, organizing faculty retreats and having them observe and/or train one another in active learning strategies, all of which can be inexpensive and effective ways to improve pedagogy across campus. At the same time, institutional researchers should continue to think about various methods for gathering data related to introductory STEM classrooms, including the use of mixed methods strategies that enable thorough, rich exploration. Finally, such assessments should result in feedback to faculty so that they have a greater context for understanding student success strategies influenced by their own behaviors; for example, faculty who participated in this mixed method study received reports about their classrooms.

In conclusion, education is fundamentally a human enterprise and, accordingly, this study focused on the unique dynamic between faculty and students within key STEM courses. At the broadest level, our findings show that the actions of faculty affect the actions of students, in this case students’ engagement with their introductory courses. Likewise, the actions of students affect their own success and faculty willingness to engage them. These combined and accumulated interactions also shape the climate or shared understanding that exists within classrooms, which can either enhance or suppress engagement. If educators are the key change agents in this dynamic, the findings suggest that introductory STEM course instructors must think just as carefully and thoroughly about how they interact with and come across to students as they do about the course content and how to assess its mastery, especially when it comes to scaling up STEM achievement and increasing student persistence.

Table 4

Table of measures

Variable	Coding
Dependent variable
Academic engagement	Factor composed of eight items: frequency with which students asked questions in class, discussed course grades or assignments with the instructor, attended professor’s office hours, participated in class discussions, tutored other students in their introductory STEM course, reviewed class material before it was covered, attended review or help sessions to enhance understanding of course content, and studied with students from this course
Course-level variables
Faculty course goal: Use technology to effectively engage students	1 = strongly disagree to 4 = strongly agree
Faculty agreement: In my classroom, there is no such thing as a question that is too elementary	1 = strongly disagree to 4 = strongly agree
Faculty agreement: I feel it is primarily up to students whether they succeed in this course	1 = strongly disagree to 4 = strongly agree
Faculty agreement: There is not enough time available to give every student individualized attention	1 = strongly disagree to 4 = strongly agree
Tenure status: Tenured (reference group: not on tenure track)	0 = no, 1 = yes
Tenure status: Not tenured, on tenure track (reference group: not on tenure track)	0 = no, 1 = yes
Student-level variables
Sex: Female	0 = no, 1 = yes
Race: White (reference group: non-White)	0 = no, 1 = yes
SAT composite score	Continuous, range 400–1,600
HS biology grade	0 = F to 4 = A
HS chemistry grade	0 = F to 4 = A
Earned college math credits in high school	0 = no, 1 = yes
Freshman (reference group: non freshman)	0 = no, 1 = yes
Degree aspiration: Medical doctorate (reference group: all other aspirations)	0 = no, 1 = yes
Frequency: Asked a teacher for advice or help outside of class in HS	1 = never to 3 = frequently
Self-rating: Communication skills	1 = lowest 10% to 5 = highest 10%
Self-rating: Initiative-taking	1 = lowest 10% to 5 = highest 10%
Self-rating: Ability to know when and whom to ask for help	1 = lowest 10% to 5 = highest 10%
Self-rating: Competitiveness	1 = lowest 10% to 5 = highest 10%
Agreement: Faculty gave students written feedback on their performance or progress in the course	1 = strongly disagree to 4 = strongly agree
Frequency: Sought tutoring from a campus office or program	1 = never to 5 = very often
Frequency: Sought a professional (off-campus) tutor	1 = never to 5 = very often
Frequency: Attended supplemental instruction sessions	1 = never to 5 = very often
Frequency: Felt excited about learning new concepts	1 = never to 5 = very often
Frequency: Felt collaboration among other students in this course	1 = never to 5 = very often
Proportion of class time devoted to lecture	1 = 0% to 7 = 100%
Proportion of class time devoted to class discussion	1 = 0% to 7 = 100%
Proportion of class time devoted to group work	1 = 0% to 7 = 100%
Agreement: Felt my hard work was reflected in my grades	1 = strongly disagree to 4 = strongly agree
Agreement: Felt comfortable asking questions in class	1 = strongly disagree to 4 = strongly agree
Agreement: I was motivated to try hard on course assignments and exams	1 = strongly disagree to 4 = strongly agree
Agreement: I received feedback that helped me learn and improve	1 = strongly disagree to 4 = strongly agree

Table 5

Qualitative institutional characteristics

Institution	Full time enrollment	Funding	Predominant racial designation	Carnegie classification	Region	SAT selectivity measure 75% percentile	Annual research dollars (million)
Southwestern Private Research University	>10,000	Private	Predominantly White	Research Universities (high research activity)	Southwest	1,290	~26
Southeastern Private Master’s College	<10,000	Private	Predominantly White	Master’s Colleges and Universities (smaller programs)	Southeast	1,310	~3
Midwestern Public Research University	>25,000	Public	Predominantly White	Research Universities (very high research activity)	Midwest	1,430	~890
Southeastern Public Master’s College	<10,000	Public	Historically Black	Master’s Colleges and Universities (larger programs)	Southeast	930	~4
Western Private Master’s College	<5,000	Private	Predominantly White	Master’s Colleges and Universities (larger programs)	West	1,290	~2
Northeastern Private Master’s College	>10,000	Private	Predominantly White	Master’s Colleges and Universities (larger programs)	Northeast	1,310	~34
Western Public Research University	>25,000	Public	Predominantly White	Research Universities (very high research activity)	West	1,400	~870
Southwestern Public Research University	>25,000	Public	Hispanic Serving	Research Universities (high research activity)	Southwest	1,140	~65