Context Matters: Social Psychological Factors That Underlie Academic Performance across Seven Institutions.

To enhance equity and diversity in undergraduate biology, recent research in biology education focuses on best practices that reduce learning barriers for all students and improve academic performance. However, the majority of current research into student experiences in introductory biology takes place at large, predominantly White institutions. To foster contextual knowledge in biology education research, we harnessed data from a large research coordination network to examine the extent of academic performance gaps based on demographic status across institutional contexts and how two psychological factors, test anxiety and ethnicity stigma consciousness, may mediate performance in introductory biology. We used data from seven institutions across three institution types: 2-year community colleges, 4-year inclusive institutions (based on admissions selectivity; hereafter, inclusive), and 4-year selective institutions (hereafter, selective). In our sample, we did not observe binary gender gaps across institutional contexts, but found that performance gaps based on underrepresented minority status were evident at inclusive and selective 4-year institutions, but not at community colleges. Differences in social psychological factors and their impacts on academic performance varied substantially across institutional contexts. Our findings demonstrate that institutional context can play an important role in the mechanisms underlying performance gaps.

INTRODUCTION

Broadening participation in science, technology, engineering, and mathematics (STEM) fields requires acknowledging the many different student experiences in higher education. Students who identify with groups historically excluded from STEM fields face barriers as they navigate these pathways into the STEM workforce (Seymour and Hewitt, 1997; Seymour and Hunter, 2019). Social challenges that disproportionately affect such students include social exclusion (Hurtado and Ruiz, 2012), a negative campus climate (Koo, 2021), and discrimination from peers and authority figures (Hurtado and Ruiz Alvarado, 2015; Park ). Meanwhile, national efforts call for more STEM graduates to address the shortage of STEM workers in the labor market and to expand underrepresented minority (URM) participation (American Association for the Advancement of Science, 2011; National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2011; President’s Council of Advisors on Science and Technology, 2012). Increased participation of students in STEM education will directly impact national science literacy, provide high-quality education to the STEM workforce, and contribute to critical scientific advances. One significant barrier to student participation and persistence in STEM is introductory STEM courses that students are required to complete before moving on to more advanced course work (Crisp ; Mervis, 2011). In such classes, instructors deliver foundational content knowledge—often through undisrupted lecture (Stains )—to relatively large lecture halls of students. Failure and attrition rates are high (Seymour and Hunter, 2019), and students perform poorly in these courses relative to non-STEM courses (Mervis, 2011; Koester ).  URM students face additional barriers in these introductory courses, in large part due to gaps in incoming preparation (Harris ; Salehi ) and limited opportunities in some institutions to interact with instructors (Hurtado ). These barriers lead to underperformance and lower probability of persisting in STEM fields for URM students compared with their overrepresented majority (non-URM) peers (National Science Foundation, 2019; Salehi ).

To address student performance barriers in introductory biology courses, recent research in biology education focuses on best practices for evidence-based teaching strategies that improve student academic performance (Offerdahl ; Freeman ; Theobald ). Though important to our understanding of student learning, the majority of current research into student experiences in introductory biology takes place at large, predominantly White institutions, such as public, 4-year PhD- and master’s degree–granting research-intensive universities (hereafter, selective institutions; Schinske ; Thompson ). This overrepresents the experiences of these students in the contemporary literature. Less research has focused on other types of institutions, such as community colleges, minority-serving institutions, or more inclusive 4-year institutions (hereafter, inclusive institutions), even though they are essential educational institutions contributing to our future STEM workforce (Thompson ). For example, while community colleges serve almost half of all undergraduate students in the United States, and an even larger percentage of URM and first-generation college students (American Association of Community Colleges [AACC], 2021), relatively little biology education research (BER) focuses on community colleges (Schinske ). Further, the authors who conduct research on community colleges are often not affiliated with community colleges and less familiar with the specific institutional context (Schinske ). The reason for this discrepancy is likely due to heavy teaching and service expectations for community college faculty, but still represents a gap in our understanding of the student experience.

By developing and implementing innovations and reforms informed primarily by research conducted at selective institutions, we overlook important contextual factors that likely differ between selective institutions and community colleges and inclusive institutions; factors such as instructor teaching loads and expectations, students’ levels of academic preparation, their educational experience based on salient social identities such as gender or race/ethnicity, and students’ socioeconomic status. Pursuing research questions across multiple institutional contexts enables research to address questions that are unattainable within a single institution. Multiple factors specific to an individual institution are likely to impact research outcomes. These could include (but are not limited to) teaching culture, student body, or class size. To demonstrate the importance of institutional context, we examined differences in demographic performance gaps, as well as two psychological factors, specifically test anxiety and ethnicity stigma consciousness (hereafter ESC; Sarason, 1961; Pintrich et al.,1993; Steele and Aronson, 1995; Picho and Brown, 2011). We tested their impacts on student performance across seven institutions from three different institution types: community colleges, 4-year inclusive institutions, and 4-year selective institutions. We selected these two social psychological factors because they are potentially impacted by institutional or classroom features (Matthews ; Ergene, 2003; Massey and Fischer, 2005; Osborne and Walker, 2006; Kellow and Jones, 2008; Ballen ) and can have consequential impacts on learning and academic performance (Kellow and Jones, 2008; Appel and Kronberger, 2012; Salehi ).

We selected test anxiety to measure across contexts because high levels of test anxiety reduce student performance outcomes, and we predicted this may vary across multiple contexts based on factors such as class size, course competitiveness, or other traits that may relate to institution type (von der Embse ). Test anxiety is characterized by feelings of tension, worried thoughts, and negative physiological reactions in an academic evaluative setting (American Psychiatric Association, 2013). Work across STEM fields shows that test anxiety negatively impacts assessment performance outcomes (Chapell ; DordiNejad ; Ali and Mohsin, 2013; Shapiro, 2014; Harris ). Results from previous research showed that both in introductory biology (Ballen ; Cotner ) and across STEM disciplines in higher education (Salehi ), for women only, test anxiety negatively influenced exam performance. Work conducted at a separate institution could not replicate these results and concluded that women underperform on exams for reasons other than gender-based differences in test anxiety (Harris ). It is possible that students with higher content-related anxiety perceive test difficulty differently, which may in turn impact test anxiety (Hong and Karstensson, 2002). The impacts of test anxiety on performance of students with underrepresented identities in biology has not been studied as extensively, to our knowledge. One study conducted mediation analysis to examine the relationship between a number of social psychological factors, including test anxiety, as well as incoming preparation, minority and first-generation status, and academic performance outcomes within a college of biological sciences (Salehi ). Results showed that URM and first-generation status was highly related to students’ incoming preparation. Lower measures of incoming preparation were associated with higher anxiety scores, and higher anxiety was associated with lower exam scores.

We also selected ESC to measure across contexts. ESC is a measure of stereotype threat, which we measured as the extent to which one feels self-conscious of the stigma associated with one’s race/ethnicity (Picho and Brown, 2011). Stereotype threat can invoke a disruptive state, undercutting performance and aspirations within a domain (Spencer ). Ever since Steele and Aronson’s (1995) work on this phenomenon, research has documented its presence among negatively stereotyped individuals in hundreds of studies. Stereotype threat has been shown to reduce an individual’s ability to learn (Rydell ; Taylor and Walton, 2011; Boucher ). It can be reduced or invoked through relatively small changes in instructor behavior or the classroom environment, such as stating that tests are difficult and performance is linked to overall ability (Steele and Aronson, 1995). Little work on stereotype threat has been conducted in the context of postsecondary biology. Taasoobshirazi administered study questions with different stereotype threat conditions in the instructions and examined biology performance based on gender, as well as self-efficacy, motivation, and domain identification. While they did not identify differences in gender outcomes by treatment, they observed women reported a greater domain identification with biology. To our knowledge, no previous work has focused on stereotype threat based on race/ethnicity in the context of postsecondary biology. Additionally, quantifying this social psychological factor as it varies across classrooms or institutions has not been explored, despite accumulating evidence that institutional context shapes student experiences (Chang ; Hurtado ; Wang, 2013; Winkle-Wagner and McCoy, 2018). For example, we may expect ESC to be higher in predominantly White spaces such as the selective institutions in our sample.

Using data generated collaboratively across multiple institutions, we tested the following research questions:

To what extent do we observe gaps in academic performance based on demographic status such as binary gender and URM status in introductory biology across different institutional contexts?

To what extent do we observe differences in social psychological factors, that is, test anxiety and ESC, based on demographic status in introductory biology across different institutional contexts?

How do these social psychological factors mediate potential differences in performance outcomes based on demographic status across different institution contexts?

Traditional views of the performance gap attribute student success primarily to personal attributes, such as deficiencies in ability or motivation. Focusing on performance gaps in this way deflects attention away from the key responsibility educational institutions play in student success and applies underlying deficit perspectives to marginalized students in STEM (Smit, 2012; Zhao, 2016). Here we stress that the opportunity gap in primary and secondary education—for example, access to high-quality curricula, instruction, and technology—contributes to gaps in academic preparation before students even enter the undergraduate biology classroom. These gaps in academic preparation are then in turn reflected in student grades in introductory courses that fail to provide students with equal opportunities to perform (Salehi et al., 2019, 2020). Understanding the extent of performance disparities in courses and their prevalence across institutional contexts is an important first step to fulfill the responsibility of higher education to educate all students. Our focus on academic disparities aims to change structural barriers within our classrooms and institutions such that student performance patterns (e.g., exam grades, total grades) are not predicted primarily by characteristics such as race, class, ethnicity, gender, proficiency in a dominant language, or other marginalized student identities.

METHODS

Participants

We solicited participation through an existing professional network from instructors who teach biology at a range of institutions (Research Coordination Network, National Science Foundation RCN–UBE Incubator: Equity and Diversity in Undergraduate STEM; grant no. 1729935 awarded to S.C. and C.J.B.; RCN–UBE grant no. DBI-1919462 awarded to S.C., C.J.B., S.F., Harshman, and C.H.). Equity and Diversity in Undergraduate STEM is a network of educators and discipline-based education researchers who work together on research and teaching projects in the context of biology curricula and is supported by the National Science Foundation. The objectives of the network are to: “(1) reveal differences, if they exist, in the cultural climate for women and minorities in STEM disciplines (initially focusing on biology) as a function of geography, institution type, and cultural profile of the participating departments; (2) increase the number of faculty in the United States that are familiar with barriers to inclusion in STEM, and can apply evidence-based techniques for countering known barriers; (3) develop a community of faculty that can serve as leaders-at their home institutions and nationally-in inclusive teaching and assessment; and (4) identify cultural factors associated with a shift toward evidence-based teaching, especially pertaining to inclusive teaching” (Thompson .

From the RCN network, we collected data from 33 biology courses at seven institutions between 2016 and 2018. Participating institutions were a convenience sample chosen from a range of institutional types (public and private, large and small) and settings (college towns to large metropolitan areas). We classified these institutions based on undergraduate profile according to the Carnegie Classification of Institutions of Higher Education. This allowed us to group institutions by admissions selectivity criteria and by whether they were 2-year or 4-year institutions. The 2-year institutions in our sample were community colleges, and so use we this term throughout this research. Within 4-year institutions, we classified institutions as “selective” if they fell into the Carnegie classification of “more selective,” indicating that the 25th percentile of first-time first-year students received greater than 21 on their American College Testing (ACT) entrance exam score. We classified institutions as “inclusive” if they fell into the Carnegie category of “inclusive” or “selective,” with 25th percentile ACT scores less than 21. Additionally, we worked with representatives at each institution to generate Table 1, which summarizes some contextual characteristics of the participating institutions.

TABLE 1.

Summary of participating institutions: each institution’s average entrance exam scores (when applicable); whether each is a minority-serving institution (year of designation); the approximate number of undergraduates enrolled; how evidence-based teaching is promoted at the institution; pedagogy of participating classes; typical class size for lower-division (LD) classes at the institution; typical class size for upper-division (UD) classes at the institution; and typical teaching load of instructors who participated in the studya

Institution	Avg. SAT or ACT	MSI? (year)	Approx. no. of undergraduates	How is evidence-based teaching (e.g., active learning) promoted at your institution, if at all?	Pedagogyb	LD class size (i.e., first and second year)	UD class size (i.e., third and fourth year)	Teaching loadc
Inclusive 4-year.1	SAT 950–1150; ACT composite score of 18–23	Yes (2009)	12,000	There are various peer-led initiatives funded internally and by grants and there is strong support in the teaching academy.	Active learning	150	75	3 to 5 classes per year
Inclusive 4-year.2	ACT composite score of 20–22	Yes (1881)	2000	One of the president’s main focus areas is creating a student-centered campus environment. The university brings in speakers during all-university conference (twice a year) to discuss teaching strategies. The last few speakers spoke about active learning and how to engage students. We have a seminar once a semester called Strategies for Effective Engagement (SEE). The speakers have been many high school teachers with 20+ year experience on engaging students. However, the teaching method is up to the faculty member.	Interactive lecture	60	40	More than 5 classes per year
Inclusive 4-year.3	SAT 1000–1190; ACT composite score of 18–24	Yes (2016)	17,000	It is promoted in many different majors and student organizations/activities university-wide. Examples include: building competitions in engineering, the great debate, the hands-on lab for future teachers, sustainability conference, first-year experience courses, ecology labs at Big Chico Creek Ecological Reserve, nursing clinical labs, Community Action Volunteers in Education (CAVE), student-run university newspaper, and many more.	Active learning	120	40	More than 5 classes per year
Selective 4-year.1	SAT EBRW 680–750; SAT Math 720–790; ACT Composite score of 32–35	No	15,000	There is an active-learning initiative, started in 2013, that is funded by donors and gives out grants to departments that apply each semester for funding. About 25% of the departments at [Institution] now have an association with the active-learning initiative.	Active learning	300	40	1 to 3 classes per year
Selective 4-year.2	SAT 1080–1260; ACT composite score of 22–28	No	12,000	Few opportunities or incentives to learn about and use evidence-based teaching exist.	Traditional lecture	200	50	3 to 5 classes per year
Selective 4-year.3	ACT composite score of 28	Yes (2016)	31,000	There are several institution-level initiatives (e.g., through the Center for Educational Innovation) as well as various collegiate and department-level programs to support professional development around making courses more “active.” While there are pockets of resistance, these are not at the administrative level.	Interactive lecture	115	20	3 to 5 classes per year
Community.college.1	N/A SAT/ACT scores are not required or collected.	No	60,000	Few opportunities or incentives to learn about and use evidence-based teaching exist.	Traditional lecture	50	15	3 to 5 classes per year
Community.college.2	N/A SAT/ACT scores are not required or collected.	No	5,600	The Center for Teaching and Learning provides regular professional development opportunities.	Active learning	25	10	More than 5 classes per year

aOriginal reports edited for clarity and succinctness.

bSelect: 1) traditional lecture; 2) interactive lecture: lectures with some opportunities for student interaction, such as instructors posing questions; or 3) active learning: lectures with iClickers, group work, formative assessments, and pre-readings.

cSelect 0–1 class per year, >1 to 3 classes per year, >3 to 5 classes per year, or >5 classes per year.

We worked with individual instructors who teach biology classes across these institutions to collect data. Our sample represented 12 courses at two community colleges (n = 454 students, average class size = 38 ± 6.68), six courses at three inclusive institutions that were also minority-serving institutions (n = 1045 students, average class size = 199 ± 49.85), and 15 courses at three selective institutions (n = 3594 students, average class size = 239 ± 40.26). During the 2-year study period, we obtained grade data disaggregated into exam scores, non-exam scores, and final grades and institutional data on gender and race/ethnicity from all courses in the study. Also, during the last few weeks of the semester for a subset of students, we administered a survey to measure test anxiety and ethnicity stigma consciousness (Table 2). Note we removed one institution, representing one class, from our analyses of demographic gaps due to the fact that 100% of student were from underrepresented racial minority groups. All aspects of research were reviewed and approved by each institution’s institutional review board.

TABLE 2.

Number of student responses to survey constructs, exam scores, and percentage of URM students for each institution typea

Institution type	Test anxiety construct (I/C)	Ethnicity stigma consciousness construct (I/C)	Exam scores (I/C)	% URM students ± SD (range)
2-year (community college)	342 (2, 12)	344 (2, 12)	454 (2, 12)	26% ± 13% (11–60%)
Inclusive 4-year	747 (2, 5)	512 (2, 4)	995 (2, 5)	50% ± 14% (36–65%)
Selective 4-year	845 (3, 6)	1937 (3,12)	3594 (3, 15)	10% ± 5% (2–25%)

aNumbers in parentheses represent number of institutions (I) and courses (C) for each measure. Percentage of URM students represents the average percentage of URM students in each class for each institution.

Survey Measures

To measure test anxiety, we used a four-item test anxiety construct from the Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich ). The MSLQ test anxiety construct asked students about their experiences during testing, such as whether they felt distracted by their anxiety during exams. To measure ESC, we used the five-item ESC construct from the Social Identities and Attitudes Scale (SIAS; Picho and Brown, 2011). ESC measured the extent to which one is conscious about one’s ethnic identity and includes items such as “My ethnicity influences how teachers interact with me.” For both constructs, students rated their agreement with each item on a seven-point Likert scale ranging from “strongly disagree” to “strongly agree.” We have included the full list of survey items in Supplemental Table S1.

Analysis

We performed all statistical analyses in R (R Core Team, 2019). We reported exam scores as the average of all exam scores earned by each student, and we normalized this average for each course to control for variation in grading and exam structure across courses. Therefore, each student’s exam score can be interpreted as how many standard deviations that score is from the mean of a given course.

Validating Constructs: Confirmatory Factor Analysis.

We used confirmatory factor analysis (CFA) in the R package lavaan to verify our survey item structure for test anxiety and ethnicity stigma consciousness. We used CFA because these survey items were previously developed and established as theoretical models and validated in previous research (Picho and Brown, 2011; Pintrich ; Rosseel, 2012). Thus, we were testing to confirm whether our data set supports the preexisting survey with established structure (Knekta et al., 2019). For advantages and more details of conducting CFA via lavaan in biology education, see Ballen and Salehi (2021). To evaluate how well the CFA model captured the variation in the survey data, we used structural equation modeling (SEM) fit indices: confirmatory fit index, root mean square error of approximation, SRMR. All the fit indices fell within the acceptable range, suggesting the survey items loaded on the defined factors and the CFA model properly captures the variation in the data.

Quantifying Performance Gaps: Mixed-Effect Analysis of Performance Gaps.

To test the extent to which gaps in academic performance are related to demographic status such as gender and URM status (together referred to as “demographic status”) in introductory biology, we ran mixed-effects multivariable regression analysis using the nlme package in R. We controlled for the random effect of courses nested within each institution.

Mediation Analysis.

We also tested: 1) whether performance outcomes based on gender and URM status were mediated by test anxiety or ESC; and 2) whether mediation varied across institution type (community college, inclusive institutions, or selective institutions). To this end, we used a SEM approach using lavaan package in R (Rosseel, 2012) on a subset of students for whom we had responses to all survey items and their exam scores (n = 337 students in 12 courses at two community colleges, n = 581 students in five courses at two inclusive institutions, and n = 756 students in five courses at three selective institutions). In lavaan, we used group analysis, with institution type as the grouping factor. Therefore, the proposed mediation model would be fit to the data of each institution type separately, and whether the model was a good fit for each institution type would be tested. We tested for mediation by using the Akaike information criterion (AIC) to compare two structural models: one including only indirect effects of demographic status mediated by social psychological factors and one that also included the direct effect of demographic status on exam scores. Within the SEM analysis, we included the latent variable structure supported by our CFA. For ease of interpretation, we normalized these construct responses for the entire sample across institution types, such that reported values reflect how many standard deviations a value is from the sample’s mean score.

RESULTS

Validating Constructs: CFA

We removed one of the survey items from the SIAS (“My ethnicity impacts how I interact with people of other ethnicities”), which had a poor loading in our sample (standardized factor loading <0.70). We evaluated model fit using several fit indices, and while RMSEA, SRMR, and CFI were consistent with cutoff values (Hu and Bentler, 1999; Knekta et al., 2019), our chi-square test of model fit was highly significant (CFI = 0.99, RMSEA = 0.03, SRMR = 0.016, χ2 = 50.91, p < 0.001). However, this is likely an effect of our large sample size, because chi-square tests are highly sensitive to sample size. In large samples, even a very small difference is highly likely to be significant (Hu and Bentler, 1999). Therefore, with a large sample size, if the model estimated covariance matrix is slightly different from the observed covariance matrix, the chi-square statistics might be still significant. Given that all three other indices fell within the acceptable range, and our constructs had high reliability (α  = 0.78; test anxiety(α) = 0.89; ESC(α) = 0.89) we accepted our CFA in lavaan as a good fit.

Quantifying Performance Gaps: Mixed-Effect Analysis of Performance Gaps

For gender performance analysis, we found that the nested structure of the random effect explained 4% of the variance in the data. This suggests that the random effects due to courses nested within institutions were small. This is partly due to the fact that the course grade is normalized within each course. We found no significant effect of gender on average exam scores (mixed-effects model: βCC = 0.09 ± 0.11, p = 0.96; βinclusive = 0.07 ± 0.08, p = 0.93; βselective = 0.08 ± 0.03, p = 0.13). While there was also variation in the extent of the gap between men and women, a greater percentage of courses showed no difference in exam scores by gender. For URM performance analysis, we similarly found that the nested structure of the random effects explained 4% of the variance in the data, which suggests that random effect is relatively small. We found that URM students underperformed relative to non-URM students on exams at both selective and inclusive 4-year institutions, but not at community colleges (mixed-effects model: βCC = −0.23 ± 0.12, p = 0.30; βinclusive = −0.27 ± 0.07, p = 0.003; and βSelective = −0.44 ± 0.05, p < 0.001). As we only found significant URM performance gaps in our sample and no gender gaps, we proceeded to only examine how social psychological factors mediate URM performance gaps.

Mediation Analysis

Our structural equation model revealed variation in mediation effects across institution types (Figure 1). We found support for partial mediation effects, as the partial mediation model was a better fit for the data (p = 0.002) and as the ∆AIC between the partial mediation and full mediation model was 9, which is larger than our cutoff ∆AIC of 2. In this model, test anxiety was universally negatively associated with exam scores (βCC = −0.27 ± 0.02, p < 0.001; βinclusive = −0.13 ± 0.03, p = 0.007; βselective = −0.33 ± 0.02, p < 0.001). However, only at inclusive institutions, URM students had higher test anxiety. For community colleges or selective institutions, there was no difference in test anxiety across URM status (βCC = 0.16 ± 0.24, p = 0.21; βinclusive = 0.30 ± 0.16, p = 0.001; βselective = 0.18 ± 0.23, p = 0.21). These relations imply that test anxiety mediated the effect of URM status on exam scores only at the inclusive institutions, but not at community colleges or selective institutions. The size of the mediation path can be calculated by multiplying the coefficients included in each path (βCC = −0.04 ± 0.04, p = 0.22; βinclusive = −0.04 ± 0.02, p = 0.04; βselective = −0.06 ± 0.05, p = 0.22). In other words, URM students at inclusive institutions had higher test anxiety than their non-URM peers, and this higher test anxiety led to lower exam performance. At community colleges and selective institutions, on the other hand, there was no difference across URM status in test anxiety.

FIGURE 1.

Structural equation model for mediation effects of test anxiety and ethnicity stigma consciousness across institution types. All continuous variables have been rescaled to have a mean of 0 and an SD of 1 and can be interpreted as positive or negative units of SD reflecting the relationship between two variables. An “e” in a circle indicates an error term in our estimations of a model variable. Orange lines indicate negative correlations; green lines indicate positive correlations. For example, the figure implies that, for inclusive institutions, URM students on average have 0.3 SD higher test anxiety compared with non-URM students, and a 1 SD increase in text anxiety leads to a 0.13 SD decrease in exam score. The size of each mediation path is calculated by multiplying the size of each coefficient included in that path. Hence, the size of mediation effect for test anxiety in inclusive institutions is 0.3 * 0.13 = 0.04. The significance levels are indicated as: *p < 0.05; **p < 0.001; ***p < 0.0001. For more information about mediation analyses using structural equation models, see research methods essay by Ballen and Salehi (2021).

ESC was not a significant mediator for the effect of URM status on exam scores for all the three institution types (βCC = 0.01 ± 0.01, p = 0.41; βinclusive = −0.01 ± 0.01, p = 0.26; βselective = −0.008 ± 0.02, p = 0.72). However, the reason for the lack of mediation effect varied by institution type. We did not observe an effect of URM status on ESC at community colleges (β = 0.12 ± 0.15, p = 0.38), but ESC positively affected exam scores (β = 0.1 ± 0.04, p = 0.02). At inclusive institutions, there was no significant correlation of URM status with ESC (β = 0.15 ± 0.12, p = 0.12), and no correlation of ESC with exam scores (β = −0.07 ± 0.04, p = 0.11). However, we did observe a correlation of URM status with ESC at selective institutions (β = 0.57 ± 0.13, p < 0.001) similar to the previous work (Salehi ), though there was still no effect of ESC on exam scores (β = −0.01 ± 0.04, p = 0.72). The observation that, for selective universities, none of social psychological factors are mediators for URM performance gaps aligns with previous findings (Salehi ) that the incoming preparation was the major mediator for demographic performance gaps in an R1 institution, and after controlling for that effect, the social psychological factors did not have a mediating effect for demographic performance gaps.

DISCUSSION

We set out to test the importance of institutional context on student experiences in biology courses using data generated collaboratively through a research coordination network (NSF DBI-1919462) from seven institutions across three institution types: community colleges, inclusive institutions, and selective institutions.

We show some patterns emerge across all institutions, regardless of institution type, such as the absence of performance gaps based on gender. However, we found that gaps in performance, test anxiety, and ESC varied across different institutions. Specifically, while we did not observe performance gaps based on race/ethnicity at the two community colleges, we did observe them at the inclusive and selective institutions in our sample. Second, we found a significant indirect effect of URM status mediated by test anxiety on exam scores at the inclusive institutions, but not at the community colleges or selective institutions. Third, we found overall that URM students at the selective institutions were more likely to report higher ESC values (but this did not in turn impact performance), while at community colleges, ESC positively affected exam scores for all students. Based on these results, we put forth a number of general and specific research recommendations.

First, we stress the importance of multiple perspectives in BER through programs that encourage research and collaborations across institution types. Other researchers have echoed the need for data-driven approaches to STEM education that span institutional contexts beyond selective institutions to address equity in STEM (Estrada ; Schinske ; Thompson ). For example, despite the important role of community colleges in educating students, only 3% of recent BER articles included either community college authors or a community college biology study context (Schinske ). And despite the unique role of community colleges in educating biology students from diverse backgrounds, Schinske showed only 7% of community college BER articles explicitly studied equity and diversity. Our results support the notion that institutional context impacts barriers to student performance and interventions to support student success will likely vary in their effectiveness based on the student population. Beyond the coarse categories of institution type, we found individual institutions and classrooms also differ from one another. Specifically, we found that URM students underperformed relative to non-URM students on exams at both the inclusive and selective institutions, but not at the community colleges; however, individual courses showed variation within institution types, with the majority of courses showing no significant gap between groups. This supports the idea that factors other than precollege background, academic preparation, and admission criteria explain a large share of performance outcomes (Massey ; Espenshade and Radford 2009; Salehi et al. 2019). Future research on factors that positively influence performance will clarify similarities across these equitable classrooms.

Second, social psychological interventions have been shown to vary in effectiveness for different students, and our results highlight the need for research into targeted interventions to support inclusion in STEM (Schwartz ). For example, we found that test anxiety was negatively associated with exam performance across institutions and mediated performance gaps for URMs at inclusive institutions. Future research would benefit from a focus on the impacts of test anxiety on performance of students with underrepresented identities in biology specifically. One study showed that lower measures of incoming preparation were associated with higher anxiety scores, which were associated with lower exam scores (Salehi ). While it is unclear why we only observed this mediating effect of anxiety at inclusive institutions, our results suggest that strategies instructors can use to mitigate test anxiety are likely to benefit all students, and especially URM students at some institutions. These include lowering the stakes of exams, expressive writing, or two-stage exams (Ergene, 2003; Ballen ; Rempel ). A meta- analysis of 56 interventions that target student behavior (N = 2482) revealed that approaches such as skill-focused strategies (e.g., test-taking skills training), behavioral strategies (e.g., relaxation training), or cognitive strategies (e.g., techniques to change negative thought patterns) are effective at reducing test anxiety. However, less work has brought these strategies into biology or STEM classrooms to test their impacts on performance outcomes. Additionally, the impacts of test anxiety on performance of students with underrepresented identities in biology has not been studied as extensively, to our knowledge.

Conversely, given our varying findings about the effects of ESC, researchers and faculty should be intentional in their implementation of stereotype threat interventions in their institutions. While past work has demonstrated that, for students who have high ESC, the threat or fear of conforming to a stereotype negatively impacts their exam performance (McFarland ; Massey and Fischer, 2005), others have suggested that this phenomenon may not be as universal as previously thought (Cromley ).

Our results showed that overall, self-reported ESC did not negatively impact academic performance for URM students, and its mediating effects varied across institutions. This suggests stereotype threat did not impact performance of URM students across contexts. In fact, at community colleges, while URM status did not predict student ESC value, the ESC value positively predicted exam score (though the effect size was small). In other words, for all students, regardless of their minority status, self-reported ESC was associated with higher exam scores. We may observe that one’s race/ethnicity does not undermine performance outcomes due to reduced negative stereotypes about race/ethnicity in more-inclusive contexts, such as community colleges. Future research would profit from an explicit focus on community colleges to understand this outcome and what other institutions can learn from equitable teaching practices that are effective at these institutions.

Fostering an inclusive environment serves to reduce the potential effects of stereotype threat (Steele, 2012) and is a leading explanation for positive performance outcomes among minoritized groups (Martin ). Murphy and Zirkel (2015) found that students in higher education anticipated a higher sense of belonging in majors where they perceived their group to be represented and that self-reported feelings of belonging in the first weeks of college predicted second-semester grades among students of color, but not White college students. We found that URM status was correlated with higher ESC at the selective institutions in our sample, supporting previous work that stresses the importance of promoting sense of belonging and social fit on campuses or institutions (Massey ; Martin ). The selective institutions in our sample include the lowest percentage of URM students (Table 2). These low numbers may lead students to feel marginalized and isolated (Hurtado and Ruiz, 2012; Patton and Croom, 2017), experience hostile learning environments (Kelly ), and possess few support networks (Palmer ). In these predominantly White spaces, we must center interventions that call upon instructors and institutions, rather than the students, to act. Previous research describes many ways to promote inclusion in the classroom, such as employing equitable teaching strategies (Tanner, 2013) and active learning (Eddy and Hogan 2014; Ballen ; Casper ; Theobald ), learning names (Cooper ), using humor (Cooper ), including positive role models for all students as examples in course content (Wood ; Yonas ), conveying that diversity is valued (Purdie-Vaughns ), and/or encouraging students to reflect on core personal values (Cohen ; Martens ; Miyake ).

Future research would profit from an understanding of how stereotypes specific to different underrepresented identities emerge in undergraduate settings, how they impact performance outcomes, and how students navigate these situations. Neal-Jackson (2020) conducted a qualitative analysis of Black women at a prestigious predominantly White institution and found that they encountered gendered racial stereotyping that negatively impacted their relationship to the academic community and their prospects for success. This, along with other work (Cho ; Morton and Parsons, 2018), also highlights the complexity associated with the multidimensionality of identity and the intersecting nature of those dimensions. We encourage researchers to continue exploring these ideas to determine what strategies increase academic performance and reduce threat for all students. Future work at community colleges and inclusive institutions will help us better understand how to alleviate ESC and inform interventions at selective institutions, where students from marginalized backgrounds are less likely to perceive social fit and more likely to experience stereotype threat.

Limitations

The work presented here is an initial step toward broadening the scope of discipline-based education research across different institutional contexts, though we faced a number of limitations that should be addressed in future work. We acknowledge that our approach here of grouping institutions by broad category of community college, inclusive, and selective suggests students share similar experiences with other students within the same broad institution type rather than across other factors, and this assumption is a limitation of our research. We do not expect that, for example, test anxiety fully mediates performance gaps across all inclusive 4 year-institutions in the United States. Our current sample includes 12 courses in two community colleges, six courses in three inclusive institutions, and 15 courses in three selective institutions. We administered surveys at a single time point, which might not be indicative of levels across the entire semester. Also, other social psychological measures (e.g., sense of belonging; Lewis ; Rainey ; Binning ) and their contribution to student performance should be explored.

Additionally, we did not have access to proxies for incoming preparation (e.g., ACT/Scholastic Aptitude Test [SAT]) from most institutions and were therefore unable to control for this factor in our analyses. Previous research demonstrates that, from the outset, capable but less academically prepared students are not provided with an equal opportunity to excel due to incoming preparation gaps and assessment norms in introductory science classes, resulting in the appearance of “performance gaps” in introductory classes, as we show here (Salehi ). However, when incoming preparation is accounted for, these apparent gaps disappear. While we were not able to control for this, we note that student transcripts, applications to graduate and professional schools, and the internalization of poor performance do not take incoming preparation into account either (Harris ). Future research may test how incoming preparation impacts demographic performance gaps across institution types and control for incoming preparation in introductory classes while examining the effect of social psychological factors. While the lack of these data is a limitation, we do not believe this impacts our main conclusion, which points to variation in outcomes across institutional contexts.

Furthermore, one of our analyses included social psychological measures that we obtained from survey data, which represented a subset of all students. This subset may not be representative of the overall sample, as suggested by the URM performance gap in the whole sample as opposed to the gap in the subsample (no URM performance gaps for community colleges in the whole sample, as opposed to the 0.2 SD URM performance gap for community colleges in the subsample; Figure 1). Thus, our analyses focusing on the impacts of these factors on performance may not be representative of the larger pool of students across these institutional contexts. Therefore, the observed differences here in student experiences and academic performance outcomes warrant further study to examine to what extent these differences are results of differences in institutional contexts.

CONCLUSIONS

We call for future research to 1) not exclusively focus on selective institutions and instead include data across different types of institutions; 2) systematically examine characteristics of institutional or classroom contexts that help or hinder equity in undergraduate education; and 3) design targeted psychological interventions and/or effective teaching strategies that fit the characteristics of institutional or classroom contexts. We found some similar trends across institution types, as well as substantial differences with respect to the size of demographic performance gaps and the relationship between social psychological factors and performance. Ongoing research programs and existing evaluative tools designed to assess learning gains and social psychological factors lay a solid foundational groundwork for testing hypotheses.

Click here for additional data file.