A social-belonging intervention improves STEM outcomes for students who speak English as a second language.

Students who speak English as a second language (ESL) are underserved and underrepresented in postsecondary science, technology, engineering, and math (STEM) fields. To date, most existing research with ESL students in higher education is qualitative. Drawing from this important body of work, we investigate the impact of a social-belonging intervention on anticipated changes in belonging, STEM GPA, and proportion of STEM credits obtained in students' first semester and first year of college. Using data from more than 12,000 STEM-interested students at 19 universities, results revealed that the intervention increased ESL students' anticipated sense of belonging and three of the four academic outcomes. Moreover, anticipated changes in belonging mediated the intervention's effects on these academic outcomes. Robustness checks revealed that ESL effects persisted even when controlling for other identities correlated with ESL status. Overall, results suggest that anticipated belonging is an understudied barrier to creating a multilingual and diverse STEM workforce.

INTRODUCTION

In the past 20 years, the number of students in the United States who speak English as a second language (ESL) has increased markedly (, ). By 2030, approximately 40% of K-12 students are projected to be ESL learners (). Despite this rapid increase, research indicates that ESL students are underrepresented in higher education, with roughly 40% of ESL students dropping out of high school and only 18% of ESL high school graduates continuing on to a 4-year college (, ). Of those who attend college, only 20 to 25% of ESL students graduate (–). Thus, it is clear that ESL students are a growing but underserved population whose academic needs are not being met.

ESL students are particularly underrepresented in the fields of science, technology, engineering, and math (STEM) (). The underrepresentation of ESL students is problematic for multiple reasons. First, the world economy is becoming more dependent on technology and advances in STEM. A diverse and multilingual STEM workforce is needed to meet the needs of an increasingly globalized economy in which international collaboration is increasingly prevalent. Second, starting salaries of STEM majors are notably higher than salaries of non-STEM majors, and many companies seek candidates with STEM bachelor’s degrees and/or some STEM education (). Therefore, increasing the number of ESL graduates in STEM will economically benefit these students and their communities while increasing the nation’s competitiveness in the global economy.

One challenge with regard to building a STEM workforce is getting people interested to pursue STEM fields of study in the first place; another challenge is finding ways to get students who are already interested in STEM to persist within STEM coursework. This project focuses on exploring an intervention aimed to do the latter. How can we encourage ESL students who are already interested in STEM fields to persist through the first year of STEM coursework—when many students choose not to pursue STEM majors ()? To understand how to promote ESL students’ participation, persistence, and performance in STEM, it is important to consider the many barriers that ESL students face, including linguistic, instructional, and psychological challenges. Although a large literature has addressed the underrepresentation of certain minority groups in STEM majors (i.e., women, Black, and Latinx students) (, ), substantially less research has examined the underrepresentation of ESL students in STEM. To our knowledge, almost all previous quantitative research on ESL students in STEM focuses on primary and secondary educational contexts and primarily examines linguistic and instruction-based barriers to success. In addition, the research that does exist on ESL students in higher education is mostly qualitative, which is highly informative, but has its limitations.

The present work fills several gaps in this literature. From a theoretical perspective, we are among the first to quantitatively examine and mitigate an important psychological barrier that ESL students face when they begin college: lower anticipated feelings of belonging in college. This study represents the first examination of how a brief online social-belonging intervention can increase undergraduate ESL students’ performance and persistence in STEM by increasing their anticipated feelings of belonging in college—setting the stage, psychologically, for STEM-interested ESL students to reach their academic potential and persist in STEM fields of study. Last, our work is the most comprehensive to date, focusing on the STEM performance and persistence of more than 2600 STEM-interested ESL students enrolled at 19 different 4-year colleges and universities.

Who are ESL students?

In the current work, we use the term ESL to refer to students who do not speak English as a first language. ESL is both one of the earliest terms used to refer to these students in the scholarly literature and the most commonly used term in the practice of higher education by colleges and universities (). However, there are many other terms that have been used to refer to these students, including English language learners (ELLs or ELs), limited English proficient students (LEPs), linguistic minorities (LMs), and bilingual/multilingual students (). Past research with ESL students at all levels of education has inconsistently used these terms and operationalized English proficiency and what it means to be an English language learner in many different ways. Different operationalizations of this group can certainly affect the relationship between ESL status and academic outcomes (). In the present work, we assessed students’ group membership with a dichotomous measure wherein college students themselves self-reported their ESL status. This operationalization allows us to capture—from the students’ perspectives themselves—the wide range of students whose native language is not English regardless of their English language proficiency or whether they are currently in the process of learning English. We also use the term ESL to avoid other terms that scholars have noted can imply that ESL students may have less potential than non-ESL students (e.g., LEPs) ().

Overall, the varying terms used to describe ESL status highlight the heterogeneity of ESL students as a group, who can also vary in their English proficiency, as well as other background characteristics such as race/ethnicity, socioeconomic status (SES), citizenship status, parental education, and high school academic performance (). For example, although a large number of ESL students identify as Black, Middle Eastern, and other racial/ethnic identities, most of the ESL students in the United States are Latinx/Hispanic and Asian (). Similarly, some ESL students come from families who are high SES, but a larger proportion of ESL college students are considered to be low SES and come from disadvantaged high schools (, ). Last, although some ESL students are citizens of the country in which their university is located (the United States, in this case), many are not (, ). Given this heterogeneity, one goal of the present work was to examine the effects of ESL identity above and beyond the demographic characteristics that intersect with ESL status through a series of robustness checks.

Barriers in STEM for students who speak ESL

A fair amount of research has focused on English language proficiency and instruction-related barriers to ESL students’ academic success. For example, the success of ESL (and non-ESL) students in STEM is contingent upon their ability to understand STEM vocabulary and apply it to the specific content they are learning. However, for some ESL students, this task may be particularly difficult, because many words used in STEM have different meanings in non-STEM contexts (e.g., volume, tangent, formula, and plane). Therefore, even if ESL students are able to speak and comprehend English effectively in their everyday lives, researchers have found that it may be more difficult for ESL students to learn STEM content relative to their non-ESL peers. That is not to say that ESL students have less ability, are unable to learn the material, or will consistently underperform; instead, the amount of motivation and effort ESL students must expend to master difficult STEM material may frequently be greater than that required of non-ESL students. Relatedly, ESL students often contend with instruction-based barriers. For example, many teachers—and especially college faculty—receive little to no training on how to meet the linguistic and instructional needs of ESL students, which can hinder the success of ESL students (). Several interventions have been developed to address these linguistic and instruction-based barriers, including curricula that use hands-on classroom activities to encourage STEM engagement (), information frameworks that assist ESL students in learning STEM vocabulary (, ), and teacher training programs that provide instructional strategies (such as using combinations of oral and visual descriptions when teaching STEM) to improve the success of STEM students (). Together, this research suggests that when ESL students are given tools and adaptable educational contexts, they are able to overcome linguistic and instruction-based barriers and flourish academically.

Although this literature provides important ways to support the performance and persistence of ESL students in STEM contexts, its focus on language proficiency and instructional strategies leaves the psychological barriers that ESL students contend with in STEM unaddressed—specifically, students’ anticipated feelings of belonging. Belonging concerns are well documented among other underrepresented groups in STEM, including women and racial minorities (, ). This research indicates that students’ concerns about belonging in college can interfere with academic performance and persistence in STEM (, ). Mirroring the academic experiences of other underrepresented students in STEM, quantitative research on ESL students in middle school and elementary school demonstrates that ESL students report high levels of belonging uncertainty and a lower sense of school belonging, which then predicts their academic performance across fields of study (, ).

In the postsecondary context, qualitative research on the lived experiences of ESL students indicates that anticipated feelings of belonging are relevant for ESL students’ success in college as well (, , ). For example, ESL students’ feelings of alienation, marginalization, and sense of connection with the university are affected by perceptions that their culture and language are not valued (–). In dozens of interviews with ESL students, central themes that emerge include difficulty acclimating to the university environment because of loneliness, a lack of social connections, and feelings of being different than native English–speaking students (, , ). In addition, many ESL students are required to pass language proficiency tests and/or enroll in special ESL courses during their first few years of college (, ). These policies can metaphorically and literally mark and physically separate ESL students from non-ESL students during the pivotal transition to college where social connections and feelings of belonging are critical to the social and academic adjustment of ESL students (). These institutional practices and policies also reinforce the message that ESL students do not belong in a traditional college setting, which recent research has linked to classroom participation (). Together, this research reveals the many ways that the degree to which ESL students belong in college and in STEM is being assessed at every turn. These studies also provide an important springboard for the current work and suggest that bolstering ESL students’ anticipated feelings of belonging in college may positively affect their STEM performance and create a positive recursive cycle in which feelings of belonging and academic performance are shored up and promoted.

Social-belonging intervention

Theoretical support for this positive recursive cycle comes from research on social-belonging interventions (–). These interventions have been effective at bolstering underrepresented students’ anticipated sense of belonging before they have matriculated into college, as well as their academic persistence and performance during college (–). The social-belonging intervention strives to promote recursive change in people by shaping their construal of the local environment. This direct-to-student psychological intervention can be powerful, because (i) many situations in college are ambiguous and can be perceived in multiple ways and (ii) the need to feel as though one belongs is a fundamental human motivation (). To this aim, the prematriculation social-belonging intervention consists of a reading and writing exercise that portrays social and academic setbacks and challenges in the transition to college as normative and temporary for students from all backgrounds and not indicative of a lack of belonging. Specifically, students read short stories attributed to juniors and seniors describing the challenges they faced in their transition to college (e.g., low grades and difficulty making friends) that made them question the degree to which they belonged there. However, these stories also revealed that either students’ belonging concerns passed with time or they described the strategies that students engaged in (e.g., joining extracurricular activities and attending office hours) that ultimately helped them feel that they belonged in college.

Overall, the social-belonging intervention aims to shift meaning-making around changes in belonging. Consistent with this aim, research demonstrates that the social-belonging intervention treatment increased socially disadvantaged students’ belonging (–, ). Central to the current work, Yeager and colleagues found that all students who received the social-belonging intervention reported significantly greater changes in belonging from their first to second year compared to students in an active control condition; however, it was only for underrepresented students (i.e., racial minority and first-generation students) that the treatment promoted grade point average (GPA) and full-time enrollment. These findings were recently replicated with more than a thousand college students at a Hispanic-serving broad-access university (). Similarly, Walton and colleagues found that this type of intervention did not improve the GPA of White students and men in STEM (respectively), whereas it did for Black students and women in STEM. Therefore, in the present work, we predicted a similar pattern of results such that ESL students (but not non-ESL students) may benefit from the intervention.

The social-belonging intervention materials that have been effective for women and underrepresented students of color included stories from these groups explicitly (, ). That is, the stories included in the intervention materials were attributed to male and female students from various racial and ethnic backgrounds. Thus, matriculating college students could literally “see” their gender and racial groups represented among the juniors and seniors who eventually came to belong in college. But is this methodological choice necessary to achieve effects? The present study used similar materials as previously used; however, by doing so, we examine a previously untested theoretical question: Can the social-belonging intervention improve the belonging, persistence, and performance of matriculating college students who come from stigmatized social group backgrounds not explicitly represented in the intervention materials?

Social identity theory suggests that messages from in-group members might be a particularly powerful force for influencing people’s perceptions of their local environment (). Of course, people have many social identities, including both readily visible (e.g., gender and race/ethnicity,) and socially prescribed identities that are often not as visible (e.g., role-based identities, such as one’s identity as a student). ESL students are unique from other minority students in that most of the ESL students have multiple socially prescribed identities that may not be readily visible such as being a non-native English speaker, being low SES, or being an immigrant as well as other visible identities such as their gender and race. Although our intervention materials communicated stories by students (creating a match to ESL students’ subscribed identities as students), they did not reflect ESL students’ particular proscribed linguistic identity (as people for whom English is a second language) or their specific adversities related to this linguistic identity. Would the psychological experiences of belonging uncertainty common among most college students positively influence ESL students’ anticipated belonging, persistence, and performance?

It is important to note that there has been research on the effectiveness of a variety of interventions on ESL students’ academic success (, ). However, these interventions have largely focused on the implementation of interventions centered around instructional strategies and pedagogical techniques, as well as outcomes such as gains in English language proficiency rather than the psychological experiences of ESL students. Given the extant qualitative research on the role of belonging in ESL students’ academic trajectories (, , ), examining psychologically based interventions to support ESL students’ anticipated belonging is warranted. Therefore, the current work makes an important contribution to the literature on interventions for ESL students.

The present study

Given the success of the social-belonging intervention for students from underrepresented racial backgrounds, we predicted a priori that a prematriculation social-belonging intervention delivered online before students arrived on campus for college would help STEM-interested ESL students anticipate a greater sense of belonging in college over time, boost the proportion of their STEM courses successfully, and earn higher GPAs in their STEM courses relative to STEM-interested ESL students who received an active control condition (see the Supplementary Materials for intervention materials). In addition, belonging is central to the theorized process by which the intervention is hypothesized to improve academic performance and persistence. Murphy and colleagues () found that anticipated changes in belonging mediate the effect of the social-belonging intervention on students’ academic outcomes. Therefore, another goal of the present work was to replicate these findings in a large, multi-institution sample and extend them by testing this process for ESL students.

Although these hypotheses were not formally preregistered, they were the central focus of a National Science Foundation grant that was funded and preceded the analyses. To test these hypotheses, we used data collected by the College Transition Collaborative (CTC) from a large-scale randomized controlled trial at 19 universities in the United States in which more than 12,000 STEM-interested students were randomly assigned to receive a brief, online social-belonging intervention (or active control materials). If the social-belonging intervention increases STEM-interested ESL students’ anticipated changes in belonging as well as their STEM outcomes, it would suggest that addressing ESL students’ sense of belonging in college can positively shape their academic trajectory.

RESULTS

Primary analyses

Anticipated change in belonging process variable

We first examined students’ anticipated change in belonging as the outcome variable (see Table 1). Consistent with the central message of the social-belonging intervention, results revealed a significant main effect of condition, b = 0.26, SE = 0.02, t(12,370) = 10.65, P < 0.001, such that students who received the social-belonging intervention anticipated feeling that they would belong more over time compared to students in the control condition. The main effect of ESL status was not significant, b = −0.03, SE = 0.02, t(12,300) = −1.15, P = 0.250; however, the interaction between ESL status and condition was significant, b = −0.09, SE = 0.05, t(12,370) = −1.97, P = 0.049. Examination of the simple effects of the interaction indicated that both ESL students, b = 0.21, SE = 0.04, t(12,370) = 4.90, P < 0.001, and non-ESL students, b = 0.30, SE = 0.02, t(12,370) = 13.59, P < 0.001, who were randomly assigned to the social-belonging intervention condition anticipated greater belonging over time than did ESL students and non-ESL students in the control condition, respectively (see Fig. 1).

Table 1

Summary of term 1 and year 1 proportion of credits earned and GPA results.

SEs are in parentheses. For main effects analyses, condition was coded as −0.5 = control, 0.5 = treatment; ESL was coded as −0.5 = non-ESL, 0.5 = ESL; and cohort was coded as 0 = cohort 1, 1 = cohort 2. To obtain condition effects among ESL students and non-ESL students individually, we performed two separate sets of analyses with condition recoded. For condition effects among ESL students, ESL status was recentered so 0 = ESL, 1 = non-ESL. For condition effects among non-ESL students, ESL status was recentered so 0 = non-ESL, 1 = ESL. See the R code at the end of the Supplementary Materials for more information about the specifications for each analysis presented. ***P < 0.001, **P < 0.01, and *P < 0.05.

		Term 1		Year 1
Predictor	Belonging change	Proportion of STEMcredits completed	STEM GPA	Proportion of STEMcredits completed	STEM GPA
Main effect ofcondition	0.26*** (0.02)	0.02** (0.01)	0.05** (0.02)	0.01* (0.00)	0.01 (0.02)
Main effect of ESLstatus	−0.03 (0.02)	0.01 (0.01)	0.13*** (0.02)	0.01 (0.00)	0.12*** (0.02)
Condition × ESLinteraction	−0.09* (0.05)	0.03** (0.01)	0.12** (0.04)	0.03** (0.01)	0.06 (0.04)
Cohort	−0.16*** (0.02)	−0.00 (0.00)	−0.02 (0.02)	−0.01 (0.00)	−0.03 (0.02)
ACT composite	0.01*** (0.00)	0.01*** (0.00)	0.09*** (0.00)	0.01*** (0.00)	0.10*** (0.00)
Simple effects for the condition × ESL interaction
Treatment effectamong ESL students	0.21*** (0.04)	0.03*** (0.01)	0.11** (0.04)	0.02** (0.01)	0.04 (0.03)
Treatment effectamong non-ESLstudents	0.30*** (0.02)	0.00 (0.00)	−0.01 (0.02)	−0.00 (0.00)	−0.02 (0.02)

Fig. 1

Anticipated change in belonging condition effects.

Means were calculated controlling for ACT scores and cohort (0 = cohort 1, 1 = cohort 2). Error bars represent SEs. *P < 0.05.

Term 1 academic outcomes

To investigate the effects of the intervention for STEM-interested students on STEM persistence through the first (post-intervention) term of college, we examined the proportion of term 1 STEM credits earned as the outcome variable (see Table 1). Results revealed a significant main effect of condition, b = 0.02, SE = 0.01, t(11,280) = 3.25, P = 0.001, such that students who received the prematriculation social-belonging intervention ultimately earned a significantly higher proportion of STEM credits at the end of their first term in college than did students in the control condition. The main effect of ESL status was not significant, b = 0.01, SE = 0.01, t(11,250) = 1.48, P = 0.139; however, the interaction between ESL status and condition was significant, b = 0.03, SE = 0.01, t(11,280) = 3.07, P = 0.002. Examination of the simple effects of the interaction indicated that ESL students randomly assigned to the social-belonging intervention condition completed a higher proportion of STEM credits at the end of their first term in college, b = 0.03, SE = 0.01, t(11,280) = 3.57, P < 0.001, compared to ESL students in the control condition (see Fig. 1). In contrast, no significant treatment effect was observed among non-ESL students, b = 0.00, SE = 0.00, t(11,280) = 0.19, P = 0.848 (see Fig. 2A).

Fig. 2

Term 1 condition effects.

(A and B) Means were calculated controlling for ACT scores and cohort (0 = cohort 1, 1 = cohort 2). Error bars represent SEs. *P < .05.

To investigate the effects of the intervention for STEM-interested students on STEM performance through the first (postintervention) term of college, we examined term 1 STEM GPA as the outcome variable. Results revealed a significant main effect of condition, b = 0.05, SE = 0.02, t(10,840) = 2.61, P = 0.009, such that students who received the social-belonging intervention earned a significantly higher GPA than students in the control condition. There was also a significant main effect of ESL status, b = 0.13, SE = 0.02, t(10,850) = 6.18, P < 0.001, such that ESL students outperformed non-ESL students. Furthermore, the interaction between condition and ESL status was significant, b = 0.12, SE = 0.04, t(10,840) = 2.86, P = 0.004 (see Table 2). Examination of the simple effects of the interaction indicated that ESL students randomly assigned to the social-belonging intervention condition earned significantly higher term 1 STEM GPAs, b = 0.11, SE = 0.04, t(10,840) = 3.08, P = 0.002, than did ESL students in the control condition (see Fig. 2B). In contrast, there was no treatment effect observed among non-ESL students, b = −0.01, SE = 0.02, t(10,840) = −0.27, P = 0.790.

Table 2

Moderated mediation results.

Condition, coded as −0.5 = control, 0.5 = treatment, was the independent variable; ESL status, coded as −0.5 = non-ESL, 0.5 = ESL, was the moderator; anticipated belonging was the mediator; and each academic outcome was the dependent variable. Consistent with previous analyses, ACT scores and cohort, coded as 0 = cohort 1, 1 = cohort 2, were included as covariates. CI, confidence interval.

	ESL students				Non-ESL students
Outcomevariable	Indirect effect	Direct effect	Total effect	Proportion oftotal effectmediated byindirect effect	Indirect effect	Direct effect	Total effect	Proportion oftotal effectmediated byindirect effect
Term 1proportion ofcredits(n = 11,279)	b = 0.001193	b = 0.031242	b = 0.032435	b = 0.035918	b = 0.001803	b = −0.000660	b = 0.001143	b = 0.206760
	95% CI	95% CI	95% CI	95% CI	95% CI	95% CI	95% CI	95% CI
	(0.00038,0.00223)	(0.01331,0.04878)	(0.01465,0.04989)	(0.01140,0.09307)	(0.00062,.00302)	(−0.00995,0.00869)	(−0.00806,0.01045)	(−5.3093,6.55635)
	P = 0.0032	P < 0.001	P < 0.001	P = 0.004	P = 0.002	P = 0.896	P = 0.825	P = 0.825
Term 1 GPA(n = 10,842)	b = 0.003548	b = 0.109542	b = 0.113072	b = 0.030710	b = 0.005293	b = −0.009890	b = −0.004653	b = −0.129269
	95% CI	95% CI	95% CI	95% CI	95% CI	95% CI	95% CI	95% CI
	(0.00036,.00757)	(0.03673,0.18097)	(0.04049,0.18478)	(0.00235,0.10483)	(0.00055,0.01012)	(−0.04760,0.02804)	(−0.04252,0.03330)	(−4.58174,3.99880)
	P = 0.029	P = 0.004	P = 0.003	P = 0.032	P = 0.028	P = 0.617	P = 0.822	P = 0.824
Year 1proportion ofcredits(n = 11,905)	b = 0.000907	b = 0.022373	b = 0.023280	b = 0.037870	b = 0.00140	b = −0.00345	b = −0.00206	b = −0.21906
	95% CI	95% CI	95% CI	95% CI	95% CI	95% CI	95% CI	95% CI
	(0.00022,0.00176)	(0.00672,0.03825)	(0.00750,0.03895)	(0.00879,0.12439)	(0.00037,0.00242)	(−0.01151,0.00464)	(−0.01013,0.00599)	(−5.8007,4.57535)
	P = 0.004	P = 0.005	P = 0.003	P = 0.007	P = 0.007	P = 0.408	P = 0.627	P = 0.628

Year 1 academic outcomes

To examine whether these academic effects persisted through the first year of college, we began by examining students’ proportion of STEM credits earned at the end of year 1 (see Table 2). Results revealed a significant main effect of condition, b = 0.01, SE = 0.00, t(11,910) = 2.32, P = 0.020, such that students who received the social-belonging intervention earned a significantly higher proportion of STEM credits at the end of their first year in college than did students in the control condition. The main effect of ESL status was not significant, b = 0.01, SE = 0.00, t(11,880) = 1.59, P = 0.112; however, the interaction between ESL status and condition was significant, b = 0.03, SE = 0.01, t(11,900) = 2.85, P = 0.004. Examination of the simple effects of the interaction indicated that ESL students randomly assigned to the social-belonging intervention condition completed a higher proportion of STEM credits, b = 0.02, SE = 0.01, t(11,900) = 2.92, P = 0.003, compared to ESL students in the control condition (see Fig. 3A). In contrast, treatment effects were not observed among non-ESL students, b = −0.00, SE = 0.00, t(11,910) = −0.57, P = 0.567.

Fig. 3

Year 1 treatment effects.

(A and B) Means were calculated controlling for ACT scores and cohort (0 = cohort 1, 1 = cohort 2). Error bars represent SEs. *P < .05.

Last, when STEM GPA was examined at the end of students’ first year in college (see Table 2), results indicated that the main effect of condition was not significant, b = 0.01, SE = 0.02, t(11,650) = 0.42, P = 0.676. The main effect of ESL status was significant, b = 0.12, SE = 0.02, t(11,670) = 6.37, P < 0.001, such that ESL students outperformed non-ESL students. However, unlike term 1 effects, the interaction between condition and ESL status was not significant, b = 0.06, SE = 0.04, t(11,650) = 1.65, P = 0.098. Moreover, examination of the simple effects of the interaction indicated that treatment effects were not observed among ESL students, b = 0.04, SE = 0.03, t(11,650) = 1.17, P = 0.243, or non-ESL students on year 1 STEM GPA, b = −0.02, SE = 0.02, t(11,660) = −1.34, P = 0.182 (see Fig. 3B). It should be noted that there may be a tradeoff between persistence and performance in STEM fields of study. Greater persistence in STEM courses can have negative consequences for GPA, as illustrated by the fact that STEM majors often earn lower GPAs than non-STEM majors because of the difficulty of STEM material (). By supporting more ESL students to persist and complete more STEM credits at the end of the first year, the intervention may have no effect (or, perhaps over the long term, a negative effect) on STEM performance over time when compared with untreated ESL students who have lower levels of persistence in STEM. However, most colleges and universities judge persistence and graduation as primary metrics of institutional success ().

Moderated mediation

Next, we examined whether treated ESL students persisted more and performed better than their nontreated peers, in part, because of the positive impact that the intervention had on students’ anticipated change in belonging (see Table 2 for a full summary of results). Consistent with predictions and the main effects discussed above, the indirect effect of anticipated change in belonging on all four academic outcome variables was significant, regardless of ESL status (bs > 0.001, Ps < 0.05). However, congruent with our moderation findings, the direct effects of the intervention on students’ persistence (the proportion of STEM credits earned at the end of term 1 and year 1) and term 1 performance (term 1 STEM GPA) were explained by anticipated changes in belonging (i.e., the indirect effects divided by the total effects), and these effects were only significant among ESL students (bs > 0.03, Ps < 0.05; non-ESL students: bs < 0.22, Ps > 0.628). Together, these results suggest that the intervention was associated with greater anticipated changes in belonging among both ESL and non-ESL students; however, these changes in anticipated belonging were associated with greater STEM persistence and performance only among ESL students.

Exploring alternative explanations through robustness analyses

Given that ESL students are a heterogeneous group (), it is possible that the observed treatment effects are explained by membership in other demographic groups that often co-occur with the ESL identity (e.g., race/ethnicity and SES). It is possible that the ESL findings are due, in part, to students’ membership in these other groups, rather than to their ESL identity, specifically. To investigate these alternate explanations directly, we performed robustness checks that examined the influence of the intervention as a function of ESL group membership as well as each of these other demographic groups on all of the academic outcomes. Below, we provide a brief overview of these results; however, the full report can be found in tables S5 to S8. Results of these analyses without ESL status or its interaction with condition can be found in table S9.

First, we tested whether the ESL findings were robust to students’ Latinx identification. Across all outcome variables (i.e., term 1 and year 1 proportion of STEM credits and term 1 and year 1 STEM GPA), there was a main effect of Latinx identification such that Latinx students earned a significantly lower proportion of credits and lower STEM GPA than did non-Latinx students (bs > 0.02, Ps < 0.001). However, the interaction between condition and Latinx was not significant (bs < 0.07, Ps > 0.10). Across the three statistically significant academic outcome variables, the interaction between ESL status and condition remained significant (bs > 0.02, Ps < 0.05; as in the previous analyses, there was no interaction on STEM GPA at the end of year 1). Moreover, the ESL treatment effects persisted above and beyond the Latinx effects when both were included in the models (bs > 0.03, Ps < 0.01).

Second, we examined whether the ESL findings were robust to students’ Asian identification. Results revealed that, compared to non-Asian students, Asian students earned a significantly greater proportion of STEM credits during both term 1 and year 1 and earned a significantly higher STEM GPA for both term 1 and year 1 (bs > 0.01, Ps < 0.05). However, Asian identification did not interact with condition to predict any academic outcome (bs < 0.06, Ps > 0.10) and, with the exception of year 1 STEM GPA (which was not statistically significant in the original analyses), the interaction between condition and ESL status remained significant (bs > 0.03, Ps < 0.01). Moreover, the ESL treatment effects persisted above and beyond the Asian identity effects when both were included in the models (bs > 0.02, Ps < 0.01).

Next, we examined whether the ESL findings were robust to students’ SES. results revealed that lower SES students and higher SES students did not significantly differ in the proportion of STEM credits earned in term 1 or year 1 (bs < 0.01, Ps > 0.10); however, lower SES students earned a lower STEM GPA at both term 1 and year 1 compared to higher SES students (bs > −0.06, Ps < 0.01). Providing support for the robustness of the ESL effects, results revealed that SES did not interact with condition to predict any academic outcome (bs < 0.05, Ps > 0.10). Moreover, the ESL treatment effects remained significant on all outcomes (except year 1 STEM GPA) when SES was included in the models (bs > 0.03, Ps < 0.01).

We next examined whether the ESL findings were robust to students’ citizenship status. Students who were citizens did not significantly differ from students who were non-citizens in their proportion of STEM credits earned in term 1 or year 1 or in term 1 STEM GPA (bs > 0.05, Ps > 0.10); however, students who were noncitizens earned significantly higher year 1 STEM GPAs than did students who were citizens (b = 0.05, P < 0.05). Consistent with the other robustness checks, across all outcome variables, the interaction between condition and citizenship status was not significant (bs < 0.01, Ps > 0.10). Moreover (with the exception of year 1 STEM GPA), the interaction between ESL status and condition and the simple effects among ESL students remained significant (bs > 0.02, Ps < 0.01).

Together, these analyses suggest that the social-belonging intervention had a unique and robust effect on ESL students’ STEM persistence (in the first term and year of college) and performance (in the first term of college). These findings could not be explained by ESL students’ membership in other stigmatized identity groups.

DISCUSSION

Overall, the results of the present work indicate that STEM-interested ESL (and non-ESL) students who received a prematriculation social-belonging intervention showed the desired psychological effect; treated students expected that they would feel like they belonged more over time than did students who did not receive the intervention materials. However, these psychological gains only bolstered the academic persistence and performance of ESL students (not non-ESL students). Specifically, STEM-interested ESL students who received the social-belonging treatment completed more of the STEM courses that they attempted at the end of their first term in college—and this effect persisted through the first year of college. Moreover, these students performed better in their first-term STEM courses (i.e., earned higher STEM GPAs in term 1) than did their untreated ESL peers. Last, we found evidence suggesting that the anticipated feelings of belonging that were engendered by the social-belonging treatment mediated students’ STEM persistence and performance.

Together, these findings fill several noteworthy gaps in the educational and psychological literature. Specifically, this study expands research to date by investigating these psychological processes among ESL college students rather than elementary or high school students—groups that comprise the bulk of the ESL literature. Moreover, it is the only quantitative study to our knowledge that examines and addresses social belonging as a psychological barrier that ESL students in STEM contend with in the college context. Last, this is the only experimental study to test a randomized controlled psychological intervention to improve the academic persistence and performance of ESL students.

This study also makes several important theoretical contributions. First and foremost, our results augment the rich body of qualitative research on ESL students in postsecondary contexts, suggesting that social belonging plays a meaningful role in the persistence of ESL students in STEM. Like students from other stigmatized groups, ESL students are faced with an academic climate that includes substantial threats to belonging, such as being numerically underrepresented, required to pass English proficiency tests, and/or required to enroll in additional language learner courses in college (, , ). These practices, necessary though they may be to support learning, may signal to ESL students that they are different from their native English–speaking counterparts and that they may not belong in college. ESL students may therefore question their belonging and leave difficult STEM fields of study as a result, which may, in turn, reinforce their belonging concerns. Although more research is needed to definitively investigate these recursive processes among ESL students, the observed longitudinal effects that persisted until the end of the first year in college are encouraging. This positive recursive cycle of sustained persistence (and in some cases, performance) has been demonstrated for low SES, first-generation, and racial/ethnic minority students, for whom previous belonging interventions have been effective (–).

Another theoretical contribution of the present work is that it suggests the content of the social-belonging intervention (i.e., the student stories and strategies) may not need not be tailored to the specific social identities of students (e.g., their ESL identity), as long as the materials include students from racially and ethnically diverse backgrounds who describe the psychological challenges of developing a sense of belonging that are common among those who share a role-based identity, such as students transitioning to college. We learned that the social-belonging intervention was effective for ESL students’ persistence in STEM, although it did not reference ESL students and the particular challenges conferred by their linguistic identity. Moreover, when accounting for ESL students’ other visible and invisible identities (i.e., race, SES, and citizenship status), ESL status was a more consistent predictor of ESL students’ belonging and academic outcomes. Therefore, researchers and educators who wish to use belonging interventions to increase the performance of underrepresented groups could potentially use a single set of materials for students who have reason to question their belonging in college because of membership in various stigmatized social groups.

Despite the theoretical contributions and applied implications of the present work, it is prudent to note some of its limitations. First, although our robustness checks suggest that the intervention is particularly effective for students for whom English is not their first language, our intervention materials do not allow us to completely rule out the possibility that our effects were due to students’ identification with the shared visible identities (i.e., race and gender) of the advanced undergraduates in these stories. The racial-ethnic and gender diversity of the student role models are likely important for a heterogeneous group such as ESL students who come from many different demographic and social backgrounds (and who share some of these visible characteristics with the student role models). A recent meta-analysis of 45 empirical studies found that exposure to in-group role models has a significant positive effect on underrepresented students’ interest and performance in STEM (). Moreover, research indicates that role models, particularly near-peer role models who demonstrate a command over the English language, increase ESL students’ academic motivation and their tendency to view mistakes as learning opportunities (, ).

Another limitation is that we used a dichotomous self-report measure to assess college students’ ESL status. Although this measure allowed us to explore how the social-belonging intervention affects all self-identified ESL students regardless of their English proficiency, research indicates that there is important variability in this group. Long-term English language learners (i.e., students who do not earn the minimum English proficiency test scores needed to be considered proficient for six consecutive years) are more likely to drop out of college and earn lower grades than English language learners who meet the threshold for proficiency (). Given the number of institutions and varying conventions each uses to operationalize ESL students, we relied on students’ self-reports and did not have access to this type of nuanced language proficiency data in the current study; however, this type of diversity within the ESL student population will be important to address in future research. Similarly, future research would benefit from obtaining additional measures related to ESL students’ precollege experiences (e.g., access to college-preparatory coursework in high school, limited curricular choices, and knowledge about the university structure) to determine whether these factors moderate the effectiveness of the intervention (, ).

Other important issues for future research are how ESL students—as a group—are operationalized and defined by institutions and researchers (and thus which students are included in studies with ESL students) as well as the role of the various labels that are used to characterize ESL students (e.g., ELL and LEP) (). Defining ESL students differently (e.g., ELL and LEP) can change the composition of the groups that are studied, rendering studies of this population difficult to compare—a problem that scholars have previously noted (). Moreover, some labels such as ELL and LEP can communicate deficiencies or suggest that ESL students have less academic potential—and students are attuned to these meanings (, , ). From a social identity threat perspective, students who are labeled with terms that denote deficiencies and lack of academic potential may be less likely than students labeled as ESL to anticipate that feeling like they belong in college. As such, a social-belonging intervention could work differently for students depending on the label that institutions confer on students. Last, future research examining the impact of the social-belonging intervention on ESL students’ psychological experiences in college should measure ESL status in multiple ways (e.g., self-identification, university designations, and continuous measures of English proficiency development) to provide a more nuanced understanding of the postsecondary school experiences of ESL students.

One strength of our study was that it spanned 19 different 4-year colleges and universities and included over 12,000 STEM-interested students, allowing us to examine various robustness checks with identities that intersect with ESL status (race, SES, and citizenship status). However, it is possible that the effects of the intervention for ESL students may be moderated by institutional characteristics. For example, institutions that offer substantial social support for ESL students (e.g., clubs, student groups, and dedicated offices that ESL students can go to for help) and institutions with high enrollment of ESL students may help ESL students accomplish the intervention message of coming to belong over time. Without these institutional and social supports, it may be harder for ESL students to engage in the positive recursive cycle in which feelings of belonging and academic performance are mutually promoted through the intervention. Although these questions are beyond the scope of the current paper, it will be important in future research to examine institutional characteristics that may bolster the intervention message.

In conclusion, the present study provides experimental evidence of an effective, scalable strategy for meeting the needs of the rapidly increasing population of ESL students in higher education. Although a consideration of linguistic and instruction-based barriers is important, researchers and educators should also address the psychological barriers with which ESL students contend. Like other underrepresented students, ESL students’ sense of belonging is a critical component of their academic success. Online psychological interventions that target students’ sense of belonging can effectively bolster the STEM persistence and performance of ESL college students. By integrating the present findings to the existing literature, researchers and practitioners can begin to understand how to more holistically support the psychological and motivational experiences of ESL students in college.

MATERIALS AND METHODS

Experimental methods

The CTC’s social-belonging intervention used materials that were slightly modified from Yeager and colleagues (), experiments two and three. Students at each of the 19 institutions were randomly assigned to one of three experimental conditions (standard social-belonging intervention, customized social-belonging intervention, or control). Across all conditions, students read stories written by upper-year undergraduates about the transition to college, wrote a short “saying is believing” essay that encouraged students to describe what they read about in their own words and internalize the intervention message, and then completed the demographic measures listed below. The specific content of the stories and essay prompt differed across conditions. In both the standard and customized social-belonging interventions, the stories that students read, and the essay prompt focused on challenges related to feelings of belonging that occur in the transition to college. The customized version included institution-specific challenges (determined via student focus groups at each of the 19 schools), whereas the standard version did not vary meaningfully by institution or previous research (). In the control condition, the content of the stories and the essay prompt pertained to the challenges that students face when transitioning to the new physical environment of college and did not address students’ belonging concerns. The standard social-belonging intervention and control materials were identical across institutions. There were no differences across any of our outcomes between the two social-belonging treatment conditions; thus, all analyses are collapsed across these two versions of the treatment. The implementation of the intervention was identical across institutions. Primary institutional review board (IRB) approval for the CTC project was obtained from Stanford University; however, some institutions required researchers to obtain local IRB approval and other institutions’ IRBs classified the intervention as quality assurance and did not require a formal IRB application.

Participants

Participants were incoming first-year undergraduates who completed the CTC’s prematriculation social-belonging intervention online after accessing it through a summer orientation checklist and/or email sent by the university before the first day of classes that were to be completed by students before classes began on campus. Among other online tasks—like specifying dorm preferences and providing immunization documents—the link to the intervention was provided and described as offering incoming first-year students a chance to learn about what it is like to be a student at that particular university. Informed consent was obtained from all participants after they were provided with information about the nature of the study and potential risks of participation. Data were collected from sixteen 4-year colleges and universities in 2015 and 2016 and 6 additional 4-year colleges and universities in 2016. By design, these institutions varied considerably in terms of selectivity, residentiality (), and region. For the purposes of the current work, 19 of these universities were included in analyses due to missing data from one university that was relevant to our hypotheses (e.g., credits earned). See table S10 for institution-level characteristics.

Our final analytic sample consisted of the 12,411 students (50.8% female) who provided an answer to the question, “Is English your first language?” (2635 ESL students and 9771 native English speakers), who expressed an interest in pursuing a STEM major, and for whom we had ACT scores to control for past performance. Participants self-identified as White/European American (41.1%), Asian (25.9%), Hispanic (16.8%), Black/African American (7.0%), multiracial (4.4%), Middle Eastern (2.7%), Native American (0.8%), or “other” (0.6%). Most of the participants were citizens (89.4%) who were, on average, from middle class families. For additional demographic variables and a breakdown of ESL status by demographic variables, see table S1. For details about any missing data that resulted in some variation in degrees of freedom, see table S2. For details about the full CTC sample, see the CONSORT diagram presented in fig. S1.

STEM interest

STEM interest was assessed in the first cohort by asking students to rate the following on a scale ranging from 1 (no interest) to 5 (a great deal of interest): “How interested are you in majoring in science, technology, engineering, or math?” Students in the second cohort rated two almost identically worded questions on a three-point scale with the same scale endpoints—one referred to science, technology, and math and the other referred to engineering. We took the highest rating of these two questions to identify the most highly STEM-interested students. To provide correspondence across cohorts, students’ responses within cohort were standardized and combined across cohorts. We then divided the standardized STEM interest values into quintiles to approximate a five-point scale. Those who were in the highest quintile (roughly equivalent to a five on the five-point STEM interest scale) were considered highly STEM interested and included in our analyses. These scores were approximately 0.6 SDs above the STEM interest mean for all students.

Anticipated change in belonging psychological process variable

Students’ anticipated change in belonging from fall of their first year of college to fall of their second year of college was assessed using a difference score between two three-item measures administered in the same online session as the intervention (). The first measure was administered in the summer before students started their first year of college and asked students questions such as “How much do you think you will feel you belong at [school name] when you arrive on campus this fall?” (α = 0.88). The second measure (α = 0.93), which was administered later in the same survey, used the same questions, but altered the wording to assess anticipated belonging in their second year, “At the end of your sophomore year, how much do you think you will feel you belong at [school name]?” All items were rated on a seven-point scale ranging from 1 (not at all) to 7 (an extreme amount). The anticipated change in belonging score was created by subtracting anticipated year one belonging from anticipated year two belonging. Higher scores indicate more anticipated growth in feelings of belonging over time.

STEM credits and GPA

Each institution provided the following information for each student’s first year of coursework: credits taken, credits earned, and grade obtained for each course. We used these data to calculate each student’s number of STEM credits taken, number of STEM credits earned, and STEM GPA during their first semester and first year of enrollment. We standardized credits taken and earned to adjust for any differences in institutional credit systems. For example, a 3-hour semester course was coded as three credits regardless of how many credits the institution used for its own record-keeping purposes. To assess persistence in STEM, we divided the number of STEM credits students earned by the number of STEM credits taken to obtain the proportion of STEM credits students ultimately earned in term 1 and at the end of year 1 in college. The significance and strength of the results were identical when using the number of STEM credits taken; however, the proportion of STEM credits earned takes into consideration the number of credits in which students originally enrolled—course enrollment took place for most students before they completed the intervention—allowing us to capture course dropping behavior.

Demographics

ESL status was assessed using the dichotomous question described above. Latinx and Asian identification was measured by asking students to select all of the racial/ethnic groups with which they identified from a list of 23 racial/ethnic identities. If students selected more than one racial/ethnic group, they were presented with a second list of 19 racial/ethnic groups and asked to select the group with which they most strongly identified. The racial/ethnic group that the student most strongly identified with was used for analysis. Students who selected Puerto Rican, Mexican American/Chicano, Central American, Hispanic, or Other Hispanic when identifying their primary identity were categorized as Latinx, and all other students were categorized as non-Latinx. Students who selected East Asian, South Asian, Southeast Asian, and Other Asian were categorized as Asian, and all other students were categorized as non-Asian. We assessed students’ subjective SES by asking, “How would you describe your family’s social class?” (1 = working class, 2 = lower middle class, 3 = middle class, 4 = upper middle class, and 5 = upper class). Students who selected working or lower middle class were categorized as lower SES, and all other students were categorized as higher SES. Citizenship was determined by asking students to identify their U.S. citizenship status. Answers to this question were then dichotomized as citizens and non-citizens. For the exact wording of all of these questions and lists of response options, see the Supplementary Materials.

Previous performance

We obtained students’ ACT/SAT scores from their university and converted SAT verbal and math scores to ACT composite scores.

Statistical analysis

Primary analyses

All analyses used multilevel modeling, conducted with the lmer package in R version 3.5.1 to account for the fact that students are nested within institutions. Restricted maximum likelihood models were used for all analyses, as recommended for this type of data. To obtain P values for fixed effects, we used the lmerTest package, which uses the Satterthwaite approximations of degrees of freedom. For each outcome variable, we ran three analyses. In the first analysis, we examined the main effect of condition (−0.5 = control, 0.5 = treatment), the main effect of ESL status (−0.5 = non-ESL, 0.5 = ESL), and the interaction between the two. For the second analysis, we conducted simple effects tests and recentered condition (0 = control, 1 = social-belonging intervention) and ESL status (0 = ESL, 1 = non-ESL) to obtain condition effects among ESL students. For the third analysis, we conducted simple effects tests and recentered condition (0 = control, 1 = social-belonging intervention) and ESL status (0 = non-ESL, 1 = ESL) to obtain condition effects among non-ESL students. Given the multitude of research demonstrating a correlation between SAT/ACT scores and college GPAs (), students’ SAT/ACT test scores were included as a covariate in all analyses. Moreover, because two cohorts of students participated in the study, cohort was included as a covariate (coded 0 = cohort 1, 1 = cohort 2). The Supplementary Materials includes the R code for all analyses, descriptive statistics and correlations among measures across condition (table S3), and results without covariates (table S4). All results remain significant when covariates are removed from the models.

Moderated mediation

We performed four multilevel moderated mediation analyses using the R package Mediation. Condition (−0.5 = control, 0.5 = treatment) was the independent variable, ESL status (−0.5 = non-ESL, 0.5 = ESL) was the moderator, and each academic outcome was the dependent variable. Consistent with previous analyses, ACT scores and cohort (0 = cohort 1, 1 = cohort 2) were included as covariates. See the Supplementary Materials for the R code used in these analyses.

Alternative analyses

For each demographic group, in the first analysis, we examined the main effect of condition (−0.5 = control, 0.5 = treatment), the main effect of ESL status (−0.5 = non-ESL, 0.5 = ESL), the main effect of each alternative demographic group (−0.5 = non-Latinx, non-Asian, high SES, or citizen; 0.5 = Latinx, Asian, low SES, or non-citizen), the interaction between ESL status and condition, and the interaction between each demographic group and condition. For the second analysis, we recentered condition (0 = control, 1 = social-belonging intervention) and ESL status (0 = ESL, 1 = non-ESL) to obtain condition effects among ESL students. For the third analysis, we recentered condition (0 = control, 1 = social-belonging intervention) and ESL status (0 = non-ESL, 1 = ESL) to obtain condition effects among non-ESL students. Consistent with our primary analyses, students’ SAT/ACT test scores and cohort (0 = cohort 1, 1 = cohort 2) were included as covariates in all analyses.