Content and Quality of Science Training Programs Matter: Longitudinal Study of the Biology Scholars Program.

Many science training programs are successful at supporting students in completing their degree programs. However, it is not clear which aspects of these programs meaningfully contribute toward achieving this goal. The current longitudinal study examined a well-established science training program, the Biology Scholars Program (BSP) at the University of California, Berkeley, to see whether social connections formed in BSP and/or enthusiasm about the BSP activities are key components in contributing to students' greater integration into their professional communities at 12 months and intentions to persist at 18 months into the program. Results indicated social connections and program enthusiasm at 6 months were unassociated with science efficacy, identity, and community values. However, social connections and program enthusiasm at 12 months were generally associated with higher levels of all these variables, with science identity and community values uniquely related to greater integration. Together, results show that students' connection to faculty, staff, and peers and enthusiasm for the program activities are both key components of successful, multiyear science training programs. Our results also suggest that, while connections and enthusiasm might develop quickly, their downstream consequences might only be observed after students build stronger social relations and enthusiasm for program activities in ways that foster greater integration.

INTRODUCTION

Race has defined who has access and who belongs in higher education since the beginning of the United States (Asai, 2020), and the persistent (and insistent) systems that perpetuate inequity remain. Meanwhile, the struggle to achieve equitable workforce development in science career pathways in the United States has been well-recognized and attributed to poor retention (rather than recruitment) of diverse scholars in academia (President’s Council of Advisors on Science and Technology, 2012; National Academies of Sciences, Engineering, and Medicine [NASEM], 2016). National data describing the pattern of retention of historically underrepresented (HU)—African-American, Hispanic or Latino/Latina, American Indian/Native American, and Alaskan Native—and first-generation undergraduates, graduates, and faculty in science, technology, engineering, and mathematics (STEM) show that ethnic disparities only increase as people progress along career pathways (Estrada ). Theoretical models have been used to increase understanding of the barriers and opportunities that exist to move toward greater inclusion and equity. In response, interdisciplinary research teams of educators, medical experts, and social scientists, guided by theory-driven research designs and hypotheses, have been advancing the science of mentorship, science training programs, and anti-racist curricula. The advances and limitations of this research are well described in several recent National Academies’ reports (NASEM, 2016, 2020), white papers (Estrada, 2014; Estrada ), and high-profile publications (Linn ). Quantitative studies that test theoretical models most often are either focused on national participant pools (e.g., the 11-year ScienceStudy) or assess the impacts of course curriculum changes (e.g., studies on course-based undergraduate research experiences, freshman research initiatives, etc.; NASEM, 2017). Longitudinal studies of one-site cocurricular science training programs with large enough cohorts of study participants to quantitatively test theoretical models are rare. When conducted, they typically are descriptive. This paper uniquely describes a longitudinal study that examines what aspects of a well-established science training program, the Biology Scholars Program (BSP) at the University of California, Berkeley, contribute toward students experiencing integration into their science professional communities and persisting in science career pathways.

Measuring Success of Science Training Programs

When assessing what aspects of a program contribute toward its success, it is worth noting that the term “success” is not always measured the same way. A review of the 31 National Institutes of Health–funded research studies that assessed the impact of cocurricular HU science training programs, showed that the most common metrics of success were measures of increased student interest and persistence. Specifically, they measured students’ increased likelihood to “major in a STEM field, graduate with a STEM degree, enroll in STEM courses, apply to graduate school including master’s, doctoral, and medical programs, work in a STEM field, have peer reviewed publications, and eventually serve as a principal investigator on RO1 grants” (Estrada, 2014, p. 3). The common feature of “success” is evidence that students do something as a consequence of their involvement in a training program, which results in continued involvement and accomplishments in STEM fields. Shorter-term metrics of successful programs includes a bevy of mediators (i.e., factors that are shown to explain what leads to longer term persistence) such as student acquisition of skills (efficacy), increased science identity, increased motivation, and intention to pursue a research career (Estrada ; Hernandez ).

While objectively measured “hard” behaviors are the gold standard of measuring improvement, a wide variety of studies have used self-reported intentions. From the field of social psychology, there is strong evidence that the leap from intention to actual engagement is not far (Kaiser and Wilson, 2004), especially when the intention is specific. Further, Lent and colleagues have repeatedly shown that intentions to pursue STEM majors do predict enrollment and persistence behaviors (Lapan ; Lent ), as well as performance of science skills (Luzzo ; Sullivan and Mahalik, 2000). Thus, research indicates that there is reason to acquire intention information as an interim measure for behavioral outcomes or as a predictor.

Biology Scholars Program

Previous research examining the BSP has measured success in multiple ways, including showing scholars are more likely to engage in science course progression, declare a science major, and graduate in that degree than other students with similar risk factors at that same institution (Estrada ). This study extends this line of work by providing longitudinal data that measure programmatic experiences, students’ psychosocial responses, and outcomes within the context of the historically successful BSP at UC Berkeley (Matsui, 2018). This program, funded by the Howard Hughes Medical Institute (HHMI) and supported by UC Berkeley administration for nearly 30 years, promotes academic and career success in science among undergraduate scholars considered to be members of HU groups based on economics, ethnicity, gender, or cultural experiences. The Departments of Molecular and Cell Biology and Integrative Biology have administered this program, which provides academic and personal advising, mentorship, career seminar series, access to paid research opportunities, academic support for “gateway” courses in the biology major, and socializing opportunities to more than 400 students annually. There is wraparound support as well to address challenges that develop beyond the classroom concerning family, financial, or personal issues. Throughout the program, students receive a consistent message that staff and faculty provide both “high expectation and high academic support” (Matsui , p. 118).

Scholars from all backgrounds can apply to this program, including low-income and/or first-generation scholars and ethnically underrepresented group members who are identified through on-campus programs and encouraged to apply. Interested students complete online written applications and participate in one-on-one interviews that focus on their passion for science and commitment to service. Analysis of 1 year of BSP enrollment shows that BSP selects scholars who without BSP would be the most at risk academically (Estrada ). While much has been written about the success of this program (Matsui ; Matsui, 2007, 2018), this paper provides a longitudinal description of how elements of the BSP program impact student integration into the scientific community and intentions to persist.

Student Integration into the Scientific Community: The Tripartite Integration Model of Social Influence

For many years, social psychologists have studied the conditions under which people will comply with the norms of a community of which they are a part. One theory, derived from the work of Herbert Kelman in 1958, described how a person’s orientation to an influencing agent or social system could predict when conformity with norms or a demand would occur (Kelman, 1958, 1961). More recent research using this theory in the context of higher education has shown that students conform through several social influence processes, such as feeling confidence in doing what the group does (which in science communities means having confidence one can do science, or science efficacy), self-identifying with the group (which in science means thinking of oneself as belonging to the scientific community), and finally, conforming because they feel alignment between their personal and the community’s values system (which would mean that the community’s values are internalized as one’s own values). These factors comprise the tripartite integration model of social influence (TIMSI). Several studies using TIMSI have shown that these factors each relate to students persisting in STEM fields (Estrada ), even 4 years after efficacy, identity, and values were measured (Estrada ).

These findings build on previous research showing that science efficacy is positively related to continuing to pursue a scientific career (Lent, 2007; Chemers ) and achievement in academia (Lent ; Hackett ). Previous research has also indicated that while science efficacy may be a necessary component for integration, it is not the most predictive of persistence when identity and values are also part of the model (Estrada ). In particular, there have been several studies showing that when students self-identify as a scientist (i.e., feel they are a part of the community of scientists), they are more likely to engage in behaviors that are in alignment with the expectations of that role and to persist in a scientific career (Chang ; Chemers ). There is some evidence that historically excluded students do not experience belonging (Hausmann ) and assume academic identity at the same rate as nonminority, “included” students (Hurtado ). As with science identity, the research has shown that when students report endorsing the values of the scientific community, they are more likely to persist years later (Estrada ).

Extending the Research

Previous research on the TIMSI has mostly drawn from national science training and mentorship programs and has shown that both research experience and quality mentorship can contribute to greater intentions to be a part of the STEM community (measured as intention to work in and actual engagement in persistence in the field; Estrada ). This study, called the Gift It Forward study, instead seeks to examine which components of a specific science training program, the BSP at UC Berkeley, impact student persistence and to examine which psychosocial factors may explain such impacts. With this in mind, and based on previous research, we hypothesize the following:

Science efficacy, identity, and values will each significantly relate to intentions to persist in science fields in terms of zero-order correlations.

Science identity and values will uniquely predict persistence in science fields (in terms of partial/regressive associations).

If these hypotheses are confirmed, the findings would replicate previous findings with a new population of students who are drawn from a wide spectrum of ethnicities and races. In addition, we aim to extend the research by asking: What characteristics of the BSP contribute toward students integrating into the scientific community and keeping their intentions to persist in science careers? For this study, we measured two components of BSP that are characteristics of most science training programs. Enthusiasm for the program activities is a measurement of the appreciation for the components of the program, which for BSP included advising, tutoring, student meetings and receptions, IDS 96: Studying the Biological Sciences, using the BSP Student Room, and online communications. The second component was perceived sense of social connection to other people in the program, including staff, faculty, and peers. Social connection includes measures of the closeness and importance of others and how much scholars would miss others in BSP if they were to go away. The purpose of assessing the impact of program activities separate from social connection is critical to better understanding why programs with similar programmatic activities can have very different outcomes. Both enthusiasm for program activities and social connections can be considered social influence agents in the TIMSI, because they are proposed precursors of social influence processes. To advance knowledge in this area, we will test the conceptual models shown in Figure 1 to answer the following research questions:

FIGURE 1.

Conceptual models tested in the current study. Model 1 examined activity enthusiasm and social connections 6 months into the program, whereas model 2 examined activity enthusiasm and social connections 12 months into the program. Months refer to the number of months students had been enrolled in the BSP.

Are social connections within the BSP (i.e., how much the staff, faculty, and other students in the program matter) and/or activity enthusiasm about BSP elements (i.e., social influence agents) associated with greater science efficacy, identity, values (i.e., social influence processes), and integration into the science community (measured as intentions to pursue a career in science)?

Do science efficacy, identity, and/or values mediate the associations between social connections and activity enthusiasm and integration into the science community?

These questions will be answered, while assessing whether the length of time enrolled in BSP (6 months vs. 12 months) influences the degree to which social connections and/or activity enthusiasm are associated with science efficacy, identity, and values 12 months into the program and integration into the science community 18 months into the program.

METHOD

Participants

The Gift It Forward study was launched in 2014 to track student longitudinal experiences across their program participation. Although the BSP has been successful in its efforts to help “at risk’ students graduate at the same grade point average (GPA) level or higher than their non-BSP counterparts (Estrada ), there has been little research to examine what makes the program work. In 2014, the Gift It Forward study was funded by the HHMI to start a longitudinal study on why BSP works.

Since 1992, UC Berkeley students have been selected into the BSP through a holistic approach rooted in a noncognitive variable framework. In contrast to using standardized measures to select students, such as GPA and Scholastic Aptitude Test scores, BSP has used a list of noncognitive variables such as resilience, persistence, authenticity, willingness to seek and give help, and ability to restrategize and regroup in the face of failure. From this list, BSP leadership and staff created a “search image” for individuals who would benefit from and be a benefit to the BSP community. Students have a chance to discuss these qualities in the two-step application process, which includes an online application and an in-person interview with program staff and faculty. The BSP selection template is based upon previous evidence showing the power of using noncognitive variables to predict academic success (e.g., Sedlacek and Adams, 1992; Sternberg, 2010; Farruggia ).

The Gift It Forward study enrolled cohorts of BSP students starting in the Fall of 2014 until the Spring semester of 2017 (total cohorts = 6). All students were surveyed every 6 months starting from when they were accepted into the program and followed longitudinally for at least 2 years. A total of 395 scholars were enrolled during the course of the study, and all available data were used to examine our research questions. For the model examining the social influence agents at 6 months (model 1), 367 scholars had available data on at least one variable and were included in the analyses. Eighty-three percent of scholars were between 18 and 21 years old, 8% were over the age of 22, and age was unavailable for 9%. Sixty-eight percent of scholars were female, 25% were male, and 6% responded “other” or did not report their gender. Fifty-three percent of students were HU, and 65% were first-generation college students. For the model examining the social influence agents at 12 months (model 2), 291 scholars had available data on at least one variable and were included in the analyses. Eighty-two percent of scholars were between 18 and 21 years old, 10% were over the age of 22, and 9% did not report their age. Sixty-eight percent of scholars were female, 25% were male, and 7% responded “other” or did not report their gender. Fifty-three percent of scholars were HU, and 67% were first-generation college students. Thus, the demographic characteristics across the model 1 and model 2 samples were generally consistent.

MEASURES

Social Influence Agents: Measured at 6 and 12 Months into the Program

Social Connections.

Students’ social connections with the BSP community were assessed via eight items measuring the extent to which students connected with BSP’s faculty, staff, and peers. Specifically, students were given the following prompt: “Take a moment to think about the BSP faculty, staff, and other students who are participating whom you have met or know.” Participants were asked to rate “How much would you miss the BSP faculty and staff if you were not able to spend time or communicate with them?” from 0 (not miss them at all) to 10 (miss them a great deal), “How close are you (in personal and emotional terms) to the members of the BSP faculty and staff?” from 0 (not close at all) to 10 (very close), “How important are the BSP faculty members to you?” from 0 (not at all important) to 10 (very important), and “How do you think the BSP faculty and staff rate you as a student?” from 0 (not at all good) to 10 (very good). Students were then prompted to “Think about the other students who are participating in BSP,” and a set of four identical questions were asked about “other BSP students”; for example, “How close are you (in personal and emotional terms) to other BSP students?” rated from 0 (not close at all) to 10 (very close). All items were averaged to derive a mean score for social connections. The scale had good reliability (α at 6 and 12 months = 0.92).

Activity Enthusiasm.

To assess how scholars experienced program activities, we asked them to indicate the extent to which different BSP program components impacted their “enthusiasm for pursuing a science-related career” on a 1 (strongly decreased enthusiasm) to 7 (strongly increased enthusiasm) scale. The program components were: “advising,” “tutoring,” “interaction with other BSP members,” “September all student meeting,” “December End of the Year reception,” “IDS 96 course: Studying the Biological Sciences,” “individual meetings with advisors,” “using the BSP Student Room (2053 Valley Life Sciences Building),” “receiving the BSP email Newsletter,” and the “BSP website.” All of these program components were readily available and applicable to all BSP scholars. All items were averaged to derive a mean score for activity enthusiasm. The scale was reliable (α at 6 months = 0.91, α at 12 months = 0.92).

Social Influence Processes: Measured at 12 Months into the Program

Science Efficacy.

Science efficacy was measured by asking participants to indicate how confident they were in their ability to carry out six research-related tasks on a scale from 1 (not at all confident) to 5 (absolutely confident; Estrada ). Sample items included “Generate a research question to answer” and “Figure out what data/observations to collect and how to collect them.” All items were averaged to derive a mean science efficacy score. The scale was reliable (α = 0.91).

Science Identity.

Science identity was measured by asking participants the extent to which they agreed with five items using a 1 (strongly disagree) to 7 (strongly agree) scale (Estrada ). Sample items included “The daily work of a scientist is appealing to me” and “I have come to think of myself as a scientist.” All items were averaged to derive a mean science identity score. The scale was reliable (α = 0.89).

Science Community Values.

Science community values were measured by asking participants to indicate the extent to which the person described in each of four statements was like them (1 = not like me at all, 2 = not like me, 3 = a little like me, 4 = somewhat like me, 5 = like me, and 6 = very much like me; Estrada ). Sample items included “A person who thinks discussing new theories and ideas between scientists is important” and “A person who feels discovering something new in the sciences is thrilling.” All items were averaged to derive a mean score for science community values. The scale was reliable (α = 0.84).

Intentions: Measured at 18 Months into the Program

Science Career Intentions.

To assess scholar’s intentions to pursue a science-related career, we used a seven-item scale previously used in Estrada . Participants were asked to rate their level of intentions to pursue a science career and goals on a 0 (definitely will not) to 10 (definitely will) scale. Sample items included: “To what extent do you intend to pursue a science-related career?” and “How likely is it that you will pursue a career in which you publish academic papers in reviewed academic journals?” All items were averaged to derive a mean science career intentions score. The scale was reliable (α = 0.87).

Data Analytic Plan, Model Fit, and Statistical Assumptions

Using the lavaan package in R (Rosseel, 2012), we tested study hypotheses by using a structural equation model (SEM) with maximum-likelihood estimation to examine the degree to which our conceptual model fit the present data for the two different models (see Figure 1). Specifically, model 1 examined whether activity enthusiasm and social connections 6 months into the program predicted science efficacy, identity, and values 12 months into the program, and whether these three latter variables predicted science career intentions 18 months into the BSP. Model 2 consisted of the same variables, except that activity enthusiasm and social connections reported at 12 months into the program were used. Each model controlled for science efficacy, identity, values, and science career intentions that were reported at the beginning of the program (i.e., baseline assessments). All available data were used in our zero-order correlation and t test analyses. Therefore, degrees of freedom may vary with respect to these analyses.

Model fit was evaluated with the following fit indices: chi-square goodness-of-fit test (χ2), comparative fit index (CFI; i.e., an incremental index), standardized root-mean-square residual (SRMR; i.e., an absolute fit index), and root-mean-square error of approximation (RMSEA; i.e., a parsimony index). In terms of the chi-square goodness-of-fit test, a nonsignificant χ2 denoted good model fit (Barrett, 2007). CFI values above 0.95, SRMR values below 0.08, and RMSEA values below 0.06 all indicated good model fit (Hu and Bentler, 1999). Statistical significance of the model parameter estimates (e.g., regression slopes) were examined following acceptable model fit (Thompson, 2004). Some researchers have recently argued that a nonsignificant χ2 statistic and the model fit criteria suggested by Hu and Bentler (1999) have several limitations for assessing model fit (for a detailed discussion of both benefits and limitation, see Yuan ; Marcoulides and Yuan, 2017; Peugh and Feldon, 2020). Therefore, we additionally evaluated model fit using equivalence testing outlined by Marcoulides and Yuan (2017) and Peugh and Feldon (2020). Equivalence testing estimates a “T-size” CFI and RMSEA denoting the size of the misspecification for these two statistics. The size of the misspecification is then used to determine the degree of model fit that is categorized as excellent, close, fair, mediocre, or poor based upon obtained model statistics such as the χ2, degrees of freedom, and number of observed variables. Model fit categorized as fair or below suggests the model does not fit the data well (Marcoulides and Yuan, 2017).

First, however, we evaluated the statistical assumptions of our planned SEM. As displayed in Table 1, the response rates of the model variables varied across time, which led us to examine whether the missing data were missing completely at random (MCAR; Enders, 2010, 2011) using Little’s MCAR test (Little, 1988). We conducted separate tests for model 1 and model 2. Results indicated that the data were missing completely at random for model 1, χ2(81) = 91.99, p = 0.189, and model 2, χ2(71) = 80.97, p = 0.196. Therefore, we used maximum-likelihood estimation without adjustments for missing data in both models. Further, no outliers were detected for either model as denoted by leverage values, studentized deleted residuals, and Cook’s D (Judd ). Residual diagnostics revealed that the linearity, normality of residuals, and homoscedasticity assumptions were met.

TABLE 1.

Intercorrelations and descriptive statistics among the variables examined in the current study

	Social influence agents				Social influence processes			Intentions
	Activity enthusiasm (6)	Social connections (6)	Activity enthusiasm (12)	Social connections (12)	Science efficacy (12)	Science identity (12)	Science values (12)	Science intentions (18)
Correlation matrixa
Activity enthusiasm (6)	—
Social connections (6)	0.44***	—
Activity enthusiasm (12)	0.46***	0.47***	—
Social connections (12)	0.33***	0.64***	0.64***	—
Science efficacy (12)	0.06	0.11	0.23***	0.27***	—
Science identity (12)	0.17*	0.14*	0.31***	0.33***	0.51***	—
Science values (12)	0.16*	0.17*	0.28***	0.30***	0.45***	0.63***	—
Science intentions (18)	0.09	0.13	0.22**	0.25***	0.33***	0.48***	0.50***	—
Descriptive statistics
N	300	291	254	256	265	268	266	231
M	5.39	6.39	5.49	6.57	3.58	5.24	4.98	6.73
SD	0.97	1.89	0.96	1.92	0.77	1.20	0.85	1.93
Cronbach’s alpha	0.91	0.92	0.92	0.92	0.91	0.89	0.84	0.87

aNumbers in parentheses refer to the number of months students had been enrolled in the program.

* p < 0.05.

** p < 0.01.

*** p < 0.001.

RESULTS

Mean Differences and Zero-Order Correlations

Intercorrelations and descriptive statistics among the variables examined in the current study are displayed in Table 1. We first examined whether there were significant differences in reported activity enthusiasm and social connections between the 6 and 12 month assessments. Paired t tests indicated that activity enthusiasm reported at 6 (M = 5.39, SD = 0.97) and 12 months (M = 5.49, SD = 0.96) did not significantly differ, t(205) = 1.13, p = 0.261. Social connections also did not differ between the 6 (M = 6.39, SD = 1.89) and 12 month (M = 6.57, SD = 1.92) assessments.

We next examined the zero-order correlations of the variables comprising our conceptual models. As shown in Table 1, the two social influence agents (i.e., activity enthusiasm and social connections) were strongly positively correlated at both 6 months (r = 0.44) and 12 months into the program (r = 0.64). activity enthusiasm at both 6 and 12 months was positively associated with science identity (6 month r = 0.17, 12 month r = 0.31) and values (6 month r = 0.16, 12 month r = 0.28). However, only activity enthusiasm at 12 months was associated with science efficacy (12 month r = 0.23). The same pattern emerged for social connections. Specifically, social connections at both 6 and 12 months was positively associated with science identity (6 month r = 0.14, 12 month r = 0.33) and values (6 month r = 0.17, 12 month r = 0.30), but only social connections at 12 months was positively associated with science efficacy (12 month r = 0.27). In terms of zero-order correlation coefficients, the associations between the social influence agents and the social influence processes were always larger for the social influence agents reported at 12 versus 6 months. As expected, zero-order correlations indicated that science efficacy, identity, and values at 12 months were all associated with science career intentions at 18 months (r values ranged from 0.33 to 0.50).

SEM Model Fit

Model fit indices indicated that our conceptual model had good model fit despite significant χ2 values for both model 1 and model 2; model 1: χ2(14) = 33.88, p = 0.002, CFI = 0.98, SRMR = 0.06, RMSEA = 0.06; model 2: χ2(14) = 29.25, p = 0.010; CFI = 0.98, SRMR = 0.05, RMSEA = 0.06. In terms equivalence testing for model 1, the T-size CFI was 0.93 and the T-size RMSEA was 0.09. This denoted fair model fit for both the CFI and RMSEA. For model 2 equivalence testing, the T-size CFI was 0.95 and the T-size RMSEA was 0.09. The T-size CFI and RMSEA denoted fair model fit for both.

Model fit was sufficient based on traditional model fit criteria, acknowledging that the equivalence testing denoted that both of our models did not fit the data well. Given the common acceptance of traditional fit criteria (Deng ), the known strengths and limitations of the equivalence testing approach (see Peugh and Feldon, 2020), and noting that the equivalence test results showed strong fit when baseline variables were not controlled, we moved forward with evaluating the parameter estimates of our models and concluded that our models were plausible to address the stated research questions.

Q1: Are Social Connections within the BSP and/or Activity Enthusiasm about BSP Elements Associated with Greater Science Efficacy, Identity, Values, and Integration into the Science Community?

The regression coefficients of our SEM models are displayed in Figure 2 (model 1) and Figure 3 (model 2). Our models controlled for science efficacy, identity, values, and science career intentions measured at baseline (i.e., at the beginning of the program). The results regarding question 1 were largely a function of how long the scholars had been enrolled in the program, particularly with respect to social connections. Specifically, activity enthusiasm at 6 months was not associated with either science efficacy (β = 0.06, p = 0.333), identity (β = 0.07, p = 0.287), or values (β = 0.09, p = 0.190) 12 months into the program. However, while activity enthusiasm at 12 months was positively associated with science values (β = 0.15, p = 0.037), it was not associated with either science efficacy (β = 0.05, p = 0.525) or identity (β = 0.10, p = 0.178). As with activity enthusiasm at 6 months, social connections at 6 months was not associated with either science efficacy (β = 0.05, p = 0.522), identity (β = 0.07, p = 0.372), or values (β = 0.09, p = 0.209) 12 months into the program. In contrast, social connections at 12 months was positively associated science efficacy (β = 0.21, p = 0.004), identity (β = 0.20, p = 0.004), and values (β = 0.17, p = 0.019) 12 months into the program. This denotes that activity enthusiasm was largely not uniquely associated with (i.e., above and beyond social connections and baseline levels) science efficacy, identity, and values. However, social connections at 12 months was uniquely associated with (i.e., above and beyond activity enthusiasm and baseline levels) science efficacy, identity, and values but was associated with neither of these when measured at 6 months.

FIGURE 2.

Conceptual model parameter estimates for model 1: activity enthusiasm and social connections 6 months into the BSP. Dashed lines indicate nonsignificant regression paths. All coefficients are standardized. Months refer to the number of months students had been enrolled in the BSP. We controlled for baseline levels of science efficacy, identity, values, and career intentions (i.e., these variables reported at the start of the program). **p < 0.01; ***p < 0.001.

FIGURE 3.

Conceptual model parameter estimates for model 2: activity enthusiasm and social connections 12 months into the BSP. Dashed lines indicate nonsignificant regression paths. All coefficients are standardized. Months refer to the number of months students had been enrolled in the BSP. We controlled for baseline levels of science efficacy, identity, values, and career intentions (i.e., these variables reported at the start of the program). *p < 0.05; **p < 0.01; ***p < 0.001.

In terms of the downstream effects of the social influence processes on science career intentions measured 18 months into the program, science identity (model 1: β = 0.22, p = 0.004; model 2: β = 0.22, p = 0.004) and values (model 1: β = 0.22, p = 0.004; model 2: β = 0.22, p = 0.004) were uniquely and positively associated with science career intentions 18 months into the program, but science efficacy was not uniquely associated with science career intentions (model 1: β = 0.07, p = 0.298; model 2: β = 0.07, p = 0.303). Each of these associations were obtained while controlling for science career intentions measured at the time of entry into the program.

Q2: Do Science Efficacy, Identity, and/or Values Mediate the Associations between Social Connections and Activity Enthusiasm and Integration into the Science Community?

Mediation analyses were conducted to test the indirect effect of the social influence agents on science career intentions through the social influence processes (all mediation and indirect effect estimates are displayed in Table 2). Bootstrapping with 10,000 iterations was used to estimate bias-corrected 95% confidence intervals (BC CI95%) around the indirect effects (Hayes, 2009) within our SEM models. In other words, all indirect effects were examined simultaneously. We also controlled for science efficacy, identity, values, and science career intentions measured at baseline. A 95% confidence interval that does not include 0 denotes mediation has occurred (Hayes, 2009).

TABLE 2.

Results of the social influence processes mediating the effects of the social influence agents on students’ science career intentionsa

	Outcome (Y): science career intentions
	a		b		a*b
Social influence agent (X)Social influence process (M)	b	β	b	β	BC CI95%[LL, UL]
Model 1
Activity enthusiasm
Science efficacy	0.05	0.06	0.17	0.07	[−0.01, 0.06]
Science identity	0.08	0.07	0.36	0.22*	[−0.02, 0.15]
Science community values	0.07	0.09	0.50	0.22*	[−0.01, 0.14]
Social connections
Science efficacy	0.02	0.05	0.17	0.07	[−0.00, 0.03]
Science identity	0.04	0.07	0.36	0.22*	[−0.01, 0.07]
Science community values	0.04	0.09	0.50	0.22*	[−0.01, 0.07]
Model 2
Activity enthusiasm
Science efficacy	0.04	0.05	0.17	0.07	[−0.01, 0.06]
Science identity	0.12	0.10	0.36*	0.22*	[−0.01, 0.16]
Science community values	0.13	0.15	0.50*	0.22*	[0.004, 0.21]
Social connections
Science efficacy	0.08*	0.21*	0.17	0.07	[−0.09, 0.06]
Science identity	0.12**	0.20**	0.36*	0.22*	[0.008, 0.11]
Science community values	0.07*	0.17*	0.50*	0.22*	[0.003, 0.11]

aModel 1 examined activity enthusiasm and social connections reported 6 months into the program, and model 2 examined these variables reported 12 months into the program. We controlled for baseline levels of science efficacy, identity, values, and career intentions (i.e., these variables reported at the start of the program), and all indirect effect within each model were tested simultaneously. Estimates were with 10,000 bootstrap replications; b = unstandardized estimate; β = standardized estimate; BC CI95% = bootstrapped bias-corrected 95% confidence intervals for the indirect effect; LL = lower limit of the confidence interval; UL = upper limit of the confidence interval.

* p < 0.05.

** p < 0.01.

Results indicated that the effect of activity enthusiasm at 6 months on science career intentions 18 months into the program was not mediated by science efficacy, identity, or values. However, while the effect of activity enthusiasm at 12 months on science career intentions was mediated by science values (BC CI95% [0.004, 0.21]), neither science efficacy nor values mediated this effect. The effect of social connections at 6 months on science career intentions was also not mediated by science efficacy, identity, or values. However, the effect of social connections at 12 months on science career intentions was mediated by science identity (BC CI95% [0.008, 0.11]) and values (BC CI95% [0.003, 0.11]) but not science efficacy.

DISCUSSION

There are many science training programs that share key features such as workshops, advising, research stipends, and other forms of instrumental support. However, it is well known that not all programs are regarded to be as successful as BSP. The present results may shine light on one key factor that differentiates these programs. The results here show that, after a year, program activities that are related to greater integration into students’ disciplinary programs, predict greater science career intentions 18 months into the program. But the results also show that an additional unique predictor (meaning that the effect is in addition to the impact of the program activities) that may have even larger implications for students’ integration and science career intentions is their sense of connection to program faculty, staff, and peers. In short, the effectiveness of science training programs may rest on both program activities and the quality of the relationships formed among people—faculty, staff, and peers—in the program. We arrived at these conclusions while aiming to answer three key research questions.

We must first recognize that the findings show that time mattered. For this reason, regarding our first research question—”Are social connections within the BSP and/or activity enthusiasm about BSP elements associated with greater science efficacy, identity, values, and integration into the science community?”—the answer is a clear “It depends.” Specifically, activity enthusiasm at 6 months was not uniquely associated with (above and beyond social connections) science efficacy, identity, or values while controlling for their levels reported at the start of the program. Activity enthusiasm at 12 months was uniquely and positively associated with science values, but it was not associated with science efficacy or identity. While social connections at 6 months was not uniquely associated with (above and beyond activity enthusiasm) science efficacy, identity, or values while controlling for their levels reported at the start of the program, social connections at 12 months was uniquely and positively associated with science efficacy, identity, and values. Our results suggests that social connections forged within the BSP program, particularly after being enrolled in the program for 1 year, has very important implications in terms of students’ integration into their disciplinary programs.

Our results also showed that activity enthusiasm and social connections were strongly associated at both 6 months (r = 0.44) and 12 months (r = 0.64), suggesting that these variables, while distinct, move together. That we still observed several unique relations between these variables and science efficacy, identity, and values despite their large covariation is a testament to the importance these variables can have in students’ integration. The stark differences between the results observed for model 1 and model 2 are somewhat surprising, because there were no significant differences between reported activity enthusiasm and social connections at 6 months and 12 months. Yet, despite equivalent mean levels of these variables at 6 and 12 months, their associations with the social influence processes and science career intentions were dramatically different, with time strengthening these relationships.

Regarding our second research question—”Do science efficacy, identity, and/or values mediate the associations between social connections and activity enthusiasm and integration into the science community?”—we found that science efficacy, identity, and values did not mediate the association between activity enthusiasm at 6 months and science career intentions 18 months into the program (see Table 2). This is not surprising, because activity enthusiasm at 6 months was not associated with either of these potential mediators (see model 1). Although the effect of activity enthusiasm at 12 months on science career intentions was mediated by science values, it was not mediated by science efficacy or identity. Thus, although the effect was small and replication of this effect is needed, enthusiasm about program activities within the BSP program 12 months into the program may influence intentions through its promotion of endorsing values consistent with the science community. The effect of social connections at 6 months on science career intentions was not mediated by science efficacy, identity, or values. As with activity enthusiasm at 6 months, this was not surprising, because it was also not associated with either of these potential mediators. However, the effect of social connections at 12 months on science career intentions was mediated by both science identity and values, but not science efficacy. Despite the small effects, this suggests that social connections forged within the BSP program 12 months into the program may lead to greater intentions via the promotion of greater science identity and endorsement values consistent with the science community. Importantly, all these models controlled for students’ levels of science efficacy, identity, values, and career intentions at the start of the program and considered the indirect effects of activity enthusiasm and social connections on science career intentions simultaneously.

Overall, the results suggest that time in a program matters in order to develop meaningful enthusiasm for the program components and also deepening social connections with others in the program. These results might also suggest that, if programs similar in nature to the BSP provide participants with only a short time in the program (i.e., less than 6 months), it is possible that students’ science efficacy, identity, and community values might not be impacted in ways that lead to greater persistence and intentions. Sustained connections, in the context of students being engaged in full-time academic commitments, may be important.

Limitations

The current study was not without important limitations. First, while the amount of time enrolled in the BSP appeared to play an important role in the relationships between the social influence agents and processes, in model 2, social influence agents and integration measures were contemporaneous. Modeling mediation with contemporaneous variables can lead to biased estimates and does not always represent true longitudinal effects (Maxwell and Cole, 2007; Maxwell ). In other words, it is not clear whether the BSP social influence agents at 12 months predict downstream science efficacy, identity, and values or if these work in concert with each other. Recent results examining these variables suggest that there may be reciprocal, contemporaneous impacts occurring (Hernandez ). The results therefore may tell us more about how effective programs simultaneously are integrating students in ways that contribute toward longer-term intentions to persist.  To gain insight into whether activity enthusiasm and social connections have downstream effects on science efficacy, identity, and values or whether our observed effects were due to the variables being contemporaneous, we examined the associations between activity enthusiasm and Social Connection at 12 months and the scholars’ reported science efficacy, identity, and values at 18 months. Every association was positive and significant (r values ranging from 0.16 to 0.31), which suggests that the contemporaneous nature of these variables might not be the sole explanation for our observed findings.

A second limitation to note is that our results pertain to a singular undergraduate program and may not generalize to other programs. This study examined a cocurricular science training program that enrolls students who are engaged in full-time academic work. They typically spend 2–8 hours a week in BSP activities and do not live together. And while they may attend one or more classes together and share a common social study space, scholars do not have mandatory attendance for most program activities. This is very different from summer intensive science training programs in which students spend time 24/7 with other people in their program and often experience intense bonding during that time. For programs that share features in common with BSP, related to developing student talent in science (as opposed to selecting for already high achieving students) across a year of programs, the results may be more confidently generalized. Additional research will be needed to see whether both enthusiasm for program activities and social connection with program faculty, staff, and peers independently predict student integration and persistence for students in a variety of training programs.

A final limitation is that the findings do not provide a clear interpretation of the impact of time in programs on student integration and persistence. One explanation of the result is that students take time, at least a year, to recognize the benefits of program activities and the social relationships they foster, at which time these become highly related to their sense of integration into their science discipline communities. Another explanation is that there is a maturation period that occurs when one enters a science training program, and 6 months simply is not enough time to experience the programs’ full activity and social connection impacts on social integration into their areas of study. Relationship qualities such as trust and appreciation may take time to grow and deepen. The first explanation suggests the outcome is a matter of student perceptions, whereas the second suggests students benefit from a full year or more of participation in a science training program. Based on more than 20 years of experience, the second explanation is most true to the experience of staff and faculty working with BSP students. And students reference their BSP relationships feeling like “family,” which also suggests that the latter explanation would be worth testing in future studies.

CONCLUSION

There is both qualitative and quantitative research demonstrating that science training programs make a positive difference for students. Previous research on the BSP has shown higher retention and graduation for the students who participate, relative to similar students at the same institution (Matsui, 2018; Estrada ). This research advances our understanding that both enthusiasm for program activities and the strength of the social connections that students forge with others in the program have important independent contributions to social integration into their science communities and persistence in their fields. These findings may also help to explain why STEM diversity programs with the same list of activities (such as mentoring, tutoring, research experiences, etc.) do not share the same successful outcomes. The relationship-building or relational aspects integral to their implementation may be a critical ingredient in the “secret sauce” of successful programs and a key component that distinguishes BSP, Meyerhoff, and other successful programs from the rest.

The results strongly suggest that directors of effective science training programs may benefit from creating programmatic activities that meaningfully (and intentionally) help students increase their science efficacy and identity and internalize the value of science. However, to do this alone, without also providing opportunities for building strong and healthy social connections between students and faculty, staff, and their peers, may deprive the program of a key additional contributor to students’ integration into their professional careers. These results, showing the positive impact of social connection, raises the question of how negative social experiences in the context of science training programs impact students who otherwise are being exposed to high-quality activities. While the results of this study cannot begin to answer this question, the results strongly suggest that the actions of educators matter. Further, the results are consistent with research that shows that making the choice to be kind and connect to learners is not only being “nice,” but an integral part of high-quality and healthy learning environments (Estrada ).