Expanding STEM opportunities through inclusive STEM-focused high schools.

Inclusive STEM high schools (ISHSs) (where STEM is science, technology, engineering, and mathematics) admit students on the basis of interest rather than competitive examination. This study examines the central assumption behind these schools-that they provide students from subgroups underrepresented in STEM with experiences that equip them academically and attitudinally to enter and stay in the STEM pipeline. Hierarchical modeling was applied to data from student surveys and state longitudinal data records for 5113 students graduating from 39 ISHSs and 22 comprehensive high schools in North Carolina and Texas. Compared to peers from the same demographic group with similar Grade 8 achievement levels, underrepresented minority and female ISHS students in both states were more likely to undertake advanced STEM coursework. Hispanics in Texas and females in both states expressed more STEM career interest in Grade 12 if they attended an ISHS. Positive relationships between ISHS attendance and grade point average were found in the total sample and each subgroup in North Carolina. Positive ISHS advantages in terms of test scores for the total student sample were found for science in both states and for mathematics in Texas. For the various student subgroups, test score differences favored the ISHS samples but attained statistical significance only for African Americans' science achievement scores in the Texas study.

INTRODUCTION

The stark contrast between the demographic composition of the U.S. population as a whole and that of science, technology, engineering, and mathematics (STEM) college majors and professionals has been a concern in economic and educational policy circles for some time (National Academies, 2005, 2011; National Science Board, 2014; President's Council of Advisors on Science and Technology [PCAST], 2010). Although more than one in nine U.S. resident adults identify as African American, less than one of 20 STEM professionals comes from this population subgroup (National Science Board, 2016). Similarly, only 6% of the science and engineering workforce in 2013 was Hispanic, the fastest growing segment of the U.S. population (National Science Board, 2016). Further, although women comprise roughly half of the college‐educated workforce and are well represented in biological and related sciences, they remain underrepresented in other STEM fields, particularly in engineering where they constitute only 15% of the workforce (National Science Board, 2016). All of these disparities raise issues not only for national economic competitiveness but also for those individuals whose opportunities are limited (National Academies, 2005; PCAST, 2010). STEM occupations are among those growing fastest in the U.S. economy (National Science Board, 2014). People in STEM jobs earn more than those in other jobs, and those with STEM bachelor's degrees have higher earnings than individuals with degrees in other fields, even when they enter non‐STEM professions (Russell & Atwater, 2005).

Inclusive STEM high schools (ISHSs) have been promoted as a strategy for increasing the representativeness of students entering the “STEM pipeline” by increasing the diversity of the student population that undertakes and completes STEM college majors. Our research examines this central assumption behind ISHSs, with particular emphasis on whether such schools provide students from population subgroups underrepresented in STEM (namely, females, Hispanics, and African Americans) with experiences that equip them academically and attitudinally to enter and stay in the STEM pipeline. We contrast outcomes for students in ISHSs with those of similar students attending comprehensive high schools without a STEM focus.

We define an inclusive STEM high school as a secondary school or self‐contained school‐within‐a‐school that (a) enrolls students on the basis of interest rather than aptitude or prior achievement, (b) provides students with more intensive STEM preparation than within conventional high schools, and (c) expresses the goal of preparing all students to succeed in STEM. Note that this definition excludes schools with intensive STEM programs in which some students participate but others do not. This definition includes schools focused on a particular STEM field requiring a college degree (e.g., Health Sciences) as well as those schools preparing students for STEM majors in general.

HISTORICAL CONTEXT

Early in the 21st century, a number of private foundations articulated a vision for secondary schools offering a rigorous curriculum and extensive supports to develop STEM interest and readiness for college‐level STEM among students from underrepresented groups choosing to attend them (Bill & Melinda Gates Foundation, 2005a; Carnegie Corporation, 2009; Means, Confrey, House, & Bhanot, 2008). State‐level initiatives to create inclusive STEM‐focused high schools have emerged in Texas, Ohio, North Carolina, Arkansas, Tennessee, Arizona, and Washington. More recently, President Obama's (2015) State of the Union speech called for a national effort to create more “next‐generation high schools” that incorporate workplace learning, closer ties to higher education institutions, and expanded STEM opportunities for groups underrepresented in STEM fields. The creation of STEM high schools with an inclusive mission was a dramatic departure from prior thinking about how to create a pipeline of students entering STEM fields. STEM‐focused schools created prior to 2000 were predominantly exam‐based schools, using competitive tests to select students who could demonstrate high levels of mathematics and science achievement by Grade 8. The oldest of these schools date back to as early as the 1930s, but they became more prominent during the post‐Sputnik era, when the United States was concerned about its ability to produce mathematics, engineering, and science elites that could compete with those of the Soviet Union (Hanford, 1997). Selective STEM schools such as Stuyvesant High School, the Illinois School of Mathematics and Science, the Bronx High School of Science, the North Carolina School of Science and Mathematics, and Thomas Jefferson High School for Science and Technology strive to attract the most talented young people into their STEM‐focused programs. These schools proudly count doctoral‐level STEM professionals and even Nobel laureates among their alumni.

The other major impetus for establishing STEM‐focused schools during the 20th century was school desegregation, following the Brown v. Board of Education Supreme Court decision. Many large districts sought to retain white students and improve their district's racial balance by creating magnet schools or other programs with an attractive set of additional education resources (Metz, 2003). STEM‐focused schools were one popular type of magnet, often articulating a mission of serving “gifted students” or offering special facilities in an effort to keep white students within a public school system serving increasing proportions of students of color. Selective STEM schools often seek to recruit qualified students from subgroups underrepresented in STEM, but with admissions based on examination scores, African American and Hispanic students are typically underrepresented (Kaser, 2006).

In contrast, the concept of inclusive STEM high schools took hold a half century later when policymakers turned their attention to the need not just for a broader pool from which elite STEM professionals could emerge but also for filling STEM‐related jobs requiring bachelor's or associates degrees and for insuring that all citizens are science literate (National Academies, 2011). The ISHS model has been articulated as a strategy for broadening participation in STEM and STEM‐related professions by recruiting students from underrepresented minorities and admitting students from their pools of applicants on the basis of interest, using lotteries rather than test scores for selection if the school is oversubscribed (Means et al., 2008). The goal of an ISHS is to develop STEM talent, rather than to select for it.

Although they all share this mission of inclusion and college preparation, ISHSs vary widely from each other in terms of many of their design features (Eisenhart et al., 2015). Some schools emphasize a particular career area, such as engineering or medicine; others seek to provide a well‐rounded STEM education, equipping their students for any STEM major. Some emphasize instruction integrating the various STEM disciplines, while others organize STEM instruction around traditional academic disciplines such as algebra and biology. In some cases, ISHSs depend on partnerships with colleges or with private industry to support significant portions of instruction. There are also ISHSs that employ a career technical education model that includes preparation for entering a baccalaureate program, and those that place great emphasis on project‐based learning (Lynch, Peters‐Burton, & Ford, 2014).

Although the term “STEM” has become familiar in education circles, its definition remains a subject for debate (Brown, Brown, Reardon, & Merrill, 2011; Committee on Integrated STEM Education, 2014; Gerlach, 2012; Kelley, 2010; Tsupros, Kohler, & Hallinen, 2009). Some scholars prefer to reserve the term for curricular approaches that integrate the science, technology, engineering, and mathematics disciplines (Morrison, 2006; Tsupros et al., 2009); others restrict “STEM” to student‐centered approaches to instruction (North Carolina State Board of Education, 2014). For the purposes of our research, we adopt a descriptive rather than a prescriptive stance toward STEM, and use the term as a category that includes biological, physical, environmental, and medical sciences as well as engineering, information technology, and mathematics (e.g., Aschbacher, Li, & Roth, 2010).

ANTECEDENTS OF THE STEM PARTICIPATION GAP

Similar proportions of African American, Hispanic, and White students express interest in STEM careers upon college entry (Herrera, & Hurtado, 2011; National Science Board, 2016), but smaller percentages of the first two of these groups eventually complete majors in these fields (National Science Board, 2016). Research exploring the reasons behind the smaller percentage of African Americans, Hispanics, and women completing STEM degree programs point not only to college experiences but also to differences in preparation and attitudes, with the roots of these beginning in high school experiences, if not before. African American students comprise 16% of the high school population but only 8% of these students are enrolled in calculus courses; for Hispanics, the corresponding percentages are 21% and 12% (U.S. Department of Education, Office for Civil Rights, 2014). Such coursetaking gaps are salient because advanced STEM courses not only make students more competitive in their college applications but also prepare them for success in college STEM courses (Bottia, Stearns, Mickelson, Moller, & Parker, 2015).

Some of the discrepancy in math and science course enrollment in high school among different student subgroups is explained by the lack of advanced course offerings in high schools with large concentrations of students from underrepresented ethnic groups. Almost one in five African American high school students attends a school that does not offer any advanced placement (AP) courses; one‐third of the high schools with the largest concentrations of African American and Hispanic students do not offer chemistry (U.S. Department of Education, Office for Civil Rights, 2014). At the same time, it is also true that students from underrepresented groups who attend high schools that do offer advanced STEM courses are still less likely than white and Asian males to have taken those courses (Laird, Alt, & Wu, 2009). A study of placement into the advanced math class in a nationally representative sample of racially diverse high schools by Muller, Riegle‐Crumb, Schiller, Wilkinson, and Frank (2010), for example, found that African American and Hispanic students were underrepresented relative to white and Asian students even after controlling for gender, parents’ education, and score on an intelligence test. Qualitative studies describe the subtle and not‐so‐subtle pressures that discourage underrepresented minorities and girls from taking courses and achieving in subject areas perceived as the domain of Asian and white males (Childress, Doyle, & Thomas, 2009; Margolis, Estrella, Goode, Holme, and Nao, 2008; Schofield, 1995).

When aiming to understand the extent to which ISHSs prepare underrepresented students for entering and persisting in the STEM pipeline, it is important then to consider both the learning opportunities present in ISHSs and the psychological and sociological factors present that may have important consequences for students’ academic and attitudinal outcomes.

A social cognitive perspective on students’ academic and career pursuits in STEM

To explore the dynamic factors at play as students pursue STEM interests and career aspirations and the ways these efforts are supported or not by ISHSs, we draw on analytic tools from Social Cognitive Career Theory (SCCT). SCCT, as described by Lent, Brown, and Hackett (1994), provides a framework for understanding persistence in pursuit of education and career goals, expanded from Bandura's social cognitive learning theory (Bandura, 1986). SCCT considers career‐related choices as the outcome of dynamic relationships among interests, goals, and expectations in the context of environmental supports and barriers. From this perspective, efficacy beliefs (i.e., the belief that I can succeed in future STEM endeavors) and outcome expectations (i.e., if I pursue STEM, what will happen?) play an important role in influencing an individual's interests and career goals. The experiences students have taking a particular course (e.g., calculus) or being placed into higher or lower level STEM course tracks have significant consequence for students’ beliefs about their ability to pursue STEM fields as a career and the kinds of opportunities this pursuit would garner (Byars‐Winston, Estrada, Howard, Davis, & Zalapa, 2010; Garg, Kauppi, Urajnik, & Lewko, 2010). Furthermore, as students’ beliefs about their capabilities and possibilities emerge, they develop STEM identities: a sense of who they are and who they would like to become, within school and STEM (Eccles, 2009).

SCCT aims to account for the role of gender and race in shaping career development, with particular attention given to the types of opportunity structures and support systems available (Lent et al., 1994). Specifically, the theory suggests that differences in interests and career choice might be mitigated through greater learning opportunities and social supports within schools. Layering SCCT with prior literature, then, can help explain some of the differences in STEM persistence across population subgroups (Bottia et al., 2015). For example, work by Steele and others on stereotype threat (Borman, & Pine, 2016; Steele, 1997, 2010) indicates that the performance of subgroups underrepresented in a particular academic specialty is undermined by their awareness that others expect them to do poorly. Female, African American, and Hispanic students may experience stereotype threat in their STEM classes, which can impair performance and consequently lower expectations of success in future STEM coursework. In contrast, members of overrepresented groups (white and Asian males) appear to have a sense of their ability to succeed in STEM that is more impervious to setbacks. Empirical research on the differential impact of STEM‐related education experiences on under‐ and overrepresented subgroups are consistent with this theoretical perspective. Male students express more math self‐efficacy than do female students with the same mathematics achievement levels (Eccles, 2009; Watt, 2006), for example. Similarly, Nora and Ramirez (2006) found that Hispanic students are more likely than non‐Hispanic white students to be discouraged by receiving a lower than expected grade. Looking longitudinally, Wang's (2013) analysis found a stronger relationship between early math achievement and math self‐efficacy for underrepresented minorities than it did for white and Asian students.

Thus, both SCCT and prior research provide grounds for hypothesizing about ways in which ISHS structure and support systems might scaffold underrepresented students’ persistence in STEM. As used here, the term “persistence” refers to continued participation in the educational activities preparing an individual for entry into a STEM career. It should not be confused with individual perseverance, which refers to continued engagement in an activity in the face of delays or difficulties, or with retention, which refers to remaining in a particular institution or program. A number of variables under a high school's control appear to be related to a student's STEM persistence—that is, taking the courses needed to enter a STEM major in college and maintaining STEM interest and self‐efficacy.

Learning opportunities, supports, and inclusive STEM high schools

Research into the factors related to students’ persistence in STEM suggests that inclusive STEM high schools might be beneficial for student groups underrepresented in STEM. As described in Means, Wang, Young, Lynch, and Peters (2016), the theory of action for ISHSs incorporates multiple features fostering student experiences known to predict entry into a STEM major in college. Ample research demonstrates that the most powerful predictor is the level of math and science courses taken in high school (see Adelman, 2006; Astin & Astin, 1993; Chen & Weko, 2009; Crisp, Nora, & Taggart, 2009; Mendez, Buskirk, Lohr, & Haag, 2008; Smyth & McArdle, 2004; Tyson, Lee, Borman, & Hanson, 2007; Wang, 2013). By design, schools can offer more or fewer courses, choose whether or not to track students by prior achievement level, and prescribe a more intensive course of study rather than making it optional.

Provision of STEM research opportunities and project‐based instructional approaches appear to increase students’ STEM interest, with the latter being particularly important for girls and underrepresented minorities (Boaler, 1998; Mergendoller, Maxwell, & Bellisimo, 2006; Ross & Hoagaboam‐Gray, 1998). Analyses of student survey responses from inclusive STEM high schools suggest that advanced coursetaking and higher achievement are associated with schools providing: student‐centered, reform‐oriented instruction; opportunities to learn advanced skills in math and science classes; integrating other STEM subjects into math instruction; and real‐world STEM experiences (Means et al., 2016).

Aschbacher et al. (2010) conducted a longitudinal, qualitative study of a diverse set of students from six public high schools who expressed strong interest in a science, engineering, or medicine career in Grade 10. Four factors distinguished between the 55% of students still interested in STEM careers at the end of Grade 12 versus the 45% who lost interest: participation in extracurricular science activities, family priorities with respect to college, family priorities with respect to STEM careers, and messages from school staff about their capabilities in STEM. Students who lost interest in STEM careers described getting the message that science is “hard” and “not for everyone” as well as experiencing poor, uninspiring teaching, often provided by a series of substitute teachers.

Nasir and Shah (2011) found that male African American students have internalized an ethnic hierarchy of mathematics achievement, with Asians at the top and African American and Latino students at the bottom. The students interviewed by these researchers were very aware of others’ expectations with respect to their performance and either embraced the narrative of low expectations for students like themselves or developed a counter‐narrative of themselves as mathematically capable. Nasir and Shah conclude that such narratives shape the ways in which students identify with mathematics and engage in math classroom activities. Such studies fit well with theoretical frameworks positing a dynamic interaction between students’ expectations of success or failure at a task and the subjective value they attach to the task as influences on the decision to engage in that task and the amount of effort they expend on it (Eccles, 2009).

Within ISHSs, the expectation is that all students in the school participate in the intensive STEM program. This means that students from underrepresented groups are more likely to experience advanced STEM coursework within these schools. The implicit and explicit expectation with such an approach is that all students are capable of completing a rigorous STEM curriculum. Such experiences may allow students from underrepresented subgroups to avoid negative messages about their STEM abilities often associated with tracking practices. Additionally, ISHSs bring together diverse student bodies with a common interest in STEM. One of the cues students use as to whether they “belong” in advanced STEM classes is simply the proportion of students in these classes of their same race, ethnicity, or gender (Larnell, 2013; Steele, 2010). Because of ISHSs’ educational philosophy, admissions policies, and organizational design, their advanced math and science courses include a sizable proportion of underrepresented minority students. Moreover, ISHSs tend to provide opportunities (and, at times, requirements) for out‐of‐school engagement in STEM and practice in real‐world contexts (e.g., medical internships), increasing students’ likelihood of seeing themselves and being recognized as “STEM people” (Bell, Lewenstein, Shouse, & Feder, 2009; Carlone & Johnson, 2007).

Investigating within states with inclusive STEM high schools

We report on two related studies conducted in North Carolina and Texas—two states that have made substantial investments in establishing ISHSs. In both studies, we addressed two basic research questions:

To what extent do STEM interests, activities, achievement, and expectations among 12th graders attending inclusive STEM high schools differ from those of similar students attending regular comprehensive high schools?

To what extent do STEM interests, activities, achievement, and expectations among 12th graders from demographic groups underrepresented in STEM fields differ between those attending inclusive STEM high schools and those attending regular comprehensive high schools?

In each state, we first identified ISHSs and non‐STEM comparison high schools serving students who were similar in terms of academic achievement prior to high school entry. We then employed HLM modeling to estimate the strength of the relationship between attending an ISHS and having positive high school attitudinal and achievement outcomes as measured by student survey scales and achievement test scores in state data systems. Both North Carolina and Texas maintain strong student longitudinal data systems that permit tracking individual students across grades. We availed ourselves of these data systems to obtain both demographic data and information on students’ achievement levels prior to high school entry. Below we describe two parallel studies, one conducted in each state, and then discuss findings of both studies, highlighting similarities and differences.

STUDY 1: NORTH CAROLINA

Study 1 investigated the relationships between attending an inclusive North Carolina STEM high school and the attitudinal and achievement variables that typically predict entry into a STEM college major. Several education initiatives in North Carolina paved the way for establishing inclusive STEM high schools. Starting in 2007 the NC New Schools Project, a joint public–private effort between the state and the Bill & Melinda Gates Foundation, opened a network of innovative new or redesigned high schools including 10 small schools with a STEM theme. The New Schools Project emphasized project‐based learning and early completion of college work as a strategy for easing the transition between high school and college for first‐generation college goers. In 2010, NC New Schools expanded from pilot‐site schools to a broader STEM initiative increasing the number of STEM‐themed schools to nearly 40. The state STEM initiative further evolved with federal Race to the Top funding, which supported converting existing comprehensive high schools into STEM‐focused schools. North Carolina's 2011 STEM Education Strategic Plan identified STEM schools and established a designation process for exemplary STEM schools.

METHOD

Instruments and data sources

Grade 12 student survey

The Grade 12 Student Survey was designed to collect data on constructs highlighted in SCCT  (i.e., science and math self‐efficacy, interests, academic expectations, and identity) and on variables shown to predict entry into STEM college majors in prior empirical research. Survey items and scales addressed students’ high school experiences in their STEM courses; extracurricular and leisure‐time activities related to STEM; overall academic and STEM orientation; academic and personal supports received through their high school; plans for the year following graduation; and interest in STEM majors and careers. Sources of items and scales for the Grade 12 Student Survey included the National Center for Education Statistics’ High School Longitudinal Study, the Consortium for Chicago School Research's Biennial Chicago Public School Student Survey, and surveys used in SRI's Program Evaluation of the Innovative Technology Experiences for Students and Teachers Program and its Evaluation of the Texas High School Project (THSP). Survey scales from these instruments have demonstrated predictive validity with respect to variables such as science self‐efficacy (Bean, Gnadt, Maupin, White, & Andersen, 2016) and high school graduation rates (Allensworth, Healey, Gwynne, & Crespin, 2016).

The student survey was designed to be completed within 30 minutes. Factual questions about topics such as courses taken were formatted as menus of options with instructions to “mark all that apply.” Attitudinal constructs were measured through scales of 4 or 5 items using a Likert scale format. The items comprising the scales used in this study are shown in Appendix A. The reliability (Cronbach's alpha) of the Grade 12 Student Survey item scales ranged from .71 to .92.

Participating schools provided a student roster so response rates could be computed. Each school chose whether it wanted students to use an electronic or a paper‐based version of the survey and the setting for survey administration, in keeping with their district's policies.

State administrative data

We relied upon state longitudinal student data from North Carolina Education Research Data Center (NCERDC) for students in our survey sample (12th graders in 2012–2013) to obtain the following: student demographic information, eighth‐grade achievement (in reading, mathematics, and science), whether the student took Algebra before ninth grade, whether the student took the ACT, and high school weighted GPA and ACT test scores. We linked survey data with the administrative data using the keys and code book that NCERDC provided.

Sample and recruiting

At the time our project began recruiting schools in North Carolina, the state did not maintain a complete list of ISHSs, so our research team had to identify the population of relevant school‐level entities. Out of approximately 600 public high schools in North Carolina, we identified 100 as potentially STEM focused based on their names. Other sources of nominations of inclusive STEM‐focused schools were NC New School Project staff and other state education leaders interviewed for the project. To make sure the STEM‐focused schools were targeting underrepresented groups, we used state data sets to narrow our list of candidate study schools to those with 35% or more low‐income and/or 35% or more underrepresented minority (African American and Hispanic) students (state average proportions across all state high schools was 49% for underrepresented minorities and 39% for low‐income students). This process reduced the list of potential North Carolina ISHSs to 73. Next, we conducted phone calls to each candidate school and used a screening protocol to establish whether the school had a more intensive STEM program than that required of North Carolina schools for high school graduation, and whether the STEM program was schoolwide, and not limited to students meeting certain criteria. We also removed schools from our list if they did not have a current class of 12th graders, or used test scores for selective admissions. These screens reduced the list of schools meeting our ISHS study criteria to 24. Four of these schools were in districts that declined to participate in our research, leaving 20 ISHSs of which 12 administered a research survey to 12th graders. (An additional six of these schools had agreed to participate in Grade 12 survey but ran out of time prior to students’ graduation.) The process of recruiting, screening, and retaining schools for the study is summarized in the left‐hand flowchart in Appendix B.

For each ISHS agreeing to participate, we then sought a matched comparison school in the same state that served similar students but did not offer a schoolwide STEM‐focused program. In seeking comparison schools in North Carolina, we reasoned that we could maximize the similarity of students in comparison schools to those in ISHSs if we took the former from similar districts that did not have an ISHS (reasoning that students like those attending ISHSs would have chosen a STEM‐focused school if one were available). Using a database containing all regular North Carolina high schools, we began a process of identifying the most closely matched comparison schools for each ISHS that were similar in terms of student demographics (percent minority and percent low income) and average test scores, giving priority to the latter variable (which is the best predictor of future achievement) when trade‐offs had to be made. We produced a prioritized list of non‐STEM school matches for each ISHS and proceeded to contact candidate comparison schools in order of quality of match to the ISHS until we found one willing to participate. The details of the screening and recruiting process, including the number of schools in the pool at each step of the process, are depicted in the left‐hand flowchart in Appendix B.

Schools were offered incentives for participation: a school‐specific report of their student survey findings and an honorarium in return for assistance in fielding the student surveys (except in a few cases where districts prohibited such payments). The honorarium was set at $500 for a small school (enrollment of 600 students or fewer) and $1000 for a larger school (enrollment greater than 600). The target student sample within each school was all students in the 12th grade as of spring 2013. The obtained student survey response rate was 77% across schools and yielded 574 ISHS and 1703 comparison school student respondents.

ANALYSIS

We used HLM to compare 12th graders in ISHSs to those in comparison schools serving students with similar academic performance in Grade 8 in terms of academic experiences and attitudes, plans and aspirations, high school STEM experiences, and academic achievement, adjusting for student demographic characteristics and eighth‐grade achievement scores. We conducted one set of analyses for all 12th‐grade students who responded to our survey and additional sets for African American and female subgroups. For each set of comparisons, we posited a hierarchical model with student and school levels for the same set of outcomes. A hierarchical linear model was posited for continuous outcomes and a hierarchical model with a logit link function for dichotomous outcomes. The ISHS impact was estimated at the school level. The HLM for student‐level outcomes took the form:

where i is students, j is schools, Yis a student outcome, and ISHS equals 1 for students in an ISHS school and 0 for students in a comparison school. e and r are student and school random effects. β 1  is the estimated ISHS impact on the student outcome. We included as student‐level covariates being female, African American, Hispanic, economically disadvantaged, limited in English proficiency, special education, either parent having a bachelor's degree, and eighth‐grade math, science, and reading achievement, as well as a variable indicating whether a student took Algebra before ninth grade. We incorporated school‐level covariates including Title I improvement status (controlling for accountability pressure) and percent economically disadvantaged students in the school. We used multiple imputation to impute missing values for student‐level predictors.1 To clearly present results, we calculated the model‐predicted values for students in ISHSs and comparison schools, respectively. The model‐predicted values represent the expected values for the average student, assuming attendance in an ISHS or comparison school, respectively, and the difference between the ISHS and comparison expected values indicates the ISHS impact on the student outcome of interest. The student‐ and school‐level equations can be found in Appendix C.

RESULTS

To examine the extent to which the ISHSs agreeing to participate in the study were representative of all the schools we identified as ISHSs, we compared school‐level data for those ISHSs that agreed and those that declined to participate in the research. The only statistically significant difference between the ISHSs accepting our invitation to participate and those declining the opportunity to participate in the study was a small difference in average attendance rate (Table 1, columns 1 and 2). The very small difference in attendance rate for the two school types is statistically significant because the variation for average attendance is very small. The ISHSs agreeing to participate in the study appeared to be more likely than those that declined to be in program improvement status and to serve a higher proportion of low‐income students, but because of the large variation across schools, these differences were not statistically significant.

Table 1

Characteristics of North Carolina study ISHSs, comparison schools, and ISHSs that did not participate

	Study ISHSs	Comparison Schools	ISHSs That Did Not Participate	All North Carolina High Schools
Characteristic	(n = 12)	(n = 12)	(n = 12)	(n = 596)
Average student enrollment	754	1120	712	804.0
Program Improvement status	0.42	0.45	0.27	0.39
Percent minority students	62.6	38.3*	63.5	38.9
Percent low‐income students	56.6	52.2	35.3	49.4
Mean incoming eighth‐grade math scorea	360.0	360.8	359.2	360.8
Mean incoming eighth‐grade science scorea	147.7	148.7	146.8	148.7
Attendance (% days)	95.6	94.0**	93.2	93.2

As reported by North Carolina Public Schools, the 2009 eighth‐grade statewide mean and standard deviation are 361.9 and 8.8 for math and 151.5 and 9.1 for science.

t‐Tests and chi‐square tests were conducted between Study ISHSs and Comparison Schools and between Study ISHSs and ISHSs that did not participate. Differs from ISHSs in the study sample at * p <.05; **p < .01.

Our school matching process was intended to yield comparison schools that were similar to the ISHSs in terms of their student populations. Comparing the ISHSs in column 1 of Table 1 to the comparison schools in column 3, we can see that the ISHSs in our sample differed from the comparison schools in having a larger proportion of underrepresented minority students—62.6% compared to 38.3% for the comparison school sample. Two aspects of our method likely contributed to this difference. First, because of our interest in whether ISHSs could address STEM participation rates for underrepresented minorities, we had eliminated any STEM‐focused school with fewer than 35% underrepresented minority students from our school recruiting. Second, in identifying matched comparison schools, we prioritized obtaining a good match in terms of the prior achievement of students entering the two types of high schools, and the North Carolina comprehensive high schools that matched the ISHSs in terms of students’ entering achievement levels had lower proportions of minority students. Thus, the ISHS and comparison school samples differed in terms of demographic composition, but not in any way that would be expected to produce an ISHS advantage in terms of high school achievement outcomes. From a comparison of columns 3 and 4 in Table 1, we can see that the comparison high schools in our sample were very similar to North Carolina high schools as a whole.

Next, we compared the students who took the survey at the ISHSs and the comparison schools. Table 2 presents the students’ survey reports on their backgrounds. The most striking difference between the student samples in the two types of school was the larger proportion of African American students in the ISHSs: Half of the ISHS students completing the survey were African American compared to only 25% of students in the comparison school sample (p < .001). A high percentage of students at both ISHS and comparison schools reported having at least one parent working in a STEM‐related field, but it should be noted that our survey item gave examples of STEM‐related jobs that do not require a 4‐year degree (e.g., computer technician) as well as those that do (e.g., civil engineer).

Table 2

Comparison of North Carolina ISHS and comparison school grade 12 survey respondents

	ISHS	Comparison School
Characteristic	(n = 574) (%)	(n = 1703) (%)
African American	50a	25
Hispanic	10	8
Female	55	50
Language other than English spoken at home	11	8
At least one parent with a bachelor's degree	37	36
At least one parent in a STEM‐related field	47	48

Source: iSTEM Grade 12 Student Survey administered in 2012–2013.

p < .001.

After examining the comparability of students in ISHS and comparison schools, we then undertook a series of analyses of school experiences, attitudes, aspirations, and achievement outcomes for African American and female students as well as for the entire Grade 12 survey sample in the two sets of schools. As shown in Appendix C, student and school background variables, such as student economic disadvantage and proportion of minority students in the school, were included in the models as control variables. Prior research has found these background variables to be related to student achievement and other outcomes. Controlling for these variables therefore supports the comparison of comparable students at both the individual and school levels, thus supporting the calculation of unbiased ISHS impact estimates.2

STEM coursework and activities

Table 3 shows data on students’ STEM coursework and activities in the ISHS and comparison schools in the study. The data in these tables, and all subsequent tables, are predicted values from the HLM models that adjust for the differences in school and student characteristics between ISHS and comparison schools, as described in the Analysis section.

Table 3

STEM coursework and activities of North Carolina ISHS and comparison school grade 12 survey respondents

	All Students		African Americans		Females
	ISHS	Comparison School	ISHS	Comparison School	ISHS	Comparison School
Item/Scale	(n = 574)	(n = 1703)	(n = 382)	(n = 445)	(n = 305)	(n = 841)
Took calculus or precalculus	60%***	38%	55%***	24%	73%***	41%
Number advanced math courses taken	0.99**	0.74	0.84***	0.43	1.17***	0.77
Took physics	32%**	12%	27%**	8%	29%*	8%
Took chemistry	79%**	58%	71%*	46%	90%***	66%
Number advanced science courses taken	0.29**	0.18	0.18*	0.08	0.32**	0.18
Took one or more technology courses	64%	58%	69%*	56%	52%	52%
Took one or more engineering courses	46%	23%	55%*	18%	28%*	8%
Number extracurricular STEM activities	1.66***	0.89	1.77***	0.95	1.37***	0.77
Number informal STEM activities outside school	2.28**	2.08	2.29**	2.05	2.12**	1.93
Took ACT or SAT	97%	94%	96%	94%	97%	95%
Took an AP exam	48%	43%	37%*	30%	51%	45%

Source: iSTEM Grade 12 Student Survey administered in 2012–2013; predicted values from HLM models for each of the dependent variables shown in the Item/Scale left‐hand column, adjusting for differences in school and student characteristics between ISHS and comparison schools. Separate analyses were conducted for the overall student sample and for subgroup samples. Covariates data obtained from North Carolina Education Research Data Center.

*p < .05; **p < .01; ***p < .001.

Perhaps the most striking aspect of Table 3 is that the overwhelming pairwise differences between ISHS and comparison school 12th graders, for students overall and for subgroups underrepresented in STEM, generally show higher participation rates for students in the ISHS sample. Not all of the differences are large, and many fail to attain statistical significance, but in no case is there a statistically significant advantage for students who attended a comparison school.

The 12th graders in our North Carolina ISHS sample reported more academic experiences relevant to becoming ready for STEM at the college level than did their counterparts in comprehensive high schools. The North Carolina high school seniors overall, the African American subgroup, and females were more likely to have taken precalculus or calculus, physics, and chemistry if they attended an ISHS rather than a comprehensive high school. In addition, African American students were significantly more likely to have taken one or more engineering courses and one or more technology courses if they attended an ISHS rather than a comparison school. African American students were more likely to have taken an AP examination if they attended an ISHS. As one might expect, students overall and both African American and female students reported having engaged in more extracurricular activities related to STEM and more self‐selected STEM activities outside of school if they attended an ISHS.

Attitudes toward STEM subjects

The Grade 12 Study Survey included items related to attitudinal constructs emphasized in SCCT, such as sense of self‐efficacy in mathematics, identity with the subject of science, and reactions after encountering difficulty in a math or science class. Student attitudes toward STEM subjects are shown in the top portion of Table 4. Students in the ISHS sample expressed a stronger science identity than did those in the comprehensive high school sample. In addition, students overall and females (but not African American students) expressed a stronger math identity if they had attended an ISHS. In contrast, there were no statistically significant differences between the two school samples in terms of students’ sense of math or science efficacy. There was a difference favoring ISHS students in terms of self‐reported perseverance when they encountered difficulties in a math or science class both for the total sample and for female students.

Table 4

STEM attitudes and plans of North Carolina ISHS and comparison school grade 12 survey respondents

	All Students		African Americans		Females
	ISHS	Comparison School	ISHS	Comparison School	ISHS	Comparison School
Item/Scale	(n = 574)	(n = 1703)	(n = 382)	(n = 445)	(n = 305)	(n = 841)
Math identity (scale)	2.43**	2.28	2.40	2.33	2.34*	2.21
Science identity (scale)	2.61***	2.37	2.38**	2.23	2.59**	2.39
Math efficacy (scale)	2.58	2.61	2.59	2.65	2.49	2.56
Science efficacy (scale)	2.90	2.83	2.80	2.82	2.94	2.75
Perseverance in math or science class	2.89*	2.59	2.88	2.62	2.99*	2.69
Plan to enter 4‐year college next fall	52%	45%	57%	47%	55%	51%
Plan to earn bachelor's or higher degree	81%	75%	88%	83%	81%	80%
Plan to earn master's or higher degree	39%*	30%	41%	29%	45%	34%
STEM career interest	89%**	82%	92%	86%	87%**	76%

Source: iSTEM Grade 12 Student Survey administered in 2012–2013; predicted values from HLM models for each of the dependent variables shown in the Item/Scale left‐hand column, adjusting for differences in school and student characteristics between ISHS and comparison schools. Separate analyses were conducted for the overall student sample and for subgroup samples. Covariates data obtained from North Carolina Education Research Data Center.

Math and science identity and efficacy scales are averages of 4‐point scale items: Strongly disagree [1] to strongly agree [4].

The perseverance in math or science class measure is a count of persistence activities. STEM career interest was scored as present for students rating themselves as “very interested” in a career in one or more of the four STEM fields.

*p < .05; **p < .01; ***p < .001.

Students’ plans and aspirations

Twelfth graders’ aspirations for college and careers are shown for the North Carolina school samples in the bottom portion of Table 4. The level of academic aspiration in terms of expectation for postsecondary degree completion tended to be higher among students in the ISHS sample. ISHS students expressed higher aspirations than students in comprehensive high schools in terms of plans to earn a master's or higher degree. ISHS students overall and ISHS female students also expressed a higher level of interest in pursuing a STEM career than did their counterparts in comparison schools. Among African American students there appears to be a similar trend toward higher STEM career interest in the ISHS sample, but it did not rise to the level of statistical significance.

Qualities of high school STEM experiences

The data in Tables 2–4 indicate that after controlling for a comprehensive set of school‐ and individual‐level variables through the HLM, STEM coursework, activities, attitudes, and career interest are stronger in most cases not only for ISHS students overall but also for female students and for African American students if they attended an ISHS. To gain some insight into the high school experiences that might contribute to these outcomes, the Grade 12 Student Survey included items asking students about their experiences with math and science classroom instruction and with school supports for making college and career plans. Differences between reports of students in the ISHS and comparison school samples were found on these measures as well, as shown in Table 5.

Table 5

High school STEM experiences reported by students in North Carolina ISHS and comparison schools

	All Students		African Americans		Females
	ISHS	Comparison School	ISHS	Comparison School	ISHS	Comparison School
Item/Scale	(n = 574)	(n = 1703)	(n = 382)	(n = 445)	(n = 305)	(n = 841)
Math instruction included advanced skills (scale)	3.54***	3.16	3.64**	3.15	3.55**	3.14
STEM integrated into math instruction (scale)	3.14**	2.84	3.13*	2.77	3.08*	2.72
Science instruction included advanced skills (scale)	3.79	3.61	3.81	3.57	3.84	3.62
STEM integrated into science instruction (scale)	3.55	3.40	3.68	3.41	3.56	3.41
Teachers’ high expectations for all students (scale)	3.11***	2.78	3.20***	2.76	3.06**	2.80
Teachers’ respect for students (scale)	3.19***	2.89	3.16***	2.82	3.17***	2.92
Number of college and career readiness supports used	5.84***	4.45	7.07***	5.68	6.29***	4.71
Talked with teachers about academic and career plans (scale)	2.86*	2.45	2.76*	2.17	2.93	2.60
Talked with counselors about academic and career plans (scale)	2.11	2.20	2.28	2.30	2.22	2.47
Talked with parents about academic and career plans (scale)	2.75	2.73	2.64	2.45	2.90	2.94

Source: iSTEM Grade 12 Student Survey administered in 2012–2013; predicted values from HLM models for each of the dependent variables shown in the Item/Scale left‐hand column, adjusting for differences in school and student characteristics between ISHS and comparison schools. Separate analyses were conducted for the overall student sample and for subgroup samples. Covariates data obtained from North Carolina Education Research Data Center.

Instruction scales are averages of 5‐point scale items: Never [0] to almost every day [4].

Teachers’ Expectations and Respect scales are averages of 4‐point scale items: Strongly disagree [1] to strongly agree [4].

Talked About Academic and Career Plans variables are counts of the number of topics discussed from the following list: (1) taking math courses, (2) taking science courses, (3) going to college, and (4) careers.

*p < .05; **p < .01; ***p < .001.

Twelfth graders in ISHSs were more likely than those in comparison schools to describe their mathematics classes as having features associated with instruction of advanced skills and deeper learning, such as use of project‐based learning and the tools used by math and science professionals. They were also more likely to describe their mathematics instruction as incorporating other STEM subjects. A similar but less pronounced (and statistically nonsignificant) pattern was found in survey reports for science classes. In terms of teachers’ expectations for student success and treatment of all their students with respect, ISHS students rated their math and science teachers more highly than did students in the comparison school sample. Finally, the number of college and career readiness supports students experienced was higher in ISHSs than in comparison schools. Importantly, ISHS African American and female students also reported these same experiences to a significantly greater extent than their counterparts in the comparison school sample. They also reported a higher frequency of talks with their teachers about their academic and career plans. Reported frequency of talks on these topics with school counselors and with parents did not vary for the two school types.

High school achievement measures

The stated mission of ISHSs is to provide a secondary education that will equip their graduates for postsecondary work, including a STEM major if they choose to pursue one. Table 6 shows the high school achievement outcomes most relevant to college readiness available from the North Carolina data system.

Table 6

High school achievement outcomes for students in North Carolina ISHS and comparison schools

	All Students		African Americans		Females
	ISHS	Comparison School	ISHS	Comparison School	ISHS	Comparison School
Item/Scale	(n = 574)	(n = 1703)	(n = 382)	(n = 445)	(n = 305)	(n = 841)
Weighted GPA	3.45**	3.25	2.97**	2.70	3.63**	3.41
ACT Matha	19.50	19.35	17.64	17.34	19.44	19.40
ACT Sciencea	19.24*	18.34	16.66	15.75	19.12	18.37

Source: State longitudinal student data system at NCERDC; predicted values from HLM models for each of the dependent variables shown in the Item/Scale left‐hand column, adjusting for differences in school and student characteristics between ISHS and comparison schools. Separate analyses were conducted for the overall student sample and for subgroup samples. Covariates data obtained from North Carolina Education Research Data Center.

As reported by ACT, the nationwide mean and standard deviation for ACT scores for the graduating class of 2013 are 21.1 and 5.3 for mathematics, and 21.0 and 5.2 for science. Corresponding means and standard deviations for African American students are 17.2 and 3.6 for mathematics and 16.9 and 4.3 for science.

*p < .05; **p < .01; ***p < .001.

Students overall, African American students, and female students in the ISHS sample had higher weighted GPAs than did students from similar backgrounds in the comprehensive high school sample. ISHS students overall had higher ACT Science scores, but they did not exceed their peers in comprehensive high schools in terms of ACT Mathematics scores. There were no significant school type differences for subgroups on ACT scores.

STUDY 2: TEXAS

Study 2 addressed the same central questions about ISHS impacts for students overall and for members of population groups underrepresented in STEM within the state of Texas. In this state, a public–private partnership for high school reform, the THSP, included a $71 million investment in starting new Texas STEM (“T‐STEM”) high schools, announced in 2005. Through THSP, charter organizations received funding starting in 2007–2008 to help defray start‐up costs for more than 51 T‐STEM high schools conforming to a T‐STEM Blueprint describing design features and best practices for inclusive STEM high schools (http://www.tstemblueprint.org). The T‐STEM Blueprint was more specific and prescriptive than the guidance and professional development offered in North Carolina, but emphasized similar instruction and school design characteristics. These features included serving large proportions of students from low‐income and underrepresented groups, an emphasis on interdisciplinary project‐based learning, and partnerships with business and higher education institutions. As in North Carolina, a public–private partnership organization (in this case, the Communities Foundation of Texas), received funding from the Bill & Melinda Gates Foundation, which was promoting strategies to increase college going among low‐income and minority students by redesigning high schools to promote “rigor, relevance, and relationships” (Bill & Melinda Gates Foundation, 2005b). After funding to support start‐up costs for T‐STEM schools ceased in 2011, the T‐STEM Blueprint has continued to have influence as the basis for a T‐STEM designation process run by the state education agency. In addition to providing funding and the T‐STEM Blueprint, the THSP set up a statewide support infrastructure for the T‐STEM schools. Seven T‐STEM Centers were established across the state to work with the T‐STEM schools, and additional technical assistance provided leadership coaching for T‐STEM school leaders. General developments in Texas education during this timeframe also influenced ISHSs. These included institution of a policy between 2007 and 2013 requiring any Texas high school student to complete four courses in each core subject area, including mathematics and science, to qualify for graduation (the “4 × 4” requirement). Some Texas ISHS leaders expressed concern that this latter policy would make their programs less distinctive and thereby less attractive to prospective students and their parents.

The Grade 12 Student Survey used in Study 1 was administered in the Texas high schools with only a few minor modifications of item wording (e.g., updating to current academic year, etc.). As in Study 1, schools provided the research team with student rosters and selected how and when they wanted to administer the survey.

In Texas, we conducted the analysis linking student administrative data with our survey data at the Texas Education Research Center (ERC) at the University of Texas at Austin. From the Texas ERC data, we obtained student demographic information and eighth‐grade achievement in reading, math and science, as well as Grade 11 scores on the Texas Assessment of Knowledge and Skills (TAKS) mathematics and science tests.

As in North Carolina, sampling and recruiting in Texas began with the identification and recruiting of inclusive STEM‐focused high schools followed by recruiting comparison schools without a STEM focus that served similar student populations.

School samples

Identification of ISHSs in Texas was straightforward because the requirements for designation as a Texas STEM high school (“T‐STEM”) include the school design and implementation criteria stipulated in our definition of an ISHS, and the Texas Education Agency maintained a list of T‐STEM schools. In 2013, there were 77 designated T‐STEM schools, 51 of which opened prior to 2010–2011, thus making them likely to have a senior class in 2013–2014 that could participate in this study. As a first step, policies for research approval were checked for the districts with jurisdiction over these T‐STEM schools. Several districts required research applications 9 months prior to data collection or had a policy of declining any nonmandated data collection that might detract from instructional time. Our researchers called T‐STEMs established prior to 2010–2011 in the remaining districts and in charter management organizations to ascertain their level of interest in being part of the study and to verify that they had a schoolwide STEM program and a 12th‐grade class. ISHS recruiting ceased once the target sample of 30 schools was achieved. School incentives were the same as those used in Study 1. Of the 42 Texas ISHSs invited to participate, 30 agreed to participate and 27 went on to administer the Grade 12 Student Survey in the spring of 2014. The details of the screening and recruiting process, including the number of schools in the pool at each step of the process, are depicted in the right‐hand flowchart in Appendix B.

In identifying potential comparison schools in Texas, where there were many more ISHSs than in North Carolina, we did not rule out districts having an ISHS from consideration for comparison school recruiting as we had done in Study 1. However, we did seek comparison schools serving students similar to those in the ISHS under consideration that were not geographically close to a STEM school (so students did not have the ready option of choosing a STEM‐focused high school). Of the 55 comprehensive high schools recruited for Study 2, 14 agreed to participate and 10 returned Grade 12 Student Surveys.

Student samples

The target student sample within each school was all students in the 12th grade as of spring 2014. The obtained student survey response rate of 77% yielded 1041 ISHS and 1795 comparison school student respondents for our analyses.

The analytic model described for Study 1 was applied with the Texas data in Study 2. The only modifications were that the Texas administrative data did not include information on the covariate on whether a student took Algebra before ninth grade or on the ACT score outcome variables. In Study 2, the latter measures were replaced with Grade 11 Texas TAKS scores in mathematics and science.

As in Study 1, our first step was understanding the extent to which our ISHS sample schools were representative of all ISHSs within the state. Table 7 shows these data.

Table 7

Characteristics of Texas study ISHSs, comparison schools, and ISHSs that did not participate

	Study ISHSs	Comparison Schools	ISHSs That Did Not Participate	All Texas High Schools
Characteristics	(n = 27)	(n = 10)	(n = 30)	(n = 1852)
Average student enrollment	923	1387	725	900
Program Improvement status	0.14	0.30	0.07	0.16
Percent minority students	73.4	69.0	70.9	51.6
Percent low‐income students	65.6	69.0	63.5	53.5
Mean incoming eighth‐grade math scorea	812.41	833.3	838.56	826.97
Mean incoming eighth‐grade science scorea	2285.17	2234.81	2299.34	2243.74
Attendance (% days)	95.2	93.9	95.7	94.5

t‐Tests and chi‐square tests were conducted between Study ISHSs and Comparison Schools and between Study ISHSs and ISHSs That Did Not Participate. No significant differences were found.

Data from the Texas Education Research Center indicate that the 2010 eighth‐grade statewide mean and standard deviation were 826.97 and 312.80 for math and 2243.74 and 281.49 for science.

Column 1 of Table 7 presents basic descriptive information for the ISHSs in our study and column 4 shows the same variables for the T‐STEM schools not in our study. There were no statistically significant differences. Column 3 provides descriptive information for the comparison schools in the study, and column 5 shows characteristics of Texas high schools as a whole. None of the school‐level variables in Table 7 differed significantly between our ISHS and comparison school samples, suggesting that they were serving students who were similar upon high school entry.

Having established the representativeness of the school samples, we proceeded to compare survey responses from ISHSs and comprehensive high schools, as in Study 1. Table 8 shows students' survey responses about their backgrounds from the two school samples. The biggest difference between students in the two types of schools was that students in ISHSs were more likely to speak a language other than English in the home (44% versus 30%, respectively, p < .001). Three additional differences were smaller in magnitude but statistically significant: ISHS students were less likely to be female (47% vs. 52%) or to have a parent working in a STEM‐related field (30% vs. 37%) but more likely to report having at least one parent with a bachelor's degree (29% vs. 23%). This third variable is the only one of the three differences that might “favor” ISHS students in terms of likelihood of attending college and declaring a STEM major, but the difference is modest and the majority of students in both types of high school did not have a parent with a college degree. Parallel sets of analyses were conducted for Hispanic, African American, and female subgroups as well as the total survey sample to examine associations between school type and high school outcomes.

Table 8

Comparison of Texas ISHS and comparison school grade 12 survey respondents

	ISHSs	Comparison Schools
Characteristic	(n = 1041) (%)	(n = 1795)
African American	11	11
Hispanic	67	66
Female	47	52*
Language other than English spoken at home	44***	30
At least one parent with a bachelor's degree	29***	23
At least one parent in a STEM‐related field	30	37***

Source: iSTEM Grade 12 Student Survey administered in 2013–2014.

* p < .05; **p < .01; ***p < .001.

Table 9 shows data on students’ STEM coursework and activities in ISHSs and comprehensive high schools. As in Study 1, the data in this and subsequent tables are model‐predicted values from the HLM analyses, described previously, and have been weighted to account for students being nested in high schools and adjusted for differences in student demographics, eighth‐grade achievement indicators, and school factors.

Table 9

STEM coursework and activities of Texas ISHS and comparison school grade 12 survey respondents

	All Students		Hispanics		African Americans		Females
	ISHS	Comparison School	ISHS	Comparison School	ISHS	Comparison School	ISHS	Comparison School
Item/Scale	(n = 1041)	(n = 1795)	(n = 703)	(n = 1183)	(n = 118)	(n = 191)	(n = 486)	(n = 907)
Took calculus or precalculus	59%*	52%	59%	50%	51%*	34%	62%	52%
Number advanced math courses taken	0.97**	0.74	0.97*	0.73	0.75	0.53	0.98*	0.73
Took physics	89%	90%	89%	90%	79%	83%	91%	91%
Took chemistry	92%	93%	92%	93%	85%	87%	94%	93%
Number advanced science courses taken	0.44**	0.24	0.43**	0.22	0.33	0.17	0.38	0.23
Took one or more technology courses	61%***	40%	64%***	37%	54%*	40%	57%***	33%
Took one or more engineering courses	57%***	17%	57%***	17%	65%***	22%	51%***	8%
Number extracurricular STEM activities	1.59***	1.05	1.72***	1.14	1.86	1.48	1.38*	0.85
Number informal STEM activities outside school	2.16**	2.02	2.18**	2.05	2.02	2.05	2.06**	1.86
Took ACT or SAT	90%*	83%	88%	85%	89%	78%	91%	85%
Took an AP exam	64%***	44%	66%***	45%	53%*	31%	63%*	46%

Source: iSTEM Grade 12 Student Survey administered in 2013–2014; predicted values from HLM models for each of the dependent variables shown in the Item/Scale left‐hand column, adjusting for differences in school and student characteristics between ISHS and comparison schools. Separate analyses were conducted for the overall student sample and for subgroup samples. Covariates data obtained from Texas Education Research Center.

*p < .05; **p < .01; ***p < .001.

As found in Study 1 for North Carolina, ISHS students in the Texas Grade 12 Student Survey sample overall reported significantly more STEM coursework and experiences in the form of a higher likelihood of having taken: calculus or precalculus, more advanced science and mathematics courses, one or more technology courses, and one or more engineering courses. They also reported more extracurricular and informal STEM activities outside of school and were more likely to have taken the ACT or SAT college admissions test and to have taken an AP exam. Female students in our sample of Texas ISHSs had the same statistically significant advantages in terms of STEM academic experiences as the total sample, with the exception of likelihood of having taken calculus or precalculus. For this latter variable, the difference for females was in the same direction as for the entire sample (60% for females in ISHSs vs. 49% in comparison schools) but was not statistically significant (p < .09).

Hispanic students in Texas ISHSs had statistically significant advantages over those in large comprehensive high schools in terms of all of the same variables that were significant for the total student sample with the exceptions of having taken calculus or precalculus (reported by 59% of Hispanic students in ISHSs compared to 49% in comprehensive high schools) and having taken the ACT or SAT (88% vs. 84%).

There were somewhat fewer statistically significant differences for the much smaller sample of African American students in Texas ISHSs and comprehensive high schools. Those variables where there were statistically significant advantages for African American students who attended an ISHS in our sample were as follows: completion of calculus or precalculus, taking one or more technology courses, taking one or more engineering courses, and getting mostly As or As and Bs in mathematics.

Student attitudes toward STEM subjects are shown in the top half of Table 10. The direction of differences in all of the attitudinal variables favored ISHS students, but there were fewer statistically significant differences in attitudes than were found in the North Carolina sample in Study 1. Hispanic and female students expressed a significantly stronger science identity if they attended an ISHS (p < .05). African American students expressed a significantly stronger math efficacy if they attended an ISHS (p < .05). Students overall and from each subgroup reported greater perseverance in the face of difficulty in a math or science class if they attended an ISHS (p < .05). Reports of other attitudinal measures (e.g., math identity, science efficacy) were similar for members of these subgroups attending ISHSs and comparison schools.

Table 10

STEM attitudes and plans of Texas ISHS and comparison school grade 12 survey respondents

	All Students		Hispanics		African Americans		Females
	ISHS	Comparison School	ISHS	Comparison School	ISHS	Comparison School	ISHS	Comparison
Item/Scale	(n = 1041)	(n = 1795)	(n = 703)	(n = 1183)	(n = 118)	(n = 191)	(n = 486)	(n = 907)
Math identity (scale)	2.33	2.27	2.35	2.25	2.32	2.19	2.24	2.19
Science identity (scale)	2.40	2.29	2.42*	2.27	2.16	2.16	2.35*	2.20
Math efficacy (scale)	2.72	2.63	2.72	2.63	2.77*	2.58	2.66	2.59
Science efficacy (scale)	2.74	2.78	2.82	2.81	2.54	2.75	2.76	2.73
Perseverance in math or science class	2.51*	2.25	2.49*	2.14	2.60*	2.06	2.69*	2.34
Plan to enter 4‐year college next fall	51%*	38%	50%*	34%	47%	40%	54%*	44%
Plan to earn bachelor's or higher degree	80%*	74%	78%	72%	76%	78%	82%*	77%
Plan to earn master's or higher degree	33%	28%	32%	25%	27%	31%	37%	34%
STEM career interest	88%**	83%	90%**	85%	89%	78%	87%**	80%

Source: iSTEM Grade 12 Student Survey administered in 2013–2014; predicted values from HLM models for each of the dependent variables shown in the Item/Scale left‐hand column, adjusting for differences in school and student characteristics between ISHS and comparison schools. Separate analyses were conducted for the overall student sample and for subgroup samples. Covariates data obtained from Texas Education Research Center.

Math and science identity and efficacy scales are averages of 4‐point scale items: Strongly disagree [1] to strongly agree [4].

Perseverance in math or science class measure is a count of persistent activities. STEM career interest was scored as present for students rating themselves as “very interested” in a career in one or more of the four STEM fields.

*p < .05; **p < .01; ***p < .001.

Twelfth graders’ aspirations for college and careers are shown by school type in the bottom half of Table 10. The level of academic aspiration in terms of expectation for postsecondary degree completion tended to be higher among students in the ISHS sample. Students in the Texas ISHS sample overall and female students expressed higher aspirations than their counterparts in the comprehensive school sample in terms of plans to enter a 4‐year college directly after high school graduation and to earn a bachelor's or higher degree. In addition, ISHS students overall, Hispanic, and female students expressed a higher level of interest in pursuing a STEM career than did students in comparison schools. The same trend toward higher STEM career interest in ISHSs is apparent for African American students, but it did not rise to the level of statistical significance.

As in Study 1, we examined student reports about their experiences with classroom instruction and school supports for making college and career plans. As shown in Table 11, differences between reports of students in the ISHS and comparison school samples in Study 2 were similar to those found in Study 1.

Table 11

High school STEM experiences reported by students in Texas ISHS and comparison schools

	All Students		Hispanics		African Americans		Females
	ISHS	Comparison School	ISHS	Comparison School	ISHS	Comparison School	ISHS	Comparison School
Item/Scale	(n = 1041)	(n = 1795)	(n = 703)	(n = 1183)	(n = 118)	(n = 191)	(n = 486)	(n = 907)
Math instruction included advanced skills (scale)	3.09	2.99	3.11	2.98	3.13	3.02	3.09	2.94
STEM integrated into math instruction (scale)	2.94**	2.73	3.00***	2.74	2.87	2.73	2.82**	2.58
Science instruction included advanced skills (scale)	3.43	3.31	3.43	3.33	3.47	3.37	3.51*	3.27
STEM integrated into science instruction (scale)	3.42	3.23	3.41	3.25	3.41	3.32	3.51***	3.21
Teachers’ high expectations for all students (scale)	2.96***	2.73	2.97**	2.74	2.86	2.68	2.94**	2.73
Teachers’ respect for students (scale)	2.99**	2.84	3.03**	2.86	2.92*	2.76	3.02**	2.83
Number of college and career readiness supports used	6.25**	5.10	6.70**	5.43	6.08	5.39	6.53*	5.41
Talking with teachers about academic and career plans	2.38	2.25	2.27	2.14	2.46	2.35	2.46	2.39
Talking with counselors about academic and career plans	2.36*	2.04	2.37	1.90	2.40*	2.07	2.55*	2.22
Talking with parents about academic and career plans	2.27	2.23	2.46	2.40	2.45	2.30	2.63	2.50

Source: iSTEM Grade 12 Student Survey administered in 2013–2014; predicted values from HLM models for each of the dependent variables shown in the Item/Scale left‐hand column, adjusting for differences in school and student characteristics between ISHS and comparison schools. Separate analyses were conducted for the overall student sample and for subgroup samples. Covariates data obtained from Texas Education Research Center data.

Instruction scales are averages of 5‐point scale items: Never [0] to almost every day [4].

Teachers’ Expectations and Respect scales are averages of 4‐point scale items: Strongly disagree [1] to strongly agree [4].

Talked About Academic and Career Plans variables are counts of the number of topics discussed from the following list: (1) taking math courses, (2) taking science courses, (3) going to college, and (4) careers.

*p < .05; **p < .01; ***p < .001.

Twelfth graders in the Texas ISHS sample were more likely than those in the comparison school sample to describe their mathematics classes as integrating content from other STEM subjects (p < .01). Texas ISHS students described their teachers as having higher expectations for student success and as having greater respect for all their students to a greater extent than did students in the comparison school sample (p < .001). As in North Carolina, Texas ISHS students reported using more college and career readiness supports than did their peers in comparison schools (p < .01). ISHS students also reported having more conversations with school counselors about their academic and career plans than comparison school students did (p < .05). There was no difference in reported frequency of conversations with teachers or with parents on these topics.

The pattern of significant differences between ISHS and comparison school students in Texas generally was the same for the Hispanic and female subgroups as for the student sample as a whole. In addition, female students in the ISHS sample reported significantly more integration of other STEM subjects into their science classes (p < .001). Texas African American students also tended to report having more of these experiences if they attended an ISHS, but with the smaller African American samples relative to Study 1 the only variable that attained statistical significance for this subgroup was math and science teachers’ respect for all students and frequency of conversations with counselors about their academic and career plans, both with p < .05.

The only high school STEM achievement measures available from the Texas longitudinal data system were Grade 11 exit‐level TAKS Mathematics and Science scores administered in spring 2013. Scale scores range from 1281 to 2876 with an average of 2262 for Grade 11 TAKS Mathematics. Scale scores range from 1338 to 2829 with an average of 2269 for Grade 11 TAKS Science (Texas Education Agency, 2013). Table 12 shows the TAKS scores for students in the ISHS and comprehensive high school samples.

Table 12

High school achievement outcomes for students in Texas ISHS and comparison schools

	All Students		Hispanics		African Americans		Females
	ISHS	Comparison School	ISHS	Comparison School	ISHS	Comparison School	ISHS	Comparison School
Item/Scale	(n = 1041)	(n = 1795)	(n = 703)	(n = 1183)	(n = 118)	(n = 191)	(n = 484)	(n = 947)
TAKS Mathematicsa	2260.55*	2220.81	2240.16	2210.92	2150.47	2136.19	2239.78	2210.40
TAKS Sciencea	2237.29*	2193.65	2216.75	2186.09	2157.42*	2087.12	2222.72	2182.90

Source: Texas longitudinal student data; predicted values from HLM models for each of the dependent variables shown in the Item/Scale in the left‐hand column, adjusting for differences in school and student characteristics between ISHS and comparison schools. Separate analyses were conducted for the overall student sample and for subgroup samples. Covariates data obtained from Texas Education Research Center data.

Data from the Texas Education Research Center indicate that the 2013 Grade 11 statewide mean was 2262.05 with a standard deviation of 212.57 for TAKS exit‐level mathematics. For TAKS exit‐level science, the mean score was 2269.06 with a standard deviation of 177.58.

*p < .05; **p < .01; ***p < .001.

As shown in the table, students in the ISHS sample overall had higher TAKS Mathematics and Science scores than those in the comprehensive high school sample. In the subgroup analyses, Texas African American students also had higher TAKS Science scores if they attended an ISHS (p < .05). None of the test score differences was large, however.

DISCUSSION

These study findings from inclusive STEM‐focused high schools implemented at scale in two states have implications for the ISHS theory of action (see Means et al., 2016). First, it should be noted that, as intended, the ISHSs were serving large proportions of students from groups historically underrepresented in STEM fields. Half of the 12th graders in the North Carolina ISHSs in this study were African American, a percentage that contrasts sharply to the 9% African American students in the class of 2013 at the state's selective STEM high school (Roberts, 2012). Two‐thirds of the students in the Texas ISHS sample were Hispanic, and in both states a majority of the ISHS student sample came from low‐income homes. Data on all ISHSs in the two states confirm that compared to all state public high schools, ISHSs are serving larger percentages of underrepresented minority students and equal (in North Carolina) or larger (in Texas) proportions of low‐income students.

Across the two studies, ISHS students provided at least equivalent and often more positive responses than their peers in comprehensive high schools in terms of the STEM goals, identity, and expectations measures on our survey. Importantly, ISHS students in general and those from subgroups underrepresented in STEM fields appeared more likely to leave high school with strong interest in pursuing a STEM career than comparable students who attended one of the comparison schools. ISHS students also expressed higher aspirations for postsecondary education, and the overall samples and several subgroups expressed stronger identities as individuals who “do” science.

From the standpoint of SCCT, however, an important component disposing individuals to perseverance in STEM was missing. In the overall state samples and most of the subgroup analyses, students attending ISHSs did not express stronger self‐efficacy in mathematics and science. It should be noted that our studies measured self‐efficacy only through brief, albeit reliable (Cronbach's alpha = .79 for mathematics and .81 for science), survey scales. More nuanced in‐depth measures of self‐efficacy might have uncovered advantages for students in the ISHS sample, but failure to find them in either the North Carolina or the Texas study raises questions about whether ISHS graduates will have a strong enough expectation of success in STEM studies to see them through postsecondary degree programs. It should be kept in mind, however, that the ISHS students were taking more advanced math and science courses than were their counterparts in the comparison high schools. In addition, some studies have found that students’ judgments of their own academic abilities are relative to those of other students in the same class or school (e.g., Marsh & Hau, 2003). It is plausible that students in ISHSs who were taking more advanced mathematics and science courses had more insight into what they did not know and understand than did students of similar backgrounds who attended comprehensive high schools and took less advanced coursework.

In terms of the education attainment and achievement variables that predict entry into college STEM majors, ISHS students were at an advantage in both studies. In North Carolina, students overall, African American, and female students took more advanced math courses and were significantly more likely to have completed calculus or precalculus, chemistry, and physics if they attended an ISHS. Students from all of these groups also had higher grade point averages than their counterparts at schools in the comprehensive school sample. Similarly, in the Texas study, students overall and female students in the ISHS sample completed more advanced mathematics courses than did their comprehensive high school peers. The data reported here for ISHSs in North Carolina and Texas show that these schools are getting a majority of their students through at least precalculus in high school.

On achievement tests, ISHS students overall had significantly higher scores on the ACT Science in North Carolina and on the TAKS Mathematics and Science in Texas. At the same time, it must be acknowledged that mathematics test scores were not significantly higher for North Carolina students on the ACT Math or for the underrepresented groups on the TAKS Math in Texas (differences all favored the ISHS samples but were small in magnitude). The fact that these large‐scale assessments do not attempt to measure calculus or precalculus may account at least in part for this lack of ISHS impact.3 It should be noted also that even for cases where ISHS students had higher scores than did similar students who attended comprehensive high schools, the ISHS students’ scores were not high compared to national and state averages. However, recent research suggests that high school coursetaking is more predictive of college success in STEM than are high school test scores (Wang, 2013), but further research is needed to determine whether the ISHS experience prepares students sufficiently for postsecondary STEM majors.

A limitation of this study is that the two school groups, ISHSs and comparison schools, diverged in ways other than whether or not they were implementing an ISHS model. The most obvious difference was that most of the ISHSs were “schools of choice.” Students opted into ISHSs while most students at comprehensive high schools were there because they lived in a defined attendance area. One might expect that even though parental education levels were similar for the two groups and were controlled in our analyses, students in the ISHSs may have had parents who placed a higher value on education or were able to offer more support for their students’ educational endeavors. One piece of data contraindicating this alternative explanation for ISHS advantages was that in both states, students in ISHSs and comparison schools reported equivalent frequencies of interactions with their parents around academic and career goals (see Tables 5 and 11).

ISHSs also tended to have smaller enrollments than the comprehensive schools in the comparison sample. In addition, the ISHSs in the North Carolina school sample served larger proportions of African American students than the comparison high schools did. Despite the use of an extensive set of covariates in our modeling of student achievement prior to high school entry, it is possible that in both states differences between the two sets of schools other than their focus on STEM may have influenced the differences in outcomes at Grade 12. Some of these differences (especially the school choice variable) seem likely to have positive influences on student attitudes and academic performance irrespective of the schools’ instructional practices while other differences (such as the greater proportion of underrepresented minorities) have been associated with poorer outcomes in other studies. Given this limitation in the comparability of the ISHS and comprehensive schools, we cannot make a strong case for causation. We are currently in the process of collecting Grade 12 student outcome data for a second cohort of ISHS and comparison school students who were surveyed 3 years ago as they began Grade 9. Analyses on this second data set will be able to control for level of science interest and STEM activity in middle school and for overall academic orientation at the start of high school.

Because STEM education outcomes for underrepresented students are an important policy and equity issue, the general similarity of the positive findings in two states with very different demographics and policy contexts is encouraging. Such replicable findings suggest that the ISHS model warrants further study.

This study provides an example of replicating studies in multiple contexts, a need that is particularly important if research findings are to play a role in guiding education policy. Replication is essential to science, and increased attention is being paid to the need for more replications in education research (Ionnidis, 2014; Makel & Plucker, 2014). Using Schmidt's (2009) replication nomenclature, the Texas study is essentially a direct replication of the North Carolina study. The purpose of running the replication was to observe the generality of ISHS outcomes with a different population of students and schools in a different state context. One important contextual difference was that in the state of Texas for the majority of the time the students in our Grade 12 sample were in high school, all high school students were required to complete four mathematics courses and four science courses for graduation. This requirement may have reduced differences between the ISHS and comparison school samples in terms of number of math and science courses taken, including taking chemistry, physics, and so on. This Texas policy, subsequently revoked in 2013, may account for the fact that students in Texas ISHSs were not more likely than their peers in comparison schools to have taken chemistry or physics. Such differences in ISHS impact on coursetaking in the two states underscore the importance of considering the broader state context when conducting research on nontraditional kinds of schools and when deriving implications for education policy.

Implications for efforts to establish new inclusive STEM‐focused schools can be found in the significantly different experiences reported by ISHS students not just in terms of more STEM coursetaking and extracurricular activities but also in terms of the kinds of instruction they received in STEM classes and their perceptions of the teachers in those classes. Instruction stressing advanced skills, incorporating project‐based work, and integrating multiple STEM disciplines may maintain or inspire student interest in STEM careers. Also important is the fact that in ISHSs students are more likely to report having mathematics and science teachers who set high standards and convey a belief that all students can achieve them. These experiences stand in stark contrast to reports in the literature of minority and female students losing confidence in their ability to pursue STEM subjects after receiving explicit or implicit messages from school staff that STEM is too hard for them (Aschbacher et al., 2010).

While promising, these findings with respect to ISHSs as a strategy for improving the equity of STEM learning opportunities call out for further research. Notably, the question remains whether the constellation of sometimes modest but consistently positive advantages in terms of STEM‐related outcomes for ISHS students are sufficiently strong and enduring to result in better postsecondary outcomes. Our research team is currently in the process of collecting and analyzing quantitative and qualitative data for the postsecondary experiences of the student cohorts in North Carolina and Texas.

Other needed research involves an in‐depth examination of the nature of mathematics instruction in these high schools and an investigation of the knowledge and skill levels of students who took advanced mathematics courses in high school but did not score highly on standardized tests. It is important to examine how these students fare in college mathematics courses and in STEM areas requiring mathematics.

Finally, we see a need for research on the public policy systems necessary to establish a consistently effective set of ISHSs and to sustain them over time in the face of budgetary pressures and shifting education priorities and leadership. Only by addressing these issues on a systemic basis can we hope to produce significant improvements in STEM learning opportunities and the representativeness of the STEM workforce.

Scale	Description	Items Included
Science instruction included advanced skills	Average of 5‐point scale items: Never [0] to almost every day [4]	Think about the SCIENCE course that you took LAST YEAR when you were a junior. In that SCIENCE course, how often did you do the following? Conducted laboratory activities, investigations, or experiments Wrote up results or prepare presentations from a lab activity, investigation, or experiment Generated your own hypotheses Used evidence/data to support an argument or hypotheses Found information from graphs and tables Worked on projects that took multiple days to complete
Math instruction included advanced skills	Average of 5‐point scale items: Never [0] to almost every day [4]	Think about the MATH course that you took LAST YEAR when you were a junior. In that MATH course, how often did you do the following? Applied mathematical concepts to “real world” problems Analyzed data to make inferences or draw conclusions Explained to the class how you solved a math problem Worked on problems with more than one solution Picked the projects or research topics you worked on Made estimates, predictions, or hypotheses Worked on projects that took multiple days to complete
STEM integrated into science instruction	Average of 5‐point scale items: Never [0] to almost every day [4]	Think about the SCIENCE course that you took LAST YEAR when you were a junior. In that SCIENCE course, how often did you do the following? Used probes, computers, calculators, or other educational technology to learn science Used engineering ideas in assignments or projects Learned some new mathematics so you could use it in science
STEM integrated into math instruction	Average of 5‐point scale items: Never [0] to almost every day [4]	Think about the MATH course that you took LAST YEAR when you were a junior. In that MATH class, how often did you do the following? Learned something about science Used technology Learned something about engineering
Perseverance in math or science classes	Count of activities	Have you ever had a difficult time understanding the content or earning the kind of grade you wanted in a science or math class? Think about the last time you had this kind of trouble. Which of the following did you do? Asked my teacher for help Got someone to tutor me Started spending more time studying/working on assignments Got help from a parent or other adult outside the school Studied with a classmate
Number of college and career preparation supports used	Count of activities	Which of these school‐offered services and experiences have you used during this academic year? College entrance exam preparation assistance Career guidance College tours Enrollment in college courses (offered on a college campus, online or at your school) One‐to‐one tutoring Classes and/or seminars on how to improve academically (for example, homework strategies, organization, time management) Academic counseling about what courses to take or how to apply to college Academic “catch up” program or class (for example, in reading or mathematics) Advanced placement strategies (for example, tutoring, prep sessions, or summer academies supporting work in AP classes) Since the beginning of the school year, which of the following people have you talked with about possible jobs or careers when you are an adult? A teacher A school counselor
Number of extracurricular STEM activities participated in	Count of activities (Never to no, all other choices to yes): Never to Almost every day	In your junior year, did you participate in any of the following types of extra‐curricular activities and if so, how often? School math, science or technology club (for example, math club or robotics club) Math or science competition Math, science, or computer camp Environmental projects (for example, monitoring water quality)
Number of informal STEM activities outside of school	Average of 4‐point scale items: Strongly disagree [1] to Strongly agree [4]	In the PAST 2 YEARS, how often have you done the following activities outside of school? Read science books and magazines Made up your own experiment Designed (thought up) and built something on your own Taken apart a toy or appliance to see how it worked Accessed Websites for computer technology information Visited a science museum, planetarium or environmental center
Teachers’ high expectations for all students	Average of 4‐point scale items: Strongly disagree [1] to Strongly agree [4]	How much do you agree or disagree with the following statements? Teachers at this school believe that all students in this school can do well. Teachers at this school have given up on some of their students (reverse coded). Teachers at this school expect very little from students (reverse coded). Teachers at this school work hard to make sure that all students are learning. Teachers at this school only care about smart students (reverse coded).
Teachers’ respect for students	Average of 4‐point scale items: Strongly disagree [1] to strongly agree [4]	How much do you agree or disagree with the following statements? Teachers at this school always try to be fair. Teachers at this school care about my opinions. Teachers at this school would be willing to give me extra help. Teachers at this school care about how I am doing in school.
Science identity	Average of 4‐point scale items: Strongly disagree [1] to strongly agree [4]	You see yourself as a science person Others see you as a science person
Math identity	Average of 4‐point scale items: Strongly disagree [1] to strongly agree [4]	You see yourself as a math person Others see you as a math person
Science efficacy	Average of 4‐point scale items: Strongly disagree [1] to strongly agree [4]	How much do you agree or disagree with the following statements about that SCIENCE course? You did well on tests in this course. You understood the most difficult material presented in the textbook used in this course.
Math efficacy	Average of 4‐point scale items: Strongly disagree [1] to strongly agree [4]	How much do you agree or disagree with the following statements about that MATH course? You did well on tests in this course. You understood the most difficult material presented in the textbook used in this course.
STEM career interest	Count of activities (very interested to yes, all other choices to no): Not interested to very interested	How interested are you in jobs related to the following subjects? Science Technology Engineering Mathematics