School Climate and the Impact of Neighborhood Crime on Test Scores.

Does school climate ameliorate or exacerbate the impact of neighborhood violent crime on test scores? Using administrative data from the New York City Department of Education and the New York City Police Department, we find that exposure to violence in the residential neighborhood and an unsafe climate at school lead to substantial test score losses in English language arts (ELA). Middle school students exposed to neighborhood violent crime before the ELA exam who attend schools perceived to be less safe or to have a weak sense of community score 0.06 and 0.03 standard deviations lower, respectively. We find the largest negative effects for boys and Hispanic students in the least safe schools, and no effect of neighborhood crime for students attending schools with better climates.

Despite the well-documented drop in violent crime in American cities, violence is part of daily life for many children. A growing body of research shows that exposure to neighborhood violence negatively affects academic performance, particularly among children living in high crime neighborhoods (Burdick-Will et al. 2011; Sharkey 2010; Sharkey et al. 2014; Schwartz et al. 2016). Whereas police, government officials, and civic organizations seek to reduce crime, schools can play a role in mitigating the negative effect of exposure to violence on students. Schools vary along many dimensions, including academic quality, student body characteristics, and school climate and environment. In some schools, disorder and conflict may contribute to feelings of fear and vulnerability among students; in other schools, students feel safe and supported. As reported in an article about New York City schools in the New York Times, “[school name] is more than just a place to learn algebra and history. A public middle school, it is seen by many families as a safe zone in a crime-plagued neighborhood” (Hu 2014). School climate, including how safe, orderly, and welcoming a school is perceived to be, may affect how youth are able to cope with traumatic events at home or in the residential community. Factors outside of school influence student success, yet little is known about whether school climate moderates the effects of these external events. In this article, we focus on the relationship between school climate and neighborhood crime and answer the question of whether school climate ameliorates or exacerbates the impact of neighborhood crime on academic performance.

To answer this question, we combine detailed administrative and survey data on neighborhood violent crime, student achievement, and student perceptions of school climate. Administrative data are key to our analysis. First, student-level data from the New York City Department of Education allow us to track test scores for the universe of public middle school students in New York City over time and observe their demographic characteristics and residential addresses. Second, incident-level crime data from the New York City Police Department (NYPD), which we geocode to individual street segments throughout the city, provide us with a daily measure of violence occurring on the blockfaces where those students reside. Finally, we use responses to an annual survey that the New York City Department of Education administers to all middle and high school students to construct measures of school climate that we link with students’ school records. This linkage provides us a unique look at how the impact of violence varies depending on school climate.

Our empirical approach capitalizes on the exogenous exposure of students to violent events in their neighborhood relative to the timing of standardized exams to estimate the causal acute effect of exposure to neighborhood violence on student outcomes (see Sharkey 2010; Sharkey et al. 2014). Within this framework, we examine how the acute effect of neighborhood crime varies with school climate, measured in different ways. We use factor analysis to combine middle school (grades six to eight) student responses to the New York City Learning Environment Survey and create three school-level scales that capture key constructs of school climate: school safety, disorder, and sense of community. We divide schools into quartiles for each of these dimensions, and estimate how the acute effect of neighborhood violence on English language arts (ELA) and mathematics test scores differs across schools with different climates.

To summarize our findings: students suffer declines in standardized test scores following exposure to a violent crime if they attend schools perceived as unsafe or having a weak sense of community. Specifically, middle school students exposed to violent crime before the test who attend schools that are less safe or have a weak sense of community score 0.06 and 0.03 standard deviations lower in the ELA exam, respectively. Students attending the schools perceived as being the safest, the least disorderly, or having the strongest sense of community suffer no visible reduction in test performance when exposed to violent crime, suggesting that schools with stronger climates might insulate students from the negative effects of neighborhood violence.

LITERATURE REVIEW

Living in a disadvantaged and dangerous neighborhood affects the lives of young people along multiple dimensions, including their health, education, and employment. A growing body of research highlights the effect of exposure to neighborhood violence on the academic attainment and achievement of students (Burdick-Will et al. 2011; Harding 2009; Rendón 2014; Sharkey 2010). In New York City, exposure to a violent assault or homicide in the week before a standardized exam decreases achievement in ELA relative to students who are exposed in the week after the exam (Sharkey et al. 2014).

School climate may also affect the academic performance of students (Thapa et al. 2013). Exposure to violence at school reduces attendance, decreases test scores, increases misbehavior, and reduces the likelihood of high school graduation and college attendance (Bowen and Bowen 1999; Burdick-Will 2013; Grogger 1997). Being the victim of an attack at school is associated with increased student misbehavior and declines in grades (Patton, Woolley, and Hong 2012). Even witnessing violence at school has consequences for student conduct, attitudes about school, and attendance (Janosz et al. 2008). School violence does not have to be extreme to have negative effects on students. Exposure to disorder in school, such as bullying, is negatively related to achievement and is also associated with more serious school violence (Arseneault, Bowes, and Shakoor 2006; Chen 2007; Juvonen, Wang, and Espinoza 2011). Schools may also be able to insulate students from violent neighborhoods by becoming a safe haven from the surrounding violence and disorder (Patton, Woolley, and Hong 2012).

The evidence suggests that four primary dimensions of school climate are likely to influence student performance: school-based violence and disorder, school safety, school discipline, and sense of community within the school. Specifically, in New York City, feeling unsafe at school decreases the academic achievement of middle school students, and the largest effects are found in schools with the most school-based violence (Lacoe 2016).

School disciplinary policy and student perceptions of the fairness of school discipline may also affect achievement. At the school level, the suspension rate is correlated with the share of students who pass competency exams (Raush and Skiba 2004). Youth who are suspended struggle to make academic progress over time and are more likely to drop out of high school than their peers who are not suspended (Arcia 2006; Lacoe and Steinberg 2018). Research has yielded mixed evidence of the efficacy of school security measures, such as metal detectors or police in schools. Some studies find these measures improve perceptions of school climate (Bhatt and Davis 2016). Others find decreases in perceived safety among students and increased involvement with the juvenile justice system (Theriot 2009). Therefore, school disciplinary policies that take zero tolerance approaches emphasizing out-of-school suspensions, or school security measures, may also affect student achievement if they make students feel less safe. Finally, the degree to which students feel connected to their school and a sense of belonging at school can affect their academic achievement. For instance, characteristics of the school environment influence students’ level of engagement and participation in school, which in turn may affect their academic achievement (Wang and Holcombe 2010).

These dimensions of school climate may affect youth differently depending on their racial and ethnic background or gender. Johanna Lacoe finds that African American and Hispanic middle school students are more likely to report feeling unsafe in the classroom and on school grounds than white and Asian peers who attend the same schools (2015). Patrick Sharkey and his colleagues find that neighborhood violence has the most pronounced effect on the achievement of black students, with little effect on Hispanic students, despite similar rates of exposure to neighborhood violence between the two groups (2014). Studies also suggest that boys and girls respond differently to school climate and neighborhood violence as well (Harding 2009).

In sum, the literature shows that community violence can be detrimental to students’ academic success. Further, research also suggests that the climate within a school (including safety, disorder and support levels) can shape students’ academic performance. This article bridges these two literatures to investigate whether and to what extent school climate moderates the effect of neighborhood crime on middle school students’ test scores.

THEORY

Several theorists have put forth models and frameworks to describe how multiple environments affect youth outcomes (Bronfenbrenner 2004; Eccles and Roeser 2011; Kirk 2009). In particular, Jacquelynne Eccles and Robert Roeser describe schools as prime developmental contexts for youth during adolescence. Schools are organizations with customs, norms, and rules that influence student interactions, learning, and development on a daily basis (Eccles and Roeser 2011). Schools, however, are not located in a vacuum, but instead are intimately connected to the surrounding neighborhoods, which for most middle school students are where they live. “By attending to the social structure of community conflict, whatever its spatial form, schools can understand, and possibly anticipate, the development of violent confrontations and possibly intervene to redirect the conflict to some other outcome” (Mateu-Gelabert and Lune 2003, 366). School climate may dictate how successfully schools manage youth responses to violence, distinguishing some schools as safe havens.

Building on this literature, our theoretical model in figure 1 describes how schools may play a moderating role in the relationship between exposure to community violence and educational outcomes. Schools may moderate this relationship if school factors change the magnitude or direction of the impact of violent crime on test scores (that is, insulate students from the full effect of exposure, or exacerbate the response to violence).

Figure 1.

Conceptual Model of the Role of School Climate

The primary school climate factors that may affect the relationship between neighborhood violent crime and academic outcomes are school-based violence and disorder, perceptions of safety, discipline, and school supports. The direction of the relationship, however, is unknown. Exposure to violence everywhere (at school and at home) may desensitize students and lessen the effects of neighborhood violence on outcomes. That is, for example, the effects of exposure to neighborhood crime on children attending schools that are perceived as unsafe or disorderly may be smaller or nonexistent. Alternatively, exposure to violence or disorder at school may compound the effect of violence in students’ home neighborhoods, causing them to perform poorly on exams, so that we would observe the largest test score losses after exposure to violent crime for children who feel unsafe at school. In contrast, if the school represents a safe haven from a violent home neighborhood, the effect of exposure to violence may be smaller.

Other aspects of the environment can also shape effects of neighborhood violence. If the disciplinary environment is strong and effective, it may support students who feel at risk in their home neighborhood. Likewise, a supportive, inclusive, and friendly school environment may insulate students from the negative effects of exposure to violence in their neighborhood. Conversely, if students feel little sense of community at the school or view the disciplinary environment as unfair or biased, then their academic performance may be more affected by violence they observe or experience outside of the school.

DATA AND MEASURES

We exploit three detailed sources of administrative data. First, we use point-specific crime data from the NYPD on daily violent crime—homicide and aggravated assault—occurrences in New York City from 2004 to 2010. The data contain the spatial coordinates of the crimes that we geocode to the blockface, a street segment between the two closest cross streets (figure A1), using ArcGIS.

Figure A1.

Blockface Geography

Second, we use longitudinal student administrative records from the city’s Department of Education. This dataset contains a wide range of student demographic characteristics including student race-ethnicity, gender, participation in special education, and receipt of free or reduced-price meals. It includes test scores on the ELA and mathematics standardized tests, as well as the school students attend and their residential address in each academic year (as of October). Our measure of exposure to neighborhood violence captures the number of violent crimes that occur on students’ residential blockface each year they remain enrolled in New York City public schools. Students who live on the same blockface are considered to be exposed to the same crimes. Note that we do not know if the child actually witnessed the crime. However, because the blockface is such a small geographic unit and homicides and violent assaults are serious offenses, it is likely that students will have either direct or indirect knowledge of the crime.

Third, to obtain measures of school climate we use student answers to the Learning Environment Survey collected by the New York City Department of Education. The survey is administered annually to students, parents and teachers in grades six to twelve during the spring semester (between mid-February and mid-March). In this article, we focus on all middle school students’ responses to the survey. The survey started in 2007, the first year of our panel.

Our analytic sample contains 16,146 students in 533 schools in grades six to eight from academic years 2006–2007 to 2009–2010. We restrict the sample to students living in high poverty census tracts who are exposed to violent crime within one week—seven days—of the ELA test. We focus on high poverty census tracts because it allows us to exclude sections of the city that have high crime rates but that are relatively wealthy, such as midtown Manhattan. Further, as prior work shows (Sharkey et al. 2014), exposure to violence has larger effects on these students, and thus understanding the role of school climate for these more vulnerable students seems especially important. We also exclude students in charter schools, in ungraded special education, and those exposed to crime both before and after the test.

Constructing School Climate Measures

To construct school climate measures we take both a theory- and data-driven approach similar to that of Matthew Kraft, William Marinell, and Darrick Yee (2016). First, we review questions in the Learning Environment Survey and select those that capture the four dimensions of school climate identified in the literature as important determinants of student outcomes: safety, disorder, sense of community, and disciplinary environment. We identify seventeen survey questions in these domains. For example, we use questions about feelings of safety in classrooms, in hallways and locker rooms, and on school grounds outside the school building. We also select questions related to bullying, fighting, substance use, gangs, perceptions of disciplinary fairness, conflict resolution, and the presence of safety agents. Finally, we also use questions about whether students feel welcome at school, treat each other with respect, or just look out for themselves. All responses consist of a scale from one to four. We code all responses so that an answer of one in the survey would be the best outcome, an answer of two would be the second best outcome, an answer of three would be the third best outcome, and an answer of four would be the worst outcome.

We use exploratory principal components factor analysis to identify whether student responses capture one overall measure of school climate or map into distinct climate dimensions. We rotate the factor loadings using oblique rotation because it assumes correlation among the factors instead of treating them as exogenous, and we expect the different climate measures to be correlated with one another. For example, a school that students perceive as highly disorderly is also likely to be perceived as unsafe. Rotating factor loadings maximizes the loadings for each factor, facilitating interpretability. After rotation, we find that the responses to the survey questions map onto three factors that capture three dimensions of school climate: safety, sense of community, and disorder and conflict. Contrary to expectations, there is no separate dimension for the disciplinary environment. The first factor (safety) explains 28 percent of the item variance, the second factor (sense of community) approximately 23 percent, and the third factor (disorder) 22 percent. Table 1 presents the relevant survey questions in each factor with the corresponding factor loadings. Consistent with the literature, we show those with factor loadings of 0.4 or greater.

Table 1.

Rotated Factor Loadings, School Climate Measures

	Factor 1: Safety	Factor 2: Sense of Community	Factor 3: Disorder-Conflict
I am safe on school property outside my school.	0.65
I am safe in the hallways, bathrooms, and locker rooms at my school.	0.68
I am safe in my classes.	0.76
Discipline in my school is fair.	0.62
There is a person or program in my school that helps students resolve conflicts.	0.71
The presence and actions of school safety agents help promote a safe and respectful learning environment.	0.50
I feel welcome at school.	0.58
Students threaten or bully other students at school.		0.51
Students get into physical fights at my school.		0.55
Most students in my school help and care about each other.		0.65
Most students in my school just look out for themselves.		0.66
Most students in my school treat each other with respect.		0.65
I stay home because I don’t feel safe at school.			0.67
Students use alcohol or illegal drugs while at school.			0.80
There is gang activity in my school.			0.71
Adults at my school yell at students.			0.51
There is conflict in my school based on race, culture, religion, sexual orientation, gender, or disabilities.			0.58

Source: Authors’ calculations using data from the New York City Learning Environment Survey.

Note: Results from factor analyses. Table shows factor loadings after oblique rotation. Loadings less than 0.4 are omitted.

To create school safety, disorder, and sense of community scales we compute factor scores for each student answering the surveys. To obtain school-level measures we follow Kraft, Marinell, and Yee and average those scores for each school across years to obtain time invariant school climate scales or indices (2016). The resulting scales are centered on zero. Higher values indicate weaker climates and lower values indicate stronger climates. Panel A of figure 2 plots the distribution of these scales across all the 593 schools in our data (including those not in high poverty census tracts). Overall, the disorder scale shows less variation: most values are concentrated around zero and below (less disorder) and no schools are perceived as having either very high levels or very low levels of disorder. Variation is greater in the sense of community measure, and the distribution is more skewed. Most schools are perceived to have a lower than average sense of community (indicated by positive values on the scale), but more schools are perceived as having very strong sense of community than as having a very weak sense. We observe a similar pattern for the school safety scale.

Figure 2.

Distribution of School Climate Measures Across Schools

The school climate in the schools the students in our sample attend might differ from the full sample of schools, but we find they look fairly similar. Panel B of figure 2 shows the distribution of school climate measures for the 533 schools in the analytical sample and demonstrates that these distributions do not look substantially different from the full sample.

EMPIRICAL STRATEGY

Our empirical strategy compares students exposed to violent crime within a one-week window around standardized testing dates. Specifically, we compare the test performance of students exposed to violent crime in the week before a standardized test with that of otherwise similar students exposed in the week after. The identifying assumption is that the occurrence of a violent crime on a student’s residential blockface one week before or after the test is conditionally random within this window. This strategy yields causal estimates of the acute effect of violent crime on test scores.

The empirical strategy relies on the assumption that students exposed to violent crime within a week of the ELA or math exam are very similar to each other. Students exposed to violent crime one week before and after the tests are quite similar demographically (see table A2). Hispanics make up the majority of students in all four samples (more than 50 percent), and black students represent approximately 40 percent. The samples are all evenly distributed between male and female students, and students receiving free or reduced-price lunch are overrepresented, as are students whose language at home is not English. To further test the assumption that students exposed to neighborhood crime before and after the test are similar, we estimate a regression model predicting exposure in the week before the ELA and math tests as a function of individual demographic characteristics. These models also include grade, year, and borough fixed effects. Results from a joint-F test on the demographic controls confirm that our sample is balanced, supporting the appropriateness of the identification assumption (table A2).

Table A2.

Student Characteristics by Exposure to Violent Crime, One-Week Window, High Poverty Sample

	ELA		MATH
	Before	After	Before	After
Race-ethnicity
Black	39.3	38.4	38.9	38.7
Hispanic	54.0	52.1	52.0	52.1
Asian	4.4	6.0	6.1	6.2
White	2.2	3.5	3.0	3.0
Gender
Female	50.3	50.6	51.8	50.4
Poverty status
Free or reduced-price lunch	94.9	93.9	94.7	94.6
Other demographics
Foreign born	14.6	16.7	17.6	17.5
Special education	11.7	11.8	11.9	11.1
Limited English proficiency	11.5	11.6	13.3	14.1
Home language not English	43.8	46.0	45.9	46.0
Overage for grade	16.5	15.7	15.8	14.8
Observations	9,071	7,075	7,941	8,735
F-stat	1.40		0.77
Prob>F	0.17		0.67

Source: Authors’ calculations using NYPD complaint data and New York City public schools student-level administrative data, provided to New York University and Syracuse University by the New York City Department of Education.

Note: Table includes column percentages. Variables included in F-test: black, Hispanic, Asian, female, free and reduced-price lunch participation, foreign born, special education, home language not English, limited English proficiency, overage for grade. Models include borough, grade, and year fixed effects. Standard errors are clustered at the school level. Sample includes students in grades six through eight between academic years 2006–2007 and 2009–2010.

We begin by estimating a baseline specification as shown in equation (1)

In this model, Test represents student i’s test score on the ELA or math exam, measured as a z-score standardized for each grade citywide with a mean of zero and a standard deviation of one. Crime takes a value of one if a student was exposed to a homicide or aggravated assault on their block in the week before the ELA test, and is zero if they were exposed in the week after the test. The coefficient of interest is β, and it can be interpreted as a causal estimate of the acute effect of exposure to violent crime. The model also includes a vector (X) of student demographic controls: gender, race-ethnicity, eligibility for free or reduced-price lunch, special education, limited English proficiency, foreign born, home language not English, and over age for grade. Grade fixed effects are γ , year fixed effects are a , and e is the usual error term. We follow this baseline specification by adding student i’s test scores lagged one year, thus controlling for a student’s prior performance.

To estimate whether school climate moderates the acute effect of neighborhood crime, we first divide schools into quartiles based on their scores on the three climate perception measures: safety, disorder, and sense of community. Schools in the first quartile are those with stronger climates (perceived as safe) and those in the fourth quartile are those with weaker climates (perceived as unsafe). Then, for each climate measure, we estimate our baseline model stratified by climate quartile. In this way, we compare the test performance of students exposed to violent crime on their block in the week before the test with those exposed in the week after attending schools that have similar climates. Specifically, we estimate as follows:

We extend our baseline specification by adding a set of interactions between the crime exposure indicator and each of the four climate quartiles. In this model, β yields estimates of the acute effect for each climate quartile by comparing students exposed before the test with those exposed after attending schools in the same quartile. For example, we compare the test performance of a student exposed to violent crime in the week before the ELA test with a similar student exposed in the week after, who both attend schools perceived as being the least safe (quartile four). If schools that are perceived as safer, or less disorderly, or with a stronger sense of community act as safe havens for students living in violent neighborhoods, we may see no difference in test performance between children exposed to homicides and violent assaults in the week before the test or after. Conversely, if schools with weaker climates (safety, disorder, and sense of community) exacerbate the effect of living in violent neighborhoods, we may see a decrease in performance after exposure to violent crime.

Schools with different climates may vary on a number of other characteristics that can influence both school climate and student performance. We examine the robustness of our main results through a series of tests. First, we add school-level, time-varying spending data, teacher-pupil ratio, and reported incidents of school violence to control for other school characteristics that might be correlated with student achievement or contribute to school climate. Second, we also estimate models with school fixed effects to control for time-invariant school characteristics that might be correlated both with perceptions of school climate and student achievement. Third, we conduct a falsification test using exposure to property crimes as our main crime exposure indicator. If students are, indeed, affected by neighborhood violent crime then we should see no effect of exposure to less serious property crimes on test scores. Finally, we test the robustness of our results using the full sample of students instead of the high poverty sample.

RESULTS

We begin by examining the demographic composition of schools with stronger and weaker climates for the three climate measures: safety, disorder, and sense of community. The most striking differences across quartiles concern the racial-ethnic composition of the students (figure 3, panel A). Black students are overrepresented in schools with weak climates (quartile four) across the three climate scales. Indeed, more than 50 percent of black students attend schools in quartile four across the three climate dimensions. In contrast, schools in quartile one are more than 60 percent Hispanic. We observe a similar pattern in schools with more mixed climates (quartiles two and three). Students who are white or Asian are the smallest group in the sample, and are also more likely to attend schools that are safer, less disorderly, and more community-oriented. As for gender, differences across quartiles are relatively small but girls are more likely to attend schools in the first quartile. This is a high poverty sample, thus more than 90 percent of students are eligible for free or reduced-price lunch, however, the percentage of poor students is slightly higher in quartile four schools than in the other three quartiles. Students whose home language is not English are less likely to attend schools with the weakest climates. In contrast, students who are over age for grade are overrepresented in quartile four schools across the three dimensions. Across all climate measures, schools are fairly similar in the percent of students with limited English proficiency, the share of foreign born, and those in special education.

Figure 3.

Student Characteristics, High Poverty Sample

Panel B of figure 3 shows differences for school spending categories: counseling, drug prevention programs, attendance and outreach, and school safety. Climate quartiles show no large differences for these school resources. In general, spending in counseling and attendance-outreach is lower in quartile one schools than in the other quartiles. As for other school resources (not shown), per pupil spending on classroom instruction is slightly higher in quartile three and four schools; pupil-teacher ratios and spending on school leadership are fairly similar across quartiles. Perhaps not surprisingly, school violence increases as we move from quartile one to quartile four schools across all dimensions.

Does the Acute Effect of Violent Crime on Test Scores Vary with the School Climate?

In the regression results that follow we show models with demographic controls, and with lagged test scores. In most cases these results are very similar to each other and we show all of them for completeness. In the discussion, however, we focus on those that account for a student’s prior performance as the preferred specification. Results from our baseline regression (table 2) show no overall average acute effect of exposure to violent crime on ELA test scores for middle school students (grades six and seven). That is, middle school students exposed to violent crime in the week before the ELA test perform no differently on average than comparable students exposed after. This average effect, however, masks significant variation across schools as figure 4 shows, suggesting that school-level factors might play a role in ameliorating or exacerbating the acute effect. Indeed, when we explore whether results vary across school climate quartiles, we find that the average estimate conceals significant heterogeneity by school climate.

Table 2.

Regression Results: Exposure to Neighborhood Violent Crime and Test Scores, One-Week Window, High Poverty Sample

	ELA		Math
DV: z-score	(1)	(2)	(3)	(4)
Crime	–0.015	–0.005	–0.015	–0.001
	(0.014)	(0.010)	(0.017)	(0.012)
Student controls	Y	Y	Y	Y
Lagged test scores	N	Y	N	Y
Observations	16,146	16,146	16,676	16,676
R2	0.229	0.459	0.193	0.532

Source: Authors’ calculations using NYPD complaint data and New York City public schools student-level administrative data, provided to New York University and Syracuse University by the New York City Department of Education.

Note: Standard errors in parentheses (clustered at the school level). Student controls include female, black, Hispanic, Asian, free lunch, reduced-price lunch, special education, limited English proficiency, foreign born, home language not English, over age for grade. All models include grade, year, fixed effects, and an indicator for missing lagged test scores. Sample includes students in grades six to eight between 2006–2007 and 2009–2010.

Figure 4.

School-Specific Random Slopes, Violent Crime

Table 3 shows that exposure to violence affects students in schools with the weakest climates (quartile four). Students attending schools deemed the least safe (quartile four) score 0.06 standard deviations lower as a result of exposure to violent crime one week before testing (column 2). Similarly, column 6 shows students in schools that have the weakest sense of community (quartile four) score 0.03 standard deviations lower after exposure to violent crime in the week before the test (significant at the 10 percent level). We do not find strong evidence of an acute effect for students attending more disorderly schools. In models not accounting for prior performance (column 3), students exposed to violent crime prior to the ELA test attending schools perceived as the most disorderly (quartile four) score 0.08 standard deviations lower (significant at the 10 percent level). Once we control for prior performance, this coefficient drops to –0.04 and is no longer statistically significant.

Table 3.

Regression Results: ELA, Exposure to Neighborhood Violent Crime, and School Climate Quartile

DV: z-score ELA	Safety		Disorder		Sense of Community
	(1)	(2)	(3)	(4)	(5)	(6)
Crime*Q1	–0.055	–0.032	–0.051	–0.002	–0.073	–0.019
	(0.038)	(0.029)	(0.039)	(0.031)	(0.055)	(0.050)
Crime*Q2	0.015	0.021	–0.006	–0.000	0.014	0.019
	(0.025)	(0.019)	(0.026)	(0.019)	(0.031)	(0.025)
Crime*Q3	–0.010	0.002	–0.009	–0.007	–0.005	0.007
	(0.022)	(0.016)	(0.017)	(0.014)	(0.021)	(0.016)
Crime*Q4	–0.070*	–0.060*	–0.080+	–0.042	–0.039+	–0.031+
	(0.030)	(0.024)	(0.044)	(0.036)	(0.021)	(0.016)
Q1	0.274**	0.156**	0.425**	0.211**	0.456**	0.216**
	(0.058)	(0.035)	(0.061)	(0.043)	(0.083)	(0.046)
Q2	0.057	0.019	0.189**	0.108**	0.121**	0.052+
	(0.040)	(0.026)	(0.049)	(0.037)	(0.042)	(0.027)
Q3	–0.020	–0.016	0.041	0.037	0.014	–0.006
	(0.043)	(0.028)	(0.043)	(0.034)	(0.034)	(0.022)
Student controls	Y	Y	Y	Y	Y	Y
Lagged test scores	N	Y	N	Y	N	Y
Observations	16,146	16,146	16,146	16,146	16,146	16,146
R2	0.241	0.463	0.252	0.465	0.245	0.463

Source: Authors’ calculations using NYPD complaint data and New York City public schools student-level administrative data, provided to New York University and Syracuse University by the New York City Department of Education.

Note: Standard errors in parentheses (clustered at the school level). Student controls include: black, Asian, Hispanic, free and reduced-price lunch, special education, limited English proficiency, foreign born, home language not English, and over age for grade. All models include grade and year fixed effects, and an indicator for missing lagged test scores. Sample excludes students exposed both before and after the ELA test. Sample includes students in grades six through eight between academic year 2006–2007 and 2009–2010.

+p < .1; *p < .05; **p < .01

Results in this section indicate that safety and sense of community are the most critical elements of school climate in moderating the effects of violence. Our findings suggest that schools have the potential to insulate students from the negative effects of exposure to neighborhood violence on academic performance, shown by the absence of any negative effect on test scores for students attending schools with strong (quartile one) or mixed climates (quartiles two and three).

Subgroup Analyses

Of course, school climate may not matter equally for all students; some students may be more sensitive to the climate of their school. In this section, we explore whether the moderating effect of school climate varies by gender and race-ethnicity. Gender analyses are motivated by the fact that boys and girls may use different coping mechanisms to deal with traumatic events, and may respond differently within these varied school climates (Rasmussen, Aber, and Bhana 2004). As for race-ethnicity, research has shown significant racial differences in sensitivity to violence. In particular, Sharkey and his colleagues find that the school performance of black students is especially sensitive to violent environments, and that exposure to violence has little effect on the academic performance of Hispanic students (2014). It is possible that differences in the school environments experienced by black and Hispanic students (and the concentration of black students in schools with weaker climates) explain these differences. In the analyses that follow, we focus on schools with weak climates (quartile four).

Our results suggest that the negative effect of exposure to neighborhood crime is concentrated among boys attending weak climate schools. As panel A of table 4 shows, boys exposed to violent crime in the week before the ELA exam score approximately 0.10 standard deviations lower when they attend the least safe schools. They score 0.08 standard deviations lower (significant at the 10 percent level) in the most disorderly schools, and 0.04 standard deviations lower in schools with weaker sense of community (also significant at the 10 percent level). There is no observed effect for girls, but these coefficients are not statistically different than those for boys.

Table 4.

Regression Results: ELA, Quartile 4 Schools by Subgroup

DV: z-score ELA	Safety		Disorder		Sense of Community
	(1)	(2)	(3)	(4)	(5)	(6)
Panel A: Gender
Crime*female	–0.043	–0.025	–0.064	0.000	–0.044	–0.023
	(0.041)	(0.030)	(0.058)	(0.053)	(0.026)	(0.019)
Crime*male	–0.094*	–0.100**	–0.094	–0.082+	–0.036	–0.041+
	(0.043)	(0.036)	(0.060)	(0.046)	(0.029)	(0.021)
Female	0.062	0.019	0.104	0.040	0.109**	0.060**
	(0.047)	(0.036)	(0.072)	(0.055)	(0.026)	(0.020)
Observations	2,550	2,550	1,013	1,013	6,244	6,244
R2	0.174	0.448	0.193	0.423	0.190	0.433
Panel B: Race-ethnicity
Crime*black	–0.066+	–0.054+	–0.047	0.014	–0.037	–0.026
	(0.038)	(0.030)	(0.048)	(0.036)	(0.027)	(0.021)
Crime*Hispanic	–0.081	–0.091+	–0.105	–0.091	–0.037	–0.040
	(0.059)	(0.050)	(0.092)	(0.083)	(0.031)	(0.026)
Black	0.008	–0.025	–0.129	–0.132*	0.027	0.018
	(0.060)	(0.046)	(0.079)	(0.058)	(0.037)	(0.028)
Observations	2,470	2,470	969	969	6,016	6,016
R2	0.169	0.449	0.183	0.419	0.184	0.431
Student controls	Y	Y	Y	Y	Y	Y
Lagged test scores	N	Y	N	Y	N	Y

Source: Authors’ calculations using NYPD complaint data and New York City public schools student-level administrative data, provided to New York University and Syracuse University by the New York City Department of Education.

Note: Standard errors in parentheses (clustered at the school level). Student controls include black, Asian, Hispanic, free or reduced-price lunch, special education, limited English proficiency, foreign born, home language not English, and over age for grade. All models include grade and year fixed effects, and an indicator for missing lagged tests scores. Sample includes students in grades six through eight between academic year 2006–2007 and 2009–2010. Panel B sample excludes students who are Asian or white.

+p < .1; *p < .05; **p < .01

Next, we explore differences by race and ethnicity. Due to sample size limitations, these models include only Hispanic and black students. Black students exposed to violent crime before the ELA test attending schools perceived to be the least safe, score 0.05 standard deviations lower on the ELA exam (table 4, panel B). The largest effect, however, is for Hispanic students in the least safe schools. These students score 0.09 standard deviations lower. This finding suggests that stronger school climates might offer some protective effect for Hispanic students. We observe no effect of exposure to violence on test scores among black and Hispanic students who attend more disorderly schools or schools with the weakest sense of community. Note that the samples in the disorder measure get very small, so we may be underpowered to detect an effect. Coefficients for black and Hispanic students are not statistically different from each other and, in general, results in this table are only significant at the 10 percent level.

Robustness and Falsification Tests

The results so far show that school climate may play an important role in moderating the effects of neighborhood violence on student outcomes. However, there may be school-level confounders that bias these results. School climate measures may reflect differences in other factors that are correlated with student perceptions and achievement. To address this issue, we add several additional sources of administrative data to construct time-varying school-level controls. First, we combine our student-level data with the school-based expenditure reports. These data provide detailed information regarding school spending by budget item as well as pupil-teacher ratios. We select spending on classroom instruction, leadership, and relevant instructional support spending categories (counseling services, drug prevention programs, attendance-outreach, and school safety). Second, we add the rate of reported school violent incidents from the violent and disruptive incident reports data. As an additional test, we reestimate our models adding school fixed effects to control for time-invariant characteristics of schools that may be correlated both with school climate and student achievement.

In these models, our results remain largely unchanged (table A3). Students in the least safe schools exposed to violent crime in the week before the ELA test score 0.056 standard deviations lower than those exposed after. As for the other measures, results are not statistically significant, but point estimates are similar as our main specifications (–0.035 for disorder and –0.021 for sense of community). Taken together, these results support our finding that school climate matters for children exposed to violent crime, but we cannot completely rule out that other unobserved school-level factors might still be at play.

Table A3.

Robustness Test: ELA, School Climate Specification with School-Level Controls

	Safety		Disorder		Sense of Community
DV: z-score ELA	(1)	(2)	(3)	(4)	(5)	(6)
Crime*Q1	–0.026	–0.018	0.009	0.021	–0.024	0.007
	(0.029)	(0.032)	(0.033)	(0.034)	(0.051)	(0.055)
Crime*Q2	0.029	0.025	0.004	–0.007	0.038	0.033
	(0.020)	(0.020)	(0.020)	(0.021)	(0.026)	(0.028)
Crime*Q3	0.004	–0.001	–0.005	–0.007	0.007	–0.009
	(0.016)	(0.018)	(0.014)	(0.016)	(0.016)	(0.018)
Crime*Q4	–0.055*	–0.056*	–0.043	–0.035	–0.026	–0.021
	(0.025)	(0.026)	(0.037)	(0.040)	(0.017)	(0.017)
Q1	0.129**		0.178**		0.165**
	(0.034)		(0.043)		(0.046)
Q2	0.011		0.093*		0.022
	(0.027)		(0.037)		(0.028)
Q3	–0.010		0.043		–0.018
	(0.028)		(0.034)		(0.023)
Student controls	Y	Y	Y	Y	Y	Y
Lagged test scores	Y	Y	Y	Y	Y	Y
School resources	Y	Y	Y	Y	Y	Y
School violence incidents	Y	Y	Y	Y	Y	Y
School FX	N	Y	N	Y	N	Y
Observations	15,032	15,032	15,032	15,032	15,032	15,032
R2	0.467	0.500	0.467	0.500	0.467	0.500

Source: Authors’ calculations using NYPD complaint data and New York City public schools student-level administrative data, provided to New York University and Syracuse University by the New York City Department of Education.

Note: Standard errors in parentheses (clustered at the school level). Student controls include black, Asian, Hispanic, free and reduced-price lunch, special education, limited English proficiency, foreign born, home language not English, and over age for grade. All models include grade and year fixed effects and an indicator for missing lagged test scores. School controls include natural log of per pupil spending in classroom instruction, leadership, attendance/outreach, drug prevention programs, counseling, school safety, and pupil teacher ratio. School violence rate per 1,000 students. All spending variables are inflation adjusted using the 2010 CPI. Sample includes students in grades six through eight, academic years 2006–07 and 2009–10.

+p < .1; *p < .05; **p < .01

If violent crimes are more salient and the key source of stress for students and not simply capturing other things happening in the neighborhood, we should see little change in school performance after exposure to property crimes (for results of this falsification test, see table A4). Indeed, we find no evidence that exposure to property crime affects test scores, or that this effect varies with the climate of the school.

Table A4.

Falsification Test: ELA, Exposure to Property Crime and School Climate Quartile

	Safety	Disorder	Sense of Community
DV: z-score ELA	(1)	(2)	(3)
Crime*Q1	–0.001	–0.014	0.013
	(0.016)	(0.018)	(0.029)
Crime*Q2	–0.000	0.005	–0.002
	(0.008)	(0.008)	(0.011)
Crime*Q3	–0.002	–0.005	–0.007
	(0.009)	(0.008)	(0.009)
Crime*Q4	0.001	0.024	0.003
	(0.012)	(0.015)	(0.008)
Q1	0.185**	0.300**	0.250**
	(0.027)	(0.030)	(0.041)
Q2	0.080**	0.145**	0.104**
	(0.018)	(0.025)	(0.022)
Q3	0.022	0.069**	0.034*
	(0.018)	(0.023)	(0.016)
Student controls	Y	Y	Y
Lagged test scores	Y	Y	Y
Observations	66,626	66,626	66,626
R2	0.482	0.485	0.483

Source: Authors’ calculations using NYPD complaint data and New York City public schools student-level administrative data, provided to New York University and Syracuse University by the New York City Department of Education.

Note: Standard errors in parentheses (clustered at the school level). Student controls include: black, Asian, Hispanic, free and reduced-price lunch, special education, limited English proficiency, foreign born, home language not English, and over age for grade. All models include grade and year fixed effects, and an indicator for missing lagged test scores. Sample excludes students exposed both before and after the ELA test. Sample includes students in grades six through eight between academic years 2006–2007 and 2009–2010.

+p < .1; *p < .05; **p < .01

The primary results are estimated on a high poverty sample. To test the robustness of our findings we estimate the school climate specifications on the full sample of students. Overall, our conclusions are unchanged (table A5). Exposure to violent crime before the ELA test lowers test scores of students in schools deemed least safe by 0.04 standard deviations. This coefficient is smaller but within the confidence interval of the estimate for the high poverty sample. We also find that children exposed to neighborhood crime who attend schools that have a weak sense of community score 0.03 standard deviations lower in ELA (all significant at the 10 percent level).

Table A5.

Robustness Test: ELA, Exposure to Neighborhood Violence and School Climate Quartile, Full Sample

	Safety		Disorder		Sense of Community
DV: z-score ELA	(1)	(2)	(3)	(4)	(5)	(6)
Crime*Q1	–0.053	–0.030	–0.033	0.003	–0.070	–0.032
	(0.038)	(0.028)	(0.039)	(0.030)	(0.056)	(0.048)
Crime*Q2	0.020	0.029	–0.009	0.004	0.011	0.022
	(0.024)	(0.018)	(0.025)	(0.018)	(0.030)	(0.022)
Crime*Q3	–0.013	–0.002	–0.007	–0.009	–0.003	0.011
	(0.022)	(0.016)	(0.017)	(0.013)	(0.021)	(0.016)
Crime*Q4	–0.046	–0.042+	–0.060	–0.020	–0.034+	–0.027+
	(0.029)	(0.022)	(0.039)	(0.031)	(0.020)	(0.015)
Q1	0.285**	0.157**	0.444**	0.229**	0.486**	0.243**
	(0.055)	(0.034)	(0.057)	(0.040)	(0.081)	(0.045)
Q2	0.067+	0.020	0.203**	0.116**	0.128**	0.051*
	(0.037)	(0.024)	(0.045)	(0.032)	(0.041)	(0.025)
Q3	–0.002	–0.010	0.065+	0.053+	0.015	–0.007
	(0.040)	(0.026)	(0.037)	(0.029)	(0.032)	(0.021)
Student controls	Y	Y	Y	Y	Y	Y
Lagged test scores	Y	Y	Y	Y	Y	Y
Observations	18,254	18,254	18,254	18,254	18,254	18,254
R2	0.252	0.469	0.263	0.471	0.258	0.470

Source: Authors’ calculations using NYPD complaint data and New York City public schools student-level administrative data, provided to New York University and Syracuse University by the New York City Department of Education.

Note: Standard errors in parentheses (clustered at the school level). Student controls include black, Asian, Hispanic, free and reduced-price lunch, special education, limited English proficiency, foreign born, home language not English, and over age for grade. All models include grade and year fixed effects, and an indicator for missing lagged test scores. Sample excludes students exposed both before and after the ELA test. Sample includes students in grades six through eight, academic years 2006–2007 and 2009–2010.

+p < .1; *p < .05; **p < .01

Results by gender and race-ethnicity are also robust to using the full sample of students. Schools that are perceived as unsafe and having a weaker sense of community seem to exacerbate the negative effect of neighborhood crime on boys. Point estimates are smaller (–0.06 and –0.04, respectively) and significant at the 10 percent level only. As for race-ethnicity, results on the full sample are also consistent with the high poverty sample (see table B7 of the online appendix).

Mathematics

So far we have only reported results for the ELA test because research shows the largest effects of community violence on reading and no effects on math (Sharkey et al. 2014; Schwartz et al. 2016). We also estimate our baseline model with math test scores as the outcome. The baseline specification—without stratifying the sample by school climate—shows no significant impact of crime exposure on math performance (table 2). Stratifying the sample by school climate quartile for each of the climate measures also yields no significant differences in test performance between students exposed to violent crime before the math test and students exposed after. That is, school climate does not moderate the effect of neighborhood violence on students’ math test scores, providing further evidence that community violence tends to affect performance in reading but not math (table 5).

Table 5.

Regression Results: MATH, Exposure to Neighborhood Violent Crime, and School Climate Quartile

DV: z-score MATH	Safety		Disorder		Sense of Community
	(1)	(2)	(3)	(4)	(5)	(6)
Crime*Q1	0.005	0.006	–0.040	–0.034	–0.040	–0.037
	(0.055)	(0.049)	(0.046)	(0.042)	(0.078)	(0.069)
Crime*Q2	0.005	0.010	0.003	0.010	–0.026	–0.024
	(0.026)	(0.023)	(0.028)	(0.025)	(0.042)	(0.038)
Crime*Q3	–0.038	–0.017	–0.011	–0.000	0.018	0.032
	(0.024)	(0.022)	(0.022)	(0.022)	(0.023)	(0.022)
Crime*Q4	–0.004	–0.009	–0.027	–0.020	–0.033	–0.023
	(0.043)	(0.037)	(0.041)	(0.037)	(0.025)	(0.023)
Q1	0.349**	0.263**	0.565**	0.407**	0.568**	0.399**
	(0.072)	(0.057)	(0.073)	(0.063)	(0.081)	(0.063)
Q2	0.225**	0.175**	0.235**	0.167**	0.279**	0.226**
	(0.050)	(0.041)	(0.062)	(0.053)	(0.057)	(0.045)
Q3	0.068	0.046	0.056	0.043	0.129**	0.104**
	(0.050)	(0.040)	(0.057)	(0.050)	(0.037)	(0.029)
Student controls	Y	Y	Y	Y	Y	Y
Lagged test scores	N	Y	N	Y	N	Y
Observations	16,676	16,676	16,676	16,676	16,676	16,676
R2	0.210	0.339	0.226	0.345	0.217	0.342

Source: Authors’ calculations using NYPD complaint data and New York City public schools student-level administrative data, provided to New York University and Syracuse University by the New York City Department of Education.

Note: Standard errors in parentheses (clustered at the school level). Student controls include black, Asian, Hispanic, free and reduced-price lunch, special education, limited English proficiency, foreign born, home language not English, and over age for grade. All models include year and grade fixed effects, and an indicator for missing lagged test scores. Sample exclude students exposed both before the math test. Sample includes students in grades six through eight between academic year 2006–2007 and 2009–2010.

+p < .1; *p < .05; **p < .01

DISCUSSION

This article investigates the role of school climate on the relationship between exposure to neighborhood violence and academic achievement for middle school students (grades six through eight) in New York City public schools. To do so, we leverage several sources of administrative data that provide advantages for this kind of analysis. Most notably, by using data on the entire city public school system we are able to generate more precise estimates than most of the literature using survey data with much smaller samples. As a result, we are able to focus our attention on very specific windows of time around public school assessments, and to make comparisons among students living in individual blockfaces within the city. By combining multiple administrative datasets, including student records, incident records from the NYPD, and school climate surveys, we are able to make progress in understanding the mechanisms by which violence in the residential environment affects students’ performance in school. Merging together multiple datasets that cover the entire city and all of its public schools allows for an analysis that would not be possible with any single source of data.

The results from our analysis provide a more nuanced picture of the impact of violence than shown by previous research. Overall, we find no significant acute effect of exposure to violence for the sample of sixth- to eighth-grade students in high poverty neighborhoods. This finding, however, masks the substantial variation in effects found in schools with different levels of disorder, safety, and sense of community. Schools with strong climates (across all dimensions—safety, disorder, and sense of community) and those with mixed climates (quartiles two and three) may insulate students from the negative effects of exposure to neighborhood violence. It follows that students experiencing decreases in ELA test scores after exposure to neighborhood violence are concentrated in schools with the weakest climates, particularly those perceived as the least safe. Specifically, students exposed to community violence before the test attending the least safe schools score 0.06 standard deviations lower, which amounts to 40 percent of the test score gap between poor and nonpoor students in our sample. For these students, attending a school with a weak climate further increases their academic disadvantage.

The analyses by race-ethnicity and gender uncover that the effect of exposure to violence is particularly salient for boys and Hispanic students in schools deemed the least safe. This last finding is interesting in light of previous research that found no effect of neighborhood crime on the test performance of Hispanic students (Sharkey et al. 2014). Indeed, it seems that although the majority of Hispanic students attend schools with strong climates, those in schools with weak climates see large declines in achievement following exposure to violence. Thus, the null average effect from previous studies obscured the finding that students in schools with weak climates may be particularly affected by neighborhood violent crime.

The magnitudes of the effects are significant. For example, a 0.10 standard deviation decrease in test scores for boys represents a 33 percent decline relative to the mean for boys in the sample. To put these numbers in context, the magnitude of this effect is comparable to the positive gains from school-level interventions such as class sizes (Chingos 2013).

We find no effects on math. This finding, while still puzzling, is not surprising. Previous work in New York City shows effects of neighborhood violence are concentrated on ELA scores (Sharkey et al. 2014; Schwartz et al. 2016). Differing psychological and cognitive processes may be involved in learning reading and math concepts, and these may be differentially affected by acute stress resulting from neighborhood violence. For example, Sharkey and his colleagues find that exposure to homicides is linked with lower attention and impulse control, which is especially important for reading instruction (Sharkey et al. 2012; Liew et al. 2008). Further, the development of language skills is more dependent on home factors, whereas math tends to be more influenced by school-level mechanisms (Bryk and Raudenbusch 1988).

We note a few key limitations in the empirical work. First, the estimation strategy provides strong identification of the acute effect of exposure to violence but does not provide evidence of whether or not this is a testing effect of it has long-term consequences for learning. Further, this article does not speak to the effects of repeated or cumulative exposure to neighborhood violence or how schools might respond. Understanding these longer-term effects may illuminate potential interventions aimed at children experiencing chronic exposure. That said, the acute effects are important, in and of themselves, due in part to the reliance on standardized tests for decisions about retention, high school admissions, or program participation for middle school students. Second, although we find no effect of exposure to violence on test scores for students at schools with strong or mixed climates, this does not imply that exposure to violence has no effect on these students. Exposure to violence may manifest in the lives of children in other ways that are not captured by test scores in the short run. Further work should investigate the effect on other outcomes, such as absenteeism, behavioral problems, or disciplinary referrals, to gain a broader perspective on how neighborhood violence affects students. Examining such outcomes would also provide insight into the mechanisms underlying the decreases in test scores for students in schools with weak climates.

Finally, more work could be done to unpack what is captured by our measures of school climate. For example, we are unable to say anything in this article about teacher quality or teacher experiences in these schools, and how their views and actions might shape school climate. As other articles in this issue suggest, the school environment is complex and factors such as teacher’s views on issues like diversity and cultural competencies (Penner et al. 2019) or school organizations such as PTAs (Murray et al. 2019) might be important determinants of school climate. Further, although schools with weaker climates have more reported incidents of violence, school climate is more than a reflection of violence in the school or in the neighborhood. Our climate constructs capture overall perceptions of school climate, but we cannot fully measure what particular factors contribute to a stronger (or weaker) school climate.

In sum, although schools are unable to control the experiences students have beyond their walls, the climate within each school can play a role in helping students cope with external forces. This article provides suggestive evidence that many schools are safe havens for young people who live in dangerous neighborhoods, insulating them from the acute effect of exposure to violence on achievement. More research is needed to understand how schools successfully foster strong climates along multiple dimensions, to identify strategies to improving school climate, and to measure how changes in school climate affect other student outcomes.

Table A1.

Response Rates, School Climate Questions

Year	Students	Mean
2007	141,897	0.52
2007	186,700	0.81
2009	181,936	0.79
2010	186,463	0.81

Source: Authors’ calculations using data from the New York City Learning Environment Survey.

Note: Mean response rate indicates share of students with no missing responses on all seventeen questions used in the school climate scales.