Exploring the Potential Campus-Level Impact of Online Universal Sexual Assault Prevention Education.

Campus sexual assault is a long-standing challenge and continues to be a severe problem for American higher education. The present study examines the short-term impact of a widely utilized sexual violence prevention course for matriculating college students as a population-level prevention approach. The course focuses on correcting misperceptions of normative behavior, increasing students' likelihood to intervene in disconcerting situations, and encouraging empathy and support for victims. Participants were 167,424 first-year college students from 80 four-year institutions who completed preintervention and postintervention surveys to assess changes in composite factor scores derived from 20 attitudinal, self-efficacy, and behavioral intention items. Employing the composite factor scores as dependent variables, individual ANOVAs were run for each of the institutions to test whether there were significant increases in mean factor scores. High percentages of institutions saw statistically significant increases (p < .05) in self-reported ability and intention to intervene to prevent sexual assault and relationship violence (98%), empathy and support for victims (84%), and corrected perceptions of social norms (75%). Fewer schools saw significant reductions in endorsement of sexual assault myths (34%). These findings suggest that when implemented as a population-level intervention for all first-year college students, the prevention course can foster accurate perceptions of positive social norms, increase empathy and support for victims, and increase students' stated ability and intention to intervene.

Sexual violence continues to be prominently featured in the national spotlight as news coverage and research involving high-profile cases with celebrities, athletes, politicians, and college campuses make headlines (Dockterman, 2017; Osborne, Sherry, & Nicholson, 2016; Terán & Emmers-Sommer, 2018; Zimmerman, 2016). In the United States, about one in three women and nearly one in six men experienced some form of sexual violence during their lifetime (Smith et al., 2017). There has been increased pressure on institutions of higher education to address this issue (Mangan, 2015). While sexual violence is not just a higher education issue, colleges and universities have a unique opportunity to provide prevention education to young people who attend their institutions.

Campus sexual assault is a long-standing challenge and continues to be a severe public health need for American higher education. Approximately 20% of college women and 5% of men experience some form of sexual assault during their time on campus (Krebs, Lindquist, Warner, Fisher, & Martin, 2008). Prevalence rates are higher among nonmajority and marginalized populations, including persons with disabilities (Elliott & Pick, 2015), certain racial/ethnic groups (Langhinrichsen-Rohling, Selwyn, & Rohling, 2012), and those who identify as lesbian, gay, bisexual, or transgender (Johnson, Matthews, & Napper, 2016).

With sexual assault survivors and student activists demanding accountability (Hoffman, 2016), the U.S. Department of Education took unprecedented steps to expand federal regulations and to investigate institutions allegedly in violation of Title IX, which prohibits sex-based discrimination at educational institutions, including sexual harassment and abuse. In response, campus administrators have intensified efforts to comply with both Title IX and the Clery Act, federal legislation that demands campus communities be fully informed regarding issues of public safety and crime prevention (Hoffman, 2016).

While there has been considerable focus on how incidents should be handled after they occur, these political developments have also increased pressure for campuses to develop, evaluate, and apply best practices for sexual assault prevention. To date, however, there is relatively little peer-reviewed research on the efficacy of sexual assault prevention programs for college students. Experts in the field recommend bystander intervention and correcting misperceived norms as promising practices (Lonsway et al., 2009), and a number of recent studies have reported encouraging results for such programs, showing changes in student attitudes and behavioral intentions (Banyard, Moynihan, & Plante, 2007; Coker et al., 2016; Coker et al., 2015), and, in some cases, actual reductions in reported perpetration of assault (Gidycz, Orchowski, & Berkowitz, 2011; Salazar, Vivolo-Kantor, Hardin, & Berkowitz, 2014).

College sexual assault prevention has traditionally been provided through workshops, lectures, and social marketing campaigns. More recently, a number of online courses have been developed that are offered primarily to entering students, but there is almost no literature evaluating their effectiveness. This study attempts to address this gap by evaluating Haven—Understanding Sexual Assault™, a popular online sexual assault prevention course offered by the education technology company, EVERFI, Inc., which is currently being used by more than 600 institutions of higher education.

Correcting Misperceptions of Social Norms

A vast research literature has established that perceptions of social norms—whether accurate or inaccurate—are a major influence on people’s behavior (Berkowitz, 2005; Berkowitz, 2010; Neighbors, Lee, Lewis, Fossos, & Larimer, 2007; Perkins, 2003). Regarding sexual assault, studies have shown that both male and female college students overestimate the prevalence of problematic attitudes and behavioral intentions among their peers, including endorsement of “rape myths” and a willingness to have sex without a partner’s consent. In addition, male students vastly underestimate other men’s positive attitudes and behavioral intentions, including their discomfort with misogynistic remarks and their willingness to intervene to prevent sexual assault or relationship violence (for a review of this research, see work by Berkowitz, 2010). These misperceptions negatively influence students’ attitudes, beliefs, and behaviors, including rape proclivity, abusive relationship behaviors, and a reluctance to intervene as bystanders. Importantly, an emerging literature also documents that perpetrators’ misperception of norms supports their self-justification for committing sexual assault (Dardis, Murphy, Bill, & Gidycz, 2016; Eyssel, Bohner, & Siebler, 2006).

Social norms interventions aim to address two principle types of misperceptions: pluralistic ignorance and false consensus. Pluralistic ignorance is the incorrect belief that one’s private attitudes, judgments, or behaviors are different from what other people think or do, which typically arises from a lack of information-sharing and open communication (Toch & Klofas, 1984). This type of misperception is common among individuals with the most positive attitudes and behaviors, who believe they are in the minority when they are actually in the majority. These individuals, if they encounter a situation where they think an expressed attitude or behavior is problematic but no one else speaks up or intervenes, incorrectly assume that others do not share their perspective and, therefore, are inhibited from acting on their concern.

In contrast, false consensus is the incorrect belief that one’s attitudes, judgments, or behaviors are representative of the majority when, in fact, they are not (Ross, Greene, & House, 1977). This type of misperception is often harbored by individuals with unhealthy attitudes and behaviors, who are emboldened to engage in negative behaviors when they overestimate the extent of peer support for their actions (Berkowitz, 2005).

Social norms interventions have been shown to reduce misperceptions across a variety of domains, including alcohol use (Berkowitz, 2005; Neighbors et al., 2007; Perkins, 2003) and sexual assault (Berkowitz, 2010), which in turn helps reduce perceived normative pressures to engage in unhealthy, high-risk, or even illegal behaviors, and perceived social barriers against bystander intervention (Berkowitz, 2013).

Bystander Intervention Approaches

Encouraging bystander intervention has gained increasing traction as a sexual assault prevention strategy (Lonsway et al., 2009). Because most college students endorse positive attitudes and behaviors, this “bystander approach” allows educators to reach students not as potential victims or perpetrators but as contributors who can take action to promote a safe and healthy campus community. These programs are informed by the five-step cognitive model for bystander intervention originally outlined by Latané and Darley (1989). To intervene, a person must (a) take note of the situation, (b) determine that the situation warrants intervention, (c) feel personally responsible for intervening, (d) believe they have the ability to intervene successfully, and (e) consciously decide to help. There are several factors that can deter people from intervening (Schroeder, Penner, Dovidio, & Piliavin, 1995), including the ambiguity of a situation; the presence of other, nonengaged bystanders; low self-efficacy; and a fear of embarrassment, all of which can be magnified when people are not aware of or misperceive widely shared prosocial norms.

Online Prevention Programs

An increasing variety of training options have become available for educating college students about important health and safety issues. In addition to (but unfortunately sometimes entirely in place of) in-person trainings, many institutions use online prevention programs because of their cost-efficiency, reliability, and scalability compared with other approaches. Delivery is standardized, and the content is often aligned with compliance requirements. Other program features may include participation tracking and built-in tools to collect evaluation data.

Most importantly, online prevention education programs have been shown to work. With respect to college student drinking, multiple studies have demonstrated the potential for online programs to mitigate both alcohol use and negative drinking consequences of incoming students (Hustad, Barnett, Borsari, & Jackson, 2010; Lovecchio, Wyatt, & DeJong, 2010; Paschall, Antin, Ringwalt, & Saltz, 2011a, 2011b; Wyatt, DeJong, & Dixon, 2013). Importantly, the strength of these positive outcomes is directly tied to implementation rates (Paschall et al., 2011a, 2011b). For that reason, these programs are often offered to students before they arrive on campus, when they are more willing to comply with requests from campus administrators.

A recent evaluation study provides initial support for the efficacy of online sexual assault prevention courses for college students, in particular when they combine the correction of misperceived social norms with bystander intervention training. Salazar and colleagues (2014) found that RealConsent, a web-based course for college men, increased knowledge, improved positive attitudes and expectancies, decreased negative attitudes and rape myth acceptance, increased intentions to intervene, and reduced self-reported violence perpetration. The benefit of combining these two approaches was further demonstrated in a study of group training sessions for college men, which found a 75% reduction in self-reported sexual aggression at 4-month follow-up (Gidycz et al., 2011). These studies suggest that there may be a synergistic effect when social norms and bystander intervention education are combined. Although the availability and use of online sexual assault prevention programs has increased in recent years, to date, there are no additional publications on their impact.

The focus of the present study is Haven—Understanding Sexual Assault, a widely used online sexual assault prevention course for college students. This two-part, 1-hr program was designed for incoming first-year students to provide prematriculation education on healthy relationships, sexual consent, individual risk and protective factors, sociocultural contributors to sexual assault, and additional content related to federal government requirements (e.g., campus-specific definitions and policies, support and reporting resources). Culminating course activities focus on bystander intervention by walking students through scenario-based exercises that align with Latané and Darley’s (1989) five-step cognitive model. Reinforcing social norms content and statistics are provided to demonstrate disconnect between actual and perceived norms related to bystander behavioral intentions and respect for students who intervene. The course was designed for and is typically administered to incoming first-year students, though ongoing students can benefit from it as well.

The present study analyzes pre- and postintervention surveys to examine the short-term impact of the course as a population-level prevention approach. We hypothesized that the course would increase accurate perceptions of positive social norms, reduce endorsement of sexual assault myths, increase empathy and support for victims, and increase students’ ability and intention to intervene to prevent sexual assault and relationship violence.

Method

Haven—Understanding Sexual Assault is a digital sexual assault prevention education course designed as a population-level prevention strategy to shift perceptions of social norms and increase empathy and bystander intervention abilities among college students. In pre- and postcourse surveys, the course participants reported their level of agreement or disagreement with several statements related to these constructs. The dependent variables were gleaned from a factor analysis of these Likert-type-style survey questions, after which pre- and postmeasures of the resulting factors were compared to determine whether shifts in attitudes and perspectives moved in the desired direction. Because Haven—Understanding Sexual Assault is a universal prevention strategy intended for all incoming students at a given institution, ANOVA was used to examine these data for each school separately.

The Sexual Assault Prevention Curriculum

The course content and messages for the course were developed using theory- and evidence-based approaches for prevention, working in collaboration with leading researchers and campus violence prevention practitioners beyond those authoring this article. Participants were gauged on their baseline knowledge regarding course content before the first learning module and then assessed again immediately after course completion to ensure they had gained sufficient understanding of the material. Most often, campus administrators provide students who are about to matriculate a link to log-in to the course platform with instructions to complete the content before classes begin. Course completion is often a prerequisite for essential integration services, such as access to housing or the ability to register for courses. Campus administrators can monitor student progress through the platform to ensure that learners complete the course and enforce the mandates.

The course contains over an hour of interactive content split into seven modules, which can be taken all at once or can be spaced out by the student to meet their individual needs. The modules focus on seven domains of prevention education:

Importance of Values: The course asks students to identify and reflect on their own personal values as a foundation for engagement with the course content.

Aspects of (Un)Healthy Relationships: The course emphasizes the importance and benefits of building healthy relationships and sets positive standards for what students should strive for in their own relationships and by which they should hold their peers accountable.

Gender Socialization: The course gives students an opportunity to explore predominant social messages about masculinity and femininity, challenges them to consider the costs and benefits of stereotypical and rigid gender identities, and provides strategies for identifying and speaking out against sexism.

Sexual Assault Misinformation and (Mis)Perceptions: The course seeks to address misinformation that students may have concerning sexual assault including what it is, how often it happens and to whom, who perpetrates and how, options for addressing the problem, and misperceptions regarding the degree of healthy/unhealthy attitudes and behaviors that exist among their peers. The scenarios and definitions of terms provided in the course, coupled with the campus-specific resources, policies, and procedures it cites, help students understand the conduct expectations and support options at their institution.

Consent: The course teaches students that the best way to obtain consent is through explicit, verbal communication with a partner. The course also addresses coercive behaviors and the impact of alcohol in obscuring sexual communication. An important focus is to teach not only consent in personal intimate situations but also signs of potential nonconsenting situations among their peers, an essential first step for bystander intervention. Importantly, the course content explicitly states that a person who is assaulted is not to be blamed for what happened to them, no matter what the particular circumstances might be.

Bystander Intervention: The instructional content provides students with realistic sexual assault and relationship violence scenarios to help them identify problematic situations, introduces them to a range of effective intervention options, and corrects misperceptions that may inhibit students from intervening in risky situations.

Ongoing Activism: The course suggests multiple ways that students can help prevent sexual and relationship violence at their institution and encourages students to be involved in prevention efforts.

Course Surveys

In addition to assessments of students’ pre- and postintervention knowledge, the course also includes attitudinal and behavioral surveys that are administered electronically to the students through the learning platform. The preintervention survey (Survey 1) was administered before any course content was delivered, which for most students occurred before they started their first year on campus. After students completed the course, there was a school-specific intersession period—typically of 30 to 45 days, by which time they had started the fall semester—after which students were recontacted and asked to complete additional education modules and a postintervention survey (Survey 2).

The surveys collected a wide range of data to assess the impact of the course and provide information to guide the institutions’ ongoing prevention efforts. Survey items were adapted from the Illinois Rape Myth Acceptance Scale (McMahon & Farmer, 2011; Payne, Lonsway, & Fitzgerald, 1999) and the Readiness to Change Scale (Banyard, Eckstein, & Moynihan, 2009), and from social norms instruments developed and used by campus professionals (Berkowitz, 2013).

Both Surveys 1 and 2 contained Likert-type-style questions that asked respondents to indicate their level of agreement or disagreement with each of 20 statements, with 1 = strongly disagree and 7 = strongly agree (see Table 1). From the national sample of data on these 20 items, Cronbach’s alpha (α) was .66 for Survey 1 and .69 for Survey 2. These survey items, while adapted from academic research literature on sexual violence prevention, were selected and updated to align closely to the behavioral and learning objectives of the intervention. Specifically, the items measured students’ ability and intention to intervene as a bystander of sexual harassment and violence, their perception of their peers’ ability to do the same, their perceptions of social norms related to sexual violence, and their belief in long-standing myths surrounding rape and sexual violence. Survey 1 also asked for demographic information, including gender identity, biological sex, race/ethnicity, sexual orientation, academic status, age, living arrangements, membership in school-based groups, and parental education level.

Table 1.

Factor Loadings and Communalities for Course Questionnaire Items Based on a Principal Components Analysis With Varimax Rotation.

Questionnaire Items[a]	Extracted Factor[b]				Communality[c]
	1	2	3	4
I can identify warning signs of abuse in dating relationships.	0.73				0.57
I would feel comfortable intervening if I witnessed abusive behavior.	0.71				0.59
I am aware of resources for relationship violence on my campus.	0.64				0.48
I am confident in my ability to support a friend who has been sexually assaulted.	0.63	0.32			0.53
I would say or do something if I heard someone using sexist language.	0.61				0.42
I can play a role in preventing relationship violence on my campus.	0.59				0.45
I have a good understanding of the definitions of sexual assault.	0.57	0.35			0.47
I would take action in a situation in which someone was trying to take advantage of another person sexually.	0.56	0.40			0.59
Explicit verbal consent is the best way to make sure a person is okay with sexual activity.		0.64			0.42
People who identify as gay, lesbian, bisexual, or transgender are also at risk of relationship violence.		0.60		0.31	0.52
A person should never be blamed for being the victim of sexual assault.		0.59			0.39
I would respect someone who intervened to prevent a sexual assault.		0.57	0.37	0.39	0.70
I genuinely feel sorry for victims of relationship violence.	0.32	0.56			0.49
Most students at my school would take action if they saw someone trying to take advantage of another person sexually.	0.31		0.77		0.69
Most students at my school would respect someone who intervened to prevent a sexual assault.			0.73		0.69
Most students at my school would feel comfortable intervening if they witnessed abusive behavior.	0.49		0.65		0.67
I don’t think sexual assault is a significant problem on my campus. (Reverse coded)			0.41	0.41	0.37
It’s not sexual assault if both people have been drinking. (Reverse coded)				0.72	0.54
Physical abuse is the only form of relationship violence. (Reverse coded)				0.72	0.54
Sexist jokes and language don’t contribute to relationship violence. (Reverse coded)				0.69	0.51

Note. Survey 1 is a preintervention survey administered at the beginning of the course. Survey 1, N = 279,929.

Respondents were asked to report their level of agreement or disagreement with each statement, with 1 = strongly disagree and 7 = strongly agree.

Factor loadings < 0.30 are not displayed. Factor 1 = Intervention and Ability; Factor 2 = Empathy and Support for Victims; Factor 3 = Perceptions of Social Norms; and Factor 4 = Sexual Assault Myths.

The communality for each variable was computed as the sum of the squared loadings for that variable; this value can be interpreted as the proportion of response variation that is explained by the four factors.

Surveys 1 and 2 also contained questions related to the prevalence of sexual assault victimization and perpetration, which were drawn from two gold-standard instruments for assessing sex crimes: The Sexual Experiences Survey (Koss et al., 2007) and the survey instrument from the Campus Sexual Assault Study (Krebs et al., 2008). These data on perpetration and victimization are not reported here.

Data Privacy and Confidentiality

Students entered the course using their institution’s assigned login ID and then used their email address to create a personal account. The account creation process is protected using Secure Sockets Layer (SSL) encryption. For campus administrators to track a student’s progress in the course, it is necessary to link the student’s personal account number and email address. Importantly, the students’ email addresses are dissociated from their survey responses, and these data are stored in a firewall-protected database at EVERFI; a student’s individual survey responses are never shared or linked to an identifying email address.

Prior to beginning the course and throughout the surveys, students were reminded that their responses to all questions were confidential. They were asked to reply honestly and to skip any questions they did not feel comfortable answering. Respondents were reminded again about this before being presented with the questions related to personal experiences of victimization and perpetration, and were also provided with trigger warnings and directions toward available resources if they found the survey questions distressing.

This study uses an existing anonymized dataset that cannot be traced to individual students and, therefore, was approved as exempt research by the Duke University Medical Center Institutional Review Board.

Results

Study Sample

We sampled from the course’s national dataset of more than 500,000 respondents during the fall semester of 2014, selecting only those students who self-identified as entering first-year students at four-year institutions of higher education. The resulting dataset included 279,929 undergraduate first-year students from 260 public (70%) and private (30%) schools. About two thirds (66%) of respondents completing Survey 1 also responded to the follow-up Survey 2. On average, students took the second survey 59 days after completing the first survey, with a standard deviation of 14 days. Only 12% of students were currently taking classes when they took Survey 1, but all students reported being in classes at the time they completed Survey 2.

The respondents self-identified as 68% White/Caucasian, 11% Asian/Pacific Islander, 9% Hispanic/Latino, 8% African American, 1% Native American/Alaskan, and 3% Other Race/Ethnicity, and as 56% female, 43% male, and 0.5% transgender or other gender. The vast majority of the sample identified as heterosexual (93%). Most students reported they were 18 years old (79%), with 11% selecting 17 years, 8% selecting 19 years, and 2% selecting 20+ years. Only 17% of respondents were first-generation college students (i.e., no parents with any college education). The demographics of this sample were quite representative of the incoming college student population in the United States when compared with the Department of Education’s statistics on undergraduates (National Center for Education Statistics, U.S. Department of Education, 2016, Table 302.62).

Factor Analysis

We conducted a principal components factor analysis on the national Survey 1 dataset to identify the underlying factor structure and then calculate composite factor scores to serve as dependent variables for the ANOVA, as opposed to item-by-item analyses. All survey items were coded so that higher scores (5-7 on the 7-point scales) represented healthier responses. In total, 18 of the 20 items correlated at least 0.30 with at least one other item, suggesting reasonable factorability. Furthermore, the Kaiser–Meyer–Olkin measure of sampling adequacy was 0.89, well above the commonly recommended value of 0.60, and Bartlett’s test of sphericity was statistically significant, χ2 (190) = 1,819,662.84, p < .05. Also, the diagonals of the anti-image correlation matrix were all more than 0.50. Finally, the communalities were all above 0.30 (see Table 1), further confirming that each item shared some common variance with other items. These indicators demonstrated that a factor analysis with all 20 questionnaire items was appropriate.

We conducted a principal components factor analysis of the 20 items using varimax rotation, extracting four factors with eigenvalues more than 1 that collectively explained 53% of the variance (31%, 11%, 6%, and 6% for Factors 1-4, respectively). All of the items had primary loadings over 0.50 with one exception: “I don’t think sexual assault is a significant problem on my campus.” The factor-loading matrix for the instrument is presented in Table 1.

Based on primary loadings for each item, the four factors were labeled as (a) Intervention Ability and Intent, (b) Empathy and Support for Victims, (c) Perceptions of Social Norms, and (d) Sexual Assault Myths, respectively. For each factor, we calculated composite scores as the sum of the items with their primary loading on that factor; higher scores indicate a greater level of agreement with healthier responses. Descriptive statistics are provided in Table 2.

Table 2.

Descriptive Statistics for Identified Factors.

Factor(Number of Items)	Cronbach’s Alpha (α)	Range[a]		M (SD)	Skewness	Kurtosis
		Min	Max
1. Intervention Ability and Intent (8)	0.84	1	56	41.54 (8.0)	−0.59	1.18
2. Empathy and Support for Victims (5)	0.68	1	35	29.57 (5.2)	−1.44	3.29
3. Perceptions of Social Norms (4)	0.64	1	28	18.16 (4.0)	−0.39	1.06
4. Sexual Assault Myths (3)	0.63	1	21	18.05 (3.1)	−1.34	2.40

Note. Survey 1 is a preintervention survey administered at the beginning of the course. Survey 1, N = 279,929.

Higher scores indicate healthier responses.

The greatest variability among students was found for the Intervention Ability and Intent factor; even so, the vast majority fell on the “Agree” side of the distribution. The Empathy and Support for Victims factor had less variance but was significantly negatively skewed so that most students expressed empathy for peers who experienced sexual assault and relationship violence. Students did not typically adhere to Sexual Assault Myths (see Table 2), leaving little room for improvement. Consistent with previous research, the distribution for the Perceptions of Social Norms factor was relatively normally distributed and did not display a great deal of variance. Importantly, all students believed they were more likely to intervene to prevent sexual assault than their peers, a misperception that the course could potentially correct (see Table 3).

Table 3.

Percentage Agreement for Course Questionnaire Items: Survey 1 Versus Survey 2.

Questionnaire Items[a]	Survey 1	Survey 2
Intervention Ability and Intent
I can identify warning signs of abuse in dating relationships.	61.2%	82.3%
I would feel comfortable intervening if I witnessed abusive behavior.	66.3%	72.6%
I am aware of resources for relationship violence on my campus.	51.6%	74.7%
I am confident in my ability to support a friend who has been sexually assaulted.	79.6%	84.3%
I would say or do something if I heard someone using sexist language.	56.1%	90.1%
I can play a role in preventing relationship violence on my campus.	68.0%	69.2%
I have a good understanding of the definitions of sexual assault.	84.3%	92.6%
I would take action in a situation in which someone was trying to take advantage of another person sexually.	83.6%	83.9%
Empathy and Support for Victims
Explicit verbal consent is the best way to make sure a person is okay with sexual activity.	76.8%	85.5%
People who identify as gay, lesbian, bisexual, or transgender are also at risk of relationship violence.	87.7%	90.2%
A person should never be blamed for being the victim of sexual assault.	68.0%	75.0%
I would respect someone who intervened to prevent a sexual assault.	93.4%	92.4%
I genuinely feel sorry for victims of relationship violence.	91.0%	91.0%
Perceptions of Social Norms
Most students at my school would take action if they saw someone trying to take advantage of another person sexually.	52.9%	56.7%
Most students at my school would respect someone who intervened to prevent a sexual assault.	81.5%	83.9%
Most students at my school would feel comfortable intervening if they witnessed abusive behavior.	43.3%	53.9%
I don’t think sexual assault is a significant problem on my campus. (Reverse coded)	48.3%	50.2%
Sexual Assault Myths
It’s not sexual assault if both people have been drinking. (Reverse coded)	86.3%	86.9%
Physical abuse is the only form of relationship violence. (Reverse coded)	91.1%	90.5%
Sexist jokes and language don’t contribute to relationship violence. (Reverse coded)	79.1%	81.6%

Note. Survey 1 is a preintervention survey administered at the beginning of the course; Survey 2 is a postintervention survey administered 30 to 45 days after course completion. N = 171,902.

Respondents were asked to report their level of agreement or disagreement with each statement, with 1 = strongly disagree and 7 = strongly agree. The reported percentages are for responses 5 to 7.

School-Level ANOVA

To assess the impact of the course, we selected institutions from the national sample that implemented the course in the fall of the 2014-2015 academic year. The analysis included only those institutions that had (a) at least 500 students who completed all course content, including all knowledge assessments and both surveys, and (b) a response rate of 60% or higher for Survey 2. This resulted in a final sample of 167,424 students representing 80 different institutions from 33 different states. The colleges ranged in size from 1,000 to 30,000 undergraduate students, with a mean undergraduate enrollment of 11,500. The average sample size per school was 2,066. For each questionnaire item, Table 3 shows the percentage of students in the total sample who reported healthy attitudes and behavioral intentions (scores = 5-7, with four items reverse-scored).

Employing the composite factor scores as dependent variables, we ran individual ANOVAs for each of the 80 institutions to test whether there was a significant increase in mean factor scores comparing pre- and postcourse survey responses. For Intervention Ability and Intent, 78 schools (98%) had significant increases in their composite factor scores (p < .05); for Empathy and Support for Victims, 67 schools (84%) had significant increases (p < .05); for Perceptions of Social Norms, 60 schools (75%) had significant increases in correctly perceived norms; and for Sexual Assault Myths, 27 schools (34%) had significant increases. See Table 4 for descriptive statistics regarding the ANOVAs and the distributions of effect sizes (partial eta squared, η2).

Table 4.

Number and Percentages of Colleges With Statistically Significant Repeated-Measures ANOVAs for Factor Composite Scores: Survey 1 Versus Survey 2.

		Distributions of Partial Eta Squared, η2
Factor Composite Score	Colleges with p < .05, N (%)	Range		M (SD)	Median	Skewness	Kurtosis
		Min	Max
1. Intervention Ability and Intent (8)	78 (98%)	0.01	0.39	0.12 (0.09)	0.11	0.86	0.24
2. Empathy and Support for Victims (5)	67 (84%)	0.00	0.09	0.02 (0.02)	0.02	1.41	1.76
3. Perceptions of Social Norms (4)	60 (75%)	0.00	0.15	0.04 (0.03)	0.03	1.42	1.99
4. Sexual Assault Myths (3)	27 (34%)	0.00	0.04	0.01 (0.01)	0.01	2.81	10.31

Note. Survey 1 is a preintervention survey administered at the beginning of the course; Survey 2 is a postintervention survey administered 30 to 45 days after course completion. Eighty institutions, N = 167,424.

Consistent with general practice, we regarded η2 values of .01, .06, and .14 as small, medium, and large effect sizes, respectively. The vast majority of institutions saw significant increases in Intervention Ability and Intent scores: 37% actually saw a large effect on this factor and 31% a medium-sized effect. Most institutions showed a significant increase in the percentages of students with more accurate Perceptions of Social Norms, but these effects were smaller in size, with only 18% of schools reaching a medium-sized effect and 58% a small-sized effect. Although more than 84% of schools saw increases in their Empathy and Support for Victims score, this impact was generally small in size (76%), due largely to students having high initial scores on this factor, leaving little room to improve. Analyses showed very little effect on Sexual Assault Myths (30% with a small-sized effect), again due to the infrequent endorsement of these myths at baseline.

Findings from these ANOVAs for each of the 80 institutions included in the analysis, using factor scores as dependent variables, indicate that an online sexual assault course such as the one evaluated here can be effective in encouraging students to intervene to prevent sexual assault and in correcting misperceptions of social norms that can inhibit them from doing so.

Discussion

This investigation evaluated the impact of a widely used online sexual assault prevention course on students’ self-reported attitudes, self-efficacy, and behavioral intentions. Principal components analysis showed that the 20 Likert-type-style survey questions employed in the preintervention survey grouped into four larger constructs that explained more than 50% of the total variance in students’ responses: Intervention Ability and Intent, Empathy and Support for Victims, Perceptions of Social Norms, and Sexual Assault Myths. These factors were used to create composite scores (each with high internal consistency) that served as dependent measures to assess the impact of the course as a population-level sexual assault prevention approach.

At baseline, students varied greatly in their intent and perceived ability to intervene to prevent sexual assault, and all students believed that they were more likely than their peers to intervene. Pre- to postintervention analysis showed that the course had the largest impact on the Intervention Ability and Intent and Perceptions of Social Norms factors, as reflected in the large number of colleges that saw statistically significant improvements and the effect sizes of those changes. These findings are especially noteworthy given the extensive literature that has focused on how misperceptions of social norms and low self-efficacy prevent college students from intervening to stop sexual assault and relationship violence. Further, more than half of the colleges also saw small but significant increases in the Empathy and Support for Victims factor. Note, however, that students reported higher levels of desirable responses to questions associated with this factor before entering the course, and so they had less room for positive change in response to the intervention.

This set of findings aligns closely with results found in similar investigations of sexual assault prevention programming, those focused on bystander intervention instruction in particular (Banyard et al., 2007; Coker et al., 2016; Coker et al., 2015) and those that combine such instruction with corrective normative feedback (Gidycz et al., 2011; Salazar et al., 2014). Emerging evidence suggests that the combination is particularly effective because misperceptions of peers’ bystander intentions and support often create significant barriers to intervening (Gidycz et al., 2011).

Although population-level sexual assault education is critical for institutions of higher education to ensure that all students possess basic knowledge of key concepts, available resources, and prevention strategies, an online course such as the one evaluated here should be considered as only one piece of a comprehensive and ongoing approach to establishing and maintaining healthy attitudes and behaviors regarding sexual assault and relationship violence.

The positive results produced by universal prevention education can diminish over time in the absence of ongoing programming and clearly communicated and enforced policies (DeGue et al., 2014). Thus, it is critical that college administrators provide supplemental awareness education and skills-based training throughout the academic year to reinforce and expand on what is covered in the online course. It is especially important to identify and reach subsets of students who may continue to have unhealthy attitudes and behaviors following an initial introductory course, and to provide tailored and targeted education to diverse subpopulations with unique needs, strengths, and experiences.

Most programs in this domain are focused on incoming student populations, and there is a dearth of evaluation research related to ongoing education for college attendees. Moving forward, current universal education programs could be adapted to reinforce prior learning, build new skills, and present developmentally appropriate content. Accordingly, the learning objectives in the current courses could be updated to build on existing knowledge and skills rather than providing a baseline of awareness, and scenarios could be modified to more closely align to the experiences of students who have spent time on campus. Recognizing that not all students learn in the same manner, critical content should be provided at multiple occasions, employing different modalities and instructional methods to accommodate all types of learners. Furthermore, the longitudinal data collected from multiple instances of sexual violence prevention education would allow administrators to track the development of these attitudes and behaviors throughout the higher education experience, thereby better informing their efforts aimed at resource allocation, impact measurement, and external reporting.

Limitations

Limitations to the present study should be noted. First, this study employed a simple pretest–posttest design with no control sites. Its strength, however, is the inclusion of multiple colleges and universities of varying size and composition, and the measurement of effect sizes at the institutional level. In effect, this investigation is a set of 80 replication studies, and while certain threats to internal validity cannot be dismissed, the large number of institutions for which statistically significant effects were found does increase confidence that the intervention was primarily responsible for these improvements. This study sets the stage for future investigations that employ either an institutional cycle design or a randomized control trial.

Second, the study focused on a restricted subsample of students: incoming first-year students at four-year institutions. Although the sample size is both large and demographically diverse, caution should be exercised when generalizing these results to other groups of college students. Future research should include a broader cross-section of students. Another limitation is that the findings reported here are based on student self-reports. However, self-reports from anonymous or confidential questionnaires are a commonly accepted method in studies with college students as they result in population-level data that are generally both valid and reliable (Brener, Collins, Kann, Warren, & Williams, 1995; Brener et al., 2002; DeJong, 2008; Dowdall & Wechsler, 2002). Finally, while the present study demonstrates positive impact on college students’ attitudes, perceptions, self-efficacy, and behavioral intentions related to sexual assault and relationship violence, changes in actual behaviors were not assessed. The possibility that students who took the course provided the socially desirable answers suggested by participation in the course cannot be entirely dismissed.

Accordingly, future research should include measures of actual bystander or consent-seeking behaviors. Additional focus on the impact of online interventions for specific student populations—particularly demographic subgroups considered to be higher risk—would provide important information for campus administrators. Ultimately, future studies of online courses and other online sexual assault prevention programs should include multisite investigations with randomization executed at the institutional level.

Conclusion

In conclusion, this study provides compelling evidence that this online sexual assault prevention course, when implemented as a population-level intervention for first-year college students, fosters accurate perceptions of positive social norms, increases empathy and support for victims, and increases students’ ability and intention to intervene to prevent sexual assault and relationship violence. Our study suggests that the most effective prevention programming includes a combination of bystander intervention training and social norms corrections, as was advised by Berkowitz (2016) and demonstrated in recent studies in which the two approaches used together produced strong outcomes (Gidycz et al., 2011; Salazar et al., 2014; Wasco, 2015). Together, these strategies can increase self-efficacy measures of bystander engagement, realign perceptions of socially acceptable behavior, and, thereby, make students more likely to stand up, step in, and speak out when they see unsettling actions or scenarios.

Universal digital education can be effective at providing students with a foundation of knowledge and awareness that can be built on by future efforts. It is important to remember that student populations are heterogeneous and will respond differentially to sexual assault prevention programming, and for this reason, colleges and universities need to put more effort into providing follow-up programming that is specific to students’ particular strengths and deficits. Two subgroups are of the greatest concern: those who are survivors of violence and the student minority that reacts negatively to this educational content (“backfire effect”) because it conflicts with their preexisting belief system (Kraft, Lodge, & Taber, 2015). Future efforts should focus on developing and evaluating effective programming for these other subgroups of students.