Development and pilot test of criteria defining best practices for organizational sexual assault prevention.

Sexual violence affects millions of Americans, and approximately one out of every three women and one out of every four men have experienced sexual violence during their lifetime. While prevention efforts have focused on implementing specific programmatic approaches, there has been relatively little focus on developing comprehensive and effective approaches to reduce sexual assault prevention across an organization. This study describes the development of the Prevention Evaluation Framework, an assessment targeting organizational best practices for comprehensive sexual assault prevention across multiple domains including human resources, collaborative relationships and infrastructure, use of evidence-informed approaches, quality implementation and continuous evaluation of programs/policies. Using the structured RAND/University of California, Los Angeles appropriateness method to develop the assessment, we conducted a literature review and solicited expert feedback about what a comprehensive organizational approach to sexual assault prevention should entail. We then pilot tested the assessment with 3 United States military service academies; and continued to improve and adapt the assessment to a range of organizations with input from 6 Department of Defense headquarters organizations, and 9 universities across the country. Given the nascent state of the evidence about what makes an effective organizational approach to sexual assault prevention, the assessment reflects one way of promoting quality in this evolving field. The consistency between the experts' ratings and the literature, and the relevance of the items across organizations suggest that the assessment provides important guidance to inform the development of comprehensive organizational approaches to sexual assault prevention and to the evaluation of ongoing efforts.

Sexual violence affects millions of Americans. Research has found that one out of every three women, and one out of every four men have experienced sexual violence in their lifetimes. Twenty-five percent of women have experienced completed or attempted rape. Beyond the immediate trauma, sexual violence often has serious long-term impacts that are both psychological (e.g., posttraumatic stress disorder, depression, anxiety, suicidal ideation, substance use, risky sex) and physical (e.g., reoccurring gynecological, gastrointestinal, cardiovascular and sexual health problems). Also, female victims are more likely to experience other forms of violence (Centers for Disease Control and Prevention, n.d.). Victims often have a diminished economic productivity. Recent estimates put the cost of rape at $122,461 per victim based on medical and legal costs, and lost productivity (Smith et al., 2018). Many assaults go unreported, given the difficulty associated with collecting accurate data, and thus the prevalence and cost of sexual violence is likely higher. The problem of sexual violence affects individuals across both civilian and military settings (Stander & Thomsen, 2016).

High profile media coverage of sexual assault on college campuses1 and military installations2 has raised public awareness of this significant public health issue and renewed the urgency of finding solutions. Despite a requirement by the Secretary of Defense that each military service academy (Navy/Air Force/Army) implement plans to prevent assaults, the 2018 bi-annual survey at academies revealed that rates of unwanted sexual contact and sexual harassment rates were continuing to increase for both men and women. Alcohol was involved in about half of all unwanted sexual contact events – used by either victims and/or offenders. While the number of sexual assault reports increased over time, the overall reporting rates remained at about 12 percent (ranging from 6 to 16% over the 2005–2006 to 2017–2018 academic years). Offenders were generally fellow academy students, and offenses occurred both off and on academy grounds (with off-grounds most likely). Similarly, the White House Task Force to Protect Students from Sexual Assault funded a Campus Climate Survey Validation Study to yield school-specific estimates of campus climate and sexual violence. The study found that 1 in 5 women and 1 in 14 men experienced sexual assault while in college and victimization rates were higher for female bisexual and transgender students; and only about seven percent of students reported their rape to authorities (Krebs, Lindquist, Berzofsky, Shook-sa, & Petersen, 2015). Mandates from Congress and the Department of Defense called for improved approaches to prevent sexual violence to address the growing public concerns. However, despite the development of primary-prevention strategies, decades of prevention and implementation research has shown that certain organizational practices are needed to support primary-prevention strategies. For example, having adequate staffing, planning, and program evaluation in place, along with evidence-based strategies, is key to improving outcomes (Livet & Wandersman, 2005).

In 2019, the Department of Defense Sexual Assault Prevention and Response Office (DoD SAPRO) partnered with RAND to create a Prevention Evaluation Framework comprised of criteria describing a set of cohesive, comprehensive, and best practices for sexual assault and harassment prevention that includes targeted primary-prevention programs and organizational capacities (e.g., leadership) and infrastructure. The purpose of this article is to describe the process for developing the criteria, pilot testing the criteria with military service academies, and continuing to improve and adapt the criteria for an array of organizations with input from civilian colleges and universities, and military headquarters organizations.

Methods

Our approach was based on the RAND/University of California, Los Angeles (UCLA) appropriateness method (Fitch et al., 2001), a systematic method for obtaining expert judgment on topics where there is limited evidence to guide practice (e.g., Acosta, Martin, Fisher, Harris, & Weinick, 2012)—for example, which organizational resources (e.g., materials, budget), infrastructure (e.g., protocols, accountability structures), workforce (e.g., expertise, training), and engagement strategies (e.g., relationships outside the organization) are needed to effectively prevent sexual assault. The method combines the best available evidence with the collective judgment of experts to develop practice guidance and has been extensively used in healthcare settings to create practice guidelines (e.g., prescribing practices, catheter use, spinal fracture care; Hirsch et al., 2018, Meddings et al., 2019, Zahra et al., 2018). The best available evidence comes from peer-reviewed and grey literature, and the collective judgment of experts is achieved through a consensus-based process where experts review and collectively develop a set comprehensive practices to help support decision-making and improve quality of care. We have adapted it for primary prevention and used it to 1) assess the “appropriateness” of including certain practices in organizational prevention efforts and 2) describe how these practices could be applied to an actual organization (i.e., military service academies and civilian universities) to determine whether the organization is consistent with the selected practices. The term applying the refers to the process of assessing whether organizations are implementing selected practices – not attempting to newly implement the selected practices. The study was approved by RAND’s Human Subjects Protection Committee. We describe the four phases of the process below.

Phase One: Generate an initial list of criteria reflecting best practices

To generate an initial list of criteria reflecting organizational best practices, we extracted best practices from across two bodies of research—sexual assault prevention-specific best practices and general primary prevention organizational frameworks and best practices. A technical package3 by the Centers for Disease Control and Prevention identified the best available evidence related to sexual assault prevention (based on n = 97 articles; Dills, Fowler, & Payne, 2016). However, this technical package was focused primarily on practices and programs across an entire community and did not include information about key organizational capacities or infrastructure needed for prevention (e.g., workforce, leadership). To augment this, we included a body of work describing the practices of an evidence-based organizational capacity building approach tailored to a variety of primary prevention areas—Getting To Outcomes (n = 48 articles4); and a complementary conceptual framework that describes drivers of organizational readiness to implement new prevention programming (n = 56 articles; Scaccia et al., 2015). Finally we reviewed the DoD SAPRO’s Prevention Plan of Action, which is characterized as a “comprehensive prevention system and process…to realize effective prevention in every military community” and relevant support documents (n = 22 articles; SAPRO, 2019b).

We created a data abstraction form to facilitate a systematic evaluation and abstraction of the best practices from across these sources. Best practices were abstracted from each source and compiled into a single list by source. These lists were then placed side-by-side in a crosswalk to allow a review and comparison of best practices from each source and to identify areas of agreement/duplication across sources. Entries were then synthesized into a single list of best practices for organizational sexual assault prevention. For example, McIntosh et al. (2010) suggested that organizational leadership should use data-driven decision making to effectively sustain prevention efforts. Therefore, we included a corresponding criteria item to ensure that leaders consistently use available research evidence to inform decisions and rely upon available evidence when briefing and approving initiatives.

From these four main sources, we identified an additional initial set of 62 organizational best practices for sexual assault prevention— 30 related to organizational capacities and infrastructure (called academy level criteria) and 32 related to specific prevention activities (called activity level criteria).

Phase 2: Convene the expert panel

We virtually convened a panel consisting of 15 experts in three key areas: sexual assault prevention, military culture and sexual assault, and organizational approaches to prevention more generally (Table 1). The expert panel was also intentionally designed to have implementation experts so that consensus-based items could reflect best practices in implementation. We sent experts a summary of the literature mentioned above and the proposed 62 best practices that included a rationale, based on the literature, for each of the proposed items via email. Proposed criteria items were presented as affirmative statements about the organizational conditions and approaches needed to develop, implement, and evaluate a comprehensive organizational approach to sexual assault prevention (e.g., Leaders at all levels consistently work together when planning prevention).

Table 1

Expert Panel Members.

Area of Expertise	Experts Names and Affiliation*
1. Sexual assault prevention	• Elizabeth Miller, Chief of the Division of Adolescent and Young Adult Medicine at the University of Pittsburgh School of Medicine • Eric R. Pedersen, Senior behavioral scientist at the RAND Corporation • Christine Gidycz, developer of The Ohio University Sexual Assault Risk Reduction Program • Brian Marx, Professor of Psychiatry at Boston University School of Medicine and staff psychologist at the National Center for PTSD, VA Boston Healthcare System • Casey T. Taft, staff psychologist at the National Center for PTSD in the VA Boston Healthcare System, and Professor of Psychiatry at Boston University School of Medicine • Jacquelyn W. White, Emerita Professor of Psychology and former director of Women’s and Gender Studies at the University of North Carolina at Greensboro, where she also served as Associate Dean for Research in the College of Arts and Sciences • Charlene Senn, Canada Research Chair in Sexual Violence and Professor of Psychology and Women’s and Gender Studies at the University of Windsor • Dorothy Edwards, founder and president of Alteristic
2. Military culture and sexual assault	• Andra Teten Tharp, Senior Prevention Advisor in the US Department of Defense Sexual Assault Prevention and Response Office • Rachel Breslin, Senior Applied Social Scientist in the Health and Resilience Research Division of the Office of People Analytics • Major Karmon Dyches, Military Deputy for Psychological Health at Military Operational Medicine Research Program • A. Monique Clinton-Sherrod, Special Advisor for the Prevention of Destructive Behaviors and Director of Prevention and Behavioral Assessments at the U.S. Navy
3. Organizational approaches to prevention	• Abigail A. Fagan is an Associate Professor in the Department of Sociology and Criminology & Law at the University of Florida • Laura F. Salazar, professor in the School of Public Health at Georgia State University

* Affiliations are those at the time they served on the expert panel. The research team comprised some of the leading experts on organizational approaches to prevention, thus the panel included fewer experts in that area.

Experts then rated each proposed criteria item twice—once on its validity and once on its importance. This rating process and similar definitions for importance and validity have been used in past similar studies (Acosta et al., 2017, Acosta et al., 2012, Farmer et al., 2015). Validity was defined as having adequate scientific evidence or professional consensus exists to support a link between the criterion and the effectiveness of sexual assault and harassment prevention efforts; and sexual assault and harassment prevention efforts with significantly higher adherence to the criterion would be considered higher quality. A criterion was deemed important if adherence to the criteria was considered a primary driver of the effectiveness of sexual assault and harassment prevention efforts or a critical influence on the development or implementation of sexual assault and harassment prevention efforts; and if there would be serious adverse consequences from not adhering to the criteria. A Likert scale (1 to 9) was used for each rating. Experts were also given the opportunity to provide comments on each of the proposed items to help clarify how items should be modified or improved, and to propose new criteria items.

After all ratings and comments were submitted via email, we convened a virtual meeting with the experts to review two newly-proposed items and items for which there was significant disagreement. Significant disagreement was defined as two or more experts rating an item with more than a three-point distance from the other panelists. After the conference call, the experts were asked to re-rate only the items that had been discussed during the virtual meeting and submit re-rating via email. The final ratings were then calculated and the final 63 criteria (31 academy level criteria and 32 activity level criteria) was assembled into the Prevention Evaluation Framework, based on whether an item received no more than one rating below a 6 (out of a possible 9) on both validity and importance.

Phase 3: Pilot test the criteria

We pilot tested the Prevention Evaluation Framework criteria with the United States Military Academy, Naval Academy, and Air Force Academy. We provided the academy staff responsible for sexual assault prevention with a summary of all the final criteria items, along with a description of how the items were developed. Participating staff at academies included the Commandant, the Military Equal Opportunity Officer, Sexual Assault Response Coordinator/Sexual Harassment and Assault Response Prevention Program Manager, Violence Prevention Integrators (those responsible for prevention of sexual assault), Victim Advocates and other leaders.

DoD SAPRO then initiated a call for data asking academy leadership about the details of each type of programming used at their Academy related to sexual assault prevention (e.g., program’s goal, frequency of evaluation, target population, concrete benchmarks, and the level of evidence). DoD SAPRO and RAND staff also conducted a series of in-person interviews of academy leadership about their prevention activities related to sexual assault and harassment. Additional site visits were conducted by DoD SAPRO personnel in June 2019 as part of DoD’s by-law assessments of the academies. A summary of interview responses and the academy’s response to the data call were then shared with expert panelists.

Given the breadth and depth of the information on each academy, expert panel members (Table 1) were each assigned to a single academy to apply the criteria (n = 5 experts per academy). After the expert panel members reviewed the materials, they were asked to rate their assigned academy using all the academy-level criteria items, as well as provide a rationale for their rating via email. They first provided an overall rating based on their review of the materials, reflecting an aggregate impression of the extent to which the academy adheres to the criteria (from 5 = Criterion has been achieved to 1 = No progress: Efforts are absent and there is no attempt to make progress). For each rating, panel members were also asked to identify the specific information that contributed to their rating (from the data call and interviews), and to provide comments on how the academy could modify or improve their organizational approach to sexual assault prevention.

To pilot test the activity-level rating, two raters from the RAND/DoD SAPRO team, using the same procedures as the expert panel, applied the criteria to a select number of specific prevention activities at the academies. The activity chosen reflected significant or flagship effort for the academy. First ratings were made independently by each rater, then revised based on a group discussion of any disagreements in the ratings.

Phase 4: Adapt the criteria for Self-Assessment by other organizations

While the initial pilot test provided critical information about the process needed to determine organizational alignment with the criteria that are part of the Prevention Evaluation Framework, it is not always feasible or preferable for an outside team of researchers and experts to apply the criteria. Therefore, the final phase was focused on developing tools to support self-assessment and exploring the criteria’s application to a broader array of organizations.

The RAND/DoD SAPRO team created a set of instructions and a guide to self-assessment tool for academies, which they used in Summer 2020 for a follow-up assessment. Fig. 1 contains a sample from an interview guide used in the Summer 2020 assessment.

Fig. 1

Sample questions from interview guide used in the self-assessment of colleges and universities.

The criteria and this guide were also adapted for use with DoD headquarters organizations as part of their Prevention Plan of Action and civilian colleges and universities to assess the criteria’s relevance in these organizations.

Adaptation study 1: DoD Headquarters Organizations. The criteria were used to identify the current state of prevention capabilities within six strategic offices at DoD headquarters. Unlike the military service academies, which represent a single military installation, DoD organizations are multi-tiered with strategic, operational, and tactical functions located within different tiers. For example, the implementation of a specific prevention activity may be directed by the strategic level but implemented at the tactical, installation level. As a result, DoD SAPRO identified only 23 of the 63 criteria, focused on strategic functions like collaboration and leadership, that were applicable for DoD headquarters organizations (Appendix A).

These 23 criteria were shared with headquarters offices within each of the military service headquarters and the National Guard Bureau. These offices provided feedback to clarify the criteria for their application to headquarters functions. Headquarters offices then used a self-assessment process similar to that used by the RAND expert panels, such that teams of raters independently rated the 23 headquarters criteria, discussed ratings and areas of discordance, and re-rated. The headquarters also compiled evidence to justify each rating, which included policy documents, interview findings, and other supporting information. To ensure internal consistency across raters, DoD SAPRO in partnership with the CDC Violence Prevention Technical Assistance Center, provided technical assistance via online workshops and in person and group meetings.

Adaptation study 2: Civilian colleges and universities. Ten individuals directly involved in sexual assault prevention, across 9 universities from the United States and Canada, reviewed the criteria items. Given the sensitivity of the subject area, a convenience sample of colleges and universities were recruited for participation in the study. Universities and colleges were asked to participate if they participated in the national discussion on sexual assault prevention (an annual event hosted by DoD) or were engaged in ongoing discussions with members of the research team about their prevention work. Examples of individuals included Title IX Coordinator, Prevention Educator, Health Educator, Vice President of Student Life, and Sexual Misconduct Officer. Individuals were asked to review an adapted version of the criteria and associated guide to self-assessment edited to replace military specific terms with more general terms (e.g., replaced military service member with student).

After reviewing, individuals were asked to rate the relevance of each criteria item for a college or university setting on a Likert scale from 1 (not relevant) to 3 (highly relevant) and to identify whether there were any items that should be added to the criteria. After providing this feedback in written form, individuals participated in a one-hour interview with members of the research team to provide general feedback on the criteria, suggest items that should be added, and identify any items that should be removed. Relevance ratings for each criteria item were averaged, and a list of suggestions for revisions or additions was compiled.

Results

Prevention evaluation Framework: Criteria of best practices for organizational sexual assault prevention

All 15 experts invited to verify the criteria participated in the initial and re-rating of the criteria’s validity and importance, and in the final rating and re-rating of the academies. Table 2 shows the final criteria describing what ‘right’ looks like when an organization uses best practices for sexual assault prevention, which includes 63 items, divided into two sections: those that relate to activities or conditions that broadly apply across an organization (“academy level”, 31 items) and those that relate more narrowly to specific prevention activities conducted by an organization (“activity level”, 32 items). In each section, items are categorized into specific domains. The final criteria contains all 62 of the items the RAND team originally proposed, three of which have been edited for clarity, and one new item proposed by expert panel members. Each item is supported directly by literature (references available upon request).

Pilot test results

Since the focus of this manuscript is on the development of criteria, we do not share specific results characterizing prevention capacity. However, a summary still provides insight into the criteria’s utility. Results of the expert panel members ratings of each academy were provided back to academy leaders. These findings highlighted important gaps in the organizational approaches currently being implemented by academies. These gaps and strengths provide important information to DoD SAPRO on opportunities to leverage strengths and resources needed to fill gaps and ultimately realize the vision of the Prevention Plan of Action. The academy level assessments revealed gaps in three key areas foundational to moving prevention forward across academies: Academies 1) lacked a strategic and coordinated approach to prevention; 2) lacked a robust and skilled prevention workforce; and 3) were not using continuous and integrated data and evaluation in their prevention efforts. The activity level assessment revealed that the prevention activities across academies lacked detailed plans for implementation and evaluation, the fit of some of the activities was not aligned with the needs of academy population, limited capacity (e.g., most academies only had one person devoted part-time to prevention) and resources at the academy level hindered the forward progress of some prevention activities, and prevention activities lacked a plan to sustain effective efforts over time (e.g., champion, funds). Academy leadership confirmed the assessments’ accuracy. Overarching results and planned next steps were included in their annual report (SAPRO, 2019a).

Results of adaptations for Self-Assessment

Adaptation study 1: Strategic Application to DoD Headquarters Organizations. Prior to applying the criteria for self-assessment, DoD SAPRO reviewed and incorporated 150 suggested changes or additions from headquarters organizations and the National Guard, which included the addition of definitions for terms used in the criteria to enhance consistency of ratings. The results of the headquarters self-assessment are outlined in the Department of Defense Fiscal Year 2019 Annual Report on Sexual Assault in the Military (DoD, 2020). Although specific assessment results varied by DoD organization, strengths identified across organizations were in the domains of leadership support and collaborative relationships. Gaps across organizations included having equipped and empowered prevention personnel in place and having measures to assess quality implementation and effectiveness of prevention activities. Each DoD organization that completed the self-assessment also identified key actions they were undertaking to address the identified gaps. For example, DoD SAPRO undertook work to build the capacity of their prevention workforce through the development of a prevention personnel training curricula, provision of technical assistance, and delivery of webinars on key prevention concepts and skills. Importantly, many DoD organizations used the criteria to assess not only their sexual assault prevention capabilities but also their capabilities in other problematic behaviors such as sexual harassment and suicide.

Adaptation study 2: Relevance to civilian colleges and universities. Feedback from reviewers indicated that each of the existing organizational-level criteria were relevant to sexual assault prevention in civilian colleges and universities; no reviewer recommended removal of any item. Reviewers suggested edits to one item to soften language related to policy enforcement since prevention staff at civilian colleges and universities may not always have the responsibility or authority to enforce sexual assault prevention policies. This Title IX authority for addressing sex discrimination often falls under to the United States Department of Education’s Office for Civil Rights. Reviewers also suggested that it may be helpful to provide clarification of certain terms used in the criteria (e.g., rapid results, collaboration).

Reviewers provided suggestions for additional items that are relevant to sexual assault prevention efforts at civilian universities. For the Leadership domain, participants suggested that leadership should: be knowledgeable about the intersection of various public health issues, be open to learning about issues they may not understand, have an intentional vision regarding sexual assault prevention, and make written statements supporting sexual assault prevention. In the Comprehensive Approach domain, reviewers indicated that faculty and staff should be knowledgeable about all local services related to sexual assault prevention and response, specialized programming should be provided for students belonging to marginalized groups (e.g., students who are Black or lesbian, gay, bisexual, transgender, queer + ), and protective factors should be assessed. In the Data domain, a reviewer emphasized the importance of using data from campus climate surveys; when considering Continuous Evaluation, one individual highlighted the need to use evaluation data for subsequent improvement. Finally, in the Workforce domain, one reviewer suggested assessing how well leadership and staff provide consistent messaging through being good role models. Several reviewers had recommendations that spanned two or more domains, suggesting that the criteria place greater emphasis on assessing protective factors in addition to risk factors and on prioritizing student collaboration and feedback.

Discussion

Sexual assault prevention is a significant public health issue affecting millions of Americans in civilian and military settings. Findings from the study suggest that the Prevention Evaluation Framework (i.e., the 63 criteria) contains valid and important criteria that fill an important gap by providing guidance for understanding and assessing what right looks like for a cohesive, comprehensive, and best practice approach to sexual assault prevention across a variety of settings. Experts consistently rated criteria we developed as both important and valid for assessing organization’s alignment with best practices for organizational sexual assault prevention, underscoring that the criteria are strong candidates for assessing organization prevention capacity. The positive feedback from the pilot test and the adaptation studies of the criteria and self-assessment guide suggest promising future applications across organizations.

Findings also suggested that the Prevention Evaluation Framework was also used in the military setting to inform their own prevention plans and begin to build prevention capacity. The pilot test of the criteria, an application of the criteria to military service academies, provided key information about the strengths and gaps of academies organizational capacity, and was used as a starting point for implementing DoD SAPRO’s prevention guidance. Applying the criteria to the strategic level (military headquarters organizations), continued to advance the work outlined in DoD’s Prevention Plan of Action by identifying organizational gaps and strengths at the strategic level, as well as highlighting which criteria were more appropriate for capturing strategic organizational capacities and which criteria were more appropriate for capturing tactical organizational capacities.

The interviews with university and college staff highlighted areas where military and civilian settings may differ and thus the criteria need to be adapted (e.g., policy enforcement). In addition to augmenting the criteria for future application in civilian settings, robust definitions will need to accompany the self-assessment guide to improve future adaptations and applications of the criteria. Both adaptations with military headquarters organizations and with civilian colleges and universities suggested definitions were needed to clarify certain terminology used – especially important given limited workforce capacity.

Additional applications of the prevention evaluation Framework

Continue to evaluate and adapt the framework. In a field where concrete guidelines are needed, the Prevention Evaluation Framework makes a valuable contribution to public health practice of what right looks like for a comprehensive prevention system for sexual assault and harassment prevention. However, continued evaluation and adaptation of these items is warranted. This could be accomplished by applying the RAND/UCLA appropriateness method to other organizational settings (e.g., high schools, community-based organizations), but could also include examining how variation in the uptake of these best practices is related to specific sexual assault outcomes. Future research could also provide valuable insight into which criteria are the most critical drivers of low sexual assault prevalence rates, so that military and civilian settings with limited resources could focus on addressing a sub-set of the criteria. Conducting a study to empirically link the criteria to sexual assault prevalence could reveal how criteria differ across settings and provide the data needed to decide whether the criteria could be narrowed to a subset of essential or core criteria that have the strongest influence on the prevalence of sexual assault.

Apply the framework to prospectively guide prevention planning. The criteria that comprise the Prevention Evaluation Framework can also be used for purposes beyond self-assessment. For public health and primary-prevention practitioners, the criteria provides an actual tool that can be used to guide the design, development, implementation, and evaluation of organizational approaches to sexual assault prevention. For those seeking to implement or evaluate sexual assault prevention policies, considering organizational capacity for prevention may be an important dimension to inform implementation planning and to consider when evaluating why certain policies are (or are not) being implemented fully or demonstrating the intended results.

Extend the framework to other primary prevention efforts (e.g., suicide prevention). As previously mentioned, DoD headquarters organizations also used the criteria to assess their capabilities in other problematic behaviors such as suicide. The extension of the criteria to these areas suggests applicability to other domains of prevention. Future adaptations to other areas of prevention could be achieved by adjusting the comprehensive approach to prevention section. Most of the other dimensions in the criteria (leadership, workforce capacity, etc.) would continue to be relevant regardless of the specific prevention area targeted. Improving organizational capacity for sexual assault prevention could also have benefits for other organizational prevention efforts, given this overlap of capacities. Given the primary prevention focus on individual prevention programs—rather than comprehensive, cohesive, and best practice approaches to prevention that cut across organizations, the use of expert guidance and a multi-faceted literature review can help provide guidance about what right looks like in this evolving field.

Continue to update the framework. The criteria will need to continue to be updated as research progresses and new insights about what right looks like emerge. Future adaptations of the criteria may even focus on a comprehensive, cohesive, and best practice approach to target the shared risk and protective factors across multiple forms of violence (Wilkins, Tsao, Hertz, Davis, & Klevens, 2014).

Limitations

The criteria (and related literature review) focuses only on organizational and programmatic level approaches to prevent sexual assault; it does not account for comprehensive community approaches often used in public health population level interventions and it does not differentiate how specific approaches should vary within military settings (e.g., academy vs. installation). The study also relied on criteria developed through literature review and the feedback of experts. A different set of experts may have developed slightly different criteria and may have rated the academies differently than the experts that developed the criteria. However, we elected to use the same group of experts to both construct the criteria and do the initial pilot test rating the academies because experts would then have a complete understanding of what was meant by each item and understand its importance to the overall organization. Finally, the current article is limited to only the findings from the application of the criteria to civilian colleges and universities, and select military settings (the three academies, a sub-set of military installations, and a single division in military headquarters). However, the fact that the criteria items remained consistent and relevant across these military and civilian settings reflects their importance.

Conclusion

Given the limited availability of evidence-based organizational approaches to sexual assault prevention, and the growing emphasis and urgency for primary prevention approaches to sexual assault, relying on expert guidance and a thoughtful literature review to develop criteria of best practices reflects an effective way of promoting what right looks like. The pilot test and adaptation study findings across military and civilian settings suggest the criteria we developed may represent an important step towards guiding future organizational efforts to prevent sexual assault and provide a formative self-assessment tool for organizations looking to understand their own strengths and areas for improvement.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.