The Dating Matters® Toolkit: Approaches to increase adoption, implementation, and maintenance of a comprehensive violence prevention model.

Purpose: This practical implementation report describes a comprehensive teen dating violence prevention model-Dating Matters®-and the approach taken to maximize its potential for widespread dissemination through development of the Dating Matters Toolkit. Dating Matters has evidence of effectiveness for preventing teen dating violence and other adolescent risk behaviors from a multi-site randomized controlled trial. Identifying strategies that reduce barriers to the dissemination and implementation of evidence-based comprehensive prevention models, like Dating Matters, is critical to their widespread adoption. Lessons learned from creating the Toolkit can inform the development and dissemination of similar comprehensive prevention strategies and speed their adoption and use in the field. Approach: We engaged in a multi-pronged, data-driven approach to maximize adoption, implementation, and maintenance of the Dating Matters model based on multiple data sources and lessons learned from the demonstration project, while retaining core components and content of the evaluated model. Potential barriers to the national dissemination of Dating Matters were identified in four key areas: training, technical assistance, model flexibility, and accessibility. A series of modifications were made to the implementation model to address these challenges and facilitate scale-up prior to national dissemination. We outline these challenges and describe solutions implemented through the development of the Dating Matters Toolkit. Outcomes: The Dating Matters Toolkit includes web-based facilitator training, enhanced implementation support through a new staff role and community of practice, online comprehensive implementation guidance and resources, and added flexibility to improve feasibility and adoption in communities during capacity-building. Findings from an initial evaluation of the Toolkit suggest users perceived the comprehensive model as feasible and accessible with enough implementation support. Most expressed interest in adopting the model. Users described cost and stakeholder buy-in as remaining barriers. Ongoing improvements to the Toolkit to address these challenges and future research to evaluate its effectiveness are planned.

The field of violence prevention continues to evolve and expand its impact by developing new prevention approaches based in research and practice evidence to address risk for violence and prevent it before it begins (Basile et al., 2016; Fortson et al., 2016; Niolon et al., 2017). Although many prevention programs have achieved positive outcomes, they often demonstrate small or time-limited effects (Community Preventive Services Task Force, 2018; DeGue et al., 2014; Whitaker et al., 2006). As such, the field of violence prevention increasingly recognizes the importance of building comprehensive prevention strategies that include multiple, coordinated prevention approaches across the social ecology to strengthen outcomes beyond the effects of single-component interventions (Basile et al., 2016; DeGue et al., 2016; Frieden, 2010; Niolon et al., 2017). Strategies that address risk and protective factors across relationship types and contexts by pairing evidence-based prevention programming with, for example, social and environmental change approaches that reinforce positive norms and behaviors may be more effective in increasing healthy behaviors and reducing rates of violence—the ultimate goal of the public health approach to violence prevention (DeGue et al., 2016; Rothman et al., 2015).

Research from the field of youth violence prevention suggests that comprehensive prevention strategies can be more effective than single-component approaches (David-Ferdon & Simon, 2014; Matjasko et al., 2012). Yet, these multi-component, integrated prevention models may be more costly and challenging to implement (Elliott & Mihalic, 2004; O’Connell et al., 2009; Rothman et al., 2015). Thus, identifying strategies that reduce barriers to the dissemination and implementation of comprehensive prevention models is critical to their widespread adoption. To advance these efforts, this practical implementation report describes one comprehensive prevention model—Dating Matters®: Strategies to Promote Healthy Teen Relationships (Dating Matters)[1]—and the approach taken by the Centers for Disease Control and Prevention (CDC) to maximize its potential for adoption, implementation, and maintenance for national dissemination while reducing costs to communities through development of the Dating Matters Toolkit. Lessons learned from this process can inform efforts to develop and disseminate similar comprehensive prevention strategies to speed their adoption and use in the field.

CDC-developed Dating Matters, starting in 2009, to address growing concerns from the public and policymakers about the risks of teen dating violence, and the need for evidence-based prevention strategies that advanced the field beyond the individual-level, school-based programs that were standard practice at the time (Teten Tharp et al., 2011). Dating Matters built on existing evidence to create multiple prevention components that address risk and protective factors for youth as well as their parents, schools, and neighborhoods. The model sought to reach adolescents at younger ages—11 to 14 years old—in middle school to build healthy relationship skills early and prevent violence before it begins. Furthermore, it was designed to leverage the infrastructure and capacity of local health departments to bring together multiple sectors (e.g., education, government, business, health, nonprofit) and stakeholders (e.g., educators, youth, parents, public health, youth-serving and violence prevention organizations) for implementation.

Dating Matters includes several core components: school-delivered programs for youth in sixth, seventh, and eighth grades; community-based programs for parents of sixth, seventh, and eighth graders; online training for school staff on their role in preventing dating violence; a youth communications program implemented by high school-age “brand ambassadors”; and activities that build prevention capacity, inform local policy, and track relevant data at the community level (see Figure 1). Each of the multi-session youth and parent programs provide interactive opportunities to build healthy relationship skills (e.g., communication, making healthy choices, emotion regulation, help-seeking and victim support, positive parenting) and develop positive norms and behaviors. Because the model was developed to build on existing evidence-based strategies, eighth-grade youth and their parents receive evidence-based teen dating violence prevention programs that pre-dated the development of Dating Matters (i.e., Safe Dates; Families for Safe Dates; Foshee et al., 2004, 2012), and parents of sixth graders receive an evidence-based sexual health promotion program adapted for Dating Matters to also address dating violence (i.e., Parents Matter!; Dittus et al., 2004). CDC developed the remaining youth and parent programs and other intervention components specifically for Dating Matters.

Figure 1.

The Dating Matters® comprehensive teen dating violence prevention model.

CDC implemented and rigorously evaluated Dating Matters in a multi-site demonstration project from 2011 to 2016. Forty-six middle schools across four sites were randomly assigned to implement either the Dating Matters comprehensive model or a standard-of-care youth program (Safe Dates) in eighth-grade only. Findings indicated that the Dating Matters comprehensive model was more effective in preventing teen dating violence perpetration and victimization as well as a range of other risk outcomes during middle school than an evidence-based, single-program approach (DeGue et al., 2020; Estefan et al., 2020; Niolon et al., 2019; Vivolo-Kantor et al., 2019). A long-term follow-up of these youth through high school (in progress; see Niolon et al., 2016 for a discussion) will provide more information on whether these effects are sustained over time, and replication of these findings overall is needed. However, current evidence suggests that Dating Matters is an effective violence prevention strategy.

Key factors in bringing effective prevention to scale

Disseminating evidence-based strategies for widespread adoption is the final, and perhaps most critical, phase of the public health approach[2] to violence prevention (Mercy et al., 1993). Yet there are multiple challenges involved in bringing evidence-based prevention strategies to scale in “real-world” settings where resources, capacity, and other key supports may be more limited than in controlled and typically well-funded research settings (Bumbarger et al., 2010). This may be particularly true of comprehensive strategies that require coordination of several activities, potentially involving multiple organizations, funding sources, populations, partnerships, and stakeholders.

Several frameworks exist to guide the design, translation, dissemination, and implementation of evidence-based interventions (Bartholomew et al., 2011; Glasgow et al., 1999; Green & Kreuter, 2005; Klesges et al., 2005; Proctor et al., 2011). One such framework, RE-AIM, identifies five dimensions for evaluating implementation efforts: reach (i.e., extent of individual-level participation), efficacy (i.e., intended health outcomes), adoption (i.e., extent of uptake in relevant settings and organizations), implementation (i.e., intervention delivered as intended or with fidelity), and maintenance (i.e., sustainability of intervention effects, implementation, and “institutionalization” within the organization or community) (Glasgow et al., 1999). Here, we focus on three key factors in implementation success that are common across existing frameworks—adoption, implementation, and maintenance—applying terminology and definitions from the RE-AIM framework, where possible, for consistency (Glasgow et al., 1999). Attention to these critical factors may help overcome the challenges unique to comprehensive prevention and boost their potential for broad dissemination and effective implementation in the field.

Adoption

Despite their potential for greater impact, comprehensive prevention strategies are inherently more complex and more resource-intensive to implement than single programs (Barton et al., 1997; DeGue et al., 2016; Nation et al., 2003). Communities often have limited resources and capacity to address multiple needs. Decisions regarding the adoption of prevention strategies are influenced by a number of factors, including perceived feasibility, intervention complexity, acceptability, availability of implementation supports, community readiness, and compatibility with the existing infrastructure (Bumbarger et al., 2010; Proctor et al., 2011; Rogers, 1995). Attention to these factors during intervention planning and development can increase the likelihood of adoption and facilitate broader uptake in the field.

The perceived feasibility of an intervention—its simplicity, accessibility, flexibility, ease of use, and cost—is a critical factor that can either facilitate or impede adoption decisions (Dusenbury & Hansen, 2004; Orlandi, 1986). The more clearly and coherently an intervention can be communicated to decision-makers, the lower its perceived complexity and the more likely it is to be adopted (Orlandi, 1986). Some evidence also suggests that more complex programs are less likely than their simplified counterparts to be implemented completely (Smith et al., 1993). Offering effective implementation supports (e.g., accessible training, clear guidance) or flexible implementation options that reflect the diverse capacities of communities may increase perceived feasibility and adoption (Bartholomew et al., 2011). Furthermore, approaches to implementation (e.g., training model, implementation mode, setting, or supports) that minimize costs without sacrificing effectiveness can place the intervention within reach of more communities. Thus, efforts to improve actual and perceived feasibility are critical to promoting adoption and may also improve the quality of implementation and potential for maintenance.

Implementation

Research has demonstrated the importance of high-quality implementation when carrying out evidence-based programs—higher fidelity results in better outcomes (Bumbarger et al., 2010; Elliott & Mihalic, 2004; Glasgow et al., 1999). However, in real-world, practice settings, maintaining fidelity can be a challenge affected by many factors, including school or community capacity, training quality or consistency, implementer supports and resources, intentional adaptations to address community needs, or unplanned adaptations in response to implementation barriers (Botvin, 2004; Bumbarger et al., 2010; Elliott & Mihalic, 2004; Kemp, 2016). Thus, addressing these potential challenges during intervention development and dissemination, and ensuring access to implementation guidance and supports, is a critical consideration when disseminating evidence-based interventions.

The importance of highly trained facilitators is well-documented (Bumbarger et al., 2010; Nation et al., 2003). Interactive training that goes beyond the “how-to’s” to help facilitators understand the theory behind an intervention, along with the provision of ongoing support, can help facilitators engage fully with their role and navigate barriers to fidelity (Bumbarger et al., 2010). For example, providing supplementary training for teacher-implementers, beyond manualized guidance, has been shown to improve fidelity and effectiveness in school-based prevention programs (Wolfe et al., 2009). Balancing fidelity with the potential need for adaptations is a well-recognized challenge in the field; unplanned adaptations can create “drift” and threaten effectiveness, while strict adherence to an “out-of-box” intervention can threaten acceptability and restrain useful innovations in real-world settings (Bumbarger et al., 2010; Kemp, 2016). Guidance on the use of planned adaptations to address the needs of special populations or respond to other challenges (e.g., time limitations) can allow implementers to make necessary adjustments without sacrificing effectiveness (Castro et al., 2004). Identifying common barriers to fidelity can also inform improvements to materials or training that reduce unplanned adaptations (Melde et al., 2006). Furthermore, continual efforts to assess and build capacity (i.e., the resources—knowledge, skills, staff, partners, tools—an organization needs to successfully implement and sustain a prevention model) can help prevent or address challenges with technical capacities, resources, or community buy-in that affect fidelity (Elliott & Mihalic, 2004). Implementing multi-component, comprehensive prevention models with fidelity poses unique challenges but addressing them during program development and refinement with an effective, efficient implementation model may improve uptake, implementation quality, and outcomes.

Maintenance

Without planning for long-term maintenance, interventions—and their effects—may be short-lived (Cooper et al., 2015; Lyon et al., 2011). Maintenance planning early in the process of intervention development is critical (Bumbarger et al., 2010). Sustainable interventions require sufficient organizational capacity and infrastructure, effective partnerships, strong leadership support; “fit” between the intervention and community needs, and access to resources and expertise to sustain the intervention (Cooper et al., 2015; Fixsen et al., 2005; Johnson et al., 2004). To be maintained, interventions must continue to meet the needs of stakeholders over time with the potential for integration into the ongoing operations, or infrastructure, of an organization (Bumbarger et al., 2010; Johnson et al., 2004). Comprehensive prevention strategies may be more challenging to maintain but attention to maintenance planning, continuously building organizational and community capacity for prevention, keeping stakeholders engaged and involved, adapting to the needs and capacity of the community, and integrating the necessary staffing into the organizational infrastructure may increase the potential for long-term maintenance.

Developing the Dating Matters Toolkit

CDC developed the Dating Matters Toolkit,[3] a comprehensive package of implementation tools, training, and guidance based on lessons learned from the demonstration project. Development of the Toolkit focused on supporting adoption, implementation, and maintenance in real-world settings through the creation of new resources as well as modifications to the implementation model. While consideration of these factors also occurred during intervention development, revisiting and attending to these concerns while planning for dissemination is an important step in bringing evidence-based interventions to scale (Bartholomew et al., 2011).

Qualitative review of several existing data sources informed the development of the Toolkit for national dissemination. First, in-person focus groups were conducted at each of the four evaluation sites with youth (N = 61; four groups) and parents (N = 27; four groups) who participated in Dating Matters programs during the demonstration project. Focus groups with youth assessed recall of the program content and the extent to which students used or shared its messages with others, as well as program strengths and areas for improvement. Parent focus groups sought to understand parent recall and use of the program content, program strengths and areas for improvement, and ways to improve parent participation for future implementation. Second, structured 1-hr interviews were completed with five individuals who implemented Dating Matters during the demonstration project. Participants provided feedback on their experience with youth and parent programs, the technical assistance model, the community-level components, and general recommendations for improvements. For both the focus groups and key informant interviews, the authors reviewed key findings summarized by federal contractors based on their qualitative analysis of participant feedback. Third, the authors reviewed implementation data collected during the demonstration project, including 4 years of session logs completed by program facilitators after each session of the youth programs (sixth grade, N = 404; seventh grade, N = 401) and parent programs (sixth grade, N = 76; seventh grade, N = 70) to identify common challenges with fidelity. Session logs contained both quantitative and qualitative data. We did not review implementation data for eighth-grade youth and parent programs as both are existing evidence-based programs distributed by a private publisher and, as such, the program content could not be modified as part of the Toolkit development (i.e., Safe Dates, Families for Safe Dates; Hazelden Publishing). However, the Toolkit was designed to support and improve implementation of the entire prevention model, including these eighth-grade programs. Fourth, two experienced program facilitators from the demonstration project reviewed the CDC-developed sixth- and seventh-grade youth and parent programs in detail and provided specific feedback based on their experience implementing the programs. They provided recommendations for revisions or improvements to the activities, messaging, and implementation guidance with the goal of improving acceptability and relevance to participants and ease of use for facilitators.

Through qualitative review of these data sources and lessons learned from the demonstration project, we identified several potential challenges for the national dissemination of Dating Matters. These challenges fell into four key areas aligned with the goals of improving adoption, implementation, and maintenance: training, technical assistance, model flexibility, and accessibility. With an intentional focus on retaining core content and hypothesized mechanisms of change, a series of modifications, updates, and improvements were made to the Dating Matters implementation model to address these challenges and facilitate scale-up prior to national dissemination. To inform implementation practice, we outline these challenges and solutions implemented to address them here as examples of strategies that could be used to improve the dissemination potential of future comprehensive prevention models (see Table 1 for a summary).

Table 1.

Summary of key modifications to the Dating Matters® implementation model for national dissemination.

Challenge	Demonstration project (original model)	Dating Matters Toolkit[a] (current model)
Training	Train-the-trainer model Training for master and local trainers (40–50hr, in person) Program facilitator training conducted by local trainer (40–50 hr, in person)	Free, web-based program facilitator training (~4 hr) In-person training activities (~2 hr) and ongoing supervision provided by a Dating Matters Coach[b] Coaches’ Playbook[c] provides training for Dating Matters Coaches
Technical assistance	Technical assistance contractor, federally funded	Dating Matters Coach provides supervision and support to program facilitators and partner organizations (e.g., schools) Team Up! for Dating Matters online community of practice[d] Dating Matters Guide to Implementation[e] Coaches’ Playbook guides Dating Matters Coaches in providing supervision and support to program facilitators
Model flexibility	All model components required by federal funding	Implementation guidance supports phased implementation of selected components while building capacity for comprehensive prevention Adaptation guidance for youth and parent programs
Accessibility	Accessible to communities receiving federal funding for implementation Many program materials requiring high-cost professional printing	Free online access to all program, training, and implementation materials and tools through the Toolkit website[a] At-a-Glance briefs provide decision-makers with easy to access, high-level information on the model Most materials available for free download and low-cost self-printing Select professionally printed materials available at no cost to communities[f]

The Dating Matters Toolkit is available at https://vetoviolence.cdc.gov/apps/dating-matters-toolkit/

Dating Matters Coaches provide training, supervision, and support to facilitators of the youth and parent programs, and are typically employed (one Coach per community) by the health department or community-based organization leading Dating Matters implementation. Coaches complete the free, online facilitator training with additional guidance provided in the Coaches’ Playbook, at no additional cost to communities.

The Coaches’ Playbook is available free online: https://vetoviolence.cdc.gov/sites/vetoviolence.cdc.gov.apps.dating-matters-toolkit/themes/dmh_ng_bootstrap/assets/pdf/DM-Coaches-Playbook-RGB-508-8-2.pdf

Team Up! is a free, peer-led, CDC-moderated online community of practice accessible through a free mobile or desktop application: https://vetoviolence.cdc.gov/apps/dating-matters-toolkit/team-up#/

The Dating Matters Guide to Implementation provides an overarching roadmap for lead organizations interested in implementing the Dating Matters comprehensive model, and is available free online: https://vetoviolence.cdc.gov/sites/vetoviolence.cdc.gov.apps.dating-matters-toolkit/themes/dmh_ng_bootstrap/assets/pdf/DM-Guide-to-Implementation-RGB-508-8-2.pdf

Some materials included in the i2i: What R U Looking 4? youth communications program were designed for professional printing to increase their appeal to youth. These materials can be ordered from CDC in print at no cost. See: https://vetoviolence.cdc.gov/apps/dating-matters-toolkit/i2i-program-facilitator#/. In addition, select additional materials that benefit from professional printing, such as spiral-bound facilitator manuals, can also be ordered for free from CDC in addition to being available for free download and self-printing. All CDC-developed program and implementation materials are available at no cost to communities in print or as a free download.

Training

Accessible, efficient, and effective training is critical to supporting the adoption, implementation, and maintenance of an intervention. Training for facilitators of the Dating Matters youth and parent programs during the demonstration project occurred in 5- or 6-day in-person sessions. Using a train-the-trainer approach, CDC trained a small group of master trainers at the national level who then trained staff at each site to serve as local trainers for facilitators throughout implementation. Continuing this training model posed several challenges for scale-up. First, it was time-intensive, costly, and required an infrastructure of certified master trainers available to train new communities. Furthermore, interviews with key staff suggested there was substantial variation in fidelity to the training curricula when carried out by local trainers at each site, indicating potential for drift in the quality or content of facilitator training over time. Finally, the potential for high turnover of program facilitators (common in most community-based interventions) posed a challenge for training to new facilitators as they are hired.

To address these challenges for national dissemination, CDC developed a free, web-based program facilitator training that substantially reduces the time and cost involved in training and ensures all facilitators receive consistent training content. Although substantially condensed, the training is interactive, covers core content from the in-person trainings, and includes videos of select sessions and skills demonstrated by experienced facilitators from the demonstration project, with actors portraying youth and parent participants. The total training time for a youth or parent program facilitator is about 6 hr. This includes about 2 hr of online material, 2 hr of self-directed “homework” activities, and 2 hr of in-person, one-on-one coaching.

To supplement the online training, a new staff role—the Dating Matters Coach—was developed to ensure facilitators receive the training, supervision, and support needed to implement the programs with fidelity. Coaches are local staff employed by the implementing organization to provide ongoing implementation support and technical assistance to facilitators and oversee youth and parent program implementation, including management tasks such as coordination with sites and data collection for program evaluation. Coaches will ideally be experienced and skilled health educators with the ability to supervise and support youth and parent program facilitators. A full-time Coach position is recommended for large-scale implementation (e.g., multiple schools and organizations), but the role can also be filled by part-time staff or a full-time staff member holding multiple roles (e.g., Coach and program facilitator) in small-scale implementation, providing communities with greater flexibility in staffing. With more time devoted to Dating Matters than a typical program facilitator, a Coach can engage more deeply with the program content, consider the need for adaptations, connect with Coaches in other communities, problem-solve, and develop best practices for their community that can be shared with facilitators to improve implementation. Coaches complete the same free online training as facilitators and have access to a Coaches’ Playbook[4] with detailed guidance for carrying out their training and supervisory roles.

By moving to an online training model with support provided by a Coach, communities have low-cost access to flexible, feasible training that is high-quality and consistent—overcoming important barriers to adoption, implementation, and maintenance. Although staffing a Coach position may require significant resources, these resources not only facilitate the initial training needed to carry out multiple programs but provide ongoing support to ensure quality implementation. Program facilitators often provide services through in-kind partnerships with local organizations, as part of their regular work as teachers or school staff, or through short-term contracts. Thus, integrating key prevention staff, like a Coach, into the infrastructure of the lead organization overseeing implementation can help build capacity and support maintenance.

Technical assistance

Beyond training, a range of implementation supports are needed to ensure organizations have the capacity, knowledge, and resources needed to carry out a multi-component prevention strategy. Technical assistance (e.g., answering questions, problem-solving, creating supplemental guidance, providing additional training or instruction) was provided to sites in the Dating Matters demonstration project through a federally funded contracting organization. Continuing this model for national dissemination would limit the number of communities that could access these supports to the capacity of a contract. In lieu of this model, CDC built alternate sources of technical assistance into the dissemination plan to ensure that all communities would have access to the guidance and support needed to implement Dating Matters with fidelity. Together, these resources aim to make technical assistance freely available to all communities—decreasing the perceived complexity of the intervention, increasing the potential for greater adoption, and improving the ability of organizations to implement the model with greater fidelity and impact.

One key source of technical assistance, as noted above, is the Dating Matters Coach, a role created to ensure program facilitators have access to an experienced, well-trained supervisor to address questions, challenges, and other issues that arise during program implementation. Rogers (2002) also highlighted the importance of peer support and activating peer networks to diffuse preventive interventions by encouraging peer communication and changing system norms. Thus, to promote peer-to-peer support and learning, CDC also created an online community of practice, Team Up! for Dating Matters,[5] moderated by CDC and accessible through a free mobile app and online. Communities implementing Dating Matters are encouraged to share questions, tips, challenges, solutions, and success stories on a secure and accessible platform. Participants can also interact with and receive assistance from CDC staff and experienced Dating Matters facilitators from the demonstration project engaged in Team Up! as consultants. Everyone involved in implementation—from lead staff to program facilitators—is encouraged to join.

The Dating Matters Toolkit also includes a comprehensive package of implementation guidance and resources for each component of the prevention model as well as for the model itself. Lessons learned from the demonstration project indicated that a key challenge in standing up a comprehensive prevention model like Dating Matters, is knowing where to start and how to build and maintain the necessary infrastructure. The Dating Matters Guide to Implementation[6] (CDC, 2019) was developed to provide complete guidance for carrying out the comprehensive prevention model, including information on building capacity and partnerships, planning a budget, hiring staff, recruiting schools and participants, adaptation, program evaluation, and sustainability. It was designed to answer common questions and challenges experienced by communities in the demonstration project, and includes detailed information to help lead staff navigate the challenges of standing up a multi-component prevention strategy.

Model flexibility

Communities vary in their initial capacity to carry out evidence-based prevention efforts in terms of expertise, resources, and stakeholder support. Building flexibility into an intervention or implementation model allows communities to start where they are and grow their capacity over time, increasing the potential for adoption and maintenance (Bartholomew et al., 2011). This could be particularly important for comprehensive prevention strategies that may require more resources and partnerships to implement than a single-program approach.

Dating Matters was designed and evaluated as a comprehensive, multi-level prevention strategy comprising seven programs and three community-level prevention activities. To achieve the preventive effects demonstrated in the evaluation (e.g., Niolon et al., 2019), implementation of the entire model is recommended. However, to increase its feasibility and adoption potential for national dissemination, the Toolkit was designed to support communities interested in implementing only select components of the model as they build capacity for comprehensive prevention. For example, communities interested in implementing only the youth programs to start can easily access the training, program materials, and implementation guidance that is specific to those programs. Staffing recommendations for the model were also designed to be flexible, allowing smaller organizations or communities to begin these efforts with existing resources and expand their infrastructure, staff, and expertise as their capacity and resources grow. The Dating Matters Capacity Assessment and Planning Tool[7] (DM-CAPT) is available to help communities assess and improve their prevention capacity over time, with the goal of integrating additional components into their efforts as capacity increases and achieving comprehensive prevention as designed. By building flexibility into the implementation model for this comprehensive prevention strategy, Dating Matters can reach more youth, families, and neighborhoods sooner while leveraging successes from those efforts to engage additional partners, secure necessary resources, and build an infrastructure that can be maintained and is ready for expansion. Providing opportunities for pre-implementation work to assess and address community readiness (e.g., needs, capacity, infrastructure) may create “fertile ground” to facilitate strategy adoption and increase the likelihood of quality implementation and maintenance (Bumbarger et al., 2010).

Accessibility

Adoption decisions are influenced by several factors, including perceived feasibility and complexity (Bumbarger et al., 2010; Orlandi, 1986). Accessible interventions—those that require fewer resources to obtain, understand, or assess for fit—may have greater potential for adoption and, thus, widespread dissemination. A key modification of the Dating Matters implementation model for national dissemination involved the creation of an online hub—the Dating Matters Toolkit website[8]—that provides free access to all CDC-developed resources and materials necessary to implement the prevention model. The interactive website is organized by staff role and prevention component to help users identify the materials they need most quickly and reduce the perceived complexity of the intervention. In addition, brief “At-a-Glance” documents are included to provide interested communities and decision-makers with a high-level introduction to the overall model and each component, including its goals and the resources required. During the demonstration model, CDC provided professionally printed program materials to each site. To reduce costs and increase access, most program materials were re-designed prior to national dissemination to allow free downloads from the Toolkit website and low-cost self-printing. Select materials that still require professional printing can be ordered from CDC for free.

In the first 6 months that the Toolkit was available, the website was viewed thousands of times and more than 100 unique organizations requested access to the program materials and facilitator training by completing a required user agreement. Requestors most often represented violence prevention or response organizations (e.g., domestic violence/sexual violence prevention, shelters, victim services), 56%; state and local governments, 12%; or K–12 schools, 9%. Although there is not currently a mechanism to track uptake and implementation of the Dating Matters model nationwide, these early indicators of interest and adoption suggest substantial potential for the reach of this comprehensive prevention approach.

Evaluating the Dating Matters Toolkit

CDC conducted a mixed methods evaluation of the Dating Matters Toolkit in Spring 2020 to examine usability and impacts on the adoption, implementation, and maintenance potential of the model. Individuals were invited to complete an anonymous 18-item survey online through a public link on the Toolkit website as well as emails sent to registered users and key CDC partners in dating violence prevention. Survey respondents were asked to provide contact information if they would be willing to participate in a 30 min follow-up interview. No incentives were provided. Interviews were administered by trained facilitators through a federal contract with informed consent from participants. Interview data was analyzed in MAXQDA 2018 for patterns relating to website navigation, program and material comprehension, interest and perceived feasibility of programmatic implementation, and user satisfaction.

Results

Survey participants (N = 70) had diverse professional backgrounds ranging from youth-serving professionals and victim advocates to law enforcement; 84% reported working in violence-related fields. Interview participants (N = 16) also represented a diverse set of stakeholders from front-line prevention staff to organizational leadership and technical advisors to federal grant recipients serving youth. Among interview participants, 19% were currently implementing Dating Matters, 44% were considering integrating a Dating Matters component into their current work, and 38% had never implemented Dating Matters and had no current plans to start. Notably, very few participants had experience with Dating Matters implementation at the time of data collection due to the recent release of the Toolkit and program materials (about 6 months prior) and the COVID-19 pandemic that closed most US schools in the Spring of 2020, side-tracking implementation planning for school-based and in-person interventions. Thus, these data reflect, primarily, the experience of Toolkit users as they utilized the website to learn about the prevention model, assess its feasibility and potential acceptability in their community, and begin planning for future implementation.

Survey and interview data provide a glimpse at perceptions of adoption and implementation feasibility. Survey data indicated that three-quarters of respondents agreed (54%) or strongly agreed (21%) that the Toolkit contained all of the implementation supports tools and materials needed for adoption and implementation. When asked whether it would be possible to implement all seven components of Dating Matters with the right resources and partnerships in place, the vast majority of respondents indicated it was possible (51%) or definitely possible (31%), with only two respondents indicating it was impossible (2.8%). A subset of 28 survey respondents were asked about potential barriers to adopting and implementing the comprehensive model and reported insufficient financial or human resources or a lack of community buy-in as concerns most often (see Table 2). Likewise, most adoption challenges that participants discussed during interviews were related to gaining buy-in from key stakeholders (including teachers, school officials, leadership in their organizations, student ambassadors, and parents) and having time and funding to carry out the programmatic activities in their entirety. Yet, responses in both the survey and interviews reflected strong interest in adopting and implementing Dating Matters. Most survey respondents indicated that they were interested (37%) or very interested (47%) in adopting and implementing one or more components of Dating Matters their community or organization.

Table 2.

Barriers to Dating Matters implementation using the Dating Matters Toolkit, reported by survey respondents (n = 28).

Barrier reported	% (n)
Too expensive/lack funding	32 (9)
Insufficient staffing or expertise	25 (7)
Lack of community support or “buy-in”	25 (7)
Lack of necessary community partnerships	18 (5)
Already implementing other complementary strategies	18 (5)
Too difficult	14 (1)
Do not see the need or potential benefit of comprehensive prevention	4 (1)

These findings suggest that the Dating Matters Toolkit and modified model described here may support improved adoption and implementation, as intended. However, anticipated challenges related to resources and engaging stakeholders may continue to be important barriers to adoption of the entire comprehensive model as well as maintenance of those components that are adopted. The Toolkit may provide a useful model for developers of multi-component prevention strategies engaged in dissemination planning. However, additional research and continual improvements to the Toolkit are needed to meet the challenges that communities face and to bring effective prevention to scale.

Considerations for future research and dissemination

Not all communities interested in dating violence prevention will have the initial capacity to carry out comprehensive prevention and may only have the resources or partnerships needed for select components (e.g., school-based programs). As such, the Toolkit was designed to support adoption and implementation of individual components, when needed, to meet the needs of more communities. However, implementation that deviates from the model, as evaluated, has implications for effectiveness. While the complete, comprehensive model has evidence of effectiveness for preventing teen dating violence and related outcomes (Niolon et al., 2019; Vivolo-Kantor et al., 2019), many of the individual components are also either evidence-based or evidence-informed (i.e., youth and parent programs) and/or have no existing evidence-based equivalents in the field (i.e., training for educators, youth communications program, policy and indicator data activities). The components have also been implemented successfully with middle school students in underserved communities where limited research is available. Thus, the individual components may still represent the best available options for some communities investing in dating violence prevention (Niolon et al., 2016, 2019). Ongoing program evaluation in the field will be critical in assessing the effects of subsets of the comprehensive model on key outcomes, as implemented using the Toolkit resources. Program evaluation guidance, including suggested process and outcome measures, is provided in the Dating Matters Guide to Implementation.

Future research that examines the impact of the Toolkit on adoption decisions, implementation quality, and maintenance is needed. Although Dating Matters has demonstrated effectiveness in preventing teen dating violence, it is possible that the Toolkit may result in strengthened outcomes through improved feasibility, fidelity, support, and maintenance planning in the future. Conversely, it is possible that some modifications could have unexpected negative impacts on effectiveness and outcomes. Implementation research conducted in the field where communities have attempted to implement Dating Matters utilizing only the Toolkit—without the additional financial and technical supports provided by CDC in the demonstration project—would greatly improve understanding of this dissemination model, its efficacy, and future improvements. Such research could also assess the extent to which communities are able to adopt the entire comprehensive model as designed or choose to focus their resources on a subset of components. Although we would anticipate reductions in effectiveness as the intervention becomes less comprehensive—and closer to a standard single-program approach—opportunities to evaluate subsets of the model as implemented in the field, particularly those components with the greatest stand-alone uptake, would be valuable. These evaluations could help inform decision-makers considering investment in comprehensive prevention by identifying core components, the unique effectiveness of subsets of interventions within the model, or quantifying reductions in effectiveness as the model is scaled back. It is possible that such research could also help identify opportunities to “trim the fat” and create additional efficiencies for implementation without sacrificing effectiveness, further improving potential for dissemination (Bumbarger et al., 2010).

It is important to balance the need for interventions to “evolve” over time in response to new evidence with the risks of modifying them in ways that could impact efficacy demonstrated in prior evaluations. As more is learned from implementation research and practice, the need for further modifications or adaptations to the Dating Matters intervention model or implementation guidance may arise. CDC is applying lessons learned from the user surveys and interviews described here to continually improve and expand the content and delivery of the Toolkit through an iterative, rapid feedback loop. For example, CDC is currently developing detailed program evaluation guidance to increase the capacity of communities to take on this crucial activity as part of their implementation planning. The design of the Toolkit website itself is also being streamlined to ensure greater usability and ease of access. In addition, to meet the needs of communities during the COVID-19 pandemic, CDC is creating adaptation guidance in 2020 to help schools and organizations implement Dating Matters programs online. New research on teen dating violence might suggest ways to strengthen outcomes by building on or revising the existing strategies in the future. It is important to balance the need for interventions to “evolve” over time in response to new evidence with the risks of modifying them in ways that could impact efficacy demonstrated in prior evaluations. Researchers have pointed to the benefits of using iterative, rapid research designs to continually optimize—rather than concretize—interventions as an innovative shift toward improving the timeliness and responsivity of research (Riley et al., 2013). Such approaches could be useful for assessing the effects of model modifications on key outcomes as those changes are implemented over time, and would be consistent with recommendations within the field that adaptations to evidence-based interventions necessitate additional evaluation (Bartholomew et al., 2011; Elliott & Mihalic, 2004).

Given the unique challenges and potential benefits of implementing comprehensive prevention approaches, learning more about what works—and what does not—in real-world settings is critical to promoting scale-up of these prevention strategies in the future. Attending to adoption, quality implementation, and maintenance during intervention development, planning for dissemination, and supporting an iterative process with ongoing evaluation, may help make the promise of multi-level, coordinated, and evidence-based prevention strategies a reality for more communities.