Letting Go: Conceptualizing Intervention De-implementation in Public Health and Social Service Settings.

The discontinuation of interventions that should be stopped, or de-implementation, has emerged as a novel line of inquiry within dissemination and implementation science. As this area grows in human services research, like public health and social work, theory is needed to help guide scientific endeavors. Given the infancy of de-implementation, this conceptual narrative provides a definition and criteria for determining if an intervention should be de-implemented. We identify three criteria for identifying interventions appropriate for de-implementation: (a) interventions that are not effective or harmful, (b) interventions that are not the most effective or efficient to provide, and (c) interventions that are no longer necessary. Detailed, well-documented examples illustrate each of the criteria. We describe de-implementation frameworks, but also demonstrate how other existing implementation frameworks might be applied to de-implementation research as a supplement. Finally, we conclude with a discussion of de-implementation in the context of other stages of implementation, like sustainability and adoption; next steps for de-implementation research, especially identifying interventions appropriate for de-implementation in a systematic manner; and highlight special ethical considerations to advance the field of de-implementation research.

Introduction

Innovation is a natural outgrowth of steady scientific progress. As scientists develop solutions to pressing public health challenges, stopping existing practices to make room for better solutions becomes a necessity. De‐implementation research focuses on the discontinuation of interventions (innovations) that should be stopped. Dissemination and implementation (D&I) science, which is dedicated to enhancing the successful uptake and implementation of research increasingly recognizes the importance of also understanding when and how it is appropriate to decrease or end interventions (Brownson et al., 2015; Niven et al., 2015; Ogden & Fixsen, 2014). For example in the United States, this topic is included in priorities identified in the NIH funding announcement, Dissemination and Implementation Research in Health (NIH, 2009).

To date, the majority of both theoretical and empirical work regarding de‐implementation of interventions has occurred within the context of medical care, primarily focusing on ineffective, overused, or harmful medical practices (Morgan et al., 2015; Niven et al., 2015). Substantially less research has focused on human service settings, like public health and social services. Although some of the theoretical frameworks developed in medical care settings may be extendable to human services, essential differences between public sector human services, like public health and social services, make comparisons with medical care difficult. Characteristics of the supporting payment systems, the level of interventions, and the ultimate goals of public health and social service efforts necessitate a separate discussion from medical care (Aarons, Hurlburt, & Horwitz, 2011; Niven, Leigh, & Stelfox, 2016). Furthermore, it is important to focus separately on de‐implementation from other phases of intervention implementation, like adoption, initial implementation, and sustainability. Though many similarities exist, empirical work is beginning to show that de‐implementation is not identical to other phases, can involve different champions and stakeholders, and is driven by distinct factors (Pierson, 1994; Van Bodegom‐Vos, Davidoff, & Marang‐van de Mheen, 2017).

This paper begins to address the gap in de‐implementation research in human service settings by providing a conceptual narrative of the field with the goals of coalescing the D&I field around the concept, easing synthesis of the literature, and recommending directions for research. In the first section, we conceptualize de‐implementation in this setting and provide criteria for identifying interventions that should be de‐implemented. We then discuss how existing D&I frameworks might be applied to help guide research endeavors and how de‐implementation fits in the context of other major implementation science concepts, especially intervention sustainability. Finally, we conclude with next steps to guide de‐implementation research and special ethical considerations that may arise while conducting de‐implementation research.

Defining De‐implementation

We define de‐implementation as the discontinuation of interventions that should no longer be provided. De‐implementation is one of many related terms that may be useful to researchers interested in this field including de‐adoption, abandonment, mis‐implementation, exnovation, disinvestment, and overuse just to name a few (Gnjidic & Elshaug, 2015; Niven et al., 2015). We build on the definition of de‐implementation provided by Niven et al. (2015) (although they used the term de‐adoption) as the abandonment of interventions that do not demonstrate efficacy or may cause harm. Building on definitions of implementation and adoption by Rabin and Brownson (2017), we use the term de‐implementation rather than de‐adoption. Emerging evidence suggests that, like implementation, de‐implementation occurs as a process (Johns, Bayer, & Fairchild, 2016; McKay, Margaret Dolcini, & Hoffer, 2017), while de‐adoption, like adoption, is a decision step situated within this de‐implementation process.

Circumstances for Appropriate De‐implementation

A key question for framing de‐implementation is identifying interventions appropriate for de‐implementation. We extend the criteria used above to include two additional criteria for appropriate intervention de‐implementation in public health and social service settings. The three criteria are: (a) When interventions lack effectiveness or are harmful, (b) when more effective or efficient interventions become available, and (c) when the health or social issue of concern dissipates. We discuss each condition and provide well‐documented examples in public health and social service settings. Table 1 provides examples with brief descriptions and supporting references in addition to the examples provided in the text.

Table 1

Examples of interventions appropriate for de‐implementation

Intervention(s)	De‐implementation criteria	Description	Intended outcomes
D.A.R.E.	Ineffective	Drug Abuse Resistance Education (D.A.R.E.) program, designed to reduce drug use among adolescents (Ennett et al., 1994; Vincus et al., 2010; West & O'Neal, 2004), was widely implemented in the 1980s and 1990s. When evidence of effectiveness became part of the criteria for obtaining federal funding, the program was revised in 2003 (Petrosino et al., 2006), but failed to demonstrate effectiveness (Vincus et al., 2010).	Prevent youth substance abuse
Infant Sleeping Position Guidelines	Harmful	In the mid‐1950s, recommendations for infant sleeping position changed to placing children in a prone position (on their stomachs) out of choking concerns (Gilbert et al., 2005). The prone position continued to be recommended in the US and abroad into the 1990s (Dwyer & Ponsonby, 1996; Gilbert et al., 2005), and has likely contributed to unnecessary infant deaths due to SIDS. In 1992, the American Academy of Pediatrics Task Force on Infant Sleep Position and SIDS officially recommended that babies be placed on the back or side when sleeping and has subsequently revised guidelines multiple times.	Prevent infant mortality due to SIDS
Healthy Families America	Ineffective	Widely implemented across the US, Healthy Families America is an intervention designed to promote child well‐being and reduce child abuse. While demonstrating effectiveness in child well‐being, several studies showed the intervention failed to effectively reduce child abuse and neglect (Duggan et al., 2004).	Prevent child abuse and neglect
Well Digging in Himalaya regions	Harmful	Tube wells are a common approach to reduce waterborne disease, but led to one of the largest mass poisonings in history in the Himalaya region of the world because of naturally occurring arsenic in the water table. In Bangladesh alone approximately 57 million people were exposed to unacceptably high levels of arsenic through tube wells as of 2000 and experienced related complications (Smith, Lingas, & Rahman, 1997). Dismantling and replacing wells has proved difficult due to social and economic factors (Hossain et al., 2015).	Prevent waterborne infection
Deinstitutionalization for mental health in the US	Low value	Community‐based care and psychotropic medications for treatment developed in the 1950s became an alternative to long‐term, often involuntary commitment in mental hospitals (Chafetz et al., 1982; Gronfein, 1985). The collection of treatment alternatives supporting the deinstitutionalization movement led to an 80% drop in mental hospital populations in subsequent decades (Mechanic & Rochefort, 1990) allowing many individuals to reintegrate and participate in society.	Provide mental health treatment
Evidence‐based HIV interventions	Low value	Around 2006 the Centers for Disease Control and Prevention disseminated a suite of evidence‐based behavioral HIV prevention interventions to local organizations (Collins & Sapiano, 2016; Johns et al., 2016). Due to continued investment in biomedical research, pre‐ and postexposure prophylaxis (PreP and PEP) have demonstrated such high efficacy that it is now more cost‐effective to provide PreP and PEP to those who are at highest risk for HIV rather than provide behavior change interventions to a more general population (Holtgrave, 2010).	Prevent HIV infection
Polio eradication through immunization	Issue has dissipated	The Global Polio Eradication Initiative (GPEI) began in 1988 when polio was still a world‐wide epidemic (World Health Organization (WHO), 2017). To date, efforts have largely been scaled back and polio only persists in a small number of countries.	Prevent polio infection
Ebola outbreak	Issue has dissipated	The response to the Ebola outbreak in several African countries in 2014 involved massive deployment of staff, infrastructure build‐up, collaborations with other relief organizations and national governments, development of treatment guidelines and training for existing clinicians, and takeover of local health systems in multiple countries (WHO, 2015b). Much of the infrastructure persists although the epidemic has now subsided.	Prevent Ebola infection
Postdisaster relief	Issue has dissipated	The postdisaster relief effort in Japan after the 2011 earthquake, tsunami, and nuclear accident required incredible disaster relief effort and reconstruction. The Japan Medical Association dispatched approximately 1400 Japan Medical Association Teams within days which remained in place to provide care in impacted communities until the healthcare system could be reconstructed. After 3 months, teams were withdrawn upon full recovery of the healthcare system (Ishii & Nagata, 2013).	Provide emergency and urgent healthcare services

Interventions Lack Evidence or Are Harmful

In an era of promoting evidence‐based interventions, demonstrating that interventions have the intended outcome is essential. Prior to the emphasis on using evidence‐based interventions, many interventions were implemented without having assessed their efficacy. Those interventions, while well‐intentioned, often were introduced and maintained via persuasion, training, or tradition. Yet as in health care, interventions often persist even when not supported by evidence of effectiveness or when harmful.

A classic, well‐known example of an intervention lacking evidence of effectiveness in the human service setting is the widely disseminated school‐based Drug Abuse Resistance Education (D.A.R.E.) program (Ennett, Tobler, Ringwalt, & Flewelling, 1994; Vincus, Ringwalt, Harris, & Shamblen, 2010; West & O'Neal, 2004). Created in the 1980s by the 1990s D.A.R.E. was the most prevalent substance use prevention program for children in the US with considerable support from the federal government (Petrosino, Birkeland, Hacsi, Murphy‐Graham, & Weiss, 2006; Shepard, 2001; Weiss, Murphy‐Graham, Petrosino, & Gandhi, 2008). The program's successful dissemination has been attributed to its fit within the accepted narrative about the national drug use problem, perceptions that the program improved relationships between students and law enforcement, absence of other acceptable alternatives for the school setting, and states’ ability to make funding available to support the program (Birkeland, Murphy‐Graham, & Weiss, 2005; Petrosino et al., 2006; Wysong, Aniskiewicz, & Wright, 1994). When evidence of effectiveness became part of the criteria for obtaining federal funding, the program was revised in 2003 (Petrosino et al., 2006), but failed to demonstrate effectiveness (Vincus et al., 2010). Although implementation of the program has declined since the 1990s (Petrosino et al., 2006), D.A.R.E. continues to be widely implemented; the program estimates that it is present in 75% of the nation's school districts and is taught in all 50 states (D.A.R.E., 2018).

Human service interventions can also be harmful in their effects, exemplified by the case of sudden infant death syndrome (SIDS) guidelines. Although the underlying causal mechanisms of SIDS are still not well understood, guidelines have changed as evidence has evolved. Prior to the early 1950s, texts recommended placing children on their backs for sleeping. However, by the mid 1950s, recommendations changed to placing children in a prone position (on their stomachs) for choking concerns (Gilbert, Salanti, Harden, & See, 2005). In the 1970s, the growing evidence on causes of SIDS, as synthesized in systematic reviews, contraindicated the prevailing practice of prone sleeping. Despite mounting evidence, the prone position continued to be recommended in the United States and abroad into the 1990s and has likely contributed to unnecessary infant deaths in the United States (Dwyer & Ponsonby, 1996; Gilbert et al., 2005). In 1992, the American Academy of Pediatrics Task Force on Infant Sleep Position and SIDS officially recommended that babies be placed on the back or side when sleeping which served as the basis for the national public health Back to Sleep campaign, later known as the Safe to Sleep campaign to raise community awareness about the emerging evidence (Kattwinkel, Brooks, & Myerberg, 1992; National Institute of Child Health and Human Development, 2017). The guidelines and campaigns were revised in 1996 (Kattwinkel, Brooks, Keenan, Malloy, & Willinger, 1996), 2000 (Kattwinkel, Brooks, Keenen, & Malloy, 2000), and 2005 (Kattwinkel et al., 2005). The most recent revisions emphasized room‐sharing without bed‐sharing, expanded the scope to include risk factors for other sleep‐related deaths, and included recommendations regarding prenatal care, maternal substance use, and infant immunizations (Moon et al., 2016; Task Force on Sudden Infant Death, S. and Moon, 2011).

More Effective or Efficient Interventions Are Available

We are constantly investing in basic science that yields evidence with implications for practice and requires practitioners to revise existing interventions. It may be desirable to de‐implement interventions, even if the intervention is evidence‐based, because the intervention is too costly, too difficult to implement, redundant with existing services, or not as effective as another intervention (Prasad & Ioannidis, 2014). We characterize these interventions as low‐value interventions, interventions that are low value in relation to more effective or efficient ones, which we suggest may occur most often in the context of chronic issues. While their effects may be inert or even beneficial for some individual members of the target population, de‐implementation of low‐value interventions more efficiently leverages public resources and maximizes the benefits of emerging science for the community.

The deinstitutionalization movement in mental health serves as an example in the United States where highly effective and less costly interventions emerged to replace long‐term institutionalization of people with chronic mental health conditions, people with developmental disabilities, and the elderly in state and county mental hospitals. Beginning in the 1950s, evidence supporting community‐based care and psychotropic medications for treatment were an alternative to long‐term, often involuntary commitment in mental hospitals and more extreme treatments such as electroconvulsion therapy and lobotomy (Chafetz, Goldman, & Taube, 1982; Gronfein, 1985). Furthermore, strong public and bi‐partisan political support promoted drastic restructuring of mental health services, such as the Community Mental Health Center Act, and payment structures, such as the Medicare, Medicaid, and Social Security Income, at the federal level (Schutt, 2016). This collection of policies encouraged community‐based mental health care and payment mechanisms that shifted funds away from long‐term institutionalization. Many elderly patients, who were also receiving care in mental institutions alongside other kinds of patients, could be moved to nursing home facilities under Medicare, yielding a more cost‐effective approach for elder care (Steven & Leah, 2013). The collection of policies and treatment alternatives supporting the deinstitutionalization movement lead to an 80% drop in county and state mental hospital populations from 1955 to 1975, and allowed many individuals to reintegrate and participate in society (Mechanic & Rochefort, 1990).

The Issue of Concern Dissipates

Although we expect this to be the least common justification for appropriate de‐implementation, sometimes public health or social service issues subside, and intervention is no longer necessary, particularly within the realm of specific infectious diseases or emergency preparedness and relief. The global efforts to eradicate polio is a prominent example of de‐implementation efforts following dramatic reductions in polio incidence, which in 2016 consisted of only 37 cases from three countries (World Health Organization, 2017). The Global Polio Eradication Initiative (GPEI), led by national governments and comprised public and private partners, began in 1988 in a world with a high polio incidence and aimed to eradicate transmission by 2000 (WHO, 2017). The efforts stalled in the early 2000s, when most of the world was polio‐free, but, due to implementation challenges in four remaining endemic countries, outbreaks in previously polio‐free areas continued to emerge (Pallansch & Sandhu, 2006). A renewal of efforts and changes in tactics by the GPEI resulted in the present gains in transmission interruption (Aylward & Tangermann, 2011). In 2013, GPEI began planning for a polio‐free world, in which only the core polio eradication and surveillance functions are maintained and the existing public health assets and infrastructure (e.g., community‐based health workers, volunteers, social mobilizers; surveillance systems; mobilized political, financial, and social support) are transitioned to address other health priorities (Rutter et al., 2017). Country‐level transition planning includes several steps: raising awareness of the transition process; identification and establishment of national coordination teams and work plans; mapping the polio assets, national priorities, and needs; conducting planning workshops and simulations with stakeholders; and establishment and generating buy‐in and funding commitments for the transition plan (Rutter et al., 2017).

A more recent example is the 2014 response to the Ebola outbreak, which over the span of months developed into a major public health emergency and was the largest emergency response to date by the World Health Organization (WHO, 2015b). The response involved massive deployment of staff, infrastructure build‐up, collaborations with other relief organizations and national governments, development of treatment guidelines and training for existing clinicians, and takeover of local health systems in multiple countries. As the epidemic resolved in 2016, remaining infrastructure from what was used to support the Ebola response remained in place, sometimes unutilized (Sieff, 2015). Even when a concern such as Ebola dissipates, public health officials need to remain vigilant and the lessons from past experience may apply to future outbreaks.

Framing De‐implementation for Research

As with any innovative direction in D&I science, theoretical models and frameworks focusing on de‐implementation are essential to help guide future research endeavors (Tabak, Khoong, Chambers, & Brownson, 2012). While this is not intended to be a systematic review, we discuss several ways to consider integrating D&I theory into de‐implementation research. We frame de‐implementation as either a process or an outcome depending on the research question and review established de‐implementation frameworks and broader well‐established D&I frameworks, especially where dedicated de‐implementation frameworks are lacking. We have listed and summarized the frameworks discussed in Table 2.

Table 2

Example frameworks for use in de‐implementation research

De‐implementation Stage	Framework	Framework Description	Level
General—Covers all stages	De‐adoption Framework (Niven et al., 2015)	A framework and conceptual model of the de‐adoption process of low‐value clinical practices in four stages: 1. Identify low‐value clinical practices, 2. Facilitate the de‐adoption process, 3. Evaluate de‐adoption outcomes, and 4. Sustain de‐adoption.	Organization, system
Step 1: Selection—Select the best candidate interventions for de‐implementation	The Reassessment Framework (Elshaug et al., 2009)	A tool for involving policy stakeholders in the identification (reassessment) of medical practices for de‐implementation (selective disinvestment). The framework contains seven criteria that are valid and feasible within existing resources from a policymaker perspective.	Policy
	Framework for identifying contradicted, unproven, and aspiring healthcare practices (Prasad & Ioannidis, 2014)	Guides the identification of candidate interventions for de‐implementation, tailored to three categories of intervention: contradicted established medical practices, unproven medical practices, and novel medical practices.	Policy, Organization
	De‐implementation Checklist (SAMHSA, n.d.)	Tool for organizational administrators to assess underperforming programs and interventions	Organization
Step 2: Assessment—Assess the context in which de‐implementation efforts will take place	EPIS (Aarons et al., 2011)	Characterizes the stages of intervention implementation and contextual factors influencing implementation outcomes.	System
	Consolidated Framework for Implementation Research (CFIR) (Damschroder et al., 2009)	Describes contextual factors influencing implementation outcomes at multiple levels within systems.	System
	De‐adoption Framework (Montini & Graham, 2015)	Catalogs and analyses extra‐scientific contextual factors that affect the process of de‐implementation including historical, economic, professional, and social resistance to de‐implementation.	System
Step 3: Active de‐implementation—Use the assessment of the context to identify the appropriate high‐level de‐implementation strategies and processes	Framework for termination of public organization (Adam et al., 2007)	Framework that organizes determinants of organizational termination along two dimensions (internal and external) and presents a typology of organizational termination strategies.	Policy, Organization
	Implementation Framework (McKay et al., 2017)	An extension of an existing implementation framework to include a distinct stage for de‐implementation, during which specific processes take place.	Organization, Individual
	Planned action model (Helfrich, 2016)	A model to develop to identify and engage key stake holders to de‐implement clinical interventions.	Cognitive, Individual
Step 4: Evaluate—Evaluate de‐implementation outcomes for success	Implementation outcomes (Proctor et al., 2011)	Describes ways to assess implementation.	System

Selected Frameworks for De‐implementation

De‐implementation as a Process

Broadly, de‐implementation occurs in a series of stages with the ultimate outcome being that an intervention is widely de‐implemented in practice settings. Frameworks to conceptualize the necessary steps for de‐implementation efforts are the most widely available, with multiple frameworks in both the clinical and policy implementation literatures. The framework by Niven et al. (2015), synthesized from existing empirical and theoretical literature from the clinical setting, is useful for structuring this process in public health and social service settings. They present a comprehensive overview of the de‐implementation process in several stages leading to complete de‐implementation throughout an entire delivery system. These steps mirror established implementation processes while making allowances for special challenges that exist for de‐implementing services and programs (Graham et al., 2006). Challenges include resistance to de‐implementation of interventions that are well‐ingrained in the system; existing and accumulated historical, economic, professional, and social factors; and balance of organizational characteristics with external pressures to de‐implement (Adam, Bauer, Knill, & Studinger, 2007; Montini & Graham, 2015).

The initial step in the de‐implementation process consists of identifying and prioritizing interventions and programs appropriate for de‐implementation. Once targeted programs have been identified, the barriers and facilitators to de‐implementation within the human service system should be assessed to develop, tailor, and put in place strategies to successfully support de‐implementing the selected practices. The final steps include evaluating outcomes of de‐implementation efforts to assess changes in practice, health outcomes, and cost. The Niven et al. (2015) framework emphasizes stakeholder engagement early during identification of de‐implementation targets and throughout de‐implementation efforts. These de‐implementation stages can be further conceptualized within organizations using interventions targeted for de‐implementation as part of a continuum of intervention delivery that includes adoption, implementation, and sustainment of interventions, as has been done by McKay et al. (2017). Their work extends a previously developed implementation framework to include a distinct stage of de‐implementation composed of several steps similar to some of those described above (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005).

Additional frameworks are available for specific stages and levels of de‐implementation. When candidate interventions must first be identified and prioritized for de‐implementation efforts, frameworks developed within healthcare settings are particularly useful. Prasad and Ioannidis (2014) present the criteria and steps necessary for identifying candidate interventions specific to contradicted established medical practices, unproven medical practices, and novel medical practices. Multiple factors influence the identification of interventions beyond the scientific evidence, such as the cost of interventions or the availability of alternatives. De‐implementation of ingrained interventions may also be met with resistance illustrating another set of sociopolitical factors that influence the identification of interventions for de‐implementation (Montini & Graham, 2015). For example, the revision of guidelines to encourage parents to place sleeping children on their backs to prevent SIDS was met with resistance over several years and required revisions in a multitude of published medical texts for practitioners and resources for parents. In addition to de‐implementing specific interventions or practices, it may also desirable to de‐implement entire policies. Paprica, Culyer, Elshaug, Peffer, and Sandoval (2015) provide seven criteria for de‐implementation that are valid and feasible within existing resources from the policymaker perspective. These frameworks help researchers and policymakers weigh these relative influences to identify policies that are the best candidates for de‐implementation. For example, the rapid deinstitutionalization movement in mental health is, in part, explained by the bi‐partisan support for removing certain policies that were viewed as both cost‐effective and in the best interest of patient rights. Although not a framework, a de‐implementation checklist developed by the National Registry for Evidence‐based Programs and Practices may help administrators within organizations assess the various contextual factors in combinations with intervention effectiveness to help determine whether an intervention should be de‐implemented and contextual issues, like financial sustainability of the intervention, that may influence ease of de‐implementation (SAMHSA, n.d.).

Correlates and Contextual Factors Influencing De‐implementation

Once a target practice, intervention, or policy has been identified for de‐implementation, it is important to assess the context in which de‐implementation efforts will take place. In the case of changing infant sleeping guidelines, the slow uptake of evidence to place infants to sleep on their backs illustrates the many contextual factors that can influence de‐implementation throughout the human service system (e.g., clinicians, public health practitioners, social workers, printed texts) in a planned and intentional manner (Dwyer & Ponsonby, 1996; Gilbert et al., 2005). Particularly helpful in assessing these contextual factors is the work by Montini and Graham (2015), in which they outline historical, economic, professional, and social factors that can hinder de‐implementation efforts. If a program or practice has been in place for a long time and is well‐integrated into public health systems, it may be highly resistant to de‐implementation, potentially also due to interdependent financial and professional interests that depend on it. Similarly, it may be difficult to de‐implement and replace some inefficient or low‐value programs without a change in the social environment around it.

In addition to existing de‐implementation frameworks, we suggest that several existing D&I frameworks may be valuable for widening the scope of factors that may be relevant for public health and social services settings. Some of the theoretical frameworks developed in medical care settings may be extendable to human services, but essential differences between public sector human services indicate a need for models informed from these perspectives. For example, characteristics of the supporting payment systems, the level of interventions, and the ultimate goals of public health and social service efforts necessitate a separate discussion from medical care (Aarons et al., 2011; Niven et al., 2016).

We use the EPIS framework and the Consolidated Framework for Implementation Research to help conceptualize the myriad of factors that specifically contribute to de‐implementation. Consistent between the two frameworks is the recognition that the outer context, or the sociopolitical ecology, in which interventions are implemented influence the likelihood that interventions are de‐implemented. Policy change and funding support, in particular, are emerging as key drivers of system‐wide de‐implementation (Johns et al., 2016), but other factors such as client demand and inter‐organizational networks may also mitigate the influence of policy and funding. For example, client demand and reliance on an intervention may promote the continuation of interventions that should be stopped, or a lack of organizational connectedness may lead different organizations to de‐implement interventions at different rates. In inner contexts, or factors occurring within organizations, organizational characteristics (e.g., culture, climate, and capacity) and individual staff characteristics (e.g., leadership, values, and expertise) may also drive the de‐implementation of interventions within organizations and among individual staff. Lastly, there is strong theoretical support that characteristics of the intervention itself may make de‐implementation more likely (Mary Ann Scheirer, 2013). For example, it may be easier to de‐implement interventions that are harmful over interventions that are simply inert or interventions that are costly over interventions that are inexpensive.

Encouraging De‐implementation

The assessment of the context in turn informs what de‐implementation strategies are appropriate. A framework developed by Adam et al. (2007) to examine persistence or termination of policy organizations links organizational characteristics and external incentives to distinct organizational survival outcomes. This framework can be adapted to examine characteristics and contexts of human service programs and identify de‐implementation strategies. Based on the framework, certain programs may be more difficult to de‐implement based on their organizational characteristics (e.g., older, larger, serving multiple purposes) and external factors that are present (e.g., low political turnover, low societal pressure, low efficiency, or no budgetary constraints). When both organizational characteristics and external factors support de‐implementation, de‐implementation may be a straightforward process. However, when external incentives for de‐implementation are high, but organizational factors hinder de‐implementation, the framework suggests that organizational reform is more likely. In such an instance, repurposing an inefficient or low‐value public health program may be a better approach than de‐implementation and replacement.

In addition to high‐level de‐implementation strategies, efforts aiming to change the practices of individual program stakeholders necessitate strategies that are tailored to individual cognitive processes. Helfrich (2016) adaptation of the framework of the Planned Action Model aids in developing de‐implementation strategies targeted to the conscious and unconscious cognitive processes at the individual level. They conceptualize the de‐implementation process as “unlearning” the intervention when practitioners are presented with evidence of ineffectiveness in order to change their existing practice. De‐implementation is conceptualized as substitution when conscious cognition is not feasible, and this process provides external environmental or emotive clues to guide practitioners to a substitute practice. When this framework is adapted to the public health practice setting, it is useful in identifying de‐implementation strategies that are focused on individual responsible for intervention implementation.

De‐implementation as an Outcome

Successful intervention de‐implementation outcomes may be examined as an indication of whether or not our efforts are successful. Key proximal outcomes as recommended by Proctor et al. (2011) for early intervention adoption and implementation are acceptability, adoption, appropriateness, feasibility, fidelity, cost, penetration, and sustainability. We suggest that outcomes indicating successful de‐implementation are similar. For example, acceptability is the perception among stakeholders that an intervention is agreeable, palatable, or satisfactory. In the context of de‐implementation, acceptability may be indicated by stakeholders’ willingness to de‐implement an intervention. Appropriateness reflects the extent to which stakeholders believe that the intervention is a good fit, relevant, or compatible with the needs of the target population and the capabilities of service providers. In the context of de‐adoption, appropriateness may be reflected in stakeholders’ and providers’ views that, given community needs, de‐implementing the intervention is warranted so that it can be replaced with an alternative intervention. The persistence of D.A.R.E. may, in part, be explained by both the low acceptability, or low agreement that it should be de‐implemented, and a perception that something more appropriate is unavailable. Penetration describes the number of organizations appropriately de‐implementing interventions and if there are differences between those successfully de‐implementing and those opting to continue interventions.

Finally, D&I research is conducted under the premise that enhancing uptake and implementation of research in practice will have downstream benefits for the public. Likewise, de‐implementing interventions should influence the quality and efficiency of services that, in turn, beneficially influence population outcomes (Proctor et al., 2009, 2011). We recommend that these may similarly serve as useful conceptual domains for understanding the influence of de‐implementation.

The Intervention Life Cycle: Sustainability and Replacement

Two other topics are worth addressing in the context of D&I science: intervention sustainability and replacement. In general, sustainability or maintenance of interventions is a phase in the implementation process where interventions are delivered in a relatively steady state within an organization, in spite of changes that may be happening either internally within an organization (e.g., funding changes or staff turnover) or externally outside of an organization (e.g., changing sociopolitical support) (Chambers, Glasgow, & Stange, 2013; Proctor et al., 2015; Scheirer & Dearing, 2011; Schell et al., 2013; Shediac‐Rizkallah & Bone, 1998). Considering the long‐term sustainment of interventions when first implementing is desirable to enhance integration and embeddedness of interventions, thus increasing the likelihood that an intervention will be sustained over time (Chambers et al., 2013; Scheirer & Dearing, 2011). However, the processes that encourage embeddedness of practices may make de‐implementing at the appropriate time more difficult. This also highlights the likely bi‐directional influences that occur between interventions and the contexts where they are implemented over time.

Theoretically, de‐implementation is often implied as the conclusion of the sustainability phase but is very rarely discussed explicitly. In other words, the implication is that if an intervention is not sustained, then it has been de‐implemented. However, this assumes that the intent was for the intervention to continue and that de‐implementation of the intervention is not a desirable outcome. Indeed, it is well documented that interventions are often ended prematurely, which we suggest is better conceptualized as intervention abandonment (Hodge & Turner, 2016; Massatti, Sweeney, Panzano, & Roth, 2008; Nadeem & Ringle, 2016; Scheirer, 1990). Because it may be justifiable to de‐implement interventions within public service sectors that are evidence‐based under the circumstance that better interventions are available or the issue dissipates, we recommend adding a de‐implementation stage to existing models describing the stages of intervention implementation to provide a more complete conceptualization of evidence‐based intervention implementation.

Furthermore, it may be useful to de‐implement so that a replacement intervention may be implemented. An interesting insight from Nigeria's rapid response to the Ebola outbreak in 2015 was successfully repurposing its polio eradication infrastructure. Nigeria saw its last case of polio in July 2014 shortly before the Ebola outbreak, and leveraged this infrastructure to control the Ebola outbreak within 93 days (WHO, 2015a). This quick response is stark in contrast to other countries where the Ebola epidemic persisted, highlighting the value of understanding de‐implementation and replacement in tandem. The incorporation of systems science theory and principles like the dynamic sustainability framework by Chambers et al. (2013) suggests that the removal of interventions may be influencing the implementation of future interventions. In addition, the presence or absence of a replacement intervention and the extent to which the replacement is like or unlike the existing intervention may influence the ease and acceptability of de‐implementing the existing intervention.

Next Steps and Research Opportunities

Leveraging existing D&I theory is critical for guiding de‐implementation research and developing a foundation of empirical evidence supporting this aspect of D&I science. As others have noted, we have minimal evidence to help us understand the extent to which interventions that should be de‐implemented abound in the public health and social service systems (Brownson et al., 2015). Using existing frameworks, we identify several areas where evidence is needed in public health and social service contexts.

An important first step for de‐implementation in public health and social services is identifying candidate interventions for de‐implementation, whether they be policies, interventions, or individual practices, in collaboration with relevant stakeholders in human service settings. Ideal candidates for de‐implementation should be selected based on the available evidence, the availability of other more effective or efficient interventions, the extent of the issue the intervention is intended to address, and the local context in which interventions are being implemented. We recognize that the strength and relevance of evidence supporting interventions may change based on the local context (Hawe, Shiell, & Riley, 2009; Schensul, 2009), and evidence from local efforts to evaluate programs may be equally as valuable as more traditional sources of evidence, like published randomized controlled trials (Fielding & Frieden, 2004).

There have been efforts within medical care to identify interventions appropriate for de‐implementation, like the Choosing Wisely campaign (American Board of Internal Medicine, 2017) and regular recommendations by the United States Preventive Services Task Force (although they do not consider cost in their recommendations) (U.S. Preventive Services Task Force, 2017), but to our knowledge, there have not been parallel efforts within public health or social services. Systematic approaches to identifying and prioritizing interventions for de‐implementation and methods to help engage stakeholders in this process would be valuable to help support this initial phase in the de‐implementation process.

Once interventions are identified, a second step for de‐implementation research in human service settings is understanding the contextual factors that influence de‐implementation and the development of evidence‐informed strategies to successfully guide and encourage de‐implementation, including the outcomes related to de‐implementation. Furthermore, explication of a de‐implementation process from identification of de‐implementation candidates through de‐implementation activities within organizations, as well as the factors and strategies that encourage de‐implementation, will have the practical benefit of helping to ensure smooth and successful de‐implementation. Early evidence suggests that encouraging de‐implementation, even using intentional strategies, may prove especially difficult (Rosenberg et al., 2015; Voorn et al., 2017). Niven et al. (2015) recognized the additional challenges associated with de‐implementation, particularly of well‐established practices, and therefore recommended an in‐depth analysis of barriers and facilitators as they relate to factors beyond the quality of the evidence, suggesting the need for qualitative approaches in this area.

Lastly, it will be important to understand the outcomes of de‐implementation. Demonstrating the benefit of de‐implementing interventions in terms of improved service quality, cost‐effectiveness, and population benefit is paramount (Proctor et al., 2011). It will also be valuable to understand some of the ancillary benefits that may have occurred because of implementing interventions, like strengthened collaborations among organizations or other infrastructure, so that these aspects of interventions might persist. Since de‐implementation likely has consequences for public health systems and organizations given the bi‐directional relationships between interventions and context, by extension, de‐implementation likely has consequences for the adoption, implementation, and success of future interventions. Explication of research methods, surveillance systems, and tools to measure both successful de‐implementation and the consequences of de‐implementation are key needs.

Although we have not fully addressed the issue here, we recognize that interventions may end rather abruptly with little to no process involved, and, as mentioned above, that many interventions are abandoned that should be continued. Mis‐implementation conceptually incorporates the premature abandonment of interventions that should continue (i.e., interventions that should have been sustained), the overuse of interventions that should be used less frequently or with a narrower target population, and the underuse of interventions that should be used more frequently or with a more general target population (Brownson et al., 2015). These are important conceptual and empirical issues distinct from de‐implementation. Further examinations of mis‐implementation or more general misuse of interventions will be a valuable direction for D&I science.

Ethical Considerations

De‐implementing interventions also raises potential ethical issues, both in practice and in research. In cases where interventions are harmful, ineffective, or unnecessary, failure to de‐implement them is unethical, as is failure to substitute those interventions with more effective and safe interventions. Thoughtful discussion has been put forth in the context of medical care to guide researchers in these circumstances (Blumenthal‐Barby, 2013; Niven et al., 2016). However, human service ethics principles must also weight overall population benefit of interventions and financial investments made given that many of these services are provided using tax‐payer dollars (Lee, 2012). Furthermore, many public health and social work interventions in practice simply lack sufficient evidence to determine whether it is effective or not. Continuation of an intervention that lacks supporting evidence either for or against simply because it has not been evaluated, especially in the absence of an evidence‐based alternative, presents a separate ethical quandary that should be further examined.

The steps surrounding the de‐implementation process present additional ethical considerations that may prove challenging for researchers conducting work in human service settings. First, before de‐implementation efforts can begin, the determination as to which intervention should be targeted for de‐implementation has to be made. This determination has significant implications related to health equity and disparities and, as such, is a principal ethical challenge of de‐implementation research and practice. The value of a particular intervention depends on who is making that determination, and not all stakeholders may agree that a particular intervention is low value. The deinstitutionalization movement serves as a cautionary tale of how de‐implementation may have differential benefits (or harm) for different individuals impacted by these changes. While the movement represented a movement toward evidence‐based treatment, patient rights, and cost‐effectiveness, this movement has also drawn sharp, yet valid, criticism. Multiple social challenges including homelessness and increases in incarcerated populations are attributable to a lack of adequate community‐based care to support community need (Chafetz et al., 1982; Gronfein, 1985; Steven & Leah, 2013). Furthermore, mental hospitals were repurposed for short‐term, acute events for patients, but this has led to a revolving door phenomenon for some with chronic conditions.

Second, researchers should be cognizant of the reasons someone may be resistant to de‐implementation efforts, some of which may be valid. Policymakers may be resistant if they feel changing policy is politically risky. Providers may be resistant because they do not feel like they have the training or infrastructure resources required to deliver the new intervention. Clients may be resistant because they feel like they are not receiving the best care or that their care is being rationed. This is especially true for minority populations who have historically not received services equitably. In implementation efforts, the client is usually the one who benefits, but in de‐implementation efforts, society or the public system may benefit more with no obvious benefit for individual clients. Therefore, researchers have an ethical obligation to understand the cultural and historical context surrounding de‐implementation efforts and use appropriate strategies and messaging that are sensitive to the reasons that organizations and practitioners may opt to continue practices. It also underscores the need to engage and build buy‐in from stakeholders at multiple levels. Engaging stakeholders such as providers, clients, administrators, and policymakers in the process of determining interventions and approaches for de‐implementation is important not only to ensure the determination is made ethically but also to improve the likelihood de‐implementation efforts will be successful (Minkler, Salvatore, & Chang, 2017). The more stakeholders who agree an intervention is ready for de‐implementation, the more buy‐in researchers and practitioners will have toward their de‐implementation efforts.

Conclusions

De‐implementation in the context of public health and social services is about more than the removal of interventions that are simply ineffective. Researchers and practitioners should also consider the quality of the evidence supporting an intervention relative to other available interventions, costs, feasibility, and community needs. Opportunities abound to conduct research focusing on this issue in public health, but it is important to use guiding theory moving forward to harmonize these efforts. Intervention de‐implementation may be thought of as either an outcome and/or as a process within a complex system of organizations and individual actors, with bi‐directional influences between the intervention and the context from which it is being removed. We encourage researchers to use available theoretical models and frameworks to guide research and develop tools to support successful de‐implementation in public health and social service practice.

Authors’ Contributions

All authors contributed to writing of all sections of the manuscript. All authors provided commentary and revision edits to the manuscript. All authors read and approved the final manuscript.

Authors’ Information

Dr. McKay is a research assistant professor in the Center for Public Health Systems Science in the George Warren Brown School of Social Work at Washington University in St. Louis. She also consults for the Dissemination and Implementation Research Core in the Institute for Clinical and Translational Science at the Washington University School of Medicine. Dr. McKay conducts mixed‐methods research focusing on the end phases of evidence‐based HIV prevention interventions.

Competing Interests

The authors declare that they have no competing interests.