Behavioral Health Workforce Development in the era of COVID-19: Examples From a State-Funded Intermediary Organization.

Intermediary and purveyor organizations (IPOs) play a key role in disseminating and implementing behavioral health evidence-based practices. The COVID-19 pandemic created a time of crisis and disruption to behavioral health care delivery. Using the conceptual framework of basic, targeted, and intensive technical assistance (TA) from the Training and Technology Transfer Centers, case studies are used to describe how programs at The Center for Practice Innovations a state funded-intermediary organization, adapted its training and technical assistance to be delivered entirely remotely, to include content related to COVID-19 and to provide guidance on telehealth-based behavioral health care.

Introduction

Intermediary and purveyor organizations (IPOs) play a key role in disseminating and implementing evidence-based behavioral health best practices (EBPs). The role of an IPO can include consultation, quality assurance, continuous quality improvement; outcome evaluation; and training (Franks & Bory, 2015; Proctor et al., 2019). Intermediary and purveyor organizations also help bridge the gap between research and practice by supporting capacity-building within an agency or system to sustain such EBPs (Franks & Bory, 2015).

Increasingly, federal agencies and state behavioral health systems partner with academic institutions or intervention developers to create IPOs to disseminate and support the implementation of EBPs (Proctor et al., 2019; Cross-Technology Transfer Center (TTC) Workgroup on Virtual Learning, 2021; Druss et al., 2021). One such example of a state-funded intermediary organization is The Center for Practice Innovations (CPI). The New York State Office of Mental Health (NYS-OMH), the New York State Psychiatric Institute and the Department of Psychiatry at Columbia University established CPI in 2007 to promote the widespread use of EBPs developed for adults with serious mental illness throughout New York State (NYS). The Center for Practice Innovations provides training and technical assistance (TA) across eight core recovery-oriented EBPs including: treating co-occurring mental health and substance use disorders, Assertive Community Treatment, 2021, supported employment, Individual Placement and Support [IPS], coordinated specialty care for first-episode psychosis, suicide prevention, cognitive health, and evidence-based care for obsessive–compulsive disorder (Covell et al., 2021; Covell et al., 2014; Dixon & Patel, 2020; Margolies et al., 2021b; Margolies et al., 2015; Margolies et al., 2021a; New York State, 2020; Thorning & Dixon, 2020). The Center for Practice Innovations also supports training in clinical core competencies and state training initiatives to support Medicaid redesign.

Since 2009, CPI offers scalable online training in the form of eLearning courses and technical assistance in the form of remote consultation with providers, learning collaboratives (including ongoing consultation, supervision, performance feedback, practice facilitation and site visits), and fidelity monitoring to over 50,000 behavioral health providers throughout New York State. Through a learning management system (LMS), a cloud-based application that manages all aspects of the online training, the Center for Practice Innovations trains providers from not-for-profit behavioral healthcare agencies, governmental facilities, community programs, and the Veteran’s Health Administration. The Center also trains the emerging workforce, including undergraduate and graduate behavioral health care students.

In March 2020, the sudden rise of SARS-CoV-2 virus and executive order in New York State required all non-essential businesses to cease in-person operations, to shelter in place, and to use remote technology to operate and provide services across many industries. In this article, we use case studies and the Technology Transfer Centers (TTC) framework that describes three types of TA (i.e., basic, targeted and intensive) (Cross-Technology Transfer Center (TTC) Workgroup on Virtual Learning, 2021) to describe how programs at CPI adapted training and TA to be delivered entirely remotely, to include content related to COVID-19 and to provide guidance on telehealth-based behavioral health care delivery. In addition, we describe how CPI supported the state mental health authority during the wake of the pandemic.

Methods

Technology Transfer Center Framework

Built upon the definition of TA from the National Implementation Research Network (NIRN; Fixsen et al., 2019), the Technology Transfer conceptual framework describes three tiers of TA: basic, targeted and intensive. Each tier describes types of training, intensity, focus and investment required to develop TA. Basic TA focuses on information dissemination through mass mailings, websites, publications of information, untargeted presentations to heterogenous groups, or social media. Targeted TA focuses on specialized groups and promotes skill building through short-term training such as online courses and webinars. Intensive TA is ongoing and includes customized training such as site visits, care consultation calls, and performance feedback to promote full integration of EBPs into practice with specialized groups.

We used the TTC framework, because CPI has used a blended (in-person and remote), practical and science informed approach to develop a range of training and TA based on the NIRN competency drivers (i.e., training, coaching, staff selection, fidelity including performance assessment) (Fixsen et al., 2019; Margolies 2021b). The tiers of TA described in the TTC framework map to our TA based on the NIRN competency drivers (see Table 1).

Table 1

CPI training and technical assistance during COVID-19

TTC category	NIRN driver	Type of training/technical assistance	Program: Output	Outcome
Basic	Training	Tips sheets and webinars	Suicide Prevention Training Implementation and Evaluation (SP-TIE): Managing suicidal clients during the COVID-19 crisis	Webinars: 1,079 attendees (live), 509 completions (archived)
			Focus on Integrated Treatment (FIT): Helping People with Co-Occurring Mental Health and Substance Use Conditions during the COVID-19 pandemic	252 attendees (live) 65, completions (archived)
			Focus on Integrated Treatment (FIT): Helping People with Behavioral Health and Tobacco Use Disorders during the COVID-19 pandemic	86 completions (archived)
		Curriculum	Wellness Self-Management (WSM): COVID-19 curriculum	Webinar:1900 attendees (live), Curriculum: 98% of users (n = 92) will continue use
Targeted	Training & coaching	Webinars	Assertive Community Treatment: COVID-19 webinar series	Webinar: 680 attendees
		Town hall meetings	ACT: Town hall series	Town hall meetings: 3
		Remote technical assistance	ACT: Clinical care and justice-informed practice consultation calls	Providers trained: 750
		Learning community	ACT: Learning Community	Meetings: 40
Intensive	Coaching, staff selection, performance measurement	Learning collaborative webinars/meetings	Individual Placement and Support (IPS)	Webinars/ meetings: 29, Sites continued IPS services: 73% of the 88 sites
		Site-specific training and consultation		Remote site visits:173, Helpfulness of training: Extremely: 18%, Reasonably: 39%, Somewhat: 28%, Slightly: 10%, Not helpful: 5%
		Training of newly hired staff		Providers trained: 26
		TA calls/online meetings	New York City Tobacco Cessation Training and Technical Assistance Center (NYC TCTTAC): Learning community	TA calls/online meetings: 68
		Capability assessment		Programs completed, Baseline assessment: 70, Follow up assessment: 40
		Training of newly hired staff		Providers trained: 162

NIRN national implementation research network, TTC technology training center

Basic TA

Basic TA disseminates information to broad audiences (Cross-Technology Transfer Center (TTC) Workgroup on Virtual Learning, 2021). The Center for Practice Innovation’s basic TA targets the state behavioral health workforce and focuses on building awareness and knowledge in core clinical EBPs (e.g., suicide prevention, integrated treatment for mental health and substance use) through sharing newsletters, posting information on its website, manuals, webinars and eLearning courses. Given the need to develop accessible training materials quickly, CPI curated resources and created a repository of information on its website (https://practiceinnovations.org/covid-19) regarding COVID-19 policy guidance from the state mental health authority, health and behavioral health resources for care recipients and operational guidance regarding telehealth-based care delivery for providers.

Case Study: Suicide Prevention Training Implementation and Evaluation (SP-TIE) and Focus on Integrated Treatment (FIT)

The Center for Practice Innovations developed tip sheets to address the increase in suicidal ideation, substance use, and tobacco use (Czeisler et al., 2020, 2021). For example, the Suicide Prevention Training Implementation and Evaluation (SP-TIE) initiative developed and disseminated “Telehealth tips: Managing suicidal clients during the COVID-19 pandemic.” This tip sheet included important basic guidelines for initiating contact via telehealth when a care recipient may be suicidal. It detailed adaptations for conducting comprehensive suicide risk assessment, for safety planning and clinical management of (e.g., steps in developing a safety plan, ongoing follow-up and monitoring), and documentation and supervision for the provider. The Center offered a live (1,079 attendees), and an archived webinar (509 completions) on this topic and how to use the tip sheet.

Given the increase in substance use and higher rates of comorbid health conditions that increase vulnerability to COVID-19 illness, housing instability and other situations (e.g., community contacts to obtain substances) that increase the risk of infection (Farhoudian et al., 2020; Volkow, 2020; Wen et al., 2020), the Focus on Integrated Treatment (FIT) initiative developed a tip sheet for co-occurring substance use and another tip sheet for co-occurring tobacco use. Developed in collaboration with state mental health and addictions authorities, these tip sheets include basic guidelines for initiating contact and applying a harm reduction approach to address COVID-19 safety, available remote services and support, safer drug and alcohol use, and, where possible, avoiding returns to use. The tip sheets were presented in live (252 attendees for substance abuse) and archived webinars in the CPI LMS (65 substance abuse and 86 tobacco cessation, respectively).

Case Study: Wellness Self-Management (WSM)

The Center for Practice Innovations collaborated with the Community Technical Assistance Center at New York University to create a six-lesson curriculum entitled “Wellness Self-Management for COVID-19 (WSM-COVID-19).” Wellness Self-Management (WSM) (Salerno et al., 2011) is the New York State adaptation of the practice known nationally as Illness Management and Recovery (Mueser et al., 2006). Available at no cost on the CPI website, WSM-COVID-19 is provided in individual and group formats and can be delivered remotely and in person. Approximately 1900 individuals attended a live webinar introducing WSM-COVID-19. A follow up survey (June 2020) of providers using this curriculum (n = 92) found that ninety-eight percent of respondents plan to use it in the future.

In its basic TA, CPI shifted from lengthier trainings to shorter and very practical types of resources. We created tip sheets, curricula that outline step-by-step procedures for delivering specific EBPs via telehealth, and webinars on using the tip sheets at the point of care. Challenges with basic TA included the need for rapid development and dissemination of tools and resources, especially for those who care for high-risk populations in a time of crisis. Future work may focus on the development of brief role-play videos to accompany tip-sheets that demonstrate specific skills such as how to develop a safety plan.

Targeted TA

Targeted TA provides training or support to specific groups such as clinical supervisors or organizations focused on building skills and promoting behavior change through didactic workshop training and communities of practice (Cross-Technology Transfer Center (TTC) Workgroup on Virtual Learning, 2021). Since its inception, targeted TA at CPI has included a combination of synchronous and asynchronous training experiences. Synchronous offerings include regional in-person meetings, on-site training, coaching in the field by implementation specialists, remote consultation and remote learning collaboratives designed to increase staff competency. Asynchronous offerings include interactive eLearning courses, and archived webinars.

After the NYS on PAUSE order, on-site training and technical assistance were conducted remotely. Notably, registration for eLearning courses more than doubled from about 25,000 in the six months before March 2020 to over 57,000 in March 2020 alone (Hinds et al., 2020). Recognizing that learners might be harder to engage remotely, CPI made several adaptations to increase the acceptability of remote training. First, remote trainings were shorter (e.g., a full day training would instead be offered as two 3 hour sessions with multiple breaks and offered on separate days, often with at least 1 day between). Second, the number of interactive activities embedded throughout the training increased including using remote platform features to engage the learners in these activities. For example, polls were used to measure knowledge (either as a baseline before a topic was presented or as a follow-up to gauge application of the concept) and to help people apply skills (e.g., choose which of the responses is an amplified reflection for the following statement, “I guess I do smoke too much sometimes, but I don’t think I have a problem with smoking”; correct response would be “Your smoking doesn’t worry you at all” compared to other types of reflections like “You don’t think your smoking has gotten so bad that it is a problem you have to do something about”). Similarly, chat boxes were used to ask people to respond to questions (e.g., where are you likely to encounter this in your practice?), to apply skills (here is a statement, write out a reply that would emphasize autonomy), and to brainstorm (e.g., here is a case study, what are the person’s strengths?). Breakout rooms were used to facilitate small group discussion and role playing. Finally, more mixed media (e.g., videos demonstrating skills) were introduced to vary the presentation of information.

Case Study: Assertive Community Treatment (ACT)

Assertive Community Treatment teams provide intensive community-based treatment for individuals with severe behavioral health challenges. ACT participants can have a history of recurrent hospitalizations, complicated by substance abuse, trauma, incarceration and other factors. ACT teams are multidisciplinary and each team member (e.g., nurse, substance use specialist, vocational specialist, family specialist, and team leader) is required to participate in role-based training through the CPI LMS. The ACT Institute provides support through an online training curriculum (Thorning et al., 2016) and remote clinical care calls and meetings for 108 ACT teams (Thorning et al., 2016, Thorning & Dixon, 2020). Remote clinical care calls for each ACT team member consisted of discussion of concerns, challenges, policy, and evidence-based practice implementation. More recently implemented, justice- informed practice calls focused on challenges (i.e., employment, housing, and overall community re-entry) teams encounter while working with individuals that have a history with the justice system. Overall, since March 2020, 580 ACT providers participated in clinical care and justice-informed practice support calls. Furthermore, the ACT Institute listserv disseminated guidance, policies, and regulations about telehealth-based care from NYS-OMH (https://practiceinnovations.org/initiatives/act-assertive-community-treatment/act-links/act-covid-19-resources-for-act-providers).

In response to the provider and team-level needs expressed during routine remote clinical care calls and meetings, the ACT Institute announced a COVID –19 webinar series on topics such as ACT Telehealth Guidance for Managing ACT Participants Who Express Suicidal Ideation During the COVID-19 Pandemic and How do we best engage with ACT participants in NYS Shelters during COVID-19. Between May 2020 and June 2020, about 680 ACT providers participated in the COVID-19 webinar series. The ACT Institute adapted their TA to host three town hall meetings for 70 ACT providers (representing 28 unique teams) from June to August 2020, focused on delivering ACT with greater flexibility. The purpose was to curate lessons learned from providers to help inform the NYS-OMH on how ACT services were adapted to meet the needs of ACT participants during the pandemic. Specific examples discussed were triaging participants who needed in-person visits for long-acting injectable medications and creating new protocols guiding daily ACT team activities.

Case Study: Focus on Integrated Treatment (FIT)

The FIT initiative continued to offer twice monthly support calls throughout the pandemic, with the frequency of total calls (40 calls from March 2020 to December 2021), similar to previous years. Providers shared successes and concerns related to providing telehealth and practice during the COVID-19 pandemic. They noted challenges such as lack of access to basic needs (WIFI, mobile phone or computer) for delivering telehealth-based care, not understanding how to use the technology to connect, and mistrust of technology among their staff and care recipients. At the same time, as others have observed (Pinals et al., 2020), many providers reported that telehealth decreased their no-show rates and allowed them to engage with and connect with care recipients who had been reluctant to travel to in-person appointments, even prior to the pandemic. In addition to an observed increase in substance abuse among care recipients, providers reported struggling to identify ways to meet people’s basic needs (e.g., food banks shut down). Additionally, the providers, who themselves were feeling socially isolated, offered each other connection and support, discussed how to manage staff fear and concerns when providing in-person care as well as managing care loads of ill staff members.

Targeted TA at CPI shifted towards using shorter training sessions. The Center convened providers for town hall meetings, initiated care and support calls to assess needs and priorities specific to their programs as well as facilitate sharing of adaptations in telehealth-based care delivery. Challenges with targeted TA included workforce lack of knowledge to deliver telehealth-based care, burnout and stress related to large careloads and staff illness. In addition to creating spaces for discussion of burnout and providing emotional support, future work may focus on the development of resources and webinars to promote workforce self-care and using remote platforms and its features (e.g., how to screen share, use Zoom white board features during telehealth sessions).

Intensive TA

Intensive TA consists of an ongoing, customized consultation to specific sites, communities, or systems to fully incorporate a new practice into real-world settings (Cross-Technology Transfer Center (TTC) Workgroup on Virtual Learning, 2021). At CPI, intensive TA includes ongoing consultation, supervision, performance feedback, practice facilitation and site visits.

Case Study: Individual Placement and Support (IPS)

Over the past decade, IPS has provided learning collaborative-related webinars and remote meetings for the Individual Placement and Support model of supported employment to programs across NYS including community psychiatric rehabilitation programs and state facility clinics (Margolies et al., 2015). During the period from March 2020-December 2021, a total of 29 collaborative-related webinars and remote meetings were provided, similar to previous years. The focus of the webinars and meetings shifted to sites sharing innovations, accomplishments, and discussing challenges in adapting the delivery of IPS. Finances and benefits became even more prominent during the pandemic with some discussions focused on stimulus payments, unemployment benefits, and ambivalence and concern expressed by some care recipients concerning the health risks related to community exposure to COVID-19 while working; a topic that received considerable attention. To address this challenge, the IPS program adapted its training and TA by using collaborative learning and educating learners about shared decision making to encourage employment staff to help care recipients to consider the pros and cons of their work options and how this aligns with the values.

Site-specific training and consultations continued, now using remote platforms. For example, there were 173 remote site visits for the period of April 2020 through December 2021. Similar to in-person site visits, remote site visits focus on reviewing accomplishments and challenges, providing guidance, and training staff in IPS knowledge and skills. One challenge with remote visits is the inability for trainers to model these skills and trainees to practice them in the community.

Training of newly hired IPS staff (n = 26), notably a higher number from previous years highlighting the challenges associated with training a workforce with a high rate of turnover, focused on modeling and role-plays of remote job development skills and reviewing the IPS fidelity scale. In a survey of IPS learners (n = 164) conducted in December 2020 asking about satisfaction with the various training events and supports provided, 18% of the responses rated efforts as “extremely helpful,” 39% “reasonably helpful,” 28% “somewhat helpful,” 10% “slightly helpful,” and 5% “not helpful.” Though tenure as an employment specialist is not formally assessed in our survey, anecdotal evidence points to the range of experience among trainees with some recently employed with little to no experience while others more experienced. Feedback indicates that some of the more experienced trainees find redundancy in various training events. In the future, stratifying training and TA experiences based on the level of experience with IPS may enhance and optimize training with varied skills and experience in IPS.

Case Study: New York City Tobacco Cessation Training and Technical Assistance Center (NYC TCTTAC)

Through a grant with The New York State Department of Health and Mental Hygiene, the FIT initiative staff partnered with Dr. Jill Williams at Rutgers-Robert Wood Johnson Medical School to help behavioral health programs implement treatment for co-occurring tobacco use forming the New York City Tobacco Cessation Training and Technical Assistance Center (NYC TCTTAC). Prior to the pandemic, in-person intensive training and blended technical assistance included monthly in-person meetings with agency workgroups, and as needed additional email correspondence and remote meetings between in-person meetings. As an example, NYC TCTTAC began working intensively with a large agency in 2019, offering in-person training for their staff, providing guidance through a baseline evaluation of capability to treat tobacco across programs, supporting development of an implementation plan, and attending monthly in-person workgroup meetings. In early 2020, the agency pivoted to providing services through telehealth. Accordingly, we continued to offer training for new staff and technical assistance remotely to support ongoing work on their implementation goals. During this time, the agency had to pause one goal, keeping free nicotine replacement in break rooms to assist staff who smoke, until staff return to the office. Another goal, providing a group to address tobacco use, was delayed until they were able to ensure that both the technology and comfort using that technology was in place for people to join a group remotely. In response, NYC TCTTAC adjusted and tailored its training and TA to feasible goals for the agency. In other areas, including increasing staff competency and improving screening and assessment of tobacco use, the agency was able to demonstrate significant progress (Covell et al., 2021).

Intensive TA at CPI was adapted by shifting the focus of training content due to emerging needs during the pandemic, conducting site visits remotely and recalibrating implementation goals according to what was feasible for the agency. Challenges with intensive TA included workforce turnover and the inability for trainers to model skills and trainees to practice them in the community. Future training may benefit from a priori assessment of skill and experience in order to tailor training and TA experiences and pairing trainees with each other to role-play specific skills while trainers observe and provide feedback during remote TA.

Supporting a State Mental Health Authority

Given the infrastructure and resources of CPI as well our relationship with the provider community, the Center was able to provide rapid assistance to support our state mental health authority during a time of crisis. Shortly after NYS on PAUSE order began the NYS Office of Mental Health launched a COVID-19 Emotional Support Helpline staffed by more than 10,000 volunteer providers. Providers (social work students and supervisors) were required to complete online training in crisis counseling and how to navigate the support line through the CPI LMS before they could begin answering helpline calls (Hinds et al., 2020). Specifically, students had to complete a course in Psychological First Aid and attend eight didactic webinars focused on disaster mental health, cultural competence, coping with grief and loss, the role of suicide and self-harm, parenting during a pandemic, substance use, providing emotional support to first responders and self-care during a crisis. CPI staff facilitated the last two webinars. The Emotional Support Helpline continues today as part of New York Project Hope (described below).

New York Project Hope is a NYS-OMH program funded by the Federal Emergency Management Agency and administered by the Substance Abuse and Mental Health Services Administration. As noted on its website, “NY Project Hope helps care recipients understand their reactions and emotions during COVID-19. Through an emotional support helpline, educational materials, and referrals, NY Project Hope helps people manage and cope with changes due to COVID-19.” For this program, CPI collaborated with subject matter experts and course developers, instructional designers, and videographers to create educational materials for the public, create an online crisis-counseling experience, and training materials for crisis counselors in several languages. For example, one set of public-facing content focuses on six topics: depression, PTSD, prolonged grief, impact of COVID-19 on children and their caregivers, pandemic as an ongoing experience, and racial justice and disparities in access to care. The eLearning courses were presented in several languages including Chinese, Korean, and Spanish. Our timeline to develop NY Project Hope curricula and eLearning courses decreased significantly (from 6–8 months to 3–4 months) compared to pre-pandemic timelines. With this shortened timeline, CPI developed archived webinars within six weeks to fill the gap between the eLearning course development and onboarding and training the crisis counselors.

Discussion

Like other intermediary organizations (e.g., Cross-Technology Transfer Center (TTC) Workgroup on Virtual Learning, 2021; Druss et al., 2021), CPI had to quickly pivot from a brick-and-mortar entity providing blended in-person and remote training and TA to fully remote delivery. In addition to revising the format of training and TA to remain engaging and effective through remote platforms, CPI also developed COVID-19 pandemic-specific content to meet the needs of NYS behavioral healthcare providers, care recipients and the state mental health authority.

While there are advantages to providing in-person training and technical assistance (e.g., higher engagement, ability to see first-hand a provider’s environment and processes), the shift to increased levels of remote training and technical assistance also yielded some benefits. First, prior to the pandemic, our intermediary organization spent an average of $45,000 annually to cover travel costs for in-person training and technical assistance. These funds were re-allocated toward the development of additional training and tools. Second, the elimination of travel increased productivity. Third, participation in training and technical assistance activities increased. Fourth, with the widespread staffing shortage that many agencies are experiencing, CPI has provided support to programs quickly and trained new staff almost immediately after hiring. Previously, there was a delay as people had to wait for scheduled training in their region. It is also noteworthy that CPI could shift quickly, because it had the existing infrastructure for remote training prior to the pandemic. Together, these advantages increased our access and availability to programs while freeing up resources to invest in additional training and supports.

The Center for Practice Innovations will likely adopt a hybrid approach to maximize the benefits of both in-person and remote training and TA. For example, training may be offered initially in person to increase engagement and understanding of their local environment, processes, and culture and then add remote technical assistance. Other intermediaries might consider adopting a similar hybrid approach to maximize the impact of their resources. However, as the pandemic evolves and remote delivery of TA and technical assistance continues, it will be critical for IPOs to evaluate the effectiveness of in person versus remote training and technical assistance strategies which may be more cost-effective and convenient for the IPO staff and behavioral health workforce.