Training Public Sector Clinicians in Competency-Based Clinical Supervision: Methods, Curriculum, and Lessons Learned.

This article describes an initiative to train public sector clinicians in competency-based clinical supervision. It was delivered as an 18-session course taught online to clinicians employed in departments of behavioral health in nine Southern California counties. The curriculum was co-constructed by a team of clinical supervision scholars and leaders who then served as instructors. Each two-hour meeting addressed a specific topic for which a training video had been prepared, usually featuring a member of the training team who had expertise in that topic. The second part of each meeting focused on a class member's supervision case presentation. Those presentations revealed 35 themes; the four most frequently occurring were: developing supervisees' clinical competencies, addressing countertransference and parallel process, balancing clinical and administrative supervisory roles, and addressing record keeping/paperwork. Participants' pre-to-post supervisory self-efficacy changes demonstrated a moderate effect size (Cohen's d = .46) for the training, with the greatest pre- to post-training changes being in the use of technology, multicultural competencies (awareness of oppression, bias, and stereotyping in clinical work and in clinical supervision), and contracting. They reported that the strengths of the course included an inclusive learning environment and opportunities to reflect on and apply new knowledge and skills, though they also reported struggling with the assignments and the course platform software. Lessons learned reflected the use of technology in this online program, the importance of obtaining buy-in from agency decision makers and being prepared to address challenges related to the use of direct observation in supervision, gatekeeping, and enacting the simultaneous roles of administrative and clinical supervisor.

Competent clinical supervision can provide public sector mental health agencies with important fiscal and clinical benefits by helping to decrease staff turnover, reduce clinicians’ emotional exhaustion (Knudsen et al., 2013) and improve the quality of services (Powell et al., 2015). But the operative term is “competent.” Only a small portion of mental health professionals enter professional practice having had the specific and systematic training required for supervisory competence (see APA, 2014, 2015; Bernard & Goodyear, 2019; Falender & Shafranske, 2021), and perhaps as a result, much supervision is ineffectual or even harmful (Ellis et al., 2014). Although some state licensure boards have begun to require licensees who supervise to participate in supervision-focused continuing education (CE), that is, on its own, insufficient as CE tends to be self-directed and characterized by isolated training events that rarely build sequentially and that focus primarily on knowledge acquisition rather than skill development. Therefore, in the absence of other mechanisms to ensure that their supervisors are qualified to supervise, some public sector mental health agencies are implementing their own policies and workforce development programs focusing on supervisor competence.

But agencies taking this step still have few models from which to draw in developing that training. The best-known training protocol is one that faculty in the Yale Supervision Program developed (e.g., Hoge et al., 2016). However, the Yale approach has been to deliver training over a relatively short period (e.g., a five-day workshop; Tebes et al., 2011), precluding the practice and consolidation of skills that can only occur over more extended training periods. As well, their use of an in-person training format can be challenging when clinicians participating in that training are dispersed across large areas.

This article describes a training program which was part of a larger project that was designed to develop supervisory competence in licensed mental health professionals in county departments of behavioral health across an area larger than that of 19 states. One of the project’s two components was a series of three- and six-hour, CE-granting webinars offered over the course of a year. These webinars were open to anyone employed in departments of behavioral health in the mental health region covered by the project and could either be attended live or viewed later as recordings (see: sites.google.com/usc.edu/bsc). Topics were determined by soliciting input from county training leadership, clinicians offering supervision in those counties, and the project co-directors’ knowledge of training needs and issues.

The project component that is the primary focus here was a nine-month program offered as an online course to a cohort of clinicians who were providing supervision or planning to do so. This article describes that program’s design and implementation, its structure and curriculum, issues addressed, and program outcomes. It concludes with a review of lessons learned.

Program Participants and Design

Project leaders Carol Falender and Rod Goodyear planned the training based on competency-based frameworks (see, esp., APA, 2014, 2015; Falender & Shafranske, 2021). They understood that to enable sustained participation of clinicians who were spread across such a large area, the program would have to be offered online, and that in so doing, they would be using an educational format that matches and even exceeds the effectiveness of in-person instruction (Means et al., 2013). An online format is also consistent with tele-mental health trends, including in supervision. Notably, Inman and her colleagues (2019) found that supervisees rated online supervision as effective as face-to-face supervision.

A distinctive feature of this training was that it targeted post-licensure professionals whose supervisees serve challenging clinical populations, including clients with serious mental illness, substance use disorders, co-occurring diagnoses, serious mental illness, foster care, prison release, family reunification, and as well as immigrant children and families. As a result, the training needs and circumstances of the clinicians in this project who were learning to supervise were very different from those of the graduate students who have been the recipients of most systematic supervision training efforts (e.g., Watkins, 2012).

For example, graduate students may take a full-semester clinical supervision course which typically addresses not only supervision skills, but theory and research as well. In addition, these courses tend to involve required readings, a writing or research project, and occur with classmates who all are training for the same profession. In contrast, supervision training for licensed clinicians tends to be offered to interdisciplinary cohorts as in-service activity that is often at a cost to other workplace responsibilities. These realities mean that clinicians especially value immediate, practical, applications of supervision strategies (vs. coverage of theory and research), and minimal requirements for outside reading, homework, or experiential learning.

To develop a training curriculum that was practitioner-centric, Falender and Goodyear assembled a team (i.e., the authors of this article) of supervision scholars and leaders who would both serve as instructors and co-developers of the course. This co-construction process occurred over a period of almost five months following an initial meeting that reviewed the essential tenets of competency-based supervision. The training team exchanged ideas and instructional resources among themselves, uploading materials to be considered for the curriculum to a shared website. The eventual program was the result of this collaborative process.

Participants

One hundred twenty-nine public sector clinicians signed up for the program. Across the nine-month course, 38 (29.4%) dropped out for a range of personal and work scheduling reasons. Ninety-one (84.6% female; 13.4% male) clinicians completed the training. The majority (51; 56.0%) were licensed marriage and family therapists. The remainder were licensed social workers (31; 34.1%), licensed psychologists (5; 5.5%), or those who had multiple degrees (e.g., marriage and family therapy, social work, psychology, or “other;” 4; 4.3%). Recruitment details are presented below.

Trainers

The online virtual format of the course made it possible to recruit trainers from across the U.S., people selected for their expertise in supervision theory, research, and practice. The 15 (including Drs. Falender and Goodyear) trainers were doctoral-level clinical or counseling psychologists or, in one case, a counselor educator. Like the trainees, they were racially and ethnically diverse and primarily female.

Recruitment of Trainees

The initial goal was to recruit 130 trainees, but to leave the methods and criteria for recruitment to the Workforce Education and Training (WET) coordinators in each county (or in the case of Los Angeles County, a small within-county region that affiliates with SCRP). They were also asked to apportion among themselves how many clinicians each would be nominating to meet the target goal of 130 participants. To support their recruitment, the project coordinators developed a brief video describing the program that WET Coordinators could distribute. Although most clinicians participated voluntarily in the training, some were mandated to attend, which potentially contributed to differences in their level of engagement and drop-out rates.

To be eligible, clinicians had to be licensed in one of the mental health professions employed in county mental health and currently be providing supervision (although 16% of those who ultimately completed the program were not supervising at the completion). Participants could earn CE credit for their respective professions by virtue of their formal course participation, for a maximum of 36 CE hours total (18 meetings; 2 h each).

Course Structure

The training was delivered as a university-based professional development studies course (Fundamentals of Competency Based Clinical Supervision offered for 0-units and on a credit/no credit basis), a structure that provided both a learning management system and university library access to participants. Participants paid no fees as their costs were covered by project funding.

Prior to the first group section meetings, participants convened for a six-hour, in-person, meeting at a site that was centrally located within the project region. That meeting introduced competency-based clinical supervision as the foundation that was to follow and gave particular emphasis to the competencies developed by the mental health region sponsoring the training (Buckles et al., 2014). A secondary purpose of this meeting was to start building group cohesion as participants were seated at tables that included other members of the group with whom they would be interacting for the 18 sessions of the online course.

The actual course launched several weeks after that meeting. It was organized as 13 sections, each with 8—11 clinicians and a member of the training team who served as the facilitator. Each section met online for two-hour sessions twice per month for nine months using the Zoom platform. Assignment to sections was made largely based on trainer and participant availability as members of both groups had highly individualized schedules; this variability resulted in the 13 sections being scheduled at 10 different times on four different days during work hours. Most participants attended from their work sites, with their participation recognized as part of their work responsibilities.

The groups were comprised of people from multiple counties (e.g., one group had eight counties represented), which meant that participants typically did not know each other prior to the class. Thus, there was the opportunity for cross-county collaboration and learning which proved to be a significant strength of the training as the counties were very diverse in their policies and practices. In a few instances, people from the same county or work setting, with different levels of administrative responsibility and power, were in the same course section. This introduced an additional layer of complexity to the dynamic, though it ultimately enhanced collaboration and perspective-taking among group members.

Once assigned to a section, participants were to remain in it for the duration of the course to promote group cohesion. They were to attend at least 14 classes to obtain credit for participating. Although there were no required readings, participants were given the option to learn more about the material covered in each class via both links to articles (typically between three to six articles for each class) and other resources, such as websites, measures, and forms.

Each class meeting began with an instructor-led presentation on a specific topic and then shifted into a discussion of a prepared case presentation by one of the class members. To standardize the presentations across sections, the instructor-led presentation occurred during the first hour of class, driven by a video presentation produced for the project. The meeting-by-meeting topics and their order of presentation were as follows: an introduction to competency-based clinical supervision; multicultural supervision; developing effective supervisory and therapeutic working alliances; identifying and repairing relationship strains and ruptures; recognizing and responding to parallel processes in supervision; ethical and legal issues in clinical supervision; helping supervisees identify and manage countertransference and reactivity; forms of direct observation in clinical supervision and their importance; gatekeeping in clinical supervision; Interpersonal Process Recall and the importance of reflective practice; the Integrative Developmental and Discrimination Models of clinical supervision; technology and telehealth in supervision; using client feedback in supervision; triadic and group supervision; self-care strategies for supervisors and supervisees using client feedback in supervision; deliberate practice as a training method; chalkboard case conceptualization process; and, self-assessment of competencies as a supervisor.

To make the videos presenting the material more engaging, they were developed in an interview format: For each one, Drs. Falender or Goodyear would interview an expert who was usually a member of the training team with specialized knowledge of that particular topic; in some instances, they would interview well-known experts who were not otherwise part of the project. Each video was between 25–30 min and included several pre-planned stopping points with discussion questions to guide class discussion. This use of the videos both standardized the content across sections and created a compendium of curricular resources that could be used for future training. Instructors were also provided with a course manual that provided a week-by-week structure and guidance, though they had considerable latitude in how they conducted their own class.

The second hour of class typically focused on a case presentation that one of the participants distributed to the other participants ahead of class. The presentations were prepared using a standard protocol developed to ensure client (and to the extent possible, supervisee) information was de-identified per HIPAA regulations. For example, participants were instructed to change at least three demographic characteristics of any client (and supervisee) who was described. The presentations were used as a springboard for discussions about the application of course material and supervisor competency development.

The instructor would begin the case presentation by asking the presenter what competency area she, he, or they wanted to see strengthened or improved as a result of making the presentation and obtaining feedback. The types of issues that were raised in the case presentations are described later in this article. At the conclusion of the discussion prompted by the presentation, the facilitator checked in with the presenter and the other participants about what they learned from the presentation to clarify any gaps in understanding and further solidify plans for incorporating the feedback into their work with supervisees. Early-on, some participants would bring a therapy rather than a supervision case, which is consistent with the behavior of a clinician who is still in the process of adopting that new supervisor lens or identity.

Instructors began each class by inquiring about any connections they had made in their recent work with their supervisees to the content from the previous week’s lesson and by asking participants how they were doing both personally and professionally. At this point, they would flag any material that might be helpful to revisit later in the meeting. Trainers reported that the check-ins promoted group cohesion and personalized the training experience. Trainers also overwhelmingly indicated that the combination of lectures and presentations kept the group engaged by providing participants with in-vivo opportunities to apply what they learned during the lecture and bring the material to life during the presentation portion of the course.

Following each meeting, participants also completed a brief quiz on the content that was presented in the video for that week. They also completed a feedback rating form to evaluate the class and facilitator.

Adapting the Training in Response to Unanticipated Issues

Any new program will face issues that were not anticipated as it was being planned. This program was no different and the following four issues required particular attention. The first two of the four demanded actions that had to be taken expediently in response; the latter two issues were less urgent, but they did require instructors to adjust the levels and types of attention they gave to the course material.

Technological Proficiency and Access

Because the project began pre-COVID, the seismic shifts to telehealth and telesupervision that forced mental health professionals to develop proficiency with internet-based technologies had not yet occurred. As a result, the first, and very pressing issue that had to be addressed concerned limitations to participant technological competence and self-efficacy. Those limitations were compounded by institutional policies and technology capacity. These issues took multiple forms:

The project relied on the University’s learning management system (LMS; Moodle) to house course materials, including the syllabus, links to week-by-week readings and to web and other resources, links to videos, supervisee case presentation uploads, and post-session quizzes. But accessing the LMS required that participants use newly issued University email accounts (which they otherwise had no reason to use) and a two-factor authentication system that many had difficulty mastering.

Later, many experienced a second round of problems accessing the LMS when six-months into the project, the University required password changes. Because participants were not accessing their university email accounts regularly, they did not receive notifications that they were to change their passwords and thus, were puzzled and often upset when they suddenly found themselves locked out of the course LMS.

Because their project involvement was a work assignment, most participants used their county work email addresses. It took some time to realize that a subset of participants was not receiving messages and notifications from project leadership or instructors. It turned out that some county firewalls were blocking email messages from addresses that had not been previously approved including those from the project.

Participants were unfamiliar with Zoom and so there were initial difficulties as they learned. This was also true for the instructors, who often found themselves struggling to master the nuances of Zoom instruction while in session with their class participants.

Counties also often prohibited the use of employees’ personal computers for this project, but then did not reliably provide equipment or space that was adequate for videoconferencing. The IT personnel in the affected counties actively worked to find eventual solutions, but it took some time to get everything resolved.

A related issue occurred when the pandemic forced almost all to work from home; several participants who otherwise had relied on their office equipment did not own a personal computer. This reality forced them to join the classes and to access the LMS from their smartphones.

Program leadership and, especially, the IT staff at the university (and sometimes, at their respective agencies) worked diligently to help participants address these various technology hurdles as they emerged. Nevertheless, some participants were sufficiently discouraged in the first weeks of the course that they dropped out.

The COVID Pandemic

The COVID-19 pandemic lockdowns began about two-thirds of the way through the course. Serendipitously, program participants who had initially struggled with, and even resisted, the technology were now the clinicians at their worksites who were most prepared to shift to tele-mental health. Trainers also reported that participants expressed gratitude for the connections and support the course provided during this turbulent time. They had formed relationships with one another and the trainer and reported high levels of investment and engagement in their learning. In fact, no attrition occurred as a result of the pandemic.

That said, flexibility and compassion were central to maintaining a positive morale and staying present amidst significant challenges. For example, some participants, due to their setting or population, were working onsite and were experiencing high levels of stress and fear. Others were balancing home-work pressures, including the homeschooling of their children, partner job losses, an influx of very high-risk clients, and staff availability to work. Many participants were impacted by the murders of George Floyd and Breonna Taylor, and the protests that followed.

Participants’ Conflicting Work Roles as Both Administrative and Clinical Supervisors

About half of the participants were serving simultaneously as clinical and administrative supervisors (52% according to the pre-training survey; a proportion consistent with earlier research by Tromski‐Klingshirn & Davis, 2007). The case material these participants presented illuminated the complexity of this type of multiple relationship, including issues related to the disclosure of personal responses and reactivity, relationship strains and ruptures, power dynamics, and evaluation.

Attitudinal and Institutional Barriers to Common Supervision Practices

Three normative supervisory practices proved challenging and, in some instances, impossible to implement in participants’ work settings: direct observation of supervisees’ work; obtaining client feedback through routine outcome monitoring to use in supervision; and gatekeeping. A very high percentage of these participants had never been observed themselves (e.g., live observation; video recordings; etc.) and had never observed their supervisees. As a result, they had no firsthand knowledge of its utility and value for clinical supervision and training. Encouraging them to begin using direct observation was made more complicated by multiple factors, including legal and setting restrictions within particular counties, concerns about confidentiality and HIPAA, storage of recordings, access to recording equipment as well as concerns raised with specific client populations (e.g., those pending incarceration or post-release, children in foster care, individuals under legal mandate for services) and settings where recording was prohibited. Almost all participants described a lack of equipment, space, technical expertise, and financial resources. As well, departmental leadership, concerned about liability issues, had rules in place that precluded direct observation of supervisees’ work.

Similar attitudinal and client population limitations also made it difficult to use client feedback through routine outcome monitoring in most settings, even though it was strongly encouraged, and participants were increasingly convinced of its value due to this course. Overall, multiple constraints precluded the implementation of these routine quality assurance measures.

Finally, gatekeeping—usually understood to be a fundamental responsibility of supervisors (APA, 2015, Bernard & Goodyear, 2019; Falender & Shafranske, 2021) – was difficult for participants because of workplace policies and a unionized work culture. In response, the video shown during the week in which gatekeeping was featured included leadership from several of the counties who were able to discuss their perspectives on supervisors’ options and responsibilities related to gatekeeping. Trainers also reported that early career supervisors in their sections had benefitted from receiving input and guidance from their more seasoned counterparts about how to navigate gatekeeping responsibilities and accompanying challenges.

As the trainers developed a better understanding of participants’ realities in their respective work settings, they were able to help participants identify small, but meaningful, changes they could make to their supervision practices in the context of these constraints. They were also able to engage participants in discussions about how they might implement new practices, such as outcome assessment, which could be more clearly linked to client progress, as longer-term goals.

Systematically Obtained Data on Training Impacts and Participants’ Issues

The training team profited from a great deal of informal feedback while delivering the training and interacting with participants. As well, there were more intentionally and systematically gathered data. These included three types of data on training impacts: pre-to-post training changes in participants’ supervision self-efficacy; participants’ post-training ratings of the likelihood with which they would use particular competencies; and participants’ satisfaction with the training. Material presented in case reports was also analyzed in order to characterize the issues and challenges most affecting the work participants were doing as supervisors.

Pre-to-Post Changes in Participants’ Supervision Self-Efficacy

The Clinical Supervisor Self-Assessment Scale (CSSAS) was used to measure supervisors’ self-efficacy expectations regarding their ability to perform 52 clinical supervisor competencies (Ellis et al., 2019). The supervisor competencies were derived from and based on the 28 guidelines delineated in the Guidelines for Clinical Supervision in Health Service Psychology (APA, 2015). Supervisors were to anchor their responses to their work with a particular supervisee and rate their degree of confidence to perform the given action in supervision with the identified supervisee using a 0 (Not at all) to 100 (Totally can do) rating scale, with the option of indicating that they cannot perform that competency at their site. Supervisors who were unable to perform the specific action at their site rated their degree of confidence that they could (or could if permitted) perform the action in supervision with the chosen supervisee. The measure is currently undergoing full psychometric development. Preliminary item analyses and confirmatory factor analyses results from a sample of 463 supervisors suggest a single dimension (i.e., total scores) comprised of 48 of the 52 items explaining 42.4% of the variance, with item-factor loadings exceeding 0.60. Cronbach’s alpha for the total scale score was 0.97.

Sixty-three participants completed the CSSAS both before and after training and 91 did so after training. They were asked to “rate your degree of confidence that you can perform the action in supervision with the supervisee you identified …(even if it is not possible to engage in that action in your work setting).” Whereas the mean pre-training score was 80.15 (SD = 5.45), the corresponding post-training scores were 87.32 (SD = 5.37). The resulting Cohen's d = 0.46, which approximates a medium effect size (see Sawilowsky, 2009).

When the 52 competencies were considered separately, the largest pre-to-post training change (pre-post change: 27.7; Cohen’s d = 1.15) was in self-efficacy in using technology in supervision, including telesupervision. The next three largest changes were in items concerning self-efficacy to provide supervision anchored in the current evidence base regarding supervision (pre-post change: 10.53; Cohen’s d = 0.58), attend to diversity issues across populations and settings in clinical work and in clinical supervision (pre-post change: 10.53; d = 0.72), intentionally infuse and integrate the dimensions of diversity in all aspects of professional practice (pre-post change: 10.13; d = 0.67), and using and adhering to the supervisory contract and to program, institutional, and legal policies and procedures related to supervisee performance evaluations (pre-post change: 9.70; d = 0.55).

On the other extreme, there was one competency for which self-efficacy actually decreased: Use live observation or review recorded sessions to monitor and provide feedback on my supervisee’s performance (pre-post change: − 3.90). Possible reasons for this result are addressed later.

Participants’ Post-Training Intention to Use Supervision Competencies

Participants were also provided with a list of 20 competencies and asked to “indicate whether you are intending to use each of the following competencies in your work as a supervisor.” The competencies were rated on a five-point scale that ranged from 1 (Definitely will not use this competency) to 5 (Definitely will use this competency). These data are summarized in Table 1, which organizes responses from highest (most likely to use) to lowest (least likely to use). It shows that the four competencies that they rated as having the highest likelihood of being used were: Attending to legal and ethical issues in supervision and in the work my supervisees do (M = 4.99; SD = 0.11); Fostering habits of self-care (M = 4.99; SD = 0.11); Multicultural supervision/cultural humility (M = 4.94; SD = 0.23); and, Recognizing/responding to relationship ruptures (M = 4.93; SD = 0.29).

Table 1

Participants' post-course ratings of the likelihood they would use specified competencies

	M	SD
The four competencies participants indicated they were most likely to use
Attending to legal and ethical issues in supervision and in the work my supervisees do	4.99	0.11
Fostering habits of self-care	4.99	0.11
Multicultural supervision/cultural humility	4.94	0.23
Recognizing/responding to relationship ruptures	4.93	0.29
The four competencies participants indicated they were least likely to use
Triadic supervision	4.19	0.92
Using session-by-session client feedback in supervision	4.01	0.91
Using live observation in supervision	3.95	0.95
Doing cotherapy with supervisees as a method of supervision	3.56	1.03
Using video recordings of supervisees' work in supervision	3.32	1.23

A five point scale where 1 = Definitely will NOT use; 5 = Definitely WILL use

The four lowest-rated competencies were: Using session-by-session client feedback in supervision (M = 4.01; SD = 0.91); Using live observation in supervision (M = 3.95; SD = 0.95); Doing co-therapy with supervisees as a method of supervision (M = 3.56; SD = 1.03); and Using video recordings of supervisees' work in supervision (M = 3.31; SD = 1.23).

Course Satisfaction and Impact Ratings

Approximately two months after the course ended, 68 (74.7%) of the participants responded to an online satisfaction and impact survey by project evaluators who were not part of the training team. Across the board, satisfaction ratings were high. For example, using a four-point scale where 4 = very satisfied, the two highest rated items concerned whether they found the class content interesting (M = 3.51; SD = 0.68) and enjoyed the instructors’ presentation of the video material (M = 3.51; SD = 0.63). The lowest satisfaction score concerned their experience using Moodle (M = 2.94; SD = 0.77), the online learning management system.

Another series of questions assessed perceived course impact using a 4-point scale where 4 = strongly agree. The four highest rated items were “the instructor facilitated meaningful discussions of the material” (M = 3.57; SD = 0.61), “the course gave me a deeper insight into the topic” (M = 3.53; SD = 0.59), “the course helped me understand skill areas I need to strengthen in my practice” (M = 3.53; SD = 0.61), and “the content was relevant to my practice” (M = 3.47; SD = 0.61). The mean score for the item, “as a result of taking this course, I have changed the ways in which I interact with supervisees,” was 3.09 (SD = 0.69).

Issues Evident in the Case Presentation Material

The case vignettes that participants presented during each class provided a window into the issues and concerns that were most pressing to them. Presenters would often align their presentations with the topic being discussed that week, even though they had autonomy in choosing the questions they wanted addressed. Therefore, to better understand what they perceived as most salient for the work they were doing, data were drawn from the written forms that presenters distributed to their respective groups ahead of the class meetings to conduct a thematic analysis. The focus was particularly on responses to these two questions:

What is it about this particular vignette or situation that led you to choose it to present?

What are the question(s) or issue(s) that you would like us to address in the supervision case conference? Please be specific. Please frame questions to relate to your supervision with the supervisee and specifically, their work with the client.

Data were the 198 questions elicited by these prompts, drawn from 118 vignettes submitted across 12 of the 13 class sections, reduced to 116 because two were strictly therapy cases, rather than supervision cases. Categorizing the issues captured by those questions resulted in 35 broad themes using key words that the participants used in their write-ups. Table 2 reports the 10 issues that were most prevalent, along with the frequency with which each was raised.

Table 2

The 10 most frequent themes (of 35) in the case material participants presented

	Theme	% of total
1	How to increase supervisee’s clinical skills, interventions, psychological mindedness	19.7%
2	Processing countertransference, transference, parallel process with supervisee	11.1%
3	Balancing clinical and administrative supervision roles	6.1%
4	Paperwork/documentation issues	5.1%
5	Cultural issues	4.5%
6	Supervisee struggles with empathy skills, process, relationship skills with clients	4.5%
7	Assessment of client crises/dealing with crises	3.5%
8	How to give feedback	3.5%
9	How to prioritize what to do in supervision	3.5%
10	Increasing supervisee’s confidence	3.0%

The complete list of themes can be found at sites.google.com/usc.edu/bsc

Participants’ most frequently presented issue concerned how to be more effective in helping their supervisee with clinical skills, interventions, and increasing their psychological mindedness. This theme accounted for a fifth of the issues (19.7%). The next three issues that participants most often raised were: addressing countertransference, transference, and parallel process with supervisees (11.1% of the issues); balancing clinical and administrative supervisory roles (6.1); and paperwork/documentation issues (5.1%). Whereas the first two of these were anticipated in what was covered in the class, the other two – and a number of the other issues listed in Table 2 – were not and so, were handled on an ad hoc basis.

Lessons Learned and Their Implications for Future Training

This project provided not only the opportunity to design and deliver a service but also to learn important lessons that can be used in development of future programs for public sector mental health clinicians. Already touched upon in preceding sections were some of those lessons. But this section draws from both data and the team’s reflections on the experience to highlight material that seems particularly important for those planning similar training initiatives.

Trainee satisfaction with the training was generally high, though the 29.4% attrition rate meant that those who persisted through the entire nine-month program were most committed to the process, which could have affected the satisfaction ratings. This overall level of satisfaction was perceived to be a general endorsement of the work done. We were especially interested in impacts, and the very substantial pre-to-post changes in participants’ self-efficacy (CSSAS) as supervisors were especially validating.

Breaking those self-efficacy ratings out by specific competency, though, gave a finer-grained picture of training impacts. The very large increase in self-efficacy related to telesupervision reflected participants having had such a low level of technological self-efficacy at the onset and then finding themselves later applying those competencies in their work during the pandemic. Although this finding is addressed more in the section below, it is useful to note here that those who plan similar, online, trainings will likely encounter a much more technologically sophisticated group than this group initially was by virtue of changes in the way mental health practitioners work that have resulted from the pandemic.

Two of the five largest increases in self-efficacy related to areas of multicultural competence. These pre-post gains in cultural competence are important to note and should be encouraging to others planning to offer similar training programs. Multicultural competence is essential for any supervisor but assumes particular importance in public sector mental health where exceptionally sociocultural, linguistic, and economically diverse populations are being served.

Of the 52 competencies, the fifth largest change in self-efficacy occurred in the use of supervision contracts. This trend no doubt reflected the instructors’ persistent message across the course about how essential contracting is to effective supervision (note: the use of the term “contract” was not allowed in some of the counties because of its legal implications, and so an alternative term was adopted, “supervision agreements”).

Use of Training Technologies

The use of videoconferencing technology (in this case, Zoom) was a planned aspect of this project and made this training program possible. Its important benefits included being able to include trainees who were spread across a large region and being able to recruit exceptionally qualified trainers who were distributed across the country. By chance, this training began prior to the seismic shifts in service delivery and training (e.g., Hames et al., 2020) prompted by the pandemic, and so participants were more prepared than their counterparts to make this shift. So, although a recommendation is that anyone doing online training such as this prepare the participants to use the technology and learn how to access help as needed, the substantial shifts toward telehealth may obviate much of this need for others who offer similar, online programs.

It will continue, though, to be important to assess and address agency-related technology issues that can affect training. In this case, this included firewalls that blocked external emails, policies that limited the use of personal computers at the worksite, while also lacking (at least initially) the technological capacity for participants to participate fully. Another recommendation is to work with the agencies to arrange internal learning management systems given that for these participants, the university-based system was a significant source of dissatisfaction.

The Influence of Agency Contexts on Supervision Practices

The challenges that were encountered with respect to (a) using direct observation of supervisees’ work; (b) the overlapping roles of administrative and clinical supervisor; and (c) gatekeeping were described earlier. Because anticipating these issues will be important to others planning similar training programs, the following suggestions are proposed.

Direct observation

It appeared that participants’ decreased self-efficacy concerning the use of direct observation underscores its importance for those offering similar trainings in the future. As already noted, most participants had little-to-no experience being directly observed in their own training or using it as supervisors and so, they made assumptions about the competency they would have if they used it based on what was taught about direct observation. As well, those discussions brought to the forefront the constraints their work settings and client populations posed to directly observing supervisees. This last point is reflected in the data in Table 1, which shows that the three competencies they indicated they were least likely to use all concerned some aspect of direct observation.

To the extent that competence in using direct observation is at issue, trainers might consider modeling the use of video by presenting videos of their own work for discussion. They might also do as was done in this program – in the video depicting Interpersonal Process Recall –explicitly addressing how to use direct observation. It can also be helpful to use sessions to practice asking clients for permission to video, so that they are themselves comfortable doing so and can then help their supervisees do the same.

To address situations in which organizational barriers or client populations make it impossible to record or observe sessions, it is useful to discuss other methods of observing the supervisee, including working with the supervisee as a co-therapy team. Moreover, providing some training in and support of their advocacy to agency leadership to make relevant policy changes may be worthwhile. If operating within a consortium or network, member identification of successful integration of innovative practices like video recording could be shared.

Supervisors in Simultaneous Administrative and Clinical Roles

About half of the participants were in simultaneous administrative and clinical supervision roles. As Hoge et al. (2011) note, this is common in public sector mental health because it is costly to separate the roles. Consequently, it would be useful for trainers to anticipate this dual role possibility and account for it in curricular planning; participants can be supported in negotiating some of the complexities that arise from this multiple role situation.

Gatekeeping

Gatekeeping is central to any definition of supervision (APA, 2015; Bernard & Goodyear, 2019; Falender & Shafranske, 2021) in the U.S. and other Western countries. Accrediting and credentialing bodies expect it. In public sector mental health, this function is relatively straightforward when the supervisees are master’s or doctoral students. But gatekeeping can be much more challenging when the supervisees are employees. It is important in addressing gatekeeping as a competency, then, to highlight the importance of careful documentation of supervision sessions and of evaluation, as well as working to ensure that supervisors have the support of the larger administrative team with which they are actively consulting. It helped in this training program to include coordinators’ perspectives from several counties in the video lesson created on gatekeeping.

The other lesson learned was that careful screening of applicants and competence assessment during onboarding (and during the probationary period) is essential.

Sustainability

A closing observation is that supervision competence initiatives that have enduring impacts need to exist within an institutional structure that supports them with necessary policy and a training infrastructure. To that end, a subset of those who participated in the program and with county leadership positions continued for a second year to implement a train-the-trainer model, facilitated by Carol Falender and Rod Goodyear. Its purpose has been to implement supervision training, including supervision-of-supervision, site-specific projects that advance supervision, and advocacy efforts for changes to agency policies that promote a culture of training and supervisory competence. A final recommendation would be add a similar train-the-trainer component or at least to devote some time during the training program to helping participants advocate for and implement these important institutional changes.

Conclusion

Most of the previous literature on training supervisors has, either implicitly or explicitly, been concerned with training graduate students and interns. Yet, only a small portion of the mental health workforce – and particularly those in public sector mental health – are beneficiaries of that training. Although almost all will eventually supervise (Rønnestad et al., 1997), they are typically thrown into that role with no specific training for it and have to rely on what they learned from observing their own supervisors. In response, some public sector agencies are committing to supervision training as an important form of workforce development. This article adds to the small literature base that describes supervisor training programs, with the hope that others will be inspired to promote supervisory competence within their work settings.