Mental Health Staff Perceptions of Improvement Opportunities around COVID-19: A Mixed-Methods Analysis.

While COVID-19 has had widespread impact on the way behavioral health services are delivered, very little research exists characterizing how providers have perceived these changes. This study used mixed-methods to understand the complex and varied experiences of staff of a psychiatric service line at a large tertiary medical center with high community spread of COVID-19. A brief convenience survey was sent to all staff of the service line and thematic analysis generated brief themes and their frequency. Qualitative focus groups were then held to elucidate greater detail on survey responses. In total, 99 individuals responded to the survey and 17 individuals attended two focus groups in which theoretical saturation was achieved. While brief survey responses generated three broad themes, including operations, telehealth and technology, and communication, focus group data provided nuanced information about these themes, including reasons underlying heightened stress and fatigue felt by staff, inadequacy of technology while finding innovative approaches for its use, and appreciation for the benefits of telehealth while expressing concern for patients not served well by it. These mixed-methods findings highlight the complexities of implementing widespread changes during COVID-19 and demonstrate how survey and focus group data can be used to evaluate rapid care transformations driven by COVID-19.

Introduction

The COVID-19 pandemic has posed significant challenges to the delivery of mental health services and rapidly changed the way care is delivered [1]. In a matter of weeks, healthcare institutions and staff have needed to adapt and rapidly transform their practice to minimize the spread of infection. Notable changes have included adapting clinical workflows and operations, transitioning care to telehealth and remote working [2], adjusting to evolving patient needs and reduced community resources, and changing environmental health and personal protective equipment (PPE) requirements.

At the same time, the pandemic has provided a unique opportunity to innovate and redesign systems of care. Institutions and their staff have come up with transformative ideas to meet the evolving needs of patients during this unprecedented time [3]. As some parts of the U.S. and other countries are beginning to come out of the acute phase of the pandemic, institutions should consider evaluating these changes in order to foster continuous quality improvement and ensure that positive changes outlive the pandemic. An important component of this improvement process is to understand how staff have experienced clinical changes and operational transformations, including which changes have had a positive impact and which continue to need improvement.

Despite the widespread impact that COVID-19 has had on the practices of mental health staff, very little research has been published to understand how staff have perceived these changes. A small qualitative study was recently published describing the experience of psychiatrists with telemedicine during the pandemic, including the barriers encountered and the overall impact on care [4]. Yet, no studies to our knowledge have sought to understand more broadly how mental health staff, including nurses and social workers, have perceived the institutional quality improvement and operational changes during COVID-19.

At the study institution, several clinical and operational changes have been described, including use of infection prevention strategies specific to inpatient psychiatric settings, creation of a COVID-specific unit for psychiatric patients [5], and rapid telehealth implementation in the intensive outpatient program [6, 7]. In addition, significant clinical changes took place, including screening and testing requirements, training on PPE, widespread transition to telehealth and teleconferencing, implementation of quarantine processes based on exposure risk and symptomatology, expansion of communication frequency, and development of new leadership reporting structures. In order to identify which changes should be maintained or improved, we sought to understand staff perceptions of these changes and gather additional ideas for future quality improvement.

In this cross-sectional study using mixed methods, we first present data gathered from a brief convenience survey given to all staff of the psychiatric service line and then present data from qualitative focus groups performed to elucidate greater detail on major themes in the survey.

Methods

Study Setting

The study was conducted at a tertiary academic medical center in the northeast region of the U.S with high community spread of COVID-19. At this institution, COVID-19 inpatients were first detected and hospitalized on March 17, 2020, with the institutional peak occurring between April 19-21st, 2020, when 447 inpatients were admitted to inpatient medicine. The psychiatric service line includes inpatient and ambulatory services for both adolescents and adults, including a 123-bed psychiatric hospital, psychiatric emergency room, consult service in a medical hospital with 1541 beds, intensive outpatient program, and an outpatient clinic.

Survey

On June 1st, 2020, a brief survey was sent out electronically to all staff of the psychiatric service line. Staff receiving the survey included licensed independent practitioners (Advanced Practice Registered Nurse (APRN), Medical Doctor (MD)), resident physicians, nurses, physical and occupational therapists, pharmacists, social workers, milieu counselors, managers, and support staff. In order to ensure anonymous responses and to encourage honest feedback, no demographic information was obtained from the respondents of the survey, similar to what has been done to improve staff engagement at other institutions [8]. The survey was administered using Qualtrics software and included three open-ended questions: 1) What has been going well that you would like to see more of? 2) What isn’t going well and can be improved? 3) What new ideas/processes/approaches need to be part of how we practice moving forward?

Qualitative Focus Groups

In July 2020, a follow-up invitation was sent electronically using Qualtrics to all staff of the psychiatric service line inviting them to participate in qualitative focus groups to gather more detailed information on the responses generated by the survey. Brief demographic information, including age, gender, race/ethnicity, and occupation were collected from each participant at the time they signed up for the focus group and data were aggregated to ensure anonymity. A total of 20 participants were sought and two focus group times were offered on August 7th, 2020, each lasting approximately 60 min. In previous research, two to four focus groups has been shown to reliably produce data saturation [9].

One main facilitator conducted each focus group (TB for one, and LL for the other), with the other research member acting as an observer in the focus groups. The observer focused on taking field notes and observing participant engagement. Focus groups were conducted virtually on Zoom, a videoconferencing technology, as COVID-related work precautions precluded the ability to conduct focus groups in person. Each focus group began with a brief introduction and was structured to spend 15–20 min on each of the three questions included in the survey. The facilitators avoided significant questioning of the group and interjected only as needed to keep the discussion going so as to avoid introducing bias into the group discussion. All focus group discussion was video recorded.

Since the survey and focus groups were designed not to collect identifiable information and the intent of the study was related to institutional quality improvement, the study was exempted by the institutional review board as a quality improvement project (HSC ID: 2,000,022,645). Participation in the survey and focus groups was voluntary and informed consent was provided at the beginning of each group. The investigation was carried out in accordance with the latest version of the Declaration of Helsinki.

Data Analysis

For the survey, responses were extracted verbatim from Qualtrics for thematic coding and analysis. One author (LL) conducted an initial review of the responses and identified eleven recurrent themes. These themes were: operations, oneness/one team, social distancing, telehealth, human resources, PPE, testing, patient care, technology, communication, and other. Next, the responses from each of the three questions were coded into the eleven themes by two authors (LL and LD). Both authors independently coded the responses and then discussed their coding to generate concordance. When responses pertained to multiple themes, they were coded into more than one theme. Frequency of themes was analyzed to understand their relative importance to staff and bar graphs of these frequencies were generated (Figs. 1a, b, and c). The three themes with the most responses were then qualitatively described.

Fig. 1

a Staff responses to what has been going well, b Staff responses to what needs further improvement, c Staff responses to new ideas or processes

With regard to the focus groups, the facilitators (TB and LL) organized field notes into main themes and subthemes. Field notes and participant engagement were reviewed by both researchers and discussed immediately following the focus groups to identify main themes and subthemes that emerged from the focus groups. Variations and discrepancies on the main themes and subthemes were discussed until concordance could be reached. One author (TB) reviewed the recordings of the focus groups and used verbatim quotes that represented the themes.

Results

The sample of the survey included 99 individuals who responded to the first question, 77 who responded to the second, and 78 who responded to the third. As shown in Fig. 1, these individuals generated 125, 90, and 97 responses to the three questions, respectively. The most common themes that respondents felt were going well (Fig. 1a) pertained to telehealth (28 responses, 22.4%), operations (19, 15.2%), communication (19, 15.2%), and oneness (17, 13.6%). In regards to what needed further improvement (Fig. 1b), similar themes emerged, including operations (23, 25.6%), patient care (14, 15.6%), telehealth (12, 13.3%), and communication (12, 13.3%). Similarly, the innovative ideas provided by staff (Fig. 1c) most commonly addressed telehealth (24, 24.7%), operations (22, 22.7%), and technology (14, 14.4%). The distribution of these responses across the three questions demonstrates three particularly prominent themes: (1) operations, (2) telehealth and technology, and (3) communications. The survey responses from these themes are described in greater detail below and supplemented by qualitative focus group data. The main themes and subthemes are described in Table 1.

Table 1

Main Themes and Subthemes of the Survey and Focus Groups

Themes	Supporting Subthemes	Key Quotes
Theme 1: Telehealth and teleconferencing provides benefits to both staff and patients but some patients are not served well by it	• Staff believe telehealth results in improved access and flexibility for patients • Staff report improved attendance and efficiency of staff meetings • Time saved by using telehealth and teleconferencing results in more time for personal and professional development • Some managers feel the switch to telehealth and teleconferencing has created an attitude change around work and accountability • System-wide practices regarding remote working and Zoom etiquette may relieve stress on individual managers • Patients with serious mental illness are harder to reach and engage using telehealth	“Telehealth services have been going better than we could have ever imagined.” “The fact that I can save 40 min of travel time by doing a zoom meeting now gives me more time and energy that I can put into a clinical need, whether that be a patient need or a project and these are things that will benefit the larger system in the long run.” “I think there is a new mindset about accountability and working. Prior to the pandemic, we were scheduled to come to work and we came to work. Now the mindset has changed now that teleworking is an option. People are pushing back saying why can’t I telework today? That was created significant challenges for me administratively.” “There are people we are not able to reach. I work with a lot of people with chronic mental health issues who don’t have the capacity to engage in the same way as other people. I worry about the number of people that aren’t getting the support they need right now.”
Theme 2: While current technology is not perceived as adequate to fully satisfy patient care needs, staff are finding new clinical applications for technology	• Poor quality of technology has made it difficult to provide the same level of patient care and to collaborate with team members • Staff feel they need more training and support on how to effectively use technology • Limitations of technology pose significant challenges for trainees • Technology improves aspects of patient care were unexpected (i.e. family meetings, innovations in group therapy)	“We need an over-arching plan for technology going forward. I still feel like we are flying by the seat of our pants with technology.” “I haven’t figured out how to incorporate trainees in a very effective way using technology.” “The use of technology has been really helpful and in ways that I didn’t expect. I have had several families who are divorced and who refuse to come to a family meeting and sit in a room together but these same parents are now able to get on a zoom call together and sit in a family meeting.” “My patients are telling me they are going to Alcoholics Anonymous meetings all over the world now”
Theme 3: The pandemic has created a renewed sense of teamwork and collaboration that is sometimes reduced by the loss of human contact	• Interdepartmental and interprofessional collaboration has improved • Weekly departmental town halls have improved sense of unity and oneness • A shared vision on a small number of goals as opposed to a large number of strategic initiatives has improved staff buy-in • Reliance on telehealth communication and loss of human contact has reduced staff sense of cohesion	“I have never experienced this level of teamwork and cohesion before. Not just within our own teams and within the department of psychiatry but also collaboration with other departments.” “We all had one goal and we were focused on it. We had to figure out how to provide patient care first and foremost in a challenging environment.” “It’s just not the same in terms of the sense of cohesiveness among staff. I appreciate the ability to telework and come together from different locations but the collegiality, ability to laugh, and have fun, spontaneous interactions with colleagues is missing from the work experience.”
Theme 4: Staff experience heightened stress and fatigue driven by uncertainty and fear, which can be mitigated in part by clear communication and transparency from administrators	• The pandemic has created new demands for staff at home and at work, which has led to anxiety and fatigue • Regular feedback from staff and adequate time for clinical implementation and debriefing can help to reduce anxiety • Contingency planning and transparent communication can help reduce staff anxiety and uncertainty • Frontline staff should have time to attend focus groups, as direct clinical responsibilities may impact participation	“There is a lot of anxiety in general. If we had more of a game plan going into the next phase of COVID and if we knew that next time, we are not going to wing it, I would feel less anxious” “I think fatigue is what I am seeing with a lot of colleagues. It is really exhausting to have to adjust things on a day-to-day, moment-to-moment basis. I don’t know how to make it better or solve it but I recognize it as something that folks across the board are struggling with.” “The feedback that I have been getting from other staff is that they like being asked their opinion and like being able to give feedback.”

With regard to the focus groups, 25 individuals signed up to participate and 17 (9 in the first group and 8 in the second) were able to attend, representing a participation rate of 68%. We obtained theoretical saturation with two focus groups, defined as having enough data to illustrate the themes clearly. Demographic characteristics of the focus group participants are shown in Table 2.

Table 2

Demographic characteristics of focus group participants

	Participants (n = 17)
	n	%
Female	16	94.1%
Race
Non-Hispanic White	12	70.6%
Black or African American	1	5.9%
Asian	2	11.8%
Hispanic	1	5.9%
Other	1	5.9%
Age range
30–39	3	17.6%
40–49	5	29.4%
50–59	5	29.4%
60–69	3	17.6%
Not given	1	5.9%
Work Setting
Inpatient	10	58.8%
Ambulatory/Intensive Outpatient Program	4	23.5%
Consult-Liaison	3	17.6%
Job Title
Licensed Independent Practitioner (APRN, MD)	6	35.3%
Social Worker	6	35.3%
Chaplain	1	5.9%
Administrator	2	11.8%
Clinical manager	1	5.9%
PT/OT	1	5.9%

Operations

The theme of operations included responses pertaining to clinical workflows, processes, and structures, and was the category that respondents most often addressed in the survey. In both the survey and focus groups, staff frequently commented that the pandemic created greater interdepartmental and interprofessional collaboration, with one staff member explaining, “I have never experienced this level of teamwork and cohesion before. Not just within our own teams and within the department of psychiatry but also collaboration with other departments.” At the same time, some staff members felt a loss of teamwork and collaboration which they primarily attributed to the loss of human contact. One staff member described “It’s just not the same in terms of the sense of cohesiveness among staff. I appreciate the ability to telework but the collegiality, ability to laugh, and have fun, spontaneous interactions with colleagues is missing from the work experience.”

In the survey and focus groups, staff commented on a number of specific aspects of operations that were going well, including the hospital incident command system, the standardization in COVID clinical workflows across sites, and the visitor restriction on inpatient units. They also identified several areas that needed improvement, including physical spaces that were not large enough to socially distance and having to attend too many meetings about operations. With regard to new ideas and future directions, staff frequently commented on the need to place more emphasis on emergency preparedness and contingency planning going forward and improve the efficiency of clinical workflow across sites, especially the process for admissions. Staff also described the need to expand the back-up pool of providers that could be called upon to fill in for staff who may be unable to work due to illness.

Telehealth & Technology

This theme primarily pertained to feedback on the use of Zoom videoconferencing and Epic MyChart (a patient portal technology used for telehealth delivery) for both patient care and for holding staff meetings/educational seminars. In both the survey and focus groups, staff largely felt positive about the expanded use of telehealth and teleconferencing. They frequently commented that telehealth improved access and reduced delays for patients and allowed for more scheduling flexibility for providers. Staff also felt that utilizing zoom for teleconferencing improved attendance and efficiency of staff meetings and educational seminars. Specifically, they noted that telehealth and teleconferencing reduced travel time for commuting between clinical sites that opened up more time for personal and professional development. One staff member observed, “The fact that I can save 40 min of travel time by doing a zoom meeting now gives me more time and energy that I can put into a clinical need, whether that be a patient need or a project and these are things that will benefit the larger system in the long run.” Many staff hoped that there would continue to be an option to work remotely while others recognized the need to establish system-wide practices on remote working and teleconferencing etiquette (i.e. cameras on/off, meeting engagement) to relieve stress on individual managers to implement these practices.

Staff also expressed concerns that many patients had trouble accessing and navigating the required technology for telehealth visits and were especially concerned that some patients with serious mental illness were difficult to reach and engage using telehealth. One staff member noted: “There are people we are not able to reach. I work with a lot of people with chronic mental health issues who don’t have the capacity to engage in the same way as other people. I worry about the number of people that aren’t getting the support they need right now.”

As related to technology, staff frequently commented that they needed more support on how to use technology effectively and often felt that internal deficiencies made it difficult to perform their job duties, including poor Wi-Fi and inadequate sound/video quality on available devices. In the focus groups, some staff expressed concerns about the quality of educational experiences that trainees were receiving and felt it was difficult to include them in patient care with telehealth technology. At the same time, staff also commented that they were finding new and unexpected applications for technology, especially for family involvement on inpatient units and ability to incorporate creative programming in group therapy.

Going forward, staff had a number of ideas for the innovative use of technology. They generally felt a hybrid model of telehealth services and in-person visits would be optimal, especially for patients who may need more intensive services. Staff highlighted the need to reimagine the deployment of technology. They expressed that patients in the outpatient setting needed greater access to technology to participate in telehealth but also called for inpatients to have more access to technology, such as community computers to allow for correspondence with outpatient providers, court-appointed attorneys, and housing authorities.

Communication

The feedback in this theme was more mixed and mostly pertained to internal communication about COVID-19. This communication primarily included weekly departmental town halls and daily emails that succinctly summarized COVID-19 updates throughout the service line. While some staff felt the communication was appropriate and timely, other staff commented that there was too much information and it was often duplicated. Some staff felt that communication regarding major changes was not appropriately communicated to staff. Responses also called for the continued use of town hall meetings to share internal changes and new ideas, meeting minutes for staff who could not attend, and continued use of Zoom for staff meetings and educational forums.

Unlike the survey responses, focus group participants commented more on the role that timely and transparent communication played in reducing staff anxiety and uncertainty during the pandemic. Staff described heightened anxiety and fatigue during this time that was often driven by uncertainty and fear. As one staff member described, “There is a lot of anxiety in general. If we had more of a game plan going into the next phase of COVID and if we knew that next time, we are not going to wing it, I would feel less anxious.” Participants also called for more opportunities for staff to provide regular feedback on clinical operations and time for staff debriefing. They emphasized the importance of including front line staff in these initiatives who may have direct clinical responsibilities that impact participation. Some staff also commented that psychiatric staff could have been called up more in medical settings to support the psychological and emotional needs of patients, staff, and families affected by COVID-19.

Other Innovative Ideas

Several innovative ideas emerged in staff responses that are important to highlight separately. Respondents felt that COVID-19 highlighted the importance of performing more small pilots of change that could foster continuous quality improvement after the pandemic. They commented that the changing needs of patients during this time brought greater awareness of the need to focus and develop more services to target the social determinants of health, including housing and substance use services. Similarly, respondents called for more training on cultural competence and anti-racism and the creation of metrics to track progress on these competencies.

Discussion

This paper uses mixed methods to characterize novel findings about staff perceptions of COVID-related changes in a psychiatric service line at a large academic medical center in the northeastern U.S. with high community spread of COVID-19. The mixed methods design was best suited to understand the complex and varied experiences of staff members during this time. While brief survey responses included a diversity of content that could be synthesized into brief themes, qualitative focus group data provided richer and more nuanced information regarding many of the initial themes generated by surveys. In addition to emphasizing many challenges posed by the pandemic, staff highlighted several important ways that disruptions have led to positive changes and innovations that they hoped could be maintained after the pandemic. To our knowledge, this is the first study that utilizes a quality improvement approach to identify and describe staff perceptions of changes to mental health care delivery during the COVID-19 pandemic.

Findings from our study highlight the complexities of implementing telehealth, communication, and operational strategies during the COVID response period. Specifically, staff frequently commented on the rapid transition to telehealth and teleconferencing and generally had positive perceptions of these changes, although focus group data helped reveal concerns that some staff had about the limitations of reaching and engaging a subset of patients with serious mental illness using this technology. Other publications have highlighted similar challenges posed by COVID-19 for engaging patients with serious mental illness who often may not have access to required technology or whose illness may not be as effectively treated by telehealth care [1, 10]. Kopelovich et al. (2020) highlight the importance of adopting a continuum of service delivery modes to more effectively reach these populations, including not only telehealth, but also community outreach, natural supports, digital interventions, and office-based visits [10].

One of the main concerns that focus group data helped illuminate was the heightened stress and fatigue felt by many frontline staff during this time. While there have been several studies demonstrating increased rates of depression, anxiety, and insomnia among healthcare workers during COVID, there has been less qualitative description of the factors driving these symptoms and possible interventions to reduce them [11-13]. Common factors cited by mental health staff in our study include feeling uncertain of what the future holds, fear of contracting the virus, perception of a lack of institutional preparedness and contingency planning, and lack of communication and transparency from hospital leadership. These factors would be difficult to measure and capture in standardized psychometric instruments. Similar factors have been found to be associated with adverse psychological outcomes among healthcare workers during prior emerging virus outbreaks, such as SARS and Ebola [14]. Staff in our study also identify several potential interventions that may reduce these symptoms, including creating more time for debriefing and reflection, asking staff for their input (including through use of focus groups and surveys), having contingency plans in place, and clear and transparent communication from leadership. Similarly, these interventions have also been found to decrease risk of adverse psychological outcomes in prior viral outbreaks [14]. Other important interventions that have been found to reduce these symptoms include allowing short breaks and adequate time off work, having supportive peers and family, providing positive feedback to staff, and providing effective training and preparation for outbreaks [14]. Mental health staff should be provided with interventions similar to frontline staff working in general medical or intensive care settings, as psychiatric settings have unique challenges and can lead to experiences of fatigue and anxiety, as demonstrated in this study.

Staff responses also led to new quality improvement ideas. These ideas highlight the need to address social determinants and anti-racism in psychiatric care delivery, which align with clinical practice findings related to COVID-19 reported outside of psychiatry [15-18]. These responses highlight growing concern among providers that the economic impact of the pandemic will continue to exacerbate existing health disparities. Additionally, staff point to novel ideas regarding the deployment of technology, both in inpatient and outpatient settings to improve the quality and accessibility of care although, as previously stated, there is concern that over-reliance on technology may exacerbate existing inequalities for some patients who may be difficult to engage with telehealth.

Other psychiatric institutions that are undergoing rapid care transformations in response to COVID-19 may find benefit from deploying a similar short survey and/or focus groups to examine staff perceptions. This type of feedback has been previously used to drive continuous quality improvement at the institutional level [8]. In addition to staff, the experience of service users should not be overlooked in this quality improvement process. A similar survey could be deployed for patients and their families. Finally, the sustainability of COVID-related changes should also be considered when planning for the post-pandemic period [3] and more rigorous quantitative and qualitative evaluation processes should be considered when planning for longer-term change.

Limitations

There are several limitations of the present study that are important to note. First, the responses generated here are in response to institution-specific changes and are not necessarily generalizable to other institutions. In addition, the relatively small sample size and lack of demographic information in the survey can introduce convenience bias into the findings. Moreover, the focus group sample may not have been representative of all staff of the psychiatric service line as there was a lack of gender and racial diversity as well as a lack of nurses, milieu counselors, and resident physicians. This could have introduced selection bias into the findings. This may suggest that greater efforts need to be made to engage frontline staff in future quality improvement processes and investigation into the barriers to participation. In addition to surveying the experiences of patients and their families, future studies could perform in-depth interviews to explore in detail the themes generated by this study. Finally, more work is needed to validate the brief questionnaire to make it more broadly applicable.