A Multi-Site Evaluation of A National Employee Wellness Initiative at the Department of Veterans Affairs.

Background: The Department of Veterans Affairs (VA) seeks to transform its health care delivery from disease-centered, episodic care to a holistic and patient-centered model known as the Whole Health System (WHS) of care. Employee engagement and buy-in are crucial to this cultural transformation. The VA aspires to provide employees with opportunities to experience whole health in their personal and professional lives through a national Employee Whole Health (EWH) program. Although there are national recommendations, different local facilities may have unique strategies and challenges as they implement this program. Objective: This study aimed to conduct a program evaluation of EWH at three different VA facilities across the United States in order to identify facilitators and barriers to the implementation of EWH.
Methods: The team used the RE-AIM framework to develop an interview guide to assess various domains of implementation. Quantitative data on whole health offerings at each site were gauged using a national employee education platform. Standardized employee-related metrics at each site were assessed using the annual, national VA employee survey.
Results: EWH has had variable implementation at the three sites. Sites noted main facilitators as employee interest as well as available skills and expertise for delivering complementary and integrative care to employees. Limited staffing for EWH and a lack of dedicated employee time were cited as barriers. The infrastructure to perform local program evaluations to demonstrate effectiveness and impact were missing.
Conclusion: Employee engagement in whole health activities has the potential to support the VA's mission to transform its health care delivery model. Currently, the use of EWH and its potential impact are difficult to discern based on available information. Local sites need guidance to conduct program evaluations and find creative solutions to enhance employee participation. A robust measurement system to demonstrate effectiveness is paramount to ensure the success of this initiative.

Introduction

As a response to the Comprehensive Addiction and Recovery Act (CARA) aimed at addressing the opioid epidemic, the Department of Veterans Affairs (VA) embarked upon transforming its health care delivery using the Whole Health System of Care (WHS) as part of its strategic goals for 2018–2024. WHS moves care from disease-centered and episodic to holistic and patient-centered, focusing on “what matters” to the Veteran by integrating mainstream, complementary and alternative medicine. Between 2016 and 2018, the VA funded 25 facilities to become design sites that focused on expanding their existing efforts to implement parts of the WHS. Using lessons learned from the design sites and several innovation projects, in 2018, 18 facilities, one from each regional Veteran’s Integrative Service Network, were designated as flagship sites to fully launch the WHS within the VA.

Employees are crucial to embracing and promoting the organizational and cultural change that the WHS intends to produce at the VA. Not only will employees need to learn skills to deliver this new model of care, but the WHS invites them to apply these principles in their personal and professional lives. Employee Whole Health (EWH) is a key component of the WHS dedicated to the health and wellness of employees, and aspires to become the basis for VA employee-related policies, services and programs pertaining to employee well-being. EWH’s mission is to “establish a collaborative culture of well-being and support employee resilience through integration of whole health into daily practices and workflow across the entire [VA] and continually evaluate value-based outcomes.”

Employee wellness has come to the forefront recently as many health care workers have suffered during the COVID-19 pandemic due to increased occupational stressors. There have been several reports describing high levels of occupational fatigue, post-traumatic stress disorder, anxiety, depression, and sleep disorders in health care workers since the pandemic.[8-11] Even prior to COVID-19, there was a plethora of evidence documenting burnout in health care workers, culminating in both the National Academies of Medicine (NAM) and the American Nurses Association (ANA) making public statements urging organizations to focus on employee wellness.[12,13] Employee wellness not only fosters a healthy work environment but is also essential for patient safety and quality of care.[14,15]

VA has tracked employee health and invested in wellness promotion even before the WHS.[16-20] Between 2010 and 2015, a national program established employee fitness centers at 86 medical facilities, with small grants supporting fitness equipment/classes, health monitoring kiosks, and sit-to-stand desks. There were also pilot programs in outpatient clinics with peer leaders championing physical activity during the workday. Such efforts showed a decrease in physical inactivity from 25% to 16% during the study period. Early outcomes from WHS implementation at flagships suggest that employees who engage in any whole health activity, either as participants or as practitioners, report greater motivation and job satisfaction, and less burnout and less intention to leave the organization.

Although the WHS and EWH are enterprise-wide initiatives, each facility may have unique implementation strategies and challenges. Our study evaluated EWH at three different VA facilities, described resources offered and utilized which support employee wellness and resilience, and identified facilitators and barriers to implementation. We aimed to identify which parts of the EWH mission that each facility embodies, and which areas need further development. These findings may serve to inform sites initiating employee wellness programs as well as future quality improvement (QI) projects in sites with established EWH programs.

Methods

The EWH program evaluation was conducted across three VA facilities that were in different phases of participation in the national EWH program. None of the sites had conducted a formal evaluation at the time of our study.

Site A is located in the Southeastern United States and consists of a 211-bed level 1A medical center and 42-bed long-term care facility, along with 20 outpatient clinics serving over 101000 Veterans. It employs nearly 5000 staff, 150 volunteers, and 250 trainees across 51 training programs. Site B is located in Northern California and consists of a 112-bed level 1A facility and a 120-bed long-term care facility, with six outpatient clinics. It employs approximately 3200 staff members, 404 volunteers, and over 700 trainees across 36 clinical training programs. Site C is located in the Mid-South United States and consists of two medical centers, including a 238-bed level 1A facility and a 347-bed level 1B facility which primarily provides long-term care, along with 12 outpatient clinics serving over 150000 Veterans. It employs approximately 4450 staff members and has two academic affiliations. All three sites provide inpatient, primary care, mental health, specialty, surgical, short- and long-term rehabilitation services.

The evaluation team was led by three interprofessional fellows in the National VA Quality Scholars (VAQS) Fellowship Program in collaboration with a VA Health Professions Education Evaluation and Research (HPEER) Fellow and VAQS program faculty. Over the course of 10 months, the team collaborated to review the background, current status, and goals of the local and national WHS, focusing on EWH components. Team members met with local stakeholders and with the national EWH leadership to define the need and gaps in evaluating program implementation and outcomes.

Key dimensions of the RE-AIM framework provided the basis for a comprehensive evaluation of program components. The evaluation team opted for a qualitative approach using standardized interview questions. Table 1 displays the focus of questions within each domain. The team conducted semi-structured interviews virtually with individuals involved with coordinating EWH programs at each site. They summarized interview responses on the interview guide at a local level and then transcribed them into a cross-site table using RE-AIM domains. The complete interview guide can be found in the Online Supplemental Material. The program evaluation proposal was reviewed according to each local system requirements and determinations concluded that the project did not meet criteria for human subjects research.

Table 1.

Employee Whole Health evaluation questions based on RE-AIM framework.

Components of RE-AIM—Definitions 22	Focus of Questions
Reach—the absolute number, proportion, and representativeness of individuals who are willing to participate in each initiative, intervention, or program, and reasons why or why not	Variety and characteristics of programs offered including topics and format
	Decision making approach including offerings, marketing, target audiences
Effectiveness—The impact of an intervention on important individual outcomes and variability across subgroups	Evaluation methods, solicitation, monitoring, and dissemination
Adoption—The absolute number, proportion, and representativeness of settings and participants	Characteristics of the participants/non-participants including those implementing the programs
	Characteristics of settings, including services used, quality of resources, champions
Implementation—The intervention agents’ fidelity to the various elements of an intervention’s key functions or components, including consistency of delivery as intended and the time and cost of the intervention	Barriers and facilitators to implementation of the programs, including insights from switching to virtual programs
	Accountability for implementation quality and effectiveness
Maintenance—The extent to which a program or policy becomes part of routine individual or organizational practices and policies	Individual level maintenance including participation in EWH and external programs and long-term effects/practices
	Setting level maintenance including sustainment or revision of programs offered, changes to organizational structures, and impact of COVID

In addition to qualitative interviews, the team gathered quantitative data from the VA’s annual All Employee Survey (AES) at all three sites regarding three metrics often linked to employee wellbeing: the Employee Engagement Index, Job Satisfaction Score, and Workplace Psychological Safety Score. Furthermore, employee education and training material related to the WHS is partly captured in the VA’s national Talent Management System (TMS). With the assistance of the national Employee Education System staff, the team queried TMS for offerings related to whole health at each of the three sites and obtained a count of the number of employees who participated in each offering. Using contextual analysis qualitative coding techniques, TMS course offerings were coded into two main categories: 1) training related to delivering whole health care and 2) employee whole health offerings on self-care and wellness topics based on the Circle of Health. Training on whole health delivery was further divided into the following sub-categories:

• Foundational education courses (i.e., Whole Health 102, Whole Health Principles etc.);

• Peer-to-Peer learning opportunities (i.e., Communities of Practice, Peer-to-Peer conferences);

• Whole health teaching and facilitation skills (i.e., Whole health facilitation, health coaching);

• Procedural skills (i.e., battlefield acupuncture, massage therapy);

• Integration/Implementation (i.e., Whole Health in Primary Care, Whole Health in Your Practice)

Results

Site Characteristics

Site A

Site A was designated as a flagship site in 2018, but had efforts in integrative health since 2012 such as a monthly interdisciplinary employee interest group and 2 half-days of integrative clinic within the Women’s Clinic. It also secured additional funding through grants for smaller projects like working with the homeless program staff. Whole Health was initially aligned under the primary care service line, but is currently its own department under the Chief of Staff office. There are 14 full-time equivalent (FTE) staff including the director, program manager, administrative staff, whole health peer coaches, and complementary/integrative practitioners. Previously, there was a .2 FTE EWH coordinator. However, since 2020, there has not been a dedicated EWH coordinator nor is there currently funding for this position.

Site B

Site B did not have a formal designation, but since 2019 has had a small group of motivated volunteers who were part of a new Integrative Health Service. This group of volunteers holds a monthly Employee Wellness Coalition meeting with interprofessional staff and providers, and reports to the Whole Health Steering Committee while also providing periodic activity reports to the Executive Leadership Team. This group has a designated Whole Health Coordinator who has volunteered extra duty time to these activities, but does not have dedicated FTE. There are no other dedicated FTE for Employee Whole Health. As of 2021, the Whole Health Coordinator position is vacant.

Site C

Site C was also designated a flagship site. While starting out with minimal staffing, the program has developed into a robust department with 65.5 FTE for fiscal year 2021. The site divides the staffing based on a “hub and spoke” model with the hub sites at four locations which support smaller outpatient clinics. At each site dedicated Whole Health personnel function at full capacity. Staffing includes section chief with administrative support, education champions, chiropractors, acupuncturists, physical therapists, Whole Health peer coaches, social workers, clerical staff, and program management staff. Despite the large staff, there is not a designated EWH coordinator, and work in this realm is completed as collateral duty.

Reach

EWH programs are open to all employees at the three sites. Each site offers synchronous experiential offerings such as yoga, tai chi, and meditation/mindfulness sessions. Site A and C have also held synchronous educational seminars, health promotion events such as health fairs and self-paced asynchronous modules. Site C created a unique Video-on-Demand web-based platform called Whole Health University which is accessible to both Veterans and employees. This platform is also used at 11 other VA facilities and the VA central office. As of April 2021, there were 826 active accounts and of these, 430 are Site C users. Approximately 30% of the accounts are registered to employees.

Email is the predominant form of advertising EWH offerings at all sites. Site B and C also highlight EWH as part of their new employee orientation. Experiential offerings at Site A and B are based on availability of staff with expertise who can lead activities. All sites have shifted most EWH activities to virtual platforms since the COVID-19 pandemic began. Site A has noted an increase in participation; pre-pandemic face-to-face activities generally had less than 10 participants while currently virtual activities generally have over 50 participants. Site C noted that shifting their pre-pandemic Employee Yoga to a virtual format met challenges due to logistical constraints and lack of engagement.

Effectiveness

Tracking of utilization of services is inconsistent at all three sites. At Site A, one of the WH staff members keeps a spreadsheet of the count of employees at different EWH offerings, but this is reliant on the employee leading the session capturing the information. At Site B, only one offering, Mindfulness-Based Stress Reduction Class, performs a pre- and post-group survey. Although Site C has a digital platform, currently there is not a mechanism to track utilization. There are plans to capture the number of enrollees and the number of hits/views per month for each session. Site C solicits EWH offering feedback using an electronic survey, but the results had not been tabulated nor analyzed at the time of this writing. All sites noted that a lack of standard measures and inconsistent audit and feedback with employee participants have thwarted displaying the effectiveness of these activities.

Adoption

Sites A and B do not currently assess the demographics or other characteristics of employees who partake or do not partake in EWH. In order to collect such information, permission from unions and other entities may be required. The virtual environment has made such an assessment more difficult. The digital platform at Site C offers the possibility to track demographics such as age, gender at birth, city of residence, service line, military branch (if served), weight, height, waist circumference, and basic questions about general health, exercise, satisfaction with life, pain, fatigue, and basic mood/mental health status. However, this data has yet to be analyzed at the time of the interview.

“It’s hard to figure out demographics of participants in the virtual setting, but when we had in person activities, most of the participants were nurses from different areas.”—WH program director

The virtual environment has allowed participants from different locations far from the medical center to participate in activities they previously could not access. In general, utilization remains higher at the larger medical centers compared to smaller outlying clinics.

At all sites, most EWH activities are mainly developed by Whole Health staff or volunteers with skills such as tai chi, yoga, or meditation. Site C has a designated Whole Health champion in each department, but time constraints are cited as a challenge to taking on collateral duties such as promoting EWH.

Implementation

At all sites, the lack of a dedicated EWH coordinator is a major barrier to EWH implementation, leading to sporadic offerings based on availability and ability of staff to coordinate. Site B noted that since most classes were led by volunteers, their workload had a major impact on class cancellation. To overcome this issue, Site C is encouraging trained whole health coaches to advocate the use of the Video-on-Demand platform and to commit to providing at least one weekly offering for both Veterans and employees. Similarly, Site A has allowed employees to partake in activities initially designed for Veterans such as tai chi class as space allows.

Lack of protected time for employees to partake in these offerings remains another challenge to the uptake of EWH activities. Site A set up two weeklong EWH “mini-retreats” during the lunch hour, where 30 min offerings were repeated twice to allow more employees to participate. Site C is hoping to pilot weekly 1 hour dedicated time for employees to partake in EWH activities.

“Most of what we are offering is within work hours so as we grow our staff, we hope to expand the times of these services.”—WH program director

Maintenance

None of the sites have rolling data collection to see whether participants of EWH sessions are coming to one-off meetings, or have ongoing participation. The long-term effects on employees participating in EWH have not been collected at a site-specific level. However, the national Whole Health program has incorporated questions regarding whole health into the All Employee Survey to address organizational benefits.

The three sites are still in the early phases of developing an EWH presence in the organizational infrastructure. Although Site A is a flagship, it notes that WHS and EWH are not yet institutionalized within the facility, mainly due to lack of stable executive leadership for the past 2–3 years.

“The flux [in executive leadership] is part of the reason why the growth of our flagship was stymied...We are slim in terms of being able to offer more for employees without someone with dedicated time. [The Education Champion] is supposed to be educating employees about Whole Health in general, but [having] an EWH Coordinator, that's a much bigger scope...but there's not bandwidth to do [more than we are currently doing].”—WH program director

All Employee Survey Data

Sites A and B noted a rise in employee engagement, job satisfaction, and workplace psychological safety during the WHS implementation period (Table 2). Site C’s metrics remained stable during this time.

Table 2.

Employee Related Outcomes by Site (FY 2020 data from AES).

	Site A		Site B		Site C
	FY 2019	FY 2020	FY 2019	FY 2020	FY 2019	FY 2020
Employee engagement index (scale 1–100)	62.7	66.8	74.6	78.5	69.7	68.8
Job satisfaction (scale 0–5)	3.79	3.87	4.14	4.23	4.09	4.03
Workplace psychological safety (scale 0–5)	3.59	3.64	3.88	3.96	3.71	3.68

The VA All Employee Survey is an annual, voluntary and confidential survey sent to all VA employees which is to be used as a tool for employees to provide feedback to management and leadership. The data helps the VA identify strengths and areas of opportunity and guides data-driven decisions at workgroup levels to improve workplace culture. Higher scores are better for all three metrics.

Whole Health Talent Management System Data

Employees from the three sites participated in 470 unique Whole Health offerings. The codification of this TMS data is displayed in Table 3. Site C reported the most course offerings, while site B reported the fewest. At all sites, the majority of courses were related to training in delivering care using the WHS rather than employee whole health. In the training category, skill building for delivering whole health (i.e., facilitation, coaching, train-the-trainer, and non-specific motivational interviewing courses) was the most populated course offering, with 2428 participants. Within EWH offerings, personal development (n = 224) and power of the mind (n = 135) offerings had the most participants.

Table 3.

Employee participation in Whole Health Offerings in TMS by site, count (% of total employees).

Category	Subcategory	Site A	Site B	Site C
Whole health training	Education	1102 (.22)	125 (.039)	234 (.053)
	Peer-to-Peer	73 (.01)	17 (.003)	52 (.012)
	Facilitation and coaching	830 (.16)	498 (.01)	1100 (.25)
	Procedural skills	57 (.01)	46 (.014)	183 (.041)
	Integration/Implementation	427 (.09)	174 (.054)	1379 (.31)
Employee whole health (self-care)	Food and drink	11 (.002)	28 (.009)	55 (.012)
	Personal development	116 (.02)	29 (.009)	84 (.02)
	Power of the mind	20 (.004)	68 (.021)	47 (.011)
	Recharge	5 (.001)	0 (0)	84 (.019)
	Relationships	6 (.001)	28 (.009)	1 (0)
	Spirit and soul	2 (.0004)	0 (0)	0 (0)
	Working your body	3 (.0006)	3 (.001)	8 (.002)

Discussion

EWH at the VA provides opportunities to realize positive personal and professional resiliency and wellbeing, as well as increase peer-to-peer, managerial, facility, and national support for VA employees and Veterans. To our knowledge, this is the first multi-site program evaluation of EWH to identify the facilitators and barriers to accomplishing its mission to incorporate the WHS into employee workflow and daily lives, and continually evaluate outcomes.

The RE-AIM framework offers a lens for assessing barriers and facilitators in each domain. Reach considers the population intended to benefit from the program, which program elements are offered and how program elements are advertised and decided. VA’s national health system can support EWH reach to its intended employee population as it already has the technology and infrastructure exist to offer synchronous and asynchronous activities which can be hosted by one facility but can potentially be attended by employees across the enterprise. The data in TMS shows that employees at all sites use the national platform to engage in EWH. A large number of trainees interact with the VA, which increases the potential reach of EWH to health care staff not formally designated as VA staff. At the time of this writing, employee feedback is not a major factor in decisions surrounding what to offer. Offerings are based on the expertise and availability of staff or volunteers.

Effectiveness describes the impact of EWH on individual and organizational outcomes, including potential negative effects. EWH’s mission focuses on improving wellbeing, resilience, and integrating whole health principles into employee daily workflow. None of the sites had conducted end-user assessments formally to comment on positive or negative outcomes. The sites did not have a robust evaluation strategy to collect utilization data, solicit employee feedback, and demonstrate impact. The National EWH program has since developed an Evaluation Toolkit which has several tools to assess individual wellbeing based on the Circle of Health and gauge organizational culture. However, in order for these tools to be effectively deployed, we recommend a standardized infrastructure to not only collect this data but analyst support to evaluate and adjust EWH activities based on employee feedback.

Prior published evidence found that employees who had greater knowledge or involvement in delivering whole health services to Veterans also reported higher rates of identifying their facility as the best place to work, lower turnover, lower burnout, and greater motivation in their jobs. Our study shows that during the implementation stage of the WHS, employee engagement, satisfaction, and workplace psychological safety scores on the AES improved at two sites and remained stable at the last site. While this data is promising, the AES results are not causal and are not specific to employee involvement in EWH activities. The most recent iteration of the AES includes two additional questions which specify the impact on employee well-being based on employees personally using whole health practices. Analysis of this data can potentially explore the relationship between WHS, EWH, and AES employee-related metrics.

Adoption refers to where and who applied EWH. All sites studied have an EWH initiative and multiple stakeholders mentioned that employee interest is a facilitator of adoption at these sites. None of the sites had available data to determine the number or percent of employees and characteristics of participants and non-participants in EWH.

Implementation refers to the consistency of program delivery, its adaptation and cost. All the sites had current employees or volunteers with expertise in a variety of complementary and integrative practices which helps to reduce the cost of the intervention. The lack of a dedicated EWH coordinator or other staff assigned to provide EWH services was cited as a frequent barrier for the lack of consistent activities at each site. Even in sites with a large number of employees, most EWH activities were coordinated by volunteers or whole health staff as collateral duties. Nationally, the Whole Health program has developed an implementation guide with recommendations for staffing in EWH.

EWH implementation is also impaired due to employee time constraints and competing work demands. Currently, the study sites could not accommodate synchronous activities outside of business hours, reducing opportunities in certain groups. Nationally, the EWH program is supporting pilot sites where employees get one hour protected time weekly to engage in EWH. Protected time may advance EWH implementation, but further analysis is needed to assess feasibility and sustainability.

The final domain, maintenance, refers to the extent to which EWH has become institutionalized at the local site as well as sustained at an individual level. The evolving presence of questions regarding the WHS and particularly EWH in the AES highlights this initiative as a national priority. Nevertheless, local sites face challenges in incorporating EWH into their facilities. One site discussed the lack of stable executive leadership as a hindrance in making EWH an institutional priority. Sites also mentioned that voluntary positions and staff turnover hamper the creation of committees or workgroups that can support long-term EWH goals. Furthermore, individual level data did not exist at any of the sites to comment on whether participation in EWH activities has led to sustained improvement in personal health behaviors or self-care.

Limitations

There are several factors that hindered the ability to gather data for this project. The lack of a dedicated EWH coordinator at each study site may have limited our access to data that may be readily available at other sites with staffing. Our data using TMS to evaluate EWH offerings is an inaccurate representation of all the offerings at each site because every activity may not be documented in TMS. Nevertheless, it is the only available standardized system across all three sites. This data represents only three VA sites so our results cannot be generalized. Finally, this project and much of the implementation of EWH across these three sites took place during the COVID-19 pandemic, which caused delays in conducting the research and affected its findings.

Conclusion

This study evaluated the implementation of a national employee wellness initiative at the largest integrated health care system in the United States. While each of the three study sites displayed unique regional- and site-specific differences, overall health care employees were interested in offerings aimed at improving their resiliency and wellbeing. In addition, many staff members already possess the skills and expertise to impart key components of EWH to other employees. In order to truly demonstrate the value-based outcomes by engaging in EWH at an individual, facility and national level, a dedicated EWH coordinator at each facility along with infrastructure for ongoing feedback and evaluation with a common set of metrics are indispensable. EWH can enable employees to embrace and espouse the transformational change that the WHS aims to generate at the VA. Investing both locally and nationally in EWH may improve employee outcomes and create a positive impact on Veteran care.

Click here for additional data file.

Supplemental Material, sj-pdf-1-gam-10.1177_21649561211053805 for A Multi-Site Evaluation of A National Employee Wellness Initiative at the Department of Veterans Affairs by Freny Shah, Joanna Sells, Jennifer Werthman, Corrine Abraham, Asma M. Ali and Carol Callaway-Lane in Global Advances in Health and Medicine