Psychological Flexibility of Nurses in a Cancer Hospital: Preliminary Validation of a Chinese Version of the Work-related Acceptance and Action Questionnaire.

OBJECTIVE: To translate the English work-related acceptance and action questionnaire (WAAQ), make cross-cultural adaptations, and examine its psychometric properties when used by Chinese oncology nurses.
METHODS: After translation, the psychometric properties of the Chinese WAAQ were analyzed among 417 nurses, and content validity was determined by six experts.
RESULTS: Item-level content validity index (CVI) values were between 0.83 and 1.00; scale-level CVI/universal agreement (S-CVI/UA) and S-CVI/average were 0.86 and 0.98, respectively, which implicated a good content validity. The correlation of the Chinese WAAQ with AAQ-II (rs = -0.247, P < 0.001) suggested criterion validity, and those with General Health Questionnaire-12 (-0.250, <0.001) and general self-efficacy scale (0.491, <0.001) and Utrecht work engagement scale (UWES) (0.439, <0.001) suggested convergent validity. Exploratory factor analysis identified a seven-item, one-factor structure of WAAQ. The Chinese version of WAAQ had high internal consistency (Cronbach's α = 0.920), with an item-total correlation coefficient of 0.702-0.828 (P < 0.05), split-half reliability of 0.933, and test-retest reliability of 0.772.
CONCLUSIONS: The Chinese WAAQ is a reliable and valid tool for assessing psychological flexibility in Chinese oncology nurses.

Introduction

Stress-related burnout among nursing professionals has been found to be higher than other health-care professionals owing to the nature of nursing work.[1] There is a significantly positive correlation between burnout and psychological symptoms (depression, anxiety, and stress) in nurses.[23] In nurses working in Norway, the prevalence of anxiety and depression has been reported to be 19.2% and 8.4%, respectively,[4] which is higher than that of the adults in Nepal (16.1% and 4.2%).[5] A high incidence of anxiety, depression, and stress has been reported in nurses working in China and Hong Kong.[67] Furthermore, the prevalence of “burnout syndrome” is significantly higher among oncology nurses than among surgical nurses.[8] Furthermore, oncology nurses have high levels of emotional exhaustion, and a large proportion of them are at risk of developing burnout.[910] However, burnout and mental health of nurses are strongly associated with the quality of patient care.[111213]

Noben et al. conducted a cost–benefit analysis of mental health programs for hospital nurses. They concluded that offering preventive intervention represents a favorable business case as seen from the employer's perspective.[14] A stress-reduced program based on mindfulness meditation was reported to improve stress, burnout, depression, and anxiety significantly.[15] Therefore, focus on nurses’ mental health, as well as reducing the prevalence of burnout of nursing staff, is valuable.

Acceptance and commitment therapy (ACT) is the “third wave” of behavioral and cognitive theory based on acceptance and mindfulness. ACT interventions aim at increasing “psychological flexibility,” which is the core concept of the ACT model of psychological health and behavioral effectiveness. Psychological flexibility is closely connected with individuals’ internal experiences, which could have great impact on psychological and physical function. With satisfying psychological flexibility, individuals could respond to negative thoughts, feelings, and events, thereby improve their well-being flexibly.[161718] ACT treatment involves six core processes (acceptance, cognitive defusion, self-as-context, being present, value, committed action), and acceptance, cognitive defusion, self-as-context, and being present belong to the acceptance and mindfulness process. ACT is becoming an increasingly popular approach that has been applied widely to treat anxiety,[19] depression,[20] diabetes mellitus,[21] chronic pain,[22] cancer,[23] and obesity.[24] The process goal of ACT in those treatments is to enhance the psychological flexibility of participants and finally to achieve better health status. Psychological flexibility was measured by the Acceptance and Action Questionnaire (AAQ)[25] and its revised version (AAQ-II).[26] Then, measures of psychological flexibility in specific thoughts were developed, such as the Chronic Pain Acceptance Questionnaire (CPAQ-20 and CPAQ-8).[27]

Bond et al. found psychological flexibility to be correlated with work-related outcomes, including mental health, better job performance, increased capacity to learn skills at work, and a lower prevalence of absence from work.[28] In 2013, Bond et al. developed the work-related AAQ (WAAQ), and then validated its good reliability and validity (mean coefficient = 0.83).[29] The WAAQ has also been validated for employees in Spanish, and good psychometric properties have been reported.[30] A validated Chinese version of WAAQ is needed to measure psychological flexibility as an assessment in mental health and job performance in workplaces among Chinese individuals.

We wished to examine the psychometric properties (content validity, criterion validity, convergent validity, exploratory factor analysis (EFA), internal consistency, split-half reliability, and test-retest reliability), as well as the performance of a Chinese version of WAAQ among a sample of Chinese nurses in a cancer hospital. The present study could provide a good tool to evaluate the psychological flexibility of staff in Chinese workplaces. Nevertheless, further research is needed to confirm its psychometric properties and performance among a more diverse population.

The aim of this study was to translate the English WAAQ, make cross-cultural adaptations, and measure the psychological flexibility of oncology nurses in workplace objectively, which will attract the nursing administrators’ attention to psychological status of oncology nurses, as well as offer preventive intervention reference to enhance their mental health in the future.

Methods

Sample

Six specialists in ACT were invited to assess the content validity of the WAAQ. A convenience sample of 417 nurses from a cancer tertiary hospital completed the questionnaire. Data from the WAAQ were used to assess criterion validity, convergent validity, internal consistency, and split-half reliability. Subsequently, the data were used to undertake EFA. Finally, 244 of these nurses completed second assessments at 2–4-week intervals to obtain test-retest reliability. The demographic data of 417 nurses are shown in Table 1.

Table 1

Sociodemographic characteristics of the study cohort (n=417)

Variable	Nurses, n (%)
Age (years)
18-30	261 (62.6)
31-40	98 (23.5)
41-50	49 (11.8)
>51	9 (2.2)
Education level
Technical secondary school	2 (0.5)
Junior college	26 (6.2)
Bachelor degree	370 (88.7)
Master's degree	16 (3.8)
Doctorate	3 (0.7)
Marital status
Unmarried	140 (33.6)
Married	268 (64.3)
Divorced	7 (1.7)
Widowed	2 (0.5)
Average monthly earnings (Yuan)
<3000	9 (2.2)
3000-5000	140 (33.6)
5000-8000	236 (56.6)
>8000	32 (7.7)
Job title
Nurse	61 (14.6)
Nurse practitioner	244 (58.5)
Supervisor nurse	63 (15.1)
Associate professor of nursing	46 (11.0)
Professor of nursing	3 (0.7)

Procedure

With the permission of the developers of the WAAQ, the English version of the WAAQ[29] (Appendix 1) was translated into Chinese Appendix 2 according to the Sousa's guideline[31] which involved six stages: forward translation, comparison, blind back-translation, comparison, pilot testing, and full psychometric testing. The first five stages were completed by cooperation between two psychology experts, two ACT experts, two highly qualified nursing specialists, and three bilingual native Chinese postgraduate students majoring in English.

Appendix 1

English version of Work-related Acceptance and Action Questionnaire

Items	1 (Never true)	2 (Very seldom true)	3 (Seldom true)	4 (Sometimes true)	5 (Frequently true)	6 (Almost always true)	7 (Always true)
I am able to work effectively in spite of any personal worries that I have	1	2	3	4	5	6	7
I can admit to my mistakes at work and still be successful	1	2	3	4	5	6	7
I can still work very effectively, even if I am nervous about something	1	2	3	4	5	6	7
Worries do not get in the way of my success	1	2	3	4	5	6	7
I can perform as required, no matter how I feel	1	2	3	4	5	6	7
I can work effectively, even when I doubt myself	1	2	3	4	5	6	7
My thoughts and feelings do not get in the way of my work	1	2	3	4	5	6	7

Appendix 2

Chinese version of Workrelated Acceptance and Action Questionnaire

条目	从未	很少	偶尔	有时	经常	几乎总是	总是
不管我是否有个人方面的烦恼，我都能有效地工作	1	2	3	4	5	6	7
我能承认工作上的失误，并且依然取得成功	1	2	3	4	5	6	7
即便因某事紧张不安，我仍然能高效工作	1	2	3	4	5	6	7
忧虑不会妨碍我取得成功	1	2	3	4	5	6	7
无论我有怎样的感受，都能按照要求完成工作	1	2	3	4	5	6	7
即使自我怀疑时，我也能有效工作	1	2	3	4	5	6	7
我的想法和感受不会干扰我的工作	1	2	3	4	5	6	7

First, two of the postgraduate students translated the English version into Chinese independently and submitted it to the psychology experts and ACT experts for examining and comparing the two versions to achieve a single version by consensus. Second, another postgraduate student finished back-translation and then compared it with the original English version to ensure conceptual equivalence.

Subsequently, the prefinal Chinese version of the WAAQ was pilot tested by a panel of experts (two psychology experts, two ACT experts, and two nursing specialists). All members of the panel were bilingual who evaluated the degree of equivalence between the English version and Chinese version. According to expert criteria, no item needed to be modified to facilitate comprehension by the participants. And then, the translated scale was pilot-tested twice with 30 nurses of different educational levels to check their understanding of the scale items and response options. Some amendments were made based on the feedback received. The modified scale was tested again with the same group of nurses: misunderstandings were not found. Finally, examine the initial full psychometric properties of the Chinese version WAAQ with a sample of the target population of interest.

Measures

Six instruments were used for data collection: the WAAQ (Chinese version), AAQ-II, General Health Questionnaire-12 (GHQ-12), general self-efficacy scale (GSES), Utrecht work engagement scale (UWES), and a general questionnaire. The latter was used to collect personal sociodemographic data such as age, education, and marital status. The AAQ-II was used to test criterion validity, and the GHQ-12, UWES, and GSES were used to test convergent validity.

Work-related Acceptance and Action Questionnaire

The WAAQ[29] is a seven-item scale that measures psychological flexibility in relation to the workplace. The items reflect the extent to which people can take goal-directed action in the presence of difficult internal experiences. The items are rated on a seven-point Likert-type scale ranging from 1 (“never true”) to 7 (“always true”). Higher scores indicate greater levels of work-related psychological flexibility. The WAAQ has shown a satisfactory one-factor structure, reliability (mean Cronbach's α coefficient = 0.83), as well as external, convergent, concurrent, and predictive validity. For instance, in comparison with the AAQ-II, the WAAQ correlates more strongly with work-specific variables.

Acceptance and Action Questionnaire-II

The AAQ-II[26] is a general measure of psychological inflexibility. It consists of seven items rated on a seven-point Likert-type scale ranging from 1 (never true) to 7 (always true). The items reflect an unwillingness to experience unwanted emotions and thoughts and the inability to be in the present moment and behave according to value-directed actions when experiencing psychological events that could undermine them. Higher scores indicate lower levels of psychological flexibility. Recent studies have shown that the AAQ-II has better psychometric properties and a clearer factor structure than the first AAQ version.[25] In the present study, we used the Chinese version translated by Cao et al., which has shown a one-factor solution (the cumulative variance contribution rate is 62.5%), good internal consistency (Cronbach's α coefficient = 0.88), retest reliability (0.80), and criterion validity with SDS and SAS (r = 0.56, 0.55, P < 0.01).[32]

General Health Questionnaire-12

The GHQ-12[33] is a short screening instrument used to measure general mental health or psychological distress. It consists of 12 items, each of which assesses the severity of a mental problem over the past few weeks using a four-point scale (from 0 to 3). The factor loadings of the scale ranged between 0.72 and 0.90. The three factors in the model modified to be strongly correlated with each other. The correlation between factor 1 (Anxiety and Depression) and factor 2 (Social Dysfunction) was 0.89, factor 2 and factor 3 (Loss of Confidence) was 0.83, and the factor 1 to factor 3 range from 0.80 to 0.90. The score is used to generate a total score ranging from 0 to 36, with higher scores indicating worse conditions. The Chinese version of GHQ-12 used in the present study has been validated.[34]

General self-efficacy scale

The GSES was developed initially by Sherer et al. in 1982,[35] which comprises 10 items rated by a Likert-4 scoring system from 1 (“absolutely incorrect”) to 4 (“totally correct”). The factor loadings ranged from 0.59–0.77, with internal consistency coefficient 0.86.[36] The Chinese version of GSES was administered to university freshmen in Hong Kong in 1995, and the internal consistency was 0.91, test-retest reliability is 0.71.[37]

Utrecht work engagement scale

Work engagement was evaluated with a shortened version of the UWES, the UWES-9,[38] which correlates highly with its original, longer counterparts. The UWES-9 has three dimensions, vigor (three items), dedication (three items), and absorption (three items), which are described as the three defining attributes of work engagement. All items are rated on a seven-point Likert scale ranging from 0 (“never”) to 6 (“always”), and the average score of each dimension is calculated. A higher score indicates a higher level of work engagement. The Chinese version of the UWES has been used for Chinese people of different occupations and has satisfactory reliability and validity.[3940] The Cronbach's α coefficient for vigor, dedication, and absorption subscales is 0.78, 0.84, and 0.80, respectively.

Data collection and analysis

Data were collected between May and November 2016. After creating the final Chinese version of the WAAQ, the six experts, based on their clinical expertise and experiences, assessed the content validity of the WAAQ. Then, a study-information package was distributed to the 417 nurses; this contained an information sheet, a consent form, and questionnaires. Criterion validity was assessed by investigating the correlation between the WAAQ and AAQ-II. Convergent validity was assessed by investigating the relationship between the WAAQ and the GSES, GHQ-12, and UWES. A total of 244 of these nurses completed the second assessments 2–4 weeks later to obtain the test-retest reliability. When the measured time period is not more than 2 times, the retest reliability is calculated using simple correlation coefficients.

SPSS version 18 (IBM, Armonk, NY, USA) was used to carry out data analyses. Descriptive statistics (mean and median values, frequencies, and percentages) was calculated to show the distribution of sociodemographic information.

Content validity was used to assess the relevance and comprehensiveness of the items. The item-level Content validity index (I-CVI) value can be computed for each item on a scale. The scale-level CVI (S-CVI) is comprised of S-CVI/UA (universal agreement) and S-CVI/Ave (average I-CVI across items).[41] The (I-CVI >0.78),[42] (S-CVI/UA > 0.8),[43] and scale-level CVI of the average calculation method (S-CVI/Ave >0.9)[44] suggest good content validity.

To explore the internal consistency of the WAAQ, corrected item-total correlations and Cronbach's α coefficient were computed, and a Cronbach's α coefficient >0.7 indicates acceptable internal consistency. An odd-even split was adopted to divide the items into two halves, in which four odd-numbered items form one half of the test and three even-numbered items form the other. Unequal length of Spearman–Brown coefficient was applied in estimating the split-half consistency of the test. The test-retest reliability was undertaken by calculating the correlation coefficient (r) between the WAAQ scores acquired at test and retest times, and r > 0.70 indicates good reliability.[45]

Spearman correlations were carried out to assess the relationship of WAAQ with AAQ-II, GHQ-12, GSES, and UWES. According to Cohen (1988),[46] interpreted correlation coefficients of 0.10, 0.30, and 0.50 can be classified as “small,” “medium/moderate,” and “large” effects, respectively.

EFA by principal component analysis was conducted. The principal component analysis was done using oblique rotation after the suitability of the data for the analyses were confirmed by a Kaiser–Meyer–Olkin Index of 0.6–1.0[47] and Bartlett's test of sphericity with statistical significance (P < 0.05).[48]

Results

Sample characteristics

A total of 417 nurses from a tertiary hospital were included in the analysis. The characteristics of nurse participants are shown in Table 1. The majority of the participants were aged between 18 and 40 years, yellow, nurse practitioner. The average proportion of single and married was 33.6% and 64.3%, respectively. The average monthly earnings were 3000–8000 RMB.

Content Validity of the Work-related Acceptance and Action Questionnaire

Six experts who specialized in ACT participated in the assessment of content validity. I-CVI and S-CVI were used to assess content validity. I-CVI values were between 0.83 and 1.00; S-CVI/UA and S-CVI/Ave were 0.86 and 0.98, respectively [Table 2]. These data suggested high relevance and good comprehensiveness of the items. The experts did not recommend removal of any items, thereby suggesting good acceptability of the items.

Table 2

Fictitious ratings on a seven-item scale by six experts: Items rated 3 or 4 on a four-point relevance scale

Item	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6	Number in agreements	I-CVI
1	X	X	X	X	X	X	6	1.00
2	X	X	X	X	X	X	6	1.00
3	X	X	X	X	X	X	6	1.00
4	X	X	X	X	--	X	5	0.83
5	X	X	X	X	X	X	6	1.00
6	X	X	X	X	X	X	6	1.00
7	X	X	X	X	X	X	6	1.00
Proportion relevant	1.00	1.00	1.00	1.00	0.86	1.00	S-CVI/UA=0.86 S-CVI/Ave=0.98

X: Items rated 3 or 4, items rated 1 or 2, I-CVI: Item-level content validity index, S-CVI/UA: Scale-level content validity index, universal agreement, S-CVI/Ave: Scale-level content validity index, average calculation method

Criterion validity and convergent validity of the Work-related Acceptance and Action Questionnaire

Table 3 shows the correlations between the scores on the WAAQ, AAQ-II, and other criteria. Consistent with previous studies,[2930] the WAAQ showed a small-to-moderate negative correlation with the AAQ-II (r = −0.247), a small negative correlation with the GHQ-12 (−0.250), and a medium correlation with the GSES and UWES (0.491 and 0.439, respectively).

Table 3

Correlations between the Work-related Acceptance and Action Questionnaire, Acceptance and Action Questionnaire-second edition and General Health Questionnaire-12, General Self-efficacy Scale, and Utrecht Work Engagement Scale

Measure		WAAQ	AAQ-II
AAQ-II		−0.247**	-
GHQ-12		−0.250**	0.656**
GSES		0.491**	−0.382**
UWES		0.439**	−0.297**

**Correlation was significant at the 0.01 level (two-tailed). WAAQ: Work-related Acceptance and Action Questionnaire, AAQ-II: Acceptance and Action Questionnaire-second edition, GHQ-12: General Health Questionnaire-12, GSES: General Self-efficacy Scale, UWES: Utrecht Work Engagement Scale

The AAQ-II revealed different patterns of correlations from that of the WAAQ [Table 3]. Specifically, the AAQ-II showed lower correlations than the WAAQ with work engagement as measured by the UWES and self-efficacy as measured by the GSES. However, the AAQ-II showed higher correlations than the WAAQ with psychological distress as measured by the GHQ-12.

Factor analysis of the Work-related Acceptance and Action Questionnaire scale

Data could be used to conduct a factor analysis according to the Kaiser–Meyer–Olkin Index (0.908) and Bartlett sphericity test (χ2 = 1951.46, P < 0.001). Table 4 shows the Chinese version of the WAAQ, which showed good communalities and loadings on the main factor ranging from 0.78 (item 1) to 0.88 (item 3). A clear one-factor solution was found according to the Kaiser criterion. This factor explained 67.86% of the variance in item scores.

Table 4

Factor loadings from principal axis factoring, scale mean and standard deviation, and alpha value

WAAQ items	Factor loading	Communalities
1	0.781	0.610
2	0.810	0.656
3	0.882	0.778
4	0.795	0.633
5	0.834	0.696
6	0.855	0.732
7	0.803	0.646
Percentage explained variance	67.86
Scale, mean (SD)	33.5 (8.84)
Cronbach's α coefficient for scale	0.920

WAAQ: Work-related Acceptance and Action Questionnaire, SD: Standard deviation

Reliability Analysis of the Work-related Acceptance and Action Questionnaire

Table 5 shows the mean score for each item and the whole scale, corrected item-total correlations, and Cronbach's α coefficient if an item was deleted. Mean scores ranged from 4.3 (standard deviation [SD] = 1.54) for item 4 to 5.3 (SD = 1.52) for item 5. The reliability of the WAAQ in terms of internal consistency was impressive (Cronbach's α coefficient = 0.920), and item-total correlations were between 0.786 and 0.878, suggesting that the items were sufficiently homogeneous. Furthermore, Cronbach's α coefficient was between 0.901 and 0.914 if each item was deleted, and the corrected item-total correlations were between 0.702 and 0.828, suggesting that no item should be deleted. The unequal length of Spearman–Brown coefficient was 0.933, which revealed a satisfied split-half reliability. The result of test-retest reliability showed appropriate stability of the instrument (r = 0.772).

Table 5

Mean scores (standard deviation) per item, corrected item-total correlations, and Cronbach's α coefficient if an item was deleted

Scale	Mean score (SD)	Corrected item-total correlation	Cronbach's α if an item is deleted
Item 1	5.1 (1.56)	0.702	0.914
Item 2	4.8 (1.51)	0.738	0.910
Item 3	4.7 (1.53)	0.828	0.901
Item 4	4.3 (1.54)	0.719	0.912
Item 5	5.3 (1.52)	0.765	0.907
Item 6	4.8 (1.51)	0.792	0.904
Item 7	4.5 (1.59)	0.728	0.911
Total score	33.5 (8.84)	-	-

SD: Standard deviation

Discussion

The prevalence of poor mental health and emotional exhaustion is high among nurses in cancer units/centers in many regions worldwide.[8910] Equally, oncology nurses based in China have to endure negative psychological conditions and burnout[46749] which hinders professional performance and affects the quality of health care provided.

Over the past decade, experimental and longitudinal studies have shown that psychological flexibility is an important determinant of mental health and behavioral effectiveness in the workplace. Higher levels of psychological flexibility correlate with better mental health, better job performance, and an increased capacity to learn skills at work.[2850] ACT interventions can improve employees’ mental health, increase innovation, and reduce the risk of burnout by increasing their psychological flexibility. Reliable and valid instruments are needed to further explore psychological flexibility in the workplace. The WAAQ is an instrument that enables assessment of psychological flexibility among individuals in a work context.

The present study showed the psychometric properties of the WAAQ among Chinese nurses working in a cancer hospital. Our data suggest that the Chinese version of the WAAQ had satisfactory reliability and validity in our sample of nurses. EFA helped us to identify a seven-item, one-factor structure of work-related psychological flexibility, a result that is in accordance with earlier studies.[2930]

Our findings revealed good reliability for internal consistency for the WAAQ, as demonstrated by Cronbach's α coefficients (0.920 for the total scale) and item-total correlation coefficients (0.786–0.878). The odd-even split-half reliability is commonly used which helps to avoid some of the potential biases that arise from simply dividing the measurement procedure into two. The odd-even split-half reliability coefficient is 0.933, which is generally considered to be satisfied. High test-retest reliability over 2–4 weeks (r = 0.772) was also found. These findings are consistent with previous studies.[29] For example, the Spanish translation of the WAAQ resulted in a Cronbach's α coefficient of 0.92 and item-total correlations of 0.65–0.81.[30]

To assess the criterion and convergent validity of the WAAQ, the correlation between the WAAQ and the AAQ-II, GSES, UWES, and GHQ-12 were examined. The correlation between the WAAQ and AAQ-II was small to moderate, which suggests that both instruments are assessing related constructs, but the correlation is not so high as to suggest that they are assessing the same one. Moreover, the AAQ-II showed lower correlations than the WAAQ with work engagement as measured by the UWES and self-efficacy as measured by the GSES. These findings demonstrated that a work-specific measure of psychological flexibility was more strongly associated with work-related outcomes than a general measure of the same construct. These data replicate the outcomes found by Bond et al.[29] and Ruiz and Odriozola-González[30] which support the hypothesis that psychological flexibility is related to a specific context as well. However, the AAQ-II showed higher correlations than the WAAQ with psychological distress as measured by the GHQ-12, which may have been because the AAQ-II is a general measure of psychological flexibility. This outcome was different to that elicited previously, possibly because of the features of our study sample. Furthermore, the mean scores of each item in the Chinese version of the WAAQ were lower than those found in the Spanish version, which suggests that the psychological flexibility of the oncology nurses was poor. The Chinese version of WAAQ can measure the psychological flexibility for oncology nurses in workplace. Therefore, the nursing administrators can have some insight into the psychological status of oncology nurses to pay more attention to their mental health in the future.

Limitations and Future Research

The psychometric properties and performance of the translated seven-item Chinese version of the WAAQ are satisfied. However, the absence of longitudinal data did not allow for testing the responsiveness of the Chinese version of the WAAQ. Future multicenter cross-sectional studies could be conducted to enhance the generalizability of Chinese version of the WAAQ. We also assume that work-related acceptance could play as a mediator between demographic data and negative emotions. The next step will figure out the mediating role of work-related acceptance.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.