Psychometric properties and scales of the Granada Burnout Questionnaire applied to nurses.

Nurses are an occupational group with extremely high levels of burnout. The most accepted definition of the burnout syndrome was proposed by Maslach and Jackson, who characterized it in terms of three dimensions: (i) Emotional Exhaustion; (ii) Depersonalization; (iii) Personal Accomplishment. This definition was the basis for the Granada Burnout Questionnaire (GBQ). The objective of this research was to evaluate the psychometric properties of the GBQ and to elaborate an evaluation scale to measure burnout in nursing professionals in Spain. A total of 1,177 nurses participated in this study and successfully completed the GBQ. Evidence of construct validity was verified by cross-validation and convergent validity, and evidence of criteria validity was checked by concurrent validity. Cronbach's alpha was used to measure internal consistency. The results obtained in our study show satisfactory fit values in the confirmatory factor analysis and in the evidence of convergent and concurrent validity. All of the Cronbach alpha values were greater than .83. This signifies that the GBQ has good psychometric properties that are applicable to nurses. For this purpose a scale of T-scores and centiles was created that permitted the evaluation of burnout in Spanish nursing professionals.

Over the last forty years, burnout has come to be acknowledged as a widespread psychological problem that has serious consequences, not only for people who suffer from it, but also for the entire workplace. Although there is no universally accepted definition for this syndrome, the one most often cited was proposed by Maslach and Jackson (1981) and its subsequent modifications (Maslach and Jackson, 1986, Maslach et al., 2001). These authors describe burnout as a response to chronic work-related stress, characterized by the dimensions of Emotional Exhaustion, Depersonalization, and Personal Accomplishment. Emotional Exhaustion alludes to a perceived lack of energy and, as its name implies, the exhaustion of emotional resources. Depersonalization is the development of indifferent and cynical attitudes towards patients, co-workers, and the organization of employment. Finally low Personal Accomplishment refers to the tendency of workers to negatively evaluate themselves in relation to their work capacity.

Although the importance of the work carried out by healthcare professionals, particularly nurses, is beyond dispute, this occupation is extremely prone to the development of burnout syndrome (Epp, 2012, Kheiraoui et al., 2012, Lin et al., 2014). Nurses care for patients, sometimes in contexts in which they are at risk of suffer physical and verbal aggressions (Gascon et al., 2013). In certain cases, this can lead to occupational illness (Trybou et al., 2014) and even to the abandonment of the profession (Farquharson et al., 2013). An added difficulty is the dichotomy which nurses must face daily in their work. Even though they are expected to have a friendly caring attitude, inherent in their vocation, they often have no choice but to be coldly objective in order to make difficult decisions in which no emotional response is possible. This can evidently become a source of conflict. For this reason, nurses are continuously obliged to deal with high levels of stress, which contributes to the appearance of burnout syndrome (Campagne, 2012, Cañadas-de la Fuente et al., 2014).

The consequences of this disorder affect all levels of society in one form or another. Nurses who suffer from burnout generally have psychosomatic problems (e.g. weakness and insomnia); emotional problems (e.g. anxiety and depression); attitude problems (e.g. hostility, apathy, and distrust), and behavioral problems (e.g. aggressiveness, irritability, and feelings of isolation) among others (Jansson-Frojmark and Lindblom, 2010, Leape et al., 2012). At the organizational level, problems can also arise that make it impossible to achieve the objectives for healthcare personnel. Because of the current economic crisis in Spain and the structural changes in the National Healthcare System, these objectives are already difficult to attain without any other added difficulties.

For example, the emotional exhaustion of these workers can cause conflict among the staff and result in absenteeism, low spirits, and decreased productivity (Trybou et al., 2014). Furthermore, those who most suffer the consequences of burnout in nurses are the users of healthcare services. It goes without saying that a harmonious workplace environment generates less stressful work conditions and this means a lower level of burnout in workers (Zhang et al., 2014). Evidently, if there is a reduction in stress at work, this will help to prevent burnout and also to increase the quality of patient care in general (Farquharson et al., 2013).

Although there are different methods of evaluating the burnout syndrome, the most frequently used is the Maslach Burnout Inventory (MBI) (Maslach & Jackson, 1981). In Spain, Seisdedos’ (1997) adaptation of this inventory used to be widely applied in burnout research. However, this version finally went out of print. In any case, even before this happened, it had drawbacks. In our opinion, some of the scales of this inventory are not valid for certain professional groups. Over the years, its psychometric evaluation and scales were applied to a sample composed of 51 primary and secondary school teachers, 149 police officers, 156 doctors with different medical specializations in a Spanish province, and 1138 people in a heterogeneous sample (Seisdedos, 1997, p. 32).

When this Spanish adaptation of the MBI went out of print, this led to the creation of the Granada Burnout Questionnaire (GBQ), which uses the same theoretical framework and structure as the MBI and from which burnout evaluation scales were established for the Spanish police force (De la Fuente et al., 2013). Because nurses have never been used as a sample to assess and validate the MBI, we wished to ascertain whether the GBQ was capable of assessing burnout levels in members of this profession.

Our hypothesis was that the GBQ could be applied to this occupational group. The advantages of the GBQ over other evaluation scales are the following: (1) it is adapted to the Spanish population; (2) it can be used by any researcher; (3) it includes scales for nursing professionals, one of the occupational groups most susceptible to burnout. The objective of this work was to thus to obtain psychometric indicators and scales of the GBQ for nurses in Spain.

Method

Participants

A total of 1,177 Spanish nurses participated in this study. The mean age of the participants was 44.25 years (SD = 8.45). Of the sample, 35.2% were male and 64.8% were female; 18.8% were single, 73.5% married, and 7.7% separated, divorced, or widowed. Of the subjects, 34% often had lengthy work shifts (24-hour on-duty service). In regards to hospital unit, 28% worked in the emergency room and intensive care; 18.9% were in surgical units; 23.9% were in patient care units; 21.8% were in primary healthcare; and 7.4% worked in the maternal and child healthcare services. In relation to work shift, 55.9% of the subjects had rotating shifts; 41.9% always worked in the morning; 0.7% always worked in the afternoon; and 1.5% always worked at night.

Instruments and development of the Granada Burnout Questionnaire

The Granada Burnout Questionnaire (GBQ) (De la Fuente et al., 2013) is a paper-and-pencil test that was created according to the guidelines in Downing (2006). It is based on the operative definition of burnout provided by Maslach et al. (2001) for the specification of the different dimensions.

The questions are answered on a five-point Likert scale, where 1 means total disagreement and 5 signifies total agreement. The items measure burnout dimensions either positively or negatively. A high score signifies a high value in the feature evaluated. Accordingly, 11 items should be redirected when the correction is performed (items 1, 4, 10, 12, 13, 19, 20, 23, 24, 25 and 26).

We also applied the MBI (Maslach & Jackson, 1981) in its version adapted to the Spanish population (Seisdedos, 1997) to examine the convergent validity of the GBQ. The MBI is composed of 22 items, which are answered on a seven-point Likert scale. Following the recommendations of meta-analytic studies (Aguayo, Vargas, De la Fuente, & Lozano, 2011), we calculated the reliability of each of the MBI dimensions in the sample. These dimensions are Emotional Exhaustion (9 items, α = .89), Depersonalization (5 items, α = .70), and Personal Accomplishment (8 items, α = .84).

The Neo Reduced Five-Factor personality inventory (NEO-FFI; Costa & McCrae, 1992) was administered in the version adapted to the Spanish population (Costa & McCrae, 2002) to evaluate concurrent validity. This inventory consists of 60 items answered on a five-point Likert scale with 12 items in each dimension. In this sample, Cronbach's alpha for each dimension was the following: Neuroticism (.79), Extroversion (.80), Openness (.70), Agreeableness (.75), and Responsibility (.83).

The Educational-Clinical Questionnaire: Anxiety and Depression (CECAD; Lozano, García-Cueto, & Lozano, 2007) was also used to assess concurrent validity. This questionnaire is composed of 50 items with a five-point Likert scale format. It provides an overall evaluation of emotional disorders based on the scores in the five dimensions. In our sample, its reliability was the following: Depression (26 items, α = .94), Anxiety (19 items, α = .92), Uselessness (9 items, α = .87), Irritability (7 items, α = .87), and Problematic Thoughts (8 items, α = .86).

Once legal permission was obtained, the guidelines of Campion and Miller (2006) were used to edit the data collection booklet.

Procedure

This research is an instrumental study (Hartley, 2012, Montero and León, 2007). We contacted the Nurses’ Union (SATSE, for its initials in Spanish). A committee composed of the authors and members of the Provincial Delegation of the SATSE in Granada were in charge of coordinating data collection. This made it possible for nurses from other provinces in the region of Andalusia to also participate. The subjects were contacted individually. Those who voluntarily agreed to be part of the study anonymously filled out the questionnaires, outside of their work schedule.

Statistical analysis

In order to evaluate the evidence of validity based on the internal structure of the questionnaire, the factorial validity was verified by cross-validation. For this purpose, an Exploratory Factor Analysis (EFA) was performed with a Minimum Rank Factor Analysis (MRFA) and oblique factor rotation (PROMIN). The polychoric correlations between items were used as the input matrix, all of which were implemented in the program FACTOR v.9.2 (Lorenzo-Seva & Ferrando, 2006).

EFA was performed on approximately 40% of the sample. A Confirmatory Factor Analysis (CFA) was conducted on the remaining 60% by using the maximum likelihood for the estimation of parameters, standard error, and goodness of fit, and was corrected when it did not show a multivariate normality. This option was suitable in our case since the items had a Likert-scale response format with moderate asymmetry indices (< 2) and kurtosis (< 7) (Finney and DiStefano, 2013, Rhemtulla et al., 2012) implemented in MPLUS 7.2 (Muthén & Muthén, 1998–2012). The goodness-of-fit criteria used were the following values: RMSEA ≤ .06, CFI ≥ .95, TLI ≥ .95, and SRMR ≤ .08 (Beauducel and Wittmann, 2005, Hu and Bentler, 1999, Yu and Muthén, 2002).

The evidence of convergent validity was evaluated by estimating the correlation of GBQ dimension scores with the MBI dimensions. Evidence of criterion-related validity was also studied through concurrent validity that analyzed the correlations between the GBQ dimensions with those of the NEO-FFI and CECAD. Attenuation formulas for measurement errors were used to correct all of the estimated correlations (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999).

An analysis was then performed of the items in the dimensions of the GBQ. More specifically, the focus of the study was on the discrimination, mean, standard deviation, and standard error of the scores of the items in each dimension. Cronbach's alpha was also calculated. Analysis of variance (ANOVA) was then performed to ascertain whether there were statistically significant differences in the burnout dimension scores among different groups.

Finally, the scores of the questionnaire were transformed to a scale of a mean of 50 and a standard deviation of 10, following a normal distribution as well as their corresponding centile scale, both widely used in Health Sciences.

Results

Construct validity

An Exploratory Factor Analysis was performed on 40% of the sample (n = 462). The Minimum Rank Factor Analysis extraction method with Promin rotation was used to extract three factors that explained 73.21% of the total variance of the questionnaire (see Table 1). The number of factors extracted was determined by a Parallel Analysis (Timmerman & Lorenzo-Seva, 2011) as well as by the theoretical model supporting the questionnaire and the suitability of the three-dimensional model.

Table 1

Factor loadings for Exploratory Factor Analysis (40% of the sample).

Item	PA	EE	D
1		.59
2		.74
3		.65
4	.68
5		.63
6		.73
7		.66
8	.87
9		.74
10		.73
11	.81
12	.81
13	.60
14		.69
15	.69
16	.69
17			.52
18			.70
19			.75
20	.70
21	.73
22	.44
23			.71
24			.82
25			.93
26			.90

Note. PA = Personal Accomplishment; EE = Emotional Exhaustion; D = Depersonalization.

The correlation between Depersonalization and Personal Accomplishment was -.63, the correlation between Depersonalization and Emotional Exhaustion was .13, and the correlation between Personal Accomplishment and Emotional Exhaustion was -.48. The mean value of the residuals of the model was -0.0006 (SD = 0.045), GFI [goodness-of-fit index] = .99, and RMSR = .045. This indicates a good fit of the three-dimensional model to the data.

The next step was a Confirmatory Factor Analysis of the remaining 60% of the sample (n = 715). The model obtained in the previous Exploratory Factor Analysis was verified and the relation between factors was taken into account. Furthermore, guided by the modification indexes, the correlations between residuals were estimated. This is common in factorial solutions with related factors. However, for the sake of simplicity, these correlations are not shown in the graphical model in Figure 1.

Figure 1

Confirmatory Factor Analysis of Granada Burnout Questionnaire (60% of the sample).

When the parameters of the model were calculated by the MLM calculation method implemented in MPLUS (Muthén & Muthén, 1998–2012), the result was χ (257) = 515.97, p < .001. Moreover, values of RMSEA = .038 (90% C.I.: .033, -.042) and p(RMSEA ≤ .05) = .99, CFI = .957, TLI = .945, SRMR = .061 were obtained. These values indicated the good fit of the model.

Because of the high correlation between Emotional Exhaustion and Personal Accomplishment as well as between Personal Accomplishment and Depersonalization, the corresponding two-dimensional models were checked with the chi-square difference test in comparison to the three-dimensional model. The result obtained was χ2diff (2) (PA + D) = 164.76, p < .001 and χ2diff (2) (PA + EE) = 272.41, p < .001, which indicated that the three-dimensional model had a better fit than the two-dimensional models (Byrne, 2001).

Evidence for convergent validity was calculated by means of the correlation between final scores of the various dimensions of the GBQ and the MBI. The values obtained were greater than .51 (Table 2). These correlations were corrected to eliminate the negative effect of the measurement error.

Table 2

Corrected correlation matrix (adjusted using the attenuation formula; N = 1,177).

		MBI-EE	MBI-D	MBI-PA
GBQ	EE	.80
GBQ	D		.52
GBQ	PA			.66
		Neuroticism	Extroversion	Openness	Agreeableness	Responsibility
GBQ	EE	.70	-.37	-.23	-.44	-.32
MBI	EE	.68	-.46	-.17	-.49	-.40
GBQ	D	.44	-.60	-.38	-.75	-.76
MBI	D	.57	-.44	-.28	-.68	-.55
GBQ	PA	-.69	.61	.36	.70	.73
MBI	PA	-.50	.56	.36	.59	.66
		Depression	Anxiety	Uselessness	Irritability	Problem thoughts
GBQ	EE	.53	.52	.41	.51	.48
MBI	EE	.69	.66	.55	.64	.65
GBQ	D	.39	.35	.39	.38	.34
MBI	D	.57	.53	.52	.53	.56
GBQ	PA	-.58	-.54	-.52	-.57	-.52
MBI	PA	-.48	-.43	-.45	-.47	-.44

Note. GBQ = Granada Burnout Questionnaire; MBI = Maslach Burnout Inventory; EE = Emotional Exhaustion; D = Depersonalization; PA = Personal Accomplishment.

Criteria validity

To obtain evidence of criterion-related validity, the concurrent validity was analyzed by calculating the correlation between the scores for the different dimensions of the GBQ as well as the MBI and the scores obtained in the various dimensions of the NEO-FI and the CECAD. Generally speaking, the values obtained for the GBQ and the MBI in relation to the NEO-FI and CECAD were similar (see Table 2). As in the previous case, the correlations were corrected.

Analysis of items and reliability

Table 3 shows the descriptive analyses, the discrimination index of the GBQ items by dimension. Also included are the α coefficients of each dimension of the GBQ and the MBI as well as their dispersion.

Table 3

Item analysis, reliability and variance of scores on the dimensions (N = 1,177).

Dimension	Item	Mean	SE	SD	Disc.	α GBQ	α MBI	σ2GBQ/SEM	σ2MBI/SEM
EE	1	2.94	0.04	1.26	.49	.84	.89	52.22/2.89	125.71/3.78
	2	2.68	0.03	1.18	.62
	3	3.43	0.03	1.19	.54
	5	2.47	0.04	1.22	.49
	6	1.97	0.03	1.14	.58
	7	2.12	0.04	1.21	.47
	9	2.32	0.04	1.24	.60
	10	2.62	0.04	1.22	.64
	14	2.96	0.04	1.21	.56
D	17	2.11	0.03	1.05	.51	.86	.70	21.01/1.71	32.67/3.13
	18	1.76	0.03	0.95	.64
	19	1.63	0.02	0.83	.56
	23	1.85	0.03	0.95	.62
	24	1.84	0.02	0.82	.65
	25	1.78	0.02	0.81	.69
	26	1.89	0.02	0.83	.70
PA	4	3.92	0.03	1.00	.55	.87	.84	48.25/2.50	73.41/3.43
	8	4.36	0.03	0.91	.63
	11	4.19	0.03	0.94	.67
	12	3.96	0.03	1.02	.72
	13	3.82	0.03	1.03	.51
	15	4.17	0.03	1.08	.57
	16	4.08	0.03	1.15	.59
	20	4.11	0.03	1.07	.60
	21	4.45	0.03	0.95	.59
	22	3.75	0.03	1.14	.41

Note. GBQ = Granada Burnout Questionnaire; MBI = Maslach Burnout Inventory; EE = Emotional Exhaustion; D = Depersonalization; PA = Personal Accomplishment; SE = Standard Error; SD = Standard Deviation; Disc. = Discrimination Index; SEM = Standard Error of Measurement.

The α coefficients obtained for the GBQ and MBI were similar. Higher reliability values corresponded to the dimensions of Depersonalization and Personal Accomplishment in the GBQ in regard to those obtained for the MBI.

When the reliability coefficient was calculated for the dimensions of the GBQ, and the variance set to that obtained in the dimensions of the MBI, the following reliability values were obtained: .99 for Emotional Exhaustion; .96 for Depersonalization; and .94 for Personal Accomplishment.

Scales

This research focused on the differences in the scores of the GBQ dimensions, depending on group membership, with a view to establishing different scales for those groups that required them. For this purpose, a fixed-effect ANOVA was performed with four factors: gender, marital status, 24-hour duty shifts, and hospital unit. No statistically significant differences were found (p > .01) for any of the dimensions as a function of the various factors or their interactions. Both T and centile scores were used for the scales of each dimension since they are frequently used in the field of healthcare (see Table 4).

Table 4

The Granada Burnout Questionnaire scales for Spanish nurses.

Emotional Exhaustion			Depersonalization			Personal Accomplishment
DS	CS	T	DS	CS	T	DS	CS	T
9	1	26	7	6	34	18	1	24
10	2	30	8	15	40	19	1	26
11	3	32	9	23	43	20	1	26
12	5	34	10	31	45	21	1	27
13	7	35	11	39	47	22	1	28
14	10	37	12	48	49	23	2	29
15	12	38	13	55	51	24	2	30
16	15	40	14	64	54	25	3	30
17	19	41	15	73	56	26	3	32
18	24	43	16	79	58	27	5	33
19	28	44	17	84	60	28	6	35
20	33	46	18	87	61	29	7	35
21	38	47	19	90	63	30	8	36
22	43	48	20	93	64	31	10	37
23	49	50	21	94	66	32	13	39
24	56	51	22	96	67	33	15	40
25	61	53	23	97	68	34	17	40
26	65	54	24	98	70	35	19	41
27	70	55	25	98	71	36	22	42
28	74	56	26	98	72	37	25	43
29	78	58	27	98	73	38	29	45
30	81	59	28	99	74	39	33	46
31	84	60	29	99	75	40	38	47
32	87	61	30	99	77	41	44	48
33	89	63	31	99	77	42	49	50
34	91	64	32	99	78	43	55	51
35	93	65	33	99	79	44	61	53
36	95	66	34	99	81	45	67	54
37	96	67				46	74	56
38	97	69				47	81	59
39	98	70				48	87	61
40	98	71				49	92	64
41	98	72				50	97	69
42	99	74
43	99	76
44	99	78
45	99	80

Note. DS = Direct Score; CS = Centil Score; T = T score.

Discussion

This study, which is a validation of the Granada Burnout Questionnaire (GBQ), was carried out using the MBI theoretical framework, along with the psychometric indicators of a sample of nurses in the Andalusian Health Service. The construct validity of the GBQ was based on the factorial validity of the questionnaire. In this way, it was possible to obtain a three-dimensional model, which provided a good fit for the data.

As in De la Fuente et al. (2013), the convergent and concurrent validity indexes were obtained by correlating the final scores of the GBQ dimensions with those of the MBI, NEO-FFI, and CECAD. Attenuation formulas for measurement errors were used to correct all correlations.

In regards to the concurrent validity of the questionnaire, the relationships between burnout dimensions and personality factors were similar to those in studies that used the GBQ as well as those that used the MBI (De la Fuente et al., 2013, Kiffin-Pettersen et al., 2011). More specifically, the dimension of Emotional Exhaustion and personality factors showed the following: (i) an intermediate and positive correlation with Neuroticism; (ii) a low or non-existent correlation with Openness; and (iii) an intermediate and negative correlation with Extroversion, Agreeableness, and Responsibility. These results are in consonance with those of other authors (Bakker et al., 2006, De la Fuente et al., 2013, Esteras et al., 2014).

Depersonalization was found to be related to personality factors. In this regard, it had a medium and positive relation with Neuroticism and a negative relation with Extroversion, Agreeableness, and Responsibility. Although these results are similar to those obtained by other authors (De la Fuente et al., 2013, Esteras et al., 2014, Goddard et al., 2004), there is a certain lack of consensus concerning the relationship between Depersonalization and Openness.

This study obtained intermediate correlations between Personal Accomplishment and personality factors. More specifically, Personal Accomplishment showed a negative correlation with Neuroticism, and a positive correlation with Extroversion, Openness, Agreeableness, and Responsibility. These results also agree with those obtained in previous research (Bakker et al., 2006, De la Fuente et al., 2013).

The concurrent validity of the questionnaire was based on the relation between the various burnout dimensions in the GBQ and the emotional disorders of the CECAD. As proposed by Papastylianou, Kaila, and Polychronopoulos (2009), this research classified emotional disorders into two large groups (Depression and Anxiety), which had been used to validate the GBQ in a sample composed of members of the Spanish police force (De la Fuente et al., 2013). Correlations between GBQ dimensions and the group of factors associated with Depression were intermediate and in the expected direction (e. g., Tourigny, Baba, & Wang, 2010). The results obtained for the variables associated with Anxiety were similar to those obtained for Depression and in line with other research (Cremades, Wated, & Wiggins, 2011).

The adequacy of the psychometric properties of the GBQ is reflected in the analysis of its items and reliability. Good discrimination indexes were obtained, such that the lowest value was .41. The reliability coefficients were found to be satisfactory (Raju, Price, Oshima, & Nering, 2007), especially considering the number of items in each dimension.

Cronbach's alpha for the GBQ dimensions of Depersonalization and Personal Accomplishment was greater than that for the corresponding dimensions of the MBI. In contrast, its reliability value was slightly lower in the dimension of Emotional Exhaustion. This occurred because the items of the MBI have a seven-point Likert format as compared to the five-point Likert scale in the GBQ. A higher number of alternatives artificially increase the variance, which also increases the reliability value (Lozano, García-Cueto, & Muñiz, 2008). The GBQ showed systematically lower Standard Error of Measurement values than those obtained for the corresponding MBI dimension. This means that the GBQ had a lower measurement error for all burnout factors. Finally, the scales of the GBQ (in T and centile scores) are included for the occupational group of Spanish nurses.

One limitation of this study was the use of an incidental sample even though it is also true that the sample was large and representative of a wide range of hospital units. In future work, we plan to assess and validate the GBQ with a sample of medical doctors. Medical doctors are somewhat different from nurses in regards to levels of burnout but they are also a group with a high risk of suffering this syndrome.

Funding

This work was funded by the Excellence Research Project P11HUM-7771 (Junta de Andalucía-Spain) and the Research Project mP_BS_6 (CEI BioTic Granada and Ministerio de Ciencia e Innovación-Spain).