Short Index of Job Satisfaction: Validity evidence from Portugal and Brazil.

Job satisfaction is an important construct that is known to be associated with workers' performance and wellbeing. As such, to properly measure it, one must use adapted measures that show adequate validity evidence for the desired context. Such measures should preferably be short to allow the parsimonious use of various measures/constructs in the same data collection. The aim of this paper is to adapt the Portuguese version for Brazil and Portugal of the Short Index of Job Satisfaction (SIJS). The SIJS is a psychometric instrument that measures job satisfaction through five items. A cross-sectional study was conducted with two multi-occupational workers samples, one from Brazil (n = 599) and one other from Portugal (n = 572). The SIJS presented good validity evidence based on its internal structure, namely dimensionality, reliability, and measurement invariance across countries and sexes. It also revealed to be positively correlated with work engagement, and quality of work life (convergent evidence). It also has shown to be negatively associated with burnout (discriminant evidence). The SIJS showed promising validity evidence. The SIJS can be useful to be used together with other instruments, due to its small number of items, producing data with good psychometric properties.

Introduction

Job satisfaction has a long history of proliferating organizational research [1] being one of the most–if not the most–studied variables in business science [2]. It was brought into the limelight in the 30s [3,4] being the most studied variable in the organizational behavior context [5,6]. It has various definitions. The most cited is the one provided by Locke [7], which considered job satisfaction as a positive emotional state that results from the worker’s job associated experiences. Researchers can be interested in an overall measure of job satisfaction or particular job satisfaction facets, being the first usually preferred [8]. Overall job satisfaction can be seen as a formative construct, aggregating satisfaction with specific facets of the job [9]. Such approach appears as a solution to measure the overarching degree of satisfaction with various job attributes, coming with various different flavors [8].

Due to its complexity–influenced by various factors–this construct is frequently used in work and organizational studies together with other dimensions. The factors that influence job satisfaction can be considered both at the individual (mainly one’s values, but also personality and mental health) and the organizational level (work, payment, promotions, peers/colleagues, supervisor, top leadership and benefits/policies) [7]. As such, a higher quality of work-life balance is expected to produce higher job satisfaction [10-12] while more job satisfaction is positively related to greater organizational commitment [13]. In the opposite direction, more stress in the workplace [14] and more surface acting [15] are related to less job satisfaction. It has been observed that workers have their job satisfaction levels decreasing parallel to the increasing tenure in a specific organization, while for people getting older–and transitioning of the organization–their satisfaction increases [16]. Lack of job satisfaction can lead to turnover [17-19], negative mood [20], reduced health and life happiness [7]. It can be related to several withdrawal behaviors [6] as absenteeism, presenteeism, and performance [21-24]. The relation with performance has long been considered the “holy grail” of satisfaction research [25] However, such relation is not unidirectional, lower performance can also lead to less job satisfaction [7]. It also has relations with several other job behaviors [26].

Despite existing many different psychometric instruments to measure the job satisfaction construct, some criticism has been made about the way this construct has been measured [27], since few of those instruments have shown satisfactory validity evidence [14]. Some of the most widespread measures are: the Index of Job Satisfaction [28] which also has a short version of five items (Short Index of Job Satisfaction [SIJS]) [29]; the Measure of Job Satisfaction (MJS) [30]; the Job in General Scale (JIG) [31] which is part of the Job Descriptive Index (JDI) [32]; the Andrew and Withey Job Satisfaction Questionnaire [33]; the Job Satisfaction Survey (JSS) [34]; the Minnesota Satisfaction Questionnaire [35]; the Michigan Organizational Assessment Questionnaire Job Satisfaction Subscale (MOAQ-JSS) [36]; and the Generic Job Satisfaction Scale (GJSS) [37]. Some of those measures are lengthy like the MJS (38 items) [30], the JIG with 18 items [31], the JSS with 36 items, or the MSQ with 100 items in the long-form, and with 20 items in the short form [35]. Other measures are shorter, like the SIJS (comprising five items), the MOAQ-JSS with three items, the Andrew and Withey Job Satisfaction with five items, or the GJSS with ten items. Van Saane et al. [14] reviewed several measures of job satisfaction and concluded that only seven of 29 reviewed measures fulfilled the authors’ minimum criteria of an adequate instrument (i.e. internal consistency and validity evidence). In such a wide variety of measures, some are specific for a specific job (e.g. MJS was developed for nurses) while others can be used with multiple occupations (e.g. SIJS, JIG, JSS).

Some instruments measure job satisfaction as a unidimensional global measure, while others assume it a multidimensional construct. Examples of former are the SIJS, the JIG, the MOAQ-JSS or the Andrew and Withey Job Satisfaction Questionnaire. While the latter has as examples the MSQ, the GJSS, the JSS, or the MJS. Global measures are more parsimonious than multidimensional constructs. It is expected that the latter explains more variance, however, it might be difficult to sort out if the broader measure assures that it is measuring the construct, or if it is also measuring elements of its causal network [38]. If one aims to measure specific job satisfaction areas, the multidimensional instruments will be more useful, since they measure job satisfaction facets. Although, the global scales are useful to have a general measure of job satisfaction [31].

There are also single-item measures, which some authors find useful in specific situations [27,39] and which can be more cost-effective than multi-item measures [40]. Despite single-item measures’ usefulness and correlation with scale measures–from a psychometric perspective, multiple-item measures are preferable [41,42]. Single-item measures do not allow to obtain internal consistency estimates, using them to measure psychological constructs can be seen as a fatal error [43]. As such, short measures that approach job satisfaction from a general perspective–as the SIJS–without entering in the peculiarities and the specifics of certain occupations might be preferable to single-item measures. These short multi-item measures maintain the overall perspective of the measure with the advantage of obtaining the latent measure through the necessary multiple items. It is preferable to select a short measure that has demonstrated validity evidence in the desired population of the study, then to adapt or select some items of a longer instrument [44]. One of these short measures is the SIJS which has some advantages over other measures. It is freely available, has shown good psychometric properties and has been used in different cultures [45-48].

No studies adapting the SIJS to Portugal or Brazil were found, as such, the aim of this study is: i) adapt the SIJS in its Portuguese version for Brazil and Portugal and assess its validity evidence; and, ii) compare the job satisfaction levels between sexes and countries. Following the guidelines of the Standards for Educational and Psychological Testing [49] two of the possible five sources of validity evidence for psychological instruments will be analyzed. The first is based on the internal structure which refers to the reliability of the scores, dimensionality and measurement invariance. The second source of validity is based on relation to other variables. Regarding the validity evidence based on the internal structure, three hypotheses were defined. Previous studies suggest that the SIJS is a single-factor measure suitable to obtain a general job satisfaction measure, presenting satisfactory validity evidence [46,50,51]. Hypothesis 1 states that the SIJS maintains its original dimensionality (five items, one factor) supported by factorial validity evidence.

Various studies showed that the SIJS presents good reliability, namely, internal consistency estimates [52-55]. Hypothesis 2 assumes that the SIJS presents adequate evidence validity in terms of reliability of the scores.

No study was found testing measurement invariance of the SIJS between sexes or countries. Nevertheless, there is evidence that other psychological instruments used in the organizational context achieved measurement invariance among countries (i.e. Brazil and Portugal) and among sexes in the two countries [56,57]. Brazil and Portugal share more than the same language, they exchange culture and they share human capital. As such, it will be likely that the SIJS has measurement invariance between countries and sexes. Thus, Hypothesis 3 states that SIJS will hold measurement invariance (at least scalar invariance) among Brazil and Portugal, and sexes within each of the countries.

The second source of validity evidence, based on the relationships with other variables, will be checked using quality of work-life, burnout, and work engagement measures. Lack of Job satisfaction is expected to be a predictor of burnout [58-60]. On the other hand, work engagement has shown to predict job satisfaction [61-63] and by the quality of work-life [12]. Nevertheless, this source of evidence is intended to assure that job satisfaction is distinguished from associated concepts [64]. Altogether, Hypothesis 4 states that the SIJS shows adequate validity evidence based on the relation to other variables—more specifically—nomological evidence [65]. As such, it hypothesized that the SIJS will present discriminant evidence with burnout (Hypothesis 4.1), and convergent evidence with work engagement (Hypothesis 4.2) and quality of work-life (Hypothesis 4.3).

Previous research studied differences among sexes for job satisfaction among countries and sexes. The meaning attached to work seems to not differ significantly among countries [6]. However, Bozionelos and Kostopoulos [66] indicated that different countries might have differences in their job satisfaction levels. Such differences have been observed between several countries [67-69]. Pichler and Wallace [70] found that differences mainly result from composition and individual-level factors of each countries’ workforce, instead of inherent cultural and historic characteristics. While, on the other hand, Hauff, Richter, and Tressin [71] indicated that some job characteristics vary significantly between countries, being partially moderated by cultural dimensions. It seems that culture might be a moderator between job satisfaction and job characteristics [72,73]. Given that Portugal and Brazil are two countries that share language and cultural features, it is expected that job satisfaction differences between countries are small or non-existent.

In terms of sex comparisons, the theoretical arguments behind potential workplace beliefs and attitudes among women and men are related to the work-life conflict [74]. Women are historically more affected by this conflict, and consequently, they might present a lower job satisfaction than men, particularly in some life events like the first marriage or the birth of the first child [75]. Nevertheless, women can present higher job satisfaction levels despite their poor quality of work conditions, something that Hakim [76] coined as the “grateful slaves” paradox. Albeit, past research suggests that men and women are increasingly similar in terms of job and life satisfaction [77]. Altogether, Hypothesis 5 states that different job satisfaction latent mean levels are observed, among different countries (Hypothesis 5.1) and different sexes (Hypothesis 5.2).

Method

Sampling and data collection

For this study, two samples of multi-occupational workers were collected (N = 1,171; S1 and S2 Datasets). One sample is composed of Brazilian workers (n = 599) working in Brazil, and one other sample is composed of Portuguese workers (n = 572) working in Portugal. The sample from Portugal had an average age of 35.83 (SD = 9.76) with 37.16% being males, 83.07% had graduation or higher academic level, the occupational group with most participants was professionals (53.63%). The sample from Brazil had an average age of 35.11 (SD = 10.13), with 32.77% of males, 74.39% had graduation or higher academic level, professionals were most represented occupational group (36.12%).

The socioeconomic status according to the 2014 version of the Brazilian Criteria [78] was obtained, consisting of a socioeconomic classification of the households ranging from A1 (best socioeconomic level) to E (worst socioeconomic level) (Table 1). Additional variables regarding the characterization of the samples are available in Table 1.

Table 1

Demographics, career, academic level and occupational group statistics across countries.

	Portugal (n = 572)	Brazil (n = 572)	Comparisons
Academic level %
High school, vocational education or less	12.26	12.52	χ2(4) = 153.682; p < .001; V = .395
Unfinished graduation	4.67	13.09
Graduation	29.57	34.16
Post-graduation (not master neither PhD)	9.34	25.62
Master	38.52	9.49
PhD	5.64	5.12
Demographics and career
Working years in the current job sector M(SD)	11.23 (9.69)	9.73 (8.61)	t(983.35) = -2.607; p = .009; d = .164
Working years in the current organization M(SD)	8.11 (8.92)	5.84 (6.80)	t(924.95) = -4.544; p < .001; d = .287
Working years in the current job M(SD)	6.14 (7.05)	4.97 (6.29)	t(986.08) = -2.760; p = .006; d = .174
Sex (males) %	37.16	32.77	χ2(1) = 2.024; p = .155; φ = .046
Age M(SD)	35.83 (9.76)	35.11 (10.13)	t(1,039) = -1.169; p = .243; d = .072
Occupational groups* %
Elementary Occupations	1.81	1.55	χ2(8) = 81.394; p < .001; V = .284
Plant and Machine Operators and Assemblers	0.60	0.78
Craft and Related Trades Workers	2.22	2.14
Skilled Agricultural, Forestry and Fishery Workers	-	-
Services and Sales Workers	6.05	6.21
Clerical Support Workers	9.48	27.38
Technicians and Associate Professionals	12.90	8.74
Professionals	53.63	36.12
Managers	8.87	15.53
Armed Forces Occupations	4.44	1.55
Socioeconomic status %
A1	14.06	9.11	χ2(6) = 38.045; p < .001; V = .191
A2	54.69	45.16
B1	25.20	29.60
B2	4.69	10.06
C1	1.17	5.31
C2	0.20	0.38
D	-	0.38
E	-	-

*Following the International Standard Classification of Occupations ISCO-08 [79].

A non-probabilistic convenience sample was collected within a cross-sectional survey through an online tool software [80]. The inclusion criteria were being workers with a contract or formal ties with their employers, literate, and with access to a device where the survey could be fulfilled.

Constructs and measures

Short Index of Job Satisfaction (SIJS)

The Index of Job Satisfaction is a self-report psychometric instrument created by Brayfield and Rothe [28]. The original version was composed of 18 items, although a shorter version with five items (SIJS) has also been proposed [29,81]. Subjects are asked to respond to each item by checking a five-point scale (1 –“Strongly Disagree”, 2 –“Disagree”, 3 –“Undecided”, 4 –“Agree”, 5 –“Strongly Agree”) two of those items are reversed (Table 2). Regarding the validity evidence based on the internal structure in terms of reliability, this shorter five items version presented a good internal consistency evidence (α = .89) [29]. Similar evidence was found in another study (α = .82 to .83) [82]. The ITC Guidelines for Translating and Adapting Tests were used to adapt the SIJS to Portuguese in a single version (using the Portuguese Language Orthographic Agreement of 1990) that could be used both for Portugal and Brazil with the same items [83]. The pilot data on the SIJS adapted version to the Portuguese language used 15 workers from Brazil and 15 workers from Portugal.

Table 2

SIJS original and Portuguese versions.

Item	Original SIJS [29]					Portuguese (Brazil and Portugal) version of SIJS
	Strongly Disagree	Disagree	Undecided	Agree	Strongly Agree	Discordo Fortemente	Discordo	Indeciso	Concordo	Concordo Fortemente
	1	2	3	4	5	1	2	3	4	5
1	I feel fairly satisfied with my present job					Sinto-me razoavelmente satisfeito com meu emprego atual
2	Most days I am enthusiastic about my work					Na maioria dos dias, estou entusiasmado com o meu trabalho
3R	Each day at work seems like it will never end (R)					Cada dia no trabalho parece não ter fim (R)
4	I find real enjoyment in my work					Sinto-me realmente satisfeito no meu trabalho
5R	I consider my job to be rather unpleasant (R)					Considero que meu emprego é particularmente desagradável (R)

RReversed items.

Quality of Working Life Scale (QWLS)

The Quality of Work Life (i.e. second-order factor) was measured with the QWLS in its Brazil and Portugal version [84]. This instrument has 16 items that are answered with a seven-point scale (Likert-type; from 1- “Very Untrue” to 7 –“Very True”).

This measure comprises a second-order factor–Quality of Work Life–and seven first-order factors (i.e. seven major needs) which are: (1) economic and family needs;(2) health and safety needs; (3) aesthetic needs; (4) actualization needs; (5) esteem needs; (6) knowledge needs; and (7) social needs [12]. This measure tries to assess the workers’ perception of how the job requirements, work environment, support programs, and supervisor behavior meet their needs. In its Portuguese version for Portugal and Brazil, it has shown good psychometric properties (measurement invariance across Portugal and Brazil, and genders) [84]. It also revealed good second-order internal consistency values both for Brazil (ω = .96; ω = .90; ω = .94) and Portugal (ω = .95; ω = .88; ω = .93).

Oldenburg Burnout Inventory (OLBI)

The Oldenburg Burnout Inventory (OLBI) in its version for Portugal and Brazil [57] was used to measure burnout. Burnout is an occupational phenomenon; a workplace syndrome that results from coping unsuccessfully with chronic stressors from the occupational context [85]. This instrument assumes a second-order latent factor–Burnout–that comprises two first-order factors, Exhaustion (eight items) and Disengagement (seven items) which are rated from 5 = “Strongly Agree” to 1 = “Strongly Disagree”. Disengagement refers to a distancing and gradual loss of concern with one’s work recipients or contents, while exhaustion refers to a state of draining energy as a result of cognitive, emotional and physical strain [86]. The selected OLBI’s version presented good validity evidence base in the internal structure (i.e. dimensionality, reliability of the scores in terms of second-order internal consistency values [ω = .91; ω = .86; ω = .93], and measurement invariance among countries and sexes) and based on the relation to other variables (i.e. nomological; discriminant evidence with work engagement).

Utrecht Work Engagement Scale (UWES)

Work engagement was measured using the Utrecht Work Engagement Scale (short-version; nine items) in its Portuguese version for Portugal and Brazil [56]. It measures work engagement as a second-order latent factor which can be defined as a fulfilling, and positive state of mind related to work, with three (first-order) dimensions: absorption, dedication, and vigor [87]. Absorption is defined as one feeling totally concentrated and happily immersed at work [88]. Dedication refers to a sense of significance, challenge inspiration, pride and enthusiasm [87]. Vigor is defined as having high levels of energy [88]. As such, each of these three dimensions has three items, which are answered using a seven-point scale, from 0 = “Never” to 6 = “Always”. This instrument has presented good validity evidence with Portuguese samples [89,90], namely in terms of dimensionality, measurement invariance (among Brazil and Portugal), reliability (second-order internal consistency values [ω = .96; ω = .93; ω = .97]) and expected relation to other variables [56].

Procedure

The participants from both countries were invited to answer the selected instruments, sociodemographic and career questions. They were contacted individually or through companies that accepted to share the study by email. The cross-sectional data (i.e. cross-sectional survey) was collected using the LimeSurvey software [80] through the website of two major universities one in each of the two countries. The participation rate of the subjects that clicked in the invitation link from Brazil was 68.07% while 74.38% of Portuguese sample that clicked on the link fulfilled the survey. The participants were presented with the electronic informed consent in the first place, and after accepting it, had access to the survey per se. This study was approved by the Committee of Ethics in Research with Human Beings (CEP) of the Faculty of Philosophy, Sciences, and Letters of Ribeirão Preto, University of São Paulo, through “platform Brazil” of CONEP (National Commission of Ethics in Research). The written consent was obtained with the approval number 33301214.2.0000.5407. This study was also approved by the Ethics Committee of the Faculty of Psychology and Education Sciences of the University of Porto, the written consent had the following statement "the Ethics Committee of the Faculty of Psychology and Education Sciences of the University of Porto, having analyzed the research project … considers that it respects all ethical principles and ethical standards of research and therefore gives a favorable opinion".

Data analysis

The statistical program R [91] through the integrated development RStudio [92] was used to perform all statistical analyses. The skimr package [93] produced descriptive statistics. The multivariate normality was verified by the multivariate kurtosis [94] which was obtained using the psych package [95].

The confirmatory factor analysis (CFA) was the statistical technique selected to test the dimensionality of the instrument model. The lavaan package [96] was used to perform CFAs and multigroup CFA (MGCFA) with the Weighted Least Squares Means and Variances (WLSMV) estimator [97]. Some goodness-of-fit indices were selected: χ2 (chi-square), RMSEA (Root Mean Square Error of Approximation), SRMR (Standardized Root Mean Square Residual), CFI (Comparative Fit Index), NFI (Normed Fit Index), and the TLI (Tucker Lewis Index). Regarding the CFI, NFI, and TLI, Hu and Bentler [98] recommended that values above .95 should be considered as indicative of good fit. The RMSEA and SRMR values below .08 should be considered as good [99,100]. Model modifications were added through the analysis of the modification indices (> 30; p < .001) together with theoretical considerations.

The convergent evidence (in terms of internal structure) was assessed through the Average Variance Extracted [AVE; 101] calculated from polychoric correlations [102] where AVE ≥ .50 was considered as adequate convergent validity evidence [100].

The reliability of the scores evidence in terms of internal consistency was approached with the ordinal α [103], Composite Reliability [CR; 101] and ω [104,105] coefficients. The ordinal α coefficient was calculated from the polychoric correlations matrix. For the three coefficients, values ≥.70 were considered as indicative of acceptable reliability of the scores [100]. In the case of the second-order latent factors, three reliability estimates were calculated: the variance of the first-order factors explained by the second-order factor (ω), the proportion of the second-order factor explaining the total score (ω), and the proportion of variance explained by second-order factor after partialling the uniqueness of the first-order factor (ω). Both the ω (both for first- and second-order factors) and the α coefficients were calculated using the semTools package [102].

The measurement invariance of the SIJS was evaluated through multigroup confirmatory factor analysis (MGCFA) which—considering the categorical nature of items–was analyzed using the theta-parameterization [106]. Five nested models were compared to test configural invariance, metric invariance, scalar invariance, strict invariance and homogeneity of factor means. For such analysis the semTools package [102] was used. The full structural equation models were analyzed through the lavaan package [96].

Using the structural equation modeling framework, the latent variables mean scores were compared using as effect size the Cohen’s d [107]. Additionally, for the job satisfaction raw scores the means, standard deviations and quartiles were calculated through the doBy package [108]. Regarding the demographics, career, academic level and occupational group variables, the frequencies were compared using the chi-square test using the Cramér’s V [109] or the φ [110] as an effect size measure using the lsr package [111] and the psych package [95] respectively. The factor means were compared using the independent two-sample t-test or the Welch’s t-test when the variances were not equal. Cohen’s d [107] was used as estimator of effect size obtained through the lsr package [111].

Results

Validity evidence based on the internal structure

Dimensionality

Items’ distributional properties. The SIJS’ items were analyzed in terms of its descriptive statistics (Table 3) which did not reveal severe violations in terms of univariate normality, since none of the items (for both samples) presented absolute values of Sk above 3 or absolute values of Ku above 7 [112]. In terms of multivariate normality, both samples had values of Mardia’s multivariate kurtosis indicative of the absence of multivariate normality: Portugal (27.61, p < .001) and Brazil (16.10, p < .001). The maximum possible range of answers was observed in the five items and there was no outliers deletion. Altogether the distributional proprieties of the items indicate adequate psychometric sensitivity, however failing to achieve multivariate normality. As such, the WLSMV estimator that does not assume multivariate normality was used being also more adequate to the ordinal nature of the SIJS items.

Table 3

Sample 1 and 2 descriptive statistics.

Item	M	SD	Min	Q1	Mdn	Q3	Max	Histogram	Sk	Ku
Brazil
Item 1 –“fairly satisfied”	3.41	1.27	1	3	4	4	5	▃▃▇▇▆	-0.46	-0.77
Item 2 –“enthusiastic”	3.49	1.19	1	3	4	4	5	▂▃▆▇▆	-0.43	-0.74
Item 3R –“it will never end”	3.54	1.25	1	3	4	5	5	▂▃▅▇▆	-0.58	-0.66
Item 4 –“real enjoyment”	3.40	1.23	1	3	4	4	5	▂▃▆▇▆	-0.43	-0.77
Item 5R –“unpleasant”	4.15	1.12	1	3	5	5	5	▁▁▂▂▇	-1.13	0.28
Portugal
Item 1 –“fairly satisfied”	3.48	1.11	1	3	4	4	5	▂▃▆▇▃	-0.49	-0.42
Item 2 –“enthusiastic”	3.51	1.13	1	3	4	4	5	▂▂▅▇▅	-0.56	-0.38
Item 3R –“it will never end”	3.68	1.13	1	3	4	5	5	▁▃▅▇▆	-0.60	-0.42
Item 4 –“real enjoyment”	3.31	1.16	1	3	3	4	5	▂▃▆▇▃	-0.40	-0.64
Item 5R –“unpleasant”	4.30	1.01	1	4	5	5	5	▁▁▂▃▇	-1.45	1.51

RReversed items; the descriptive statistics of such items refer to the recoded scores.

Factor related validity evidence. The factorial validity evidence of the original proposed single latent factor was acceptable for the joint sample (Fig 1; χ2(5) = 108.469; p < .001; n = 1,171; CFI = .994; NFI = .993; TLI = .987; SRMR = .055; RMSEA = .133; P(rmsea) ≤ .05) < .001; 90% CI ].112; .155[). Although, after checking the modification indices it was noticed that the correlation path between the residuals of the items 3 and 5 could improve the model’s fit. That correlation was added (r = .322; p < .001) and the fit to the data was very good (χ2(4) = 11.228; p = .024; n = 1,171; CFI = 1.000; NFI = .999; TLI = .999; SRMR = .021; RMSEA = .039; P(rmsea) ≤ .05) = .698; 90% CI ].013; .067[). All factor loadings (λ) and the items 3 and 5 residuals’ correlation were statistically significant. There were no items removed. The minimum item loading for the SIJS was .57. The fit of the data to the model with one correlation between the referred items in the Brazilian sample was good (χ2(4) = 11.486; p = .022; n = 599; CFI = .999; NFI = .998; TLI = .997; SRMR = .030; RMSEA = .056; P(rmsea) ≤ .05) = .339; 90% CI ].019; .095[) the same was observed in the Portuguese data (χ2(4) = 6.726; p < .001; n = 572; CFI = 1.000; NFI = 1.000; TLI = 1.000; SRMR = .019; RMSEA = .035; P(rmsea) ≤ .05) = .661; 90% CI ].000; .078[).

Fig 1

SIJS latent structure (five items).

Factor loadings for each item are shown (Brazil | Portugal). RReversed items.

Convergent validity evidence. The AVE value for the job satisfaction factor was .58, which is indicative of good convergent validity evidence for this factor. Good AVE values were also observed for each country (AVE = .54; AVE = .65). Globally, these results indicate good convergent validity evidence in terms of internal structure for SIJS. In other words, the items contained in the job satisfaction factor relate to each other.

Reliability of the scores

The reliability of the scores was assessed through internal consistency estimates which revealed to be globally very good values for both samples (Table 4). The values of the internal consistency estimates were very good (≥ .84) for the Brazilian data, with the CR obtaining the highest value. In the case of the Portuguese sample all internal consistency estimates were also very good (≥ .88). Such results are indicative of very good evidence in terms of reliability, suggesting that items’ scores are homogenously crowded together around the job satisfaction latent variable.

Table 4

Internal consistency estimates for both samples.

SIJS dimension	SIJS—5 items
	Brazil			Portugal			Total
	α	ω	CR	α	ω	CR	α	ω	CR
Job Satisfaction	.86	.84	.87	.90	.88	.90	.88	.86	.88

Measurement invariance

The measurement invariance among countries and sexes was tested through a group of nested models with increasing constraints (Table 5). Scalar invariance was achieved among countries considering the ΔCFI < .010 criterion [113] while using the Δχ2 criterion [114] only metric invariance was achieved. Regarding the measurement invariance among sexes in the Brazilian sample, full uniqueness measurement invariance was obtained using the ΔCFI < .010 criterion, while the Δχ2 criterion only supported metric invariance. The Portuguese sample showed that full uniqueness measurement invariance was supported by both criteria (i.e. ΔCFI < .010 and Δχ2). That is to say that the SIJS measures job satisfaction in the same way across countries and sexes, allowing to interpret the differences in terms of group differences in the job satisfaction construct.

Table 5

Measurement invariance between countries.

Countries
Model	χ2	df	CFI scaled	Δχ2	ΔCFI scaled
Configural	106.14	10	.981	-	-
Metric	113.33	14	.985	7.820ns	-.004
Scalar	149.57	28	.982	49.056***	.003
Full uniqueness	333.08	33	.959	233.143***	.023
Latent means	346.43	34	.975	2.125ns	-.016
Sex Brazil
Configural	76.147	10	.955	-	-
Metric	82.685	14	.962	6.021ns	-.007
Scalar	92.337	28	.963	14.871ns	-.001
Full uniqueness	102.484	33	.962	11.675*	.001
Latent means	107.401	34	.975	0.863ns	-.013
Sex Portugal
Configural	27.010	10	.993	-	-
Metric	27.468	14	.994	1.260ns	.001
Scalar	41.714	28	.994	20.350ns	.000
Full uniqueness	45.361	33	.994	5.572ns	.000
Latent means	50.736	34	.997	0.863ns	-.003

nsp > .05

*p ≤ .05

***p < .001.

Validity evidence based on relations to other variables

Three additional measures were used to investigate the validity evidence based on relations to other variables, particularly the nomological validity both in terms of convergent and discriminant evidence [49]. Using the OLBI second-order factor model which revealed an acceptable fit to the data (χ2(86) = 1,039.970; p < .001; n = 1,109; CFI = .984; NFI = .983; TLI = .981; SRMR = .065; RMSEA = .100; P(rmsea) ≤ .05) < .001; 90% CI ].095; .106[) after adding three pairs of correlations between items’ residuals (i.e. Item1—Item7; Item10−Item12; and Item4−Item16). Such correlations were added after the analysis of modification indices because the items in each pair belong to the same factor. Since it seems plausible that residuals of the same factor are correlated [115]. The UWES-9 second-order model also obtained an acceptable fit (χ2(25) = 437.948; p < .001; n = 1,100; CFI = .998; NFI = .998; TLI = .997; SRMR = .041; RMSEA = .123; P(rmsea) ≤ .05) < .001; 90% CI ].113; .133[) after adding a constraint to the variance of one of the first-order factors–Dedication–fixing it to 0.001 to avoid negative variance. The QWLS-16 presented good fit (χ2(97) = 931.475; p < .001; n = 1,108; CFI = .991; NFI = .990; TLI = .988; SRMR = .064; RMSEA = .088; P(rmsea) ≤ .05) < .001; 90% CI ].083; .093[). Both in terms of first- and second-order factors the reliability of the scores in terms of internal consistency for QWLS, UWES-9 and OLBI presented acceptable to very good evidence (Table 6).

Table 6

UWES-9, OLBI, and QLWS internal consistency estimates.

Internal consistency estimates
Estimate	UWES-92L	OLBI2L	QWLS2L
ω	Vigor = .94	Disengagement = .68	Health & Safety = .43
			Economic & Family = .64
	Dedication = .91		Social = .46
			Esteem = .77
		Exhaustion = .86	Actualization = .91
	Absorption = .88		Knowledge = .90
			Aesthetics = .83
ωpartial L1	.97	.93	.96
ωL1	.93	.86	.92
ωL2	.96	.92	.96

Second-order latent version.

In nomological terms, the convergent evidence was good, since both the model of quality of work life correlating with job satisfaction (χ2(180) = 1,126.605; p < .001; n = 1,108; CFI = .993; NFI = .992; TLI = .992; SRMR = .054; RMSEA = .069; P(rmsea) ≤ .05) < .001, 90% CI ].065; .073[) and the model of work engagement correlating with job satisfaction (χ2(73) = 669.163; p < .001; n = 1,100; CFI = .998; NFI = .997; TLI = .997; SRMR = .038; RMSEA = .086; P(rmsea) ≤ .05) < .001, 90% CI ].080; .092[) presented significant and strong correlations (r = 0.82; r = 0.86, respectively).

The model where job satisfaction was correlated with burnout showed good fit to the data (χ2(165) = 1,558.286; p < .001; n = 1,109; CFI = .988; NFI = .987; TLI = .986; SRMR = .061; RMSEA = .087; P(rmsea) ≤ .05) < .001, 90% CI ].083; .091[). The hypothetical model indicates a large and negative correlation between job satisfaction and burnout (r = -0.90; p < .001). Such result suggests good discriminant evidence in nomological terms. Altogether, the SIJS’ validity evidence based on relations to other variables was satisfactory, showing associations in the expected directions in other associated measures.

Job satisfaction comparisons among sex and country

The job satisfaction latent means were compared through the structural equation modeling framework. The comparison among countries did not reveal statistically significant differences (Δχ2(1) = 2.125; p = 0.145; d = 0.086). Also, the comparison among sexes did not reveal statistically significant differences (Δχ2(1) = 0.126 p = 0.723; d = -0.003). For potential comparison with other studies, job satisfaction means, standard deviations, and quartiles of the raw scores per country and sex are provided (Table 7).

Table 7

Job satisfaction raw scores: Quartiles, means, and standard deviations for countries and sexes.

Brazil
Female (n = 355)					Male (n = 173)					Total (n = 599)*
M	SD	P25	P50	P75	M	SD	P25	P50	P75	M	SD	P25	P50	P75
3.58	0.95	3.00	3.80	4.20	3.68	0.87	3.00	3.80	4.20	3.60	0.92	3.00	3.80	4.20
Portugal
Female (n = 323)					Male (n = 191)					Total (n = 572)*
3.68	0.91	3.20	3.80	4.40	3.61	0.89	3.00	3.80	4.20	3.66	0.90	3.00	3.80	4.40

*Some subjects did not inform their Sex.

Discussion

The SIJS presented a promising fit to data from multi-occupational workers from Portugal and Brazil. The minimum λ value was high, and the instrument presented acceptable convergent validity evidence. Globally, the SIJS presented acceptable values of internal consistency, with scalar invariance obtained among the Portuguese and Brazilian samples and sexes. This is in line with the good psychometric properties observed in different countries such as the USA [82], Italy [116], New Zealand [47] or the UAE [52]. Due to its good validity evidence, the SIJS has been reported as a commonly used instrument of job satisfaction in studies published in top journals [117].

The dimensionality of the SIJS was successfully confirmed (H1), presenting an excellent fit to the data. However, a modification was added to the structure: the correlation among items’ 3 and 5 residuals. Such correlation can be justified since both items belong to the same latent factor, and additionally both items are reversed. Interestingly, a study by Jabeen et al. [52] revealed problems with this pair of reversed items (i.e. unexpected factor loadings), which lead the authors to remove those items from the analysis. The discussion about the inclusion of reversed items reveals that there are specific trade-offs that researchers should be aware of [118-120].

The reliability evidence confirmed to be good (H2). Such findings are corroborated by other studies where the SIJS revealed adequate to good reliability evidence in terms of internal consistency [45-48]. Our study’s internal consistency estimates values were good in the three different used estimates (i.e. α, ω and CR).

Regarding Hypothesis 3, it was also confirmed. A minimum of scalar invariance was achieved both for countries and sexes. As far as we know, none of the other studies that used the SIJS tested the measurement invariance of this instrument. As such, this finding is an interesting contribution to the literature. Strong measurement (scalar) invariance is an essential condition to directly compare an instrument score among groups [121].

The validity evidence based on the relation to other variables revealed to be satisfactory, since the tested full structural equation models presented paths that occurred as expected, confirming Hypothesis 4. More specifically, job satisfaction was associated with burnout (H4.1). The burnout association with overall job satisfaction is well known, albeit some argue that they can have bidirectional relations [122]. Others [e.g., 123] reported a strong negative correlation between emotional exhaustion (burnout’s core dimension) and job satisfaction. Emotional exhaustion has been also observed to explain specific types of job satisfaction (e.g. satisfaction with the communication) [124].

Job satisfaction also correlated with work engagement (H4.2) a finding that has parallel in other studies [125,126]. Conversely in other studies, some authors tested work engagement as a predictor of job satisfaction, finding also a positive relationship between both variables [127,128].

The relation with quality of work-life was successfully verified (H4.3) similarly to previous research which showed that quality of work-life predicts job satisfaction [12,52]. The correlation magnitude is large which is somehow expected, since the used quality of work-life measure approaches the perception of needs satisfaction related to the supervisory behavior, job requirements, and work environment which–when occurs–conducts to job satisfaction [12].

Portugal and Brazil did not present statistically significant differences regarding job satisfaction (H5.1). Since the sample was mainly constituted by workers with higher education, such finding was plausible. Others [e.g., 129], found that job satisfaction was rather similar across various countries among the higher education graduates. Countries with different cultural backgrounds can display different perceptions of job satisfaction, and such potential differences should be controlled before making comparisons [130]. Something that—regarding the Brazil and Portugal comparison—was guaranteed both by the procedure of adaptation of the items which considered both countries’ cultural specificities and through the verification of the scalar invariance of the SIJS. Only after such evidence, direct comparisons of the scores can be established [131].

The hypothesis tested in H5.2 was not supported by the data. In Brazil, a previous study among health-care workers did not find statistically significant differences among genders [132]. In Portugal there are mixed findings, with some studies not finding statistically significant differences among men and women [133,134] while others reported that men had higher job satisfaction than women[135]. Past studies in other countries found differences between sexes both with males having greater job satisfaction after controlling for similar job ranks [136], or the reverse effect [137], or even the absence of differences between women and men [138,139]. Kaiser [135] found that job satisfaction differences among sexes are expected to be insignificant within a higher job status segment. This seems a plausible explanation since the collected samples in the present paper have high frequencies of higher academic levels, which are expected to be associated with higher job status. Zou [140] reported that when work orientations are taken into account, job satisfaction differences among women and men are eliminated. Also, there is evidence that women and men are similar in the crossover and spillover dynamics regarding job satisfaction, job insecurity, own family and partner’s family life satisfaction [141]. Various explanations exist for such absence, or even the possible greater job satisfaction perceived by women [142] which would be seen as a job satisfaction paradox [135]. Nulling effect of covariate variables of sex, like education, age and paygrade can explain the absence of differences [143],. perhaps due to the evolution of the demographics and women’s attitudes towards work [77]. Such findings can be also explained by the less apparent differences among sexes in terms of job roles, qualifications, earnings and other indicators [144,145]. Such a fading-out process might be explained by appropriate labor market interventions by the governing institutions [135]. Hodson [142] suggested that women can report equal or greater job satisfaction than men, despite women’s worst job conditions because of two possible processes that may operate in conjunction. First, women can use different comparison groups. Second, men can be more inclined to report situations of dissatisfaction than women.

Conclusions

The short measure (i.e. SIJS) can be taken as a good measure, since it has been used successfully in several studies [e.g. 6], and it also demonstrated promising validity in this study. Despite being shorter than several other instruments versions that measure job satisfaction [146] the SIJS revealed very promising validity evidence, both based in the internal structure as also based on the relations to other variables. When a researcher wants to use several measures, the length of the selected instruments is a fundamental issue to avoid respondents’ fatigue. As such, having short scales that show good validity evidence is crucial since lengthy surveys can lead to fatigue and consequently have increased insufficient effort responding [147,148].

Some limitations of the current study should be mentioned. The sample was a convenience sample (i.e. non-probabilistic). Desirably in future studies probability sampling techniques should be used. Additionally, the two current samples do not have the same proportions of occupational groups, sex and academic level in each country. And since, the conclusions regarding invariance should be analyzed with caution. In this cross-sectional study, no information can be extracted regarding the temporal stability of the SIJS structure. In future research, researchers might address this issue by testing longitudinal measurement invariance [149]. Research with other samples (i.e. more specific occupational groups) is expected to occur in future studies, as also in other countries/cultures, to approach the transcultural invariance robustness of the SIJS.

The dataset used.

(CSV)

Click here for additional data file.

The dataset codebook.

(CSV)

Click here for additional data file.

26 Nov 2019

PONE-D-19-28151

Short Index of Job Satisfaction: Validity Evidence from Portugal and Brazil

PLOS ONE

Dear Dr Marôco,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Your paper has been commented by three reviewers that are acknowledged academics in the topic addressed in the study. You will find that two of them see potential in the paper but recommend a considerable set of revisions from you, and one of the reviewers advised against its consideration for being published in PLOS ONE. As, personally, I see this study as valuable and probably useful for occupational researchers worldwide, I decided to give you the opportunity to revise and resubmit an improved version of the manuscript, as long as all the received comments be properly addressed, the changes/amendments done properly detailed and explained, and the contents considerably updated, in accordance to the suggestions provided by our reviewers. Please also consider to put a bigger effort on the presentation of the paper for non-expert readers.

We would appreciate receiving your revised manuscript by Jan 10 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Sergio A. Useche, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2.  Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

3. Thank you for including your ethics statement: "Human Subject Research (involving human participants)

Brazil data:

Name: Committee of Ethics in Research with Human Beings (CEP) of the Faculty of Philosophy, Sciences and Letters of Ribeirão Preto, University of São Paulo, through “platform Brazil” of CONEP (National Commission of Ethics in Research).

Approval number: CAAE no. 33301214.2.0000.5407

Form of consent obtained: written

Portugal data:

Name: Ethics Committee of the Faculty of Psychology and Education Sciences of the University of Porto (on

Statement indicating approval of this research: "A Comissão de Ética da Faculdade de Psicologia e de Ciências da Educação da

Universidade do Porto, tendo analisado o projeto de investigação de Jorge Fernando Pereira Sinval, intitulado “Saúde Ocupacional: metanálise, modelos e instrumentos”, orientado na FPCEUP pela Prof.ª Cristina Queirós, considera que ele respeita todos os princípios éticos e normas deontológicas da investigação pelo que emite um parecer favorável à sua realização."

Form of consent obtained: written"

Please amend your current ethics statement to confirm that your named institutional review boards / ethics committees specifically approved this study.

Once you have amended this/these statement(s) in the Methods section of the manuscript, please add the same text to the “Ethics Statement” field of the submission form (via “Edit Submission”).

For additional information about PLOS ONE ethical requirements for human subjects research, please refer to http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research

4.  Thank you for stating the following in the Acknowledgments Section of your manuscript:

The authors would like to thank the Portuguese national occupational health program of the

Directorate-General of Health (DGS). This work was produced with the support of INCD funded by

FCT and FEDER under the project 22153-01/SAICT/2016. We thank to Dr. Edwin A. Locke (Robert

H. Smith School of Business [Emeritus], University of Maryland, College Park, MD, United States of

America) for helpful comments.

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

Authors: JS and JM

Grant number: UID/PSI/04810/2019

Full name of the funder: Portuguese Science Foundation

URL of funder website: https://www.fct.pt/

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The paper looks more like an exercise in statistical analysis as the scale is only a five item scale and it should work as a uni-dimensional scale. Thus the bigger question is what can we learn for the entire exercise.

Reviewer #2: Main claims of the paper

The authors aim to validate the SIJS in a new version in Portuguese in two samples in Brazil and Portugal. This has been done via convenience samples in both countries.

The manuscript is well organized and has a clear presentation of the background and the ideas behind.

Introduction, hypotheses

The introduction is clear and thorough and the development of the hypotheses is comprehensible. Regarding the hypotheses for sex and countries, these should be restated. Both hypotheses in H5 are formulated as null-hypotheses which cannot be proven: “Altogether, similar job satisfaction latent mean levels are expected to be observed (Hypothesis 5) among different countries (Hypothesis 5.1) and different sexes (Hypothesis 5.2).”

This point also can be seen in the results and discussion, see: “Portugal and Brazil showed no differences regarding job satisfaction, which supported H5.1.”. Again, “no differences” cannot be proven, so this topic should be considered. The same as for sexes is for H 5.2 in the comparison between the countries and here the authors themselves write about mixed results (p. 21), which again points to the principal problem of stating and trying to prove a null-hypothesis.

Methods

The data set does not provide information for occupation, school level, socioeconomic level. If PLOS needs the complete data set then this information should be added.

The description of the data set not satisfying for re-analyses as e.g. the scale information is lacking (which is understandable as this is not possible in an xls-file in comparison to an SPSS-file).

The internal consistency values for the instruments for the previous versions should be included in the methods section.

Data and analyses

As minor point, in “Table 3. Sample 1 and 2 Descriptive Statistics.” It would be helpful to have at least an abbreviation of the item content.

Discussion

The discussion follows the argumentation line, refers to the literature and research findings and discusses the results in this context.

Minor points

The language should be checked. E.g. the following sentence is not fully clear: „The meaning attached to work seems that not differs significantly among countries (Judge et al., 2001), as so it expected that job satisfaction differences between countries are not so great or inexistent.“

Some other examples.

„As such, each of these three dimensions was three items“

Reviewer #3: The main aim of this paper was to adapt the Portuguese version for Brazil and Portugal of the Short Index of Job Satisfaction (SIJS) in two different population in a cross-sectional manner. The authors have tested the convergent and discriminant validity and sex differences in SIJS in both countries. The results seem to indicate a good psychometric properties but some concerns remained before publishing the article, which are listed below.

1. Readers may feel difficult to understand what is SIJS in the abstract. Please add some more information about it for readability.

2. The rationale for analyzing sex differences seems weak to persuade the reviewer.

3. I don’t see how the sample was selected. How many was recruited and what was the response rate?

4. Table 1 doesn’t show the statistical differences between Portugal and Brazilian sample.

5. Introduction is well-written but the rationale for selecting SIJS among other measures is weak.

6. Table 1: Please explain socioeconomic status under table 1.

7. Was translation of English version of SIJS to Portuguese was back translated to English to check the proper meaning?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes: Akinori Nakata

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

16 Dec 2019

REVISION NOTES

“Short Index of Job Satisfaction: Validity Evidence from Portugal and Brazil”

MS# PONE-D-19-28151

Thank you for reviewers’ comments. We found the comments very helpful and made a sincere effort to revise the paper based on the feedback from the reviewers.

****************************************************************************

Editor

****************************************************************************

Editor’s comments

Your paper has been commented by three reviewers that are acknowledged academics in the topic addressed in the study. You will find that two of them see potential in the paper but recommend a considerable set of revisions from you, and one of the reviewers advised against its consideration for being published in PLOS ONE. As, personally, I see this study as valuable and probably useful for occupational researchers worldwide, I decided to give you the opportunity to revise and resubmit an improved version of the manuscript, as long as all the received comments be properly addressed, the changes/amendments done properly detailed and explained, and the contents considerably updated, in accordance to the suggestions provided by our reviewers. Please also consider to put a bigger effort on the presentation of the paper for non-expert readers.

----------------------

Authors’ response

We tried to make the paper more presentable for non-expert readers.

****************************************************************************

Reviewer 1

****************************************************************************

Reviewer 1’s comments

The paper looks more like an exercise in statistical analysis as the scale is only a five item scale and it should work as a uni-dimensional scale. Thus the bigger question is what can we learn for the entire exercise.

----------------------

Authors’ response

This is a psychometric manuscript, where the SIJS Portuguese version for Portugal and Brazil is adapted and deeply evaluated in its psychometric properties. It is a useful measure for various reasons, it is one of the most measured constructs in the organizational contexts, this particular measure (i.e. SIJS) is widely used, it is free for use, there is no published study evaluating the psychometric properties of this instrument, it showed strong psychometric properties (e.g. measurement invariance among sexes and countries).

****************************************************************************

Reviewer 2

****************************************************************************

Reviewer 2’s comments

1 - The authors aim to validate the SIJS in a new version in Portuguese in two samples in Brazil and Portugal. This has been done via convenience samples in both countries. The manuscript is well organized and has a clear presentation of the background and the ideas behind.

2 - Introduction, hypotheses

The introduction is clear and thorough and the development of the hypotheses is comprehensible. Regarding the hypotheses for sex and countries, these should be restated. Both hypotheses in H5 are formulated as null-hypotheses which cannot be proven: “Altogether, similar job satisfaction latent mean levels are expected to be observed (Hypothesis 5) among different countries (Hypothesis 5.1) and different sexes (Hypothesis 5.2).”

This point also can be seen in the results and discussion, see: “Portugal and Brazil showed no differences regarding job satisfaction, which supported H5.1.”. Again, “no differences” cannot be proven, so this topic should be considered. The same as for sexes is for H 5.2 in the comparison between the countries and here the authors themselves write about mixed results (p. 21), which again points to the principal problem of stating and trying to prove a null-hypothesis.

3- Methods

The data set does not provide information for occupation, school level, socioeconomic level. If PLOS needs the complete data set then this information should be added.

The description of the data set not satisfying for re-analyses as e.g. the scale information is lacking (which is understandable as this is not possible in an xls-file in comparison to an SPSS-file).

The internal consistency values for the instruments for the previous versions should be included in the methods section.

4 - Data and analyses

As minor point, in “Table 3. Sample 1 and 2 Descriptive Statistics.” It would be helpful to have at least an abbreviation of the item content.

5 - Discussion

The discussion follows the argumentation line, refers to the literature and research findings and discusses the results in this context.

6 - Minor points

The language should be checked. E.g. the following sentence is not fully clear: „The meaning attached to work seems that not differs significantly among countries (Judge et al., 2001), as so it expected that job satisfaction differences between countries are not so great or inexistent.“

Some other examples.

„As such, each of these three dimensions was three items“

----------------------

Authors’ response

1 - Thank you.

2 - The hypotheses were reformulated as per the reviewer’s suggestion.

3 - The occupational group, academic level, and socioeconomic status were added to the dataset (i.e. S1_Dataset.csv).

Unfortunately, csv files do not allow to added labels to variables or values.

The internal consistency values for the instruments previous versions were added.

4 -The items’ content is presented in Table 2. However, if the reviewer finds it strictly necessary, the items content can be repeated in this table.

5 - Thank you.

6 - We carefully reviewed the writing of the manuscript.

****************************************************************************

Reviewer 3

****************************************************************************

Reviewer 3’s comments

The main aim of this paper was to adapt the Portuguese version for Brazil and Portugal of the Short Index of Job Satisfaction (SIJS) in two different population in a cross-sectional manner. The authors have tested the convergent and discriminant validity and sex differences in SIJS in both countries. The results seem to indicate a good psychometric properties but some concerns remained before publishing the article, which are listed below.

1. Readers may feel difficult to understand what is SIJS in the abstract. Please add some more information about it for readability.

2. The rationale for analyzing sex differences seems weak to persuade the reviewer.

3. I don’t see how the sample was selected. How many was recruited and what was the response rate?

4. Table 1 doesn’t show the statistical differences between Portugal and Brazilian sample.

5. Introduction is well-written but the rationale for selecting SIJS among other measures is weak.

6. Table 1: Please explain socioeconomic status under table 1.

7. Was translation of English version of SIJS to Portuguese was back translated to English to check the proper meaning?

----------------------

Authors’ response

Thank you.

1. Information about the SIJS was added to the abstract.

2. The rationale for analyzing sex differences was improved.

3. Information about how the sample was obtained was added, and the response rates from the participants in this study (see Procedures section).

4. The statistical tests of the frequencies and means comparisons were added.

5. The SIJS is freely available, it’s a short easy to apply measure, and it shows good psychometrics in several countries. This information was added to the introduction.

6. The socioeconomic status was evaluated according to the the “Critério Brasil” (Associação Brasileira de Empresas de Pesquisa 2012) which consists in a standard socioeconomic status criteria.

7. Yes, it was back-translated as suggested by the ITC guidelines (International Test Commission 2018) (see subsection “Short Index of Job Satisfaction (SIJS)” inside the section “Constructs and measures”).

References

Associação Brasileira de Empresas de Pesquisa. (2012). Critério de classificação econômica do Brasil [Criteria for economic classification in Brazil]. http://www.abep.org/

International Test Commission. (2018). ITC Guidelines for Translating and Adapting Tests (second edition). International Journal of Testing, 18(2), 101–134. https://doi.org/10.1080/15305058.2017.1398166

24 Feb 2020

PONE-D-19-28151R1

Short Index of Job Satisfaction: Validity Evidence from Portugal and Brazil

PLOS ONE

Dear Dr Marôco,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The revised version of your paper has been scientifically judged by three reviewers. Overall, they all agree on the potential of the paper and its value for occupational and organizational research. Furthermore, two of them consider that their previous comments have been adequately responded, and only one of the reviewers asked for some amendments, most of them minor. Please carefully check and amend/clarify these points, including the response to the reviewer' queries and the detail of the changes made in your rebuttal letter.

We would appreciate receiving your revised manuscript by Apr 09 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Sergio A. Useche, Ph.D.

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: (No Response)

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have carried out the necessary revisions and the paper looks better than the original thus more acceptable.

Reviewer #2: Main claims of the paper

1 The authors aim to validate the SIJS in a new version in Portuguese in two samples in Brazil and Portugal. This has been done via convenience samples in both countries.

The manuscript is well organized and has a clear presentation of the background and the ideas behind.

Answer Authors

1 - Thank you.

2 Introduction, hypotheses

The introduction is clear and thorough and the development of the hypotheses is comprehensible. Regarding the hypotheses for sex and countries, these should be restated. Both hypotheses in H5 are formulated as null-hypotheses which cannot be proven: “Altogether, similar job satisfaction latent mean levels are expected to be observed (Hypothesis 5) among different countries (Hypothesis 5.1) and different sexes (Hypothesis 5.2).”

This point also can be seen in the results and discussion, see: “Portugal and Brazil showed no differences regarding job satisfaction, which supported H5.1.”. Again, “no differences” cannot be proven, so this topic should be considered. The same as for sexes is for H 5.2 in the comparison between the countries and here the authors themselves write about mixed results (p. 21), which again points to the principal problem of stating and trying to prove a null-hypothesis.

Answer Authors

2 - The hypotheses were reformulated as per the reviewer’s suggestion.

Reviewer

The hypotheses H5 are now stated in a correct way, principally. The authors now rewrote the argumentation lines before which is a bit contrary to the hypotheses H5.1 and H5.2. It is not clear why the main difference should be the case as the cited references go into the direction of a moderation effect. Also, it is not clear why there should be differences in countries. There are no references except [5] about that. As the main message at page 6 is that the differences between men and women rely on moderating variables the corresponding hypotheses are not fitting. In a whole, these hypotheses now look (at a second reading) incongruous. Possibly this part should not be included as the hypotheses imply a whole research question and the main point of the study wants simply to make a validation study of the SIJS.

This point also regards the discussion.

Reviewer

2.a, now additionally

After reading the introduction again in the light of the other comments I have to admit that this critique is important: “Reviewer 3. 5. Introduction is well-written but the rationale for selecting SIJS among other measures is weak. „

The authors write „These short multi-item measures maintain the overall perspective of the measure with the advantage of obtaining the latent measure through the necessary multiple items. [..] One of these short measures is the SIJS which has some advantages over other measures. It is freely available, has shown good psychometric properties and has been used in different cultures [41-44].“

This text sounds like as there was a presentation of the other instruments which has not been done. As there is a lot of literature available about job satisfaction in general but few about single-item measurements or facet-items or facet scales measurements of job satisfaction this aspect should be presented more clearly. To present the SIJS and argue that this has advantages implies that the presented arguments can be validated a bit better in comparison to other (here not presented) scales. This is important especially for a scientific text as it always should rely on transparency.

3 Methods

The data set does not provide information for occupation, school level, socioeconomic level. If PLOS needs the complete data set then this information should be added.

The description of the data set is not satisfying for re-analyses as e.g. the scale information is lacking (which is understandable as this is not possible in an xls-file in comparison to an SPSS-file).

The internal consistency values for the instruments for the previous versions should be included in the methods section.

Answer Authors

3 - The occupational group, academic level, and socioeconomic status were added to the dataset (i.e. S1_Dataset.csv).

Unfortunately, csv files do not allow to added labels to variables or values.

The internal consistency values for the instruments previous versions were added.

Reviewer

Ok, done partly. Besides a .csv a description of the labelling could be provide. It is not clear, e.g., whether the authors used the scale anchors between the marking points 1 and 5: “Subjects are asked to respond to each item by checking a fivepoint scale ranging from 1 - “Strongly Disagree” to 5 - “Strongly Agree” (Table 2), two of those items are reversed.”

So e.g., have the points 2, 3, 4 also been labelled for the respondents?

There should be a description of the data set. Typically this is a codebook. In SPSS this is done partly automatically via the labels etc.

4 Data and analyses

As minor point, in “Table 3. Sample 1 and 2 Descriptive Statistics.” It would be helpful to have at least an abbreviation of the item content.

Answer Authors

4 -The items’ content is presented in Table 2. However, if the reviewer finds it strictly necessary, the items content can be repeated in this table.

Reviewer

I still find it much more helpful for a reader to find abbreviations like “fairly satisfied” in Table 3 in comparison to “Item 1”.

5 Discussion

The discussion follows the argumentation line, refers to the literature and research findings and discusses the results in this context.

Answer Authors

5 - Thank you.

6 Minor points

The language should be checked. E.g. the following sentence is not fully clear: „The meaning attached to work seems that not differs significantly among countries (Judge et al., 2001), as so it expected that job satisfaction differences between countries are not so great or inexistent.“

Some other examples.

„As such, each of these three dimensions was three items“

Answer Authors

6 - We carefully reviewed the writing of the manuscript.

Reviewer

Ok.

Reviewer #3: I think the authors have made necessary revision. From next time, I recommend authors to indicate "where in the main text the revision was made."

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Paul Jimenez

Reviewer #3: Yes: Akinori Nakata

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

19 Mar 2020

Thank you for Reviewer #2 comments. We found the comments very helpful and made a sincere effort to revise the paper based on the received feedback.

####################################################

Reviewer 2

####################################################

2 Introduction, hypotheses

The hypotheses H5 are now stated in a correct way, principally. The authors now rewrote the argumentation lines before which is a bit contrary to the hypotheses H5.1 and H5.2. It is not clear why the main difference should be the case as the cited references go into the direction of a moderation effect. Also, it is not clear why there should be differences in countries. There are no references except [5] about that. As the main message at page 6 is that the differences between men and women rely on moderating variables the corresponding hypotheses are not fitting. In a whole, these hypotheses now look (at a second reading) incongruous. Possibly this part should not be included as the hypotheses imply a whole research question and the main point of the study wants simply to make a validation study of the SIJS.

This point also regards the discussion.

*******************************************************************

*******************************************************************

####################################################

Authors’ response

####################################################

We rewrote the hypothesis as per the referee's request. Now the research hypotheses are in line with the Statistical Alternative Hypothesis. Additionally, the rationale behind the hypotheses was changed to make the literature sound more logical regarding the established hypotheses. However, our idea of quoting the literature was not to support a particular form of the research hypotheses, but rather to present what other researchers have found on the subject.

*******************************************************************

*******************************************************************

####################################################

Reviewer 2

####################################################

2.a, now additionally

After reading the introduction again in the light of the other comments I have to admit that this critique is important:

“Reviewer 3. 5. Introduction is well-written but the rationale for selecting SIJS among other measures is weak. „

The authors write „These short multi-item measures maintain the overall perspective of the measure with the advantage of obtaining the latent measure through the necessary multiple items. [..] One of these short measures is the SIJS which has some advantages over other measures. It is freely available, has shown good psychometric properties and has been used in different cultures [41-44].“

This text sounds like as there was a presentation of the other instruments which has not been done. As there is a lot of literature available about job satisfaction in general but few about single-item measurements or facet-items or facet scales measurements of job satisfaction this aspect should be presented more clearly. To present the SIJS and argue that this has advantages implies that the presented arguments can be validated a bit better in comparison to other (here not presented) scales. This is important especially for a scientific text as it always should rely on transparency.

*******************************************************************

*******************************************************************

####################################################

Authors’ response

####################################################

We give information about other instruments: freely available or not, length, dimensions of the referred scales (see Introduction; 3rd and 4th paragraphs).

*******************************************************************

*******************************************************************

####################################################

Reviewer 2

####################################################

3 Methods

Ok, done partly. Besides a .csv a description of the labelling could be provide. It is not clear, e.g., whether the authors used the scale anchors between the marking points 1 and 5: “Subjects are asked to respond to each item by checking a fivepoint

scale ranging from 1 - “Strongly Disagree” to 5 - “Strongly Agree” (Table 2), two of those items are reversed.”

So e.g., have the points 2, 3, 4 also been labelled for the respondents?

There should be a description of the data set. Typically this is a codebook. In SPSS this is done partly automatically via the labels etc.

*******************************************************************

*******************************************************************

####################################################

Authors’ response

####################################################

Yes, points 2, 3 and 4 were also labeled. That information is presented in Table 2, nevertheless, we completed that information in the text (see the Subheading “Short Index of Job Satisfaction (SIJS)”):

Subjects are asked to respond to each item by checking a five-point scale (1 - “Strongly Disagree”, 2 - “Disagree”, 3 – “Undecided”, 4 – “Agree”, 5 - “Strongly Agree”) two of those items are reversed (Table 2).

The CSV file “S2_Dataset.csv” was added. All the variables have a description, the inverted items have that information of their description.

*******************************************************************

*******************************************************************

####################################################

Reviewer 2

####################################################

4 Data and analyses

I still find it much more helpful for a reader to find abbreviations like “fairly satisfied” in Table 3 in comparison to “Item 1”.

*******************************************************************

*******************************************************************

####################################################

Authors’ response

####################################################

We added a small item vignette to each of them. We also emphasized that for items 3 and 5 the presented descriptive statistics (Table 3) are presented for the recoded scores:

“Note. R – reversed items; the descriptive statistics for such items refer to the recoded scores.”

*******************************************************************

*******************************************************************

25 Mar 2020

Short Index of Job Satisfaction: Validity Evidence from Portugal and Brazil

PONE-D-19-28151R2

Dear Dr. Marôco,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Sergio A. Useche, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Thanks for submitting the minor changes requested by one of the reviewers. After a careful revision of them, I am happy with the job done by the authors and their responses, and consider the article is publishable in PLOS ONE.

Reviewers' comments:

31 Mar 2020

PONE-D-19-28151R2

Short Index of Job Satisfaction: Validity Evidence from Portugal and Brazil

Dear Dr. Marôco:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Sergio A. Useche

Academic Editor

PLOS ONE