Elementary school climate through teachers' eyes: Portuguese adaptation of the Organizational Climate Description Questionnaire Revised for Elementary schools.

Strong links have been observed between professionals' occupational health and their perceived organizational climate. However, in Portugal, one of the European Union countries where teachers present higher levels of occupational stress, no measures have been found to assess perceived school climate in elementary-school teachers. This study aimed to evaluate the psychometric qualities of the Portuguese adaptation of the Organizational Climate Description Questionnaire Revised for Elementary Schools (OCDQ-RE). To test its factor structure, 687 elementary-school teachers (85.2% female, M Age = 46.15 years, SD Age = 8.88) completed the Portuguese OCDQ-RE. An additional sample of 81 participants (96.3% female, M Age = 46.21 years, SD Age = 4.82) responded at two points in time and completed external measures, ensuring test-retest reliability and validity analyses. Confirmatory factor analysis supported the hypothesized factor structure. Coefficient omegas suggested adequate internal consistency of the composites. Adequate test-retest reliability was sustained through high correlation scores between the two data collection waves. Evidence of discriminant validity against external measures was also observed. Despite the need for further studies, the results support the adequacy and reliability of the Portuguese OCDQ-RE which may be an important research and intervention resource.

Introduction

Teachers’ occupational health has emerged as one of the main topics in international research, where various studies indicate the teaching profession as highly emotionally demanding with a high risk for occupational stress and burnout (Collie et al., 2012; Roeser et al., 2013). Although the study of teachers’ occupational stress and burnout began in the 1970s (Kyriacou, 2001), with the new millennium, the working conditions for teachers have become even more difficult as a result of higher and more diverse job requirements, globalization, electronic development, and less stability in the family structure and life trajectory (Lambert et al., 2009). Thus, the literature has invested efforts in identifying and mitigating the explanatory factors that underlie teachers’ ill-health. In accordance with the Job Demands-Resources (JD-R) model (Demerouti et al., 2001), teachers’ occupational stress and burnout stem from an imbalance between perceived high job demands (e.g., workload, excessive paperwork requirements, pressure, role conflict, job insecurity, interpersonal conflicts) and the scarcity of resources to accomplish the job requirements (e.g., lack of organizational support, lack of autonomy, poor relationships with colleagues) (Kyriacou, 2001; Schaufeli & Taris, 2014). Thus, an emphasis has been placed on the design of interventions to reduce the harmful and chronic effects of teacher burnout (Iancu et al., 2018; Oliveira et al., 2021a). However, while studies have shown that teachers’ occupational ill-health may be predicted by personal (e.g., self-regulation, time management skills), interpersonal (e.g., positive interpersonal relationships), and contextual variables (e.g., organizational social support networks) (Collie et al., 2012; Schaufeli & Taris, 2014), interventions have mainly acted on an individual level (Iancu et al., 2018; Oliveira et al., 2021a, b). Hence, less research and fewer interventions have focused on contextual variables (Schaufeli & Enzmann, 1998). Despite the promising impacts of the latest interventions to promote teachers’ resources, enhance teachers’ occupational health and well-being, and prevent teacher burnout (Iancu et al., 2018; Oliveira et al., 2021a, b), there is still considerable variance that is yet to be explained (Oliveira et al., 2021a). Since teaching-specific stressors (i.e., job demands for which teachers perceive poor resources) are mainly of a social and emotional nature (McCarthy et al., 2016; Roeser et al., 2013), contextual variables may play an important role within research and practice in this field. In fact, the prosocial classroom model (Jennings & Greenberg, 2009) supports this idea, since it describes the influence of contextual elements on teachers’ social and emotional competence and well-being (e.g., perceived principal support appears to impact teachers’ goal setting ability which is a relevant social and emotional specific skill; Collie 2017).

In this scenario, organizational climate has been a primary focus of research, given the strong links between professionals’ occupational health and the perception of their own work context (Schaufeli & Taris, 2014). Within the context of teaching in particular, the organizational climate (also referred to as “school climate” when specifically addressing teachers’ organizational climate, i.e., schools; Cohen et al., 2009) has been identified as a predictor not only of burnout, but also of teacher stress, job satisfaction and teaching efficacy (Collie et al., 2012; Schaufeli & Enzmann, 1998). School climate is defined by teachers’ shared perception of the unique characteristics that compose their work environment (Hoy & Tarter, 1992), as a result of their experience of organizational norms, goals, values, interpersonal relationships, procedures and practices (Cohen et al., 2009; Hoy & Clover, 1986). School climate is, therefore, “the set of internal characteristics that distinguishes one school from another and influences the behavior of its members” (Hoy & Tarter, 1992, p. 74). A literature review on school climate has identified four inherent main spheres, namely physical, social and emotional safety (e.g., clearly communicated rules and procedures, effective conflict management, tolerance and appreciation for individual differences), teaching and learning quality (e.g., quality of instruction, professional development support, positive and supportive leadership), relationship and collaboration (e.g., positive adult-adult relationships between/among teachers, administrators, and staff, mutual support and ongoing communication) and environmental structure (e.g., adequate space and materials) (Cohen et al., 2009). Thus, school climate is mainly represented by the quality of interpersonal relationships across school employees (e.g., teacher-teacher and principal-teacher relationships; Hoy & Clover 1986; Hoy & Tarter, 1992), where the leaders play a main role (Amorim et al., 2020; Sanchez et al., 2020).

Nonetheless, school climate is a double-edged sword, on the one hand a powerful resource to mitigate job demands, on the other a risk factor itself (Collie et al., 2012). Owing to its strong influence on employees’ emotions and behaviors (Collie, 2017; Hoy & Clover, 1986), an open and healthy school climate can buffer socio-cultural pressures and promote teachers’ motivation, engagement, and identification with their school (Hoy & Tarter, 1992). In contrast, a closed or unhealthy school climate can negatively impact teachers’ motivation, engagement, performance, and turnover (Cohen et al., 2009; Schaufeli & Bakker, 2004), the overall quality of school activities and students’ learning and achievement (Amorim et al., 2020; Collie et al., 2012; Maxwell et al., 2017; Vos et al., 2013). Consequently, research focusing on teachers’ occupational stress assessment and mitigation should use a comprehensive and systemic approach which considers job demands and resources at both a contextual (e.g., peer and leadership perceived social support) and individual level (e.g., emotional and behavioral self-regulation strategies).

To such end, some self-report measures of organizational climate, namely the Organizational Climate Description Questionnaire Revised for Elementary Schools (OCDQ-RE; Hoy & Clover 1986) and for Secondary Schools (OCDQ-RS; Hoy et al., 1991) have been developed and studied across different countries and cultures with good psychometric properties (e.g., Sanchez et al., 2020; Vos et al., 2013; Yilmaz & Altınkurt, 2013). These measures assess the perceptions of schools’ professional personnel with regards to both the leadership team and the teachers’ behaviors (Hoy et al., 1991). Measures assessing the subjective perception of school climate are particularly relevant since the literature has shown that teachers’ stress and burnout experience are mainly influenced by their own perceptions of their environment conditions, rather than the objective features of their professional context (McCarthy et al., 2016).

Nevertheless, despite the existence of several well-established self-report measures of organizational climate, the international research on school climate still lacks methodologically robust studies which use psychometrically validated measures to assess teachers’ subjective experience of their work environment in a comprehensive approach (Cohen et al., 2009). In the particular case of Portugal, despite being one of the European Union countries where teachers present higher levels of occupational stress (European Commission, 2011; Marques-Pinto & Alvarez, 2016), to the best of our knowledge, school climate has been mostly studied with middle- and high-school teachers (e.g., Amorim et al., 2020; Castro-Silva et al., 2017), and few studies have addressed the role of contextual variables in relation to teachers’ occupational health (e.g., Costa, 2010; Lopes & Oliveira, 2020). Additionally, these studies tend to use small samples and do not present a robust methodological analysis of the psychometric qualities of the measures used (e.g., Amorim et al., 2020; Costa, 2010). Moreover, no valid and reliable measures specifically developed or adapted to assess perceived elementary school climate in the Portuguese context have been found. Therefore, due to the apparent relevance of school climate perceptions with regards to teachers’ occupational health and performance, to further understand Portuguese elementary-school teachers’ perception of school climate may be an important resource not only to expand the knowledge on this area but also to develop culturally adapted interventions to improve the work environment of this professional group. Furthermore, global events such as the COVID-19 pandemic have created additional challenges for schools. Therefore, the development of cross-cultural studies with psychometrically robust measures which help shed light upon teachers’ perceptions of the contextual variables of their workplace may serve to extend the knowledge in this research domain, providing a more comprehensive and expanded lens. Moreover, the study of the psychometric properties of a measure in different cultures adds to the current theoretical and empirical knowledge by sustaining its cross-cultural stability and robustness and its validity in measuring the proposed construct. Hence, this study aims to investigate the psychometric qualities of the Portuguese adaptation of the OCDQ-RE regarding its factor structure, reliability, and validity. In light of prior research on this measure, the following hypothesis was established with regards to the OCDQ-RE Portuguese version expected factor-structure:

H1: The OCDQ-RE Portuguese version will present a 6-factor solution with three scales addressing leadership behaviors and three scales addressing teachers’ behaviors.

Furthermore, to address the validity of the Portuguese OCDQ-RE against external measures, some additional instruments were used. To test for discriminant validity, taking into consideration prior research on the impacts of school climate already addressed, external measures evaluating stress, well-being and burnout were selected. Considering the specific features of these variables in relation to school climate, occupational stress, social and psychological well-being, and (reduced) personal accomplishment (e.g., Bahrami et al., 2013; Cohen et al., 2009; Collie et al., 2012; Collie, 2017; Jennings & Greenberg, 2009; Polat & İskender, 2018; Skaalvik & Skaalvik, 2009) measures were used. These dimensions represent the eudaemonia stream of well-being and were chosen since they refer specifically to how individuals perceive their positive functioning in life (e.g., personal growth, positive relations with others, autonomy, environmental mastery; Keyes et al., 2008). In the same vein, personal accomplishment (a dimension of burnout; Maslach et al., 1996) was selected for its relationship with one’s perception of personal resources and ability to respond to job demands/lack of resources (Bresó et al., 2007; Schaufeli & Taris, 2014). Since no psychometrically robust measures of contextual variables adapted specifically to the Portuguese elementary school context were found, the convergent validity of the Portuguese OCDQ-RE was not assessed. Hence, in line with prior literature (e.g., Arnetz et al., 2011; Bahrami et al., 2013; Collie et al., 2012; Polat & İskender, 2018; Skaalvik & Skaalvik, 2009), the following hypotheses were established regarding the discriminant validity of the Portuguese OCDQ-RE:

H2a: Weak and negative intercorrelations between school climate and occupational stress are expected.

H2b: Weak and positive intercorrelations between school climate and social and psychological well-being, and personal accomplishment are expected.

Method

Participants

A convenience sample of 687 Portuguese elementary-school teachers (85.2% female, M = 46.15 years, SD = 8.88, range: 22 to 65 years) participated in the study. The participants had a mean of 21.99 years of teaching experience (SD = 10.08) and 86.1% were practicing in state elementary schools. Although a non-probability sampling method was used, the set of participants included teachers from all Portuguese counties, thus ensuring a national representation, and the data were organized considering the Territorial Units for Statistical Purposes (NUT II). Table 1 presents all the sociodemographic data of the sample in comparison to the Portuguese population of elementary-school teachers. To test for test-retest stability and discriminant validity, an additional convenience sample of 81 elementary-school teachers from the Lisbon Metropolitan Area (96.3% female, M = 46.21 years, SD = 4.82) participated in a 2nd phase of the study.

Table 1

Participants’ sociodemographic characteristics (percentage of the most frequent category, mean and standard deviation)

Variable	National reference(N = 29.861)		Total sample(N = 687)
	%	μ	%	M	SD
Gender (Female)	87.1		85.20
Age		47.00		46.15	8.88
Years of teaching experience		n.s.		21.99	10.08
Highest Educational Qualification (Graduation)	83.2		76.0
Type of School (State)	89.0		86.1
NUT II of residence
North	33.4		15.9
Center	20.4		23.3
Lisbon Metropolitan Area	26.6		24.5
Alentejo	7.1		19.2
Algarve	4.4		2.8
Autonomous Region of Madeira	3.4		3.6
Autonomous Region of Azores	5.2		6.1

The national reference data regarding the sociodemographic characteristics of the Portuguese population of elementary teachers was recovered from the latest statistical report of the Contemporary Portugal Database (PORDATA, 2019) and from the Teacher Profile latest report (DGEEC, 2019).

Measures

Organizational Climate Description Questionnaire Revised for Elementary Schools (OCDQ-RE)

Developed by Hoy and Clover (1986), the OCDQ-RE is a 42-item research instrument to measure elementary school climate which aims to identify the quality of teacher-teacher and teacher-principal interactions (Hoy et al., 1991). Therefore, the OCDQ-RE conceptualizes two main features of the construct: an administrative / leading level and a teacher level. For each of these two aspects, three dimensions are considered. The administrative level focuses on the principal’s behavior which may be perceived as supportive (i.e., behaviors which reflect concern for teachers through open communication, active listening and constructive feedback, e.g., “The principal listens to and accepts teachers’ suggestions.”; ⍺ = 0.95), directive (i.e., behaviors which are rigid and imply inflexible and close supervision of teachers’ work, e.g., “The principal monitors everything teachers do.”; ⍺ = 0.89), and / or restrictive (i.e., behaviors which handicap rather than facilitate teachers’ work mainly through a bureaucratic workload, routine and other demanding requirements, e.g., “Administrative paperwork is burdensome at this school.”; ⍺ = 0.80). The teacher level also encompasses three indicators which refer to teachers’ behavior being collegial (i.e., behavior which supports open and collaborative professional interactions among teachers, e.g., “Teachers help and support each other.”; ⍺ = 0.90), intimate (i.e., behavior which refers to cohesive and strong social relations among teachers, e.g., “Teachers provide strong social support for colleagues.”; ⍺ = 0.85), and / or disengaged (i.e., behavior which implies a sense of perceived lack of meaning and focus regarding professional activities which leads to time investment in non-productive group efforts and no common goals, e.g., “Faculty meetings are useless.”; ⍺ = 0.75). Each item was evaluated on a 4-point Likert scale (from 1 – Occurs rarely to 4 – Occurs very frequently).

A factor analysis computed by the original authors revealed that the six-factor structure explained 67.2% of the obtained variance. The six factors presented mainly moderate correlation values (|0.20| < r < |0.63|), good reliability coefficients and stability over time evidencing construct validity (Hoy et al., 1991). Also, the authors did not find cross-loads of items between factors. A second-order factor analysis suggested that the three indicators of the administrative level could be grouped in a general dimension referring to the degree of openness in principal-teacher relations, and the three indicators of the teaching level could also be grouped in a 2nd general dimension representing the degree of openness in teachers’ behavior. Moreover, these two general factors were presented as orthogonal.

Occupational stress

An indicator of occupational stress (adapted from Kyriacou & Sutcliffe, 1978) was used to measure occupational stress. The item (i.e., “To what extent do you consider being a teacher as a stress-generating activity?”) was evaluated on a 5-point Likert scale (from 1 – Not at all stressful to 5 – Extremely stressful).

Mental Health Continuum – Short Form (MHC-SF)

The social well-being (5 items, e.g., “In the last month, how often have you felt that you belong to a community (such as a social group, your school or your neighborhood)?”; ωT1 = 0.89, ωT2 = 0.84) and psychological well-being (6 items, e.g., “In the last month, how often have you felt confident to think or express your own ideas and opinions?”; ωT1 = 0.91, ωT2 = 0.92) scales of the MHC-SF (Keyes et al., 2008; Portuguese version by Matos et al., 2010) were used to evaluate how well individuals perceive their functioning in life. Items were evaluated on a 6-point Likert scale (from 0 – Never to 5 – Everyday).

Maslach Burnout Inventory – Educators Survey (MBI-ES)

We used the personal accomplishment (8 items, e.g., “I feel that I am positively influencing other people’s lives through my work.”; ωT1 = 0.83, ωT2 = 0.84) scale of the MBI-ES (Maslach et al., 1996; Portuguese version by Marques-Pinto et al., 2005) to measure participants’ feelings of efficacy and successful achievement in their work. This subscale is composed of positively phrased items and low scores are considered indicators of burnout symptoms. Items were evaluated on a 7-point Likert scale (from 0 – Never to 6 – Everyday).

Sociodemographic information

Sociodemographic information regarding participants’ gender, age, years of teaching experience, highest educational qualification, type of school (i.e., state or private school) and the geographic area (considering the Portuguese NUT II) in which they were working at the time of data collection, was gathered by means of a general questionnaire.

Procedures

The Deontology Committee of the Faculty of Psychology, University of Lisbon approved this study, as did the original authors of the OCDQ-RE. After the authorizations to conduct the research had been granted, the translation and linguistic adaptation of the original items were initiated, following the process proposed by Hill and Hill (2012). To ensure linguistic equivalence, the 42 items were primarily translated from English to Portuguese by two independent researchers with a master’s degree in educational psychology and fluent in both languages. During this process, to guarantee cultural adaptation, besides the translation, we also adapted the items to represent the specific features of the Portuguese elementary school context. When first developed by Hoy and Clover (1986), the OCDQ-RE’s items regarding leadership and management style only referred to the school principal’s behaviors. Since, in Portugal, elementary schools are part of school clusters (which aggregate schools from diverse teaching levels) with one common principal, they are not independently managed bodies, and the administrative and leadership roles are distributed across different individuals who compose the managing team of the organization (Ministry of Education, 2008). More specifically, the general council, the pedagogical council, the administrative council, and the principal compose the management and administration team which govern all the schools within the school cluster, regardless of the teaching level (Ministry of Education, 2008). Additionally, the roles of school coordinator, department / pedagogical coordinator, and administrative services team represent an intermediate management / administration team which are specific to the elementary-school level (Ministry of Education, 2008). These specific features were taken into consideration when adapting the items of the OCDQ-RE Portuguese version. Furthermore, the scales concerning the leadership-level behaviors described not only the principal’s behaviors but those of all the management team members, considering the specific role of each. Two other experts with a PhD in educational psychology then analyzed the translated and adapted items to create a single Portuguese version. Subsequently, this final Portuguese version was back-translated into English by a bilingual researcher with a PhD in educational psychology, and the two English versions of the OCDQ-RE (the original and the back-translated version) were compared and deemed similar, ensuring that each of the 42 items maintained their meaning (Hill & Hill, 2012). Finally, to ensure face and cultural validity of the final OCDQ-RE Portuguese version, an expert evaluation process (adapted from Escobar-Pérez & Cuervo-Martínez, 2008) was conducted with five Portuguese elementary-school teachers (M = 20.4 years of teaching experience, SD = 4.93) who were asked to read the items and rate each one of them from 1 (Does not meet the criteria) to 4 (Meets the criteria at a high level) regarding its perceived clarity (i.e., whether the item was easily understood, had adequate syntax and semantics, and was suited to teachers’ professional reality). Whenever an item was rated below 4, the experts were asked to comment on their understanding of what alterations were required. All the items were unanimously perceived as clear, adequate, and easy to understand.

The final OCDQ-RE Portuguese version, along with the sociodemographic information general questionnaire were then uploaded as an online survey using the Qualtrics platform. The online survey link, along with information regarding the study’s purpose, was launched via email and the social network groups of Portuguese elementary-school teachers and disseminated via email by FENPROF (Portuguese National Federation of Teachers). This method enabled us to contact Portuguese elementary-school teachers from all the Portuguese counties. The only eligibility criterion was that the participants had to be Portuguese elementary-school teachers, and this was the population we targeted in our contacts. Participants were included in the sample based on their voluntary decision to participate. The additional sample of 81 participants also responded to the occupational stress measure and to the aforementioned MHC-SF and MBI-ES subscales, to allow for a discriminant validity analysis. Within this sample, the data were collected online in two waves, with an interval of two months to allow for the test-retest reliability analysis. Since the data were collected online, to improve data validity a data validation protocol was considered. The following quality criteria were checked: consistency of response (i.e., age and schooling combinations were plausible); uniqueness of IP addresses (i.e., all participations had a unique IP address to exclude multiple submissions); completion time (i.e., a threshold of a 5 min response minimum was considered) and progress (i.e., only responses with 100% completion progress were considered); and use of text entry boxes (i.e., information regarding age and area of residence were collected through text entry boxes to facilitate the detection of random answers, spam, or the use of autofill software) (Aust et al., 2013; Dewitt et al., 2018). Responses which did not meet the data validation protocol criteria were considered invalid and excluded from the sample. Prior to partaking in the study, the participants had to agree with the informed consent which guaranteed voluntary participation, the possibility of dropping out at any moment and also ensured the confidentiality and anonymity of the participation. Participation in the study had a mean duration of 10 min (and around 15 min for the 81 participants in the additional sample). Despite using a non-probabilistic sampling method, it was possible to reach state and private schools across the country. The data collection process lasted three months. No compensation was offered to the participants.

Data analysis

Data analyses were performed using the following software packages designed for R environment (R Core Team, 2018): lavaan (Rosseel, 2012), MBESS (Kelley, 2022), semTools (semTools Contributors, 2015), semPlot (Epskamp, 2015) and userfriendlyscience (Peters, 2019).

A confirmatory factor analysis (CFA) with recourse to a maximum likelihood estimator (ML) was computed to test whether the six first-order factors proposed by previous authors revealed an adequate fit. This model was compared to four additional solutions [i.e., unidimensional structure, two second-order models (a traditional second-order structure with one second-order factor which includes the six first-order factors, and the second-order solution advanced by the original authors with two second-order orthogonal factors each composed of three first-order factors), and a bifactor model] to determine which of the alternatives best fit the data. Model adjustment was evaluated through the following fit indices: Chi-squared test (χ2), chi-squared/degrees of freedom (χ2/df), the comparative fit index (CFI), the Tucker-Lewis index (TLI), the standardized root mean square residual (SRMR), the root mean square error of approximation (RMSEA) with a 90% confidence interval, Akaike information criteria (AIC) and Bayesian information criteria (BIC). An adequate fit was considered for a χ2/df value below 3 (Arbuckle, 2009), CFI and TLI values close to 0.90 or above (Bentler, 1990; Bentler & Bonett, 1980), and SRMR and RMSEA values below 0.08 (Arbuckle, 2009; Hu & Bentler, 1999). As for model comparison, smaller AIC and BIC values (thus suggesting a more parsimonious solution; Arbuckle, 2009; Byrne, 2010) and the chi-square difference test (Bollen, 1989) were taken into account.

Before computing the CFA, the data were screened for missing values, outliers, univariate and multivariate normality across the 42-items, also examining the correlation matrix (Tabachnick & Fidell, 2013). The MCAR de Little test revealed a completely random distribution of the missing values [χ2 (325) = 354.533, p = .125], which had little expression (< 1%). Therefore, missing imputation did not occur, and a complete case analysis was performed (Tabachnick & Fidell, 2013). Regarding univariate normality and outliers, the analysis of the Q-Q plots evidenced that the presence of outliers did not have a marked impact on the data distribution, as the variables presented a tendency towards normal distribution (i.e., |z| < 3; Kline, 2011). The study of multivariate normality was conducted through the analysis of the inverse correlation matrix, revealing a VIF range from 1.11 to 4.05, which was considered adequate (i.e., < 5; Marôco, 2007). Lastly, the data presented a statistically significant Bartlett’s sphericity test [χ2 (861) = 12,009.05, p < .001] (Marôco, 2007), good KMO values (overall KMO = 0.92, and items’ KMO > 0.60; Reis, 2001) and an adequate mean of communalities of 0.46 (range: 0.15 to 0.77; MacCallum et al., 1999), thus ensuring good quality of the data.

An additional scale diagnosis analysis of the final factor structure of the best fit model of the OCDQ-RE Portuguese version was computed. Concerning reliability, internal consistency was assessed through coefficient omega (ω; Peters, 2014), since this is considered the coefficient which presents the more accurate correction for multidimensional scales with multiple 1st order subscales (Revelle & Zinbarg, 2009). Reliability was considered adequate for values above a minimum of 0.50, and good when scores were equal to or above 0.70 (Crutzen & Peters, 2017). Test-retest reliability was performed through the re-administration of the measures under analysis on a second occasion. Pearson correlations were computed to evaluate the association between the sets of scores. With regards to validity, discriminant validity was tested against external measures. Evidence of discriminant validity was considered for weak intercorrelations (i.e., r < .30; Kline, 2011). Lastly, we studied the association between the Portuguese OCDQ-RE scales and the sociodemographic variables through Pearson correlations.

Results

Factor structure

The initial CFA suggested that, when comparing the five solutions, the first-order structure with six factors (Model B) presented the best fit for the data (χ2 = 2214.063, p < .001, χ2/df = 2.75, CFI = 0.87, TLI = 0.86, SRMR = 0.08, RMSEA = 0.05, 90% CI [0.05, 0.06]). However, two items (i.e., item 06 and item 37) did not present a statistically significant standardized factor loading (p(>|z|) > 0.05). An analysis of the Modification Indices (MI; cut-off of > 30) was also computed. All suggested MI were carefully analyzed regarding their theoretical suitability and were only integrated if conceptual meaning was presented. Hence, the examination of the MI suggested that the model fit could improve with the following adjustments: item 39 needed to be reversed and integrate Factor 1 (MI = 174.106; item 39 refers to an autocratic management style while Factor 1 refers to a generally supportive and regardful management); items 39 and 41 (MI = 54.241) and 39 and 05 (MI = 30.675) (i.e., the three items referring to leadership style which requires the subordination of teachers), items 20 and 33 (MI = 38.006; both items assessing the degree to which teachers socialize with each other while at school), items 02 and 07 (MI = 31.783; both related to the existence of personal relationships between teachers, beyond the school space), and items 01 and 26 (MI = 31.159; both items assessing teachers’ engagement with their school and job activities) should display error covariances. Thus, a new CFA was computed for the five re-specified models (i.e., items 06 and 37 were removed and the MI were integrated in the models). Table 2 depicts the goodness-of-fit indices for the original and re-specified models. The model comparison suggested that Model B showed the best fit and complied with the recommended cut-off values (χ2 = 1883.267, p < .001, χ2/df = 2.61, CFI = 0.90, TLI = 0.90, SRMR = 0.07, RMSEA = 0.05, 90% CI [0.04, 0.05]).

Table 2

Goodness-of-fit statistics for the CFA models of the Portuguese OCDQ-RE (N = 687)

	χ2	df	χ2/df	CFI	TLI	SRMR	RMSEA	90% CI	AIC	BIC	df, Δ χ2	Model comparison
CFA for the original structures tested (42 items)
Model A	5688.505***	819	6.95	0.57	0.55	0.11	0.09	[0.09, 0.10]	67347.218	67459.993
Model B	2398.691***	804	2.98	0.86	0.85	0.08	0.05	[0.05, 0.06]	64087.404	64220.318	15, 3289.8***	Model A
Model C	2512.689***	812	3.09	0.85	0.84	0.08	0.06	[0.05, 0.06]	64185.402	64307.576	8, 114.0***	Model B
Model D	2517.763***	813	3.10	0.85	0.84	0.08	0.06	[0.05, 0.06]	64188.476	64309.307	1, 5.074*	Model C
Model E	3322.088***	819	4.06	0.78	0.77	0.15	0.07	[0.06, 0.07]	64980.801	65093.577	6, 804.32***	Model D
CFA for the re-specified models (40 items and modification indices)
Model A	4931.377***	735	6.72	0.63	0.61	0.11	0.09	[0.09, 0.09]	63223.542	63337.661
Model B	1883.267***	720	2.61	0.90	0.90	0.07	0.05	[0.04, 0.05]	60205.433	60339.689	15, 3048.1***	Model A
Model C	2026.537***	729	2.78	0.89	0.88	0.07	0.05	[0.05, 0.05]	60330.702	60452.876	9, 143.27***	Model B
Model D	2005.299***	729	2.75	0.89	0.88	0.07	0.05	[0.05, 0.05]	60309.465	60431.638	0	Model C
Model E	2819.413***	735	3.84	0.82	0.81	0.15	0.07	[0.06, 0.07]	61111.579	61225.697	6, 814.11***	Model D

χ2 = Chi-squared test; df = degrees of freedom; CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; SRMR = Standardized Root Mean Square Residual; RMSEA = Root Mean Square Error of Approximation; AIC = Akaike Information Criteria; BIC = Bayesian information criteria. Model A (unidimensional structure), Model B (six first-order factors structure), Model C (two second-order factors each with three first-order factors each), Model D (one second-order factor with six first-order factors), Model E (bifactor model). * p < .05, *** p < .001

All the standardized factor loadings of the six first-order factor structure were statistically significant (p(>|z|) < 0.001) and the loadings for the items within each factor ranged as follows: 0.41 to 0.87 on Factor 1 (Professional relationships management); 0.25 to 0.84 on Factor 2 (Pedagogical tasks management); 0.28 to 0.66 on Factor 3 (Bureaucratic tasks management); 0.43 to 0.77 on Factor 4 (Professional interactions among teachers); 0.30 to 0.82 on Factor 5 (Personal interactions among teachers); and 0.40 to 0.64 on Factor 6 (Dynamic of the teachers’ group). Figure 1 shows the final factor structure of the Portuguese version of the OCDQ-RE (a six first-order factor solution with 40 items). The final measure is presented in Table 3.

Fig. 1

Factor structure and factor loadings of the final 40-item six-factor model of the OCDQ-RE Portuguese version

Table 3

Final 40-items of the OCDQ-RE Portuguese version

Item	Raramente	Algumas vezes	Frequentemente	Quase sempre
1. Os/As professores/as concretizam o seu trabalho com energia, vigor e prazer.	1	2	3	4
2. Os amigos mais próximos dos/as professores/as são outros docentes da escola.	1	2	3	4
3.* As reuniões de professores são inúteis.	1	2	3	4
4. A minha coordenação faz tudo o que pode para ajudar os/as professores/as.	1	2	3	4
5. O/A coordenador/a da escola gere a equipa com “punho de ferro”.	1	2	3	4
7. Os/As professores/as convidam os colegas para irem à sua casa.	1	2	3	4
8.* Há uma minoria de professores/as que se opõe sempre à maioria.	1	2	3	4
9. A minha coordenação utiliza críticas construtivas.	1	2	3	4
10. O/A coordenador/a da escola verifica a presença dos/as professores/as diariamente.	1	2	3	4
11.* As tarefas rotineiras interferem com a atividade de ensino.	1	2	3	4
12. A maioria dos/as professores/as aceita as imperfeições dos colegas.	1	2	3	4
13. Os/As professores/as conhecem o contexto familiar dos colegas.	1	2	3	4
14.* Os/As professores/as exercem pressão de grupo sobre os colegas contestatários.	1	2	3	4
15. A minha coordenação explica as suas razões quando critica os/as professores/as.	1	2	3	4
16. A minha coordenação ouve e aceita as sugestões dos/as professores/as.	1	2	3	4
17. A minha coordenação agenda o trabalho dos/as professores/as.	1	2	3	4
18.* Os/As professores/as têm demasiadas exigências por parte dos órgãos de gestão.	1	2	3	4
19. Os/As professores/as ajudam-se e apoiam-se entre si.	1	2	3	4
20. Os/As professores/as divertem-se, socializando entre si, enquanto estão na escola.	1	2	3	4
21.* Os/As professores/as divagam quando intervêm nas reuniões de professores.	1	2	3	4
22. A minha coordenação zela pelo bem-estar pessoal dos/as professores/as.	1	2	3	4
23. A minha coordenação trata os/as professores/as como iguais.	1	2	3	4
24. A minha coordenação corrige os erros dos/as professores/as.	1	2	3	4
25.* O trabalho administrativo é uma sobrecarga na minha escola.	1	2	3	4
26. Os/As professores/as estão orgulhosos da sua escola.	1	2	3	4
27. Os/As professores/as organizam momentos de convívio uns com os outros.	1	2	3	4
28. A minha coordenação elogia os/as professores/as.	1	2	3	4
29. A minha coordenação faz-se compreender com facilidade.	1	2	3	4
30. A minha coordenação verifica de perto a prática docente.	1	2	3	4
31. Os serviços administrativos reduzem o trabalho burocrático dos/as professores/as.	1	2	3	4
32. Os/As novos/as professores/as são prontamente aceites pelos colegas.	1	2	3	4
33. Os/As professores/as socializam entre si regularmente.	1	2	3	4
34. A minha coordenação supervisiona de perto os/as professores/as.	1	2	3	4
35. A minha coordenação verifica os planos de aula / planificações diárias.	1	2	3	4
36.* Os/As professores/as estão sobrecarregados com trabalho.	1	2	3	4
38. Os/As professores/as fornecem um forte apoio aos colegas.	1	2	3	4
39.* O/A coordenador/a da escola é autocrata (tem poder absoluto na equipa).	1	2	3	4
40. Os/As professores/as respeitam a competência profissional dos colegas.	1	2	3	4
41. A minha coordenação monitoriza tudo o que os professores fazem.	1	2	3	4
42. A minha coordenação faz tudo o que pode para mostrar apreço pelos/as professores/as.	1	2	3	4

* Reversed items

Reliability, correlation between the composites and validity study

Table 4 presents the descriptive statistics, internal consistency and intercorrelations between the composites under study. All the factors presented adequate levels of internal consistency across the different assessment points. Moreover, scores were highly correlated between the two data collection points suggesting test-retest reliability. Associations between the composites were weak to moderate in size and statistically significant, except for the correlations among Factor 4 (Professional interactions among teachers) and Factor 1 (Professional relationships management), and Factor 5 (Personal interactions among teachers) which were strong (≥ 0.50) and statistically significant. Intercorrelations between the study measures and occupational stress, social and psychological well-being, and personal accomplishment were weak, suggesting discriminant validity (see Table 5).

Table 4

Descriptive (mean and standard deviation), reliability (ω) and association (Pearson r) measures of the six OCDQ-RE Portuguese version composites (N = 687); and test-retest reliability with reliability (ωT1 and ωT2) and correlation (r) at the two points in time for the study measure (n = 81)

Composites	M (SD)	ω [95% CI]	1.	2.	3.	4.	5.	ωT1	ωT2	rT1−T2
Factor 1	2.80 (0.80)	0.94 [0.94, 0.95]						0.90	0.91	0.76**
Factor 2	2.08 (0.57)	0.77 [0.74, 0.79]	0.11**					0.78	0.76	0.63**
Factor 3	1.77 (0.52)	0.62 [0.58, 0.67]	− 0.19**	− 0.07*				0.68	0.65	0.54**
Factor 4	2.77 (0.57)	0.78 [0.76, 0.81]	0.51**	0.09*	0.19**			0.82	0.81	0.71**
Factor 5	2.30 (0.56)	0.80 [0.78, 0.83]	0.37**	0.17**	0.12**	0.65**		0.80	0.82	0.61**
Factor 6	2.75 (0.57)	0.54 [0.48, 0.60]	− 0.25***	− 0.04	0.40**	0.37**	− 0.16**	0.74	0.73	0.61**

* p < .05, ** p < .01

Table 5

Intercorrelation between OCDQ-RE composites and stress, social and psychological well-being, and personal accomplishment scales (n = 81)

	Factor 1		Factor 2		Factor 3	Factor 4	Factor 5	Factor 6
Occupational stress	− 0.20		− 0.12		− 0.16	− 0.16	− 0.14	− 0.03
Social well-being	0.05		0.14		0.17	0.10	0.10	− 0.01
Psychological well-being	0.14		− 0.01		0.05	0.19	0.05	− 0.03
Personal accomplishment	0.04		0.03		0.05	0.25*	0.08	0.14

* p < .05

Correlations Between the Portuguese OCDQ-RE and Sociodemographic Indicators

Table 6 depicts the correlation values between the Portuguese OCDQ-RE composites and age and gender. Associations were weak, positive, and statistically significant between age and Factor 1 (Professional relationships management) and Factor 4 (Professional interactions among teachers), and weak, negative and statistically significant between age and Factor 2 (Pedagogical tasks management). The remaining correlations were extremely weak, with variables being hardly related (since they were below the threshold of .10).

Table 6

Intercorrelation between Portuguese OCDQ-RE composites and sociodemographic indicators (N = 687)

	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	Factor 6
Age (in years)	0.10*	-0.19**	-0.03	0.19**	0.03	0.06
Gender	-0.02	-0.04	-0.03	0.00	0.04	-0.02

* p < .05, ** p < .01

Discussion

Over the past decades, teachers’ occupational health has become one of the central concerns of research and practice (European Commission, 2011). In this scenario, the literature has emphasized the potentially crucial role of contextual variables in improving teachers’ health and well-being. However, research and practice have mainly acted on an individual level, whereby contextual variables have been less investigated and intervened (Iancu et al., 2018; Oliveira et al., 2021a; Schaufeli & Enzmann, 1998). Particularly in Portugal, the development of a culturally adapted and reliable measure enabling the assessment of schools’ professional personnel’s perceptions of elementary school climate has been sorely needed. Hence, this study aimed to adapt and investigate the psychometric qualities of the OCDQ-RE Portuguese version in terms of its factor structure, reliability, and validity.

The results supported the established hypothesis regarding the factor structure of the OCDQ-RE Portuguese version (H1), sustaining the cross-cultural stability and robustness of the OCDQ-RE. In line with that found by previous adaptations of this measure (e.g., Vos et al., 2013; Yılmaz & Altınkurt, 2013), a 2nd general dimension of two orthogonal factors was not observed. Therefore, the culturally adapted Portuguese OCDQ-RE consists of six first-order factors which assess elementary school climate through both leadership-level and teacher-level behaviors. Thus, in line with the original version but considering the cultural adaptations described in the "Method" section, the OCDQ-RE Portuguese version comprises three scales pertaining to management team behaviors (at the leadership-level). The first is Professional relationships management (originally referred to as Supportive principal behavior) which refers to a democratic leadership style, where the management team demonstrates concern, respect and interest for the schoolteachers, through open communication, active listening, and constructive feedback (e.g., “The management team listens to and accepts teachers’ suggestions.”). The second is Pedagogical tasks management (originally referred to as Directive principal behavior) which describes a leadership style geared towards the achievement of results and goals in which the management team supervises and monitors the pedagogical tasks carried out by the teachers in a rigid, inflexible, and constant manner, controlling the deadlines and accomplishment of those tasks (e.g., “The coordinators closely supervise the teachers’ work”). Finally, the last scale is Bureaucratic tasks management (originally referred to as Restrictive principal behavior) which considers a leadership style that focuses on bureaucratic aspects associated with the school context in which the management team behaviors can handicap teachers’ work through a workload of administrative requirements which interfere with their teaching activities (e.g., “Administrative paperwork is burdensome in my school.”). These cultural adaptations and renaming of the scales were important to ensure an adequate representation of the present-day Portuguese school system. An example is that, contrary to what was originally proposed (Hoy & Clover, 1986), the Portuguese teachers perceived the Pedagogical tasks management as a positive and helpful behavior of the managers and this scale presents a positive association with Professional relationships management and a negative association with Bureaucratic tasks management. Additionally, concerning teachers’ behavior (at the teacher-level), the OCDQ-RE Portuguese version is composed of three scales, namely Professional interactions among teachers (originally referred to as Collegial teacher behavior) which concerns informal teacher-teacher interactions that are established within the professional context and reflects open and collaborative professional interactions among teachers (e.g., “Teachers help and support each other.”); Personal interactions among teachers (originally referred to as Intimate teacher behavior) which refers to informal teacher-teacher interactions that are established outside / beyond the professional context, through the maintenance of cohesive and strong social relationships where teachers consider their colleagues as close friends, socialize regularly with them and provide them with strong social support (e.g., “Teachers’ closest friends are other teachers at the school.”); and lastly Dynamic of the teacher’s group (originally referred to as Disengaged teacher behavior) which describes the dynamics that occur among the school teachers when in formal working periods, reflecting a critical perspective of the perceived meaning and focus regarding professional activities (e.g., “Faculty meetings are useless”).

The coefficient omegas suggested an adequate internal consistency of the six composites of the Portuguese OCDQ-RE. However, the internal consistency of the Bureaucratic tasks management and the Dynamic of the teachers group scales was lower, which, despite being similar to that found across other studies (e.g., Vos et al., 2013; Yılmaz & Altınkurt, 2013), calls for further research. It is possible that these results reflect the transformation of teachers’ work activities across time (Lambert et al., 2009). In fact, when developed, the OCDQ-RE items were constructed not only considering that management and leadership roles were centered on the principal, but also that teachers’ responsibility and central work activity was teaching per se (Hoy et al., 1991). Therefore, it is possible that the items integrated in these two scales do not fully represent the dimensions being assessed (e.g., concerning formal meetings, it may be important for the items to discriminate meetings related to school procedures versus those related to the pedagogical department), particularly since the changes to the Portuguese education structure (Ministry of Education, 2008) specifically impacted the bureaucratic dimension of teachers’ work (Costa, 2010).

The majority of intercorrelations among the six scales were weak to moderate, albeit statistically significant, thus reinforcing that they are correlated but distinct dimensions. However, we observed strong, positive and significant correlations between Professional interactions among teachers and Professional relationships management (r = .50) and Personal interactions among teachers (r = .65) suggesting that these dimensions are associated. The stronger associations between these scales are in line with the original studies (Hoy et al., 1991) and support the relationship between leadership style and principal-teacher interaction and the subsequent teacher-teacher interactions (Sanchez et al., 2020). Moreover, in the Portuguese educational context, the principal and management team are regular teachers within the school cluster who will return to teaching at the end of their mandate. Therefore, it is likely that members of the management team simultaneously maintain informal interactions with the teachers. Strong scores for test-retest reliability also sustained the psychometric quality of the Portuguese OCDQ-RE. The findings also provided support for discriminant validity against external measures, with negative and weak intercorrelations found between perceived positive school climate and occupational stress (corroborating H2a), and positive and weak intercorrelations found between perceived positive school climate and social and psychological well-being, and personal accomplishment (sustaining H2b). These findings are in line with prior literature which supports these related yet distinct constructs (e.g., Bahrami et al., 2013; Collie et al., 2012; Polat & İskender, 2018; Skaalvik & Skaalvik, 2009).

Finally, intercorrelations between the Portuguese OCDQ-RE scales and the sociodemographic indicators suggested that teachers’ perceptions of school climate are not associated with their gender, which is in keeping with the findings of Sanchez et al. (2020) and Skaalvik and Skaalvik (2009). Moreover, with regards to age, there were weak, positive, and statistically significant associations between teachers’ age and their perception of Professional relationships management and Professional interactions among teachers, suggesting that older teachers appear to perceive higher democratic leadership style behaviors from the management team and more open and collaborative professional interactions among teachers in the school-context. Teachers’ age presented a negative, weak, and statistically significant association with Pedagogical tasks management, suggesting that younger teachers tend to perceive less rigid and regular supervision of their pedagogical tasks. Although studies addressing the association between age and perceived school climate were not found for elementary schools, within the secondary school context, the findings were slightly different with younger teachers (< 30 years) perceiving interactions among schoolteachers as more friendly than the older teachers (41 + years) (Sanchez et al., 2020). No associations were found between teachers’ age and their perceptions of leadership styles, in that study (Sanchez et al., 2020).

Overall, these findings provide support for the adequacy, reliability, and validity of the Portuguese version of the OCDQ-RE as a self-report measure to assess perceived elementary school climate. They sustain the cross-cultural stability and robustness of the measure and its validity in measuring the proposed construct in different contexts. Since the literature has emphasized the need to value and consider contextual variables within research and practice (Cohen et al., 2009; Oliveira et al., 2021a; Schaufeli & Enzmann, 1998), this measure may also be an important resource for researchers and practitioners in this field.

Limitations and future research

This study presents some limitations which should be taken into consideration. Firstly, despite covering all the Portuguese counties and the use of an online data validation protocol, the study used a non-probabilistic online sample. Therefore, future studies are needed to validate the Portuguese OCDQ-RE factor structure and its psychometric properties resorting to different data collection methods and using representative samples of the Portuguese population of elementary-school teachers. Moreover, due to the sample’s imbalance of gender, age, and type of school representation, it was not possible to test for structural invariance across the groups and this should be accomplished in future research. Future studies should also test for convergent validity of the Portuguese OCDQ-RE against external measures. Lastly, following the main school climate spheres identified by Cohen et al. (2009), the OCDQ-RE does not consider teacher-student interactions or the environmental structure features. Despite previous attempts to integrate new items, they did not fit into the models (e.g., Hoy et al., 1991). Additionally, the OCDQ-RE was developed before the main transformations in teachers’ job requirements that accompanied the new millennium (Lambert et al., 2009), which gave rise to a broader spectrum of teachers’ work activities that go beyond pedagogical tasks to also include bureaucratic and management tasks. Hence, it may be important for future studies to improve the Portuguese OCDQ-RE version by seeking to cover all the main spheres of the school climate and teachers’ work activities.

Study impact

Despite the aforementioned limitations, this study supports the validity and adequacy of the Portuguese OCDQ-RE, thus advancing important contributions to both research and practice. With the observed strong associations between teachers’ occupational health and contextual variables, such as the organizational climate (Collie et al., 2012; Schaufeli & Taris, 2014), the literature has emphasized the need to develop psychometrically validated measures to assess these latter variables (Cohen et al., 2009). By presenting itself as a culturally adapted, adequate, reliable, and valid theoretically grounded measure of perceived organizational climate, the Portuguese OCDQ-RE contributes to research by facilitating methodologically robust studies that aim to assess the role of this variable across settings. As far as practice is concerned, this measure may be a useful resource for practitioners within Portuguese educational contexts, since it may be used to screen and diagnose contextual needs, thus helping to define and plan intervention goals and practices to improve the work environment for the professional personnel in schools. The Portuguese OCDQ-RE is also easily administered and simple to score and can therefore be used without being overly time consuming (a common barrier within educational contexts). Moreover, since Portuguese is one of the most spoken languages worldwide and the most spoken language in the Southern Hemisphere, with the necessary adaptations, the Portuguese OCDQ-RE may also facilitate cross-cultural studies and the expansion of research and practice across different educational contexts worldwide.