Development and Validation of a Safety Climate Scale for Manufacturing Industry.

BACKGROUND: This paper describes the development of a scale for measuring safety climate.
METHODS: This study was conducted in six manufacturing companies in Iran. The scale developed through conducting a literature review about the safety climate and constructing a question pool. The number of items was reduced to 71 after performing a screening process.
RESULTS: The result of content validity analysis showed that 59 items had excellent item content validity index (≥ 0.78) and content validity ratio (> 0.38). The exploratory factor analysis resulted in eight safety climate dimensions. The reliability value for the final 45-item scale was 0.96. The result of confirmatory factor analysis showed that the safety climate model is satisfactory.
CONCLUSION: This study produced a valid and reliable scale for measuring safety climate in manufacturing companies.

Introduction

Safety climate is an important indicator of safety performance, and it is used for predicting safety related outcomes such as safety behavior and occupational accidents/injuries [1,2]. The existence of a valid scale for measuring the safety climate is very important and it can facilitate the collection of accurate data [3,4]. Validity test of a safety climate scale is considered as a real test to reveal the safety level in an organization, and the test aims to improve the quality of required data [5]. The assessment of reliability only describes the level of measurement errors of a scale.

Many studies have investigated the construction of the safety climate in organizations. However, they have not reached a common agreement on safety climate dimensions [6-9]. The review of previous studies showed that management commitment to safety is a common dimension for safety climate [10-13]. Seo et al [3] indicated that the safety climate dimensions can be categorized into five themes: management commitment to safety, supervisor safety support, coworker safety support, employee participation in safety decision making and activities, and competence level of employee with regard to safety. A review of 18 safety climate surveys by Flin et al [5] revealed that safety system, management/supervision, risk, work procedure, and competence were the most frequent dimensions. Flin et al [11] also identified work pressure as another frequently used dimension. Safety communication, safety training, supportive and supervisory environments, in addition to safety rules and procedures were found as other dimensions of the safety climate [10,12,13].

Several methods are typically used to assess the validity of a measurement instrument. The content validity of an instrument can be examined in development and judgment stages. The development stage is usually carried out through performing a comprehensive literature review or conducting interviews with focus groups. The judgment stage is accomplished through the application of either quantitative or qualitative methods. The quantitative analysis of the content validity is determined by the application of statistical methods. The qualitative approach only depends on the opinion of experts. Several studies have investigated the content validity analysis by reviewing the literature and by using an expert panel [3,13]. Many researchers have examined the content validity of safety climate scales using a qualitative method. However, few of them presented enough evidence for the analysis of the content and the construct validity [3]. Therefore, the quantitative examination of the content validity is not a common method for analysis of the safety climate scales. In addition, experts conduct the face validity analysis through the review of an instrument. They check the instrument to ensure it measures what it is supposed to measure [14]. The construct validity is examined using statistical methods. A large number of researchers have employed the exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) to evaluate the construct validity of the safety climate scales [15-18].

Many instruments have been developed to measure the safety climate in various industries worldwide. To the authors' knowledge, this study is the first one to develop and to validate a safety climate scale for manufacturing industry in Iran. Because of the unique nature of safety climate and context culture in countries, industries, companies, and even different sectors of an organization [19,20], we found a need to develop a new scale to examine the safety climate. Kudo et al [21] identified the necessity to produce a standard safety climate questionnaire to collect appropriate data. The authors also recognized the need for specific safety climate dimensions for each occupation. Therefore, it is important to develop an original scale to measure the safety climate in Iranian manufacturing companies. In this study, we developed a new safety climate scale and explored the validity and the reliability of the scale.

Materials and methods

The present study was conducted to test the validity and the reliability of a newly developed scale for measuring safety climate in the manufacturing industry. A total of 50 people participated in the content and the face validity analyses. The first group of participants were faculty members (n = 14) who researched occupational health and safety (OHS) and worked at two universities in Tehran (the capital city) and Urmia (the capital of the west Azerbaijan province) in Iran. The second group were OHS officers (n = 10) who worked at manufacturing companies in Urmia. The last group were employees (n = 26) who worked at three manufacturing companies in Urmia. Other group of employees (n = 26) from the companies participated in a test–retest reliability study, and they refilled questionnaires after a 3-week period. The employees were randomly chosen for the validity and the reliability analyses. A total of 269 employees participated in this study who worked in six manufacturing companies in the West Azerbaijan Province in Iran to collect required data for performing EFA, CFA, and final reliability analysis. The authors obtained written permission from the companies to conduct this study and asked respondents to participate voluntarily in the survey.

A literature review was conducted and a total of 662 safety climate items were generated from the available questionnaires in the published articles [7,8,10,17,18,22-37]. The number of items reduced to 71 after conducting a screening process for redundancy and the general aim of our study. This 71-item scale was translated to the Farsi language (the official language in Iran). Then, we examined the validity and reliability of the translated scale. All safety climate items were rated on a 5-point Likert-type scales with phrases of strongly disagree and strongly agree on Point 1 and Point 5 to conduct the reliability analysis and EFA.

The content and the face validity of the scale were examined by the OHS experts (faculty members and OHS officers) and by the employees. We used a different measurement criteria for examining the content validity. The criterion for measuring the content validity by the OHS experts included three categories: (1) essential; (2) useful, but not essential; and (3) not necessary [38]. Further, we asked the OHS experts to write their comments about the ambiguity and the clarity of the items to evaluate the face validity. A different criterion was used for the employee sample [39]. The employees were asked to rank each of the safety climate items for relevancy, clarity, and simplicity using a 4-point Likert-type arrangement: (1) not relevant (clear or simple); (2) item needs some revision; (3) relevant (clear or simple) but need minor revision; and (4) very relevant (clear or simple).

We employed descriptive statistics to describe the individual characteristics of the participants and to examine the content validity of the scale. Content validity ratio (CVR) was calculated for each item of the questionnaires, which filled out by the OHS experts [CVR = (ne−N/2)/(N/2)]. The mean of item CVRs was computed to calculate the content validity index (CVI) [38]. For each item of the questionnaires, which were filled out by the employees, we calculated an item content validity index (I-CVI) as the number of “3” and “4” responses/number of experts × 100 [39]. After that, the scale content validity index (S-CVI) was calculated for whole items of each questionnaire through obtaining the average of all I-CVIs. We conducted EFA to identify the safety climate’ underlying dimensions. Intraclass correlation coefficient (ICC) and Cronbach α were calculated. Then, CFA was performed to confirm the identified dimensional structure of the scale. The statistical analyses were performed using SPSS version 21 software (SPSS Inc., Chicago, IL, USA), and AMOS version 21 (IBM) was used for conducting CFA.

Results

Fifty people participated in the content and the face validity analyses of the safety climate scale. As shown in Table 1, the majority of the faculty members (92.9%) of employees (84.2%) were male. The age pattern revealed that most respondents of the three groups of the participants were aged 30–39 years. Most of the OHS experts had 1–5 years of working experience and most of the employees (36.4%) had 6–10 years of working experience. The majority (96.2%) of the employees who participated in the test–retest reliability analysis were male. Most of these employees were aged 40–49 years, and 34.6% of them had > 20 years of working experience.

Table 1

Demographics of the participants in the content validity and the reliability analyses

Variables	Validity analysis
	Faculty members (n = 14)	OHS officers (n = 10)	Employees (n = 22)	Reliability analysis (n = 26)
Gender
Male	13 (92.9)	5 (50)	18 (81.8)	25 (96.2)
Female	1 (7.1)	5 (50)	4 (18.2)	1 (3.8)
Age (y)	40.7 (10.7)∗	32.7 (7.00)∗	35.5 (10)∗	41.85 (8.05)∗
< 30	2 (14.3)	4 (40)	5 (22.7)	3 (11.5)
30–39	6 (42.9)	4 (40)	16 (72.7)	5 (19.2)
40–49	3 (21.4)	2 (20)	—	14 (53.8)
50–59	2 (14.3)	—	2 (4.5)	4 (15.4)
≥ 60	1 (7.1)	—	—	—
Working experience (y)	10.6 (9.5)∗	8 (6.05)∗	11.6 (7.70)∗	15.73 (7.65)∗
< 1	1 (7.1)	1 (10)	—	1 (3.8)
1–5	6 (42.9)	4 (40)	5 (22.7)	4 (15.4)
6–10	2 (14.3)	2 (20)	8 (36.4)	3 (11.5)
11–15	—	2 (20)	1 (4.5)	2 (7.7)
15–20	1 (7.1)	1 (10)	6 (27.3)	7 (26.9)
> 20	4 (28.6)	—	2 (9.1)	9 (34.6)

Data are presented as n (%), unless otherwise indicated.

Mean and standard deviation in years provided for age and working experience of the participants.

The analysis of the content validity of the scales, which rated by the OHS experts showed that 61 of the 71 items (85.92%) had an excellent content validity. The acceptable level of CVR for 24 experts is > 0.38 [38]. Four items that rated by the employees had a low CVI. The recommended value for the acceptable I-CVI is no lower than 0.78 [39]. Two out of these four items rated with unacceptable CVR simultaneously. Therefore, 12 items were kept out from the initial scale, and 59 items retained. The S-CVI was 0.89. Further, four out of 26 questionnaires filled out by the employees were removed from the final analysis due to missing data.

Minor remarks were given by the OHS experts regarding to improve the clarity of the wording. The result of the test–retest reliability analysis showed that there is no difference between safety climate scores (F (1, 25) = 0.60, p = 0.81), and the degree of reliability is high (ICC = 0.93). The Cronbach α for the retest group was 0.95.

The EFA using principal component analysis with varimax rotation method resulted in the retention of eight factors with 48 items (Table 2). The analysis showed that Kaiser–Meyer–Olkin measure of sampling adequacy was 0.92, which indicates that the data were appropriate for this analysis. Bartlett's test of sphericity was significant (χ2 = 8.561E3, p < 0.01), indicating that correlations exist among some of the safety climate dimensions. Nine items were removed from the scale because there were fewer than three loaded items for each factor [3,40,41]. Likewise, the value of the loading for other two items was < 0.4 and kept out from the scale [3,41]. The final dimensions were identified as safety commitment and communication, safety involvement and training, positive safety practices, safety competency, safety procedures, accountability and responsibility, supportive environment, and safety prioritization (Appendix 1).

Table 2

The results of the exploratory factor analysis

Dimension variables	Corrected item-total correlation	Factor loading	Eigen value	Variance explained (%)	Cumulative variance explained (%)
Factor 1: Safety commitment and communication			8.46	14.34	14.34
12. Feedback for safety proposals	0.60	0.72
8. Managers/supervisors interest for safety issues	0.66	0.68
11. Openly discussions about safety problems	0.64	0.66
3. Sufficient resource allocation for safety	0.60	0.65
9. Managers/workmates respected who work safely	0.60	0.65
7. Management looked for underlying factors of incidents	0.60	0.64
2. Management decisive and quick actions for safety concerns	0.63	0.64
16. Interest of company for views of employee about safety	0.67	0.63
6. Getting the equipment needed to do job safely	0.50	0.62
14. Effectively communicate of changes in safety procedures	0.67	0.61
1. Real cares about the employee safety	0.66	0.61
13. Dissemination of safety information to appropriate personnel	0.64	0.60
10. Workers were consulted about safety issues	0.62	0.58
21. Involvement of unit manager in safety activities	0.64	0.48
17. Influence on safety performance	0.55	0.42
31. Management understand impact of operations on safety	0.71	0.41
Factor 2: Safety involvement and training			4.62	7.84	22.18
18. Involvement in the development or review of safety procedures	0.41	0.67
25. Training about new procedures or equipment	0.64	0.65
20. Encouragement to report unsafe conditions	0.44	0.61
29. Consult workers to establish their training needs	0.59	0.59
23. Safety training at regular intervals	0.66	0.56
19. Encourage to make suggestions on safety improvement	0.63	0.56
24. Training provide skills and experience to do operations safely	0.66	0.51
33. Investigate accidents for finding their causes	0.58	0.44
Factor 3: Positive safety practices			4.14	7.01	29.19
51. Availability of enough people to do job safely	0.59	0.65
57. Feel challenged and motivated by work tasks	0.40	0.63
35. Safe work site	0.57	0.57
54. Balanced workload	0.57	0.52
52. Stop working due to safety concerns	0.59	0.52
34. Appropriate feedback about performance	0.60	0.50
38. Safety regulations are performed in my workplace	0.65	0.46
50. Control for safety rule violations	0.69	0.45
Factor 4: Safety competency			3.35	5.69	34.88
26. Clear about safety responsibilities	0.53	0.84
27. Understand the safety risks of responsible works	0.52	0.81
28. Understand the job safety procedures	0.59	0.70
Factor 5: Safety procedures			2.64	4.48	39.36
43. Follow safety procedures to do job safely	0.41	0.71
40. Safety is number one priority when completing a job	0.41	0.63
44. Safety procedures reflect how do jobs safely	0.48	0.53
47. Clear procedures appropriate to the user needs	0.57	0.42
Factor 6: Accountability and responsibility			2.21	3.74	43.10
48. Workmates react against people who break safety procedures	0.47	0.64
46. Safety instructions are easy to understand and implement	0.51	0.58
15. Co-workers give tips on how to work safely	0.47	0.45
Factor 7: Supportive environment			1.95	3.30	46.40
39. Safety considers to be equally as important as production	0.58	0.56
45. Rules describe the safest way of working	0.53	0.45
30. Manager/supervisor bring safety information to my attention	0.67	0.44
Factor 8: Safety prioritization			1.93	3.28	49.68
37. Untidy work site	0.27	0.71
36. Required to work in an unsafe manner	0.19	0.71
41. Difficult to do some jobs safely	−0.12	−0.54

We used Cronbach α to measure the internal consistency reliability of the scales. The desired accepted value for Cronbach α is 0.70, but when there are five or fewer items, the acceptable level is 0.60 [42]. Safety prioritization excluded from the final scale because of a low reliability. The reliability measure for the final 45-item scale was 0.96, and the reliability coefficients of the dimensions ranged from 0.63 to 0.93 (Table 3).

Table 3

Cronbach α values, mean and standard deviations for the safety climate dimensions

Safety climate dimension	Number of items	Cronbach α	Mean	SD
Safety commitment and communication	16	0.93	3.34	0.79
Safety involvement and training	8	0.87	2.94	0.82
Positive safety practices	8	0.85	3.25	0.79
Safety competency	3	0.89	3.66	1.00
Safety procedures	4	0.73	3.74	0.76
Accountability and responsibility	3	0.62	3.38	0.82
Supportive environment	3	0.71	3.15	0.91

As shown in Fig. 1, the safety climate dimensions were considered as latent variables in CFA. The result of CFA showed that the model that previously identified by EFA is satisfactory (χ2(931) = 1907.72, p < 0.01). The root mean square error of approximation (RMSEA) index was 0.06, which is lower than the recommended critical limit of 0.08 [3]. The comparative fit index (CFI) was 0.85, and incremental fit index (IFI) was 0.85. These values were reasonable compared with the acceptable value of two fit indices that are ≥ 0.90 [3]. The modification indices were assessed to determine the possible modifications in the initial safety climate model. The results showed that 11 error terms were allowed to correlate, and the modified model was satisfactory [χ2(920) = 1723.02, p < 0.01]. The CFI and IFI increased slightly, but RMSEA decreased to 0.05 (Table 4).

Fig. 1

Confirmatory factor analysis of safety climate scale.

Table 4

Goodness of fit indicators of the safety climate model (n = 269)

Models	χ2	df	χ2/df	IFI	CFI	RMSEA	PCLOSE
Initial	1907.72*	931	2.05	0.85	0.85	0.06	0.00
Modified	1723.02*	920	1.87	0.87	0.87	0.05	0.00

*p < 0.05.

CFI, comparative fit index; IFI, incremental fit index; RMSEA, root mean square error of approximation.

Discussion

The main objectives of this study were the development and the validation of a safety climate scale. Initial investigation of the validity and the reliability of the developed scale resulted in 59 items. After conducting the EFA and the reliability analysis, the items were reduced to 45.

The present study investigated the content validity by application of the quantitative method. This study also examined the construct validity of the scale by the application of EFA and CFA. Further, the internal consistency reliability of the scale was also satisfactory. Therefore, the scale proved to be a valid and reliable tool to measure the safety climate. It is important to note that the assessment of the concurrent validity between the safety climate and participants' accident experience did not provide a significant result. It may result from the point that respondents asked for their’ experience of accidents in the past 3 years.

The EFA was performed to reduce the safety climate attributes into dimensions. The safety climate dimensions were labeled as safety commitment and communication, safety involvement and training, positive safety practices, safety competency, safety procedures, accountability and responsibility, and supportive environment. These findings are consistent with the results of the previous studies that reported the safety commitment and communication, safety training, employee involvement, competency, safety procedures [5,43,44], accountability [45], responsibility [46,47], and supportive environment [5] as safety climate dimensions. The findings of CFA support the application of a seven-dimension model for measuring the safety climate. The assessment of the major fit indices revealed that the dimensional structure of the safety climate scale was satisfactory. The result of the Chi-square test for the examination of the CFA model showed a statistically significant result. The Chi-square test is one indicator of good model fit; however, it is more sensitive to trivial misspecifications in the model's structure [48] and sample size [49-51]. Prior studies employed other indices to prove the model fit when the Chi-square result was significant [48,52,53]. Tharaldsen et al [18] also used other fit indices and they did not report the Chi-square result. We thus used CFI, IFI, goodness of fit, and RMSEA to assess the CFA model fit.

The application of a large number of participants is more satisfactory for conducting factor analysis. Furr [49] described the problem of sample size in CFA as following: “The appropriate sample size for CFA is a complex issue. Recommendations for absolute sample sizes vary from a minimum of 50 participants to 300 or more, while other recommendations are framed in terms of ratios such as a five-to-one or a twenty-to-one ratio of participants-to-variables”. Other scholars have suggested that using the sample size between 200 and 300 is good for conducting factor analysis [54]. Therefore, the sample size of this study was satisfactory for conducting the CFA.

The qualitative evaluation of the safety climate scales by a group of experts is a common approach to assess the content validity of the scales [3]. The application of a quantitative method for conducting such analysis facilitate the decision making process regarding retention or rejection of the items of the scale. The authors employed a high number of experts and a Likert-type scale for rating the items in the validation process. These were conducted to consider the recommendations given by Wynd et al [55] for overcoming the limitations of CVI.

In conclusion, the result of this study showed that the validity and the reliability of the developed scale were satisfactory. The scale was developed in response to a need for a safety climate scale in the manufacturing industry in Iran. It can be used to investigate the perception of manufacturing employees about safety. For future research, we would recommend re-examining the validity and the reliability of the scale with a larger and more diverse sample of manufacturing employees. Such examination will be warranted for the validity and reliability of the safety climate dimensions' structure across various companies. Future research may examine the discriminant validity of the scale by conducting a correlation analysis between the safety climate dimensions and other contributing occupational or organizational factors.

Conflicts of interest

All contributing authors declare no conflicts of interest.

Factor 1: Safety commitment and communication
1	Workers were given sufficient feedback regarding safety proposals
2	In my workplace managers/supervisors show an interest in safety issues
3	Workers were able to openly discuss safety problems with supervisors or managers
4	Management allocated sufficient resources to health and safety
5	People who work safely are respected by their managers/workmates
6	Management looked for underlying factors that contributed to safety incidents rather than blame the people involved
7	Management acts decisively and quickly when a safety concern is raised
8	The company shows interest in my views on health and safety
9	I always get the equipment I need to do the job safely
10	Changes in working procedures and environment and their effects on safety are effectively communicated to workers
11	The company really cares about the health and safety of the people who work here
12	Safety and health information (outcome of OHS meetings, causes of accidents/incidents, …) is effectively disseminated to all appropriate personnel
13	Workers were consulted about health and safety issues
14	On my unit, senior level management gets personally involved in safety activities
15	I can influence health and safety performance here
16	Management had a good understanding of operational issues that impacted on work safety
Factor 2: Safety involvement and training
17	I get involved when health and safety procedures/instructions/rules are developed or reviewed
18	I received related training when new procedures or equipment were introduced
19	I am strongly encouraged to report unsafe conditions
20	People here are consulted to establish their training needs
21	Safety training was received at regular intervals to refresh and update knowledge
22	The company encourages suggestions on how to improve health and safety
23	Company training provided adequate skills and experience to carry out operations safely
24	Accident investigations aim at finding causes of accidents rather than blaming individuals
Factor 3: Positive safety practices
25	There are always enough people available to get the job done safely
26	I generally feel challenged and motivated by my work tasks
27	My work site is often safe
28	My Workload is reasonably balanced
29	The company would stop us working due to safety concerns, even if it meant losing money
30	I receive appropriate feedback about my performance
31	The regulatory requirements on health and safety are performed in my workplace
32	My supervisor always has control over safety rule violations
Factor 4: Safety competency
33	I am clear about what my responsibilities are for health and safety
34	I fully understand the health and safety risks associated with the work for which I am responsible
35	I fully understand the health and safety procedures/instructions/rules associated with my job
Factor 5: Safety procedures
36	Some health and safety procedures/instructions/rules need to be followed to get the job done safely
37	Safety is the number one priority in my mind when completing a job
38	Most of the health and safety procedures/instructions/rules reflect how the job is now done
39	Procedures are written in clear unambiguous language appropriate to the needs of the user
Factor 6: Accountability and responsibility
40	My workmates would react strongly against people who break health and safety procedures/instructions/rules
41	The written safety rules and instructions are easy for people to understand and implement
42	Co-workers often give tips to each other on how to work safely
Factor 7: Supportive environment
43	In my company safety considerations are equally as important as production
44	The rules always describe the safest way of working
45	Safety information is always brought to my attention by my line manager/supervisor