Qualitative and quantitative differences between common occupational health risk assessment models in typical industries.

OBJECTIVE: The differences in the methodologies of various occupational health risk assessment (OHRA) models have not been extensively reported. We aimed to understand the qualitative and quantitative differences between common OHRA models in typical industries.
METHODS: The Environmental Protection Agency (EPA), Australian, Romanian, Singaporean, International Council on Mining and Metals (ICMM), and the Control of Substances Hazardous to Health (COSHH) models were evaluated, and a theoretical framework was established for a comparative study.
RESULTS: Qualitative comparisons showed that each OHRA model had its own strengths and limitations, and exhibited a diverse distribution at different levels for each evaluation indicator. The Singaporean, COSHH, and EPA models had a much higher comprehensive advantage than the other models for all indicators. Quantitative comparisons demonstrated that these three models also had a stronger ability to distinguish the difference in risk ratios between different industries. The Singaporean model had the strongest correlation with the other models.
CONCLUSION: Each model possessed its own strengths and limitations depending on its unique methodological principles. Combining the EPA, Singaporean, and COSHH models might be advantageous for developing an OHRA strategy. More studies comparing multiple models in key industries are required.

Introduction

Workers exposed to occupational hazards are at a greater risk of developing work-related diseases and injuries[1)]. Globally, there are 2.3 million deaths associated with work-related diseases every year[2)] and in many countries the economic costs of these injuries and illnesses ranges from 1.8% to 6.0% of the gross domestic product[3]-[5)].

Occupational health risk assessment (OHRA) in the workplace is essential for implementing risk control for occupational activities and maintaining worker health[6],[7)]. Therefore, OHRA models have recently been developed by many industrialized countries and international organizations. A series of risk assessment guidelines have been established by the U.S. Environmental Protection Agency (EPA)[8],[9)] and the National Institute for Occupational Safety and Health (NIOSH)[10)]. Furthermore, Australia[11)], Romania[12)], Singapore[13],[14)], the International Council on Mining and Metals (ICMM)[15)], and the United Kingdom (UK)[16],[17)] have all developed their own OHRA models. Generally, OHRA models are established based on four core steps: hazard identification, hazard characterization, exposure assessment, and risk characterization.

Currently, there is little guidance for choosing the most suitable model for a given application, which relies on the expert's individual judgment, and therefore may lead to very different results depending on the experience of the consulted experts. It is therefore desirable to strengthen and solidify the theoretical framework for assessing and minimizing occupational risks, which is dependent on understanding the similarities and differences in the methodologies of the different ORHA models. However, at present there is very little information on the quantitative or qualitative differences among the different OHRA methods. Zhou et al.[18)] systematically reviewed the ORHA models used by the EPA, Singapore, Australia, Romania, and the ICMM, as well as the Control of Substances Hazardous to Health (COSHH) model developed in the UK, and concluded that the scope and principles of these OHRA models are not exactly the same, and that each has its own strengths and limitations. Therefore, quantitative, semi-quantitative, and qualitative methods can be applied in combination when conducting OHRA. The International Chemical Control Toolkit (ICCT), based on the COSHH model, was recently tested in parallel with the Singaporean model to evaluate the utility of both models and to compare them based on their theoretical and empirical aspects, and found that the assessed risk levels were largely consistent between the two models[19)]. Another study compared risk level assessments using the EPA, Singaporean, and Romanian models in many industries such as chemical engineering, electroplating, and furniture manufacturing[20],[21)]. The authors found that although each model had advantages and disadvantages, the Singaporean model possessed certain advantages when evaluating sawdust exposure.

The majority of reported work-related diseases and accidents occur in developing countries[1],[22)], demonstrating the importance of improving OHRA practices in these countries[23]-[28)]. Although some progress has been made in this area, the qualitative and quantitative differences in OHRA methodologies are poorly understood. The purpose of this study was to qualitatively and quantitatively compare six common OHRA models (i.e. the EPA, Australian, Romanian, Singaporean, ICMM, and COSHH models) in three typical industries (wood furniture manufacturing, electroplating, and crane manufacturing) by establishing a theoretical framework for comparative study.

Materials and Methods

Description of Typical Industries and Enterprises

The wood furniture manufacturing, electroplating, and crane manufacturing industries were selected as the ideal sample industries for this study for the following reasons. First, the three industries are classified as having the "most severe" or "relatively severe" occupational health risks. According to the "Management catalogue of occupational hazard risk classification of construction projects" issued by the State Administration of Work Safety of China (2012 edition)[29)], the risk levels for the electroplating and crane manufacturing industries are classified as "most severe" and "relatively severe", respectively. The wooden furniture manufacturing industry has been downgraded to "relatively severe" from the original "most severe" risk level based on recent changes in administrative rules and regulations. Qualified local institutes for occupational health and technical services were unable to detect the presence of benzene, which is the most toxic chemical used in typical wood furniture manufacturing processes (e.g. paint spaying, polishing, and packing). Second, these industries are dominated by small to medium enterprises (SMEs), which in China usually lack comprehensive occupational disease prevention and control measures because of the employer's poor legal knowledge, leading to a high probability of occupational health hazards.

A total of nine SMEs in the Zhejiang province of East China were selected as the ideal sample factories (three enterprises per industry). These specific manufacturing enterprises were selected because they had many features typical of other SMEs in each industry, such as similar types of work, production processes, occupational hazards, and exposure levels, as well as inadequate control measures[30],[31)]. Together, the nine factories had a total of 600 workers at various positions.

Identification of Risk Factors

Based on field investigations, air sampling, and laboratory tests, the hazardous occupational factors were identified in the three industries. Table 1 shows that each industry has its own characteristic processes and hazardous factors. The levels of risk factors from the majority of processes in the three industries were qualified using the Chinese standard requirements (Occupational Exposure Limits for Hazardous Agents in the workplace, GBZ 2-2007). Only the levels of sawdust, noise, hydrochloric acid, and toluene generated from certain processes were disqualified. Air sampling for chemical poisons and dust was performed according to the sampling standard in China described in "The sampling specification for hazardous substances monitoring in workplace air (GBZ 159-2004)." Laboratory tests for these chemicals were based on a series of standards (The determination of toxic substances in the workplace; GBZ/T 160-2004). Onsite measurement of noise was conducted according to the standard "The physical factor measurement in the workplace (GBZ/T 189.8-2007)".

Table 1.

Identification of the main risk factors in three typical industries

Industries	Processes	Hazardous Factors	Exposure levels (mg/m3 or dB (A))	OELs (mg/m3 or dB (A))	Results
The sample size of each risk factor is 3.The exposure level of chemical factor is expressed by mg/m3, and the exposure level of noise is expressed by dB (A).OELs: Occupational Exposure Limits, obtained from the occupational health standards in China (Occupational Exposure Limits for Hazardous Agents in the workplace, GBZ 2-2007). OELs for dust and chemicals were expressed in PC-TWA (permissible concentration-time-weighted average); OELs for noise were expressed in LAeq.8h.a The concentration of benzene was below the detection limit in the wood furniture manufacturing and crane manufacturing industries.
Wood furniture manufacturing	Preparation, splicing	Sawdust	8.9 (4.6 - 27.2)	3	Disqualified
		Noise	80.8 (78.3-82.5)	85	Qualified
		Formaldehyde	0.3 (0.09 - 0.36)	0.5	Qualified
	Assembling	Sawdust	1.9 (0.87 - 2.5)	3	Qualified
		Benzene	< 6a	6	Qualified
		Toluene	4.5 (1.1 - 8.7)	50	Qualified
		Xylene	6.1 (1.5 - 11.7)	50	Qualified
	Paint spraying	Benzene	< 6a	6	Qualified
		Toluene	5.8 (1.5 - 11.3)	50	Qualified
		Xylene	16.1 (4.9 - 28.8)	50	Qualified
		Sawdust	2.3 (1.5 - 3.1)	3	Qualified
	Polishing	Sawdust	3.6 (1.5 - 8.6)	3	Disqualified
		Benzene	< 6a	6	Qualified
		Toluene	3.8 (1.3 - 8.9)	50	Qualified
		Xylene	9.8 (3.6 - 20.3)	50	Qualified
	Packing	Formaldehyde	0.15 (0.09 - 0.24)	0.5	Qualified
		Benzene	< 6a	6	Qualified
		Toluene	6.7 (4.3 - 11.9)	50	Qualified
		Xylene	15.5 (13.9 - 25.4)	50	Qualified
Electroplating	Oil removing	Sodium hydroxide	0.28 (0.21 - 1.50)	2	Qualified
	Pickling	Hydrochloric acid	7.7 (3.5 - 7.8)	7.5	Disqualified
	Copper plating	Hydrogen cyanide	0.57 (0.32 - 0.82)	1.0	Qualified
		Sulfuric acid	0.42 (0.30 - 0.52)	1.0	Qualified
	Chrome plating	Hexavalent chromium	0.02 (0.01 - 0.03)	0.05	Qualified
Crane manufacturing	Welding	Welding dust	1.2 (1.0 -1.8)	4	Qualified
		Manganese	0.02 (0.01- 0.03)	0.15	Qualified
		Noise	85.9 (85.1- 86.2)	85	Disqualified
	Polishing	Grinding wheel grinding dust	2.5 (2.2 - 3.0)	5	Qualified
		Noise	86.7 (86. 1 - 87.5)	85	Disqualified
	Paint spraying	Benzene	< 0.49a	6	Qualified
		Toluene	1.68 (0.49-2.52)	50	Qualified
		Xylene	2.91 (0.68 - 4.33)	50	Qualified
		N-butyl alcohol	< 0.47	100	Qualified
		Noise	82.6 (81.4-83.4)	85	Qualified
	Smelting	Sulfur dioxide	1.7 (1.6 - 1.9)	5	Qualified
		Slag ash	1.7 (1.5- 2.1)	8	Qualified
	Dip coating	Benzene	< 0.49a	6	Qualified
		Toluene	68.01	50	Disqualified
		Xylene	37.15 (30.12- 40.21)	50	Qualified

Methodology for ORHA Modeling

Based on a literature review, the EPA, Singaporean, Australian, Romanian, ICMM, and COSHH Essential models were identified as the six most common OHRA models. The detailed principles of these six models have been previously described in the literature[9],[11]-[17)]. Briefly, these principles are as follows.

(1) The EPA model: The EPA inhalation risk assessment includes two components: carcinogenic and non-carcinogenic risk assessments. In this study, only the non-carcinogenic risk assessment was used.

a) Estimating exposure concentrations (EC):

EC= (CA×ET×EF×ED) ÷ AT (Equation 1)

In this equation, EC (μg/m3) is the exposure concentration; CA (μg/m3) is the contaminant concentration in the air; ET (hours/day) is the exposure time; EF (days/year) is the exposure frequency; ED (year) is the exposure duration; and AT is the averaging time (ED [years] × 365 days/year × 24 h/day).

b) Non-carcinogenic risk assessment:

HQ = EC / RfC (Equation 2)

In this equation, HQ is the hazard quotient and RfC represents the reference concentration for inhalation toxicity. The limit for HQ is considered to be 1.

(2) The Singaporean model: The risk levels are calculated based on the hazard ratings (HR) and exposure ratings (ER), as shown in Equation 3:

The HR is assigned based on the carcinogenicity classifications established by the International Agency for Research on Cancer (IARC). The ER is based on the ratio of the exposure level (E) and permissible exposure limit (PEL) or OEL.

If the exposure concentration is not available, exposure indices (EIs) can be used to determine the ER, as shown in Equation 4:

ER=[EI1×EI2×.....EI]1/n (Equation 4)

EIs are determined using exposure factors or parameters of chemicals, such as vapor pressure, hazard control measures, the amount used per week, and duration of work per week.

(3) The Australian model: The risk levels can be assessed using a manual diagram method or a calculator by analyzing the identified exposure levels, the possible consequences of exposure, and the likelihood of exposure for each hazard.

(4) The Romanian model: Based on the severity of a hazard and probability of its occurrence, the concept of a risk acceptability curve was proposed. A matrix method is applied to qualitatively estimate the risk level.

(5) The ICMM model: This model applies a matrix method to assess risk levels, including matrix combinations of health hazards and the probability of an exposure occurring in a similar exposure group or process, as well as matrix combinations of health hazards and exposure levels with existing control measures.

(6) The COSHH Essential model: This model simultaneously considers both the health hazards and exposure levels of chemical substances (solid or liquid), and uses a generic risk assessment to recommend the control level (one of the four types of approaches needed to achieve adequate control). The health hazard is determined based on allocation of the evaluated substance to a hazard band using a Risk-phrase. The exposure potential is determined by allocating the substance to a dustiness or volatility band as appropriate, and another band is used for the scale of use.

Theoretical Framework for a Comparative Study of the Different Models

The theoretical framework for a comparative study of the different OHRA models consisted of two parts: a qualitative and a quantitative comparison. An analysis of the key information and a multi-criteria analysis were performed to compare the qualitative differences. Quantitative comparisons were performed based on the analysis of risk ratios (RRs) to test the agreement between models and the reliability of different models. Agreement between the different models was tested using correlation analysis. Reliability was verified by evaluating the consistency of the RRs in industries with the industries' current risk classifications. Finally, the consistency of both comparisons was evaluated.

Qualitative Comparisons

Key information regarding the principles, attributes, scope, risk classifications, strengths, and weaknesses of the six models was qualitatively analyzed based on review of the literature and discussion among experts. The literature databases queried were Web of Science, PubMed, Medline, Scopus, and related official websites. Search terms used were "risk assessment", "occupational health", "methodology", and "model".

A multi-criteria qualitative analysis was subsequently established based on this analysis of key information[32)] and included the following steps: determination of evaluation indicators, assignment of indicator values and weights, expert consultation, interview with key informants, and comprehensive analysis. The evaluation indicators were determined based on the literature review and expert consultation, in which 30 experts in the field of health management or occupational health were asked for advice on evaluating indicators in two rounds. The seven selected indicators are shown in Table 2. Rather than using different quantification scores, most of the consulted experts (86.7%) considered it appropriate to divide each indicator into low, medium, and high levels, which were assigned 1, 2, and 3 points, respectively. However, the practicability (whether or not the model provides strategies for control) and operability (whether or not the model is easy to use) indicators were only divided into 2 levels (high and low), because the medium level was difficult to define. To assign indicator weight, 83.3% of experts agreed that the weight of the seven indicators should be equivalent, meaning that each indicator was equally important. The rationality of the framework for qualitative comparisons was further discussed by 15 additional core experts as key insiders.

Table 2.

Scoring system used for the multi-criteria analysis.

Criteria (Indicators)	Scores (levels)
	1 (Low)	2 (Medium)	3 (High)
Evaluated Substance (The model that evaluates more types of substances is more useful.)	Chemicals	Chemicals, dust	Chemicals, dust, physical agents
Attribute (A quantitative model can report the probability of risk occurrence.)	Qualitative	Semi-quantitative	Quantitative
Validation (The model is validated by documents containing independent data and may be more accurate.)	No	The model is validated by a few documents	The model is validated by adequate documents with independent data
Reliability (The model based on experimental or epidemiological data is more reliable.)	Depends on subjective judgments	Partly depends on experimental data	Depends on experimental or epidemiological data
Guidance (The model provides explanatory guidance that helps implementation.)	No guidance available	Guidance manuals are available, but lack examples of applications	Guidance manuals are available and give many examples of applications
Practicability (The model that provides a control strategy to reduce health risks is more practical.)	No control strategy is available	-	Control strategy is available
Operability (The model is convenient to use.)	Complicated to use	-	Easy to use

A radar diagram was drawn to directly reflect the distribution of the six methods at different levels for each evaluation indicator. The total score for each model in the seven evaluation indicators was calculated to evaluate whether there was a comprehensive advantage for each model.

Quantitative Comparisons

The RR is defined as the ratio between the risk level of a particular risk factor (obtained through the given model) and the maximum risk level for that model. For example, in the Singapore model the risk level for benzene at a paint spraying location is 3, while the maximum risk level is 5. Hence the RR of benzene is 0.6 (3/5). RRs represent the relative risk levels and are therefore comparable across different models.

Each model has its own maximum risk level based on its methodology. For example, while the maximum risk level for the Romanian model is 7, it is 5 for the Singaporean model. However, the EPA model only provides two risk levels (< 1 or ≥ 1), and the COSHH model only provides four risk control levels. To calculate the RRs of the EPA and COSHH models, their risk rank was converted based on the classification criteria of the Singapore model. In the Singapore model, four specific cut points (i.e. 0.1, 0.5, 1.0 and 2.0 times the permissible exposure limit [PEL]) are used to categorize the exposure ratings (ER). The five total risk levels are then calculated based on the five levels of ER and HR. Generally, amounts 0.1 and 0.5 times greater than the PEL as established by the NIOSH and OHSA in the USA are considered as the safety and action levels, respectively. Based on these considerations, the two risk levels (< 1 or ≥ 1) of the hazardous quotient (HQ) for the non-carcinogenic evaluation in the EPA model were re-categorized into five maximum risk ranks (e.g., < 0.1, 0.1-0.5, 0.5-1.0, 1.0-2.0, and ≥ 2). The four risk control levels of the COSHH model were converted into five maximum risk ranks based on a comparative study[19],[32)] which assessed a parallel between the risk control levels obtained from the COSHH and the Singaporean models, in which the control strategy (CS) levels of 2, 3, and 4 were equivalent to risk levels of 3, 4, and 5.

Statistical Analysis

One-way analysis of variance (ANOVA) was used to analyze the RRs for each individual hazard from the various OHRA models using the LSD comparison method when variances were equal, or the Dunnett T3 comparison method when variances were heterogeneous. The Spearman correlation analysis (abnormal distribution) was utilized to analyze the correlation of RRs.

Results

Qualitative Differences in Key Information between Different Models

Table 3 summarizes the key information for the different OHRA models. The methodological principles of the various OHRA models are different in their hazard and exposure assessment approaches. For example, while the EPA model uses a quantitative dose-response assessment, the COSHH Essential model is based on the hazard or exposure banding approach, the Singapore model uses a semi-quantitative risk calculation based on hazard and exposure classifications, and many qualitative models like the Australian, Romanian, and ICMM models are based on a matrix method. Each model has its own strengths and limitations.

Table 3.

Qualitative differences in key information between the different models.

Model	Attribute	Scope	Assessment method	Risk classification	Strengths	Weaknesses
Numbers in brackets indicate literature references.
EPA [9]	Quantitative	Chemicals	Dose- response assessment	2 levels	1. Carcinogenic and non-carcinogenic assessment2. Reliability based on epidemiological or toxicological data	1. Limited to chemical poisons with IUR and RfC values2. Difficult to differentiate multiple risk levels
Australia [11]	Qualitative	Chemicals, physical factors, and dust	Manual diagram	5 levels	1. Good operability and ease of use2. Broad scope of evaluated substances3. Appropriate for moderate- and small-sized businesses	1. Relies on subjective judgment2. Requires professional knowledge
Romania [12]	Qualitative	Chemicals, physical factors, and dust	Matrix	7 levels	1. Broad scope of evaluated substances2. Calculation of total risk level	1. Relies on subjective judgment2. Difficult to judge the probability of consequences or adverse events
Singapore [13, 33]	Semi- quantitative	Chemicals and dust	Semi- quantitative calculation	5 levels	1. Uses both quantitative and qualitative methods2. Uses an exposure index method when air monitoring data is absent	1. The exposure index classification is relatively crude
ICMM [15]	Qualitative	Chemicals, physical factors, and dust	Matrix, quantitative rating	4 levels	1. Broad scope of evaluated substances2. Application to various industries	1. Relies on subjective judgment2. Overestimates risk using the quantitative rating method
COSHH Essentials [17, 22, 33]	Qualitative	Chemicals and dust	Banding	4 levels	1. Good operability and ease of use2. Focuses on middle- and small-sized businesses.	1. Overestimates risk levels2. Occurrence of bias when judging liquid volatility

Qualitative Differences Obtained from the Multi-criteria Analysis

The radar diagram (Fig. 1) directly shows a diverse distribution of the six methods at different levels in each evaluation indicator. Each model has its own score for the seven evaluation indicators. For the validation, reliability, and guidance indicators, the EPA, Singapore, and COSHH models were ranked at relatively higher levels than the other models, and thus had greater scores. For the practicability and operability indicators, the Singaporean, Australian, and ICMM models' levels were relatively higher and consequently got higher scores. On the whole, the total scores for the Singaporean, COSHH, and EPA models were 19, 17, and 15, respectively, which were greater than that for the Australian, Romanian, or ICMM models (13 for each).

Fig. 1.

Radar diagram showing the diverse distribution of the OHRA models across the different evaluation indicators. The total scores for the Singaporean, COSHH, and EPA models were 19, 17, and 15, respectively, which were greater than the scores for the Australian, Romanian, and ICMM models (13 for all).

Quantitative Differences in Risk Ratios between the Different Models

Table 4 and Fig. 2 show the results of the quantitative comparisons between the different models. Fig. 2 illustrates that the risk ratios (RRs) obtained from the EPA, COSHH, and Singapore models in the electroplating industry were significantly higher than those in the wood furniture manufacturing industry or crane manufacturing industry (P < 0.05). This finding was consistent with the electroplating industry's own risk assessment classification of the "most severe" level. Likewise, the relatively lower RRs in the wood furniture manufacturing industry or crane manufacturing industry agreed with their "relatively severe" level risk classification. The other three models did not differentiate between the RRs for the different industries. Table 4 shows that the EPA model yielded the highest average RR (0.83 ± 0.29) in all three industries (P < 0.05). The RRs of the COSHH, Singaporean, and Australian models were second-highest, and the Romanian and ICMM models had the lowest RR values. Thus, the order of the RRs for the six models is RR EPA > RR Singapore, RR COSHH, and RR Australia > RR ICMM and RR Romania (P < 0.05).

Table 4.

Quantitative differences in the risk ratios between the different models (Mean ± SD).

Industry	Risk classification	n	EPA model		Australian model		Romanian model		Singaporean modelg		ICMM model		COSHH model
			Risk level	RR	Risk level	RR	Risk level	RR	Risk level	RR	Risk level	RR	Risk level	RR
RR: risk ratio; n: total number of risk levels for all risk factors in three enterprises of each industry using a risk assessment model; Risk level: the range of risk levels for all risk factors evaluated by each model. Risk calculations are according to the "Management catalog of occupational hazard risk classification of construction projects" (2012 edition) issued by the State Administration of Work Safety of China.aP < 0.05 compared to the ICMM model; bP < 0.05 compared to the Romanian model; cP < 0.05 compared to the Singaporean model; dP < 0.05 compared to the Australian model;eP < 0.05 compared to the COSHH model; fP < 0.05 compared with the wood furniture manufacturing or crane manufacturing industry. g The average risk ratios were calculated based on the exposure ratio (ER) and exposure index (EI) methods in the Singapore model.
Wood furniture manufacturing	Relatively severe	60	Low~ extremely high	0.80±0.30	Moderate~ high	0.52±0.92	Very low~ medium	0.38±0.73	Low~ high	0.53±0.14	Low~ medium	0.44±0.25	Low~ extremely high	0.64±0.27
Electroplating	Most severe	15	Extremely high	1.00±0.00f	Moderate~ high	0.54±0.15	Minimal~ low	0.30±0.12	Medium~ high	0.64±0.90f	Low~ medium	0.38±0.18	Medium~ extremely high	0.80±0.20f
Crane manufacturing	Relatively severe	93	Negligible~ extremely high	0.84±0.30	Low~ high	0.65±0.14	Very low~ low	0.43±0.10	Low~ medium	0.50±0.13	Low~ high	0.36±0.17	Low~ extremely high	0.63±0.22
Total	-	168	Negligible~ extremely high	0.83±0.29abcde	Low~ high	0.57±0.23ab	Minimal~ medium	0.40±0.10	Negligible~ high	0.59±0.16ab	Low~ high	0.39±0.20	Low~ extremely high	0.65±0.24ab

Fig. 2.

The risk ratios of each model in the three industry types. Risk ratios derived from the EPA, COSHH, and Singapore models in the electroplating industry were significantly greater than those in the other two industries, which is consistent with the industries' own risk classifications (*P < 0.05).

The correlation analysis of RRs among the six models showed that the EPA model did not correlate with the other five models (P > 0.05). The COSHH RRs only correlated with those of the Singaporean model (correlation coefficient 0.437, P < 0.05). The RRs of the Singaporean model correlated with those of the Romanian, Australian, and ICMM models (correlation coefficients 0.802, 0.887, and 0.693, respectively; P < 0.01). Similarly, there was a positive correlation between the Romanian, Australian, and ICMM models (P < 0.05).

Discussion

Theoretical frameworks for comparative studies of different OHRA models have not been widely reported. This study aimed to explain the qualitative and quantitative differences in the methodologies using approaches such as literature review, expert consultation, multi-criteria analysis, and quantitative analysis using RRs. The theoretical framework established in this study proved to be effective.

Analysis of key information for the different OHRA models showed that the methodological principles for individual hazard or exposure assessments can be quite different between models. For example, while the EPA model uses a quantitative dose-response assessment, the COSHH Essential model is based on a banding approach, the Singaporean model uses a semi-quantitative risk calculation, and many qualitative models are based on matrix methods. As a result, each model possesses its own strengths and limitations based on their methodologies. The results obtained from our key information analysis strategy are consistent with other reviews on OHRA methodology[18],[28)]. More studies should be conducted to examine the strengths and weaknesses of different models and assist in their further refinement and utility[34)].

The multi-criteria analysis further evaluated the qualitative differences between the different models. A radar diagram showed that the OHRA models exhibited a diverse combination of high and low rankings for the different evaluation indicators, suggesting that several factors must be considered when using multiple models to perform OHRA. Our results suggest that the EPA, COSHH, and Singapore models might achieve more accurate outcomes since they are based on independent experimental or epidemiological data, and thus may exhibit better reliability and validity. These three models also provide good guidance for their implementation through multiple approaches like official websites or published documents. Both the COSHH model and the Singapore model were considered more practical than other models since they provide detailed control strategies to reduce occupational health risks. In addition, all of the qualitative and semi-quantitative models were relatively easy to use in terms of operability. When all of the evaluation indicators were considered, the Singapore, COSHH, and EPA models got higher total scores, suggesting that these models might be the most appropriate for OHRA practice in the workplace due to their comparative advantages, especially in reliability.

The qualitative reliability assessments for these three models were supported by quantitative comparisons. Fig. 2 shows that the RRs derived from the EPA, COSHH, and Singaporean models are consistent with the current risk classifications in the examined industries, suggesting that in some industries these models are able to more accurately identify high occupational risk than the Romanian, Australian and ICMM models. The quantitative comparisons also validated the qualitative comparison results. Since the EPA, Singapore and COSHH models use quantitative, semi-quantitative, and qualitative methods, respectively, combining these three models might be advantageous when performing OHRA. Our research team has previously proposed that quantitative, semi-quantitative, and qualitative methods can be applied in combination when conducting OHRA[18)]. Table 4 shows that the RR for the EPA was significantly greater than the RRs for the COSHH, Singaporean, and Australian models; which in turn were greater than the RRs for the ICMM and Romanian models. This indicates that the use of different models will yield diverse risk assessment results. This phenomenon also reminds users of the necessity for careful selection of evaluation models. The relatively smaller RRs of the Australian, ICMM, and Romanian models might be due to underestimation of risk levels, which are usually determined based on the subjective judgments of the users.

Correlation analysis using the RRs was used to test the agreement between the different models, and found that the EPA model was strongly independent, with no correlation with the other five models. This is because the EPA model only applies the IUR and RfC values when determining chemical toxicity, resulting in a relatively narrow scope[9)]. Additionally, the COSHH model only correlated with the Singapore model, indicating that the COSHH model was also relatively independent. This might be due to the unique banding evaluation method used in the COSHH model, which is quite different from the matrix method used by the other qualitative OHRA models. The correlation between the RRs of the COSHH and Singapore models is supported by a previous parallel study which concluded that the CS levels of 2, 3, and 4 in the COSHH model were equivalent to the risk levels of 3, 4, and 5 in the Singapore model[19],[33)]. The Singapore model correlated with all models except for the EPA model, suggesting the Singapore model has good overall compatibility. This is because the Singapore model, as a semi-quantitative method, possesses characteristics of both the quantitative and qualitative models, and thus is able to make up for the shortcomings of the quantitative and qualitative methods. Finally, good consistency was found between the three similar qualitative models, i.e. the ICMM, Australian, and Romanian models.

The main limitation of this study is the small number of enterprises tested in each industry. This case study considered only nine factories in three industries. It would be useful to replicate the study in many more factories to further compare the models and to see if they perform similarly across multiple samples.

Conclusion

The following conclusions can be drawn from this study: (1) the theoretical framework developed here can distinguish qualitative and quantitative differences between the different OHRA models, (2) each model possesses its own strengths and limitations depending on its unique methodological approach, (3) due to their comprehensive advantages, it may be advantageous to combine the EPA, Singapore and COSHH models when developing an OHRA strategy, and (4) the Singapore model best parallels the other OHRA models in terms of RRs, while the EPA model is highly independent. This study lays a foundation for strengthening the theoretical framework of these OHRA models, and also provides a recommendation for joint application of risk assessment methods, which will benefit the establishment and improvement of OHRA technical specifications in developing countries. More comparative studies using multiple methods should be conducted in key industries with a high probability of occupational health hazards.

Acknowledgments: This work was sponsored by Zhejiang Provincial Program for the Cultivation of High-level Innovative Health Talents. Additional support was provided by the Natural Science Foundation of China (81472961), the Co-constructed Projects by the National Health and Family Planning Commission of China and the Health Bureau of Zhejiang Province (No. WSK2014-2-004),the Key Research and Development Program of Zhejiang Province of China (No. 2015C03039), the Natural Science Foundation of Ningbo (No.2017A610272), and the Tianjin Municipal Natural Science Foundation (15JCQNJC45100).

Conflicts of interest: None declared.