Indicators to assess physiological heat strain - Part 3: Multi-country field evaluation and consensus recommendations.

In a series of three companion papers published in this Journal, we identify and validate the available thermal stress indicators (TSIs). In this third paper, we conducted field experiments across nine countries to evaluate the efficacy of 61 meteorology-based TSIs for assessing the physiological strain experienced by individuals working in the heat. We monitored 372 experi-enced and acclimatized workers during 893 full work shifts. We continuously assessed core body temperature, mean skin temperature, and heart rate data together with pre/post urine specific gravity and color. The TSIs were evaluated against 17 published criteria covering physiological parameters, practicality, cost effectiveness, and health guidance issues. Simple meteorological parameters explained only a fraction of the variance in physiological heat strain (R2 = 0.016 to 0.427; p < 0.001), reflecting the importance of adopting more sophisticated TSIs. Nearly all TSIs correlated with mean skin temperature (98%), mean body temperature (97%), and heart rate (92%), while 66% of TSIs correlated with the magnitude of dehydration and 59% correlated with core body temperature (r = 0.031 to 0.602; p < 0.05). When evaluated against the 17 published criteria, the TSIs scored from 4.7 to 55.4% (max score = 100%). The indoor (55.4%) and outdoor (55.1%) Wet-Bulb Globe Temperature and the Universal Thermal Climate Index (51.7%) scored higher compared to other TSIs (4.7 to 42.0%). Therefore, these three TSIs have the highest potential to assess the physiological strain experienced by individuals working in the heat.

Introduction

Working in hot environments directly threatens people’s ability to live healthy and productive lives. Thirty-five percent of people who work in the heat experience physiological heat strain, which is associated with unfavorable health out-comes [1]. A number of heat mitigation measures have been proposed to address this issue, most of which require the adoption of thermal stress indicators (TSIs) [2-5]. These indicators are mathematical equations that combine two or more meteorological factors (temperature, humidity, wind, and solar radiation) to explain the physiological heat stress/strain experienced by someone.

In a series of three companion papers published in this Journal, we addressed the following three objectives. We identified the TSIs developed since the dawn of scientific research (part 1) [6], we conducted a Delphi exercise to understand what is important to consider when adopting a TSI to protect individuals who work in the heat (part 2)[7], and we conducted field experiments across nine countries to evaluate the efficacy of each TSI for quantifying the physiological strain experienced by individuals who work in the heat (part 3; present article). The first article [6] of this series identified 340 unique TSIs previously developed to assess the heat stress and strain experienced by people performing various activities over a wide operating range and ambient conditions. Of these, 61 TSIs are meteo-based [i.e., can be calculated utilizing only meteorological data (air temperature, relative humidity, wind speed, and solar radiation)] and have been designed for use in occupational settings. Many of these TSIs bear the same name but use different calculation methods (e.g., Wet-Bulb Globe Temperature for indoor and outdoor conditions) and, therefore, we considered these as unique TSIs. Some of these TSIs have been frequently used to quantify the magnitude of physiological strain experienced by workers [8-13]. However, the vast majority of occupational meteo-based TSIs have not undergone such testing and, therefore, it remains unknown whether they could be used to improve the efficacy of occupational heat stress mitigation strategies.

Previous studies assessing whether TSIs reflect the physiological heat strain experienced by working people have typically used body core temperature, skin temperature, and/or heart rate as criteria to judge the validity of the TSIs [9,14-21]. But, until recently, it remained unclear if body core temperature, skin temperature, heart rate, or hydration state are criteria that a TSI should be able to accurately reflect. Also, we remained naive as to the relative importance of each of these criteria in the assessment of a TSI. In the second article [7] of this three-paper series, we performed a Delphi exercise identifying 17 criteria with varying weights that a TSI should be able to align with for protecting individuals who work in hot environments. The Delphi methodology is a well-established structured communication technique based on collecting opinions from a panel of experts while eliminating the influence of participants upon one another (i.e., using the rules of brainstorming where participants withhold judgment) [22]. A necessary next step, therefore, would be to evaluate the above-mentioned 61 occupational meteo-based TSIs against these 17 criteria.

Identifying the most efficacious meteo-based TSI is critical since occupational heat stress affects workers’ health and productivity [1,14,15,23-25]. Working under occupational heat stress conditions leads to progressive rises in core temperature, cardiovascular strain, and fluid depletion, which can have acute and chronic effects, and may lead to heat stroke or death [1,26-28]. Also, meta-analytic data shows that 30% of individuals who work under heat stress report productivity losses [1] with potentially large spill over effects on the entire economy [29]. In the coming decades, individuals working in currently temperate climates may be at risk for heat illness on nearly 40 days/year [30].

Based on knowledge gained in the previous parts of this series of companion papers, where all occupational meteo-based TSIs were identified [6] and the 17 criteria to consider when adopting a TSI were determined [7], the aim of this paper was to evaluate the capacity of all occupational meteo-based TSIs for quantifying the magnitude of physiological heat strain experience by workers in a large-scale series of field studies performed in multiple industries across several countries. The testing hypothesis was that these TSIs would reflect the magnitude of physiological heat strain experienced by workers. It is hoped that this work will facilitate the use of accurate environmental monitoring strategies in occupational settings to protect workers and the general population.

Methodology

The experimental protocol (ClinicalTrials.gov ID: NCT04160728) used in this study was approved by the Charles Darwin University Human Research Ethics Committee (no. H13144), the National Bioethical Review Board of Cyprus Ethics Review Board (no. 2017.01.61), the University of Ottawa Health Sciences and Science Research Ethics Board (no. H10–04–04b), the Danish National Committee on Health Research Ethics (no. 55907_v3_02012017), the University of Thessaly, School of Exercise Science (no. 1217), and the National Committee for Medical Ethics at the Ministry of Health of the Republic of Slovenia, in accordance with the Declaration of Helsinki. The study involved monitoring 372 (males: 87.3% and females: 12.7%) experienced (assessed work was their primary occupation) and acclimatized (i.e., living and working in the area for ≥2 months) workers (Table 1) during one to five work shifts (in total 893 full work shifts) performed in nine countries (Australia, Canada, Cyprus, Denmark, Greece, Qatar, Slovenia, Spain, and the United States of America). It is important to note that some jobs were tested across multiple countries at different time periods. Parts of the data from Cyprus [9,15], Greece [15], Qatar [15,17], Slovenia [31], Spain [15], and North American [16,18,19] were previously published to investigate the physiological heat strain experienced by workers in these countries. Before their participation in the study, written informed consent was obtained from all volunteers after detailed explanation of all the procedures involved.

Table 1.

Participant characteristics. Numbers in squared brackets correspond to International Standard Classification of Occupations (ISCO-08) codes.

Job [ISCO-08 code]	Number	Mass (kg)		Height (m)		Age (years)
	worker/s	Mean	SD	Mean	SD	Mean	SD
Agriculture
Grape loading worker [9211]	4	80.4	8.5	1.72	0.03	33.3	5.7
Grape picking worker [9211]	9	72.6	18.2	1.69	0.07	41.8	12.2
Plowing (hand tractor) worker [9331]	2	81.0	4.2	1.79	0.04	24.5	5.0
Potato picking worker [9211]	10	72.4	16.2	1.67	0.04	38.7	12.2
Potato loading worker [9211]	2	83.0	1.4	1.75	0.01	46.0	1.4
Straw harvesting worker [9211]	33	66.0	9.2	1.70	0.06	31.6	6.7
Construction
Carpenter [7115]	22	63.6	8.9	1.64	0.06	36.2	7.9
Helper [9313]	14	65.7	11.0	1.66	0.06	29.1	8.2
Mason [9313]	18	66.4	5.6	1.65	0.06	35.6	7.6
Steel fixer [9313]	19	68.8	11.8	1.66	0.07	37.3	8.7
Rigger [7215]	1	60.4		1.66		27.0
Electrician [7411]	1	71.1		1.69		31.0
Scaffolder [9313]	3	72.5	15.7	1.70	0.07	24.0	8.7
Framework [9313]	14	79.2	13.3	1.76	0.11	41.7	10.1
Brick layer [9313]	6	83.3	11.9	1.71	0.04	43.3	10.4
Crane worker [8344]	1	97.1		1.78		45.0
Forklift driver [8344]	1	72.4		1.60		49.0
Manufacturing
Aluminum - remelt operator [7211]	10	80.0	23.6	1.69	0.11	42.2	12.3
Aluminum - supervisor in smelting furnace (electrolysis) [3135]	3	79.3	12.5	1.76	0.03	39.7	11.6
Aluminum - worker in smelting furnace (electrolysis) [8121]	8	86.4	14.2	1.83	0.08	36.6	7.4
Services
Electrical utilities (engineer) [3113]	3	87.3	13.4	1.70	0.07	52.3	3.3
Ground work [7413]	8	100.0	18.1	1.80	0.09	37.6	11.6
Bucket work [7413]	7	98.0	20.4	1.81	0.13	40.0	8.1
Ground/bucket and pole work [7413]	12	92.8	14.7	1.78	0.06	32.5	10.4
Military
Military personnel [0110] †	37	73.5	12.9	1.72	0.09	27.8	5.7
Mining
Miner [8111] *	51	88.9	11.3	1.80	0.05	37.0	6.6
Mining rescue [2269]	9	87.4	12.1	1.78	0.07	47.0	9.0
Tourism
Bar man/woman [5132]	13	72.4	10.2	1.70	0.08	32.6	9.3
Waiter [5131]	16	67.9	13.6	1.73	0.11	29.6	9.7
Laundry service [8157]	5	73.0	10.8	1.66	0.07	52.4	12.3
Bus driver/hotel driver [8331]	6	92.8	13.3	1.78	0.06	38.5	6.5
Parking worker [5414]	1	80.0		1.72		43.0
Cook/chef [3434]	5	91.3	19.4	1.76	0.08	32.6	8.9
Dish washing [9412]	2	59.0	12.7	1.67	0.04	44.5	0.7
Cook/charcoal grilling [5120]	4	86.3	12.1	1.75	0.05	31.5	7.0
Baker/pastry chef [7512]	2	81.0	8.5	1.82	0.02	37.5	0.7
Butcher [7511]	1	93.0		1.80		36.0
Gardener [9214]	4	78.0	12.6	1.74	0.08	41.0	12.3
Hotel maid [5151]	2	58.5	0.7	1.67	0.02	42.0	8.5
Mini market [5211]	1	64.0		1.59		44.0
Hotel manager [1411]	1	78.0		1.74		29.0
Pool boy [9112]	1	75.0		1.80		18.0
Overall	372	76.14	15.90	1.72	0.09	35.36	9.89

Notes:

† data were collected during two different military duties (i.e., hike and guard duty).

* anthropometric data available for 32 out of 51 mine workers

Data collection

Self-reported age, body stature, and body mass were collected prior to the experiment. During the field studies, continuous heart rate, core temperature, and mean skin temperature data were collected using wireless heart rate monitors (Polar Team2, Polar Electro Oy, Kempele, Finland), telemetric capsules (BodyCap, Caen, France), and wireless thermistors (iButtons type DS1921H, Maxim/Dallas Semiconductor Corp., USA), respectively. Telemetric capsules were ingested approximately 30 minutes prior to each work-shift, as the time following ingestion does not significantly affect the validity of core temperature measurement in the absence of fluid consumption [32]. Skin temperature data were collected from four sites (chest, arm, thigh, and leg) and were expressed as mean skin temperature according to Ramanathan: mean skin temperature = 0.3 × (chest + arm) + 0.2 × (thigh + leg) [33]. For the North American studies, core temperature was recorded continuously using a wireless integrated physiological monitoring system (VitalSense, Mini Mitter, Bond, OR), while skin temperature was continuously measured at three sites (forearm, abdomen, and calf) using wireless telemetric dermal patches (VitalSense, Mini Mitter, Bond, OR) and mean skin temperature was expressed as the mean of the three sites [18]. The adoption of different methods to estimate mean skin temperature (i.e., three vs. four measuring sites, different sites) did not interfere with our findings (please see “Results” section). Mean body temperature was computed using core temperature and mean skin temperature data according to Burton: mean body temperature = 0.35 × mean skin temperature + 0.65 × core temperature [34]. Urine samples were collected and analyzed by experienced researchers pre- and post-work-shift to assess urine specific gravity using a refractometer (PAL-10S, Atago Co., Ltd., Fukaya, Saitama Prefecture, Japan) and urine color according to a urine color scale [35]. Differences in urine specific gravity and urine color between pre- and post-work-shift samples were calculated to assess the labor-induced dehydration.

The collected data included: heart rate (614 work shifts; 4868 work hours), body core temperature (610 work shifts; 3949 work hours), mean skin temperature (729 work shifts; 5719 work hours), urine specific gravity (532 work shifts), and urine color (484 work shifts). Throughout the monitored work shifts, continuous meteorological data [air temperature (°C), relative humidity (%), wind speed (m/s), and solar radiation as expressed by globe temperature (°C)] were collected using portable weather stations (Kestrel 5400FW, Nielsen-Kellerman, Pennsylvania, USA) installed in close proximity to the workers. For the North American studies, air temperature and relative humidity were monitored with a portable weather station (Vantage Vue, Davis Instruments Corp, Hayward, CA) installed in close proximity to the workers. Then, we used published methods [36] to obtain wind speed and globe temperature for each work shift based on the date, time, and location provided.

All data were calculated as hourly means to minimize the influence of instantaneous events (e.g., drinking cold water, dropping cold water on the body, eating, moving to a different environment for limited amount of time) according to previous literature [8]. Thereafter, the meteorological data were used to compute the 61 TSIs found to be suitable for use in occupational settings using the “Thermal Stress Indicators calculator” developed in the first paper of this series [6] that can be freely downloaded using the following link: www.famelab.gr/research/downloads.

Data analysis against meteorological parameters

Pearson’s correlation was utilized to explore potential relationships between the physiological heat strain (core temperature, mean skin temperature, and heart rate) and the magnitude of dehydration (differences in the urine specific gravity and color between pre- and post-work-shift urine samples) of the workers at the end of the work shift against the collected meteorological data (temperature, humidity, wind, and solar radiation). For this purpose, hourly and daily means were utilized to conduct the correlations analysis for the thermal strain variables and dehydration level, respectively. Moreover, stepwise linear regressions were performed to determine the influence of all four (temperature, humidity, wind, and solar radiation) meteorological factors on the physiological heat strain (mean body temperature, core body temperature, mean skin temperature, heart rate, urine specific gravity, and urine color) experienced by people who work in the heat.

Data analysis to rank the 61 TSIs

The list of the 61 TSIs found to be suitable for use in occupational settings in the first article of this three-paper series [6] were combined with the results of Delphi exercise presented in the second article of the series [7] which identified 17 criteria to be considered for adopting a TSI in occupational settings. A detailed description of the 17 Delphi criteria is shown in Table 2.

Table 2.

The 17 Delphi criteria determined in the second article of this three-paper series [7] used for the quantitative assessment of the 61 TSIs.

#	Delphi criteria	Description	Weightings
1	Pearson’s correlation coefficient with core temperature	Pearson’s r value (range: 0 to 1)*	0.1170
2	Pearson’s correlation coefficient with mean skin temperature	Pearson’s r value (range: 0 to 1)*	0.0622
3	Pearson’s correlation coefficient with mean body temperature	Pearson’s r value (range: 0 to 1)*	0.0605
4	Pearson’s correlation coefficient with heart rate	Pearson’s r value (range: 0 to 1)*	0.0560
5	Area Under the ROC curve for increased (>36.7°C) mean body temperature	Area Under the Curve value (range: 0 to 1)	0.0413
6	Sensitivity for increased (>36.7°C) mean body temperature	Sensitivity value (range: 0 to 1)*	0.0710
7	Specificity for normal (≤36.7°C) mean body temperature	Specificity value (range: 0 to 1)*	0.0220
8	Risk ratio for increased (>36.7°C) mean body temperature	Risk ratio value (range: 0 to 1)*†	0.0415
9	Providing categories characterizing the level of heat stress	No categories = 0; Categories = 1	0.1117
10	Use heat stress categories to provide guidance	No guidance = 0; Guidance = 1	0.1435
11	Practicality and cost-effectiveness during year 1	Ratio to the most cost-effective index in year 1 found in our systematic search (range: 0 to 1)	0.0415
12	Practicality and cost-effectiveness in a 10-year period	Ratio to the most cost-effective index in 10-years found in our systematic search (range: 0 to 1)	0.0533
13	Pearson’s correlation coefficient with hydration status (average correlation of urine specific gravity and urine color)	Pearson’s r value (range: 0 to 1)*	0.0595
14	Area Under the ROC curve for extreme (>39.0°C) mean body temperature	Area Under the Curve value (range: 0 to 1)	0.0235
15	Sensitivity for extreme (>39.0°C) mean body temperature	Sensitivity value (range: 0 to 1)*	0.0450
16	Specificity for non-extreme (≤39.0°C) mean body temperature	Specificity value (range: 0 to 1)*	0.0190
17	Risk ratio for extreme (>39.0°C) mean body temperature	Risk ratio value (range: 0 to 1)*†	0.0315

Note: * = non-significant values (p > 0.05) were marked as “0”; † = values greater than “1” were marked as “1”.

Correlation coefficient (r)

Pearson’s correlation coefficients (r) were utilized to investigate the relationships between each of the 61 TSIs and the physiological data [core temperature (Delphi criterion 1), mean skin temperature (Delphi criterion 2), mean body temperature (Delphi criterion 3), heart rate (Delphi criterion 4), and hydration level (computed as the average correlation of urine specific gravity and urine color; Delphi criterion 13)]. The magnitude of associations was determined as “very weak” (r = 0.00 to 0.19), “weak” (r = 0.20 to 0.39), “moderate” (r = 0.40 to 0.59), “strong” (r = 0.60 to 0.79), and “very strong” (0.80 to 1.00) [37].

Receiver operating characteristics (ROC) curve

ROC curves were used to calculate the diagnostic performance of a binary classifier tool considering its sensitivity (see “2.3.c. Sensitivity” subsection) and specificity (see “2.3.d. Specificity” subsection) at various threshold settings. In the present study, ROC curves were utilized to investigate the ability for each of the 61 TSIs to diagnose increased (>36.7°C; Delphi criterion 5) or extreme (>39.0°C; Delphi criterion 14) mean body temperature using the Area Under the ROC curve. The Area Under the ROC curve ranges from zero to one and evaluated the capacity of each TSI to diagnose when workers have increased mean body temperature. A value of “0.5” corresponds to random chance, while a value of “0” or “1” corresponds to a perfect predictor. In this light, all values were converted to reflect their predictive capacity by adding 0.5 to the absolute difference between each value and 0.5. For example, the values of 0.4 and 0.6 are characterized by the same predictive capacity (ROC score = 0.6) despite being different. The equation used to calculate the ROC score can be found below:

Mean body temperature was used as a marker of excessive heat strain as it incorporates both core temperature and mean skin temperature. The threshold of 36.7°C mean body temperature [36.7°C > (38.0°C × 0.65) + (34.1°C × 0.35)] was calculated using: (1) Burton’s equation [34] (mean body temperature = core temperature × 0.65 + mean skin temperature × 0.35), (2) the safety threshold of 38°C, which according to the World Health Organization [38] “it is considered inadvisable for the deep body temperature to exceed 38°C for prolonged daily exposures in heavy work”, and (3) the standard value of 34.1°C mean skin temperature recommended in the ISO 7933 [39] for occupational settings.

Sensitivity

Sensitivity (also called the true positive rate) for each of the 61 TSIs was defined as the capacity to identify the proportion of workers with increased (>36.7°C; Delphi criterion 6) or extreme (>39.0°C; Delphi criterion 15) mean body temperature, who are correctly identified as having increased or extreme mean body temperature, respectively. Sensitivity values for each TSI between the thermoneutral category (as defined by each TSI) and each category pertaining to heat stress were calculated. Our analysis did not consider categories pertaining to cold stress. The overall sensitivity score for each TSI was calculated as the average sensitivity of all heat stress categories used by each TSI, with sensitivity values demonstrating non-significant Cohen’s Kappa statistic being marked as “0”.

Specificity

Specificity (also called the true negative rate) for each of the 61 TSIs was defined as the capacity to identify the proportion of workers with normal (≤36.7°C; Delphi criterion 7) or non-extreme (≤39.0°C; Delphi criterion 16) mean body temperature who are correctly identified as not having increased or extreme mean body temperature, respectively. Specificity values for each TSI between the thermoneutral category (as defined by each TSI) and each category pertaining to heat stress were calculated. Our analysis did not consider categories pertaining to cold stress. The overall specificity score for each TSI was calculated as the average specificity of all heat stress categories used by each TSI, with specificity values demonstrating non-significant Cohen’s Kappa statistic being marked as “0”.

Risk ratio/relative risk

The capacity for each of the 61 TSIs to diagnose the increase in risk for having increased (>36.7°C; Delphi criterion 8) or extreme (>39.0°C; Delphi criterion 17) mean body temperature when being exposed to meteorological parameters characterized by different heat stress categories were calculated using Excel spreadsheets. Risk ratios between all the heat stress categories and the thermoneutral category were calculated. If the risk for having increased or extreme mean body temperature was more than 1 compared to the thermoneutral category, it was considered that the current heat stress category was characterized by a “perfect” risk assessment capacity and was given a score equal to 1 divided by the number of categories utilized by the current TSI. For a TSI to receive a “perfect” risk ratio score, it should be characterized by a “perfect” risk assessment capacity in each one of its heat stress categories. For instance, in a hypothetical case where one TSI uses four heat stress categories (low, moderate, high, and very high heat stress) and two of them were given a “perfect” risk assessment score of 1 while the other two were scored with 0, the overall risk ratio score of that TSI would be 0.5. It is important to note that risk ratios characterized by no statistically significant results were given a score of 0. The equation used to calculate the overall risk ratio score can be found below: n = number of heat stress categories used by the current TSI; cat1 = risk ratio between the first heat stress category and the thermoneutral one:

Categories

TSIs that provided categories indicating the level of heat stress (e.g., a WBGT value of 30°C indicates high heat stress) experienced by workers (Delphi criterion 9) scored 1 in this criterion. TSIs that did not provide heat stress categories scored 0 in this criterion.

Guidance

TSIs that provided health-related recommendations associated to the heat stress categories (Delphi criterion 10) scored 1 in this criterion. TSIs that did not provide health-related recommendations scored 0 in this criterion.

Cost-effectiveness

Cost effectiveness was calculated as the ratio of Cost and Effectiveness. Cost effectiveness analysis of the first year (Delphi criterion 11) of using a TSI incorporates all three cost-related parameters presented below, while cost effectiveness analysis for 10 years (Delphi criterion 12) of using a TSI incorporates the cost of buying the equipment only during year 1. Cost was calculated based on three parameters:

Cost of equipment: TSIs requiring more than one meteorological factor are more expensive to be computed. Based on an extensive internet search in relevant providers of scientific, commercial, and industrial equipment, the average cost for obtaining the equipment to assess the necessary meteorological parameters with acceptable accuracy [air temperature: ~0.5°C; relative humidity: ~6.0% (accuracy of partial vapor pressure should be ± 0.15 kPa in a measuring range between 0.5 kPa and 3.0 kPa); wind speed: ~(0.1 + 0.05 × wind speed) m/s; and solar radiation: ~5 W/m [2,40] was as follows: air temperature and relative humidity = 73.9 EUR; wind speed = 46.1 EUR; solar radiation = 144.8 EUR. For temperature and humidity, the average cost derived from obtaining prices for nine different handheld instruments ranging from 22.7 EUR to 172.9 EUR. Similarly, the average cost for a wind speed instrument was obtained as the average of 10 different handheld wind speed meters ranging from 7.7 EUR to 232.1 EUR. Finally, the average cost for assessing solar radiation resulted after collecting data for nine different handheld solar radiation meters ranging from 47.3 EUR to 295.8 EUR. Therefore, the cost of equipment to operate a TSI was calculated as the sum of the costs associated with the environmental parameters needed to compute this TSI.

Time required to measure each meteorological factor: A TSI that incorporates more meteorological factors is considered more expensive to measure. The global average wage (12.94 EUR/hour or 0.22 EUR/minute) in EUR was calculated based on data obtained from the Organization for Economic Co-Operation and Development (OECD) [41] and the International Labor Organization [42,43] for the 58 largest economies worldwide. This calculation incorporates all OECD countries, as well as other countries, such as Bangladesh, Brazil, China, Egypt, India, Iran, Mexico, Pakistan, Russia, and Turkey. Thereafter, the cost of measuring each TSI on an hourly basis was calculated based on the amount of time required to be measured (we assumed two minutes for each meteorological factor). For instance, if a TSI incorporates only two meteorological parameters (0.86 EUR/hour) is considered cheaper to be measured compared to a TSI incorporating all four meteorological factors (1.73 EUR/hour).

Training cost for an occupational health and safety specialist: Here we assumed that 20 EUR/year are required for each meteorological parameter. For instance, if a TSI incorporates three meteorological factors, we estimated 60 EUR/year for the occupational safety and health specialist to be trained on using it.

Effectiveness was calculated as a utility value based on the criteria described in the Delphi exercise (Table 2) [7]. Specifically, criteria 1–10 and criterion 13 of the Delphi survey were used to compute a utility value describing the effectiveness of a TSI to protect individuals who work in the heat. Delphi criteria 14 to 17 were not considered, since no workers were identified to experience a mean body temperature higher than 39°C in our field experiments. Hence, the utility value describing the effectiveness of each TSI was calculated based on 11 Delphi criteria (1–10 and 13) and their accompanied weightings as derived from the Delphi survey (Table 2). The percentage points of importance were proportionally adjusted to account for the weightings of Delphi criteria 11, 12, and 14–17. Thereafter, the following equation (where i indicates a Delphi criterion) was used to calculate the effectiveness characterizing each TSI:

To extend the analysis on effectiveness, we calculated the quality-adjusted life years (QALY; indicating a year of perfect health in a worker’s life) [44] gained by using the different heat indices to protect individuals who typically work in the heat. The calculation was based on a 40-hour work-week across the average 40.6 years working life across the world [45] as follows:

Overall score

The overall score for each of the 61 TSIs was calculated using the weightings of the Delphi criteria 1 to 13 (Table 2). As mentioned above, Delphi criteria 14 to 17 were not used as no workers experienced a mean body temperature higher than 39°C in our field experiments. Therefore, the weightings were proportionally adjusted to account for excluding Delphi criteria 14 to 17. Thus, the following equation (where i indicates a Delphi criterion) was used to calculate the overall score characterizing each TSI:

Comparisons against the benchmark

Despite its simplicity, air temperature (i.e., dry bulb temperature) is the most well-known meteorological parameter that both scientists and civilians use in their daily routines to describe the surrounding ambient conditions. Therefore, all the analyses described above were also conducted to investigate the capacity of air temperature for quantifying the physiological heat strain experienced by people who work in hot environments.

Results

Work shifts performed in a broad spectrum of ambient conditions were included in the current study. Specifically, air temperature (31.7 ± 7.4°C) ranged from −16.4°C during a military hike in the Slovenian Alps to 43.8°C during a work shift in the construction sector of Qatar (Figure 1 and Table S1). Relative humidity (45.8 ± 17.9%) ranged from 8.8% during a work shift in the construction sector of Qatar to 100% during a guard duty performed by military personnel in Slovenia (Figure 1). Similarly, water vapor pressure (2.1 ± 0.7 kPa) ranged from 0.1 kPa during a work shift in the mining sector of Canada to 4.5 kPa during a work shift in the Australian mining sector. Wind speed (1.3 ± 2.1 m/s) ranged from 0 m/s during work in indoor environments to 30.5 m/s during a military hike in the Slovenian Alps (Figure 1). Solar radiation (251.3 ± 292.8 W/m [2]) ranged from 0 W/m [2] during work in indoor environments to 1097 W/m [2] during grape-picking at mid-day under a clear sky on the top of a mountain in Cyprus (Figure 1). This wide range of environmental conditions allowed for a comprehensive testing of all TSIs since high clothing insulation and/or metabolic rate can lead to hyperthermia even in apparently cold conditions [46]. Indeed, our military personnel demonstrated core temperature values as high as 38.8°C while hiking at sub-zero air temperatures. Another reason why we used a wide range of environmental conditions was to test the capacity of TSIs to identify true negatives (i.e., specificity), which is an important component of efficacy.

Figure 1.

Associations between air temperature and other environmental factors: wind speed (green color), solar radiation (orange color), and relative humidity (blue color). Curved lines and shaded areas correspond to three-term polynomial regressions and their 95% confidence intervals, respectively.

There were poor relationships between the core body temperature of the participants and the meteorological factors of air temperature and wind speed (Table 3). Mean body temperature was found to be moderately related with air temperature and poorly related with relative humidity and solar radiation (Table 3). Mean skin temperature was moderately related with air temperature and relative humidity, and was poorly related with wind speed and solar radiation (Table 3). There was a poor relationship between heart rate and all four meteorological factors (Table 3). Urine specific gravity was poorly related with air temperature, relative humidity, and solar radiation (Table 3). Similarly, urine color was poorly related with air temperature, relative humidity, and wind speed and did not correlate with solar radiation (Table 3).

Table 3.

Relationships between occupational heat stress and physiological strain experienced by people who work in the heat.

		Air temperature	Relative humidity	Wind speed	Solar radiation
Core body temperature	r	−0.034	0.027	0.035	0.032
	p	0.034	0.087	0.031	0.102
	n	3871	3880	3871	2643
Mean body temperature	r	0.527	−0.268	0.019	0.107
	p	<0.001	<0.001	0.258	<0.001
	n	3483	3492	3482	2267
Mean skin temperature	r	0.574	−0.325	0.071	0.173
	p	<0.001	<0.001	<0.001	<0.001
	n	5630	5639	5632	3770
Heart rate	r	−0.137	0.101	0.034	0.168
	p	<0.001	<0.001	0.017	<0.001
	n	4832	4832	4838	3635
Urine specific gravity	r	−0.127	0.098	−0.004	0.120
	p	0.003	0.025	0.927	0.013
	n	527	528	526	423
Urine color	r	−0.179	0.191	−0.185	0.024
	p	<0.001	<0.001	<0.001	0.631
	n	480	480	480	393

Results from stepwise linear regressions are shown in Table 4. The meteorological parameters explained less than 10% of the variance in core temperature (F(3,2640) = 13.90, p < 0.001), dehydration magnitude assessed via urine specific gravity (F(1,422) = 9.65, p = 0.002), dehydration magnitude assessed via urine color (F(3,392) = 9.77, p < 0.001), and heart rate (F(4,3631) = 74.60, p < 0.001). On the other hand, the meteorological parameters explained significant amounts of variation in mean body temperature (F(2,2665) = 654.17, p < 0.001) and mean skin temperature (F(3,3768) = 933.81, p < 0.001).

Table 4.

Percent of the variance in physiological heat strain explained by meteorological parameters in individuals who work in the heat. Hyphen (-) indicates no statistically significant predictor.

Physiological heat strain parameters	Variance explained in stepwise regression (%)
	Total	Air temperature	Relative humidity	Wind speed	Solar radiation
Core body temperature	1.6	1.0	0.4	-	0.2
Mean body temperature	36.6	33.3	-	3.3	-
Mean skin temperature	42.7	39.1	-	3.2	0.4
Heart rate	7.6	3.7	0.2	0.1	3.6
Dehydration magnitude (via urine specific gravity)	2.2	2.2	-	-	-
Dehydration magnitude (via urine color)	7.0	4.0	-	2.0	1.0

More than half of TSIs (36/61) were significantly associated with the core temperature of the workers (p < 0.05), however the strength of that association was “very weak” [37] (r: 0.083 ± 0.042; range: 0.032 to 0.216; Table S2). On the other hand, almost all TSIs were significantly (p < 0.05) related with the mean skin (60/61; r: 0.454 ± 0.137; range: 0.058 to 0.602) and mean body (59/61; r: 0.437 ± 0.118; range: 0.110 to 0.557) temperatures with “moderately strong” [37] relationships (Table S2). The adoption of different methods for estimating mean skin temperature (i.e., three vs. four measuring sites, different sites) did not interfere with our findings, since the magnitude of the correlations between the tested TSIs and mean skin temperature was similar (d = 0.013) when using all data (r = 0.454 ± 0.137, ranging from 0.058 to 0.602) and when using only four skin measurement sites (r = 0.444 ± 0.146, ranging from 0.027 to 0.599). Similarly, the majority of TSIs (56/61) were significantly (p < 0.05) related with the workers’ heart rate, however the strength of the association was “very weak” [37] (r = 0.107 ± 0.061; range: 0.031 to 0.388). Many of the TSIs were significantly (p < 0.05) associated with the magnitude of dehydration assessed via urine specific gravity (31/61; r: 0.114 ± 0.024; range: 0.087 to 0.171) and color (20/61; r = 0.149 ± 0.038; range: 0.101 to 0.208), however the strength of these associations was also “very weak” [37] (Table S2).

More than half (36/61) of TSIs provide categories to characterize the level of heat stress experienced by workers (Table S2). In total, 80% (92/115) of these heat stress categories demonstrated statistically significant Cohen’s Kappa agreement (p < 0.05) with a high (>36.7°C) mean body temperature. These results indicate that higher heat stress-level categories indicate an increased mean body temperature. Further evaluation demonstrated, on average, moderate/acceptable Area Under the ROC curve (0.690 ± 0.061; range: 0.501 to 0.746) and sensitivity (0.615 ± 0.182; range: 0.342 to 0.998), but low/unacceptable specificity (0.415 ± 0.239; range: 0.034 to 0.859) for the categories provided by the TSIs.

Only three out of the 61 TSIs provide health-related guidance/recommendations for their categories of heat stress (Table S2). Of note, the identified health-related recommendations for the indoor and outdoor Wet-Bulb Globe Temperature were published by widely reputable organizations [47,48], while the identified health-related recommendations [49] for the Universal Thermal Climate Index are not officially published by the organization that developed the thermal index.

The cost for hourly calculation for each of the 61 TSIs at a single workplace over the first year of operation ranged from 1,373.4 to 2,863.8 EUR (1,497.0 to 3,121.5 USD), with an average of 2,151.9 ± 665.8 EUR (2,345.6 ± 725.7 USD; Table S2). The cost of performing hourly measurements of different TSIs at a single workplace over a decade ranged from 13,068.8 to 26,254.7 EUR (14,245.0 to 28,617.6 USD) with an average of 19,982.6 ± 5,898.6 EUR (21,781.1 ± 6,429.5 USD).

The results for effectiveness demonstrated an average of 27.6 ± 14.8%, indicating a relatively low average capacity of TSIs for detecting physiological heat strain and protecting individuals who work in the heat. However, it is important to note that the effectiveness of TSIs ranged from 5.7 to 58.4%, suggesting that some TSIs were moderately effective. The same analysis indicated that, in a hypothetical scenario where all health-related recommendations and measures are followed, adopting a TSI to protect workers who typically work in the heat would lead to total gains of 0.6 to 5.6 quality-adjusted life years (depending on the TSI adopted) of uninterrupted work. Extrapolating this over the 40.6 years of average working life [45] results in gains of 2.3 to 23.7 (11.2 ± 6.0) quality-adjusted life years, depending on the TSI adopted.

The overall score for each of the 61 TSIs against the Delphi criteria 1 to 13 and the associated weightings is illustrated in Figure 2, with detailed scores presented in Table S2. Twenty-six of the 61 TSIs (42.6%) demonstrated overall scores below 25%, and 32/61 TSIs (52.5%) demonstrated scores between 25.1% and 50%. Finally, three TSIs (4.9%) scored above 50%, indicating that they satisfied (at least partly) the criteria to be considered for adopting a TSI to protect individuals who work in the heat. These TSIs were the indoor (WBGTIN; overall score: 55.4%) and outdoor (WBGTOUT; overall score: 55.1%) Wet-Bulb Globe Temperature, as well as the Universal Thermal Climate Index (UTCI; overall score: 51.7%).

Figure 2.

Overall score of the 61 TSIs. Black (minimum) and yellow (maximum) colors indicate the score of each TSI across the different Delphi criteria listed in Table 2. Detailed scores and full references are provided in Table S2 in the supplementary material.

Comparisons of the TSIs against the benchmark

Air temperature, which is considered the benchmark in occupational heat exposure, demonstrated very weak associations with core temperature (r = −0.034, p < 0.05), heart rate (r = −0.137, p < 0.05) and the magnitude of dehydration (r = 0.153, p < 0.05), as well as moderate associations with mean skin temperature (r = 0.574, p < 0.05) and mean body temperature (r = 0.527, p < 0.05). The association of air temperature was higher than 42.6% of the TSIs for core temperature, 82.0% of the TSIs for mean body temperature, 85.2% of the TSIs for heart rate, 93.4% of the TSIs for mean skin temperature, and 95.1% of the TSIs for the magnitude of dehydration. Similarly, air temperature had a greater Area Under the ROC curve (AUC = 0.740) than 88.5% (54/61) of TSIs (criterion #5). On the other hand, air temperature scored 0 in the Delphi criteria 6 to 10 since no published categories or guidance are available. Air temperature was found to be more cost-effective than 68.9% and 72.1% of TSIs during the first year of operation and over a decade, respectively. The cost for hourly calculation of air temperature at a single workplace over the first year of operation was 723.6 EUR (788.8 USD), while the cost of performing hourly measurements of air temperature over a decade escalates to 6571.4 EUR (7162.8 USD), indicating that measuring air temperature alone is considerably cheaper compared to the average cost required for measuring a TSI. In summary, the overall score of air temperature was 15.5%, therefore higher than 41.0% (25/61) of the occupational meteo-based TSIs.

Discussion

Of the 61 occupational meteo-based TSIs evaluated in this study, we found that the Wet-Bulb Globe Temperature and the Universal Thermal Climate Index score best when ranked against the criteria determined by our previous Delphi exercise [7]. Therefore, these TSIs have the highest potential to reflect the physiological strain experienced by workers and should therefore be adopted in occupational settings. Adoption of these TSIs will also facilitate cross-sector and -country comparisons allowing for further improvements in workers’ health and safety recommendations.

The total score of the analyzed TSIs ranged from 4.7 to 55.4% with an average of 26.5 ± 14.5%. Three TSIs scored relatively higher in the Delphi criteria, namely the indoor and outdoor Wet-Bulb Globe Temperature and the Universal Thermal Climate Index. The difference between these three TSIs and the remaining 58 was primarily due to the fact that they have published health-related recommendations – a Delphi criterion that was deemed as the most important by experts (some of whom are authors of this article) for adopting a TSI to protect individuals who work in the heat. Overall, the standard deviation of the weightings across all Delphi criteria was 5.1 ± 1.8%, with a range from 3.0% to 9.5% [7]. It is logical to suspect that fluctuations on the weighting assigned to the Delphi criterion related to health-related recommendations may have affected the ranking of the TSIs. However, even if this Delphi criterion was given, for instance, a 10% weighting by experts (making it the 3rd criterion of importance), the indoor and outdoor Wet-Bulb Globe Temperature and the Universal Thermal Climate Index would still rank at #1, #2, and #4, respectively. Even if equal weightings were given to all Delphi criteria, the indoor and outdoor Wet-Bulb Globe Temperature and the Universal Thermal Climate Index would rank among the best TSIs at #2, #3, and #10, respectively. This is in line with previous studies that have shown the increased capacity of these TSIs in assessing parameters of physiological heat strain and in predicting productivity losses in different environments [1,9,15,20,50]. Moreover, these notions are further supported by the fact that the Wet-Bulb Globe Temperature (cited/used in 8970 articles) and the Universal Thermal Climate Index (cited/used in 2920 articles) are two widely-accepted and -adopted TSIs (Google Scholar metrics on 19 January 2022; Fig. S1). A Google Scholar search for “Wet Bulb Globe Temperature” in the article title resulted in 547 published papers from 36 countries, while a similar search for the more recent “Universal Thermal Climate Index” resulted in 134 articles published from 25 countries (country of first author was considered; Fig. S2). Therefore, by adopting these TSIs, future studies will allow for direct comparison with previously published data across many countries.

Nearly all the tested TSIs showed statistically significant correlations with the mean skin temperature (98%), the mean body temperature (97%), and the heart rate (92%) of the workers, while 59% of them were correlated with core body temperature. The reduced correlations observed between the tested TSIs and workers’ core body temperature may be due to the fact that the latter is mostly related with work intensity [15], a parameter that was intentionally left uncontrolled in the present study. Almost half of the TSIs were associated with hydration status (urine specific gravity and/or urine color), and more than half provide categories to characterize the level of heat stress experienced by workers. On the other hand, only the indoor and outdoor Wet-Bulb Globe Temperature [23,51-53], as well as the Universal Thermal Climate Index [54] have been included in health-and-safety recommendations. The annual cost of using a TSI on a daily basis ranges from 1,373 to 2,864 EUR, reaching 13,068.8 to 26,254.7 EUR over a decade. Air temperature alone was cheaper to measure than any other TSI and had a stronger association with mean skin temperature, mean body temperature, heart rate, and hydration status than 82% of the studied TSIs. However, these associations were higher for the indoor and outdoor Wet-Bulb Globe Temperatures, as well as the Universal Thermal Climate Index. Also, there are no official occupational recommendations for air temperature, probably because of its limited relevance for workplaces with high humidity and/or radiant heat (e.g., blast furnaces and dry cleaning). In contrast, indoor and outdoor Wet-Bulb Globe Temperatures, as well as the Universal Thermal Climate Index have been widely accepted and adopted across all occupational environments, and they have been included in health-and-safety recommendations [6].

Technological advancements are continuously reducing the costs associated with heat stress monitoring. In many cases, companies have installed sensors that wirelessly transmit data which are logged and relayed in real-time to company supervisors and health-and-safety experts. While such monitoring may be more expensive to implement, it does not require person-hours each week and may render the above estimates as exaggerated. Regardless, implementing heat stress monitoring can be challenging for many industries (e.g., across large and/or remote agricultural fields). Also, these systems may be less affordable in low-income countries, as they require significant technological infrastructure to support their use. Although these costs may seem high for small enterprises, they are quite reasonable since the tested TSIs can increase the workers’ healthy life by 11.2 years on average if implemented at a workplace characterized by elevated heat stress. This is especially important if we consider that the risk for life-threatening diseases [1] and fatal accidents [55] is exponentially increased when people work in adverse ambient conditions. The ongoing global warming is projected to cause about one fatality from occupational heat stress every 14 to 24 minutes during the years 2020 to 2030 [56]. This may be only the tip of the iceberg, since the economic burden related to heat-induced decrease in labor productivity [1,9,14,15,20,57] may exacerbate the problem [58]. It is therefore imperative that urgent measures should be taken to protect the billions of people who are currently exposed to adverse climatic conditions, as well as the even larger number of people who will be affected in the near future [59]. Some of the countries in southern Europe have already implemented regulations [52] or they are currently developing relevant legislation that includes the adoption of TSIs to safeguard their workforce in the face of rising global temperature associated with climate change. Similarly, the International Labor Organization is supporting the enactment of legislation in countries situated in the Gulf region [23,53]. These legislations are either fully or partly based on TSIs, and as such they are an important component of heat-health risk assessment. The adoption of TSIs as a proactive method to protect people working under heat stress is encouraged by reputable organizations and it is further supported by the results of the present study.

We investigated the capacity of occupational meteo-based TSIs to quantify the physiological heat strain experienced by hundreds of workers across multiple industries in nine countries. While such a large-scale analysis provides many advantages in terms of ecological validity and statistical power, we acknowledge that certain considerations should be taken into account. Our study shows that the best TSIs address only 55% of the criteria determined by our Delphi exercise and that the correlations between TSIs and physiological heat strain parameters are low or modest. Previous studies evaluating whether TSIs reflect the physiological heat strain experienced by working people have reported higher correlations [9,14-16,18-21]. These studies were either well-controlled laboratory experiments [14,20,21] or small-scale field studies evaluating a group of workers at a single work site, wearing similar clothing, and performing similar jobs [9,15,16,18,19]. Therefore, it is likely that the 45% of the Delphi criteria that has not been addressed by even the best TSIs in our study is explained by differences that modify the heat strain response and the associated health outcomes. These are (1) altered work characteristics (work intensity, clothing, work schedule, in- versus out-door work, etc.) between industries as well as between countries within the same industry [1,15], (2) varying technological advancements and regulations between countries [1,15,60], (3) different inter- and intra-individual factors of the workers [60-62], and (4) availability of heat mitigation strategies (hydration regimes, provision of electric fans, air conditioning, work-rest cycles, etc.) [15,63]. To maintain the ecological validity of our study, we opted not to split our workers into similar groups (e.g., based on metabolic rate and/or clothing). Despite the seemingly low performance of even the best TSIs against the Delphi criteria, it is remarkable that some TSIs based only on four pieces of meteorological information (temperature, humidity, wind, and solar radiation) effectively satisfied these strict criteria in a series of field studies across multiple industries and countries.

In the last few years, our workplaces have been rapidly evolving through new digital technologies including artificial intelligence, widespread connectivity, the internet of things, big data applications, wearables, mobile devices, and online platforms [64]. These advancements allow the collection of workers’ data combined with the use of big data and algorithms for decision-making, changing the nature and location of work, who works and when, as well as how work is organized and managed. While monitoring workers is probably as old as work is, digitalization allows for new, continuous, and lower-cost forms of worker monitoring and management, through mobile devices, wearable or embedded sensory or other monitoring devices from various sources both in and outside the workplace.

The findings from this series of three companion papers demonstrate that work-related applications and systems should employ the indoor and outdoor Wet-Bulb Globe Temperatures, or the Universal Thermal Climate Index to inform management and make evidence-based decisions. This can lead to a rationalized organization of work and production as well as improved worker health and safety, where the real-time workers’ data collected are used to provide automated feedback to workers on their health and positively influence their behaviors. For example, in construction and agriculture, automated scheduling software can use historical and real-time data on weather and seasonal patterns for scheduling tasks and employees based on the indoor and outdoor Wet-Bulb Globe Temperatures, or the Universal Thermal Climate Index. This will allow employers to adjust schedules and to allocate shifts in response to the present and future conditions in the workplace. Also, analytics could identify associations between workplace practice and safety and health risk, reducing worker exposures to heat stress and producing individually tailored prevention or job resources to manage heat stress. Such practices can be effectively applied to traditional forms of work that require physical presence as well as to a dispersed workforce, such as teleworkers, or moving workers including drivers and couriers.

The data presented in this series of studies demonstrate that the indoor and outdoor Wet-Bulb Globe Temperature [51], as well as the Universal Thermal Climate Index [54] can support the development of effective occupational heat mitigation strategies in different countries and industries, particularly considering their relevance for big-data analytics used in climate services for the occupational and the general populations. However, it should be noted that our assessments involved monitoring people working in developed countries, and thus our results do not necessarily reflect the practices and conditions under which workers perform their jobs in other countries. Also, we did not examine the capacity of TSIs to quantify the physiological heat strain experienced by workers wearing heavily protective clothing that reduces heat dissipation, such as firefighters.

In conclusion, based on knowledge gained in the previous parts of this series of companion papers, where all occupational meteo-based TSIs were identified [6] and the 17 criteria to consider when adopting a TSI were determined [7], the present evaluation of 61 occupational meteo-based TSIs in a series of field studies performed in multiple industries across nine countries showed that the Wet-Bulb Globe Temperature and the Universal Thermal Climate Index have the highest potential to quantify the physiological strain experienced by individuals working in the heat. Adoption of these TSIs will facilitate more effective cross-sector and -country comparisons promoting and standardizing improvements in occupational health and safety.

Click here for additional data file.