Accuracy of metabolic rate estimates from heart rate under heat stress-an empirical validation study concerning ISO 8996.

The standard ISO 8996 provides methods for the determination of metabolic rate from measured oxygen consumption (MVO2), as well as simplified estimation algorithms based on heart rate (MHR). We quantified the accuracy of these methods by comparing MHR with MVO2 measured in 373 climatic chamber experiments under different workloads and widely varying heat stress conditions. While our results confirmed the 5% accuracy level for MVO2, MHR considerably overestimatedMVO2 due to the rise in core temperature concomitantly increasing heart rate by approximately 30 bpm/°C resulting in an overall error of 43%. After individually correcting for this bias the accuracy was 10-15% as stipulated by the standard. Thus, methods correcting for the thermal component of heart rate, e.g. by introducing intermittent resting periods of sufficient length of at least five min when investigating heat stress at workplaces, should become a mandatory element in the ongoing revision of the relevant standards.

In addition to the physical parameters of the thermal environment (air temperature, humidity, air velocity, thermal radiation) and the thermal properties of clothing, the rate of metabolic heat production (M) associated with occupational activities is a crucial input to assessment procedures for the thermal environment. This applies to the thermal comfort index PMV (Predicted Mean Vote, ISO 7730)1) as well as to the cold stress index IREQ (required clothing insulation, ISO 11079)2) and to heat stress indices like the WBGT (wet bulb globe temperature, ISO 7243)3) and the currently revised PHS (Predicted Heat Strain, ISO/DIS 7933)4).

The international standard ISO 89965) describes methods for the determination of M at increasing levels of expertise, effort and presumed accuracy. They start at Level 1 (screening) with a simple categorization of workload (resting, low, moderate, high, very high)3, 5), and tables of metabolic rates for different professions6) associated with a ‘very great risk of error’5). Level 2 (observation) uses tables5) or predictive equations of energy consumption for simple activities like walking7) with an assumed accuracy of 20%. Level 3 (analysis) provides algorithms to estimate M from heart rates (HR) recorded during workplace studies considering the influence of gender, age and body constitution (weight, body fat). A recent review8), which will serve as a guideline document for the ongoing revision of ISO 8996, analysed the uncertainty of the different components of the underlying algorithms using Monte Carlo simulations. The authors concluded from the resulting percentage coefficient of variation (CV) that the overall accuracy of metabolic rate estimates based on HR was 10−15%.

For assessing their accuracy, these estimates should be ideally compared to M determined from measured rates of oxygen consumption (VO), which are considered Level 4 (expertise) methods and supposed to provide the highest level of accuracy with a typical error of 5%5).

As VO measurements require sophisticated equipment and expert knowledge, the recording of heart rates with subsequently applied estimation algorithms are considered as a cost-effective alternative with acceptable accuracy8) and are recommended for the evaluation of the impact of dynamic work when assessing heat strain at workplaces by the revised draft of the PHS standard4).

However, the widely acknowledged5, 8, 10, 12) influence of further components, like static work, thermal and mental load etc., that increase heart rate and thus potentially trigger overestimation errors, is not considered in depth by these documents4, 8). More specifically, they do not quantify the effects of the rise in core temperature (ΔTre) under heat stress, which will increase both M and HR. For the first effect, Q coefficients9) describe the percentage change in M (%M) due to ΔTre by

(1)

Thus, the common setting Q=29) implies that a 1°C increase in core temperature will rise M by approximately 7%, which might increase the variability of M measured under heat stress. The Q effect on M will then also lead to a concomitant increase in HR for this M increment. Independently from this effect, the thermoregulatory response to rising body temperature, an increase in blood flow to the skin, will require an increase in HR. This impact of ΔTre on HR, called thermal cardiac reactivity10) or the thermal component of heart rate (ΔHR)5, 10) or ‘thermal pulses’11, 12), will increase HR by 30−40 bpm per 1°C rise in Tre8, 10, 12). This could introduce an overestimation bias and thus inflate the prediction error of M estimated from HR8) under heat stress conditions.

This validation study aimed to assess the claimed levels of accuracy for the different methods to determine M quantifying the influence of physiological strain under heat stress conditions by comparing M estimates to measured values from controlled climatic chamber experiments.

We compiled a database from human heat stress experiments performed previously9, 12,13,14,15) in the climatic chambers at IfADo, which had been conducted according to the ethical principles of the Declaration of Helsinki. Data originated from 373 laboratory sessions consisting of 11 individual series with 11 to 78 trials performed by six non-acclimated young fit males (Table 1) who had provided informed consent to the studies. Their averaged characteristics (mean ± SD) were 20.8 ± 0.9 yr of age, 1.83 ± 0.04 m of body height (H), 72.1 ± 8.4 kg of body weight (W), 1.9 ± 0.1 m2 of body surface area (A), 21.5 ± 2.5 kg/m2 of body-mass-index (BMI), and 55.3 ± 8.1 ml/min/kg of peak oxygen consumption (VO).

Table 1.

Means () and coefficient of variation () of metabolic rate from measured oxygen consumption () measured in 11 series with six participants (ID1−ID6) under different workload conditions (W1, W2, W3)

Influence of the increase in rectal temperature from rest (ΔTre) on a) the prediction error for metabolic rates estimated from heart rates (MHR), and on b) the increase in heart rate from rest (ΔHR). Regression lines are shown for the individual series with the six participants (ID1–ID6) under the different workload conditions (W1, W2, W3).

Series						MVO2		Error MHR		Error MHR corrected for ΔTre		Error Mpan
Work-ID		Nexp	ΔTre (°C)	TCR (bpm/°C)	Q10 (nd)	AM (W)	CV	%bias	%rmse	%bias	%rmse	%bias	%rmse
W1-ID1		15	1.0	28.8	1.47	193	11%	63%	66%	0%	14%	−12%	16%
W1-ID2		18	0.9	16.2	1.07	224	4%	41%	45%	0%	14%	−23%	23%
W1-ID3		15	1.0	24.8	1.12	210	7%	61%	63%	0%	11%	−23%	24%
W2-ID1		44	1.0	38.8	1.19	221	11%	56%	60%	0%	14%	1%	11%
W2-ID2		35	1.0	17.9	1.13	265	6%	27%	31%	0%	12%	−14%	15%
W2-ID3		78	1.0	20.9	1.05	242	5%	44%	47%	0%	13%	−11%	12%
W2-ID4		52	0.9	39.3	1.19	289	7%	35%	45%	0%	15%	−0%	7%
W2-ID5		38	0.9	23.3	1.00	272	5%	57%	59%	0%	12%	−7%	8%
W2-ID6		51	0.8	28.7	1.09	277	4%	8%	19%	0%	9%	−5%	7%
W3-ID1		11	0.9	40.4	0.95	323	5%	10%	14%	0%	6%	5%	7%
W3-ID2		16	1.1	11.0	0.97	329	4%	8%	14%	0%	9%	5%	7%
Subtotals for workload
	W1	48	1.0	22.8	1.21	210	7%	54%	57%	0%	13%	−20%	21%
	W2	298	0.9	28.0	1.10	260	6%	38%	43%	0%	13%	−6%	10%
	W3	27	1.0	23.0	0.97	327	4%	9%	14%	0%	8%	5%	7%
Total		373	1.0	27.0	1.11	258	6%	38%	43%	0%	12%	−7%	11%

Together with averaged rectal temperature increase (ΔT), thermal cardiac reactivity (TCR) representing the slopes from Fig. 1b) and non-dimensional (nd) Qcoefficients. Percentages of mean prediction error (%bias) and root-mean-squared error (%rmse) for metabolic rates estimated from heart rates (M) in comparison to the errors after correcting for bias due to rectal temperature increase (ΔT) using the relationship from Fig. 1a), and to predictions from the Pandolf equation (M). Subtotals for workload and total figures were calculated from the individual series as means weighted by the number of experiments (N).

Influence of the increase in rectal temperature from rest (ΔTre) on a) the prediction error for metabolic rates estimated from heart rates (MHR), and on b) the increase in heart rate from rest (ΔHR). Regression lines are shown for the individual series with the six participants (ID1–ID6) under the different workload conditions (W1, W2, W3).

Fig. 1.

Influence of the increase in rectal temperature from rest (ΔTre) on a) the prediction error for metabolic rates estimated from heart rates (MHR), and on b) the increase in heart rate from rest (ΔHR). Regression lines are shown for the individual series with the six participants (ID1–ID6) under the different workload conditions (W1, W2, W3).

Following a one-hour bed rest under neutral conditions, the participants donned a cotton coverall combined with underwear, socks and gym shoes providing a thermal insulation of 0.7 clo13). Then they moved into the climatic chamber set to heat stress conditions characterized by different combinations of air temperature (range 15−55°C), water vapour pressure (0.4−1.8 kPa), air velocity (0.5−2.0 m/s), and values of mean radiant temperature varying between 0 and 128.5 K above air temperature. The participants performed treadmill work for at least 3 h with three levels of workload: 3 km/h on the level (W1), 4 km/h on the level (W2), and 4 km/h with 2.5° inclination, corresponding to a grade of 4.4% (W3). The protocol included short interruptions (3 min) after each 30 min period for weighing the participants. In addition, rates of oxygen consumption (VO) in l/min were determined from the expired air collected in Douglas bags towards the end of each full hour according to ISO 89965). As physiological responses usually stabilized after two hours12), we calculated averages over the third hour of exposure from continuously recorded rectal temperatures (Tre) and HR, thus representing steady-state values. These were matched with VO measurements from the third exposure hour. We also computed resting values (Tre, HR) from the last 15 min of the rest period.

We calculated Q coefficients from Tre and VO for each series as described recently9) using an expanded version of (1) shown in (2).

(2)

More precisely, Q coefficients were obtained by exponentiating the slopes of the logarithmized equation (2) fitted by linear regression to the measured VO using Tre as predictor with Tre = 36.8°C. It should be noted that choosing a different value for Tre or applying a multiplicative transformation on VO, e.g. calculating metabolic rates using a constant standard energy equivalent as in (3) below, would only affect the intercept VO, but result in identical Q values.

The metabolic rates in watts from VO measured in l/min (M) were calculated using the standard energy equivalent of 5.68 W/(l/h) according to ISO 89965) as:

(3)

Estimates of metabolic rate based on HR (M) rely on the linear relationship between metabolic or work capacity with cardiac capacity or cardiac reserve. Detailed algorithms considering different populations depending on gender, age and body constitution are provided in the literature5, 8, 16). Here, we will focus on the equations needed for our sample of young fit males and suggested by the standard5, 8) as follows:

(4)

In (4), M and HR denote resting values of metabolic rate and heart rate, respectively; MWC is the maximum work capacity in watts; and HR denotes maximum heart rate. As only HR and HR were measured, we estimated the other parameters following the published guidelines8) underlying the current revision of ISO 8996. We assumed M for males as 60 W/m2 and converted it to watts by multiplication with A. HR was calculated depending on age in years as HR = 208–0.7 × age. MWC can be estimated for males depending on age and lean body mass (LBM) as:

(5)

As recently reviewed8), there are several options to estimate LBM. We adopted the approach as advised for males in the current draft for the revision of ISO 8996 and calculated LBM from body weight (W) and height (H) as:

(6)

In the sample under study, LBM thus varied between 77% and 85% of body weight.

Representing a Level 2 method, we also applied the widely used Pandolf equation7) for treadmill walking. Equation (7) predicts the metabolic rate (M) in watts considering body weight (W), the load due to the weight of clothing and sensors (L=2 kg13)), the grade of the treadmill (G) in %, and walking speed (v) in m/s, which was calculated by dividing the values given in km/h by 3.6.

For each of the 11 series, based on calculations for each individual session, we determined intra-series means and CV as presented in Table 1. Mean M increased with workload from 210 W (W1) over 260 W (W2) to 327 W (W3), whereas CV showed slightly decreasing values with 7% (W1), 6% (W2) and 4% (W3). A closer inspection revealed that CV significantly increased with Q (r=0.81, p=0.002). Though rectal temperature increase from rest (ΔTre=Tre − Tre) varied between 0.1−1.7°C (Fig. 1a), we observed similar averages of approximately 1°C in all series (Table 1), indicating prevailing steady-state conditions. Together with the overall Q of 1.11, this implies an average increase in M of about 1% due to ΔTre (1), contributing to 6% overall CV, with CV values conforming to the accuracy level of 5% claimed by the standard5) for Q ≈ 1, i.e. if M were independent of ΔTre.

We calculated the prediction error=M−M for all 373 experiments and computed intra-series mean error (bias) and root-mean-squared error (rmse) summary statistics presented in Table 1 as percentage values relative to mean M. Metabolic rates calculated from heart rates considerably overestimated M (Table 1) with mean %bias ranging from 54% (W1) over 38% (W2) to 9% (W3). Consequently, overall %rmse amounted to 43% (Table 1), which further increased to 64% when replacing lean body mass (LBM) by body weight (W) in eq. (5), as suggested8) for the prevailing lean persons in our study. These figures were far above the accuracy level of 10−15% stipulated by the standard5) and guideline document8).

The error was highly sensitive to ΔTre as illustrated by Fig. 1a. However, the slopes exhibited a considerable inter-series variability, which mirrored the variation in thermal cardiac reactivity (TCR), i.e. the increase in heart rate (ΔHR=HR−HR) due to core temperature rise (ΔTre) shown in Fig. 1b and summarized in Table 1. TCR varied between 11.0−40.4 bpm/°C, whereas the overall average (27.0 bpm/°C) was close to the value of 33 bpm/°C reported in ISO standard 988610). The slopes of Fig. 1a significantly correlated with TCR (r=0.96, p<0.001), suggesting that the bias was largely attributable to the thermal component of heart rate (ΔHR)5, 10), also termed ‘thermal pulses’11, 12), which were neglected by the proposed algorithms8).

There are procedures to correct for ΔHR, either using the correlation with core temperature, if those measurements were available11), or by estimating ΔHR from HR recorded during resting periods intermitting the heat exposure10, 17,18,19). The latter method is advantageous under field conditions, as it does not require any core temperature measurements. The basic idea is to calculate ΔHR as the difference of the HR recorded after at least five minutes break from work to HR, which could be measured before start of work or estimated by the 1st percentile of all measured HR8). Linear interpolation approximates intermediate values of ΔHR over the working periods, which are then subtracted from recorded HR before estimating M17,18,19).

As Tre was available in our study, we applied a bias correction to the M estimates for each series using the individual regression functions from Fig. 1a. As shown in Table 1, this did not only remove the bias, but also reduced %rmse considerably to values between 9−15%, thus conforming to the requested accuracy level of 10−15%5, 8).

Interestingly, in our study a comparable overall performance was observed for the simple estimates M (Table 1) using the Pandolf equation7) (7), which represents a Level 2 method5) for this type of treadmill work in the laboratory.

A recent field study on forest workers18) achieved a similar level of accuracy by estimating ΔHR from intermittent resting periods17) and using individual M-to-HR relations calibrated with procedures deemed representative for forest work18). This is important, as the M-to-HR relationship intra-individually depends on the type of work: for the same heart rate, work with great muscles (legs) shows a metabolic rate 23−30%8) higher than activities involving smaller muscles (work with arms), and the difference may even increase for static muscular work. So the overestimation bias shown in Table 1 for M8) may be actually higher in field situations frequently including work of small muscle groups and also static muscular work than for our comparison with walking subjects predominantly concerning leg muscles.

This validation study followed the concept of expressing accuracy levels as CV5, 8), but supplemented this by calculating bias in combination with rmse, as these quantities provide more appropriate error figures in case of non-zero bias. Adhering to this concept, we could largely confirm the accuracy level of 5% for M as stipulated by the standard5), although the influence of body temperature on VO was smaller (Q=1.11) compared to recent studies9) with semi-nude subjects showing Q=2.

However, we observed large overestimation by the M algorithm due to thermal cardiac reactivity inflating the overall error up to 43% on average and above 60% for single series. After bias correction at the individual level, the error conformed to the accuracy level of 10–15% found in the recent simulation study assuming zero bias8).

Our results reinforce recent findings11, 16,17,18,19) on two essential requirements for the application of M estimation algorithms to work scenarios. First, individual HR-to-M-relationships derived from controlled cardiac stress tests are desirable, that should reflect the actual workload and type of work under consideration8, 11, 18). Secondly, a correction for the thermal heart rate component is necessary, especially under heat stress conditions. Thus a bias correction, e.g. by introducing intermittent resting periods of sufficient length of at least five minutes10, 17,18,19) should become mandatory for Level 3 studies in ISO 89965) and PHS4). Otherwise, the accuracy of Level 3 studies might fall behind simpler Level 2 methods, as indicated by the smaller error for M in our study.

Finally, we like to add that measurement repetition is one method to reach a requested level of accuracy. Based on the CV of an unbiased estimate, the formula (actual accuracy level/requested accuracy level) approximates the required number of repetitions. This implies that two measurements would be necessary to achieve the 10% accuracy level with a method actually providing 14%, while four repetitions would be needed with 20% accuracy, and even 9 with 30%, making such a method inefficient for field applications.