Cognitive demands and mental workload: A filed study of the mining control room operators.

Cognitive demand and mental workload assessment are essential for the optimal interaction of human-machine systems. The aim of this study was to investigate the cognitive demands and mental workload as well as the relationship between them among the mining control room operators. This cross-sectional study was performed on 63 control room operators of a large mining plant located in Iran. Cognitive demands and mental workload were assessed using cognitive task analysis (CTA) and NASA Task Load Index (NASA-TLX), respectively and the analysis was performed using SPSS version 21. Independent samples T-test, Mann-Whitney U test and multivariate linear regression were used for data analysis. Twelve cognitive demands were extracted after observing the tasks and conducting semi-structured interviews with the control room staff. The mean scores of total cognitive demands and MWL were 6.60 and 72.89, respectively, and these two indicators showed a positive and significant correlation (r = 0.286; P = 0.023). The participants' demographic characteristics such as age, education, and work experience did not affect mental workload, but the two cognitive demands (memory and defect detection) affected MWL. High cognitive demands and mental workload indicate poor interaction between humans and machines. Due to the effect of memory load and defect detection on mental workload, it is recommended to assign cognitive tasks based on memory and defect detection to the machine to reduce the mental workload and improve human-machine interaction.

Introduction

The development and application of new technologies have made systems and organizations more complex (Johnsen et al. 2017). Changing analog systems to computer systems in modern control rooms has led to greater complexity and increased human error (Stanton et al., 2017). Studies show that increasing the probability of human error increases the risk of accidents (Schumacher et al. 2011). Most accidents that result from human error are influenced by the complex design of socio-technical systems or the mismatch between the complexity of tasks and the capabilities of the human operator (Gomes et al., 2015). Errors are often caused by defects in mental processes such as distraction, low motivation, low alertness, mental overload, and fatigue (Aricò et al., 2016). Errors made by control room operators have been one of the leading causes of major accidents in the world, including Bhopal in India (1984), Piper Alpha in the UK (1988), Chernobyl in Ukraine (1986), and the Texaco Refinery in Wallace (1994) (Jahangiri et al., 2016).

Due to its effect on human error and performance, the mental workload is an important emergency issue in complex systems such as control rooms (Fallahi et al., 2018). Factors such as high information flow, the complexity of information, many difficult decisions, and stress due to time constraints are effective in increasing the mental workload of control room operators (Hwang et al., 2008). Hart and Staveland have defined mental workload as “a hypothetical construct that represents the cost incurred by a human operator to achieve a particular level of performance” (Mouzé-Amady et al., 2013). Mental workload is usually measured by methods of measuring primary and secondary tasks, psycho-physiological methods, and mental methods (Stanton et al., 2017). The NASA-TLX method is a valid and widely used mental method for measuring mental workload (Mouzé-Amady et al., 2013).

The present study focuses on mental workload that can be defined as the cognitive demand of a task (Miyake 2001). Mental workload is affected by three important factors: the nature of the task, the environment, and human characteristics. Task features such as the difficulty and complexity level have a greater effect on mental workload, thus, controlling it can have a greater effect on reducing mental workload (Galy et al. 2012). Backs (1995) showed that the difficulty of a task, determined by the number of items to remember, affects the performance of the in the memory task. The average error rate increased from 1.09% to 5% if the subject had to remember three items instead of one. More generally than, the level of inherent cognitive load depends primarily on the number of elements that are assimilated at the same time, especially the degree of interactivity of the elements (Ayres 2006; Kalyuga 2009; Sweller 1994). By keeping other causes of (external) cognitive load constant, Ayres (2006) showed that subjective measurements strongly correlated with the error rate of problem-solving task, with subject having an intrinsic cognitive load. It suggested that we evaluated the changes directly. Similarly, the difficulty of task affects psychophysiological measurements, especially component, which is under the control of the autonomic nervous system. High cognitive loads have been shown to result in increased cardiac activity, increased breathability, and increased blood glucose levels to provide the energy needed to complete the task(Ayres 2006; Carroll et al. 1986; Miyake 2001).

The control room operation is a complex job that depends on the operator's cognitive demands such as attention, decision making, problem solving and etc. (Vitório et al. 2012; Schumacher et al. 2011). Besides, the responsibility for maintaining the safety of the system and human resources is the main function of a control room operator, which puts extra mental workload on them (Aricò et al., 2016).

The job requirements of control room operators are different. For example, there are differences between supervisors and operators in temporal pressure. Task difficulty directly affects cognitive load, while temporal pressure activates emotional elements and thus indirectly affects cognitive load. Temporal pressure is a contradiction between the required processing time of a task, and the actual time required to complete the task, leading to a very emotional reaction (Gomes et al., 2015); Increased cognitive demands creates a mental overload of and reduces work performance. Therefore, in order to improve the well-being and safety of control room operators of the mining, it is essential to investigate the factors of mental workload and how they interact (Galy et al. 2012). Accordingly, this study aims to identify and quantify the cognitive demands and mental workload and also examine cognitive demands affecting mental workload in the staff of the control room of a mining and industrial complex.

Method

Participators

Seventy of 76 control room operators working in a mining and industrial complex participated in the study. Seven operators were excluded from the study due to inaccuracy in completing the questionnaire, and finally, the data from 63 operators (63 men; Mage = 32.79; SD = 4.24; work experience = 8.71; SD = 4.11) were used in the data analysis. The inclusion criteria were having acceptable general and mental health and having no addiction to drugs and alcohol. The exclusion criteria were having a low level of alertness, consumption of caffeinated substances, alcohol or drugs up to 24 h before participating in the study, and work experience less than one year. before the study, written informed consents were obtained from the participants.

Task analysis

The first step in this study was task analyzing which was carried out by hierarchical task analysis (HTA) (Stanton et al., 2017). To implement this technique, the related documents (organizational chart and available instructions) were reviewed, a short interview was conducted with the control room staff and the HTA was run to identify the tasks and subtasks performed in the control room.

Cognitive demands assessment

After identifying the tasks and subtasks of the control room operators, the job cognitive demands of each task and subtask were determined using the CTA method that was performed in three steps: (1) Initial review and review of job-related documents, (2) Identification of job-related information indexes, and (3) Selection and implementation of the data related to the knowledge required for the job (Clark and Estes 1996; Ghanbari et al. 2014).

In the first step, the initial review and review of job-related documents, the general information related to the knowledge required for the job of the control room was obtained, and also the subject-matter experts (SMEs) were identified to participate in the data collection process. In the second step, information indexes related to each task such as concept maps, flowcharts, virtual networks, etc. were identified. In the third step, the method of collecting data related to operating the control room was selected. Then, according to the guidelines (Clark and Estes 1996), the interview and observation method was identified to be appropriate for cognitive task analysis.

In this study, to accurately define and explain the cognitive demands and psychological processes required by the job, the cognitive demands identified in the previous step were quantified. Accordingly, like Fleishman Job Analysis Survey (FJAS) (Ghanbari et al. 2014), all cognitive demands were quantified on a 7-point scale. In this way, the participants quantified the cognitive demands in their job by choosing a score from 1 to 7. Finally, the average cognitive demands were calculated as the total cognitive demand.

Mental workload assessment

The last step was mental workload assessment. The NASA Task Load Index (NASA-TLX) as a multidimensional assessment tool that rates the perceived workload associated with tasks was used in this study. The tool consists of two parts. The first part measures the workload demand of a given task using six sub-scales: Mental demand, physical demand, temporal demand, performance, effort, and frustration. To this end, after getting familiar with the six scales, the participants rated the impact of each of them on their job in the range of 100 points. The mean scores obtained at this stage as Raw TLX values are used in many studies including the present study for the analysis of the results.

NASA-TLX in the second part of the TLX allows participants to determine the importance of each of these scales by comparing 6 scales of the first part by pairs (binary comparison). In this way, between the two choices, the person chose the scale that he considered most involved in his job. Therefore, the number of times each is chosen is the weighted score which is multiplied by the scale score for each dimension and then divided by 15 to get a workload score from 0 to 100 as the overall task load index.

Statistical analysis

Based on the results of the Kolmogorov–Smirnov test concerning the normal distribution of the data, appropriate statistical tests were used at a significance level of 0.05. Accordingly, the independent samples t-test was used to evaluate the differences between the mental workload index of the tasks performed by the participants (the control room managers and operators) and their marital status. The cognitive demands of the participants were also analyzed based on their task type using the Mann-Whitney U test. The differences in the mental workload of the participants according to the working unit (six units) and their level of education were also examined using the one-way ANOVA. Besides, the Kruskal-Wallis test was used to examine the possible differences in the cognitive demands of employees working in different factories. Pearson and Spearman correlation tests were also used to investigate the relationship between quantitative variables. Finally, NASA-TLX and Raw-TLX modeling was performed with twelve cognitive demands and six TLX subscales (the independent variables) using stepwise multivariate regression. All analyses were performed using SPSS21 software. The effects of cognitive demands (identified via CTA method) and NASA-TLX subscales on the mental workload of control room staff were also investigated. To this end, three models were developed. In the first model, the effect of the six NASA-TLX subscales of mental workload (the independent variables) on NASA-TLX and Raw-TLX (the dependent variables) was analyzed. In the second model, the effect of twelve cognitive demands on TLX raw and total mental workload scores was measured. In the final model, both the cognitive demands and the TLX subscales were analyzed to predict the dependent variables.

Results

Tasks analysis

The hierarchical task analysis (HTA) was performed after conducting the interviews and review of the documents, and the tasks and subtasks of the control room managers and operators. Two main tasks and 17 subtasks were identified as shown in Table 1.

Table 1

Determining the cognitive demands of each task using the CTA technique.

Tasks	Subtasks	Cognitive demands
supervisor	Supervising the operators	Visual, auditory, defective diagnosis, situational awareness, decision making, attention, accuracy, experience
	Starting the production line	Visual, auditory, situational awareness, decision making, attention, experience
	Stopping the production line	Visual, auditory, situational awareness, decision making, attention, speed of action, and experience
	Coordinating loading	Visual, auditory, defective diagnosis, position awareness, decision making, problem solving, attention, accuracy, memory, experience
	Coordinating unloading	Visual, auditory, situational awareness, decision making, attention, accuracy, memory, speed of action and experience
	Coordinating repairs	Visual, auditory, defective diagnosis, situational awareness, decision making, problem solving, attention, memory
operator	Running the start command	Visual, auditory, defective diagnosis, awareness of the situation, problem solving, work tricks (smart solution), attention, accuracy, memory
	Running the stop command	Visual, auditory, situational awareness, attention, memory, speed of action and experience
	Loading	Visual, auditory, defective diagnosis, position awareness, decision making, problem solving, work tricks (smart solving), attention, accuracy, memory, speed of action and experience
	Unloading	Visual, auditory, defective diagnosis, position awareness, decision making, problem solving, work tricks (smart solving), attention, accuracy, memory, speed of action and experience
	Reporting to the control room manager	Visual, auditory, attention, accuracy, memory
	Reporting to the shift supervisor	Visual, auditory, attention, accuracy, memory
	Shift delivery	Visual, auditory, attention, accuracy, memory
	Answering the phone calls from production line workers	Visual, auditory, defect detection, position awareness, attention, accuracy, memory, speed of action
	Answering the phone calls from the shift supervisor	Visual, auditory, defect detection, position awareness, attention, accuracy, memory, speed of action
	Product quality control (especially in pelletizing unit)	Visual, auditory, defect detection, position awareness, problem solving, work tricks (smart solving), attention, accuracy, memory, speed of action and experience
	Troubleshooting	Visual, auditory, defective diagnosis, position awareness, decision making, problem solving, work tricks (smart solving), attention, accuracy, memory, speed of action and experience

Cognitive demands

The cognitive task analysis (CTA) technique was used to identify the cognitive demands of the control room managers and operators using observations and semi-structured interviews with six control room managers and eight control room operators. Based on the findings of the present study, a total of 12 cognitive demands including visual, auditory, defect detection, position awareness, decision-making, problem-solving, work tricks (smart solutions), attention, accuracy, memory, speed of action, and experience related to the control room operators were identified (Table 1). The mean scores of cognitive demands were also determined for the tasks of the control room managers and operators (Table 2). The results of the Mann-Whitney U test did not show a statistically significant difference in the twelve cognitive demands of the control room operators by type of task. However, more cognitive demands were evident in the tasks performed by the control room managers (Figure 1). There was no significant difference between these two tasks in terms of the total cognitive demand (Table 3). There was also no significant difference in the mental workload of the operators with different levels of education (P > 0.05). The results of Spearman's correlation analysis also showed a positive and significant correlation between the total cognitive demand and the two indicators of NASA-TLX (r = 0.286; P = 0.023) and Raw-TLX (r = 0.343; P = 0.006).

Table 2

Comparison of cognitive demands and the mental workload in two tasks.

		Operator (n = 51)	Supervisor (n = 12)	P-value	Total (n = 63)
Mental demand	Mean (SD)	96.17 (7.15)	76.67 (11.23)	0.059∗∗	95.24 (9.31)
Physical demand	Mean (SD)	49.08 (26.16)	43.33 (35.47)	0.715∗	48.81 (26.33)
Temporal demand	Mean (SD)	93.58 (9.30)	76.67 (13.63)	0.053∗∗	92.78 (10.65)
Performance	Mean (SD)	9.98 (9.42)	22.00 (19.67)	0.259∗∗	10.56 (10.16)
Effort	Mean (SD)	88.17 (16.80)	78.33 (24.66)	0.317∗∗	87.70 (17.11)
Frustration	Mean (SD)	70.67 (29.54)	65.00 (35.20)	0.755∗	70.40 (30.32)
Raw TLX	Mean (SD)	67.94 (8.92)	60.33 (12.90)	0.190∗	67.58 (9.76)
NASA TLX	Mean (SD)	73.40 (10.50)	67.94 (8.92)	0.102∗	72.89 (11.23)
Total Cognitive Demand	Mean (SD)	6.59 (0.51)	6.83 (0.17)	0.542∗∗	6.60 (0.51)

Independent t-test.

Mann-Whitney U test.

Figure 1

Comparison of the average cognitive demands in the two tasks of supervisor and control room operator.

Table 3

Stepwise multiple regression analysis results of NASA-TLX and Raw-TLX (n = 63).

	NASA-TLX					Raw-TLX
	Output	Regression coefficients	Standardized coefficients	P-value	Adjusted R Square	Output	Regression coefficients	Standardized coefficients	P-value	Adjusted R Square
Model 1a	Constant	15.006	7.384	0.047	0.668	Constant	1.245E-013		<0.001	1
	Efforts	0.290	0.056	<0.001		Mental Demand	0.167	0.159	<0.001
	Frustration	0.140	0.030	<0.001		Physical Demand	0.167	0.450	<0.001
	Temporal Demand	0.240	0.086	0.007		Temporal Demand	0.167	0.182	<0.001
						Efforts	0.167	0.292	<0.001
						Frustration	0.167	0.518	<0.001
						Performance	0.167	0.174	<0.001
Model 2b	Constant	30.470			0.161	Constant	46.763			0.131
	Memory	6.303	0.418	0.004		Defect Detection	3.354	0.381	0.002
Model 3c	Constant	15.006	7.384	0.047	0.668	Constant	1.245E-013		<0.001	1
	Efforts	0.290	0.056	<0.001		Mental Demand	0.167	0.159	<0.001
	Frustration	0.140	0.030	<0.001		Physical Demand	0.167	0.450	<0.001
	Temporal Demand	0.240	0.086	0.007		Temporal Demand	0.167	0.182	<0.001
						Efforts	0.167	0.292	<0.001
						Frustration	0.167	0.518	<0.001
						Performance	0.167	0.174	<0.001
						Visual	1.070E-013	0.001	<0.001
						Working Tricks	-1.034E-013	0.001	<0.001

Inputs of model: NASA-TLX subscales include Mental Demand, Physical Demand, Temporal Demand, Efforts, Frustration, Performance.

Inputs of model: Cognitive demands include Visual, Audial, Defect Detection, Situation Awareness, Decision Making, Problem Solving, Working Tricks, Attention, Accuracy, Memory, Action pace, Experience.

Inputs of model: NASA-TLX subscales + Cognitive demands.

Mental workload

The results of the NASA-TLX index showed that the control room operators were exposed to a high mental workload. The control room operators also reported a higher mental workload than the control room managers, although this difference was not statistically significant (Table 2). Besides, the control room managers were more affected by the mental workload than the control room operators based on the NASA-TLX's six subscales (Figure 2). The results of the one-way ANOVA also showed that there was no significant statistical difference in the mental workload of the staff according to their level of education (P > 0.05).

Figure 2

Comparison of mental workload and its subscales by task in supervisors and control room operators.

Multivariate analysis and modeling

Three models were developed based on the analysis of the impact of cognitive demands (identified through the CTA method) and the NASA-TLX subscales on the mental workload of the control room staff (Table 3).

Discussion

The findings of this study indicated the high values of both the total cognitive demand and the workload indexes among the control room managers and operators. Moreover, the total cognitive demand index was found to have a positive and significant correlation with NASA-TLX and Raw-TLX indicators, implying the importance of the role of cognitive demands in inducing mental workload and also confirming the effectiveness of CTA technique in identifying the cognitive demands of the tasks and the accuracy of its implementation in this study. CTA has currently found widespread applications in complex and dynamic settings such as nuclear energy, aviation, military, firefighting, and health and emergency services (Jahangiri et al., 2016; Stanton et al., 2017). House conducted a study to identify the impact of the First ATC Support Tools Implementation (FASTI) on the tasks performed by air traffic controllers and the cognitive demands associated with these tasks. They identified the cognitive demands of the tasks with error potential using the CTA tool (House 2007). In line with the findings of the present study which suggested that the task of managing the control room was more cognitively and mentally demanding than operating in the control room, Walker and Bergmann also used the CTA method to inform the teaching of psychomotor skills and cognitive strategies in clinical tasks in dental education. They found that the highest cognitive load was associated with the final tooth implantation task (Walker and von Bergmann 2015). The present study also showed the tasks such as coordinating and implementing loading and unloading and troubleshooting imposed the greatest cognitive load on the control room staff (Table 2). Gomes et al. used CTA to identify cognitive ergonomic factors affecting helicopter pilots in the Air Transport System of Campo Region of Brazil. They showed that the lack of color aerial maps and not understanding very short communication messages increased the cognitive workload on the pilots (Gomes et al., 2015). In another study, Dionne et al. also used CTA to identify and describe the psychological processes involved in making difficult decisions to end life in the intensive care unit (ICU). Their results suggested that three main and frequent psychological aspects, i.e., body memories, feelings of fear and insight, and moral considerations, influence the judgment of decision-makers in admitting the patients or their conditions (Dionne-Odom et al., 2015). Keeney et al. investigated the readiness of miners to escape danger in an emergency using CTA. The results of this study indicated very high cognitive demands for the decision-making process to escape in times of danger (Keeney et al., 2018).

The results of the present study showed that cognitive demands associated with a task have a significant relationship with the mental workload of the control room staff, which is consistent with the findings of Tubbs-Cooley et al. in their study of the relationship between the mental workload of NICU nurses and negligence in performing care tasks. The results of the NASA-TLX analysis showed that NICU nurses' negligence in performing their tasks had a significant relationship with their mental workload (Tubbs-Cooley et al., 2019). Hollands et al. conducted a study to assess the mental workload of Canadian soldiers in the face of the rate of messages delivered from the war management system. The results of the NASA-TLX test showed a high mental workload due to receiving messages at a fast rate (Hollands et al. 2019). In another study, Wijsman et al. evaluated the effect of a robotic camera holder on the reduction of the mental workload of surgeons in laparoscopic surgery using the NASA-TLX tool. Their results showed that using a camera-holding robot significantly reduced the surgeons’ workload (Wijsman et al., 2019). The present study showed that among the NASA-TLX subscales, mental demand (95.24) imposed the highest workload on the control room staff (Table 3). Similarly, Merkle and colleagues found that the maximum workload was imposed on cardiac surgeons by mental demand (Merkle et al., 2019). However, Bazazan et al. reported the highest workload of emergency nurses was caused by effort (86.80%) and mental demand (83.73%), which was lower than the rate reported in the present study (Bazazan et al., 2019). It is noteworthy that the maximum workload reported by the managers of the control room in this study was related to the effort scale, which is in line with the findings of Bazazan et al. As it was stated earlier, the control room operators experienced a higher workload than the control room managers. However, the difference between the two groups was not statistically significant (Table 3). The reason is perhaps the lack of transparency and distinction between these two tasks since in some cases, some of the managers' tasks, such as coordinating with the shift supervisor for loading and unloading are performed by the operators (Table 2). In general, the control room staff, regardless of the type of task (managing or operating), experienced the highest workload in the mental demand, temporal demand, and effort subscales, respectively, which indicates the importance of mental and cognitive workload in this job.

According to the regression models. (1) and (3), the effort, frustration, and temporal demand subscales were the strongest predictors of the NASA-TLX index. Among the cognitive demands, only the individual's memory was retained in the Model (2), and the other eleven cognitive demands did not have the necessary fit to predict the NASA-TLX index. Besides, in the analysis of the regression of the twelve cognitive demands with the Raw-TLX index, only defect detection had the necessary predictive power. In the first regression model predicting the Raw-TLX, all NASA-TLX subscales were effective. In the third model, the NASA-TLX subscales and cognitive demands entered at the same time along with the visual cognitive demands and work tricks to predict Raw-TLX, but due to the poor regression coefficient, their predictive power was not significant. In models 1 and 3, all NASA-TLX subscales were able to predict Raw-TLX, which is clearly because Raw-TLX is derived from averaging these scales and the results of these two models were the same. Overall, the models developed using cognitive demands had a low fit (Adjusted R Square = 0.13 to 0.16). Accordingly, further studies are needed to shed light on this issue. One of the limitations of this study was that the data were collected under the normal working conditions of the control room, as the staff's workload increases especially in emergencies and at the time of the system failures. Another limitation of the study was the failure to receive a permit from the factory management for performing the electroencephalography (EEC) analysis of the staff's mental workload. Therefore, due to the high mental workload of the control room staff in this study, it is suggested that EEG analysis be used as a supplementary test in future studies.

Conclusion

The results of the study indicated both the total cognitive demand and the NASA-TLX indicators were high among the control room staff, and the two indicators were positively correlated. Moreover, the fact that the operators' workload was influenced by the cognitive demands of working memory and defect detection implies the lack of proper interaction between the tasks and abilities of the control room staffs, which requires some modifications to reduce the need for working memory due to the error-proneness of working memory. Therefore, to reduce the mental workload, it is recommended that working-memory and problem-solving-based tasks be assigned to computer systems in the control room.

Declarations

Author contribution statement

Mostafa Mohammadian: Performed the experiments; Wrote the paper.

Hosein Parsaei: Analyzed and interpreted the data.

Hamidreza Mokarami: Contributed reagents, materials, analysis tools or data.

Reza Kazemi: Conceived and designed the experiments; Wrote the paper.

Funding statement

This work was supported by (IR.SUMS.REC.1398.1233).

Data availability statement

Data included in article/supplementary material/referenced in article.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.