Erik G Prytz1, Jonas Rybing1, Carl-Oscar Jonson2. 1. Department of Computer and Information Science, Linköping University, Linköping, Sweden. 2. Centre for Teaching and Research in Disaster Medicine and Traumatology, Department of Clinical and Experimental Medicine, Linköping University, Linköping, Sweden.
Abstract
OBJECTIVE: This study reports on an initial test using a validated workload measurement method, the NASA Task Load Index (TLX), as an indicator of joint emergency exercise effectiveness. Prior research on emergency exercises indicates that exercises must be challenging, ie, result in high workload, to be effective. However, this is often problematic with some participants being underloaded and some overloaded. The NASA TLX was used to test for differences in workload between commanders and subordinates and among three different emergency response organizations during a joint emergency exercise. DESIGN: Questionnaire-based evaluation with professional emergency responders. SETTING: The study was performed in conjunction with a large-scale interorganizational joint emergency exercise in Sweden. PARTICIPANTS: A total of 20 participants from the rescue services, 12 from the emergency medical services, and 12 from the police participated in the study (N=44). Ten participants had a command-level role during the exercise and the remaining 34 were subordinates. MAIN OUTCOME MEASURE(S): The main outcome measures were the workload subscales of the NASA TLX: mental demands, physical demands, temporal demands, performance, effort, and frustration. RESULTS: The results showed that the organizations experienced different levels of workload, that the commanders experienced a higher workload than the subordinates, and that two out of three organizations fell below the twenty-fifth percentile of average workload scores compiled from 237 prior studies. CONCLUSIONS: The results support the notion that the NASA TLX could be a useful complementary tool to evaluate exercise designs and outcomes. This should be further explored and verified in additional studies.
OBJECTIVE: This study reports on an initial test using a validated workload measurement method, the NASA Task Load Index (TLX), as an indicator of joint emergency exercise effectiveness. Prior research on emergency exercises indicates that exercises must be challenging, ie, result in high workload, to be effective. However, this is often problematic with some participants being underloaded and some overloaded. The NASA TLX was used to test for differences in workload between commanders and subordinates and among three different emergency response organizations during a joint emergency exercise. DESIGN: Questionnaire-based evaluation with professional emergency responders. SETTING: The study was performed in conjunction with a large-scale interorganizational joint emergency exercise in Sweden. PARTICIPANTS: A total of 20 participants from the rescue services, 12 from the emergency medical services, and 12 from the police participated in the study (N=44). Ten participants had a command-level role during the exercise and the remaining 34 were subordinates. MAIN OUTCOME MEASURE(S): The main outcome measures were the workload subscales of the NASA TLX: mental demands, physical demands, temporal demands, performance, effort, and frustration. RESULTS: The results showed that the organizations experienced different levels of workload, that the commanders experienced a higher workload than the subordinates, and that two out of three organizations fell below the twenty-fifth percentile of average workload scores compiled from 237 prior studies. CONCLUSIONS: The results support the notion that the NASA TLX could be a useful complementary tool to evaluate exercise designs and outcomes. This should be further explored and verified in additional studies.