Lukasz M Mazur1, Prithima R Mosaly2, Lesley M Hoyle2, Ellen L Jones2, Lawrence B Marks2. 1. Department of Radiation Oncology, University of North Carolina, Chapel Hill, North Carolina. Electronic address: lmazur@med.unc.edu. 2. Department of Radiation Oncology, University of North Carolina, Chapel Hill, North Carolina.
Abstract
PURPOSE: To quantify, and compare, workload for several common physician-based treatment planning tasks using objective and subjective measures of workload. To assess the relationship between workload and performance to define workload levels where performance could be expected to decline. METHODS AND MATERIALS: Nine physicians performed the same 3 tasks on each of 2 cases ("easy" vs "hard"). Workload was assessed objectively throughout the tasks (via monitoring of pupil size and blink rate), and subjectively at the end of each case (via National Aeronautics and Space Administration Task Load Index; NASA-TLX). NASA-TLX assesses the 6 dimensions (mental, physical, and temporal demands, frustration, effort, and performance); scores > or ≈ 50 are associated with reduced performance in other industries. Performance was measured using participants' stated willingness to approve the treatment plan. Differences in subjective and objective workload between cases, tasks, and experience were assessed using analysis of variance (ANOVA). The correlation between subjective and objective workload measures were assessed via the Pearson correlation test. The relationships between workload and performance measures were assessed using the t test. RESULTS: Eighteen case-wise and 54 task-wise assessments were obtained. Subjective NASA-TLX scores (P < .001), but not time-weighted averages of objective scores (P > .1), were significantly lower for the easy vs hard case. Most correlations between the subjective and objective measures were not significant, except between average blink rate and NASA-TLX scores (r = -0.34, P = .02), for task-wise assessments. Performance appeared to decline at NASA-TLX scores of ≥55. CONCLUSIONS: The NASA-TLX may provide a reasonable method to quantify subjective workload for broad activities, and objective physiologic eye-based measures may be useful to monitor workload for more granular tasks within activities. The subjective and objective measures, as herein quantified, do not necessarily track each other, and more work is needed to assess their utilities. From a series of controlled experiments, we found that performance appears to decline at subjective workload levels ≥55 (as measured via NASA-TLX), which is consistent with findings from other industries.
PURPOSE: To quantify, and compare, workload for several common physician-based treatment planning tasks using objective and subjective measures of workload. To assess the relationship between workload and performance to define workload levels where performance could be expected to decline. METHODS AND MATERIALS: Nine physicians performed the same 3 tasks on each of 2 cases ("easy" vs "hard"). Workload was assessed objectively throughout the tasks (via monitoring of pupil size and blink rate), and subjectively at the end of each case (via National Aeronautics and Space Administration Task Load Index; NASA-TLX). NASA-TLX assesses the 6 dimensions (mental, physical, and temporal demands, frustration, effort, and performance); scores > or ≈ 50 are associated with reduced performance in other industries. Performance was measured using participants' stated willingness to approve the treatment plan. Differences in subjective and objective workload between cases, tasks, and experience were assessed using analysis of variance (ANOVA). The correlation between subjective and objective workload measures were assessed via the Pearson correlation test. The relationships between workload and performance measures were assessed using the t test. RESULTS: Eighteen case-wise and 54 task-wise assessments were obtained. Subjective NASA-TLX scores (P < .001), but not time-weighted averages of objective scores (P > .1), were significantly lower for the easy vs hard case. Most correlations between the subjective and objective measures were not significant, except between average blink rate and NASA-TLX scores (r = -0.34, P = .02), for task-wise assessments. Performance appeared to decline at NASA-TLX scores of ≥55. CONCLUSIONS: The NASA-TLX may provide a reasonable method to quantify subjective workload for broad activities, and objective physiologic eye-based measures may be useful to monitor workload for more granular tasks within activities. The subjective and objective measures, as herein quantified, do not necessarily track each other, and more work is needed to assess their utilities. From a series of controlled experiments, we found that performance appears to decline at subjective workload levels ≥55 (as measured via NASA-TLX), which is consistent with findings from other industries.
Authors: Denny Yu; Cem Dural; Melissa M B Morrow; Liyun Yang; Justin W Collins; Susan Hallbeck; Magnus Kjellman; Mikael Forsman Journal: Surg Endosc Date: 2016-08-05 Impact factor: 4.584
Authors: Daniel S Lemke; Ann L Young; Sharon K Won; Marideth C Rus; Nadia N Villareal; Elizabeth A Camp; Cara Doughty Journal: AEM Educ Train Date: 2021-08-01
Authors: Jie Xu; Carrie Reale; Jason M Slagle; Shilo Anders; Matthew S Shotwell; Timothy Dresselhaus; Matthew B Weinger Journal: Nurs Res Date: 2017 Sep/Oct Impact factor: 2.364
Authors: Johannes Prottengeier; Johann Georg Keunecke; Christine Gall; Christian Eiche; Andreas Moritz; Torsten Birkholz Journal: Scand J Trauma Resusc Emerg Med Date: 2019-08-16 Impact factor: 2.953
Authors: Michael Wagner; Peter Gröpel; Felix Eibensteiner; Lisa Kessler; Katharina Bibl; Isabel T Gross; Angelika Berger; Francesco S Cardona Journal: Pediatr Res Date: 2021-07-21 Impact factor: 3.953
Authors: Lukasz M Mazur; Lawrence B Marks; Ron McLeod; Waldemar Karwowski; Prithima Mosaly; Gregg Tracton; Robert D Adams; Lesley Hoyle; Shiva Das; Bhishamjit Chera Journal: Adv Radiat Oncol Date: 2018-02-07