Grace Hickam1, Jaime Jordan2, Mary R C Haas3, Jason Wagner4, David Manthey5, Stephen John Cico6, Margaret Wolff7, Sally A Santen8. 1. Medical Education Fellow Clinical Instructor Department of Emergency Medicine Virginia Commonwealth University Richmond Virginia USA. 2. Associate Professor of Clinical Emergency Medicine Associate Director of Residency Training Program Vice-Chair Acute Care College Department of Emergency Medicine David Geffen School of Medicine at UCLA Ronald Reagan UCLA Medical Center Los Angeles California USA. 3. Assistant Residency Director Instructor Department of Emergency Medicine University of Michigan Medical School Ann Arbor Michigan USA. 4. Residency Program Director Assistant Professor of Emergency Medicine Washington University in St. Louis School of Medicine Saint Louis Missouri USA. 5. Professor of Emergency Medicine Wake Forest School of Medicine Winston-Salem North Carolina USA. 6. Assistant Dean for Graduate Medical Education Professor of Clinical Emergency Medicine & Pediatrics Indiana University School of Medicine Indianapolis Indiana USA. 7. Associate Professor of Emergency Medicine and Pediatrics University of Michigan Ann Arbor Michigan USA. 8. Senior Associate Dean, Assessment, Evaluation, and Scholarship and Professor Emergency Medicine Virginia Commonwealth University School of Medicine Professor of Emergency Medicine and Medical Education University of Cincinnati College of Medicine Cincinnati Ohio USA.
Abstract
BACKGROUND: COVID necessitated the shift to virtual resident instruction. The challenge of learning via virtual modalities has the potential to increase cognitive load. It is important for educators to reduce cognitive load to optimize learning, yet there are few available tools to measure cognitive load. The objective of this study is to identify and provide validity evidence following Messicks' framework for an instrument to evaluate cognitive load in virtual emergency medicine didactic sessions. METHODS: This study followed Messicks' framework for validity including content, response process, internal structure, and relationship to other variables. Content validity evidence included: (1) engagement of reference librarian and literature review of existing instruments; (2) engagement of experts in cognitive load, and relevant stakeholders to review the literature and choose an instrument appropriate to measure cognitive load in EM didactic presentations. Response process validity was gathered using the format and anchors of instruments with previous validity evidence and piloting amongst the author group. A lecture was provided by one faculty to four residency programs via ZoomTM. Afterwards, residents completed the cognitive load instrument. Descriptive statistics were collected; Cronbach's alpha assessed internal consistency of the instrument; and correlation for relationship to other variables (quality of lecture). RESULTS: The 10-item Leppink Cognitive Load instrument was selected with attention to content and response process validity evidence. Internal structure of the instrument was good (Cronbach's alpha = 0.80). Subscales performed well-intrinsic load (α = 0.96, excellent), extrinsic load (α = 0.89, good), and germane load (α = 0.97, excellent). Five of the items were correlated with overall quality of lecture (p < 0.05). CONCLUSIONS: The 10-item Cognitive Load instrument demonstrated good validity evidence to measure cognitive load and the subdomains of intrinsic, extraneous, and germane load. This instrument can be used to provide feedback to presenters to improve the cognitive load of their presentations.
BACKGROUND: COVID necessitated the shift to virtual resident instruction. The challenge of learning via virtual modalities has the potential to increase cognitive load. It is important for educators to reduce cognitive load to optimize learning, yet there are few available tools to measure cognitive load. The objective of this study is to identify and provide validity evidence following Messicks' framework for an instrument to evaluate cognitive load in virtual emergency medicine didactic sessions. METHODS: This study followed Messicks' framework for validity including content, response process, internal structure, and relationship to other variables. Content validity evidence included: (1) engagement of reference librarian and literature review of existing instruments; (2) engagement of experts in cognitive load, and relevant stakeholders to review the literature and choose an instrument appropriate to measure cognitive load in EM didactic presentations. Response process validity was gathered using the format and anchors of instruments with previous validity evidence and piloting amongst the author group. A lecture was provided by one faculty to four residency programs via ZoomTM. Afterwards, residents completed the cognitive load instrument. Descriptive statistics were collected; Cronbach's alpha assessed internal consistency of the instrument; and correlation for relationship to other variables (quality of lecture). RESULTS: The 10-item Leppink Cognitive Load instrument was selected with attention to content and response process validity evidence. Internal structure of the instrument was good (Cronbach's alpha = 0.80). Subscales performed well-intrinsic load (α = 0.96, excellent), extrinsic load (α = 0.89, good), and germane load (α = 0.97, excellent). Five of the items were correlated with overall quality of lecture (p < 0.05). CONCLUSIONS: The 10-item Cognitive Load instrument demonstrated good validity evidence to measure cognitive load and the subdomains of intrinsic, extraneous, and germane load. This instrument can be used to provide feedback to presenters to improve the cognitive load of their presentations.
Authors: Paul A Harris; Robert Taylor; Robert Thielke; Jonathon Payne; Nathaniel Gonzalez; Jose G Conde Journal: J Biomed Inform Date: 2008-09-30 Impact factor: 6.317
Authors: Adam Szulewski; Andreas Gegenfurtner; Daniel W Howes; Marco L A Sivilotti; Jeroen J G van Merriënboer Journal: Adv Health Sci Educ Theory Pract Date: 2016-10-27 Impact factor: 3.853
Authors: Grace Hickam; Sally A Santen; Stephen John Cico; David Manthey; Margaret Wolff; Joel Moll; Alexandra Lambert; Jaime Jordan; Mary R C Haas Journal: AEM Educ Train Date: 2020-10-09
Authors: David A Cook; Anthony J Levinson; Sarah Garside; Denise M Dupras; Patricia J Erwin; Victor M Montori Journal: JAMA Date: 2008-09-10 Impact factor: 56.272