David A Cook1, Rachel H Ellaway2. 1. a Mayo Clinic College of Medicine , USA . 2. b Northern Ontario School of Medicine , Canada.
Abstract
BACKGROUND: The absence of a standard, comprehensive approach to evaluating technology-enhanced learning (TEL) limits the utility of individual evaluations, and impedes the integration and synthesis of results across studies. PURPOSE: To outline a comprehensive framework for approaching TEL evaluation in medical education, and to develop instruments for measuring the perceptions of TEL learners and instructors. METHODS AND RESULTS: Using both theoretical constructs of inquiry in education and a synthesis of existing models and instruments, we outlined a general model for evaluation that links utility, principles, and practices. From this we derived a framework for TEL evaluation that identifies seven data collection activities: needs analysis; documentation of processes, decisions, and final product; usability testing; observation of implementation; assessment of participant experience; assessment of learning outcomes; and evaluation of cost, reusability, and sustainability. We then used existing quality standards and approaches to develop instruments for assessing the experiences of learners and instructors using TEL. CONCLUSIONS: No single evaluation is likely to collect all of this information, nor would any single audience likely find all information elements equally useful. However, consistent use of a common evaluation framework across different courses and institutions would avoid duplication of effort and allow cross-course comparisons.
BACKGROUND: The absence of a standard, comprehensive approach to evaluating technology-enhanced learning (TEL) limits the utility of individual evaluations, and impedes the integration and synthesis of results across studies. PURPOSE: To outline a comprehensive framework for approaching TEL evaluation in medical education, and to develop instruments for measuring the perceptions of TEL learners and instructors. METHODS AND RESULTS: Using both theoretical constructs of inquiry in education and a synthesis of existing models and instruments, we outlined a general model for evaluation that links utility, principles, and practices. From this we derived a framework for TEL evaluation that identifies seven data collection activities: needs analysis; documentation of processes, decisions, and final product; usability testing; observation of implementation; assessment of participant experience; assessment of learning outcomes; and evaluation of cost, reusability, and sustainability. We then used existing quality standards and approaches to develop instruments for assessing the experiences of learners and instructors using TEL. CONCLUSIONS: No single evaluation is likely to collect all of this information, nor would any single audience likely find all information elements equally useful. However, consistent use of a common evaluation framework across different courses and institutions would avoid duplication of effort and allow cross-course comparisons.
Authors: Tobias Mill; Shefali Parikh; Archie Allen; Gemma Dart; Daniel Lee; Charlotte Richardson; Keith Howell; Andrew Lewington Journal: BMJ Simul Technol Enhanc Learn Date: 2021-05-25
Authors: Brent Thoma; Michael Gottlieb; Megan Boysen-Osborn; Andrew King; Antonia Quinn; Sara Krzyzaniak; Nicolas Pineda; Lalena M Yarris; Teresa Chan Journal: Cureus Date: 2017-05-04