Owen B J Carter1, Brennen W Mills1, Nathan P Ross2, Alecka K Miles2, Jonathan M Mould3, Robert P O'Brien4. 1. Office of the Pro Vice-Chancellor (Health Advancement), Edith Cowan University, Joondalup Western Australia, Australia. 2. School of Medicine, Deakin University, Geelong, Victoria, Australia. 3. Division of Health Sciences, School of Nursing, Midwifery and Paramedicine, Curtin University, Bentley, Western Australia, Australia. 4. School of Medicine, The University of Melbourne, Parkville, Victoria, Australia.
Abstract
Introduction: Simulation-based education (SBE) literature is replete with student satisfaction and confidence measures to infer educational outcomes. This research aims to test how well students' satisfaction and confidence measures correlate with expert assessments of students' improvements in competence following SBE activities. Methods: N=85 paramedic students (mean age 23.7 years, SD=6.5; 48.2% female) undertook a 3-day SBE workshop. Students' baseline competence was assessed via practical scenario simulation assessments (PSSAs) administered by expert paramedics and confidence via a questionnaire. Postworkshop competence and confidence plus self-reported students' satisfaction were remeasured. Results: PSSA scores increased significantly between baseline and post workshop (35.7%→53.4%, p<0.001), as did students' confidence (55.7%→60.5%, p<0.001), and their workshop satisfaction was high (71.0%). Satisfaction and postworkshop confidence measures were moderately correlated (r=0.377, p=0.001). However, competence improvements were not significantly correlated with either satisfaction (r=-0.107 p=0.344) or change in confidence (r=-0.187 p=0.102). Discussion: Students' self-reported satisfaction and confidence measures bore little relation to expert paramedics' judgements of their educational improvements. Satisfaction and confidence measures appear to be dubious indicators of SBE learning outcomes. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.
Introduction: Simulation-based education (SBE) literature is replete with student satisfaction and confidence measures to infer educational outcomes. This research aims to test how well students' satisfaction and confidence measures correlate with expert assessments of students' improvements in competence following SBE activities. Methods: N=85 paramedic students (mean age 23.7 years, SD=6.5; 48.2% female) undertook a 3-day SBE workshop. Students' baseline competence was assessed via practical scenario simulation assessments (PSSAs) administered by expert paramedics and confidence via a questionnaire. Postworkshop competence and confidence plus self-reported students' satisfaction were remeasured. Results: PSSA scores increased significantly between baseline and post workshop (35.7%→53.4%, p<0.001), as did students' confidence (55.7%→60.5%, p<0.001), and their workshop satisfaction was high (71.0%). Satisfaction and postworkshop confidence measures were moderately correlated (r=0.377, p=0.001). However, competence improvements were not significantly correlated with either satisfaction (r=-0.107 p=0.344) or change in confidence (r=-0.187 p=0.102). Discussion: Students' self-reported satisfaction and confidence measures bore little relation to expert paramedics' judgements of their educational improvements. Satisfaction and confidence measures appear to be dubious indicators of SBE learning outcomes. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.