Daniel J Ackil1, Amanda Toney2, Ryan Good3, David Ross4, Rocco Germano5, Linda Sabbadini5, Molly Thiessen6, Colin Bell7, John L Kendall6. 1. Department of Surgery Division of Emergency Medicine University of Vermont Larner College of Medicine Burlington VT USA. 2. the Department of Pediatric Emergency Medicine University of Colorado School of Medicine Denver Health Medical Center Denver CO USA. 3. the Department of Pediatrics Section of Critical Care University of Colorado School of Medicine Anschutz Medical Campus and Children's Hospital Colorado Aurora CO USA. 4. the Rocky Vista University College of Osteopathic Medicine Parker CO USA. 5. the Facolta di Medicina e Chirurgia Universita degli Studi di Brescia Spedali Civil di Brescia Brescia Italy. 6. and the Department of Emergency Medicine University of Colorado School of Medicine Denver Health Medical Center Denver CO USA. 7. and the Department of Emergency Medicine Queen's University Kingston Ontario Canada.
Abstract
OBJECTIVES: Assessment of competence in technical skills, including point-of-care ultrasound (POCUS), is required before a novice can safely perform the skill independently. Ongoing assessment of competence is also required because technical skills degrade over time, especially when they are infrequently performed or complex. Hand-motion analysis (HMA) is an objective assessment tool that has been used to evaluate competency in many technical skills. The purpose of this study was to demonstrate the feasibility and validity of HMA as an assessment tool for competence in both simple and complex technical skills as well as skill degradation over time. METHODS: This prospective cohort study included 36 paramedics with no POCUS experience and six physicians who were fellowship trained in POCUS. The novices completed a 4-hour didactic and hands-on training program for cardiac and lung POCUS. HMA measurements, objective structured clinical examinations (OSCE), and written examinations were collected for novices immediately before and after training as well as 2 and 4 months after training. Expert HMA metrics were also recorded. RESULTS: Expert HMA metrics for cardiac and lung POCUS were significantly better than those of novices. After completion of the training program, the novices improved significantly in all HMA metrics, knowledge test scores, and OSCE scores. Novices showed skill degradation in cardiac POCUS based on HMA metrics and OSCE scores while lung POCUS image acquisition skills were preserved. Novices deemed competent by OSCE score performed significantly better in HMA metrics than those not deemed competent. CONCLUSION: We have demonstrated that HMA is a feasible and valid tool for assessment of competence in technical skills and can also evaluate skill degradation over time. Skill degradation appears more apparent in complex skills, such as cardiac POCUS. HMA may provide a more efficient and reliable assessment of technical skills, including POCUS, when compared to traditional assessment tools.
OBJECTIVES: Assessment of competence in technical skills, including point-of-care ultrasound (POCUS), is required before a novice can safely perform the skill independently. Ongoing assessment of competence is also required because technical skills degrade over time, especially when they are infrequently performed or complex. Hand-motion analysis (HMA) is an objective assessment tool that has been used to evaluate competency in many technical skills. The purpose of this study was to demonstrate the feasibility and validity of HMA as an assessment tool for competence in both simple and complex technical skills as well as skill degradation over time. METHODS: This prospective cohort study included 36 paramedics with no POCUS experience and six physicians who were fellowship trained in POCUS. The novices completed a 4-hour didactic and hands-on training program for cardiac and lung POCUS. HMA measurements, objective structured clinical examinations (OSCE), and written examinations were collected for novices immediately before and after training as well as 2 and 4 months after training. Expert HMA metrics were also recorded. RESULTS: Expert HMA metrics for cardiac and lung POCUS were significantly better than those of novices. After completion of the training program, the novices improved significantly in all HMA metrics, knowledge test scores, and OSCE scores. Novices showed skill degradation in cardiac POCUS based on HMA metrics and OSCE scores while lung POCUS image acquisition skills were preserved. Novices deemed competent by OSCE score performed significantly better in HMA metrics than those not deemed competent. CONCLUSION: We have demonstrated that HMA is a feasible and valid tool for assessment of competence in technical skills and can also evaluate skill degradation over time. Skill degradation appears more apparent in complex skills, such as cardiac POCUS. HMA may provide a more efficient and reliable assessment of technical skills, including POCUS, when compared to traditional assessment tools.
Authors: Dimitrios Stefanidis; James R Korndorffer; Sarah Markley; Rafael Sierra; Daniel J Scott Journal: J Am Coll Surg Date: 2006-04 Impact factor: 6.113
Authors: Megan A Hayter; Zeev Friedman; M Dylan Bould; John G Hanlon; Rita Katznelson; Bruno Borges; Viren N Naik Journal: Can J Anaesth Date: 2009-04-02 Impact factor: 5.063
Authors: David Clinkard; Matthew Holden; Tamas Ungi; David Messenger; Colleen Davison; Gabor Fichtinger; Robert McGraw Journal: Acad Emerg Med Date: 2015-02-09 Impact factor: 3.451
Authors: Charles A Rappaport; Bryan C McConomy; Nicholas R Arnold; Aaron T Vose; Gregory A Schmidt; Boulos Nassar Journal: Crit Care Med Date: 2019-12 Impact factor: 7.598
Authors: Ryan J Good; Kelly K Rothman; Daniel J Ackil; John S Kim; Jonathan Orsborn; John L Kendall Journal: J Vasc Access Date: 2018-10-14 Impact factor: 2.283
Authors: Vicki E Noble; Lionel Lamhaut; Roberta Capp; Nichole Bosson; Andrew Liteplo; Jean-Sebastian Marx; Pierre Carli Journal: BMC Med Educ Date: 2009-01-12 Impact factor: 2.463