Scott J Millington1, Robert T Arntfield2, Michael Hewak2, Stanley J Hamstra3, Yanick Beaulieu4, Benjamin Hibbert5, Seth Koenig6, Pierre Kory7, Paul Mayo6, Jordan R Schoenherr8. 1. Division of Critical Care Medicine, University of Ottawa, Ottawa, Ontario, Canada smillington@toh.ca. 2. Western University, London, Ontario, Canada. 3. Accreditation Council for Graduate Medical Education, Chicago, Illinois USA. 4. Hôpital du Sacré-Coeur de Montréal, Montréal, Québec, Canada. 5. University of Ottawa Heart Institute, Ottawa, Ontario, Canada. 6. Hofstra North Shore-LIJ School of Medicine, Hempstead, New York USA. 7. University of Wisconsin, Madison, Wisconsin USA. 8. Carleton University, Ottawa, Ontario, Canada.
Abstract
OBJECTIVES: Increased use of point-of-care ultrasound (US) requires the development of assessment tools that measure the competency of learners. In this study, we developed and tested a tool to assess the quality of point-of-care cardiac US studies performed by novices. METHODS: In phase 1, the Rapid Assessment of Competency in Echocardiography (RACE) scale was developed on the basis of structured interviews with subject matter experts; the tool was then piloted on a small series of US studies in phase 2. In phase 3, the tool was applied to a sample of 154 point-of-care US studies performed by 12 learners; each study was independently rated by 2 experts, with quantitative analysis subsequently performed. RESULTS: Evidence of the content validity of the RACE scale was supported by a consensus exercise, wherein experts agreed on the assessment dimensions and specific items that made up the RACE scale. The tool showed good inter-rater reliability. An analysis of inter-item correlations provided support for the internal structure of the scale, and the tool was able to discriminate between learners early in their point-of-care US learning and those who were more advanced in their training. CONCLUSIONS: The RACE scale provides a straightforward means to assess learner performance with minimal requirements for evaluator training. Our results support the conclusion that the tool is an effective means of making valid judgments regarding competency in point-of-care cardiac US.
OBJECTIVES: Increased use of point-of-care ultrasound (US) requires the development of assessment tools that measure the competency of learners. In this study, we developed and tested a tool to assess the quality of point-of-care cardiac US studies performed by novices. METHODS: In phase 1, the Rapid Assessment of Competency in Echocardiography (RACE) scale was developed on the basis of structured interviews with subject matter experts; the tool was then piloted on a small series of US studies in phase 2. In phase 3, the tool was applied to a sample of 154 point-of-care US studies performed by 12 learners; each study was independently rated by 2 experts, with quantitative analysis subsequently performed. RESULTS: Evidence of the content validity of the RACE scale was supported by a consensus exercise, wherein experts agreed on the assessment dimensions and specific items that made up the RACE scale. The tool showed good inter-rater reliability. An analysis of inter-item correlations provided support for the internal structure of the scale, and the tool was able to discriminate between learners early in their point-of-care US learning and those who were more advanced in their training. CONCLUSIONS: The RACE scale provides a straightforward means to assess learner performance with minimal requirements for evaluator training. Our results support the conclusion that the tool is an effective means of making valid judgments regarding competency in point-of-care cardiac US.
Keywords:
assessment tool; echocardiography; medical education; point-of-care ultrasound
Authors: P Mayo; R Arntfield; M Balik; P Kory; G Mathis; G Schmidt; M Slama; G Volpicelli; N Xirouchaki; A McLean; A Vieillard-Baron Journal: Intensive Care Med Date: 2017-03-07 Impact factor: 17.440
Authors: Kiyetta H Alade; Jennifer R Marin; Erika Constantine; Atim Ekpenyong; Susan E Farrell; Russ Horowitz; Deborah Hsu; Charisse W Kwan; Lorraine Ng; Perry J Leonard; Resa E Lewiss Journal: AEM Educ Train Date: 2021-08-01
Authors: Robert F Reardon; Elliott Chinn; Dave Plummer; Andrew Laudenbach; Andie Rowland Fisher; Will Smoot; Daniel Lee; Joseph Novik; Barrett Wagner; Chris Kaczmarczyk; Johanna Moore; Emily Thompson; Craig Tschautscher; Teresa Dunphy; Thomas Pahl; Michael A Puskarich; James R Miner Journal: Acad Emerg Med Date: 2021-11-06 Impact factor: 5.221
Authors: Scott J Millington; Robert T Arntfield; Robert Jie Guo; Seth Koenig; Pierre Kory; Vicki Noble; Haney Mallemat; Jordan R Schoenherr Journal: Crit Ultrasound J Date: 2017-11-22
Authors: Jessica Buesing; Yingjie Weng; John Kugler; Libo Wang; Ondrej Blaha; Jason Hom; Neera Ahuja; Andre Kumar Journal: J Grad Med Educ Date: 2021-12-29