Colin Bell1, Andrew K Hall1, Natalie Wagner2,3, Louise Rang1, Joseph Newbigging1, Conor McKaigney4. 1. Department of Emergency Medicine Queen's University Kingston Ontario Canada. 2. Department of Biomedical & Molecular Sciences Queen's University Kingston Ontario Canada. 3. Office of Professional Development & Educational Scholarship Queen's University Kingston Ontario Canada. 4. Department of Emergency Medicine University of Calgary Calgary Alberta Canada.
Abstract
OBJECTIVES: Point-of-care ultrasound (POCUS) has become an integral diagnostic and interventional tool. Barriers to POCUS training persist, and it continues to remain heterogeneous across training programs. Structured POCUS assessment tools exist, but remain limited in their feasibility, acceptability, reliability, and validity; none of these tools are entrustment-based. The objective of this study was to derive a simple, entrustment-based POCUS competency assessment tool and pilot it in an assessment setting. METHODS: This study was composed of two phases. First, a three-step modified Delphi design surveyed 60 members of the Canadian Association of Emergency Physicians Emergency Ultrasound Committee (EUC) to derive the anchors for the tool. Subsequently, the derived ultrasound competency assessment tool (UCAT) was used to assess trainee (N = 37) performance on a simulated FAST examination. The intraclass correlation (ICC) for inter-rater reliability and Cronbach's alpha for internal consistency were calculated. A statistical analysis was performed to compare the UCAT to other competency surrogates. RESULTS: The three-round Delphi had 22, 26, and 26 responses from the EUC members. Consensus was reached, and anchors for the domains of preparation, image acquisition, image optimization, and clinical integration achieved approval rates between 92 and 96%. The UCAT pilot revealed excellent inter-rater reliability (with ICC values of 0.69-0.89; p < 0.01) and high internal consistency (α = 0.91). While UCAT scores were not impacted by level of training, they were significantly impacted by the number of previous POCUS studies completed. CONCLUSIONS: We developed and successfully piloted the UCAT, an entrustment-based bedside POCUS competency assessment tool suitable for rapid deployment. The findings from this study indicate early validity evidence for the use of the UCAT as an assessment of trainee POCUS competence on FAST. The UCAT should be trialed in different populations performing several POCUS study types.
OBJECTIVES: Point-of-care ultrasound (POCUS) has become an integral diagnostic and interventional tool. Barriers to POCUS training persist, and it continues to remain heterogeneous across training programs. Structured POCUS assessment tools exist, but remain limited in their feasibility, acceptability, reliability, and validity; none of these tools are entrustment-based. The objective of this study was to derive a simple, entrustment-based POCUS competency assessment tool and pilot it in an assessment setting. METHODS: This study was composed of two phases. First, a three-step modified Delphi design surveyed 60 members of the Canadian Association of Emergency Physicians Emergency Ultrasound Committee (EUC) to derive the anchors for the tool. Subsequently, the derived ultrasound competency assessment tool (UCAT) was used to assess trainee (N = 37) performance on a simulated FAST examination. The intraclass correlation (ICC) for inter-rater reliability and Cronbach's alpha for internal consistency were calculated. A statistical analysis was performed to compare the UCAT to other competency surrogates. RESULTS: The three-round Delphi had 22, 26, and 26 responses from the EUC members. Consensus was reached, and anchors for the domains of preparation, image acquisition, image optimization, and clinical integration achieved approval rates between 92 and 96%. The UCAT pilot revealed excellent inter-rater reliability (with ICC values of 0.69-0.89; p < 0.01) and high internal consistency (α = 0.91). While UCAT scores were not impacted by level of training, they were significantly impacted by the number of previous POCUS studies completed. CONCLUSIONS: We developed and successfully piloted the UCAT, an entrustment-based bedside POCUS competency assessment tool suitable for rapid deployment. The findings from this study indicate early validity evidence for the use of the UCAT as an assessment of trainee POCUS competence on FAST. The UCAT should be trialed in different populations performing several POCUS study types.
Authors: Jason R Frank; Linda S Snell; Olle Ten Cate; Eric S Holmboe; Carol Carraccio; Susan R Swing; Peter Harris; Nicholas J Glasgow; Craig Campbell; Deepak Dath; Ronald M Harden; William Iobst; Donlin M Long; Rani Mungroo; Denyse L Richardson; Jonathan Sherbino; Ivan Silver; Sarah Taber; Martin Talbot; Kenneth A Harris Journal: Med Teach Date: 2010 Impact factor: 3.650
Authors: Daniel J Kim; Jonathan Theoret; Michael M Liao; Emily Hopkins; Karen Woolfrey; John L Kendall Journal: Acad Emerg Med Date: 2012-09 Impact factor: 3.451
Authors: Jonathan Sherbino; Glen Bandiera; Ken Doyle; Jason R Frank; Brian R Holroyd; Gord Jones; Joanne Norum; Carolyn Snider; Kirk Magee Journal: CJEM Date: 2020-01 Impact factor: 2.410
Authors: Ross Prager; Paul Pageau; Timothy Hodges; Christina Yan; Michael Woo; Marie-Joe Nemnom; Scott Millington; Matthew Holden; Raphael St-Gelais; Warren J Cheung Journal: AEM Educ Train Date: 2022-04-01
Authors: Delia Gold; Marla Levine; Deborah Hsu; David P Way; Allan E Shefrin; Samuel H F Lam; Resa Lewiss; Jennifer R Marin Journal: AEM Educ Train Date: 2021-08-01