Mia L Østergaard1, Kristina R Nielsen2, Elisabeth Albrecht-Beste3, Lars Konge4, Michael B Nielsen2. 1. Department of Radiology, Copenhagen University Hospital, Rigshospitalet, Blegdamsvej 9, afd. 2023, 2100, Copenhagen O, Denmark. mlo@dadlnet.dk. 2. Department of Radiology, Copenhagen University Hospital, Rigshospitalet, Blegdamsvej 9, afd. 2023, 2100, Copenhagen O, Denmark. 3. Department of Clinical Physiology, Nuclear Medicine and PET, Copenhagen University Hospital, Rigshospitalet, Blegdamsvej 9, 2100, Copenhagen, Denmark. 4. Copenhagen Academy for Medical Education and Simulation CAMES, The Capital Region of Denmark, Blegdamsvej 9, 2100, Copenhagen, Denmark.
Abstract
BACKGROUND: This study aimed to develop a test with validity evidence for abdominal diagnostic ultrasound with a pass/fail-standard to facilitate mastery learning. METHOD: The simulator had 150 real-life patient abdominal scans of which 15 cases with 44 findings were selected, representing level 1 from The European Federation of Societies for Ultrasound in Medicine and Biology. Four groups of experience levels were constructed: Novices (medical students), trainees (first-year radiology residents), intermediates (third- to fourth-year radiology residents) and advanced (physicians with ultrasound fellowship). Participants were tested in a standardized setup and scored by two blinded reviewers prior to an item analysis. RESULTS: The item analysis excluded 14 diagnoses. Both internal consistency (Cronbach's alpha 0.96) and inter-rater reliability (0.99) were good and there were statistically significant differences (p < 0.001) between all four groups, except the intermediate and advanced groups (p = 1.0). There was a statistically significant correlation between experience and test scores (Pearson's r = 0.82, p < 0.001). The pass/fail-standard failed all novices (no false positives) and passed all advanced (no false negatives). All intermediate participants and six out of 14 trainees passed. CONCLUSION: We developed a test for diagnostic abdominal ultrasound with solid validity evidence and a pass/fail-standard without any false-positive or false-negative scores. KEY POINTS: • Ultrasound training can benefit from competency-based education based on reliable tests. • This simulation-based test can differentiate between competency levels of ultrasound examiners. • This test is suitable for competency-based education, e.g. mastery learning. • We provide a pass/fail standard without false-negative or false-positive scores.
BACKGROUND: This study aimed to develop a test with validity evidence for abdominal diagnostic ultrasound with a pass/fail-standard to facilitate mastery learning. METHOD: The simulator had 150 real-life patient abdominal scans of which 15 cases with 44 findings were selected, representing level 1 from The European Federation of Societies for Ultrasound in Medicine and Biology. Four groups of experience levels were constructed: Novices (medical students), trainees (first-year radiology residents), intermediates (third- to fourth-year radiology residents) and advanced (physicians with ultrasound fellowship). Participants were tested in a standardized setup and scored by two blinded reviewers prior to an item analysis. RESULTS: The item analysis excluded 14 diagnoses. Both internal consistency (Cronbach's alpha 0.96) and inter-rater reliability (0.99) were good and there were statistically significant differences (p < 0.001) between all four groups, except the intermediate and advanced groups (p = 1.0). There was a statistically significant correlation between experience and test scores (Pearson's r = 0.82, p < 0.001). The pass/fail-standard failed all novices (no false positives) and passed all advanced (no false negatives). All intermediate participants and six out of 14 trainees passed. CONCLUSION: We developed a test for diagnostic abdominal ultrasound with solid validity evidence and a pass/fail-standard without any false-positive or false-negative scores. KEY POINTS: • Ultrasound training can benefit from competency-based education based on reliable tests. • This simulation-based test can differentiate between competency levels of ultrasound examiners. • This test is suitable for competency-based education, e.g. mastery learning. • We provide a pass/fail standard without false-negative or false-positive scores.
Authors: Lars Konge; Paul Frost Clementsen; Charlotte Ringsted; Valentina Minddal; Klaus Richter Larsen; Jouke T Annema Journal: Eur Respir J Date: 2015-07-09 Impact factor: 16.671
Authors: Ann Sofia Skou Thomsen; Jens Folke Kiilgaard; Hadi Kjaerbo; Morten la Cour; Lars Konge Journal: Acta Ophthalmol Date: 2015-02-26 Impact factor: 3.761
Authors: Iman Ghaderi; Farouq Manji; Yoon Soo Park; Dorthea Juul; Michael Ott; Ilene Harris; Timothy M Farrell Journal: Ann Surg Date: 2015-02 Impact factor: 12.969
Authors: M R Torloni; N Vedmedovska; M Merialdi; A P Betrán; T Allen; R González; L D Platt Journal: Ultrasound Obstet Gynecol Date: 2009-05 Impact factor: 7.299
Authors: Leizl Joy Nayahangan; Kristina Rue Nielsen; Elisabeth Albrecht-Beste; Michael Bachmann Nielsen; Charlotte Paltved; Karen Gilboe Lindorff-Larsen; Bjørn Ulrik Nielsen; Lars Konge Journal: Eur Radiol Date: 2018-01-09 Impact factor: 5.315
Authors: Mia Louise Østergaard; Kristina Rue Nielsen; Elisabeth Albrecht-Beste; Annette Kjær Ersbøll; Lars Konge; Michael Bachmann Nielsen Journal: Eur Radiol Date: 2019-01-07 Impact factor: 5.315
Authors: John Bailitz; James O'Brien; Matt McCauley; David Murray; Christine Jung; Gary Peksa; Michael Gottlieb Journal: AEM Educ Train Date: 2022-08-03
Authors: Pia Iben Pietersen; Kristian Rørbæk Madsen; Ole Graumann; Lars Konge; Bjørn Ulrik Nielsen; Christian Borbjerg Laursen Journal: Crit Ultrasound J Date: 2018-09-03
Authors: Mia Louise Østergaard; Lars Konge; Niklas Kahr; Elisabeth Albrecht-Beste; Michael Bachmann Nielsen; Kristina Rue Nielsen Journal: Diagnostics (Basel) Date: 2019-05-06