Isaac A Thangasamy1, Stacy Loeb2, Niranjan J Sathianathen3, Michael Leveridge4, Brian Stork5, Benjamin J Davies6, Henry H Woo7. 1. Division of Cancer Surgery, Peter MacCallum Cancer Centre, University of Melbourne, Melbourne, VIC, Australia; Faculty of Medicine, University of Queensland, Brisbane, QLD, Australia. 2. Department of Urology and Population Health, New York University, New York City, NY, USA. 3. Division of Cancer Surgery, Peter MacCallum Cancer Centre, University of Melbourne, Melbourne, VIC, Australia. 4. Department of Urology, Queen's University, Kingston, ON, Canada. 5. Department of Urology, University of Michigan, Michigan, MI, USA. 6. Department of Urology, University of Pittsburgh Medical Center, Pittsburgh, PA, USA. 7. Sydney Adventist Hospital Clinical School, University of Sydney, Sydney, NSW, Australia; Department of Uro-Oncology and Robotic Cancer Surgery, Chris O'Brien Lifehouse, Camperdown, NSW, Australia. Electronic address: hwoo@urologist.net.au.
Abstract
BACKGROUND: The International Urology Journal Club attracts global participation; however, its effectiveness as an educational resource is undetermined. OBJECTIVE: To evaluate the effectiveness of the international Twitter-based urology journal club (#urojc) using Kirkpatrick's evaluation model and to quantify the development of #urojc into a robust educational platform. DESIGN, SETTING, AND PARTICIPANTS: A 13-item survey was sent to 3127 followers of #urojc. Twitter metrics including number of users, tweets, and impressions during 24 mo of operation were obtained via a third-party service "Symplur." Tweets were qualitatively analyzed to assess for relevance. Kirkpatrick's four-level evaluation model (reaction, learning, behavior, and results) was applied to analyze impact and effectiveness. RESULTS AND LIMITATIONS: Survey participation rate was 48% (after accounting for spam and inactive accounts). The reaction (Kirkpatrick level 1) to #urojc was overwhelmingly positive. Consultant urologists were the most active participants while trainees reported highest educational value. As much as 45% of junior consultant urologists and trainees reported a change in practice as a result of #urojc participation. The number of unique users of #urojc almost doubled from yr 1 to 2. The number of tweets per month and author participation increased from yr 1 to 2. The proportion of content-relevant tweets remained stably high at approximately 60%. This study was unable to document any impact on knowledge, behavior, or outcomes of #urojc (Kirkpatrick levels 2-4). Silent observers who do not follow #urojc were not captured. CONCLUSIONS: To our knowledge, this is the first time Kirkpatrick's evaluation model has been applied to a journal club. The reaction to #urojc among survey responders is positive with change in practice also reported. It appears that #urojc achieves level 1 of Kirkpatrick's evaluation model. Further investigation is required for levels 2-4. Twitter-based journal clubs draw robust global participation and have inspired journal clubs in other specialties. PATIENT SUMMARY: This report highlights the importance of the International Urology Journal Club as a potential educational resource of consultant urologists and trainees alike. Global participation was a major advantage of this platform. This journal club has inspired other medical specialties to follow suit.
BACKGROUND: The International Urology Journal Club attracts global participation; however, its effectiveness as an educational resource is undetermined. OBJECTIVE: To evaluate the effectiveness of the international Twitter-based urology journal club (#urojc) using Kirkpatrick's evaluation model and to quantify the development of #urojc into a robust educational platform. DESIGN, SETTING, AND PARTICIPANTS: A 13-item survey was sent to 3127 followers of #urojc. Twitter metrics including number of users, tweets, and impressions during 24 mo of operation were obtained via a third-party service "Symplur." Tweets were qualitatively analyzed to assess for relevance. Kirkpatrick's four-level evaluation model (reaction, learning, behavior, and results) was applied to analyze impact and effectiveness. RESULTS AND LIMITATIONS: Survey participation rate was 48% (after accounting for spam and inactive accounts). The reaction (Kirkpatrick level 1) to #urojc was overwhelmingly positive. Consultant urologists were the most active participants while trainees reported highest educational value. As much as 45% of junior consultant urologists and trainees reported a change in practice as a result of #urojc participation. The number of unique users of #urojc almost doubled from yr 1 to 2. The number of tweets per month and author participation increased from yr 1 to 2. The proportion of content-relevant tweets remained stably high at approximately 60%. This study was unable to document any impact on knowledge, behavior, or outcomes of #urojc (Kirkpatrick levels 2-4). Silent observers who do not follow #urojc were not captured. CONCLUSIONS: To our knowledge, this is the first time Kirkpatrick's evaluation model has been applied to a journal club. The reaction to #urojc among survey responders is positive with change in practice also reported. It appears that #urojc achieves level 1 of Kirkpatrick's evaluation model. Further investigation is required for levels 2-4. Twitter-based journal clubs draw robust global participation and have inspired journal clubs in other specialties. PATIENT SUMMARY: This report highlights the importance of the International Urology Journal Club as a potential educational resource of consultant urologists and trainees alike. Global participation was a major advantage of this platform. This journal club has inspired other medical specialties to follow suit.
Authors: Mara Taverna; Julian Nicolaus Bucher; Maximilian Weniger; Roswitha Gropp; Serene M L Lee; Barbara Mayer; Jens Werner; Alexandr V Bazhin Journal: GMS J Med Educ Date: 2022-02-15