Educational Impact of #IDJClub, a Twitter-Based Infectious Diseases Journal Club.

BACKGROUND: Journal clubs have been an enduring mainstay of medical education, and hosting these on social media platforms can expand accessibility and engagement. We describe the creation and impact of #IDJClub, an infectious diseases (ID) Twitter journal club.
METHODS: We launched #IDJClub in October 2019. Using the account @IDJClub, an ID physician leads a 1-hour open-access Twitter discussion of a recent publication. All participants use the hashtag #IDJClub. Sessions started monthly, but increased due to demand during the coronavirus disease 2019 (COVID-19) pandemic. We used Symplur 's Healthcare Hashtag project to track engagement of #IDJClub per 60-minute discussion plus the following 30 minutes to capture ongoing conversations. We also conducted an online anonymous survey using Likert scales and open-ended questions to assess educational impact.
RESULTS: In its first 20 months, 31 journal clubs were held, with medians of 42 (interquartile range [IQR], 28.5-60) participants and 312 (IQR, 205-427.5) tweets per session. 134 participants completed the survey, of whom 39% were ID physicians, 19% pharmacists, 13% ID fellows, and 10% medical residents. Most agreed or strongly agreed that #IDJClub provided clinically useful knowledge (95%), increased personal confidence in independent literature appraisal (72%), and was more educational than traditional journal clubs (72%). The format addressed several barriers to traditional journal club participation such as lack of access, subject experts, and time.
CONCLUSIONS: #IDJClub is an effective virtual journal club, providing an engaging, open-access tool for critical literature appraisal that overcomes several barriers to traditional journal club participations while fostering connectedness within the global ID community.

Journal clubs have been an important component of continuing medical education (CME) since Sir William Osler [1]. Initially created as a way for physicians to pool resources to purchase journal subscriptions, they have evolved into a means to stay current with new scientific research and to hone and maintain skills in critical appraisal of medical literature [1]. Journal clubs are ubiquitous in medical training [2]. For clinicians who leave academia, however, there are fewer opportunities for structured discussions to maintain these critical appraisal skills. Moreover, the explosion of medical literature of dubious quality during the coronavirus disease 2019 (COVID-19) pandemic—including manuscripts shared publicly prior to peer review—has highlighted the need for these skills more than ever [3]. There has thus been a need to expand access to journal clubs beyond the confines of traditional academic venues.

Virtual journal clubs have found fertile ground in Twitter (San Francisco, CA), a free, publicly accessible “microblogging” website whereby users post messages of 280 characters or fewer and interact with one another in real time [4]. Synchronous and asynchronous discussions on Twitter are made possible by the ability to index tweets using hashtags, which allow tweets to be easily discoverable by users who search for the hashtag, irrespective of whether the users have pre-existing relationships. Groups within the medical community have made use of Twitter for professional discussion and collaboration, such as #ASPChat, a long-running question-and-answer series featuring antibiotic stewardship experts from across the globe [5]. Other specialties’ virtual journal clubs have noted high rates of attendee-reported educational value and identified key advantages of the virtual format, including cross-institutional networking and global outreach [6, 7]. Herein, we describe IDJClub, the first sustained Twitter-based infectious diseases (ID) journal club, and assess the impact of this novel educational platform.

METHODS

The @IDJClub Twitter account was created in May 2019 by one of the authors (I. S. S.) who recruited 3 co-founders (N. W. C.-P., L. E. W.-C., and T. M.) 1 month later via Twitter. The other co-authors (J. B. C and B. K. T) joined as moderators over the subsequent year. The official public launch of the IDJClub virtual journal club was in October 2019. The general format includes a 1-hour synchronous live chat about a recently published ID research article led by one of the co-moderators. The chat consists of pre-prepared tweets released on a timed basis from the @IDJClub account using TweetDeck (Twitter) as well as live engagement between the co-moderators and attendees. Moderators select articles through consensus. In some cases, articles are nominated by journal club participants, or when multiple recent articles are felt to be good candidates for discussion, articles are selected by Twitter poll. Article selection and links are announced on Twitter from @IDJClub approximately 1 week prior to the event. Open-access articles are preferred; when articles are not open-access, they are shared ahead of time by e-mail with self-identified participants who lack access. Although the discussions occur synchronously, all tweets remain visible and users can review the discussion after the live chat. Journal clubs were initially hosted monthly, but frequency increased during the pandemic in response to the rapid pace of preprint and peer-reviewed publication of important manuscripts.

Metrics to track engagement for each Twitter journal club are collected using Symplur’s Healthcare Hashtags Project (Real Chemistry, San Francisco, CA; available at Symplur.com/healthcare-hashtags/). For each session, we track the number of impressions, tweets, participants, and the engagement rate (average tweets/participant) using the hashtag #IDJClub per 60-minute journal club session plus the following 30 minutes to capture ongoing discussions.

For purposes of assessing the global reach of #IDJClub, we obtained a cross-sectional snapshot of the geographic distribution of followers of the moderating account (@IDJClub) on 26 July 2021 using Tweepsmap (Toronto, ON; available at tweepsmap.com). We also conducted an online anonymous survey in May 2020 using Qualtrics (Provo, UT) to determine the demographics, occupations, practice settings, number of #IDJClub discussions attended, and level of engagement of participants. We used Likert scales and multiple-choice questions to assess perceived barriers to traditional journal club participation addressed by #IDJClub and the overall educational impact to our participants. Open-ended response questions solicited constructive feedback on ways to improve the sessions.

RESULTS

As of 7 August 2021, the moderating account (@IDJClub) had garnered 9467 followers from 114 countries (Figure 1). In its first 20 months, 31 journal clubs were held. At the time of discussion, 5 studies were preprints. One study was subsequently retracted. The study design of articles included primary analyses of phase 3 randomized controlled trials (RCTs) in 25 journal clubs (81%), phase 1 or phase 1/2 clinical trials in 2 journal clubs (6%), secondary analyses of phase 3 RCTs in 2 journal clubs (6%), and a registry study and systematic review with meta-analysis in 1 journal club each (3%). The subject of the articles was COVID-19 in 15 journal clubs (48%); orthopedic infections in 3 journal clubs (10%); cellulitis, non–COVID-19 pneumonia, tuberculosis, and bacteremia in 2 journal clubs (6%) each; and Clostridioides difficile infection, fungal infection, malaria, human immunodeficiency virus (HIV), and Staphylococcus aureus disease in 1 journal club (3%) each. Studies predominantly focused on treatment (n = 20 [65%]), followed by pharmacologic and nonpharmacologic prophylaxis (n = 5 [16%]), vaccines (n = 4 [13%]), and diagnostics (n = 1 [3%]).

Figure 1.

Geographic distribution of @IDJClub followers. Data as of 7 August 2021 (n = 9467).

Journal clubs had a median of 42 (interquartile range [IQR], 28.5–60) participants and a median of 312 (IQR, 205–427.5) tweets per session (Figure 2). The median number of tweets per participant was 6.9 (IQR, 6.3–8.1) and the median number of impressions (instances in which a tweet is presented on users’ timelines) was 1 258 000 (IQR, 786 000–1 806 000). A breakdown of participants, tweets, and impressions by article type is shown in Table 1.

Figure 2.

IDJClub engagement. Shown are the number of participants tweeting with the hashtag #IDJClub during the 60-minute chat and 30 minutes immediately thereafter. Articles discussed are annotated [8–38]. BRIEF TB, Brief Rifapentine-Isoniazid Evaluation for TB Prevention; DANCE, Duration of ANtibiotic therapy for Cellulitis; PO Vanco, per os vancomycin; PrEP, pre-exposure prophylaxis; S. aureus, Staphylococcus aureus; PCT, procalcitonin; LPVr, lopinavir-ritonavir; RDV, remdesivir; ACTT, Adaptive COVID-19 Treatment Trial; Prelim, preliminary; Ad5 vax, recombinant adenovirus type-5 vectored COVID-19 vaccine; Ph1, phase 1; RECOVERY, Randomised Evaluation of COVID-19 Therapy; Dex, dexamethasone; CRP, c-reactive protein; Rx, treatment; ChAdOx1, chimpanzee adenovirus vectored SARS-COV-2 vaccine. Ph1/2, phase ½; MERINO, Meropenem vs Piperacillin-Tazobactam for Definitive Treatment of Bloodstream Infections Due to Ceftriaxone Non-susceptible Escherichia Coli and Klebsiella Spp; 3 v 6 DFO, 3 versus 6 weeks of treatment for diabetic foot osteomyelitis; BNT162b2, BioNtech SARS-COV-2 mRNA vaccine; Ph3, phase 3; COLCORONA, Colchicine for coronavirus SARS-COV-2; IL6, interleukin-6 (antagonist); 2 ppx PJI, secondary prophylaxis for prosthetic joint infections; SAFER, Short-Course Antimicrobial Therapy for Pediatric Community-Acquired Pneumonia; Posa v Vori IA, posaconazole versus voriconazole as primary therapy for invasive aspergillosis; 4m TB Rx, 4-month tuberculosis treatment; DATIPO, Treatment of the Infections on Osteo-articular Prostheses by 6 Versus 12 Weeks of Antibiotic therapy.

Table 1.

Attributes and Engagement of Research Articles Discussed in #IDJClub

Article Attributes	No.	Participants	Tweets
Study design
Phase 3 RCT	25	41 (28.5–58)	312 (205–412.5)
Other	6	45 (26.3–54)	282.5 (201–400)
Topic
COVID-19	15	65 (43.5–70)	434 (381–483)
Other	16	30.5 (21.75–43)	240.5 (179.5–280)
Intervention
Treatment	20	46 (38–65)	354 (274–447)
Other	11	27 (21.0–46.5)	225 (178.5–333)
Population
Adults	28	46 (36.5–65)	354 (238.3–437.3)
Pediatric	3	21 (20.5–21)	225 (169–245)
Publication status
Peer reviewed	26	39.5 (27.25–48.75)	286 (183.5–397.5)
Preprint	5	69 (65–69)	447 (421–460)

All data are median (interquartile range) unless otherwise indicated. Abbreviations: COVID-19, coronavirus disease 2019; RCT, randomized controlled trial.

The survey was completed by 134 participants. The occupations of respondents are shown in Figure 3. Thirty-nine percent of respondents were ID physicians, 19% pharmacists, 13% ID fellows, and 10% medical residents. The majority of respondents (72.5%) were from the United States.

Figure 3.

Professional composition of survey respondents (N = 134). The category of pharmacy trainee includes ID pharmacy fellows (n = 2), pharmacy residents (n = 3), and pharmacy students (n = 1), while the category of “Other” includes non-ID practicing physicians (n = 6), other subspecialty fellows (n = 1), microbiologists (n = 1), microbiology students (n = 2), public health professionals (n = 1), and not otherwise specified (n = 1). Abbreviation: ID, infectious diseases.

Perceived barriers to participating in traditional in-person journal clubs that are mitigated by #IDJClub are shown in Table 2. The Twitter-based format of #IDJClub addressed several barriers, such as lack of access to traditional in-person journal clubs, lack of access to subject experts at one’s own institution, and lack of time attend traditional journal clubs.

Table 2.

Survey Respondents’ Perceived Barriers to Traditional Journal Club Participation That Were Addressed by IDJClub

Barrier to Participation in Traditional Journal Clubs	Percentage of Survey Respondents
Lack of journal club forum	44
Lack of institutional subject experts	52
Lack of time to read new research	43
Lack of time to attend journal clubs	33
Traditional journal club uninteresting	30
Traditional journal club intimidating	29

Respondents’ perceptions of the educational value of #IDJClub are summarized in Table 3. Most respondents strongly agreed or agreed that #IDJClub provided clinically useful knowledge (95%), increased personal confidence in reviewing literature (72%), and was more educational than traditional journal clubs (72%). Open-ended question responses highlighted improved access for non-physician ID specialists, insight into regional and global practice variations within the specialty, and access to colleagues with content expertise as important benefits of #IDJClub.

Table 3.

Educational Value of #IDJClub Participation, as Assessed by Survey Respondents

	Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree
I learn more from #IDJClub participation than traditional in-person journal clubs	30%	42%	22%	5%	1%
I gain clinically useful knowledge from #IDJClub participation	72%	23%	5%	0%	0%
I gain confidence in independent evaluation of literature from #IDJClub participation	33%	39%	26%	1%	1%

Data are presented as percentage of 134 survey respondents.

Respondents’ open-ended feedback for improving #IDJClub centered on a few key themes. Requests for additional journal club sessions occurring at different times to accommodate participants across time zones were common. Several participants requested regular input on the articles to be discussed in the form of a public poll, indicating that the types and topics of articles chosen were too limited. Finally, several participants thought #IDJClub could be enhanced by additions to the current format, such as a regularly published visual abstract, video presentation, or podcast.

DISCUSSION

For over a century, journal clubs have been an important and enduring part of (continuing) medical education, although their need has perhaps never been greater. The quantity of literature pertaining to ID has increased exponentially over the past century: for example, a PubMed search using the term “infection” yields 16 results from the year 1900, 108 results from 1940, 26 317 results from 1980, and 223 782 results from the year 2020. This flood of research of varying quality poses a challenge for busy clinicians hoping to stay current with emerging pathogens, therapies, and diagnostics. In addition to their function in identifying and analyzing important research articles, journal clubs provide a forum to develop and maintain proficiency in critical appraisal of research studies. In the current study, we have shown that #IDJClub, a Twitter-based journal club, is an accessible, engaging, and effective platform for critical appraisal of ID research.

#IDJClub has been enthusiastically received by the ID community, evidenced by the supportive survey responses, the moderating account’s approximately 10 000 followers, and the consistent attendance of dozens of active participants at each session. We believe this is because #IDJClub fills an important void. Nearly half of respondents to our survey lacked access to a traditional journal club, and where available, barriers to journal club participation such as limited time or access to content experts were highlighted. The transition of many journal clubs to remote video-conferencing during the COVID-19 pandemic has reduced some barriers, like allowing participation from home, but in our experience, this has added to “Zoom fatigue” and comes at the cost of decreased engagement and discussion. In contrast, #IDJClub allows participation from home, provides a low-pressure interface where one can follow discussions without feeling the need to constantly contribute, and is engaging.

In addition to being highly accessible, the survey confirmed that #IDJClub is an effective tool for teaching about ID and critical appraisal of research studies. The majority of survey respondents answered that the forum improved their confidence in independent literature appraisal and conferred clinically useful knowledge. Remarkably, nearly three-quarters of respondents answered that they learned more from IDJClub than from traditional journal club forums. This could reflect better accessibility (since nearly half of respondents did not have access to a traditional journal club), the unique attributes of the platform, or execution of the journal club.

The survey also highlighted some areas in which #IDJClub could be improved. Although moderators try to lead discussants through analysis and discussion of the article in an organized progression, the nonlinear format of Twitter-based discussions can seem cacophonous or frenetic to some participants, particularly when they are new to the platform and format of this style of journal club. To an extent, this can be a function of the number of participants. In our experience, chats with 20–40 participants can seem more manageable than those with a greater number of participants. We, and others [4], have found that the user experience during Twitter-based journal clubs is enhanced by participation via TweetDeck, a freely accessible Twitter client that enables participants to have concurrent columns and allows automatic refreshing of hashtag searches. Another limitation of IDJClub is that journal clubs are currently held only once per article and the time zone of the chat is not conducive to participation from the Eastern Hemisphere. Other Twitter-based journal clubs (eg, #NephJC, a nephrology journal club) have successfully implemented a second chat to occur at a time tailored to Eastern Hemisphere participation, and these have increased global engagement [4]. Additionally, the articles discussed in #IDJClub chats could be more diverse in study design and content. We select articles that we predict will generate the most interest, and these tend to be those that address common clinical questions and may potentially change practices. Consequently, the vast majority of journal articles discussed have been phase 3 studies evaluating interventions. Although other study designs and/or article subjects tend to draw fewer participants, we are committed to expanding the range of studies to better reflect the evidence on which ID clinicians are frequently required to make decisions. Often these involve rare diseases or questions less commonly addressed in RCTs, and thus it is important for trainees and practitioners to have the skills to appraise other article types, including observational studies and systematic reviews. Based on our experiences organizing #IDJClub, we recommend that others contemplating organizing similar virtual journal clubs consider a few key points (Table 4).

Table 4.

Tips for Creating a Virtual Journal Club

Phase	Task	Comments
Planning	Market research	• Robust community support is a prerequisite for sustainable and engaging virtual journal clubs
		• Gauge interest in the online community: consider a Twitter poll to assess whether users think there is a need and whether they would likely participate
		• Organizers do not necessarily need to have many followers but should be sufficiently engaged with an online community of peers to gauge support for this venture
	Assembling organizers	• Time demands of moderating may be difficult to juggle with other clinical and academic commitments
		• A team of co-moderators, at least 4 and ideally 6 or more, should be assembled
	Identifying a hashtag	• Search Twitter to ensure hashtag is not already in use
		• Registering hashtag with Symplur’s Healthcare Hashtag project (https://www.symplur.com/healthcare-hashtags/) allows for a user-friendly way to track engagement
	Creating a moderator account	• This does not necessarily need to be the same as the hashtag, though it can be
		• This is the account that moderators will use to announce article selection and guide discussion
Pre–journal club preparation	Selecting a journal article	• Timely articles with potentially practice-changing or dogma-challenging findings—especially about commonly encountered challenges—seem to garner the most engagement
		• Open-access journals improve accessibility, and should be prioritized
		• Articles should be announced with at least several days’ notice, and periodic reminders should be tweeted in the days ahead of the journal club
Moderating the discussion	Preparing the discussion	• Tweets can be pre-written and loaded into Twitter or Tweetdeck.com, scheduled to be published at various predetermined intervals
		• Interspersing key questions every 5–10 minutes effectively drives engagement
		• Thematic examples of questions include asking about participants’ typical practice regarding the article’s subject in the introduction section, potential weaknesses of the design in the methods, whether results were surprising or expected, and perceived clinical impact of the article in the discussion)
		• Polls can also be an engaging way to assess practices or opinions (although these cannot currently be pre-scheduled in TweetDeck)
	Moderating	• Having pre-scheduled the main tweets guiding the discussion, the moderator can focus on replying to and amplifying participants
		• Highlight salient comments from participants by retweeting (with or without comment). If possible, make an effort to amplify new participants
		• Co-moderators can help welcome participants, set the tone for the discussion (from personal accounts or from the moderating account)
	Housekeeping	• Remind participants to use the designated hashtag on all tweets to ensure tweets are visible to those following the conversation via the hashtag
		• Asking participants to use a question/answer numbering system in their responses (eg, A1 indicating an answer to question Q1) makes discussions easier to follow both during and after the journal club sessions
		• Where relevant, it may be prudent to ask participants to disclose potential conflicts of interest at the beginning of the discussion
Leveling up	Improving engagement	• Visual abstracts can be effective for promoting the discussion ahead of time, and can help remind discussants about some of the salient features of the study
		• Compiling highlights from the discussion into a Twitter “Moment” can provide a more linear summary to the chat and can be useful for individuals who missed the discussion but would like to understand key take-aways
Quality improvement	Seeking and incorporating feedback	• Organizers should seek feedback from participants about various aspects of the journal club
		• Reflecting upon engagement (eg, as measured by number of participants who use the hashtag during the chat, as measured by Symplur Healthcare Hashtags) can be helpful, but different journal clubs may value different metrics; for example, organizers may determine that number of participants is less important that the quality of the discussion, or vice versa
		• Continually consider and re-consider how your journal club can be more inclusive

In the future we plan to continue to expand and innovate in how we use #IDJClub to foster CME in the community in the following ways: (1) introduce a second chat to engage participation in other time zones; (2) harness TwitterSpaces, a new function for live audio conversations on Twitter, for debate-style sessions on challenging topics in ID with discussions guided by subject matter experts; and (3) explore ways in which to harness our content for CME (eg, through partnerships with existing podcasts, CME quality infographics, and archived and organized collections of #IDJClub discussions).

In conclusion, #IDJClub effectively leverages the accessibility, agility, and interactivity of Twitter, and provides an engaging, open-access tool for critical appraisal of ID literature and a glimpse into the incredible potential of social media for medical education.