Five years of #MedRadJClub: An impact evaluation of an established twitter journal club.

INTRODUCTION: Twitter journal clubs are a relatively new adaptation of an established continuing professional development (CPD) activity within healthcare. The medical radiation science (MRS) journal club 'MedRadJClub' (MRJC) was founded in March 2015 by a group of academics, researchers and clinicians as an international forum for the discussion of peer-reviewed papers. To investigate the reach and impact of MRJC, a five-year analysis was conducted.
METHODS: Tweetchat data (number of participants, tweets and impressions) for the first five years of MRJC were extracted and chat topics organised into themes. Fifth anniversary MRJC chat tweets were analysed and examples of academic and professional outputs were collated.
RESULTS: A total of 59 chats have been held over five years with a mean of 41 participants and 483,000 impressions per hour-long synchronous chat. Ten different tweetchat themes were identified, with student engagement/preceptorship the most popular. Eight posters or oral presentations at conferences, one social media workshop and four papers have been produced. Qualitative analysis revealed five core themes relating to the perceived benefits of participation in MRJC: (1) CPD and research impact, (2) professional growth and influencing practice, (3) interdisciplinary learning and inclusion, (4) networking and social support and (5) globalisation.
CONCLUSION: MRJC is a unique, multi-professional, global community with consistent engagement. It is beneficial for both CPD, research engagement, dissemination and socialisation within the MRS community.

Introduction

The Journal Club (JC) is a long‐established professional development activity. Journal clubs provide a forum for members to meet regularly to critique research publications, develop their critical appraisal skills and improve their knowledge on specific topics.

Journal clubs began the transition to web technology in early to mid‐2000s. Widening participation and the use of online platforms to host discussions in real‐time followed. The most successful format is the Twitter JC enabling professionals to engage with participants outside their locality. This format has grown steadily over the last decade. Twitter JCs are denoted by a hashtag (a searchable phrase or acronym prefixed by the ‘#’ symbol) and are usually accompanied by a designated Twitter account. A recent analysis found over 40 Twitter JCs with two‐thirds still active as of January 2020.

During a JC session a discussion typically takes place about a pre‐selected paper guided by pre‐determined themes or questions. Formats for Twitter JCs vary from synchronous hour‐long sessions to asynchronous week‐long tweetchats and iterations in‐between with the conversation steered by the volunteer chat moderator(s). It is common to have a JC affiliated with, or organised by, an academic journal. For example, the #EBNJC centres on research articles and commentaries from Evidence‐Based Nursing. A wide range of specialisms in medicine and healthcare are represented on Twitter from geriatric medicine (#GeriMedJC), nephrology (#NephJC) and medical toxicology (#firesidetox).

Medical radiation science (MRS) professionals employ various forms of radiation in the diagnostic and therapeutic care of patients. MRS specialisations include radiography, radiation therapy, nuclear medicine, ultrasound and magnetic resonance imaging. The MRS journal club or #MedRadJClub (MRJC) was founded in March 2015 by a group of academics, researchers and clinicians as an international forum for the discussion of peer‐reviewed papers. Monthly papers are chosen from the three main international MRS journals, the Journal of Medical Radiation Sciences (Australia and New Zealand), Radiography (United Kingdom and European Union) and the Journal of Medical Imaging and Radiation Sciences (Canada).

MRJC facilitates a monthly one‐hour synchronous tweetchat inspired by a selected paper with an accompanying blog which is posted on the MRJC website (www.medradjclub.wordpress.com). The discussion is framed by several themes related to the paper and is moderated by one of the MRJC organisers. Authors of the selected paper and blog are encouraged to participate in the tweetchat. The tweetchat transcript is collated via Wakelet® (www.wakelet.com), a content curation platform, for archive and easy retrieval of the chat content. In addition to the tweetchats, a number of social in‐person events have been organised at MRS international conferences. MRJC organisers have published several papers about the journal club, presented at conferences or used tweets as data for research.

Analyses of Twitter JCs thus far have focussed on the success of the JC based on participant metrics and perceptions of benefits and drawbacks. What is less well reported in the literature is the professional and educational outcomes stemming from participation in online JCs. This paper aims to build on previous work by exploring various domains of engagement and impact in the context of a long‐standing Twitter‐based JC.

Methods

This study sought to explore the impact of MRJC across a number of domains to provide an overview of its impact in the first five years.

Analysis of audience and engagement metrics

The number of participants, the number of tweets and number of impressions for each MRJC tweetchat for the first 5 years of operation (March 2015 to February 2020) were obtained using healthcare social media analytics platform Symplur (www.symplur.com). Metrics were collated for individual chats for the chat duration and one hour post‐chat (for post chat engagement) and collated for overall analysis. MRJC blog traffic was obtained from the number of reads from the Wordpress site (www.medradjclub.wordpress.com).

The number of countries represented in the MRJC follower‐base were obtained using the Twitter analytics platform Tweetsmap (www.tweepsmap.com). Finally, the tweetchat topics since the inception of MRJC were categorised by theme and analysed for level of engagement.

Review of MRJC‐related research outputs

To explore contributions to the MRS evidence‐base stemming from MRJC, a review of research outputs (journal articles, conference presentations and posters) relating to MRJC was conducted. Research that had been facilitated by MRJC attendees or that had resulted from a MRJC tweetchat was also collated.

Qualitative analysis of 5‐year anniversary tweetchat

A MRJC tweetchat was held on 24th March 2020 to mark the fifth anniversary of the journal club. MRJC coordinators sought to celebrate and reflect on the JC’s history. Discussion points included ‘What are the pros and cons of an online club?’ and ‘What impact has MRJC had?’ The individual tweets from the chat were retrieved using the Twitter advanced search function querying ‘#MedRadJClub’ from the 23rd to the 25th March 2020. The extended time frame was selected to enable capture of the chat plus any relevant pre‐ and post‐chat tweets across time zones. Tweets relating to impact were extracted into a Microsoft Word document. The tweetchat was then manually examined via Twitter to extract additional data from replies and sub‐chats and where the hashtag had been omitted. These posts were combined with the original dataset. All tweets were organised in chronological order and anonymised prior to review. Initial inductive analysis and development of themes was carried out by one of the authors. An independent review of the dataset and preliminary themes by a second author enabled merging and refining of a number of themes. Peer debriefing led to a consensus on the overarching themes represented in the data.

Ethical considerations

Twitter account holders agree to the terms of service on account activation. This involves acknowledgement that their content and user profile are by default in the public domain and as such can be reproduced, modified and published in all media and distribution methods.

As a measure to ensure all potential participants were informed about the anniversary tweetchat data collection, the MRJC coordinators posted on the MedRadJClub website and on Twitter prior to the tweet chat stating that tweets would be captured and analysed for research purposes. The privacy of participants was protected by removing all identifiable information from the data. In line with recommendations by Fielser and Proferes, no Twitter usernames or identifiable information were disclosed and tweets from the participants were not quoted verbatim to ensure published data could not be linked to an individual’s Twitter account. In this way, the anonymity of the MRJC participants was prioritised, accepting a bearable cost to the authenticity of the quotes.

Results

Audience and engagement metrics

Twitter followers

At the current time (December 2021) MRJC has over 4500 Twitter followers from 99 countries. Over 50% of MRJC followers are located in the United Kingdom (UK). The other top four countries are:

Australia (9%).

United States (7%).

Canada (6%).

Nigeria (4%).

Participants, tweets and impressions

Since MRJC’s inception in March 2015 there has been a tweetchat each month with only one exception (June 2017) totalling 59 unique online journal club discussions. Over the 59 tweetchats MRJC has attracted an average of 41 participants, 448 tweets and 483,200 impressions per chat (Table 1).

Table 1

Summary of MRJC engagement metrics.

	Year 1	Year 2	Year 3	Year 4	Year 5	All time
	Mar 2015 –Feb 2016	Mar 2016 –Feb 2017	Mar 2017 –Feb 2018	Mar 2018 –Feb 2019	Mar 2019 –Feb 2020	Mar 2015 –Feb 2020
No of tweet chats per year	12	12	11	12	12	59 total chats
Participants per chat Mean (Min‐Max)	42 (27–56)	42 (21–71)	49 (25–92)	38 (23–82)	32 (15–49)	41 (22–70)
No. of tweets per chat Mean (Min–Max)	672 (308–907)	498 (192–747)	452 (185–734)	358 (160–677)	262 (159–382)	448 (201–689)
No. of impressions 1 Mean (Min‐Max)	610,832 (149,338–1,341,602)	420,462 (100,482–537,891)	483,472 (188,039–1,005,000)	481,315 (133,836–862,540)	420,149 (253,981–738,129)	483,246 (165,135–897,032)
No. of Blog reads per chat. Mean (Min–Max)	35 (1–126)	118 (13–757)	178 (8–485)	233 (17–1305)	170 (26–436)	147 (13–622)

Impressions: calculated by the number of tweets from each participant multiplied with the number of followers that participant currently had then summed up for all participants.

No. of impressions

Engagement per tweetchat category

There have been ten categories of tweetchat topics identified (Table 2). These tweetchat categories were analysed for level of engagement. The most popular category of chat in terms of numbers of participants was students/preceptorship. In terms of tweet totals, wellbeing was the most popular chat category (Fig. 1).

Table 2

MRJC tweetchat categories.

MRJC Tweetchat category	No. of chats	Example chat
Education/professional development	7	Factors that Influence Radiographers’ Decisions to Pursue Postgraduate Education: An Exploratory Qualitative Study (Nov 2017)
Novelty (Christmas chat)	5	The survival time of chocolates on hospital wards: An observational study (December 2015)
Patient Experience/Involvement	8	A snapshot of patients’ awareness of radiation dose and risks associated with medical imaging examinations at an Australian radiology clinic (May 2017)
Patient Safety/Quality Improvement	5	A culture of safety? An international comparison of radiation therapist error reporting (March 2015)
Productivity	4	A Two‐Pronged Trial to Eliminate No Shows in Diagnostic Imaging at a Community‐Based Hospital (April 2019)
Research Skills	6	Doctoral profile of the medical radiation sciences: a baseline for Australia and New Zealand (August 2017)
Scope of Practice	11	Assistant radiographer practitioners: Creating capacity or challenging professional boundaries (August 2018)
Students/Preceptorship	3	Understanding student radiographer attrition: Risk factors and strategies (May 2020)
Technology/Techniques	4	Assessment of lung tumour motion and volume size dependencies using various evaluation methods (June 2016)
Wellbeing	6	A cross‐sectional study of stressors and coping mechanisms used by radiation therapists and oncology nurses: resilience in cancer study (August 2015)

Figure 1

Number of participants and tweets by categories of tweetchat.

MRJC research outputs

A number of research outputs demonstrate MRJC engagement and dissemination beyond Twitter such as journal articles, poster presentations and oral presentations discussing the JC’s value and impact. Traditional research outputs such as these are summarised in Table 3.

Table 3

Summary of MRJC research outputs.

Title	Format & source
MedRadJClub social media workshop	Oral Presentation: 2016 ASMMIRT Annual Scientific Meeting of Medical Imaging and Radiation Therapy (Australia)
The Twitter journal club for Medical Radiation Professionals (#medradjclub) – Experiences from the first 12 months	Oral Presentation: 2016 ASMMIRT Annual Scientific Meeting of Medical Imaging and Radiation Therapy (Australia)
The Twitter journal club for Medical Radiation Professionals (#medradjclub): CPD/CE for the 21st century	Oral Presentation: Society of Nuclear Medicine and Molecular Imaging Conference (US) J Nucl Med. 2016,57(S2):2658
Bloom’s taxonomy of Twitter: Higher order thinking and the Twitter Journal Club	Poster: Society of Nuclear Medicine and Molecular Imaging Conference (US) J Nucl Med. 2016;57(S2):2653
Twitter Journal Club in Medical Radiation Science	Paper: J Med Imaging Radiat Sci. 2017;48(1):83‐89. https://doi.org/10.1016/j.jmir.2016.09.001
A Twitter Journal Club for Medical Radiation Professionals (#medradjclub): CPD/CE for the 21st Century	Oral Presentation: RTi3 conference (Canada) J Med Imaging Radiat Sci. 2017;48(1):S9. https://doi.org/10.1016/j.jmir.2017.02.028
Twitter journal clubs and continuing professional development: An analysis of a #MedRadJClub tweet chat	Paper: Radiography. 2018;24(1):3‐8. https://doi.org/10.1016/j.radi.2017.09.005
#MedRadJClub: A Twitter Journal Club	Oral Presentation: 2018 European Congress of Radiology (Austria)
MedRadJClub: An impact analysis of an international Twitter journal club	Poster: 2018 UK Radiological and Radiation Oncology Congress
Improving the experience of LGBTQ patients in the healthcare environment: an analysis of a Twitter chat	Poster: 2018 UK Radiological and Radiation Oncology Congress
The Education and Practice Environment for Medical Radiation Science Professionals Caring for Lesbian, Gay, Bisexual, and Transgender Patients: An Analysis of a #MedRadJClub Tweet Chat	Paper: J Med Imaging Radiat Sci. 2018;49(4):428‐435.e1. https://doi.org/10.1016/j.jmir.2018.10.006
What Medical Imaging Professionals Talk About When They Talk About Compassion	Paper: J Med Imaging Radiat Sci. 2020;51(4S):S44‐S52. https://doi.org/10.1016/j.jmir.2020.08.009

MRJC has influenced research, education and practice in a number of novel ways, such as:

To provide qualitative data for a doctoral study.

For research support for novice practitioners through a moderated exchange between established researchers and those wishing to get involved in research. In addition, the live chat was incorporated into an undergraduate MRS research course and allowed the students to learn from qualified professionals and other students in different institutions and countries.

As part of a ‘Clinical Competence and Continuous Learning’ course by a group of MRS Masters students.

To provide qualitative and quantitative data on improving the healthcare experiences of lesbian, gay, bisexual and transgender patients in diagnostic imaging and radiation therapy departments. This chat resulted in a successful collaboration with a novice researcher resulting in a conference presentation and a subsequent journal article.

A total of 366 tweets were posted during the hour‐long five‐year anniversary tweetchat in March 2020; 70 tweets relating to impact and benefits of MRJC were extracted for thematic analysis. Five core themes emerged relating to the perceived benefits of participation in MRJC: (1) CPD and research impact, (2) professional growth and influencing practice, (3) interdisciplinary learning and inclusion, (4) networking and social support and (5) globalisation (Fig. 2).

Figure 2

Themes developed from analysis of five‐year anniversary tweetchat.

CPD and research impact

Participants shared how MRJC had influenced their education and professional development endeavours. This included using MRJC as evidence of CPD and examples of MRJC discussions inspiring further research:

I have often cited involvement in #MedRadJClub as good practice. It’s a good example of how I engage in CPD

So grateful to #MedRadJClub for just being there; you made such a big difference to my #PhD journey

That idea inspired my MSc research and subsequent first publication

Professional growth and influencing practice

Some posts related to how MRJC discussions had translated into clinical practice. It was evident that knowledge sharing between peers through MRJC was creating opportunities for improving education and clinical practice.

So many good ideas have been raised from all the different ways of working. I have personally taken a lot from them in terms of my teaching and clinical practice

Taking part in #MedRadJClub made me a lot more confident. I have been able to take ideas from here and discuss with colleagues and even implemented some of them in the workplace.

Interdisciplinary learning and inclusion

The benefits of interprofessional learning were raised by the chat participants. The JC allowed for erosion of hierarchical boundaries and promoted discussion and learning between the MRS disciplines.

Geographical and professional boundaries are broken down

I love how it’s both therapeutic and diagnostic radiographers not to mention the other professions all talking together no matter the subject we can always learn from each other!

Networking and social support

Many participants offered valuable insights on the supportive community created within MRJC and some commented on connections formed through MRJC leading to collaborations offline.

The supportive nature of #MedRadJClub is superb and to see so many research ideas going forward is inspiring!

I’ve also met people and maintained contact ‘offline’ to share resources and ideas

Globalisation

MRJC members commented that participation had broadened their outlook and encouraged them to consider global variations in practice. The global reach of the JC was also seen to eliminate geographical boundaries creating a close‐knit community.

I love hearing from radiographers across the globe! It’s a small community in a big world. Sharing knowledge, problems and funny stories.

Discussion

This paper investigates various domains of engagement and impact in the context of a long‐standing Twitter‐based journal club. Results demonstrated consistency in participant engagement, evidence of global reach and growing impact beyond the typical Twitter metrics of numbers of tweets and impressions.

The five‐year average of MRJC engagement metrics examining 59 chats revealed a mean of 41 participants and 448 tweets per chat. Mean participant numbers per session vary across other well‐established Twitter‐based JCs, from 21 participants in the Paediatric Hospital Medicine tweetchat (#PHMFellowJC) to 73 joining the Journal of Hospital Medicine tweetchat (#JHMChat). , MRJC compares favourably with #urojc which reported a mean number of participants over two years as 44. One drawback of Twitter‐based JCs is the fast pace of the chat therefore in terms of numbers, it is possible that bigger isn’t always better. The MRJC organisers have noticed that there is a ‘sweet spot’ for participant numbers of about 40 to 45 people for chat moderation. More than that makes it harder to follow conversations and highlight or retweet pertinent comments. For a traditional face‐to‐face JC, five to nine participants are considered to be optimal for full involvement. It is unsure, however, what this would translate to for an online JC.

According to Roberts et al. ‘there is … no current way to measure the passive value of journal clubs, specifically relating to users who are following the journal club discussion’ In appraising MRJC’s performance, it is important to consider the proportion of passive engagement, for instance, people who follow MRJC chats without tweeting. Despite passive engagement being difficult to quantify, MRJC blog metrics demonstrate an average of 3.5 times more blog views compared to the number of active participants during the chat. This helps to illustrate MRJC’s passive audience through interaction with the blogs.

Average impressions per MRJC chat over the five‐year period were 483,000. Twitter impressions indicate the number of times the tweet has been seen, including the times the tweet appears in a followers' timeline, in a search or as a result of someone liking the tweet. Analyses of other Twitter‐based JCs revealed a variety of impressions per chat. For example, #JHMChat attracts over 2 million impressions. A more modest engagement is reflected by #PHMfellowsJC where an evaluation of the first seven months revealed an average of 205,000 impressions per chat.

MRJC chat participation was variable. Fluctuations in engagement are likely linked with the diversity of topics discussed and the relative interest to the followers. The most popular topics are wellbeing, students and preceptorship, research skills and education (Fig. 1). Topics attracting the least engagement were technology and technique and the annual novelty chats held each December where perhaps a downturn in tweets during the festive season is expected. The MRS encompasses a number of different sub‐specialities and the technology and technique chats are focused on a subset of followers which may account for lower engagement rates.

Despite fluctuations in participation, the MRJC follower count continues to rise, surpassing 4500 in December 2021. This suggests that global reach is also growing with followers in 99 different countries. The majority of followers are from the United Kingdom, which may be due to the relatively larger number of MRS professionals when compared to the other nations with participating journals (Canada, Australia and New Zealand). However, the continued growth of MRJC followers compares very favourably with a study of online JCs where the median number of followers was 374.

Apart from the monthly chats and blogs, MRJC participants and organisers have contributed to the professional knowledge base in various ways (Table 3). These aspects of MRJC’s activity help engage organisers in promoting the JC as well as provide opportunities for novel research. Research activity with broader community members provides mentorship for less experienced researchers. There were also examples given during the five‐year anniversary tweetchat of research inspired by exchanges during past chats, or insights from the chosen paper.

Lawson and Cowling state that ‘social media provides a new means of inclusive professional development’. (e78) Continuing professional development was highlighted by participants as a positive benefit of MRJC during the five‐year anniversary tweetchat. This resonates with several published evaluations of online JCs. , , ,

Direct improvements to clinical practice from JC discussions are hard to quantify. Thangasamy et al. assessed educational utility of the #urojc using Kirkpatrick’s evaluation model. The Kirkpatrick model is an educational assessment tool that evaluates programs using four levels: reaction, learning, behaviour and results. Thangasamy et al. ascertained that their JC achieved level one, namely that participants found the JC engaging, informative and helpful for fostering professional relationships. Kirkpatrick levels two and three relate to participant knowledge and behavioural change respectively. Analysis of the MRJC five‐year anniversary tweetchat demonstrated that participants felt that ideas learned from chats have improved their knowledge and have been implemented in their practice. This is suggestive of impact at levels two and three; however, further study with a methodology designed to specifically utilise the Kirkpatrick model in relation to Twitter JCs is needed before firmer conclusions may be drawn.

One major advantage of an online JC is the flattened hierarchy. MRS students regularly join the chat and several student‐centred events have been held over the past five years to encourage student participation (e.g. the annual ‘student takeover’ where undergraduates pick a paper of interest and moderate the chat). Student engagement and preceptorship is a popular chat theme and supports those supervising trainees as well as those transitioning to practice.

‘Globalisation’ of learning is another major advantage of an online JC. MRJC chat times are selected to maximise attendance across time zones and the chat is archived for those unable to attend. The elimination of hierarchical and geographical boundaries was a prominent topic during the five‐year anniversary tweetchat.

Traditional JCs offer ‘personal and professional growth and the opportunity for professional socialisation’ (p.23) Feedback from participants stressed that socialisation was also seen as a core benefit of MRJC. The MRS professions differ from medicine in that fewer practitioners are actively engaged in research, either in conducting novel research or using research evidence regularly as part of their clinical practice. MRJC offers a flexible and friendly way for those interested in research to join a welcoming group and find mentorship if needed. In this way it operates as a community of practice, a collection of people who share an interest and who come together to learn. The social aspect of MRJC allows the development of mentoring and collaborative relationships between colleagues as well as a supportive environment for novice researchers. It also provides a unique opportunity to discuss publications with their author(s), gaining a behind the scenes understanding of the research.

One potential predictor of success identified by Stoneman and Hiremath was ‘having a large team, or support (such as from a professional society or journal)’. (p.266) We would argue that in addition to a supportive team, having an engaged and encouraging membership is also an important factor.

Future directions/research

The most popular category of chat in terms of the number of participants was students/preceptorship. To further engage this important group, more chats for and about students and undergraduate education have been added to the annual calendar. This includes a popular ‘student take over’ where students choose the paper, write the accompanying blog and moderate the tweetchat.

To build on the CPD value of MRJC, the 2022 calendar will include a practical session on critiquing a paper. Making MRJC sessions easier to formally record as CPD activities using learning objectives and a reflective framework is an avenue that could be explored. This may lead to increased engagement and is an interesting facet of measurable impact.

Blogs are a rich and diverse dimension of MRJC with potential to reach a wider population outside of the Twitter platform. Since each blog is typically only promoted for limited time pre‐chat, they receive limited exposure. Making these commentaries more widely accessible and citable would be of benefit to the MRS community and provide another metric of impact.

In terms of further evaluation research, the monthly MRJC chosen papers receive focused social media attention which may lead to an increased citation rate. Quantifying this along with article page views, Altmetric scores and citation rates of chosen papers would be an important next step. Another avenue of research previously mentioned would be the development and evaluation of a bespoke methodology designed to utilise the Kirkpatrick model in relation to Twitter JCs.

Limitations

Metrics presented via Symplur can be easily skewed by individuals with high follower count and can appear as a falsely large spread. The tweets gathered from the five‐year anniversary tweetchat are from self‐selected participants.

Conclusion

Evaluation shows that MedRadJClub is a unique, multi‐professional, global community with consistent engagement. The community is supported by planned social interactions and profile raising at professional events. MRJC has contributed directly to research in various fields and facilitated research collaborations and mentorship. It has proven to be beneficial to both professional development and socialisation and is an important educational resource for the medical radiation community.

Conflict of interest

The authors have no conflicts of interest to declare and no funding was obtained for this project.