Effectiveness of a web-based virtual journal club to promote medical education (Web-Ed): protocol of a multicentre pragmatic randomised trial.

INTRODUCTION: A journal club (JC) is a commonly used medical educational tool. Videoconferencing technology can facilitate the delivery of JCs, however, there remains no evidence on the role of web-based virtual JCs in promoting the acquisition and retention of medical knowledge. The Web-Ed trial aims to evaluate the educational benefits, feasibility and acceptability of web-based virtual JCs compared with traditional face-to-face ones. METHODS AND ANALYSIS: Web-Ed is a multicentre pragmatic parallel-group randomised trial across teaching hospitals within the UK National Health Service (NHS). We will enrol qualified doctors or medical students who are >18 years old, proficient in English and able to use online videoconferencing software. Block randomisation will be used to allocate participants in 1:1 ratio to either intervention group. Both groups will be presented with the same educational material and follow a standardised JC structure hosted by nominated moderators and medical faculty members.The primary outcome is the difference in participants' knowledge acquisition and retention 7 days after the JCs evaluated using standardised multiple-choice questions. We will report secondarily on the feasibility and acceptability of the JCs using Likert scale questionnaires. Assuming a 30% drop-out rate, we aim to enrol 75 participants to detect a 20% improvement in knowledge acquisition at 80% power and 5% significance. We will report using mean difference or risk ratio with 95% CIs and assess significance using parametric/non-parametric testing. Where relevant, we will adjust for predetermined characteristics (age, grade of training and session duration) using multivariate regression analyses. ETHICS AND DISSEMINATION: Web-Ed was designed by doctors in training to address their learning needs and evaluate the preferred mode of learning. The trial results will be published in peer-reviewed journals and presented at relevant scientific conferences. The trial has been approved by the NHS Health Regulation Authority (21/HRA/3361). TRIAL REGISTRATION NUMBER: ISRCTN18036769. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The Web-Ed trial is designed by doctors in training to evaluate the role of web-based virtual journal clubs in modern medical education.

The trial multicentre design and use of standardised interventions in both comparison groups will increase the generalisability of its findings.

The trial results will help to inform the future provision of postgraduate medical education using modern videoconferencing technology.

The trial findings might be limited by subtle variations in intervention delivery across groups and participating sites.

Introduction

Journal clubs (JCs) are commonly used in medical education as dedicated events where clinicians meet to share medical knowledge and critically appraise recent updates in the medical literature.1 JCs were first introduced by Sir James Page in London between 1835 and 1854 and Sir William Osler at McGill University in 1875 as a mean to share periodicals with physicians in training since texts were very expensive and the literature could not be easily accessed.2 3

Since then, the educational goals and format of JCs have evolved from simple dissemination of new knowledge to focus on relevant skills acquisition to encourage the application of new evidence into the practice of evidence-based medicine such as critical appraisal and evidence grading skills.4 5

Unlike other static educational tools, the JC facilitates interactive learning via structured and informal exchange among the attendees. Hosting this safe exchange medium can increase knowledge retention and the integration of knowledge into everyday clinical practice.6 While attractive, the optimal use, format and structure of a JC in modern medical education remains unknown.

Several features were proposed to increase the educational benefits of a JC such as presenting original articles, having clear objectives, nominating a facilitator, holding frequent predetermined sessions (eg, monthly), maintaining high attendance and fostering a conducive safe environment to enable interactive debate among the participants.5–9 In the UK, the majority of JCs take the format of in-person group discussions held in-house, during working hours and focused on specified content chosen by local faculty members. However, the educational benefits of this format are often limited by poor attendance and disruptions due to increasing clinical service demands on both the attendees and the faculty. Additionally, the hierarchical delivery of JC contents limits trainees’ input to address their knowledge gap and desired learning objectives.10

The use of online telecommunication and videoconferencing platforms to host and deliver educational materials is gaining momentum, especially during the COVID-19 global pandemic.11 Online education is now readily available thanks to rapid advances in videoconferencing technology with interactive features such as online polling, live questions and answers, breakout rooms and peer-to-peer sharing.6 The last couple of years saw rapid adoption of various online JC formats based on Twitter or other social media platforms.5 The uptake in web-based JCs is likely to grow as the videoconferencing technology continues to improve offering a more conducive online meeting environment. However, there remains no evidence on the role of web-based virtual JCs in promoting the acquisition and retention of knowledge in modern medical education. We designed the Web-Ed multicentre randomised trial to address this knowledge gap and provide guidance on the optimal design and delivery of educational JCs in medical training.

Aim and objectives

We aim to evaluate the effectiveness, feasibility and acceptability of a web-based virtual JC compared with face-to-face group-based JC as an educational tool among medical trainees.

Our primary objective is to assess the acquisition and retention of medical knowledge among participants in both groups. Secondarily, we will evaluate the feasibility and participants’ acceptability of the web-based virtual JC as an educational tool.

Methods and analysis

Study design

Web-Ed is a multicentre pragmatic parallel-group randomised trial. Recruitment is planned between March and October 2022 with a predicted trial end date in April 2023.

Setting

Ten accredited medical teaching hospitals within the UK National Health Service (NHS).

Participants

Qualified doctors undertaking formal clinical training in an accredited medical training programme within the NHS. Medical students who are affiliated with an accredited UK medical university and undertaking a clinical attachment within the NHS are eligible to participate in Web-Ed. Participants must be over 18 years of age at the time of enrolment, able to provide written consent, have a good command of the English language and be proficient in using simple online meeting and videoconferencing software.

Screening, consenting and enrolment

The Web-Ed trial will be coordinated by each site’s primary investigator (PI). All PIs are members of the UK Audit and Research Collaborative in Obstetrics and Gynaecology (UKARCOG) and will receive formal training in Good Clinical Practice. Local PIs will screen and identify eligible trainees within their department to enrol them to the trial after providing the electronic participant information sheet (ePIS). Trainees will be given sufficient time to consider the trial and provide consent after reading the ePIS. All participants will receive clear information on the use and storage of any personal data including policies for safeguarding information as per the European Union General Data Protection Regulation. All participants will be provided with a personal identifying digit and will be asked to complete written consent forms. Following consent, local PIs will collect the participants’ baseline information and randomise them to either intervention group.

An independent research assistant will perform block randomisation using online computer software (randomise.com) in 1:1 ratio. Concealed participant allocation will be performed using an electronic messaging service. Participants will be informed immediately of their allocated group and will be provided with electronic diary invitations and instructions to attend their allocated JC group.

Procedures

Participants in both groups will receive an electronic copy of a scientific article published in a peer-reviewed medical journal selected a priori. The selected article was chosen in consensus among the Trial Management Group (TMG) based on its relevance and suitability for the trial target cohort. The article will be sent to all participants 2 weeks before the planned JC. An additional reminder will be sent to prompt participants to read the shared article 1 week before their allocated JC date.

We aim to keep the size of each JC session limited to 5–10 participants to ensure equal participation among attendees, prompt debates and boost interactions. The duration of JC sessions in both arms of the study will also be limited to 30–40 min. Local PIs will have the flexibility to amend the date and time of the face-to-face JCs according to the local availability (eg, venue availability, participant preference) to ensure maximum participation. The face-to-face JC will be hosted at each participating site in a venue dedicated for education with facilities for electronic slide presentations. The local PIs will try to host each face-to-face JC within protected teaching time to minimise interruptions to participants on active clinical duty.

Participants allocated to the virtual JC will be provided with login details and instructions on how to operate the trial’s online video-conferencing software. The chosen software enables live video and audio streaming with slide presentation abilities, live question and answer, and live polling features. Video and camera sharing will be encouraged to promote active discussion and participation, but participants will have the option of keeping their video camera turned off if desired (table 1).

Table 1

Structure and format of the web-based journal club compared with the face-to-face journal club

Web-based journal club(Intervention)	Face-to-face journal club(Control)
Article for discussion shared electronically with attendees 2 weeks in advance	Article for discussion shared electronically with attendees 2 weeks in advance
An electronic reminder to read the highlighted article 1 week before the journal club	An electronic reminder to read the highlighted article 1 week before the journal club
Journal club hosted online using a dedicated Webinar software	Journal club hosted in a dedicated teaching room at the training unit
Mode of attendance: Virtual with video and audio sharing	Mode of attendance: Face-to-face
Hosted by a moderator and the article author	Hosted by a presenter and faculty member
Journal club structure (both groups): Introductions of moderator and faculty Presentation of a clinical problem relevant to the article presented Presentation of the article objectives, research question and methodology Presentation of the study results Discussion of the study limitations Discussion on the applicability of the study to clinical practice Questions and answers prompted by the faculty Written discussion summary points shared electronically at the end

Both JCs types will be hosted by nominated moderators who will receive training by the TMG to ensure they maintain fidelity with the trial’s procedures, published protocol, and strictly follow a JC structure as highlighted in figure 1 and table 1. Where possible, JCs will be run by the same group of moderators to minimise variations across planned JC sessions. Faculty members will consist of volunteer trainers with experience in clinical teaching and advanced knowledge in the clinic topic of chosen article. To standardise both the intervention and control all JC sessions will follow the same format outlined below in figure 1. Moderators and faculty will promote discussion and dialogue among the participants using set open ended questions based on elements of critical appraisal based on the CASP tool12 (table 1).

Figure 1

Flow chart of the Web-Ed randomised trial to evaluate the educational benefits, feasibility and acceptability of a web-based virtual journal club compared with face-to-face journal club (JC). MCQ, multiple-choice question.

Outcome measures

We will report primarily on the participants’ knowledge acquisition evaluated immediately after the JCs using standardised multiple-choice questions. The assessment will include five questions each including four true or false statements. The questions were developed by the trial TMG to test direct knowledge acquisition based on the discussed article as well as elements of critical appraisal based on the CASP tool.12 The questions were piloted among volunteers from the UKARCOG network to ensure their face and content validity as well as suitability to the trial’s target cohort. We will also evaluate knowledge retention 7 days post the JC using the same multiple-choice questions, however, with different wording to reduce memory bias and accurately evaluate knowledge retention.

To assess feasibility, we will report on the attendance rate at each JC, the number of questions asked by the participants, the duration of each JC session and the participant attrition rate from the start to the end of the session. Finally, we will report on participants’ satisfaction and acceptability of the intervention using Likert scale questionnaires. The questionnaire will assess if participants had sufficient time to attend the JC, if they found the experience easy and feasible, if the educational content was useful to them, if they are likely to attend the same JC format in the future, and if they are happy to participate in a focus group to explore their JC experience further. Each questionnaire will include five statements anchored between (strongly disagree) and (strongly agree) with an option to choose (not sure).

Data collection, handling and confidentiality

All submitted case record forms will be collected electronically using an online case submission database. Participants will receive email notifications to complete the knowledge tests and acceptability outcomes using their personal computer or smart mobile phone. Feasibility outcomes will be completed by the JC moderators directly into the trial electronic database.

All data collected during the study will be entered onto a dedicated, password-protected, electronic database using a secure computer and internet connection. Only prespecified members of the trials study team will be able to access the electronic database. Regular automated backups will be conducted to ensure a trail of data entry into the database. All participants’ data will be pseudoanonymised for the analysis stage using unique trial personal identification digits. We will not publish any data which could lead to the identification of any study participants. PIs at all sites will share the same duty to prevent unauthorised disclosure of personal information to any unauthorised body. During the study, all records collected from participants are the responsibility of the chief investigator and will be kept in secure conditions. On completion of the study, all records will be kept securely and confidentially in accordance with the Data Protection Act 1998-UK, NHS Caldicott Guardian principles, The Research Governance Framework for Health and Social Care and the Health Research Authority by the study sponsor (The Shrewsbury and Telford Hospital NHS Trust). The chief investigator has the overall responsibility to maintain and protect the participants’ anonymity throughout the study.

Quality assurance and auditing

The chief investigator will ensure the trial is conducted in compliance with the principles of the Declaration of Helsinki (1996) and the regulatory requirements including, but not limited to, the Research Governance Framework, Good Clinical Practice. Non-compliance may be captured from a variety of different sources including intervention session monitoring, data entry auditing, communications and updates. The sponsor will maintain a log of any non-compliance to ascertain if any trends are developing or escalating. The sponsor of the study will assess the non-compliances and action a time frame in which they need to be dealt with. Each action will be given a different time frame dependent on the severity of the event. If the actions are not dealt with accordingly, the sponsor will agree on appropriate action, including an on-site audit.

Sample size

Previous studies evaluating the educational value of JCs report a mean difference in knowledge acquisition scores between cohorts ranging between 8% and 25%.13–15 Assuming a 10% mean difference between groups, we need to enrol 32 participants to detect a 20% improvement in knowledge acquisition immediately after the JC. Similar numbers are expected to be sufficient to detect a meaningful mean difference for knowledge retention 1 week post-JC after allowing for a 30% drop-out rate. Therefore, we need to randomise 75 participants to achieve 80% power at 5% significance.

Statistical analysis

Data normality will be assessed using a Kolmogorov-Smirnov test. We will compare the mean difference of continuous outcomes between groups using a t-test for parametric data and a Mann-Whitney U test for non-parametric data. We will compare the distribution of categorical data between groups using a χ2 test with a significant p value set at <0.05. We will report using mean difference for continuous outcomes with 95% CIs and risk ratio for dichotomous outcomes. Where relevant we will adjust the mean difference to predetermined population characteristics (eg, age, grade of training and session duration) using multivariate linear regression analysis.

Ethics and dissemination

The trial has been approved by the Health Regulation Authority in the UK and is exempt from Research Ethics Committee approval. The trial is registered prospectively and will report any amendment of its protocol. At the conclusion of the Web-Ed trial, the steering and management committees will hold a meeting to discuss the main results and dissemination policy. The success of this study depends on the participation of trainees and local faculty members within the UKARCOG network. As such, the credit for publishing the study’s results will be dedicated to all collaborators equally. The TMG will be responsible for publishing the findings of the Web-Ed trial in a peer-reviewed journal as per the ICMJE guidelines. Open-access publication will be sought where possible to maximise impact. We will also aim to disseminate the trial’s findings via oral and poster presentations at relevant national and international conferences.

Patient and public involvement

Key stakeholders (medical trainees and trainers in the NHS) were consulted on the trial design, research objective and outcome reporting to address the highlighted training priorities. The trial findings will be communicated to all participants to scope their feedback before publication in peer-reviewed journals. Patients and members of the public were not involved in the trial design, conduct or reporting.

Discussion

Postgraduate medical education is a cornerstone of clinical training globally and it ensures clinicians can respond to the fast-changing field of medical practice.16 The medical literature is changing rapidly with daily updates and about 50% of evidence-based recommendations being reversed or updated within 5 years.17 Unlike the undergraduate setting where there is dedicated time for personal study and provision of up-to-date learning materials, the postgraduate clinical practice offers little time outside clinical duties to review up to date medical literature and implement newly available evidence. Traditionally, undergraduate training programmes make use of ‘spiral learning’ where the material is covered multiple times throughout the course to reinforce key new concepts and medical knowledge.18 This runs in contrast to the sporadic and fragmented nature of learning materials presented in a JC. As such, a structured JC format with clear learning objectives is essential to maximise the learning benefit for those attending.

The Web-Ed trial is specifically designed to test knowledge retention between in-person and online JCs which will inform future practice on how to deliver a JC. It is anticipated that the results of Web-Ed will inform future provision of postgraduate medical education.

The educational value of locally lead JCs may be limited by the lack of practitioners with adequate teaching skills, or indeed sufficient time to deliver regular JCs to a high quality. Enabling virtual delivery of JCs could facilitate centralised wide-scale participation in specialised JCs delivered by content experts to enrich the learning experience and maximise value. We hope that our findings will provide the much-needed evidence base to formalise the use of virtual teaching technology in medical education. The use of other online social media platforms may also have a role in the advertisement of online JCs as well as the dissemination of their learning outcomes to those who were unable to attend.

Several elements may limit the generalisability of the Web-Ed findings. Variations between intervention arms and across sites in content delivery may affect performance bias. We aim to minimise this risk by implementing a structured JCs format using standardised educational materials. The knowledge sharing and debate will invariably differ within and across groups depending on each group’s specific dynamics and participants background knowledge on the topic of discussion. Specifically, participants’ familiarity and affinity with the use of web-based video conferencing software may affect groups’ interactions. Still, we adopted a pragmatic design for our trial to provide evidence sought from real-life examples of the educational process in clinical settings.