Findings from an OMFS journal club: is COVID-19 the catalyst we have needed to embrace technology?

The COVID-19 outbreak has rapidly progressed into a worldwide pandemic, and the need for social distancing has changed the way we learn and work. Our monthly OMFS journal club has been no different, and is currently meeting on the video conferencing application Microsoft Teams. The use of a virtual setting for training in medicine and dentistry is not new and, as in the case of our recent move to a virtual medium, it may be that COVID-19 has fast-tracked this digital transformation. There are of course disadvantages to online teaching that traditional face-to-face teaching overcomes. We conducted a survey to examine how trainees' attitudes and experiences have altered with this change, and to understand whether some elements of this new style of training may be advantageous in the post-pandemic world. We aimed to assess trainees' attitudes towards online teaching, and which elements, if any, would be beneficial once face-to-face teaching becomes possible again. A survey was created for all trainees taking part in journal club meetings at Bradford Teaching Hospitals. Multiple-choice and Likert scale questions were designed to ascertain the differences in experience between online and face-to-face settings. A Wilcoxon matched pairs signed test was used to analyse the results. Responses were kept anonymous. Results showed that the majority of trainees found it easier to attend the online journal club, and also indicated that the most found Microsoft Teams easy to use, though we did not have another online application for comparison. There was no significant difference in participation comfort between the two settings, though trainees felt that the online setting considerably improved learning effectiveness. Furthermore, 79% (11/14) thought that online tutorials and meetings should replace traditional face-to-face meetings in the future. The use of internet technology such as video conferencing is not new, and although journal clubs are typically held in academic institutions, online and virtual clubs are flourishing. With an array of advantages, there is no shying away from the trend to move our teaching to a virtual medium. COVID-19 may have just provided the stimulus that has forced this transformation to accelerate.

Introduction

This paper explores the effect on training and skills acquisition in the context of a journal club within an oral and maxillofacial surgery department.

A number of studies have found that critical appraisal skills can have a positive impact on the evidence-based medicine skills of healthcare professionals.1, 2 Journal clubs are a common example of this, being described as a ‘well-recognized quality improvement strategy used by health practitioners to critique and keep up-to-date with relevant health literature.’ When critical appraisal checklists are applied, they can be an effective approach to the teaching of critical appraisal skills; a study involving Canadian paediatric surgery trainees showed a significant increase in the knowledge of evidence-based medicine following journal club meetings. In a systematic review, Deenadayalan et al found that ‘journal club activities made a significant impact on at least one outcome measure when compared with a control activity (such as lectures or general reading)’. The study did concede that there was no evidence of knowledge application, and Harris et al were also unable to conclude that journal clubs effectively supported evidence-based decision-making. Other studies have reported no significant effect on outcomes from a journal club intervention.

The traditional journal club takes place as a discussion in a small group setting led by faculty. This same system has been utilised for junior dental trainees based at Bradford Teaching Hospitals NHS Foundation Trust. As part of this, juniors take turns to present and critique papers selected by a senior trainee. Prior to the arrival of COVID-19, these meetings took place face-to-face once a month within the maxillofacial department.

However, the impact of the coronavirus has resulted in the cancellation of almost all routine operations, meaning that the day-to-day running of the department and staff timetables has been drastically affected. While the effects of this are wide-ranging, the impact on training is considerable. On top of this, the advent of ‘social distancing’ has prevented numerous meetings and conferences from taking place, with many users instead moving to video conferencing.

The monthly OMFS journal club has been no different, and currently meets using the video conferencing application Microsoft Teams. In addition to the change in setting, the meetings have increased in frequency to once a week, owing to the change in clinical activity and consequent flexibility for clinicians. We conducted a survey to examine how trainees’ attitudes and experiences have altered with this change, and to understand whether some elements of training may be advantageous in the post-pandemic world.

Methods

A structured survey was created for all dental trainees taking part in journal club meetings at Bradford Teaching Hospitals NHS Trust. The online survey platform, surveyhero.com, was used to run it. A mix of Likert scale and multiple-choice questions aimed to ascertain the differences in experience between online and face-to-face settings for the club, along with opinions regarding the setting for future meetings. A Wilcoxon matched pairs signed test was used to assess the significance of differences in the responses. Responses were kept anonymous.

Results

Results were collected from all trainees involved in the journal club, providing a response rate of 100% (n=14). Results from the survey and statistical analysis indicate that the majority of trainees found it significantly easier (p<0.05) to attend the club after it had been moved to an online setting, with only one participant indicating more difficulty in accessing it currently (Fig. 1 ).

Fig. 1

Bar chart demonstrating trainees' Likert scale responses to question about ease of attendance (0 = not at all easy, 10 = extremely easy). Wilcox analysis: Z-value -3.1074.

Trainees overall found Microsoft Teams easy to use, with the majority (11/14) giving a score of eight or more (Fig. 2 ). Furthermore, trainees generally reported that learning through the virtual journal club was more effective than it was face-to-face (Fig. 3 ).

Fig. 2

Bar chart demonstrating trainees' Likert scale responses to questions on ease of use of Microsoft Teams (0 = not at all easy, 10 = extremely easy) (n=14).

Fig. 3

Bar chart demonstrating trainees' Likert scale responses to question about the effectiveness of learning (0 = not at all effective, 10 = extremely effective).

When asked how comfortable they felt when interacting during journal club, trainees on the whole gave relatively similar answers between face-to-face and online meetings. The difference was statistically insignificant, though an accurate p-value could not be calculated because of the number of identical responses for both questions (Fig. 4 ).

Fig. 4

Bar chart demonstrating trainees' Likert scale responses to the question about level of comfort when interacting (0 = not at all comfortable, 10 = extremely comfortable).

Trainees generally found that they had more time to read over and critique papers in preparation for journal club (Fig. 5 ), with 12 of 14 trainees giving a Likert score of seven or more for the current online club. There was a statistically significant increase in Likert scale scores for the online club. Most (10/14) trainees also felt that journal club would benefit from remaining online (Fig. 6 ).

Fig. 5

Bar chart demonstrating trainees' Likert scale responses to the question about time to prepare for journal club (0 = not at all sufficient, 10 = definitely sufficient).

Fig. 6

Bar chart demonstrating responses to the question 'When normal clinical activity resumes, do you feel that journal club would benefit from remaining online?'.

When asked ‘Do you think that online tutorials and meetings should replace traditional face-to-face meetings in the future?’, 11 trainees responded with a combination of both, two said yes, and only one said no.

Discussion

Trainees found the virtual, teams-based journal club considerably easier to access than the ‘pre-COVID’ face-to-face arrangement. This may be due, in part, to the logistical difficulties in attending a meeting in a specific location at a certain time in the working day, as many trainees will be timetabled to clinical activity in settings away from the main hospital. This finding concurs with the comments of McGlacken-Byrne et al: ‘Face-to-face meetings that adhere to rigid schedules do not suit healthcare professionals who work different shift patterns or who are geographically isolated.’

The use of a virtual setting for training in medicine and dentistry is not new and has taken many forms. Early proponents made use of online discussion forums and blogs, and progressed to Twitter.12, 13 These types of internet-based forums have the potential to reach a global audience. A recent study by Jabotinsky and Sarel found that a move from face-to-face to virtual communication does not necessarily lead to a loss of information. While we did not test retention of information, the participants considered Microsoft Teams web-based learning to be significantly more effective than face-to-face.

Some studies have highlighted that a move away from a formal, hospital setting can add to the acceptance of the journal club itself, and this indeed was a characteristic of the very first journal club held outside St Bartholomew’s Hospital. One element that is likely to affect the acceptability of the club is the approach and attitude of the senior faculty or trainers. The face-to-face journal club had minimal senior consultant input whereas the virtual club has, more often than not, had consultant input. This previously was not possible, as it was held at the same time as the monthly consultants’ meeting. The change has resulted in an injection of experience and knowledge to the discussion.

Though the subject of each paper is carefully chosen by the organising senior trainee, the authors can unequivocally state that the added expertise of a consultant will only benefit the learning experience of all involved. Ahmadi et al stated that ‘faculty involvement may be a key factor to both online and moderated journal clubs.’ However, it is important to remain mindful that some authors have warned that senior clinicians might contribute disproportionately to discussions,10, 18 so maintaining a flat hierarchy when discussing papers is important. Our survey found that the trainees’ level of comfort when participating did not change significantly between face-to-face and online meetings. However, it did not consider the fact that no consultant was present at any face-to-face meeting, and this may therefore have affected the responses about online meetings.

Traditional face-to-face training allows participants to be part of a group and to share thoughts and ideas in a discussion. Although this social element can be replicated online, and a team can still collaborate just as effectively, the interaction that comes with web-based learning can result in participants feeling isolated. This has already been identified as a problem with recent social distancing and isolation guidelines as a result of the pandemic. A study of an instant-messaging-based journal club also found that some participants felt intimidated or not included. The facilitator for the online learning needs to be aware of this potential isolation, and also of the lack of non-verbal communication, and will need strategies to include participants such as asking questions and inviting comments.

Though our results found a significant improvement in the time available for trainees to prepare for the meeting, this may have had more to do with increased flexibility in the timetable due to the changes brought about by the pandemic than inherent differences between face-to-face and online settings.

Additionally, virtual journal clubs may be advantageous to those more comfortable with the use of technology in terms of access and utilising the application’s features. A successful meeting in this setting also requires each participant to have a stable internet connection, which might mean that the learning experience of those with a weaker connection may suffer. Our survey indicated that the majority of trainees found Microsoft Teams easy to use when accessing journal club (Fig. 2), though we did not compare it with another application.

Conclusion

The use of internet technology such as video conferencing is not a new phenomenon, and although journal clubs are typically held in academic institutions, online and virtual clubs are flourishing. Our findings have shown considerable advantages of online clubs over face-to-face settings, though our small sample size may pose questions over their actual significance. There is no shying away from the trend to move our teaching to a virtual medium, and it may be that COVID-19 was the catalyst that has fast-tracked this digital transformation.

Conflict of interest

We have no conflicts of interest.

Ethics statement/confirmation of patients’ permission

Not needed.