Systematic literature search, review and dissemination methodology for the COVID-19 pandemic.

Purpose of the study: SARS-CoV-2 has caused healthcare systems globally to reorganise. A pandemic paradox emerged; while clinicians were desperate for information on a new disease, they had less time to find and evaluate the vast volume of publications at times of significant strain on healthcare systems.A multidisciplinary team undertook a weekly literature search capturing all COVID-19 publications. We also monitored free open access medical education (FOAMed) sources for emerging themes. Title and abstract screening pooled the most relevant papers for emergency medicine. Three summary types were created, a 'Top 5 Flash Update', a journal club and a rapid response to emerging FOAMed themes. From these summaries, three modes of dissemination were used: short written summaries, blogs and podcasts. These were amplified through social media. Study design: A retrospective review was conducted assessing the impact of this knowledge dissemination strategy for the period of March to September 2020.
Results: In total, 64 687 papers were identified and screened. Of the papers included in the 'Top 5', 28.3% were on epidemiology, 23.6% treatment, 16.7% diagnostics, 12% prognosis, 8.7% pathophysiology with the remaining 10.7% consisting of PPE, public health, well-being and 'other'. We published 37 blogs, 17 podcasts and 18 Top 5 Flash Updates. The blogs were read 138 343 times, the Top 5 Flash Updates 68 610 times and the podcasts had 72 501 listens.
Conclusion: A combination of traditional academic and novel social media approaches can address the pandemic paradox clinicians are facing. © Author(s) (or their employer(s)) 2021. No commercial re-use. See rights and permissions. Published by BMJ.

Since December 2019, the world has faced a new virus, SARS-CoV-2, and a new disease, COVID-19.

In 2020, 103 892 articles were published on the topic and translating the knowledge generated through scientific study to busy front-line clinicians is challenging.

A pandemic paradox has emerged where clinicians are desperate for information on a new disease, with less time than ever to absorb vital knowledge that is hidden in a vast volume of publications.

A multimedia approach that uses the power of different formats amplified through social media enabled an audience of 279 454 to be reached in an 18-week period.

FOAMed highlighted emerging themes which were popular compared with our journal club and Top 5 Flash Update summary types.

The pandemic paradox can be countered by a multimedia strategy encompassing blogs, flash updates and podcasts.

Background

SARS-CoV-2, the cause of COVID-19, emerged in China in late 2019 from a zoonotic source.1 As of January 2021, it has become a global pandemic infecting nearly 100 million people of which two million died.2

There has been a need to find answers to this new disease at a pace and scale that matches the spread of the virus itself, which in turn led to a rapid increase in the number of publications on the topic. This haste has, at times, led to publications in high-impact journals that were a lower level of evidence than usual and of lower quality than concurrently published non-COVID literature.3 This unprecedented scenario has resulted in some very high-profile retractions, which in turn has led many to question the very ethics of scientific publication.4 5

The increase in new research about COVID-19 parallels the day-to-day clinical challenge for healthcare professionals tasked with managing an acute severe illness which is not only a threat to patients but also to our colleagues and indeed ourselves. Keeping up with the unprecedented pace and scale of new publications presents a major challenge. Combining this with working in a difficult clinical environment creates the pandemic paradox. Such difficulty in managing large numbers of publications has been described as trying to ‘drink from a firehose’.6 7

Non-traditional knowledge dissemination methods have developed online in forms such as blogs and the FOAMed community. The St Emlyn’s blog and podcast are a free and open access platform that publishes topics related to emergency and critical care.8 It was started in 2012 and is rated as one of the most influential international blogs/podcasts in emergency care.9 In 2020, it was accessed over 860 000 times worldwide.

This paper describes a systematic and collaborative approach to filtering the flow of information from original sources through peer review and critical appraisal and then through to dissemination using both traditional and non-traditional methods.

Methods

In March 2020, we convened a multidisciplinary team (MDT) of emergency physicians, medical virologists and public health specialists. We aimed to help bridge the evolving science and clinical practice in relation to COVID-19 with those practicing emergency medicine (EM) as the intended audience. We initiated a surveillance programme to continually identify new published evidence, which was maintained at the same pace until the end of September 2020. To survey the emerging literature, we undertook a weekly, automated search of PubMed using prospectively agreed search terms to perform a structured systematic literature search (see box 1).

(“COVID-19”(All Fields) OR “COVID-2019”(All Fields) OR “severe acute respiratory syndrome coronavirus 2”[Supplementary Concept)] OR “severe acute respiratory syndrome coronavirus 2”(All Fields) OR "2019-nCoV”(All Fields) OR “SARS-CoV-2”(All Fields) OR "2019nCoV”(All Fields) OR ((“Wuhan”(All Fields) AND (“coronavirus”(MeSH Terms) OR “coronavirus”(All Fields))) AND (2019/12(PDAT) OR 2020(PDAT)))) OR sars-cov-19(All Fields) OR (“coronavirus”(MeSH Terms) OR “coronavirus”(All Fields)

Following a title screen and abstract review, papers considered the most relevant and informative to EM were pooled. We also undertook a specific review of high-impact and relevant journal sites including the Lancet, the New England Journal of Medicine (NEJM), the British Medical Journal (BMJ), Medical Virology and the Journal of the American Medical Association. Similarly, given the speed of publication, preprints were identified on the medRxiv website and reviewed in full when deemed relevant. Further, we distributed a call for submissions of relevant literature via a Google Form portal, which was publicised by the Royal College of Emergency Medicine (RCEM) and disseminated through social media, webinars and word of mouth. The weekly writing team initially consisted of the MDT, but this was quickly expanded to include research teams from around the UK (each team is listed and thanked in the Acknowledgements). This democratised the process and decreased the risk of bias that would be unavoidable with a single team. They screened identified papers based on (1) relevance to EM practice, (2) level of evidence and (3) methodological quality. Any conflicts were resolved by consensus; if this was not possible, then SC cast the deciding vote (figure 1). Of those papers selected, they were given a categorical impact rating which was agreed by consensus: worth a peek, head turner and game changer. ‘Game changer’ was a paper that could potentially warrant a change in clinical practice. ‘Head turner’ papers were directly relevant but lacking in specific areas that preclude them from directly changing practice. ‘Worth a peek’ papers were indirectly relevant to an aspect of EM practice but contained potentially useful insights. The MDT retained editorial oversight and peer-reviewed the weekly writing team’s submission.

Figure 1

Flowchart of the search, selection and dissemination strategy from the core activities. FOAMed, free open access medical education; MDT, multidisciplinary team; RCEM, Royal College of Emergency Medicine.

The MDT also responded rapidly to developing topics within the pandemic. These were identified by monitoring for themes within the FOAMed community, including social media and other FOAMed websites. We/they provided an in-depth analysis of large trials, for example, the Randomised Evaluation of COVID-19 therapy trial (RECOVERY)10, and emerging clinical phenomena such as the association of venous thrombosis and COVID-19.11 This also provided experiential information from those involved in the early phase of the pandemic. This experiential information was used as a way of sharing ideas and good practice and was aimed at those yet to experience an influx of patients.

To present the most relevant literature, we generated three weekly outputs: the RCEM Top 5, journal club and rapid responses (see figure 1). These were disseminated through written summaries, podcasts and blogs. The ‘Top 5 Flash Update’ was created with the aim of giving succinct written summaries and an informal review of the papers. This was disseminated by RCEM to all members and shared widely on social media. We conducted a journal club and disseminated it as a podcast and blog. This was an in-depth review of papers with expert discussion and an audience question and answer. It was a joint venture between the University of Manchester, RCEM and the St Emlyn’s blog and podcast. Rapid responses were curated into blogs and podcasts.

We used non-traditional means of dissemination. This was initially through synchronous and asynchronous Twitter and Facebook posts linked to the St Emlyn’s blog and podcast. This reach was further amplified through retweets and likes from members of the FOAMed community. We collated data on the reach and impact of the process using the following methods :

Email conversion rate (click-throughs) from the Top 5 Flash Update was obtained from RCEM records.

Page views of COVID-19 specific articles from the St Emlyn’s blog were obtained from the WordPress platform.

Audience for the St Emlyn’s podcast were obtained from the Podbean platform.

Results

Results are based on the period from 1 March 2020 and 1 September 2020. During this period, 64 687 papers were identified using the aforementioned search terms (figure 2). These were screened by title, abstract and, if appropriate, reviewed in depth.

Figure 2

Publication count by week from 1 January 2020, the beginning of the outbreak, according to our search criteria from PubMed.

There were a total of 279 454 views and listeners across all mediums; by medium, this was 138 343 for blogs, 72 501 for podcast and 68 610 for written summaries (exclusively the Top 5 Flash Update). By source, there were 180 867 for rapid responses and 18 314 for journal club (table 1).

Table 1

Engagement stratified by the different modes and summary type

Mode of dissemination	Total views	Summary type	Total views
Emailed summaries	68 610	Top 5 Flash Update	68 610
Blog	138 343	Journal Club	18 314
Podcast	72 501	Rapid responses	180 867

The Top 5 Flash Update was the only type to use written summaries and therefore the numbers are equal. Journal Club and Rapid Responses were both disseminated in blog and podcast forms

In this period, the Top 5 Flash Update ran for 18 weeks and included 90 papers. The full list of top five bulletins can be downloaded from the RCEM website here. Of the included studies. 28.3% were on the topic of epidemiology, 23.6% treatment, 16.7% diagnostics, 12% prognosis, 8.7% pathophysiology and the remaining 10.7% consisting of PPE, Public Health, well-being and other. This dissemination was read 68 610 times by clinicians over the 18 weeks, averaging 4036 readers a week, which would represent 41.0% of the RCEM’s membership.

A total of 37 blog posts were published on St Emlyn’s during the study period. Twenty-three were rapid responses to new emerging themes; 14 of the blogs were related to the journal club. One of the themes was the early experience of EM clinicians in the outbreak in northern Italy, and an interview with an EM physician, Dr Robert Cosentini from Papa Giovanni XXIII Hospital in Bergamo, Italy, was conducted and received 30 098 listens. Table 2 shows the geographical distribution and unique users from the top 10 countries accessing the St Emlyn’s blog during the study period.

Table 2

Unique users accessing the St Emlyn’s website by country (blog and podcast)

Rank	Country	Users
1	UK	63 472
2	US	50 124
3	Australia	17 420
4	India	7497
5	Canada	6325
6	South Africa	3773
7	Germany	2351
8	Ireland	2628
9	Italy	2100
10	New Zealand	1885
–	Other	39 312
Total		196 887

Seventeen podcasts were published on the St Emlyn’s site during the study period. These were downloaded 72 501 times, with 7 covering the journal club and 10 covering rapidly emerging topics. Of these rapid response topics, the interview with an Italian EM clinician was an outlier accounting for 41.5% of all downloads.

Discussion

This paper describes the broad international reach of a multimodal and multimedia approach to evidence collation, appraisal and dissemination that straddles traditional academic and novel social media platforms. The data demonstrate users across a wide geographical distribution who engaged with the evidence-based summaries.

It is commonly stated that the time taken for evidence to translate from publication through to clinical practice is measured in years.12 This delay may not apply during a novel pandemic when there is little established practice and all knowledge is to a degree ‘new’. Paradoxically, the absence of prior knowledge about COVID-19 led to an explosion in the number of papers published, many of which were of questionable scientific merit.13 It is clearly ineffective for every individual, especially those tied up in clinical work, to seek, sort, critique and summarise this body of emerging evidence. There is a logical argument for this task to be centralised and organised into a systematic approach to knowledge translation with outputs presented in an easily accessible format. Our data presented demonstrate that this can be achieved using a combination of techniques that span traditional information dissemination methods such as email, combined with social media platforms, blogs and podcasts.

We have also demonstrated the appetite for variety in the type of media from the audio-only podcast to the short ‘bite size’ Top 5 Flash Update. Interestingly, one of the most popular resources deployed was the rapid response experiential narrative of an Italian EM physician. While expert opinion does represent a lower level of evidence, it adds value when there is a paucity of evidence. This insight appears to have been sought by physicians, perhaps in part because the Italians were hit early in the global pandemic.14

Throughout the pandemic, other traditional publishing groups (such as BMJ, Lancet and NEJM) and #FOAMed sites have collated and curated COVID-19 information, but few have adopted a formalised multimodal collaborative approach that targets specific clinical groups (in this case emergency care clinicians) as described in this paper. This multimodal approach to knowledge dissemination reflects how we now live our lives through technology, and we believe this is a significant factor in the success of the project.

Limitations

In this paper, we have focused on data obtained from blogs and podcasts on the St Emlyn’s and RCEM platforms, which are where the information was accessible to clinicians. However, a significant degree of sharing/dissemination took place through other social media platforms such as Facebook and Twitter. As we did not create a specific hashtag for these evidence-based updates at the beginning of this project, we have been unable to track the dissemination through these and other online platforms.

While the search terms used were broad and consistent, the study selection was not according to the rigorous systematic review methodology. This was deliberate, as its purpose was to filter large volumes of evolving literature for research likely to have the most impact and relevance for those working in EM trying to remain up to date with the COVID-19 pandemic.

We were unable to determine how much cross-over existed between platforms. The St Emlyn’s and RCEM sites undoubtedly have a cross-over of users, but the degree to which this occurs is unknown. However, the broad geographical reach of the St Emlyn’s site does not match the geographical distribution of members and fellows of RCEM.

Conclusion

A clinician-led approach incorporating both traditional and social media-based platforms allowed the rapid dissemination of research to clinicians in EM. During a period of rapidly changing research information, this became an important and widely received source of information translation.