Disseminating implementation science: Describing the impact of animations shared via social media.

BACKGROUND: Dissemination is an under-researched activity that is important to researchers and funders and may have a role in creating conditions for implementation. We aim to study the impact of two animations shared via social media upon dissemination.
METHODS: We developed two short animations to increase the awareness of healthcare positional leaders of work undertaken to enhance a much-used implementation intervention. We measured both how frequently the related articles were accessed, and engagement with the research team, before and after the intervention. We analysed variation over time using statistical process control to identify both trend and periods of significant change.
RESULTS: We found evidence that the animation increased how frequently the articles were accessed, with a significant increase (p = <0.01) during the two weeks after release of each animation. One animation was associated with an increase in positional leaders' engagement with the research team.
CONCLUSIONS: Animations shared via social media can enhance dissemination. We describe lessons from the work to develop the intervention and support calls for work to increase the understanding and adoption of effective dissemination interventions. Our findings provide support for further work using randomised study designs.

Introduction

Dissemination of research helps towards meeting societal, funder and researcher goals: At the societal level, dissemination has a role in increasing research use across sectors such as health, social care, criminal justice and education, towards the goals of changing behaviour, increasing the quality of services and improving outcomes [1]. Dissemination may have this effect by addressing awareness, knowledge, perceptions, and motivation [2], leading to behaviour change [3-5]. Nilsen [6] describes a diffusion-dissemination-implementation continuum, where “dissemination is the active spread of new practices to the target audience using planned strategies” (p2). Such dissemination requires consideration of the recipients’ needs, tailoring dissemination to cultural and structural features, using appropriate style, imagery, communication channels [7] and the settings in which research findings are to be received [8].

Funders expect publicly-funded researchers to undertake activities to disseminate their research to multiple research users as part of the work to develop impact [9-11]. Funders often provide resource to support dissemination of findings, and many provide structures to transmit the findings [12, 13]. Capturing the impact of dissemination activities is important to funders [14] and provides the opportunity for researchers to monitor and improve effectiveness of dissemination activities. The number of times a paper has been accessed can be used as a measure of ‘user pull’ uptake of research [15] and provides a measure of dissemination [16, 17].

Wilson et al. [8] found that dissemination was rated as important or very important by 93% (n = 216) of respondents to their survey of principal investigators of applied and public health research. The most common reported communication channel was through academic journals (98%) and conference presentations (96%). The use of animation or social media was not included in the list of options in their 2003–8 study, which pre-dated the commonplace use of social media, although 5% referred to using ‘other’ communication channels and one respondent described the use of a DVD.

Twitter is an important social media channel for communicating research findings: 9.4% of PubMed and Web of Science papers 2010–2012 had been tweeted at least once by the end of 2012 [18]. An emailed survey of people who tweet academic articles found that 47% (n = 856) sought to communicate to the public, 43% sought to communicate to peers [19]. A further survey of clinicians found they described using social media both to get research evidence (26.9%; n = 852) and to disseminate research evidence (15.0%) [20]. This supports earlier findings that the most identified reasons for health professionals social media use were extending colleague network of colleagues, updating colleagues about work and sharing information on medical conferences with my colleagues and marketing [21].

Literature review

McGuire’s Persuasive Communication Matrix [22] proposes five dissemination variables: the channel, source, message, audience, and setting. Social media is a valuable channel for dissemination and is associated with increased downloads and citations [23, 24]. Social media enables the use of more accessible content (for example, through using more visual presentation style [24]) and can provide information through a route that is more consistent with how clinical staff access information (for example, on a smartphone rather than through articles viewed on a hospital computer [25]). Twitter is the most commonly studied social media platform in correlational studies seeking to increase the impact of health research [26] and a valuable social media channel for health research [27]. There are conflicting findings about the effect of Tweets alone (without animation) upon downloads, attention (as measured by Altmetrics) or citations [28-30]. Adding infographics or podcasts [31] or a graphic summary of a research article’s question, methods, and major findings [32] to promotional tweets may increase article accesses.

Animation shared via social media, as part of health promotion campaigns to patients and the public, have reported positive outcomes including to reduce alcohol use [33] and increase awareness of neurosurgery [34] and of COVID [35]. We have not been able to identify previous work using animation shared via social media to target healthcare professionals. Animations have, however, been used as part of educational interventions targeting healthcare workers to improve clinical assessment [36], to improve pain management [37] and to improve nurses’ response to cardiac arrest [38]. A pilot study by Attin and colleagues [38] randomised nursing students to receipt of an animation and discussion and reported faster responses to a cardiac arrest compared to a control group. A before-and-after study [39] delivered an animation to in-patients and nurses; it found that over 65 year olds fell significantly less after delivery of the animation, although the effect was not seen in patients younger than 65. Interestingly, there were no reported differences in knowledge, suggesting that this may not be the mechanism through which the animations had an effect.

There have been calls for research that extends dissemination science and practice [2], including the effectiveness of dissemination activities at achieving defined goals [7]. The effectiveness of dissemination activies might be influenced by various factors, these include the skills of those developing the dissemination materials, method of delivery and the wider context; as such, dissemination activies are complex interventions [6]. Guidance recommends that those developing complex interventions incorporate evidence, theory and stakeholder views; consider implementation and use iterative design methods [40, 41].

The current paper describes work to disseminate research findings from two studies to positional leaders in healthcare through animations shared via social media (Twitter). The source was an applied health researcher, describing work undertaken as part of a PhD to describe and enhance audit and feedback. The work was undertaken with a supervisory team, and involved substantial stakeholder involvement through co-production and advisory groups [42, 43]. The message of the first animation was a multisite description of what currently happens when a national audit reaches the hospital and how it could be enhanced through an intervention to support recipients to analyse performance, select strategies and generate commitment. The message of the second animation was a multisite description of what happens during an audit of the quality of care on wards across a hospital. Both messages were delivered through the medium of animation, a form of non-text output [44] and time-based visual artwork that illustrates a story unfolding over time [45]. The target audience for both animations was positional healthcare leaders (for example, directors, clinical audit leads). It was anticipated that they would be viewing social media in work, home or travel settings. In this paper, we describe the effect of the animation upon the frequency with which particular open-access papers [42, 43] were accessed.

Materials and methods

Aim

To study the impact of two animations shared via social media upon dissemination.

Study design and theoretical framework

This is a phase II [46] before-and-after study using statistical process control to describe the impact of an animation intervention upon the weekly number of accesses of the target articles. We conceptualise dissemination as a process, an outcome of which is whether an article was subsequently accessed. We propose that whether the article is cited or acted upon is influenced by the article content, rather than the dissemination activity. We recognise that the accessing of articles will vary over time (e.g. due to holiday periods); by monitoring the significance of variation we are able to describe the impact of the animation as an event in the dissemination process [47].

Setting and participants

This was an online study where participants were Twitter users. The source Twitter account had approximately 530 followers, with further participants able to view the tweet as a result of likes, retweets and searches. The followers were predominantly described in their biographies as healthcare improvement leaders, clinicians, clinical positional leaders, clinical academics and people interested in implementation science.

The study was approved by the Newcastle University Ethics Committee (Ref: 3917/2020).

Intervention

The animations, which are described in Box 1 and in the TIDieR checklist (S1 Checklist), are available at: https://twitter.com/Msykes09/status/1330556748210515968?s=20 and https://twitter.com/Msykes09/status/1408675290176491521?s=20.

Box 1. A description of the animation content and delivery

The source: An applied health researcher, describing work undertaken as part of a PhD to describe and enhance audit and feedback in dementia care. The work was undertaken with a supervisory team, and involved substantial stakeholder involvement through co-production and advisory groups [25, 26].

The channel: Twitter account with approximately 530 followers.

Article 1 message: A description of what currently happens when a national audit reaches the hospital and how it could be enhanced through an intervention to support recipients to analyse performance and select strategies.

Article 2 message: A description of what currently happens during a nurse-led hospital-wide audit of the quality of care in wards and steps to enhance the audit.

The audience: Positional healthcare leaders (for example, directors, clinical leads). For article 1, they were targeted through a Tweet directed to regional and national organisations leading healthcare improvement (e.g. @TheIHI @FabNHSStuff @ImprovementCym @Aqua_NHS @Improve_Academy), national improvement and regulation agencies (@HIQA @HSCQI @noca_irl @online_his) and an umbrella organisation for English hospital providers (@NHSConfed). For article 2, they were targeted through Tweets directed to national nursing leads (e.g. @CNOEngland, @CharlotteMcArdl), directors of nursing and nursing research leaders (e.g. @AlisonProf, @DrJoanne_Cooper).

The setting: It was anticipated that they would be viewing social media on a phone or tablet in work, home or travel settings.

Intervention development involved initial scoping of the content to be delivered, consideration of the message, the audience, the channel and the style, followed by iterative stakeholder engagement and refinement (Fig 1). Stakeholder engagement resulted in refinements: to increase appropriateness by extending the diversity in the images to more closely reflect the target audience; to increase accessibility by abridging the text, changing the duration of each scene and simplifying the language (e.g. changing ‘sites’ to ‘hospitals’); and to increase engagement and recipient sensemaking by amending the language and adding the learning outcomes to the beginning of the animation.

Fig 1

A flowchart describing intervention development.

The animations were developed to be engaging: They included content to support the viewer to compare the findings to their own practice, by describing current practice and asking whether this “sounds like what happens at your hospital”; They included content relating the work to priorities, by presenting benefits for patients, clinicians and positional leaders. The animations used accessible language and employed a positive, improvement-focussed tone intended to be acceptable to recipients. It was anticipated that describing the funder, university and stakeholder involvement would develop perceived credibility and trustworthiness. The animations were produced using Adobe Illustrator and After Effects.

The animations were released at different times of the year, with the link to the article in a linked tweet and the first author’s (MS) biography (Table 1).

Table 1

The source, date and content of the Tweets delivering the animations.

Animation 1 was tweeted by the first author (MS) on 17th November 2020	Here’s a short animation of our @NIHRresearch funded work to describe and enhance a national audit. The work was undertaken with @TracyLFinch @niinamk @drlouiseallan and Richard Thomson #NUPHSI. It would not have been possible without the input from a large number of stakeholders, including @afMetaLab and many others whom I am unable to name here.
Animation 1 was tweeted by MS on 22nd November 2020	Enhancing what happens when a national audit reaches the hospital. May be of interest to @THIS_Institute @TheIHI @FabNHSStuff @HIQA @HSCQI @noca_irl @online_his @ImprovementCym @Aqua_NHS @Improve_Academy @FNightingaleF @NHSConfed
Animation 1 was tweeted by MS on 23rd November 2020	Those following #CAAW20 may be interested in this. Note: Using #CAAW20 sought to bring it to the attention of people following the Clinical Audit Awareness Week
Animation 2 was tweeted by MS on tweeted on 26th June 2021	NEW animation: Describing a monthly audit of ward quality at NHS [National Health Service] hospitals. The thread continued: You can read more about what we did, what we found and what we propose here: [Link to paper]
Animation 2 was tweeted by MS on 2nd July 2021:	Directors of nursing, matrons + ward managers described ward audit data as ‘meaningless’ & ‘a sea of green’. @DavidFMelia @RivkahMiar @Antonialynch @mapFlynn @AlisonSmith2306 The animation & paper describe that the audit is costly, may have adverse effects & could be enhanced @Karen_Goudie @Day2H @lzredfernsoecno @KarenDunderdale @LeesLizzie @angelawooduk @PeteWRN
Animation 2 was tweeted by MS on 4th July 2021:	The paper and animation describe the discomforting issue of punitive feedback and its impacts upon patient care, staff wellbeing, improvement, assurance and cost.
Animation 2 was tweeted by MS on 13th July 2021:	A multi-site study using interviews, observations & doc analysis, supported by 2 groups of stakeholders. Funded by @NIHRresearch Found opportunities to improve care, staff well-being, costs & assurance. @AlisonProf @DrJoanne_Cooper @jorycroftmalone @PorteousDr @Evidence4QI

Data collection

The primary and target behavioural outcome of the animation was the number of times the paper had been accessed. This was retrieved from the target journals’ website. We captured events that may confound the results (e.g. conferences, newsletter or non-study tweets). The secondary outcome was direct requests to the first author (MS) for further information. For Article 2, we recorded how often the video was viewed each week (S1 Table).

Analysis

Statistical process control (SPC) charts describe system performance over time. We used SPC charts to describe the weekly number of accesses of the target papers before and after the animation. In line with guidance [30], we labelled the SPC with events in the dissemination process (potential confounders), for example, when the work described in article 1 was presented by the research team at an international conference, or when article 2 was highlighted in a newsletter independent of the research team.

The weekly number of accesses was analysed using an Excel-based statistical process control c-chart tool [30].

Control limits =

The trend in the first three weeks after publication demonstrated special cause variation and, in line with guidance for the use of SPC, was removed from the graph [48]. The SPC charts describe the subsequent trend for the period before and after the release of the animations. Visual analysis of the SPC charts sought temporal association between the intervention and the primary outcome (article accesses) and secondary outcome (contacts to corresponding author).

Results

Statistical process control charts are interpreted by looking for occasions when the frequency line crosses the standard deviation line. Crossing the three standard deviation line is evidence for significant difference (p = <0.01), and is referred to as special cause variation. Remaining within the standard deviation line is evidence that whilst there might be variation, this is not significantly different from prior frequency and is referred to as common cause variation.

The mean number of weekly accesses for article 1 was 25, with a significant decrease evident during the last week of August. In the two weeks after the intervention, there was a significant increase in the weekly number of accesses, as described in Fig 2. There was common cause variation (that is, no significant change) after presenting the paper at an international conference or by tweeting links to the paper without the animation during weeks 2 and 13.

Fig 2

A statistical process control c-chart showing weekly accesses of Sykes et al.

(2020) and numbers of contacts [c] made to first author by stakeholders [s.d. = Standard deviation].

The mean number of weekly accesses for article 2 was 43, with a significant decrease evident during the third week of July. In the two weeks after the intervention, there was a significant increase in the weekly number of accesses, as described in Fig 3. Article 2 also had significant increases in the number of accesses following publicity about the paper by @Evidence4QI (a project seeking to implement evidence in quality improvement projects) and after posting in the Q Community (https://q.health.org.uk/) discussion forum. Whilst study tweets were re-tweeted; we sought but did not identify novel non-study tweets.

Fig 3

A statistical process control c-chart showing weekly accesses of Sykes et al.

(2021) and numbers of contacts [c] made to first author by stakeholders.

The secondary outcome was direct requests to the first author for further information. We received seven requests for further information about article 1 during the study period. Five of the requests came during the 8 days after the animation was released (Table 2).

Table 2

A table to show when requests for further information about article 1 were received.

Date (2020)	Contact to first author
27th July	Request to speak to a deputy director of nursing about the study
13th October	Request to speak to a national clinical lead about the work
19th November	Invite to speak to a Trust-level committee
21st November	Invite to speak to a national network of clinical audit leads
	Contact from the clinical lead of a national audit
24th November	Invite to speak to a Trust-level committee
25th November	Request to use the animation in a presentation by a non-executive to a Trust-level committee

In addition, we received five requests for further information about article 2 during the study period, one of which, to speak to an improvement lead from a UK healthcare provider organisation, came in the period immediately after the release of the animation (Contact received on 26h June). The other four came later and shortly after sharing the animation via a quality improvement virtual discussion forum: National leads 4th and 10th August; Organisational leads 5th and 10th August.

Discussion

There is evidence that the animation intervention initiated the intended response in the target audience. We found that each animation was associated with a significant increase in the weekly number of accesses. Specifically, during the two weeks after the intervention there was a significant increase. Over these two weeks, there were 60 accesses above the mean for the proceeding period for article 1 and 24 accesses above the mean for the proceeding period for article 2. There is evidence that the animation 1 was temporally associated with new requests for information and discussion from the target audience.

There are strengths and limitations to the work: Consistent with McGuire’s Persuasive Communication Matrix, we identified the source, channel, message, audience and setting [22]. The animations were designed to meet a specific outcome (for viewers to be motivated to gain more information about the research described in the animation) and to meet specific dissemination outcomes (to be engaging, accessible and acceptable). The iterative, multi-method stakeholder engagement is a further strength. The overall effect of the engagement was to support the use of animations delivered via Twitter, and both to refine delivery and focus the message so that messages perceived as being more peripheral were removed. This highlights the role of stakeholder engagement in identifying the messages to be disseminated.

It is possible that the findings are not generalisable to different animations, articles, sources or audiences. Article 1 was published in an open-access journal (Implementation Science). Implementation Science is the third ranked health policy journal [49] with a 2-year impact factor of 4.525 [50]. Article 2 was published in an open-access journal (BMC Health Services Research), the 66th ranked health policy journal [49] with a 2-year impact factor of 2.655 [51]. It is possible that accesses would have been reduced if access were behind a paywall, or if published in a different journal. The animations were disseminated from a personal account. It is possible that the impact of the animations was affected by the source; future work should consider the influence of changing the source, for example, to an organisational account. Due to Twitters limited number of characters, the associated papers and the inclusion of target recipients were tweeted as a reply to the animation, thereby requiring an additional action to view the animation and to open the paper; it is possible that this acted as a barrier to accessing the article. Future work could explore the impact of targetting individuals through the inclusion of Twitter handles. We did not collect data on impressions, engagements, details expands or profile visits, and only collected views for article 2. It is possible that this would have provided a richer picture of how recipients responded to viewing the animation. Data collection was extracted manually from the journal website; like previous studies [17] we are not able to confirm the accuracy of the count. During data collection for the second animation, the counter did not change during the period 2nd to 5th July (cumulative 871 accesses) and 6th to 13th July (cumulative 882 accesses). These were the only multi-day periods when the journal counter recorded that the articles were not accessed; this hiatus was followed by a large increase (88 accesses). It is anticipated that this was an error in the data collection by the journal. Email correspondence with the journal sought, unsuccessfully, to correct this. As a result, the data analysis included an estimate for data distribution for this two week period based upon article 1. For transparency, the uncorrected data is presented in S1 Fig and S1 Table. Both the corrected and uncorrected graphs illustrate a significant increase in accesses in two of the three weeks after the release of the second article. Consistent with the guidance [48, 52], we re-calculated the mean and standard deviation after the initial period of special cause variation.

The current study focussed on usual dissemination and the social media delivery of animations targeted at positional leader knowledge users. The main outcome was for recipients to seek further information, as measured through article access. It cannot be inferred that those accessing the paper where the target audience or that they behaved differently as a result of accessing the article. However, subsequent contacts were from members of the target audience. The animations cost £508; we did not monitor time costs, but these were estimated to be 25 hours of first author time. The animation intervention was associated with a significant increase in accesses that lasted two weeks. Future work should consider how to enhance the response, for example, a multi-faceted approach may result in a more sustained increase [53]. Future research should also consider the link between increased article accessed and author contact, and impact upon target audience behaviour and patient outcomes.

There is a diverse range of dissemination methods that vary by complexity and level of interaction [54]; for example, Coon et al. [55] describe creative communication methods associated with systematic reviews, including methods for use in dissemination (illustrations, podcasts, blog posts, briefing papers, board games, social media shareable content). Video abstracts, which used the plain language summaries as the spoken script, can lead to greater comprehension, reported understanding and a positive affective response than original or graphic abstracts [56]. Our study extends this work by finding that they may also increase engagement with research articles and researchers.

There were important lessons from developing the animations. The mode of delivery impacted upon number and content of the scenes: there is a 140 second limit to embedded videos on Twitter and we found there was a minimum length of time needed to understand a scene. This constrained the number and cognitive load of scenes, including the volume of text and use of movement. In developing the animation, the role of movement was a key consideration: we sought to balance the use of movement to gain attention with the impact of movement on cognitive load that could distract from participant engagement with the message. The animation was anticipated to be viewed on tablets and mobile phones, which impacted upon the size of the font. It was anticipated that this could be viewed in work, public or home settings, as such sound needed to be optional. Providing both text and voice may have added to the cognitive load. As a result, we opted for gentle music consistent with the intended tone rather than narration. The feedback from the consultation with stakeholders highlighted opportunities to increase accessibility by improving the clarity about the research findings and addressing international differences in how audiences may view the findings.

Conclusion

We studied the impact of animations to increase the frequency with which two open-access papers [42, 43] were accessed. In designing the intervention, we drew upon McGuire’s Persuasive Communication Matrix [22] to consider the source, channel, message, audience, and setting. We found that the release of the animation was associated with a significant increase in the number of times the article was accessed. We observed that neither tweeting article links without the animation nor presentation at an international conference led to a significant increase. We propose that the use of animation distributed via Twitter may provide an effective way to disseminate research findings and increase stakeholder engagement with study findings. Further work to test the use of animation using randomised study design, and to investigate the impact upon dissemination outcomes from evidence newsletters and delivering animations via professional digital forums, would be valuable.

The TIDieR (Template for Intervention Description and Replication) Checklist*.

(DOCX)

Click here for additional data file.

The brief developed by MS, LC and JC.

(DOCX)

Click here for additional data file.

The unadjusted statistical process control c-chart showing weekly accesses of Sykes et al. (2021).

(DOCX)

Click here for additional data file.

Weekly data per article.

(DOCX)

Click here for additional data file.

22 Apr 2022

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Christopher Carroll, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following financial disclosure:

“This report is independent research arising from a Doctoral Research Fellowship (DRF-2016-09-028) supported by the National Institute for Health Research (NIHR). The views expressed in this presentation are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health and Social Care.”

Please state what role the funders took in the study.  If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

If this statement is not correct you must amend it as needed.

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

3. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

4. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please move it to the Methods section and delete it from any other section. Please ensure that your ethics statement is included in your manuscript, as the ethics statement entered into the online submission form will not be published alongside your manuscript.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for your submission, I found it most interesting. I have left various comments attached to the PDF which largely should be easy to attend. There are some early issues around formatting of references and there are a couple of areas that need rewriting for clarification. I also would have liked to have seen more data around the Tweet analytics that went beyond the publication views, so for example Tweet impressions and engagements as well as how many times the embedded videos were viewed over the same time you explored article access. Also, it is interesting to note that the publication was not Tweeted as part of the animation, but as a reply in a thread. It would be interesting to know why you took that approach and used the @ to gain attention in the animation but not as a subsequent Tweet. It is also important to note that the push for increased visibility will have potentially increased views of the paper if it was in the first Tweet rather than not include the handles of notable users, adding them to a reply. A comparison between different types of Tweets (with handles and animations as opposed to just article URL) and their impact on views would be useful for comparison and future exploration. This can be covered in the Discussion. It is a useful paper, although I do sometimes wonder whether these kinds of exercises are better served as a blog post than academic paper. That said, there is broad appeal for this kind of work and it will be of interest to those outside of your discipline.

Good luck with your amendments and going forward.

Reviewer #2: Thanks for the opportunity to review this paper. This study addresses an important question in academia with an innovative way. The research findings are valuable to the literature and the practice. To enhance this paper for publication, I would like to suggest the following revisions:

1. It would be helpful for readers if the introduction and literature review are put into two separate sections. The first three paragraphs could be the introduction and the rest paragraphs could be the literature review.

2. In the introduction, more discussion about why dissemination is important is needed. The authors touched on this in the fourth paragraph (“Dissemination activity may address antecedents of behaviour change, including awareness, knowledge, perceptions, and motivation [12].”). But more detailed discussion is preferred.

3. The literature review discussed social media as a useful tool for research dissemination. It would be better if the authors provide more examples about how social media have been used for research dissemination. There are a bunch of survey studies investigating how health researchers and professionals use social media to disseminate and learn about trending research.

4. A review of literature on using animations to enhance dissemination should be added to the literature review section. If there is little research on using animations for research dissemination, the authors could review literature about using animations to enhance dissemination of other types of messages (e.g., health messages, news reports, etc.). Also, formal hypotheses should be proposed following the literature review.

5. Moving to the method section, the details about animation development is appreciated. But it would be more clear to readers if the process is summarized and presented with a flow chart. Also, the purpose of each step and the and justification of each decision should be clarified. For example, what is the role of the co-production group and why the authors decided to make the animations "descriptive, non-critical and offer solutions"? In addition, it would be better to replace the name initials with first, second, and third author. It is somewhat confusing to understand what MS, JC, and LC refer to when reading the paragraph.

6. The list of animation tweets could be summarized in a table.

7. The discussion about the generalizability should also consider the influence of source. The animation shared by different sources (e.g., organizational vs. individual researcher, source affiliation, source expertise, etc.) may have different impact on dissemination. With the single source in this study, the generalizability of the finding is undermined.

8. As a reader, I would also like to see more discussion of using animation in comparison with other possible media formats to enhance dissemination (e.g., podcast interview, infographics).

9. The appendix numbering should be checked and fixed. The A, B, C numbering system and the 1, 2, 3 numbering system are used at the same time.

10. I would suggest deleting appendix B because the study is not a randomized control trial.

11. Some of the sentences are difficult to understand due to complicated sentence structures. Please edit the manuscript to enhance the readability.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Andy Tattersall

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: PONE-D-22-05141_reviewed.pdf

Click here for additional data file.

26 May 2022

Response to review feedback: Disseminating implementation science: Describing the impact of social-media delivered animations [PONE-D-22-05141]

Feedback 1: Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Response 1: Apologies, we have amended the manuscript in line with the formatting template.

Feedback 2. Thank you for stating the following financial disclosure:

“This report is independent research arising from a Doctoral Research Fellowship (DRF-2016-09-028) supported by the National Institute for Health Research (NIHR). The views expressed in this presentation are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health and Social Care.”

Please state what role the funders took in the study. If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

If this statement is not correct you must amend it as needed.

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

Response 2: We can confirm that the funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. (Line 349)

Feedback 3. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

Response 3: All available data is provided in the manuscript in graphical form and the new S3.

Feedback 4. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please move it to the Methods section and delete it from any other section. Please ensure that your ethics statement is included in your manuscript, as the ethics statement entered into the online submission form will not be published alongside your manuscript.

Response 4: We have added the ethics statement to the method. (Line 197)

Reviewers' comments:

Feedback a1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

________________________________________

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

Response 3: All available data is now provided in the manuscript.

________________________________________

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

Response 5: We have amended the manuscript to increase clarity, as detailed below.

________________________________________

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for your submission, I found it most interesting. I have left various comments attached to the PDF which largely should be easy to attend. There are some early issues around formatting of references and there are a couple of areas that need rewriting for clarification.

Response 1.1: We are extremely grateful to the reviewer for this approach, and have amended the document in line with their comments:

• Change to ‘animation shared via social media’ (Lines 4, 24, 34, 90, 93, 112, 129)

• Added additional keywords

• Removed e.g. from references

• Clarified that Wilson study largely pre-dates commonplace social media usage (Line 65)

• Amended sentence describing the effectiveness of dissemination activities, so as to use the suggested wording (Line 106)

• Removed speech marks where not a quote

• Added study finding the tweets alone can be effective (Line 86)

• Clarified that MS refers to the first author on first usage within the manuscript (Line 169)

• Changed description of intervention development to a flowchart (Line 157)

• Described the animation production platform (Line 167)

• Added supplemental data of animation views to the Appendix (S3)

• That tweeting the paper as a reply may have impact upon engagement (Line 265)

• We have amended the discussion to note, “We did not collect data on impressions, engagements, details expands or profile visits, and only collected views for article 2. It is possible that this would have provided a richer picture of how recipients responded to viewing the animation.” (Line 267)

• I regret that we are not able to state the time that they were tweeted, as we did not record this information

• Included details of all contacts with the first author for article 2 (Line 230).

2. I also would have liked to have seen more data around the Tweet analytics that went beyond the publication views, so for example Tweet impressions and engagements as well as how many times the embedded videos were viewed over the same time you explored article access.

Response 1.2: We agree that this could have provided greater insight into how recipients responded to the animation, and have included in the discussion that, “We did not collect data on impressions, engagements, details expands or profile visits, and only collected views for article 2. It is possible that this would have provided a richer picture of how recipients responded to viewing the animation.” (Line 268)

3. Also, it is interesting to note that the publication was not Tweeted as part of the animation, but as a reply in a thread. It would be interesting to know why you took that approach and used the @ to gain attention in the animation but not as a subsequent Tweet. It is also important to note that the push for increased visibility will have potentially increased views of the paper if it was in the first Tweet rather than not include the handles of notable users, adding them to a reply.

Response 1.3: We agree with the reviewer and have amended the discussion to state that, “Due to Twitters limited number of characters, the associated papers and the inclusion of target recipients were tweeted as a reply to the animation, thereby requiring an additional action to view the animation and to open the paper; it is possible that this acted as a barrier to accessing the article”. (Line 263)

4. A comparison between different types of Tweets (with handles and animations as opposed to just article URL) and their impact on views would be useful for comparison and future exploration. This can be covered in the Discussion.

Response 1.4: We agree and have included a statement that, “Future work could explore the impact of targeting individuals through the inclusion of Twitter handles.” (Line 267)

5. It is a useful paper, although I do sometimes wonder whether these kinds of exercises are better served as a blog post than academic paper. That said, there is broad appeal for this kind of work and it will be of interest to those outside of your discipline. Good luck with your amendments and going forward.

Response 1.5: We would like thank the reviewer for agreeing that there is a broad appeal to the work. Our view is that dissemination is an under-explored facet of implementation science, and that such peer-reviewed work will serve to ensure the quality of learning about interventions to increase reach and impact.

Reviewer #2: Thanks for the opportunity to review this paper. This study addresses an important question in academia with an innovative way. The research findings are valuable to the literature and the practice. To enhance this paper for publication, I would like to suggest the following revisions:

1. It would be helpful for readers if the introduction and literature review are put into two separate sections. The first three paragraphs could be the introduction and the rest paragraphs could be the literature review.

Response 2.1: We would like to thank the reviewer for this feedback. We have amended the introduction to reflect the two sections, whilst remaining consistent with journal format requirements. (Line 76)

2. In the introduction, more discussion about why dissemination is important is needed. The authors touched on this in the fourth paragraph (“Dissemination activity may address antecedents of behaviour change, including awareness, knowledge, perceptions, and motivation [12].”). But more detailed discussion is preferred.

Response 2.2: We agree and have extended the introduction to describe how dissemination helps towards meeting societal, funder and researcher goals. (Line 41-67)

3. The literature review discussed social media as a useful tool for research dissemination. It would be better if the authors provide more examples about how social media have been used for research dissemination. There are a bunch of survey studies investigating how health researchers and professionals use social media to disseminate and learn about trending research.

Response 2.3: We agree and have included a new paragraph describing Twitter as an important social media channel for communicating research findings. (Line 67)

4. A review of literature on using animations to enhance dissemination should be added to the literature review section. If there is little research on using animations for research dissemination, the authors could review literature about using animations to enhance dissemination of other types of messages (e.g., health messages, news reports, etc.). Also, formal hypotheses should be proposed following the literature review.

Response 2.4: We have included a brief review describing the use of animations as an educational tool and noting that we have not been able to identify studies exploring the use of animation delivered through social media targeting healthcare professionals. (Line 92)

5. Moving to the method section, the details about animation development is appreciated. But it would be more clear to readers if the process is summarized and presented with a flow chart. Also, the purpose of each step and the and justification of each decision should be clarified. For example, what is the role of the co-production group and why the authors decided to make the animations "descriptive, non-critical and offer solutions"? In addition, it would be better to replace the name initials with first, second, and third author. It is somewhat confusing to understand what MS, JC, and LC refer to when reading the paragraph.

Response 2.5: We agree and have added a flowchart illustrating intervention development. We have incorporated into the flowchart the rationale for each step and into the narrative before the flowchart the justification for decisions. I note that we tried re-writing MS, JC and LC with first second and third author, but this became repetitive and disrupted the flow. Instead, we have sought to increase clarity by using “first author (MS)”, “second author (JC)” and “third author (LC)” on the first occasion, then using initials. (Fig 1)

6. The list of animation tweets could be summarized in a table.

Response 2.6: We agree and have included them in a table. We have also tabulated the requests for further information. (Line 170, Table 1)

7. The discussion about the generalizability should also consider the influence of source. The animation shared by different sources (e.g., organizational vs. individual researcher, source affiliation, source expertise, etc.) may have different impact on dissemination. With the single source in this study, the generalizability of the finding is undermined.

Response 2.7: Whilst we think that the impact of the animation would be generalizable to similar Twitter accounts, we agree that the source is a possible influence upon generalizability and have include this as a limitation: “The animations were disseminated from the personal account. It is possible that the impact of the animations was affected by the source; future work should consider the influence of changing the source, for example, to an organisational account.” (Line 260)

8. As a reader, I would also like to see more discussion of using animation in comparison with other possible media formats to enhance dissemination (e.g., podcast interview, infographics).

Response 2.8: We did not examine alternative approaches. We have therefore summarised related findings about a range of dissemination methods, including podcasts and infographics. (Line 87, Line 295)

9. The appendix numbering should be checked and fixed. The A, B, C numbering system and the 1, 2, 3 numbering system are used at the same time.

Response 2.9: We apologise for this error and note that we have corrected the numbering.

10. I would suggest deleting appendix B because the study is not a randomized control trial.

Response 2.10: We agree and have removed the checklist

11. Some of the sentences are difficult to understand due to complicated sentence structures. Please edit the manuscript to enhance the readability.

Response 2.11: We agree and have reviewed the manuscript to improve readability.

14 Jun 2022

Disseminating implementation science: Describing the impact of animations shared via social media

PONE-D-22-05141R1

Dear My Sykes,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Christopher Carroll, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Thank you for the thorough and careful revisions completed.

Please look again at Box 1/Figure 1 - there seems to be a little confusion here (Box 1 is not a flowchart, just a list; Figure 1 is a flowchart, and represents something different) - should they both be there? Do they need separate labelling?

Reviewers' comments:

16 Jun 2022

PONE-D-22-05141R1

Disseminating implementation science: Describing the impact of animations shared via social media

Dear Dr. Sykes:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Christopher Carroll

Academic Editor

PLOS ONE