The impact of Twitter promotion on future citation rates: The #TweetTheJournal study.

Social media (SoMe), including Twitter, are emerging as important marketing tools and have become indispensable for many organizations. SoMe offers a different approach for promotional purposes, e.g. of corporate products, but also for dissemination of science-related information from the academic world. Twitter has been found to be the favorite SoMe platform for cardiologists [1], [2]. However, only 10–20% of articles listed in PubMed are tweeted at least once, suggesting a persistently low utilization of Twitter [2] with a clear need for validation of the causal relationship between promotion of scientific work on Twitter and online visibility (i.e. citation scores) for cardiovascular medicine journals [3].

In a retrospective analysis, we captured the association between SoMe interactions and the number of artifacts of the IJC Heart & Vasculature between 2014 and 2020. SoMe interactions included retweets, likes and comments about individual artifacts published in IJC Heart & Vasculature. Artifacts refer to any research output which is accessible online. In the case of IJC Heart & Vasculature, metrics for three artifacts were gathered via PlumX Analytics: articles, reviews and corrections. Of note, up until 2019, IJC Heart and Vasculature did not have a dedicated Twitter handle. Therefore, in the years before 2018, the Tweets related to IJC Heart and Vasculature artifacts were made by third parties only. In 2019, promotion of artifacts on Twitter by the handle @IJC_Heart_Vasc was initiated.

Fig. 1 depicts the number of SoMe interactions (blue) per year associated with individual manuscripts (artifacts) published in IJC Heart & Vasculature. The number of artifacts (gray) per year is displayed simultaneously (in total 624 artifacts). SoMe interactions increased in quantity from 2014 to 2020 independent of the annual number of artifacts, which fluctuate substantially over the same time period, suggesting that IJC Heart & Vasculature becomes increasingly recognized on SoMe every year, which ultimately increases its online visibility. Although Twitter is the main SoMe medium propagating knowledge in the cardiovascular environment [2], [3], we have also noticed some activity on Facebook during the same observation period (data not shown). Additionally, the increasing number of SoMe interactions was associated with a continuous increase in CiteScore, an alternative journal evaluation metric to the generally used Web of Science impact factors, to 2.8 in 2019 (Scopus Preview: https://www.scopus.com/sourceid/21100394792#tabs=2) [4].

Fig. 1

Number of published artifacts published in the IJC Heart & Vasculature Journal (gray) and Social Media interactions on Twitter (black) per year (https://plu.mx/ijc-heart-and-vasculature/analytics/g?report=subgroups_by_metric) [30].

Promoting a manuscript on Twitter may impact its future citation rate. Well-established journals have tested their SoMe strategy in a number of randomized studies [3], [5], [6], [7]. In the field of cardiovascular medicine, an association between Twitter promotion and an increased online visibility and number of citations was shown in a randomized trial of ESC journal family articles [3]. Similar results are expected in a prospective, randomized trial including a SoMe strategy through the Thoracic Surgery Social Media Network (TSSMN), where publications are highlighted in collaborative effort of leading journals in cardiothoracic surgery [6]. A moderate correlation of SoMe attention with the numbers of citations for cardiovascular research articles, published in the eight highest Web of Science impact factor journals, was demonstrated in a retrospective study [8]. However, the positive impact of SoMe promotion on citation rates is not a consistent finding. A randomized trial on articles receiving SoMe exposure from Circulation showed no increase on the article page view [5], [9]. Also studies outside the cardiovascular field could not confirm a significant effect of SoMe exposure on traditional metrics as article citation rates [7], [10]. or demonstrated only very weak associations [11], [12], [13]. Therefore, the impact of tweets on citation performance remains unclear.

One important consideration for the design of a SoMe promotion study is the selection of an optimal and reliable primary outcome. Citation counts or formulae based upon citation counts are widely used to assess the scholarly impact of a manuscript, but it can take several years for a typical article to generate a sufficient number of citations to consider them as having a long-term impact. In response to the need for early estimates of long-term impact, a range of faster impact indicators has been proposed [14], [15]. One of the most dissiminated and popular of this new metrics is the reference manager Mendeley [16], [14], [17]. The count of Mendeley readers appear earlier than citations and have moderate to strong long-term correlations with the number of citations in most fields [12], [18], [19], [20], [21].

Within the herein proposed #TweetTheJournal study, we will investigate the impact of SoMe promotion on early indicators of long-term impact of manuscripts in IJC Heart & Vasculature. The methodology follows a randomized controlled trial design, in which out of three subsequent issues of IJC Heart & Vasculature, published every two months between April 2021 and August 2021, a total of 90 artifacts are randomized into (1) Twitter promoted artifacts and (2) non-Twitter promoted control group artifacts. The used tweeting schedule is derived from previous studies [3], [5], [7], [9]. Following each Journal release, 15 artifacts will be randomized into the intervention arm and tweeted within the first week (Monday to Friday) of the publication. After 2, 4 and 6 weeks a repost (including #ICYMI “In Case You Missed It”) [9] will follow up to increase visibility and dissemination, until the release of the next journal issue [9], [22]. Tweets will be posted at different times of the day (e.g. 11 am, 3 pm, 8 pm CET) to overcome bias of daytime and increase visibility of tweets [9], [22]. Weekends will be excluded. Articles with a press release or presentation at a scientific session will be excluded to intervene possible bias due to a generated higher attention, as implemented in prior studies [3].

The Twitter promotion intervention is performed via the @IJC_Heart_Vasc Twitter handle. Every tweet will be retweeted by the @ELS_Cardiology account (27,7K followers, 08.03.2021).

The layout of the respective tweets is supported by implementing appropriate Hashtags, number of characters and a link (shortened via www.bitly.com) to the given artifact with an open-access to the manuscript. As previous research has shown a significant impact on the dissemination and view of tweeted artifacts including a visual abstract or central figure [23], we included this in all artifacts in the intervention arm. The precise layout derives from findings of the Healthcare Hashtag Project () [24], knowledge gathered from SoMe engagement tools (https://buffer.com/) [25] and suggestions of previous research [3], [5], [23].

Non-Twitter promotion refers to non-active tweeting from the selected Twitter channel and held to pierce whether tweeting can be aligned with an increase in the Mendeley reader count at one year, which is a well-established early impact indicator (primary endpoint). The analysis of the secondary endpoints will include citations, SoMe interactions (retweets, likes) and Twitter impressions after one year (April 2022-August 2022) in both groups obtained through PlumX Metrics (https://plumanalytics.com/learn/about-artifacts/) [26]. The approximate same period between online publication and collection of Mendeley readers counts and citations in the artifacts (1 year period) should overcome bias caused by different time windows for accumulation of Mendeley readers and citations, as mentioned in previous studies on SoMe impact on citations throughout different research fields [12], [14], [18], [27].

In the past years, the online availability and therefore wide dissemination of scientific articles (Open Access Websites, SoMe) might have lead to a decrease of the expressiveness of quality in traditional metrics, as citations and also the Journal Impact Factor [28]. Therefore, new indicators for the quality and impact of a Journal, as alternative metrics like Mendeley reader counts, SoMe interactions, are rising up [28]. With the herein described study, we expect to get further insights into the value of these new journal metrics. Since IJC Heart & Vasculature is a very recently founded journal, the relative impact of tweets on its visibility in the academic landscape is likely to be greater and therefore, the effect of tweets on the citation rate is expected to be stronger than performing a similar study with a well-established journal, which may get attention via several options. The growing usage of Twitter in cardiovascular medicine, particularly in times of rising online presence due to the Covid-19 pandemic, may offer a new future strategy to improve the spread and visibility of very recent research insights [29].