Altmetrics: Broadening Impact or Amplifying Voices?

Alarge portion of research investment across the globe is generated from public funds, and there is thus considerable effort aimed at demonstrating to the public the impact of their investment. Broader impacts, however, are not easily measured. Efforts have been made to use translation, application, and technology transfer as evidence of broader impact. Open access publishing—such as practiced by ACS Central Science—has been heralded as another potential avenue to achieve this broader dissemination. As science has moved online, so too have its indicators. Advances in digital technology have given rise to a new form of indicators—altmetrics—which are based on quantitative evidence of interactions with research objects online, primarily through social media platforms.

The goal of altmetric indicators is to turn attention from the impact of science on science (measured through citations) to the impact of science on society. Advocates imply that the altmetric audience is demographically different from the citing audience for the object. Measuring the audience on social media, however, is not quite as straightforward as measuring a citing audience. Active signals include those who demonstrate explicit interaction with the object (e.g., through tweeting and retweeting). Passive signals are indicated by those to whom the object is implicitly disseminated (e.g., subscribers to the New York Times and followers on Twitter). Following these patterns of dissemination and interaction allows scholars to investigate the diversity of the social media audience. Altmetric aggregators have provided specialized indicators to investigate these effects: for example, the platform ImpactStory will tell you how visible your work has been in the Global South, compared to other scholars on the platform; Altmetric.com provides a similar geographic outlay of the attention to a specific article. In doing so, these sources work to incentivize outreach activities in ways that citation indicators have not traditionally done.

Despite these initiatives, it has been demonstrated that altmetrics tend to replicate preexisting networks of scholarly influence and inequalities in science. Most of the data found in altmetric aggregators are retrieved from two platforms: social reference manager Mendeley and microblogging platform Twitter.[1] Mendeley is used largely by graduate students. High volume science tweeters tend to be highly educated[2] individuals from North America and Europe, who favor publications from these continents.[3] Top-followed scientists on Twitter are dominated by science popularizers, men, and those in STEM fields,[4] and tweeting networks tend to reinforce disciplinary boundaries.[5] These findings urge caution for the interpretation of altmetrics as indicators of broader impact.

However, we argue that broader impact is not a singular concept with a corresponding indicator. Broader impact can be measured in terms of increased diversity of audience, but it can also be measured in terms of the increased diversity of the research that is disseminated. Broader impacts should not be conceived only as a distinction of the audience which receives the work, but the broadening of the scientific voices which are disseminated and garner attention. Although altmetrics have been well investigated according to audience, there has been less emphasis on evaluating the authorship of the work that receives attention on social media platforms.

As a young OA journal, ACS Central Science provides a unique case study of the potential amplifications or disparities that can be revealed when using certain indicators. For example, research has demonstrated consistent disparities in terms of citation received by female-authored papers.[6] Do such disparities hold true in terms of tweets rather than citations? That is, do certain indicators favor different types of authors? And does OA mitigate or amplify these disparities? As an exploratory analysis, we compared ACS Central Science with a number of other journals: large and established chemistry journals (Journal of the American Chemical Society and RSC Advances), high-impact generalist journals (Nature and Science), and their chemistry (Nature Chemistry) and OA (Science Advances, Nature Communications, and Scientific Reports) offspring. We focus on gender, using the algorithm developed in Larivière et al. to assign a proxy for gender based on name.[6]

As noted, citation disparities by gender have been demonstrated across fields. This trend is reinforced in the citations to work published from 2015 to the present in the journals we looked at with Journal of the American Chemical Society, Nature, and RSC Advances closest to gender parity in terms of citations (Figure ). For most journals, the advantage for first and last authors is nearly always male. The strongest exceptions are ACS Central Science and Nature Chemistry, which demonstrate citation advantages for females in last-author (typically senior) positions.

Figure 1

Male/female difference in mean number of citations (left panel) and mean number of tweets (right panel), by journal, January 2015 to June 2017 papers. Numbers of papers are those to which we assigned gender of the first or last author. Citation data from Web of Science. Twitter data from Altmetric.

The same data from the perspective of altmetrics reveals different dimensions. In this case, nearly half of the journals have a female advantage in at least one of the dominant authorship roles: ACS Central Science, RSC Advances, Science Advances, and Scientific Reports. These journals all have one thing in common: they are OA journals. The only OA journal that does not follow this trend is Nature Communications. Across all the OA journals, the female advantage is stronger with first- rather than last-authored positions, suggesting that these venues provide particular visibility for younger female academics on social media.

This altmetric advantage for young female academics is more readily observed when analyzing the tweet to citation ratio of papers (Figure ). Women, who are underrepresented in terms of citation, are nearly at parity with their male colleagues in terms of the ratio between tweets and citations. For ACS Central Science and Science Advances, both fully open access journals, tweets to women’s work currently exceed citations to the same body of work. This effect is seen in last-author positions in Science Advances as well. Of course, the timeliness of altmetrics may be a factor here: ACS Central Science and Science Advances were both launched in 2015. Therefore, it may be that the altmetric advantage is one that will lessen over time. The larger journals, Journal of the American Chemical Society and RSC Advances, observe no large difference between the receipt of citations and tweets, for either gender, irrespective of the author order.

Figure 2

Tweet to citation ratio of first- (left panel) and last- (right panel) authored papers, by gender and journal, January 2015 to June 2017. Numbers of papers are those to which a gender was assigned to either first or last author. Citation data from Web of Science. Twitter data from Altmetric.

This cursory analysis is by no means comprehensive nor generalizable. However, this snapshot illuminates a critical dimension of science indicators: academic capital is not equally distributed, regardless of the metric used. Indicators favor certain types of authors, and some venues may serve as a riskier option for those seeking to generate traditional forms of academic capital.

Therefore, those who mentor and evaluate individual scholars should be critically informed about the effects of sociodemographic characteristics on indicators and the compounding effects of intersectionality, where scholars are working under multiple and intersecting dimensions of disparity.

Capturing broader voices in who gets to produce and evaluate science should be at the heart of broadening impact. The research community must begin to break open its indicators to ensure that they match the ideologies of science: values of openness, access, and broader participation. Altmetrics are likely not the panacea for our measurement woes. They are merely one proxy for attention received in the online social environment.[7] However, our exercise demonstrates that altmetrics reveal not only different patterns of impact but different patterns for different authors. We should continue to seek new, robust sources which give voice to all authors in the scientific ecosystem.