Top Altmetric Scores in the Parkinson's Disease Literature.

A new class of social web-based metrics for scholarly publications (altmetrics) has surfaced as a complement to traditional citation-based metrics. Our aim was to study and characterize those recent papers in the field of Parkinson's disease which had received the highest Altmetric Attention Scores and to compare this attention measure to the traditional metrics. The top 20 papers in our analysis covered a variety of topics, mainly new disease mechanisms, treatment options and risk factors for the development of PD. The main media sources for these high attention papers were news items and Twitter. The papers were published predominantly in high impact journals, suggesting a correlation between altmetrics and conventional metrics. One paper published in a relatively modest journal received a significant amount of attention, reflecting that public attention does not always parallel the traditional metrics. None of the most influential papers in PD, as reviewed by Ponce and Lozano (2011) made it to our list, suggesting that recent publications receive higher attention scores, and that altmetrics may omit older, seminal work in the field.

INTRODUCTION

Social media, such as Facebook and Twitter, have profoundly changed our cultural and societal landscapes, and dramatically altered the way the news is reported and how opinions are conveyed. By 2020, the number of worldwide social media users is expected to reach 2.95 billion, around a third of the Earth’s population [1]. Nearly all aspects of life have been changed by this phenomenon, politics being one example that may come to mind, but the advent of social media has also affected science.

Academics have long relied on citation-related metrics as the main indicators of a publication’s impact [2]. With the advent of social media, a new class of social web-based metrics (“altmetrics”) has surfaced as a fast, format-agnostic complement to traditional indicators, showcasing attention amongst many different stakeholder groups. There is discussion regarding altmetrics’ significance, specifically relating to how altmetrics correlate with traditional, citation-based metrics, as well as concerning its general usefulness [3, 4]. The current view is that even though there may be a correlation between altmetrics and traditional metrics, the two most likely represent different aspects of a publication’s reach [5].

Our aim was to study which recent papers in the field of Parkinson’s disease (PD) had received the highest Altmetric Attention Scores, a weighted score assigned by an altmetrics reporting service, Altmetric, and to compare this to the traditional metrics.

METHODS

Scientific articles indexed in PubMed with the MeSH term “Parkinson’s Disease” or with the keyword “Parkinson” in the title were selected and ranked according to their Altmetric Attention Score using Altmetric Explorer database (Version 2 for Institutions)—updated 29/12/2016. Altmetric Attention Scores are calculated via an automated algorithm created by the company Altmetric, representing a weighted count of the amount of attention a particular paper has received in various online sources. The score is weighted to reflect the relative reach of each type of source. A news story is worth 8 points, a tweet is worth 1 point, and a Facebook post is worth a quarter point. This is based on the assumption that news stories are likely to generate more attention that the single Facebook post. Regarding news sites, each is assigned a tier, which determines the weight of their contribution to the overall attention score meaning that newspapers with worldwide readership contribute more heavily to the score than do local news outlets. Other sources are weighted differently [6]. Articles’ traditional citations were obtained from Scopus, and the two scores were compared qualitatively.

RESULTS

Table 1 shows the 20 publications on PD with the highest Altmetric Scores. Mainstream media mentions and Twitter mentions comprised a majority of the attention that these articles received online. The three highest scoring papers in field of PD have received a considerable amount of attention from news media and Twitter, even though they are fundamentally very different articles. The first (#1) reports a new disease mechanism for PD, establishing a potentially exciting link between intestinal microbiota, motor symptoms and neuro-inflammation in a cleverly designed animal model of PD [7]. The second (#2) reports a significant increase in the incidence of PD over the last decade, which understandably raises concern among policy makers and the general public [8]. In both cases, the scientific merits and public attention went hand in hand. This was somewhat different for the third paper (#3), which describes a unique and remarkably consistent gait pattern (a unilaterally reduced arm swing on the right side, which could potentially signal the presence of early parkinsonism) in a series of five highly ranked Kremlin officials, among them Russian President Vladimir Putin and Prime Minister Medvedev. The worldwide fascination with the Russian President, the excitement of the possible implication of early parkinsonism in elite Russian politicians (as well as the alternative explanation that the absent arm swing was actually a “gunslinger’s gait”, explained by KGB or other forms of weaponry training), the compelling videos that came with the article, and the public attention given to the special edition of the British Medical Journal (where the paper was published) arguably contributed to the paper’s relatively generous amount of online attention [9]. The strict scientific merits were more modest for this paper, although it does draw attention to the presence and importance of the earliest stages of PD.

Table 1

Papers in PD literature with the highest Altmetric Attention Scores

Paper	Rank	[Ref]	Altmetric	Altmetric Score	Journal	Impact	Year of	Topic	Main	Main social	Citations	Raw
			Score	context by paper			publication		findings	media outlet		mentions
				age and Journal**
Gut microbiota regulate motor deficits and neuroinflammation in a model of Parkinson’s disease	1	[7]	926	#3	Cell	28.710	2016	Disease mechanism	Gut microbiome may influence risk of PD.	Twitter	1	654
Time trends in the incidence of Parkinson disease	2	[8]	905	#1	JAMA Neurology	8.230	2016	Epidemiology	Incidence of PD is rising.	News	1	220
“Gunslinger’s gait”: a new cause of unilaterally reduced arm swing	3	[9]	735	#8	British Medical Journal (BMJ)	19.697	2015	Semiology	Highly ranked Russian politicians exhibit reduced arm swing.	Twitter	1	553
α-Synuclein binds to TOM20 and inhibits mitochondrial protein import in Parkinson’s disease	4	[12]	588	#5	Science Translational Medicine	16.264	2016	Disease mechanism	New pathogenic mechanism in PD.	News	5	109
Fatigue in Parkinson’s disease: report from a mutidisciplinary symposium	5	[10]	490	#1	NPJ Parkinson’s Disease	NA	2016	Fatigue	Fatigue is prevalent and merits recognition by experts, policy-makers and the general public.	News	NA	72
Parkinson’s disease-related fatigue: A case definition and recommendations for clinical research	6	[11]	479	#1	Movement Disorders	6.010	2016	Fatigue	Recommendation for diagnosis/investigation PD-related fatigue.	News	NA	69
Structural brain connectome and cognitive impairment in Parkinson’s disease	7	[25]	476	#3	Radiology	6.798	2016	Neuro-imaging	Altered brain networks in MCI-PD.	News	NA	62
Cannabis (medical marijuana) treatment for motor and non-motor symptoms of Parkinson disease: an open-label observational study	8*	[15]	385	#1	Clinical Neuropharmacology	1.748	2014	Treatment	Cannabis may improve symptoms of PD.	Twitter	22	1116
Pharmacological treatment of Parkinson disease: a review	9*	[16]	385	#6	JAMA	37.684	2014	Treatment	Review of treatment options for PD.	Twitter	202	737
Midlife milk consumption and substantia nigra neuron density at death	10	[22]	373	#1	Neurology	8.166	2016	Risk factors	High milk-intake correlated with substantia nigra neuron loss.	News	2	98
Placebo effect of medication cost in Parkinson disease: A randomized double-blind study	11	[17]	338	#1	Neurology	8.166	2015	Treatment	Perception of cost influences motor function and brain activation in PD patients.	Twitter	23	185
Nilotinib effects in Parkinson’s disease and dementia with Lewy bodies	12	[18]	336	#1	Journal of Parkinson’s Disease	3.015	2016	Treatment	Nilotinib is safe and well tolerated and should be evaluated in clinical trials.	News	5	76
Aldehyde dehydrogenase variation enhances effect of pesticides associated with Parkinson disease	13	[23]	325	#3	Neurology	8.166	2014	Risk factors/Genetics	Pesticides associated with certain enzyme inhibition increase risk of PD in genetically susceptible individuals.	Facebook	33	759
Mitochondrial pyruvate carrier regulates autophagy, inflammation, and neurodegeneration in experimental models of Parkinson’s disease	14	[13]	324	#17	Science Translational Medicine	16.624	2016	Disease mechanism	Possible new therapeutic target for PD.	News	0	75
Alzheimer’s and Parkinson’s diseases: The prion concept in relation to assembled Aβ, tau, and α-synuclein	15	[14]	317	#25	Science	34.661	2015	Disease mechanism	Conceptualization of PD and Alzheimer as prion-like diseases.	Twitter	67	415
Tai chi and postural stability in patients with Parkinson’s disease	16	[19]	271	#5	NEJM	59.558	2012	Treatment	Tai chi improves balance in PD.	Twitter	233	346
Human ESC-derived dopamine neurons show similar preclinical efficacy and potency to fetal neurons when grafted in a rat model of Parkinson’s disease	17	[20]	266	#1	Cell Stem Cell	22.387	2014	Treatment	Human embryonic stem cells – derived midbrain neurons in a rat model of PD show long-term survival and functionality.	Twitter	68	137
Reports of pathological gambling, hypersexuality, and compulsive shopping associated with dopamine receptor agonist drugs	18	[26]	265	#6	JAMA Internal Medicine	14.000	2014	Neuropsychiatric aspects of PD	Dopamine agonists correlate with impulse control disorders.	Twitter	45	210
Preclinical and clinical assessment of inhaled levodopa for OFF episodes in Parkinson’s disease	19*	[21]	264	#16	Science Translational Medicine	16.624	2016	Treatment	Inhaled levodopa ameliorates OFF periods in PD.	Twitter	0	112
Identification of TMEM230 mutations in familial Parkinson’s disease	20*	[24]	264	#6	Nature Genetics	31.616	2016	Genetics	New locus for familial PD.	Twitter	77	106

*Ties in Altmetric score are broken by considering raw number of mentions. **Altmetric Attention Scores in context of paper age and journal (comparison between all papers in same journal published within the same 3-month period). Citation counts were obtained from Scopus. NA, not available.

Fatigue was the subject of two very highly ranked papers (#5, #6). Paper #5 reports on a symposium in which members of the general public participated [10]. This prompted the attention of the Movement Disorders Society with the publication of paper #6, considering fatigue as being one of the most important, albeit neglected aspect of PD [11]. Both were highly reported upon in news outlets, suggesting this is also an important issue for the general public.

The fields most consistently covered in this list are new disease mechanisms (#1, #4, #14, #15) [7, 12–14], treatment options (#8, #9, #11, #12, #16, #17, #19) [15-21] and risk factors for the development of PD (#10, #13, #20) [22-24]. These are topics that rank very highly among the concerns of PD patients and their families, because no curative treatment exists. Other subjects included MRI changes in patients with cognitive impairment associated with PD (#7) [25] and impulse-control disorders associated with dopaminergic agonists (#18) [26].

Virtually all papers were published in journals with high impact factors, indicating a potential correlation between traditional metrics and altmetrics. One exception worth mentioning is the paper in 8th position—dealing with cannabis (medical marijuana), as a possible treatment for motor and non-motor symptoms of PD—with the highest number of raw mentions by far, mainly due to Twitter. The use of medical marijuana is still the subject of lively debate in many parts of the world [27], which likely contributed to the paper’s attention in social media. It is also interesting to note that the study that was published in the New England Journal of Medicine (the journal with the highest Impact Factor in this analysis) ranked only 16th, in the Altmetric-score ranking [19], suggesting that traditional metrics and altmetrics do not measure the same construct, and indeed offer complementary perspectives.

DISCUSSION

Altmetric provides a new perspective on the attention surrounding scholarship. The scores reflect the online attention received by recent scholarly publications. Indeed, the papers included in this analysis on the subject of PD are all newly-published work: all 20 papers were published recently, with the majority having come out in the last two years. Notably, Altmetric started collecting content from publishers only during the second half of 2011 [6], and none of the articles referenced by Ponce and Lozano in their 2011 review of the most influential works in PD [28] made it to the top 20 list of altmetrics. Even though Ponce and Lozano’s paper was published in a time when altmetrics was only starting to collect content from publishers, it would be reasonable to expect that at least some of those very influential publications had received higher Altmetric Attention Scores. This finding emphasizes that altmetrics is particularly sensitive to recent news. This suggests that more recent publications may receive higher Altmetric Attention Scores, and that altmetrics may overlook seminal work previously published in the field.

Contributing to the apples-to-oranges challenges in comparing altmetrics to citations is the fact that altmetrics accrue at a much faster rate than citations: within a year of publication, it is reasonable to expect that any given paper has reached the end of its social-media-buzz cycle [29]. In order to contrast this with the much-longer lifecycle of scholarly impact, we looked at the 1,639 PD papers published in 2006 which have been cited at least ten times and found that 1,362 of them, or 83%, were still being actively cited a decade after going to press.

Additionally, papers that were very highly cited in the scientific literature—including results from important clinical trials published since 2012—have received unexpectedly low Altmetric Attention scores. One example is the paper by Okun and collaborators, which described the results from constant-current subthalamic brain stimulation (STN-DBS) in PD [30]. This paper, published in 2012 in Lancet Neurology, received a low attention score of 24, with 141 citations. Another influential treatment study from Olanow and collaborators—again published in Lancet Neurology, now in 2013—addressed the effects of intrajejunal infusion of levodopa-carbidopa intestinal gel with advanced PD [31]. Despite high citation counts (124), this publication also received a low attention score of 34. These observations highlight that Altmetric Attention Scores may not always showcase the most important articles in terms of scientific merits alone, but rather reflect a different construct. Impact factors reflect the actual use of scientific results, whereas the disposable nature of current press releases may have less value in evaluating the impact of science itself. Perhaps seminal work is not picked up by the altmetrics scores, simply because the authors paid less attention to issuing a press release, or because they were themselves less active on social media. Clearly, high altmetrics scores can be a reflection of good quality research, but as our present paper shows, marked discrepancies also exist, and altmetrics scores must be interpreted accordingly.

It is reasonable to use traditional citation-based metrics to measure the impact of a scholarly article, in the sense that citations document the fact that new evidence was built upon a given paper’s findings. Altmetrics, on the other hand, appear to reflect public interest, rather than the scientific merits of an article. There are cases, however, when word of an interesting new finding is spread online among the social-media-savvy subset of a given scientific community. In these instances, there is evidence that the volume of tweets in the first three days after publication can be predictive of future citations in the scientific literature [32]. While some investigators may be more effective than others in their ability to persuade fellow scientists to cite their work, it is likely that public relations strategies in the world of altmetrics can skew the results in a more dramatic fashion. Internet presence is highly dependent on the number of viewers and followers, and news outlets often report misleading, eye-catching information masquerading as science, which could influence web-based attention scores [33]. As such, the altmetrics really offer a new index of scientific success, complementary to traditional citation-based measures.

The majority of the papers were published in prestigious neurology journals, but we also observed a notable exception (the cannabis paper, where public attention led to high altmetric scores for a paper published in a relatively modest neurology journal). Most papers provided new insights regarding treatment, pathophysiology or risk factors for PD. Most online attention received by these articles was found in mainstream media and on Twitter.

While all of the articles considered in the analysis have been published within the last five years, there is not a level playing field when it comes to the varying amounts of time each paper has had to accrue attention. A paper from 2012 has had significantly more time to have been discovered by a broader audience than has a paper from 2016. On a similar note, a paper appearing in a highly prestigious journal receives an automatic head start in getting noticed by the general public when compared to a paper published on the same day but appearing in a more obscure journal. In order to aid with journal and published-date normalization, we have included a column in Table 1 which lists where the article in question ranks when compared to the Altmetric scores of all the other papers published in that same journal within the same three-month period. If you consider this column in aggregate, you can see that for a PD paper published in Neurology to have made our list, it must have been among the top three of its peers, while for a PD paper appearing in Science or Science Translational Medicine, being in the high teens or even mid-twenties seemed to suffice.

In the current era of “alternative facts” and “post-truths” [34], the shift from traditional metrics to altmetrics must be approached judiciously. Social media represent a uniquely powerful tool for the widespread dissemination of a message, and the work presented here suggests that scholarly publications are no exception. The business of illegitimate science is a profitable one [35], and subpar publications could theoretically end up receiving high Altmetric Attention Scores. In addition, fake accounts in Twitter and Facebook may artificially broaden a publications’ reach [36], and even generate unjustified hype regarding a diagnostic test or treatment. In our list, however, all papers came from respected publishers. However, credible organizations should remain alert for the possibility of “false papers” receiving undeserved social media attention. These considerations underscore the urgent need for the introduction of some informed and unbiased organization to filter all the social media buzz around PD, and to highlight what is credible news, and what is “fake” news or poor science. Thus, traditional metrics and altmetrics can jointly help to weigh both the evidence and the resultant buzz, and shape the ultimate interpretation. Also, regarding news outlets, while Altmetric does not attempt to understand whether or not the same news story is being reprinted by a number of news outlets, an attempt is made to account for news outlet quality and to make sure that multiple stories from the same news outlet do not falsely increase the Altmetric Attention Score for a given paper [37].

Finally, an interesting direction for future investigation would be to compare the typology of PD articles with the highest Altmetric Attention Scores to analogous sets of articles for related diseases in an attempt to determine whether disease characteristics are in any way correlated to the types of scientific findings which garner the most online attention.

DISCLOSURES

RA and BB co-authored one of the papers listed in Table 1. AAS and SK are employed by Digital Science and Altmetric LLC, the two companies behind Altmetric.com.

CONFLICT OF INTEREST

No conflict of interest.