Impact of Wikipedia on citation trends.

It has been suggested that the "visibility" of an article influences its citation count. More specifically, it is believed that the social media can influence article citations.Here we tested the hypothesis that inclusion of scholarly references in Wikipedia affects the citation trends. To perform this analysis, we introduced a citation "propensity" measure, which is inspired by the concept of amino acid propensity for protein secondary structures. We show that although citation counts generally increase during time, the citation "propensity" does not increase after inclusion of a reference in Wikipedia.



Citation analysis is of central importance in research evaluation (Adam, 2002[1]; Bird, 2008[3]; Bornmann et al., 2008[4]; Fava et al., 2004[13]; Garfield, 1972[14]; Gisvold, 1999[16]), in spite of its shortcomings (Sancho, 1992[26]; Seglen, 1997[27]). Moreover, almost all bibliometric measures, including impact factor (Garfield, 2006[15]; Whitehouse, 2002[28]) and h-index (Glänzel, 2006[17]; Hirsch, 2007[18]), are functions of article citations. As a result, it is also important to study factors and variables which can influence citation counts.

It is generally believed that the “visibility” of an article can influence its citation count. Lay media are previously suggested to have a role in increasing the citation counts of a scientific article (Callaham et al., 2002[6]; Phillips et al., 1991[25]). Some authors have reported that, on average, open access articles are cited more than non-open access articles (Lawrence, 2001[20]; Norris et al., 2008[24]), although this claim is disputed by others (Calver and Bradley, 2010[7]; Davis, 2011[9]; Davis et al., 2008[10]; Lansingh and Carter, 2009[19]). On the other hand, it is believed that many authors do not necessarily read the articles to cite them (Braun et al., 2010[5]; Marashi, 2005[22]), but rather adopt citations from the reference list of some subsequent papers (Braun et al., 2010[5]). All these findings suggest that citations to an article might strongly depend on the visibility, rather than the merit of the article.

With the popularity of Web 2.0 in recent years, it has been suggested that the social media can also influence scientific article citations (Bar-Ilan et al., 2012[2]; Eysenbach, 2011[12]; Li and Thelwall, 2012[21]). One of these social websites is the free online encyclopedia Wikipedia (http://en.wikipedia.org). Although scientific entries in Wikipedia are typically written by anonymous authors, many scientists may still try to find information in this website, because of its high visibility and the simplicity of the language used (Figure 1(Fig. 1)).

Figure 1

Number of publications citing Wikipedia from 2004 to 2010. The data are obtained from Scopus (http://www.scopus.com/) by searching for the word “Wikipedia” in “source title”.

Figure 1(Fig. 1) suggests that Wikipedia entries have high visibility to the authors of scholarly articles, especially in recent years. In the present work, we investigate whether the opposite is also true, i.e., inclusion of scholarly references in Wikipedia significantly affects the citation trends.

The Materials and Methods of this work are presented in the Supplementary Information.

In the first part of this study, we investigated the effect of inclusion of articles in Wikipedia on their citation counts. The inclusion time was defined as the origin of time, i.e., year zero. Citation analysis was performed on the five-year period of [-2,-1,0,+1,+2]. Normalized citation count in year i, C, was computed for each of the analyzed articles. Average C values are shown in Figure 2(Fig. 2).

Figure 2

Normalized citation count before and after inclusion in Wikipedia. The reported R2 values are related to fitting of the data to the best-fit linear model.

From this figure(Fig. 2), it is obvious that relatively high coefficients of determination (R values) in the linear regression of data are observed. In a closer look, the analyzed articles, on average, have an increasing trend in the years [-2,-1,0]. However, this increasing trend does not continue in years +1 and +2. In all three cases in Figure 2(Fig. 2), namely “Social systems”, “Chaos theory” and “Systems biology”, either in year +1 or in year +2, a decrease in the citation trend is observable. This observation may question the increasing trend of the citations.

One may argue that the observed increase in the number of citations occurs independently of inclusion of article in Wiki pedia. For example, such a trend is expected because of the inclusion of an increasing number of articles in Scopus during time. Therefore, the observed increase in Figure 1(Fig. 1) may not be more than what is expected by chance.

In order to study the citations while correcting for the intrinsically increasing citation counts, we computed the citation propensity of the articles (see the Materials and Methods section). The citation propensity in year i, P, measures the tendency of an article to be cited more than what is expected by chance. P>1 means high citation tendency in year i compared to a random year. P<1 means low citation tendency in year i compared to a random year. Finally, P=1 shows a citation tendency in year i equal to a random year. This concept is comparable to the amino acid propensity for protein secondary structures, which is the tendency of an amino acid to be included in α-helix or β-sheet structures (Chou and Fasman, 1974[8]; Marashi et al., 2007[23]).

Figure 3(Fig. 3) shows the results of the propensity analysis. In years 0 and +1, in all of the three cases, especially in case of “Systems biology”, citation propensity is almost constant and close to 1. However, in year +2 in all of the cases we observe a considerable decrease in the propensities.

Figure 3

Propensity of citations (P) before and after inclusion of the reference in Wikipedia

Figure 3(Fig. 3) suggests that the increase in the number of citations after inclusion in Wikipedia (Figure 2(Fig. 2)) can be simply due to the general increase in the number of citations during time.

In the present study, we show that inclusion of articles in Wikipedia does not increase the propensity of articles to be cited. Interestingly, the reverse is reported to be true, i.e., Wikipedia selectively lists high impact articles shortly after their publication (Evans and Krauthammer, 2011[11]).

It has been previously observed that visibility of an article, e.g., Mendeley user counts (Bar-Ilan et al., 2012[2]; Li and Thelwall, 2012[21]), bookmarks in CiteULike (Bar-Ilan et al., 2012[2]) and tweets in Twitter (Eysenbach, 2011[12]) are reported to be correlated to the citation counts of scholarly articles. Does the article “impact” increase these visibility measures, or the visibility is the cause of the increase in the scholarly citation counts of the article? We believe that this interesting question is yet to be answered.

Notes

All authors contributed equally to the manuscript.