Bibliometric trends in ophthalmology 1997-2009.

AIMS: To track citation patterns in ophthalmic journals and contrast them with major medical and surgical journals from 1997 to 2009. In addition, we want to familiarize the ophthalmic community with bibliometrics indices.
MATERIALS AND METHODS: Data retrieved from Institute for Scientific Information and related websites include 2-year journal impact factor JIF, 5-year impact, Eigenfactor score, H-factor, Article Influence score, and SCImago factor.
RESULTS: JIF rose steadily around 10% annually in ophthalmic journals, and likewise for major medical and surgical journals. JIF correlated with recent bibliometric indicators like 5-year impact, H index, and SCImago factor but not with Eigenfactor. Ophthalmic journals publishing reviews, basic science, or large volume on broad range of topics ranked at the top for JIF, while subspecialty journals tended to have low JIF. JIF of subspecialty journal Retina rose from 0.740 (rank 23) in 2000 to 3.088 in 2007 (rank 6).
CONCLUSIONS: JIF tends to rise annually by 10% in medical, surgical, and ophthalmic fields. Journals publishing reviews, basic science, or large volume on broad range of topics rank at the top for JIF. The rapid rise of JIF for Retina unlike other subspecialties that stayed status quo is multifactorial: Change in editorial policies (introduction of review articles and omission of case reports) and technological advances in the retinal field.

Publications across the world appear in thousands of peer-reviewed scientific journals. Citations are currently the currency of science,[12] and the performance of an article is measured by the number of citations; likewise, a journal is currently assessed by a Journal Impact Factor (JIF) which is a quality assessment instrument monitored yearly by Thomson Scientific (formerly International Scientific Institute-ISI, Philadelphia, USA, founded by Eugene Garfield[2]). Thomson Scientific calculates scientometric data of more than 9,000 journals provided by Journal Citations Reports (JCR) through ISI Web of Science. Additional bibliometric indices were recently introduced.[3]

The top-ranked journals in biology and surgery have impact factor scores around 10 times that for top-ranked journals in ophthalmology.[4567891011] The average ophthalmology journal appears to fare like the average internal medicine or surgery in terms of JIF and aggregate cited half-life [Table 1]. Prior bibliometric analyses in ophthalmology have observed a rise in JIF in some of the ophthalmic literature;[456789101112] hence the present paper aims at elucidating these trends and analyze temporal patterns from 1997 to 2009. Also our aim is not to propose a new scientometric index but rather to familiarize ophthalmologists with current bibliometric indices.

Table 1

2009 bibliometric profile of ophthalmology, surgery and internal medicine journals

Materials and Methods

This is a bibliometric analysis of papers published in select major ophthalmic journals from 1997 to 2009, using Journal Citation Reports with JIF calculated by taking the number of citations to articles published by the journal in the previous two years and dividing this by the total number of articles published by the journal during those same years. The total number of yearly publications includes research articles, original case reports, technical notes, and reviews. These constitute the citable items of a journal. commentaries, editorials, correspondence, letters to editors are noncitable and were not incorporated in the total number of publications. JIF ranking was done from a total of 49 visual sciences journals that have a Science Citation Index.

Because of shortcomings of JIF,[123] we added new bibliometric characteristics available over the past 3 years and provided by Thompson Scientific: Five-year impact, Article Influence, and Eigenfactor. Five-Year Impact Factor gives a broader range of citation activity for a more informative snapshot over time where citation activity continues to rise through several years. Eigenfactor Metrics, comprising the Eigenfactor Score (http://eigenfactor.org/) and Article Influence Score, are designed to reflect the prestige and citation influence of journals by considering scholarly literature as a network of journal-to-journal relationships. Article Influence Score is derived from Eigenfactor Score and conceptually similar to JIF in that there is a numerator as well as a denominator (i.e. number of citable papers) except that it uses Eigenfactor Score (rather than the total number of citations) as the numerator. Thus, dissimilar to JIF where all citations are counted equally regardless of their source, in Article Influence Score, each citation is multiplied by the “quality” of the citing journals, resulting in greater weights for citations that come from highly cited journals, and less weight to poorly cited journals.

Other web sites have provided newly popularized bibliometric measures such as the Scientific Journal Rankings (SJR) indicator with H index, provided by the SCImago Journal and Country Rank web site (http://www.scimagojr.com), and developed by the SCImago research group. SJR is a novel instrument by Scopus that provides unrestricted (open) access, is based on a larger source journal database (87 journals in ophthalmology for Scopus versus 49 for Thompson Scientific; hence Scopus finds more cites per paper), and focuses on the quality of citations that a journal receives by other journals, rather than the absolute number. Scopus also introduced the H index proposed by Hirsch.[3] The SJR indicator of a specific journal for a 3-calendar-year period is calculated through an iteration process that computes the “prestige” gained by the journal through the transfer of prestige from all the other journals included in the network of journals, by their citations during the past 3 years to all articles of the specific journal published in the past 3 years, divided by the total number of articles of the specific journal during the 3-year period in regard. The amount of prestige of each journal transferred to another journal in the network is computed by considering the percentage of citations of the former journal that are directed to articles of the latter journal.

Additional indices were included such as immediacy index which is the average number of times an article is cited in the year it is published.

Journal selection included 7 major ophthalmology journals, 8 subspecialty journals, 2 ophthalmology review journals, 2 basic science ophthalmology journals, 5 major surgery journals, and 5 major medical journals (all top rank for JIF in their respective fields). Year selection was from 1997 to 2009 because the JCR database was available for that period only.

Statistical analysis

JIF values were averaged based on subspecialties and ophthalmic journal specialization (basic science, review, general scope referred to as “major”). Thirteen-year trends in JIF of the various journal subcategories were referenced relative to their corresponding 1997 JIF values. Linear correlations were used to determine associations between JIF and H index or other indices. Determination of significance was set at the 5% level. All analyses were done using Sigma Plot version 11 (2008; Systat Software Inc., San Jose, CA).

Results

JIF increased steadily by around 10% annually in the ophthalmic literature as well as in the select major surgical and medical journals [Figs 1–3]. This follows also the large increase in published papers over the 13-year study. The mean annual number of articles per journal from 1997 to 2009 was 130 in 8 subspecialty eye journals vs. 39 for 2 review eye journals vs. 382 for basic eye journals and 236 for 7 large volume wide-spectrum eye journals, vs. 410 in 5 major medical journals and 208 in 5 major surgical journals. The percentage of increase in number of articles was 169% for subspecialty journals, 91% for review journals, 197% for basic journals, 120% for large volume wide-spectrum journals, 46% in major medical journals, and 104% in major surgical journals. There was a positive correlation between JIF and the annual number of published articles in subspecialty ophthalmic journals (r = 0.68; P = 0.000), major ophthalmic journals (r = 0.63; P = 0.000), basic ophthalmic journals (r = 0.63; P = 0.0005). There was a negative correlation between JIF and the annual number of articles in the category of ophthalmic review journals (r = −0.56; P = 0.003) and internal medicine journals (r = −0.33; P = 0.007). There was no correlation between JIF and the number of articles in the category of Surgery journals (r = 0.068; P = 0.6).

Figure 1

Logarithmic scale of JIF change across 13 years for ophthalmic, medical, and surgical specialties

Figure 3

JIF temporal trend for subspecialty ophthalmic journals and major wide-spectrum ophthalmology journals. Clin Exp Ophthalmol followed the same absolute rise in JIF as the rest of major ophthalmic journals. Retina journal witnessed a sharp increase in JIF from 2006 to 2009

Journals publishing reviews, basic science, or large volume on broad range of topics rank at the top in JIF. The rapid rise of JIF for Retina unlike other subspecialties that stayed status quo is multifactorial: Change in editorial policies (introduction of review articles and omission of case reports) and technological advances in the retinal field. Retina JIF in 1997 was 0.836 with 105 publications and a rank of 16. This gradually increased in 2005 to 1.286 with 137 publications and a rank of 22, in 2006 to 1.403 with 107 papers and a rank of 23. JIF jumped abruptly in 2007 to 3.088 with 188 publications and a rank of 6 [Table 1]. To further elucidate the swift rise in JIF, we compared the subspecialty journal of Retina to that of other subspecialties journal like Cornea and to major journals: Basic like Investigative Ophthalmology Visual Sciences, a wide-spectrum journal like American Journal of Ophthalmology, and a review journal like Progress in Retinal and Eye Research [Table 2]. JIF of Cornea rose mildly from 1997 to 2007 from 1.056 to 1.358, while JIF of American Journal of Ophthalmology rose moderately from 1.715 to 2.628. JIF of Progress in Retinal and Eye Research rose markedly from 1.844 to 7.725 while JIF of Investigative Ophthalmology and Visual Sciences decreased from 5.250 to 3.528 [Table 2].

Table 2

Comparison between 1997 and 2007 for JIF and rank of representative ophthalmology journals

Clinical and Experimental Ophthalmology had the highest relative increase in JIF over the 13-year period and this relates partly to the decrease in number of articles [Fig 2] and more so to the universal scope of the journal. A similar trend was noted for Indian Journal of Ophthalmology with steady rise in citation while a third Asian journal the Japanese Journal of Ophthalmology had a drop in citation pattern after 2008 [Fig 4].

Figure 2

Relative JIF change over 13 years (1997 JIF level considered as one). Note the sharp increase in ophthalmic review journals and mild decrease in basic science ophthalmic journals. The largest increase was for Clin Exp Ophthalmol

Figure 4

Comparison of progress of bibliometric data in three Asian journals of ophthalmology

JIF correlated with H factor (r = 0.83; P = 0.000) [Fig 5], Article Influence Score (r = 0.95; P = 0.000), 5-year impact factor (r = 0.64; P = 0.000), immediacy (r = 0.84; P = 0.000), but not with Eigenfactor (r = 0.159; P = 0.2).

Figure 5

Correlation between JIF and H index for 2007-2008

Discussion

JIF is a systematic objective means to evaluate journals based on citation data. It is clear that reduction in the number of citable articles in the denominator will increase JIF. As an example, Investigative Ophthalmology and Visual Sciences had 320 articles in 1997 with JIF of 5.250 and rank 1. Thereafter in 2007, JIF fell to 3.528 (32.8% decrease) with 724 articles (226% increase in number of articles) and a rank of 5 [Table 2]. Consequently some editors aimed at decreasing the number of published manuscripts, thereby selecting the higher priority articles leading ultimately to a higher earned JIF. Among citable articles, reviews are more commonly cited than original research papers[13] because reviews are authoritative, and are the synthesis of past multiple original research works of a topic in depth. This accounts for the fact that Progress in Retinal and Eye Research tops the list for JIF. Case reports are clearly of lesser academic value than original and review articles[14] and tend to lower JIF. Hence some editors (like Retina and American Journal of Ophthalmology) stopped publishing case reports. Basic science subjects like biochemistry, cell biology, and immunology have a very high JIF. Similarly, basic science journals in ophthalmology had high JIF with Investigative Ophthalmology and Visual Sciences ranking first over several years. Journals with high volume of original basic and clinical research and covering wide topics had high JIF, like Ophthalmology, Archives of Ophthalmology, American Journal of Ophthalmology, and British Journal of Ophthalmology.

The fact that the field of ophthalmic subspecialties targets a rather specific and limited medical audience, and that related articles usually get quoted by this audience, partly explains the low impact factor values. Furthermore, the elongated turnaround time in many of these subspeciality journals contributes to their low impact factor. Hence journals should speed up their publication times to help these values rise.

The phenomenon of journal proliferation has had also a profound effect on JIF. During the past decade an increased JIF was observed in a majority of top-ranking major journals. This echoes the rise in JIF in most ophthalmic journals listed. Andersen et al.[15] found 39% increase in new journal listings in the infectious diseases category over a 10-year period.

The sharp increase in JIF for the journal Retina stems from this journal proliferation, online access, editorial changes with introduction of review articles,[14] transfer of case reports to another related forum,[14] prominent editorial board, and the revolutionary diagnostic and therapeutic advances in the field of rapidly growing retinal research like vascular endothelial growth factor antagonists (47 papers in 2009, 31 papers in 2008, 20 papers in 2007, and 15 papers in 2006 were published in journal Retina relating to bevacizumab) and optical coherence tomography (10 papers in 2009, 28 papers in 2008, 34 papers in 2007, 16 papers in 2006 were published in journal Retina relating to optical coherence tomography).

One can categorize objections to the use of JIF in evaluation of research: (1) Some scientific works are only recognized several years after their publication, while any citation analysis is limited to a predetermined citation window; (2) papers that are never cited do not necessarily have zero impact; (3) negative citations are counted in the same way as positive citations; (4) JIF relies on select English literature while Google has a wider multilingual scope with nearly double journal coverage; (5) the distribution of citations is nonparametric and need to take into account the quality as well as the quantity of citations. The journals should be ranked according to their eigenvector centrality in a citation network. With the recent success of Google's ranking system for web pages, this concept has been modified to include algorithms based on a PageRank system.[16] Although there are several different algorithms in use, the two that have gained the most attention in recent years are SCImago Journal Rank (SJR) (http://www.scimagojr.com/index.php) and Eigenfactor score (ES)[17] (http://eigenfactor.org/), both of which use an iterative weighting system to calculate a summary index that reflects both the “quality” and the “quantity” of citations received by these journals based on a PageRank algorithm. JIF indicator correlated with new bibliometric factors like 5-year impact, H index, and SCImago factor but not with Eigenfactor, according to the present study. In conclusion, while bibliometrics is attractive by providing numbers that relate to scientific productivity, it is often misused because the activity of a scientist does not equate the number of publications or citations, hence the need to analyze the importance and impact of the work itself.[181920] Future studies can further define the role and limitations of the new scientometric indicators in medicine and ophthalmology.