Correlation between Self-Citation and Impact Factor in Iranian English Medical Journals in WoS and ISC: A Comparative Approach.

BACKGROUND: In this study, the impact of self-citation (Journal and Author) on impact factor of Iranian English Medical journals in two international citation databases, Web of Science (WoS) and Islamic world science citation center (ISC), were compared by citation analysis.
METHODS: Twelve journals in WoS and 26 journals in ISC databases indexed between the years (2006-2009) were selected and compared. For comparison of self-citation rate in two databases, we used Wilcoxon and Mann-whitney tests. We used Pearson test for correlation of self-citation and IF in WoS, and the Spearman's correlation coefficient for the ISC database. Covariance analysis was used for comparison of two correlation tests. P. value was 0.05 in all of tests.
RESULTS: There was no significant difference between self-citation rates in two databases (P>0.05). Findings also showed no significant difference between the correlation of Journal self-citation and impact factor in two databases (P=0.526) however, there was significant difference between the author's self-citation and impact factor in these databases (P<0.001).
CONCLUSION: The impact of Author's self-citation in the Impact Factor of WoS was higher than the ISC.

Introduction

The idealistic aim of every journal is dissemination in international famous indexes. Citation indexing databases entitle journals to index their citations if the journal’s structure coincides with the platform of that database and on condition that the journal contains a valid number of citations. Authors need to present their names in these databases too, in order to gain higher scientific reputations (1, 2).

Journal Impact Factor (IF) is the most popular measurement use by Thomson Reuters in comparative assessment of journal’s function (3). This criterion has also been used for journals in ISC (4). This criterion is an aspect of every article’s mean value in that journal and is calculated by dividing the number of current Year citations to the source items published in that journal during the previous two years (5).

Self-Citation is one of the disputed subjects in scientific evaluation or citation analysis (6–8). Self-Citation is a natural phenomenon but this cannot be neglected in citation analysis. Journal self-citation is the giving of reference to articles published in a certain journal. This phenomenon can be seen either positively or negatively.

The mean author self-citation in diabetic literatures is approximately 20% (9). This finding was also similar for medical literatures (9). Larcombe and Voss have found that 17.15% of scholarly articles in ten journals published by the American Physiological Society between 2000 and 2010 contained author’s self-citation. They found that this rate of author self-citation was very similar to the degree reported by others for biomedical journals (10).

Some studies imply that self-citation affects the Impact Factor positively in two databases. In other words, they have reported that there is positive correlation between this citation behavior and Impact Factor (6, 11–13), But some studies showed different results (14, 15).

The aim of this study was identifying the correlation between journal and author self-citation rate and Impact Factor, separately, in Iranian English Medical Journals in two international citation databases, Web of Science (WoS) and Islamic World Science Citation center (ISC) and to distinguish whether results of self-citation would have an effect in the journal’s Impact Factor and also to find out if two types of self-citations would have an impact on indexing in any of the two databases.

Methods

This study was accomplished by citation analysis. The population of this study was all English Medical Journals with Impact factor in WoS and ISC since the establishment of English Journals Citation Reports in Iranian database, starting from (2006) to the last journal citation reports of this database, which is in 2009. There were 12 journals in WoS and 26 journals in ISC. Journal and author self-citation and Journal Impact Factor were dependent variables.

Data gathering

We used JCR Science Edition for collecting Impact Factors and Modified Impact Factors (Impact Factor without Journal Self-Citation) limited to Iranian journals in WoS. In ISC we used English Journal Citation Reports or EJCR for the same purpose. Journal Impact Factor without journal Self-Citations was extracted from EJCR then the author Self-Citation was excluded from the calculation and afterwards, the Impact Factor of the journal was finally calculated and recorded.

For calculating author Self-Citations, we used Cited Reference Search Link in WoS. Two links in ISC, Islamic Countries SCI and English Current Contents, were used for this part of data gathering. In order to calculate self-citations in the ISC database, for journals whose citations were still not recorded in these two databases, the calculated citations and their Impact Factors were calculated with the help of the following two tables namely; Journals’ Cited tables and Citing Journal tables. Data collection lasted from November 28, 2011 to January 9, 2012.

Accounting of Author Self-Citation

For calculating of author Self-Citations, we used Cited Reference Search Link in WoS. Author Self-Citation was collected from the Citation Index database of Islamic Countries (SCI) and from the table of contents of the English Current Contents in ISC. For journals whose citations were still not recorded in these two databases, the calculated citations and their Impact Factors were calculated with the help of the Journal’s Cited table and Citing Journal table and were evaluated by browsing article resources. Then, the author Self-Citation was deleted from the calculation formula. It is important to note that the formula used for calculating the journal’s impact factor in the Journal Citation Reports in each of these two databases was calculated and reported in a complete mathematical method for each year of evaluation.

Calculation of Impact Factor without author self-citation was done in the same way for journal self-citation. In this study, only the first author of every article was considered for author self-citation.

Data analysis

Due to the reason that in 2009, in one database, a journal has had two different titles so, with the help of an existing table of guidelines from the JCR database (WoS) a unified Impact Factor and the mean self-citations for this journal was calculated (International Journal of Fertility and Sterility) (Table 1). The unified impact factor has been calculated and then compared based on the guidelines (16).

Table 1:

Computing of Unified Impact Factor for International Journal of Fertility and Sterility in WoS (2009)

Variable	A Citation to journals published in 2007&2008	B Number of articles published in 2007&2008	C Journal Impact Factor 2009 (Unified IF) Column A/Column B
Old Title	0 +7	25 + 0	0.28
New Title	7 + 0	0+ 32	0.219
Compound (Old + New)	7 + 7	32 +25	0.246

In order to obtain the mean Impact Factors and the mean of the two types of self-citations, the Microsoft Excel and spreadsheets were used. Variable test was done using SPSS version.20. The meaning surfaces were also 0.05 in all of tests for union paraphrasing (interpretation).

Wilcoxon singed-ranked test was used for comparison between self-citation rates (journal and author separately) for journals both indexed in WoS and ISC which amounted to 6 journals. Man-Whitney test was used for journals, which were indexed separately, so there were 6 journals in WoS and 16 Journals in ISC.

Correlation between self-citations and Impact factor in WoS: The values were obtained using Pearson test.

Correlation between self-citations and Impact factor in ISC: Since data volume in ISC was few, this part of analysis was performed using the Spearman-ranked test.

In order to identify the correlation between journal self-citations and author self-citations and Impact Factors in two databases, correlation between the two test mentioned above, the covariance analyzes was employed.

Journal Ranking

In comparing the two databases’ journals’ rankings based on Impact Factors with and without self-citations for the year 2006, no calculation has been implemented for the reason that no Iranian journal had been indexed in the Web of Science database on that year. Furthermore, in 2007, the only journal indexed in this database was Daru, which was not included in the ranking in that year. Therefore, this part of the study was evaluated separately and was excluded in statistical analysis.

Results

Twenty-four Iranian English Medical Journals were indexed in WoS from 2007 to 2012. Only 13 journals have Impact Factor in this years’ study. One journal has its title changed (from Iranian Journal of Fertility and Sterility to International Journal of Fertility and Sterility). Since this journal was recorded twice, therefore, the number of journals in WoS for this study was 12. In 2006, the Iranian Journal of Public Health and in 2007 the Daru journal, were indexed in this database but the impact factor calculation and the entrance of Iranian Journal of Public Health to JCR has been since 2008.

An overview of journal self-citation and author self-citation rate in this study showed that only Daru had 0 percent for journal self-citation rate in 2007. Journal Self-Citation and author self-citation rate are shown in Table 2.

Table 2:

Mean of journal and author self-citation rates in Iranian English medical journals indexed in WoS based on journal titles (2007–2009)

Row	Abbreviation of Journal Title	Author self-citation		Journal self-citation		Mean of Impact Factor
		Mean	Mean of Rate (%)	Mean	Mean of Rate (%)
1	Arch. Iran.Med.	15	10.27	9	6.16	0.874
2	Daru	24	23.05	19	23.77	0.365
3	Hepat.Mon.	33	62.26	8	15.09	0.716
4	Int. J. Fertil. Steril.	5	35.71	4	28.57	0.246
5	Iran. J. Allergy Asthma Immunol.	2	3.28	5	8.2	0.968
6	Iran. J. Arthropod-Borne Dis.	5	41.67	1	8.33	0.353
7	Iran. J. Pediatr.	6	28.57	3	14.29	0.131
8	Iran. J. Pharm. Res.	4	21.05	4	21.05	0.235
9	Iran J. Public Health	11	13.71	8	9.76	0.292
10	Iran. J. Radiat. Res.	2	33.33	3	50	0.125
11	Iran. J. Reprod. Med.	3	23.08	2	15.38	0.183
12	Iran. Red Crescent Med. J.	4	57.14	2	28.57	0.071

Mean journal self-citation rates and mean author self-citation rates in these journals are shown in Table 3. Unlike the author self-citation rate, journal self-citation rate was higher in WoS. The mean of journal self-citation rates were 27.53% in this database and the mean author self-citation rates were 19.1% in this database too (Table 3).

Table 3:

Mean of journal and author self-citation rates for Iranian English medical journals indexed in WoS 2007–2009

Mean of journal self-citation rate (%)	Mean of author self-citation rate (%)	Year
0	33.33	2007
16.11	20.52	2008
31.73	17.67	2009
27.53	19.1	Total

In ISC, 40 Iranian English Medical Journals were indexed from 2006 to 2012. The number of journals included in this study was 26. In other words, 26 journals had Impact Factor in this database (2006–2009) including the following 6 Iranian journals in the field of health which were extracted by searching the EJCR database page by page without filtering the broader category; Dental Research Journal; Iranian Journal of Clinical Infectious Diseases; Iranian Journal of Dermatology; Iranian Journal Environment Health Science and Engineering; Iranian Endodontic Journal; and Iranian Journal of Medical Ethics and History of Medicine.

Mean journal self-citation rates and mean author self-citation rates in these journals are shown in Table 4.

Table 4:

Mean of journal and author self-citation rates in Iranian English medical journals indexed in ISC based on journal tittles (2006–2009)

Row	Abbreviation of Journal’s Title	Author self-citation		Journal self-citation		Mean of Impact Factor
		Mean	Mean of Rate (%)	Mean	Mean of Rate (%)
1	Acta Med. Iran.	1	100	1	100	0.003
2	Arch. Iran.Med.	5	38.57	3	21.43	0.056
3	Daru	1	33.33	1	33.33	0.01
4	Dent. Res. J.	0	0	1	0	0.01
5	Hepat.Mon.	0	0	0	0	0.019
6	Iran.Biomed. J.	2	25	1	25	0.031
7	Iran.Endod. J.	0	0	0	0	0.005
8	Iran. J. Allergy.Asthma Immunol.	2	33.33	0	0	0.019
9	Iran. J. Basic Med.Sci.	0	0	1	100	0.021
10	Iran. J. Child Neurol.	2	25	2	75	0.02
11	Iran. J.Clin. Infect. Dis.	0	0	0	0	0.021
12	Iran. J. Environ.Health Sci.Eng.	3	88.33	0	0	0.012
13	Iran.J.Immunol.	0	0	4	100	0.016
14	Iran. J. Med.Sci.	0	0	2	38.89	0.013
15	Iran.J.Parasitol.	0	0	0	0	0.024
16	Iran.J.pathol.	2	100	1	50	0.009
17	Iran. J. Pharm. Res.	1	8.33	3	75	0.024
18	Iran. J. Psychiatry Behav.Sci.	0	0	1	100	0.013
19	Iran. J. Public. Health	0	0	1	4.17	0.037
20	Iran.J.Radiol.	2	50	1	25	0.045
21	J. Res. Med.Sci.	0	0	1	50	0.005
22	Urol. J.	3	100	1	25	0.01

The data, which were ambiguous, were excluded. So, the comparison was implemented for 22 journals in this database. The pattern of author self-citation rate, and journal self-citation rate were irregular in this database. The mean of journal self-citation rates was 25.24% in this database for all journals included in this study while the mean author self-citation rate was 34.81% (Table 5).

Table 5:

Mean of journal and author self-citation rates for Iranian English medical journals indexed in ISC 2006–2009

Mean of journal self-citation rate (%)	Mean of author self-citation rate (%)	Year
41.67	41.67	2006
29.59	34.69	2007
34.76	29.76	2008
8.85	36.72	2009
25.24	34.81	Total

Comparison between self-citation rates in two databases

No difference was observed between self-citation rates in Iranian English Medical Journals in WoS and in ISC. Results of Wilcoxon tests also indicated no difference between journal self-citation rate (P=0.6) and author self-citation rate (P=0.917) in journals indexed in both databases.

In addition, the Man-Whitney tests showed no difference between journal self-citation rate (P=0.194) and author self-citation rate (P=0.602) for journals indexed separately.

Correlation between self-citation and journal Impact Factor

WoS: For the reason that only few journals were indexed in WoS between 2007 and 2008, therefore no test was available for computing these years. But there was a meaningful difference between journal self-citation and Impact Factor in Iranian English medical journals in WoS in 2009 (P=0.002, r=0.77). a) This shows that, Impact factors would increase if journal self-citation rate increases. The share of journal self-citation in this increase was 0.59% (R2= 0.59).

Approximately, 60% of the variation in journal Impact factors was due to journal self-citation.

Generally, there was a meaningful difference between journal self-citation and Impact Factor (P= 0.007, r=0.64) in this section of the statistical population indexed in WoS from 2007 to 2009 (Table 6). On the other hand, there was no difference between author self-citation rates and Impact Factor in Iranian English medical journals in WoS in all of these years (Table 7).

Table 6:

Correlation between journal self-citation and Impact Factor in Iranian English medical journals in WoS (2007–2009)

Number of journals	Meaningless	Pearson correlation coefficient ®	Year
13	0.002	0.77	2009
16	0.007	0.64	Total(2007–2009)

Table 7:

Correlation between author self-citation and Impact Factor in Iranian English medical journals in WoS (2007–2009)

Number of journals	Meaningless	Pearson correlation coefficient ®	Year
13	0.11	0.46	2009
16	0.06	0.69	Total(2007–2009)

a) ISC: Results of Spearman tests in this section of research showed that, there was low correlation between journal self-citation and Impact Factor in Iranian English medical journals in ISC (P= 0.13, r=0.43). The correlation was meaningful at 0.1 level of significance only for journals published in 2009 (P=0.09, r=0.43). This correlation was weak but positive. The share of increased journal self-citation was 18%. The result was different for 2008. The correlation was negative, but was not significant in this current year (P=0.92, rsp= −0.03).

There was no meaningful difference between journal self-citation and Impact Factor (P= 0.13, r=0.23) totally, in these parts of the statistical population which indexed in ISC from 2006 to 2009 (Table 8).

Table 8:

Correlation between journal self-citation and Impact Factor in Iranian English medical journals in ISC (2006–2009)

Number of journals	P	Spearman correlation Coefficient (rsp)	Year
6	0.35	0.46	2006
7	0.3	0.45	2007
14	0.92	−0.03	2008
16	0.09	0.43	2009
43	0.13	0.23	Total

There was a weak correlation between author self-citation and Impact Factor in Iranian English medical journals in ISC except for 2007. In this year, there was a positive and meaning correlation between the two variables in meaning surface test (P=0.04, r=0.75). In this year, the share of author self-citation was 0.56% (R2= 0.56). The fifty six percent of the variations in journal Impact factors were by author self-citation. There was no meaningful difference between author self-citation and Impact Factor (P= 0.63, r=0.06) totally, in these parts of the statistical population which indexed in ISC from 2006 to 2009 (Table 9).

Table 9:

Correlation between author self-citation and Impact Factor in Iranian English medical journals in ISC (2006–2009)

Number of journals	meaningless	Spearman correlation Coefficient (rsp)	Year
6	0.85	−0.09	2006
7	0.04	0.75	2007
14	0.5	−0.19	2008
16	0.96	0.014	2009
43	0.63	0.06	Total

Analysis of two correlation tests (by covariance test), which was performed on self-citation rates and Impact Factor showed that there was no difference between journal self-citation and Impact Factor in WoS and ISC (P=0.0526). In contrary, there was a meaningful difference between author self-citation and Impact Factor in these databases (P<0.001). This means, the correlation between author self-citation and Impact Factor in WoS was stronger than ISC.

Journals Ranking

a). WoS: After the elimination of Author self-citations, Daru’s Impact Factor has decreased in 2007, it was the only Iranian medical journal indexed in WoS in the current year. Results of journal evaluation ranking in the last 2 years (2008–2009) showed that 2 journals (14.28%) remained with the same ranks their ranking after removing journal self-citations while 3 journals have increased their ranks (21.43%) and the ranking of nine journals (64.29%) remain unchanged. The elimination of author self-citations has resulted to an increase in the ranking of 2 journals (14.28%) but decreased the ranking of 1 journal (7.14%) while 11 out of the 14 journals have maintained their previous rankings. The impact of journal self-citations in journal rankings was higher than the impact of author self-citations in this database.

b). ISC: Results of the four year evaluation conducted on the journal ranking of 43 journals showed that rankings of 26 journals (60.46%) have increased after elimination of journal self-citations, 9 journals (20.93%) decreased their rankings while 8 journals (18.61%) have their rankings maintained. The elimination of author self-citations has (27.91%) increased the rank of 17 journals (39.53%), decreased the rank of 12 journals while maintaining the ranks of 14 journals (32.56%). The impact of journal self-citations in journal rankings was higher than the impact on author self-citations in this database too. After elimination of journal self-citations, ranking stability in WoS has increased more than threefold in comparison to ISC (64.29% VS 18.61%). Deviation towards a decreasing trend was less while deviation towards an increasing trend was 3 times higher in WoS in comparison to ISC (60.46% VS 21.43%). In addition, the ranking stability in WoS has increased after removing author self-citations (78.57% VS 32.56%). Variations toward Ranking deviation to a decreasing trend was less in about fourfold (7.14% VS 27.91%). Variations toward an increasing trend in ranking has increased to fourfold in ISC after eliminating authors self-citations (39.53% VS 14.28%).

Discussion

According to this study, there was no significant difference between journals self-citation rate and author self citation rate of the two databases whereas, an important difference was observed in correlation between journal self-citation and the impact factor of the two databases.

Thus, the notion that journals indexed in WOS database with the help of self-citation are well established is incorrect, but the authors’ self-citation for articles in medical journals published in WOS are most likely to increase the impact factor of these journals. Despite the strong and positive correlation between a journal’s self-citation and the journal’s impact factor in WOS, the impact of journal self-citation was not so strong since most of the journal rankings have decreased. In contrast, in the ISC, there was no correlation between self-citation and the impact factor but after removing the self-citation; the journals rankings in this database were higher in comparison to the other database.

According to citation analysts, self-citation cannot be denied. Despite this, an increase in the share of self-citation in scientific credible journals has proved their effectiveness. Gami et.al. in their assessment on authors’ self citation in the field of diabetes have expressed that, authors’ self-citation in this field amounted to 20%. They also expressed that authors’ self-citation had negative impact on important scientific medical processes important scientific medical processes, communication and growth of medical literatures and in discovering innovations (9).

Rate of authors’ self-citation for American journal in the field of physiology is on the average (17.75%) and very close in ranks to each other (10).

The rate of authors ‘self-citation in WOS and ISC databases are 19.1% and 34.81% and the rate of journal self-citation in WOS and ISC are 27.53 and 25.24 respectively. Despite the fact that the rate of authors’ self-citation in ISC database is higher than WOS, the relationship between author self-citation and the journal’s impact factor in this database was lower in comparison to Web of Science.

The average rate of journal self-citation in Persian medical journals in ISC database was reported 21.93%. Their study showed that there was significant correlation between journal self-citation and impact factor. The share of journal self-citation on impact factor changes was 30%. They also reported the same positive correlation among journals in the field of agriculture and veterinary medicine in this database (P<0.01, r=0.425), 18% of the coefficient changes were under the influence of journal self-citation. The average rate of journal self-citation was 30.05% (16).

The share of journal self-citation on impact factor changes in ISC on 2009 alone was remarkable (18%) despite the fact that test results was not on the significant level, the effect of journal self-citation on WOS just like its results, were remarkable (P=0.007, r=0.64).

Ghane (11) found a significant correlation of 0.01% between impact factor, journal self-citation and author self-citation in Persian journals in the field of Engineering in ISC database. The average author’s self-citation was 41% and the average journal self-citation was 66%.The influence of author’s self-citation on impact factor was slightly higher than the journal self-citation. This study did not find any significant correlation between authors self-citation and impact factors on Medical British journals in this database (P=0.13, rsp= 0.06). Also, there was no significant correlation between journal self-citation and impact factor (P=0.63, rsp = 0.06).

In the study conducted by McVeigh, a weak correlation was observed between the rate of journal self-citation, credibility and the journal’s subject. In his findings, the elimination of journal self-citation from the IF calculation had little influence on the relative rankings of prestigious journals (17).

Theory of Anseel and colleagues regarding the rate of author self-citation on the impact factor of psychology journals in 2001was rejected. The changes in the impact factor after the elimination of journal self-citation on journals having a high IF was insignificant (18). In the present study, it was only on the year 2006–2008 that correlations between two variables were on the negative direction but, the correlations were not significant (P>0.05).

Frandsen indicated a positive correlation of 0.01% between journal self-citation and impact factor among journals in the field of economics (from 1986–2004) (6). In this present study, this relationship was shown in WOS, but this correlation during the shift of journal rankings in this database was not remarkable. In ISC database, despite the presence of more changes in journal rankings, this correlation was not significant (P=0.13, rsp=0.23). Furthermore, there was no significant difference observed between the correlation of these two databases (P=0.526).

There was negative correlation between journal self-citation and impact factor among journals in the field of ecology in 2004 (14). However, this relationship was not significant for high impact accredited journals. For journals with high IF, self-citation was lower but, these journals, after eliminating the journal self-citation had undergo several changes. He also expressed that there exist an increasing trend towards journal self-citation. The average rate of journal self-citation in the journal’s impact factor in this field was 16.2%. In this present study, the share of journal self-citations in increasing the impact factor for 2009 in WOS was 59%. Moreover, after eliminating journal self-citations in this database, the rank of their 3 top journals was not affected. This finding was similar to the finding in our study.

The authors used impact factor in making decisions to publish their articles and as well as finding other journals in their own specialty. Also, directors of scientific institutions and scientific festivals such as the Razi Medical Sciences Research Festival have used the impact factor in decision making process for improvement, promoting the faculty members’ rank and in choosing the outstanding researchers (19).

Although the impact factor is useful, but like any other tool should be used correctly. This represents an average rate for a journal and serves as possible criterion but there are other available tools that can be used to complete this impact factor. In truth, the impact factor measures the level of the journal but neither the level of the article nor the authors. Based on the statement from the Declaration on Research Assessment (DORA), “Journal-based metrics such as Journal Impact Factors should not be used as a surrogate measure of the quality of individual research articles, in assessing scientist’s contributions and in hiring, promoting and funding decisions (20).

In this digital era, research shows that relationship between the impact factor and paper’s citation is weakening and according to the authors this can pose serious consequences. “If this pattern should continue, it might bring an end to the use of impact factor as a way to evaluate the quality of journals, papers and researcher” (21).

Conclusion

The authors of articles published in journals indexed by WoS have gained more self-citations than authors having articles published in ISC. Therefore, we can conclude that journals in WoS may be indexed in this database with the help of author self-citation.

Ethical considerations

Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.