Citation Analysis for Biomedical and Health Sciences Journals Published in Korea.

OBJECTIVES: A citation analysis of biomedical and health sciences journals was conducted based on their enlistment in journal databases to identify the factors contributing to the citation metrics.
METHODS: Among the 1,219 academic journals managed by the National Center for Medical Information and Knowledge at the Korea Centers for Disease Control and Prevention, 556 journals were included for analysis as of July 2016. The characteristics of the journals include history years, publication media, language, open-access policy as well as the status enlisted in international and domestic databases, such as Science Citation Index (SCI), Scopus, Medline, PubMed Central, Embase, and Korea Citation Index (KCI). Six bibliometric measures were collected from SCI, Scopus, and KCI as of 2015, the most recent disclosure year. Analyses of group differences and influential factors were conducted using t-tests, Mann-Whitney tests, and multiple regression.
RESULTS: Journal characteristics, such as history years, publication media, and open-access policy, were not significant factors influencing global or domestical citation of the journals. However, global citations were higher for SCI and Medline enlisted journals than for their counterparts. Among KCI journals, the KCI impact factors of journals published in English only were lower.
CONCLUSIONS: Efforts by journals to be enlisted in international databases, especially in SCI and Medline, are critical to enhance their global circulation. However, articles published in English only hinder the use of domestic researchers. Different strategies are required for enhancing international and domestic readerships.

I. Introduction

Globally, the number of scholarly journal publications in the biomedical and health science field has risen sharply in recent years [1]. The number of journals indexed in Medline increased from 3,823 in 1995 to 5,623 in 2016. Also, biomedical journals published in Korea are greatly increasing [2]; between 2000 and 2017, the number of journals indexed by the Korean Association of Medical Journal Editors (KAMJE) increased from 57 to 256 [3]. However, the citation performance of journals published in Korea is relatively low [4] and has remained stagnant over decades [5].

A citation is often used as a criterion to evaluate an article's quality and even as a criterion to assess the research competence of an individual or a research team [4]. Therefore, researchers try to publish their study results in a journal that has a high citation rate. Also, editors of journals make various efforts to increase the citations of their journals. Some journals permit open access to enhance readers' access and attract more readers [6]. In non-English-speaking countries, such as South Korea, some journals are published in English to extend the scope of readers and to increase the probability of an article to be cited [7]. In addition, editors of journals try to enlist their journals in international databases [8] because the enlistment of a journal in international databases not only guarantees, to some extent, the quality of the journal but also makes an article easily attract readers' attention. Hence, enlistment in international databases may help a journal to acquire trust from researchers and thereby increase the probability of citation.

Over the past few decades, many bibliometric measures that reflect academic journals' scientific quality and influence were developed [9]. Traditionally, the major source of citation data has been the Web of Science (WoS), which reports impact factors (IFs) Eigenfactor scores, and article influence scores (AISs) for the world's leading journals enlisted in databases such as the Science Citation Index (SCI) [10]. Another major database is Scopus, providing SCImago journal rank (SJR), source normalized impact per paper (SNIP), h-index, and impact index [10]. Also, the Korean government has established the Korea Citation Index (KCI) to analyze the citation rates of domestic research [11]. The KCI reports Korea Citation Index impact factors (KCI IFs) and Web of Science-Korea Citation Index integrated impact factors (KCI-WoS IFs).

Although the Korean government aims to enhance the quality of domestic journals through various national policies, no empirical study has been conducted about the effectiveness of those policies. Also, little is known about the factors influencing the citation of journals in biomedical and health sciences. Investigation of the factors of citation rates may contribute to the spread of domestic research works and relevant future policies. Therefore, the objectives of the present study were to compare citation rates according to database enlistment and publication characteristics and to identify the factors influencing journal citation rates.

II. Methods

1. Target Journals

A journal database from the National Center for Medical Information and Knowledge at the Korea Centers for Disease Control and Prevention (personal communication, July 12, 2016), which includes 1,219 biomedical journals, was used. Among them, only currently published journals with major journal information were included as of July 2016. After duplicated journals were excluded based on paper ISSN and electronic ISSN, 556 journals were included for analysis. The earliest published journal was printed in 1948, and the most recent journals were published in 2016. The selection process of target journals for analysis is summarized in Figure 1.

Figure 1

Selection process of target journals for analysis. SCI: Science Citation Index, KCI: Korea Citation Index.

2. Journal Bibliometric Measures

Bibliometric measures were collected as of 2015 from each pertinent database website for journals enlisted in the SCI, Scopus, and KCI databases. For journals enlisted in SCI, their IF and AIS data were obtained from the Institute of Scientific Information. For the citation rates of journals enlisted in Scopus, SNIP and SJR scores were obtained from SCImago. The Korean bibliometric measures of journals enlisted in KCI are KCI-WoS IF and KCI IF, and these scores were obtained from KCI. Then, these data were merged with the database by each journal's paper ISSN or electronic ISSN.

1) Impact factor

The IF of a journal is defined as the number of times articles from the journal are cited, according to WoS, within 2 years divided by the total number of articles published in the same journal during that 2-year period [10]:

2) Article influence score

The AIS is an indicator that measures the relative average influence of a given article within the first five years of its publication [4]. It calculates the relative importance of the journal on a per-article basis. The AIS is obtained by dividing the Eigenfactor score by the number of articles published in the journal and normalized to make the overall AIS of all journals 1.0. It is roughly analogous to the 5-year IF; it is the ratio of the journal's citation influence to the size of the journal's article contribution over a period of 5 years.

3) Source normalized impact per paper

The SNIP corrects for differences in topicality between subject fields. It is the ratio of a journal's citation impact and the degree of topicality of its subject field. SNIP's numerator gives a journal's raw impact per paper, which is very similar to the IF. Its denominator is the citation potential in a journal's subject field, a measure of the citation characteristics of the journal's field, determined by how often and how rapidly authors cite other works, and how well their field is covered by the database [12].

4) SCImago journal rank

The SJR is a prestige metric inspired by Google PageRank that represents a measure of scientific influence of journals; it constitutes the number of citations received by a journal and the prestige of the journals where such citations made. It expresses the average number of weighted citations received in the selected year by the documents published in the journal in the three previous years [12].

5) Korean Citation Index-Web of Science impact factor

The representative citation evaluation indicator, IF, is computed by dividing the number of times a journal has been cited by others over the past two years by the total number of published articles during the same time. The KCI-WoS IF of a journal is defined as the number of citations in KCI and WoS within a certain year to all articles published in that journal during the previous 2 years (the numerator), divided by the number of articles in that journal published during the previous 2 years (the denominator) [13].

6) Korean Citation Index impact factor

The KCI IF of a journal is defined as the number of citations in KCI within a certain year to all articles published in that journal during the previous 2 years (the numerator), divided by the number of articles in that journal published during the previous 2 years (the denominator) [13].

3. Publication Characteristics

The publication characteristics of the journals were collected from the database, and the missing values were verified from the online website of the individual journals. The journal information includes publication media (whether a journal has paper ISSN or electronic ISSN numbers); open-access policy; publication frequency (issues per year); language (Korean, English, or both); history years (period or year since its first issue); status enlisted in SCI, Scopus, KCI, Medline, PMC, and Embase.

4. Data Analysis

Descriptive statistics were calculated for each variable using frequencies and means for categorical and continuous variables, respectively. The citation rates between groups were compared by t-tests and Mann-Whitney U tests depending on sub-group sample sizes. For an overall summary, multiple regression analyses were performed to analyze the factors influencing the citation rates. Effects were considered significant when the p-value was less than 0.05. The analyses were conducted with SPSS ver. 23.0 (IBM, Armonk, NY, USA).

III. Results

1. Descriptive Characteristics of Journals

Among the 556 journals finally included, 276 journals (49.64%) had only paper ISSNs; 183 journals (32.91%) were open-access; and 180 journals (32.37%) were written in only English. Twenty-five journals were enlisted in Medline (4.50%), 50 in SCI (8.99%), 98 in Scopus (17.60%), 60 in Embase (10.79%), 102 in PMC (18.30%), and 272 in KCI (48.92%). The mean IF of the 50 journals enlisted in SCI was 1.54 ± 0.92, and their mean AIS was 0.29 ± 0.22. The mean SNIP of the 98 journals enlisted in Scopus was 0.59 ± 0.36, and their mean SJR was 0.38 ± 0.23. The mean KCI-WoS IF of the 272 journals enlisted in KCI was 0.65 ± 0.64, and their mean KCI IF was 0.45 ± 0.45 (Table 1).

Table 1

General characteristics of biomedical journals published in Korea (n = 556)

SCI: Science Citation Index, IF: impact factor, AIS: article influence score, SNIP: source normalized impact per paper, SJR: SCImago journal rank, PMC: PubMed Central, KCI: Korea Citation Index, WoS: Web of Science.

2. Citation Rates according to Publication Characteristics and Enlistment in Databases

Table 2 shows the citation rates according to publication characteristics and database indexing status of biomedical journals published in Korea. The magnitudes of IF, SNIP, and SJR of open-access journals were higher than those of non-open-access journals (1.75 vs. 1.10; 0.62 vs. 0.50; and 0.40 vs. 0.31). However, the KCI IFs of open-access journals were lower than those of non-open-access journals (0.34 vs. 0.59). With regard to language, the SNIP and SJR of journals published in only English were higher (0.64 vs. 0.31; 0.41 vs. 0.18), but their KCI IFs were lower than those of other journals (0.21 vs. 0.64).

Table 2

Average citation rates according to publication characteristics and database indexing status of biomedical journals published in Korea

SCI: Science Citation Index, IF: impact factor, AIS: article influence score, SNIP: source normalized impact per paper, SJR: SCImago journal rank, KCI: Korea Citation Index, WoS: Web of Science, PMC: PubMed Central.

*p < 0.05, **p < 0.01.

The journals enlisted in SCI showed high SNIP, SJR, and KCI-WoS IF scores (0.74 vs. 0.47; 0.48 vs. 0.29; and 1.40 vs. 0.49), but low KCI IFs (0.30 vs. 0.48). The journals enlisted in Scopus showed high KCI-WoS IFs (0.83 vs. 0.56), but low KCI IFs (0.29 vs. 0.53). The journals enlisted in Medline showed high AIS, SNIP, SJR, and KCI-WoS IF scores (0.43 vs. 0.22; 0.75 vs. 0.55; 0.53 vs. 0.34; and 1.32 vs. 0.58). The journals enlisted in Embase showed high IF, SNIP, SJR, and KCI-WoS IF scores, but low KCI IFs. The journals enlisted in PMC showed high IF, AIS, SNIP, and SJR scores but low KCI IFs.

3. Factors Influencing Journal Citation Rates

To identify the factors influencing citation rates, a multiple regression analysis was performed with the journal characteristics and the enlistment in databases for each of the six bibliometric measures taken from SCI, Scopus, and KCI. As shown in Table 3, F-tests showed that all six models were statistically significant at the 0.05 level. Journal characteristics, such as history years, publication media, and open-access policy, were not significant factors influencing global or domestic citation of the journals.

Table 3

Coefficient estimates for factors influencing citation rates of biomedical journals published in Korea

SCI: Science Citation Index, IF: impact factor, AIS: article influence score, SNIP: source normalized impact per paper, SJR: SCImago journal rank, KCI: Korea Citation Index, WoS: Web of Science, PMC: PubMed Central.

*p < 0.05, **p < 0.01, ***p < 0.001.

For each model, the statistically significant factors were the following: for SCI IF, non-enlistment in KCI (B = –1.29); for AIS, enlistment in Medline (B = 0.30); for SNIP, enlistment in SCI (B = 0.20); for SJR, enlistment in SCI (B = 0.12) and PMC (B = 0.11); for KCI-WoS IF, language in English only (B = –0.46), and enlistment in SCI (B = 1.12) and Medline (B = 0.47); and for KCI IF, the frequency of publication (B = –0.02) and language in English only (B = –0.43). Overall, global citations were higher for SCI, Medline, and PMC enlisted journals. Among KCI journals, the KCI impact factors of journals published in English only and higher issues were lower.

IV. Discussion

This study was conducted to analyze the factors influencing the citation rates of biomedical and health science journals published in Korea. The results show that enlistment in an international database enhances the citation rate. Most of the citation rates of the journals enlisted in prominent international databases, such as SCI and Medline, were higher than those of the journals that were not enlisted. This result could be explained by the fact that researchers in the biomedical field try to cite the articles of journals enlisted in prominent international databases to support the validity and legitimacy of their studies. In addition, journals enlisted in prominent international databases are more likely to be searched, attracting more international readers [6].

The multiple regression analyses confirmed the results that the key factor influencing international citation rates was enlistment in prominent international databases. Enlistment in SCI was found to be effective in increasing SNIP and SJR, even after other variables were adjusted. Enlistment in PMC was found to be effective in increasing SJR, even after other variables were adjusted. On the contrary, enlistment in KCI was found to be a factor decreasing IF. Only three journals were enlisted in SCI but not enlisted in KCI, and the IF of these journals was high because the major targets of these journals were international readers.

Enlistment in Medline was found to influence AIS and KCI-WoS-IF, even after other variables were adjusted. The effect of enlistment in Medline may be based on not only the fact that Medline is a prominent international database but also the fact that Medline is one of the databases most frequently used by researchers of medical science [14]. Moreover, the articles of the journals enlisted in Medline undergo MeSH (Medical Subject Headings) indexing by MeSH indexing experts, which may increase the probability to be searched by researchers. Researchers want to rapidly find desired articles by using the MeSH indexing languages as search keywords to increase search specificity [1516]. In studies such as systematic reviews, MeSH indexing is often used for search keywords to increase comprehensiveness [17]. However, the causal relation between MeSH indexing and citation rates should be clarified in further studies.

In addition, open-access journals showed higher IF, SNIP, and SJR scores than non-open-access journals. This result is consistent with those of previous studies that found that open-access journals are easily accessed by researchers and are more likely to be cited [18]. The Scopus citation rates, SNIP, and SJR scores of the journals published only in English were higher than those of other journals, which is consistent with the findings of previous studies [619]. This suggests that journals published in English are viewed by international readers and thus may be cited more frequently.

However, the KCI IFs were lower for journals published only in English. This could be explained by the fact that journals published in Korean are easier for Korean readers to read; thus, the domestic citation rates of these journals were higher. This explains the low KCI IFs of journals enlisted in SCI, Embase, and PMC as well as open-access journals, which is opposite to other citation rates. Most of the journals enlisted in these databases are published in English, and publication in English is a prerequisite for an open-access journal to be enlisted in one of these databases. Journals published in English showed lower KCI IFs than journals published in Korean. These results suggest that publishing a journal in Korean is advantageous to enhance domestic circulation in Korea. In addition, journals having higher publication frequency showed lower KCI IFs. Therefore, to improve the quality of a journal published in Korean, publishing high-quality articles may help to increase domestic citation rates, such as KCI IFs, rather than simply increasing the number of articles.

Various citation indices were considered in the analysis performed in the present study. These citation indices are calculated by using different equations and have different implications [20]. Therefore, the predictors of individual citation indices are different. For example, the SCI citation index, the popularity index, IF, and the prestige index, AIS, were found to have different predictors. Although IF is the most widely known citation index, and it is extensively applied due to its simple calculation, IF has many limitations, for example, lack of quality assessment of citations, English language bias, and influence of self-citation [21]. Also, other indices have their own advantages and disadvantages [22]. Therefore, in the evaluation of the quality of a journal, it is important to understand the calculation equations of individual citation rates and to utilize a citation index appropriate for the evaluation purpose, rather than relying on a single citation index.

The present study had following limitations. First, since this was a cross-sectional study, it may have been limited in identifying a causal relation between database enlistment and citation rates. A further longitudinal study needs to be conducted to verify the direct impact of database enlistment. Second, this study targeted the biomedical and health science journals published in Korea as a whole, but the distribution of journal characteristics and citation behaviors might be different in sub-areas of biomedical sciences [23]. Therefore, research in this area may be expanded by considering the specialty categories of biomedical and health sciences. Despite these limitations, the present study has value in that the relationship between database enlistment and citation rates was empirically analyzed to provide important information to enhance future journal circulation.

In conclusion, enlistment of a journal in prominent international databases is critical to international circulation. The results of the present study also showed that only low proportion of Korean journals in biomedical and health science fields were enlisted in prominent international databases. One strategy for enlistment is to publish journals in English, which would satisfy this prerequisite for enlistment in a prominent international database and help to attract more international readers. On the other hand, publishing domestic journals in Korean is advantageous for enhancing the circulation in Korea. Thus, the publication language should be carefully chosen by a journal with consideration of the target readers. In addition, more than 50% of Korean journals of biomedical science have not been enlisted even in KCI. To spread the content of biomedical journals published in Korea, more work should be done, such as indexing and bibliographic services initiated by reputable, governmental agencies, e.g., the National Center for Medical Information and Knowledge to provide validation and public trust.