Identifying the Factors Affecting Papers' Citability in the Field of Medicine: an Evidence-based Approach Using 200 Highly and Lowly-cited Papers.

INTRODUCTION: Nowadays, publishing highly-cited papers is important for researchers and editors. In this evidence-based study, the factors influencing the citability of published papers in the field of medicine have been identified.
MATERIAL AND METHODS: 200 papers indexed in Scopus (in two groups: highly-cited and lowly-cited) with 100 papers in each were studied. Needed data were manually collected with a researcher-made checklist. Data analysis was done in SPSS using descriptive and inferential statistics.
RESULTS: Variables such as journal IF, journal rank, journal subject quartile, the first/corresponding author's h-index, the number of documents produced by the first/corresponding author, SJR and SNIP had significantly positive correlation with paper citability (p< .05). Other variables, including among others, paper age, paper type, the number of references, the number of authors, indexing institute and journal kind had not any relationship with paper citability (p> .05).
CONCLUSION: the factors affecting the citability are among indicators relating to authors, publishing journals and published papers. Determining the extent to which these factors influence the citability of a paper needs further large-scaled research. Authors and editors searching for high-citedness should consider these factors when authoring and publishing papers.

INTRODUCTION

Issues on science nature, science classification and science structure have longer been considered and due to the vast development of science and technology in recent years, these issues have been developed and deepened gradually and named under “science studies”. Science studies have two main aspects: qualitative and quantitative aspects. The former considers the philosophy, history and sociology of the science. The latter studies the measurable aspects of science named scientometrics (1).

As one of the methods for evaluating the scientific activities and research management, scientometrics is based on four main variables, including authors, scientific publication, citations and references (2). Most scientometric indicators are based on the citation, as a basic scientific indicator (3). It has been argued in the scientific societies that whether or not the citation can be a good scientific indicator. However, it cannot be ignored in evaluating scientific output as it considers both quality and quantity of scientific output.

Citation connects scientific works and determines their scientific value. Citation is used as an indicator for evaluating the scientific development of scientific associations and fields and the scientific value of published papers (4). By connecting scientific papers, citation makes researchers inform of the previously-published related works (5). Such connection shows the relatedness of the paper at hand with previously published papers in the scientific network (6). Citation is a scientific indicator for measuring the scientific quality of any research paper, too (7).

Many scientometric studies have been conducted in Iran and other countries. Literature review shows that since scientific indicators such as impact factor (IF) and h-index are used for citation analyses by most scientific associations, their completeness and accuracy is challengeable. In other words, suggesting other complementary indicators during recent years clearly shows that main scientific indicators are problematic in depicting the whole picture of the scientific production.

There is no research inclusively conducted on the factors affecting the citability of scientific papers. In few researches, the effect of some variables on papers’ citability has been considered (8). showed that the journal impact factor is not at work in citation rate of papers, especially those with high-prestige authors. Country contribution does increase the citation rates (9, 10). Hopewell and Clarke found that high-reprint articles of the Lancet received more citations than smaller-reprint articles of the journal (11). Yegros-Yegros et al. argued that interdisciplinary articles encounter a challengeable dualism in which multidisciplinary creates a creative work as well as weak work as to its content and scientific value, both affecting the citation rates (12). Multi-authored papers received more citations than single-authored papers (13).

The co-operation among authors and contributing institutions has a significant relationship with citation rate. More the authors interest in collaboration, more their papers receive citations from other papers (14). A study on papers published by FAO revealed that open-accessed journals received more citations than non-open-accessed journals (15). Open-access is one of ways for accessing millions of papers published in many scientific conferences worldwide (16). Davis and Fromerth studied 2,756 papers published in four mathematical journals during 1997- 2005 and found that papers archived in ArXive were cited 25% higher than ones did not accessed in it, especially in case of highly-cited papers (17). For investigating the high-quality of papers published in peer-reviewed journals (18), studied the rates of citations received by accepted vs. rejected manuscripts in these journals, but published by other journals. They found that the accepted papers in pee-reviewed journals received more citations and concluded that these journals could select appropriate papers for publication. In a study on 300 top papers published in economic journals, Moosa found that multi-authored papers have not significantly received more citations than single-authored ones (19).

Each researcher studied one or two variables affecting the citability of scientific papers. No researcher has comprehensively studied the factors affecting the citability. In this study, we aimed to identify the factors affecting the citability of papers in medicine based on the evidence-based approach and answer the following questions:

Does the age of a paper affects its citability?

Does the type of a paper (original research, review, etc.) affect its citability?

Do the scientometric indicators (such as IF, journal rank and SNIP) of a paper affect its citability?

Does the number of contributing countries in authoring a paper affect its citability?

Does the number of references of a paper affect its citability?

Does the h-index of the first / corresponding author of a paper affect its citability?

Dos the accessibility of a paper (open-accessed vs. non-open-accessed) affect its citability?

Does the indexing institute of a paper affect its citability?

MATERIAL AND METHODS

In July 2016, a non-restricted search was done in Scopus under the keyword “medicine” and 878,615 papers were retrieved. As in some scientometric indicators such as IF, at least two years of interval is needed for receiving citations, the searched results were refined and the years 2015-2016 were excluded. The retrieved papers were ranked by their citation count. The first 100 highly-cited papers were identified as Group 1. Considering the restriction of the search engine in showing retrieved records (maximum 2000 records), 100 papers at the end of the list were selected as lowly-cited papers and identified as Group 2.

A researcher-made checklist was used for data collection. It included 19 variables in three main groups:

Variables relating to the author(s), including the country of the first/corresponding author, the number of contributing countries, the h index of the first/corresponding author, the number of papers published by the first/corresponding author;

Variables relating to the paper, including the number of citations, paper age, the number of references, the number of authors, and paper type;

Variables relating to publishing journals, including its kind (open-access or non-open-access), IF, IF without self-citation, journal quartile, journal rank, indexing institute, SNIP and SJR.

As the information on journal rank and IF was not accessible via Scopus, all journals publishing the 200 papers of groups 1 and 2 was searched in JCR. Scopus citation report was used for calculating the number of the first/corresponding author’s h index and published papers. When the corresponding author was not mentioned, the first author conceived as the corresponding author. For determining whether or not the studied papers were indexed in other databases including WoS and PubMed, all of them were searched in these two citation databases. If a paper had a PMID in its bibliographic information, it was not searched in PubMed.

All data needed for completing the checklist were collected manually and direct observation. After data collection, the variables were defined in SPSS and needed data were entered. Some approaches of descriptive and inferential statistics were used for data analysis.

RESULTS

The mean rate of citations received by the studied papers was 1594.27±1816.61. The mean age of the papers was 14.66±9.49 years. The mean IF of publishing journals was 19.48±17.69. The mean rates of h-indices belonged to the first author and the corresponding author were 47.82±32.88 and 54.65±35.41, respectively. Table 1 shows the descriptive statistics of the studied variables.

Table 1.

Descriptive statistics of the studied variables in all studied papers.

Cahracteristic	N	Minimum	Maximum	Mean±SD
Received Citations	200	322	12607	1594.27±1816.61
Paper Age	200	2	51	14.66±9.49
References	177	2	1478	115.20±173.37
Authors	196	1	83	9.19±12.18
Impact Factor (IF)	193	.31	137.57	19.48±17.69
IF without Self-citation	193	.31	137.11	18.87±17.52
Journal Rank	193	1.12	207.25	12.03±22.76
First/Corresponding Author’s Country	195	1	15	1.77±2.12
First Author H-Index	196	1.00	185.00	47.82±32.88
Corresponding Author H-Index	196	1.00	185.00	54.65±35.41
First Author’s Docs	196	1	2263	211.60±268.60
Corresponding Author’s Docs	196	1	2263	245.81±274.70
SJR	191	.10	32.24	8.22±7.26
SNIP	190	.26	265.00	6.31±19.45

As shown in Table 2, 188 (94%) papers were published in non-open-accessed journals. 60%, 24%, and 11% of corresponding authors were from America, Africa and Asia, respectively. Most first authors were from America and Africa. 54% and 43.5% of the papers were original research and review articles, respectively.

Table 2.

Frequency (%) of quantitative indicators of the study

Cahracteristic	variable	Frequency (%)
Journal Kind	open access	12(6)
	non-open access	188(94)
Corresponding Author’s Country	America	120(60)
	Europe	49(24.5)
	Asia and other	11(5.5)
Q	Q1	173(86.5)
	other	20 (10)
First Author’s Country	Usa	135(67.5)
	Europe	48(24)
	Asia and others	13(6.5)
Paper Type	review	87(43.5)
	original	108(54)
	other	5(2.5)
Indexing Institute	PubMed	8(4)
	ISI	4(2)
	both	188(94)

For determining the possible relationship of citation rate with variables such as journal type and the first/corresponding author’s country, journal subject quartile was used. K2 test was used for determining the possible relationship between citation rate and the corresponding author’s country. Fisher’s exact test was used for determining the possible relationship between the citation rate and paper type. Mann-Whitney test was used for comparing the citation rates of the two paper groups as to variables such as paper age, the number of co-authors, the number of references, journal IF, journal IF without self-citation, journal rank, contributing countries, the h indices of the first and/or the corresponding author, the number of documents produced by the first/corresponding author, SNIP and SJR (Table 3).

Table 3.

Comparing characteristics of the two paper groups (highly-cited vs. lowly-cited)

Cahracteristic	variable	High citation (N=100)	Low citation (N=100)	p-value
Journal Type	Open access (%)	7 (7)	5(5)	0.552
	Non-open access (%)	93 (93)	95 (95)
Paper Age	Mean±SD (median)	14.13±18.81(12)	15.19±10.14(12)	0.855
References	Mean±SD (median)	140.77±221.19(53)	88.75±97.39(62)	0.548
Authors	Mean±SD(median)	10.73±13.56(6)	7.68±10.51(5)	0.072
Corresponding Author’s Country	Usa(%)	63(66.3)	57(67.1)	0.159
	Europe(%)	29(30.5)	20(23.5)
	Asia and other(%)	3(3.2)	8(9.4)
IF	Mean±SD(median)	23.63±15.30(18.39)	15.28±18.99(8.16)	<0.001
IF without Self- citation	Mean±SD(median)	22.98±15.23(17.93)	14.72±18.74(7.74)	<0.001
Journal Rank	Mean±SD(median)	6.12±9.83(3.15)	18.00±29.62(5.18)	<0.001
Q	Q1 (%)	94(96.9)	79(82.3)	<0.001
	other (%)	3 (3.1)	17(17.7)
First Author’s Country	America (%)	65(67)	70(70.7)	0.059
	Europe (%)	29(29.9)	19(19.2)
	Asia and others (%)	3(3.1)	10(10.1)
Corresponding Author’s Country	Mean±SD (median)	1.95±2.06(1)	1.59±2.15(1)	0.008
Paper Type	review (%)	45(45)	42(42)	0.358
	original (%)	51(51)	57(57)
	other (%)	4(4)	1(1)
Indexing Institute	PubMed (%)	3(3)	5(5)	0.894
	ISI (%)	2(2)	2(2)
	both (%)	95(95)	93(93)
First Author’s H-Index	Mean±SD (median)	57.39±36.80(51)	38.45±25.41(36)	<0.001
Corresponding Author’s H-Index	Mean±SD (median)	65.16±36.80(63)	44.36±31.25(38)	<0.001
First Author’s Docs	Mean±SD (median)	249.46±270.62(163)	174.51±262.69(103)	0.015
Corresponding Author’s Docs	Mean±SD (median)	292.25±272.35(212)	200.31±270.68(136)	<0.001
SJR	Mean±SD (median)	10.05±7.51(6.99)	6.34±6.52(3.69)	<0.001
SNIP	Mean±SD (median)	8.4±26.68(4.91)	4.18±5.73(2.5)	<0.001

As Table 3 shows, journal type, paper age, number of references, number of authors, paper type, and indexing institute had not any significant relationship with citation rates of the papers (p>.05). The other variables, including among others, journal IF, journal rank, journal quartile, h index of the first/corresponding authors, the number of documents produced by the first/corresponding author, SJR and SNIP had significantly positive relationship with the citation rates of the studied papers (P<0.05).

DISCUSSION

Many factors are at work in the citability of scientific papers that studying them are important as to their influence on the citation rate. These factors may be different in scientific disciplines. In the field of medicine, it appears that the kind of journal access (open access vs. non-open access) does not affect the citability. The possible reasons of the finding needs further research as previous research showed a positive relationship between open access papers and their citability in some non-medical fields (15, 20, 21).

There was no relationship between paper age and citation rate. This depends on the nature of scientific disciplines and fields. The reason of the finding may be that because of ever-changing nature of information in medicine, there is low tendency to cite old papers and authors tend to cite newly-published works in the field (7).

In this study, there was no significant relationship between the number of references and that of citation received. The possible relationship is challengeable and some researchers argue that more references in a paper do not mean that it will receive high citation (7). Studying co-authorship and co-citation relations can be beneficial to this aspect.

We found that the number of authors had not any effect on paper citability. It is expected that the quality of multi-authored papers is high and they receive more citations. As there is contradictory findings in this aspect (22, 23), achieving appropriate results needs further research.

There was not any citation bias in citing papers published in different geographical areas in this study. Factors other than geographical one can be effective in citing medical papers. This finding is not in line with the finding that more productive countries receive more citations (24).

There was a significant relationship between paper citability and publishing journal IF-with or without taking self-citations into account. This finding is justifiable considering the definition and functionality of IF (25). Considering a significantly positive relationship between paper citability and journal rank, it can be said that this relationship reflects the function of IF as high IF means high journal rank (26). Similar to our study, García et al. found that papers published in the journals included in the first subject quartile had more citation rate (27).

Since we did not find any significant relationship between paper citability and paper type in medical field, other researcher found that review articles receive more citations due to their nature (28).

As most studied papers were indexed in both ISI and PubMed, there was no relationship between paper citability and their being indexed in each of these citation databases. Another study can be done for comparing papers indexed in ISI vs. PubMed.

We found a significant relationship between paper citability and the first/corresponding author h-index. As h-index manifests authors’ ability to being cited, the co-authorship of the first/corresponding author with authors with high h-indices can increase the citability of documents co-authored (29).

The positive relationship was seen between the number of documents produced by the first/corresponding author and the number of citations received by his/her paper at hand. However, the number of citations received by previously published papers of the first/corresponding author did not studied here. It is notable that in different scientific fields, multi-authored papers receive more citations than single-authored papers (30).

CONCLUSION

Our study confirmed the relationship between paper citability and SJR on one hand and SNIP on the other hand. It can be said that both indicators are disciplinary-based and have high visibility. Subject connection and visibility are of factors affecting the citability in this regard.

Determining and explaining the factors affecting citability of papers published in the field of medicine are important for determining highly-cited authors, works and journals in the field. Ones administrating medical research can use these criteria in macro and micro levels. It is appeared that citability factors at work in author, paper and journal levels and clearly defining them and accurately measuring their influence need further large-scaled researches in the field.