Predictors of citation rates for research publications in Neurosciences.

OBJECTIVES: To identify the predictors of citation rates for research publication in Neurosciences.
METHODS: All original articles including meta-analyses (MAs) and systematic reviews (SRs) that were published in Neurosciences during 2011 to 2019 were reviewed. The impact of several predictors on citation rates was assessed using correlation coefficient and mean difference tests.
RESULTS: This study examined 231 articles. The mean article citation number was 11.6. The correlation analysis showed a significant association between citation rates and duration from publication in years (p<0.0001), sample size (p<0.0001), study design (p=0.0353), and level of evidence (LOE) (p=0.03). The comparative analysis showed significantly more citations for articles that were published 6-10 years ago (p<0.0001), had a sample size >91 (p=0.0359), were randomized controlled trials (p=0.0353), MAs and SRs (p<0.0001), and level of evidence (LOE)-I (p=0.0004). Retrospective case series had significantly lower citations. The higher and lower citation numbers for publications from Iran and rehabilitation, respectively, may have been influenced by the duration from publication.
CONCLUSION: The most significant predictors of citation rates for Neurosciences publications were the age of articles, population size, study design, and LOE. Awareness of the predictors of citation rates may help researchers enhance the academic impact of their work. Copyright: © Neurosciences.

The number of citations an article receives, also referred to as the citation rate, is arguably the most important indicator of its impact and clinical significance. Identification of the predictors of citation rates is useful for researchers and journals in order to boost the impact of their work. This topic has been the subject of several studies in recent years. Most such publications have focused on identifying the predictors of citation rates in the literature relating to certain fields, including neurosurgery, spine, orthopedics, neurosciences, plastic surgery, urology, radiology, and glioblastoma multiforme trials. Several variables were identified as potential citation rate predictors. Their significance however, differed between various reports. These included sample size, study design, level of evidence (LOE), topic, journal IF, field, and publishing country, as well as the number of authors, institutions, countries, and references listed on the paper. In a recent study, several bibliometrics besides journal IF were identified as predictors of citation counts in neurosciences journals. These included the Eigenfactor score, cited half-life, immediacy index, and number of articles. Furthermore, in the era of digitization, open access, and social media activities are increasingly recognized as drivers of citation activities.

Neurosciences Journal, which is referred to as Neurosciences (Riyadh) in PubMed and will heretofore be referred to as Neurosciences in this article, is an open access, peer-reviewed, quarterly publication that was launched in 1996. The journal is published by the Armed Forces Hospital, Riyadh, Kingdom of Saudi Arabia (KSA). The Scimago Journal and Country Rank (SJR) web site, ranked KSA as fortieth in the world and fourth in the Middle East based on the total citations of its clinical neuroscience literature during 1996- 2020. However, to date, the factors that influence citation rates have not been addressed in Neurosciences or in any other Saudi medical journal. The objective of this study is to identify the predictors of citation rates for Neurosciences publications. It is hoped that the awareness of the factors that impact citations will help researchers enhance the impact of their work.

Methods

The Neurosciences web site was searched on 13th September 2021 for suitable articles. The inclusion criteria were all original research publications in Neurosciences from 2011 to 2019. The duration was chosen to find an acceptable number of papers and to give the more recent publications an adequate period to get cited. Articles reporting meta-analysis (MAs) and systematic reviews (SRs) were included as they frequently reflect higher LOE. Conversely, review articles, case reports, brief reports, and clinical notes were excluded. Using the full text, the selected articles were analyzed, and relevant data were collected. To minimize bias, two of the authors conducted independent analysis. The 2 lists were compared, and any discrepancies were resolved by consensus. The article citation numbers were determined using Google Scholar. In view of the regular changes in the citation numbers with time, the search findings on a single day (September 15, 2021) were used for the analysis in this study.

The following data were collected: duration from publication in years, sample size, study design [randomized controlled trials (RCTs), prospective, MAs and SRs, experimental, cross sectional, survey questionnaire, case-control, and retrospective case series], study LOE [I- IV], study field [clinical, or non-clinical], publishing specialty [neurology, neurosurgery, psychiatry, pediatric neurology, rehabilitation, basic neuroscience, and miscellaneous], publishing country [KSA, Turkey, Iran, China, and miscellaneous], authors academic affiliation [university, or non-university], number of authors, number of centers, and number of references. The LOE was ranked according to Oxford’s LOE scale as follows: I: RCTs and MAs of RCTs, II: prospective studies and SRs of prospective studies, III: case-control studies, IV: cross-sectional, retrospective case series and SRs of case series, and undetermined: experimental and survey questionnaire studies. The publishing specialty, country, and authors’ academic affiliations were based on the corresponding author’s data.

The data were analyzed by correlating articles’ citation numbers with various predictors’ findings using the Pearson correlation coefficient test in Social Sciences Statistics. The data were analyzed further by comparing the different predictors’ subgroups using the mean difference test in MedCalc. The median was taken as a cut-off point in the numerical parameters. In all statistical analyses, significance was determined when p≤0.05.

Results

A total of 231 Neurosciences publications were selected. At the time of the analysis, the articles had a total of 2,669 citations. The mean (±SD) article citation number was 11.6 (±14.2). The median (range) article citation number was 7 (0-129). Ten (3.8%) articles did not receive any citations, while 153(66.2%) articles received ≤10 citations. The articles citation distribution was skewed with 4 (1.7%) receiving 335 (12.6%) citations, 17 (7.4%) receiving 825 (30.9%) citations, and 78 (33.8%) receiving 1,954 (72.3%) citations. All articles except 2 were published by one country; hence, the number of countries was not included as a citation rate predictor. The median (range) findings were duration from publication: 6 (2-10) years, sample size (reported in 222 studies only): 91 (5-6777), number of authors: 5 (1-17), number of centers: 1 (1-8), and number of references: 25 (5-61).

summarizes the findings relating to the other predictors. summarizes the correlation analysis results. A significant correlation was observed between citation rates and the duration from publication in years (R=0.3423) (p<0.0001), sample size (R=0.467) (p<0.0001), study design (R= -0.1408) (p=0.0353), and study LOE (R=-0.1428) (p=0.03). None of the other predictors (study field, publishing specialty, publishing country, authors academic affiliation, number of authors, centers, and references) reached significance.

Table 1

- Findings relating to several citation rate predictors in the selected Neurosciences publications.

Factors/Features	Articles (%)
Duration from publication
10 years	29(12.6)
9 years	26(11.3)
8 years	24(10.4)
7 years	22(9.5)
6 years	24(10.4)
5 years	25(10.8)
4 years	23(10)
3 years	28(12.1)
2 years	30(13)
Study design
RCTs	10(4.3)
Prospective	17(7.4)
MAs and SRs	12(5.2)
Experimental	27(11.7)
Cross-sectional	65(28.1)
Survey questionnaire	15(6.5)
Case control	14(6.1)
Retrospective case series	71(30.7)
Study level of evidence (LOE)
I	15(6.5)
II	18(7.8)
III	14(6.1)
IV	144(62.3)
Undermined	42(30.7)
Study field
Clinical	204(88.3)
Non-clinical	27(11.7)
Publishing specialty
Neurology	77(33.3)
Neurosurgery	34(14.7)
Psychiatry	24(10.4)
Pediatric neurology	19(8.2)
Rehabilitation	20(8.7)
Basic neuroscience	30(13)
Miscellaneous*	27(11.7)
Publishing country
KSA	91(39.4)
Turkey	48(20.8)
Iran	33(14.3)
China	25(10.8)
Miscellaneous**	34(14.7)
Authors academic affiliation
University	182(78.8)
Non-university	49(21.2)

*Miscellaneous specialties (27 articles) [Medicine (others): 7, Family and Community Medicine: 6, Radiology: 6, Public Health: 2, Anaesthesia: 2, Epidemiology: 2, Dentistry: 1, Nursing: 1].**Miscellaneous countries (34 articles) [Iraq: 7, Jordan: 6, India: 3, Kuwait: 2, Pakistan: 2. Egypt: 2, Brazil: 2, Oman: 1. Tunisia: 1, Morocco: 1, Nigeria: 1, Rwanda: 1, USA: 1, Canada: 1, Malaysia: 1, Taiwan: 1, Thailand: 1]

Table 2

- Correlation analysis between citation numbers and findings relating to several predictors using pearson correlation coefficient (R).

Predictors	R-value	P-value
Duration from publication	0.3423	<0.0001*
Sample size	0.467	<0.0001*
Study design	-0.1408	0.0353*
Study Level of Evidence	-0.1428	0.03*
Study field	-0.0593	0.3696
Publishing specialty	0.0414	0.5313
Publishing country	0.0161	0.8077
Authors academic affiliation	0.0158	0.8112
Number of authors	-0.0141	0.8312
Number of centers	0.0489	0.4595
Number of references	0.0676	0.3063

*p-values ≤0.05 are significant

summarizes the comparative analysis results. Significantly higher citation rates were observed in articles that were published 6-10 years ago, compared to 2-5 years ago [16.4 versus (vs.) 5.8] (p<0.0001), had sample size >91 compared to ≤91 (13 vs. 9.1) (p=0.0359), reported RCTs compared to other study designs (20.9 vs. 11.1) (p=0.0353), reported MAs and SRs compared to other study designs (28.8 vs. 10.6) (p=<0.0001), had LOE-I compared to other LOE (24.1 vs. 10.6) (p=0.0004), and publications from Iran compared to those from other countries (21.2 vs. 10) (p<0.0001). Furthermore, significantly lower mean citation rates were observed in retrospective case series compared to other study designs (10 vs. 15.9) (p=0.0327) and publications by rehabilitation compared to those by other specialties (4.7 vs. 12.2) (p=0.0261). The influence of the age of the paper on the abovementioned findings was assessed. A significantly higher mean duration from publication was observed in articles published from Iran compared to those from other countries (7.6 vs. 5.7 years) (MD=-1.9) (p=0.0002). Further, a significantly lower mean duration from publication was noted in articles published by rehabilitation compared to other specialties (3.5 vs. 6.2 years) (MD: 7.5) (p<0.0001). The difference in mean duration from publication between all other subgroups did not reach significance.

Table 3

- Comparative analysis of the mean citation numbers (± SD) between the various subgroups in several predictors using mean difference (MD) test

Predictors/Variables	Articles Number	Mean Cites (±SD)	MD	P-value
Duration from publication (years)
2-5 years	106	5.8(±5.7)	10.6	<0.0001*
6-10 years	125	16.4(±17.5)
Sample size
≤91	110	9.1(±8.8)	-3.9	0.0359*
>91	112	13(±17.3)
Study design
RCTs	10	20.9(±14.3)	-9.8	0.035*
Other designs	221	11.1(±14.3)
Prospective	17	6.8(±8)	5.1	0.1621
Other designs	214	11.9(±14.8)
MAs and SRs	12	28.8(±37.6)	18.2	<0.0001*
Other designs	219	10.6(±11.5)
Experimental	27	9.8(±9.3)	2	0.5003
Other designs	204	11.8(±15)
Cross-sectional	65	11.2(±10.7)	0.5	0.8136
Other designs	166	11.7(±15.7)
Survey questionnaire	15	13.8(±11.5)	-2.4	0.534
Other designs	216	11.4(±14.6)
Case-control	14	14(±20.5)	-2.6	0.5146
Other designs	217	11.4(±14)
Retro- case series	71	8.5(±10)	4.4	0.0327*
Other designs	160	12.9(±15.9)
Study level of evidence (LOE)
I	15	24.1(±31.5)	-13.5	0.0004*
Other levels	216	10.6(±12)
II	18	7.1(±7.9)	4.9	0.1671
Other levels	213	12(±14.8)
III	14	14(±20.5)	-2.6	0.5146
Other levels	217	11.4(±14)
IV	144	10.6(±11.8)	2.6	0.1849
Other levels	87	13.2(±17.9)
Undetermined	42	11.2(±10.2)	0.4	0.8718
Other levels	189	11.6(±15.3)
Study field
Clinical	204	11.8(±15)	2	0.5003
Non-clinical	27	9.8(±9.3)

*p-values ≤0.05 are significant

Discussion

The SJR index expresses the number of citations articles received in a selected year by the number of documents published in the journal in the previous three years. In 2020, the SJR index for Neurosciences was 0.235. The journal was ranked 284th and 11th among clinical neuroscience journals in the world and the Middle East, respectively. It is generally accepted that article citation rates correlate well with the publishing journal IF. In this study, Neurosciences publications had mean citations of 11.6 and total citations of 2,669.

The distribution of citations was skewed, with a small percentage (1.7%) of articles receiving a disproportionately high portion of total citation numbers (12.6%). The unequal distribution of citation numbers for Neurosciences publications is not unusual and has been reported by other journals.

We observed a significant age effect on citation numbers for Neurosciences publications. The post-publication period here (median 6 years) was relatively longer than those of other reports (3-5 years). In general, older papers have longer exposure and are likely to receive more citations. However, the number of citations per year an article receives rises quickly in the first few years and decreases as time passes. This suggests that the exact post-publication duration maybe relevant in determining the impact of the article’s age on citation numbers. A significant correlation with sample size was observed confirming it as a definite predictor of citation rates for Neurosciences publications. Similar findings were reported by others. The most frequent study designs were retrospective case series and cross-sectional studies, while the most frequent LOE was IV. A significant correlation between citation numbers and study design, and LOE was observed. This is in agreement with reports that identified high LOE and study designs such as RCTs and MAs, as strong predictors of citation counts.

The majority of selected articles were linked to clinical fields, and the most frequently publishing specialties were neurology, neurosurgery, and psychiatry. No significant correlation was observed between citation rates and publishing specialty or study field. In the comparative analysis, however, the mean citation number for articles published by rehabilitation was significantly lower than those published by others. This may be related to the fact that the rehabilitation papers were significantly younger than those from other specialties (mean age 3.5 vs. 6.2 years). Nevertheless, citation rates may vary considerably between different subjects and topics of a discipline. The chance of being cited correlates with the number of papers published in different subjects. Therefore, papers from small fields may achieve fewer citations than those from more general fields.

The most frequently publishing countries were KSA and Turkey. No significant correlation was observed between citation rate and publishing country. In the comparative analysis, however, the mean citation number for articles published from Iran was significantly higher than those from other countries. This could be related to papers from Iran being significantly older than others (mean age 7.6 vs. 5.7 years). The variation could also be related to the well-recognized high country self-citation rate for Iran, which was reported as 36.57% (ranked 3rd in the world during 1996- 2017). The disparity in citation rates of articles published in different countries is well-known. Authors who are affiliated with certain countries achieve more citations than others. Privileged countries with strong scientific backgrounds and adequate financial support can conduct higher-quality research that gets published in high-IF journals and receives more citations.

The association between the authors’ academic rank and citation rates is not supported in the literature. In this study, authors’ university affiliations had no significant effect on citation rates. This could reflect quality research contributions by health care facilities that are not affiliated with universities. The citation numbers of Neurosciences publications were not impacted by the numbers of authors, centers, countries, and references, which is in agreement with the findings of previous research. However, few publications identified the number of authors, number of organizations, and number references as factors that could influence citation rates.

The study has several limitations. First, the study was reliant on the accuracy of the Neurosciences web site. The citation numbers were taken from Google Scholar which is arguably be less comprehensive than the Scopus or Web of Science databases. Article citations were taken at a certain point that is likely to change. The number of selected articles could be considered small. The exclusion of review articles and other reports could be interpreted as selection bias. The wide study duration may have influenced citations in favor of older papers. There may have been potential errors in the subgrouping of the study design and LOE. Defining the specialty and academic affiliation based on the corresponding author may not reflect all authors of multi-disciplinary papers. The impact of self-citation on the citation numbers was not examined.

In conclusion, citation rates for Neurosciences publications were significantly impacted by the age of the paper, sample size, study design, and LOE. The citations were not affected by the study field, specialty, publishing country, author academic affiliation, number of authors, centers, or references. The citation rates for articles from Iran and concerning rehabilitation may have been influenced by their duration from publication. Awareness of the factors that influence citations may be help researchers enhance the impact of their work.

Predictors/Variables	Articles Number	Mean Cites (±SD)	MD	P-value
Publishing specialty
Neurology	77	12(±17.6)	-0.6	0.7673
Other specialties	154	11.4(±12.7)
Neurosurgery	34	8.9(±7.8)	3.1	0.2496
Other specialties	197	12(±15.3)
Psychiatry	24	13(±10.8)	-1.6	0.6081
Other specialties	207	11.4(±14.8)
Pediatric neurology	19	13.4(±12.4)	-2	0.5636
Other specialties	212	11.4(±14.6)
Rehabilitation	20	4.7(±4)	7.5	0.0261*
Other specialties	211	12.2(±14.9)
Basic neurosciences	30	11.5(±9.7)	0.1	0.9718
Other specialties	201	11.6(±15)
Miscellaneous	27	16.4(±21.6)	-5.5	0.0635
Other specialties	204	10.9(±13.2)
Publishing country
KSA	91	11.4(±14.5)	0.2	0.9185
Other countries	140	11.6(±14.5)
Turkey	48	8.5(±10)	3.9	0.0956
Other countries	183	12.4(±15.3)
Iran	33	21.2(±23.4)	-11.2	<0.0001*
Other countries	198	10(±11.7)
China	25	9(±7.7)	2.9	0.343
Other countries	206	11.9(±15)
Miscellaneous	34	8.8(±6.8)	3.2	0.2353
Other countries	197	12(±15.4)
Authors academic affiliation
University	182	11.7(±15.3)	-0.5	0.8308
Non- university	49	11.2(±11.1)
Number of authors
≤5	150	11.8(±15.6)	-0.8	0.6889
>5	81	11(±12.1)
Number of centers
1	214	11.5(±14.7)	1.1	0.7635
>1	17	12.6(±11.4)
Number of references
≤25	117	10.2(±10.5)	2.8	0.1422
>25	114	13(±17.6)

*p-values ≤0.05 are significant