Barriers to Impact Factor Growth in Two Major Psychiatry Journals in India.

BACKGROUND: Despite exponential growth in Indian research, Indian journals have low impact factors. A previous study by one of the authors (CA) of this paper showed that articles published in the Indian Journal of Psychiatry (IJP) under-referenced previously published relevant papers in the same journal. Based on this, we decided to investigate the citation characteristics of contemporary scientific articles published in the Indian Journal of Psychological Medicine (IJPM).
METHODS: The citation characteristics of scientific articles published in 2018 (Vol 40, issues 1-6) in the IJPM were examined to determine how well the authors cited relevant articles published during the past ten years in the IJPM and the IJP.
RESULTS: There were 145 and 142 citation-worthy articles in the IJPM and the IJP, respectively; of these, 85.5% and 65.5%, respectively, had not been cited.
CONCLUSIONS: Authors publishing in the IJPM under-reference previous relevant research published in the IJPM and IJP. This suggests unawareness of, deliberate disregard of, or even disdain for prior Indian research in the field. Additionally, if Indian researchers do not cite previous Indian research in the field published in Indian journals, the citation metrics of Indian journals will not grow.

Key Messages:

For every paper published in the IJPM during 2018, there were very few citation-worthy articles from the same journal and from the IJP during the past ten years (barring the past one year). Of these citation-worthy articles, about 85% of articles from the IJPM and 65% of articles from the IJP were omitted. We suggest that Indian researchers make efforts to acquaint themselves with and cite relevant regional research to overcome the vicious cycle of under-publication and under-citation.

A bulletin from the Centre for Scientific and Industrial Research in 2014 described an exponential increase in the publication of research in India, which is now one of the top 10 countries in the world in terms of the number of research papers published.[1] This increase in scientific productivity, however, has not been accompanied by a proportionate increase in citations, with Indian journals continuing to have lower impact factors in comparison with publication indices of international journals.[2]

In 2010, a review in the Indian Journal of Psychiatry (IJP) observed that much of Indian psychiatric research replicates Western research and cites it extensively, even when published in Indian journals, while often neglecting to cite related, relevant, and contemporary Indian scientific work.[3] A study examining the contribution of Indian psychiatrists to PubMed-listed mental health literature between 1995 and 2013, published in the Indian Journal of Psychological Medicine (IJPM), similarly noted that Indian psychiatry’s contribution to indexed literature remained modest, and despite a rising number of indexed publications, Indian psychiatric journals continued to have low citation indices.[4]

A previous study by one of the authors (CA) of this paper showed that articles published in the IJP under-referenced previously published relevant papers in the same journal[5]; four years later, there was little change.[6] Based upon this, it was hypothesized that articles published in the IJPM would, similarly, demonstrate a continued trend by Indian researchers in psychiatry to under-reference Indian scientific literature, and more specifically, literature published in the IJPM and IJP, which are the two leading psychiatric journals in the country.

The primary aim of this study was to investigate the citation of previously published and relevant Indian research by Indian researchers currently publishing in the IJPM. The secondary aim of the study was to investigate self-citation by Indian researchers currently publishing in the IJPM. The primary objective was to examine the citation patterns of original research published in one year in the IJPM with respect to articles published in the preceding ten years of the IJPM and IJP. The secondary objective was to examine self-citation patterns in original research published in the same year of the IJPM with respect to articles published in any journal.

Materials and Methods

This study was conducted in 2019. No ethical review was considered necessary because the study examined secondary data that did not involve human subjects. In this study, issues 1–6 of Volume 40 of the IJPM, spanning the calendar year from January to December 2018, were chosen for recency. All scientific articles published in these six issues were extracted and divided into five categories; editorial/review/viewpoint, original research/brief communication, case series/case report, commentary on previously published research, and letters to the editor that did not fall into a previously listed category. Articles excluded were learning curve articles and book reviews, as these were intended to be didactic and were not required to cite relevant research. The references of each article were examined for citations of Indian research.

Our plan was (a) to extract all original research published in one calendar year (2018) of the IJPM and examine their references to identify the total number of citations from the IJPM and the IJP, including articles we defined as citation-worthy (Box 1) (b) to manually search through the archives of the IJPM and the IJP to extract articles defined as citation-worthy (c) to calculate the ratio of citation-worthy articles referenced to those excluded, (d) to extract the number of self-citations by authors in 2018.

One of the authors (MD) performed a manual search of the electronic archives of the IJPM and the IJP. All scientific articles published in 2018 were read extensively. The references of these articles were then examined and compared with the archives of the IJPM and the IJP to identify citation-worthy articles, as defined below. We believed that a manual search, though painstaking, would be more sensitive than an electronic search based on keywords. In case of doubts regarding the citation-worthiness of an article, a discussion was held between both authors (MD and CA).

Citation-worthy articles were defined as relevant research published in the IJPM and IJP in the last ten years (dating back to January 2008), excluding the previous calendar year (January to December 2017); this is further explained in Box 1. This definition was based upon previous studies with similar methodology, by one of the authors.[5, 6] We chose to make the criterion of “citation-worthiness” independent of the quality of previously published research as we believed that it would be better to cite and critique poorly conducted but locally relevant research than to ignore it altogether. Articles published in 2017 were excluded from consideration as the submission, review, and publication process for articles can take up to a year. So, the authors would not have had access to these articles for reference.[5, 6] Self-citation was defined as any publication which had at least one author in common with the paper under consideration; this is further explained in Box 2. The descriptors examined for each published article are presented in Box 3. Data are represented using the mean (standard deviation), and the median, where the distributions were skewed.

Results

There were 134 articles published in the IJPM in 2018. Excluding the learning curve articles (3 articles) and book reviews (1 article), other 130 articles were eligible for selection for the study. These 130 articles included ten editorials, reviews, and viewpoints, 61 original research articles (including brief communications), 36 case series and case reports, 17 commentaries on original research published in the previous issues of the IJPM, and 6 other letters to the editor (including errata). Except for a case report from Sri Lanka, all articles were by Indian authors.

For the 130 articles examined, the number of citation-worthy articles identified in the previous issues of the IJPM and IJP was 145 and 142, respectively. However, of these citation-worthy articles, only 21 articles from the IJPM and 49 articles from the IJP were actually cited. A total of 48 IJPM (including 12 self-citations) and 96 IJP (including six self-citations) articles were cited. There were 57 self-citations in all, across all journals.

The percentage of cited to citation-worthy articles was 14.5% (i.e., 21/145) for the IJPM and 34.5% (i.e., 49/142) for the IJP. That is, about 85% and 65% of citation-worthy articles were omitted from the IJPM and the IJP, respectively.

The mean (standard deviation) citation-worthy articles from the IJPM was 1.12 (1.69), and that from the IJP was 1.09 (2.03). Of these, 0.16 (0.51) IJPM and 0.38 (0.92) IJP articles were actually cited.

The mean (standard deviation) total articles cited from the IJPM was 0.37 (0.71) and from the IJP was 0.74 (1.94). These included 0.09 (0.29) IJPM self-citations and 0.05 (0.27) IJP self-citations. The mean (standard deviation) total self-citations, including self-citations from all other journals, was 0.44 (1.06).

Interestingly, the median value for all the above variables was 0. The inter-quartile range was not examined as the median value was 0. A full list of the IJPM 2018 articles, citation-worthy articles, articles cited, and citation-worthy articles not cited is available as an online-only supplementary file. The results of our study have been summarized in Table 1.

Box 1.

Operationalization of the Criterion “Citation Worthiness”

1. Citation worthiness was defined to include any publication in the IJP or the IJPM during the past ten years (barring the past one year), examining a similar research question in the same area. For example, a paper on treatment outcomes in patients with alcohol dependence syndrome (ADS) need not cite a previous publication on domestic violence in spouses of patients with ADS. However, it must cite a previous publication on impulsivity and time to relapse in patients with ADS because the time to relapse is a treatment outcome.

2. Citation worthiness was also defined to include any publication in the IJP or the IJPM in the past ten years (barring the past one year) that was similar to an article already cited in the references of the paper under consideration. For example, if a study on treatment-resistant schizophrenia cites a foreign publication on the genetics of schizophrenia, then it should also cite a similar publication in the IJP or the IJPM on the genetics of schizophrenia.

Discussion

There are two salient findings. One is that for each paper published in the IJPM during 2018, there were actually very few citation-worthy articles from the same journal and the IJP during the past ten years (barring the past one year); the mean value was approximately 1.0 for both journals. This indicates the paucity of continuity of Indian research in a single area, at least as judged by research published in these two journals. It may be that Indian researchers, at least in psychiatry, develop and investigate a hypothesis in relative isolation, without an adequate review of literature and appreciation of what has already been done and achieved in the field. This would be problematic both because it would be difficult to contextualize current Indian research and because it may lead to redundancy in research (e.g., multiple and overlapping candidate gene studies in schizophrenia).

A limitation of this conclusion is that it applies only to papers published in the IJPM during 2018. We note, however, that there were two fields in which, subjectively, there seemed to be a continuity of research: schizophrenia and alcohol use disorders.

The other salient finding is that about 85% and 65% of citation-worthy articles were omitted from the IJPM and the IJP, respectively. This is a matter of concern because it suggests that persons who publish in a specific area do not seem to build upon what has already been done by other investigators in the country; at the worst, it suggests that investigators are not even aware of the previous research.

A limitation of the latter conclusion is that the authors may have examined a different angle in their study and may, therefore, have considered that the previous publication was irrelevant for citation. Whereas we accept that there could be some subjectivity in choosing whether or not a previous article needs to be cited, there were many examples where there was a close similarity between previous and present papers with the previous paper having been omitted from the citation.

We have attempted to address the potential for subjectivity in the definition of “citation-worthiness” by operationalizing the definition (Box 1) to identify Indian studies examining a similar research question or similar to an international publication cited by the authors in their references. This operationalization may have led to the over-inclusion of articles. However, it is worth noting that despite the potential for over-inclusion, the median number of both citation-worthy and actually cited articles is zero.

Thus, an IJPM publication in 2018, examining the prevalence of unrecognized macrocytosis in patients with their first episode of psychosis or depression, bore close similarity to, but did not reference, an earlier IJPM publication which examined the prevalence of Vitamin B12 deficiency in patients presenting with neuropsychiatric symptoms.[7, 8] Similarly, an IJPM publication in 2018, examining the prevalence of burnout in Indian psychiatrists, could have cited an earlier IJPM publication which identified compassion fatigue and burnout in Indian clinicians, of whom psychiatrists are a subset.[9, 10]

The omission of citation of 65%–85% of citation-worthy articles in the IJP and the IJPM is of further concern because the impact factors of journals grow only when articles published in the journals are cited. Effectively, what seems to be happening is that authors publish in Indian journals but cite articles from other journals, leading to the question of whether Indian researchers truly read or respect Indian research.[5, 6]

In this context, it is noteworthy that there were 57 self-citations in 2018, which suggests the whimsical conclusion that Indian researchers are aware of their own previous work but not of the work of their Indian colleagues.

Earlier, similar studies have demonstrated a 1:2 ratio of cited to citation-worthy papers in the IJP. Our study indicates a cited to citation-worthy ratio of 1:7 for IJPM publications and 1:3 for IJP publications.[5, 6] This is indicative of a worrisome decline in the citation of Indian psychiatric research by Indian researchers.

Why might such under-referencing happen? One possible explanation is that international journals have larger reputations and higher impact factors. So, research published in such journals is considered to be of higher quality or of greater credibility than research published in Indian journals, most of which do not have an impact factor or have a low impact factor. So, Indian psychiatric researchers may prefer to cite international publications over contemporary, published Indian literature. Another possible explanation is that Indian researchers in psychiatry may simply be unaware of Indian research because of limited discussion of such research in contemporary teaching and professional practice. It is worth noting that discussions in post-graduate teaching programs, such as seminars and journal clubs as well as professional conferences, often prioritize Western literature published in international journals over Indian research. While acquaintance with international peer-reviewed literature is imperative, it may often come at the cost of overlooking Indian research.

Authors may have been restricted by limitations placed upon the number of references in some types of scientific articles, such as case reports and letters to the editor (up to 10 references) in the IJPM. However, there have been instances where the permitted number of references have been exceeded.[11, 12] Hence, we presume that this is unlikely to be a major barrier to the citation of Indian research.

The implications of this under-referencing are several. Indian researchers and Indian journals are cauin a vicious cycle of under-publication and under-citation, which adversely affects the professional growth of the Indian researcher and the impact of Indian journals in the field.[13] In addition, cultural and epidemiological nuances of Indian psychiatry that are inadequately captured in the Western literature would, therefore, be neglected.

Based on the findings of this study, and in the context of related previously published literature, we suggest that Indian researchers make efforts to acquaint themselves with and cite relevant regional research and that Indian research be made an essential part of the academic curriculum in postgraduate education and professional society meetings.

Reviewers and editors of Indian journals must draw the attention of authors to neglected Indian research in manuscripts that are under consideration for publication. We would like to note that the reviewers and editors of some journals often do suggest the inclusion of previously published research from the same journal, during the peer-review process. However, we would like to add the caveat that this is of benefit only when the references are “citation-worthy” and are of relevance and add benefit to the manuscript under review. It would be unethical if the intent behind this were to inflate the impact factor of journals without relevance to research. An alternative would be for suggestions to be made during the peer review process, independent of the publishing journal.

Our argument is that citation of relevant and contemporary Indian research by authors would help the research fraternity place current research in the context of what has already been done in the field in the country. To this extent, we believe a continuity of Indian research would be more beneficial than research questions developed and investigated in isolation, without an adequate review of Indian literature. Further, we make a case for the citation of Indian research irrespective of methodological rigor and quality of research, since it would serve better to cite and critique poor methodology rather than to omit the previous research altogether. That said, we acknowledge that in the case of some types of original research such as case reports, the restrictions placed upon the number of references would naturally merit the citation of well-conducted research over studies with poor research methodology. Similarly, journals may also choose to limit the number of citations due to space constraints, which may require the omission of studies of poorer quality.

Finally, for practical reasons, we limited ourselves to citations from the IJP and IJPM, the two leading journals in Indian psychiatry; we did not capture publications in other Indian or international journals. We limited ourselves to an examination of citation practices during 2018; there may be temporal variations in citation patterns in other years. We may have underestimated under-referencing by deliberately excluding from our purview the articles published during 2017.

Box 2.

Operationalization of the Criterion “Self-citation”

Self-citation was defined as the citation of a publication, which had at least one author in common with the paper under consideration, regardless of the journal of publication or the date of publication.a

aUnlike citation-worthy articles, the audit of self-citation was not limited to the IJP or IJPM or to a specified time period. IJP: Indian Journal of Psychiatry, IJPM: Indian Journal of Psychological Medicine.

Box 3.

Descriptors Extracted for Each Published Paper

1. Type of article (editorial/review/viewpoint, original research/brief communication, case series/case report, commentary on previously published research, and letters to the editor that did not fall into a previously listed category)

2. Country of origin (India, outside India)

3. Number of citation-worthy IJPM articles identified for that articlea

4. Number of citation-worthy IJPM articles actually cited

5. Total number of IJPM citations (regardless of when the cited article was published)

6. Number of IJPM self-citations

7. Number of citation-worthy IJP articles identified for that articlea

8. Number of citation-worthy IJP articles actually cited

9. Total number of IJP citations (regardless of when the cited article was published)

10. Number of IJP self-citations

11. Total number of self-citations (in all journals)

Table 1.

Summary Statistics of the Descriptors Extracted for Each Published Paper

Variable	Total	Range	Mean	Median	Standard Deviation
Citation-worthy articles in IJPM	145	0–9	1.12	0	1.69
Actually cited IJPM articles	21	0–4	0.16	0	0.51
Total IJPM articles cited	48	0–4	0.37	0	0.71
IJPM self-citations	12	0–1	0.09	0	0.29
Citation-worthy articles in IJP	142	0–11	1.09	0	2.03
Actually cited IJP articles	49	0–5	0.38	0	0.92
Total IJP articles cited	96	0–13	0.74	0	1.94
IJP self-citations	6	0–2	0.05	0	0.27
Total self-citations	57	0–4	0.44	0	1.06

aIn the past ten years, barring the past one year. IJP: Indian Journal of Psychiatry, IJPM: Indian Journal of Psychological Medicine.