Contribution of Indian Psychiatrists to PubMed Listed Mental Health Literature During 1995-2013: an Exploratory Study.

INTRODUCTION: Contribution of Indian Psychiatrists as publications in peer-reviewed journals listed with PubMed and their impact has not been studied. The aim of this manuscript is to assess such contribution using a new article level metric measure. The relative citation ratio (RCR) has been used to assess the quality, quantity, and impact of research output of Indian Psychiatrists.
MATERIALS AND METHODS: Publications by Indian psychiatrists in PubMed during 1995-2013 were collected, their RCR and associated factors estimated. The nationality of the journals, type of manuscripts, PubMed Central (PMC) visibility and the type of the journals were factored in. The data collected was analyzed. Descriptive statistics, Chi-square tests, correlations, and linear regression were performed. P ≤ 0.05 was taken as significant.
RESULTS: Using the criteria set, 1914 manuscripts were identified. Of the 1914 manuscripts, 1007 were cited at least once and among this, 40.7% were listed with PubMed while of the 907 non-PMC listed manuscripts, only 180 were never cited (P = 0.000). Of the 1032 manuscripts published in Indian journals, 474 were never cited while 214 of the manuscripts published with non-India based journals were never cited even once (P = 0.000). DISCUSSION: The difference in terms of manuscripts visibility in PMC, nationality of journals and article type in analysis indicate that there exists an innate difference between the cited and noncited manuscripts. The probable explanation behind this and its associated phenomenon are discussed.

INTRODUCTION

The science of studying citations and information of published literature, referred as bibliometrics[1], has critically pointed that Indian biomedical research output has been inadequate in terms of scope and quality in relative to India's size and capabilities.[12] To the best of our knowledge, there is very limited data on the impact of the contribution of Indian Psychiatrists to India and Global health research.[34567] Such data would help the policy framers and journal editors to pen future policies for the overall improvement of the contribution of the Indian mental health fraternity.

With publications being now made a part of the review process for academic pursuits such as grant awards, and promotions, a need to convert the impact of the publications from a subjective to objective measures arose. Quality of the scientific journals is often calculated by several entities such as Journal Impact Factor, Source Normalized Impact per Paper, H-index, and Impact per Publication. These measures have been postulated as a measure of the quality of the journals in which a particular manuscript has been published or the impact of a manuscript.[89] Each of such measures is plagued by several drawbacks as most of these measures do not consider the individual impact of the article. This necessitated to develop a new citation metric that is at an individual article level, field-normalized in a way that it measures a variety of journals (from low-reach to large reach) without any bias, compares the reach of the manuscripts to its peers, repeatable, easily reproducible, withstand the scrutiny of peer-review, freely accessible, and calculated in a transparent way. In search of such an endeavor, the scientists with the National Institutes of Health, in October 2015 advocated a new citation metric, called relative citation ratio (RCR). This ratio bases its foundation on the estimation of using the co-citation network of each article to the field and time-normalized by calculating the expected citation rate (ECR) from the overall sum of citation behavior of an article in question.[9] Furthermore, such an individual metric could facilitate general inferential statistics be applied for a group of manuscripts classified by certain criterion.

The aims of this manuscript are to analysis and present the contribution of the Indian Psychiatrists in the reputed, freely accessible, widely referred PubMed database using the RCR as a measure of the impact of the articles published by Indian psychiatrists.

MATERIALS AND METHODS

In the last week of December 2015 (December 25, 2015–December 28, 2015), using the National Institute of Health, USA's PubMed database interface (www.ncbi.nlm.nih.gov/pubmed), all articles published by Indian psychiatrists between January 1, 1995, and December 31, 2013, were collected. These dates were chosen for following reasons: In mid-1990's, there was an acute interest among Indian Psychiatrists, especially in academics to publish around this time period, as publications then were deemed to be a part of the “desired” qualification for career promotion as well as introduction of the “open access” model of publications. As with the selection of December 31, 2013, as cut-off date for inclusion, there were two reasons: (1) To allow a 2 years window period for a manuscript to get cited.[4] (2) The β-version of ICITE did not permit assessment of manuscripts published after this date. The search filters of author “affiliation” of author and custom dates were employed. The search criteria used was “India” and “Psychiatry.” The results obtained from the search were stored as a working file. From this working file, the unique identifier PubMed-ID (noted as PMID) were collated. From this, those manuscripts that were e-published before the cut-off dates, those manuscripts that were not found in journals listed with PubMed database were eliminated.

The corresponding PubMed Central (PMC) (a voluntary, free, PubMed repository for depositing of manuscripts which follows “open access” and widely accessed) ID were collected from PubMed and matched with PMID. Based on this all manuscripts were classified as PMC visible (PMC-visible) or non-PMC visible. The journals, based on country of publication were classified as Indian or non-Indian and based on their scope classified either as psychiatry and related field journals or nonpsychiatry medical journals or basic sciences or allied fields. Based on the manuscript, PubMed's and ICITE's articles criteria, manuscripts were classified as true articles (including all peer-review materials) and false articles (all nonpeer reviewed materials). All these were taken as predictor variables.

These PMIDs were then fed into the ICITE (www.icite.od.nih.gov) database in prescribed format. This β-version was used to collect the following individual details:

Articles that were cited at least once were classified as “cited” for those article that has been cited at least once and if never been cited, it was classified as “never cited”

PubMed's classification of Manuscripts as False (Reviews, Letter to Editors etc.,) or True (Original Research/Case reports)

Total citations received: Refers to the entire number of citations received for the unique PMID. This variable was not used for any analysis as annual variations may impede the further calculations

Mean citations per year (MCY): Refers to the average number of citations per year received for the unique PMID

ECR: It is the comparison of the citation record of the unique PMID to the citation averages of similar items published in the same journal during the same year, as estimated by the ICITE β-version

Field citation rate (FCR): It is the comparison of the citation record of the unique PMID to the citation averages of similar items published in the same year, as calculated by the ICITE β-version

RCR: Is a new citation measure that is independent of the Journal Impact factor, and relies on the co-citation network of the peers, as described elsewhere.[9]

These values formed the outcome variables. The data, thus collected were entered and analyzed using Social Package for Statistical Services, version 20 (SPSS, IBM, IL, USA). Descriptive statistics are presented for the predictor and outcome variables. The Chi-square was used to identify the difference between the cited and never-cited manuscripts based on the outcome variable. Kruskal–Wallis and Mann–Whitney tests were used to identify the existence of difference between median of each predictor with regards to outcome variable. Appropriate correlation tests were employed to find the association between predictor variables and outcome variable. As MCY, ECR, and FCR are contributory functions of RCR, their relationship was not computed, as they would be related mostly significantly. Considering the assumptions stated, multiple linear and log linear regression analysis were performed to assess the association of independent variable's effect on the dependent variables. P ≤ 0.05 was taken as significant.

RESULTS

Using the search criteria and limits, a total of 2002 manuscripts were identified. Of these, 88 were found to be published beyond the dates and or were not listed with the citation databases including PMC, European PMC, CrossRef, and Web of Science. Hence, the remaining 1914 were included for the study. These 1914 unique manuscripts were fed into the ICITE database to get the MCY, ECY, FCR, and RCR.

Table 1 shows the basic characteristics of the manuscripts used in this study. Of the 1914 manuscripts, 907 (47.4%) were not listed in the PMC, 135 (7.1%) were identified as false manuscripts. Of these 1914 manuscripts, 1032 (53.9%) were published in Journals published in India. Of all the manuscripts 77.2% were published in core-psychiatry journals, 2.9% in allied specialties, 3.7% in basic sciences journals, and 16.2% in nonpsychiatry medical journals. These 1914 manuscripts were published in 297 journals. Of this 1914 manuscripts, 466 (24.34%) were published with Indian Journal of Psychiatry, 161 (8.4%) with Indian Journal of Psychological Medicine, and 100 (5.22%) with Industrial Psychiatry Journal. Of the 297 Journals, only single manuscripts were published in 149 journals, 2 manuscripts in 54 journals, and 3 to 10 manuscripts in 66 journals. The year wise distributions of the manuscripts are listed in Graph 1 along with the citation trends. Graph 2 shows the annual changes in Mean and Median RCR with outlier data with 95% confidence interval.

Table 1

Characteristics of the manuscripts published and listed with PubMed by Indian psychiatrists between 1995-2013

Graph 1

The yearly variation in number of manuscripts by Indian Psychiatrists published in journals listed with PubMed database

Graph 2

Bar plot showing the annual variation in mean and of median relative citation ratio during study period

Of the 1914 manuscripts, 1007 were cited at least once and among this, 40.7% were listed with PubMed while of the 907 non-PMC listed manuscripts, only 180 were never cited. The difference was statistically significant (P = 0.000). Of the 1032 manuscripts published in India-based journals, 474 were never cited while 214 of the manuscripts published with non-India based journals were never cited even once. The difference was statistically significant (P = 0.000) [Table 2]. Of the 1032 manuscripts published in India-based journals, 558 (54.1%) were cited at least once, and 848 (82.2%) were listed with PMC. Of the PMC included manuscripts, 421 (49.6%) and 137 (74.5%) of non-PMC manuscripts were cited. Similarly, in the 723 manuscripts published with non-India based journals, 159 (18%) were PMC visible. Among this 78 (49.1%) were cited while in the non-PMC manuscripts, 590 (81.6%) were cited.

Table 2

Comparison of the cited and never cited manuscripts in the study group based on other variables

Of the 1226 manuscripts that were cited at least once, in altogether received a mean citation of 4.53 ± 10.26. For the study group, the MCY was 0.84 ± 1.39, ECR was 1.69 ± 0.69, FCR was 2.9 ± 1.17, and the RCR was 0.49 ± 0.75. The MCY, FCR, and RCR means varied significantly in terms of the visibility of the journals. All outcome parameters were statistically significant between the article types. Nationality of the journal significantly influenced MCY, FCR, and RCR [Table 3].

Table 3

Non - Parametric tests of the predictor and outcome variable in the study group

Most of the predictor and outcome variables were significantly correlated with each other, and the extent of correlation and its significance are listed in Table 4. The RCR was significantly associated with, article type (r2 = 0.07, P = 0.002), nationality of journal published with (r2 = 0.280, P = 0.000), core type of journal (r2 = 0.150, P = 0.000), and PMC visibility (r2 = 0.298, P = 0.000). Multiple regression were run to predict RCR from visibility, article type, nationality, and core type of journals. These variables with statistical significance predicted the dependent variables as given in Table 5. RCR was significantly influenced by the predictors and was correlated as F (4.1909) =54.689, P = 0.000; r2 = 0.103. Of the variables considered, visibility of the journal (β =0.204, P = 0.000) and the country of publication (β =0.145, P = 0.000) were highly influencing the outcome. However, the distribution of the data remained obscure. Hence a log linear regression analysis was preferred. A five-way log-linear analysis was performed to determine a hierarchical unsaturated model for the associations between visibility (PMC/Non-PMC), article type (true/false), nationality (Indian/non-Indian) of journal and core type of journals. There were 1914 articles of which 688 had no RCR while remaining 1226 had RCR with several cells having less than five expected frequencies, few outliers and approximately normally distributed adjusted residuals for the chosen model. An unsaturated model was chosen using SPSS Statistics' hierarchical log-linear model selection procedure with a backwards elimination stepwise procedure. This produced a model that included all main effects and multiple -way associations – visibility in PMC*article type*Nationality of Journals*core type of journals, Nationality of Journals*core types of Journals*cited, visibility in PMC *cited, article type*cited. The likelihood ratio goodness-of-fit test indicated that the model was a good fit to the observed data, χ2 (22) =0.000, P = 1. The data indicated that none of the dependent variables considered or their interactions were statistically significant as in the model. However, the absence of several cells meeting the assumptions need to be factored in before taking the results of this model. As both models have limitation, the exact association cannot be accurately predicted.

Table 4

Correlation between the variables considered in the study population

Table 5

Multiple regression analysis of the outcome variables

DISCUSSION

Medical journals are an integral part of academic information and knowledge dissemination chain. With advent of the Internet and open access model of publications, more authors were encouraged to be a part of this chain. In currently prevalent Indian context, original research publications in peer-reviewed, “Indexed” journals stand as one of the criteria for academic promotions.[10] With numerous citation metrics available, comparison of published manuscripts becomes an uphill task. Moreover, most of the currently available metrics use the journal's impact rather than the effect of the individual manuscripts. This allows several types of biases while estimating the impact of the research. To address such bias, a relatively new type of article level index, RCR was postulated.[9] Such a metric would be helpful to (i) quantify the impact of an article (ii) understand the actual dimensions and the rate of distribution of information among other authors/scholars (iii) effectively markup most productive ideas especially when vertically spread and (iv) track pattern of productivity over a period of time and serve as a proxy marker of progression of science.[9] Very little knowledge and introspection have been done with regards to the Indian psychiatrist's contribution to biomedical literature.[4567] Hence, most of the results presented and discussed cannot be supported or refuted with support of existing literature. However, unless such introspection is performed, future policies cannot be relied on past experiences and shortcomings. In this regard, this study may provide an incremental addition to existing insights.

The number of manuscripts published with PubMed has been steadily increasing since 1995, which is on a positive and a welcome trend. The trend to get published with PubMed rapidly shot up with the publication becoming mandatory for promotion in academia by 2007.[10] However, the mean citation rate of the manuscripts considered, as seen in Graph 1, several of the manuscripts, in post 2000 scenario were never cited. Though citations are not the only indicator of the quality of a manuscript, it remains as a one of the main surrogate markers of the contemporariness, quality, usability, and reach of the manuscript, thus used widely as a reliable, indirect marker of the quality of manuscripts.[8] Hence, the study of citation metrics at an individual level becomes more important. In this regard, the use of RCR is gaining more prominence and would serve the purpose.[9]

The existence of high degree of statistical difference in terms of manuscripts visibility in PMC (P = 0.000), nationality of journals (P = 0.000), and article type (P = 0.001) indicate that there exists an innate difference between the cited and noncited manuscripts [Table 2]. In an attempt to understand this phenomenon, comparison, and correlation of the predictor variables in terms of the outcome variables was performed [Tables 3 and 4]. From such an analysis in Table 3, it could be inferred that the RCR was significantly different among the subgroups of the predictor variables. The cause of this was relatable to the ECR and FCR, which played a major role in computation of the RCR.

India, though not an Anglophone region, in the postindependence era, retained English as the lingua franca for scientific communication and to the best of our knowledge, no regional or Indian language based psychiatric medical exists as in case of other western countries. Hence, the entirety of Indian biomedical publications exists in English and thus no citation bias from language could have crept in.[11] The PMC, a PubMed depositary initiative has made several of journals visible to the world. However, in this study, non-PMC article appears to have a distinct, statistically significant higher RCR. While worldwide, open access manuscripts are associated with higher citation rate; presently such a facility does not appear to infer higher RCR to Indian Psychiatrists. This could be explained by the fact that PMC advantage is probably greater for the more citable, high-quality articles, created not from the author end but probably only by quality advantage, from readers and future authors self-selecting what to use and cite. In the present study of the all the India-based PMC included manuscripts, 421 (49.6%) and 137 (74.5%) of non-PMC manuscripts were cited. While in the 723 manuscripts published with non-India based journals, 159 (18%) were only PMC visible. Among this 78 (49.1%) were cited while in the non-PMC manuscripts, 590 (81.6%) were cited. These results indicate that the RCR of manuscripts published with non-Indian and non-PMC listed journals are relatively higher than the others.

It should be considered that majority (~40%) of manuscripts are published by three reputed Indian journals, listed with PubMed and PMC. It also should be considered that in a study, “high-impact-factor journals of (≥28) only account for 11% of papers that have an RCR of 3 or above” which translates to “overlooking 89% of similarly influential papers published in less prestigious venues.”[9] This observation is against populist idea of working hardest to get published with the high-impact factor journals and being attracted by international journals. The results herein underline the fact that the Indian authors were not quite successful in garnering many citations in spite of being in PMC. These findings were not in concurrence with earlier reports.[1213] Hence, one could account that relatively and inherently the qualities of such publications are being perceived inferior by potential future authors. None the less, the knowledge that emanates from the noncited publications are also important, and their service to local community and peers are immeasurable by any current trends.[11] As with any citation-based metrics, fundamentally RCR can never estimate the fullest, nonmeasurable impact such as clinical translational or newer dimension or a paradigm shift or importance of making progress in solving the problem being addressed.[9] Additionally, values such as RCR are dynamic and highly susceptible to variation with time.

Indian Psychiatrists contribution to PubMed database and its quality though currently modest in RCR measures but has been shown to be incrementally varying with time (Graph 1 and 2). At the same time, the policy framers and psychiatric associations, in unison need to explore ways to increase the measurable quality of their research outputs as estimated by the publication industry and the international peers. The stakeholders need to consider the evidence presented here to take urgent measures to be instituted to improve the quality of manuscripts added by Indian Psychiatrists to the PubMed Database. The need of the hour is to turn Indian Psychiatrists contribution, whose quality needs to be accepted by international and Indian parameters and at the same instance trying to preserve our unique loco-regional-cultural, knowledge identity, and work culture. This is an uphill task as it would be a new trail full of obstacles/conflicts that are often difficult to reconcile. Furthermore, most of the Indian psychiatrists scientific output may largely remain as unpublished thesis, dissertations, or in nonvisible journals. This needs to be strongly addressed as they may help to solve loco-regional mental health issues. This existing “scientific (im) mobility” may be a result of improper training, errors in study design, level of evidence, and probably “uneven opportunities afforded to investigators.”[9]

The major limitations of the present study are that it (i) relies only two search terms India and Psychiatry. If author (s) have not included their country or departmental affiliations, then they would be overlooked; (ii) relies exclusively on β-version of ICITE and RCR concept, which may not be free from errors; (iii) citations from other sources (apart from PubMed, PMC, European PMC, CrossRef, and Web of Science) including nonindexed journals were not considered.(iv) The presence of a number of manuscript with no citations poses a significant disadvantage while performing regression analysis as they skew the data. Future studies need to factor in this critical area.

CONCLUSION

Robust estimate of popularity, quantification, and usability of manuscripts published by Indian psychiatrists in PubMed has been presented with new citation metric RCR for the first time. The results can be used safely by officials for chalking their official policy, providing training formats besides improving the overall quality of manuscripts by Indian Psychiatrists.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.