H-indices of Academic Pediatricians of Mashhad University of Medical Sciences.

BACKGROUND: Web of Science, Scopus, and Google Scholar are three major sources which provide h-indices for individual researchers. In this study we aimed to compare the h-indices of the academic pediatricians of Mashhad University of Medical Sciences obtained from the above mentioned sources.
METHOD: Academic pediatrician who had at least 5 ISI indexed articles entered the study. Information required for evaluating the h-indices of the included researchers were retrieved from official websites Web of Science (WOS), Scopus, and Google Scholar (GS). Correlations between obtained h-indices from the mentioned databases were analyzed using Spearrman correlation coefficient. Ranks of each researcher according to each database h-index were also evaluated.
RESULTS: In general, 16 pediatricians entered the study. Computed h-indices for individual authors were different in each database. Correlations between obtained h-indices were: 0.439 (ISI and Scopus), 0.488 (ISI and GS), and 0.810 (Scopus and GS). Despite differences between evaluated h-indices in each database for individual authors, the rankings according to these h-indices were almost similar.
CONCLUSION: Although h-indices supplied by WOS, SCOPUS, and GS can be used interchangeably, their differences should be acknowledged. Setting up "ReasercherID" in WOS and "User profile" in GS, and giving regular feedback to SCOPUS can increase the validity of the calculated h-indices.

INTRODUCTION

Assessment of researchers scientific impact has gained importance in today‘s scholarly circle which is commonly achieved by different traditional bibliometric indices: the number of publications, citation count, etc (1). H (Hirsch) - index is one these indices which was initially proposed in 2005 as a valid criterion for evaluating the physicists’ outputs quality (2). It is now widely recognized and used in different scientific communities as a useful tool for rating the academic achievements of scientists (3). The H index of a researcher is defined as “the number of articles published by the researcher with citations higher than or equal to that number” (1) and can be easily obtained from several online sources. Initially Web Of Science (WOS) was the only source for computing h-indices, but recently other databases (Scopus and Google Scholar (GS)) can provide h-indices too (2, 4). Considerable variability could exist in calculated h-indices from different databases (5). The aim of our study was to compare calculated h-indexes through 3 different databases (WOS, Scopus, and GS) for academic pediatricians of Mashhad University of Medical Sciences.

METHOD AND MATERIAL

Academic pediatricians of the Mashhad University of Medical Science who had at least 5 ISI indexed articles were included in this study. To obtain h-indices of the selected pediatricians, citation analyses were collected from each of the three mentioned databases official websites (6, 7).

In WOS, h-index of each researcher was obtained from researchers’ profiles at Researcher ID (8) or manually extracted if they didn’t have any profile, by using “Author Finder” application of the WOS. H-indices were retrieved from Scopus using “Author search” feature for each author name. For those with several author matches, all the found matches were used collectively.

“Advanced search” option of GS was used to retrieve “user profile” for each researcher in order to obtain the h-indices. For those researchers without a “user profile”, we searched manually to find each authors’ publications and calculate the h-index.

Correlations of h-indices with each other were evaluated using spearman correlation coefficient. Statistical analyses were done using SPSS version 11.5. P-values less than 0.05 were considered statistically significant.

RESULTS

We found 16 pediatricians from Mashhad University of Medical Science who had at least 5 articles indexed in ISI. All the relevant information regarding h-index calculation for each author (total number of articles, total citations, and ranks of researchers according to h-indices from WOS, Scopus, and GS) are detailed in Table 1.

Table 1

Detailed information of Mashhad University of Medical Science pediatricians retrieved from WOS, Scopus, and Google Scholar. *: Research activity duration based on the first published article.

Spearman correlation coefficients between h-indices were: 0.439 (ISI and Scopus), 0.488 (ISI and GS), and 0.810 (Scopus and GS).

DISCUSSION

Measuring the academic impact of scientists has gained considerable importance for several purposes: rapid comparison of researchers, evaluating scientific performance, assessing academic achievements, etc. Academic impact of the researchers has been approached using different measures. H-index is one of the strongest tools for measuring the quality of researchers’ scientific output which is dependent on the number of citations received by the published articles (9). Higher values of h-index are representatives of more successful researchers.

In previous studies, h-index was introduced as a reliable method to evaluate the academic performance of anesthesiologists in Canada by comparing obtained h-indices from WOS and Scopus (10, 11, 12) and comparing GS and Scopus h-indices of academic US neurosurgeons showed a positive relation between this metric with academic ranks and scientific age (13).

H-indices can be retrieved from several sources. WOS, Scopus, and GS are three major databases which provide h-index for individual researchers. H-index is highly dependent on its source (5). This was evident in our results too. For the studied authors, obtained h-index, citation count, and total number of the articles were different from each of the mentioned sources.

Different coverage and citation counts of each source are the major cause of this difference (10). For example GS covers citations from several types of documents (such as books) which are not included in Scopus or WOS. Due to this fact, h-index provided by GS is usually higher than the ones provided by WOS or Scopus. As an example, an article by author 1 of our study (14) has been cited 11 times in GS; 3 of which were books. Number of citations to this article was 6 in WOS and 9 in Scopus. However GS does not always have higher citation counts compared to Scopus or WOS: for example another publication by the first author of our study (15) has been cited 3,4, and 4 times in GS, Scopus, and WOS respectively. Two of these citations in Scopus and WOS were a comment and the author’s reply to the comment (16, 17). These two citations were considered as one in GS.

Journals covered by each database can also be different from each other. Although the three mentioned sources cover most of the scientific disciplines, Scopus covers more journals compared to WOS. GS covers the widest range of scientific journals with no language limit; however exact coverage of GS is not clear (4, 5, 11, 18). Our study was also consistent in this regard since total articles authored by each researcher and total citations to each article was usually highest in GS and lowest in WOS which in turn resulted in variability of h-indices.

In WOS, “author finder” option quickly provided the author’s records for those who had ResearcherID with automatically calculated h-index. However authors should set up the ID themselves and should upgrade it regularly since WOS would not upgrade the publications automatically (in contrast to Scopus and GS). For those researchers who did not have ResearcherID, finding the correct number of articles was difficult. In our study, only author number 2 had ResearcherID, so finding the other researchers’ records without ResearcherID was very difficult and inaccurate. For these authors, several “author sets” could be found and we were forced to calculate the h-indices manually which was a very hard task. Actually, calculating h-index can be almost impossible without ResearcherID for those authors who have familiar and commonly used first (such as Ali or Mohammad) or last names (such as Mohammadi or Razavi).

Scopus has its own author locating program which is very powerful and is updated automatically. This makes Scopus very user friendly compared to WOS. Actually authors do not need to set up their own profiles (in contrast to ResearcherID for ISI and “user profile” for GS) since Scopus automatically generates its own author profile for each researcher. However, this process is far from being ideal and has many shortcomings. The researchers should give their feedback to Scopus using “author feedback” tools in order to correct SCOPUS inaccuracies in assigning publications to their profile. “Author feedback” tools is very flexible and give the researchers the opportunity to “merge” several profiles, “add” or “delete” publications, etc. Actually, we had problem in detecting publications of those researchers who had not given feedback to SCOPUS. For author number 14, one of his articles was in another researcher profile with similar last name. Finding the author number 8 and 16 were difficult, because of the different affiliations they have used without feedback to Scopus to merge them.

In GS, each author can setup “user profile” which helps them to gather their articles in one profile which will be updated by GS itself. However setting up “user profile” is not an automatic process in contrast to SCOPUS. In our study, most problems occurred in searching for authors who did not have ‘user profile’ in GS. Only the first two authors of our study had “user profile”, for the others we faced many problems in extracting articles and sometime it was not possible to retrieve them at all due to huge number of publication yielded by each search.

Significant relation has been reported in previous studies between h-index and scientific age (19-21) which was not observed in our researchers h-indices. For years, Iranian researcher’s scientific productivity has not been considered as a key factor for assessing scientific achievements and improving academic ranks but now this issue has become the major point of focus for researchers. Therefore in this study our senior researchers have not obtained higher h-index in comparison with juniors.

In our study, it was very hard to locate publications of authors who did not have researcherID in WOS or user profile in GS and those with different author sets in Scopus. Most Iranian names (first name or surname) are composed of two parts, which may cause inaccuracies during the indexing process. For author number 6, the first name and for author number 4, the surname consisted of two parts; so we should have tried different parts of their name to find their publications in each database. Some researchers had frequently used first and last names and we had to search several different profiles or author sets to find their publications. For example author number 3 and 16 had similar last names with many other researchers and some of their publications were mistakenly assigned to other author sets or profiles. GS search was the hardest in this regard and we could not be sure about the exact number of the authors’ total articles and citations, thus the calculated h-indices may be inaccurate.

Correlations between h-indices of three databases were sub-optimal in our study which shows that each author can have different h-indices depending on the database used to obtain them. However ranking of the researchers according to each database h-index were almost similar. Since ranking is very important in scientific proliferation, it seems that each of the mentioned sources could be used interchangeably because of the similarity between their ranking. However the differences between the h-indices provided by different databases should be acknowledged.

CONCLUSION

H-indices of pediatric researchers provided by WOS, Scopus, and GS can be different from each other. This is most likely due to different coverage of each database and inaccuracies in assigning publications to each author. By setting up ResearcherID in WOS, User profile in GS, and giving regular feedback to Scopus; the accuracy of h-index calculation can be increased considerably.