Comparative research performance of top universities from the northeastern Brazil on three pharmacological disciplines as seen in scopus database.

OBJECTIVES: Postgraduate programmes around the world are periodically subjected to research performance evaluation through bibliometric indicators. In this research, we characterized and compared the research performance of 15 universities from Northeastern Brazil, in which 13 were among the top Universities of the Latin America.
METHODS: Specifically, total documents, citations and the h-index of each university were retrieved from the Elsevier Scopus database and were analysed not only for historical scientific achievement but also across the period of the past 6 years (2010-2015). Using these bibliometric indicators, we also investigated the performance of programmes at these Universities that have their papers indexed in the Scopus database under the category of "Pharmacology, Toxicology and Pharmaceuticals" for the same period.
RESULTS: We found that the Federal University of Pernambuco (UFPE) and the Federal University of Ceará (UFC) were the most productive institutions, producing 17847 and 15048 documents, respectively. The number of papers published by each of these universities in the past six years represented more than 50% of their entire productivity. With regards to their scientific output in "Pharmacology, Toxicology and Pharmaceutics", UFC showed the highest number of published documents followed by UFPE and the Federal University of Paraíba (UFPB). UFC received the highest h-index (with and without self-citations) and number of citations and shared their most cited papers with foreign institutions from the USA and Germany. However, papers from UFC were published in journals with lower impact factors (2.322).
CONCLUSIONS: The present study shows where each of these universities stands and can be helpful in identifying potential collaborators in these areas of knowledge.

Introduction

In the past several years, more attention has been given to evaluating the research performance or scientific output of postgraduate programmes within institutions of higher education.1, 2 Such evaluation generally uses bibliometric parameters, such as the number of papers published by a program and the number of citations produced, excluding self-citations. In addition, this analysis can be helpful in identifying potential collaborators and to inform research strategy development. For instance, through bibliometric analysis, Noyons et al. identified centres of excellence in life science research in Europe.

More recently, citation analysis has been cogitated as a way to identify research groups. Therefore, in Brazil, research groups from Postgraduate Programmes are experiencing a period of increasing competition in terms of scientific productivity, as productivity constitutes one important factor in getting financial resources from the main funding agencies CAPES (Coordination for the Improvement of Higher Education Personnel) and CNPq (National Council for Scientific and Technological Development). In this context, research collaboration has steadily increased over the past years.6, 7, 8

Numerous databases have been used for bibliometric analyses, among which Scopus (from Elsevier) and Web of Science (from Thomson Reuters) have been at the forefront due to their multidisciplinary character. Although a comparative study of the two databases revealed no clear difference, a recent study demonstrated that they differ in terms of their scope and the number of documents they contain within a tolerable margin of deviation. Nevertheless, the databases are well-accepted by the scientific community as efficient tools for assessing and comparing the performance of researchers, institutions, and countries among others.3, 11

The Scopus subject category “Pharmacology, Toxicology and Pharmaceuticals” contains some important subareas of the biopharmaceutical sector. A look at the scopus database revealed a great increase in the number of papers produced by Brazilian Higher Institutions (data not shown) in the past 6 years (i.e., from 2010 to 2015) in this subject category, especially from the northeastern region (where there is scarcity of financial resources that does not favour research development). The formation of new research groups with consolidated and highly productive investigators and/or the consolidation of pre-existent research groups are some possible explanations for this increase. This phenomenon has certainly boosted the rankings of some institutions from northeastern Brazil, placing them among the best universities of the Latin America, with University of São Paulo (from the southeastern region) in the first position.

In spite of the growth of research and development in the chosen Scopus subject category (Pharmacology, Toxicology and Pharmaceuticals) by institutions from the northeastern region of Brazil, there are no bibliometric studies on this Scopus subject category particularly focused on this region. Thus, the present study was undertaken to characterize and compare the research performance of top institutions from northeastern Brazil in the “Pharmacology, Toxicology and Pharmaceuticals” subject category by evaluating not only their historical achievements (which may not be relevant in the actual scenario) but also their achievements in the last 6 years (2010–2015), as represented by two periods of assessment of programmes by the CAPES. Particularly, we analysed these institutions' total documents indexed in the Scopus database, citations (with and without self-citations) and h-index (with and without self-citations).

Materials and Methods

Selection of institutions

The institutions or universities used were selected from a publication by the consultancy QS (Quacquarelli Symonds) Media Research Company, which is a British company specializing in education and study abroad. According to the QS World University Ranking (2016), 13 universities from the northeastern region out of 89 total Brazilian institutions appeared to be among the best of their kind in Latin America. It is important to note that the criteria for classification included research, number of publications, number of lecturers with PhD degrees, academic reputation, and impact of the institution on the web. The best rated institution in Latin America was the University of São Paulo from Brazil. The thirteen institutions from the northeastern region of Brazil that appeared in the list were ranked as follows: Federal University of Pernambuco-UFPE (44th) > Federal University of Bahia-UFBA (69th) > Federal University of Ceará-UFC (87th) > Federal University of Rio Grande do Norte-UFRGN (105th) > Federal University of Paraíba-UFPB (113th) > Federal University of Campina Grande-UFCG (136th) > Universidade Federal of Sergipe-UFS (169th) > Federal University of Alagoas-UFAL (189th) > Federal University of São Francisco Valley-UNIVASF (243th) > State University of Bahia-UNEB (244th) > Federal University of Maranhão-UFMA (246th) > Paraíba State University-UEPB (294th) > State University of Maranhão-UEMA (299th).

Because the majority of the authors of this paper are from the Regional University of Cariri-URCA, we decided to include URCA in this study. In addition, the Ceará State University-UECE was also included in this study. Both of these universities are in the State of Ceará in the northeastern region. It should be stressed that these universities were not classified among the 89 Brazilian Institutions.

Data collection

For bibliometric studies, the Elsevier Scopus database was used to retrieve citable documents (articles, reviews, letters and notes) produced by each university. The scientific productivity of each institution was evaluated by assessing their total published documents, and the number published over two periods: 2000–2015 and 2010–2015. The database search was performed on December 24, 2016.

Scientific productivity in the subject category “Pharmacology, Toxicology and Pharmaceuticals”

Starting from the whole scientific productivity of the University, we limited our search to the subject area of “Pharmacology, Toxicology and Pharmaceuticals”, as categorized in the Scopus database. From this selection, the total documents, total citations, average number of citations per paper and the total h-index were retrieved. For comparative purpose, the annual number of publications was calculated, considering the academic age of the university (obtained by the difference between the actual year (i.e., 2016) and the year of the first indexed document). In addition, the number of documents published in this subject area between the years 2010 and 2015 was analysed, as well as their citations (with and without self-citations) and h-index (with and without self-citations).

Publication output and journal impact factor

The impact of national and/or international collaboration in the “well-performing university” was evaluated by considering its 30 most cited papers. In addition, the top 20 journals in which the “well-performing university” published their pharmacological, toxicological and pharmaceuticals research were evaluated.

Results and discussion

Documents

As can be seen in Table 1, apart from UECE and URCA (which were added to the list of the best institutions in Latin America from Northeastern Brazil), three institutions were from the state of PB (UFCG, UFPB and UEPB), two institutions were from the state of PE (UFPE and UNIVASF), BA (UFBA and UNEB) and MA (UFMA and UEMA), while only 1 (one) institution was from the state of CE (UFC), RN (UFRGN), SE (UFS) and AL (UFAL). The number of total documents published by each institution, as seen in the Scopus database, revealed that the Federal University of Pernambuco (UFPE) from the state of Pernambuco (PE) was the most productive followed by the Federal University of Ceará (UFC) from the state of Ceará (CE), represented by 17847 and 15048 total published documents, respectively. However, the least productive institution was the State University of Maranhão (UEMA), producing a total of 548 documents followed by the Regional University of Cariri (URCA), which produced a total of 593 documents (Table 1). The same tendency was observed for the time period of 2000–2015. When analysing the scientific output of each institution over the period of the last 6 years (from 2010 to 2015), it is possible to see that the total number of publications in this time period dramatically increased, generally representing more than 50% of the institution's entire productivity. This effect probably demonstrates the necessity of having a research grant or better institutional visibility (reputation).

Table 1

Total productivity of the institutions, as seen in Scopus.

Institutions	State	Number of documents indexed in Scopus
		Total	From the last 10 years (2000–2015)	From the last 5 years (2010–2015)
Federal University of Ceará-UFC	CE	15048	12406	7649 (50.83%)
Ceará State University-UECE	CE	2434	2111	1384 (56.86%)
Regional University of Ceará-URCA	CE	593	508	401 (67.62%)
Federal University of Pernambuco-UFPE	PE	17847	13822	8412 (47.13%)
Federal University of Bahia-UFBA	BA	12904	10271	6246 (48.40%)
Federal University of Rio Grande do Norte-UFRGN	RN	12014	9942	6765 (56.31%)
Federal University of Campinas Grande-UFCG	PB	5440	4888	3240 (59.56%)
Federal University of Sergipe-UFS	SE	4183	3527	2753 (65.81%)
Federal University of Paraíba-UFPB	PB	9805	7718	4986 (50.85%)
Federal University of Alagoas-UFAL	AL	3872	3017	2057 (53.13%)
Federal University of São Francisco Valley-UNIVASF	PE	920	N/A	647 (70.33%)
State University of Bahia-UNEB	BA	778	651	449 (57.71%)
Federal University of Maranhão-UFMA	MA	3021	2451	1582 (52.37%)
Paraíba State University-UEPB	PB	1553	1317	974 (62.72%)
State University of Maranhão-UEMA	MA	548	450	319 (58.21%)

The percentage in the last column represents the percentage productivity of the last 6 years (2010–2015) in relation to the total number of documents. N/A-Not Applicable because the first document indexed in subject category started in 2005. CE-Ceará, PE-Pernambuco, BA-Bahia, RN-Rio Grande do Norte, SE-Sergipe, PB-Paraíba, AL-Alagoas, MA-Maranhão.

“Pharmacology, Toxicology and Pharmaceuticals” publication records of the top universities of the northeastern region of Brazil

Long-term analysis

The number documents published in Scopus-indexed journals up to December 24, 2016 in the subject category “Pharmacology, Toxicology and Pharmaceuticals” with their total citations and h-index is shown in Table 2. UFC exhibited the highest number of documents (1193) (representing 7.93% of the total productivity of the institution (i.e., 15048)), citations (18490) and greatest h-index value (51). The lowest performing institution was the UEMA with nine published documents representing 1.64% of the University's total productivity, and a total of 23 citations with an h-index value of 2. The order of effectiveness in terms of published documents was UFC (1193) > UFPE (777) > UFPB (679) > UFRGN (417) > UFBA (415) > UFS (387) > UECE (248) > UFAL (223) > URCA (124) > UEPB (119) > UFMA (89) > UFCG (78) > UNIVASF (56) > UNEB (38) > UEMA (9). It is noteworthy that this Scopus subject category represents 20.91% of URCA's total productivity, 10.19% of UECE's and 9.25% of UFS's.

Table 2

Productivity in the subject area of pharmacology, toxicology and pharmaceuticals.

Institutions	Total documents	Total citation	Total h-index	Citation per paper	Academic age (year)	Annual publication
UFC	1193 (7.93%)	18490	51	15.50	49 (2016–1967)	24.35
UECE	248 (10.19%)	3546	31	14.30	19 (2016–1997)	13.05
URCA	124 (20.91%)	928	17	7.48	8 (2016–2008)	15.50
UFPE	777 (4.35%)	8120	41	10.45	49 (2016–1967)	15.86
UFBA	415 (3.22%)	5028	33	12.12	41 (2016–1975)	10.12
UFRGN	417 (3.47%)	4150	29	9.95	41 (2016–1975)	10.17
UFCG	78 (1.43%)	466	12	5.97	13 (2016–2003)	6.0
UFS	387 (9.25%)	4448	32	11.49	19 (2016–1997)	20.37
UFPB	679 (6.93%)	8468	39	12.47	36 (2016–1980)	18.86
UFAL	223 (5.76%)	2928	27	13.13	41 (2016–1975)	5.44
UNIVASF	56 (6.09%)	428	11	7.64	11 (2016–2005)	5.09
UNEB	38 (4.88%)	260	9	6.84	20 (2016–1996)	1.90
UFMA	89 (2.95%)	863	17	9.70	23 (2016–1993)	3.87
UEPB	119 (7.66%)	986	16	8.29	23 (2016–1993)	5.17
UEMA	9 (1.64%)	23	2	2.56	8 (2016–2008)	1.13

The number in parenthesis represent the percentage that the Scopus subject category “Pharmacology, Toxicology and Pharmaceuticals” occupies in the whole productivity of the institution.

Based on the fact that citations accumulate with time, it is theoretically expected that older institutions should have more citations than newer ones. In fact, we found a positive linear relationship between the number of published documents and the academic age of an institution (Y = 18.16X − 162, R2 = 0.6325). Therefore, a possible way to make a better comparison in this case is to determine the average annual number of publications and citations per paper. Although UFC and UFPE had the same academic age (49 years), the annual number of documents published by UFC was 1.5 times more than that of UFPE (24.35 vs. 15.86) with an average of 15.50 citations per paper in comparison to the 10.45 citations produced by UFPE. Similarly, among the institutions with an academic age of 41 years, UFBA and UFRGN had almost the same number of annual documents (10) and a comparable number of citation per paper (12.12 vs. 9.95). However, UFAL (an institution with the same academic age of 41 years) produced only an average of 5.44 citations per year but had the highest number of citations per paper (13.13). The URCA appears to be a relatively younger institution, which is nevertheless notably productive, producing an average of 15.50 publications but a lower average of citations per paper (7.48) (Table 2).

In the past several years (2010–2015)

Figure 1 illustrates the total amount of documents published in Scopus-indexed journals over the past 6 years (2010–2015). The results indicate that UFC had the highest number of published documents in the Scopus subject category “Pharmacology, Toxicology and Pharmaceuticals” (499 documents) followed by UFPE and UFPB with 368 and 355 documents, respectively (Figure 1). The least productive institution was UEMA (6 documents) followed by UNEB (28 documents) and UNIVASF (32 documents).

Figure 1

Total documents published in the last 6 years period (2010–2015) in Scopus subject category “Pharmacology, Toxicology and Pharmaceuticals”.

Citation analysis is generally regarded as a quality factor that can be used to evaluate research performance and citation without self-citations of the authors can be of great importance, since it reflects the interest that other people from the scientific community show in a research group's published paper. UFC (which showed the highest number of published documents) received the highest number of citations (4401) (Figure 2A), even after excluding self-citations (3222) (Figure 2C) followed by UFPE and UFS with 2539 and 2440 total citations, respectively (Figure 2A). When self-citations were removed from the total number of citations related to the institutions, the number of citations ranged from 4401 to 3222 for the most productive institution (UFC) and remained unchanged (7) for the least productive (UEMA) (Figure 2A and 2C). We also evaluated the research impact of these institutions by measuring their average number of citations per paper. We found that UFS has the highest average number of citations per paper (9.31), followed by UFRGN (8.87) and UFC (8.82). However, UFRGN appeared to have the highest number of citations per paper when self-citations were excluded (6.84), followed by UFC (6.46) and UFS (6.31). The order of effectiveness in the average number of citations per paper was as follows after excluding self-citations: UFS (9.31) > UFRGN (8.87) > UFC (8.82) > UFAL (7.88) > UFBA (7.78) > UFMA (7.27) > UFPE (6.90) > URCA (6.86) > UNIVASF (6.31) > UECE (6.29) > UNEB (6.21) > UEPB (5.26) > UFPB (5.06) > UFCG (4.63) > UEMA (1.17) and UFRGN (6.84) > UFC (6.46) > UFS (6.31) > UFBA (6.0) > UFMA (5.42) > UFPE (5.26) > UFAL (5.16) > URCA (4.78) > UECE (4.68) > UNEB (4.29) > UFPB (3.36) > UEPB (3.28) > UNIVASF (3.0) > UFCG (2.92) > UEMA (1.17).

Figure 2

Total citation with (A) and without self-citations (C) associated with the h-index with (B) and without (D) self-citations of published documents for the last years (2010–2015) in the Scopus subject category “Pharmacology, Toxicology and Pharmaceuticals”.

The h-index is a bibliometric parameter that simultaneously measures the quantity (number of documents) and quality (in terms of citations) of published papers and their increase over time. Using the h-index alone as a bibliometric parameter, Huang was able to obtain a classification of Chinese institutions similar to that obtained through the Academic Ranking of World Universities (ARWU). As shown in Figure 2B, UFC showed the highest h-index value (27), even when self-citations were excluded (23). In contrast, the UFPE (which showed a higher number of documents (2539) than UFS (2440, see Figure 1)) presented a lower h-index value (21) in comparison to UFS (25) (Figure 2B); however, they produced the same h-index value (19) when self-citations of authors were excluded (Figure 2D). The classification of these institutions over the last 6 years (2010–2015) on the basis of their h-index values without considering self-citations was as follows: UFC (23) > UFPE = UFS (19) > UFBA=UFRGN (17) > UFPB (16) > UECE (14) > UFAL (13) > URCA (11) > UFMA = UEPB (9) > UFCG = UNIVASF (7) > UNEB (6) > UEMA (1) (Figure 2D).

Impact factor of top 20 journals in which UFC published in scopus category “Pharmacology, Toxicology and Pharmaceuticals”

Considering that UFC exhibited the highest scientific research performance across almost all of the parameters used, we decided to investigate the impact factor of journals where UFC's papers are published and to attempt to understand the collaborative network behind their performance. The impact factors of journals measures the average number of citations of the articles published in the journals in question. In this study, we used the 2015 impact factor published by Web of Science from Thomson Reuters.

Table 3 shows the list of the top 20 journals in which UFS published scientific research (from 1976 to December 2016) indexed in the Scopus category “Pharmacology, Toxicology and Pharmaceuticals” and the impact factor of each of the journal. As can be seen, the number of papers (i.e., documents) published in the top 20 journals was 618, representing 51.80% of the total of total number of documents (1193) published since 1976. The following journals had the best records of publication, representing, respectively, 4.86, 3.86 and 3.69% of the total publications (i.e., 1193) (Table 3): Toxicon (58), Phytotherapy Research (46) and the Journal Of Ethnopharmacology (44). The average impact factor (AIF) of the top 20 journals, as calculated by dividing the sum of the impact factor of the journal by the total number of journals, was 2.322. This finding indicates that postgraduate programmes from UFC working in the areas of pharmacology, toxicology and pharmaceuticals publish their research in low impact factor journals and consequently still require improvement to afford at least relatively good journals (with impact factor, IF = 4–5).

Table 3

List of the top 20 journals and impact factor where UFC mostly published since 1967.

Position	Journal name	Number of documents	% of total	Impact factor (2015-JCR)
1	Toxicon	58	4.86	2.309
2	Phytotherapy Research	46	3.86	2.694
3	Journal of Ethnopharmacology	44	3.69	3.055
4	Journal of Natural Products	38	3.19	3.662
5	Brazilian Journal of Pharmacognosy	36	3.02	0.956
6	Planta Medica	35	2.93	1.990
7	Brazilian Journal of Medical and Biological Research	33	2.77	1.146
8	Natural Product Communications	33	2.77	0.98
9	Phytochemistry	32	2.68	2.779
10	Journal of Pharmacy and Pharmacology	31	2.60	2.363
11	Life Sciences	28	2.35	2.685
12	European Journal of Pharmacology	25	2.10	2.730
13	Fitoterapia	24	2.01	2.408
14	Pharmacology Biochemistry and Behaviour	24	2.01	2.537
15	Biological and Pharmaceutical Bulletin	23	1.93	1.8
16	Naunyn Schmiedeberg S Archives of Pharmacology	23	1.93	2.376
17	Fundamental and Clinical Pharmacology	22	1.84	2.156
18	Phytomedicine	22	1.84	2.937
19	Inflammation Research	21	1.76	2.557
20	Revista Brasileira De Farmácia	20	1.68	N/A
	Total	618	51.82	AIF = 2.322

The number of documents is from 1967 to ending 2016. 2015 Journal Citation Reports® by Thomson Reuters, 2016. N/A-Not Applicable. Only journals with impact factor were considered for the calculation of the average of impact factor (AIF). The total indexed documents since 1967 was 1193.

Share of the 30 most cited papers of UFC in “Pharmacology, Toxicology and Pharmaceuticals” by institutions and countries

To understand the collaborative network behind the scientific performance of UFC, we analysed the countries collaborating with UFC using their 30 most cited papers. As depicted in Figure 3 and Table 4, UFC collaborated with other Brazilian institutions for 65.52% of their 30 most cited papers followed by Belgium (6.9%) and the United State of America (USA, 6.9%). Nevertheless, among the 30 most cited papers published by UFC, the first two (2) papers are co-authored by foreign institutions from the USA (Medical College of Georgia) and Germany (German Cancer Research Centre), which are English-speaking countries with good scientific reputations. This finding indicates that choosing collaborating partners from high-performing English-speaking countries can improve institutional or country research performance; this property was found to have a positive effect on profits and economic development.18, 19, 20, 21 Although collaborative international papers can attract higher numbers of citations, we are not assuming that this can be used as a measure of research performance.

Figure 3

The share of Federal University of Ceará (UFC) publications (for the 30 most cited papers) by country in the Scopus subject category “Pharmacology, Toxicology and Pharmaceuticals”.

Table 4

Publication output of the 30 most cited papers. Overview of institutions that collaborated in the 30 most cited papers of UFC in the sub-area of pharmacology, toxicology and pharmaceuticals.

Institution	Number	Institution	Number
Medical College of Georgia	1	Federal University of Sergipe	1
German Cancer Research Center	1	Univ. Fed. Rio Janeiro	1
Universidade Federal de São Paulo	1	Federal University of Pará	1
Universidade de Sorocaba	1	Federal Rural University of Rio de Janeiro	1
Universidade Federal de São Carlos	1	Universidade Estadual de Londrina	1
Universidade Estadual de Campinas	1	Universidade Federal de Pernambuco	1
Universidade Federal Do ABC	1	Univ. Estadual do Ceará	2
Univ. Sci. and Technol. Chittagong	1	University of Leuven	1
University of Karachi	1	University of Antwerp	1
University of Ferrara	1	Universidade de Brasília	1
University of Campinas	1	Université Paris Descartes	1
University of São Paulo	2	Universidade do Estado do Rio de Janeiro	1
Natl. Inst. Biol. Standards/Contr.	1	Total	29
Federal University of Piaui	1
University of Mississippi	1

The institutions are listed from the highest cited paper to the lowest (out of 30 papers). The total citations from these 30 most cited papers were 2982 and 2644 when self-citations was excluded.

Conclusion

The comparison of research performance across top institutions from northeastern Brazil in the “Pharmacology, Toxicology and Pharmaceuticals” Scopus subject category based not only on their historical achievements (which may not be relevant to the actual scenario), but also on their recent achievements (i.e., in the last 6 years, 2010–2015); identified the Federal University of Ceará as one of the most active/productive institution. The hierarchy of research output based on h-index values (which simultaneously evaluates the quantity and quality of research) without considering self-citations of all authors was UFC (23) > UFPE = UFS (19) > UFBA=UFRGN (17) > UFPB (16) >UECE (14) > UFAL (13) > URCA (11) > UFMA = UEPB (9) > UFCG = UNIVASF (7) > UNEB (6) > UEMA (1). Although UFC showed the highest research performance, the average impact factor of journals in which they mostly published is relatively low (2.322), suggesting that there is a necessity to publish in high-impact journals in order to possibly attract more citations. For low-performing institutions, a tentative solution could be the combination of knowledge from different areas to produce high-impact papers with good visibility. It should be stressed that our study has several limitations, including the fact that we focused only on existing literature in the Scopus database, which may not include all scientific literature produced by these institutions. However, the same problems also occur in other databases, such as Web of Science and Google Scholar.

Recommendation

Based on the results obtained in this study, it is recommended that the combination of knowledge from different areas can produce high-impact papers with good visibility.

Authors' contributions

JPK conceived and designed the study. JPK, KRF, RGDSN and IFA retrieved the data. FABC, AED and LMB analysed and interpreted the data. JPK, KRF and RGDSN helped to write manuscript. APP and IRAM critically reviewed the manuscript for important intellectual content. OOE revised the manuscript for English editing and improvement. All the authors critically read and approved the final version of the submitted manuscript.

Conflict of interest

The authors have no conflict of interest to declare.