Worldwide inequality in production of systematic reviews.

BACKGROUND: Investment in science is vital for the development and well-being of societies. This study aims to assess the scientific productivity of countries by quantifying their publication of systematic reviews taking the gross national income per capita (GNIPC) into account.
METHODS: Medline and ISI Web of Science were searched for systematic reviews published between 1st January 2006 and 31st December 2010. The productivity of each country was quantified by exploring the authors' affiliation. The GNIPC was used according to the World Bank Report. Concentration index (CI) was calculated as the index of inequality.
RESULTS: CI of percentage of systematic reviews as a function of percentage of countries ranked by GNIPC was 0.82 which indicates inequality in production of systematic reviews in pro rich countries. Countries with high income produced 206.23 times more systematic reviews than low income countries, while this ratio for lower middle and upper middle countries was 9.67 and 12.97, respectively. The highest concentration index was observed in clinical sciences (0.76) and the lowest in public health (0.61).
CONCLUSION: This study demonstrates a significant gap between industrialized and nonindustrialized countries in the production of systematic reviews. Addressing this gap needs tremendous national and international efforts.

Introduction

It is a well acknowledged fact that investment in science is vital for the development and well-being of societies. Studies have shown that investment in research and science would eventually pay off overwhelmingly for the community (1).

It is of vital importance for policy makers to assess national scientific productivity which is defined by outputs of scientific researches, in order to make better decisions on priorities and resource allocation. The Scientific productivity of institutions in individual countries or groups of countries has been traditionally assessed by measuring the amount of governmental or industrial investments, number of post-graduate programs offered, quantity of patents or publications in peer reviewed journals indexed in popular biomedical databases, citations received by the publications, extent of scientific awards granted, and number of active researchers in the target institutions (1-8). Although these measures show appropriate validity for this purpose, their accurate evaluation is difficult or even impossible in some cases, such as the number of post-graduate positions offered or scientific awards granted. Furthermore, there might not be an absolute definition for some of these measures; therefore, the number of patents or publications seems to be more reproducible parameters. Although these two measures precisely reflect scientific productivity of countries, taking the quality of the publications into account would further increase the accuracy of such evaluations (2,4).

Systematic reviews are regarded as the highest level of evidence in quality and strength of recommendations in biomedical literature (9,10). Findings of systematic reviews are most valuable in health policy and clinical decision making. Publications of public health systematic reviews can be considered as high quality researches of countries that can be used for local decision making. Previous studies have demonstrated that there is an unequal contribution in production of evidence in science among different nations, however according to our best knowledge currently there is no study on the productivity of systematic reviews as the source of the most valuable evidence for decision making (10).

In this study, we aimed to assess the scientific productivity of countries by quantifying the publication of systematic reviews in journals indexed in popular biomedical databases worldwide and to determine the amount of inequality in this measure as an indicator of gross national income per capita (GNIPC) inequality.

Methods

Quantifying the publication of systematic reviews

Two scientific bibliographic databases, Medline and ISI Web of Science, were searched for systematic reviews published by each country listed by World Bank (11). Medline and ISI Web of Science were used since they implement acceptable international standards.

For searching Medline, the term ‘systematic review’ was entered in the search field and the search was limited to publications between 1stJanuary 2006 and 31stDecember 2010 using the ‘limits’ facility of PubMed.

The ISI Web of Science search was carried out by searching ‘systematic review’ in the title or abstracts of indexed items published.

Upon retrieving the indexed items, the following procedure was performed:

1) Duplicate abstracts were found and the Web of Science copy of each pair of duplicates was kept for further assessments‏.

2) Two investigators assessed and put away items which were not systematic review studies (i.e. comments or author replies to a systematic review and publications on the importance or methods of performing a systematic review) as well as items which were not published in journals‏.

3) Systematic reviews were further divided into the categories of (1) biological sciences (systematic reviews of papers evaluating molecular pathophysiology of diseases), (2) clinical (systematic reviews on diagnostic or prognostic studies, or effectiveness of interventions in terms of clinical outcomes, or patient education), (3) public health (systematic reviews on burden of disease, or risk factors of diseases, or acceptability or efficacy of interventions in terms of public health, or medical education), (4) unrelated to biomedical sciences or combinations of first four subgroups. This process was carried out by two investigators independently and in case their classifications did not match and they didn’t reach a consensus after the discussion, the decision of the third investigator was applied‏.

4) Productivity of each country was quantified by manually exploring the authors’ affiliations listed in each abstract. All authors’ affiliations were taken into account regardless of the position of the author in the paper. Hence if the authors were from different countries the publications would be attributed to more than one country‏.

Gross national income per capita

Gross National Income Per Capita (GNIPC) is the gross national income, converted to U.S. dollars using the World Bank Atlas method, divided by the midyear population. Data regarding countries GNI was taken from the World Bank statistics (11). Countries were divided into low, lower middle, upper middle, and high income based on World Bank statistics and classification in 2010 (11).

Statistical Analysis

The Distribution of number of published systematic reviews was explored as number and percentage. Concentration index (CI) was calculated as the index of inequality using STATA version 9 software. The range of the CI index is between -1 to +1. Zero value of CI demonstrates no inequality, while increase of CI toward +1 indicates pro rich countries inequality and decrease of CI toward -1 indicates pro-poor countries inequalities. The details of the calculation of CI is explained elsewhere (12).This index was also calculated for each of the systematic review categories and each classification based on income.

Results

Upon initial search in PubMed and ISI Web of Science, 28618 items were retrieved. After putting away duplicate abstracts, items which were not systematic reviews, as well as items which were not published as articles in journals, 18,352 abstracts remained for further evaluation. Data on the country of origin of the production was available for 17,842 (97.22%) out of 18,352 systematic reviews of which 4939 (27.68%), 4356 (24.41%), 2308 (12.93%) belonged to the United States, United Kingdom, and Canada, respectively. Table 1 shows the first ten countries publishing the highest absolute number of systematic reviews as well as number of systematic reviews in different fields of biomedical sciences. These countries all belonged to the upper class, except for Brazil that falls in the upper middle class and China that falls in the lower middle class.

Table 1

Top ten countries producing the highest number of systematic reviews in the world in journals indexed in PubMed or Web of Science during 2006-2010

Country	Number of systematic reviews
	Total	Clinical	Public Health	Biological
United States	4939 (27.68)	3487 (28.17)	1395 (26.94)	89 (24.93)
United Kingdom	4356 (24.41)	2930 (23.67)	1455 (28.10)	71 (19.88)
Canada	2308 (12.93)	1589 (12.83)	714 (13.79)	29 (8.12)
Australia	1320 (7.39)	776 (6.27)	536 (10.35)	23 (6.44)
Netherlands	1305 (7.31)	907 (7.32)	378 (7.30)	57 (15.96)
Germany	864 (4.84)	615 (4.96)	234 (4.52)	9 (2.52)
Italy	811 (4.54)	662 (5.34)	150 (2.89)	27 (7.56)
China	621 (3.48)	509 (4.11)	105 (2.02)	17 (4.76)
Spain	594 (3.32)	436 (3.52)	151 (2.91)	14 (3.92)
Brazil	518 (2.90)	369 (2.98)	129 (2.49)	21 (5.88)
All Countries	17842 (100)	12376 (100)	5177 (100)	357 (100)

Data are presented as numbers (percentage).

Figure 1 depicts the concentration curve of scientific productivity in terms of numbers of publication of systematic reviews as a function of GNIPC. Concentration index of percentage of systematic reviews as a function of percentage of countries ranked by GNIPC was 0.82 (SE=0.31) which indicates a high concentration of systematic review production in rich countries.

Fig. 1

Cumulative percentage of systematic reviews according to cumulative percentage of countries ranked by their gross national income per capita

Countries with high income produced 206.23 times more systematic reviews than low income countries (Table 2), while this ratio for lower middle and upper middle countries was 9.67 and 12.97, respectively.

Table 2

Number and ratio of systematic reviews published by countries in each group of gross national income

GNI level	Mean systematic review (SD)	Systematic review ratio*	Concentration index†
Low income	1.93 (3.43)	1	0.34 (0.23)
Lower middle income	18.66 (86.59)	9.67	0.26 (0.31)
Upper middle income	25.04 (79.60)	12.97	0.27 (0.42)
High income	398.02 (977.84)	206.23	0.20 (0.17)
Total	-	-	0.82 (0.30)

GNI: Gross national income

* The ratio of mean number of systematic reviews in each GNI level to Low income countries.

† Data are presented as concentration index (standard error)

Table 2 illustrates the concentration index of high, upper middle, lower middle, and low income countries in terms of quantity of systematic reviews according to GNIPC.

The CI in each type of systematic review was also calculated. The highest concentration index was observed in the clinical sciences category (CI= 0.76, SE = 0.30), while the lowest was for public health research category (CI= 0.61, SE = 0.28). Concentration index in biological sciences was 0.73 (SE = 0.25).

Discussion

According to the best of our knowledge, this study evaluated the inequality in production of systematic reviews among countries worldwide for the first time. Our findings demonstrate a significant gap between industrialized and non-industrialized countries in producing systematic reviews, although in many cases there is a need of local evidence for addressing and solving regional problems. Our study showed that there is a relatively small intra-variability in the scientific productivity of countries in each strata of gross national income, termed as low, lower middle, upper middle, and high income countries.

Previous reports showed that highly populous countries or regions may show higher quantity of publications. However, taking the population of countries or regions into account may elucidate that regions with lower populations are also productive. Hence taking into account the ratio of the number of publications to the population can clarify important points (7,13-15). Therefore, in this study we applied Gross national income per capita which considers the midyear population in each country.

In the current study, we manually assessed the authors’ affiliations since previous reports demonstrated that automated search may miss substantial numbers of publications affiliated to certain institutions due to incomplete addresses (16,17).

The relatively lower concentration index in production of systematic reviews in public health might indicate that developing countries are making more efforts in targeting local health issues rather than studying biological or clinical problems, which are more portable and can be solved by applying foreign research findings. This is in concordance with the findings of Falagas et al who showed that in parasitology, which is rather a local problem in Africa, this region ranks second in terms of publication adjusted by GNIPC (18). Although the concentration index for the field of public health is relatively lower than other fields, it is considered very high which implies that there is still a long way to go for developing countries to address other local problems. Notably, in line with our findings, even among high income countries a great difference in scientific productivity in diverse fields of biomedical sciences is evident (CI=0.20) (19-23). Similar inequalities were seen in systematic review productions in other layers of countries as well. This reflects the difference in resource allocation and research prioritization which was even seen between more similar countries regarding GNI.

Although the gap between higher and lower income countries can be regarded as a consequence of resources available in these countries, it can itself lead to further increase in the gap between health problems. It is also critical to note that in addition to production of systematic reviews, developing countries would need efficient systems to critically appraise these studies.

Our findings are similar to those of ‘Scimago Journal & Country Rank’, and also to sporadic studies on scientific productivity in certain biomedical fields including cardiovascular diseases, radiology, and reproductive health in particular regions of the world (20-25) which indicate higher scientific productivity of higher income countries. In 2004, King published a study on the scientific publications of 31 countries producing about 98% of the world’s highly cited papers during 1993-2001. He found a skewed distribution between the ratio of the citations to all papers per unit national GNP as a function of the GDP per capita for overall scientific publications in these countries (8). On the same track, this study showed that 8 countries in the world publish 84.5% of the highly cited papers, whereas the second 9 countries produce 13% (8). The skewed distribution observed in this report as well as the significant difference in the scientific impact between the world’s top first and second group of countries producing science with the highest impact is comparable with the high positive concentration index found in our study. In line with this finding, it is shown that African countries and low income countries of Latin America produce the lowest number of biological sciences publications (7).

Inequality in biomedical scientific productivity in the world, as shown in this study, is further enhanced by the fact that a great proportion of the total global burden of diseases is related to developing countries which have approximately five times the population of high income countries (26), therefore a major misdistribution of resources in biomedical sciences is apparent which may act as an important barrier to progress. Many of the aforementioned diseases are infectious, and evidence related to them is not produced in rich countries; there is a need for the local production of such evidence.

One reason for the lower scientific productivity might simply be allocation of economic resources to the basic needs of societies or instability of regions in some developing countries, judged by the variability in number of publications in some regions during conflicts (17). However, the major issue of brain drain leading to fewer numbers of qualified innovative researchers in developing countries, access to biomedical databases and journals (27), conduction of studies of inadequate quality, and poor collaboration with international scientific communities which is regarded as the strongest predictor of scientific productivity and academic promotion in scientific institutions (28,29), may all play important roles in the lower scientific productivity of developing countries (30,31). Lower international interest in studies targeting local challenges and therefore lower chances of publishing such researches in highly viewed popular journals, and language barriers cannot be put aside.

The Findings of this study are only a rough estimate of scientific productivity of countries. The Quality and usefulness of systematic reviews published is not assessed in this study. Furthermore, in contrary to Web of Science, PubMed registers the address of the first author; therefore, our study may underestimate scientific productivity of some countries which have more collaboration. Considering these limitations, our results should be generalized cautiously.

Conclusion

Lower income countries may benefit from the experience of other countries with similar resources which outperform their financial resources in biomedical sciences. However, addressing the gap between developing and industrialized countries needs tremendous national and international efforts.

Acknowledgements

The Authors would like to express their gratitude to FatemehAdiliadghdam and KamyarMollazadeh-Moghaddam for their technical assistance in data gathering. This study was supported by Tehran University of Medical Sciences (grant number 89-04-102-11901).

Conflict of interest

The authors declared no conflict of interest.