Measuring Efficiency of Knowledge Production in Health Research Centers Using Data Envelopment Analysis (DEA): A Case Study in Iran.

INTRODUCTION: Efficiency analysis is necessary in order to avoid waste of materials, energy, effort, money, and time during scientific research. Therefore, analyzing efficiency of knowledge production in health areas is necessary, especially for developing and in-transition countries. As the first step in this field, the aim of this study was the analysis of selected health research center efficiency using data envelopment analysis (DEA).
METHODS: This retrospective and applied study was conducted in 2015 using input and output data of 16 health research centers affiliated with a health sciences university in Iran during 2010-2014. The technical efficiency of health research centers was evaluated based on three basic data envelopment analysis (DEA) models: input-oriented, output-oriented, and hyperbolic-oriented. The input and output data of each health research center for years 2010-2014 were collected from the Iran Ministry of Health and Medical Education (MOHE) profile and analyzed by R software.
RESULTS: The mean efficiency score in input-oriented, output-oriented, and hyperbolic-oriented models was 0.781, 0.671, and 0.798, respectively. Based on results of the study, half of the health research centers are operating below full efficiency, and about one-third of them are operating under the average efficiency level. There is also a large gap between health research center efficiency relative to each other.
CONCLUSION: It is necessary for health research centers to improve their efficiency in knowledge production through better management of available resources. The higher level of efficiency in a significant number of health research centers is achievable through more efficient management of human resources and capital. Further research is needed to measure and follow the efficiency of knowledge production by health research centers around the world and over a period of time.

1. Introduction

Measuring efficiency has been common in a large range of activities, including health care (1, 2), insurance services (3), universities (4, 5), and research activities (6). Analyzing efficiency through DEA (data envelopment analysis) is an assessment of output-input ratio in order to find relative efficiencies of decision-making units (DMUs) (7). DEA is one of the best approaches for comparative performance assessment (8). DEA is well known as a powerful and also easily understandable analytical tool (9, 10). Further, its superiority compared with other techniques has been proven by several studies (11, 12). During the last few decades, the health sector has experienced a dramatic growth in cost in developed and developing countries, and inefficiency of health sectors, at least in part, has contributed to this growth (13, 14). In response to this situation, extensive empirical studies have focused on the measurement of efficiency of health institutions around the world. While hospitals are the subject of most of these studies, there are few studies about health research center efficiency. Nowadays, due to the rapid progress of research systems, the regular collection and reporting information about the performance of research centers are common practices. Research center information and reports can provide a general overview about their performance (15, 16), and the next logical step is to begin comparison of the efficiency of research centers. Through such comparisons, the most efficient research units can be identified and considered as benchmarks for improving the efficiency and effectiveness of others (17). Iran also has experienced an impressive advance in knowledge production during the past decades (18, 19). The numbers of health research centers increased from 53 in 2001 to 359 in 2010 during the last decade. Research centers are growing rapidly in Iran and will continue to provide excellent opportunities in knowledge production (15). Also Iran’s government provides great financial support for health research centers and research projects in order to advance knowledge production more progressively (15, 20). Therefore, using efficiency analysis is necessary in the area of health knowledge production. Our study is the first step in analyzing efficiency of knowledge production in health areas and can be a good model for analyzing efficiency of knowledge production, especially for developing and in-transition countries. The aim of our study is to measure efficiency of knowledge production by a sample of health research centers using data envelopment analysis. This paper used a comprehensive methodology of data envelopment analysis (DEA) to measure health research center efficiency in Iran.

2. Material and Methods

This retrospective and applied study conducted in 2015 used input and output data of 16 health research centers affiliated with a health sciences university during 2010–2014. The 16 research centers included virology, nephrology and urology, nano biotechnology, microbiology, gastroenterology and hepatology, behavioral sciences, neurosciences, human genetics, sports physiology, trauma, molecular biology, biotechnology, nutrition and hygiene, chemical injuries, medicine and religion and health management. The input and output data of the health research centers for years 2010–2014 were collected from the Iran Ministry of Health and Medical Education (MOHE) profile. The input data included the average amount of annual budget, faculty researchers, and staff researchers employed in each health research center during five years (2010–2014). These inputs could well represent labor and capital used in the knowledge production process and have been widely considered in previous studies as the main factors affecting knowledge production in research centers (20, 21). The output data included the average number of published papers indexed in ISI web of science, paper citations according to Scopus website report, and the textbooks written by each research center during 2010–2014. These outputs introduced and reported by the Iran Ministry of Health as the main indicators of measuring knowledge production of health research centers. These outputs also have been used in previous studies for measuring knowledge production of research centers (15, 16). Using the average five years (2010–2014) of inputs and outputs data was because the result of some research activities appear in the long-run (five year). The three main models of DEA, included input-oriented model, output-oriented model, and hyperbolic-oriented model in the VRS (variable returns to scale) approach, were used to study the relative efficiency of health research centers. The input-oriented model focused on minimizing the resources for a certain level of production, while the output-oriented model focused on maximizing production in a certain level of resources. The hyperbolic-oriented model focused on both resource minimizing and production maximizing simultaneously (22). In DEA models, the efficiency scores 1 means complete efficiency and the efficiency scores smaller than 1 means relatively not complete efficacy. The input and output data of each research center were entered into R software, and the numerical score of each research center’s efficiency were calculated and reported.

3. Results

The numerical values of each research center’s efficiency in three main models, including input-oriented, output-oriented, and hyperbolic, are reported in Table 1. Eight research centers have complete efficiency in all the three models, including chemical injuries, nutrition and hygiene, trauma, human genetic, medicine and religion, gastroenterology and herpetology, microbiology, and virology. Dispersion indexes of efficiency scores in each of the three models are shown in Table 2. The minimum efficiency score in input-, output-, and hyperbolic-oriented models was 0.043, 0.048, and 0.11, respectively. The applied biotechnology research center have the minimum efficiency level in the input-oriented model with a score of 0.043 and the nano biotechnology research center has a minimum efficiency level in both output-oriented and hyperbolic-oriented models with 0.048 and 0.11 efficiency score, respectively. The mean efficiency score in input-oriented, output-oriented, and hyperbolic-oriented models was 0.781, 0.671, and 0.798, respectively. In Table 3, research centers with less than the mean scores reported as an under-average efficiency research center. According to Table 3, seven research centers were below the average efficiency in the input-oriented model, and six research centers were below the average efficiency in output-and hyperbolic-oriented models. Five research centers (including applied biotechnology, sports physiology, behavioral sciences, health management, and nano biotechnology) were under the average efficiency in all the three models.

Table 1

Scores of Each Research Center’s Efficiency in DEA Models

Health Research Center	Input-Oriented	Output-Oriented	Hyperbolic-Oriented
Chemical injuries	1.000	1.000	1.000
Nutrition and hygiene	1.000	1.000	1.000
Applied biotechnology	0.043	0.144	0.130
Molecular biology	0.445	0.870	0.852
Trauma	1.000	1.000	1.000
Human genetic	1.000	1.000	1.000
Sports physiology	0.503	0.294	0.603
Medicine and religion	1.000	1.000	1.000
Neurosciences	1.000	0.148	0.880
Behavioral sciences	0.569	0.602	0.703
Gastroenterology and hepatology	1.000	1.000	1.000
Health management	0.735	0.266	0.735
Microbiology	1.000	1.000	1.000
Nano biotechnology	0.449	0.048	0.110
Nephrology and urology	0.756	0.368	0.756
Virology	1.000	1.000	1.000

Table 2

Dispersion Indexes of Efficiency Scores in Each DEA Model

Method	Min	1st Qu	Median	Mean	3rd Qu	Max
Input-Oriented	0.043	0.536	1.000	0.781	1.000	1.000
Output-Oriented	0.048	0.287	0.935	0.671	1.000	1.000
Hyperbolic-Oriented	0.110	0.719	0.940	0.798	1.000	1.000

Table 3

Under Average Efficiency Research Centers in Each DEA Model

Input-Oriented	Output-Oriented	Hyperbolic-Oriented
Under average efficiency	Under average efficiency	Under average efficiency
Applied biotechnology	Applied biotechnology	Applied biotechnology
Molecular biology	Sports physiology	Sports physiology
Sports physiology	Neurosciences	Behavioral sciences
Behavioral sciences	Behavioral sciences	Health management
Health management	Health management	Nano biotechnology
Nano biotechnology	Nano biotechnology	Nephrology and urology
Nephrology and urology

4. Discussion

The considerable advances in health science in the twentieth century have become obvious, and it is really necessary to maintain this capacity for continued scientific advance in the twenty-first century. This goal is not achievable without a greater focus on efficiency of health science researches. Therefore, focusing on the efficiency of a health research center seems to be critical. Improving the efficiency of health research centers can provide a suitable foundation for these crucial health scientific advances. In recent years, the number of health research centers has expanded, which have had a significant role in health knowledge production. Assessing the efficiency of research centers is vital for effective performance and utilization of knowledge production activities, and our study is a first step is this area. The application of DEA can enable health research centers to identify deficiencies and taking appropriate actions for improvement. In this study, the efficiency of health research centers was evaluated based on three basic data envelopment analysis (DEA) models: input-oriented, output-oriented, and hyperbolic-oriented. In the input-oriented model, the mean of efficiency score was 0.781. It means that, among 16 research centers, seven research centers (44%) have been working below average efficiency level and should reduce their inputs for producing the same level of outputs. In the output-oriented model, the mean of efficiency score was 0.67. This means that six research centers (37%) have been working under the average efficiency level and should increase their outputs by the same level of outputs, and the average performance can be 0.33 percent better. Based on the hyperbolic-oriented model, the mean of efficiency score was 0.798. So, according to the results of the model among 16 research centers, six research centers (37%) have been working under the average-efficiency level. These research centers should reduce their inputs and increase their outputs simultaneously.

Based on the findings of this study, only 50% percent of health research centers (eight from 16) were completely efficient. In all the studied models, 31% of research centers (including applied biotechnology, sports physiology, behavioral sciences, health management, and nano biotechnology) were identified as under-average efficiency research centers. So it is necessary for health research centers to improve their efficiency through better management of available resources. The application of DEA to assess RandD efficiency of 29 universities in China indicated that universities that have improved their management achieved a high efficiency value regardless of whether their original RandD strengths were strong or weak. They expressed that such a measure is proved to be helpful for motivating universities to keep on improving their RandD management (23). Also Agasisti and Haelerman in their study analyzed the performance of public universities in Italy and The Netherlands. According to their findings, the relative efficiency of Italian and Dutch universities is largely dependent on policy adoption. When Italian universities were much more efficient in cost minimizing for a given level of activity, Dutch universities were more efficient in realizing European goals and minimizing resources to obtain a high number of graduates (4).

While many studies have measured the efficiency of hospitals and health service institutions, there are only a few studies on measuring the efficiency of health research centers. In a similar study by Dabagh (24), efficiency and productivity of the research sectors of 31 capital universities of Iran have been investigated using data envelopment analysis (DEA); results showed that most universities (74%) were technically inefficient in research filed in 2007. In another study by Afonso and Santos (25) on relative efficiency of Portuguese public universities by DEA, output efficiency scores were 0.77, which means that, with the same inputs, the average faculty seems to be obtaining 23-percent-less performance than it should. Also in a study by Kuah and Wong (26) in Malaysia, the efficiency of 30 universities was assessed through DEA in teaching and research activities separately. The results showed that three universities were efficient in their teaching activities and 11 universities were efficient in their research activities. Out of these 30 universities, only one university was efficient in all the criteria. According to the results of our study, a large gap exists between maximum and minimum efficiency scores of different research centers. This has been confirmed by studies conducted in developing and lower income countries. For example, Taylor and Harris (27) showed considerable and time progressive differences in relative efficiency among South African universities. Also in a study by Dabagh (24), the result of five years following efficiency suggests that, despite large differences in relative efficiency of universities in Iran, there has been no significant improvement in research efficiency in these five years. A study of Australian public university efficiency suggests that Australian universities are operating at a fairly high level of efficiency relative to each other (28). However, an empirical study by Zhigang et al. on total factor productivity in China showed that the country’s eastern regions were significantly and constantly more efficient than the western regions during the years 1978–2003 (29). Due to the undesirable efficiency level in a large number of health research centers and also due to the large gap between efficiency of health research centers, our study suggests future studies should focus on identify factors affecting efficiency in health research centers. A balanced efficiency improvement is necessary in order to make health science progress more desirable. In order to reach this goal, it is necessary that future studies focus on the institutional and environmental factors affecting health research center efficiency. Also some factors include manager’s experience and skills mentioned previously by some studies (30); still there are many unknown factors affecting efficiency of health research centers.

5. Limitations

The present study has limitations that we would like to note as follows: first, in our study, we compare research centers together without considering their field of activity. Comparing efficiency of research centers in different field of sciences separately—such as clinical and none clinical research centers—can provide more accurate comparisons. Second, we use only a nonparametric method to measure efficiency of research centers, but using parametric methods also can be useful for such comparisons. Our study was just a first step in analyzing efficiency of knowledge production in health areas, and future studies should be done without such limitations.

6. Conclusions

Despite increasing the number of health research centers in recent years and their role in producing knowledge and health promotion in society, based on the results of this study, one-half of the studied research centers are operating below full efficiency. Also, a significant number of research centers (one-third) are operating under an average efficiency level, and still there is a large gap between research center efficiency relative to each other. Thus, the higher level of efficiency in a significant number of the health research centers will occur through more efficient management of human resources and capital. Further researches are needed for measuring and following the efficiency of knowledge production by health research centers around the world and over a period of time using DEA techniques. Because many health research centers regard themselves in global competition, international comparisons are also necessary in future studies.