Health Research Governance: Introduction of a New Web-based Research Evaluation Model in Iran: One-decade Experience.

BACKGROUND: Governance is one of the main functions of Health Research System (HRS) that consist of four essential elements such as setting up evaluation system. The goal of this study was to introduce a new web based research evaluation model in Iran.
METHODS: Based on main elements of governance, research indicators have been clarified and with cooperation of technical team, appropriate software was designed. Three main steps in this study consist of developing of mission-oriented program, creating enabling environment and set up Iran Research Medical Portal as a center for research evaluation.
RESULTS: Fifty-two universities of medical sciences in three types have been participated. After training the evaluation focal points in all of medical universities, access to data entry and uploading all of documents were provided. Regarding to mission - based program, the contribution of medical universities in knowledge production was 60% for type one, 31% for type two and 9% for type three. The research priorities based on Essential National Health Research (ENHR) approach and mosaic model were gathered from universities of medical sciences and aggregated to nine main areas as national health research priorities. Ethical committees were established in all of medical universities.
CONCLUSION: Web based research evaluation model is a comprehensive and integrated system for data collection in research. This system is appropriate tool to national health research ranking.

Introduction

In 2001, WHO defined the health research system (HRS)as the people, institutions, networks and activities whose primary purpose is to generate and promote the utilization of scientifically validated knowledge that can be used to enhance or maintain the health status of populations(1). This system has four principal functions consist of stewardship or governance, financing, building up human and physical resources and finally research utilization (2).

In this system, for making appropriate health policies, it is needed to strengthen four essential elements such as defining vision, mission, goals(VMG) based on strategic planning, identifying appropriate health research priorities, monitoring ethical standards for research partnership and setting up health research system evaluation(2).

Strategic planning as an essential element in stewardship is a systematic management approach to set the priorities based on resources with stakeholder’s cooperation. On the other hand, determination of health priorities guides policy makers to implement interventional programs to provide maximum social welfare for population (3).

Based on upstream documents in Iran, health research goals in science and technology for 2025 have been determined as follows:

- Production of 20000 indexed medical science articles per year

- Establishment of 700 research centers

- Employment of 20000 researchers in research centers, universities of medical sciences and health R&D units.

Moreover, in Iran, in 2010, Deputy for Research and Technology – Ministry of Health and Medical Education (MOHME) extracted the national health research priorities based on mosaic method and Essential National Health Research approach (ENHR). In this way, local research priorities with stakeholders’ participation were determined by universities of medical sciences and then each local priorities package was considered as a mosaic or unit. A collection of units formed a layer of information to set the national health research priorities (4).

In Iran, in 1999, local ethical committees in universities of medical sciences were established in order to assess and evaluate the observance of ethical standards for researches (5). One of the most important elements in stewardship is HRS evaluation that it has an important role in growing science and technology in the world (6).

In recent years, the international ranking of institutions such as Academic Ranking of World Universities (ARWU)(7), Times Higher Education-QS(8), Webometrics Ranking of World Universities(9), SCImago Institutions Ranking(10) have received wide attention. In these systems, scientific institutions based on some criteria are evaluated and ranked.

In Iran, integration of medical education and the health care delivery system, made MOHME as the health authority is responsible to health research policy and resource allocation. Therefore, it is important, to assess the system efficiency based on comprehensive evaluation system. Over the past decade, Iran has developed one of the most advanced research evaluation and ranking systems in developing countries. Since two years ago, the mentioned evaluation information is available to web based system as medical science information portal that information will be updated monthly. In this regard, easy and prompt access to research information can cause to evaluate the HRS goals and visions based on planned mission.

In this article, as an experience, we are going to introduce the method of mission based on line evaluation which applicable to assess goals achievement.

Materials and Methods

In this observational study, data collection forms were designed based on stewardship function in HRS. The indicators for evaluation of this function consisted of having mission based planning, identifying health research priorities, approving the projects based on priorities, establishment the ethical committee and refer the projects to these committees.

For designing an appropriate research evaluation model in governance, three main steps were performed as follows:

1-Developing of mission-oriented program:

This step has several activities consist of:

- Invitation from all of UMSs to participate in designing the mission oriented program

- Establishment a national committee to direct and monitor the process

- Review upstream documents such as the Fifth Development Plan, Holistic Scientific Map and comprehensive plan review by national committee.

- Selection the main areas to governance monitoring and evaluation consist of:

- Priority setting (4)

- Ethical committee (5)

- Enabling environment (6)

- Selection the main indicators for each area

- Designing a form or datasheet for data gathering related to governance indicators.

- Form the research puzzle and assign the missions based on stakeholder capabilities.

2-Creating enabling environment by:

- Setting up the PhD by Research courses

- providing facilities to establish research institute

- development research networks

- providing easy access to scientific data bank

3-Set up Iran Research Medical Portal as a center for research evaluation by:

- set up appropriate software based on selected indicators

- filling the forms through research portal as a pilot study and finding the challenges

- training the instruction of research portal to representatives from UMSs

- implementation the main phase

1- In 2012, a pilot study was conducted by deputy for research and development to establish the applicability of on line evaluation system. The web based evaluation was carried out by 3 teams consisting of research experts in universities of medical sciences and research centers, conceptual experts at the office of undersecretary for research and software experts in medical portal office.

For setting this portal, follow activities have been done:

- Carrying out common sessions among conceptual and software experts for finding the common language.

- Clarifying the research indicators based on mission.

- Designing an appropriate software with follow characteristics:

- Open access

- Web based

- High security function

- User friendly

- With training support

- With Nas solution backup

- With back up from user interface

- Multi interface

- With reporting capabilities

- With news section

- Training research experts at university level for understanding the medical research portal, login mode and data entry.

- Doing pilot study by two universities of medical sciences (Qazvin and Arak) to provide feedback, correction, fix flaws and elimination bugs.

- Designing the judgment panel for scoring the research activities.

- Presenting the report ranking related to research activities.

It is mentioned that the traditional evaluation system was formed in 2002 and all of research documents were evaluated manually and annually. In 2013, this system was developed to web based ones.

Results

All of the universities of medical sciences were participated.

The results are divided in two parts. In part one, results of indicator’s evaluation in stewardship function and in part two, features of online evaluation system as one of the most important element in HRS governance will be presented.

The results of indicator’s evaluation in steward-ship function are as follow:

- Mission based planning:

The contribution of type one to goal achievement was 70%, second type 25% and for third type only 5 % has been considered.

According to web based evaluation in 2013, the number of indexed medical science articles was 20945 that this number is ahead of its objectives (104%). The contribution of medical universities in knowledge production was 60% for type one, 31% for type two and 9% for type three.

In early of 2015, the number of approved research center is 587 and this number is ahead of its objective (111%).

The number of adjusted researchers in universities of medical sciences and research centers in early of 2015 is 23099 and is ahead of its objective.

- Identifying health research priorities:

All of universities of medical sciences set the health research priorities .The total number of collected priorities from the universities was 6723 that were aggregated to nine main areas consist of communicable diseases, non-communicable disease, Health System Research, pharmaceutical sciences and industry, basic science, traditional medicine and herbal medicine, nutrition, environmental health, and dentistry were extracted(4).

- Approved projects based on priorities:

In 2013, the total number of approved projects was 14423, which 86% of them were based on priorities. Only for 4 universities of medical sciences, this percent was less than 70.

- Establishment the ethical committee:

All of universities of medical sciences established the ethical committee in their universities.

- Refer the projects to ethical committee:

In type one of medical universities, 85% of projects have been referred to the ethical committees. This percent for type two was 57 and for type three was 49.

Unique features of web based evaluation system are as follow:

- Creating a comprehensive and integrated system of data collection in research consist of faculty member’s CV, research articles, books, paper conference, innovation, research ceremony prizes, research projects, ethics and student research activities of universities of medical science, research centers and research institute.

- Conducting the web based qualitative arbitration to verify the research output.

- Ability to Quick and updated research evaluation reports. These reports are available in various formats such as table, figure and so on. These reports allow simultaneous comparison among universities.

- Creating a centralized portal of research management

- Reducing information gaps in research products

- Creating the evidence based decision making

- Providing the dynamic self-evaluation research system for medical universities and research centers

Discussion

In this study, two main processes have been explained. At first, the process of governance indicators’ evaluation and other, web based evaluation system as the main component of HRS functions.

In related to the first, some issues are important. Using ENHR approach as an integrated and systematic method for organizing and managing the global health research priorities at national level is very noticeable. This approach was introduced by the Commission on Health Research for Development (COHRED) in 2001 and it is an appropriate tool for health promotion based on equity and social justice(4). Gathering all of research priorities from universities of medical sciences and integrated to national priorities were done based on mosaic model. This model is a communication model that was introduced by Becker. In this approach, information mosaic or unit forms the information layer and this layer is the evidence for decision-making or judgment(12).

Considering the upstream documents such as map of health science, fifth development plan and so on is another issue that is very valuable for designing the main bases of governance indicators(13).

Set the health research priorities and approving the projects based on them, is effective step towards health promotion and it is the same goal of HRS(4).

Reviewing the results showed that almost all of indicators are ahead of their objectives, it’s may be due to creating facilities such as increasing the number of indexed journals in ISI, Pub med and Scopus, expedite the arbitration process, facilitate the article publishing in national accredited journals.

It seems that, this is the time for action to increase quality of scientific documents and it is necessary to revise some research indicators such as providing clinical guidelines, designing policy brief to facilitate decision making based on evidence.

Comparison of online evaluation system with traditional method implies that web based evaluation has more speed, accuracy, responsive and cost effective. This system as a unique portal can facilitate access to health research output for all of researchers and appropriate evidence and input to decision making for policy makers (14).

Conclusion

Web based research evaluation is a rapid, accurate and cost effective method to research promotion.

Ethical considerations

Ethical issues regarding plagiarism, informed consent, misconduct, data fabrication, double publication and/or submission, and redundancy have been completely observed by the author.