Assessment of research systems in universal health coverage-related organizations.

Background: Universal health coverage (UHC) is the desired goal of achieving universal access to health services without having to endure pain and financial difficulties. Multiple factors can help steer countries toward UHC. One of the most important factors is the production of valid quality evidence that can be achieved through research. The present study aimed at outlining the status of research systems in UHC-related organizations and identifying the barriers faced by research.
Methods: The key individuals and organizations that could provide rich, relevant, and diverse data in response to the research question were purposively selected for the interviews. Thematic analysis was used to analyze the interviews. Categories and subcategories were deductively extracted from the text based on research system performance as follows: resource provision, production and utilization of knowledge, existing resources, and stewardship. Then, the themes were inductively extracted from the interviews.
Results: Many barriers existed for performing research in UHC-related organizations. The stewardship barrier seemed to play a key role such that structural changes in organizations affected the production & utilization of evidence. Limited financial and human resources were evident in most of the organizations. Research questions were not comprehensively identified. The conducted studies either were not designed to answer the relevant questions and/or were not appropriately reported to policy makers. As a result, their implementation in decision- making did not reach the ideal status.
Conclusion: Research utilization aimed at achieving UHC will come to realize only when and if research is conducted to produce evidence required for decision-making and implementation. Therefore, in addition to the interventions recommended by the World Health Organization, we should design and implement interventions tailored to the local barriers and needs of UHC-related organizations.

↑ What is “already known” in this topic:

Many factors can help countries come closer to Universal Health Coverage. One of the most important of which is the production of valid quality evidence. Research for Universal Health Coverage is crucial for exploring, developing, and proposing interventions to sustain public health.

→ What this article adds:

This paper outlined the status of research systems in UHC– related organizations and identified their system’s existing barriers.

Introduction

Universal health coverage (UHC) is the desired goal of achieving universal access to health services without having to endure pain and financial difficulties. ‘Coverage’ refers to a wide range of services including promotion, prevention, treatment, rehabilitation and palliation, particularly health-related Millennium Development Goals (MDGs), and those related to non-communicable diseases and injuries (1). Three general indicators have been defined for achieving UHC: (1) equity in access to health services (those who need health services irrespective of whether they can or cannot afford them should receive them); (2) quality of healthcare services (health services should be good enough to improve the health status of those receiving them); and (3) financial risk protection (the guarantee that health service costs do not expose people to financial problems) (2).

Many factors can help countries come closer to UHC. One of the most important of which is the production of valid quality evidence. Production of evidence through research and its synthesis and analysis for the purpose of informed decision- making and health system management can be helpful. A wide range of research types is warranted for achieving UHC. To achieve UHC, research should be able to respond to 2 groups of questions. The first group focuses on how we should select the services that must be delivered and how we can increase the coverage of those already provided. How can we promote the financial risk protection status? How can we sustain and promote people’s well-being? The second group of questions focuses on which method is the best for measuring UHC and financial risk protection under any circumstances? How do we know when we have achieved the UHC goals? (2).

Research for Universal Health Coverage is not a luxurious concept. Indeed, it is crucial for exploring, developing, and proposing interventions to sustain public health. Many of the questions concerning access to healthcare services and financial risk protection status require local answers. Hence, in addition to the utilization of research evidence, countries need to conduct research in this field (2). In addition to the availability of research evidence to policy makers, they need to be utilized if UHC is to be achieved. To this end, to encourage UHC – related organizations to produce and utilize local evidence, we should first identify and acknowledge them. Thus, the current study was conducted to outline the status of research systems in UHC – related organizations and identify their existing barriers.

Methods

The following steps were taken:

1. Identification of UHC–related Organizations: The relevant organizations in Iran were identified keeping in mind the “result chain for UHC, focusing on outcomes” (2) (Fig. 1).

Fig. 1

A Representation of the Results Chain for Universal Health Coverage, focusing (2)

2. Development of a Guide for the Semi-structured Interview: The guide was developed based on the stages of research (identification & prioritization of research questions, research conduction, assessment and evaluation of results, dissemination of results, and implementation of results) and a research prioritization procedure assessment checklist (3, 4‏).

Interview

Sampling: The key individuals and organizations that could provide rich, relevant, and diverse data in response to the research question were purposively selected for the interviews. To increase the number of the initial sampling, snowball sampling was used. The researcher interviewed a particular participant and after recording his/her views on the topic, then, the interviewer asked the participant who to interview next to obtain more information on the topic.

Data Collection

The main data collection tool in this study was in-depth interviews. We used in-depth interviews to study interviewer’s experiences and explore personal and sensitive themes. The researchers interviewed the participants in their workplaces separately from February 2015 to April 2015. The approximate length of the interviews was 45 to 60 minutes. To adhere to scientific principles in the interviews, an interview guide was prepared to facilitate the process and was pilot-tested. The main question asked was about the participant’s respective organization’s research system performance. All the interviews were recorded upon obtaining verbal consent from the interviewees. The audio file of each interview was numbered by a special code, archived, and later transcribed.

Data Analysis

Thematic analysis was used to analyze the interviews (5). Categories and subcategories were deductively extracted from the text based on the research system performance as follows: resource allocation, productivity and utilization of research, existing resources, and stewardship (6). Then the themes were inductively extracted from the interviews.

Results

We found that the UHC–related organizations in Iran are Ministry of Health & Medical Education (MOHME) and other organizations including the Social Security Organization, Ministry of Welfare, the Imam Khomeini Relief Foundation (IKRF) and the Red Crescent Society. The MOHME further consists of the following divisions: Deputy for Public Health, Deputy for Treatment affairs, Food & Drug Organization (FDO), Medical Equipment Division, Health Technology Assessment (HTA) Office, Office of Standard & Tariff Affairs, and National Institute of Health Research (NIHR). Nine key individuals (experts/representatives) from the following organizations participated in the interviews: MOHME, Social Security Organization (SSO), Iranian Health Insurance Co., Armed Forces Insurance Co., and IKRF. Then the interviews were analyzed on the grounds of 4 aspects of health research systems including barriers in health system financing, productivity and utilization of research, resources, and barriers at the stewardship level‏.

Barriers Related to Financing: (sustainable & transparent procedures for attaining and allocating financial resources to research) (6).

The research budget was provided by the organization itself in most interviewed cases. The MOHME divisions did not have an independent budget for research, and financial resources were provided through 3 channels: MOHME contingency plans, international support, and universities.

Barriers Related to Productivity and Utilization of Research: (production of knowledge required to solve the health system’s problems and the translation of produced evidence into policies and tools) (6).

Results of the thematic analysis are as follow

In this section, the barriers are presented in 6 parts: identification of research questions, prioritization, research conduction, assessment and evaluation of research, dissemination of research results, and implementation of research results‏.

Identification of Research Questions

In most organizations, there was no structured guide to explain in detail and step-by-step how to perform the process of identifying research questions. Furthermore, the sources used to identify research questions were experienced individuals working inside the organization. Exceptions were the Armed Forces Insurance Co. and IKRF, which used data mining methods to identify problems, and the HTA Office, where questions were sent by the MOHME deputies and relevant companies. In the other divisions of MOHME, research questions were not systematically identified and were designed based on problems that had arose and the demands that had been made.

Prioritization of Research Questions

In most organizations, there was no structured guide to explain in detail the process of prioritizing research questions. The results of prioritization and the method with which it was done (method of selection) were not presented to extra-organizational stakeholders. However, the lists of research priorities were displayed on the organization’s website by the SSO and Iranian Health Insurance Co. annually. The HTA office prepared a list of research priorities and sent it to the NIHR every few months, and that list, too, was displayed on the institute’s website.

In some deputy offices of MOHME, priority-setting was not desirable either. The authorities believed that the plans underway were either necessary or being executed as a result of insistence of higher-level managers. As there were many priorities that had not been tended to, they did not need new priority-settings.

In most organizations interviewed, no protocol had been foreseen for objecting to priority-setting results and/or debate by extra-organizational stakeholders.

Research Conduction

Mostly, the proposals were not written in coordination with groups or individuals who had put forth the research question. This collaboration only took place in the SSO. In some organizations, applied researches (that needed quick results) were conducted by administrative and not research staff; however, no contracts were signed for such studies either. MOHME divisions directly placed their research orders directly with academic researchers. This leads to the diversity in methods and procedures and also raises the possibility of duplication. Only HTA research topics were handed over to researchers through the NIHR.

Research Assessment and Evaluation

Generally, assessment and evaluation were done through presentation in meetings, and there was no documented protocol for the review process. Only the SSO had a protocol in which a scoring system was used, and the reports that scored lower than 60 out of 100 were rejected.

Dissemination of Research Results

Most proposals and final reports did not specify the stakeholders of the results. There was no explicit place for outlining the research message and stakeholders in the final report format. Active dissemination of results to extra-organizational stakeholders was almost absent. Even in MOHME, the results were not systematically presented to the other sectors and only the HTA office displayed its research results online. Moreover, there was no specific protocol to object to research results, and no focused center existed where policy-oriented research studies were searchable.

Implementation of Research Results

No specific plan existed for implementing research results. On the other hand, policy makers and managers did not have an appropriate attitude toward evidence-informed decision- making, and used research results as they liked. No research had been conducted on how much of the research results had been implemented in any of the organizations interviewed.

Barriers Related to Resources (expert human resources for research and organizational capacity for research) (6‏).

Some research studies were conducted by the administrative personnel, but a large proportion was conducted by extra-organizational researchers. The interviewees believed there was a grave shortage of human resources with an expertise in research. They also thought the research studies conducted by well-known researchers were not acceptable because they were conducted by co-investigators and students, and the principle investigator did not directly supervise the investigation. Organizations had either given up on this matter or signed contracts with final-year doctorate students who they trusted would conduct the projects entirely by themselves‏.

Barriers Related to Health System Stewardship (a strong leadership that can steer, coordinate, and manage research) (6).

Non-homogeneity existed among the definitions of activities of various MOHME offices. Some were of evidence production nature, and some were of executive nature. This led to a variation in evidence utilization in different decision-makings. Moreover, over the years, structural changes followed by changes in research capacity had hindered the conduct of research. For example, in one non – MOHME organization, many projects had reached the stage of being handed out to extra-organizational researchers. However, political and managerial changes and financial challenges hindered the contracts from being signed. Furthermore, the lack of an outlined structure for integrating research into MOHME led to a state in which decision-makers acted based on their personal preference toward implementing research results in their decisions. Most interviewees from MOHME were of the belief that the Applied Research Secretariat was responsible for the research process in MOHME, which is currently non-existent. Ever since the Applied Research Secretariat (ARS) was dissolved and became a subgroup of other sectors, it did not have an outlined and effective job description, as its budget and mechanism were not specified at the time of changing the structure. In the interviewees’ opinions, the reasons behind dissolving the ARS were as follow‏:

Lack of clear-cut criteria for prioritizing topics in the Health Deputy’s research council

Inappropriate method of writing Request for Proposals (RFP), and because academic researchers researched independently from the offices and departments, the results were not in line with the policy-makers’ problems and demands‏.

The overall demand was in the form of large projects that would be broken down into smaller components, each of which being independent of each other and of the policy-maker, hence, the desirable result would not be product‏.

Among the other stewardship-related barriers was lack of a need for implementation of policy or program. In the best case scenario, the pilot study was shortly conducted. They may have foreseen implementation consideration, while developing the programs, but it is crucially important to pay significant attention to this issue independently.

Despite the introduced barriers, some of the organizations interviewed have taken certain steps to improve their research system toward meeting their needs. These interventions are as follow:

Establishing the Center for Insurance Studies

Re-establishing the SSO’s Higher Institute for Research to meet the organization’s research needs

Establishing the Center for Insurance Studies‏

Restoring the cultural deputy that is responsible for facilitating work with extra-organizational stakeholders‏

Establishing the research body under the title of ‘Professional Bureau for Health Insurance‏’

Developing the horizon scanning process to identify HTA research questions‏

Developing HTA-relevant guidelines and incorporating guideline directives into the information technology system overlooking diagnostic and therapeutic services

Providing full insurance coverage of guideline-based services‏

Ordering projects to researchers who personally do the research work

Developing RFP content by organizational managers

Designing the economic assessment form that should be filled by the HTA applicant

Beginning to develop priority-setting criteria and the procedure guide in some organizations

Removing limitations in the number of ‘call for proposals’ by the NIHR

Discussion

The decisions made by UHC–related organizations have profound effects on universal health coverage. Upon reviewing the research procedure in these organizations, we concluded that 2 main functions (production of research evidence and its implementation) are influenced by financial and human resources and the stewardship approach.

Based on our findings, there are many barriers to conducting research in UHC–related organizations such as limitations in financial and human resources that are evident in most organizations. Research questions are not comprehensively identified, projects are either not appropriately designed to respond to the relevant questions, or not appropriately reported to policy- makers, so their implementation in decision-making is not ideal. The stewardship-related barrier plays a key role in organizations such that structural changes in organizations undermine research. However, the reverse also holds true. With the given information, the following question arises: What should we do to prevent changes in the research structure and capacity subsequent to changes in management?

Although some organizations outside the MOHME do have an outlined structure for research, MOHME does not have a central structure for this purpose. These organizations have a specific structure for stewardship and leadership, which is in fact their R & D (Research & Development) unit. The barriers identified in the knowledge production and utilization sector all fall under the umbrella of 1 organization (Although academic researchers conduct the studies, the stewardship of the research process is in the hands of the same organization.). However, in case of MOHME, the research projects are conducted by the university, and hence the barriers of knowledge production section lie outside the MOHME’s jurisdiction.

According to the current study, one of the important reasons behind the dissolution of research bodies was the lack of achievement of goals set by those bodies. Therefore, in addition to thriving to succeed, the leadership of any newly-formed research body should measure and document its rate of success.

General solutions have been recommended by the WHO in 2013 to improve the research production and utilization procedure to promote UHC (2) as presented below:

The recommended steps to take toward research at national level include research priority-setting for UHC, strengthening research capacity, setting research conduction standards, translating knowledge into policy and practice, ensuring collaboration, and a general understanding of research‏.

The recommendations made to support research at national and international level include research monitoring, research coordination and data exchange, research financing, health research management, and stewardship‏.

Although these solutions are correct and appropriate, they should be applied collaterally along with those solutions that target the identified barriers.

Barriers to knowledge utilization in policy- making are not a new phenomenon and it is not limited to specific countries either. Multiple studies have been conducted in developing countries so far, and as expected, the barriers identified here overlap with those already identified (Table 1). Some of these barriers exist in the knowledge-producing organizations, others are present in the knowledge -utilizing organizations, and some barriers influence the way these 2 interact. Nevertheless, UHC research can improve UHC only if the previously recognized barriers are dealt with.

Table 1

Barriers Identified in the Utilization of Evidence in Health System Policy Makings

Author	Study Design	Barriers Identified
Oliver K(2014) (7)	A systematic review of barriers & solutions to evidence utilization by policy-makers	‏- Lack of clear relevant evidence and costs‏- untimely presentation of report results- policy-makers’ lack of research skills and/or awareness
El-Jardali F (2012) (8)	A survey of researchers from 12 EMRO countries who had produced articles on health system & policy-making	- Lack of budget for health research- Overt political pressure, low political will & corruption- Insufficient communication, exchange & inappropriate dialogues- Too few trained policy-makers empowered with evidence utilization
Majdzadeh R(2011) (9)	In-depth interviews & focus, and group discussions with Iranian policy-makers & researchers	Policy-makers’ characteristics:Policy-makers’ selection criteria/ incentive mechanism/ knowledge of & belief in evidence-based policy-mak2014break />Organizational values/ limited vision in decision-makings/ the effect of non-technical issues/ the executive capacity of policy/ context/ the small effect of evidence-informed policy-making on resource allocation/ resistance toward innovation/ lack of coordination between various sectors of the decision-making organizationResearch system:Lack of a priority-setting system for health research/ resource shortage/ insufficient communications between knowledge producers and decision-makers
Nedjat S (2014) (10)	A survey of Iranian authors and in-depth interviews & focus group discussions with managers, policy-makers, health service providers & researchers	Barriers related to researchers’ capabilities:Lack of awareness of knowledge translation/ lack of cooperation among researchers resulting from mistrust/ methods of research topic selection/ lack of hope toward change in target audiences/ lack of appropriate communication between researcher & decision-makerKnowledge transfer barriers:Shortage of applied & useful research/ low research quality/Lack of delivery of results to target audiencesBarriers to human resource management:Inappropriate promotion criteria for researchers/ shortage of human resources and difficulties in recruitmentBarriers to research management:Irrational priority-setting of research/ absence of a single pre-defined mechanism for delivery of research results
Imani-nasab MH(2014) (11)	Structured interviews with producers of policy-informed evidence, managers and MOHME experts	Determinant factors of evidence utilization by policy-producers:Knowledge & innovation development/ performance management/ physical environment/ hiring technicians/ colleaguesDeterminant factors of evidence utilization by health policy-makers:Use of jargon / composition of documentations & evidence/ time restrictions/ timely & relevant evidence/ administrative framework for evidence-based policy-making/ limited resources to support evidence-based policy-making
Hyder AA (2010) (12)	In-depth interviews with policy-makers through purposive sampling at national level (Iran)	Communication & dissemination of evidence:Communication/ different goals & languages of researchers and policy makers/ lack of policy-makers’ technical capacities/ limited capacity for conducting policy-related research/ resource shortage/ organizational cultureThe impact of political conditions:Legislating trend/ parliamentary machinery/ financial policies

Under the current circumstances, it seems that decision-making without research evidence is inevitable; and promoting the use of research after decision- making at organizations (persuasive use), and steering the organizations toward documentation and development of standard protocols for the research process seem appropriate. What must be kept in mind is that this approach of producing evidence for adopted decisions will only come into effect when there is a mechanism to improve decisions and policies on the basis of what is achieved from research.

The most important limitation of the present study was that no written documentation was available on the conduct of research and its implementation in organizations. Hence, the sole source of information was the interviewees, so triangulation could not be done for the data. We also tried to contact more experts, but the number of Universal Health Coverage – related organizations was low, and a small number of organizations accepted our interview invitation.

One of the strengths of this study was that a member check was done, meaning that the final report was sent to the interviewees to implement their opinions in the final version to raise the validity of our findings and interpretations.

Conclusion

If we are to exploit research to achieve UHC, we should steer organizations, whose decisions affect UHC, toward the production of evidence required for decision- making and implementation. To this end, in addition to the interventions recommended by the World Health Organization, we should design and implement interventions tailored to the country’s local barriers and needs.

Acknowledgements

This article is based on a research project that has been financially supported by National Institute of Health Research for Research under contract number 241/M/9392.

Conflict of interests

The authors declare that they have no competing interests.