Health Care Managers' Perspectives on the Sources of Evidence in Evidence-Based Hospital Management: A Qualitative Study in Iran.

BACKGROUND: Evidence-based management (EBMgt) has been developed as a management framework for improving the quality of management decisions. To use that, we need to identify the source of evidence in decision-making. Therefore, the purpose of this study was to identify the sources of evidence in managing hospitals.
METHODS: Qualitative methods were used to explore the sources of evidence and to identify hospital managers' attitudes towards evidence-based management. A series of semi-structured interviews (n=48), with a purposive sample of 48 participants, were conducted in 2016. Also, four focus group discussions (FGDs) were conducted with health managers and specialists in the field of management. A questionnaire was used for collection of demographic characteristics and managers' perspectives.
RESULTS: Six main themes emerged from the interviews including: scientific and research evidence, facts and information of hospital, political-social development plans, managers' professional expertise and ethical-moral evidence. Also, the results showed that the majority of participants believed to use the evidence-based hospital management (95.83%).
CONCLUSIONS: Our study suggested that a full evidence-based hospital manager someone who is using all the sources of evidence for making hospital decisions. Using hexagon of evidence sources, managers can identify the best available evidence for hospital decisions and to make the best decision in the process of evidence-based decision making.

Introduction

In the last 20 years, there has been an intensive attempt toward expanding new frameworks and patterns for organization and administration, especially in health care organizations (1–6). One of these patterns is Evidence-Based Management (EBMgt) for managing health care organizations, especially hospitals (2,7–9). ‘Evidence-based’ is originally derived in the 1990s in medicine, though nowadays, its doctrines develop across disciplines such as training, nursing, criminology, social science, health economics, health policy and public policy. In hospital management, however, EBMgt is still in its infancy (8,10).

The basic concept of EBMgt is that high-quality decisions need both critical thinking and the use of the ‘best available evidence’. This ‘evidence’ may come from scientific research, but internal hospital information and even professional experience of administrators also constitute ‘evidence’. Similar to evidence-based medicine (EBM), evidence in EBMgt supports a tool to answer the questions regarding to likely outcome of a decision (11–13).

On the other hand, hospital administrator decisions in healthcare management have a significant impact on the effectiveness of quality patient care delivery and the success of hospitals. Researchers stated that an EBMgt would improve the competence of the decision-makers and their motivation to utilize more scientific methods in hospital management decision-making. Therefore, hospital managers are forced to use the Evidence-Based Health Care Management (EBHCMgt) in order to attain the effective management. EBHCMgt improves quality of organizational decisions as a bridge from theory to practice, and it has a decisive impact on hospital performance (2,8,12–18).

Actually, we live in an era of “evidence-based everything”, and that everything-medicine, management, economics, nursing, dentistry, health policy and management, decision making, and the other sciences-have become evidence-based (1, 15). Hospital manager decisions have an important impact on the effectiveness of quality care delivery and the success of hospitals (8).

Hospitals continue to confront dynamic surroundings, partially from recent changes in the political and regulatory evolutions (19). According to powerful evidence, hospital administrators need to develop the professional skill, however, a specialized discipline demanding fit training and skills and cannot be particularly acquired on the work. Nonetheless, the majority of hospital administrators are not managers (20,21).

There are many perspectives about EBMgt. These perspectives have been inspired by researchers and experts in management and organization sciences. Axelsson (1998), Konver (2000), Walshe and Rundall (2001), Pfeffer and Sutton (2006), Rousseau (2006), Briner (2009), Barends et al (2015) and Wright et al (2016) have all explained the concept, applications and components of EBMgt in their research in recent years.

Based on the evidence searched from the literature, the movements of EBMgt have been split into three phases (8,15,22,23): First Phase (1998 – 2005), the first phase of the movement of EBMgt ranged from 1998 to 2005. During this first phase, researchers began to introduce the idea of EBM and to argue the applications of the principles of EBM to healthcare management practice. Second Phase (2006 – 2012), the second phase of the EBMgt movement continue from 2006 through 2012. During this period, more scholarly publications about EBMgt, including research articles and books, were produced. Third Phase (2013– future), the third phase of the EBMgt movement in healthcare management has gone forward from 2013 and will continue in the future.

Finally, EBMgt has been defined as “making decisions through the conscientious, explicit, and judicious use of four sources of information: administrator expertise and judgment, evidence from the local context, a critical assessment of the best available research evidence, and the perspectives of that population who might be affected by the organizational decisions” (5). The purpose of this article is to identify the sources of evidence in evidence-based hospital management. The goal of this study is to contribute to understanding of managers' attitudes to EBHMgt and identifying sources towards the use of research results, in order to increase its use and adoption in hospital management. Identifying the sources of evidence can be useful in management and hospital decisions. The sources of evidence specify the type of management style regarding the EBHMgt. This study follows the best sources of evidence to determine the best available evidence.

Methods

Study design and sampling: We used qualitative methods to explore the sources of evidence and to identify managers' attitudes to EBHMgt. A series of semi-structured interviews (n=48), with a purposive sample of 48 participants, were conducted in 2016. Also, the study team conducted four focus group discussions (FGDs) (nine participants in three FGDs and six participants in one FGD) with health managers and specialists in the field of management. Participants consisted of policy-makers and administrators of the MOHME (Ministry of Health and Medical Education), research managers and policy-makers elsewhere, hospital administrators, specialists in the field of health policy and management, experts of health research centers and experienced administrators. Main descriptive characteristics of these participants are summarized in Table 1.

Table 1

Characteristics of the research participants

States (n= 7)	Organizations (n= 15)	Number
East Azerbaijan	ICEHM	9
East Azerbaijan	Iranian EBM Centre of Excellence	4
Tehran	MOHME	3
West Azerbaijan, Qazvin and Tabriz	Health Care Organizations	9
East Azerbaijan	Tabriz University of Medical Sciences	4
Tehran	Tehran University of Medical Sciences	4
Yazd	Yazd University of Medical Sciences	1
Hamadan	Hamadan University of Medical Sciences	3
West Azerbaijan	Urmia University of Medical Sciences	1
Tehran	Iran University of Medical Sciences	5
Ardabil	Ardabil University of Medical Sciences	1
East Azerbaijan	University of Tabriz	1
Tehran	University of Tehran	1
Tehran	Tarbiat Modares University	1
Tehran	Allameh Tabataba'i University (ATU)	1
Total		48

A formal letter was sent by Iranian Center of Excellence in Health Management (ICEHM) about the objective of the study and satisfaction of experts for participating in the interview and FGDs to the seven states (Tehran, West Azerbaijan, East Azerbaijan, Qazvin, Ardabil, Yazd and Hamadan). Fifty-nine experts were invited by the ICEHM; forty-eight staff agreed to be interviewed. Table 1 shows the characteristics of the research participants.

Data collection: For the aim of data collection, FGDs were used for health managers, and in-depth interviews were held with policy-makers and managers. Data was gathered through verbatim transcripts of semi-structured recorded interviews. Before interviews were conducted, connections were contacted with several top and middle managers of each organization to identify, name and define a set of reform initiatives, which were currently progressed. Each interview and FGD took between 60 and 90 minutes to finish. The interviews continued until data saturation was achieved. The interviews and FGDs were conducted by one researcher and one note-taker.

The interview guides included questions on the sources of EBHMgt and implementation procedures. To designate the guide for the in-depth interviews and FGDs, the knowledge translation model in the Tehran University of Medical Sciences (TUMS) was used (22). Our research questions (RQ) to consider the managers' perspectives on the sources of evidence in EBHMgt include the following items:

RQ1: What are the managers' perspectives towards EBHMgt?

RQ2: What are the sources of evidence in the process of evidence-based decision making?

RQ3: What are the managers' perspectives towards the applicability of sources of evidence?

Also, the questionnaire was used for collection of demographic data and managers' perspectives on the EBMgt. These questions are regarding the sources of evidence, current knowledge and attitude of participants about EBMgt.

Analytical approach: The structure of this qualitative analysis is based on previous knowledge about EBMgt with the purpose of surveying the sources of evidence in the EBMgt for better understanding EBMgt sources in the hospitals. Therefore, the research team applied the deductive content analysis process when coding the interview data, using the theoretical framework of previous studies.

Firstly, all the texts were read, and then sources of evidence were coded as themes. Finally, homogeneous themes were combined and categories were created. All the themes and categories transpired from the data. To perform triangulation, a weekly research meeting was held to evaluate interview status and reflect on and to reach consensus related to any interview coding subjects. During the coding process, the researchers made an initial cross through the transcripts followed by an expressiveness of coding and assignment criteria. Next, they re-evaluated the code assignments and made corrections based on the definitions resulting from discussion in research sessions.

All the documented in-depth interviews and FGDs were reviewed by two researchers of the study group separately to ensure the reliability of the analysis. In matters of disagreement, the team made the final decision on choosing the theme. Also, descriptive statistics were used to present quantitative and qualitative variables respectively. Data entry and analysis were done using SPSS-18.

Ethical considerations: The Ph.D. proposal was approved by the ethical committee of Tabriz University of Medical Sciences (Ethical code of project: TBZMED.REC.1395.497). After the aims of the study had been explained, verbal agreement was obtained from the participants at the beginning of each meeting. They were also told that the voice-recording could be paused at any matter they supposed necessary.

Results

The 48 interviews were managed between June and December of 2016 and included participants who had between 1 and 39 years of health care experience. The interviewees were about gender including 19 women and 29 men. The majority of the participants were Ph.D. holders (n=29). The findings showed that the mean age was 39.55 (9.37) years. Also, the mean work experience of participants were 11.12 (8.48) years. The mean health care management experience of participants were 5.77 (6.49) years. For a more detailed profile of the interviewees, please see Table 2.

Table 2

Profile of interviewees

Demographics (n = 48)		Frequency	%
Gender (percentage)	Male	29	60.42
	Female	19	39.58
current occupation group (percentage)	Managers	13	27.08
	faculty members	21	43.75
	Both	14	29.17
Highest level of educational degree (percentage)	Masters	3	6.25
	Ph.D.	29	60.42
	MD	9	18.75
	MD, Ph.D.	5	10.42
	MD, Specialists	2	4.16
Main expertise and skill (percentage)	Strategic planning	11	22.92
	Change management	7	14.58
	Process improvement	5	10.42
	HRM	11	22.92
	Quality management	3	6.25
	Policy making	4	8.33
	Financing	5	10.42
	Accreditation	2	4.16

In Table 3, descriptive data have been shown. As Table 3 indicates, most respondents base their decisions on management literature (95.83%), scientific research (91.66%), knowledge acquired through formal education (84.41%), personal judgment (68.75%) and advice from colleagues (more than 60%).

Table 3

Descriptive information of participants about attitude toward EBHMgt

Questions		Frequency	Agree %
On what do you base the decisions you make as an administrator?	Test-error	6	12.5
	Intuition-insight	11	22.92
	Personal judgment	33	68.75
	Acquired knowledge	41	84.41
	Consult with internal colleagues	30	62.50
	Consult with external colleagues	31	64.58
	Management literature	46	95.83
	Internet	22	45.83
	Scientific research	44	91.66
	Business Source Premier from EBSCO	10	20.83
With which of the online databases are you familiar?	Science Direct from Elsevier	19	39.58
	PsycINFO	11	22.92
	Inter Science	10	20.83
	ProQuest	31	64.58
	Cochrane	9	18.75
	CINAHL	11	22.92
	Springer	29	60.42
	Ovid	26	54.16
	PubMed/Medline	41	85.41
	Scopus	33	68.75
	Web of Science	41	85.41
	Embase	10	20.83
	Emerald	20	41.66
	SID (Iranian)	46	95.83
	Magiran (Iranian)	45	93.75
	Google Scholar	48	100.00
Do you have experience conducting scientific research?		41	85.41
Was there special attention given to scientific research in your formal education?		29	60.42
What percentage of your decisions as a manager, do you feel is based on findings from scientific research?		30	62.50
Do you believed ethical evidence?		46	95.83
Would you like health care management be evidence-based?		46	95.83
Do you regularly search the databases online? (Once a week)		35	72.91
Do you regularly read a research article? (once a day)		12	0.25
Do you feel is based on findings from scientific research in your daily?		36	0.75
Are you familiar with journal of health management? (More than twenty Journals)		20	41.66

The minority of those participants who were interviewed replied that they base their decisions on results from test-error (12.50%). The findings showed that most participants are familiar with online databases. According to online databases, familiarity level with them was different among participants. Hundred percent, however, are familiar with Google Scholar. Also, they were familiar (more than 85%) with other databases include PubMed/Medline, Web of Science, SID and Magiran. Only 18.75% were familiar with Cochrane database. Other results are shown in Table 3.

The results of the synthesis show the main themes, sub-themes and quotations for sources of evidence in EBMgt. According to the final codes which these were extracted from the text, sources of evidence were divided into six categories that include the following: Scientific and Research Evidence (SRE), facts and information of hospital, political-social development plans, managers' professional expertise and Ethical-Moral Evidence (EME).

Scientific and Research Evidence (SRE): The first source of evidence is scientific and research evidence published in academic journals. Also, scientific evidence related to hospital management is main-quality in decision-making and managing hospitals. According to three of the participants:

-“The scientific and research evidence are the first source of evidence. What matters is managers have to learn the search and research skills.” (MD, Ph.D., manager and faculty member)

-“Research evidence is so much important such that a number of the academy of management and decision-making in the world accepted only the research evidence as evidence. [...] it must be stated that observations on some of the problems can be enough evidence.” (PhD, manager and faculty member).

-“Absolutely, research articles that have been published in the journals of high level (High impact); those are the best source of evidence.” (PhD, manager and faculty member).

The sub-themes for this source are academic journals, scientific evidence, research evidence and sometimes observational evidence. This evidence assists hospital managers in organizational decisions.

Facts and information of hospital: Hospital information system, management dashboard, internal evidence, data and facts, supportive team of hospital and questionnaires and checklists have been identified as facts and information of hospital and a source of evidence in EBHMgt. One participant stated, “One of the best sources of evidence is a supportive team for managing the hospitals that it should be able to collect data and documentations. Indeed, the supportive team should be fact-based. [...] For example, the employee motivation may be low that we can identify their reasons using questionnaire.” (PhD, and faculty member).

Political-social development plans: According to one participant stated, “If, we want to decide based on the evidence for managing our hospital effectively, we should use a political-social program for strategic decision-making. For example, Health Sector Evolution Plan (HSEP) is a political-social program.” (MD and manager).

We identified three sub-themes for this source of evidence: government laws, political-social programs and programs of MOHME. Hospital managers and faculty members believed that political-social development plans are a high-quality source in decision-making.

Managers' professional expertise: A forth source of evidence is the professional experience and judgment of hospital administrators. Managers' professional experience differs from intuition and personal opinion because it reflects the specialized knowledge acquired. Another participant highlighted the importance of the source in managing hospital, stating: “Although, I'm professor of health system management, but I have to say that share of the experience in hospital administration is 50 percent that refer to skill and profession.” (PhD, Specialist, manager and faculty member).

Ethical-Moral Evidence (EME): One participant added: “Depending on the situation, I believe that religious and moral principles will help to administrators in the financial management, human resources management and organizational Behavior […]. For example, holy books are a good evidence for some of the problems especially, in organizational behavior.” (PhD, and faculty member).

A fifth source of evidence is ethical-moral evidence. The EME can help to manage organizational behavior and interaction among staff (clinical and non-clinical). This evidence is rooted in the beliefs of the staff and managers that easily influence the attitude and consequently the skill and competency. The source of evidence considers the ethics and religious beliefs. Another participant stated: “Ethics in EBM means that physicians must follow the best evidence for diagnosis and treatment. Also, hospital administrators need to look the best evidence for running hospitals.” (MD, Specialist and manager).

Values and expectations of all stakeholders: A sixth source of evidence is values and expectations of all stakeholders. Stakeholders are any individuals or groups who may be affected by an organization hospital's decisions and their outcomes. Internal stakeholders include employees (clinical and non-clinical staff), administrators and board members. Stakeholders outside the hospital such as suppliers, patients, outpatients, other hospitals and health care organizations, shareholders, the government and the public sector at large may also be affected. Values and expectations of all stakeholders reflect what stakeholders believe to be significant, which in turn affects how they attend to react to the possible outcomes of the hospital's decisions. According to one participant, “The difference between EBMgt and EBM is that sources of evidence used for decision-making in EBMgt are stakeholders' concerns and values, while EBM use the patient values and preferences.” (Master, manager).

Discussion

In this study, we identified the sources of evidence in evidence-based hospital management, from researchers' and managers' perspectives. In Barend's study, most respondents base their decisions on personal experience (94%), knowledge acquired through formal education (71%), intuition (67%), advice from colleagues (64%), or insights provided by experts (63%). The minority of those managers interviewed replied that they base their decisions on findings from scientific research (33%) (24).

Our study indicated that most respondents' base their decisions on management literature (95.83%), scientific research (91.66%), knowledge acquired through formal education (84.41%) and personal judgment (68.75%). According to the principles of evidence-based practice, evidence from four sources should be taken into account: scientific evidence, organizational evidence, experiential evidence and stakeholder evidence (2,3,8,10,13).

In our study, this classification was reformed, and according to interviews the sources of evidence was classified into the six scopes. Evidence-based hospital managers are similar to spiders in that they are surrounded on all six sources of evidence (Spider's webs). As shown in Figure 1, we recognized the sources of evidence and management scopes in EBHMgt. According to utilization rate of the evidence sources, hospital administrators were classified into six domains. Depending on the type of problem, using the sources of evidence in EBHMgt, managers will select the best available evidences and their sources.

Figure 1

The sources of evidence and management zones in evidence-based hospital management

As shown in Figure 1, according to the six sources of evidence, the managers place in the areas sextet of web. The managers that use the source of SREs are known as scientific-oriented managers. The hospital managers who utilize only the fact and information of hospitals are known as data and fact-based managers. Plan-based managers are those who rely on the political-social development plans. The managers are highly regulatory and inflexible. Many managers take strictly the need to reflect critically on their experiences and seep the practical examples. They are expertise-oriented managers. Also, in this study, we identified the new sources of evidence in the EBHMgt. The hospital administrators who use the EMEs are known as ethics-oriented managers. Finally, the last management area in the EBHMgt is related to shareholder-oriented managers. They are extremely customer-focused and respect their values, specifically.

A full evidence-based hospital manager is someone who utilizes all the sources of evidence in decision-making. It should not be disregarded that hospital administrators due to the problem and population use the best sources of evidence (15,25). Hospital administrators who use only one source of evidence are monopoles. Depending on the type of sources, managers may decide on one or several areas in all the managerial decisions. In this study, the sources of evidence have been identified for improving the use of evidence in managerial decision making in hospital management and suggested that evidence-based decision-making is important in improving the quality of management decisions.

In totality, positive attitude toward evidence-based hospital management, adequate and suitable knowledge from hospital and management and EBHMgt skill are necessary for implementation of EBHMgt model. Hospital administrators are like a fisherman. Therefore, a professional fisherman should be interested, trained and mastered. Afterwards, the fisherman can catch the best available fish. Almost all the participants (n=46) believed that health care management can be evidence-based and had positive attitude towards EBHMgt. Also, it must be noted that evolutions should be based on the evidence; we require the evidence-based health managers that EBHMgt foundation is professional and expertise educations (25–27).

In order to develop the adoption and utilization of evidence-based management, hospital managers need to promote a culture that facilitates managers to dedicate time to consult scientific evidence; educational institutions need to focus on improving EBHMgt skills that are needed to find, read, assess, and apply scientific evidence. Also, universities need to train future academics methods needed to critically appraise and summarize the best available evidence on a issues relevant to the best practice(11,17,25).

Our study suggests that most participants have positive attitudes towards EBMgt and that an enormous majority believes that using evidence-based hospital management can improve the quality of hospital decisions. Since EBHMgt approach is emerging, its practice among hospital administrators remain limited. Several factors existing at the extra-organizational level, the organizational level and the personal level have played considerable roles in affecting such lack of skill.

We know that many hospital managers lack the skills of sources identification required to use EBHMgt. Thus, it must be trained by educational organizations, academics and research institutes. Using hexagon of evidence sources, managers can identify the best available evidence for hospital decisions and in an evidence-based decision-making process to make the best decision. Furthermore, to increase the benefit and utilization of EBHMgt, training organizations, universities of medical sciences, health care centers and research institutes must more actively involve hospital managers in setting research plans and guiding the appropriate presentation of research evidence to facilitate the interpretation of research evidence created to improved management practice in the hospitals.