An Evidence-Based Framework for Evidence-Based Management in Healthcare Organizations: A Delphi Study.

BACKGROUND: Evidence-based management (EBMgt) is a growing literature concept in management sciences which claims that management decision-making must be based on the best available evidence. The aim of this paper is to present and provide an evidence-based framework for EBMgt to improve decision-making in healthcare organizations.
METHODS: A two-round Delphi survey was used to collect the factors affecting EBMgt. Purposive and snowball sampling methods were used in both rounds. In round 1, we conducted a systematic review and a series of semi-structured interviews (n=45). In round 2, a specific questionnaire with four main parts was designed. The experts (n=21) were asked to rate on a 9-point Likert scale the importance of each factor. The data was collected through Google Forms (n=11) and paper forms (n=10).
RESULTS: Participants were mostly men (73%). Overall, 126 factors were selected in round 1. Factors were classified into 4 categories: facilitators, barriers, the sources of evidence and EBMgt process that consisted of 48, 46, 22 and 10 factors, respectively. In round 2, based on median scores, many factors (n=114) were found to be very important. Only, 12 factors have a median score of less than 3 and were excluded from the study. Finally, 114 factors were confirmed.
CONCLUSIONS: Confirmed factors played significant roles in affecting the practice of EBMgt among healthcare managers. We tried to facilitate interaction between these factors in the framework. Depending on the type of problem, using six steps of EBMgt process, managers will select the best evidence among six sources of evidence.

Introduction

There has been an intense attempt to develop new models for organization and management, especially in the last decade (1–6). One of these models is evidence-based management (EBMgt) for different organizations. The EBMgt was an evolution in the practice of management and organizations that was coined from the “Evidence-Based Medicine (EBMed)” by Sackett in the 1990s (1,3,7–11).

According to several founders of this movement, “evidence-based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients” (5,7). Now, its principles are developing across disciplines such as education, criminology, disaster management, health policy, economics and public policy (5,12). In management and healthcare organizations, however, EBMgt is still in its infancy and a large number of researchers demonstrate that it needs to develop more of a comprehensive framework (13–15).

Evidence-based management is the systematic application of the best available evidence to management decision-making, aimed at improving the performance of healthcare organizations (16–20). The use of an evidence-based approach has been proposed to get better practice of health care management by progressing in the quality of managerial decisions (8,21–23). The EBMgt seeks to have managers who make judgments based on facts and information by analyzing facts appropriately to make these judgments to a greater degree (4,11,24). Healthcare managers should learn to search and critically appraise evidence from management research as a basis for their practice(5,10,24).

Patient safety, quality of healthcare, widespread demands for decreasing the cost of care and service, and value-based purchasing all require healthcare managers to take an evidence-based approach when making decisions. It is essential to adopt an EBMgt approach for health administration. Health managers are accountable and responsible for both their patients and their healthcare organizations (25).

There are many viewpoints regarding EBMgt. These viewpoints have been inspired by researchers and experts in management and organization. Many researchers have investigated the concept, applications, gaps and components of EBMgt in their research during recent years(1, 3–5,24,26–30). Although these studies have provided extensive knowledge about EBMgt and its importance, the practical application of this topic from these studies is limited. Therefore, it is essential to introduce a practical model or framework in EBMgt.

Additionally, to facilitate the uptake of EBMgt, we need to know about managers' viewpoints and perceived barriers and facilitators towards it. Recognizing the specific viewpoints and perceived barriers and facilitators to EBMgt can help to develop new practice and skill in the workplace. In addition, to achieve this aim, it is necessary to identify factors affecting the EBMgt. Therefore, an evidence-based framework is needed to practically develop the process of decision-making. Finally, we present the framework of EBMgt for healthcare organizations that it helps to improve the decision-making process. Therefore, managers of all levels of the healthcare organizations can rely on the evidence-based framework in their decisions.

Methods

We conducted a two-round Delphi study among health managers and other experts in management sciences to confirm the sources of evidence, EBMgt process, key barriers and facilitating factors for EBMgt implementation in Iran.

The Delphi technique is based on a structured process for collecting and purifying knowledge from a team of experts by means of a series of questionnaires integrated with controlled viewpoint feedback. It is considered to be a strong methodology for achieving a rigorous consensus of experts on a specific theme. There is no consensus on the panel size for Delphi studies. In the classic Delphi approach, four rounds are used, although in previous studies, two or three rounds were preferred (31–33).

Participants consisted of policy-makers and managers of the Ministry of Health and Medical Education (MOHME), research managers and policy-makers elsewhere, hospital managers, specialists in the field of health policy and management, experts of health research centers and experienced administrators. Purposive and snowball sampling techniques were used to identify experts who could provide insight into the phenomenon from all Iranian provinces. Basic descriptive characteristics of the participants are displayed in Table 1.

Table 1

Characteristics of the research participants in Delphi.

Organizations (provinces)	Positions	Participants (round 1)	Participants (round 2)
ICEHM (East Azerbaijan)	Expert of management sciences in the center, associate professor in health policy, economics and management, managerial experience, ICEHM manager	6	6
Iranian EBM Centre of Excellence (East Azerbaijan)	Senior managers in Iranian EBM Centre of Excellence, expert in method of systematic reviews	3	0
Ministry of Health and MedicalEducation (MoHME)	Office director in the hospital management and clinical service excellence, deputy of MoHME in the field of planning, senior manager in MoHME	3	2
Health care organizations (Qazvin, West and EastAzerbaijan)	Hospital managers, faculty members in health services management	11	3
Tabriz University of Medical Sciences	Associate professor in health information management	4	3
Tehran University of Medical Sciences	Associate professor in health policy, economics and management; managerial experience	4	2
Yazd University of Medical Sciences	Associate professor in health services management	1	0
Zanjan University of Medical Sciences	Associate professor in health services management	0	1
Hamadan University of Medical Sciences	Associate professor in health servicesmanagement, Managerial experience in health sector	2	1
Uremia University of Medical Sciences	Managerial experience in health sector	1	1
Iran University of Medical Sciences	Managerial experience in hospital, senior manager in faculty of health management, faculty members in health services management	5	1
Ardabil University of Medical Sciences	Professor in the field of management science, managerial experience	1	1
University of Tabriz (East Azerbaijan)	Associate professor in the field of management science (organizational policy making)	1	0
University of Tehran	Professor in the field of strategic management	1	0
TarbiatModares University (Tehran)	Professor in the field of management science	1	0
AllamehTabataba'i University (Tehran)	Professor in the field of management science	1	0
Total		45	21

Ethical consideration: The project proposal was approved by the Ethical Committee (Ethical code of project: TBZMED.REC.1395.497). Verbal and written confirmation of guaranteed anonymity was given to the experts. A formal letter was sent through email and telegram messenger by Iranian Center of Excellence in Health Management (ICEHM) about the aim of the study and the satisfaction of experts for participating in Delphi.

Design of the Delphi study: As it is necessary to obtain the factors for designing framework of EBMgt, to identify the factors influencing EBMgt, a series of semi-structured interviews and systematic review were conducted.

Round 1: The systematic review and interviews

To identify the factors (barriers and facilitators) influencing EBMgt, a series of semi-structured interviews and systematic review were conducted. The systematic review was conducted on six electronic databases (PubMed, Web of Science, Cochrane, ProQuest, Embase, and Scopus). In addition, we searched Google Scholar, Emerald, Academy of Management (AOM), and the website for the Center for Evidence-Based Management (CEBMa) for articles related to EBMgt. Among the 26,011 records identified from database searches, 17,278 records remained after we removed duplicates, and 197 articles remained for full-text assessment. Overall, 26 studies were selected for synthesizing. Moreover, 45 semistructured interviews were conducted of which 7 were conducted by telephone. Barriers and facilitators identified in the systematic review and semi-structured interviews were categorized into 5 domains each.

The facilitators were categorized into the following areas: organizational factors, managers' characteristics and individual factors, factors related to research productions and external or environmental factors and social/interpersonal factors. The barriers to managers' use of EBMgt were categorized into the following areas: barriers about managers' characteristics, decision-making environment, training and research system and organizational barriers. On the other hand, the sources of evidence were divided into six categories that included: scientific and research evidence (SRE), facts and information of hospital, political-social development plans, managers' professional expertise, ethical-moral evidence (EME) and values and expectations of all stakeholders.

Round 2

According to our classification above, we designed a specific questionnaire which consisted of four main parts. The experts were asked to rate on a 9-point Likert scale (1 = not at all important, 9 = essential) the importance of each factor. For each category of factors, an open-ended question was added to capture factors that were missing in the list. Questionnaires were collected through the internet by the Google Forms (n=11) and others were collected through paper forms (n=10).

Data analysis: In round 1, the synthesis involved interpretative analysis following the principles of meta-synthesis. Included studies were read, reread and details of the studies recorded. In addition, all the texts were read and then challenges and components were coded as themes. In the next step, homogeneous themes were composed and then categories were created. All themes and categories emerged from the obtained data.

In round 2, median scores were calculated as indicators of the importance of factors for each category of the sources of evidence, EBMgt process, barriers and facilitators. Factors with a median score of 7 or higher were considered very important. Factors with a median score of 4 to 6 were considered important that were entered in the next round. In addition, scores of 3 or lower were considered not important enough to be included in the framework.

Results

Among the 94 experts invited to join one of the two Delphi study groups, 66 participants responded to all questions. The participants were mostly men (70%). They were selected from 18 organizations (research institutes, training organizations and universities, hospitals and MoHME) in 9 Iranian provinces. In round 1, 56 experts were invited by the ICEHM; forty-five 45 persons consented to be participated (response rate of 80%). In round 2, the number of invited persons was 38 of whom 21 agreed to be participated (response rate of 55.26%).

We recruited a sufficient number of participants from the senior and executive managers in healthcare sector, academy members, professors of health policy, management, economics and other management sciences, experts in systematic review, and health information professionals.

Overall, 126 factors were selected using a synthesis of previous studies and interviews in round 1. In other words, facilitators, barriers, the sources of evidence and EBMgt process consisted of 48, 46, 22 and 10 factors, respectively. We described the categories of EBMgt process and other categories in more details in Table 2. In round 2, we used the median score. Based on median scores, many factors were found to be very important. Among the 48 factors identified related to facilitators from first-round 1, median scores of 6 factors were less than 3 and were excluded. Also, four of the factors related to EBMgt process and two of the factors related to sources of evidence were excluded.

Table 2

The confirmed and excluded factors by experts in round 1 and 2.

Main themes	Factors	Identified factors	Number of factors		Median scores
			Included	Excluded	Included	Excluded
Facilitators	Organizational factors	10	7	3	≥7	≤3
	Manager's characteristics and individual factors	12	11	1	≥8	≤3
	Factors related to research productions	10	9	1	≥8	≤3
	External or environmental factors	8	7	1	≥8	≤3
	Social/interpersonal factors	8	8	0	≥7	-
Barriers	Decision-makers characteristics	12	12	0	≥7	-
	Decision-making environment	10	10	0	≥7	-
	Training and research system	10	10	0	≥7	-
	Organizational barriers	10	10	0	≥7	-
	Team barriers	4	4	0	≥7	-
The sources ofevidence	Scientific and research evidence (SRE)	4	4	0	≥8	-
	Facts and information of hospital	6	4	2	≥8	≤3
	Political-social development plans	3	3	0	≥8	-
	Managers' professional expertise	3	3	0	≥8	-
	Ethical-moral evidence (EME)	3	3	0	≥8	-
	Values and expectations of all stakeholders	3	3	0	≥8	-
Process of	Asking	2		1	≥8	≤3
evidence-based	Acquiring	2		1	≥8	≤3
management	Appraising	2		1	≥8	≤3
	Aggregating	1		0	≥8	-
	Applying	2		1	≥8	≤3
	Assessing	1		0	≥8	-
Total		126	114	12	-	-

On the other hand, all of the 46 factors related to barriers were confirmed by experts (median score higher than 7). Among the factors identified related to sources of evidence and EBMgt process, median scores of 6 factors were less than 3. These factors were deleted, as the majority of the participants believed that these factors could be combined in other factors. Finally, 46 barriers, 42 facilitators, 20 sources of evidence and 6 factors related to EBMgt process were selected and confirmed that in total included 114 factors.

Discussion

Using the Delphi technique, we confirmed the factors of EBMgt in the Iranian context with the known factors found in the scientific literature. The results showed that many of the factors found in the systematic review are also applicable to the Iranian context. In this study, the barriers to EBMgt, sources of evidence, facilitators, and EBMgt process were identified in the hospitals.

Evidence-based medicine movement has influenced prominent researchers and health managers in health services management. The EBMgt was not broadly used by healthcare administrators in the U.S up to 2006 (34–36). Nowadays, Guo's study showed that healthcare managers have adopted an evidence-based management in their decision-makings (25). It is important for healthcare managers to adopt an EBMgt approach and use in decision-making process. Healthcare managers' decisions have a significant impact on the effectiveness of delivering quality care and on the success of healthcare organizations (22,25). In our study, 4 categories were obtained that it is necessary to use in order to develop an evidence-based framework. As shown in Figure 1, we identified the sources of evidence and management areas in EBMgt. Indeed, the evidence pyramid of EBMed has been converted to an evidence hexagon in EBMgt. The hexagonal points present the sources of evidence.

Figure 1

Evidence-based framework for evidence-based management in healthcare organizations

Evidence is an important part of practicing evidence-based management. Liang et al. listed seven sources of evidence for participants to rate their importance and frequency of use in relation to the decision-making process. These included internally developed data, best practice, stakeholders'/clients' expectations, instances of external practice, expert opinions, quantitative research, and qualitative research (16,22). Moreover, Barends et al. listed four main evidence types which include following items: the best available scientific research, organizational data, professional experience and judgment, and stakeholders' values and concerns (29).

In our study, according to type of evidence sources (six main sources with 20 sub-sources), healthcare managers were classified into 6 categories. The web space is divided into six areas that each area represents a particular style of management. For example, managers that use scientific-research evidence are scientific-oriented managers. Other areas of the web belong to data and fact-based, plan-based, expertise-oriented, ethics-oriented and shareholder-oriented managers.

Liang et al. and Barends et al. in their studies showed that EBMgt process is based on six steps (22, 29). In our study, depending on the type of problem, managers will select the best available evidence and its sources using EBMgt process. The EBMgt process includes the following steps: 1) asking, 2) acquiring, 3) appraising, 4) aggregating, 5) applying and 6) assessing. We named them 6A that is an abbreviation of the words.

Based on the problem or the population, managers should use the best sources of evidence. As shown in Figure 1, a full evidence-based manager is a person who utilizes all the sources of evidence in a six-step process for decision-making. This process is evidence-based decision making (EBDM). The EBDM is axis of evidence-based management, and this is decision-making process that solves the problem.

In EBDM, it is essential to understand the context in order to know better the interaction among these factors. In that context, factors such as facilitators, barriers and training organizations and research institutes must be fully considered. The framework of evidence-based management can be considered as a good tool to better manage all the healthcare organizations. In this study, we tried to create an evidence-based framework for management of healthcare organizations. In addition, this framework is a practical model. It means that we can determine the management zones for each manager based on the sources of evidence confirmed in the Delphi.

In conclusion, based on the results of this study, more efforts in developing a culture that embraces evidence-based management among healthcare managers are needed. Healthcare managers need to more often consult scientific-research findings for decision-making, including both qualitative and quantitative study in healthcare management, in addition to professional experience, political-social development plans, fact and information of hospital, and stakeholders' values and expectations.

Professional associations (academy of management, international scientific hospital associations, etc.) universities, private sectors and research agencies, practitioners and educators together need to conduct more EBMgt research, collect and organize EBMgt information resources and provide the synthesized information for practicing healthcare managers. This effort may help healthcare managers to more easily consult the best available evidence for their decision-making. Finally, an effort should be given to translate evidence into management practice so that healthcare managers can have a meaningful use of research findings in their healthcare organizations. It help guide healthcare leaders and managers in making better decisions.

Our study suggested that most participants had positive attitudes towards EBMgt and that a large majority of them believed that using evidence-based management can improve the quality of management decisions. Evidence-based management is important to improve the quality of management decisions, and hence, to improve service delivery system plus effectiveness and efficiency. Since EBMgt is an emerging approach, its practice among healthcare managers has remained limited. Several factors exist at the healthcare level, among which, the organizational level and the personal level ones play different and considerable roles in affecting causing this lack of skill. We know that many healthcare managers lack of skills about adopting EBMgt. Thus, they must be trained by training organizations and research institutes.

This study has important implications for healthcare administrators. The findings of this study could help policy-makers and professional associations better understand healthcare managers' decision-making style. The outcomes of the study can be used for exploring some possible interventions that might increase the use of an evidence-based approach in healthcare management and reduce barriers to the adoption of EBMgt in the future; for example, to reduce the gap between research and practice in healthcare management. Moreover, few studies on the framework of EBMgt in healthcare management have been conducted. The EBMgt framework helps healthcare managers to view the pursuit of multiple evidence sources on a continuum based on levels of knowledge utilization. By using the hexagon of evidence sources, managers can recognize the best available evidence for management decisions, and in an EBDM process, make the best decisions.

This study has two limitations. The first limitation is that we only surveyed one criterion based on the importance of each factor. The second limitation is that the response rate in round 2 was low than in round 1. Higher Response rate could increase consistency of the study.