The Diffusion of Discrete Event Simulation Approaches in Health Care Management in the Past Four Decades: A Comprehensive Review.

This study systematically examines the diffusion of the discrete event simulation (DES) approach in health services and health care management by examining relevant factors such as research areas, channels with the objective of promoting the application of DES in the health field. We examined 483 journal papers referencing this approach that were published in 230 journals during 1981 to 2014. The application of DES has extended from health service operational research evaluation to the assessment of interventions in diverse health arenas. The increase in the number of adopters (paper authors) of DES and the increase in number of related channels (journals publishing DES-related articles) are highly correlated, which suggests an increase of DES-related publications in health research. The same conclusion is reached, that is, an increased diffusion of DES in health research, when we focus on the temporal trends of the channels and adopters. The applications of DES in health research cover 22 major areas based on our categorization. The expansion in the health areas also suggests to a certain extent the rapid diffusion of DES in health research.

Discrete event simulation (DES) models the operation of a system as a discrete sequence of events in time. Each event occurs at a specific instant in time and marks a change of state in the system.[1,2] Some researchers have characterized it as “a form of computer-based modelling that provides an intuitive and flexible approach to representing complex systems.”[3] Having some subtle difference with DES, agent-based approach and multi-agent system is aimed to minimize patient waiting time and health care cost in the emergency department. In the process of implementation, their differences were somewhat downplayed.[4,5] Since its appearance in 1950, DES as a typical operational research (OR) technique has been widely used in many areas including production process diagnosis, program evaluations, evaluation of investment decisions, network simulator in computer science, hospital operations, and health care capacity planning at national and regional levels, among others.[6]

The original idea of using DES in health service operation management, health service resource planning, and medical decision-making processes, and so on, can be dated back to the early 1980s.[7] From 1981 onwards, a rapid diffusion of DES simulation was witnessed in health research at different levels, which include, but not limited to, health care capacity planning at the national level; process reengineering at local clinics, pharmacies, and hospitals; patient flow and the reduction of waiting time; staffing and scheduling at emergency departments and outpatient clinics; the logistics of medical supplies (e.g., blood, vaccines);[8] the evaluation of disease screening;[9] the evaluation of disease prevention and treatment;[10] and the impact of health on the economy.[11] Recently, Karnon and colleagues[3] reported that DES has been used in a variety of health care applications. Most early applications involved analyses of systems with constrained resources, where the general objective was to improve the delivery service in hospitals, clinics, and pharmacies. But more recently, DES was employed to assess specific technologies in the context of health technology assessment.[3,12]

By compressing time and space, DES enables researchers and policy designers in the health sector to gain a better understanding on the dynamic interaction of system components and of the system as a whole.[13] Like other simulation tools, DES has advantages over other techniques due to its flexibility, ability to handle variability and uncertainty, and its dialog interfaces that facilitate communication among researchers, health professionals, and policy makers.[14] DES has the capability of modeling complex patient flow through health care clinics. Most important, it facilitates scenario planning by answering “what-if” questions through changing patient flow patterns and service policies.[15]

Given the rising interests in and importance of using DES in health research, it is necessary to have a clear picture for the status of this research approach. Specifically, it is necessary to have a better understanding of the factors that possibly affect the diffusion of this OR simulation technique in health research. It is also imperative to identify what areas have been investigated, which areas are the main focus of this type of study, which new areas have been explored recently, and whether an identifiable shift of focus in research areas exists.

Being distinct from a conventional literature review, this research provides a systematic and comprehensive overview and analysis of the DES diffusion in health research with a purpose of identifying the diffusion pattern and factors driving that pattern. This study provides several sources of information on related research. The present study has four key objectives: 1) to understand the status quo and general diffusion process of this OR technique in health research; 2) to explore the dynamics of adopters* (first authors of papers and all other participating authors) and channels (journals) over time and how they have affected the diffusion process of DES in health studies; 3) to examine the health research areas investigated by DES and identify whether the changes in research areas have influenced the DES diffusion in health research; we then study the research trends in different areas over recent years; and 4) to project the future application of this OR technique in health research. Although research has been conducted to review the application of DES in certain health areas, a systematic and comprehensive research like ours has not been conducted previously to examine the diffusion process of DES in health study from the above-mentioned three perspectives, that is, adopters, channels, and research areas. Thus, the present study helps provide important insights that offer useful implications for future research.

In the next section, we provide the research motivation and objectives for this study along with a literature review regarding the application of DES in different areas of health research. The subsequent section presents the diffusion status of DES publications over the years. We also analyze the changes of adopters and channels and how they have affected the diffusion process of DES in health research. The next section offers the categorization of the research areas and the publications in each area. Finally, we conclude this study with major findings and research limitations and provide a discussion of the challenges of applying DES in health research. This section also presents ideas for future research directions.

Literature Review

Actually, there are two group of reviews in this section. One group of literature review has been conducted regarding the application of DES in particular areas in the field of health. Another group of reviews has compared DES with other simulation tools such as agent-based models (ABM), Markov models (MM), decision trees (DT), Monte Carlo simulation (MC), and system dynamics (SD) in health research. In what follows, we present each group in chronological order in Table 1.

Table 1

Two Major Groups of Literature Review Regarding DES’s Application in Health Studies Done During 1981 to 2014.

Application of DES in Particular Health-Related Areas	Comparison Between DES and Other Simulation Tools in Health Research
DES to resources allocation and planning in the health sector[16]	DES and SD are preferable to capture interactions between individuals in economic evaluation of health interventions[32]
Situations where DES can be applied in health care clinics[17]	SD and DES to address strategic health care delivery issues[33]
Breadth, quality, and value of DES for health service delivery[18]	DT, MM, and DES for cost-effectiveness analysis of interventions using coronary heart disease as an example[34]
Employing DES to identify real problems in critical care[19]	Providing guidelines for using DES, ABM, SD, and MM in pharmacoeconomic and health technology assessment[35]
Using DES for pharmacoeconomic analysis and to inform policy makers of the economic implications of pharmaceutical intervention[20]	Suggest to use SD, SNA, ABM, DES, MM, and Soft System to integrate the behavioral and social sciences into the NIH scientific enterprises for addressing health disparities among populations[36]
DES’s application in economic evaluation of the health care system[21]	Use of DES and ABM in different levels of implementations[37]
Potential of using DES in health care settings[22]	Use SD, ABM, DES, and social networking analysis to develop public health policies and inform their implementation[38]
Using DES to evaluate economic and clinical outcomes of imaging technology in health services[23]	To facilitate the understanding of how SD, ABM, and DES can address various health care issues[39]
Using DES to address typical administrative issues faced by health care managers[24]	Application of SD, ABM, and DES in modeling patient flow issues in surgical care[40]
Whether DES in health care has the necessary sense of urgency to deal with a much-needed reality check[25]	Regression models, time-series analyses, queuing theory, and DES in studying emergency department patient flow/crowding[41]
DES in performance modeling in health care[26]	MC, MM, and DES to assess the cost effectiveness of drug treatment strategies for HIV infection[42]
DES in assessing the cost-effectiveness of health technology for heart failure[27]	SD, DES, and ABM in childhood obesity research[43]
Operational issues of using DES in different health care contexts[28]	Cohort MM and DES in evaluating clinical outcomes and utility for the treatment of chronic lymphocytic leukemia[44]
DES to address patient flow issues in health care clinics and integrated health care systems[29]	Application of DES and MM in economic evaluation of pharmacotherapeutics in patients with bipolar disorder[45]
DES to model full guideline pathways for cost-effectiveness analysis of diagnostic and treatment pathways[30]	Comparison of the advantages and disadvantages of using MM and DES in conducting a cost-effectiveness analysis[46]
DES models for conducting cost and benefit analysis of health technologies in health care[31]	Relationship among DES, MM, and DT in evaluating the cost-effectiveness of the treatment of chronic schizophrenia[47]

ABM, agent-based models; DES, discrete event simulation; DT, decision trees; MC, Monte Carlo simulation; MM, Markov models; SD, system dynamics; SNA, social network analysis.

Methods

We conducted a bibliometric analysis on all retrieved publications. Specifically, our statistical analysis includes plotting trends over time, tabulating the ordered categories, calculating correlation coefficients, and conducting qualitative analysis.

Table 2 offers the item selection criteria used in searching and screening literature. Figure 1 presents how the data were collected and analyzed. The adopter indicates the author that participated in the writing of the retrieved journal paper. Annual new adopters indicate the number of authors that started to publish DES-based papers in health research in a certain year. The channel refers to the journal that published DES-based papers in health study. Annual new channels refer to the number of journals that started to publish DES-based papers in health research in a certain year. The research areas are categorized based on the MeSH headings of PubMed. Some areas having unique characteristics are labelled as different areas.

Table 2

List of Inclusion and Exclusion Criteria and Search Terms[a]

Exclusion Criteria	Inclusion Criteria
1. Non-journal papers2. Non-English papers3. Editorials, commentaries, letters to the editor, academic dissertation, conference proceedings4. Animal health related processes	1. English language2. Journal articles3. Paper published from 1980 through 20144. Human-related health research
Search Items
1. Healthcare2. Hospital management3. Emergency management4. Public health5. Health services6. Health technology7. Medical systems8. Health policy9. Medial economics10. Treatment	a. Simulationb. Operations managementc. Operational researchd. Discrete event simulatione. Queueingf. Waiting timeg. Process improvement

For search items, they have “OR” relationship in a column, the items have “AND” relationship in different columns. For example, “hospital management” OR “public health” OR “healthcare.” The items in different columns have AND relationship such as “emergency management” AND “discrete event simulation.”

Figure 1

Flowchart of the process of literature review, data acquisition and analysis.

By searching PubMed, we identified 587 relevant articles. Criteria in Table 2 were used to filter out nonconforming ones. Thereafter, Wiley database, ScienceDirect database, Cochrane Library, Scopus, and Google Scholar were used to glean articles that were not indexed by PubMed. Using the same filtering criteria, together, we retrieved 483 journal papers (1981–2014) published in English. Then we examined the 483 studies and obtained the number of annual publications, cumulative publications, all the adopters, all the channels, citations for each paper, and the research areas of each paper. We then found the distribution of annual new adopters over time, the distribution of annual new channels over time, the impact factor of involved journals, the categorization of research areas, and the distribution of publications in each area over time.

We also calculated the Pearson correlation coefficients between annual publications and annual new adopters as well as the correlation coefficient between annual publications and annual new channels to demonstrate that the increase in the adopters and channels drives the increase of annual publications. Then we demonstrated that adopters, channels, and research areas individually and jointly contribute to the rapid diffusion of DES in health research. Finally, we projected the application of this OR techniques for future health research.

Analysis of the Diffusion of DES in Health Research

The Diffusion of DES Publications

As shown in Figure 2, from 1981 through 2014, 483 journal papers have been published with an average of 14.2 papers per year. In addition to using the yearly number, we also used a 3-year period to smooth out the curve. Among all of the retrieved papers, 40 of them are review papers accounting for 8.3% of the total, in which two types of reviews are presented. One of them is a review of the DES’s application in different perspectives of health research. The other is a comparison between DES and other simulation methodologies such as MM, SD, regression, and DM when evaluating the cost-effectiveness and clinical outcomes of health care interventions.

Figure 2

Trend in health-related research publications using discrete event simulation (DES) during 1981 to 2014.

Staring from 1994, the number of publications has been increasing steadily, even with some variation. This is partially ascribed to the rapid popularization of the PC and the enhanced capability of simulation software in parallel with significant improvement in computing power.[6] Compared with previous periods, 2007 started to witness an accelerated increase in publications with the largest number, 57, appearing in 2012.

Adopters

The fast growth of DES publications in health research is ascribed to the rapid increase in the adopters and diffusion channels for this kind of research approach. Table 3 shows the new adopters and channels each year for the covered publication period. The trend in the increase of adopters and diffusion channels correspondingly matches the increase in publications shown in Figure 1. In the past 34 publication years, 360 first authors adopted the DES approach in health-related research, with 10.6 new first authors per year. Within our data, the earliest adopters authoring DES research in health fields are Kutzdrall et al.[48]; Look, Schriber, Nawrocki and Murphy[49]; and Miller[50], all of whom published their studies in 1981. They studied space requirements and scheduling polices in surgical suites, immune responses to malignant lymphoid cells, and the evolution of the lifespan, respectively. As indicated in Table 3, in the early stage of DES diffusion (1981–1995), very few new first authors appeared each year using the method in health-related research, with about 1.3 average new first authors being added each year. Starting from 1996, however, more authors began to adopt this OR technique and the adopters increased moderately, with some small variation. The number of new first authors apparently increased at an accelerated pace, starting from the beginning of the 2000s. The past 10 years has witnessed an average increase of 16.2 new first authors per year.

Table 3

Annual New Adopters and Channels for Publishing Discrete Event Simulation (DES)-Based Health-Related Research during 1981 to 2014[a]

Year	Annual New Channels	Percentage	Cumulative Percentage	Annual New Adopters	Percentage	Cumulative Percentage
1981	3	1.30%	1.30%	10	0.66%	0.66%
1982	0	0.00%	1.30%	0	0.00%	0.66%
1983	0	0.00%	1.30%	0	0.00%	0.66%
1984	0	0.00%	1.30%	0	0.00%	0.66%
1985	1	0.43%	1.74%	1	0.07%	0.72%
1986	3	1.30%	3.04%	3	0.20%	0.92%
1987	1	0.43%	3.48%	5	0.33%	1.25%
1988	1	0.43%	3.91%	2	0.13%	1.38%
1989	1	0.43%	4.35%	3	0.20%	1.57%
1990	1	0.43%	4.78%	2	0.13%	1.71%
1991	0	0.00%	4.78%	0	0.00%	1.71%
1992	2	0.87%	5.65%	4	0.26%	1.97%
1993	2	0.87%	6.52%	5	0.33%	2.30%
1994	3	1.30%	7.83%	3	0.20%	2.49%
1995	2	0.87%	8.70%	2	0.13%	2.62%
1996	7	3.04%	11.74%	18	1.18%	3.81%
1997	5	2.17%	13.91%	12	0.79%	4.59%
1998	3	1.30%	15.22%	18	1.18%	5.77%
1999	3	1.30%	16.52%	12	0.79%	6.56%
2000	3	1.30%	17.83%	25	1.64%	8.20%
2001	4	1.74%	19.57%	13	0.85%	9.06%
2002	4	1.74%	21.30%	24	1.57%	10.63%
2003	9	3.91%	25.22%	40	2.62%	13.25%
2004	10	4.35%	29.57%	57	3.74%	16.99%
2005	8	3.48%	33.04%	65	4.27%	21.26%
2006	11	4.78%	37.83%	41	2.69%	23.95%
2007	16	6.96%	44.78%	113	7.41%	31.36%
2008	21	9.13%	53.91%	98	6.43%	37.80%
2009	15	6.52%	60.43%	89	5.84%	43.64%
2010	10	4.35%	64.78%	132	8.66%	52.30%
2011	13	5.65%	70.43%	114	7.48%	59.78%
2012	24	10.43%	80.87%	223	14.63%	74.41%
2013	23	10.00%	90.87%	172	11.29%	85.70%
2014	21	9.13%	100.00%	218	14.30%	100.00%
Total	230			1524

Annul new channels indicate the annual number of journals that started to publish DES-based papers in the health-related field, which did not publish previously. Annual new adopters indicate the annual number of authors that started to publish DES-based papers in the health-related field.

Since the number and increase rate of total participants are also important indicators in reflecting the popularity and the DES diffusion pace, with further analysis we found a total of 1524 different authors with an average of 44.8 new adopters per year. The increasing pattern in this respect is similar to the increase pattern of first authors, that is, from a stagnant status from 1981 through 1995, to an obvious increase starting in 1996 and a significant increase beginning from 2003. The average number of new adopters per year in the past 10 years is 126.5. The correlation coefficient (CC) between the number of new adopters per year and the number of new publications per year is 0.996 (nearly 1), indicating that the variables are almost perfectly correlated. This means that the increase in number of new adopters is followed in tandem by the increase in number of new publications. We also found that 36 adopters authored more than 4 journal papers. By looking at the publication periods, many early adopters (starting from the beginning of the 2000s) are still very actively using DES in health research. In addition, 14 of 36 authors publishing more than 4 papers are new adopters in the past 5 years. This indicates that the insularity of this community to new adopters or junior researchers is low, which suggests the outreach potential of DES and consequently the network of researchers is increasing and being applied to explore more health issues.

The above-mentioned 36 authors published 266 papers accounting for 55% of all retrieved papers, while the other 1488 authors participated in 45% of our retrieved papers. The reason for this result might be the existence of coherent subgroups, and the existence of strong insularity or resistance of established research groups to the admission of new adopters outside their circles.

Channels

Regarding the diffusion channels, during the 34 years covered by this study, 230 journals became diffusion channels of DES in health-related research, that is, around 6.8 new journals per year. Corresponding to the earliest adopters, the earliest channels for accepting DES publications were Simulation, Immunology, Mechanisms of Ageing and Development, with two of these not being OR-related journals. Starting from 1985, mainstream OR journals like the Journal of the Operational Research Society, European Journal of Operational Research, and Omega began to publish health studies using DES. In Table 3, in the years 1981 through 1995 the maximum annual new channels accepting DES-based health research was 3, and the average new channels increase was about 1.2 journals each year. From 1996 through the early 2000s, more journals started accepting papers that used DES in health-related research although the increase in the new channels was mild and had some fluctuation occasionally. The very rapid increase in new channels began in 2003, with the largest number of 24 occurring in 2012. The number of new channels from 2003 to 2014 accounted for 78.7% of all 230 journals.

The CC between the number of channels and the number of publications is 0.996 (nearly 1). Consequently, this suggests that the increase in the number of channels was one of dominant forces for driving the increase in the number of publications. We also found that among all the 230 journals, only about 11.7% (27 out 230, with 73 papers accounting for 15.1% of 483 papers) are pure OR-related journals, and the majority are mainstream health journals such as Pharmacoeconomics, BMJ, Vaccine, and Gastroenterology. This indicates that DES was well recognized by researchers in the health field as a method to better understand and manage health service operations.

The rapid increase in new adopters provided increased chances to have access to new channels where they did not have interactions before. Since new adopters came from a variety of areas, researchers knew the specific issues demanding OR techniques. Therefore, they tended to introduce DES to those particular areas where they had expertise. By plotting the lines of annual increase of new adopters and new channels, we get a CC of r = 0.930, which suggests that the two time series have a strong linear relationship. The result suggests that our premise is well founded. However, the opposite explanation might also sound plausible, that is, that the increase of new channels might have brought to the limelight issues of interest, which consequently encouraged more adopters to examine health issues. Hence, we think that the increase in both new adopters and new channels exerted a mutually reinforcing pressure.

Table 4 lists 29 journals that published >2 papers, in which a total of 248 papers were published, accounting for 51.35% of all retrieved publications. Among them, the four most popular ones were Health Care Management Science, Pharmacoeconomics, Journal of the Operational Research Society, and Medical Decision Making, wherein each of them published more than 20 papers. Table 4 also lists the publication period for DES papers. The earliest one could be dated back to 1981. Ten journals spanned more than 10 years in publishing health-related DES papers, and 17 have been active in the years of 2013 and 2014, which indicates the popularity of and increasing need for using DES in health research.

Table 4

The Ranking of Journals in Terms of the Number of Health Research Papers Using Discrete Event Simulation (DES) Published during 1981 to 2014[a]

Rank	Journal Name	# of Papers[b]	%	Cumulative %	Publication Period	Impact Factors, 2013–2014[c]
1	Health Care Management Science	34	7.02%	7.02%	1998–2014	0.871
2	Pharmacoeconomics	27	5.58%	12.60%	2002–2014	3.338
3	Journal of Operational Research Society	21	4.34%	16.94%	1985–2011	0.91
4	Medical Decision Making	20	4.13%	21.07%	1993–2014	2.698
5	Value Health	15	3.10%	24.17%	2007–2014	2.891
6	European Journal of Operational Research	11	2.27%	26.45%	1987–2012	1.843
7	PLoS One	11	2.27%	28.72%	2010–2012	3.534
8	Simulation	10	2.07%	30.79%	1981–2012	0.656
9	Health Technology Assessment	9	1.86%	32.64%	2003–2014	5.116
10	Journal of Medical Systems	9	1.86%	34.50%	2002–2012	1.372
11	International Journal Health Care Quality Assurance	8	1.65%	36.16%	2007–2014	0
12	Journal Medical Economics	7	1.45%	37.60%	2010–2014	0
13	Annals of Emergency Medicine	6	1.24%	38.84%	1989–2014	4.333
14	Current Medical Research Opinion	6	1.24%	40.08%	2006–2010	2.372
15	Anesthesia & Analgesia	5	1.03%	41.12%	2001–2009	3.422
16	Studies in Health Technology Informatics	5	1.03%	42.15%	2004–2014	0
17	Academic Emergency Medicine	4	0.83%	42.98%	2004–2013	2.198
18	Applied Health Economics Health Policy	4	0.83%	43.80%	2013–2014	0
19	Computer Methods and Programs Biomedicine	4	0.83%	44.63%	2001–2011	1.093
20	International Journal Technology Assessment in Health Care	4	0.83%	45.45%	2007–2014	1.556
21	Simulation Modelling Practice and Theory	4	0.83%	46.28%	2011–2014	1.05
22	Biotechnology Progress	3	0.62%	46.90%	2006–2013	1.883
23	BMC Health Services Research	3	0.62%	47.52%	2008–2014	1.659
24	BMC Public Health	3	0.62%	48.14%	2007–2014	2.32
25	BMJ	3	0.62%	48.76%	1992–2011	16.3
26	Gastroenterology	3	0.62%	49.38%	2000–2003	13.926
27	Journal Health Services Research & Policy	3	0.62%	50.00%	2001–2004	2.087
28	Journal of Theoretical Biology	3	0.62%	50.62%	2005–2011	2.303
29	Vaccine	3	0.62%	51.24%	2010–2013	3.485

The table also uses 2013–2014 impact factors for these journals.

The journals that published <3 papers are not listed, but are available from the first author if requested.

Journals with a 2013–2014 impact factor >3 are highlighted.

Journal Impact Factors

To identify the impact of journals that published health-related papers using DES, we examined the impact factors (IF)* of all retrieved journals and found that the weighted average IF[†] (WAIF) of all journals is 2.40. The top 10 journals that published the largest number of DES papers on health had a WAIF of 2.16. The 29 journals in Table 4 have a WAIF of 2.31. This is an interesting finding indicating that the journals that published the largest number of DES papers in health do not have high WAIFs, which means that health-related DES papers have tended to be accepted by many low-IF journals (with <3 papers each) instead of concentrating on a very limited number of high-IF journals. For instance, the papers we retrieved were published in journals such as the Journal of Allergy and Clinical Immunology (IF = 12.24, 2 papers), Alzheimer’s and Dementia (IF = 17.47, 1 paper), Annals of Internal Medicine (IF = 16.10, 1 paper), Stroke (IF = 6.01, 1 paper), Journal of the National Cancer Institute (IF = 15.16, 1 paper), and Journal of Clinical Oncology (IF = 17.87, 1 paper). Since these journals are targeting very specialized areas of health and disease, this may indicate that DES is penetrating these areas or these areas have a need for this OR technique. However, there may be barriers preventing the diffusion of DES to potential adopters in these very specialized areas, but the specifics behind this are beyond the scope of this paper.

Analysis on the Research Areas and Research Trends

Based on the MeSH headings in PubMed and the areas having obvious distinction from others, DES publications in health research were categorized into 22 subareas (see Tables 5 and 6; readers may have suggestion of other type of categorizations). The numbers 1 through 22 in column 1 of both Tables 5 and 6 are used to label the 22 categorized areas.

Table 5

Distribution of Health Research Areas Using Discrete Event Simulation (DES) during 1981 to 2014[a]

No.	Research Areas	Number of Papers	%	Cumulative %	First Year to Publish
1	Improving health service at hospital/clinics/pharmacy	167	34.58%	34.58%	1981
2	Evaluating different treatment strategies and/or medicines for diseases	102	21.12%	55.69%	2000
3	Modeling progression/evolution of disease and treatment outcome	30	6.21%	61.90%	1997
4	Planning health service capacity at community or state	25	5.18%	67.08%	1985
5	Compare DES with other modeling tools	23	4.76%	71.84%	2003
6	Evaluating screening technology and policy for disease	22	4.55%	76.40%	1996
7	Review of DES in different areas of health research	17	3.52%	79.92%	1995
8	Modeling logistics of health service products	13	2.69%	82.61%	2007
9	Microlevel simulation, body physiology, drug dynamics	13	2.69%	85.30%	1981
10	Planning health service capacity for emergency event	12	2.48%	87.78%	1999
11	Modeling spread of contagious disease	12	2.48%	90.27%	1990
12	Evaluating the impact of health technology on health service	12	2.48%	92.75%	1988
13	Evaluating prevention strategies for contagious diseases (contagious disease and infection at hospital	7	1.45%	94.20%	2005
14	Improvement of clinical trial	5	1.04%	95.24%	2008
15	Understanding health care systems beyond hospital/clinics/pharmacy	5	1.04%	96.27%	2001
16	Evaluating the interventions to substance abuse	4	0.83%	97.10%	1996
17	Modeling medicine development and production process	4	0.83%	97.93%	2004
18	Modeling disease epidemiology	3	0.62%	98.55%	2011
19	Evaluating prevention strategies for noncontagious diseases	2	0.41%	98.96%	2002
20	Improvement of medical education	2	0.41%	99.38%	2000
21	Improving diagnosis process and procedures (sampling policies)	2	0.41%	99.79%	2012
22	Negative externalities of health issue	1	0.21%	100.00%	2014
Total		483

Research areas are primarily categorized based on the MeSH headings on PubMed. Some areas having unique characteristics are also labeled as different areas. Table 6 provides more detailed explanation on the area categorization.

Table 6

Health Research Areas Using Discrete Event Simulation (DES) during 1981 to 2014[a]

No.	Research Areas	Specific Research Questions, Topics, and Content
1	Improving health service at hospital/clinics/pharmacy/emergency department (ED)	Issues related to patient flow, appointment scheduling, bed (other capacity) planning, changes in policies, operating room utilization, resource allocating, access/admission management, performance management, waiting time, boarding in ED, training improvement, ambulance diversion, ambulance response time
2	Evaluating different treatment strategies and/or medicines for diseases	Evaluating cost-effectiveness and clinical outcomes of different medicines and/or treatment strategies & protocols & courses, benefits (quality of life) and risk evaluation of treatment and surgery operation
3	Modeling progression/evolution of disease and treatment outcome	Modeling progression of disease (including external factor on evolution) and intervention costs (including cancer and malignant tumor, major depression, asthma, Alzheimer, diabetes, heart disease, coronary events)
4	Planning health service capacity beyond hospital/clinics	Planning and budgeting for health service capacity beyond hospital level, i.e., health care network, pharmacy system, community, state, national levels
5	Evaluating screening technology and policies for disease	Economic and/or risk evaluation of screening policies, diagnosis, risk assessment schemes, technologies
6	Compare DES with other modeling tools	Compare DES with SD, DT, MM in the application of modeling patient flow, evaluating cost-effectiveness and clinical outcome of different medicines and treatments, allocating health care resources
7	Review of DES in different areas of health research	Review of DES in resource allocation/planning, economic evaluation of health care system, cost and benefit analysis of health technologies, pharmacoeconomic analysis, performance modeling
8	Modeling logistics of health service products	Modeling supply chain management and logistics, i.e., blood, vaccine, medical oxygen
9	Microlevel simulation, body physiology, drug dynamics	Human body microlevel simulation, i.e., metabolic network, gene expression, nucleation process, physiology process, drug dynamics, in vivo tumor response to therapy
10	Planning health service capacity for emergency events	Incidents of urban terrorism, mass casualty, bioattack, outbreak of epidemic diseases such as smallpox
11	Modeling spread of contagious disease	Spread and transmission of contagious diseases, e.g., AIDS pandemic, malaria, Lyme disease, TB, H5N1
12	Evaluating impacts of health technology on health service	Impact of health technology on service performance, cost-effectiveness, efficiency of health service delivery
13	Evaluating prevention strategies for contagious diseases (contagious disease and infection at hospital)	Prevention strategies for mother-to-child transmission of HIV, pandemic influenza, herpes zoster, infection control such as methicillin-resistant Staphylococcus aureus
14	Improvement of clinical trial	Improving design and study of clinical trials such as phase I pediatric oncology, agile study design
15	Understanding health care systems beyond single hospital/clinics/pharmacy	Hospital-at-home initiatives, physician network setting, communication process of stakeholder in health care systems, change program at national level, performance of family health units versus primary health care centers
16	Evaluating the intervention to substance abuse	Evaluating health and/or economic impact of interventions, i.e., smoking cessation program and treatment
17	Modeling medicine development and production process	New medicine development, production process of antibody production, facility design for medicine
18	Prevention strategies for noncontagious diseases	Evaluating prevention strategies for coronary heart disease, occupational disease, i.e., low back pain
19	Modeling disease epidemiology	Estimating prevalence of disease and health care costs (acquired brain damage, hypertension, chronic diseases)
20	Improvement of medical education	Improving understanding on the workflow processes using simulation tool
21	Improving diagnosis process and procedures	Evaluating different sampling policies to find cost-effective alternatives to pathologists
22	Negative externalities of health issue	Evaluating impact of health issues on national labor supply

DT, decision trees; MM, Markov models; SD, system dynamics; TB, tuberculosis.

Research areas are primarily categorized based on the MeSH headings on PubMed. Some areas having unique characteristics are also labeled as different areas.

Given the strengths of DES in addressing issues of patient flow, scheduling, and queuing (Area 1), publications in these areas accounted for 34.58% of all retrieved papers, which was the highest percentage among all involved areas. For treatment strategies and/or medicines for diseases (Area 2), even though the publications started at the beginning of the 2000s, evaluating different treatment strategies and/or medicines for diseases using DES quickly became a hot topic, with publications amounting to 21.12% of all retrieved papers. Together the aforementioned two areas account for 55.69% of all publications. Regarding the research trend of using DES in health research, referring to Table 7, Area 1 had an average of 3.9 publications per year. From 1981 through 2006, the increase in publications in Area 1 was stagnant, with an average of less than 2 papers per year. From 2007 to 2014, the number jumped to an average of 14.9 papers per year, with the largest number of 21 in 2013 indicating an accelerating pace of DES diffusion in Area 1. With 5 years of slow penetration at the inception (1.2 papers/year), DES publications in Area 2 witnessed an obvious increase starting from 2005 and kept increasing till 2014 (9.6 papers/year). Please refer to Figure 2 for the trend and distribution of relevant areas.

Table 7

Publications Over Years in Applying Discrete Event Simulation (DES) to Different Areas of Health Research during 1981 to 2014a

Year ↓	Publication Number Over Years in Areas of Health-Related Study Using Discrete Event Simulation
Areas →	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22
1981	1								2
1982	0								0
1983	0								0
1984	1								0
1985	0			2					0
1986	0			2					1
1987	2			1					0
1988	0			0					0			1
1989	1			0					0			0
1990	0			0					0		1	0
1991	0			0					0		0	0
1992	1			1					0		0	0
1993	1			0					0		0	1
1994	3			1					0		0	0
1995	1			1			1		0		0	1
1996	5			0		2	0		0		1	0				1
1997	3		1	1		0	0		0		0	0				0
1998	2		0	1		0	0		0		1	2				0
1999	2		0	0		0	1		0	1	2	0				0
2000	1	2	0	0		1	0		0	0	1	0				0				1
2001	4	0	0	0		1	0		0	1	1	0			2	0				0
2002	2	1	2	1		2	0		0	1	0	0			1	0			1	0
2003	6	2	0	0	1	3	2		1	0	1	0			0	0			0	0
2004	5	1	0	1	1	1	1		2	1	0	1			0	0	1		0	0
2005	3	6	0	3	1	2	1		2	1	0	0	1		0	0	0		0	0
2006	4	6	3	0	1	1	2		0	0	0	1	0		0	0	1		0	0
2007	12	9	3	1	2	1	1	1	1	0	0	1	1		0	0	0		0	0
2008	13	8	0	3	3	1	0	1	1	0	2	0	0	3	0	0	0		0	0
2009	13	7	2	1	1	0	2	0	0	0	1	0	3	1	0	0	0		1	1
2010	13	11	3	0	2	1	1	0	1	0	0	0	1	1	0	0	0		0	0
2011	14	7	3	0	3	1	1	1	0	0	0	1	0	0	1	1	0	1	0	0
2012	15	16	5	0	1	2	0	4	1	5	1	3	1	0	0	1	1	0	0	0	1
2013	21	11	4	3	3	0	3	4	0	0	0	0	0	0	0	1	1	0	0	0	1
2014	18	15	4	2	4	3	1	2	1	2	0	0	0	0	1	0	0	2	0	0	0	1
Total	167	102	30	26	23	22	17	13	11	12	12	12	7	5	5	4	4	3	2	2	2	1

We categorized two types of review papers covering DES applications in health-related areas. One type compares DES with other simulation modeling tools (e.g., SD, DT, and MM) in modeling patient flow and allocation of health care resources, evaluating cost-effectiveness and clinical outcomes of different treatment strategies (Area 6). The other type conducts a general review of the application of DES to different health-related issues, for example, resource allocation, capacity planning, economic evaluation of health care systems, cost-benefit analysis of health care technologies, pharmacoeconomic analysis, and performance modeling (Area 7). Areas 6 and 7 so far have had an average of 1.2 and 0.9 papers per year, with publications starting from1996 and 1997, respectively.

Another five areas of study (namely, microlevel simulation, capacity planning for emergency events, modelling of contagious disease, evaluating impact of health technology, and evaluating intervention to substance abuse) that had publications before the 2000s showed very low proliferation over the time span of more than 15 years. Areas 13 through 22 only had less than 10 publications, making it hard to identify whether increasing patterns exist or not. Among these, “evaluating interventions for substance abuse” had the longest publishing history and the lowest proliferation rate. Publications for the remaining 9 areas appeared after year 2000. “Evaluating the impact of health issues on the national labor supply” is the latest area; it had one publication in 2014 that evaluated the negative externalities of health care issues. It was attempting to pioneer a broader perspective, which incorporated more factors of the socioeconomic system into the health-related DES model to gain a better understanding of the interactions between the health care and socioeconomic systems and their dynamic impacts on each other.

By using 10 years as an interval, we found that publications in four areas (namely, improving health, planning service capacity beyond hospital, microlevel stimulation, and evaluating impact of health technology) appeared from years 1980 through 1989; publications in six new areas (i.e., progression of disease, compare DES with other modelling tools, review of DES in different areas of health, planning health service capacity for emergency events, modelling infectious diseases, and evaluating interventions to substance abuse) came out from years 1990 through 1999; publications in nine new areas (namely, evaluating treatment of diseases, evaluating screening technology for diseases, modelling logistics of health service products, evaluating prevention strategies for infectious disease, improvement of clinical trials, gaining better understanding on health care systems beyond hospital, modelling development of medicine, modelling disease epidemiology, and improvement of medical education) appeared from years 2000 to 2009; and publications in another three new areas (namely, evaluating prevention strategies for noncommunicable disease, improving diagnosis process, and negative externalities of health issue) came into view from years 2010 to 2014. New areas after 2000, accounting for 50% of all categorized 22 health-related areas, showed that DES had been penetrating new areas in the health field at an expedited pace. More important, the number of publications in new areas (emerging after 2000) accounted for 35% of all retrieved 483 papers. This further substantiates that the increase of research areas did help get the attention of more potential adopters and channels, which rapidly advanced the diffusion of DES into health research and vice versa.

In terms of shift trend of research areas, improving health services at hospital and evaluating different treatment strategies of diseases are still the main areas employing DES; publications in these areas have showed an accelerated increase in the past 10 years. Although research areas including planning health service capacity beyond hospital, microlevel simulation, modelling infectious diseases, and evaluating impact of health technology all had more than 20 years of publication history, they all have had rather a low proliferation rate of less than 1 paper per year. They should not be considered as mainstream areas to which DES can be applied. Apart from areas in evaluating screening technology for diseases and review of DES in different areas of health, which were conducting the reviews of DES research, modelling disease progression and modelling logistics of health products were the only two areas showing a noticeable increase and continuous publication frequency in recent years, which signals the possibility that both of them might be able to enter the pool of mainstream research areas.

Discussion and Conclusion

This systematic and comprehensive analysis had examined how adopters, channels, expansion of research areas, and the combination thereof might help advance the diffusion of DES in health research. Major findings are found as follows. Publications addressing the application of DES in health-related research emerged in the early 1980s. The field experienced stagnant growth before 1994 and mild increase between 1994 and 2007. The diffusion of DES in health research was very slow before the early 1990s. We think there are four reasons that might explain this phenomenon. First and foremost, like any diffusion process, the early potential adopters were not familiar with the application of this OR simulation technique in health research.[51,52] Second, due to the unavailability of personal computers (PC), particularly commercial off-the-shelf simulation software/platforms, and computing constraints,[6] potential adopters had difficulty in programming for the health care issues in which they were interested. Additionally, the interdisciplinary prerequisites of applying DES in health research require scholars to possess skills and knowledge not only in OR techniques and software programming but also in health care system and associated critical issues, and health service operation management.[53,54] This undoubtedly created challenges for researchers who wanted to adopt DES to investigate pertinent issues in health care systems, and it consequently deterred the diffusion process. The third reason might be the unavailability of appropriate channels for accepting this type of research.[55] It is not uncommon for many incumbent mainstream journals to barely accept a new research paradigm in its very early stage. Without publicly recognized platforms, this prevented the quick dissemination of DES in health-related studies. Last but not the least, given the aforementioned reasons and the limited research experiences using DES in health studies, not many research areas in health fields could be explored. This also prevented the diffusion of DES in health research, which will be substantiated by the following analysis.

From 2007 onwards, publications witnessed an explosive increase. DES diffusion is now at the stage of fast growth in a typical S-shaped diffusion process, suggesting that there is a high possibility that this area will be staying active in the future. In our analysis, we found 1524 authors who participated in the 483 publications published in 230 journals from 1981 to 2014. The increase in adopters was stagnant from 1981 to 1995 but had an obvious increase starting from 1996. Adopters have had significant increases every year since 2003. Given the fact that the publications of 36 authors (with >4 papers/author) account for 55% of all retrieved papers, we think there might be need to promote the method in potential adopters or junior researchers.

From the channel perspective, nearly the same pattern was followed as the adopters. Publications were concentrated in 12.6% of all available journals (29 out of 230 with >2 papers), which published 51.35% of all retrieved papers. Moreover, DES papers in health research were recognized by many high-IF journals. In terms of research areas, areas of scheduling, and queuing (Area 1) and treatment strategies and/or medicines for diseases (Area 2) contributed 55.69% of all publications. According to the trend analysis, Areas 1 and 2 are still very active areas under investigation. Areas in modelling progression of diseases, modelling logistics of health care products, comparing DES with other modelling tools, and planning health service capacity for emergency events have the potential to have more publications in the future. We also found obvious increases in research areas in the past four decades, with four areas appearing in the 1980s, six new areas in the 1990s, nine new areas in the 2000s, and another three new areas from the years 2010 to 2014. The increase in adopters, channels, research areas, and their mutual re-enforcement jointly advances the diffusion of DES in health research.

Two broad factors, that is, internal and external factors, influence the shape of the diffusion pattern of DES in health-related studies. We have focused here our attention on the internal factors, namely, adopters, channels, research areas, and the high specialization of health-related issues. Since our data only cover components included in the internal factors, components in external factors (computing power, software, research funding, collaboration across disciplines, OR education in health-related fields) are beyond the scope of this paper. Therefore, we only provide a brief discussion of those factors.

As far as the internal factors are concerned, the increase in both adopters and channels is highly correlated (with CC approximating 1) with the increase of DES publications in health-related areas. In addition, the high correlation between adopters and channels could imply that they have jointly driven the increase of publications, a point that can be substantiated from the data: more adopters induced an increase in the channels, while more channels attracted more adopters for studies and publications. Both adopters and channels experienced an explosive growth starting from 2003, which as a result significantly increased the number of publications from that year onwards, which could be interpreted as something other than coincidence.

Nevertheless, there are also some issues associated with the adopters and channels that could suggest the existence of negative trends associated with the diffusion of DES in health-related research. Although the insularity of the DES research community to junior researchers is low, high barriers for potential adopters exist outside the established coherent groups in terms of recognition by mainstream journals given the fact that 36 authors participated in about 55% of all retrieved publications and the other 1488 authors participated in 45% of all retrieved publications. Regarding channels, among all of the 230 journals, 29 journals published more than 51.35% of all retrieved papers. One possible reason for this imbalance may be that new channels have only appeared rather recently. Second, due to the novelty of the DES method in the new channels and the lack of qualified reviewers, those new channels may not have been ready to review more DES-related papers during that period of time. Moreover, the newly acquired channels using DES in health-related research are targeting at highly specialized areas in health-related studies where not many researchers are familiar with DES techniques. Apart from the adopters and channels, the fast penetration of DES into more research areas (especially after 2000) has significantly advanced the diffusion of the use of this method in health-related research.

Although a detailed discussion of the impact of external factors is beyond the scope of this study, these factors could have a major influence on the diffusion of DES in the health research field. The availability of PCs and switching the operating system (OS) from DOS to Windows in the 1990s enhanced the graphic-user interface of DES software, which in turn lowered the barrier for using DES. These external changes may have driven the mild annual increase in the number of adopters, channels, and, consequently, the number of publications in the 1990s. With the functional expansion in the use of queuing theory and resource allocation issues in the newly developed DES software platform in the 2000s, DES was able to simulate more issues in health-related research. Major software platforms, namely (appearing in chronological order), SIMAN, Arena, ExtendSim, and Anylogic, have progressively increased the usability of DES in health-related research. Together, these developments in computer hardware and software have made significant contribution to the accelerated increase in DES publications in the 2000s[56,57] (Kelton et al., 2003; Siebers et al., 2010). Moreover, those systematic and extensive trainings offered by graduate programs, courses, or workshops in engineering schools, medical schools, schools of public health, independent health care–related research centers, and societies (e.g., SMDM [Society of Medical Decision Making] and ISPOR [International Society for Pharmacoeconomics and Outcomes Research]) undoubtedly have provided vehicles to disseminate knowledge and modelling techniques regarding the DES’s applications in health-related research to many scholars (potential new adopters).[3,58]

Despite the promising application prospects, the diffusion of DES in health-related research still faces challenges. First, the deficiency of teaching OR techniques including DES to students in health-related degree programs is apparent. Many researchers who want to address certain issues in health care systems lack the requisite OR expertise, which affects the increase of potential adopters. Second, given the high degree of specialization, departments or agents in health care tend to function independently and tight coordination among distinct departments is prohibitively difficult, which prevents the opportunity of using DES to address issues in processes involving multiple departments. The third challenge is the difficulty to validate DES models, disseminate the results, implement changes, and generalize the model. The outcomes directly affect how decision makers might perceive the method’s credibility and dependability, which affect the potential adopters and channels.

Although extensive search and analysis have been conducted, there still exist some limitations in this research. First of all, due to the choice of search terms and unintentional ignorance of some terms representing or resembling DES (such as multi-agent systems or agent-based approaches), some publications may have been missed. Second, since the objective of this research is to identify the diffusion pattern and factors driving the diffusion process through systematic review, the PRIMSA framework was not used even though it is a standard that is really expected in systematic reviews. The third limitation of our study is the visualization of our results. As the review covers 34 years data, 483 papers, 230 journals, 1524 authors, and 22 research areas, using figures to report the results makes them very hard to capture the meaning even when we attempted to use different ways. Therefore, we have to tabulate the results using large tables.

In conclusion, the accelerated increase in the number of adopters, channels, and research areas is jointly driving the rapid diffusion of DES in health-related research. Some challenges continue to exist for the application of DES, while interdisciplinary and transdisciplinary research is definitely bringing solutions to address those challenges. The diffusion of DES into the health-related research arena will remain active for many years in the future.

Click here for additional data file.

Supplemental material, DES_literature_dataset_new for The Diffusion of Discrete Event Simulation Approaches in Health Care Management in the Past Four Decades: A Comprehensive Review by Shiyong Liu, Yan Li, Konstantinos P. Triantis, Hong Xue and Youfa Wang in MDM Policy & Practice

Click here for additional data file.

Supplemental material, Table_A1.rjf_online_supp for The Diffusion of Discrete Event Simulation Approaches in Health Care Management in the Past Four Decades: A Comprehensive Review by Shiyong Liu, Yan Li, Konstantinos P. Triantis, Hong Xue and Youfa Wang in MDM Policy & Practice

Click here for additional data file.

Supplemental material, Table_A2.rjf_online_supp for The Diffusion of Discrete Event Simulation Approaches in Health Care Management in the Past Four Decades: A Comprehensive Review by Shiyong Liu, Yan Li, Konstantinos P. Triantis, Hong Xue and Youfa Wang in MDM Policy & Practice