Patients, primary care, and policy: Agent-based simulation modeling for health care decision support.

Primary care systems are a cornerstone of universally accessible health care. The planning, analysis, and adaptation of primary care systems is a highly non-trivial problem due to the systems' inherent complexity, unforeseen future events, and scarcity of data. To support the search for solutions, this paper introduces the hybrid agent-based simulation model SiM-Care. SiM-Care models and tracks the micro-interactions of patients and primary care physicians on an individual level. At the same time, it models the progression of time via the discrete-event paradigm. Thereby, it enables modelers to analyze multiple key indicators such as patient waiting times and physician utilization to assess and compare primary care systems. Moreover, SiM-Care can evaluate changes in the infrastructure, patient behavior, and service design. To showcase SiM-Care and its validation through expert input and empirical data, we present a case study for a primary care system in Germany. Specifically, we study the immanent implications of demographic change on rural primary care and investigate the effects of an aging population and a decrease in the number of physicians, as well as their combined effects.

Highlights

We present the hybrid agent-based simulation model SiM-Care, which aims to serve as a decision support tool for the analysis of the quality of primary care systems

Assessments are based on multiple key performance indicators such as patient waiting times and physicians’ utilization

Effects of interventions such as the use of mobile medical units or centralized appointment systems can be quantified and validated before an actual action is taken

The simulation model is very generic and can be easily adapted to individual needs and regional specifics

A case study demonstrates the application of SiM-Care for the analysis of a rural primary care system in Germany

Introduction

Primary care systems are the foundation of accessible health services. Following the definition of the American Academy of Family Physicians [5], primary care systems “serve as the patient’s first point of entry into the health care system and the continuing focal point for all needed health services”. To that end, they feature a set of primary care physicians (PCPs) who provide “primary care services to a defined population of patients”. These include “health promotion, disease prevention, health maintenance, counseling, patient education, diagnosis and treatment of acute and chronic illnesses”.

Demographic change challenges the functioning of primary care systems: Medical and technological progress paired with improved living conditions and reduced birth rates leads to an increased share of elderly citizens. In the United States, the percentage of individuals aged 65 and older is predicted to exceed 21 % of the total population by 2030 [65]. As populations age, their demand for primary care services tends to increase due to the prevalence of chronic illnesses, which disproportionately affect older adults [4, 46]. Simultaneously, primary care physicians are also aging; e.g., 34.1 % of all primary care physicians in Germany were 60 years or older by the end of 2017 [2] and thus about to retire. Moreover, fewer medical students are willing to practice primary care [46], let alone open a private primary care practice [35]. This reduces treatment capacities and exacerbates the risk for supply disruptions.

In the United States, the “confluence of a rising demand for primary care services and a decreasing supply of professionals providing these services” is considered a “crisis in primary care” [46]. In order to manage this crisis, existing systems have to adjust fundamentally [50]. Various new concepts and policies to maintain the standard of health care provision are discussed by the statutory health insurances, governments, and the Associations of Statutory Health Insurance Physicians [46, 56]. This discussion commonly distinguishes (i) microsystem improvements, which aim at enhancing a single server of the system and can be implemented at an individual level, and (ii) macrosystem reforms, which are fundamental, system-wide changes that must be implemented by policy makers [69]. Both types of system changes require validation and evaluation prior to their potentially costly implementation [50]. Naturally, this leads to the pressing question: How can we quantify the quality of primary care systems and the effects of changes?

The predominant solution to this problem is to assess the ratio between physicians and the population. In the United States or example, the Health Resources and Services Administration defines adequate health care supply based on profession- and region-specific population-to-provider ratios [16]. Similar ratio-based measures are applied in several European countries like Germany, Italy, and Spain [24, 45]. Such ratio-based assessments have several shortcomings: Even if they incorporate the local situation, ratios can only provide a very rough estimate. Furthermore, adjustment criteria are highly dependent on the definition of the underlying zones or geographic areas. They neglect factors such as the accessibility of practices and PCPs’ individual workloads. Finally, ratio-based assessments cannot account for new concepts such as telemedicine, mobile medical units, or centralized appointment scheduling.

To overcome these limitations, this paper contributes the hybrid agent-based simulation tool SiM-Care (Si mulation M odel for Primary Care). SiM-Care represents patients and PCPs on an individual level as illustrated by Fig. 1. It models patients and primary care physicians via a geo-social system, in which patients decide whether and where to request an appointment and PCPs handle appointment requests, manage patient admission, and treat patients. By tracking the resulting interactions in SiM-Care, planners can identify dependencies of subproblems, evaluate new planning approaches, and quantify the effects of interventions on the basis of multiple key performance indicators. As such, SiM-Care can serve as a versatile decision support tool for primary care planning that is very generic, can be easily modified, and can be extended to meet individual needs. Based on empirical data from a German primary care system, we illustrate how to generate simulation scenarios and showcase SiM-Care through a case study. To the best of our knowledge, SiM-Care is the first simulation model that captures entire primary care systems with all physicians and patients as individual agents and allows for the simultaneous consideration of microsystem improvements as well as macrosystem reforms. The open source release of SiM-Care is currently in preparation.

Fig. 1

Geo-social system of patients and physicians. Note: Map tiles by Stamen Design, under CC BY 3.0. Data by OpenStreetMap, under ODbL

The remainder of this paper is structured as follows. Section 2 discusses related work. We introduce SiM-Care on the basis of the ODD framework [26] in Section 3. Section 4 presents a case study based on real-world data to aid model validation and showcases the application of SiM-Care in health care planning. Section 5 summarizes the use and requirements of SiM-Care and further model applications before outlining future research.

Related work

Decision support for health care planning is of increasing importance [29]. Related tools have to deal with the detail complexity that is inherent to the health care sector, finding it difficult to rely on the principle of “keep it simple, stupid” (KISS, [12]). Simulation modeling can deal with this complexity by “simulating the life histories of individuals and then estimating the population effect from the sum of the individual effects” [23]. As such, simulation models represent a powerful tool to inform policy makers: They can provide valuable insights into the dependencies within health care systems and allow for the prediction of the outcome of changes in strategy ahead of potentially costly and risky real-world interventions [23, 29].

Given these potentials, the use of computer simulation in health care delivery has significantly increased over the recent years [69]. The resulting body of literature is rich, as shown by several surveys of existing contributions. Examples include [15, 23], who review the use of simulation modeling for health care in general. Other surveys are mostly focused on particular simulation paradigm, e.g., system dynamics [11, 32], discrete event simulations [29, 36], agent-based modeling [7, 64], and hybrid simulations [13, 14]. Most recently, with general research attention being focused on the matter of pandemics in general and COVID-19 in particular, [20] point out opportunities for health care simulation modeling for pandemics beyond epidemiological modeling. The authors list a variety of decisions in emergency health care that simulation modeling, such as exemplified by SiM-Care, can support. Nevertheless, some sources remark that research on health care modeling continues to be under-reported [10] and highlight a “lack of real-world involvement in published simulation modelling” [15].

As background for the primary contribution of presenting a novel simulation system, we consider several examples of the computational study of primary care systems. Related references stem from a literature research featuring the keywords {simulation, decision support, system dynamics, discrete event, agent based model} + {primary care, health care}. Table 1 lists the resulting sources and differentiates the simulation paradigm, the modeling objective, and information on stakeholder involvement and maintenance. Accordingly, we broadly partition the considered models into two groups: those studying microsystem improvements and those investigating macrosystem reforms.

Table 1

Classification of related simulation models in primary care

Ref.	Method	Setting	Objective	Stakeholder Involvement	Maintenance
[18]	DES	Single primary care clinic	Eval. sequencing- and appointment rules	No information	No information, only management recommendations
[25]	DES	Single outpatient clinic	Testing a new scheduling approach	Stakeholder involvement through action research	No information, implemented recommendations
[59]	DES	Single primary care clinic	Eval. of appointment systems	No information	No information on maintenance, emphasize adaptability
[61]	DES	Single primary care clinic	Eval. effects of six factors on clinic’s performance	Management involvement in data collection	Researchers provided only recommendations, no system
[67]	DES	Single primary care clinic	Eval. implications of capacity allocations and appointment scheduling	Aimed to support stakeholders, no explicit involvement	No information on availability and maintenance
[69]	DES	Single pediatric clinic	Eval. effects of scheduling templates, staff ratios, room assignments	Analysis of exemplary clinic, no information on stakeholder involvement	No information
[31]	ABM	Entire health care system	Investigate paradox of primary care	Cooperation between academics and patients, caregivers, and clinicians	Model, software, and worksheets available for download and discussion
[47]	SD	Entire primary care sector	Eval. effects of system-wide policy changes	Group model building, development workshop	Model handed over to Regional Health Systems

Studies of microsystem improvements include [18, 25, 59, 61, 67, 69]. In contrast to SiM-Care, these models only feature a single primary care practice. Moreover, all of these models adopt a different approach to the representation of patients: While SiM-Care models a persistent patient population that is shared by all providers, the referenced models represent patients only as they arrive at the practice and disregard their evolution when they are discharged. As a result, such models do not account for the effects of individual microsystem improvements on the entire system.

Other references, such as [31, 47], investigate macrosystem reforms and feature entire primary care systems. Still, the agent-based model [31] differs from SiM-Care in its objective: It investigates the external effects of treatments in primary care on the entire health care system, whereas SiM-Care focuses on the processes within primary care systems. Hence, [31] does not model internal processes, such as appointment scheduling. Model [47] implements the system dynamics paradigm and thus focuses on a higher level system representation than SiM-Care. While system dynamics models do not consider the level of micro-detail offered by agent-based simulations, they require less computational effort to run simulation experiments. In addition, they may provide a more concise model that is easier to communicate to stakeholders. This motivates us to specifically consider aspects of model validation in the case study.

To the best of our knowledge, no existing simulation model allows the simultaneous consideration of microsystem improvements and macrosystem reforms in primary care systems that SiM-Care provides.

Simulation model

Creating a simulation model means both formalizing what the model includes and deciding what to leave out [64]. Therefore, this section first discusses the process of creating the model and the involvement of stakeholders before listing the resulting model assumptions and limitations. Subsequently, we formally describe all modeled components and relationships.

SiM-Care is designed to meet the requirements of various stakeholders. Researchers access the model to evaluate outcomes from prescriptive planning approaches based on mathematical modeling. The modeling team regularly consulted with health care practitioners including primary care physicians, health insurance representatives, as well as representatives from industry associations and administrative authorities. Generally, we find that explaining the simulation model through the agent-based paradigm and presenting results from related studies allows for in-depth discussions, where the simulation provides a helpful tool for illustration.

At an early modeling stage, it became evident that the model would never be able to mirror all intricacies of a primary care system. Therefore, development focused on the idea of “modeling the problem, not the system”, as recommended by [49]. Here, the primary problem is evaluating the macro-level effects from combining of health care supply in the form of a population of physicians versus a demand in the form of a population of patients.

Thereby, we model the trade-offs between the objectives pursued by three stakeholder groups: patients, PCPs, and policy makers. SiM-Care assumes that PCPs strive to efficiently utilize their time, whereas patients strive for a quick response to their health concerns. Thereby, the model illustrates the trade-off between efficiency and patient-centered care. Policy objectives can range from minimizing the cost of health care to maximizing the degree of patient-centered care. Policy makers are not represented by agents within SiM-Care. Instead, policy decisions set relevant model parameters such as the number of physicians in the system and treatment standards. To model interactions on a micro-level, SiM-Care thus features two populations of agents: potential patients and primary care physicians .

Every patient resides at a specific location, belongs to a certain age group and has an individual health status and treatment preferences; compare Fig. 2. Patients develop acute illnesses that depend on their age and health status and require treatment. Additionally, patients may suffer from long term chronic illnesses, which need to be monitored by a physician. To receive medical attention, patients either schedule an appointment or visit a PCP’s practice without prior notice. Patients’ decisions depend on their individual preferences and health status. These factors determine the choice of physician, the type of the visit (walk-in/ appointment), and the time of the visit.

Fig. 2

Concept of SiM-Care showing both types of agents with their main attributes as well as interactions between agents

All PCPs practice at a certain location and have weekly opening hours; see Fig. 2. Moreover, every physician follows individual strategies that govern how they manage appointments, admit patients, and perform treatments. As patients and physicians interact, they influence each other and adjust their preferences and strategies.

In the following, we list – to the best of our knowledge – the underlying assumptions and the limitations that may restrict the application of SiM-Care.

SiM-Care focuses on the adult population and neither models pediatric care nor gender differences. While we do differentiate patients by health status, age, and illnesses, we assume that all patients implement the same strategies when arranging appointments or becoming walk-ins. Furthermore, the model assumes that all patients attend their appointments, i.e., there are no no-shows patients.

As it stands, the model does not consider cross-effects between illnesses that may occur, e.g., when a chronic illness worsens the progression of an acute illness. As there is no model of direct patient interaction, SiM-Care does not include an explicit infection model, i.e., the probability of a patient developing an acute illness is independent of their interaction with other patients and physicians. While patients who suffer from illnesses seek treatment, the duration of an illness is not directly affected by treatments.

On the provider side, we do not model a relationship between primary care systems and specialists or hospitals. Physicians do not differentiate patients according to their insurance policy. The physicians’ service times do not depend on the patients’ number or types of illnesses and physicians do not offer home visits. We assume that PCPs are never late or absent and the model includes neither seasonality nor holidays. Finally, we assume independence of surrounding municipalities, such that the modeled primary care system is a closed system.

In the remainder of this section, we provide the model’s formal description based on the ODD framework described by Grimm et al. [26]. For the sake of brevity, some of the very technical modeling details are only presented in Appendix A.

Simulation environment

SiM-Care’s environment entails the geographical and temporal structure as well as policy effects. Within the model, locations are represented using the geographic coordinates latitude and longitude.

The modeled time period is considered as a continuum structured by points in time and durations. For any time object , indicates the day and specifies the time as an increment of day known as decimal time. That is, we use the same encoding for points in time and durations as context uniquely defines which of the former a time object refers to. For example, corresponds to day 38 and 24 ⋅ 60 ⋅ 0.55 = 792 minutes, i.e., 1:12 p.m. as a point in time or, analogously, to a duration of 38 days, 13 hours, and 12 minutes. To ease notation, we associate every point in time and duration with the non-negative value .

In addition to the continuous representation of time, we structure each day into a morning and an afternoon session as it is common practice in primary care [40]. Each session is uniquely defined by a day and a binary indicator γ ∈{0,1} that defines whether it is the morning (γ = 0) or the afternoon (γ = 1) session. Sessions reoccur on a weekly basis which yields an equivalence relation on the set of sessions Λ via The resulting equivalence class for a session λ ∈Λ defined as contains all sessions sharing the same day of the week and time of the day, e.g., all Thursday afternoon sessions. Thus, we model and distinguish 14 sessions each week, i.e., Monday to Sunday with a respective morning and afternoon session that we associate with the set of all equivalence classes . Particularly, this allows for a distinction between sessions on weekdays and weekends.

Entities and state variables

Patients and PCPs are the active entities in the simulation. Their interaction is motivated by patients’ suffering from illnesses and therefore seeking treatment with PCPs via appointments or walk-in visits.

Going from simple to more elaborated, we begin by describing the self-containing entities of SiM-Care and end with the description of the agents representing patients and physicians.

Objectives

When patients suffer from an acute illness, they want to be treated as soon as possible, ideally by their preferred physician. For the treatment of chronic illnesses and the follow-up care of acute illnesses, patients prefer treatment by the same physician through appointments in regular intervals. Physicians, on the other hand, aim at efficiently utilizing their available time while minimizing overtime. Thus, patients’ and physicians’ objectives conflict, as it is ineffective for physicians to fully comply with patient demands: To ensure that all short-notice appointment requests can be accommodated, PCPs would have to withhold too much treatment time. Providing follow-up appointments in strict intervals would prevent PCPs from reacting to demand fluctuations.

Policy makers follow a multitude of conflicting objectives. On the one hand, they need to ensure a certain minimum standard in health care quality to guarantee patients are treated when necessary. On the other hand, they cannot afford to subsidize an excessive number of physicians. Thus, policy makers necessarily aim at a trade-off.

SiM-Care represents policy decisions through their resulting parameter values, e.g., the number of physicians and their distribution.

Illnesses and families of illnesses

Illnesses are health concerns that cause discomfort to patients and require treatment. They belong to a certain illness family (e.g. cold or heartburn), have a certain seriousness (e.g. mild or severe), persist over a certain period of time, and require initial treatment within an acceptable time frame as well as subsequent follow-up visits in regular time intervals. SiM-Care formalizes illnesses as tuples with attributes as shown in Table 2. Thereby, s ∈ [0,1] defines the seriousness of the illness, defines the illness family, and defines the duration. The parameter defines the willingness to wait, which is the patient’s maximum accepted waiting time for the initial treatment of this illness. The parameter defines the follow-up interval, which specifies the frequency of the required aftercare that follows the initial treatment of this illness. For some illnesses , the characteristics duration and follow-up interval do not apply. This is indicated by setting d = ∅ and ν = ∅.

Table 2

Attributes of illnesses

Attribute	Type	Unit
seriousness	si ∈ [0, 1]
illness family	f_i∈F
duration	d_i∈T	[days]
willingness to wait	ω_i∈T	[days]
follow-up interval	ν_i∈T	[days]

While emerging illnesses vary in their manifestation, families of illnesses define the common constant traits of all illnesses belonging to the same family. In our model, the common constant traits of all illnesses with seriousness s ∈ [0,1] belonging to family are the expected duration , the expected willingness to wait , and the follow-up interval . The expected duration and expected willingness to wait serve as means for the distributions, from which we sample each stochastic duration d and stochastic willingness to wait ω. Thus for all emerged illnesses , it generally holds that and . Only the follow-up interval of emerged illnesses derives from the illness family in a deterministic way, i.e., .

In order to define the common traits of illnesses, families of illnesses are specified by the three linear functions shown in Table 3. As above, we indicate the inapplicability of the characteristics duration or follow-up interval to families of illnesses by setting D = ∅ and N = ∅, respectively.

Table 3

Attributes of families of illnesses

Attribute	Type
linear function for expected duration	D_f:[0,1]→T
linear function for expected willingness	W_f:[0,1]→T
linear function for follow-up interval	N_f:[0,1]→T
chronic attribute	κf ∈{0, 1}

To illustrate this concept, consider the family “common cold” defined by the functions D(s) = 10s + 3, W(s) = − 3s + 3, and N(s) = − 2s + 7. When a patient develops a mild case of “common cold” (s = 0.2), the illness family “common cold” defines the expected duration, expected willingness to wait, and follow-up interval of the mild cold as D(s) = 5 days, W(s) = 2.4 days, and N(s) = 6.6 days. The actual values of the specific cold are stochastic and vary around their expected counterparts, e.g., d = 5.5 days and ω = 2.7 days. The follow-up interval is deterministic and derives from the illness family via days. Note, that the particular mild cold in this example will thus not require a follow-up visit, as its duration is shorter than the follow-up interval, i.e., d < ν.

To model chronic health concerns that persist over an extended period of time, such as diabetes, a chronic attribute κ ∈{0,1} identifies families of chronic illnesses. Thereby, κ partitions into an acute set and a chronic set . This directly induces a partition of the set of illnesses into acute and chronic illnesses and .

Acute illnesses develop and subside over time and patients can simultaneously suffer from an arbitrary number of acute illnesses . Chronic illnesses are static in SiM-Care – they neither develop nor heal. Instead, each patient suffers from at most one chronic illness throughout the modeled time period. To distinguish patients suffering from a chronic illness, we refer to them as chronic patients.

Ideally, attributes of families of illnesses should be estimated from empirical data. For example, follow-up intervals can be estimated from collections of individual treatment histories that indicate the times at which patients were treated for a particular diagnosis. In case empirical data is not available, estimates can be based on expert knowledge or official treatment guidance such as disease management programs [6]. Moreover, if one does not aim to make detailed predictions for a specific primary care system, it is possible to infer attributes or model entirely artificial health concerns as long as they are sufficiently validated in a baseline analysis. This last and most basic approach is adopted in the case study presented in Section 4.

Appointments

Appointments specify the point in time when the treatment of a specific patient is scheduled to take place. To that end, appointments are defined by the time of the appointment , the attending primary care physician , and the patient receiving treatment. At any point in time, non-chronic patients can have at most one scheduled appointment , called the acute appointment. Acute appointments are intended for the initial treatment of acute illnesses, the follow-up treatment of acute illnesses, or both. Chronic patients may additionally have a regular appointment to treat their chronic illness. While chronic illnesses are only treated during regular appointments, all acute illnesses are treated during every appointment.

Age classes

Age classes define common characteristics of patients. For patients of age class , these characteristics are deviations from the expected illness duration and from the expected willingness to wait , the probability to cancel an appointment after full recovery p ∈ [0,1], and the expected number of annual acute illnesses defined through the linear function ; see Table 4. The deviation from the expected illness duration is a multiplicative factor that determines whether the expected illness duration extends () or shortens () for patients of age class ; analogously for . The linear function defines the expected number of annual acute illnesses for patients in age class which depends on the patient’s health condition c ∈ [0,1] which can range from perfectly healthy (c = 0) to extremely delicate (c = 1).

Table 4

Attributes of age classes

Attribute	Type
linear function expected annual acute illnesses	I_a:[0,1]→ℝ_≥0
deviation from expected illness duration	Δad>0
deviation from expected willingness to wait	Δaω≥0
probability to cancel appointments	pa ∈ [0, 1]

Age-class-illness distribution

The age-class-illness distribution builds the connection between the set of age classes and the set of acute families of illnesses . To that end, πact defines the expected distribution of acute illness families per age class, i.e., among all developed acute illnesses by patients of age class , a fraction πact(a, f) ∈ [0,1] is expected to belong to illness family .

Patients

Patients are the driving force of the simulation, as their health concerns trigger most events. All non-chronic patients are characterized by their geographical location , health condition c ∈ [0,1], acute illnesses , age class , acute appointment , and preferences. While the location, health condition, and age class of each patient remain constant throughout a simulation experiment, a patient’s acute illnesses, acute appointment and preferences can change over time. Chronic patients are additionally characterized by a constant chronic illness and a variable regular appointment . Table 5 summarizes the attributes shared by all patients as well as the attributes specific to chronic patients.

Table 5

Attributes of (chronic) patients

Attribute	Domain	Type
location	ℓ∈L	constant
health condition	c ∈ [0, 1]	constant
age class	a∈A	constant
acute illnesses	I^act⊆I^act	variable
emergency flag	ε ∈{0, 1}	variable
acute appointment	b^act∈B	variable
considered PCPs	G^con⊆G	constant
availabilities	α:Λ/∼→{0,1}	constant
appointment ratings	rρapp(ϕ)≥0,∀ϕ∈G^con	variable
walk-in ratings	rρwalk(ϕ,[λ])≥0,	variable
	∀ϕ∈G^con,∀[λ]∈Λ/∼
chronic illness	I^chro={ς}⊆I^chro	constant
regular appointment	b^reg∈B	variable
family physician	ϕ^fam∈G^con	variable

Patients’ preferences determine when, where, and how they pursue treatment. Specifically, each patient considers a set of PCPs and never seeks treatment with PCPs outside the consideration set. Since continuity in the treatment of chronic illnesses is particularly important, chronic patients arrange all regular appointments with a distinguished family physician . While every patients’ consideration set remains constant throughout the modeled time period, patients reevaluate and vary their family physician. Naturally, patients have personal schedules and cannot attend all weekly sessions. Thus, the model assumes that each patient has a constant set of weekly-reoccurring session availabilities given by , where 0 encodes unavailability. Finally, patients maintain individual appointment ratings as well as session-specific walk-in ratings for every weekly session and every considered physician .

Via internal ratings, patients track their satisfaction with a physician’s services. Whenever a patient seeks consultation, the choice of physician is determined by their current ratings. Ratings incorporate patients’ sense of geographic distance, matching of opening hours with availabilities, and previous positive and negative experiences. As patients adjust their ratings over time, they adjust their choice of PCP. If a physician is unable to meet an appointment request, incurs excessive waiting time, or rejects patients due to capacity overruns, patients reduce their rating. Positive experiences such as successful appointment arrangements or short waiting times increase ratings. Note that ratings are only internal valuations and not communicated to other patients or physicians.

When patients begin to suffer from a new illness, they always seek treatment. To that end, patients first request an appointment from the set of considered PCPs, . Patients make up to two appointment requests in order of the appointment rating . If both requested PCPs fail to offer a feasible appointment within the patient’s willingness to wait, patients forgo an appointment and visit a PCP as a walk-in. They select the PCP for the walk-in visit based on the walk-in rating of the targeted session λ ∈Λ.

Upon arrival, a PCP may reject patients due to, e.g., capacity overloads. Following a rejection, patients update their rating of this PCP and attempt a new walk-in visit at the then-highest-rated PCP. Rejected patients are flagged as emergencies (ε = 1) for as long as they unsuccessfully continue to seek treatment. PCPs may include the emergency state in their decision making.

Until an illness subsides, patients continuously arrange follow-up appointments with the attending physician in the follow-up interval . Analogously, chronic patients continuously arrange regular appointments with their family physician in the follow-up interval of their unique chronic illness .

Primary care physicians

PCPs’ practices feature an uncapacitated waiting room. The model characterizes physicians by their geographic location , opening hours, as well as an individual set of strategies to schedule appointments, manage patient admission, and organize treatments. Table 6 summarizes the attributes of PCPs.

Table 6

Attributes of PCPs

Attribute	Type
location	ℓ∈L
opening hours	o:Λ/∼→H×H
appointment scheduling strategy	S∈S^app
admission strategy	S∈S^adm
treatment strategy	S∈S^tmt

SiM-Care assumes that all physicians operate in clinical sessions. Opening hours for these sessions are weekly-reccurring and therefore defined over the session of the week via where denotes the set of decimal times; cf. Section 3.1. Opening hours specify for each session λ ∈Λ the time window during which patients are admitted. The beginning of session λ = (δ, γ) ∈Λ is defined as , the session’s end as . To encode that a PCP is closed, we set o([λ]) = ∅. Physicians use the first hour after the end of each session as time buffer to compensate for possible delays and walk-in patients. Figure 3 visualizes a PCP’s working day.

Fig. 3

Schematic representation of a PCP’s morning (λ0) and afternoon (λ1) session visualizing service-, idle- and overtime

PCPs implement a set of strategies to schedule appointments, decide on patient admissions, and organize the treatment of patients. These strategies are interchangeable model components that are defined via interfaces. They govern the physicians’ interactions with patients in terms of sensing, predicting, adapting, and learning. In the following, we summarize the main functionality of each strategy. All technicalities and the exemplary strategies that are used as part of our case study can be found in Appendix A.1.

The PCP’s appointment scheduling strategy defines how they allocate consultation time to appointment slots and how the resulting slots are assigned to requesting patients.

The PCP’s treatment strategy defines the order of treatment based on patients’ waiting times. To account for the observation that physicians consciously or unconsciously adjust service times depending on demand [28], treatment policies define when and how physicians adjust their consultation speed and thereby service times.

The PCP’s admission strategy determines whether they admit an arriving patient based on the current workload. SiM-Care requires PCPs to treat all admitted patients. Thus, when physicians underestimate their workload, they may have to work overtime. On the other hand, physicians that overestimate their workload fail to fully utilize their available time. At the end of a session’s buffer, physicians may learn by reevaluating their predictions and adapting their admission policy.

Process overview and scheduling

SiM-Care models the progression of time via the discrete event paradigm. Thus time passes between discrete events, at which the system state is updated. The model stores events of the form (t, e) in a sequential queue where is the point in time an event of type occurs. By construction, the event queue never runs empty.

Every simulation run follows the structure depicted in Fig. 4, chronologically processing the events from up to a specified point in time . In this, the specific process depends on the event type . Appendix A.2 details the modeled events and their processing.

Fig. 4

Structure of simulation run with time horizon T

Modeling variability

SiM-Care relies on stochastic values to approximate real-world variability and control the frequency of events. This applies to aspects of illnesses as well as to patient arrivals, appointment cancellations and service times. In consequence, every simulation experiment includes multiple stochastic repetitions of the modeled time period, termed simulation runs. When examining simulation output, we account for the resulting variability through confidence intervals.

Table 7, lists all probabilistic model aspects. Appendix A.3 details the parameterization of the distributions underlying the random values.

Table 7

Probabilistic model aspects

Aspect	Distribution
frequency of acute illnesses	exponential distribution
type of acute illnesses	age-class-illness distribution
seriousness of acute illnesses	triangular distribution
duration of acute illnesses	log-normal distribution
patients’ willingness to wait	Weibull distribution
patient punctuality	normal distribution
walk-in arrivals	beta distribution
service time	log-normal distribution
appointment cancellations	binomial distribution

Emergence and observation

SiM-Care tracks multiple key performance indicators to illustrate the trade-offs between the stakeholders’ objectives. These indicators emerge from agent interactions based on patients’ evolving preferences and physicians’ evolving strategies. Appendix A.4 provides formal definitions of all evaluated key performance indicators. In the following we summarize them briefly.

From the patients’ point of view, key performance indicators include access time, access distance, and waiting time. To evaluate the patients’ indicators, SiM-Care keeps track of the total access time of arranging acute and regular appointments, the total number of arranged acute and regular appointments, the total number of attended appointments, the total number of walk-in patients, the total access distance of patients, and the total waiting time for both patients with appointment and walk-ins.

From the physicians’ point of view, key performance indicators include the utilization, overtime, number of treatments, and number of rejected patients with and without appointment. To evaluate the physician’s indicators, SiM-Care collects on physician level the total service time spent treating patients, the total number of performed treatments, the total overtime, and the total number of rejected patients with and without appointment. The total available working time per PCP required to compute the utilization, can be derived from the opening hours o and the modeled time horizon T.

Input, initialization, and warm-up

SiM-Care codes many values as flexible parameters. Setting up a simulation experiment requires specifying the parameter values. Each simulation scenario represents a particular setting, in which a specific set of patients interacts with a specific set of physicians under specific circumstances.

As part of every simulation scenario, the modeler specifies the families of illness , the age classes , the age-class-illness distribution πact, and the set of physicians with all their attributes. The set of patients is only partially defined through the simulation scenario: Each scenario specifies every patient’s location , health condition c ∈ [0,1], age class , availabilities , and, for chronic patients, a chronic illness . The remaining attributes of patients are derived as described in Appendix A.5.

To initialize a simulation experiment, modelers can broadly choose one of two approaches: empty and interim initialization. An empty system state is inherently unrealistic, as it sets all parameters that are subject to simulation dynamics to zero. An ideal interim initialization would mean that there is no period when the system state does not align with the real-world observations. However, this creates additional challenges for validation. For a structurally valid simulation, a valid interim state should automatically emerge from an empty state initialization after a warm-up period. Thus, we rely on an empty-state initialization and include a warm-up period, where the simulation state does not align with any plausible real-world state. The duration of the warm-up and the length of the modeled time horizon are both variable per experiment.

Submodels

We consider various aspects of SiM-Care that rely on an internal logic as submodels. One of the most basic submodels describes the logic of distances and travel times. More complex examples include the logic underlying patients’ behavior when requesting appointments and visiting practices as walk-ins. Appendix A.6 provides the specifics of all submodels.

Structural validation and verification

We carried out validation and verification for SiM-Care according to the best practices documented in the literature [42, 57]. To ensure a correct model implementation (verification), we followed established programming practices. We used object oriented programming to write modular code, implementing SiM-Care in Java 8. All random distributions rely on the Apache Common Math library [48]. We verified each module individually through unit testing. Assertions ensure that variables remain within their specifications at runtime. To detect undesired model behavior, SiM-Care can trace the entire simulation process. Traces are specialized logs that contain all information about the model’s execution. In SiM-Care, traces are textual and comprehensible to modelers. They can track agents throughout the model and contain all information that would be required for an animated visualization. By analyzing traces and input output relationships, we performed dynamics tests for multiple simulation scenarios of various sizes with different system setups.

To ensure that the conceptional model serves as an adequate representation of real primary care systems (validation), we took several measures. With regard to face validity, we presented the conceptual model to physicians and decision makers from health insurers as well as public authorities. Furthermore, SiM-Care builds on data from the literature as well as empirical data collected on-site. Moreover, we visited a primary care practice and interviewed staff to capture and understand the daily processes and routines of PCPs. For the specific scenarios featured in the case study, we validated the simulation output with available empirical data. Details on this historical validation can be found in the baseline analysis of the following case study.

Case study

To demonstrate the potential use of SiM-Care, we present a case study evaluating the effects of changes in the population of a primary care system. Specifically, we create a baseline scenario representing a real-world primary care system and investigate two possible changes from the status quo. On the one hand, we let the number of PCPs decline as a result of a decreasing interest in opening a primary care practice in rural areas [35]. On the other hand, we let an aging population cause a shift in the quality and intensity of illnesses and the resulting health care requirements. By considering both changes individually and in combination, we create three “what-if” scenarios that we compare to the baseline scenario.

Each scenario models a time period of one year preceded by warm-up period. As SiM-Care relies on stochastic values, every simulation experiment includes 20 independent runs. While 20 repetitions are generally at the low end of the suggested number of runs, the resulting confidence intervals are sufficiently small to assess the qualitative effects of changes between scenarios.

Baseline scenario

The real-world primary care system that serves as the template for our study comprises three predominantly rural municipalities in western Germany (Roetgen, Simmerath, and Monschau) with a total population of approximately 36000 inhabitants and 20 primary care physicians. For the considered primary care system, empirical data concerning the physicians’ distribution and opening hours was provided by the responsible department of public health or obtained from the responsible association of statutory health insurance physicians [37]. The distribution of patients and their demographic composition is available from the national census [33] and official population projections by the federal state [34]. The distribution of illnesses and their characteristics can be estimated from publications of health insurances and federal government agencies [27, 53]. All unavailable data was either empirically collected in a primary care practice or, where this was not possible, inferred. For the sake of clarity, Table 8 summarizes our basis for the selection of each input parameter.

Table 8

Basis for the selection of input parameters

Attribute	Basis (Source)
PCPs
location	empirical (dept. public health)
opening hours	empirical ([37])
strategies	literature ([17, 39, 40])
Patients
location	empirical ([33])
age class	empirical ([33])
health condition	inferred
Age classes
exp. annual acute illnesses	inferred
dev. illness duration	inferred
dev. willingness to wait	inferred
availabilities	inferred
appointment cancellation	inferred
chronic patients	empirical ([53])
Families of Illnesses
characteristics	inferred
age-class-illness dist.	empirical ([27])

In the following, we discuss how the available empirical data translates into a simulation scenario. To that end, we detail the input parameter choices, i.e., the modeled physicians, patients, age classes, families of illnesses, and age-class-illness distributions.

According to data provided by the Aachen department of public health in 2017, there are 20 primary care physicians with health insurance accreditation in the three municipalities. The physicians’ exact locations are specified as part of the provided dataset (cf. Fig. 5) and the physicians opening hours were obtained from the Association of Statutory Health Insurance Physicians Nordrhein [37]. All considered physicians are closed on Saturdays and Sundays. Concerning the employed strategies, all physicians apply the individual-block/fixed-interval appointment scheduling strategy, priority first come, first served treatment strategy (PFCFS), and priority threshold admission strategy; cf. Appendix A.1.

Fig. 5

Locations of PCPs with health insurance accreditation and population cells reported by the 2011 census [33]

The latest publicly available high resolution population data for the considered region is the German Census conducted in 2011 [33]. At a resolution of 2754 population cells measuring one hectare each, the 2011 Census reports a total population of 35542 for the three municipalities; compare Fig. 5. This population includes children under the age of 16 who the Census records on municipality level: Roetgen 1390, Simmerath 2383, and Monschau 1794. To exclude children under the age of 16 from our patient population, we proceed as follows: First, we fix one adult per population cell as we assume that children under the age of 16 do not live on their own. Then, we sample the number of under 16-year-olds from the remaining population of each municipality according to a uniform distribution.

Performing this procedure for each municipality individually, we obtain the final patient population consisting of 29975 patient agents distributed over 2754 population cells.

The location for each patient is sampled from the associated population cell according to a uniform distribution. Patients’ health conditions c ∈ [0,1] are sampled from a beta distribution with shape parameters p = q = 25 such that all patients have an expected health condition of .

The baseline scenario differentiates three patient age classes: young (16-24), middle-aged (25-65), and elderly (> 65). The characteristics of the modeled age classes are shown in Table 9.

Table 9

Age classes

	16-24	25-65	> 65
exp. illnesses	Ia(c)= 6c	Ia(c)= 7c+ 1	Ia(c)= 9c+ 1
dev. duration	Δad=0.8	Δad=1.0	Δad=1.2
dev. willingness	Δaω=1.2	Δaω=1.0	Δaω=0.8
prob. cancel	pa= 0.95	pa= 0.8	pa= 0.7

Based on Census data [33], the age class of each patient depends on the discrete probability distribution shown in Table 10. The age-class-dependent attributes of each patient agent are subsequently determined as follows: Each patient’s session availabilities α are determined by performing a Bernoulli trial based on the age-class dependent success probabilities from Table 10. To decide whether a patient is chronically ill, we perform a Bernoulli trial using the success probabilities from Table 10 that were estimated based on [53].

Table 10

Age specific parameters for patient generation

	16-24	25-65	> 65
age class distribution	0.1196	0.6318	0.2486
availability probability	0.85	0.55	0.95
chronic illness probability	0.12	0.33	0.52

Families of illnesses

The most important classification system for illnesses world-wide is the International Classification of Diseases and Related health Problems (ICD) maintained by the World Health Organization. In its current revision, ICD-10 [3] distinguishes more than 14000 codes. For the purpose of SiM-Care, such a granular illness distinction is generally not necessary. Thus, we can aggregate ICD-10 codes, e.g., using the 22 chapters of ICD-10, or considering only a subset of all ICD-10 codes, e.g., the ones most frequently reported. In the baseline scenario, we consider a subset of the 100 ICD-10 codes most frequently reported to the Association of Statutory Health Insurance Physicians Nordrhein [38]. The attributes of families of illnesses can be estimated based on historical treatment data which is commonly available to health insurers. This data is protected by confidentiality and cannot be published. Thus, we only estimate all attributes which yields the families of illnesses listed in Table 11.

Table 11

Characteristics of considered families of illnesses

ICD	Name	Exp. willingness Wf	Exp. duration Df	Treatment frequency Nf	Is chronic
I10	high blood pressure	Wf(s) = − 10s + 20	not applicable	Nf(s) = − 20s + 100	true
E11	diabetes	Wf(s) = − 4s + 14	not applicable	Nf(s) = − 10s + 90	true
I25	ischemic heart disease	Wf(s) = − 4s + 10	not applicable	Nf(s) = − 30s + 100	true
E78	high cholesterol level	Wf(s) = − 5s + 8	Df(s) = 4s + 8	Nf(s) = − 2s + 11	false
M54	back pain	Wf(s) = − 3s + 4	Df(s) = 9s + 5	Nf(s) = − 4s + 11	false
Z25	vaccination	Wf(s) = 40	not applicable	not applicable	false
J06	cold	Wf(s) = − 2s + 2	Df(s) = 5s + 4	Nf(s) = −s + 6	false

Age-class-illness distributions

The age-class-illness distribution is estimated on the basis of the reported incidence rates of a large German health insurer published in [27]. We aggregate [27] by gender and age to obtain the age-class-illness distribution shown in Table 12. Analogously, we determine the expected distribution of chronic families of illnesses among the modeled age-classes denoted by shown in Table 12.

Table 12

Age-class-illness distributions πact and πchro

	16-24	25-65	> 65
high cholesterol level	0.02	0.24	0.36
back pain	0.32	0.38	0.28
vaccination	0.14	0.14	0.27
cold	0.52	0.24	0.09
high blood pressure	0.17	0.65	0.61
diabetes	0.33	0.16	0.2
ischemic heart disease	0.5	0.19	0.19

The distribution πchro is not part of the baseline scenario, as it is only required to generate the unique chronic illness of chronic patients. This process is analogous to the process of generating acute illnesses described in Appendix A.3.

Duration of warm-up

To decide an appropriate length for the warm-up period, we simulate the baseline scenario for a time period of 70 years and track all performance indicators for each year individually. Figure 6 shows the resulting data series for the average access time, average weekly overtime, and average waiting time of walk-ins. The illustrated series suggest that the performance indicators stabilize after 20 to 40 years. To formalize this assessment, we employ the Schruben-Singh-Tierney test for initialization bias [60] to determine an appropriate truncation point for the warm-up period. Specifically, we use a two-sided test with a significance level of 5%. The scaling parameter is estimated using the batch means method with 5 batches based on the last half of the data series as suggested in [41]. Increasing the warm-up duration in steps of 5 years, the null-hypothesis of no initialization bias is first not rejected for all performance indicators for a warm-up duration of 30 years. Consequently, we choose a warm-up period of 30 years in the following.

Fig. 6

Data series of performance indicators in the baseline scenario for every year in a 70 year time period

Baseline analysis

Table 13 reports the resulting expected key performance indicators as well as the associated exact 95 %-confidence intervals for each tracked performance indicator; compare Section 3.5. In the status quo, each physician performs, on average, 10137 treatments per year. This amounts to an average number of 6.76 physician contacts per patient, which is slightly above the 6.6 annual PCP contacts reported back in 2006 [1]. Roughly 47 % of patients visiting a physician in our baseline scenario are walk-in patients, which is consistent with the observed 48 % share of walk-ins in our empirical dataset of service times; compare Appendix A.3. We were unable to obtain empirical data on overtimes, as most primary care physicians are self-employed and even the definition of overtime is unclear. However, the estimated average overtime per physician (according to our definition) seems to be too low at just 4 minutes per week. This can be explained by the incorporated buffers and the exclusion of mandatory physician’s activities, such as reporting and accounting, from the simulation model.

Table 13

Mean performance indicators and 95 %-confidence intervals obtained by repeating each simulation experiment 20 times for each simulation scenario variant

	Baseline Scenario		Decline in PCPs Short-term Shift		Decline in PCPs Medium-term Shift		Aging Patients Short-term Shift
	Mean	95 %-CI	Mean	95 %-CI	Mean	95 %-CI	Mean	95 %-CI
avg. # treatments	10137	[10128, 10146]	12444	[12431, 12457]	15094	[15085, 15104]	10237	[10229, 10245]
avg. # walk-ins	4747	[4738, 4756]	6967	[6954, 6980]	9449	[9440, 9459]	4846	[4838, 4854]
avg. # acute appts.	3214	[3212, 3216]	2764	[2761, 2768]	2331	[2324, 2339]	3192	[3191, 3193]
avg. # regular appts.	2176	[2175, 2178]	2713	[2709, 2716]	3313	[3306, 3321]	2199	[2198, 2200]
avg. utilization [%]	72	[72, 72]	81	[81, 81]	89	[89, 89]	73	[72, 73]
avg. weekly overtime [min]	4	[4, 4]	15	[15, 16]	58	[57, 60]	4	[4, 4]
avg. # rejected walk-ins	17	[16, 18]	91	[88, 93]	473	[463, 482]	17	[16, 18]
avg. access time [d]	2.5	[2.5, 2.5]	3.2	[3.2, 3.2]	4.1	[4.1, 4.2]	2.5	[2.5, 2.5]
avg. access time regular [d]	1.5	[1.5, 1.5]	1.6	[1.6, 1.6]	1.9	[1.8, 1.9]	1.5	[1.5, 1.5]
avg. access distance [km]	4.8	[4.8, 4.8]	6	[6.0, 6.0]	7.2	[7.2, 7.3]	4.8	[4.8, 4.8]
avg. waiting time appt. [min]	2	[2, 2]	2	[2, 2]	2	[2, 2]	2	[2, 2]
avg. waiting time walk-in [min]	40	[40, 40]	52	[52, 52]	67	[67, 67]	40	[40, 41]
on-time appts. [%]	61	[61, 61]	59	[59, 59]	59	[58, 59]	61	[61, 61]
# acute illnesses	136405	[136234, 136575]	136553	[136386, 136720]	136344	[136227, 136460]	137731	[137554, 137908]
# chronic patients	10662	–	10662	–	10662	–	10776	–
total PCP capacity [h]	32617	–	26455	–	22139	–	32617	–
	Aging Patients Medium-term Shift		Combined Effects Short-term Shift		Combined Effects Medium-term Shift
	Mean	95 %-CI	Mean	95 %-CI	Mean	95 %-CI
avg. # treatments	10325	[10313, 10337]	12579	[12567, 12591]	15368	[15356, 15380]
avg. # walk-ins	4934	[4921, 4946]	7102	[7090, 7114]	9723	[9711, 9734]
avg. # acute appts.	3160	[3158, 3162]	2741	[2736, 2745]	2258	[2251, 2264]
avg. # regular appts.	2231	[2229, 2233]	2736	[2732, 2741]	3388	[3381, 3395]
avg. utilization [%]	73	[73, 73]	81	[81, 81]	90	[90, 90]
avg. weekly overtime [min]	4	[4, 5]	17	[16, 18]	67	[66, 68]
avg. # rejected walk-ins	20	[19, 21]	100	[97, 103]	560	[548, 572]
avg. access time [d]	2.6	[2.6, 2.6]	3.3	[3.3, 3.3]	4.4	[4.4, 4.4]
avg. access time regular [d]	1.5	[1.5, 1.5]	1.7	[1.6, 1.7]	2	[1.9, 2.0]
avg. access distance [km]	4.9	[4.9, 4.9]	6.1	[6.1, 6.1]	7.3	[7.3, 7.3]
avg. waiting time appt. [min]	2	[2, 2]	2	[2, 2]	2	[2, 2]
avg. waiting time walk-in [min]	41	[40, 41]	53	[53, 53]	68	[68, 68]
on-time appts. [%]	61	[61, 61]	59	[59, 59]	59	[58, 59]
# acute illnesses	138696	[138478, 138913]	137886	[137686, 138086]	138595	[138468, 138722]
# chronic patients	10931	–	10776	–	10931	–
total PCP capacity [h]	32617	–	26455	–	22139	–

Patients in the baseline scenario are expected to travel almost 5 km to visit a physician and have to wait an average 2.5 days for their appointments. With regard to waiting times, we obtain an average expected waiting time of 2 minutes for patients with appointment and 40 minutes for walk-in patients. In comparison to the average waiting times observed when recording our service time dataset (4 minutes with appointment, 15 minutes without appointment), simulated waiting times are strikingly unfavorable for walk-in patients. We take this to suggest that physicians in the real world avoid excessive waiting times for walk-in patients through more sophisticated treatment strategies like [62].

Scenario 1: Decline in PCPs

Scenario 1 models a decline in the number of PCPs for a short- and a medium-term shift in time. To that end, we exclude those PCPs from the baseline PCP population that reached the statutory retirement age of 65 by this point. Specifically, we consider the year 2023, by which 4 out of 20 PCPs will reach the statutory retirement age, and the year 2027, by which 7 out of 20 PCPs will reach the statutory retirement age. Assuming that none of the excluded physicians are replaced by a successor, we obtain a decimated population of primary care physicians for the short-term and for the medium-term shift. By replacing the physician population in our baseline scenario by and , respectively, we obtain two variants for Scenario 1; see Table 14.

Table 14

Populations in each simulation scenario variant

	Scenario 1		Scenario 2		Scenario 3
	s	m	s	m	s	m
patients	P	P	P^s	P^m	P^s	P^m
physicians	G^s	G^m	G	G	G^s	G^m

s = short-term shift, m = medium-term shift

The simulation results for Scenario 1 in Table 13 show a severe deterioration of all patient and physician indicators compared to the baseline scenario. The physicians’ expected workload measured through the average number of treatments increases by 23 % for the short-term and 49 % for the medium-term shift. Due to the increased scarcity of appointments, more and more patients are forced to visit physicians as walk-in patients (56 % for short-term and 63 % for medium-term shift). The average weekly overtime for physicians increases by 11 minutes for the short-term and 54 minutes for the medium-term shift. On average, patients wait 30 % longer for their appointments in the short-term and even 67 % longer in the medium-term shift scenario variant. Similar increases can be observed for the patients’ average access distance, which increases by 26 % for the short-term and by 51 % for the medium-term shift. The average waiting time for patients with appointment is almost unaffected by the decline in the number of physicians, which can be explained by the strict prioritization in PFCFS. The average waiting time for walk-in patients increases by 30 % for the short-term and 66 % for the medium-term shift.

Scenario 2: Aging patients

Scenario 2 models the ongoing aging of the patient population for a short- and medium-term shift in time. For this purpose, we adjust the discrete probability distribution determining the patients’ age classes to generate two new patient populations. More precisely, we use current population projections [34] for the years 2025 and 2030 to obtain the two adjusted discrete probability distributions in Table 15. Using these distributions, we generate the aged patient population for the short-term and for the medium-term shift. By replacing the patient population in our baseline scenario by and , respectively, we obtain two scenario variants for Scenario 2; compare Table 14.

Table 15

Age class distributions for aged patient population

	16-24	25-65	> 65
short-term shift	0.1051	0.6283	0.2666
medium-term shift	0.1025	0.6033	0.2942

The simulation results for Scenario 2 (Table 13) paint a similar picture as in Scenario 1, i.e., the majority of patient and physician indicators deteriorate, albeit far less severely. The average number of treatments per physician increases by 1 % for the short-term and by 2 % medium-term shift. In contrast to Scenario 1, additional treatments distribute more evenly across appointment and walk-in patients and thus the expected ratio of walk-in patients remains almost unchanged. Physicians seem to accommodate the additional treatments within their regular opening hours, as their average overtime is almost unaffected. Due to the increased demand, patients wait on average 2 % longer for their appointments in the short-term and 5 % longer in the medium-term shift variant. Moreover, they are willing to accept 1 % longer average access distances in both scenario variants. Patient waiting times with appointment are unaffected by the increased demand. The average waiting times of walk-in patients in the short-term shift remain almost unchanged, while they increase by 1 % for the medium-term shift.

Scenario 3: Combined effects

Scenario 3 models a combined decline in the number of PCPs and aging of the patient population for a short- and medium-term shift. By replacing both populations from the baseline scenario with the adjusted patient and physician populations from Scenarios 1 and 2, we obtain two variants for Scenario 3; compare Table 14.

Analyzing the simulation results in Table 13 for Scenario 3 confirms that the combined effects lead to the greatest deterioration of indicators. The effect of the combined changes compared to the combination of effects from Scenarios 1 and 2 varies between indicators: For the average number of treatments and the ratio of walk-in patients, the effects of the combined changes correspond to the sum of the effects for the individual changes, e.g., a 24 % increase in the average number of treatments in short-term shift variant of Scenario 3 versus a 23 % and 1 % increase in the respective variants of Scenarios 1 and 2. For the physicians’ average overtime, a combined consideration of both changes has an amplifying effect. For example, in the medium-term shift variants of Scenarios 1 and 2 the average overtime increases by 54 and 0 minutes, respectively, while the combined changes in Scenario 3 lead to an increase of 63 minutes. Similar amplifying effects can be observed for the patients’ average access time, access distance, and walk-in waiting time.

Sensitivity analysis

SiM-Care’s complexity and large number of input parameters make the model versatile, but also create the risks of instability and high sensitivity towards small changes in the input values. To ensure that such undesired behaviors do not invalidate the outputs of simulation experiments, we perform a sensitivity analysis.

The sensitivity analysis considers the baseline scenario in the setup of the case study, i.e., 20 independent runs modeling one year preceded by a warm-up of 30 years. As the complexity of the model prohibits a full sensitivity analysis within the scope of this paper, we demonstrate the process for those input parameters from Table 8 that are least anchored in empirical data. Specifically, we study the model’s sensitivity towards the patients’ health condition c ∈ [0,1] and the age classes’ deviation from the illness duration , deviation from the willingness to wait , and probability to cancel an appointment after recovery p ∈ [0,1].

We vary each input parameter relative to its original value between ± 20 % in increments of 1 %. We analyze the resulting impact on the PCPs’ average utilization and number of rejected walk-in patients. Figures 7 and 8 show the resulting average values and 95 % confidence intervals for both performance indicators.

Fig. 7

Mean average utilization and corresponding 95 % exact confidence intervals

Fig. 8

Mean average number of rejected walk-in patients and corresponding 95 % exact confidence intervals

These results do not indicate that the complexity of SiM-Care causes instabilities. Instead, both performance indicators behave as expected towards variations of the input parameters, e.g., increasing the patients’ deviation from the willingness to wait causes patients to wait longer for appointments, which in turn reduces treatments and thus decreases utilization and rejected walk-ins.

The shape of the relationship between changes in input and output reveals two phenomena. For the average utilization shown in Fig. 7, the relationships is almost linear with small confidence intervals. Moreover, the slopes within these relationships are relatively flat, which indicates a low sensitivity towards small changes in the input values. For the average number of rejected walk-in patients (Fig. 8), the relationship is non-linear with large confidence intervals. The sensitivity and the variation in the number of rejected walk-in patients furthermore increase with the system’s utilization, which seems intuitive. Still, for small changes (± 5 %) of the considered input values, the sensitivities are not extreme given the width of the confidence intervals.

Conclusion and future work

SiM-Care aims to support decision makers in planning, analyzing, and adapting primary care systems. It produces meaningful performance indicators that enable a far more detailed assessment of primary care systems compared to the current approaches based on patient-physician ratios. SiM-Care can predict and quantify the influence of policy decisions and changes in the systems population, e.g., an aging of the population or a decline in the number of PCPs as illustrated in Section 4. Because SiM-Care can particularly model multiple simultaneous system changes, it enables the analysis of combined effects. As all components of a simulation scenario can be easily adjusted, this opens up a broad field of potential applications ranging from physicians’ location planning to the evaluation of specific PCP strategies, e.g., in the field of appointment scheduling. However, as pointed out in [68], we must note that the resulting predictions can never be completely accurate and are therefore intended to scan the horizon and “inform policy makers about future problems and development”. Finally, the modular design of SiM-Care and its planned availability as open source code allows for easy model extensions, e.g., to model prospective new supply concepts such as mobile medical units or telemedicine. Moreover, one could also change the setup of simulation experiments, e.g., by using the batch means method instead of independent runs to trade robustness for computational efficiency; compare [42].

The greatest challenge to using SiM-Care in practice is the complex and time consuming task of generating and validating the input scenarios. As SiM-Care models each agent individually, it requires detailed empirical data, which has to be obtained from various parties or may be unavailable. Additionally, even the estimation of parameters from empirical data is generally a challenge on its own that we neglected in this work. The current version of the system tailors some model components such the service time distributions to the German system. These components might have to be adjusted when using SSiM-Care to analyze, e.g., a primary care system in the United States. Such model changes potentially change the model’s behavior and thus require a new validation process to ensure that insights derived from SiM-Care are viable.

To help overcoming this challenge, we exemplified the scenario generation and validation process for a real-world primary care system. Particularly, we detailed the generation process of all simulation entities and provided available empirical data sources. Although data availability may vary for other primary care systems, this may hint at where the required empirical data can be obtained. We validated our simulation scenario by comparing its output to available empirical data. To show internal validity, we captured the model variability through confidence intervals. However, we need to stress that additional validation should be performed before actual policy decisions are derived from the presented case study. Such validation measures should particularly include an expert validation which was out of scope for the purpose of this study.

Conceptual modeling needs to be judged by validity, feasibility, utility, and credibility [54, 55]. So far, we discussed validity at length and demonstrated a degree of feasibility and utility in our case study. However, trust can not come from validation alone, but must also rely on the credibility of modelers and stakeholders [30]. Thus, we conclude by turning to the subject of credibility.

Credibility can only be given from the client’s perspective and requires stakeholder involvement and case studies, as emphasized in [63]. For a stakeholder definition and consideration of diverse stakeholder groups, we rely on [12]. Based on flow charts and early prototypes of SiM-Care, we discussed the conceptual model and experimental designs with professionals (PCPs), health care providers, and a municipal government agency. However, up to the time of this writing, the project neglected, i.a., patient interest groups. Furthermore, we did not implement a systematic framework for stakeholder involvement as proposed, e.g., in [44, 63]. One reason for this was that SiM-Care resulted from a project aiming to quantify the effects of new mathematical planning approaches, e.g., to schedule patient appointments. Therefore, the related project constituted what [12] term “curiosity-driven academic research” rather than practice-based research.

Some arguments on generality and agility speak against a strictly sequential development involving stakeholders such as the PartiSim framework introduced in [63]: As [63] point out, the health care domain is driven by “many decision-makers” and “busy stakeholders”. At the same time, SiM-Care aims to support applications from “horizon scanning” (cf. [68]) to evaluate the impact of changing specific submodels, e.g., a PCP’s approach to scheduling patients. We consider it as highly challenging to elicit views and involvement from a sufficiently broad selection of decision-makers and stakeholders to create such a generally applicable simulation model. Even more importantly, an agile rather than sequential view of developing simulation systems may reduce the risk of project failure; compare [58].

Agile development does not strictly adhere to a process that lays down the conceptual model before the coding phase. Instead, it does allow for adjusting the model even in the experimentation phase. To use an agile approach to development, we suggest a systematic stakeholder involvement in the post-model-coding-stages as exemplified in [43]. In other words, we recommend emphasizing stakeholder involvement more strongly when designing case studies that rely on SiM-Care.

Finally, to implement more stakeholder-driven research means working to overcome the challenges of stakeholder “identification, contact, and elicitation” [12]. Somewhat paradoxically, in doing that, having implemented and published research with the current version of SiM-Care might help engagement. This did appear to be the case when presenting SiM-Care to the general public at a university outreach event in 2019. During this event, participants could generate appointment scheduling rules and discuss the effects of their ideas as computed by the simulation. As suggested in [43], a pick-and-mix approach might support such a view and enable facilitated simulation modeling, coding, and experimentation.

Future work on SiM-Care will include further effort towards model validation and calibration as well as the implementation model extensions. Currently, illness distributions are considered as being static by SiM-Care. By modeling dynamic illness distributions, we can incorporate seasonality or the patients’ previous history of illnesses. In the current model, the duration of an illnesses is independent of the actual treatment. Interestingly, the results are convincing even without this causal link. In the future, we want to compare whether implementing this link in the conceptual model significantly affects our findings. A similar comparison shall investigate the influence of no-show patients, who introduce unexpected idle time into the physicians’ schedules. While the sole integration of patient non-attendance into SiM-Care is straightforward, the actual difficulty lies in the need for empirical no-show probabilities as well as the necessity to decide how no-show patients continue their course of treatment. Yet other possible model extensions include: Illness specific appointments, intentional physicians’ breaks, implementation of additional patient attributes such as gender, and mobile patient agents that move between different locations, e.g., their home and work. Finally, we are currently preparing the open source release of our model implementation that comes with a graphical user interface such that SiM-Care can be easily accessed, studied, and adapted to the individual requirements of all modelers.

Table 16

Adaptation of patient ratings and

Positive Event	Adjustment
Waiting time <7min	+ 5
Successful arrangement of appointment	+ 4
Successful treatment as walk-in	+ 3ζ
Successful treatment with appointment	+ 2ζ
Negative Event	Adjustment
Waiting time >30min	− 10
No appointment within willingness available	− ω
Rejected as walk-in	− 10
Rejected with appointment	− 20

Parameter describes patient’s willingness to wait and ζ ∈ [0, 1] the physician’s consultation speed