Adoption costs of new vaccines - A Stackelberg dynamic game with risk-perception transition states.

Vaccination has become an integral part of public health, since an increase in overall vaccination in a given population contributes to a decline in infectious diseases and mortality. Vaccination also contributes to a lower rate of infection even for nonvaccinators due to herd immunity ((Brisson and Edmunds, 2002)). In this work we model human decision-making (with respect to a vaccination program in a single-payer health care provider country) using a leader-follower game framework. We then extend our model to a discrete dynamic game, where time passing is modelled by risk perception changes among population groups considering whether or not to vaccinate. The risk perception changes are encapsulated by probability transition matrices. We assume that the single-payer provider has a given fixed budget which would not be sufficient to cover 100% of a new vaccine for the entire population. To increase the potential coverage, we propose the introduction of a partial vaccine adoption policy, whereby an individual would pay a portion of the vaccine price and the single payer would support the rest for the entire population. We show how this policy, together with changes in risk perceptions regarding vaccination, impact the strategic decisions of individuals in each group, the policy cost under budgetary constraints and, ultimately, how it impacts the overall uptake of the vaccine in the entire population.

Introduction

Vaccines stimulate the immune system to build a defense against infectious diseases ((World Health Organization Report, 2010)). Even though vaccines are proven to be effective (smallpox and polio are eradicated, and vaccination significantly lessened the number of cases of measles, diphtheria, pertussis, etc. (Asim, Malik, Yousaf, Gillani, & Habib, 2012)), personal beliefs about them in populations at large vary greatly, based on an array of nonmedical factors, such as education level, economic status, religious beliefs, etc. These factors are incorporated in population health and epidemiological models to draw conclusions about individuals' possible vaccinating decisions ((Helbing, 1992, pp. 330–348)). In previous works (see (Cojocaru & Bauch, 2007; Cojocaru, Bauch, & Johnston, 2007) and the references therein), the authors used game theory to study the dynamics of potential vaccinating decisions in a population consisting of distinct groups of parents, where each group has different perceptions of vaccine and disease risks. These models discussed the cumulative effect of the groups' decisions on the uptake of vaccine.

The Nash equilibrium concept provides a resolution to noncooperative nonzero-sum games when no single player's decision weighs more than any others'. There is a vast literature on Nash games to date, so we limit ourselves to a few fundamental references and the references therein (Basar & Olsder, 1998; Ichiishi, 1983; Nash, 1950; Von Neumann & Morgenstern, 1945). There are noncooperative decision problems where one of the players can enforce their strategy on the other player(s), and for such decision problems, one has to introduce a different equilibrium concept known as a Stackelberg1 equilibrium (Basar & Srikant, 2002; von Stackelberg, 2011, p. 134). Examples of Stackelberg games are found in a vast number of references (see for instance (Basar & Srikant, 2002), (Gibbons, 1992), (Fudenberg & Tirole, 1991) and references therein).

A Stackelberg game has a distinct player (usually called the “leader”) who can anticipate the actions of the others (called “followers”) and use this knowledge in selecting their optimal strategy (Zhi-Quan, Pang, & Ralph, 1996). Here the followers' strategies depend on the particular strategy of the leader. In general, the followers behave according to a Nash non-cooperative principle. The Stackelberg game is a sequential game, where the leader chooses an action before the others choose theirs. Importantly, the followers must have information about the leader's choice, otherwise the difference in time would have no strategic effect. There are some variations of this game. If all players are followers, then it is called the Cournot game where decisions are simultaneous. If all players in this game believe themselves to be the leader, then it will become a leader-leader game, and it will lead to a Stackelberg disequilibrium.

The Stackelberg game has been used in the literature for modelling vaccine designs (Swetasudha & Vorobeychik, 2015). In (Shen, Sobzcyk, Buchanan, Wu, & Duggan-Goldstein, 2011, pp. 12–18), it is shown that the gap between the immunization and its affordability should be reduced in order to increase vaccine coverage in a population. In (Mylene, Haines, & Palmer, 2007), authors discussed the impact of conditional cash transfer programs on immunization coverage in low- and middle-income countries.

In this paper we investigate how a copay under a single payer health provider (such as Canada, U.K. or Australia) may influence the likelihood of vaccination in population groups. The provider is contemplating the introduction of an immunization program for a hypothetical new vaccine. In countries where a single-payer health care system is present, new vaccines are considered to be adopted under the single-payer health insurance program and adoption is understood to mean complete cost coverage for all individuals under the insurance program. The decision whether or not to adopt depends on many factors such as the expected cost of the program, and its expected efficacy and coverage (Erickson, De Wals, & Farand, 2005). In our work here we explore the possible benefits when single-payer adopts and funds a portion of the cost of a new vaccine. In this scenario, individuals who wish to get vaccinated would have to pay some fraction of the treatment price out of pocket. The single-payer provider acts as the “leader” in our model, and wishes to minimize the difference (gap) between the likely overall coverage under a given budget, and the expected overall coverage in the population. The latter expected coverage is calculated based on the probability of vaccinating of various population groups. The population groups, differing by perceptions of vaccine and infection risks and costs, act as the “followers”. Individuals have direct knowledge of the program being introduced by the leader, and take this into consideration in their decision-making, however our framework does not account for the decisions made by health care provider(s).

The structure of the paper is as follows: in Section 2 we introduce our leader-followers game model for the single-payer health provider and a partially adopted vaccination program. Section 3 expands our model to incorporate time evolution of individuals groups' perceptions of risk, both towards the vaccine and towards the consequences of getting the infection. We study the effects that changes in risk perceptions together with the partial vaccine adoption policy have on the vaccination coverage. We close with conclusions and some remarks on future work.

Model set up and background

In (Cojocaru & Bauch, 2007; Cojocaru et al., 2007), the authors presented a Nash vaccinating game played by cohorts of parents who consider whether or not to vaccinate their offspring against pediatric diseases such as measles, mumps, rubella, polio, etc. The game proposed is played among a finite number of groups of parents () where parents in a group are considered to share common perceptions of the risk of vaccinating their offspring, as well as the risk of not vaccinating. The game was further extended into a generalized Nash game in (Cojocaru, 2008) and placed in a dynamic context in (Cojocaru & Greenhalgh, 2012). None of the works, however, accounted for the other decision maker acting in some countries (for instance Canada, U.K.), namely a single-payer health program. Below we present a mathematical model that incorporates the perspective of the single-payer health provider, along with a model of population groups contemplating not only the perceived risks of vaccinating, but also sensitivity to the cost of vaccinating and perceived risks of getting sick in the case of not vaccinating.

Nash and Stackelberg games

In general, a multiplayer Nash game involves a finite number of players, denoted here by . A generic player is thought to have a strategy set , with strategy vectors , and a payoff function , where we denote by . A Nash equilibrium of a multiplayer game is defined as follows:

Assume each player is rational and wants to maximize their payoff function . Then a Nash equilibrium is a vector which satisfies the inequalities:where .

In general, the definition of a Stackelberg game can be given as follows (we assume k followers as above, involved in a noncooperative Nash game):

Let be the objective function of the Leader, usually with good continuity and convexity properties (see for instance (Leyffer & Munson, 2010)) and a closed, convex set as the Leader's strategy set. The Leader's problem is then an optimization problem, such as:and where given a fixed , is a Nash equilibrium of the followers’ problem:

Followers game

Let be the number of population groups, which are the players (followers) in our model. Each group represents a fixed proportion in the population (thus ). The population groups are differentiated by their risk perception towards the adverse effects of the vaccine, , and risk perception toward significant adverse outcomes upon infection, , but people in one group are thought to share the same risk perceptions. By we denote the perceived probability of becoming infected given that a proportion p of the population is vaccinated, considered to be the same in all groups. The overall perceived probability of experiencing significant adverse outcomes because of not vaccinating is thus .

In order to find a mathematical expression for one could take a theoretical approach (as in (Bauch & Earn, 2004)). However, individuals do not have perfect knowledge of their probability of being infected. Here, we assume for ease of analysis that is a decreasing function of p given byi.e., disease prevalence is a function of how many individuals have been vaccinated, and a greater perceived coverage in the population means a reduced perceived infection risk for susceptible individuals. In other similar models assumptions on the functional form of are different (see for instance (Cojocaru & Bauch, 2007) where they were distinct among groups: , or (Bauch, 2005) where the functional form involves linearity).

In general, each group has two pure strategies (0 = not vaccinate and 1 = vaccinate), while their mixed strategy is given by their probability of vaccinating, denoted by . In the notation of Definition 2.1, and for all .

The vaccine coverage in the population is considered to be (excluding time lags between vaccination and uptake of the vaccine) . Each group has an expected utility of vaccinating, defined as:Let us now scale the utility in each group by introducing the to be the relative perceived risk of vaccination versus getting infected in group i, thus leading us to consider each group's utility to be of the form:

We now let be the average sensitivity of an individual in group i to supporting the full cost of the vaccine out of pocket. Here a 0-sensitivity means the cost of the treatment has no influence on the individual's decision to vaccinate, while a denotes an individual for whom the out of pocket cost is the main deterrent in deciding to not vaccinate. In this case, the utility of a group becomes:and each player i is solving the following problem:where .

Leader's game

In this paper we assume the single-payer program funds a portion of a new vaccine cost from public sources, while individuals have to pay some fraction of the price per treatment, which is typically called a copay. Let us assume cost per treatment 2 is denoted by and that the copay is a fraction of the cost of the new vaccine; for simplicity, we denote it by . The decision to cover of the cost is motivated by a desire of the single-payer provider to better use an assumed fixed budget B allocated for the adoption of this vaccine.

Assume the budget B is used to cover the population under consideration (of size N) with a cost per treatment of , thus the number of treatments that can be acquired for the population will be . This means that the coverage level for this vaccine under budget B and copay will be

We also have that is the Nash equilibrium coverage depending on the Nash equilibrium strategies of followers: .

Then the Leader (the single-payer health provider) wishes to minimize the gap between , and the expected Nash equilibrium vaccine coverage of the followers, . In this case, the Leader's problem is:where solves the followers problem:

Note that we implicitly have for any value of δ. In order to solve the Leader's problem (2) we start by solving the followers' problem (3) for all values of δ, where now the expected utilities depend on the copay δ and the individuals' perception of the vaccine risks and costs . Here we assume that a reduction in out-of-pocket cost per treatment induces a reduction in sensitivity to the treatment's perceived cost .

We know (see (Cojocaru et al., 2007)) that Nash equilibrium vaccination strategies exist and are unique for the followers' game (3) for any fixed values of δ and ε. This means that the function , evaluated at the equilibrium strategies will attain its optimal value at a for which . Thus an optimal solution to our model would be a with with is covered by the single-payer budget whenever each individual copays .

Copay size and equilibrium vaccine coverage

In this subsection we show the implementation of our game above in a population with three groups, differ by their relative risk values and their relative vaccine cost sensitivities.

In general, we pick a value of for the reference behaviour, representing a situation where there is a significant level of trust in vaccination in group 1, and the disease is thought to be 100 times more dangerous than becoming vaccinated (the actual value is much higher for most vaccine-preventable infections, but is a perceived relative risk, not an actual relative risk). In essence, the relation between the relative risk and in a group comes from an assumption we make for the model, namely, that lower values of the perceived probability of infection in a group correspond in general to larger values of the perceived relative risk: individuals who think having the disease is less dangerous may also believe that their risk of becoming infected is lower. In the other groups with variable behaviour, we assume that and . In this context we call group 1 vaccine inclined, group 2 undecided and group 3 vaccine adverse, (). We keep these values throughout the examples in the Sections below.

We analyze first the impact of reducing the vaccine cost in a population where , i.e., the group is a majority; this is clearly our assumption, albeit an educated one. In countries such as Canada, the public vaccination schedules are well-known to the population and uptake of vaccines for instance for pediatric infections is generally between 85% and 92%, which is to say that the population does not contain large adverse majorities against vaccines in general. Although refusal to vaccines is known to exist (Smith et al., 2011), it is reasonable to assume that a lot of the population falls under the group when presented with a new treatment. We examine what happens when the group proportions vary in the next section.

In our examples we assume a treatment (vaccine) against an infectious disease of strength comparable to measles, thus we choose , in formula (1) above, as detailed in (Cojocaru et al., 2007).

We further assume that the single payer has a fixed budget B so that the coverage offered under this B is only of the population (i.e., B cannot offer a complete adoption of the vaccine for the entire population with a copay of ). Last but not least, we consider the following values . We report our results in Fig. 1 next.

Fig. 1

Left panel: Overall coverage versus copay δ. Right panel: This the Leader's problem, where its objective function is minimized for a copay value of of the cost per treatment, leading to a coverage of .

We see that we find an optimal value of which leads to both minimizing the function for the leader, as well as to significantly increasing the expected vaccination coverage in the population, from a for (i.e., when individuals would have to support the entire cost of the treatment) to (when they would support only 5% of the cost).

Time dependent game with varying risk perceptions

In this section we consider the fact that individuals' perceptions of risks towards the vaccine may change over time due to several factors that we expand upon below. We consider a finite number of discrete time steps, generally denoted by the time division of a finite time interval , and we continue to assume that the population is divided into three groups with risk perceptions discussed in Subsection 2.4 above.

Each average individual of a group has a probability of changing their views on their relative risk perception from stage to stage . This assumption means that an individual has a probability of switching from group to group in one time step; let us denote this transition probability . Given probabilities we build a square matrix , such that is right stochastic with each row summing up to 1, denoted in general by:

Our model now is formulated as follows:where solves the problem:

To model the expected vaccination coverage under copay when the population's risk perceptions change, we follow the steps below:

STEP 0:

Start with the initial distribution of people in each group for the initial time: , ;

Solve the followers game and obtain the equilibrium strategies for each δ value; Then find the value(s) which solves the leader problem on .

Apply the transition matrix to compute the distribution of people in each group at .

STEP m: Repeat (1), (2) & (3) at . If , repeat (1) and (2) and STOP.

Policy-independent transition matrix

Changes in risk perceptions towards a vaccine could happen over time and independent of this policy being introduced, for instance in cases of vaccine scares, of public health education campaigns, of previous infectious outbreaks in the population and/or of changes in the socio-politic environment of the population (Opiski, 2012). To account for these possibilities, and to study the effect these changes may have on the long-term health of the policy, we introduce a specific form of the transition matrix T, described below.

Let us denote by the probabilities of an individual in group 2 (an ) to switch their perception of vaccine risks towards vaccine () and respectively towards vaccine (). We assume for simplicity that both and individuals are unchanged in their perceived risks estimates over time. Then the matrix becomes:

We show next a first scenario, where we vary the probabilities of changing risk assessment groups: , over a time window , where the time unit is taken to be 1 year. We note that if both , then no individual in any group changes their risk perceptions over time; if however , then the group is non-existent, thus we would be in the case where the population is either vaccine inclined of vaccine adverse. We run this scenario considering the values of group sizes in Subsection 2.4 as the initial values of ε at .

In our simulations and discussions below we also plot the functionto clearly show the sign difference between the possible coverage given by a budget B and the expected Nash equilibrium coverage of followers. We present our results of this simulation in Fig. 2 below.

Fig. 2

Overall coverage as a function of the probability of changing risk perceptions between and groups, and time . Lower panel shows the change in population group sizes.

We see that (upper right panel) the optimal values of decrease as the values of increase. This means that as the population becomes more polarized (lower panel), the cost burden of the single payer (i.e., ) has to increase. However, the polarization between and also results, over the 5 years, in a decline of the overall coverage (upper left panel). Thus we conclude here that having a majority of results in the best coverage, as they may be swayed to vaccinate by being offered a shared cost policy. However, relying on such a policy in the face of big opinion changes is not enough to motivate vaccinating behaviour.

Coverage-dependent transition matrix

Assume now that is the coverage for the control of disease in the given population and is the Nash equilibrium coverage depending on the Nash equilibrium strategies . Assuming for all , we now define the transition matrix as:

An individual in the Undecided group may change their relative risk perception and cross over to the Inclined group; this accounts for a change in the group size given by , where c is the sensitivity of the Undecided group to the difference in coverage. Similarly, an individual in the Adverse group may change their risk perception and cross over to the group; this accounts for a change in group sizes equal to , where is the sensitivity parameter for the Adverse group. We assume that the group is not influenced by the year-to-year coverage changes.

We run simulations based on varying , and . At we have the same conditions as in Subsection 2.4, with .3 We report our results in Fig. 3 below. We note first that each figure depicts a 4-dimensional representation of the expected Nash coverage (upper figure), respectively of the leader's gap function . While the first two parameters are represented in a standard manner (δ on x-axis, respectively t on y-axis), the other two parameters are represented by color variation (for ) and color intensity (dark to light for c).

Fig. 3

Overall coverage as a function of the copay , time and sensitivities c and .

Here we see that the copay δ has the smallest positive effect for all values of , and (upper and lower panels, blue, dark to light), i.e. the coverage increases little with δ and t when the group's opinion is unchanged over time. When this group's opinion is influenced over time (i.e., for ) we see that there is an increasing trend in coverage levels as . The biggest time effect is displayed for (i.e., the copay to the individual is 0), while time's effect is less remarkable for .

In the lower panel, we can see the values of the leader's true objective function (i.e., without the absolute value imposed). In general we see that for there are cases where the single-payer budget is insufficient for complete coverage, even for larger values of δ. However, as time evolves, the budget is balancing to cover the expected vaccination uptakes. To see this, look for instance at the values and . At there is a budgetary shortage although the individual is asked to support of the cost. However, as time progresses, by , and the value of the Gap function is 0, thus for all subsequent years, there will be a budgetary surplus, if is maintained as individual copay.

The second panel in Fig. 3 gives further insights in what a time-dependent implementation of this policy might look like. Let us for instance assume that in year 2 we have achieved the Leader's goal to have the budget needed to support the expected vaccine uptake. Then, in the following years, given that a surplus might form, the leader can seek to further improve their policy by reducing δ from 0.3 to perhaps 0.2. Once budgetary balance is achieved for , then they could further reduce it in subsequent years. Our model then can be used as a mechanism to evaluate how long it might take (and for what population types) to reach adoption of a treatment with .

Conclusions and future work

In this paper we introduce a model of a partial adoption of a new vaccine in countries with a single-payer health provider, such as Canada, the U.K., Australia etc. As the prices of new treatments tend to increase, the costs of public programs increases as well. In order to help keep costs under control, yet still promote and help increase the overall treatment coverage in a population, we propose that the price of a new treatment, such as a vaccine, be split between the single-payer provider and individuals in the population. We are thus able to determine proportions of price per dose needed to be supported by the single payer program so that both the expected vaccination coverage increases in the population, as well as costs are maintained within fixed budgets.

We show that a populations' perception of the risks of a new treatment can interfere with the policy. To see this we introduced a time-dependent element in the model (via a discrete transition matrix) which reflects the changes of views on the perceived risks in the population per year. Such changes could be either: 1) independent of the policy being introduced, as in cases of vaccine scares, of public health education campaigns, of previous infectious outbreaks in the population and/or of changes in the socio-politic environment of the population; 2) or changes could be dependent on each individuals' estimation of the level of vaccine coverage in the population, i.e., consequently their estimation of how protected they may be by herd immunity.

In the former case, if the population is highly polarized, with most individuals being adverse or inclined toward the vaccine, the policy introduction will not necessarily induce an increase in the expected coverage. This is because the adverse individuals are not simply persuaded to vaccinate just by lowering the monetary cost of vaccine; their adverse perception toward a vaccine is likely a combination of other factors, not easily changed by a change in cost. On the other hand, if the population has a large majority of undecided individuals (i.e., individuals who are not intrinsically pro or against a vaccine, but who most of the time fail to vaccinate due to economic and personal costs), then they may be persuaded to consider vaccination by lowering the cost per dose.

In the latter case, we see that the introduction of the partial adoption policy in populations which are more sensitive to the vaccine coverage year by year achieves very high overall vaccine coverage levels. In contrast, while the introduction of the policy helps in general, it is less effective in populations which are generally less sensitive to overall yearly coverage.

A major assumption of the work is that the willingness to pay for the vaccine is additive to relative risk. Our assumption here is directly linked with being able to take advantage of two results: first, the existence and uniqueness of Nash strategies for all δ values, using the proofs in (Cojocaru et al., 2007); second, the result in (Barbagallo & Cojocaru, 2009) showing that the Nash strategies must vary continuously with the parameter δ, thus giving us a solid foundation for the plots of Fig. 1. A nonlinear-type dependency can be explored as well for the willingness to pay. However, in this case care must be taken in re-analyzing the vaccinators' game from existence as well as uniqueness of solutions points of view as the payoffs of each group will change in a nonlinear fashion the above results will not necessarily apply anymore.

In this work we present the theoretical framework to analyze the proposed policy under various parameter sensitivity scenarios. The scope of our analysis was to show that, qualitatively: 1) coverage will be impacted if a copay is introduced; 2) there is a “common ground” spot between a single payer programs wish to expand vaccination and to keep costs under control; 3) time and changes in peoples opinions are very likely to be extra factors in the success or failure of the proposed policy.

In a future exploration of this topic, one may consider using available data for a currently existing vaccine which is not yet adopted in the overall population. The example here could be the herpes zoster vaccine and the targeted population would be all individuals over the age of 50. Our framework could be used to explore a partial adoption policy for all at-risk individuals in the population.