Population-level intervention and information collection in dynamic healthcare policy.

We develop a general framework for optimal health policy design in a dynamic setting. We consider a hypothetical medical intervention for a cohort of patients where one parameter varies across cohorts with imperfectly observable linear dynamics. We seek to identify the optimal time to change the current health intervention policy and the optimal time to collect decision-relevant information. We formulate this problem as a discrete-time, infinite-horizon Markov decision process and we establish structural properties in terms of first and second-order monotonicity. We demonstrate that it is generally optimal to delay information acquisition until an effect on decisions is sufficiently likely. We apply this framework to the evaluation of hepatitis C virus (HCV) screening in the general population determining which birth cohorts to screen for HCV and when to collect information about HCV prevalence.

Introduction

There is currently no guidance for determining the optimal schedule for collecting additional information regarding a decision to invest in a health program or technology [1, 2]. Current practice in the health decision science literature assumes that model parameters are fixed across cohorts and the value of additional information is calculated assuming the information-collection effort is initiated immediately [3-5]. However, in many cases the cost-effectiveness of a health program or technology – and, therefore, the value of additional information about one or more model parameters – may be changing over time because of trends affecting the cohort or the intervention [6]. In these cases, collecting additional information immediately may not be optimal and value-of-information calculations based on static parameter assumptions are likely to be biased. Planning over longer horizons is particularly important in health policy because, once established, clinical practice is difficult to change due to high switching costs (re-training and potentially new capital equipment expenditures), particularly if it appears that the level of service is being reduced [7].

In this paper we apply a stochastic dynamic programming approach to identify both the optimal time to change the current health intervention policy and the optimal time to collect decision-relevant information. We consider a hypothetical medical intervention for a cohort of patients. At each time, a new cohort of patients becomes eligible for the intervention and one parameter varies across the cohorts with imperfectly observable linear dynamics. We assume that the value of the intervention is linear in the dynamic parameter. In general, the (incremental) net monetary benefit of an intervention is linear in parameters with a one-time effect (e.g., the prevalence of a disease at one point in time or the outcome of a one-time screening test). When an effect accrues over time, such as for a reduction in the annual transition rate of a disease complication or death, linearity is often used as an approximation (see, e.g., [8]). At each time, the policy-maker can choose to invest in the medical intervention and/or to purchase sample information about the uncertain dynamic parameter. We demonstrate that information acquisition is best delayed until the signal is sufficiently likely to affect the optimal policy decision.

We apply this framework to the evaluation of hepatitis C virus (HCV) screening. Prior to the development of highly-effective treatments, HCV screening in the general population was not considered cost-effective [9] and universal screening was not recommended [10]. The advent of more effective therapy has changed the value of identifying infected individuals early to initiate treatment [11-15]. Recently released guidance by the Centers for Disease Control and Prevention (CDC) and the US Preventive Services Task Force (USPSTF) recommends one-time HCV screening for all individuals born between 1945 and 1965 [16, 17] although screening individuals born after 1965 may also be cost effective [13-15]. Based on our primary analysis of the National Health and Nutrition Examination Survey (NHANES), in the US general population, HCV prevalence is highest in people born around 1956 and declines thereafter at a rate of approximately 11% per birth year. Since HCV prevalence is decreasing across birth cohorts, HCV screening will only be cost-effective for a limited time or for a limited set of birth cohorts. We apply our model to simultaneously evaluate the optimal HCV-screening and information-acquisition policy.

Specifically, we apply our model to the policy decision of whether or not to perform one-time HCV screening in successive cohorts of healthy 50-year olds, who have not previously been tested for HCV, at a routine preventive health visit. Applying a traditional health economics framework, the policy-maker could decide today how many cohorts will be screened (e.g., each cohort of 50-year olds until those born in 1965 turn 50) or, to inform this decision, the policy-maker may seek additional information to be collected immediately. Our framework differs from the traditional paradigm in that each year the policy-maker makes a decision about whether to continue the one-time HCV screening program (whether or not to screen the new cohort of healthy 50-year olds) and whether to collect information about disease prevalence in this current cohort. If information is never collected, the optimal policy does not differ across frameworks. However, in our framework, the immediate decision is not limited to the decision of when to change policies, but it also includes when to collect information to inform a future change of policy. For example, the (immediately) optimal policy might be to screen each cohort of 50-year olds for the next 6 years and then collect information about HCV prevalence to inform future decision making. Delaying information acquisition until a time that the information is sufficiently likely to affect the decision increases the value of the information. In addition, from a practical perspective, collecting information years before it is likely to influence a policy change wastes immediate resources and, should something occur in the lag-time between the information-acquisition effort and the policy change, implementing the pre-determined policy change may not be optimal.

Related literature and contribution

The relevant literature spans technology adoption, dynamic decisions in healthcare, and the value of information in healthcare.

Technology adoption

In technology-adoption models, a decision-maker considers the adoption of a technology of unknown profitability. Jensen [18] introduced a model in which information about a new technology is costlessly observed and the decision-maker can decide to adopt the new technology at any point in time. McCardle [19] presented a model in which collecting information is associated with a fixed cost; in each period the decision-maker can defer and collect information, or make a final decision to accept or reject the new technology. The optimal policy in each period is characterized by two thresholds: if the expected benefit is above the upper threshold, it is optimal to adopt the technology; if the expected benefit is below the lower threshold, it is optimal to reject the technology; and, if the expected benefit is between these two thresholds the optimal strategy is to gather information. Uncertainty about the technology’s value decreases over time and the two thresholds converge to the cost of adoption. Smith and McCardle [20] provided several meta-results, some of which we use, describing how properties of the value function of a stochastic dynamic program are preserved and propagated through finite-horizon Markov-reward and decision processes. Ulu and Smith [21] extended this work by relaxing the assumption that the decision-maker’s value of the technology can be summarized by the expected benefit, and they use more general monotone-comparative-statics techniques in terms of likelihood orders to generalize the class of signals that are observed prior to making an adoption decision.

Another line of research considered technologies, like ours, with uncertain and changing value. Rosenberg [22] found that expectation of technological improvement may delay a firm’s irreversible technology investments. Bessen [23] calculated the option value of delay for such a problem. Kornish [24] considered the choice between two uncertain technologies where each is subject to a positive network effect and explored the impact of the network effect on the optimal adoption policy. Chambers and Kouvelis [25] formulated a technology-adoption problem incorporating expected learning-curve effects.

Stochastic dynamic programs in healthcare

Sequential decisions under uncertainty are common in healthcare [26, 27]. Most healthcare applications of stochastic dynamic programs have focused on optimizing the timing of interventions for an individual patient: the decision to accept or reject an offered kidney for transplantation [28]; the optimal treatment plan for mild spheroctosis [29]; the optimal surveillance and management of ischemic heart disease [30]; the optimal time to perform a living-donor liver transplant [31, 32]; the optimal time to initiate treatment for HIV [33, 34]; the optimal timing and frequency of HCV testing from the patient perspective [35]; the optimal use of statins in patients with type 2 diabetes [36, 37]; the optimal prostate biopsy referral [38]; and, optimal cancer screening programs [39, 40]. Dynamic programming has also been applied to complex appointment scheduling problems in healthcare, including problems with patients of different clinical types/priority [41, 42]; incorporating patient no-shows [43]; and problems of sequential appointment scheduling with the objective of closely adhering to a prescribed schedule (e.g., sequential chemotherapy appointments [44]) or with the objective of satisfying patient preferences [45, 46]. Fewer examples of application to population-level policy exist. Kornish and Keeney [47] and Özaltın et al. [48] formulated the influenza-strain selection problem in a finite-horizon optimal-stopping framework. Similar to our problem, the influenza-vaccine composition decision is also an optimal-stopping problem with information acquisition; however, it has many unique characteristics that distinguish it from the problem discussed here such as an inventory deadline (finite horizon), a product useful for one season only, and a time-consuming production process. Similar to many of the technology-adoption models discussed above but unlike our framework, in the influenza-vaccine composition models, information is collected in every period in which a final decision has not yet been made.

Health economics and value of information in healthcare

Cost-effectiveness analysis is an economic method for comparing the lifetime discounted costs and health benefits associated with two or more medical interventions or health programs [1, 2]. In theory, the optimal allocation of resources across a portfolio of health interventions is determined by solving a constrained optimization problem with the objective of maximizing health benefits subject to a budget constraint [49-51]. In reality, regional and national health policy bodies routinely compare the incremental cost effectiveness ratio of candidate interventions to a pre-determined threshold intended to approximate the shadow price of the budget to determine if the intervention is ‘cost-effective’ as one component of their policy-making process [52]. Cost-effectiveness analysis is widely used to evaluate general population screening for relatively rare conditions because these programs impose a small cost on everyone who is screened and provide substantive healthcare gains for only a small number of individuals who are identified (or identified earlier than they would be otherwise); calculating the population-level costs and benefits can require detailed natural history models, extensive model calibration and validation, and thorough analysis.

Bayesian decision theory approaches to value-of-information assessment were first introduced by Raiffa and Schlaifer [53]. Weinstein [54] proposed the widespread adoption of value-of-information analysis to research priority setting in health policy and medicine. Hornberger et al. [55], Claxton and Posnett [56], and Claxton [57] introduced a Bayesian approach to identifying the optimal trial sample size and to assessing the value of additional information for technology-adoption assessments. Several approaches to increasing the accuracy of value-of-information calculations continued to relax assumptions implicit in the original formulation (see examples in [58-62]). One common assumption in these studies is that the currently estimated per-person value of information can be applied to individuals in all future cohorts. Recognizing some of the implications of this assumption, Philips et al. [6] discussed the impact that intervention-horizon uncertainty, price changes, and technological development can have on the per-person value of information for future cohorts. They find that delaying information collection may be desirable but do not provide a framework for determining the optimal time to collect information.

Contribution

In this paper, we extend the technology-adoption literature by allowing for a technology that is changing in value over time, for the opportunity to ‘wait’ without collecting information, and for the possibility of optimally determining the collected amount of information in each period. We also incorporate the possibility of an imperfect information-collection technology. We broaden the scope of applications of stochastic dynamic programs in the area of healthcare in an important way – focusing on population policy rather than patient-level decisions. We extend the health decision science literature on value-of-information assessment by developing an approach to identify the optimal information-acquisition policy when model parameters are varying across cohorts. Finally, as an example, we apply our framework to the timely public policy problem of developing a population screening program for HCV. We find that considering the opportunity to collect information in the future leads to a substantially different policy recommendation than current guidelines because it explicitly considers and addresses the parameter uncertainty which is changing over time.

The model

A policy-maker faces recurring decisions for cohorts arriving at times t ∈{0,1,2,…} about whether to invest in a health intervention delivered once per cohort (of size N). By cohort we mean a group of individuals with a certain medical presentation (i.e., individuals with a new diagnosis of cancer) or of a certain status (i.e., individuals who turned 50 this year). The policy-maker’s objective is to maximize net monetary benefit from a societal perspective. The per-person incremental net monetary benefit (INMB) of performing the intervention compared to the status quo is assumed to be affine in an uncertain parameter that varies across the cohorts, with realizations in [0,1] and known dynamics. So , for all t ≥ 0, where 𝜃 is the marginal INMB (with respect to the parameter ) and − γ is the fixed INMB, both measured on a per-person basis.

At the beginning of period t, the policy-maker simultaneously decides whether to invest in a medical intervention for the individuals in cohort t and whether to conduct a study of sample size n over the period to obtain a better estimate of the uncertain parameter . Information, if sought, arrives at the end of the current period and is used, together with the known dynamics of , to inform the intervention decision for future cohorts. Let denote the intervention decision at time t, where d = 0 indicates ‘No intervention’ and d = 1 indicates ‘Intervention.’ The amount of information collected is measured in terms of the sample size ; it is obtained at the cost K(n ), where K(⋅) is an increasing function including a fixed and a variable cost when n > 0 and K(0) = 0. Thus, at each time t the policy-maker implements the control . The per-person current reward for the cohort in period t is

The application in Section 4 features the decision problem of when to stop a once-in-a-lifetime disease-screening program where is the uncertain disease prevalence in the t-th cohort which, in expectation, is geometrically decreasing over time; 𝜃 > 0 denotes the marginal benefit of early diagnosis and treatment for an affected individual, γ > 0 is the per-person cost of the program, and the current-period INMB g is increasing in . Beyond our leading example, the framework can accommodate a wide variety of problems. As formulated, the uncertain parameter needs to lie in a compact interval (which can be mapped via bijection to [0,1]). Thus, the parameter can represent not only a probability but also other model parameters, such as a quality-of-life weight or cost. Additionally, our analysis assumes that the parameter value is decreasing over time. To model a situation where the expectation of the uncertain parameter is increasing (e.g., obesity prevalence), the problem can be formulated as one in which a parameter of opposite definition is decreasing (e.g., prevalence of individuals who are not obese). Our exposition involves an example of when to stop a health intervention. However, the framework can also be used in situations in which the decision-maker wishes to identify the optimal time to initiate a new intervention (e.g., when to adopt a new surgical technique). More broadly, our framework can be applied in settings in which the decision-maker wishes to identify the optimal time to stop the current intervention or initiate a new intervention; the uncertain parameter is geometrically increasing or decreasing across intervention cohorts; and the current-period reward function is linearly increasing or decreasing in the uncertain parameter. Examples are shown in Table 1.

Table 1

Examples of alternative cases in which our framework applies

Case	Uncertain time-varying parameter p~_t=1−q~_t	INMBt of “Intervention” vs. “No Intervention” ∗	Setting ‡	Example †
A	p~_t, decreasing in t	INMBt is increasing in p~_t 𝜃,γ > 0	μ_p(x_0)>γ𝜃 “Intervention” is currently implemented. Optimal stopping problem.	“Intervention”: General population HCV screening at age 50 (Section 4). Period reward function: INMB_t=𝜃p~_t−γ; p~_t, prevalence of HCV in cohort t; 𝜃, marginal benefit of early diagnosis and treatment for an infected individual; γ, fixed cost of screening.
B	p~_t, decreasing in t	INMBt is decreasing in p~_t 𝜃,γ < 0	μ_p(x_0)>γ𝜃 “Intervention” not currently implemented. Optimal starting problem.	“Intervention”: New surgical device vs. old device. Period reward function: INMB^_t=−𝜃^p~_t+γ^; p~_t, probability of an adverse event in device iteration t; 𝜃^, incremental cost of an adverse event; γ^, benefit of surgical intervention without an adverse event. Problem transformation to framework: 𝜃=−𝜃^, γ=−γ^.
C	q~_t, increasing in t	INMBt is decreasing in q~_t (increasing in p~_t) 𝜃,γ > 0	μ_q(x_0)<1−γ𝜃 ⇒μ_p(x_0)>γ𝜃 “Intervention” is currently implemented. Optimal stopping problem.	“Intervention”: Pap smear for early identification of pre-cancerous lesions on the cervix from HPV infection. Period reward function: INMB^_t=−𝜃^q~_t+γ^; q~_t, prevalence of HPV vaccine coverage in cohort t; 𝜃^, difference in benefit of Pap smear in a vaccinated person (compared to an unvaccinated person); γ^, value of Pap smear in an unvaccinated person. Problem transformation to framework: p~_t=1−q~_t, 𝜃=𝜃^, γ=−γ^+𝜃^.
D	q~_t, increasing in t	INMBt is increasing in q~_t (decreasing in p~_t) 𝜃,γ < 0	μ_q(x_0)<1−γ𝜃 ⇒μ_p(x_0)>γ𝜃 “Intervention” not currently implemented. Optimal starting problem.	“Intervention”: Peanut-free spaces regulation (in schools, airplanes, etc.). Period reward function: INMB^_t=𝜃^q~_t−γ^; q~_t, prevalence of severe peanut allergy at time t; 𝜃^, benefit of peanut-free spaces to individuals with peanut allergies; γ^, fixed cost of creating and enforcing peanut-free public spaces. Problem transformation to framework: p~_t=1−q~_t, 𝜃=−𝜃^, γ=γ^−𝜃^.

* When 𝜃 ≤ 0 ≤ γ, the “Intervention” is dominated by the alternative for all realizations of . For γ ≤ 0 ≤ 𝜃, the “Intervention” dominates the alternative for all realizations of

‡ μ (x 0) is the expectation of the initial belief ; μ (x 0) is the expectation of the initial belief ; μ (x 0) = 1 − μ (x 0)

† In each of the examples, the period reward function is linear in the time-varying parameter. That the mean and variance of the time-varying parameter satisfy the dynamics presented in Section 2.3 should be verified empirically for each case

The information-acquisition problem

The policy-maker’s prior belief about at t = 0 is beta-distributed with distribution parameters x 0 = (a 0,b 0). The posterior distribution when a beta-density is updated in a Bayesian manner with information collected using an imperfect information-collection technology is a mixture of beta-densities. Thus, in general, the policy-maker’s prior beliefs about at time t are in where denotes the set of measures which are a mixture of beta-densities. Specifically if , then there exists parameters for all i where , and a set of non-negative weights ω such that , where the distribution of is a mixture of beta-densities of the form .

The policy-maker has the option to update his beliefs about the parameter by testing n individuals at cost K(n ). The information-collection technology has binary test characteristics q = (q 1,q 2), where q 1 is the sensitivity, q 2 is the specificity, and q 1 + q 2 > 1 (indicating the test is properly labeled). The terms ‘sensitivity’ and ‘specificity’ are often used to describe test accuracy in the medical literature. For clarity, we state their relationship to Type I and Type II error: ‘Specificity’ = 1 −‘Type I error’ = 1 −‘False positive rate’ and ‘Sensitivity’ = 1 −‘Type II error’ = 1 −‘False negative rate’. The number of positive samples is an uncertainty with realization v ∈{0,…,n }. Based on the collected information the policy-maker updates his beliefs about in a Bayesian manner.

Proposition 1

If the policy-maker’s prior belief f (⋅)is a mixture of beta-densities, i.e., , then for any number of positive observations from n samples, the Bayesian posterior belief f (⋅|v ) is also a mixture of beta-densities, i.e., f is in .

Proof

See Appendix A.1. □

If is a mixture of m ≥ 1 beta-densities and if the information-collection technology is imperfect (i.e., ), then the true posterior distribution is also a mixture of beta-densities, containing between m + n and m × (n + 1) unique beta-distributions (see Appendix A.2.1). The resulting probability density function (pdf) is with updated weights

Explicit expressions for the conditional mean and variance, μ and σ p|v2, are provided in Appendix A.2.2.

Remark 1

If follows a mixture of m ≥ 1 beta-densities and the information-collection technology is perfect (i.e., q 1 = q 2 = 1), then the distribution of sample information, , is a mixture of m beta-binomial distributions with the same weights ω . Updating results in a posterior distribution that is a mixture of m beta-densities with pdf with updated weights for all i ∈{1,…,m}.

Approximate Bayesian inference

For practically relevant sample sizes n and an imperfect information-collection technology, the number of beta-densities in the posterior distribution can become very large, thus requiring approximation. The need for distributional approximations in decision models has been recognized by Smith who proposed moment matching to replace continuous distributions by appropriate discrete ones [63]. More recently, moment-matching methods have also been used in a Markovian setting, to approximate vector-autoregressions [64]. In our Markov dynamic programming setting, we apply moment matching to approximate the exact posterior distribution which is a mixture of beta-densities with a single beta-distribution. This greatly simplifies the belief propagation compared to dealing with mixtures of beta-densities which feature an increasingly large number of coefficients with each information-collection effort and ultimately an infinite-dimensional state space.

Thus, instead of carrying forward full distribution information about the posterior mixture of beta-densities caused by an imperfect information-collection technology, the policy-maker’s posterior belief about is approximated by a single beta-distribution with the same mean and variance as the exact posterior distribution. The policy-maker’s prior belief is represented by the distribution parameters x = (a ,b ) and the posterior belief incorporating any information collected at time t is represented by the updated parameters . Using the mean and variance of the exact posterior distribution, μ and σ p|v2, the approximate posterior belief parameters are determined using the one-to-one relationship between the standard parameters of the beta-distribution and its mean and variance1. We let ψ(x ,n ,v ,q) denote the function that generates the approximating parameters, with In the case of a perfect information-collection technology, the preceding relations describe the policy-maker’s posterior beliefs exactly.

Mixtures of beta-distributions can be fitted to any continuous distribution on [0,1]. Thus, a single beta-distribution with the same mean and variance as a distribution formed by the mixture of beta-densities, will not always provide a satisfactory approximation. However, we focus on the special case where the time-t belief has been obtained via Bayesian updating from a single beta-prior. In this special case, approximating the mixture of beta-densities with a single beta-distribution with the same mean and variance maintains unimodality2 and stationarity of the state space over time.

We assessed the approximation quality using simulation in the policy-relevant region for our application (Appendix A.3). We found that the maximum distance between the cumulative density function of the exact posterior distributions and that of the approximation with matching mean and variance were generally small (< 2%), but became large when the mean was approaching zero and the standard deviation was relatively large. The quality of the approximation was very good (< 0.5%) when the mean was greater than 2%. We deemed the approximation to be of sufficiently high quality for our numerical analysis because our initial conditions and predicted trajectory without information acquisition rely on the regions in which the approximation is good. Also, because of relatively high fixed costs associated with information acquisition, optimal sample sizes in our numerical analysis tended to be sufficiently large that information would likely only be collected once which reduces concerns about compounding the approximation error over successive information-collection efforts.

System dynamics

The belief state x , containing the parameters of the distribution of , represents the policy-maker’s current beliefs about the uncertain parameter and follows a law of motion of the form where z ∈ (0,1) is the decay rate. These dynamics imply a geometrically decreasing expected value, increasing coefficient of variation, and decreasing variance for (Fig. 1). In the mean-variance space, the equivalent state dynamics become

Fig. 1

Sample state trajectory with decay z = 0.8, with (dashed) and without (solid) information acquisition at time t (for n = 50), respectively

Derivations of these equations are presented in Appendix A.4. The features of these dynamics can represent a wide variety of settings in which the expectation of a parameter is geometrically decreasing over time (e.g., a health condition that is decreasing in prevalence over time; see Section 4). To model a situation where the expectation (and variance) of the uncertain parameter is increasing (e.g., obesity prevalence), the problem can be re-formulated as one in which a parameter of opposite definition is decreasing (e.g., prevalence of individuals who are not obese).

The policy-maker’s problem

Given a social discount factor δ ∈ (0,1), the policy-maker’s objective is to maximize the net present value of the stream of expected INMBs, given the initial belief x 0 = (a 0,b 0) and admissible policy decisions . To achieve the objective, the policy-maker seeks to find the best of all possible policies π (⋅), t ≥ 0, with u = π (x ) for all , which at each time t maps the state space to admissible current-period actions u , so that the implemented path of actions U = (u 0,u 1,…) lies in the control-constraint set . The number of positive observations in the testing sample of n is a random variable with realization v ∈{0,…,n }. Based on the collected information the policy-maker updates his beliefs about in an (approximate) Bayesian manner using the function ψ(x ,n ,v ,q). Because of the decreasing trend of the uncertain parameter (z < 1), it is never optimal to restart an optimally stopped program.3 We consider stationary policies to solve the optimal control problem Provided the value function V (x) satisfies the Bellman equation, for all admissible states , the corresponding maximizer π ∗(x) on the right-hand side defines an optimal policy.

Remark 2

To reflect the policy-maker’s ongoing concern for the health-intervention decision, the problem is formulated in an infinite-horizon setting. Given a time-invariant system, this implies that the optimal policy can be described as a mapping from states to actions, without explicit consideration of time. If more information about the system becomes available over time, for example, relating to the decay rate in the system dynamics (see Eq. 4), then it is possible for the policy-maker to re-solve the problem and update the policy accordingly.

Dynamic healthcare decisions

Policies without information acquisition

If information is prohibitively costly or practically infeasible to collect, Eq. 6 simplifies to for all , as there is no Bayesian updating and therefore ψ reduces to an identity map. For all states x for which the optimal strategy is to not do the intervention, this action remains optimal in the future because of the decreasing trend of . Indeed, since for z ∈ (0,1), μ(ϕ(x)) = z μ(x) < μ(x), we have that for all states where V (x) = 0, it is also the case that V (ϕ(x)) = 0. Hence, for it is optimal to stop the intervention. This defines a threshold policy of the form for all t ≥ 0. Restricting attention to the interesting case where and using the fact that μ(x ) = z μ(x 0), we can identify the optimal time T(x 0) to stop the intervention, which is the first period in which the intervention has a nonpositive expected INMB (see Appendix A.5): Hence, given any initial state x, the value of implementing the optimal stopping policy for t ∈{0,...,T(x) − 1} is

Proposition 2

When information is prohibitively costly or practically infeasible to collect, the optimal value function V NoInfo(x ) is non-decreasing and convex in μ(x ) .

See Appendix A.6. □

Remark 3

The above result depends only on the decay in the mean of the uncertain parameter distribution and is otherwise distribution-free. In other words, it does not depend on the policy-maker’s beliefs other than that is expected to decrease over time.

Policies with information acquisition

When the policy-maker has the option to acquire information, the value function is determined by the Bellman equation (Eq. 6). Its properties in the no-information case (Proposition 2) carry over to the more general situation.

Proposition 3

The optimal value function V (x ) is nondecreasing and convex in μ(x ) , and nondecreasing in σ 2(x ) .

See Appendix A.7. □

Special case: one-time information collection

Assume for now that information can be collected at most once. Given a one-time size- η experiment (with η ≥ 1) and, briefly, ignoring the cost of information collection K(η), the value with information exceeds the no-information value, as a consequence of Jensen’s inequality. This insight is also useful for the comparison of experiments. A higher confidence in the information, i.e., for a larger sample size and/or better test characteristics, produces a mean-preserving spread of the random-variable in the original experiment, and thus, by the convexity of the (monotone) value function and second-order stochastic dominance, a larger value with information.

Because of the monotone system dynamics, the optimal time to collect information of sample size η, at cost κ(η), is obtained by finding a period k where information acquisition is preferred to waiting until the next period, k + 1. In other words, find the smallest k for which or equivalently The positivity of the right-hand side of the last inequality indicates that information acquisition may, on certain trajectories, never be optimal. This is confirmed in our application in Section 4, where the stopping region and the region with information acquisition have a common boundary, transversal to expected state trajectories.

General case: information collection in any period

Based on Proposition 3, the intervention is desirable for greater μ(x ) and greater σ(x ); the latter increases the upside of the policy-maker’s asymmetric (convex) payoffs, as if holding a call option. The dynamics presented in Eq. 4, with decreasing expectation and decreasing variance, imply monotonicity of the intervention decision, .

Corollary 1

Consider x t(1), x t(2)with μ(x t(1)) < μ(x t(2))and σ 2(x t(1)) = σ 2(x t(2)), then if it is optimal to do the intervention with μ(x t(1)), it is also optimal to do the intervention with μ(x t(2)).

Proof

See Appendix A.8. □

Corollary 2

Consider x t(1), x t(2)with μ(x t(1)) = μ(x t(2))and σ 2(x t(1)) < σ 2(x t(2)), then if it is optimal to do the intervention with σ(x t(1)), it is also optimal to do the intervention with σ(x t(2)).

See Appendix A.9. □

A direct consequence of Proposition 3 and Corollaries 1 and 2 is that an optimal policy, as a map from states to actions, features three regions (Fig. 2). We describe, in detail, the features of the optimal policy for the case of an optimal stopping problem (Fig. 2A). In region I, an optimal policy is ‘no intervention (and do not sample).’ In region II, an optimal policy is ‘do intervention and sample n individuals.’ In region III, an optimal policy is to ‘do intervention and do not sample.’

Fig. 2

Policy regions for a an optimal-stopping problem and b an optimal-starting problem. In either case, the initial belief is in region II or III; over time, the belief moves towards region I

The boundary between regions I and III is (Section 3.1). For , the policy-maker is indifferent between ‘no intervention (and do not sample)’ and ‘do intervention and sample n individuals’ when the rewards of the two regions are equal:

Focusing on the region , the policy-maker is indifferent between ‘do intervention and sample n individuals’ and ‘do intervention and do not sample’ when the rewards of the two regions are equal. Removing common terms from each side, this occurs when

For each σ 2(x ), there can exist more than one μ(x ) where satisfying Eq. 11 because is increasing, but neither concave or convex, in v . The existence of the section of region III between regions I and II (the location of point A) can be obtained using intuition. Consider two points, A and B, with the same standard deviation (Fig. 2). Compared to point B, if information were to be gathered at point A, the distribution of possible posterior states includes a higher proportion of states in region I (with a reward of 0) and a lower proportion of high-reward states (those with high mean and high standard deviation) and, therefore, information acquisition is less likely to yield a value exceeding its cost. Now consider two points, A and C, with the same mean. Compared to point C, if information were to be gathered at point A, the distribution of possible posterior states is narrower. In both of these cases, increased spread on the side of low mean has no impact on the expectation and increased spread into the high-reward states substantially increases expectation. Therefore, information acquisition is more likely to yield a value exceeding its cost for the state with higher standard deviation.

Proposition 4

For a fixed sample size η (so n ∈{0,η}for all t), misclassification in the information-collection technology decreases the value function and reduces the number of states for which information acquisition is optimal.

See Appendix A.10. □

This result is consistent with Blackwell’s result that a less informative signal cannot increase the value of a single-person decision problem [66].

Application

Background and motivation

Chronic HCV infection is a slowly progressing blood-borne disease that causes liver fibrosis, cirrhosis, and liver cancer. It is the principal cause of death from liver disease and the leading indication for liver transplantation in the United States (US) [67, 68]. Between 2.7 and 5.2 million Americans (1.1% to 2.1% of the adult population) are chronically infected with HCV [69, 70]. In the non-injection drug using US population, prevalence peaks in the 1945 to 1965 birth cohorts and decreases thereafter (Fig. 3). Approximately half of all chronically infected individuals are unaware of their disease status [71].

Fig. 3

Prevalence of HCV by birth year in a men and b women. Estimated using the National Health and Nutrition Examination Survey (NHANES) (1999-2010). For details, see Appendix A.11.3

Recent model-based analyses concluded that one-time screening of individuals born between 1945 and 1965 is cost-effective [11-15] and the CDC and USPSTF recently released new guidance in support of one-time screening of these birth cohorts [16, 17]. Several studies indicate that screening individuals born later than 1965 is also likely to be cost-effective [13-15]. Since HCV prevalence is decreasing in birth year after the 1956 birth cohort (Fig. 3), there may be a time at which screening is no longer cost-effective. To improve the decision about the best time to stop screening, additional information about prevalence of the current and future cohorts may be desirable. However, standard approaches to finding the value of information do not usually include the option to delay the information acquisition.

Note that the population we model were predominantly infected decades ago [72, 73] and do not have ongoing risk factors for HCV re-infection. Many historically significant modes of disease incidence have been virtually eliminated including transmission by surgical or other hospital equipment prior to modern sterilization procedures and blood transfusion [73, 74]. Injection equipment sharing among people who actively use injection drugs (PWID) is currently the principal cause of HCV transmission [76]. Although a history of injection drug use is relatively common among individuals with chronic HCV infection (approximately 40% [71]), re-infection and disease transmission to others via injection drug use are not an ongoing risk for a large proportion of these individuals as three-quarters of HCV infected individuals with a self-reported history of injection drug use report last injecting greater than 5 years ago (median time since last injection = 20 years) [75]. Our model does not include PWID and so we do not consider the possibility of re-infection. PWID are a high-risk population and guidelines, separate from those otherwise discussed here, recommend routine annual HCV screening in this population [77].

We now apply the stochastic dynamic programming framework developed in Section 2 to the case of one-time HCV screening at a routine medical appointment at age 50 for successive birth cohorts. We consider screening at age 50 because one-time screening at this age had the lowest incremental cost-effectiveness ratio in an analysis of single birth cohort screening [14]. Waiting to perform a one-time screening in older individuals is less cost-effective because their disease may have progressed further and treatment is less effective in more severe disease states. One-time screening of younger individuals is less cost-effective because younger individuals are further away from the long-term consequences of HCV which screening and treatment hope to avoid. We transform the unbounded state space in terms of x = (a ,b ) to the compact policy-relevant space μ(x ) and σ(x ). Using value iteration implemented in R version 2.15.0 [78], we numerically determine an optimal HCV-screening and information-collection policy for US adults.

At each time, we consider the actions of ‘do not screen for HCV and do not collect information about HCV prevalence in the current cohort;’ ‘screen for HCV and collect sample information about HCV prevalence in the current cohort;’ ‘screen for HCV and do not collect information about HCV prevalence in the current cohort.’ We compare this optimal strategy to the policies identified by various alternative approaches: a slightly modified version of the new CDC and USPSTF recommendation; an optimal policy without information acquisition; and an optimal policy with (possibly immediate) information acquisition. A policy of HCV screening does not inherently provide additional information about HCV prevalence to policy-makers, because only positive test outcomes are reported to the CDC and the reason for the medical test is private health information (the test may have been performed for a reason other than routine screening at age 50). Estimating prevalence among asymptomatic individuals seeking routine preventive medical care therefore requires a study with random sampling of those individuals. The (quasi-)linearity of INMB for this example is established in Appendix A.11.1. Parameter values and ranges used in sensitivity analysis are presented in Table 2. Details of parameter estimation are presented in Appendix A.11.2 and A.11.3.

Table 2

Model parameter values and range used in sensitivity analysis

	Males	Females
Variable, Description	Value (Range)	Value (Range)	Sources
Annual Cohort: Individuals aged 50 years
Eligible for a preventive health exam (PHE)	2.1 million (1.8 − 2.3 million)	2.1 million (1.9 − 2.4 million)	[79]
Proportion who attend a PHE	24.4% (19.3 − 29.5%)	43.3% (37.3 − 49.3%)	[80]
N Annual number of PHE	508, 222 (386, 600 − 630, 000)	920, 706 (753, 000 − 1, 100, 000)	Calculated
HCV Screening Test
q 1 Sensitivity	0.97 (0.950 − 0.999)		[81]
q 2 Specificity	0.9996 (0.990 − 1.0)		[82]
C S Cost of HCV-antibody test	$ 28 ($ 20 − 40)		[83]
C FP Cost of false positive	$ 230 ($ 200 − 250)		[83]
B S Quality-of-life change, event of screening	0 (not varied)		Assumed
B FP Quality-of-life change, false-positive result	0 (not varied)		Assumed
Lifetime discounted costs per person
C 1 HCV+, identified through screening	$ 146, 928 ($ 140, 000 − 154, 000)	$ 161, 121 ($ 153, 000 − 170, 000)	[14]
C 2 HCV+, not identified through screening	$ 126, 943 ($ 120, 000 − 133, 000)	$ 143, 900 ($ 136, 000 − 152, 000)	[14]
C 3 HCV- individual	$ 181, 314 ($ 172, 250 − 190, 000)	$ 192, 135 ($ 182, 500 − 202, 000)	[14]
Lifetime discounted quality-adjusted life-years (QALYs) per person
B 1 HCV+, identified through screening	10.42 (10.16 − 10.68)	11.71 (11.41 − 12.0)	[14]
B 2 HCV+, not identified through screening	10.06 (9.80 − 10.31)	11.44 (11.15 − 11.72)	[14]
B 3 HCV- individual	15.69 (15.30 − 16.08)	16.24 (15.83 − 16.64)	[14]
Incremental net monetary benefit (INMB) per person
𝜃 Variable component of INMB	$ 7, 030 ($ 4, 000 − 12, 000)	$ 2, 962 ($ 3, 000 − 10, 000)	Eq. 20
γ Fixed component of INMB	$ 28.05 ($ 22 − 40)	$ 28.05 ($ 22 − 40)	Eq. 21
Cost of collecting information
K F Fixed cost per sampling study	$ 50, 000 ($ 25, 000 − 250, 000)		Estimated
K V Variable cost per sample	$ 100 ($ 50 − 500)		[83]
K(n t) Cost of sampling	(K F + K V n t) 1 {n>0}		Assumed
Other
x 0 Initial belief, HCV prevalence in undiagnosed individuals	a 0 = 75.1, b 0 = 2350.5	a 0 = 51.5, b 0 = 3756.3	Appendix A.11.3
(1960 birth cohort in 2010)	μ(x 0) = 0.031, σ(x 0) = 0.0035	μ(x 0) = 0.0135, σ(x 0) = 0.0019
z Rate of prevalence decay	0.893 (0.871 − 0.915)		Appendix A.11.3
λ Willingness-to-pay threshold	$ 75, 000/QALY gained ($ 50, 000 − 100, 000/QALY gained)		[84]
r Annual discount rate	0.03 (0 − 0.05)		[1]

PHE – Preventive health exam; QALY – Quality-adjusted life-year; INMB – Incremental net monetary benefit

Results

For the purposes of our analysis, we assume the current time to be the year 2010 and the initial cohort to be born in 1960.

Policies identified by alternative approaches

The expected value of the CDC and USPSTF recommendation was obtained by substituting T = 6 into Eq. 9. The sum of the discounted expected INMBs for screening 6 cohorts at age 50, until the 1965 birth cohort turns 50 years of age, is $399.1 million for men and $15.4 million for women (Table 3). The large difference between men and women is attributable to higher HCV prevalence and higher marginal INMB of early diagnosis and treatment in men.

Table 3

Comparison of optimal policies indicated by various analytic approaches for men with initial belief μ(x 0) = 0.0310 and σ(x 0) = 0.0035 and women with initial belief μ(x 0) = 0.0135 and σ(x 0) = 0.0019

Case	Optimal Policy	Value (Expected INMB)	Increase in Expected INMB
Males
CDC/USPSTF recommendation ∗	Screen until 1965 birth cohort turns 50	$ 399, 140, 000	Reference
No information available	Screen until 1978 birth cohort turns 50	$ 566, 470, 000	$ 167, 330, 000
Information only available immediately	Sample 910 men now, then identify optimal action	$ 566, 490, 000	$ 167, 350, 000
Information available in all periods	Sample 4,000 men in 16 years (1976 birth cohort), then identify optimal action	$ 567, 940, 000	$ 168, 800, 000
Females
CDC/USPSTF recommendation ∗	Screen until 1965 birth cohort turns 50	$ 15, 390, 000	Reference
No information available	Screen until 1963 birth cohort turns 50	$ 21, 720, 000	$ 6, 330, 000
Information only available immediately	Sample 4,930 women now, then identify optimal action	$ 22, 320, 000	$ 6, 930, 000
Information available in all periods	Sample 4,500 in 1 year (1961 birth cohort), then identify optimal action	$ 22, 500, 000	$ 7, 110, 000

* The new guidelines recommend screening all individuals born between 1945 and 1965 for HCV at their next routine medical visit [16, 17]. We ignore the screening of individuals born prior to 1960; for all others, we assume HCV screening occurs at age 50

We identify the threshold prevalence value below which the HCV-screening program should be terminated and the best time to terminate the screening program, assuming no opportunity to collect information using Eqs. 7–8. In men, the program should be terminated when prevalence falls below 0.4%, which will occur in 18 years (95%CI: 16-19 years). In women, the program should be terminated when prevalence falls below 0.1%, which will occur in 3 years (95%CI: 0-5 years). The expected INMB of these policies is $566.5 million for men and $21.7 million for women (Table 3).

The traditional approach to value-of-information assessment in the health policy literature assumes immediate information collection [3, 4]. For men and women, we find the optimal sample sizes to be 910 and 4,930 individuals from the current cohort, respectively (Fig. 4). The expected INMB of immediate information followed by the optimal policy based on the information collected increases by $20,000 for men and $600,000 for women. Women have a greater value of immediate information because they are closer to the intervention stopping region threshold and, therefore, immediate information is more likely to result in a policy change.

Fig. 4

The value of collecting sample information immediately for various sample sizes. The gain in the expected INMB of the policy () , the cost of information (κ(η)), and the net gain in the expected INMB of an HCV-screening policy if sample information is collected in the first period only as a function of sample size for a men with initial belief μ(x 0) = 0.031 and σ(x 0) = 0.0035 and b women with initial belief μ(x 0) = 0.0135 and σ(x 0) = 0.0019

Model results

Implementing the full model, we considered the possibility of collecting sample information at each decision period. For computational and illustrative reasons, we restricted the policy-maker’s choice to two sample sizes . We considered several possible values for η (2000, 2500, 3000, ..., 8000) and we present the results for the sample size that maximized the value at the initial condition for each gender. We also performed analyses using multiple study sample size levels available at each period. We do not present these analyses, as they led to the same optimal policies indicating that our restriction to two sample sizes was not material for this application.

The optimal policy is characterized by the three main regions described in Section 3.2.2 (Fig. 5a). At low prevalence and relatively low uncertainty, it is optimal to not screen and not collect information. At high prevalence, it is optimal to screen and not collect information. At prevalence close to the threshold and relatively high uncertainty, it is optimal to both screen and collect information.

Fig. 5

a Optimal policy given any current belief about HCV prevalence and the opportunity to sample 4,000 men (left) and 4,500 women (right) at any time. b Time to the next policy action for men (left) and women (right). For states below the solid line, the next action is to stop screening. For states above the solid line, the next action is to collect information. c The marginal value of collecting information, 4,000 samples for men (left) and 4,500 samples for women (right), in the current period.

For each state in the region where it is optimal to screen without information acquisition, we can identify the optimal next action and the time when it should occur (Fig. 5b). We subdivide this region by a solid line. Above the solid line, which is the region with higher uncertainty, it is optimal to screen without information acquisition for a specified number of periods and then to collect information. In the region with lower uncertainty, it is optimal to screen without information acquisition for a specified number of periods and then to stop screening without ever collecting information. The current prevalence estimates for men and women indicate that it is optimal to screen without information collection for 16 years and 1 year, respectively, and then to collect sample information to inform the next action. The expected INMBs of these policies are $567.9 million and $22.5 million for men and women, respectively (Table 3).

For each state, we also computed the marginal value of collecting a specific amount of information (Fig. 5c). The marginal value of information in the current period is near-zero for states in which collecting information in the future is optimal. Consistent with our expectations, in the ‘Screen and Collect Information’ region, the marginal value of information is greatest close to the -threshold and increases with uncertainty. In the ‘Screen and Do Not Collect Information’ region, the value of information is highest along the boundary that divides the region into points with trajectories leading to information collection and points with trajectories leading to ‘No Screening’ without information collection.

Sensitivity analysis identified that the general conclusions of our numerical analysis are robust to uncertainty in the inputs (details in Appendix A.11.4).

Discussion of application

Evaluating an HCV-screening policy over its entire lifecycle using a stochastic dynamic programming approach has led to several important policy-relevant insights. Our analysis indicates that recommendations by the CDC and USPSTF to screen individuals born between 1945 and 1965 at their next routine medical visit are conservative for men. Specifically, our analysis shows that, for men, screening should continue until at least the 1976 birth cohort turns 50 (in 2026), at which point 4,000 individuals should be sampled to inform about the continuation of the program. Screening men at least 10 years longer will enable early diagnosis in an estimated 50,500 additional individuals, thus preventing an expected 767 additional liver cancers and about 212 additional liver transplants. For women, we find that a large information-acquisition effort should take place when the 1961 birth cohort turns 50 (in 2011),4 as it is likely not cost-effective to screen women, per guidance, to the 1965 cohort because of relatively low prevalence (Fig. 3) and slower disease progression in women [85]. Compared to the CDC and USPSTF recommendation, our model increases the expected INMB by $168.8 million in men and $7.1 million in women.

Our analysis has several limitations. First, we assume only the current cohort can be sampled to learn about subsequent cohorts, relying on the correlation between cohorts (as implied by the system dynamics). In practice, for our example, it is possible to sample the next cohort (49-year olds) directly. We chose this assumption because the individuals who make up the ‘next cohort’ are typically unknown (e.g., the next cohort of patients with a heart attack, the next cohort of pregnant women, or the next cohort of cancer patients). Second, we consider one-time screening at age 50 based on a cost effectiveness analysis of once-in-a-lifetime HCV screening [14]. However, this analysis (and, consequently, ours) assumed that the cohort being screened has not been previously screened. Our model does not identify the optimal age at which to perform one-time screening. Third, we assumed that the individuals who attend a preventive health exam and participate in recommended HCV screening are an unbiased sample from the cohort–that is, individuals are not more or less likely to attend their preventive health exam if they are HCV-positive. However, if individuals at higher-risk of HCV disproportionately self-select for general population screening, then we have underestimated the duration for which screening will be cost-effective. If individuals at lower-risk disproportionately self-select for screening (often called the “worried well”), then we have overestimated the duration for which screening will be cost-effective. Fourth, we focus on HCV screening policy in the non-injection drug using population only because they were the focus of the recent change in HCV screening policy. Finally, while uncertainty (and related information acquisition) with respect to model parameters other than prevalence can be treated in an analogous manner, the details are left for future work.

Conclusion

Our analysis shows that when parameters vary across intervention cohorts, it may be optimal to delay information acquisition. This is a significant improvement over the current paradigm which only considers one-time immediate information collection. More specifically, we provide a framework for optimal information acquisition, in terms of timing and precision of the acquired signal (sample size). Further, we incorporate misclassification from an imperfect information-collection technology into our framework, which is an important real-life complexity of information gathering that adds substantial analytical difficulty.

The common assumption that the per-person value-of-information remains constant for future cohorts may result in a significant error when estimating the population value of additional information. It may indicate immediate expensive information collection when, incorporating the system dynamics, the optimal action is to collect information in the future or never at all. When a parameter is evolving across intervention cohorts, ignoring the opportunity to wait and collect information in the future, when the information collected is more likely to result in action, is a missed opportunity for increased efficiency. As seen in our example, adding the option of delaying information acquisition until a time when the signal is more likely to justify a policy shift can increase the expected value compared to a policy of immediate information collection. The dynamic programming framework developed in this paper enables an accurate assessment of the marginal value of additional information and identifies an optimal information-acquisition policy.

In this work, we assumed that the dynamics are monotonically increasing or decreasing and that they are deterministic. In future work, we plan to consider the more realistic assumption of uncertainty in the dynamics. This would then enable learning about the evolution of the parameters, rather than just their current state. Furthermore, our model does not consider the possibility of intervening on a cohort at a different time in the course of their disease or lives (i.e., at an earlier or later age) or the possibility of the intervention modifying the population-level dynamics. Although true for our application, this latter assumption does not hold in general for an infectious disease. Including the additional benefits of reduced disease transmission from prevention and treatment interventions may generate more near-term benefits and may dramatically alter the value of the intervention over time.

With strained resources for health programs and population-health monitoring, this type of analysis may ensure an optimal implementation horizon for health programs together with guidance on when and how much information should be collected to inform health-program adjustments. Beyond health, many application areas face limited resources for investment and information acquisition, high-quality decision-relevant information is often difficult or expensive to collect, and population or environmental trends influence the preferences and behavior of customers across industries. Facing a dynamic consumer, competitive, or physical environment, the optimal timing of high-quality information acquisition may provide competitive advantage.

Table 4

Glossary of Symbols

Symbol	Definition
N	Annual number of preventive health exams
HCV Screening Test
q 1	Sensitivity
q 2	Specificity
C S	Cost of HCV-antibody test
C FP	Cost of false positive
B S	Quality-of-life change, event of screening
B FP	Quality-of-life change, false-positive result
Lifetime discounted costs per person
C 1	HCV+, identified through screening
C 2	HCV+, not identified through screening
C 3	HCV- individual
Lifetime discounted quality-adjusted life-years (QALYs) per person
B 1	HCV+, identified through screening
B 2	HCV+, not identified through screening
B 3	HCV- individual
Incremental net monetary benefit (INMB) per person
𝜃	Variable component of INMB
γ	Fixed component of INMB
Cost of collecting information
K F	Fixed cost per sampling study
K V	Variable cost per sample
K(n t)	Cost of sampling
Other
x 0	Initial belief, HCV prevalence in undiagnosed individuals (1960 birth cohort in 2010)
z	Rate of prevalence decay
λ	Willingness-to-pay threshold
r	Annual discount rate

Table 5

Logistic regression predicting HCV prevalence for 1956-1980 birth cohorts using NHANES 1999–2010 (n = 12,607)

Variable	Estimate	SE	p-value	Odds Ratio (95% CI)
Intercept	217.7	23.85	< .0001
Birth year	− 0.113	0.012	< .0001	0.893 (0.872 – 0.915)
Race: Black (vs. non-Black)	0.226	0.073	0.0021	1.57 (1.18 – 2.10)
Gender: Female (vs. Male)	− 0.256	0.071	0.0003	0.6 (0.46 – 0.79)

SE – Standard error; CI – Confidence interval

Table 6

Logistic regression predicting the proportion of HCV-positive individuals unaware of their infection status using NHANES 1999–2010 (n = 206)

Variable	Estimate	SE	p-value	Odds Ratio (95% CI)
Model 1:
Intercept	0.00022	0.167	0.998
Race: Black (vs. non-Black)	0.161	0.157	0.304	1.40 (0.75 – 2.55)
Gender: Female (vs. Male)	− 0.332	0.140	0.017	0.51 (0.30 – 0.89)
Model 2 (Final Model):
Intercept	− 0.111	0.144	0.444
Gender: Female (vs. Male)	− 0.329	0.140	0.019	0.52 (0.30 – 0.90)

SE – Standard error; CI – Confidence interval

Table 7

Comparison of optimal policies indicated by various analytic approaches

Case	Adjusted Parameter Value	Optimal Policy ∗	Increase in expected INMB compared to screening for 5 years ∗∗
Males
Base Case		Sample 4,000 men in 16 years (1976 birth cohort), then identify optimal action	$ 168, 800, 000
Low 𝜃	𝜃 = 3000	Sample in 11 years (1971 birth cohort)	$ 51, 410, 000
High 𝜃	𝜃 = 10, 000	Sample in 20 years (1980 birth cohort)	$ 390, 240, 000
Low γ	γ = 22	Sample in 18 years (1978 birth cohort)	$ 197, 990, 000
High γ	γ = 40	Sample in 13 years (1973 birth cohort)	$ 120, 800, 000
Low z	z = 0.871	Sample immediately	$ 185, 690, 000
High z	z = 0.915	Sample in 21 years (1981 birth cohort)	$ 261, 390, 000
High κ F	κ F = 250, 000	Sample in 16 years (1976 birth cohort)	$ 168, 600, 000
Low μ(x 0)	μ(x 0) = 0.024	Sample in 13 years (1973 birth cohort)	$ 106, 040, 000
High μ(x 0)	μ(x 0) = 0.038	Sample in 18 years (1978 birth cohort)	$ 235, 620, 000
Scenario 1	𝜃 = 8500;	Sample immediately	$ 257, 240, 000
	γ = 25;
	z = 0.88
Scenario 2	𝜃 = 5500;	Sample in 14 years (1974 birth cohort)	$ 108, 430, 000
	γ = 35;
	z = 0.905
Females
Base Case		Sample 4,500 women in 3 years (1963 birth cohort), then identify optimal action	$ 7, 110, 000
Low 𝜃	𝜃 = 1500	Never initiate screening / Stop screening	$ 63, 330, 000
High 𝜃	𝜃 = 6000	Sample in 7 years (1967 birth cohort)	$ 22, 550, 000
Low γ	γ = 22	Sample in 4 years (1964 birth cohort)	$ 610, 000
High γ	γ = 40	Sample in 1 years (1961 birth cohort)	$ 46, 040, 000
Low z	z = 0.871	Sample immediately	$ 14, 570, 000
High z	z = 0.915	Sample in 3 years (1963 birth cohort)	$ 2, 460, 000
High κ F	κ F = 250, 000	Sample in 1 years (1961 birth cohort)	$ 6, 900, 000
Low μ(x 0)	μ(x 0) = 0.024	Sample immediately	$ 29, 320, 000
High μ(x 0)	μ(x 0) = 0.038	Sample in 4 years (1964 birth cohort)	$ 340, 000
Scenario 1	𝜃 = 8500;	Sample immediately	$ 7, 090, 000
	γ = 25;
	z = 0.88
Scenario 2	𝜃 = 5500;	Never initiate screening / Stop screening	$ 69, 020, 000
	γ = 35;
	z = 0.905

* Sample size for men = 4,000; Sample size for women = 4,500

** Expected value of the optimal policy with the parameter change compared to the expected value of the CDC/USPSTF recommendation of screening for 5 years (calculated with the adjusted input parameters)