Personalized Evaluation of Biomarker Value: A Cost-Benefit Perspective.

For a patient who is facing a treatment decision, the added value of information provided by a biomarker depends on the individual patient's expected response to treatment with and without the biomarker, as well as his/her tolerance of disease and treatment harm. However, individualized estimators of the value of a biomarker are lacking. We propose a new graphical tool named the subject-specific expected benefit curve for quantifying the personalized value of a biomarker in aiding a treatment decision. We develop semiparametric estimators for two general settings: (i) when biomarker data are available from a randomized trial; and (ii) when biomarker data are available from a cohort or a cross-sectional study, together with external information about a multiplicative treatment effect. We also develop adaptive bootstrap confidence intervals for consistent inference in the presence of nonregularity. The proposed method is used to evaluate the individualized value of the serum creatinine marker in informing treatment decisions for the prevention of renal artery stenosis.

Introduction

Advances in lab technology have led to the discovery of a large number of biomarkers and medical tests that are potentially useful for guiding treatment decisions. However, these biomarkers and tests may be costly, invasive, or otherwise burdensome. Examples include the Oncotype Dx test for predicting the response to adjuvant chemotherapy, which currently costs around $4,000 USD (http://www.breastcancer.org/symptoms/testing/types/oncotype_dx), and Amniocentesis in pregnant women for detecting chromosomal abnormality, which is associated with a risk of miscarriage. Therefore, when a biomarker or test is optional, patients and clinicians must determine whether the biomarker provides useful information for treatment selection beyond readily available information such as patient demographics and standard test results. Hereafter, we use the term “covariate” for any other readily available patient information.

To make an informed decision regarding whether to have a biomarker-based diagnostic test, it is important to understand how the test affects a patient’s clinical consequence. In particular, one needs to quantify the impact of the additional marker using a metric that is intuitive and interpretable to patients. The metric should also flexibly account for individual differences to facilitate personalized decision making. Classical statistical measures for biomarker evaluation are not satisfactory for this purpose. For example, common measures of classification performance like sensitivity, specificity, and the receiver operating characteristics curve [22, 35] are not directly relevant for individual patients’ decision making because they are defined conditional on disease status and do not reflect an individual’s risk of disease. Common measures of risk prediction performance like the predictiveness curve [16, 23] and positive and negative predictive values [1] are informative about disease risk, but by themselves do not account for other important information for decision making such as the effects of available treatments on the targeted disease and additional treatment costs (e.g., the side effect and the monetary cost). Many other recent measures of a marker’s value in a treatment decision, e.g., the overall reduction in the disease rate through treatment selection [27, 31–34], the conditional distribution of the risk difference between comparative interventions in a population [7, 14, 18] and in the marker-positive group [9, 34], focus on the marker’s average effect on the entire population only with respect to the burden of the targeted disease.

There are multiple factors affecting a patient’s clinical consequence, including the patient’s expected disease outcome with or without treatment and the patient’s tolerance of the disease harm and the treatment harm. Decision theory provides an appealing framework for incorporating these cost-benefit considerations into model evaluation and has received much attention in recent years. Examples include the decision curve analysis developed for characterizing the performance of disease risk prediction models [2, 29] and the net benefit for characterizing treatment effect prediction models [4, 25, 30]. Nevertheless, existing work in this paradigm focuses primarily on quantifying the average value of a marker to the whole population for guiding individualized treatment selection instead of allowing the evaluation of the marker value itself to be individualized. For example, the decision curve quantifies a risk prediction model’s utility in units of the benefit for a true positive, which does not account for patient-specific treatment effects. The net benefit quantifies the utility of a marker-based treatment-selection strategy in units of the disease harm, which is intuitive and easy to communicate to patients and clinicians. The utility, however, is measured relative to the strategy of no treatment, which may not be a patient’s best choice based on the available covariate information.

In this paper, we propose a new statistical measure, the subject-specific expected benefit curve, for characterizing the additional value of a biomarker to individual patients from a cost-benefit perspective. As will be shown in Sect. 2, it is built upon an existing decision-theoretic framework [17, 25, 30] but is expanded to allow individualized evaluation of a marker by accounting for individual differences in treatment effect on disease and in tolerance of disease and treatment harm. In particular, this curve quantifies the personalized expected benefit of measuring a marker, which is defined as the reduction in the minimum total disease and treatment cost by incorporating the marker information into an individual’s treatment decision. The points on the curve are defined conditional on an individual’s baseline covariates and one’s choice of treatment/disease cost ratio. We develop semiparametric estimators for this measure that are broadly applicable to both randomized trials and cohort or cross-sectional studies. We also develop adaptive bootstrap confidence intervals that are consistent even in the nonregular setting in which a subject’s specific treatment/disease cost ratio coincides with one’s expected treatment effect on disease outcome.

In Sect. 2, we introduce the concept of the subject-specific expected benefit curve. We develop estimation methods and theoretical results for both randomized trials and cohort studies. We examine finite sample performance of the proposed estimators using simulation studies in Sect. 3. In Sect. 4, we apply the proposed methodology to demonstrate the personalized evaluation of serum creatinine as a biomarker for guiding treatment decision in preventing renal stenosis. Concluding remarks are made in Sect. 5.

Method

Let denote an undesirable binary outcome that we call “disease,” coded 0 for non-diseased and 1 for diseased. Assume that there is a treatment available, and let indicate the decision or recommendation for a subject to have or not have the treatment based on some model. Let denote baseline subject covariates, possibly multivariate, which can be used to inform the treatment decision. Suppose there is a biomarker or a marker combination that, if measured, has the potential to further inform the treatment decision. If the marker is costly to measure, the decision to collect the marker should be informed by an estimate of the individual-specific benefit offered by the marker. In our motivating example of serum creatinine evaluation, a female subject must decide whether to use renal angioplasty to prevent renal stenosis. A model based on her baseline characteristics , such as age, body mass index, smoking status, etc., can be used to predict the probability of having renal stenosis with or without the treatment. Based on this prediction, she can then incorporate her tolerance of the side effects of renal angioplasty to make a treatment decision. If she was also offered the option of having a lab test measuring serum creatinine level, , she should only opt to have the lab test if the predictive model incorporating both and performs significantly better than the model using alone so as to justify the cost of the test.

We propose to quantify the individual-specific value of a biomarker with the expected reduction in an individual’s total disease and treatment cost as the result of measuring the biomarker, under the assumption that an individual would make the optimal treatment decision based on available information either before or after the biomarker measurement. To put the disease cost and the treatment cost on a common scale, we follow the decision-theoretic framework of [30] by allowing each individual to input a ratio of the cost per treatment relative to the cost per disease event, namely , which reflects the individual’s tolerance of the treatment burden relative to his/her tolerance of the disease burden.

Define the cost per targeted disease event as one so that the total disease and treatment cost will be represented in units of the burden per disease event. For a subject with covariate , this total cost, given a treatment decision rule that is a function of either alone or plus , can be computed as follows.

Let and indicate the potential disease outcome if a subject does or does not receive the treatment. First, if a subject with covariates makes a treatment decision based on alone, i.e., , then his/her expected disease rate if receiving/not receiving treatment according to decision rule would equal , where is the indicator function; the corresponding cost of treatment is since the cost of treatment will only be incurred when . As a result, the total disease and treatment cost based on equalsThe optimal treatment-selection rule, say , that minimizes (1) iswhere is the risk difference between non-treated and treated conditional on . That is, a subject will choose to treat the disease only if one’s risk difference is greater than the cost ratio and will not treat otherwise. Equivalently, is the risk difference at which the treatment benefit (i.e., the risk difference for disease) exactly compensates for harm of treatment [30]. The total disease and treatment cost given equalswhere is the positive-part function.

Second, if a subject with makes a treatment decision based on both covariates and the additional marker, i.e., , then the total disease and treatment cost is It follows that the treatment-selection rule that minimizes (3), say , iswhere is the risk difference between non-treated and treated conditional on and . Corresponding total cost due to either disease or treatment for a subject with equalsWe define the expected benefit for a subject with covariates and cost ratio as the reduction in the minimum total cost by adding marker into the disease risk model conditional on :The subject-specific expected benefit curve is defined as the curve of versus . One can also plot versus to show the relative reduction in cost by measuring . Examples of the curves of and for different levels are displayed in Fig. 1a, which will be discussed later in the data example. Corresponding curves of and versus are displayed in Fig. 1b and c. Note that a change-point exists in the curve of versus and in the subject-specific expected benefit curves, where there is a dramatic change in the shape of the curve. This point corresponds to , the treatment effect conditional on , which is the specific cost ratio where a subject’s optimal treatment absent biomarker changes.

Fig. 1

Subject-specific cost curves (a), expected benefit curves (b), and curves of relative reduction in cost by measuring serum creatinine in guiding the treatment of stenosis (c)

Estimation

In this section, we consider estimation of and , the subject-specific expected benefit on absolute and relative scales. We develop semiparametric estimation methods for data either from a randomized trial or from a cohort or cross-sectional study.

Estimation with Data from a Randomized Trial

Biomarker studies nested within randomized trials are natural sources for estimating the subject-specific expected benefit curve. Let indicate the assignment to the untreated and the treated arm respectively in a two-arm randomized trial. Suppose we observe i.i.d. samples of , one observation for every participant in the trial. The estimation of cost and expected benefit consists of two steps: (i) estimation of the disease risk conditional on and (ii) estimation of the distribution of conditional on .

The task in the first step is to estimate and . We make the following assumptions: (i) stable unit treatment value assumption (SUTVA) and consistency: {D(0),D(1)} of one subject is independent of the treatment assignments of other subjects, and the observed outcome is equal to the potential outcome under treatment actually received; (ii) ignorable treatment assignments assumption: . Assumption (i) is plausible in trials where participants do not interact with one another, and assumption (ii) is ensured by randomization. Under these assumptions, for , and . Thus, and can be estimated by modeling the disease risk as a function of , , and . We adopt a generalized linear model (GLM)where is a link function such as logit link or inverse normal-CDF link. We derive as the maximum likelihood estimator (MLE) of .

In the second step, there are many options to estimate the distribution of conditional on . Here we adopt a semiparametric location-scale model [12] for the distribution of conditional on : and , such that , where is some baseline residual distribution with mean 0 and variance 1, and are the location and scale functions, respectively.

Let and with and being specified functions of . For example, for binary , and could be , while for continuous , and could be a B-spline basis for . As in [12], we use the following estimating equations for and :Write and , then under the Gaussian higher moment relationship [12], the above estimating equations equal from which we can solve for and . We then estimate empirically using residuals .

Finally, note that in (2) and (5), can be represented as , the expectation of conditional on . Thus, we estimate in (2) and in (5) withwhere , is the estimated risk of disease, and is the estimated risk difference between untreated and treated conditional on and based on MLE .

Estimation with Data from a Cohort or Cross-sectional Study

Under the SUTVA, consistency, and ignorable treatment assignments assumptions, the subject-specific expected benefit curve can also be derived when we have biomarker data from a cohort or a cross-sectional study that can be used to estimate risk prediction models, if in addition we have external information about a multiplicative treatment effect. The latter can be obtained, e.g., from historical data or from a different study. The assumption of a relative risk for treated versus untreated has frequently been made in evaluating the use of risk models for recommending therapy, including the Gail model for advising the use of tamoxifen to prevent breast cancer [3, 10, 11]. It implicitly requires a monotone treatment effect, i.e., in terms of the disease, the treatment will not cause harm [17]. This multiplicative treatment effect assumption implies a one-to-one correspondence between the risk of disease in the population and the risk difference between untreated and treated, i.e., , where . In other words, in this scenario, we can interpret the subject-specific expected benefit for an individual as the reduction in the total disease and treatment cost by measuring the biomarker, under the condition that a subject with a treatment/disease cost ratio opts to take treatment whenever the predicted disease risk is greater than a threshold value of .

Estimation of and again consists of two steps. In the first step, we model the disease risk as a function of and using a GLM . In the second step, we model the distribution of conditional on using a semiparametric location-scale model as in Sect. 2.1.1. Finally, we estimate and as in (6), where we estimate with .

We note that the proposed estimation procedures can be easily extended to allow for subsampling of , e.g., when the biomarker is only measured from a case-control sample nested within a randomized trial or a cohort study. Let be the probability of sampling from the subject in the study, we can estimate the subject-specific expected benefit curve by weighting the contribution of subject inversely by when estimating both the risk model and the conditional biomarker distribution.

Note that in the special case where is discrete and sample sizes among each level are large, the expected benefit of a marker conditional on a particular can be estimated by modeling the disease risk as a function of and estimating the distribution of [15] within each stratum of separately. This approach, however, does not use data as efficiently, nor is it applicable when is continuous. The semiparametric approach, in contrast, provides a way to borrow information across levels of and applies to general including both discrete and continuous components.

Inference

In this section, we develop confidence intervals for the subject-specific expected benefit. Let and indicate the prevalence of conditional on with or without treatment, respectively. Under standard regularity conditions (RCs) specified in Appendix, when , and are asymptotically normally distributed as stated in Theorems 1 and 2.

Theorem 1

Assume RCs, when , converges to a mean-zero normal random variable as .

Theorem 2

Assume RCs, when , converges to a mean-zero normal random variable as .

Proofs of Theorems 1 and 2 follow from standard arguments and are sketched in Appendix. We also provide expression for the asymptotic variances of and in the Appendix. The asymptotic normality of , the estimator of the relative cost reduction, follows from the Delta method when .

When , it can be shown that , a constituent of and , converges to a mixture of 0 and a truncated normal distribution. As a result, asymptotic normality of , , or no longer holds. Even in the scenario where asymptotic normality of these estimators does hold, we recommend the bootstrap for constructing confidence intervals since computation of asymptotic variances of these estimators requires numerical differentiation.

The foregoing abrupt change in the limiting distribution at signals nonregularity, which is anticipated by the nonsmoothness of the max operator [13, 28]. Thus, standard bootstrap percentile confidence intervals can lead to undercoverage when [8]; to avoid undercoverage, we develop an adaptive bootstrap confidence interval by extending the ideas of [5, 21, 26] Specifically, we use a pretest to determine if . If the test rejects, then there is strong evidence that is different from and we apply the standard percentile bootstrap; if the test accepts, we use a projection interval formed as the union of bootstrap intervals as described below.

Let index bootstrap samples drawn from the original data with replacement. We add a superscript , to indicate that a statistic has been computed using a bootstrap sample. We construct an adaptive projection confidence interval as follows. For any define , . Let and denote percentile bootstrap confidence intervals formed by taking empirical percentiles of and over bootstrap samples, respectively. Let denote an asymptotically valid confidence interval for . The projection intervals for and are given respectively by and .

We construct an adaptive bootstrap CI for or by constructing a standard 95 % bootstrap percentile interval when , and constructing a projection interval described above with otherwise, where is a sequence of positive constants satisfying and . As presented in Theorem 3, this adaptive bootstrap CI will have asymptotic coverage equal to 95 % when and be conservative otherwise. In this sense, the interval is adaptive.

Theorem 3

Let be a sequence of positive random variables satisfying and almost surely as . Define if and otherwise. Let denote probability taken with respect to the bootstrap sampling algorithm, conditional on the observed data. Assume RCs, we havewhere for , and refer to standard bootstrap confidence intervals for and . If then the right hand side of the foregoing inequalities can be replaced with equalities. A proof of Theorem 3 is sketched in Appendix.

;

,

Simulation Studies

In this section, we conduct simulation studies to investigate performance of the proposed estimators. Suppose we have a continuous covariate and a continuous biomarker that jointly follow a bivariate normal distribution with mean 0, standard deviation 0.5 each, and correlation 0.2. We consider two simulation settings: one from a randomized trial and the other from a cohort/cross-sectional study. We consider the estimation of and for equal to the 25, 50, and 75 % percentiles in the distribution of .

In the first setting, we consider a two-arm 1:1 randomized trial where the risk of a binary disease conditional on , and follows a linear Probit model . The differences in disease prevalence between untreated and treated () conditional on the 25, 50, and 75 % percentiles of are , and , respectively.

In the second setting, we consider a cohort/cross-sectional study where subjects receive no treatment. Suppose the risk of a binary disease conditional on and in the cohort follows a linear Probit model . Moreover, suppose the treatment under consideration has a relative risk . The differences in disease prevalence between untreated and treated conditional on the 25, 50, and 75 % percentiles of are , and , respectively.

In each simulation setting, sample sizes of 200, 500, and 2000 are used. Standard bootstrap and adaptive bootstrap confidence intervals are constructed with 1,000 resamples. For the adaptive CI, we use and use the projection interval when , where is the quantile of the standard normal distribution and is estimated from a standard bootstrap procedure with the disease risk model and the conditional distribution of re-estimated in each resampled data.

Performances of our semiparametric estimators of and for simulation setting 1 are presented in Tables 1 and 2 for equal to its 25 and 50 % percentiles. Corresponding results for simulation setting 2 are presented in Tables 3 and 4. In general, our estimators have minimal bias with a sample size as large as 200. Coverage of the percentile bootstrap CI is close to the nominal level when is away from . An undercoverage, however, is observed for standard percentile bootstrap CI when is close to , which is not alleviated with the increase of sample size. The adaptive bootstrap CI fixes the undercoverage problem. The same pattern is observed for equal to its 75 % percentile (results omitted).

Table 1

Performance of estimator based on randomized trial data

	x_1,ρ_0x_1-ρ_1x_1=0.036						x_2,ρ_0x_2-ρ_1x_2=0.097
δ-(ρ_0x-ρ_1x)	-0.036	-0.016	-0.006	0	0.004	0.024	-0.097	-0.027	-0.007	0	0.003	0.023
δ	0	0.02	0.03	0.036	0.04	0.06	0	0.07	0.09	0.097	0.1	0.12
EB_x(δ)	0.028	0.036	0.04	0.043	0.042	0.033	0.014	0.035	0.044	0.047	0.046	0.036
n	Bias×100
200	-0.29	-0.94	-1.37	-1.68	-1.5	-0.68	0.5	-0.5	-1.25	-1.56	-1.4	-0.59
500	-0.04	-0.55	-0.94	-1.26	-1.09	-0.38	0.25	-0.17	-0.81	-1.12	-0.96	-0.26
2,000	0.07	-0.12	-0.4	-0.68	-0.52	-0.03	0.07	0.03	-0.32	-0.6	-0.45	-0.01
	SE×n
200	0.28	0.28	0.28	0.28	0.28	0.28	0.25	0.3	0.3	0.3	0.3	0.3
500	0.33	0.33	0.32	0.32	0.32	0.32	0.26	0.34	0.34	0.33	0.33	0.34
2,000	0.4	0.38	0.36	0.36	0.36	0.4	0.27	0.4	0.36	0.36	0.36	0.4
	Coverage of 95 % bootstrap CI
200	94.42	89.5	84.82	81.34	83.34	91.4	95.84	93.96	88.24	85.12	86.72	93.52
500	95.54	92.32	87.44	82.82	85.62	94.4	95.1	95.34	90.16	86.02	88.34	95.04
2,000	95.56	95.96	91.74	83.58	88.72	96.42	94.44	95.82	93.46	86.38	90.52	96.4
	Coverage of 95 % adaptive bootstrap CI
200	97.1	97.32	97.06	96.7	96.8	97.24	96	97.84	97.88	97.44	97.54	97.58
500	96.78	97.26	97.36	97.12	97.34	97.32	95.24	97.62	97.98	97.58	97.8	97.42
2,000	95.68	97.18	97.64	97.44	97.8	96.66	94.44	95.98	97.88	97.86	98.24	96.72

Here and correspond to the 25th and the 50th percentiles of , respectively

Table 2

Performance of estimator based on randomized trial data

	x_1,ρ_0x_1-ρ_1x_1=0.036						x_2,ρ_0x_2-ρ_1x_2=0.097
δ-(ρ_0x-ρ_1x)	-0.036	-0.016	-0.006	0	0.004	0.024	-0.097	-0.027	-0.007	0	0.003	0.023
δ	0	0.02	0.03	0.036	0.04	0.06	0	0.07	0.09	0.097	0.1	0.12
EB_x(δ)Costx1(δ)	0.214	0.24	0.254	0.263	0.252	0.198	0.113	0.183	0.207	0.216	0.208	0.167
n	Bias×100
200	-1.06	-4.48	-6.45	-7.81	-7.02	-3.26	3.67	-1.76	-4.57	-5.67	-5.06	-2.1
500	-0.17	-2.72	-4.51	-5.88	-5.12	-1.86	1.68	-0.63	-3.03	-4.12	-3.54	-1.01
2,000	0.35	-0.58	-1.84	-3.11	-2.39	-0.21	0.5	0.16	-1.14	-2.16	-1.62	-0.06
	SE×n
200	2.05	1.89	1.82	1.77	1.75	1.64	1.84	1.54	1.45	1.42	1.41	1.34
500	2.3	2.07	1.97	1.92	1.89	1.8	1.91	1.7	1.56	1.52	1.51	1.46
2,000	2.6	2.27	2.07	1.99	1.97	2.04	1.97	1.88	1.61	1.56	1.54	1.61
	Coverage of 95 % bootstrap CI
200	96.54	94.42	92.06	89.96	90.9	94.88	95.48	96.08	92.76	91.14	92.14	95.6
500	96.2	94.38	91.46	88.32	90.14	95.32	95	95.8	92.62	90	91.46	95.4
2,000	95.62	95.8	93.06	87.74	91.1	96.4	94.08	95.64	94.02	88.96	91.82	96.1
	Coverage of 95 % adaptive bootstrap CI
200	97.58	98.14	98.12	98.12	98.08	98.06	95.62	98.02	98.34	98.24	98.3	98
500	97.28	98.04	98.04	97.92	97.98	97.72	95.28	97.62	98.32	98.2	98.4	97.52
2,000	95.82	97.26	98.08	98.1	98.3	96.94	94.08	95.86	98.18	98.36	98.5	96.74

Here and correspond to the 25th and the 50th percentiles of , respectively

Table 3

Performance of estimator based on cohort data

	x_1,ρ_0x_1-ρ_1x_1=0.053						x_2,ρ_0x_2-ρ_1x_2=0.080
δ-(ρ_0x-ρ_1x)	-0.043	-0.023	-0.003	0	0.007	0.026	-0.07	-0.026	-0.01	0	0	0.02
δ	0.01	0.03	0.05	0.053	0.06	0.08	0.01	0.053	0.07	0.08	0.08	0.1
EB_x(δ)	0.004	0.016	0.029	0.032	0.03	0.025	0.003	0.025	0.036	0.042	0.042	0.036
n	Bias×100
200	0.02	0.04	-0.28	-0.45	-0.2	0.01	0.01	0.06	-0.07	-0.43	-0.42	-0.01
500	0.01	0.02	-0.14	-0.29	-0.07	0.01	0	0.02	0	-0.28	-0.27	0.01
2,000	0	0.01	-0.03	-0.14	0	0.01	0	0.01	0.01	-0.14	-0.13	0
	SE×n
200	0.01	0.03	0.06	0.07	0.09	0.11	0.01	0.05	0.06	0.08	0.08	0.11
500	0.01	0.03	0.06	0.07	0.1	0.11	0.01	0.05	0.06	0.07	0.08	0.12
2000	0.01	0.03	0.05	0.07	0.12	0.11	0.01	0.05	0.06	0.08	0.08	0.12
	Coverage of 95 % bootstrap CI
200	90.46	97.18	92.18	84.06	95.9	94.5	90.86	95.34	98	87.94	88.78	95.1
500	93.02	94.14	93.8	82.64	96.18	94.72	93.5	93.62	97.98	86.58	87.74	94.76
2,000	94.2	94.38	96.98	81.8	94.48	94.3	94.62	94.48	95.94	85.74	88.34	94.28
	Coverage of 95 % adaptive bootstrap CI
200	90.42	96.06	97.56	95.84	96.08	94.46	90.86	94.34	98.8	97.08	97.12	95
500	93.02	93.7	98.24	96.26	96.02	94.72	93.5	93.34	98	97.5	97.5	94.76
2,000	94.2	94.38	98.02	96.42	94.32	94.3	94.62	94.48	95.4	96.82	96.9	94.28

Here and correspond to the 25th and the 50th percentiles of , respectively

Table 4

Performance of estimator based on cohort data

	x_1,ρ_0x_1-ρ_1x_1=0.053						x_2,ρ_0x_2-ρ_1x_2=0.080
δ-(ρ_0x-ρ_1x)	-0.043	-0.023	-0.003	0	0.007	0.026	-0.07	-0.026	-0.01	0	0	0.02
δ	0.01	0.03	0.05	0.053	0.06	0.08	0	0.01	0.07	0.08	0.08	0.1
EB_x(δ)Costx1(δ)	0.045	0.143	0.225	0.238	0.225	0.189	0.023	0.146	0.188	0.211	0.211	0.18
n	Bias×100
200	0.59	1.1	-1.16	-2.18	-0.96	-0.02	0.22	0.76	0.12	-1.45	-1.42	0.01
500	0.21	0.43	-0.59	-1.49	-0.38	0	0.07	0.28	0.16	-1	-0.96	0.03
2,000	0.06	0.12	-0.1	-0.78	-0.02	0.02	0.01	0.08	0.09	-0.52	-0.49	0.02
	SE×n
200	0.29	0.61	0.46	0.41	0.35	0.37	0.12	0.47	0.41	0.32	0.32	0.3
500	0.25	0.6	0.51	0.42	0.33	0.36	0.11	0.45	0.47	0.32	0.32	0.29
2000	0.24	0.58	0.63	0.45	0.33	0.36	0.11	0.45	0.52	0.33	0.33	0.29
	Coverage of 95 % bootstrap CI
200	92.18	92.94	97	90.86	96.42	96.64	91.88	92.7	96.76	92.76	92.9	96.36
500	93.98	93.9	97.64	88.06	96.48	95.26	93.82	93.92	95.7	90.88	91.1	95.5
2,000	94.48	94.3	96.74	87	96.64	94.4	94.5	94.38	94.36	89.3	90.08	94.4
	Coverage of 95 % adaptive bootstrap CI
200	92.18	92.76	98.16	98.22	98.58	95.96	91.88	92.62	96.26	98.52	98.64	96.2
500	93.98	93.9	98.1	98.02	98.04	95.06	93.82	93.92	95.12	98.52	98.5	95.08
2,000	94.48	94.3	96.18	97.6	96.6	94.4	94.5	94.38	94.36	97.76	97.82	94.4

Here and correspond to the 25th and the 50th percentiles of , respectively

Example

We illustrate the methodology by evaluating serum creatinine as a risk prediction marker for renal artery stenosis in hypertensive patients from a cohort study. The original cohort consists of 426 hypertensive patients undergoing renal angiography [19, 20]. Baseline risk is modeled with age, smoking status (ever versus never), and their interaction, gender, recent onset of hypertension, body mass index (BMI), abdominal bruit, atherosclerosis disease, and hypercholesterolaemia.

Suppose an individual patient needs to determine whether measuring serum creatinine is useful in guiding the treatment of renal artery stenosis. Here, for illustrative purposes, we assume the presence of a treatment that can lead to a 30 % reduction in the risk of stenosis. The treatment also has additional costs associated with it. An example of the treatment could be the renal angioplasty, which can stabilize a patient’s renal function but has the potential risk of causing bleeding, additional kidney damage, and total renal failure. The value of serum creatinine in guiding the choice of therapy differs across individuals because of differences in individual patients’ baseline risk, distribution of serum creatinine conditional on baseline covariates, and tolerance about treatment harm relative to disease harm. To illustrate the personalized benefit of serum creatinine, we display the subject-specific cost and expected benefit for two individuals using the method described in Sect. 2.1.2. We model the risk of stenosis conditional on baseline covariates and serum creatinine using a linear logistic model, and model the distribution of serum creatinine conditional on baseline covariates with a location-scale model. For the latter, a multiple linear regression model is used for the location parameter conditional on baseline covariates, and a constant scale parameter is assumed. Subject 1 is a 33-year-old male with BMI 19.9 kg who has a smoking history. His baseline risk of renal stenosis is 15.7 %. Subject 2 is a 65-year-old female with BMI 26.0 kg who has vascular diseases. Her baseline risk is 38.3 %.

Figure 1a shows the minimum total disease and treatment cost for each subject as a function of the cost ratio with or without the serum creatinine measurement. Corresponding subject-specific expected benefit curves are shown in Fig. 1b, and the curves of the relative reduction in cost by measuring serum creatinine are shown in Fig. 1c. These curves provide useful information to an individual for determining whether to have the new marker measured. For example, if subject 2 has a high tolerance for treatment cost with a cost ratio , she will have an expected benefit of 1e4 [with 95 % CI (0, 0.005)] and a relative cost reduction of 0.04 % [95 % CI (0–2.6 %)] by measuring serum creatinine, which might be deemed too minimal to justify the measurement of the marker. In contrast, if subject 1 has the same cost ratio of , his expected benefit will be 0.007 [95 % CI (4e4, 0.068)], amounting to a relative cost reduction of 5.2 % [95 % CI (0.2, 32.6 %)], which may be large enough for him to choose the marker measure. On the other hand, suppose subject 1 has a low tolerance for treatment cost with , his expected benefit will decrease to 0.002 [95 % CI (0, 0.012)], which corresponds to 1.3 % reduction in cost [95 % CI (0, 4.3 %)] by measuring serum creatinine. At this cost ratio, subject 2 will have a higher expected benefit of 0.022 [95 % CI (0.002, 0.083)] with a relative cost reduction of 5.9 % [95 % CI (0.5, 26.7 %)].

In practice, given an individual’s baseline covariates, the subject-specific expected benefit curves can be generated. The individual can then find out the value of measuring the additional marker by inputting his/her choice of the cost ratio or a range of , to assist with the decision regarding the diagnostic test.

Concluding Remarks

A biomarker that is useful in guiding treatment decisions for the general population will have different values to individual patients due to individual differences in their responses to treatment and in their tolerances of the disease harm and treatment harm. In this paper, we propose new graphical tools for personalized evaluation of a biomarker’s value in guiding clinical decisions from a cost-benefit perspective. In particular, based on an individual’s baseline covariate level and personal input of treatment/disease cost ratio, we propose to evaluate the biomarker-elicitated reduction in total disease and treatment cost, assuming a person will make the most cost-effective treatment decision before and after measuring the biomarker. Besides the absolute reduction in cost, the relative reduction is also recommended to help with personal decision making. In practice, a subject can compare the expected benefit of a biomarker with the cost of measuring the marker to make a final judgment call about whether to have the biomarker measured.

A caveat in cost-benefit analyses at a population level is the choice of the treatment/disease cost ratio, which is typically difficult to fix across individuals. Different approaches have been taken to reach a common cost ratio. For example, in the setting where treatment harm is characterized by other diseases caused by the treatment, one can estimate the cost ratio using data on different types of disease outcomes, under an assumption about the relative severity between the targeted disease events and the side effect of treatment [10]; other researchers have proposed to elicit a common cost ratio across subjects using a questionnaire [30]. For personalized evaluation of a biomarker, on the other hand, allowing for subject-specific input of the cost ratio is important in order to flexibly account for individual differences. The expected benefit conditional on covariates and a personal input essentially quantifies the value of measuring the marker to individuals in the population who have the same potential disease risk and cost ratio as this particular subject.

Estimators for our proposed measures were derived both in a randomized trial setting where the disease risk is modeled as a function of the marker, the covariates, and the treatment, and in a cohort study setting where the disease risk model in the absence of treatment is modeled as a function of the marker and the covariates. An adaptive bootstrap procedure was developed for inference to account for nonsmoothness in the estimands.

While described in a personalized decision setting, the methods we developed can be readily applied to evaluate the value of biomarkers in treatment decision making among sub-populations, which can differ with respect to baseline covariates and cost ratio.