Literature DB >> 30288409

Data-Driven Markov Decision Process Approximations for Personalized Hypertension Treatment Planning.

Greggory J Schell1,2,3, Wesley J Marrero1,2,3, Mariel S Lavieri1,2,3, Jeremy B Sussman1,2,3, Rodney A Hayward1,2,3.   

Abstract

Background: Markov decision process (MDP) models are powerful tools. They enable the derivation of optimal treatment policies but may incur long computational times and generate decision rules that are challenging to interpret by physicians.
Methods: In an effort to improve usability and interpretability, we examined whether Poisson regression can approximate optimal hypertension treatment policies derived by an MDP for maximizing a patient's expected discounted quality-adjusted life years.
Results: We found that our Poisson approximation to the optimal treatment policy matched the optimal policy in 99% of cases. This high accuracy translates to nearly identical health outcomes for patients. Furthermore, the Poisson approximation results in 104 additional quality-adjusted life years per 1000 patients compared to the Seventh Joint National Committee's treatment guidelines for hypertension. The comparative health performance of the Poisson approximation was robust to the cardiovascular disease risk calculator used and calculator calibration error. Limitations: Our results are based on Markov chain modeling. Conclusions: Poisson model approximation for blood pressure treatment planning has high fidelity to optimal MDP treatment policies, which can improve usability and enhance transparency of more personalized treatment policies.

Entities:  

Keywords:  Markov models; decision rules; decision support techniques; provider decision making

Year:  2016        PMID: 30288409      PMCID: PMC6124941          DOI: 10.1177/2381468316674214

Source DB:  PubMed          Journal:  MDM Policy Pract        ISSN: 2381-4683


Optimal medical decision making is challenged by uncertainty regarding both treatment effects and patient risks in the absence of treatment.[1,2] Such uncertainties create even greater problems for situations in which decisions must be made sequentially over time, such as making medication recommendations for chronic conditions. Markov decision processes (MDPs) are well-established tools for aiding such sequential decision making.[3-7] MDP models are frequently used in operations engineering and have potential applications in medicine, in which they can enable the decision maker to develop personalized treatment plans to optimize a patient’s health outcome based on the best available information, with due consideration for parameter uncertainty. However, using MDPs can be difficult for real-world clinical practice. They often require specialized software, such as MATLAB or CPLEX,[8] that are unavailable to most clinicians and hospitals. Using an MDP model often requires long computational times, especially when the medical problem studied has high dimensional state (many clinical variables) and action spaces (the possible decisions that could be made), which make look-up tables infeasible.[9] Moreover, the decision rules obtained with an MDP may be complex and not easily interpretable. Together, these limitations also create a black box for clinicians, limiting acceptance and making communicating the results of the MDP to patients difficult. Therefore, a tool that harnesses the improved decision making capabilities of an MDP model (without the burden of specialized software, long computation time, and complexity) would prove useful in the management of patients with chronic disease requiring serial decision-making. One such chronic disease is hypertension, or high blood pressure (BP), which is a major risk factor for cardiovascular disease (CVD; consisting of heart attack, angina, stroke, and other heart and blood vessel diseases), the leading cause of death in the United States and worldwide.[10-12] We have created and previously reported on hypertension treatment policies using an MDP model with the goal of maximizing a patient’s expected discounted quality-adjusted life years (QALYs), and we found that it saved significantly more QALYs than the Seventh Joint National Committee’s (JNC7’s) proposed treatment policy.[13] This was accomplished by combining the power of MDP operations research modeling with a benefit-based tailored treatment strategy,[14-16] which simultaneously considers all known factors that affect an individual’s untreated risk and the treatment’s relative effects on those risks. In this article, we develop a novel framework that approximates the optimal hypertension dosage decision from the MDP model using Poisson regression and generalized linear mixed effect modeling. We then compare the performance and accuracy of our approximation to the original MDP model.

Methods

All methods are described in more depth in the supplemental appendix.

Data Source and Study Design

We sampled 100,000 simulated patients from the Third National Health and Nutrition Examination Survey (NHANES III),[17] a survey of U.S. patients, to generate a representative sample of the U.S. population. NHANES III was chosen because BP was treated much less aggressively in the early 1990s, allowing for easier estimation of untreated BPs. Our data set included adults between the ages of 40 and 84 years that were examined over a 10-year horizon. The QALY MDP for CVD was solved for all 100,000 patients over a 10-year planning horizon (assuming a 0.001 disutility per prescribed medication) to obtain the MDP treatment policy for each patient. We also applied the current hypertension treatment guidelines in the United States, known as JNC7,[18] to each patient in the sample.

Markov Decision Process Formulation

We modeled the process of sequentially determining hypertension treatment medications over a planning horizon as a discrete-time, finite horizon MDP formulation.[19] The objective of the MDP was to determine the optimal treatment strategy π* for a single patient that maximizes his/her expected discounted QALYs over the planning horizon, t = 1, . . ., T. We considered a 10-year planning horizon with annual treatment decisions. The MDP formulation was characterized by four features: state space, action space, state transition probabilities, and rewards. Table A1 summarizes the inputs and data sources for the model.

State Space

The MDP utilizes a state-space representation of a single patient to fully describe his/her characteristics at each time period t = 1, . . ., T in the planning horizon of length T. The state s consists of demographic information, clinical observations, and the patient’s health state. The demographic information includes the patient’s age, sex, smoking status, and diabetes status. The clinical observations are measurements of the patient’s untreated systolic blood pressure (SBP), high-density lipoprotein (HDL), total cholesterol (TC), and if the patient has left ventricular hypertrophy as determined by electrocardiogram. Last, there are 10 mutually exclusive patient health states: 1) healthy (no history of coronary heart disease (CHD) or stroke); 2) history of CHD but no CHD event this period; 3) history of stroke but no stroke this period; 4) history of CHD and stroke but no adverse event this period; 5) survived a CHD event this period; 6) survived a stroke this period; 7) death from a non–CVD-related cause; 8) death from CHD event this period; 9) death from stroke this period; and 10) dead.

Action Space

The physician makes annual treatment decisions in accordance with policy π, which depends on the patient’s state s at each decision period t. We assumed the physician is able to prescribe between 0 and 5 antihypertensive medications at full dosage. These antihypertensive medications may include thiazide diuretics, beta-blockers, calcium channel blockers, angiotensin-converting enzyme inhibitors, and angiotensin II receptor antagonists. However, we constrain the set of possible actions by a minimum allowable SBP threshold which may depend on the patient’s state. The minimum allowable SBP threshold prevents the physician from prescribing medication that would, in expectation, lower the patient’s SBP below the threshold, so as to prevent treatment harm from excessive medication. We used a threshold of 120/90 mmHg. For example, a patient with an untreated SBP of 130 mmHg taking one medication would expect a BP reduction of 6.7 mmHg and a BP reduction of 12.73 mmHg if taking two medications.[20] With a threshold of 120 mmHg SBP, the patient could be prescribed one medication (resulting in a new SBP of 123.3 mmHg), but could not be prescribed two medications (since the new SBP of 117.27 mmHg is below 120 mmHg).

Transition Probabilities

Before treatment decisions are made at the beginning of each decision period, the patient’s state s is mapped to a pretreatment, one-period likelihood of CHD, , and stroke, , event using the risk calculators found.[21] After a treatment decision is made at the beginning of each decision period, we assume the antihypertensive medication has a near-immediate effect on the patient’s SBP. The expected change in SBP from treatment is calculated using the equations published in previous studies.[22] Given the expected change in SBP, we compute the patient’s posttreatment SBP (posttreatment SBP = pretreatment SBP − expected changed in SBP). Using the patient’s posttreatment SBP, we determine the post-state , which includes all the other elements of s. We further utilize the expected change in SBP to compute the relative risk reduction (RRR) for CHD, , and stroke, , from treatment using equations from previously published studies.[22] These equations utilize age- and event-specific risk slopes (see Table A2) and the reduction in SBP from treatment relative to a 20 mmHg reduction to determine the RRR. Multiplying the patient’s pretreatment risk by the RRR, we computed the posttreatment risk for CHD, , and stroke,. If the patient has a history of CHD and/or stroke, we multiply the patient’s CHD and stroke odds by a scaling factor and/or .[23,24] We used a scaling factor of 3 for CHD and stroke odds. With fatality likelihoods for CHD events, , and strokes, ,[25-27] and total mortality likelihood, , we compute the probability of transitioning from one health state to another health state The fatality likelihoods are presented in Table A3. We assume only one type of event may happen in a decision period, for example, a patient cannot have both a CHD event and a stroke. In order to guarantee valid probability distributions, we prioritize death from stroke over other outcomes and adjust the health state transition probabilities accordingly. In addition to the computing health state transition probabilities, we model the dynamics of the other elements of the patient’s state s over the planning horizon. To forecast the patient’s untreated SBP, HDL, and TC, we utilized linear regression on the NHANES III data. We considered the following candidate predictors for each linear regression model: intercept term, age, squared age, sex, smoking status, diabetes status, race (white, black, Hispanic), history of CHD, and history of stroke. Using Akaike information criterion (AIC), the best models for SBP, HDL, and TC were determined. Table A4 presents the coefficients for each model. We applied the linear regression model to each patient to forecast that patient’s SBP, HDL, and TC over the next 10 years. We adjusted the forecasts by applying the difference between the linear regression fitted value and the observed value in the NHANES data to all forecasted values (effectively changing the intercept term for each risk factor regression model). The health state transition probabilities and linear regression models jointly determine the state transition probabilities , which depend on the treatment decision .

Rewards

Our objective is to maximize the patient’s expected discounted QALYs, that is, find the policy , which maximizes Z: where is the discount factor and is the reward resulting from treatment when the patient is in state s at decision t. We used a discount factor of 0.97. The reward is equal to the quality of life associated with the patient’s state minus the burden from the treatment . We assume the quality of life weight for each state s depends only on the patient’s health state Quality of life weights come from previously published literature.[28-30] Table A5 reports the weights used.

Poisson Model Development

We split the full sample into a training set of 20,000 patients, a threshold set of 20,000 patients, and a testing set of 60,000 patients. Using the training set, we parameterized a Poisson regression model using linear mixed-effects modeling to account for the correlation of the sequential measurements taken to each patient during their annual examination, such as BP and cholesterol readings. The optimal medication count at each decision period from the MDP policy was used as the response variable, while demographic information and risk factor data were used as predictor variables. Poisson regression is a regression model where the outcome variable is a count variable (e.g., number of medications). By expressing the expected value of the outcome variable as an exponential function of a linear combination of predictor variables, Poisson regression guarantees that the predicted outcome is nonnegative—in contrast to traditional regression that would allow for predictions of negative medications. Linear mixed-effects modeling combines fixed effects (e.g., SBP, age, race) with random effects (e.g., patient). In biostatistics, the fixed effects represent population averages while the random effects represent patient-specific effects. Since our data contain multiple readings per patient, we require a random effect to control for the within patient correlation of outcome variables. We considered two Poisson regression models: 1) full model and 2) risk-only. The full Poisson model used an initial candidate set of predictors of age, sex, smoking status, diabetes status, pretreatment SBP, diastolic blood pressure (DBP), HDL, TC, 5-year CVD risk (heart attack plus stroke) as computed by the Framingham risk calculator,[21] and an interaction term for each variable with CVD risk. The risk-only Poisson model just considered 5-year CVD risk, determined using the Framingham risk calculator, as a predictor.[21] The final set of predictors and regression coefficients of the full Poisson model were determined using the AIC. Since the optimal medication count comes from solving a finite-horizon discounted MDP, the optimal policy is time-dependent. However, since age is a state variable of the MDP, which increases at the same rate as time, the optimal policy can be viewed as approximately age-dependent. Therefore, the time-dependence of the optimal policy is approximately accounted for by including age as a predictor in the Poisson regression models.

Number of Medications to Prescribe: The Rounding Threshold

Within the models, we determined the number of medications to prescribe under the Poisson policies examining a sequence of rounding thresholds (0 to 1 by 0.1 increments). The sequence of thresholds was first applied to each parameterized Poisson regression function in the threshold set of 20,000 patients. If the Poisson regression fitted value was less than or equal to the rounding threshold, we rounded down to the nearest integer. For each rounding threshold, we computed the expected QALYs per 1000 patients. The rounding threshold that maximized QALYs was set as the preferred threshold.

Feasibility Adjustments

Because many believe hypertension is especially dangerous at higher levels, we treated all patients whose BP was >150/90 mmHg. Also, since there is no evidence that BP treatment is beneficial at especially low levels, our models do not allow treatment if the BP is below 120/90 mmHg.

Model Evaluation

Given the preferred threshold found in the threshold set (n = 20,000), we evaluate the performance of the Poisson policy model in the testing set (n = 60,000). We computed the expected QALYs saved and expected number of CVD events prevented (when compared to no treatment) under the two Poisson policies, the optimal MDP policy, and the JNC7 guidelines. We also compared the number of medications prescribed by the Poisson and MDP policies to determine the rate at which the policies matched.

Sensitivity Analyses

To evaluate the robustness of the Poisson regression approach to approximating optimal MDP treatment policies, we also investigated how the Poisson policies performed when using a different CVD risk calculator, the ASCVD calculator.[31] This analysis allowed us to determine if the Poisson policy would be effective in other countries where the risk calculators are calibrated to a different population. We also evaluated the treatment policies under scenarios of CVD risk calculator miscalibration. Given the reliance on CVD risk as a predictor variable in both Poisson policies and the optimal MDP policy, it is important to evaluate the performance of these treatment policies when the risk computed by the calculator is ±25% off from the patient’s true CVD risk.

Results

Comparison of Sample Populations

Table 1 summarizes the patient characteristics of the training and testing set. We found that randomization was successful (i.e., no clinically significant differences between the populations).
Table 1

Summary Statistics of Population

VariableTraining (n = 20,000)Testing (n = 60,000)
Age, median (IQR)60.0 (22.0)60.0 (22.0)
Male (%)47.0%47.1%
Diabetic (%)8.7%8.2%
Smoker (%)23.6%23.7%
SBP (mmHg), median (IQR)133.0 (27.7)133.1 (27.7)
DBP (mmHg), median (IQR)77.1 (14.5)77.3 (14.4)
HDL (mg/dL), median (IQR)49.4 (20.2)49.4 (20.0)
TC (mg/dL), median (IQR)216.4 (58.7)216.4 (57.8)
5-Year % CVD risk, median (IQR)7.17% (11.03%)7.23% (11.02%)

Note: IQR = interquartile range; SBP = systolic blood pressure; DBP = diastolic blood pressure; HDL = high-density lipoprotein; TC = total cholesterol.

Summary Statistics of Population Note: IQR = interquartile range; SBP = systolic blood pressure; DBP = diastolic blood pressure; HDL = high-density lipoprotein; TC = total cholesterol.

Poisson Regression Analysis

We applied the Poisson policies to the testing set and compared the number of medications prescribed against the optimal MDP policy for each patient. Table 2 provides the percentage of decision periods where the Poisson policies exactly matched, prescribed fewer medications, and prescribed more medications than the optimal MDP policy. We found that the full Poisson model has 99% accuracy and a very low mean squared error (0.006), while the risk-only Poisson model (the model just considering the 5-year CVD risk as computed with the Framingham risk calculator) has 92% accuracy and a higher mean squared error (0.859). Further evidence of the accuracy of the Poisson policies is plotted in Figure 1, which shows the distribution of the number of medications for both Poisson policies and the optimal MDP policy. The frequency plotted is the number of times a patient was prescribed that number of medications for 1 year, based on the testing population of 60,000 patients followed for 10 years. The full Poisson policy and MDP policy do not have substantively different distributions. On the other hand, the risk-only Poisson policy differs significantly when the optimal number of medications to prescribe is 4 or 5.
Table 2

Comparison of MDP and Poisson Treatment Policies

Compared to MDP, Percentage of Decision Periods in WhichFull Poisson ModelRisk-Only Poisson Model
Equal meds prescribed (%)99.691.6
Fewer meds prescribed (%)0.08.1
More meds prescribed (%)0.40.3

Note: MDP = Markov decision process.

Figure 1

Number of medications prescribed for MDP and Poisson treatment policies

Comparison of MDP and Poisson Treatment Policies Note: MDP = Markov decision process. Number of medications prescribed for MDP and Poisson treatment policies Results in Table 2 and Figure 1 only examine discordant prescribing decisions, but a much more important issue is how different prescribing decisions, using the Poisson policies instead of the optimal MDP policy, affects health outcomes for patients. Table 3 provides a comparison of the expected QALYs and expected number of CVD events when prescribing decisions are based on the two Poisson policies, the MDP policy and JNC7. We found that using the full Poisson policy results in nearly identical health performance for patients compared to the optimal MDP policy. Furthermore, the reduction in accuracy to 92% for the risk-only Poisson policy only reduced CVD events prevented by 1.0 and QALYs saved by 5.4 per 1000 patients (10,000 patient-years), compared to the MDP policy. In contrast, the full Poisson policy prevents 17.9 fewer events and saves 109.2 more QALYs per 1000 patients when compared to the JNC7 treatment policy.
Table 3

Health Performance of Treatment Policies

Expected Number of CVD Events Prevented per 1000 Patients[a]
Expected Discounted QALYs Saved per 1000 Patients[b]
Treatment PolicyTotalReduction Compared to MDPTotalReduction Compared to MDP
MDP47.4Ref271.9Ref
Full Poisson model47.4<0.1271.9<0.1
Risk-only Poisson model46.41.0266.55.4
JNC729.517.9162.7109.2

Note: CVD = cardiovascular disease; QALY = quality-adjusted life year; MDP = Markov decision process; JNC7 = Seventh Joint National Committee.

CVD events prevented when compared to no treatment.

QALYs saved when compared to no treatment.

Health Performance of Treatment Policies Note: CVD = cardiovascular disease; QALY = quality-adjusted life year; MDP = Markov decision process; JNC7 = Seventh Joint National Committee. CVD events prevented when compared to no treatment. QALYs saved when compared to no treatment. Table A7 in the supplemental appendix presents the regression coefficients and P values of the full and risk-only Poisson models. The regression coefficients of the final set of predictor variables included in the full Poisson model were obtained using the AIC method. Based on these final regression models, we found that the optimal rounding threshold is 0, that is, always round the fitted value from the Poisson regression down to the nearest integer. To determine if the Poisson policies perform well when CVD risk is calculated based on a different population, we evaluated the performance of the four treatment policies when the ASCVD risk calculator is used to estimate CVD risk instead of the Framingham equations (see Table 4). We found that the improvement over JNC7 remained substantial for both CVD events prevented (17 fewer CVD events per 1000 patients) and QALYs gained for the full Poisson model and MDP policies when using the ASCVD calculator. The risk-only Poisson model continued to perform only slightly worse than the optimal MDP policy.
Table 4

Health Performance of Treatment Policies Under ASCVD and Framingham Risk Calculator Miscalibration

Expected Number of CVD Events Prevented per 1000 Patients
Expected Discounted QALYs Saved per 1000 Patients
Risk CalculatorTreatment PolicyTotalReduction Compared to MDPTotalReduction Compared to MDP
ASCVDMDP47.8Ref185.3Ref
Full Poisson47.70.1185.10.2
Risk-only Poisson46.81.0181.34.0
JNC730.617.2115.769.6
+25% (Framingham calculator overestimates true CVD risk)MDP58.5Ref336.0Ref
Full Poisson58.5<0.1336.0<0.1
Risk-only Poisson57.70.8331.34.7
JNC736.322.2199.8136.2
−25% (Framingham calculator overestimates true CVD risk)MDP35.9Ref204.8Ref
Full Poisson35.9<0.1204.8<0.1
Risk-only Poisson34.81.1199.35.5
JNC722.413.5123.481.4

Note: CVD = cardiovascular disease; QALY = quality-adjusted life year; MDP = Markov decision process; JNC7 = Seventh Joint National Committee.

Health Performance of Treatment Policies Under ASCVD and Framingham Risk Calculator Miscalibration Note: CVD = cardiovascular disease; QALY = quality-adjusted life year; MDP = Markov decision process; JNC7 = Seventh Joint National Committee. We next examined the effects of risk calculator miscalibration, that is, the calculated CVD risk systematically over- or underestimates the patient’s true CVD risk. Table 4 reports the performance of the treatment policies under overestimation of CVD risk (+25% calibration error) and underestimation of CVD risk (−25% calibration error). For both over- and underestimation of true CVD risk, the Poisson policies yield higher expected CVD events prevented (22.2 and 13.5 fewer events per 1000 patients, respectively) and QALYs gained (136.2 and 81.4 more QALYs per 1000 patients, respectively) than JNC7. The differences between MDP and the full Poisson model remain negligible and the differences between MDP and the risk-only Poisson model remain small.

Discussion

Our analyses point the way for how decision analysis can become much more practical for daily clinical use, by finding that Poisson regression can be used effectively to approximate a fully optimized MDP model for the prescription of BP medications. The full Poisson model, which includes interactions effects, resulted in negligible differences in the treatment decisions and health outcomes compared to the optimal MDP treatment policy. The risk-only Poisson model (including only 5-year CVD risk as a factor) reduced CVD complications almost as much as the full Poisson model. Furthermore, both Poisson models dominated JNC7 treatment guidelines in QALYs and CVD events. We also found that the performance of the MDP and Poisson models was robust to the CVD risk calculator used (Framingham or ASCVD) and to calculator miscalibration (±25%). While not directly studied here, our QALY maximizing decisions likely lead to more cost-effective decisions than JNC7 given the high medical cost of CHD events and strokes and the low cost of BP medications. More broadly, our study revealed that policy approximation can offer very comparable performance to MDP optimized treatment policies without the drawbacks of MDP models. MDP models generally require specialized software and lengthy computational times that would prohibit their use in many clinical settings. In this study, computational times were short (within seconds) but we required software capable of implementing recursive algorithms in order to solve the MDP. Approximations to these complex models limit these drawbacks by generating simple equations that can be solved nearly instantly using standard software or a calculator. With the advent of big data, decision support tools can develop approximations to the optimal treatment policy by training statistical models on very large data sets of patients. By doing so, the computational burden is removed from the end-user (i.e., the clinician or facility) and frontloaded to the decision support developer who has greater computational capabilities and access to the required software. Furthermore, policy approximation improves the interpretability of the optimal treatment decision rules for clinicians. For Poisson regression, the coefficients of the predictors provide insight into how sensitive the optimal number of medications is to changes in patient risk factors. This insight may translate to improved trust of the decision support tool and increase its utilization and acceptance in clinical practice. This is particularly true for the risk-only Poisson model since there is only one regression coefficient to interpret, whereas the full Poisson model may have multicollinearity, which prevents direct interpretation of the sign and magnitude of the coefficients. For the risk-only Poisson model, the intercept coefficient is 1.167 and the coefficient for 5-year CV risk is 1.006. The positive sign of the risk coefficient implies that increases in CV risk lead to increases in the number of medication prescribed. This matches physician intuition and guidelines, which bolsters the trustworthiness of the Poisson regression. The intercept coefficient can be interpreted to mean the risk-only Poisson model would prescribe at least 3 medications to patients (before applying the minimum SBP threshold constraint). The risk coefficient implies that the patient’s pretreatment 5-year CV risk would need to be at least 22% (44%) to recommend 4 (5) BP medications. The full list of the Poisson models’ coefficients is available in the supplemental appendix. In general, our policy approximation methodology could be applied to any MDP formulation developed with a large structured dataset. For instance, the optimal prescription planning derived with MDP formulations for patients with type 2 diabetes[20] or for patients with heart disease[7] could be approximated using Poisson regression, as long as there is enough data to fit the model. Our methodology can also be applied to other objective functions, including maximizing cost-effectiveness and minimizing the number of CVD events, and application domains. Our study’s limitations derive mainly from questions about the inputs in our models. Most notably, the precise details of who benefits from BP lowering are debatable. Our modeling choices, however, are based on the best available clinical trial data from an individual-level meta-analysis[32] and fit the SPRINT Trial well, which found treatment benefit to 120 mmHg when using 3 to 4 BP medications in very high CVD risk patients.[15] We chose to base our comparison on the JNC7,[18] because American practitioners often know it well, and JNC8 was not a consensus guideline and is unlikely to be relevant for much longer. Efforts are already underway to update these guidelines, and our results suggest that our benefit-based tailored treatment approach should at least be strongly considered in deliberations of how to update BP guidelines post-SPRINT.[14] Critics of using CVD risk in clinical decision making have pointed to risk prediction tools’ well-known susceptibility to calibration problems when developed in one population and applied to different populations.[33] Our results, however, demonstrate that a benefit-based treatment approach remains superior to traditional treatment target–based approaches, such as JNC7, even in the presence of substantial calibration problems. Of course, it is always preferable to use a risk prediction tool that is well calibrated to the patient population being treated, and in the era of the modern electronic medical records, it is becoming much more feasible for integrated health systems to develop or recalibrate risk tools to be optimally fitted to their own population.[34] Furthermore, critics have pointed out that risk prediction tools with similar discrimination, as measured by the area under the receiver operating curve, often given substantively different risk estimates for individual patients. Our study demonstrates that whether the Framingham and ASCVD tool was used,[21,31] the Poisson models prevented many more CVD events and saved many more QALYs compared to the JNC7 approach.

Conclusion

We developed a Poisson regression model to approximate the optimal treatment decisions from an MDP model. We found that our policy approximation enables fast, easily interpretable, and comparable decision support without a need for specialized software. Our application to hypertension treatment planning for patients at risk for CVD indicates high fidelity to optimal treatment policies and high performance. Treatment policy approximation resulted in more QALYs and fewer CVD events than current clinical practice. Furthermore, the improvement in health outcomes over JNC7 was robust to which risk calculator was used, as well as to systematic calculator error. While this study specifically addressed hypertension treatment, we believe the methodology of using Poisson regression to approximate fully optimized MDP policies has the potential to reduce computational time, improve acceptability by clinicians, and maintain health-optimization performance for treatment decisions for chronic diseases more generally.
  21 in total

1.  2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines.

Authors:  David C Goff; Donald M Lloyd-Jones; Glen Bennett; Sean Coady; Ralph B D'Agostino; Raymond Gibbons; Philip Greenland; Daniel T Lackland; Daniel Levy; Christopher J O'Donnell; Jennifer G Robinson; J Sanford Schwartz; Susan T Shero; Sidney C Smith; Paul Sorlie; Neil J Stone; Peter W F Wilson; Harmon S Jordan; Lev Nevo; Janusz Wnek; Jeffrey L Anderson; Jonathan L Halperin; Nancy M Albert; Biykem Bozkurt; Ralph G Brindis; Lesley H Curtis; David DeMets; Judith S Hochman; Richard J Kovacs; E Magnus Ohman; Susan J Pressler; Frank W Sellke; Win-Kuang Shen; Sidney C Smith; Gordon F Tomaselli
Journal:  Circulation       Date:  2013-11-12       Impact factor: 29.690

2.  Deaths: final data for 2009.

Authors:  Kenneth D Kochanek; Jiaquan Xu; Sherry L Murphy; Arialdi M Miniño; Hsiang-Ching Kung
Journal:  Natl Vital Stat Rep       Date:  2011-12-29

3.  Aspirin for the primary prevention of cardiovascular disease in women: a cost-utility analysis.

Authors:  Michael Pignone; Stephanie Earnshaw; Mark J Pletcher; Jeffrey A Tice
Journal:  Arch Intern Med       Date:  2007-02-12

4.  Heart disease and stroke statistics--2009 update: a report from the American Heart Association Statistics Committee and Stroke Statistics Subcommittee.

Authors:  Donald Lloyd-Jones; Robert Adams; Mercedes Carnethon; Giovanni De Simone; T Bruce Ferguson; Katherine Flegal; Earl Ford; Karen Furie; Alan Go; Kurt Greenlund; Nancy Haase; Susan Hailpern; Michael Ho; Virginia Howard; Brett Kissela; Steven Kittner; Daniel Lackland; Lynda Lisabeth; Ariane Marelli; Mary McDermott; James Meigs; Dariush Mozaffarian; Graham Nichol; Christopher O'Donnell; Veronique Roger; Wayne Rosamond; Ralph Sacco; Paul Sorlie; Randall Stafford; Julia Steinberger; Thomas Thom; Sylvia Wasserthiel-Smoller; Nathan Wong; Judith Wylie-Rosett; Yuling Hong
Journal:  Circulation       Date:  2008-12-15       Impact factor: 29.690

5.  Optimizing statin treatment for primary prevention of coronary artery disease.

Authors:  Rodney A Hayward; Harlan M Krumholz; Donna M Zulman; Justin W Timbie; Sandeep Vijan
Journal:  Ann Intern Med       Date:  2010-01-19       Impact factor: 25.391

6.  Cardiovascular disease risk profiles.

Authors:  K M Anderson; P M Odell; P W Wilson; W B Kannel
Journal:  Am Heart J       Date:  1991-01       Impact factor: 4.749

7.  Prevalence of hypertension and controlled hypertension - United States, 2007-2010.

Authors:  Cathleen D Gillespie; Kimberly A Hurvitz
Journal:  MMWR Suppl       Date:  2013-11-22

8.  Improved cardiovascular risk prediction using nonparametric regression and electronic health record data.

Authors:  Edward H Kennedy; Wyndy L Wiitala; Rodney A Hayward; Jeremy B Sussman
Journal:  Med Care       Date:  2013-03       Impact factor: 2.983

9.  Markov decision processes: a tool for sequential decision making under uncertainty.

Authors:  Oguzhan Alagoz; Heather Hsu; Andrew J Schaefer; Mark S Roberts
Journal:  Med Decis Making       Date:  2009-12-31       Impact factor: 2.583

10.  Aspirin, statins, or both drugs for the primary prevention of coronary heart disease events in men: a cost-utility analysis.

Authors:  Michael Pignone; Stephanie Earnshaw; Jeffrey A Tice; Mark J Pletcher
Journal:  Ann Intern Med       Date:  2006-03-07       Impact factor: 25.391

View more
  2 in total

1.  Personalizing Second-Line Type 2 Diabetes Treatment Selection: Combining Network Meta-analysis, Individualized Risk, and Patient Preferences for Unified Decision Support.

Authors:  Sung Eun Choi; Seth A Berkowitz; John S Yudkin; Huseyin Naci; Sanjay Basu
Journal:  Med Decis Making       Date:  2019-02-15       Impact factor: 2.583

Review 2.  A Promising Approach to Optimizing Sequential Treatment Decisions for Depression: Markov Decision Process.

Authors:  Fang Li; Frederike Jörg; Xinyu Li; Talitha Feenstra
Journal:  Pharmacoeconomics       Date:  2022-09-14       Impact factor: 4.558

  2 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.