Improving diabetes prevention with benefit based tailored treatment: risk based reanalysis of Diabetes Prevention Program.

OBJECTIVE: To determine whether some participants in the Diabetes Prevention Program were more or less likely to benefit from metformin or a structured lifestyle modification program.
DESIGN: Post hoc analysis of the Diabetes Prevention Program, a randomized controlled trial.
SETTING: Ambulatory care patients. PARTICIPANTS: 3060 people without diabetes but with evidence of impaired glucose metabolism. INTERVENTION: Intervention groups received metformin or a lifestyle modification program with the goals of weight loss and physical activity. MAIN OUTCOME MEASURE: Development of diabetes, stratified by the risk of developing diabetes according to a diabetes risk prediction model.
RESULTS: Of the 3081 participants with impaired glucose metabolism at baseline, 655 (21%) progressed to diabetes over a median 2.8 years' follow-up. The diabetes risk model had good discrimination (C statistic=0.73) and calibration. Although the lifestyle intervention provided a sixfold greater absolute risk reduction in the highest risk quarter than in the lowest risk quarter, patients in the lowest risk quarter still received substantial benefit (three year absolute risk reduction 4.9% v 28.3% in highest risk quarter; numbers needed to treat of 20.4 and 3.5, respectively). The benefit of metformin, however, was seen almost entirely in patients in the top quarter of risk of diabetes. No benefit was seen in the lowest risk quarter. Participants in the highest risk quarter averaged a 21.4% three year absolute risk reduction (number needed to treat 4.6).
CONCLUSIONS: Patients at high risk of diabetes have substantial variation in their likelihood of receiving benefit from diabetes prevention treatments. Using this knowledge could decrease overtreatment and make prevention of diabetes far more efficient, effective, and patient centered, provided that decision making is based on an accurate risk prediction tool. © Sussman et al 2015.

RCT Entities:   Population Interventions Outcomes

Introduction

The Diabetes Prevention Program was a groundbreaking randomized controlled trial in which the incidence of diabetes was dramatically reduced with a structured lifestyle intervention and significantly reduced with prophylactic metformin in patients at high risk of developing diabetes.1 This study and others like it introduced the prevention of diabetes with metformin and structured lifestyle programs to clinical practice.2 3 The American Diabetes Association now recommends intensive lifestyle interventions or metformin to prevent diabetes for people at high risk.2 However, very few Americans with pre-diabetes being treated with metformin or receiving a structured lifestyle intervention remain.3 Among the factors limiting implementation of diabetes prevention interventions have been the structural difficulties of organizing and financing large scale lifestyle improvement programs and concern that the benefits of preventive metformin treatment might not outweigh its side effects. Understanding which patients receive more or less benefits than average could make treatment both more effective and more efficient.

In many clinical trials, the average benefit reported may be concentrated in a small subset of patients, usually those with the greatest risk of developing the study outcome.4 Because risk of the outcome in untreated patients is a key determinant of the potential for benefit, large clinical trials should routinely conduct a risk stratified assessment of heterogeneity in treatment effect by using a multivariable risk prediction tool to facilitate intelligent tailoring of treatment to individual patients.5 Such information can help us to better personalize the results of clinical trials. By estimating the individual patient’s risk if untreated through use of a risk prediction tool and applying the treatment’s relative risk reduction for people at a similar risk level from a risk stratified trial analysis, we can better estimate a person’s likelihood of benefit in clinical practice.4 5 This is a form of personalized medicine often referred to as benefit based tailored treatment.6

We hypothesized that a risk stratified analysis of the Diabetes Prevention Program could identify people with much higher or much lower than average benefit from interventions to prevent diabetes and that this information could inform better clinical decisions using the principles of benefit based tailored treatment.5 6 7

Methods

The Diabetes Prevention Program

The design, rationale, outcomes, and loss to follow-up of the Diabetes Prevention Program have been described in detail elsewhere.1 8 In brief, the final Diabetes Prevention Program consisted of 3234 patients. Data from the Diabetes Prevention Program were made available through a National Institutes of Diabetes and Digestive and Kidney Diseases repository. Owing to some local institutional review boards’ decisions not to distribute data, our dataset had 3081 participants (95% of full population). All participants had a body mass index of 24 or higher (22 or higher in Asians) and a fasting plasma glucose concentration of 95 to 125 mg/dL (impaired fasting glucose) and a concentration of 140 to 199 mg/dL two hours after a 75 g oral glucose load (impaired glucose tolerance). These criteria were designed to identify patients who were at high risk for developing diabetes, but they are not the same as the American Diabetes Association’s diagnostic criteria for pre-diabetes, which use a cut-off point of 100-125 mg/dL for impaired fasting glucose and do not have a body mass index requirement.2 Eligible patients gave consent and were randomized to standard lifestyle recommendations plus 850 mg of metformin twice daily, an intensive program of lifestyle modification that included 16 lessons with a case manager, or standard lifestyle recommendations plus placebo twice daily. After a median follow-up period of 2.8 (range 1.8-4.6) years, progression to diabetes was reduced by 58% (95% confidence interval 47% to 66%) in the lifestyle modification arm and 31% (17% to 43%) in the metformin arm, both compared with the placebo arm.1

Diabetes prediction model development

Following an established methodology and a pre-specified analytic plan,5 we developed a diabetes risk prediction model based on well established risk factors for diabetes. Following this approach, we developed an internal model using proportional hazards regression to predict risk for progression to diabetes in the Diabetes Prevention Program population. Other research has shown that development of internal models directly on a complete trial population can be done without creating any bias in the estimation of heterogeneity of treatment effect as long as standard processes are used to prevent “over-fitting.” In this technique, the risk model is developed on the entire population of the study. The effect of the treatment arm is added in a second step, along with an interaction term between estimated risk and treatment arm.9

Our model started with 17 potential baseline risk factors that we chose a priori on the basis of their having been found predictive in at least three previous diabetes risk models assessed in a high quality review article10: fasting blood sugar, hemoglobin A1c, age, body mass index (in kilograms per meter of height squared), waist:hip ratio, waist circumference, height, triglycerides, high density lipoprotein cholesterol, systolic blood pressure, and physical activity metabolic equivalent as linear variables; and sex, race (white, black, non-black Latino, and other), family history of diabetes, self reported history of hypertension, self reported history of high blood glucose (including “borderline” high glucose or only during pregnancy), smoking status, and assigned study arm (table 1). (Repeat analyses considering high glucose during pregnancy as not being a high risk condition did not change the results.) We used continuous variables in their original scale. We determined selection of risk factors for inclusion in the final multivariable prediction model by backwards selection, excluding variables that had P>0.1 in the multivariate model. The model had more than 30 outcomes per independent variable considered, making over-fitting very unlikely.11 We used SAS version 9.3 for data management and regression model building. We used the “rms” package in R software version 3.0.17 to perform bootstrapped internal validation. Once the model was developed, we excluded the 17 (0.6%) patients with missing data in the selected predictors from the analysis.

Table 1

 Characteristics of participants. Values are numbers (percentages) unless stated otherwise

Characteristics	Overall (n=3081)	Placebo (n=1030)	Metformin (n=1027)	Lifestyle (n=1024)
Overall diabetes cases	655 (21.3)	292 (28.4)	215 (20.9)	148 (14.5)
Mean (SD) age, years	50.6 (9.0)	50.4 (8.8)	50.8 (8.9)	50.6 (9.3)
Female sex	2053 (66.6)	699 (67.9)	669 (65.1)	685 (66.9)
Race/ethnicity:
White	1768 (57.4)	586 (56.9)	602 (58.6)	580 (56.6)
Black	644 (20.9)	219 (21.3)	221 (21.5)	204 (19.9)
Hispanic	508 (16.5)	168 (16.3)	162 (15.8)	178 (17.4)
Other non-white	161 (5.2)	57 (5.5)	42 (4.1)	62 (6.1)
Family history of diabetes	2127 (69.0)	715 (69.4)	699 (68.1)	713 (69.6)
Current smoker	216 (7)	80 (7.8)	69 (6.7)	67 (6.5)
Diagnosis of hypertension	835 (27.1)	282 (27.4)	267 (26.0)	286 (27.9)
History of hyperglycemia	614 (19.9)	216 (21.0)	192 (18.7)	206 (20.1)
Mean (SD) fasting plasma glucose, mg/dL	107.2 (7.7)	107.4 (7.8)	107.3 (7.9)	107 (7.5)
Mean (SD) hemoglobin A1c, %	5.9 (0.5)	5.9 (0.5)	5.9 (0.5)	5.9 (0.5)
Mean (SD) body mass index	33.5 (5.8)	33.7 (5.9)	33.5 (5.8)	33.4 (5.7)
Mean (SD) systolic blood pressure, mm Hg	124.2 (14.7)	123.8 (14.4)	124.5 (14.9)	124.2 (14.8)
Mean (SD) triglycerides, mg/dL	162.9 (93.5)	167.2 (92.6)	158.4 (90.3)	163 (97.2)
Mean (SD) high density lipoprotein cholesterol, mg/dL	45.6 (11.8)	44.7 (11.4)	46 (11.5)	46.1 (12.5)
Mean (SD) leisure physical activity, MET-hour/week of exercise*	15.9 (25.4)	16.5 (28.9)	16.1 (25.2)	15.2 (21.6)
Mean (SD) height, cm	166.8 (9.2)	166.5 (9.2)	167 (9.2)	166.8 (9.2)
Mean (SD) waist circumference, cm	105.0 (14.6)	105.1 (14.4)	104.8 (14.4)	105.2 (14.9)
Mean (SD) waist:hip ratio	0.9 (0.1)	0.9 (0.1)	0.9 (0.1)	0.9 (0.1)

*MET denotes metabolic equivalent; MET-hours represent average amount of time engaged in specified physical activities multiplied by MET value of each activity.

We divided the trial populations into quarters of pre-intervention risk on the basis of model predictions and assessed them for discrimination, calibration, Pearson’s median skewness coefficient, and the median:mean risk ratio. The last two of these both assess the amount of skew in a risk score (that is, the amount of “tail”). Pearson’s median skewness coefficient uses a normalized value calculated as 3×(mean−median)/standard deviation. The median:mean risk ratio is a simpler measure—the median risk divided by the mean. It is meant to capture in an intuitive way the risk of the typical patient relative to the average risk in the population. We also used the extreme quarter risk ratio to assess the ability of the risk prediction tool to predict a wide variation of risks. This value has particular benefit in this analysis because patients at extreme predicted probabilities of developing diabetes should have a more straightforward decision about the benefit of treatment.

Assessing heterogeneity of treatment effect

We then analyzed the absolute risk reductions for the interventions after stratification by risk of diabetes.12 We divided all three trial groups into quarters by pre-treatment risk and analyzed them independently. As no pre-specified factors strongly predict treatment-by-risk factor interactions in diabetes prevention, we did not test additional interaction effects as per recommended standards.5

We also did a net benefit assessment on these data.13 A net benefit assessment compares the effects on overall treatment benefit of treating all patients with those of treating smaller, targeted groups of patients at high risk. In the net benefit assessment, we assume that patients have a “cut-off point” below which mildly effective treatments are not worthwhile. The analysis then adjusts for minimum necessary benefit to understand the value of a treatment across the population. This cut-off point represents an estimate of the totality of benefits and harms from treatment. Benefits above the cut-off point represent net benefit. Because of its inherent subjectivity, we assessed a wide range of possible cut-off points (see web appendix for details).

External validation

As an internally developed model will generally have excellent calibration for the population on which it was developed and may also have overly optimistic discrimination, we also repeated the analysis by using the Framingham diabetes risk model.14 The Framingham model was created on a very different population (most notably without the requirement that all patients be at high risk of developing diabetes) and was intended to predict diabetes over seven years. To recognize this, we first evaluated the Framingham score as published with an adjustment for length of follow-up. Because the Framingham results were poorly calibrated to our population (as has often been seen for diabetes prediction models),15 we re-calibrated the score to the trial population by adjustment of both the intercept and the overall slope. The details of this analysis are in the web appendix.

Results

As previously described, the patient population had a high risk of progression to diabetes (table 1).1 In our sample, the study was 21% African-American and 16.5% Latino. The average fasting plasma glucose was 107 mg/dL, and 69.1% of participants had a family history of diabetes.

Assessment of risk models

The internal risk model had seven predictor variables (table 2). When we used this model, a patient’s baseline fasting plasma glucose was by far the most important predictor of development of diabetes. A self reported history of high blood glucose was also quite influential, and the patient’s hemoglobin A1c was predictive independently of the other two factors.

Table 2

 Comparison of prediction models

Variable	Internal model (hazard ratio)	Framingham model (odds ratio)*
Age, years	—	0.99
Sex, male	—	0.65
Fasting plasma glucose, per 10 mg/dL	1.98	1.15
History of high blood glucose	1.67	—
Hemoglobin A1c, %	1.93
Parental history of diabetes	—	1.55
Height, cm	0.985	—
Waist:hip ratio, per 0.1 unit	1.173	—
Waist circumference, per 10 cm	1.11	1.05
Body mass index	—	1.04
Systolic blood pressure, mm Hg	—	1.01
Triglycerides, per 10 mg/dL	1.02	1.00
High density lipoprotein cholesterol, mg/dL	—	0.96
Intercept	—	−18.607
3 year baseline survival†	0.999998	—
C statistic	0.73	0.69
Mean:median risk ratio	0.72	1.09
Pearson’s median skewness coefficient	1.05	−0.94
Extreme quarter risk ratio	6.48	2.18

*Odds ratios for Framingham model continuous variables are per 1 unit increase.

†Can be used to calculate person’s risk predictions on basis of hazard ratios; practitioners can use nomogram in web appendix.

The internally developed model had a C statistic of 0.73 and excellent calibration (fig 1). People classified in the lowest quarter of risk had between 1.1% and 9.5% three year predicted probability of developing diabetes (mean 6.9%), and those classified in the highest quarter had between 27% and 99% predicted probability (mean 45%) (table 3). This means that the highest risk quarter had a risk that was 6.5 times greater than the risk in the lowest quarter (extreme quarter risk ratio=6.5) (table 2). The risk predictions were strongly skewed to the right, with a mean three year progression rate to diabetes of 21%, but the median patient had only a 15% probability (median:mean risk ratio=0.72) (table 2).

Fig 1 Calibration plot: black dots represent deciles of risk

Table 3

 Three year Diabetes Prevention Program outcome events and Kaplan-Meier event rates by treatment and risk quarter

Characteristics	Quarter 1 (n=766)	Quarter 2 (n=766)	Quarter 3 (n=766)	Quarter 4 (n=766)	All (n=3064)
Predicted probability, range	1.1-9.5%	9.5-15.1%	15.1-27.0%	27.0-99.8%	—
Predicted probability, mean	6.9%	12.0%	20.1%	44.6%	20.9%
Observed rate, mean (events)	7.1% (46)	13.7% (88)	23.4% (155)	42.9% (297)	21.9% (586)
Placebo rate, mean (events)	8.3% (19)	17.8% (37)	29.1% (73)	59.6% (140)	29.1% (269)
Metformin rate, mean (events)	9.6% (21)	14.9% (32)	24.4% (53)	38.2% (86)	21.9% (192)
Lifestyle rate, mean (events)	3.4% (6)	8.6% (19)	15.5% (29)	31.3% (71)	14.8% (125)
Hazard ratio, metformin (95% CI) (versus placebo)	1.07 (0.57 to 2.01)	0.79 (0.49 to 1.28)	0.82 (0.57 to 1.18)	0.44 (0.33 to 0.59)	0.67 (0.55 to 0.81)
Hazard ratio, lifestyle intervention (95% CI) (versus placebo)	0.30 (0.12 to 0.75)	0.45 (0.26 to 0.79)	0.43 (0.28 to 0.67)	0.34 (0.25 to 0.46)	0.42 (0.34 to 0.52)

Heterogeneity of treatment benefit

On average, patients in the metformin arm had a substantial average overall reduction in incidence of diabetes, but this benefit was almost entirely a result of those at the highest risk experiencing a dramatic relative risk reduction (as measured by the hazard ratio) and absolute risk reduction (table 3; fig 2). For the lowest quarter of risk of progression to diabetes, the metformin arm had a slightly higher risk of developing diabetes (9.6%) than did the control arm (8.3%). The hazard ratio was non-significant (1.07, 95% confidence interval 0.57 to 2.01). Even in the third quarter (second to highest risk of developing diabetes), patients in the metformin arm only had a 4.7% observed absolute risk reduction (number needed to treat (NNT)=22; hazard ratio 0.82, 0.57 to 1.18). However, in the quarter with the highest risk, patients in the control group had a 59.6% observed rate of developing diabetes and the metformin group had an absolute risk reduction of 21.4% (NNT=4.6; hazard ratio 0.44, 0.33 to 0.59). A significant interaction existed, in which patients with greater risk of diabetes incidence had a greater relative risk reduction from metformin use (P<0.001).

Fig 2 Efficacy plots

Substantial variation in absolute risk reduction also occurred in the lifestyle arm (fig 2). Patients in the lowest quarter of predicted risk had a 4.9% absolute risk reduction (NNT=20.4]), and the absolute risk reduction in the highest risk quarter was 28.3% (NNT=3.5). These findings were entirely due to variation in participants’ baseline risks, as no significant variation was seen in the hazard ratio in the lifestyle arm. The overall population had a hazard ratio of 0.42 compared with placebo. Patients in the lowest risk quarter had an average hazard ratio of 0.30 (0.12 to 0.75), and those in the highest risk had a hazard ratio of 0.34 (0.25 to 0.46). We found no interaction between risk of diabetes incidence and relative risk reduction due to the lifestyle intervention (P=0.16).

Results of sensitivity analysis using Framingham risk score

We examined our results by using an externally developed diabetes prediction model from the Framingham cohort and a recalibrated version of the same model.14 This model had similar discrimination to the internal model, with a C statistic of 0.69. However, after adjustment for length of follow-up, the Framingham model would have predicted that 33.3% of patients in the placebo group would develop diabetes; only 26.2% did. Recalibration (re-estimation of the intercept and slope of the model) resolved this problem.16 Our results were not substantively changed by use of the externally developed Framingham risk tool (details are in the web appendix).

The net benefit assessment found that a tailored approach using risk prediction was substantially more effective than treating everyone in the trial, particularly when the doctor’s or patient’s decision threshold was in the range of 0.05 to 0.2 for metformin and 0.1 to 0.2 for the lifestyle intervention. These values mean that below these thresholds all patients benefit from treatment. For example, if a patient sees no net negatives to taking metformin (including cost or side effects), then the treatment would be worthwhile for the entire population. Above these cut-off points, nobody benefits. For example, if a patient were to want metformin only if the absolute probability that it will prevent diabetes is greater than 20%, then nobody should take it. (Further details are in the web appendix.)

Discussion

In this study, we used well established clinical risk factors to identify which patients at elevated risk of diabetes are the most likely to have incident diabetes prevented by lifestyle modification and metformin. We found that average reported benefit for metformin was distributed very unevenly across the study population, with the quarter of patients at the highest risk for developing diabetes receiving a dramatic benefit (21.5% absolute reduction in diabetes over three years of treatment) but the remainder of the study population receiving modest or no benefit. Although patients at lower risk also received a much smaller absolute reduction from the lifestyle intervention than did higher risk patients, the relative effects of the lifestyle intervention were fairly constant across risk groups (hazard ratio 0.42). These results could decrease drug overuse, help to prioritize lifestyle programs, and be a model for the secondary analysis of randomized trials.

Comparison with other studies

Our risk based analysis is more effective than the conventional “one variable at a time” subgroup analysis, which is plagued by false negative results from poor statistical power and by false positives from multiple comparisons.12 17 In contrast, recent research has shown how risk based examination of treatment heterogeneity examines a critically important clinical question in a single comparison with much better statistical power.5 12 The Diabetes Prevention Program has previously been shown to have subgroup-by-treatment interactions. However, even when a treatment has a consistent relative risk reduction across risk levels, as we found for the Diabetes Prevention Program’s lifestyle intervention, the technique used in this study allows clinicians to recognize large variation in a treatment’s absolute risk reduction between patients on the basis of multivariate risk prediction. The presence (or absence) of a treatment interaction on a proportional scale does not give any direct information about the clinically most important measure of benefit and which patients to treat or not treat.

Our study differs from other analyses of the Diabetes Prevention Program by the method, which we believe should be routinely applied to large randomized trials, and the implications of our findings for the prevention of diabetes. We and others have proposed that a risk stratified analysis such as this one should be a critically important component of the primary analysis of all large clinical trials for three reasons: it provides clinicians and patients with important information on the distribution of the study population’s risk of the outcome; combining risk factors into a single risk score greatly improves statistical power for detecting whether the treatment’s relative benefit (such as hazard ratio or relative risk reduction) is similar in study participants at lower versus higher risk; and it provides clinicians and patients with the ability to better estimate the individual patient’s chance of benefiting from a treatment. This benefit can be estimated by multiplying the relative benefit by the best available estimate of the patient’s risk for the outcome if not treated.

Clinical and research implications

Each of the benefits of a risk stratified analysis is apparent from our analysis of the Diabetes Prevention Program. Firstly, our results show a wide and highly skewed distribution of risk for diabetes within the Diabetes Prevention Program study population, ranging from participants with a 1-2% risk of developing diabetes in the next three years to those with a greater than 90% probability of developing diabetes. Secondly, the risk stratified results showed that the relative benefit (hazard ratio) of the lifestyle intervention was consistent across the population but that the relative benefit for metformin was significantly greater for the highest risk quarter (hazard ratio 0.44) than it was for the remaining 75% of the study population (hazard ratio ranged from 0.79 to 1.07). Most importantly, these results allow for much better estimates of an individual patient’s likelihood of benefiting from intervention (fig 2).

Implementation of these results could make prevention of diabetes much more efficient while lowering rates of adverse events. In fact, large risk based variation in treatment benefit with metformin suggests that outcomes nearly equivalent to treating the entire Diabetes Prevention Program cohort can be achieved by treating only the quarter of the patients at highest risk. As metformin has substantial gastrointestinal side effects and may increase rates of lactic acidosis,18 justifying its use in people who are at lower risk might be difficult.

Our study’s relevance for lifestyle interventions is different from that for metformin. Unlike metformin, most of the lifestyle intervention’s “side effects” are likely to be beneficial, including improved mental health, reduced cardiac risk, enhanced quality of life, and the cosmetic effects of weight loss and improved fitness. Unfortunately, few large scale programs for providing the lifestyle support used in the Diabetes Prevention Program exist, and they are expensive.18 19 Our results suggest that these programs’ cost effectiveness will vary dramatically on the basis of a patient’s risk of developing diabetes, but it remains possible that a lifestyle intervention is cost effective for other reasons.19

This study also showed that progression to diabetes could be effectively predicted with multivariable models. We primarily based our analysis on a diabetes prediction model that was developed using Diabetes Prevention Program data and established risk factors, using an approach that minimizes over-fitting.11 The risk score itself had novel findings. We found that fasting plasma glucose was the strongest predictor for developing diabetes, but that hemoglobin A1c and self reported history of hyperglycemia were also independent risk factors. Waist circumference, height, and waist:hip ratio were additional independent predictors of diabetes. We hypothesize that the differences between these variables and other existing risk prediction models may be due to the very high risk of the Diabetes Prevention Program cohort, the short term follow-up, and the close measurement of more variables than other studies examined. Our risk prediction tool could be used by programming it into an electronic health record, by developing an app, or by using the nomogram provided in the web appendix, similar to what has been done with existing risk tools.14

Our results also highlight the importance of having an accurate risk prediction tool in implementing benefit based tailored treatment. We found that the Framingham score had good discrimination in the Diabetes Prevention Program population but suboptimal calibration (that is, it over-estimated study participants’ risk by about 25%). It is very common for a prediction model to have good discrimination but poor calibration in cohorts that are dissimilar to the population in which they were developed.15 20

A reliable estimate of a patient’s risk if not treated, however, is needed for all good clinical decision making, not just for benefit based tailored treatment. Regardless, following the benefit based tailored treatment strategy will still outperform the “treat all” strategy even using the miscalibrated Framingham model. Appropriate selection of patients would improve with a better risk tool.

Strengths and limitations of study

The primary strength of this study is the high quality, randomized evidence of the Diabetes Prevention Program. Another strength is the analytic technique used. Our primary technique to assess the heterogeneity of treatment effect provides substantial new insight in addition to the analyses used in most clinical trials. This analysis does not have the statistical biases that are found in observational studies, and it minimizes the risks of spurious false positives in approaches based on serial “one variable at a time” subgroup analysis.5 6 12 21

The inconsistent calibration of risk scores is a limitation of this study. The internally derived tool we used was pre-specified but not externally validated. The poor calibration of most diabetes risk tools implies that clinicians should use a risk prediction tool developed in or recalibrated on a patient population similar to their own.15 Ideally, local health systems will test and recalibrate established risk tools in their populations.22 The rise of electronic health records and accountable care organizations should make such internally developed risk tools more practical. Existing examples include risk tools developed in the Kaiser system and those being developed by the US Department of Veterans Affairs.22 23 Furthermore, clinicians often have additional information not accounted for by the risk tool, such as the patient’s personal dislike of drugs or the components of a lifestyle intervention, the severity of a patient’s family history, or the patient’s frailty and comorbidities. The results of the benefit based tailored treatment analysis can help to guide decisions, but good clinicians should also consider other factors.24

Furthermore, although we believe that this type of analysis should be performed regularly, this particular study was not planned until well after the Diabetes Prevention Program completed. Ideally, the relations between risk and estimated benefit would be verified in future research.

This study also has all the limitations of the Diabetes Prevention Program randomized trial from which we obtained the data. Among these limitations are the short term follow-up of the study and that the study showed a reduction in diagnosis of diabetes but was not powered to see a difference in complications of diabetes or the long term stability of the reduction in diabetes. Also, these effects may merely be short term benefits of being on treatment, although the benefit of metformin was seen to continue at least through a two week washout period.25 Finally, our study examines the role of metformin and a lifestyle intervention in prevention of diabetes, but both of these interventions have broader clinical benefits.

Conclusions and policy implications

This study has implications for the clinical prevention of diabetes, for diabetes prevention policy, and for the standard interpretation of randomized trials. We have shown that among patients with a high risk for diabetes, those at the highest risk should be treated most aggressively. Even among patients in the Diabetes Prevention Program, however, most were unlikely to develop diabetes over 2.8 years. Through better risk targeting, lifestyle interventions could become dramatically more efficient and the side effects of metformin could be limited while retaining the effectiveness of the programs.

In conclusion, the Diabetes Prevention Program proved that a lifestyle intervention and metformin can prevent diabetes in some patients with pre-diabetes. Our results show how widely that benefit varies between individual patients with pre-diabetes as a function of their risk of developing diabetes in the near future. Understanding these dramatic variations can enhance our ability to use these interventions more effectively and efficiently by tailoring our decisions about treatment to individual patients’ circumstances and preferences.

The Diabetes Prevention Program randomized controlled trial showed that metformin or a structured lifestyle intervention could reduce transition to diabetes in a population of patients at high risk of diabetes

Most patients in the Diabetes Prevention Program did not develop diabetes

Progression to diabetes can be predicted with risk prediction scores

The benefit of metformin in preventing diabetes is focused almost entirely in people with an extremely high risk of developing diabetes

Patients at high risk of developing diabetes had substantially greater benefit from preventive treatment than did those at lower risk

Preventive treatments could be used far more efficiently with risk stratified analyses of randomized trials