A review of the NG17 recommendations for the use of basal insulin in type 1 diabetes.

AIMS: To revisit the data analysis used to inform National Institute of Health and Care Excellence (NICE) NG17 guidance for initiating basal insulin in adults with type 1 diabetes mellitus (diabetes).
METHODS: We replicated the data, methodology and analysis used by NICE diabetes in the NG17 network meta-analysis (NMA). We expanded this data cohort to a more contemporary data set (extended 2017 NMA) and restricted the studies included to improve the robustness of the data set (restricted 2017 NMA) and in a post hoc analysis, changed the index comparator from neutral protamine Hagedorn (NPH) insulin twice daily to insulin detemir twice daily.
RESULTS: The absolute changes in HbA1c were similar to those reported in the NG17. However, all 95% credible intervals for change in HbA1c point estimates crossed the line of null effect, except for detemir twice daily (in the NICE and extended 2017 NMAs) and NPH four times daily. In the detemir twice-daily centred post hoc analysis, the 95% credible intervals for change in HbA1c crossed the line of null effect for all basal therapies, except NPH.
CONCLUSIONS: In NG17, comparisons of basal insulins were based solely on efficacy of glycaemic control. Many of the trials used in this analysis were treat-to-target, which minimize differences in HbA1c . In the NMAs, statistical significance was severely undermined by the wide credible intervals. Despite these limitations, point estimates of HbA1c were used to rank the insulins and formed the basis of NG17 guidance. This study queries whether such analyses should be used to make specific clinical recommendations.

This study found no significant differences in HbA1c reduction between twice‐daily detemir and modern basal insulin comparators in efficacy trials; the apparent wide variation in HbA1c undermines the statistical robustness and the clinical relevance of the recommendation in the current National Institute of Health and Care Excellence (NICE) guidelines for type 1 diabetes in adults (NG17).

The analyses highlight the importance of the quantity and quality of data used in network meta‐analyses to allow clinically meaningful recommendations.

With the lack of differentiating evidence to support twice‐daily detemir as the basal insulin of choice for type 1 diabetes, selection of basal insulin should be personalized to individual needs.

Introduction

One of the ongoing controversies in the treatment of people with type 1 diabetes mellitus (diabetes) is the choice of regimen and type of basal insulin that should be used to provide optimum glycaemic control. There is also pressure on clinicians to identify the most appropriate therapy, and with respect to basal insulin, this is in an environment that increasingly has multiple therapeutic options for the same clinical indication. The decision on the most appropriate basal insulin requires analysis of a diversity of information on the different insulin analogues and formulations with regards to the assessment of efficacy, value and cost‐effectiveness from a variety of randomized controlled trials (RCTs) and independent studies that differ in design.

A robust process for an objective assessment is essential to compare therapies based on a disparate data set. Such analyses require the use of sophisticated statistical methodology such as Bayesian network meta‐analyses (NMAs) and cost‐effectiveness analysis (for example, using the Core Diabetes Model) 1, 2, 3, 4. NMA combines both direct and indirect evidence for the therapies being compared and allows for the ranking of different therapeutic options based on efficacy.

This study aims to review the available evidence supporting the choice of basal insulin therapy in type 1 diabetes, in the context of recommendations made by the National Institute for Health and Care Excellence (NICE) in 2015 (NG17) 1, 5. In the NICE NG17 NMA, efficacy was defined as the change in HbA1c and rate of severe/major hypoglycaemia. Our study re‐examined the original data set utilizing the same statistical methodology and analysis used in the NG17 NMA. We then determined the impact of updating the NMA with newly available studies that included new, second‐generation, long‐acting, basal insulin analogues, with and without applying more restrictive inclusion criteria for the RCTs.

Methods

Study design

We performed three NMAs following the original analysis, as reported by NICE in NG17 5. The first update used an extended data set to include studies reported up to 2017 (which included an additional therapy, e.g. insulin degludec and insulin glargine 300 U/ml), and the second update involved a re‐analysis of the extended data set but restricting the inclusion criteria for the RCTs. The third NMA, a post hoc analysis, repeated the above‐mentioned NMA updates but used insulin detemir twice daily as the comparator instead of neutral protamine Hagedorn (NPH) insulin twice daily. Through these new analyses, we explored the primary conclusion from the NG17 analyses that recommend insulin detemir twice daily as the standard of care for all adults with type 1 diabetes initiating insulin therapy.

NMA methodology

A systematic literature review was performed in accordance with the protocol utilized in the literature review for the effectiveness of insulin regimens undertaken for NG17 6. The details of the systematic literature review are presented in the PRISMA‐NMA diagram (Fig. 1), and a summary of all the studies included in the systematic literature review is presented in Table S1.

Figure 1

PRISMA flow diagram.

Data on HbA1c reduction and severe/major hypoglycaemia were extracted from papers selected from the literature review. The NMA statistical analysis followed the methodology reported in NG17 Appendix M 5. In summary, theoretical networks were produced for each outcome and each analysis, based on the interventions and comparators in the included trial. Model fit was verified using the Gelman–Rubin R statistic, random‐ or fixed‐effect models were chosen based upon the deviance information criterion, and study heterogeneity was examined with various methods including the I 2 statistic.

The WinBUGS code provided in Appendix M was used to fit Bayesian random effects to the study networks, using Markov chain Monte Carlo estimation 5. This produced estimates of (relative) treatment effects for the difference in change in HbA1c and rate ratios of severe/major hypoglycaemic events, with NPH and insulin detemir twice daily (in the post hoc analysis) as reference treatments. Absolute treatment effects were also estimated.

In NG17, a single‐arm meta‐analysis of the efficacy of NPH twice daily was conducted, providing an estimated treatment effect as a mean reduction in HbA1c of 3 mmol/mol (–0.3%) 5. This estimate was then used as a baseline treatment to provide estimates of absolute effects for the remaining treatments. The same approach was used for the analyses presented here, when NPH and insulin detemir twice daily were used as the comparators. Similarly, single‐arm meta‐analyses of the effect of NPH twice daily were performed on the rate of severe/major hypoglycaemia.

Analyses were also carried out to assess model fit, to compare the fixed‐ and random‐effects models, and to assess study heterogeneity and inconsistency (data not shown).

Original NMA as performed by NICE

The NMA endpoints, change in HbA1c from baseline and rate of severe/major hypoglycaemic events per person‐year (number of events and person‐years of exposure) were defined in Appendix M of the NG17 guideline 5. The treatment effect of severe/major hypoglycaemia was defined as the rate of severe/major hypoglycaemic events calculated from the number of events/episodes per person‐year of follow‐up time 5. For some studies, the person‐years were estimated for each trial arm by dividing the number of major/severe hypoglycaemic events by the rate per person‐year. Where rates were not reported, the person‐years were approximated by the mean follow‐up time multiplied by the sample size 5.

In the base case analysis, a total of 25 RCTs were included for the change in HbA1c network, and 16 RCTs were included for the rate of severe/major hypoglycaemic events network 5.

NMA update: 2017 extended analysis

The first NMA analysis update followed the same methodology as the original NG17 analysis 5 with the following changes. The data source was a literature review that exactly matched the protocol used for the literature review for NG17 6, but updated to include studies from 1 January 2014 to 30 May 2017, as outlined in the PICOS‐T table (Table S2). The search terms were identical to those used in NG17 with the addition of terms to identify studies that included the newer, second‐generation, long‐acting basal insulins 6. The MEDLINE, Embase and Cochrane Library databases were searched as per the NG17 literature review. Study screening and selection was undertaken by two reviewers, with a third resolving any disagreements. Extraction was undertaken by a single reviewer and checked by a second.

Additionally, for the hypoglycaemia analyses, both number of events and person‐years of exposure were required. However, as the data were not consistently available, person‐years were calculated for each RCT by multiplying the intended follow‐up time (in years) by the number of enrolled participants. This estimation ignores participant dropout rates, which may result in inflated person‐years of exposure and, thus, underestimate absolute‐rate estimates. However, by using a common method of calculation, it should increase consistency between RCTs and, therefore, decrease bias in the rate–ratio comparisons. This deviates from the methodology used by NICE, described above, where the person‐years calculations were based on two different methods depending on data availability.

NMA update: 2017 restricted analysis

For the second NMA update, in addition to including RCTs up to 2017, the inclusion criteria were restricted to include RCTs with > 100 participants and a follow‐up of ≥ 6 months.

NMA update: insulin detemir‐centred analysis

In a post hoc analysis, the impact of the NMA was centred on insulin detemir twice daily (as the current standard of care) as opposed to NPH twice daily, as in the original NMA undertaken by NICE for NG17.

Results

The literature search identified 1343 unique records, of which 1302 were excluded at abstract selection. Six papers were included after assessment of the 41 full texts reviewed, as detailed in the PRISMA flow diagram (Fig. 1). Consequently, in the first NMA update, the data set used by NICE was extended to include three additional studies for the HbA1c endpoint 7, 8, 9 and four additional studies for the hypoglycaemia endpoint published up to 2017 7, 8, 10, 11. This allowed the inclusion of large phase 3 trials that were published since the original analysis and added a further comparator insulin regimen into the analysis (insulin glargine 300 U/ml, which was previously excluded by NICE).

For the extended 2017 analysis, in which there were five additional studies, absolute changes in HbA1c were similar to those reported in the NICE NG17 NMA (Table S3). The HbA1c point estimates, relative to NPH twice daily, ranged between +4 mmol/mol (+0.4%) for NPH once daily and –2 mmol/mol (–0.2%) for insulin detemir once/twice daily (Fig. 2). All 95% credible intervals crossed the null line (representing no statistically significant difference), except for insulin detemir twice daily and NPH four times daily (Fig. 2). For the severe/major hypoglycaemia endpoint, the 95% credible intervals all crossed unity and, apart from insulin detemir twice daily, were also very wide and overlapped with each other, such that no distinction could be made between the various insulins.

Figure 2

Extended network meta‐analysis (NMA) 2017: change in (a) HbA1c and (b) severe/major hypoglycaemia relative to neutral protamine Hagedorn (NPH) twice daily. Note: the extended NMA 2017 included randomized controlled trails (RCTs) up to 2017 including insulin glargine 300 U/ml. *Differences were calculated from the median of the posterior distribution for the mean change or mean rate change from the NMA. †Person‐years of exposure were calculated as the number of enrolled participants multiplied by the intended follow‐up time in years for each study to ensure consistency between RCTs; however, this ignores participant dropout rates. For the hypoglycaemia endpoint, NPH four times daily was not included as a network could not be formed with this basal insulin regimen.

NMA update: 2017 restricted analyses

The 2017 NMA focused on the effect of applying more restrictive inclusion criteria, whereby only RCTs with more than 100 participants and with a follow‐up time of 6 months or longer were included. This was to maintain the quality of the data analysed and the robustness of the analysis conclusions. Nine studies for the HbA1c endpoint 12, 13, 14, 15, 16, 17, 18, 19, 20, 21 and nine for the severe/major hypoglycaemia endpoint 12, 13, 14, 21, 22, 23, 24, 25, 26 were excluded for the revised data set. Furthermore, one study for the HbA1c endpoint 9 and two studies for the severe/major hypoglycaemia endpoint 10, 11, included in the extension to 2017, were also excluded.

Application of the more restrictive inclusion criteria also meant that the reference data for insulin detemir twice daily in the NMA became restricted to a single study 27. Review of that study showed that the design was based on treatment escalation, whereby participants were commenced on insulin detemir once daily and, if not controlled, escalated to insulin detemir twice daily 27. As this study had significant potential for introducing heterogeneity, it ideally should have been excluded; however, as this was the only reference point for insulin detemir twice daily in the NMA, the study was retained.

Limiting the RCTs included in the analysis also diminished the beneficial effects in HbA1c observed (based on the HbA1c point estimates) compared with the extended analysis and the NICE NG17 analysis (Table S3). However, the relative NMA outputs for the 2017 restricted analysis (Fig. 3) were very similar to the extended analysis; the HbA1c point estimates, relative to NPH twice daily, ranged between +4 mmol/mol (+0.4%) for NPH once daily and –2 mmol/mol (–0.2%) for insulin detemir once/twice daily. All 95% credible intervals crossed the line of null effect, except for NPH four times daily. Comparisons for the severe/major hypoglycaemia endpoint could not be calculated as data were not available for the comparator, NPH twice daily.

Figure 3

Restricted network meta‐analysis (NMA) 2017: change in HbA1c relative to neutral protamine Hagedorn (NPH) twice daily. Note: The restricted NMA 2017 included randomized controlled trials (up to 2017 including insulin glargine 300 U/ml) with > 100 participants and a follow‐up of ≥ 6 months. *NPH twice daily (the reference in the NICE NG17 guideline) and insulin detemir twice daily could not be included in the hypoglycaemia network; therefore, comparisons could not be calculated. Differences were calculated from the median of the posterior distribution for the mean change from the NMA.

The two previous analyses followed the NICE methodology, using NPH twice daily as the comparator (index) regimen. The NICE recommendation in NG17 defined insulin analogues, and insulin detemir twice daily in particular, as the new standard of care for people with type 1 diabetes 1. As part of the post hoc analysis and, in an attempt to sensitivity check this recommendation, the NMA 2017 extended and restricted analyses were rerun with the index comparator changed to insulin detemir twice daily.

The point estimates for change in HbA1c relative to insulin detemir twice daily for the 2017 extended and restricted data sets are shown in Figs 4 and 5. For both data sets, the 95% credible intervals cross the line of null effect for all therapies, demonstrating no statistical difference between them, except for NPH twice daily (extended data set) and NPH four times daily (extended and restricted data sets).

Figure 4

Twice‐daily insulin detemir‐centred network meta‐analysis (NMA): change in HbA1c relative to insulin detemir twice daily for the 2017 extended data set. Note: The 2017 extended data set included randomized controlled trials up to 2017 including insulin glargine 300 U/ml. *Differences were calculated from the median of the posterior distribution for the mean change from the NMA.

Figure 5

Twice‐daily insulin detemir‐centred network meta‐analysis (NMA): change in HbA1c relative to insulin detemir twice daily for the 2017 restricted data set. Note: The 2017 restricted data set included randomized controlled trials (up to 2017 including insulin glargine 300 U/ml) with > 100 participants and a follow‐up of ≥ 6 months. *Differences were calculated from the median of the posterior distribution for the mean change from the NMA.

Discussion

This study extends the NMA published by NICE in 2015, which is the basis of the NG17 guidance for the treatment of adults with type 1 diabetes 5. Since 2004, NICE has provided formal assessment and guidance on treatments, and its recommendations are a powerful influence on clinical care in the UK and beyond, and are often viewed as defining ‘best practice’. The NICE NG17 series of recommendations included adults with type 1 diabetes who should be offered insulin analogues, with twice‐daily insulin detemir as the basal insulin of choice in a multidose insulin regimen and once‐daily insulin detemir or insulin glargine 100 U/ml as an alternative 1.

This recommendation was consistent with recommendations from the national UK Dose Adjustment for Normal Eating (DAFNE) structured education programme, which recommends twice‐daily basal insulin 28. This approach was supported by observational data from graduates of the DAFNE programme, which found that those on twice‐daily basal regimens had better HbA1c outcomes than those on once‐daily basal insulin 28. However, a specific recommendation for a twice‐daily basal insulin regimen is not a universally accepted clinical practice. For instance, NICE guidance for children adopted a recommendation based on the best choice for the person living with diabetes 29, which clearly includes once‐daily basal analogue insulin.

The NICE guidance based on the NMA conducted in 2014–2015 recommending insulin detemir twice daily as basal insulin therapy for adults with type 1 diabetes 1 did not, however, expand on the important issues of statistical and clinical significance, which are important in the assessment and implementation of the NG17 guidance. The original NMA, conducted by NICE, compared insulin glargine 100 U/ml, insulin degludec, insulin detemir and NPH from a variety of different trials with different administration schedules, with NPH administered twice daily 5. Based on the NMA outputs from that analysis, when the differences in mean change in HbA1c were compared with NPH twice daily, the credible intervals crossed the line of null effect, except for insulin detemir once daily and NPH four times daily, indicating that there was no statistical difference in HbA1c lowering between these insulins and NPH twice daily. In comparing the effectiveness of the different insulins, the point estimates differ and can be ranked; however, it is important to bear in mind the overlapping credible intervals, which occur for all interventions, and that most therapies crossed the line of null effect. Additionally, the hypoglycaemia outcomes showed no differences in the estimated rate ratios of severe or major hypoglycaemia between any of the therapies; all credible intervals crossed the line of unity.

The updated NMAs conducted here sought to investigate if newly reported data would alter the findings. The 2017 extended NMA was performed to determine the effect of adding new studies, which included the new, second‐generation, long‐acting basal insulins, and the 2017 restricted analysis sought to determine the effect of limiting the RCTs to improve data quality. The insulin detemir‐centred NMA (post hoc) was performed to determine the effect of changing the comparator. Insulin detemir twice daily was selected as the alternative index against which other treatments are compared based on its recommendation by NICE in NG17 1. All the NMAs performed presented issues with lack of statistically significant differences in HbA1c and hypoglycaemia endpoints when compared with NPH or insulin detemir twice daily, and wide, overlapping credible intervals, similar to the original NMA.

One of the most important observations is that data from the RCTs related to hypoglycaemia are unreliable and have such statistical uncertainties that they cannot be used for further analysis. This observation has also been made for the NMA that informed the NG17 recommendations, where it is also suggested that the recommendations for insulin detemir rest on the analysis of efficacy (HbA1c lowering) alone 30. With respect to HbA1c lowering, it should be noted that the major insulin RCTs use a treat‐to‐target design. This study design minimizes differences in HbA1c by driving insulin dose to achieve a defined treatment target; this does not reflect general clinical practice and undermines and severely limits these trials in establishing comparative efficacy. In addition, these studies inconsistently report the nature and adjustment of the prandial insulins, the ability of the patient to carbohydrate count and self‐adjust dosing, or the access to and completion of educational programmes. These interventions have a major clinical impact on therapeutic success but are not captured in the NMA analysis, which attributes HbA1c lowering solely to the insulin and insulin regimen described in the source data. This has a major impact on the applicability of these findings to accurately inform clinical practice. The data presented in the original NG17 analysis illustrate that, although there is an apparent difference in the point estimate of HbA1c decrement, the statistical significance of this is severely undermined by the wide credible intervals. Indeed, the only statistically significant result of this analysis is that insulin detemir twice daily is statistically superior to NPH twice daily, and NPH four times daily is statistically inferior.

In this current study, NICE methodology was followed to allow for further investigation of the evidence behind the NG17 recommendation of insulin detemir twice daily. It is important to note that the NG17 recommendation was based on both the NMA and cost‐effectiveness analyses. However, the NMA outputs (relative point estimates and rate ratios) are used as the treatment‐effectiveness inputs for the cost‐effectiveness analysis. Thus, the lack of statistical differences seen in the NMAs also limits the conclusion that can be drawn from cost‐effectiveness analysis. Additionally, the review of studies used for the 2017 restricted NMA showed that there was only one study for insulin detemir that could be included in the NMA 27. This study had a design based on escalation of insulin detemir therapy, which introduced significant bias in the insulin detemir treatment group. Despite these reservations, it was necessary to include this study in the NMA; however, we note that it was also included in the NICE NG17 analysis.

Our study acknowledges the importance of NMA and hierarchical analysis in making comparisons between therapeutic interventions, but it is clear that these techniques require large quantities of high‐quality data to make them robust. Data related to point estimates are particularly dependent on the quality and robustness of the endpoint, and interpretation should include consideration of the range of the credible intervals.

Conclusions

This study presents the results of the original NMA used to inform NG17 guidance for basal insulin use in type 1 diabetes and additional NMA outputs. The analyses show no statistically significant difference between any insulin analogue in relation to the outcome measures used (HbA1c and severe/major hypoglycaemia). Furthermore, important clinically relevant confounders such as prandial insulin use and educational support are not accounted for in this modelling. Although recognizing the hypotheses upon which the NG17 guidance for basal insulin use in type 1 diabetes was based, there was no robust statistical basis for a recommendation for a specific insulin analogue regime for people with type 1 diabetes. There is evidential support that better source data would increase the power of these analyses, and additionally it is appropriate to consider a more uniform view of study design including clinically important confounders, such as that used in phase 3 registration studies, which could allow all studies to be included in subsequent meta‐analyses (and more accurately inform clinical practice).

In view of the lack of statistical evidence of a difference in the key outcomes between basal analogue insulin regimens in the NMA, a cost‐minimization analysis may be more appropriate than a cost‐effectiveness analysis. If all analogue insulins are at a similar cost, while awaiting stronger evidence, the clinical community should be encouraged to take an individualized view of therapy, matching the treatment to the requirements and choices of individuals with type 1 diabetes. This pragmatic approach is used by most clinicians in clinical practice and explains the range of insulin preparations that are used to treat people with type 1 diabetes. The lack of statistical evidence also serves as a reminder that conclusions from meta‐analyses can only reflect the quantity and quality of the data available, and there is a real need to design studies in a manner that allows them to be used in meta‐analyses. The more robust data and statistical methods would better inform guideline recommendations as well as individual therapeutic choices and reassure people with type 1 diabetes of a personalized approach to their therapy.

Funding sources

This study was funded by Sanofi UK.

Competing interests

SB has received research grants (includes principal investigator, collaborator or consultant) from Healthcare and Research Wales (Welsh government) and Novo Nordisk. He has received other research support and honoraria from Boehringer Ingelheim, Eli Lilly, Merck, Novo Nordisk and Sanofi, and has ownership interest in Glycosmedia (diabetes online news service). MFe has received payment for advisory boards and/or lecturing from AstraZeneca, Eli Lilly, NOVO and Sanofi, and research support from AstraZeneca, NOVO and Sanofi. MFi has received payment for advisory boards and / or lecturing from AstraZeneca, Boehringer Ingelheim, Eli Lilly, Mylan, NAPP, Novo Nordisk and Sanofi. JM and NH are employees of York Health Economics Consortium Ltd and consultants for Sanofi. DR‐J has received payment for advisory boards, speaker honoraria and research funding from AstraZeneca, Dexcom, Eli Lilly, Novartis, Novo Nordisk and Sanofi. HS, KCSL and MB are employees of Sanofi. EGW has received personal fees from Abbott Diabetes Care, Dexcom, Eli Lilly, Medtronic, Novo Nordisk and Sanofi Aventis.

Table S1. Included studies and outcomes incorporated in NMA for HbA1c analyses and hypoglycaemia, and risk of bias from new included studies.

Table S2. PICOS‐T table.

Table S3. Absolute change in HbA1c and absolute rate of severe/major hypoglycaemic events according to the NMA with NPH twice daily as the comparator.

Click here for additional data file.

Click here for additional data file.