Assessing performance of closed-loop insulin delivery systems by continuous glucose monitoring: drawbacks and way forward.

BACKGROUND: We investigated whether continuous glucose monitoring (CGM) levels can accurately assess glycemic control while directing closed-loop insulin delivery. SUBJECTS AND METHODS: Data were analyzed retrospectively from 33 subjects with type 1 diabetes who underwent closed-loop and conventional pump therapy on two separate nights. Glycemic control was evaluated by reference plasma glucose and contrasted against three methods based on Navigator (Abbott Diabetes Care, Alameda, CA) CGM levels.
RESULTS: Glucose mean and variability were estimated by unmodified CGM levels with acceptable clinical accuracy. Time when glucose was in target range was overestimated by CGM during closed-loop nights (CGM vs. plasma glucose median [interquartile range], 86% [65-97%] vs. 75% [59-91%]; P=0.04) but not during conventional pump therapy (57% [32-72%] vs. 51% [29-68%]; P=0.82) providing comparable treatment effect (mean [SD], 28% [29%] vs. 23% [21%]; P=0.11). Using the CGM measurement error of 15% derived from plasma glucose-CGM pairs (n=4,254), stochastic interpretation of CGM gave unbiased estimate of time in target during both closed-loop (79% [62-86%] vs. 75% [59-91%]; P=0.24) and conventional pump therapy (54% [33-66%] vs. 51% [29-68%]; P=0.44). Treatment effect (23% [24%] vs. 23% [21%]; P=0.96) and time below target were accurately estimated by stochastic CGM. Recalibrating CGM using reference plasma glucose values taken at the start and end of overnight closed-loop was not superior to stochastic CGM.
CONCLUSIONS: CGM is acceptable to estimate glucose mean and variability, but without adjustment it may overestimate benefit of closed-loop. Stochastic CGM provided unbiased estimate of time when glucose is in target and below target and may be acceptable for assessment of closed-loop in the outpatient setting.