Performance of the FreeStyle Libre Flash glucose monitoring system in patients with type 1 and 2 diabetes mellitus.

OBJECTIVE: To evaluate the performance of the FreeStyle Libre Flash continuous glucose monitoring (FSL-CGM) system against established central laboratory methods. RESEARCH DESIGN AND METHODS: 20 subjects (8 type 1 diabetes mellitus, 12 type 2 diabetes mellitus) were analyzed. FSL-CGM sensor measurements (inserted in arm and abdomen) were compared with capillary blood glucose results analyzed with StatStrip as semigold standard. The glucose response after a standardized oral glucose load was measured by FSL-CGM and capillary samples analyzed by perchloric acid hexokinase (PCA-HK) method, StatStrip and FSL test strip (FSLC), and a commonly used CGM system (iPro2).
RESULTS: FSL-CGM arm sensor readings showed 85.5% of paired readings falling within Clarke Error Grid (ISO 15197:2013) zone A when compared with StatStrip. For FSL-CGM abdomen and FSLC, these percentages were 64% and 98%, respectively. The overall correlation of FSL-CGM in the arm and the StatStrip indicates a performance with lower results with the FSL-CGM in the arm than expected based on the StatStrip in the lower glucose ranges, and higher results than expected in the higher ranges. Following a standardized glucose load, a slower rise in glucose level was observed for FSL-CGM arm as compared with PCA-HK, StatStrip, FSLC, and iPro2 during the first 45-60 min after glucose load ingestion.
CONCLUSIONS: Certain matters need attention while using the FSL-CGM in daily life including the observed lower values in the lower ranges, and the underestimation of the effect of a meal on glucose response. These effects of such deviations can partly be overcome by optimizing the available user instructions. TRIAL REGISTRATION NUMBER: TC5348; results.

FreeStyle Libre Flash continuous glucose monitoring (FSL-CGM) is an alternative approach to glucose monitoring, more or less comparable to continuous glucose monitoring; independent validation testing has not been performed yet. Its accuracy is inadequately studied.

Accuracy of FSL-CGM is moderate in comparison to standard glucose measurement techniques, showing lower than actual results in the lower ranges, and higher than actual results in the higher ranges; after standardized glucose load glucose measurement results lag in comparison with other glucose measurement techniques. In general, users are satisfied.

Results of the FSL-CGM should always be considered in clinical context; when in doubt, use another glucose measurement technique to confirm or refute the reported FSL-CGM glucose concentration.

Introduction

Self-monitoring of blood glucose levels requires intermittent capillary blood sampling and a blood glucose measurement device. However, many patients experience barriers to frequent testing, among others including the pain and discomfort associated with the finger-stick blood samples along with accumulated trauma to the fingers. Also, intermittent blood glucose monitoring through intermittent capillary blood sampling provides only snapshots of glucose concentrations.

Another important point of attention is the accuracy of the strips used to measure capillary blood glucose concentrations. International standards have agreed on several criteria with regards to accuracy: an accuracy of ±15% for glucose levels ≥100 mg/dL and ±15 mg/dL for glucose levels <100 mg/dL of the actual blood glucose level are the most relevant criteria from a patient point of view,1 since hypoglycemias do have the most immediate impact on patient's well-being and the degree of hypoglycemia does influence the measures needed to counteract the hypoglycemia.

In the past 15 years, continuous glucose monitoring (CGM) systems have become available.2 These CGM systems measure interstitial fluid glucose levels at rather closely spaced intervals to provide semicontinuous information on glucose levels, allowing identification and signaling of glucose level fluctuations to a degree that cannot be obtained with intermittent capillary blood glucose measurements. While improved glycemic control has been demonstrated with the use of CGM systems,3–5 CGM accuracy also remains a challenge; most of the available systems need calibrating at least twice daily to allow a sufficiently reliable correlation between interstitial and capillary glucose results. Nevertheless, with the advances in the development of highly accurate and easy to use CGM systems, the ultimate use of ‘artificial pancreas’ moves closer to become a reality.6

Recently, a different variety of continuous glucose monitoring (CGM; FreeStyle Libre Flash (FSL-CGM), Abbot Diabetes Care, Alameda, California, USA) for interstitial glucose fluid monitoring has been introduced in Europe that is compact, lightweight, has a 2-week period of use, and according to the producer does not require calibration by the user (factory-calibrated). This flash glucose monitoring system measures interstitial glucose via disposable electronics and a subcutaneous sensor, with a button-like structure firmly adhering to the skin to allow the inserted sensor to stay in place. The sensor is put in place by a single-use applicator, and automatically measures glucose every minute for up to 14 days. Scanning of the sensor by a separate reader collects the glucose measurements and trend at the moment of scanning plus up to 8 hours of prior readings every 15 min. The reader used for FSL-CGM also supports glucose and ketone capillary blood measurements using FreeStyle Precision glucose/ketone strips. In principle, the glucose sensing technique is based on the technique of the FreeStyle Navigator, which has been shown to be a reliable CGM measurement technique.7–9

Whenever a new CGM device becomes available, it is essential to critically evaluate its accuracy and usability. Independent accuracy assessments of the FSL-CGM are scarce; previous assessments were performed comparing the FSL-CGM to capillary blood evaluated with a point of care (POC) measurement.10 Whether such a comparison can be seen as sufficient accurate remains to be seen, however. As already alluded, capillary measurements are allowed to deviate for a maximum of 15% for glucose levels ≥100 mg/dL and ±15 mg/dL for glucose levels <100 mg/dL.1 Furthermore, glucose measurements with POC devices can be highly inaccurate, especially in critically ill patients.11–13 Therefore, the use of an appropriate reference method is a key factor when assessing the quality of POC glucose device accuracy studies.14

The present study was designed to assess the accuracy and usability of the FSL-CGM by comparing its scanned sensor results with various standardized reference methods in a subjects with type 1 diabetes mellitus (T1DM), type 2 diabetes mellitus (T2DM) and subjects without diabetes; this article presents the results in the diabetes population.

Research design and methods

Study design

This study had a prospective design. Inclusion and study procedures took place at the Department of Internal Medicine of the Isala hospital (Zwolle, the Netherlands) in the period between 30 August 2015 to 29 September 2015. Primary aim of this study was to establish the accuracy of the FSL-CGM against an established central laboratory technique.

The study was performed according to Good Clinical Practice and the Principles of the Declaration of Helsinki. The Ethics Review Committee of the hospital approved the protocol and written informed consent was obtained from every participant prior to enrolment. The study protocol was registered prior to the start of the study (http://www.trialregister.nl (TC 5348)).

Patients

Persons with T1DM, T2DM, and persons without DM between 18 and 75 years of age who were able to undergo the requested study procedures were eligible for study participation. Main exclusion criteria were the inability to understand the Dutch language and the presence of a severe or unstable medical condition. In this article only the results in persons with T1DM and T2DM are described.

Study procedures

The overall study duration for each participant was 14 days. After obtaining informed consent, baseline characteristics were collected using a standardized case record form during the first study visit. Additionally, one FSL-CGM sensor was inserted on the back of the upper arm and one in the abdominal wall. In addition, a retrospectively calibrated blinded CGM (iPro2 Professional CGM; Medtronic, Northridge, Pennsylvania, USA) device was inserted in the abdominal wall, not to be seen as a comparison as gold standard, but to allow comparing two CGM systems. The FSL-CGMs remained in situ for 14 days and the blinded CGM for 7 days. Patients were instructed to perform a total of four capillary self-measurements of blood glucose using the FSLC, the FSL-CGM test strip and the StatStrip Xpress monitoring system (Nova Biomedical, Waltham, Massachusetts, USA). In our facility, the StatStrip POC glucose measurement systems (Nova Biomedical, Waltham, Massachusetts, USA) have been traced and aligned to the highest level order of methodology: isotope dilution gas chromatography, mass spectrometry.15 The preferred testing sequence was on waking, before lunch, before dinner and at bedtime. Following each test, subjects were asked to report the blood glucose results in a diary along with the glucose results of the FSL-CGM scans (using separate reading devices for the arm and abdominal wall FSL-CGM). It was recommended to confirm the glucose value with a capillary measurement in case of (imminent and/or suspected) hypoglycemia, glucose levels changing rapidly, or when symptoms did not match the systems reading.

All device-related procedures were performed by one trained investigator (MJF).

The second study visit was performed during the 14-day study period, at least 1 day after insertion of the FSL-CGMs and the iPro2 device. During this 4-hour in-clinic visit, an oral glucose challenge test with repeated blood sampling was performed. Capillary blood samples were taken at 0, 15, 30, 45, 60, 75 and 90 min after ingestion of a 75 g oral glucose load. Capillary blood glucose tests during the glucose challenge test were performed within a 2 min time frame in relation to the FSL-CGM readings, and the iPro2 readings. Capillary blood analyses were performed using StatStrip, FSL test strip and the reference method: perchloric acid hexokinase (PCA-HK) method (Roche Modular P800, Roche Diagnostics GmbH, Mannheim, Germany).16 Body composition was measured and fat percentage estimated by using an Omron BF306.

Outcomes

Primary outcome was the accuracy of the FSL-CGM in the arm against StatStrip measurements during the 14-day study period and the results of the glucose load testing. Correlation between FSL-CGM and StatStrip was calculated using Clarke Error Grid analysis for the combined T1DM and T2DM group.

The readings of the FSL-CGM in the arm after the 75 g glucose load were compared with the FSL test strip, the StatStrip, and the PCA-HK method as comparison with capillary blood glucose measurements, and with the iPro2 as a comparison with another CGM device. The iPro2 was retrospectively calibrated (1–3 hours) before starting the glucose load test. No data from the glucose load test were used for calibration of the iPro2.

The test results of FSL-CGM in the abdominal wall were assessed in an identical way, to assess the degree of accuracy of the FSL-CGM device on an alternative site.

Statistical methods and ethical considerations

Results are expressed as mean (with SD) or median (with IQR) for normally distributed and non-normally distributed data, respectively. A significance level of 5% (two-sided) was used. Normality was examined with Q-Q plots.

Measurements were also assessed through mean absolute differences (MAD) and mean absolute relative differences (MARD) for all ranges.

For the glucose challenge test, repeated measures mixed-model analyses were applied to assess MARD between FSL-CGM and PCA-HK, with subjects as a random effect and type of diabetes as fixed effect. Wilcoxon signed-rank test were applied for paired differences between methods at the different time points, that is, 0, 15, 30, 45, 60, 75 and 90 min. Outcomes from FSL-CGM and capillary blood glucose readings were superimposed on the error grids as described by Clarke et al.17 The point accuracy of sensor-based glucose values versus finger stick blood glucose was determined as per cent within consensus error grid zone A. Bootstrap analyses were performed to determine 95% CIs around the percentage of values within zone A. Values in zones A and B are deemed clinically acceptable, whereas those in zones C, D and E are considered potentially unsafe. Analyses were performed using SPSS (IBM SPSS Statistics for Windows, V.20.0, Armonk, New York, USA: IBM Corp) and Microsoft Excel 2010 with Analyse-It statistical add-on (V.2.30).

Results

A total of 28 subjects were included in this study, 8 subjects with T1DM, 12 subjects with T2DM (in total 20 subjects with DM) and 8 non-diabetic subjects. Two persons (one T1DM, one T2DM) did not end the study due to a detachment of the sensors of the FSL-CGM, and one T1DM withdrew from the study because of personal reasons. Therefore, the presented results represent 18 subjects with diabetes in the comparison study (7 T1DM, 11 T2DM); furthermore, 17 out of 18 subjects (6 T1DM, 11 T2DM) successfully finished the 75 g glucose load testing. Baseline characteristics are presented in table 1.

Table 1

Baseline characteristics

	Type 1 diabetes	Type 2 diabetes
	N=8	N=12
Age (years)	63 [24–74]	56 [39–65]
Female sex (%)	3 (38)	3 (25)
BMI (kg/m2)	23.2 [21.6–24.5]	28.3 [26.4–30.7]
Fat percentage (%)	27.1 [23.6–33.0]	28.9 [25–35.1]
Insulin therapy (n): CSII/MDI/BD	4/4/0	3/8/1
Oral blood glucose-lowering agents (%)
Metformin	–	10
SU	–	1
Other	–	0
HbA1c (mmol/mol)	59 [46–70]	55 [42–82]
Presence of microvascular complications
Nephropathy (n)	0	1
Neuropathy (n)	2	4
Retinopathy (n)	2	1
Diabetes duration (years)	14 [7–24]	13 [4–25]

Data are presented as n (%), mean (SD) or median [range].

BD, twice daily mix insulin; BMI, body mass index; CSII, continuous subcutaneous insulin infusion; HbA1c, glycated hemoglobin; MDI, multiple daily injections; SU, sulfonylurea derivate.

Accuracy of FSL-CGM versus StatStrip

Figure 1 demonstrates the Clarke Error Grid analyses (ISO 15197: 2013) of the StatStrip and coincident readings of FSL-CGM arm, FSL-CGM abdomen and FSLC in the combined population of 18 subjects with T1DM and T2DM (figure 1). For the paired StatStrip-FSL-CGM arm readings, the percentages of results in zone A was 85.5 (95% CI 77.6 to 89.5) as determined by bootstrap analyses.

Figure 1

Clarke Error Grids (ISO15197; 2013) of StatStrip and coincident FSLTab-CGM arm (upper left panel), FSL-CGM abdomen (lower left panel) and FSLC readings (right panel) in the combined population of type 1 and 2 diabetes. Number and percentages of paired readings within error grid zones A, B, C D and E are presented. FSL-CGM, FreeStyle Libre Flash continuous glucose monitoring.

FSL-CGM in the arm versus abdomen and FSL-CGM arm versus other techniques

Percentages of paired readings falling in zone A were found to be substantially lower for FSL-CGM abdomen, that is, 63.9% (95% CI 56.8% to 68.7; figure 1).

As presented in figure 1, comparing the FSLC test strip results with the StatStrip showed a tight correlation between both glucose measurement techniques, the percentages of results in zone A being 97.8% (95% CI 97.1% to 98.6%) as determined by bootstrap analyses. The percentages of results in zone B was 2.0% and in zone C 0.2%.

The overall correlation of FSL-CGM in the arm and the StatStrip indicates a performance with lower results with the FSL-CGM in the arm than expected based on the StatStrip in the lower glucose ranges, and higher results than expected in the higher ranges. This resulted in a correlation equation of Y=1.07X−13.48. Comparing glucose measurement data at various ranges confirmed this observation: MAD and MARD at these ranges are shown for the comparisons of FSL-CGM and FSL capillary, and for FSL-CGM versus StatStrip (table 2).

Table 2

Glucose measurement data at various concentrations (mean+SD; and median with 25–75 IQRs; paired t-tests)

	FSLA (arm)			FSLC (capillary)			StatStrip			FSLA vs FSLC			FSLA vs StatStrip
	N	Mean (SD)	Median (25–75)	N	Mean (SD)	Median (25–75)	N	Mean (SD)	Median (25–75)	MAD	MARD	p Value	MAD	MARD	p Value
≤70	56	60 (10)	65 (62, 69)	45	68 (16)	70 (59, 75)	56	70 (16)	69 (62, 79)	11	19	<0.0001	13	24	0.0059
71–80	39	76 (2)	76 (74, 78)	39	73 (26)	78 (69, 85)	39	83 (16)	83 (74, 94)	10	13	0.0071	13	17	0.0264
81–90	53	86 (3)	85 (83, 88)	53	85 (23)	90 (83, 96)	53	95 (12)	96 (87, 101)	9	11	<0.0001	12	14	0.0067
91–100	48	97 (3)	97 (94, 99)	48	88 (33)	97 (87, 111)	48	106 (15)	105 (96, 117)	11	12	<0.0001	14	14	0.0001
101–110	44	107 (3)	106 (105, 110)	39	113 (24)	106 (101, 121)	44	117 (12)	115 (109, 126)	13	13	<0.0001	13	12	0.1083
111–125	77	117 (4)	115 (112, 121)	77	116 (39)	126 (112, 133)	77	127 (15)	126 (117, 136)	15	13	<0.0001	15	12	0.2237
126–150	124	138 (7)	139 (132, 144)	124	123 (49)	137 (119, 149)	124	143 (18)	142 (130, 155)	13	9	0.0106	14	10	0.0032
151–180	105	166 (8)	166 (159, 173)	105	141 (61)	160 (144, 177)	105	169 (23)	169 (156, 184)	16	9	0.1751	19	10	<0.0001
>180	169	235 (49)	220 (196, 259)	199	178 (92)	202 (159, 236)	199	215 (45)	211 (186, 239)	23	10	<0.0001	25	10	0.1298

MAD, mean absolute difference (mg/dL); MARD, mean absolute relative difference (%).

Oral glucose loading test

Part of the results of the measurements of the glucose loading test with the FSL-CGM and the PCA-HK, real-time CGM, FSLC glucose strip and the StatStrip are presented in figure 2. The results demonstrate a slower rise and generally lower glucose values for the FSL-CGM as compared with the established laboratory method PCA-HK and StatStrip, as evidenced by significant differences between methods at time points 15, 30, 45 and 60 min (figure 2). For patients with DM the MARD was 8.3% (95% CI 13.1% to 3.5%); 12.7% (95% CI 21.2% to 4.3%) for subjects with T1DM and 6.4% (95% CI 12.0% to 0.7%) for subjects with T2DM.

Figure 2

The 75 g glucose loading test. Comparison of FSL-CGM versus PCA-HK, StatStrip, FSLC test strip, and iPro2; and PCA-HK versus FSLC test strip, and iPro2. Glucose concentrations (mg/dL; median, interquartile Q1 and Q3) at time points 0, 15, 30, 45, 60, 75 and 90 min in the combined population of type 1 and 2 diabetes. *p Value ≤0.05; **p Value ≤0.01. FSL-CGM, FreeStyle Libre Flash continuous glucose monitoring; PCA-HK, perchloric acid hexokinase.

The CGM comparator iPro2 CGM readings showed glucose levels being comparable to those of the PCA-HK and StatStrip on virtually all points (figure 2).

Discussion

In this study, the performance of the FSL-CGM was evaluated against capillary blood glucose reference measurements traced and aligned against established laboratory methods, both in daily life practice and during a glucose loading test. The results demonstrated that the use by subjects with T1DM and T2DM of the FSL-CGM in a daily life setting was associated with an acceptable accuracy, with 85.5% and 97.8% of the paired FSL-CGM—StatStrip readings falling within consensus Clarke Error Grid zone A and clinically acceptable zones A and B, respectively. Acceptable accuracy could only be demonstrated for the FSL-CGM readings in the upper arm, data obtained from the abdomen was not reliable (only 62.9% of readings in zone A). Findings of this study are in line with recently published data on the performance of the FSL-CGM (85.5% and 99.0% of readings falling in error grid zone A and zones A and B, respectively),10 and previous studies evaluating the accuracy of other CGM systems.17–21 Furthermore, results in the lower ranges are more or less in line with the findings of Linong and colleagues, who found a MAD in the ≤70 mg/dL of 14 mg/dL (translating into a 20% relative difference in that range) when comparing FSL-CGM with the Free Precision strip capillary glucose measurement.

In lower glucose ranges, during hypoglycemia or in periods of rapid variation in glucose levels, values provided by CGM systems may be inaccurate.22 In our study, lower than expected glucose values were observed in the lower glucose ranges for the FSL-CGM compared with reference method. This is and will remain a point of attention for various reasons. When a user is alert and sees a low reading in the absence of clinical signs of hypoglycemia, glucose control with another technique is advisable.

As an observational fact, some of the study subjects reported low readings especially during the night, not supported by the control measurements. Therefore, this should also be a point of attention when on hindsight low FSL-CGM readings are found after sleeping/in nighttime. Again, this will not necessarily be a true representation of the actually existing glucose concentration. Since this observation was no part of the study design, a more formal and properly performed study is necessary to elucidate this observation.

Accurate hypoglycemia measurements are clinically important for obvious reasons. It is recognized that currently available CGM devices generally are least accurate in the hypoglycemic range11 23—further refinements in technology may be required before these systems could be used to sufficiently accurate warn for (impending severe) hypoglycemia. Whether such status can ever be reached, remains an important issue, which is not only dependent on the efforts of the manufacturers to improve algorithms but may also be due to the inherent inability to predict interstitial fluid changes, both in flow and in glucose concentrations (see also below).

This study also demonstrates a slower rise in glucose levels along with an (initially) significantly aberrant MARD following a standardized glucose load for the FSL-CGM in comparison with results obtained from the PCA-HK and StatStrip analyses, but also in comparison with the FSL test strip and the iPro2. Consequently, one may underestimate the effect of a meal on the glucose response when only taking the FSL-CGM readings into account.

Using subcutaneous interstitial fluid for glucose monitoring provides closeness to the vasculature while still being minimally invasive and has become the mainstay of CGM devices. Although sensors are evolving in the past years, subcutaneous sensing still has inherent limitations (physiological lag time, sensitivity to local fluctuations) and can be slow and variable.24 25 Apparently, the producer of the iPro2 has been able to address this issue differently compared with the producer of the FSL-CGM; a part of the observed differences is explained by the calibration used for the iPro2, but definitely not completely.

Technologies for CGM require patient education for proper use of the device and correct interpretation of the data.22 26 The observed deviations of the FSL-CGM system from the norm identified in this study can partly be addressed by appropriate patient instruction at first use of the device. It also places a responsibility with the producer to bring such deviations under the attention of the users.

The FSL-CGM has various features that distinguish the device from currently available CGMs.27 The compact and easy to use FSL-CGM system is factory calibrated and does not need fingers stick calibration during the 14 days of wear. FSL-CGM could be considered as CGM on demand or a viable alternative to frequent finger prick readings, and might also benefit individuals who have ceased sensor use due to alarm fatigue.28 Owing to its 14-day longevity, costs are lower compared with available CGM systems,27 29 30 bringing the FSL-CGM within easier reach for out-of-pocket expenses and reimbursement systems. In line with these observations it was stated previously that patients using the device were satisfied with its performance with numerous reports available in patient blogs.27 It is anticipated that the FSL-CGM system will contribute to the ability of adequate self-management in appropriate target groups.

This study is the first to describe the accuracy of the FSL-CGM in a clinical and real-life setting in an independent study. Comparisons with various blood glucose measurement techniques are made, including the gold standard and an often used real-time CGM method. Limitations should be mentioned. First and foremost, numbers are small. In particular, the relatively low number of readings below 80 mg/dL, due to the non-blinded nature of our study patients were able to act on low glucose concentrations, could be of importance with respect to our aforementioned concerns about the influence of low readings. The accuracy of the FSL in the lower ranges, and also in the higher ranges, should be subject of future studies which include an acceptable amount of readings and, ideally, data concerning clinical tests and symptoms.

Second, as our study was applied in a daily life setting, the accuracy of the FSL-CGM could not be established against the ‘real gold standard’ to measure arterial blood glucose concentrations. Further study limitations included the risk of individual subject errors in the study procedures (eg, use of the device, reporting of glucose results, intake study drugs, despite correct instructions). However, we intentionally used a broad range of methods as well as Abbott's own FSL test strip for comparison of the FSL-CGM.

Conclusions

Our findings indicate that the FSL-CGM system can be used as a reasonably suitable adjunct in the management of diabetes, but only when used if inserted in the upper arm. In this prospective study, a reasonable accuracy of the FSL-CGM readings in the upper arm was demonstrated against capillary values that were traced and aligned against recognized laboratory reference values. However, certain matters need attention while using the FSL-CGM in daily life including the observed lower values in the lower ranges and higher values in the higher ranges, and the underestimation of the effect of a meal on glucose response. These weaknesses can partly be overcome by optimizing the available user instructions. Further evaluation is needed to identify the proper target population most likely to benefit from the FSL-CGM.