Accuracy of the intermittently scanned continuous glucose monitoring system in critically ill patients: a prospective, multicenter, observational study.

OBJECTIVE: Continuous glucose monitoring (CGM) has the potential to improve glucose control in the intensive care unit (ICU) setting. We sought to evaluate the accuracy of the intermittently scanned CGM (isCGM) system in critically ill patients. RESEARCH DESIGN AND METHODS: Adult patients were consecutively enrolled from three ICUs from August 2020 to January 2021. The performance of FreeStyle Libre Pro was evaluated against the venous blood glucose samples as a reference. Numerical accuracy was examined by the mean absolute relative difference (MARD), the Bland-Altman analysis, and the International Organization for Standardization criteria. Clinical accuracy was assessed by performing the Clarke and consensus error grid analysis.
RESULTS: A total of 122 patients were included and 3416 matched glucose pairs were analyzed. The overall MARD was 18.0%, and the highest MARD (33.1%) was observed in the hypoglycemic range (<70 mg/dL). The Bland-Altman analysis revealed a mean bias of -11.7 mg/dL, with the 95% limits of agreement of -73.0 to 49.5 mg/dL. The percentages of isCGM glucose values within ±15%/15, ±20%/20, and ±30%/30 mg/dL were 49.8%, 64.7%, and 84.5%, respectively. The Clarke and consensus error grid analysis showed acceptable clinical accuracy with 98.5% and 98.8% of glucose values falling into zones A and B.
CONCLUSIONS: Our study demonstrated suboptimal overall accuracy of isCGM for critically ill patients. Whether the adjunctive use of isCGM could improve glucose management and health outcomes in the critically ill needs further investigation. CLINICAL TRIAL REGISTRATION: ChiCTR2100042036, Chinese Clinical Trial Registry.

Introduction

Hyperglycemia is common in the intensive care unit (ICU) and may be secondary to either diabetes or critical illness-associated hyperglycemia. Since the landmark study of Van den Berghe et al. [1], it is generally accepted that maintenance of blood glucose levels within a specified range is associated with improved morbidity and mortality in inpatients including the critically ill. Needless to say, glucose monitoring is essential for the adjustment of treatment regimens and optimal glucose control.

Currently, bedside fingertip blood glucose measurement is still the standard of care for the evaluation of blood glucose levels in the ICU. According to the latest American Diabetes Association guidelines, hospitalized patients need 3–6 finger blood glucose tests every day. More frequent monitoring, approximately every 30 min to 2 h, is required for patients who use intravenous insulin [2]. Compared with fingertip blood glucose testing, continuous glucose monitoring (CGM) has the advantage of comprehensive 24-h monitoring, allowing for the assessment of glucose patterns and trends, and the identification of asymptomatic hypoglycemia and nocturnal hypoglycemia [3] that are unrecognized by conventional methods. Although CGM has been demonstrated to improve glycemic control in outpatients with diabetes [4, 5] and in hospitalized patients in non-critical care settings [6], the use of CGM in the ICU is still limited, with accuracy being a major concern. In this regard, numerous previous studies have evaluated the performance of multiple CGM sensors, and the results varied widely [7-9]. Moreover, only a few have assessed the performance of isCGM in ICUs [8, 10–12]. Therefore, this study aimed to evaluate the analytical and clinical accuracy of an isCGM device in a large group of critically ill patients.

Research design and methods

Study design

This was a prospective, observational, multicenter study performed at the Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Jinshan Branch of Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, and Lingang Campus of Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine (names of principal investigators listed in Supplementary Table 1). The study protocol was approved by the ethics committee of Shanghai Sixth People’s Hospital, and all subjects gave informed consent before the study enrollment (ChiCTR2100042036, Chinese Clinical Trial Registry).

Participants

The study involved patients from three central ICUs from August 2020 to January 2021. Eligible participants were adult patients with acute physiology and chronic health evaluation II (APACHE II) scores over 15 and good venous access and who were expected to remain in the ICU for at least 3 days. Key exclusion criteria included using glucocorticoids and immunosuppressants; pregnancy or planned pregnancy; severe anemia (hemoglobin <6 g/dL); allergy to the adhesive used in the monitoring; adverse skin diseases (such as cowhide moss or rash) in the instrument placement area; a need for MRI examination; or an eventual ICU stay less than 3 days due to various reasons (such as death or ICU discharge).

Procedures

The study team approached patients shortly after hospital admission to explain the design and potential risks of the study. After informed consent was obtained, a trained team member placed a FreeStyle Libre Pro (Abbott Diabetes Care, Alameda, CA, USA) in the posterior upper arm according to the instructions. Data can be obtained starting within 60 min after the sensor is successfully initiated, and the glucose value can be collected every 15 min for 14 days (hereafter referred to as the isCGM value). The research team took 0.5 mL of venous blood every 15 min for 7 consecutive hours on a random day after enrollment. Venous plasma blood glucose was measured immediately as the reference glucose values with a blood glucose lactic acid analyzer (EKF BIOSEN-C line series, German) using the glucose-oxidase method after centrifugation at bedside (hereafter referred to as the venous blood glucose [vBG] value). The isCGM values were matched with the vBG values measured simultaneously, and the corresponding glucose pairs were used to assess the accuracy of isCGM.

Patient age, gender, medical history, medication, use of renal replacement therapy, and use of ventilator were collected. Blood gas, leukocytes, and creatinine were measured by the alcohol method, and glycosylated hemoglobin was measured by the high-pressure liquid method. The patient’s heart rate and blood pressure were collected from the intensive care system.

The patients and the study team were blinded to the isCGM glucose results during the research. According to the guidelines of the American Diabetes Association, these three centers adopted a unified glycemic control program. All isCGM data were downloaded after ICU discharge using LibreView software (Abbott Diabetes Care, Alameda, CA, USA) after removal from this study.

Statistical analysis

For the evaluation of analytical accuracy, we calculated the absolute relative difference (ARD) ([sensor glucose – reference glucose]/reference glucose, %) between matched glucose pairs, and then mean ARD (MARD) was reported. We also conducted the Bland–Altman analysis and calculated the percentages of sensor glucose levels within ±15%/15, ±20%/20, and ±30%/30 mg/dL from the reference glucose value. These metrics were evaluated for all paired glucose values and in different glucose ranges (<70, 70–180, and >180 mg/dL). Clinical accuracy was assessed by performing the Clarke and consensus error grid analysis. The error grid analyses were performed with MATLAB R2021a (MathWorks). All other analyses were conducted with SAS 9.4. A P value of <0.05 (two-tailed) was considered statistically significant.

Results

Basic information of the participants

A total of 122 patients were finally included for analysis. The mean age of the study population was 65.4 ± 15.5 years, and 43 (35.2%) were female. Twenty-three (18.9%) patients were diagnosed with type 2 diabetes (Table 1). The mean hemoglobin A1c (HbA1c) level was 6.2 ± 1.5% (44.5 ± 16.2 mmol/mol), the median length of hospital stay was 27.5 days, and the median length of ICU stay was 20 days. During the study, 32 patients received antihypertensive drugs, 32 patients received vasopressors, 29 patients received heart rate-lowering drugs, 90 patients were intubated, and 34 patients were on renal replacement therapy.

Table 1

Characteristics of the study population

Characteristic	Value
n	122
Age, years	65.4 ± 15.5
Male, n (%)	43 (35.2)
Type 2 diabetes, n (%)	23 (18.9)
Admission hemoglobin A1c, %	6.2 ± 1.5
Admission hemoglobin A1c, mmol/mol	44.5 ± 16.2
Length of ICU stay, days	20 (13.8, 36)
Length of intubation, days	15 (5, 26)
Length of renal replacement therapy, days	0 (0, 8)
APACHE II score	20.2 ± 7.1
The use of (during the study), %
Antihypertensive drugs	32 (26.2)
Vasopressors	32 (26.2)
Antiarrhythmic drugs	29 (29.5)
Respirator	90 (73.8)
Renal replacement therapy	34 (27.9)
MAP	91.1 ± 23.7
pH	7.46 ± 0.20
Hematocrit	31.4 ± 8.5
Glasgow coma score	9.0 ± 5.4
Lactose, mmol/L	1.44 ± 0.78
Mean vBG value, mg/dL	150.7 ± 62.0
Mean isCGM value, mg/dL	139.0 ± 63.7

ICU intensive care unit, APACHE II acute physiology and chronic health evaluation II, MAP mean artery pressure, pH potential of hydrogen, vBG venous blood glucose, isCGM intermittently scanned continuous glucose monitoring

Accuracy analysis

A total of 3416 matched pairs of isCGM/vBG values were analyzed. The average vBG value was 150.7 ± 62.0 mg/dL, and the average isCGM value was 139.0 ± 63.7 mg/dL (Table 1). The overall MARD was 18.0 ± 13.4% (Table 2). The Bland–Altman analysis revealed a mean bias of −11.7 mg/dL, with the 95% limits of agreement of −73.0 to 49.5 mg/dL (Fig. 1). For 3212 of the 3416 paired values (94.0%), the difference was within the 95% limits of agreement. The percentages of isCGM values within ±15%/15, ±20%/20, and ±30%/30 mg/dL of vBG values were 49.8%, 64.7%, and 84.5%, respectively (Table 2).

Table 2

The point accuracy of FreeStyle Libre Pro across different glucose ranges

Glucose range	Glucose pairs (n)	MARD (%)	±15%/15 mg/dL (%)	±20%/20 mg/dL (%)	±30%/30 mg/dL (%)
Overall	3416	18.0	49.8	64.7	84.5
<70 mg/dL	82	33.1	46.3	58.5	75.6
70–180 mg/dL	2411	18.2	48.1	63.5	84.3
>180 mg/dL	923	16.3	54.7	67.9	85.6

CGM continuous glucose monitoring, MARD mean absolute relative difference

Fig. 1

Bland–Altman plot. In Bland–Altman analysis, the mean of paired readings is on the X-axis, and the difference is on the Y-axis. The solid line represents the mean difference. The dashed line represents the cutoff of two standard deviations from the mean difference. isCGM intermittently scanned continuous glucose monitoring, vBG venous blood glucose, SD standard definition

The Clarke and consensus error grid analysis showed acceptable clinical accuracy, with 98.5% (zone A: 64.0%, zone B: 34.5%) and 98.8% (zone A: 65.2%, zone B: 33.5%) of glucose values falling into zones A + B, respectively (Fig. 2).

Fig. 2

Clarke and consensus error grid analysis. A In Clarke error grid analysis, the vBG value (X-axis) was compared to isCGM values (Y-axis). Zone A: isCGM <20% deviation from vBG value or both isCGM and vBG value <70 mg/dL; zone B: deviation from vBG value >20% but lead to no inappropriate treatment; zone C: lead to unnecessary treatment; zone D: a potentially dangerous failure to detect hypo- or hyperglycemic state; zone E: erroneous treatment. The Clarke error grid analysis showed acceptable clinical accuracy, with 98.5% of glucose values falling into zones A + B. B In consensus error grid analysis, zone A: no effect on clinical action; zone B: little or no effect on clinical outcome; zone C: likely to affect the clinical outcome; zone D: could have significant medical risk; zone E: could have dangerous consequences. 98.8% of glucose values falling into zones A + B. vBG venous blood glucose, isCGM intermittently scanned continuous glucose monitoring

Factors affecting accuracy

The accuracy was higher in the hyperglycemic range (>180 mg/dL, n = 923, MARD 16.3 ± 12.3%) than in the target range of 70–180 mg/dL (n = 2411, MARD 18.2 ± 12.9%) (P < 0.001) (Table 2). The highest MARD was observed in the hypoglycemic range (<70 mg/dL, n = 82, MARD 33.1%) (P < 0.001).

For glucose readings <70 mg/dL (n = 82), ±15%/15, ±20%/20, and ±30%/30 mg/dL were 46.3%, 58.5%, and 75.6%, respectively (Table 2). The percentages for glucose values within 70–180 mg/dL (n = 2411) were 48.1%, 63.5%, and 84.3%, respectively. For glucose readings >180 mg/dL (n = 923), ±15%/15, ±20%/20, and ±30%/30 mg/dL were 54.7%, 67.9%, and 85.6%, respectively.

For further evaluation, data were stratified by subgroups according to selected clinical factors. Linear regression analysis with the interaction component was performed to explore the relationship between MARD and these clinical variables (Supplementary Table 2). Significantly lower MARD was observed in patients with diabetes when compared with those without diabetes (interaction P = 0.015). The accuracy of isCGM was not influenced by other characteristics including age, gender, pH, APACHE II score, or the use of vasopressors.

Discussion

To date, most hospitalized patients undergo fingertip capillary blood glucose testing as the primary method of glucose monitoring. However, the accuracy of blood glucose meters varies greatly [13] and may be affected by several factors [14]. Moreover, frequent measurements in those patients who use insulin will simultaneously increase the workload of medical workers and the risk of infection, especially in the context of the COVID-19 pandemic. On the other hand, there is a large body of evidence that isCGM can improve glucose control and quality of life [15-17]. In a randomized controlled multicenter study, Bolinder et al. found that using isCGM (freestyle Libre) for 2 weeks reduced the whole-day hypoglycemic events (<70 mg/dL) by 25.8%, the nocturnal hypoglycemic events by 33.2% and the severe hypoglycemic events (<40 mg/dL) by 55% in patients with type 1 diabetes with good glycemic control (HbA1c ≤ 7.5% [58.5 mmol/mol]) [18]. This suggests that isCGM may improve blood glucose control in critically ill patients.

The accuracy of isCGM in critically ill patients has been reported in some studies, but these studies have some limitations, such as a small sample size, few matched pairs, the single-center and/or retrospective study design, and inconsistent reference methods, which also partially explains for the conflicting results concerning the MARD values (14–41%). For instance, Ancona et al. analyzed 185 pairs in 8 critically ill patients with type 2 diabetes and showed that the MARD was 14% [8]. The analysis of the accuracy of isCGM in the hypoglycemic range was precluded in that study due to the lack of hypoglycemic glycose pairs. Several previous studies evaluated the accuracy of isCGM in a pediatric ICU (n = 16, 711 pairs) [10], a cardiology ICU (n = 15, 149 pairs) [11], and COVID-19 patients in an ICU (n = 17, 171 pairs) [12], respectively, but the results could not be extended to all critically ill patients.

In this prospective study, we analyzed 3416 pairs from 122 patients, and the MARD was 18.0%, implying suboptimal overall accuracy for critically ill patients. Furthermore, we found that the accuracy of blood glucose monitoring decreased as vBG decreased but was not influenced by patients’ characteristics, such as age, gender, pH, APACHE II score, or the use of vasopressors, suggesting that these results could be generalized to a wide range of critically ill patients.

Of note, together with the current glucose value, CGM systems provide trend arrows to indicate the direction and rate of glucose change, to inform decision making. Therefore, the ideal assessment of CGM accuracy should include both point accuracy and trend accuracy [19]. However, there is no well-accepted metric for evaluating trend accuracy. Interestingly, in a small study investigating the accuracy of isCGM, visual inspection revealed that FGM values closely followed the trend of the reference method (arterial blood gas) despite the poor point accuracy [9]. Taken together, although the point accuracy of isCGM in our study does not justify the nonadjunctive use of this device in ICU, it is possible that patients may benefit from a hybrid management protocol incorporating both isCGM (including glucose values and trend) and point-of-care measurements while reducing the workload and infection risk of health providers. The impact of such protocols on the outcomes of critically ill patients warrants further investigations.

The advantage of this study lies in the inclusion of a mixed population of medical-surgical ICU patients, the multicenter study design, a large number of glucose pairs comprising the full range of glycemia, the use of venous plasma glucose as the only reference standard, and the unified equipment used to measure vBG. Nevertheless, there are some limitations to this research. First, most of the samples in this study were taken during the day. Therefore, we could not examine the difference in the accuracy of isCGM between daytime and nighttime. However, the relatively large sample size enabled us to investigate the accuracy of FGM in the full range of glucose levels (i.e., hypoglycemic, euglycemic, and hyperglycemic ranges), which partially mitigates this concern. Second, trend accuracy was not evaluated in this study, which may be an interesting issue to address in the future. Finally, the effect of isCGM on glycemic outcomes could not be determined in this study.

In summary, our study demonstrated suboptimal overall accuracy of isCGM for critically ill patients. Whether the adjunctive use of isCGM could improve glucose management and health outcomes in the critically ill needs further investigation.

Supplementary_material