Variation in hypoglycemia ascertainment and report in type 2 diabetes observational studies: a meta-epidemiological study.

INTRODUCTION: Observational studies constitute an important evidence base for hypoglycemia in diabetes management. This requires consistent and reliable ascertainment and reporting methodology, particularly in studies of type 2 diabetes where hypoglycemia risk is heterogeneous. Therefore, we aimed to examine the definitions of hypoglycemia used by observational studies of patients with type 2 diabetes. RESEARCH DESIGN AND METHODS: We conducted a meta-epidemiological review of observational studies reporting on hypoglycemia or evaluating glucose-lowering medications in adults with type 2 diabetes. MEDLINE and Google Scholar were searched from January 1970 to May 2018. The definitions of non-severe, severe and nocturnal hypoglycemia were examined.
RESULTS: We reviewed 243 studies: 47.7% reported on non-severe hypoglycemia, 77.8% on severe hypoglycemia and 16.9% on nocturnal hypoglycemia; 5.8% did not specify. Among 116 studies reporting non-severe hypoglycemia, 18.1% provided no definition, 23.3% used glucose values, 38.8% relied on patient-reported symptoms, 17.2% accepted either glucose values or patient-reported symptoms and 2.6% relied on International Classification of Disease (ICD) codes. Among 189 studies reporting severe hypoglycemia, 11.1% provided no definition, 53.4% required symptoms needing assistance, 3.7% relied on glucose values, 14.8% relied on ICD codes, 2.6% relied on ICD codes or glucose values and 15.9% required both symptoms needing assistance and glucose values. Overall, 38.2% of non-severe and 67.7% of severe hypoglycemia definitions were consistent with the International Hypoglycemia Study Group.
CONCLUSIONS: The marked heterogeneity in how hypoglycemia is defined in observational studies may contribute to the inadequate understanding and correction of hypoglycemia risk factors among patients with type 2 diabetes. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Since 2016, the International Hypoglycemia Study Group (IHSG) defined hypoglycemia severity levels and recommended their use for assessment and report in research studies.

The definitions of hypoglycemia used by randomized clinical trials of diabetes therapies are still diverse and inconsistent.

In the 243 observational studies of type 2 diabetes therapies reviewed, the hypoglycemia definitions reported were heterogeneous.

More than half of the observational studies published after the IHSG hypoglycemia definition were compliant with their recommendations.

Almost a fifth of the observational studies reporting hypoglycemia outcomes did not provide a specific definition for the event.

Heterogeneity in hypoglycemia definitions hinders the comparison of observational studies.

Using standardized hypoglycemia definition, ascertainment and reporting in observational studies could lead to a better comprehension of these events in a real-world setting, as well as supporting the development of better risk stratification and prevention strategies.

Introduction

Hypoglycemia is a serious and potentially preventable adverse event in diabetes management, leading to morbidity, impaired quality of life, high costs for patients and society, and death.1–3 While hypoglycemia is more common among people with type 1 diabetes,4–6 it also affects people with type 2 diabetes, particularly those with multiple or advanced comorbidities and those treated with insulin (including concentrated insulin) and/or insulin secretagogues.7–11 Efforts to better understand hypoglycemia risk factors and develop interventions for those at highest risk are predicated on reliably, accurately and consistently identifying events as they occur. In epidemiologic assessments and in research, such efforts have been hindered by the lack of standardized and universally used reporting parameters for clinically significant hypoglycemia.

The International Hypoglycemia Study Group (IHSG), on behalf of the American Diabetes Association (ADA) and European Association for the Study of Diabetes (EASD), has defined level 1 hypoglycemia as any glucose value ≤70 mg/dL (3.9 mmol/L), level 2 hypoglycemia as glucose <54 mg/dL (3.0 mmol/L) and level 3 (‘severe’) hypoglycemia as any glucose value associated with severe cognitive impairment requiring external assistance for recovery.12 13 Recognizing the importance of consistent and standardized reporting of hypoglycemia as an adverse event in diabetes management and data demonstrating marked variability in how hypoglycemia is described in clinical (ie, interventional) trials,14 the IHSG recommended that level 2–3 hypoglycemia, but not level 1 hypoglycemia, be reported in such trials.12 13 However, real-world data and observational studies also constitute an importance evidence base for clinical decision making.15 16 This is particularly important for an outcome like hypoglycemia, the incidence of which is likely to be higher in real-world settings than in closely monitored trials that enroll narrowly defined, and often lower risk, populations. Standardization of hypoglycemia reporting is therefore equally, or even more, important in observational studies and those that rely on real-world data.

To contextualize the existing evidence base regarding hypoglycemia, we examined and summarized the definitions of hypoglycemia used in observational studies centered on patients with type 2 diabetes. We focused specifically on type 2 diabetes because the risk of hypoglycemia in this population is more heterogeneous and treatment-dependent and context-dependent than in type 1 diabetes.7

Research design and methods

Data sources and selection

We conducted a meta-epidemiological review of the literature in MEDLINE and Google Scholar for observational studies published between January 1970 and May 2018. Our search strategy for the bibliographic databases combined terms for hypoglycemia, glucose-lowering drugs and observational studies of type 2 diabetes in adults and was limited to English language studies and full-length articles. The applied search terms were: ‘Diabetes mellitus type 2’, ‘Hypoglycemia’, ‘Adverse event’, ‘Insulin’, ‘Sulfonylurea’, ‘Thiazolidinedione’, ‘Dipeptidyl peptidase 4 inhibitor’, ‘Glucagon-like peptide-1 receptor agonist’, ‘Sodium glucose cotransporter-2 inhibitor’, as well as different combinations and associated Medical Subject Headings. Two researchers working independently screened papers for eligibility, with a third one resolving discrepancies. We selected observational (ie, not interventional) studies that reported hypoglycemic events or evaluated glucose-lowering drugs (any type of insulin, sulfonylurea, thiazolidinedione, dipeptidyl peptidase 4 inhibitor, glucagon-like peptide-1 receptor agonist and sodium glucose cotransporter-2 inhibitor) in adults with type 2 diabetes. We excluded studies that did not report on hypoglycemic events.

Data extraction

From a total of 243 observational studies,17 two reviewers, working independently and in duplicate, reproducibly (kappa >0.6) extracted the exact definitions used to define non-severe (alternatively named mild/moderate), severe and nocturnal hypoglycemia. Severity of hypoglycemia was assigned on the basis of the definitions used in the reported studies. Unspecified hypoglycemia was defined as any reported hypoglycemic event without a clear definition of being non-severe, severe or nocturnal.

Statistical analysis

Data are presented as frequencies and percentages for categorical variables. Univariate between-group comparisons were performed using χ2 tests for categorical and binary variables. P values ≤0.05 were considered statistically significant. IBM SPSS V.22.0 (SPSS, Inc, Amonk, New York, USA) was used to perform all analyses.

Results

A total of 4862 research papers were retrieved from the databases search, with 1809 duplicates removed. After the title and abstract screening, 2182 studies did not meet the eligibility criteria and were eliminated. A total of 871 studies underwent full-text review, of which 628 were eliminated for not meeting eligibility criteria, mainly not reporting on hypoglycemia events (figure 1). A total of 243 studies were ultimately reviewed: 53 cross-sectional (21.8%), 99 prospective cohort (40.7%), two case–control (0.8%) and 89 retrospective cohort (36.6%). Overall, 47.7% (n=116) reported on non-severe hypoglycemia, 77.8% (n=189) reported on severe hypoglycemia and 16.9% (n=41) reported on nocturnal hypoglycemia. In 5.8% (n=14) of studies, hypoglycemia type was not specified (table 1). A total of 32 of the analyzed studies were published after November 2016 when IHGS recommendations on standardizing hypoglycemia definitions were published.12 13 Overall, 38.2% of non-severe and 67.7% of severe hypoglycemia definitions were consistent with the IHSG recommendations. Among studies published before November 2016, 33.3% of studies reporting on non-severe hypoglycemia and 69.6% of studies reporting on severe hypoglycemia were consistent with what would be the IHSG recommendations. After IHSG recommendations were published, 62.5% and 54.1% of non-severe and severe hypoglycemia definitions were consistent with them.

Figure 1

Flow chart of database research, screening and study selection.

Table 1

Study design, type of data source and treatment*

Type of hypoglycemia	Not defined	Non-severe	Severe	Both	Nocturnal†	Total
	(n=14)	(n=40)	(n=113)	(n=76)	(n=41)	(n=243)
Study design
Cross-sectional	0 (0)	15 (37.5)	18 (15.9)	20 (26.3)	8 (19.5)	53 (21.8)
Prospective cohort	5 (35.7)	14 (35.0)	38 (33.6)	42 (55.3)	31 (75.6)	99 (40.7)
Retrospective case–control	0 (0)	0 (0)	1 (0.9)	1 (1.3)	0 (0)	2 (0.8)
Retrospective cohort	9 (64.3)	11 (27.5)	56 (49.6)	13 (17.1)	2 (4.9)	89 (36.6)
Type of data source
Administrative data	5 (35.7)	3 (7.5)	33 (29.2)	4 (5.3)	0 (0)	45 (18.5)
EHR	3 (21.4)	3 (7.5)	12 (10.6)	9 (11.8)	0 (0)	27 (11.1)
Interview	0 (0)	1 (2.5)	2 (1.8)	3 (3.9)	1 (2.4)	6 (2.5)
Registry	2 (14.3)	5 (12.5)	19 (16.8)	2 (2.6)	0 (0)	28 (11.5)
Study cohort	4 (28.6)	15 (37.5)	34 (30.1)	42 (55.3)	33 (80.5)	95 (39.1)
Survey	0 (0)	13 (32.5)	13 (11.5)	16 (21.1)	7 (17.1)	42 (17.3)
Type of treatment
Not specified	7 (50)	10 (25.0)	42 (37.2)	17 (22.4)	3 (17.6)	76 (31.3)
Insulin	4 (28.6)	17 (42.5)	41 (36.3)	33 (43.4)	36 (87.8)	95 (39.1)
Sulfonylurea	0 (0)	1 (2.5)	7 (6.2)	9 (11.8)	0 (0)	17 (7)
Insulin+SU	1 (7.1)	2 (5.0)	10 (8.8)	3 (3.9)	0 (0)	16 (6.6)
Other (TZD, DPP-4, GLP1 and SGLT-2)	1 (7.1)	6 (15.0)	8 (7.1)	7 (9.2)	0 (0)	22 (9.1)
Insulin+other (TZD, DPP-4, GLP1 and SGLT-2)	1 (7.1)	2 (5.0)	3 (2.7)	3 (3.9)	2 (4.9)	9 (3.7)
SU+others (TZD, DPP-4, GLP1 and SGLT-2)	0 (0)	2 (5.0)	2 (1.8)	4 (5.3)	0 (0)	8 (3.3)

*Data are presented as number (percentage) unless specified otherwise.

†Studies that reported nocturnal hypoglycemia were the same studies that evaluated severe and/or non-severe episodes and therefore were not included in the statistical analysis.

DPP-4, dipeptidyl peptidase 4 inhibitor; EHR, electronic health record; GLP1, glucagon-like peptide 1 receptor agonist; SGLT-2, sodium glucose cotransporter-2 inhibitor; SU, sulfonylurea; TZD, thiazolidinedione.

Defining non-severe hypoglycemia

Among the 116 studies reporting non-severe hypoglycemia (table 2), 18.1% (n=21) did not provide a specific definition, stating only that hypoglycemia was ‘mild’, ‘moderate’ or ‘not severe’; 23.3% (n=27) relied on a range of glucose values; 38.8% (n=45) relied on patient-reported symptoms alone; 17.2% (n=20) accepted either glucose values or patient-reported symptoms; and 2.6% (n=3) relied on International Classification of Disease (ICD) diagnosis codes for hypoglycemia.

Table 2

Hypoglycemia definitions*

	Not defined	Non-severe	Severe	Total
	(n=14)	(n=116)	(n=189)	(n=319)†
Hypoglycemia definition
Not specific definition	14 (100)	21 (18.1)	21 (11.1)	56 (17.6)
Glucose only	–	27 (23.3)	7 (3.7)	34 (10.7)
Symptoms only	–	45 (38.8)	0 (0)	45 (14.1)
Symptoms requiring ED or health provider assistance	–	0 (0)	101 (53.4)	101 (31.7)
ICD codes only	–	3 (2.6)	28 (14.8)	31 (9.7)
Glucose and/or symptoms	–	20 (17.2)	30 (15.9)	41 (12.9)
Glucose and/or ICD codes	–	0 (0)	2 (1.1)	5 (1.6)

*Data are presented as number (percentage) unless specified otherwise.

†This denominator represents all the hypoglycemia definitions for non-severe or severe events across all the analyzed studies.

ED, emergency department; ICD, International Classification of Disease.

Although 47 studies (40.5% of all studies reporting on non-severe hypoglycemia) relied on specific glucose ranges to define non-severe hypoglycemia, only 13.8% (n=16) and 1.7% (n=2) used glucose values consistent with ADA/EASD guidelines and IHSG definitions of level 1 (<3.9 mmol/L) and level 2 hypoglycemia (<3.0 mmol/L), respectively. Remaining studies used thresholds ranging between <50 mg/dL (<2.8 mmol/L) and <70 mg/dL (<3.9 mmol/L) (figure 2A). None of the studies defined hypoglycemia at glucose levels ≥70 mg/dL (3.9 mmol/L).

Figure 2

(A) Blood glucose cut-off values for non-severe hypoglycemia (mmol/L, mg/dL). (B) Blood glucose cut-off values for severe hypoglycemia (mmol/L, mg/dL). *The International Hypoglycemia Study Group (IHSG) level 1 of hypoglycemia definition. **IHSG level 2 of hypoglycemia definition. Figure part A shows the glucose values used by the analyzed studies to define non-severe hypoglycemia episodes. Figure part B shows the glucose values used by the analyzed studies to define severe hypoglycemia episodes.

Severe hypoglycemia

Of the 189 studies reporting severe hypoglycemia (table 2), 11.1% (n=21) did not provide a specific definition for what was considered to be ‘severe’; 53.4% (n=101) required symptoms requiring external or medical assistance, including some specifying need for emergency department (ED) care or hospitalization; 3.7% (n=7) relied on a range of glucose values only, without mention of symptoms; 14.8% (n=28) relied on ICD diagnosis codes only; 2.6% (n=2) relied on ICD diagnosis codes and/or glucose values; and 15.9% (n=30) required both symptoms requiring medical assistance and a specific glucose value. In the 30 studies that relied on ICD diagnosis codes, hypoglycemic events were deemed to be ‘severe’ if the hypoglycemia diagnosis code was present in encounters in the ED only (n=1); hospital only (n=4); ED or hospital (n=3); outpatient clinic only (n=1); or either ED, hospital or outpatient clinic (n=9). Remaining studies did not specify the setting where hypoglycemia diagnoses were ascertained (n=12). Moreover, the diagnosis codes could be in any position of the claim (n=1), primary or secondary position (n=6) or primary position only (n=2); in the vast majority of cases (n=21), the position was not specified. Only 10 of the studies that relied on ICD diagnosis codes used the validated Ginde algorithm for hypoglycemia ascertainment,18 with the remainder using alternate code sets.

The 39 studies (20.6% of the 189 reporting on severe hypoglycemia) that included glucose values in their definition of severe hypoglycemia (figure 2B) did not have a consistent threshold for what glucose level constituted severe hypoglycemia. The most prevalent threshold was <50 mg/dL (2.8 mmol/L; n=15, 38.5%), followed by <55.8 mg/dL (3.1 mmol/L; 33.3%, n=13) but ranged between <72 mg/dL (4.0 mmol/L) and <36 mg/dL (2.0 mmol/L).

Nocturnal hypoglycemia

While 41 studies reported on nocturnal hypoglycemia, 41.5% (n=17) did not provide any definition for how these events were defined. In 53.7% (n=22) of the studies, nocturnal hypoglycemia was defined based on temporality without specification of symptom severity, for example: ‘A nocturnal hypoglycemic event was defined as an individualized symptomatic event consistent with hypoglycemia that occurred while the patient was asleep, between bedtime (±after the evening insulin injection) and before getting up in the morning (±before morning determination of fasting plasma glucose and morning injection)’.19 20 Blood glucose levels were required to confirm hypoglycemia in only 4.9% (n=2) of the studies.

Conclusions

Hypoglycemia is a common, serious, yet potentially preventable, adverse health outcome in the management of type 2 diabetes.7–9 Hypoglycemia prevention is predicted on the ability to capture, track and evaluate events as they occur in real-world practice. In this meta-epidemiological review of observational studies of patients with type 2 diabetes, we found substantial heterogeneity in the definition, ascertainment and report of hypoglycemia, particularly for non-severe events.

Recognizing the importance of a uniform definition for hypoglycemia, on 21 November 2016, the IHSG proposed a taxonomy for non-severe (further subdivided into level 1 and level 2) and severe (level 3) hypoglycemia.12 13 The IHSG further advised that all clinical trials of diabetes management report on level 2 and level 3 hypoglycemia, with an option to also report level 1 hypoglycemia.12 13 However, observational (ie, non-randomized) studies are an invaluable source of information on adverse events in real-world settings and as such it was critical to examine how hypoglycemia was defined in such studies. This is especially important for retrospective studies that rely on secondary analyses of existing data collected for other reasons, whether for clinical care or billing/administrative purposes. To our knowledge, this is the first study to systematically examine the definitions of non-severe and severe hypoglycemia used by observational studies that form the evidence base for hypoglycemia prevention among adults with type 2 diabetes. Although most of the recent studies tend to adhere to the IHSG recommendations, the hypoglycemia definitions remained inconsistent. Overall, almost a fifth of the studies provided no definition of hypoglycemia at all. An additional 17% were loosely adherent, as they relied on ICD diagnosis codes from clinical encounters and may be construed to indirectly imply need for medication attention. Such heterogeneity in hypoglycemia reporting hinders comparisons across studies and precludes generalizable inferences about the safety of diabetes management across populations and settings.

Our study builds on earlier work demonstrating heterogeneity in hypoglycemia definitions in randomized controlled trials (RCTs) of diabetes therapies.14 Despite the IHSG recommendation, Balijepalli et al14 found that 40% of RCTs included in the Canadian Agency for Drugs and Technologies in Health report for second-line and third-line therapies for type 2 diabetes either did not report on hypoglycemia or did not specify the definition of reported events. Of the 60% that reported and defined hypoglycemia, only 14% used the IHSG definition for level 1 hypoglycemia and 20.8% for level 2 hypoglycemia.14 In contrast, our analyses that were restricted to the post-IHSG recommendation period found that hypoglycemia reporting in observational studies of type 2 diabetes was better, with 62.5% of studies on non-severe hypoglycemia and 54.1% of studies on severe hypoglycemia consistent with IHSG definitions. Nevertheless, substantial opportunity for improvement remains.

An important consideration for studies that leverage real-world data is how to optimally use administrative claims and electronic health records (EHRs) for large-scale hypoglycemia ascertainment and reporting. Doing so requires accurate and reliable identification of events, which in turn is predicated on patient’s reliably reporting events, healthcare providers consistently and uniformly documenting them and such documentation to be available in a format amenable to large scale ascertainment. Our analysis included both prospective and retrospective observational studies, and both study designs demonstrated heterogeneity in how hypoglycemia is defined. However, while prospective studies can homogenize their approach to hypoglycemia ascertainment by adopting IHSG definitions, retrospective studies that rely on secondary analysis of data collected for other reasons (eg, billing or routine care) require that hypoglycemia be uniformly defined and documented across all settings and not just research.

In our analysis, 3 of 116 (2.6%) studies that reported on non-severe hypoglycemia and 30 of 189 (15.9%) studies that reported on severe hypoglycemia used ICD codes to identify events. Because the IHSG definition of non-severe hypoglycemia is predicated solely on glucose levels, diagnosis codes and claims data cannot be used to establish a corresponding definition. Severe hypoglycemia is characterized by the need for third party or medical assistance. It can be inferred that ED or hospital encounters for hypoglycemia represent acute severe events. However, ambulatory documentation of hypoglycemia may reflect prior events being discussed in the office, both severe and non-severe, and not convey the frequency or timing of those events relative to the encounter. Diagnosis codes from ED-based or hospital-based encounters are less likely to be misclassified, particularly if the hypoglycemia code is listed as the primary or principle diagnosis for the acute event. This is the approach used by the Centers for Disease Control and Prevention to quantify severe hypoglycemic events.21 In contrast, many of the studies examined here did not specify the position of the hypoglycemia code in the encounter, what date range of claims was considered (ie, only from the date of hospital admission, only on the date of discharge or any day throughout the hospitalization) or even the setting(s) eligible for inclusion (ie, office evaluation and management visit, any ambulatory visit, ED visit, observation or inpatient hospital stay). Finally, there is heterogeneity in the specific ICD codes used to define hypoglycemia and whether studies relied on the Ginde algorithm,18 a modified version of the Ginde algorithm, or other codes entirely. Each of these parameters has the potential to alter event rates and study inferences. Nevertheless, it is important to note that up to 95% of severe hypoglycemic events do not culminate in an ED visit or hospitalization,7 22 23 and as such, studies that rely solely on claims data greatly underestimate their frequency.

Many observational studies rely on events documented as part of routine care (eg, registries, EHR and claims), yet collecting data about hypoglycemia in the real-world is challenging. Patients rarely volunteer information about hypoglycemia to their clinicians,24–28 and clinicians do not routinely screen their patients for hypoglycemia even when they are at risk for these events.7 29 As a result, patient-reported hypoglycemia is not easily captured in clinical practice, despite its association with increased all-cause mortality and impaired quality of life.30 31 Data from glucometers and continuous glucose monitors (CGMs), while valuable, is also not commonly available in the EHR, whether due to patients not using these devices (particularly in developing countries) or the inability of many practices, particularly in primary care, to consistently download device information into the EHR. Additionally, CGM use among patients with type 2 diabetes remains uncommon particularly when not treated with intensive insulin therapy.32 Reliance on events that do come to medical attention, whether in the ambulatory setting, ED or hospital, will miss most events and patients who experience them. Thus, it is critical to raise awareness among clinicians, patients and policy makers about the importance of routine and standardized hypoglycemia ascertainment and documentation, in accordance with ADA guidelines.33

This study should be considered in the context of its limitations. We focused on observational studies conducted among patients with type 2 diabetes. A large number of studies were excluded from analysis because they did not specify diabetes type and thus included patients with both type 1 and type 2 diabetes, as reliable classification of diabetes is often challenging in real-world data sources (references 214–243 of the online supplemental material).17 This contributed to the relatively small number of observational research studies analyzed. Our analyses included studies through May 2018, and hypoglycemia reporting may have improved over the past 2 years with greater attention and awareness paid to hypoglycemia by clinicians, professional societies and regulatory agencies. Nevertheless, our data point to the substantial gap in the quality of hypoglycemia ascertainment and reporting in research. This is confounded by persistent gaps in clinical hypoglycemia ascertainment7 and ultimately may contribute to inadequate understanding and correction of hypoglycemia risk factors among patients with diabetes.

Observational studies and real-world data are an invaluable evidence base for comparative effectiveness and safety research that complement knowledge gleaned from interventional trials. They are particularly useful when studying adverse drug events such as hypoglycemia. The marked heterogeneity in how hypoglycemia is defined, documented and reported is a major barrier to assessing its prevalence, identifying highest risk subpopulations, promoting screening for and disclosure of events and developing prevention strategies. As such, this work reinforces the urgent need to promote, facilitate and use standardized ascertainment, documentation and reporting of hypoglycemia in observational studies and in the data sources that feed them. Using tools such as the IHSG hypoglycemia definitions in research studies could homogenize hypoglycemia reporting and evaluation. Furthermore, patients’ education to recognize, report and manage hypoglycemia is a very important tool we can use right now to decrease mortality and morbidity. Ultimately, the ability to reliably study hypoglycemic events in real-world settings will support better risk stratification and prevention strategies aimed to stopping these common, harmful yet potentially preventable adverse events.