Point-of-Care Blood Glucose Meter Accuracy in the Hospital Setting.

Point-of-care (POC) tests provide analytical information that can be used to make decisions at patients' bedside, as opposed to laboratory tests that must be run at a central laboratory. POC testing is a widely used tool to enable immediate determination of glucose levels in hospitalized patients and facilitate rapid treatment decisions in response to fluctuations in glycemia. Accurate POC glucose testing requires attention to various factors before, during, and after performance of tests. These include 1) proper preparation of test sites to avoid preanalytical errors, 2) proper identification of tested patients whose physiological status permits sampled capillary specimens to correlate with central venous blood glucose levels to avoid analytical errors, and 3) proper documentation of the fidelity of meter results with the medical record to avoid postanalytical errors.

Diabetes and hyperglycemia are common problems for hospitalized patients. It is necessary to monitor blood glucose levels in these patients so that appropriate types and quantities of medications and food can be delivered to them. Two trends in hospital care are making it more important than ever for patients to be accurately tested for abnormal glucose levels while in the hospital. First, an ever-increasing acutely ill mix of patients can be found in hospitals, and these patients can break down quickly if they develop either hypo- or hyperglycemia. Second, achieving target glycemic levels using accurate blood glucose monitoring is necessary to improve outcomes of hospitalized patients.

Three types of initiatives are underway to improve the accuracy of point-of-care (POC) blood glucose monitoring in hospitals, including efforts to 1) improve the accuracy of blood glucose monitoring devices, 2) eliminate sources of error, and 3) improve the safety of testing. Improved accuracy of blood glucose monitors has been shown to lead to a higher quality of insulin dosing decisions. Modeling studies have demonstrated that, when monitors are inaccurate, resulting insulin dosing errors can lead to iatrogenic hypoglycemia in some cases. It would not be ethical to conduct trials of inaccurate meters on human subjects and make treatment decisions using meters known to measure erroneously.

Improving Accuracy

In the United States, there has been a movement to demand more accurate blood glucose meter performance for hospitalized patients. This point was made at a public meeting held by the U.S. Food and Drug Administration (FDA) in March 2010, where many diabetes professionals requested a new two-track regulatory approach to distinguish the needs of patients who use meters at home from those of health care professionals (HCPs) working in hospitals or other acute care settings where POC testing is performed.

On 7 January 2014, the FDA released two draft guidance documents for blood glucose monitoring test systems.[1,2] At the time this article was written, the two documents were being distributed for a 90-day public comment period and were not yet finalized. One guidance is for prescription POC use by professionals in settings that include hospitals and acute and chronic care settings. The other guidance is for over-the-counter (OTC) self-monitoring devices used by laypeople. The first guidance document is relevant to hospital monitoring of blood glucose in both critical and noncritical care settings.

Draft FDA Guidance for Prescription POC Monitors

Before release of the prescription POC guidance, most blood glucose monitoring devices, including those intended for use by HCPs in a hospital setting, were submitted to the FDA based on accuracy claims for OTC devices use by patients. Performance studies needed for FDA approval were conducted on outpatients with diabetes. Hospital-specific issues, including extreme states of hydration or blood pressure, imbalances of analytes, or severe undercurrent illnesses or other unusual physiological states associated with hospitalization, were not a significant part of the approval process. In this guidance, the FDA has specified that the performance of prescription blood glucose monitors must now be tested based on issues that arise in the intended-use population for which FDA approval of the device is sought.

The proposed prescription POC blood glucose monitor guidance specifies that, to demonstrate that a blood glucose monitoring system is sufficiently accurate to be used safely by HCPs, the manufacturer should demonstrate that 99% of all values are within a range of ± 10% of the reference method for glucose concentrations > 70 mg/dl and within ± 7 mg/dl at glucose concentrations < 70 mg/dl. Also, to avoid crucial patient management errors, no individual result should exceed ± 20% of the reference method for samples > 70 mg/dl or ± 15 mg/dl for samples < 70 mg/dl.The previous standard, which the FDA used for clearance of both OTC and prescription POC devices was the 2003 International Organization for Standardization (ISO) standard that required 95% of the measured glucose values to fall within ± 15 mg/dl of the average measured values of the reference measurement procedure at glucose concentrations < 75 mg/dl or within ± 20% at glucose concentrations ≥ 75 mg/dl.[3]

ISO 15197-2013 (E) Standard

ISO 15197-2013 (E) is an international standard that delineates requirements for in vitro glucose testing measuring capillary blood glucose concentrations and performance validation by intended users (e.g., people with diabetes). It should be noted that the latest ISO standard, which was released in 2013 and is expected to be adopted by the European Union in 2016, does not specify separate accuracy requirements for patient systems and professional systems. Furthermore, the degree of accuracy required for both types of system in this ISO standard is less stringent than the proposed FDA requirements.[4] The FDA POC guidance states specifically that, “FDA believes that the criteria set forth in the ISO 15197 standard do not adequately protect patients using BGMS [blood glucose monitoring system] devices in professional settings, and does not recommend using these criteria for BGMS devices.”[1]

According to ISO 15197-2013 (E), also known as “in vitro diagnostic test systems—requirements for blood glucose monitoring systems for self-testing in managing diabetes mellitus,” the required accuracy is as follows: 95% of the measured glucose values shall fall within ± 15 mg/dl of the average measured values of the reference measurement procedure at glucose concentrations < 100 mg/dl or within ± 15% at glucose concentrations ≥ 100 mg/dl. Also 99% of individual glucose measured values shall fall within zones A and B of the Parkes Error Grid for type 1 diabetes. The FDA guidance does not mention any error grids. Figure 1 shows the Parkes Error Grid.[1]

Figure 1.

Parkes Error Grid for patients with type 1 diabetes.

Blood glucose monitor performance can be described by analytical accuracy or clinical accuracy. Analytical accuracy is a quantitative method for describing how closely the result of a measurement by a device compares with a measurement by a reference method for this assay. Clinical accuracy is a qualitative metric for describing the clinical outcome of making a treatment decision using the result of a measurement from a device being assessed. A blood glucose error grid is a graph that plots blood glucose monitor values on the y-axis and reference blood glucose values on the x-axis. A set of risk zones is superimposed on the graph such that each zone represents a likely clinical outcome resulting from inaccuracy in the measured glucose values. Outcomes can range from appropriate treatment to slightly inadequate treatments requiring no corrections to serious errors causing life-threatening outcomes, with many gradations in between. Error grids permit a data set to be defined on the basis of the percentage of data points that fall into each zone of risk. An error grid developed by the Diabetes Technology Society in cooperation with the FDA, the American Diabetes Association (ADA), The Endocrine Society (TES), and the Association for Advancement of Medical Instrumentation (AAMI), known as the Surveillance Error Grid, was released this year and is discussed in more detail below. This metric is expected to be used by the FDA to assess the performance of cleared blood glucose monitors.[5]

Evaluation Studies for Prescription POC Monitors

The recent 2014 guidance for prescription POC glucose monitors specifies five types of evaluation studies: 1) a precision evaluation, 2) a linearity evaluation, 3) a method comparison/user evaluation, 4) interference evaluations, and 5) flex studies.[1]

The precision evaluation must express the precision as a mean value, standard deviation, and percentage coefficient of variation (defined as the ratio of the standard deviation divided by the mean). The linear evaluation should demonstrate adequate performance of the system across the range of glucose levels for which clearance is sought. In addition to the percentage of data points falling within a percentage of the reference method, other methods for expressing the liner analytical accuracy of a blood glucose monitor include calculating the correlation with reference method based on the bias, slope, intercept, and r2, as well as the extent of agreement or consistent bias comparing the monitor method to a reference method using a Bland Altman plot.[6] Multiple-method comparison and user performance studies for blood glucose monitoring systems include multiple users and multiple devices. An interference evaluation studies the effect of potentially interfering endogenous and exogenous substances on device performance. These include icterus, hyperlipemia, and varying hematocrit levels, as well as some common medications. Such potential interfering medications include acetaminophen (which is being increasingly used in hospitals for pain relief), ascorbic acid, L-dopa, and xylose. Flex studies stress the operational limits of a test system and should be used to validate the insensitivity of the test system to performance variation under stress conditions. The FDA recommends four types of flex studies, including 1) mechanical vibration testing, 2) shock testing, 3) electromagnetic compatibility, and 4) electrostatic discharge/electromagnetic interference testing.[1]

Several POC devices meet current FDA standards for accuracy. Future “next-generation” POC devices will likely be required to meet stricter standards for FDA clearance, especially if the 2014 draft guidance for professional POC devices is adopted. No POC monitor is currently approved by the FDA for critically ill patients, which includes patients who are typically in the intensive care unit, operating room, recovery room, or emergency room of a hospital. This means that use of POC devices in these settings is considered an off-label use.

The Centers for Medicare & Medicaid Services (CMS) regulates all U.S. laboratory testing except research through the Clinical Laboratory Improvement Amendments (CLIA) program.[7] The purpose of this program is to ensure quality laboratory testing. POC blood glucose testing in hospitals is under the purview of CMS. Although use of POC devices is a standard of care in virtually every hospital in the United States, the off-label status of this practice is not in accordance with CLIA regulations.[8] Therefore, a solution to this situation will be needed to permit continued legal use of these products in these settings.

Hypoxemia, poor perfusion, and tissue edema can artificially alter the measured blood glucose results of POC devices. Blood glucose monitors require oxygen to complete the chemical reaction that generates an electric current that is proportionate to the blood glucose concentration. The accuracy of some glucose meters is degraded by states of hypoxemia or low partial pressure of oxygen concentration. Also, glucose is metabolized when blood transitions from arteries to capillaries to veins. Therefore, venous samples (compared to capillary samples) will produce lower glucose results from many blood glucose monitors. During states of poor perfusion, blood may be shunted away from skin toward central organs. The oxygen concentration in skin in these cases is low. Therefore, blood glucose measurements from POC devices may be artificially lower in blood samples from patients with poor perfusion. It is necessary to always follow the instructions on the label of the specific device used. If a device is approved for capillary glucose testing, it should not be used on blood samples from other sources. Edematous patients might be subject to dilution of a capillary fingerstick sample by fluid in the edematous tissue overlying the capillary, which can artificially lower the measured blood glucose concentration.[9]

Critically ill patients may be affected by hypoxemia, poor capillary perfusion, or tissue edema. In these states, the accuracy of the POC blood glucose devices may be inadequate, and blood glucose readings in these situations can lead to insulin dosing errors. Little empirical research has been conducted in critically ill patients to assess the contribution of the above three risk factors or other potential risk factors for inaccurate performance of POC blood glucose monitoring devices.

Food residue on fingers can contain glucose and lead to artificially high readings from POC testing. Therefore, hands should be cleaned with soap and water and dried (rather than cleaned with alcohol) before a fingerstick blood glucose test is performed.[10]

Analytical Versus Clinical Accuracy

As previously mentioned, the performance of blood glucose monitors can be described in terms of either their analytical or their clinical accuracy. Analytical accuracy describes how closely the result of the measurement method being assessed correlates to a measurement from a reference method. Clinical accuracy describes the clinical consequences of a treatment decision based on a result by the measurement method being assessed. The need to consider clinical accuracy, especially in postmarketing analyses of specific scenarios of analytically inaccurate performance by an approved product or by a specific lot of an approved product by regulatory agencies, arises because not all blood glucose errors of a particular percentage range from the reference method have equal clinical significance.[11]

Analytical accuracy is measured by quantitative statistical metrics such as precision, bias, and percentage of data points lying within a certain percentage of the reference value. The term “reference value” in assessment of a blood glucose monitor refers to a laboratory-based glucose measurement method that has been well validated for precision and accuracy and is traceable to a higher order, such as an internationally recognized reference material or method. The traceability chain should include as few stages as possible to reduce bias. Clinical accuracy is measured by comparing paired data points of the results of the measurement being evaluated, along with the results from a reference method as defined above, with the paired data points plotted on a grid, which is known as an error grid.[11]

Clinical Accuracy Metrics for Blood Glucose Monitor Performance

Error grid analysis of blood glucose monitors was developed in 1987 by Clarke et al.[12] at the University of Virginia to quantify the clinical accuracy of patient-determined blood glucose values. This first error grid, known as the Clarke Error Grid, analyzed the clinical relationship between a patient-generated blood glucose value and a reference blood glucose value when a treatment decision is based on the patient-generated blood glucose value. Later in 1995, Parkes et al.[13] developed an updated error grid (known as the Parkes Error Grid or the Consensus Error Grid), which they published in 2000.

The good clinical accuracy of the GlucoWatch device, as determined by error grid testing, was an appealing feature of the product.[14] This contributed to its approval as this first continuous glucose monitor marketed in the United States.

As mentioned above, the Diabetes Technology Society is working with the FDA, ADA, TES, AAMI, and representatives from the diabetes diagnostics industry on an updated error grid.[15] Such a new metric is needed because new practices and treatments that are now standards of care were not well established when the earlier error grids were developed. This new metric, which will be known as the Surveillance Error Grid, will be based on work that started in 2012 and is expected to be published in 2014.

Preanalytical, Analytical, and Postanalytical Factors

The performance of blood glucose monitors can be adversely affected by preanalytical (obtaining the sample), analytical (performing the assay), or postanalytical (reporting the results) factors. The main preanalytical factors are related to operator performance and test strips. Operator activities that affect preanalytical accuracy include overlooked food residue on the fingertips, site selection (if arterial or venous blood is sampled instead of capillary blood, for which most monitors are indicated), improper lancing technique, and improper sampling of a hanging blood drop. Strip-related factors that can adversely affect preanalytical accuracy include product use past the expiration date, exposure to heat and humidity, or imprecision resulting from lot-to-lot variability.

In addition to any inherent inaccuracy of the blood glucose monitoring system itself (e.g., an inaccurate monitor, inadequate calibration process, or improper testing method), analytical factors can also adversely affect blood glucose monitoring accuracy.[16] These include 1) extreme physical environments (e.g., high altitude, humidity, heat, or cold, especially if vasoconstriction occurs), 2) nonstandard physiology of the patient (e.g., extreme hematocrit, extreme partial pressure of oxygen, or severe hypertriglyceridemia), and 3) concomitant use of certain medications, including in some cases ascorbic acid, acetaminophen, or D-xylose.[17] Environmental, physiological, and pharmacological factors that affect monitor performance are known as “interfering substances.”

Postanalytical factors that can adversely affect blood glucose monitor performance include display of incorrect units of glucose (e.g., mg/dl instead of mmol/l or vice versa), failure to upload data into an electronic database, incorrect data transmission to an electronic database, presentation of a misleading message, or failure to respond appropriately to glucose data. At a hospital in Italy, it was determined that, when 1,966 individual POC glucose test results were performed and then manually entered into an electronic database, there were errors of missing information (12.1% of measurements), missing time of testing (7.2%), and incorrect results (3.2%). [18] Newer blood glucose monitors for hospital systems offer real-time electronic transfer of data, thus eliminating data entry errors.

The preanalytical, analytical, and postanalytical factors that are most likely to occur in a hospital setting are not the same as those that might typically occur during blood glucose testing in an outpatient setting.[19] Plebani[20] reported a series of hospital laboratory errors divided into preanalytical, analytical, and postanalytical categories. The causes and distributions of that hospital's errors are presented in Table 1. Because of the risk and liability of measurement errors, many hospitals have written policies prohibiting the use of information from OTC blood glucose meters that patients bring into the hospital and operate themselves. The FDA has categorized the most common blood glucose monitor errors in terms of their potential sources (e.g., errors caused by monitor design, production, or use). Six error source categories and examples of each are presented in Table 2.

Table 1.

Phases in Diagnostic Processing Leading to Missed Diagnoses

Table 2.

Potential Sources of Error in Blood Glucose Monitors Based on FDA Experience

Conclusions

Proper inpatient glycemic management requires timely blood glucose results with careful consideration of sample size, patient comfort, test time, nursing work flow, cost, and ability to automatically transfer results into the electronic medical record so they are readily available to clinicians to make treatment changes. Prescription blood glucose monitors for use by HCPs at the bedside are already widely used for this purpose. Regulatory bodies in the United States and Europe are requiring progressively greater levels of accuracy for these products. Even with accurately performing monitors, it is necessary to follow proper procedures to avoid errors. Preanalytical errors resulting from poor sampling or strip storage can cause inaccuracy. Measurement errors caused by perturbations in patient’s physiological state or interference from concomitant use of medications can result in analytical errors. Improper data-handling can result in postanalytical errors. Proper training for health care staff and knowledge of how these devices work are both necessary to get the highest quality and most useful information from blood glucose monitoring in the hospital setting.