Analyse Accuracy of Glucose Monitors without any Dedicated Software Package.

Sir,

Diabetes mellitus (DM) is a global health issue, and India is among the top contributor to the global burden. Patients with diabetes, especially those on insulin therapy and/or oral hypoglycaemic agents with the potential to cause hypoglycaemia, need home blood glucose monitoring. This is possible in the current day thanks to low-cost portable blood glucose monitors. These consumer-grade devices need a small drop of blood to measure the glucose level in the blood with the advancement of the biosensors.[1] However, these devices should have a minimum level of accuracy for their successful usage in the detection of blood glucose. There are several criteria for finding the accuracy of the devices as shown in Table 1.[234567891011]

Table 1

Methods for observing accuracy of monitors for home glucose monitoring

Method	Name	Criteria
Number-based	ISO 15197: 2003	95% of BGM reading must be within±15 mg/dL of reference reading if glucose level is <75 mg/dL and within 20% if glucose level is ≥75 mg/dL[2]
	ISO 15197: 2013; reviewed in 2018	95% of BGM reading must be within±15 mg/dL of reference reading if glucose level is <100 mg/dL and within±15% if glucose level is ≥100 mg/dL; 99% of the values should be within zones A and B in consensus error grid[3]
	FDA (2003)	95% of BGM reading must be within±15 mg/dL of reference reading if glucose level is <75 mg/dL and within 20% if glucose level is ≥75 mg/dL[2]
	FDA (2016)	95% of BGM reading must be within±15% of reference reading and 99% within±20%, regardless of glucose level[4]
	ADA (1987)	100% of BGM reading must be within±10% of the reference reading, regardless of glucose level[5]
	ADA (1994)	100% of BGM reading must be within±5% of the reference reading, regardless of glucose level[5]
	CLIA (1988)	80% of the BGM reading must be within±10% or±6 mg/dL of the reference reading, whichever is larger[6]
Graph-based	CEGA (1987)	Grids divide the upper and lower zone of A (<±20% of reference; or both SMBG and reference is<70 mg/dL; clinically appropriate decision), B (>± 20% of reference value; no impact on the clinical decision), C (overcorrection of acceptable blood glucose level), D (dangerous failure in detection or treatment), E (erroneous value - opposite to the reference value; leads to opposite treatment [treatment of hyperglycaemia when patients are actually hypoglycaemic and vice versa]) [Figure 3a][78]
	PEGA (2000)	Separate error grids for Type 1 and Type 2 DM. Grids divide upper and lower zone of A (clinically accurate; no effect on clinical action), B (altered clinical action; little or no effect on the outcome), C (altered clinical action; likely affect the outcome), D (altered clinical action; significant clinical risk), E (altered clinical action; dangerous consequences) [Figure 3b][9]
	SEGA (2014)	Divided into risk levels of 0-0.5 (None), 0.5-1 (Mild, Lower), 1-1.5 (Mild, Higher), 1.5-2 (Moderate, Lower), 2-2.5 (Moderate, Higher), 2.5-3 (High, Lower), 3-3.5 (High, Upper), 3.5-4 (Extreme) [Figure 4b and 4c][10]
Other calculation	Modified Bland-Altman (1983)	Agreement between the reference and BGM reading by plotting reference blood glucose and difference (reference - meter) in glucose and separating accepted and non-accepted values with the help of gridlines according to defining criteria like ISO [Figure 2], FDA, ADA, and CLIA[11]

ADA: American Diabetes Association, BGM: Blood glucose monitor, CEGA: Clarke Error Grid Analysis, CLIA: Clinical Laboratory Improvement Amendments, FDA: Food and Drug Administration, ISO: International Organization for Standardization (https://www.iso.org/standard/54976.html), PEGA: Parkes Error Grid Analysis, SEGA: Surveillance Error Grid Analysis

In this article, we describe the method of calculation of accuracy tests without any statistical software package. This would help researchers from developing countries or resource-limited settings to carry out those statistical tests.

NUMBER-BASED ACCURACY

Calculation and tabular presentation

Accuracy according to the International Organization for Standardization (ISO),[23] Food and Drug Administration (FDA),[24] American Diabetes Association (ADA),[5] and Clinical Laboratory Improvement Amendments (CLIA)[6] can be calculated manually or in spreadsheet software like Microsoft Excel®. An example of the calculation according to ISO 15197:2013 is shown in Figure 1. In this example, there were 10 pairs of values of blood glucose. Among the reference glucose reading, four were less than 100 mg/dL; hence, these were tabulated according to the absolute difference. There were six reference glucose values ≥100 mg/dL; hence, these were tabulated according to the absolute difference in percentage. Overall (in all reference glucose levels), the monitor had 70% of the results within the limits as per ISO 15197:2013 criteria. Percentages of results within the limits were 75% for glucose concentrations <100 mg/dL and 66.67% for glucose concentrations ≥100 mg/dL.

Figure 1

Example of a calculation of the accuracy of a blood glucose monitor according to ISO 15197:2013 criteria

Visual output - modified Bland-Altman plot

The tabular presentations may be accompanied by a graphical presentation of the result. The original Bland–Altman plot has an “average of two measures” on the X-axis and “difference between two measures” on the Y-axis and a mean horizontal line with ± 1.96 standard deviation lines on both sides of the mean.[11] However, this plot is not used for presenting the result according to ISO 15197:2013. A modified (modification of X-axis) Bland–Altman plot is used. In this plot, the “reference glucose” (not the average of reference and meter glucose) is plotted on the X-axis, and the difference between reference and meter glucose is plotted on the Y-axis. A template of the modified Bland–Altman plot is shared in Figshare and available at the end of this letter. This template can be used to get the plot by entering reference and meter glucose values. An example of a modified Bland–Altman plot is shown in Figure 2.

Figure 2

A modified Bland–Altman plot according to ISO 15197:2013 criteria. The area within the green dotted lines indicates accepted limits. The straight horizontal line from 0 to 100 on X-axis indicates the criteria “within ± 15 mg/dL of reference reading if glucose level is <100 mg/dL” and the tangent lines indicate the criteria “within ± 15% if glucose level is ≥100 mg/dL”

GRAPH-BASED ACCURACY

Clarke and Parkes error grid analysis

Clarke Error Grid Analysis (CEGA) and Parkes Error Grid Analysis (PEGA) (also known as Consensus error grid analysis) can be carried out manually on a graph paper.[789] For the analysis, a large graph paper, ruler, pen, pencil, and a piece of white paper are needed. We have shared detailed coordinates for drawing grid lines of CEGA and PEGA in the template file at the end of this letter. According to the coordinates, the grid lines should be drawn on a graph paper with a pen, and the name of the zones should be written.

Take each data pair and plot it on the graph paper. For example, if we see Figure 1, the first pair of data is 73/80. To put the point on the graph paper, go 73 on X-axis and then 80 on Y-axis and mark the point with a pencil. This point would fall in a certain zone bounded by the gridlines. The zone (A, B, C, D, or E) should be noted down on the white piece of paper. Similarly, after putting the next point on the graph paper, write its zone on the white paper immediately. Follow this for all the points. Otherwise, it will be difficult to locate their location later due to overlapping or any point may be missed. This completes the calculation of zone wise distribution of coordinates (x, y pairs).[7]

For presenting the image in a manuscript, use the Excel® sheet we provided and put the data on the designated column, and the points will appear on the scatterplot. Save this as a portable document format for presenting it in the manuscript. An example figure for CEGA is shown in Figure 3a and PEGA is shown in Figure 3b.

Figure 3

Examples of error grid analysis graph – (a) Clarke Error Grid, (b) Parkes or Consensus Error Grid (for type 2 diabetes mellitus)

Surveillance error grid analysis

The Surveillance Error Grid Analysis (SEGA) can be conducted online from the following website: https://www.diabetestechnology.org/seg/.[10] A sample of comma-separated values (CSV) file can be downloaded from the website. The data of blood glucose from the reference method and glucose monitor should be entered (i.e., copy from the source file and paste) into the CSV file. While saving the file, either save it as “CSV UTF-8” or “CSV (Comma Delimited)” format. On the web page, click on the “Browse,” select the file, and wait for “Upload complete.” Click on the “Create Summary Tables” and “Create SEG Tables.” The process is shown briefly in Figure 4a. The graph can be accessed from the “SEG” tab, and zone-wise distribution can be seen from the “Summary Tables” tab. For downloading the image of the SEGA, click on the “Download SEG.” A truncated version of a result according to the SEG risk category is shown in Figure 4b, and the SEGA graph is shown in Figure 4c. The risk category may be presented in tabular format and SEGA graph as a figure in the manuscript.

Figure 4

(a) Brief steps for SEGA on the website (https://www.diabetestechnology.org/seg/), (b) SEG risk category-wise distribution of an example data, (c) an example SEGA graph

NOTE ON REFERENCE METHOD

For the measurement of blood glucose, the methods are categorized into three groups:- ”definitive,” “reference,” and “field” methods. The definitive method uses isotope dilution mass spectrometry for glucose. The reference method uses the hexokinase method of glucose estimation.[12] Finally, there are several field methods like the copper reduction method (modified Folin Wu), Ortho-Toluidine, and glucose oxidase-peroxidase (GOD-POD). The majority of the laboratories in developing countries use these field methods to estimate blood glucose. However, these methods are not as accurate as the hexokinase method.[13] Even if the commercially available device based on the hexokinase method is considered the reference method, it is not a perfect reference method as the “reference hexokinase method” needs a manual method of estimation of glucose using a protein-free filtrate.[12] Hence, in resource-limited settings, it is difficult to test the accuracy of glucose monitors. However, measuring surveillance accuracy is inevitable and should be checked to find if the glucose meters give an acceptable result. Wherever possible, the hexokinase method should be used. When this is not available, other field methods (e.g., GOD-POD) may be tried if that field method is the only available method for blood glucose estimation in that region for diagnosis and treatment purposes of patients suffering from DM. However, that surveillance accuracy should be interpreted with caution.

APPLICATION IN DIABETES CARE

Self-monitoring of blood glucose is crucial during the initial adjustment of insulin or drug dose. It also helps patients to detect hypo-/hyperglycaemia at home and can adjust insulin according to the guidance provided by the physicians. In an emergency, they may seek immediate help from telemedicine centres. In developing countries like India where test laboratories are not available in remote places, these devices can help in the management of patients suffering from DM. However, many of the marketed devices may not have an acceptable level of accuracy.[141516] Hence, post-marketing or surveillance accuracy tests are needed. However, in resource-limited settings, researchers may face technical difficulty in analysing graph-based accuracy.[16] In this article, we provided a brief guide to overcoming those technical difficulties in data analysis. We have described how researchers from any corner of the world can analyse the data without a computer. Additionally, we prepared templates for the graphs. Any researchers can use the template to present the result in their manuscript.

TEMPLATES

The modified Bland–Altman plot, CEGA, and PEGA template in Microsoft Excel® (version 2010) sheets are available from the following Figshare (https://figshare.com/) links:

Modified Bland–Altman plot: https://doi.org/10.6084/m9.figshare.19126844

CEGA: https://doi.org/10.6084/m9.figshare.17166620

PEGA: https://doi.org/10.6084/m9.figshare.17166644

CONCLUSION

We presume that this short guide would help the researchers from developing countries to conduct the tests for checking the accuracy of the glucose monitor without any premium statistical software packages.

Details of contribution

Himel Mondal: Concept, Literature search, Writing manuscript.

Shaikat Mondal: Concept, Revising manuscript.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.