Validity and reliability of a point-of-care nerve conduction device in diabetes patients.

AIMS/
INTRODUCTION: Although nerve conduction study (NCS) using a standard electromyography system (EMGS) is considered to be the gold standard in evaluating diabetic polyneuropathy, this examination requires expensive equipment and well-trained technicians. We aimed to validate a point-of-care device, NC-stat/DPNCheck™, that has been developed for widespread use of NCS in diabetic polyneuropathy.
MATERIALS AND METHODS: Diabetes patients underwent two kinds of NCS: DPNCheck™ and electromyography system. Inter-/intrarater reliability of DPNCheck™ were also determined by the intraclass correlation coefficient.
RESULTS: A total of 57 patients were evaluated. The parameters of NCS between the two methods correlated well (r = 0.7734 for the sural nerve conduction velocity, r = 0.6155 for the amplitude of sural nerve action potential). The intraclass correlation coefficients were excellent (intrarater: the velocity 0.767, the amplitude 0.811; interrater: the velocity 0.974, the amplitude 0.834).
CONCLUSIONS: The point-of-care device has excellent reproducibility and good agreement with standard electromyography system. The device might be useful to evaluate diabetic polyneuropathy.

Introduction

Among diabetic complications, diabetic polyneuropathy (DPN) develops in the earliest stage of diabetes and often progresses asymptomatically1, 2. As the early detection and severity classification of DPN are important to maintain high quality of life for diabetes patients3, nerve conduction study (NCS) using a traditional electromyography system (EMGS) has been developed as the gold standard examination for quantitative evaluation of DPN4, 5, 6. However, as a standard EMGS is costly and requires an advanced examination technique, EMGSs are available only in limited facilities. In order to overcome the poor availability of NCS, a point‐of‐care device, NC‐stat/DPNCheck™, was developed as a specialized NCS device to examine the sensory nerve conduction velocity (SNCV) and amplitude of sensory nerve action potential (SNAP) of sural nerves7, 8, 9. The device examines only a single nerve function, but it has already been shown that an individual NCS could detect impairment of the peripheral nervous system comparable with composite scores of nerve conduction abnormalities10. Therefore, this handheld device, which is inexpensive and requires no special training, might ultimately become indispensable in the diagnosis of DPN. Indeed, the device has been utilized as the most definitive examination technique for the diagnosis of peripheral neuropathy including DPN and other clinical settings in several reports11, 12, 13, 14. In addition, the NC‐stat/DPNCheck™ device performs very well when it is used simply to detect DPN, as compared with clinical examination15. However, only a few reports have investigated the reliability and validity of the device to diagnose DPN16, 17. Although a recent report validated the diagnostic ability of DPNCheck™ in Japanese18, no report has validated the reliability of the device itself in Asian diabetes patients. The current research is the first study to examine the reliability and validity of the nerve conduction parameters acquired by DPNCheck™: SNCV and amplitude of SNAP in sural nerves.

Methods

Study participants and clinical diagnosis of DPN

From July 2014 to November 2014, all patients who were previously diagnosed as type 1 or 2 diabetes mellitus and were hospitalized at Aichi Medical University Hospital to improve their hyperglycemia were invited to participate in the study. A total of 57 participants signed a document of consent for the study. Participants were screened for neurological dysfunction of the peripheral nervous system using the simple diagnostic criteria proposed by the Diabetic Neuropathy Study Group in Japan19. In brief, the criteria consist of a prerequisite condition and three neurological examination items. The prerequisite condition includes two items: (i) diagnosed as diabetes mellitus; and (ii) other neuropathies than DPN can be excluded. The criteria require any two or more of the following three items: (i) the presence of symptoms considered to be due to DPN; (ii) decreased vibration in the bilateral medial malleoli; and (iii) the decrease or disappearance of bilateral ankle tendon reflexes. Additionally, the criteria include important references in which, if either one of the following reference items is met, even if the above criteria are not met, diabetic polyneuropathy can be diagnosed: (i) presence of any abnormality in two or more nerves in the NCS; and (ii) presence of clinically apparent diabetic autonomic dysfunction. However, in the protocol of the current study, these two reference items were not applied due to the lack of normal limits in each nerve conduction parameter and the lack of definitions of autonomic dysfunction. Patients with diabetic ketoacidosis, severe infection or severe injuries were excluded.

NCS

The NCS in bilateral sural nerves was carried out utilizing DPNCheck™ (NeuroMetrix Inc., Waltham, MA, USA) and a standard EMGS (Neuropack X1, MEB‐2312; Nihon Kohden, Tokyo, Japan). The NCS using the standard EMGS was carried out in an air‐conditioned electrically shielded room by trained technicians. DPNCheck™ has been developed to evaluate the SNCV and amplitude of SNAP of sural nerves as described in previous reports16, 17. In brief, DPNCheck™ consists of three components: (i) a handheld device; (ii) a computer for data analysis; and (iii) a docking station that transmits data from the device to the computer. The device needs to be attached to a disposable biosensor that detects surface temperature, facilitates electrical stimulation and integrates the nerve conduction data. The NCS can start only under a surface temperature of 28 ± 5°C, and the acquired values are automatically corrected by the temperature. Well‐trained technicians who already had advanced skills for the NCS utilizing conventional EMGS evaluated each patient with the EMGS followed by DPNCheck™. The skin temperature was measured at the ankle, and the foot was warmed with a hot towel before testing when the temperature was below 32°C. For interrater analysis, another advanced technician examined sural nerve conduction functions using DPNCheck™ at the same time. For intrarater analysis, the same technician repeated the NCS using DPNCheck™ twice on the same day with each patient. Clinical information for each participant was withheld from all examiners. The cohort with 57 patients was divided into two groups; one group was for intrarater analysis and another was for interrater analysis. If SNAP were undetectable, no value of conduction velocity or amplitude was included.

Statistical analysis

SPSS Statistics version 20 for Windows (IBM SPSS, Chicago, IL, USA) and the statistical software R for Windows version 3.4.3 (The R Foundation for Statistical Computing, Vienna, Austria) were utilized for data analyses. Characteristics including age, sex, chemical laboratories, physiological findings and NCS parameters were presented as raw data. Student's t‐tests and χ2‐tests with Yates’ correction were used for analyses of differences in continuous and categorical variables, respectively. Correlation of the SNCV or SNAP amplitude between DPNCheck™ and the traditional EMGS were analyzed using Pearson's correlation coefficients. Intra‐/interrater reliability were analyzed with the intraclass correlation coefficient (ICC) and 95% confidence intervals (CI). The level of agreement was evaluated according to the six‐level nomenclature given by Landis and Koch20: (i) poor: 0.00; (ii) slight: 0.00–0.20; (iii) fair: 0.21–0.40; (iv) moderate: 0.41–0.60; (v) substantial: 0.61–0.80; and (vi) almost perfect: 0.81–1.00. Diagnostic validity was analyzed using a receiver operating characteristic curve and the area under the receiver operating characteristic curve (AUC).

Ethics

This study was carried out in accordance with the Declaration of Helsinki. Study procedures were approved by the ethics committee of Aichi Medical University Hospital (NO. 14‐019), and participants submitted their informed consent before participation.

Results

Clinical characteristics

The clinical characteristics of the participants are presented in Table 1. In total, one patient with type 1 diabetes and 56 patients with type 2 diabetes were included in the study (28 men, 29 women; 58.1 ± 14.4 years). The mean duration of diabetes was 8.9 ± 9.4 years. Participants had a mean glycated hemoglobin of 9.6 ± 2.0% and a mean body mass index of 25.9 ± 5.3 kg/m². Based on the simple diagnostic criteria, DPN was detected in 28.1% of the patients (n = 16) and not detected in 45.6% of the patients (n = 26). The assessments were not completed in 26.3% of the patients (n = 15). There were no significant differences in clinical parameters between patients with or without DPN except for the higher mean age in patients with DPN.

Table 1

Characteristics of the study population

Variables	Total cohort	Patients with DPN	Patients without DPN	P
No. patients	57	16	26
Sex (male/female)	28/29	6/10	14/12	0.4765
Age (years)	58.1 ± 14.2	63.9 ± 11.0*	53.2 ± 13.7	0.0137
Body mass index (kg/m2)	25.9 ± 5.3	25.2 ± 5.2	26.0 ± 6.1	0.3665
Type of diabetes (type 1/type 2)	1/56	0/16	1/25	0.8040
Duration of diabetes	8.9 ± 9.4	10.5 ± 11.2	7.5 ± 8.5	0.3670
HbA1c (%)	9.6 ± 2.0	9.7 ± 1.7	9.6 ± 1.9	0.8275

Data are presented as number of patients or mean ± standard deviation. *P < 0.05 versus patients without diabetic polyneuropathy (DPN). HbA1c, glycated hemoglobin; ND, not determined; P, P‐value between patients with diabetic polyneuropathy and without diabetic polyneuropathy.

Quantitative accuracy of DPNCheck™

Both NCSs showed decreased amplitudes of SNAP in patients with DPN (Table 2). However, no significant differences of nerve conduction velocities between patients with and without DPN were proven.

Table 2

Parameters of nerve conduction studies

Variables	Total cohort	Patients with DPN	Patients without DPN	P
Standard nerve conduction study
Sural nerve conduction velocity (m/s)	45.0 ± 5.4	44.5 ± 5.0	45.7 ± 5.2	0.3379
Sural nerve action potential (μV)	7.1 ± 5.5	5.7 ± 3.6*	8.9 ± 5.9	0.0003
Point‐of‐care device
Sural nerve conduction velocity (m/s)	48.4 ± 6.4	47.3 ± 5.9	48.8 ± 5.9	0.2864
Sural nerve action potential (μV)	9.6 ± 5.7	8.6 ± 5.7*	10.8 ± 5.9	0.1159

Data are presented as mean ± standard deviation. The participants whose action potential was not detected were excluded from calculation of mean values. *P < 0.05 versus patients without diabetic polyneuropathy (DPN). P, P‐value between patients with diabetic polyneuropathy and without diabetic polyneuropathy.

The correlations between the values acquired by traditional EMGS and DPNCheck™ were excellent in the sural nerve conduction velocity (SuNCV; r = 0.8160, 95% CI 0.7388–0.8721) and fair‐to‐good in the amplitude of sural nerve action potential (SuNAP; r = 0.6255, 95% CI 0.5000–0.7246). The linear regression equations of SuNCV and the amplitude were (SuNCV by DPNCheck™) = 0.9876 × (SuNCV by EMGS) + 3.6039 and (amplitude by DPNCheck™) = 0.6513 × (amplitude by EMGS) + 4.9222, respectively. The comparisons are shown in scatterplots and Bland–Altman plots (Figure 1). The scatterplots showed strong correlations of conduction velocities or amplitudes between the methods (Figure 1a,b). The Bland–Altman plots evaluated the agreement of values (Figure 1c,d). Although these analyses showed good correlations between standard EMGS and DPNCheck™ in SuNCV and SuNAP, DPNCheck™ produced higher values compared with EMGS (mean difference of SuNCV: +3.03 m/s, SuNAP: +2.41 μV). As the difference appears to become larger in the range of high values both in velocities and amplitudes, these disagreements might be systemic bias, which depends on the magnitudes of measurements.

Figure 1

Agreement analyses of nerve conduction studies between the point‐of‐care device (POCS) and standard electromyography system (EMGS) in the sural nerve. (a,b) Scatterplots of (a) sensory nerve conduction velocities (SNCVs) and (b) amplitudes of sensory nerve action potential (SNAP) between two methods. (c,d) Bland–Altman plots showing the (c) difference of SNCVs and (d) amplitudes of SNAP between two methods. The solid red lines represent the mean difference and the dotted red lines represent the upper or lower limits of the 95% confidence interval.

Intrarater repeatability of DPNCheck™ was “almost perfect” in SuNCV (ICC 0.877, 95% CI 0.792 ‐0.928) and the amplitude of SuNAP (ICC 0.842, 95% CI 0.763–0.893; Table 3, Figure 2a,b). Similarly, interrater reliability showed substantial agreement for SuNCV and “almost perfect” agreement for SuNAP, with ICC values of 0.783 (95% CI 0.640–0.874) and 0.807 (95% CI 0.703–0.872), respectively (Table 3, Figure 2c,d).

Table 3

Intra‐/interrater reliability outcomes for DPNCheck™ parameters

Parameter	Measurement 1	Measurement 2	Bland‐Altman blots	ICC
Intrarater
Amplitude (μV)	9.0 ± 4.6	10.1 ± 6.0	1.1 (−5.1, 7.3)	0.842 (0.763, 0.893)
Conduction velocity (m/s)	47.9 ± 6.7	48.4 ± 6.6	0.5 (−6.1, 7.1)	0.877 (0.792, 0.928)
Interrater
Amplitude (μV)	11.0 ± 6.5	10.6 ± 6.6	−0.3 (−8.5, 7.8)	0.807 (0.703, 0.872)
Conduction velocity (m/s)	48.4 ± 5.5	49.0 ± 5.9	0.6 (−6.9, 7.8)	0.783 (0.640, 0.874)

Data of measurements 1 and 2 are presented as mean ± standard deviation, Data of Bland–Altman blots are presented as mean difference (95% lower and upper limits of agreement). Data of intraclass correlation coefficient (ICC) are presented as ICC (95% confidence intervals).

Figure 2

Interclass reliability analyses of nerve conduction studies by the point‐of‐care device in the sural nerve. (a,b) Intrarater reproducibility analyses with scatterplots of (a) sensory nerve conduction velocities (SNCVs) and (b) amplitudes of sensory nerve action potential (SNAP) between two measurements carried out by one rater. (c,d) Interrater reliability analyses with scatterplots of (c) SNCVs and (d) amplitudes of SNAP between two raters.

To determine the diagnostic potential of DPNCheck™, we evaluated each NCS parameter utilizing receiver operating characteristic analysis. The AUC of the amplitudes by DPNCheck™ showed moderate accuracy (0.696), which was comparable with that by the standard EMGS (0.721; Figure 3a,b). The threshold values with maximized accuracy were ≤6 μV for DPNCheck™ (sensitivity 86.5%, specificity 43.8%) and ≤3 μV for the standard EMGS (sensitivity 96.2%, specificity 40.6%). In contrast, the conduction velocities had low diagnostic accuracy in DPN. The AUCs were 0.582 in DPNCheck™ and 0.615 in the EMGS (Figure 3c,d). The threshold value with maximum accuracy was ≤44 m/s (sensitivity 71.2%, specificity 53.6%) for values by the EMGS. However, the threshold value for DPNCheck™ could not be ascertained because of the lack of a distinguishable prominence on the convex curve.

Figure 3

Receiver operating characteristic curves validating the diagnostic potential of nerve conduction methods in the sural nerve to identify diabetic polyneuropathy. (a,b) Receiver operating characteristic curves of amplitudes of sensory nerve action potential obtained by (a) the point‐of‐care device and (b) the standard electromyography system. (c,d) Receiver operating characteristic curves of sensory nerve conduction velocities obtained by the (c) point‐of‐care device and (d) electromyography system. *An optimal threshold for diagnosis.

Discussion

We examined a cohort of 57 diabetes patients with or without DPN to assess the validity and reliability of the point‐of‐care nerve conduction device. The validity of the device was expressed by correlation between the values of the SNCV or SNAP in sural nerves retrieved from the device and conventional EMGS. The correlation coefficient was good: r = 0.8160 for SNCV and r = 0.6317 for SNAP. Furthermore, intra‐/interrater reliability of the devices that were analyzed using ICC were substantial or almost perfect according to Landis and Koch's classification. As all four ICCs were >0.75, the reliability of the simplified device appeared to be comparable with or might be better than the reliability of standard EMGS (Table 4)21.

Table 4

Comparison of intraclass correlation coefficients in sural nerve conduction studies between DPNCheck™ and a standard electromyography system

Parameter	ICC of DPNCheck™		ICC of EMGS†
	Intrarater	Interrater	Intrarater	Interrater
Amplitude	0.87	0.84	0.69–0.88	0.74
Conduction velocity	0.88	0.78	0.52–0.72	0.79

†Data were extracted from Yasuda et al.19 EMGS, electromyography system; ICC, intraclass correlation coefficient.

DPNCheck™ is a device that allows the NCS of sural nerves to be examined within only several minutes without requiring a trained technician. There are several differences in the inspection method between the device and standard EMGS. First, DPNCheck™ stimulates sural nerves orthodromically, whereas standard EMGS uses antidromic stimuli. The SNAP evoked by orthodromic stimulus would be principally smaller than that by antidromic stimulus. However, in the current study, orthodromic stimuli by DPNCheck™ resulted in higher SNAPs compared with those by the EMGS with antidromic stimuli. The unexpected results could be explained by a difference of utilized stimulation intensity. DPNCheck™ utilizes stronger electric stimuli up to 70 mA. In accordance with our protocol for the standard EMGS, maximum stimuli for sural nerves are approximately 20 mA. Therefore, the stronger stimulus by DPNCheck™ could avoid technical obstacles caused by a lower leg edema or the increase in skinfold thickness. The second difference is whether the temperature correction algorithm was applied. The biosensor of DPNCheck™ monitors skin temperature using an infrared digital thermometer, and the SNCV is normalized to 28°C using a temperature correction factor of 1 m/s per 1°C. In contrast, in cases utilizing the standard EMGS, when skin temperature was maintained >32°C, no correction algorithm for skin temperature was applied. Third, the definition of latency is different. The latency in DPNCheck™ begins at the end of the stimulation pulse, whereas that in the standard EMGS begins at the start of the pulse. These second and third differences might account for the disagreement of the conduction velocity between DPNCheck™ and standard EMGS. These measurement biases should be considered when interpreting the results to expand the versatility of the device in future.

Notwithstanding the systemic measurement bias, the current study showed good validity of DPNCheck™, which will allow quantitative evaluation of sural nerve functions by the device in diabetes patients. In the diagnostic validity analysis, amplitudes of SNAP showed higher AUC compared with the SNCV. The AUC of the amplitude with a sensitivity of 86.5% and a specificity of 43.8% were comparable with the previous report using standard EMGS5. The positive predictive value of sural nerve amplitude for concurrent DPN was 71.4% and the negative predictive value was 66.7% using the threshold value 6.0 μV, which was higher than the threshold value 3.0 μV in amplitude of sural nerves derived from standard EMGS. This disagreement of threshold values was probably caused by systemic bias between two methods, as explained above. Some previous studies reported threshold values for maximum accuracy of DPN diagnosis5, 16, such as Lee et al., which reported a threshold value of ≤6.0 μV with 88% sensitivity and 94% specificity. Although their value is fortuitously the same as the present result, we should keep in mind the differences in the definition of DPN in their study protocol, in which DPN was identified by one or more abnormal nerve conduction result(s) in the sural or peroneal nerves. Furthermore, we should recognize the basic background for DPN diagnosis. The new point‐of‐care device is insufficient for assessment of systemic sensorimotor neuropathy in diabetes. As the device is just suitable for a screening of neuropathy, comprehensive NCS is essential for detailed assessment of DPN. Aside from these discussions, the excellent validity of the device might actualize widespread availability of the NCS in various medical settings using DPNCheck™ as a substitute for conventional EMGS.

The present study had some limitations. First, the study cohort was relatively small and examined at a single academic medical center. However, the margins of error are acceptable in a NCS that has generally low reproducibility (margins of error with 95% CI in SuNCV by the point‐of‐care device: 1.7, margins of error in SuNAP: 1.6). Although the current study might have low generalizability, the study design achieves high fidelity on procedures of the NCS, and showed the accuracy and reproducibility of the device. Second, although DPNCheck™ is expected to be used in general facilities including primary care clinics, technicians who were professionals of neuroelectrophysiological examinations engaged in the study. However, encouragingly, excellent validity of DPNCheck™ has been achieved by two non‐technical personnel in a previous report from Toronto, Canada16. Future studies should investigate the validity in Japanese, or pan‐Asian, contexts carried out by non‐technical staff. Third, although the simple diagnostic criteria proposed by Diabetic Neuropathy Study Group in Japan were used for the diagnosis of clinical DPN, the criteria consist of physical signs and symptoms of peripheral neuropathy. As it has been reported that diagnosis of DPN by experts using signs and symptoms was excessively variable22, the Japanese criteria should be exploited with an identical method. Fortunately, as all physical examinations and clinical interviews in the present study were carried out by diabetologists who belong to one institution, it is considered that the criteria were applied almost homogeneously.

In conclusion, DPNCheck™ sufficiently provides valid measurements of the NCS in sural nerves. The parameters derived from the device could be used to indicate concurrence of clinical DPN in diabetes patients. In future, this device might develop into a useful diagnostic tool for DPN.

Disclosure

Jiro Nakamura received a research grant from Fukuda Colin Company Limited. The other authors declare no conflict of interest.