Performance evaluation of all analytes on the epoc® Blood Analysis System for use in hospital surgical and intensive care units.

OBJECTIVE: To evaluate the performance of the epoc hand-held analyzer against the RAPIDPoint 500 blood gas analyzer and laboratory analyzers where applicable.
METHODS: Venous or arterial whole blood samples collected in balanced heparinized syringes were obtained from 69 patients (35 females, 34 males) predominantly (77%) from the surgical unit and intensive care unit (ICU). Method comparison was performed for all analytes on the epoc System against the RAPIDPoint 500 Blood gas analyzer or laboratory analyzers where applicable.
Results: Mean bias was <5% for blood gases, electrolytes, lactate and glucose. Hematocrit showed a bias of -6.76% (95% CI ​= ​-8.91, - 4.61) compared to the HemataSTAT-II method, whereas calculated total hemoglobin showed a bias of 1.51% (95% CI ​= ​-1.04, 4.06) against the Sysmex XN-10 hematology analyzer. Creatinine showed the largest bias relative to laboratory analyzers, Abbott Architect c8000 Jaffe method (13.54%, 95% CI ​= ​5.43, 21.65) and Roche Cobas c702 enzymatic method (30.01%, 95% CI ​= ​12.64, 47.38). Conclusions: The epoc system is fit for use in the surgical and ICU setting for the measurement of all analytes except for creatinine. Crown

Introduction

The epoc® Blood Analysis System (Siemens Healthineers, Oakville, Ontario, Canada) is a relatively newer, portable, point-of-care (POCT) device and has recently been compared to various other POCT, blood gas, and hematology instruments [[1], [2], [3], [4], [5]]. This system is comprised of credit card-sized test cards and a wireless card reader. The test cards contain sensors and their associated fluidics compartments, with a sample entry port, and a sealed calibrator fluid reservoir [6]. A single test card can be used to measure blood gases (pCO2, pO2 and pH), electrolytes (sodium, potassium, ionized calcium, chloride), lactate, glucose, creatinine and hematocrit. The epoc® system provides calculated values including bicarbonate, total carbon dioxide, oxygen saturation, anion gap, estimated glomerular filtration rate and total hemoglobin [6]. These values are resulted within 30 ​s of sample introduction. The epoc® system imparts key advantages including the availability of single-use cards that can be stored at room temperature, wireless connectivity and fast turnaround time, making it a suitable system for several settings. As a result, various reports have validated the use of the epoc® in surgical, intensive care, emergency and outpatient hospital units [[1], [2], [3],5], where these analyzers are in high demand.

Studies evaluating epoc® performance show varying degree of concordance between epoc® and other POCT devices. Samples obtained from cardiopulmonary bypass patients showed good correlation between epoc® and GEM4000 (Instrumentation Laboratory, Bedford, Massachusetts, USA) and epoc® and iSTAT (Abbott Point of Care, Princeton, New Jersey, USA) in measurements of pH, pCO2, pO2, sodium, potassium, ionized calcium, glucose, lactate, and hemoglobin [2]. However, a study comparing epoc® and RAPIDPoint 500 (Siemens Healthineers, Oakville, Ontario) analyzers showed significant biases for hemoglobin and lactate and borderline biases for pCO2 and glucose [7]. Additionally, most studies on the epoc® system have not evaluated all analytes available on the platform. Notably, there is a paucity of studies that compare the performance of creatinine on the epoc® against routine laboratory analyzers. POCT creatinine measurements are important in several hospital settings, in imaging units as well as in hospital-at-home assessments.

In this study, we aimed to evaluate the performance of all twelve analytes available on the epoc® system against routine laboratory analyzers. Importantly, we report a bias in creatinine measurements between epoc® and routine laboratory analyzers, Architect c8000 (Abbott, Abbott Park IL, USA) and Cobas c702 (Roche, Basel, Switzerland). Further, we elucidate the potential impact of this bias in creatinine measurements on estimated glomerular filtration rate (eGFR) calculations and the classification of patients in different stages of chronic kidney disease (CKD).

Methods

Study participants and samples

Venous or arterial whole blood samples collected in balanced, heparinized syringes (Portex, 23.5 IU/ml heparin syringes) were obtained from 69 patients (35 females, 34 males) predominantly (77%) from the surgical and ICU units (Supplemental Table 1: patient locations). Requirement for Research Ethics Board (REB) approval has been waived.

Laboratory instruments and tests

The epoc® system measures pH, pCO2, sodium, potassium and ionized calcium potentiometrically; pO2, glucose, and lactate amperometrically, whereas hematocrit is determined conductometrically. Hemoglobin is calculated from the measured hematocrit using the formula: Hemoglobin (g/L) ​= ​Hematocrit (decimal fraction) ​× ​340 [6]. Creatinine on the epoc® is measured amperometrically via a creatinine sensor based on an enzymatic reaction [6]. Two different epoc® readers (serial number 08173 and 23,512) were used in our method comparison analysis. Two different lot numbers of blood gas electrolyte and metabolite (BGEM) test cards were used as the sample collection was staggered and batches of samples were analyzed on different days. Our analysis of precision (data not shown) was conducted prior to the start of the study using 2–3 levels of quality control material and 10 runs per level for each analyte. For both epoc® readers, our coefficients of variation were ≤5% for all analytes across all levels except for the low level of pO2 and creatinine, which were ≤15%.

The epoc® system was compared to RAPIDPoint 500 (Siemens, ON, Canada) for all analytes, except hematocrit and creatinine. Hematocrit measurements on epoc® were compared to those obtained on HemataSTATII (EKF Diagnostics, Cardiff, U.K.). Creatinine measurements on epoc® were compared to those obtained on Architect c8000 and Cobas c702. Total hemoglobin measurements were also compared against the Sysmex XN-10 (Sysmex Corp, Kobe, Hyogo, Japan). Whole blood samples were analyzed on the epoc®, RAPIDPoint 500, HemataSTATII and Sysmex XN-10. These whole blood samples were then spun, and plasma was collected for creatinine analysis on Architect c8000 and Cobas c702 laboratory analyzers. Samples were visually inspected to ensure that samples included in the study did not exhibit lipemia or hemolysis that may interfere with the analysis. Measurements of analytes on the epoc® and the comparator method were carried out within the same hour for comparisons between the epoc® versus RAPIDPoint 500, HemataSTATII and Sysmex XN-10. Particularly, comparisons between the epoc® and the RAPIDPoint 500 took place in the same location and were performed within 10 ​min of each other. Measurements of creatinine on core laboratory analyzers took place within 3 ​h of the sample being analyzed on the epoc®. Plasma samples were stored at 4 ​°C during this delay. All creatinine assays (epoc®, Architect c8000 and Cobas c702) are traceable to the isotope dilution mass spectrometry reference method. Samples were analyzed in a single replicate.

Data analyses

The R program version 3.5.2 Copyright (C) 2018 was used to perform all data analysis and produce graphs. The mcr package (version 1.2.1) was used for method comparison analyses (slope and intercept from unweighted Deming regression) and to generate figures. The blandr package was used to calculate mean bias as well as upper and lower limits of agreement (LOA) and 95% confidence intervals (Table 1). The fmsb package was used to calculate the kappa-statistic for the assessment of agreement between instruments in creatinine-based CKD staging. Bias was calculated by subtracting the value obtained on the epoc® from the comparator method (RAPIDPoint500, HemataSTATII, Sysmex XN-10, Architect c8000, Cobas c702). Percentage bias was calculated by dividing the bias by the value for the concentration obtained on the comparator method. Power calculation was done assuming IQMH APL limit for effect size (cohen d) and combined SD for both methods using both the blandPower and the pwr packages in R. All analytes, except for the creatinine comparison against cobas c702 had at least 80% to detect IQMH APL difference assuming a significance level of 0.05 (two-sided). Power calculation for Bland-Altman analysis as proposed by Lu et al. [8] could not be performed due to wide SD of the differences, which often exceeded allowable difference.

Table 1

Summary of epoc® method comparison analysis.

Analyte (Unit)	Method X (Method Y ​= ​epoc)	N	Analyte Concentration Ranges (epoc)	Slope of regression line (95% C.I.)	Intercept (95% C.I.)	Mean Bias (95% C.I.)	Mean Bias range	% Bias (95% C.I.)	% Bias range	Upper LOA (95% C.I.)	Lower LOA (95% C.I.)
PCO2 (mmHg)	RAPIDPoint500	69	21.4, 133.8	1.00(0.93, 1.06)	2.2(-2.0, 6.5)	1.97(0.38, 3.55)	-26.5, 17.3	3.98 (1.61, 6.35)	-21.4, 28.6	14.89 (12.17, 17.62)	-10.96 (-13.68, -8.24)
PO2 (mmHg)	RAPIDPoint500	69	12.8, 299.5	1.00 (1.01, 1.08)	-3.4(-6.8 to 0.0)	0.14 (-1.48, 1.76)	-13.0, 25.8	-1.00(-2.63, 0.63)	-14.7, 21.3	13.38 (10.6, 16.17)	-13.1 (-15.88, -10.31)
pH	RAPIDPoint500	69	6.98, 7.48	1.00 (0.96, 1.03)	0.1 (-0.2 to 0.3)	0.00 (0.00, 0.01)	-0.0, 0.0	0.04 (-0.02, 0.10)	-0.4, 0.6	0.04 (0.03, 0.04)	-0.03 (-0.04, -0.02)
Ionized Calcium (mmol/L)	RAPIDPoint500	69	0.83, 1.4	1.20 (1.1, 1.29)	-0.2 (-0.3 to -0.1)	0.00 (0.01, 0.02)	-0.8, 0.1	1.17 (0.40, 1.94)	-8.8, 8.1	0.09 (0.07, 0.1)	-0.06 (-0.07, -0.04)
Chloride (mmol/L)	RAPIDPoint500	69	88, 121	1.10 (1.03, 1.21)	-10.9 (-19.8 to -2.0)	1.39 (0.88, 1.91)	-3.0, 7.0	1.35 (0.86, 1.84)	-3.0, 6.7	5.59 (4.71, 6.48)	-2.81 (-3.69, -1.93)
Potassium (mmol/L)	RAPIDPoint500	69	2.4, 6.4	1.00 (0.99, 1.07)	-0.3 (-0.4 to -0.1)	-0.15 (-0.17, -0.13)	-0.4, 0.0	-3.78 (-4.36, -3.20)	-10.4, 0.5	0.03 (-0.01, 0.07)	-0.33 (-0.37, -0.29)
Sodium (mmol/L)	RAPIDPoint500	69	126, 156	1.00 (0.94, 1.16)	-6.1 (-21.4 to 9.3)	0.56 (-0.12, 1.23)	-5.6, 6.5	0.42 (-0.06, 0.90)	-4.0, 4.5	6.09 (4.92, 7.25)	-4.98 (-6.14, -3.81)
Lactate (mmol/L)	RAPIDPoint500	69	0.81, 17.30	1.10 (1.02, 1.11)	-0.3 (-0.5 to 0.0)	0.02 (-0.14, 0.17)	-0.9, 2.6	-4.88 (-8.85, -0.91)	-41.5, 38.1	1.26 (1, 1.52)	-1.23 (-1.49, -0.96)
Glucose (mmol/L)	RAPIDPoint500	69	1.7, 29.2	1.00 (0.97, 1.01)	0.0 (-0.2 to 0.3)	-0.05 (-0.16, 0.06)	-1.1, 1.0	-0.57 (-2.06, 0.92)	-21.6, 12.2	0.86 (0.67, 1.05)	-0.96 (-1.15, -0.77)
Hematocrit (%)	HemataSTAT-II	32	17, 60	1.10 (1.02, 1.15)	-4.7 (-6.9 to -2.5)	-1.91 (-2.62, -1.19)	-6.0, 3.0	-6.76 (-8.91, -4.61)	-17.9, 6.8	1.96 (0.73, 3.19)	-5.77 (-7, -4.55)
Total Hemoglobin (g/L)	Sysmex XN-10	68	57, 198	1.40 (1.00, 1.75)	-9.8 (-19.3 to -0.2)	1.60 (-0.69, 3.89)	-21.0, 28.0	1.51 (-1.04, 4.06)	-21.2, 37.9	20.15 (16.22, 24.09)	-16.95 (-20.88, -13.01)
Total Hemoglobin (g/L)	RAPIDPoint500	69	57, 204	1.20 (1.10, 1.26)	-24.5 (-33.2 to -15.8)	-6.06 (-8.41, -3.71)	-27.0, 26.0	-6.73 (-9.11, -4.35)	-25.7, 40.0	13.13 (9.09, 17.16)	-25.24 (-29.28, -21.2)
Creatinine (μmol/L)	Architect c8000	68	28, 947	1.00 (0.93, 1.04)	14.9 (1.3–28.5)	12.15(2.27, 22.02)	-75.0, 190.0	13.54(5.43, 21.65)	-44.0, 118.8	92.12 (75.16, 109.08)	-67.83 (-84.79, -50.87)
Creatinine (μmol/L)	Cobas c702	30	28, 488	1.10 (0.92, 1.20)	20.0 (-10.6 to 50.6)	29.23 (9.19, 49.27)	-34.0, 216.0	30.01 (12.64, 47.38)	-35.4, 161.2	134.43 (99.79, 169.06)	-75.96 (-110.6, -41.32)
eGFRml/min/1.73 m2	Architect c8000	66	4, 136	1.00 (0.91, 1.06)	-5.1 (-10.15 to -1.06)	-5.58 (--9.04, -2.12)	-47.0, 20	-7.67 (-12.54, -2.8)	-60.5, 33.3	22.01 (16.06, 27.95)	-33.16 (-39.1, -27.22)
eGFRml/min/1.73 m2	Cobas c702	29	7, 118	0.90 (0.75, 1.09)	-7.2 (-20.64 to 0.77)	-12.90 (-19.89, -5.9)	-51.0, 16.0	-17.77 (-26.41, -9.13)	-68.1, 18.9	23.16 (11.06, 35.26)	-48.95 (-61.05, -36.85)

Reference interval concordance analysis was performed using reference intervals listed in Table 2. Our analysis tested whether samples that were within the reference interval when measured on epoc® were also within the reference interval when measured on the comparator instrument. The reference interval concordance, therefore, indicates the percentage of agreement between epoc® and the comparator method for each specific analyte. Reference interval concordance analysis was performed using intervals that were validated for the RAPIDPoint 500, Architect c8000, HemataSTAT-II and Sysmex XN-10 instruments.

Table 2

Summary of epoc® reference interval concordance and acceptability criteria analysis.

Analyte	Units	Reference Method	N	Reference Interval	Concordance based on R.I. (%)	Samples passing acceptability criteria (%)
						BV	CLIA	CAP (blood gas survey)	IQMH
PCO2	mmHg	RAPIDpoint500	68	A: 35–45V: 40-52	88	14	64	64	61
PO2	mmHg	RAPIDpoint500	68	A: 80–100V: 30-50	94	NA	96	NA	91
pH	None	RAPIDpoint500	68	A: 7.35–7.45V: 7.31–7.41	96	NA	99	99	93
Ionized Calcium	mmol/L	RAPIDpoint500	69	1.12–1.32	84	7	99	NA	93
Chloride	mmol/L	RAPIDpoint500	69	100–110	91	20	96	96	91
Potassium	mmol/L	RAPIDpoint500	69	3.2–5	96	19	94	100	84
Sodium	mmol/L	RAPIDpoint500	69	135–145	78	7	81	81	81
Lactate	mmol/L	RAPIDpoint500	69	0.5–2	93	48	70	75	67
Glucose	mmol/L	RAPIDpoint500	69	3.8–7	97	19	81	91	83
Creatinine	umol/L	Architect c8000	68	F: 50–98M: 64-110	76	15	49	69	37
Creatinine	umol/L	Cobas c702	30	F: 50–98M: 64-110	80	17	40	53	33
Hematocrit	%	HemataSTAT-II	32	F: 33–47M: 42-54	97	13	31	38	59
Total Hemoglobin	g/L	Sysmex XN-10	68	F: 120–160M: 140-180	96	21	38	57	90
Total Hemoglobin	g/L	RAPIDpoint500	68	F: 120–160M: 140-180	96	9	17	32	87

We also assessed the percentage of samples passing acceptability criteria, i.e. whether the difference between measurements from epoc® and the comparator instrument was within the allowable error criteria set by biological variation in healthy individuals (European Federation of Clinical Chemistry and Laboratory Medicine or Westgard), CLIA, CAP and IQMH. These criteria are detailed in Supplemental Tables 2A–B.

To investigate the impact of the bias in creatinine measurements between the epoc® system and routine laboratory analyzers, we estimated the glomerular filtration rate for each patient using creatinine measurements obtained from the various instruments. We used the CKD-EPI equation [9] for the estimated glomerular filtration rate (eGFR), based on which, patients were classified in different CKD stages. For each CKD stage, we calculated the percentage of concordance between the stage assigned using eGFR based on creatinine obtained from epoc® versus routine laboratory analyzers.

Results

Method comparison experiments were performed using venous or arterial whole blood samples from 69 patients (35 females, 34 males) predominantly (77%) from the surgical and ICU units (Supplemental Table 1). Deming regression analyses results are summarized in Table 1 and can be visualized in the graphs in Fig. 1A–I and Supplemental Figs. 1–3. Good correlation was observed between tests on the epoc® and RAPIDPoint 500 analyzers.

Fig. 1

Bias in creatinine measurements between epoc® and routine laboratory analyzers.

A-I. Deming regression and Bland-Altman plots showing comparisons between Creatinine (A–D) and eGFR (E–I) measurements on epoc® vs. Architect c8000 and Cobas c702 systems. J. Bias in creatinine measurements was seen in some of the repeated samples (n ​= ​11) but not others. Bland-Altman plot shows the difference Method Y-Method X along with mean bias 95% C.I. in dotted lines. Plot shows lines of agreement (LOA) and their associate 95% C.I. in dotted lines. ARCHITECT ​= ​Architect c8000, Cobas ​= ​Cobas c702.

Blood gases, electrolytes and chemistries

Analyses of blood gases (pCO2,pO2, pH), electrolytes (Na, K, Ca and Cl), lactate and glucose measured on the RAPIDPoint 500 blood gas analyzer and compared to the epoc® analyzer showed slopes of regression between +1.0 and ​+ ​1.2 and correlation coefficients (R) between 0.9 and 1.0. Intercepts were more variable and ranged from -10.9 (Chloride) to +2.2 (pCO2).

Overall, few measurements showed relatively small bias in epoc® compared to RAPIDPoint 500 (pCO2 +3.98%, K: 3.78%, Ca: +1.17%, Cl: +1.35%, lactate: 4.88%).

Total hemoglobin and hematocrit

Total hemoglobin calculated on the epoc® showed relatively weaker correlation with measurements obtained on the Sysmex XN-10 analyzer (n ​= ​30, slope: +1.4, R: 0.7, intercept: 9.8, mean bias: +1.6 ​g/L). Although total hemoglobin obtained on the epoc® correlated better with RAPIDPoint 500, the epoc® had a significant negative bias compared to the RAPIDPoint 500 (n ​= ​69, slope: +1.2, R ​= ​1.0, intercept: 24.5, mean bias: 6.06 ​g/L). Hematocrit measured on the epoc® analyzer correlated well with the measurements performed on the HemataSTAT-II hematocrit analyzer (n ​= ​32, slope: +1.1, R: 1.0, intercept: 4.7).

Compared to the HemataSTAT-II hematocrit analyzer, epoc® showed a bias of -6.76% in hematocrit measurements. Compared to the Sysmex-XN-10 analyzer, epoc® showed a bias of +1.15% in total hemoglobin measurements.

Creatinine

Creatinine measured on epoc® enzymatic method correlated well with creatinine measured using the Architect c8000 Jaffe method (n ​= ​68, slope: +1.0, R: 1.0) and with creatinine measured by Cobas c702 enzymatic method (n ​= ​30, slope: +1.1, R: 0.9). However, the intercepts observed were large (+14.9 ​μmol/L for Architect c8000 and ​+ ​20.0 ​μmol/L for Cobas c702). Estimated GFR, calculated using the CKD-EPI equation based on creatinine from the epoc® showed a slightly lower correlation and a notable intercept with eGFR obtained on the Architect c8000 (R: 0.9, intercept: 5.1 ​ml/min/1.73 ​m2) or the Cobas c702 analyzers (R:0.8, intercept: 7.2 ​ml/min/1.73 ​m2).

Creatinine measurements between epoc® and Architect c8000 showed a significant bias of +12.1 ​μmol/L (+13.54%), whereas epoc® and Cobas c702 systems showed a significant bias of +29.2 ​μmol/L (+30.01%). The large bias in creatinine also resulted in a negative bias in eGFR of -5.58 ​ml/min/1.73 ​m2 (-7.67%) in epoc® compared to the Architect c8000 and a negative bias of -12.90 ​ml/min/1.73 ​m2 (-17.77%) in epoc® compared to the Cobas c702. Notably, a large range of differences in creatinine measurements were seen in comparisons of epoc® against both the Abbott Jaffe (-75.0 to +190.0 ​μmol/L) and the Cobas c702 enzymatic method (-34.0 to +216.0 ​μmol/L).

To assess whether the bias in creatinine measurements between epoc® and the Architect c8000 and Cobas c702 platforms was specific to certain patients, we selected a subset of samples from our study where two or more consecutive creatinine measurements were performed on a single patient (Fig. 1J). Our data shows that when there is a significant bias between the analyzers in a patient sample, it is consistently seen in other samples obtained from the same patient. In Fig. 1J, patients P3, P5 and P11 are good examples of this effect.

We further calculated eGFR using creatinine values from different instruments and classified patients into the five CKD stages as determined by the KDIGO guidelines (Table 3). A similar percentage of patients were classified into CKD stage I (Architect c8000: 93%, Cobas c702: 100%) and stage V (Architect c8000: 92%, Cobas c702: 86%). However, there was greater variability and less agreement in percentage of patients classified into stage II (Architect c8000: 35%, Cobas c702: 17%), IIIa (Architect c8000: 0%, Cobas c702: 40%), IIIb (Architect c8000: 86%, Cobas c702: 67%), and IV (Architect c8000: 57%, Cobas c702: 33%). Additionally, we calculated the kappa statistic to assess the agreement in CKD classification using creatinine values from epoc® versus routine laboratory analyzers. CKD staging using creatinine obtained on the Architect yielded a kappa statistic of 0.53 (95% C.I. of 0.38–0.67) when compared to the epoc® creatinine-based staging. CKD staging using creatinine obtained on the Cobas c702 yielded a kappa statistic of 0.49 (95% C.I. of 0.28–0.71) when compared to the epoc® creatinine-based CKD staging. These kappa-statistics indicate only a moderate agreement between creatinine-based CKD staging derived from the epoc® compared to that obtained on routine laboratory analyzers. Additionally, compared to the Architect c8000, the creatinine measurements obtained on the epoc® tended to classify patients at a higher stage of CKD overall as shown in Fig. 2.

Table 3

Concordance in CKD staging between epoc®, Architect c8000 and Cobas c702 analyzers.

CKD Stage	eGFR range (ml/min/1.73 m2)	N (Architect c8000 comparison)	% Concordance with Architect c8000	N (Cobas c702 comparison)	% Concordance with Cobas c702
I	≥90	14	93	5	100
II	60–89	17	35	6	17
IIIa	45–59	8	0	5	40
IIIb	30–44	7	86	3	67
IV	15–29	7	57	4	33
V	<15	13	92	7	86
OVERALL		66	62	30	59

Fig. 2

CKD staging based on eGFR calculated from serum creatinine measured on Architect c8000 and epoc® analyzers. Numbers on the side of each bar graph represent the number of patients classified in a specific CKD stage using eGFR based on serum creatinine from either the Architect c8000 or epoc® analyzers. Circles represent eGFR obtained from Architect c8000 creatinine measurements and squares represent eGFR obtained from epoc® creatinine measurements. ARCHITECT ​= ​Architect c8000, Cobas ​= ​Cobas c702.

Acceptability criteria

We next investigated whether the difference we observed between the epoc® analyzer and the RAPIDPoint 500 or routine laboratory analyzers were within allowable limits as determined based on biological variation, CLIA, CAP and IQMH. The most up-to-date criteria from each source were used for this analysis and are shown in Supplemental Table 1. Table 2 shows the percentage of samples where the difference between the instruments was within the acceptable criteria tested.

Only a minor percentage of samples ranging from 7% for sodium and calcium to 48% for lactate showed a bias that was less than what would be observed from biological variation alone. This finding is expected since the limit for the desirable bias allowable based on biological variation is small, ranging from 0.3% for sodium and 3.7% for creatinine (Supplemental Table 2A). The percentage of samples that passed the acceptability criteria for CLIA, CAP and IQMH was similar for blood gases (61–64% for pCO2, 91–96% for pO2 and 93–99% for pH), electrolytes (93–99% for calcium, 91–96% for chloride, 84–100% for potassium and 81% for sodium) and metabolic analytes (67–75% for lactate and 83–91% for glucose). The percentage of samples passing the acceptability criteria was smaller and more variable across CLIA, CAP and IQMH for creatinine. The values ranged between 37% and 69% in our comparison between epoc® and Architect c8000 and 33%–53% in our comparison between epoc® and Cobas c702.

Reference interval concordance

Blood gases and pH measured on the epoc® demonstrated high reference interval concordance with RAPIDPoint 500 measurements (pCO2: 88%, pO2: 94% and pH: 96%) as shown in Table 2. Hematocrit showed high concordance between epoc® and HemataSTAT-II (97%). Total hemoglobin also showed a high concordance between epoc® and both RAPIDPoint 500 (96%) and Sysmex XN-10 (96%). Reference interval concordance was more variable in electrolytes, ranging from 78% for sodium and 96% for potassium, but was higher for metabolic analytes (93% for lactate and 97% for glucose). Creatinine measurements on epoc® had the lowest reference interval concordance (76%) with those from the Architect c8000 and a relatively lower concordance with creatinine from Cobas c702 (80%).

Discussion

The epoc® Blood Analysis System showed good agreement with RAPIDPoint 500 for blood gases, electrolytes, lactate, glucose and with Sysmex XN-10 and HemataSTAT-II for total hemoglobin and hematocrit, respectively. The bias in measurements of blood gases, electrolytes, lactate and glucose, between the epoc® and RAPIDPoint500 was minimal (-4.9 to +4.0%). Overall, our findings resemble the range of biases reported for these analytes in comparison studies between epoc® and GEM4000 (-2.1% to +5.1%) and epoc® and iSTAT (-5.2 to +6.7%) [2,10]. Reports on values obtained in emergency and community paramedic settings also showed similar agreement between epoc® and routine laboratory analyzers for blood gases, electrolytes, hemoglobin and hematocrit [4,11].

We observed the largest bias in creatinine measurements between epoc® and Architect c8000 (+12.1 ​μmol/L or +13.5%) and epoc® and Cobas c702 systems (+29.2 ​μmol/L or +30.0%). A recent review of different POCT instruments that measured creatinine also showed a wide range of biases between handheld and blood gas POCT analyzers and routine laboratory analyzers [12]. Significant proportional and absolute bias have been reported between the i-STAT hand-held analyzer and the Vitros 750 [13], Roche Cobas c702 enzymatic method [[14], [15], [16]], Beckman Synchron CX7 Jaffe method [17] and the Beckman DxC800 Jaffe assay [18]. Moreover, significant bias has been reported between the creatinine measured on Nova StatSensor and routine laboratory enzymatic methods [16,[19], [20], [21], [22], [23]]. The StatSensor and the iSTAT, have been reported to exhibit a bias in creatinine of +16% and +6% respectively against the routine laboratory analyzers [5]. Thus, biases in creatinine measurement have been shown across different POCT devices compared with other POCT or routine laboratory analyzers.

The differences in creatinine that we observed resulted in differences between in eGFR between the epoc® and both the Architect c8000 (-5.58 ​ml/min/1.73 ​m2, -10.5%) and Cobas c702 systems (-12.90 ​ml/min/1.73 ​m2, -23.4%). Our findings are similar to those from a recent study where three hand-held point-of-care devices were compared against the Siemens IDMS calibrated enzymatic creatinine assay [5]. This study showed +8% bias in eGFR values between epoc® and the routine laboratory analyzer. Although both epoc® and the Cobas c702 methods use the enzymatic assay based on creatininase, their detection methods are different. The epoc® analyzer and most other POCT devices, uses amperometric biosensors that detect current produced in the enzymatic reaction, whereas the enzymatic creatinine assay on the Cobas c702 is a colourimetric method. Thus, the detection method used in the creatinine sensor could be a contributing factor to the bias observed in creatinine measurements between POCT and routine laboratory Cobas c702 analyzers. However, this idea requires further testing to be established as a root cause of the differences observed.

A recent report attributed a positive bias in creatinine to an increase in hematocrit; this study compared five POCT creatinine methods to the Roche enzymatic creatinine assay using samples (n ​= ​40) from hemodialysis patients before and after dialysis [24]. The authors reported a +18% increase in creatinine results on the epoc® with an increased hematocrit, whereas decreased hematocrit showed no effect [24]. Our findings do not show any association between hematocrit and bias in creatinine values between epoc® and routine laboratory analyzers (Supplemental Fig. 4A–D). However, our analyses demonstrate that the differences in creatinine measurements appear to be patient specific. This finding is consistent with a study by Straseski et al. where a comparison between the Nova StatSensor vs. the Roche enzymatic assays was evaluated, the authors identified a subgroup of 22 patients with ≥44 ​μmol/L between the two methods [20]. Additionally, four of the 22 patients had more than one discrepant sample. An investigation into the discrepant samples revealed 17/22 (77%) were from individuals with renal insufficiency or renal-related diagnosis, 5/22 (23%) were categorized with some type of renal dysfunction and the 4 patients with multiple discrepant samples were all diagnosed with end-stage renal disease or renal failure. Among our samples with consistent bias in creatinine, 3/11 (27%) had a calculated eGFR of <15 ​ml/min/1.73 ​m2 which falls into the CKD Stage V category and 2/11 (18%) had an eGFR between 30 and 44 ​ml/min/1.73 ​m2 which falls under CKD Stage IIIb category. However, other than the association with a diagnosis of renal dysfunction, the Straseski et al. report does not show any contribution of an interfering substance or differences in the whole-blood matrix for the discrepant patients.

Different interferences can potentially affect Jaffe creatinine assays, such as bilirubin, glucose, or protein, or enzymatic assays, such as dopamine, although the susceptibility and extent of interference can vary depending on the assay configuration [[25], [26], [27], [28], [29]]. However, none of the samples included in our study showed any lipemia or hemolysis when running the creatinine assay on either the Architect c8000 or Cobas c702 instruments. Bias in creatinine measurements between epoc® and routine laboratory analyzers did not correlate with the concentration of glucose in the samples (Supplemental Fig. 4C and D). Additionally, we observed a bias of -11.2 ​μmol/L (-8.9%) in the Cobas c702 enzymatic assay relative to the Architect c8000 Jaffe assay (Supplemental Fig. 5A–D). However, Jaffe assays have been shown to be more susceptible to interferences and have been demonstrated to result in higher creatinine values compare to enzymatic and IDMS methods [30,31]. There was a higher level of reference interval concordance (90%) and a higher percentage of samples passing acceptability criteria (CLIA: 63%, CAP: 67%, IQMH: 70%) (Supplemental Tables 3A–B) between routine laboratory analyzers than between epoc® and each of the Architect c8000 or Cobas c702 analyzers.

Creatinine has a wide-ranging utility at the point of care including the evaluation of renal function prior to contrast-enhanced imaging or drug dosing (specially in patients prescribed nephrotoxic drugs) [5,32]. A number of studies have discussed the use of creatinine POCT in screening for CKD and acute kidney injury and in monitoring of renal function in kidney transplant recipients [19,23,33]. However, there are currently no clinical guidelines on the use of POCT creatinine measurements in these settings. Although the bias in eGFR observed in some patients is within allowable error in our study, it has the potential to directly impact patient care. In fact, drug dosing and contrast media administration protocol guidelines often have medical decision points at eGFR values between 10 ​ml/min/1.73 ​m2 and 60 ​ml/min/1.73 ​m2 where most discrepancies are occurring between epoc® and routine laboratory analyzers [5,34].

The strengths of our study include our comparison of all analytes offered on the epoc® analysis system against the RAPIDPoint 500 in a hospital population including patient samples from the surgical unit and ICU where there is an increasing demand for its use. Our findings highlight important differences differences in creatinine measurements between the epoc® and routine laboratory analyzers. We have further investigated the clinical impact of this bias by elucidating how it affected CKD classification. However, the limitations of our study include the fact that the comparator methods used in our experiments are not reference or gold standard methods. In analytes where we found a larger bias such as creatinine, our study could have merited from a comparison across platforms using reference material, or comparison against a reference laboratory method. Therefore, it is important to interpret our results with caution when implementing the epoc system with assays on different laboratory platforms. As well, with regards to our investigation of the effect of the bias bias in creatinine on eGFR and CKD staging, our analysis would benefit from additional samples in each CKD stage to better understand concordance between the instruments.

In conclusion, we have shown that the performance of the epoc® system is acceptable for the measurement of all analytes except for creatinine in various hospital units including the surgery and ICU.