Performance Evaluation of the i-SmartCare 10 Analyzer and Method Comparison of Six Point-of-Care Blood Gas Analyzers.

Blood gas, electrolyte, glucose, and lactate level measurement have an immediate and critical impact on patient care. We evaluated the performance of i-SmartCare 10 (i-SENS Inc., Seoul, Korea) and conducted a method comparison study of five point-of-care (POC) analyzers with i-SmartCare 10 as the comparator, according to the CLSI guidelines. Ten analytes (pH, pCO2, pO2, Na+, K+, Cl-, iCa2+, glucose, lactate, and Hct) were tested on six analyzers: i-SmartCare 10, ABL90 FLEX PLUS (Radiometer Medical ApS, Copenhagen, Denmark), i-Stat (Abbott Point of Care Inc., Princeton, NJ, USA), RapidLab 1265 (Siemens Healthcare Diagnostics Inc., Tarrytown, NY, USA), Stat Profile pHOx Ultra (Nova Biomedical, Waltham, MA, USA), and Gem Premier 5000 (Instrumentation Laboratory, Bedford, MA, USA). The total imprecision and linearity (r2>0.99) were excellent, except for a few analytes that narrowly escaped the preset criteria. Interference was noted for Na+ in the presence of a high K+ level and for iCa2+ in the presence of high K+ and Mg2+ levels. Forty of 48 items demonstrated either a proportional or systematic difference in regression analysis; the relative mean difference (%) of 14/48 items escaped the allowable total error in the difference plot analysis. i-SmartCare 10 shows acceptable performance, and using a single POC blood gas analyzer is recommended for monitoring.

Given the immediate and critical impact of blood gas, electrolyte, lactate, and glucose level measurements on metabolic and respiratory management, most blood gas analyzers are used on-site for point-of-care (POC) testing [1-3]. As POC analyzer users are generally non-laboratory personnel who are unfamiliar with instrument maintenance and QC, a cartridge-type analyzer with simple maintenance and QC is preferable for POC testing [4]. Recently, i-SmartCare 10 (i-SENS, Seoul, Korea), a novel POC analyzer that includes all components necessary for sample analysis in a single, multi-use cartridge, requiring one min or less for analysis, has been released.

In many institutions, various POC analyzers are used in different sites. Given the concerns about the interchangeability of POC analyzer results, a comparative study of POC analyzers is required. A few comparative studies of POC analyzers have been reported; however, those on currently available analyzers are limited [5-10]. We evaluated the performance of i-SmartCare 10 and conducted a method comparison study of five commercial blood gas analyzers with i-SmartCare 10 as the comparator in testing 10 analytes according to the most up-to-date CLSI guidelines [11-15].

The i-SmartCare 10, ABL90 FLEX PLUS (Radiometer Medical ApS, Copenhagen, Denmark), i-Stat (Abbott Point of Care, Princeton, NJ, USA), RapidLab 1265 (Siemens Healthcare Diagnostics Inc., Tarrytown, NY, USA), Stat Profile pHOx Ultra Blood Gas Analyzer (Nova Biomedical, Waltham, MA, USA), and Gem Premier 5000 (Instrumentation Laboratory, Bedford, MA, USA) analyzers were used in this study.

Residual whole blood samples (N=209) were randomly collected from patient samples. As samples were used anonymously, the need for informed consent was waived according to the local ethical guidelines. This study was approved by the Institutional Review Board (IRB) of Samsung Medical Center, Seoul, Korea (IRB No. 2018-12-055) and conducted in accordance with the Declaration of Helsinki. Ten analytes (pH, pCO2, pO2, Na+, iCa2+, K+, Cl−, glucose, lactate, and Hct) were tested by all analyzers except i-Stat, which does not test Cl− and lactate.

Total imprecision was evaluated according to the CLSI EP5-A3 guideline using the i-Smart QC materials (i-SENS, Seoul, Korea) [11]. Two levels of QC materials were used for Hct and three levels for the other nine analytes. The obtained precision estimates were compared with the Ricos desirable specifications for imprecision and allowable total error (TEa) and the specifications provided by the manufacturer [16, 17].

Linearity was evaluated according to the CLSI EP6-A guideline with five levels of calibration materials (Calibration Verification Controls; RNA Medical, Devens, MA, USA) [12]. The measured mean of duplicate measurements was calculated and compared with the expected value to yield recovery. Tests were considered acceptable if the recovery ranged 90-110% and the 95% confidence interval (CI) for the slope of the linear regression line included 1.00 [18]. Carry-over was estimated with sequential measurements of high- and low-concentration i-Smart QC material (H1, H2, H3, H4, L1, L2, L3, and L4) and calculated using the following equation:

% carry-over=[L1-(L3+L4)/2]/[(H2+H3)/2-(L3+L4)/2)]*100

Carry-over of <1% was considered acceptable.

We assessed the interference of high K+, iCa2+, and Mg2+ levels with the measurement of Na+, K+, iCa2+, that of a high lactate level with Cl− measurement, and that of a high uric acid level with the glucose and lactate measurement according to the CLSI EP07‐A2 and EP37 guidelines [13, 14]. Interference was considered significant when the difference exceeded the Clinical Laboratory Improvement Amendments (CLIA) criteria for acceptable performance [19].

The method comparison of five blood gas analyzers was performed according to the CLSI EP9-A3 guideline using i-SmartCare 10 as the comparator [15]. We also performed method comparison study of i-SmartCare 10 with Gem Premier 5000 as the comparator. To minimize bias due to pre-analytical factors, samples were analyzed as soon as possible (within 2 hours) after sample collection [20]. Measurements were conducted in random order by a single operator. The instruments were installed side-by-side to minimize the time interval (<1-2 minutes), maintaining the total measurement time per sample within 10 minutes.

The results of total imprecision, linearity, and carry-over assessment of the i-SmartCare 10 analyzer are summarized in Table 1. The repeatability and within-laboratory CVs ranged 0.00-2.12% and 0.00-9.41%, respectively. The within-laboratory CVs of pH, pCO2, Cl−, iCa2+, and Hct escaped the preset criteria for imprecision; however, they were all within the TEas, except for iCa2+ (low and high level). All within-laboratory CVs were within the manufacturer’s specifications. For all analytes, the linear regression plots showed linear responses with the coefficient of determination (r2) exceeding 0.99. The slope of the linear regression plot ranged 0.98-1.23, with the 95% CIs of the slopes for pO2, Cl−, iCa2+, and lactate failing to include 1.0. The ranges of recovery of pO2, iCa2+, glucose, and lactate narrowly escaped the acceptance criteria. Carry-over was within the acceptance criteria for all analytes. Significant interference was observed for Na+ in the presence of a high K+ level, and for iCa2+ in the presence of high K+ and Mg2+ levels (Supplemental Data Table S1).

Table 1

Analytical performance of i-SmartCare 10

Analyte	Level	Imprecision				Linearity		% Carry-over
		Within-laboratory CV (%)	Criteria			Slope (95% CI)	% Recovery
			Ricos* (%)	Ricos TEa[†] (%)	Manufacturer[‡] (%)
pH	Low	0.12	0.10	N/A	0.28	1.01 (0.96-1.06)	99.91-100.15	−0.05
	Medium	0.07			0.27
	High	0.07			0.26
pCO2	Low	2.18	2.40	5.70	4.10	1.05 (0.95-1.14)	97.78-106.90	−0.01
	Medium	1.67			6.10
	High	2.77			11.4
pO2	Low	4.31	N/A	N/A	6.20	1.15 (1.12-1.19)	72.67-111.57	−0.02
	Medium	1.60			5.00
	High	1.79			5.00
Na+	Low	0.70	0.90	4.60	2.20	0.98 (0.94-1.02)	98.93-100.88	0.00
	Medium	0.45			1.50
	High	0.64			1.30
K+	Low	0.00	2.30[§]	5.60[§]	12.50	1.02 (1.00-1.04)	100.00-101.96	0.00
	Medium	0.00			5.80
	High	0.30			4.20
Cl−	Low	0.68	0.60[§]	1.50[§]	2.60	1.04 (1.02-1.06)	98.67-101.65	0.00
	Medium	0.53			2.50
	High	0.45			2.50
iCa2+	Low	2.85	0.90[§]	2.00[§]	5.20	1.23 (1.14-1.31)	88.24-117.90	0.00
	Medium	1.16			5.30
	High	3.22			9.80
Glucose	Low	1.92	2.80[§]	7.00[§]	5.00	1.03 (0.96-1.11)	78.87-101.91	−0.01
	Medium	1.50			5.00
	High	1.33			5.00
Lactate	Low	9.41	13.60	30.40	28.60	1.11 (1.05-1.16)	105.00-111.43	0.00
	Medium	3.80			7.40
	High	2.02			7.60
Hct	Low	3.33	1.40	4.00	8.00	1.01 (0.98-1.05)	91.67-98.55	0.00
	High	2.26			3.80

Deviations from the preset criteria are indicated in bold.

*Ricos desirable specification for imprecision; †Ricos desirable specification for allowable total error; ‡Manufacturer’s desirable specification for imprecision; §Criteria for serum analytes were applied.

Abbreviations: TEa, allowable total error; N/A, not available; Hct, hematocrit.

The results of method comparison of six blood gas analyzers are summarized in Table 2. In correlation analysis, the correlation coefficients ranged 0.84-1.00, with those for Hct in ABL90 FLEX PLUS and RapidLab 1265 and pO2 in RapidLab 1265 and i-Stat failing to exceed 0.9 (Supplemental Data Table S2). The correlation coefficients exceeded 0.90 for all 10 analytes in Gem Premier 5000 and Stat Profile pHOx Ultra. In Passing–Bablok regression analysis, the slope ranged from 0.68 to 1.28, and the intercept ranged from –19.08 to 16.70 (Table 2). Forty out of 48 items demonstrated either a proportional or systematic difference. In Bland–Altman difference analysis, the relative mean difference (%) of 14 out of 48 items escaped the TEas. We estimated bias at medical decision points based on the institution’s reference ranges. For most analytes, the 95% CI of the estimated value failed to include the predicted value at the medical decision point.

Table 2

Results of Passing–Bablok regression and Bland–Altman difference plot analysis between five blood gas analyzers and the comparator (i-SmartCare 10)

Analyte		ABL90 FLEX PLUS	Gem Premier 5000	Stat Profile pHOx Ultra	RapidLab 1265	i-Stat	TEa*
pH	Slope	0.88	0.86	0.80	0.92	0.86
	Intercept	0.85	1.00	1.51	0.61	1.02
	Mean difference (%)	0.13	0.01	0.40	0.41	0.53	N/A
pCO2	Slope	1.06	1.09	1.28	1.13	1.07
	Intercept	−4.35	−6.94	−19.08	−11.01	−6.24
	Mean difference (%)	−2.31	−3.65	−5.92	−7.42	−6.55	5.70%
pO2	Slope	0.91	0.90	0.99	0.96	0.93
	Intercept	0.12	1.84	0.03	2.15	0.37
	Mean difference (%)	−8.00	−7.93	1.06	−0.82	−4.52	N/A
Na+	Slope	1.00	1.00	0.92	0.98	0.93
	Intercept	2.50	−1.00	11.73	2.06	9.33
	Mean difference (%)	1.92	−0.47	−0.01	−0.20	0.11	4.60%
K+	Slope	0.89	1.00	0.93	0.93	0.91
	Intercept	0.44	0.10	0.46	0.34	0.34
	Mean difference (%)	0.27	1.56	3.87	1.10	−0.65	16.00%
Cl−	Slope	1.06	1.12	0.85	1.08	N/T
	Intercept	−9.24	−0.12	16.70	−10.15	N/T
	Mean difference (%)	−3.08	−1.41	1.00	−2.18	N/T	1.50%[†]
iCa2+	Slope	1.03	0.93	0.86	1.00	1.23
	Intercept	−0.45	5.40	0.23	−0.05	−0.29
	Mean difference (%)	−0.99	0.83	5.46	−4.72	−3.02	2.00%[†]
Glucose	Slope	1.00	1.03	0.93	0.97	0.97
	Intercept	−3.28	−5.10	4.86	−3.51	−4.60
	Mean difference (%)	−3.93	−3.05	−1.82	−7.76	−8.88	6.96%[†]
Lactate	Slope	1.11	1.13	1.11	0.98	N/T
	Intercept	−0.22	−0.21	−0.21	0.31	N/T
	Mean difference (%)	7.98	10.48	6.45	8.56	N/T	30.40%
Hct	Slope	0.79	0.89	0.68	0.76	0.81
	Intercept	7.47	−1.53	10.05	4.59	0.31
	Mean difference (%)	−7.74	−16.75	−1.48	−10.94	−19.34	3.97%

The 95% confidence interval of slope and intercept failing to include 1 and 0, respectively, and relative mean difference (%) exceeding the TEa are indicated in bold.

*Ricos desirable specification for TEa; †Criteria for serum parameters analytes were applied.

Abbreviations: TEa, allowable total error; N/A, not applicable; Hct, hematocrit.

In the method comparison study of i-SmartCare 10 with Gem Premier 5000 as the comparator, the correlation coefficients all exceeded 0.9. In regression analysis, eight out of 10 analytes demonstrated either a proportional or systematic difference, and in difference plot analysis, all mean differences were within the TEas, except that for Hct. For Hct, both proportional and systematic biases were observed with a relative mean difference exceeding the TEa (Fig. 1).

Fig. 1

Passing–Bablok regression (A) and Bland–Altman difference (B) plots of i-SmartCare 10 (test analyzer) and Gem Premier 5000 (comparator) for hematocrit. In the Passing–Bablok regression plot (A), the solid and dashed lines represent the regression line and its 95% confidence interval, respectively. The dotted line represents the identity line (y=x). In the Bland–Altman difference plot (B), the y-axis represents the relative mean difference (%) of results obtained with the test analyzer and comparator, and the x-axis represents their average value. The solid and dot-dashed line represent the relative mean difference (%) and its 95% CI, respectively. The dashed and dotted line represent the limit of agreement (±1.96 SD of the differences) and y=0, respectively.

Overall, the i-SmartCare 10 analyzer showed an acceptable performance. The repeatability and within-laboratory CVs of all 10 analytes were within the TEas, except those for iCa2+ at low and high levels. However, this is less likely to cause error in a real clinical setting, as the expected values for both low- and high-level iCa2+ QC materials (0.5 mmol/L and 1.5 mmol/L, respectively) were outside the reference range of our institute (1.15-1.33 mmol/L). Moreover, despite narrowly escaping the TEa, iCa2+ satisfied the manufacturer’s imprecision target.

Although interference was observed for Na+ at high levels of K+ and for iCa2+ at high levels of K+ or Mg2+, such observed interference is not very likely to result in error in the real clinical setting as the K+ and Mg2+ levels in the spiking solution (12 mmol/L and 5 mmol/L, respectively) were significantly higher than the test levels recommended by the CLSI EP07-A2 guideline (5 mmol/L and 2.6 mmol/L) and the upper limits of the reference ranges (5.4 mmol/L and 1.1 mmol/L) [13]. However, caution should be taken in patients when interpreting the Na+ level with an elevated K+ level and the iCa2+ level with an elevated K+ or Mg2+ level.

Previous method comparison studies of blood gas analyzers have reported discrepant results. Although some studies showed negligible differences among analyzers, some did show significant bias [5-10]. In the present study, we found critical and often large differences exceeding the preset TEas. Although there were differences in the principle of measurement among the analyzers evaluated (Supplemental Data Table S3), these differences were not associated with the degree of agreement among the analyzers.

Our study has some limitations. For K+, Cl−, iCa2+, and glucose, targets for whole blood samples were not available in the Ricos database; therefore, targets for serum samples were considered. As the true value of the blood gas test was unknown, we were unable to analyze the effect sizes of the differences in the correlation analysis of the six blood gas analyzers.

In conclusion, the i-SmartCare 10 analyzer shows overall satisfactory performance, suggesting its suitability for clinical use. For monitoring using a POC analyzer, the use of a single analyzer is recommended to avoid analytical differences and misinterpretation of the results.