Comparative Performance Analysis of Urised 3 and DIRUI FUS-200 Automated Urine Sediment Analyzers and Manual Microscopic Method.

OBJECTIVE: Microscopic examination of urine sediment is necessary for evaluation of renal and urinary tract diseases. In this study, we evaluated and compared analytic and diagnostic performances of DIRUI FUS-200 and a new image-based automated urine sediment analyzer Urised 3.
METHOD: A total of 440 urine samples, submitted to our laboratory, were evaluated by two automated urine sediment analyzers and a standardized manual microscopy. Precision, linearity and method comparison studies were performed according to CLSI guidelines.
RESULTS: Considering the red blood cell (RBC) and white blood cell (WBC) counts, strong correlations existed between FUS-200 and manual microscopy (r=0.993 vs 0.861), Urised 3 and manual microscopy (r=0.962 vs 0.818), FUS200 and Urised 3 (r=0.961 vs 0.961). Clinical non-concordance ranged from 7% to 14.16% among all methods.
CONCLUSIONS: The concordance between the analyzers and manual microscopy for WBC was better than that of RBC. The concordance between the two analyzers was better for WBC and RBC, with respect to the manual microscopy. Although the Urised 3, FUS-200 and manual microscopy counts were in agreement; confirmation of the results of automated analyzers with manual microscopy is particularly helpful, for pathological samples with near cut-off values. Copyright Istanbul Medeniyet University Faculty of Medicine.

Introduction

Urinalysis is one of the most important screening tests in clinical practice. Microscopic examination of urine samples is necessary for the evaluation of urinary system diseases[1]. Microscopic analysis of urine sediments is a time consuming method that requires experience. Several preanalytical and analytical processes contribute to its imprecision. These factors are related to sampling, centrifugation and inter-observer variability. Despite all these disadvantages, manual microscopy is the reference method for the evaluation of urine sediment[2]. The automated urinalysis can save labor and time and is more feasible for the high volume laboratory workload[3,4]. Two systems based on different technologies which are image-based and flow cytometry are available. We are currently using FUS200 in our laboratory. However, we plan to install a new system, Urised 3. The objective of this study is to compare DIRUI FUS-200 (DIRIU Industrial Co., China) and Urised 3 (77 Elektronika Kft, Budapest, Hungary) with manual microscopy and with each other, using similar parameters [cell or particle counts/μL under HPF (high-power field)] for evaluation. Because we want to see if the Urised 3 has sufficient and better analytical and diagnostic performance with respect to FUS-200. Otherwise, it will not be advantageous to use Urised 3 instead of FUS200.

Materials and Methods

This study approved by the Istanbul Medeniyet University, Goztepe Training and Research Hospital, Clinical Studies Ethics Committee, (6 June 2017, 2017/0200).

Sample

This study was performed in the central laboratory of Istanbul Medeniyet University Goztepe Training and Research Hospital. The laboratory has large sample volume and workflow. We performed this study using 440 urine samples in June 2017. We collected the urine samples in sterile, preservative and antiseptic-free containers and transferred them into test tubes. Specimens were analyzed consecutively using these two automated analyzers and a standardized manual microscopy (by a single experienced technician) within 2 hours after submission to the laboratory. Since two urine samples were insufficient for evaluation using three devices, and their manual microscopic analysis could not be performed so they were not included in the analysis.

Automated urine analyzers

The analytical principle of the DIRUI FUS-200 is based on flow cell digital imaging and identification using software. As the urine passes through the flow cell, it is illuminated by a special light source, and the images are recorded by a digital camera placed into the eyepiece of the microscope and then they are login a computer-based system. The software classifies these images and displays them on the screen for the evaluation of images of these sediments by the operator.

The analytical principle of the Urised 3 is image- based microscopic analysis of the urine sediment. The Urised 3 analyzer pipettes urine samples into an individual disposable cuvette without any reagents. The filled cuvette is centrifuged at 2000 rpm for 10 seconds. An automated built-in camera then takes 15 images of the settled monolayer of urine particles. Images are recorded in three types, including bright-field, phase contrast, and composite. Images are then evaluated by the Auto Image Evaluation Module, an automatic, real-time, image processing software. After the images are obtained, the operator can evaluate particles from the images displayed on the screen.

Manual method

We centrifuged all urine specimens in a conical measuring tube to measure urine volume at 400g for 5 minutes and discarded supernatant leaving 200 μl of sediment for further analysis. The remaining 200 μl of urine specimen was resuspended and 20 μl of sediment was pipetted onto a microscope slide (Olympus CX41RF) and covered with a coverslip (20 mm x 20 mm) for standardized conventional urine sediment analysis. We counted number of RBCs and WBCs on 10 small squares under 400× magnification and gave the results as an average per HPF and classified as shown in Table 1. We converted these numbers as cells per liter according to the following formula:

Table 1

Reference values of urine WBC and RBC.

Cells/HPF	Negative	Positive
		Few	Moderate	High	Many
WBC	< 5	6-10	11-20	21-50	> 50
RBC	< 5	6-10	11-20	21-50	> 50

WBC: white blood cell; RBC: red blood cell; HPF: high power field (x400)

In this transformation formula, n is the mean count of cells/HPF, HPFSlide is “ the ratio of area of the slide and area of one HPF”. VolCentr is the volume of the pellet after centrifugation, VolTube is the total urine volume in the test tube and VolSlide is the volume under the coverslip (20x10-6 ??)[5].

Study: Precision, Linearity, Comparison

The between-run, within-run precisions and linearity were determined according to the CLSI protocols for the instrument Urised 3 for RBC and WBC counts and for the instrument FUS-200 for total particle counts because of the features of different materials used for internal quality control[6,7]. Two levels of control materials were run 20 times on the same day for within-run precision study; twice daily on 20 separate days in duplicate for between-run precision study. The imprecision was stated as the coefficient of variation (CV%). For stability reasons, instead of urine samples, Kova Liqua-Trol level 1 and level 2 control materials (fixed RBC and WBC) were used for the Urised 3; and negative and positive control samples (particles) were used for the FUS-200. To determine linearity for WBC, erythrocyte- lysed whole blood samples were used as high level samples. Saline was used as the blank sample. We mixed blank and high level specimen in ratios of 0:4, 1:3, 2:2, 3:1 and 4:0; and these mixtures were run for five times. Detection capabilities of the FUS-200 and Urised 3 were calculated according to EP17-A2[8].

Data Analysis

Deming Regression analysis was performed to establish differences between the analyzers. We classified the samples semi-quantitatively (≤5, 6-10, 11-20, 21-50, >50 cells/HPF) and as positive, and negative (≤5 vs >5 cells/HPF) according to erythrocyte and leukocyte counts[9-11]. Within the same grade Cohen’s kappa coefficients were calculated for concordance between the methods and the McNemar test was used to measure changes in the distribution of two dichotomous variables. Values for Cohen’s kappa coefficient were defined as poor (0-0.20), fair (0.21-0.40), moderate (0.41-0.60), good (0.61-0.80) and very good (0.81-1.00) agreement[12]. We determined the diagnostic power of both devices for RBC and WBC with reference to manual microscopy. EP Evaluator (David G. Rhoads Associates, Kennett Square, PA) was used for statistical analysis. P values of ≤0.05 value were considered as significant test results. The diagnostic performance parameters; sensitivity, specificity, positive (PPVs) and negative predictive values (NPVs); were determined.

Results

Precision, Linearity, Comparison

The reproducibility of the FUS-200 and Urised 3 were shown in Table 2. WBC showed good linearity up to 1090 cells/µL with the following regression equations:

Table 2

Results of precision study of FUS-200 and Urised 3.

Analyzer	Particle/pL	Within-run Imprecision				Between-run Imprecision
		Low Level		High Level		Low Level		High Level
		Mean±SD	CV%	Mean±SD	CV%	Mean±SD	CV%	Mean±SD	CV%
FUS-200	Particle	0*	*	993.05±25.46	2.56	0*	*	1001.4±32.0	3.4
Urised 3	RBC	0*	*	72.16±15.85	22	0*	*	44.77±11.02	24.61
	WBC	9.67±2.3	23.8	73.99±14.19	19.2	2.2±0.96	44.06	140.1±31.14	22.23

* These values could not be calculated because the average cell count was 0 RBC: red blood cell; WBC: white blood cell; CV: coefficient of variation

Y=0.925X-2.4 (R[2]=0.99) for the FUS-200 and Y=1.159X+16.67 (R[2]=0.97) for the Urised 3 (Figure 1).

Figure 1

Linearity of FUS-200 (A) and Urised 3 (B) for WBC (cells/µl).

We compared the FUS-200, Urised 3 and manual microscopy results for method comparison study. The Deming regression analysis was performed for RBC (range: 0 -2695 cells/µL) and WBC (range: 0 -1110 cells/µL) by comparing FUS-200, Urised 3 and manual microscopy methods (Figure 2) (Table 3).

Figure 2

Deming regression analysis for RBCs (A) and WBCs (B) among the FUS-200, Urised 3 and manual microscopy.

Table 3

Deming regression analysis between FUS-200 and Urised 3 for RBC and WBC.

		Manual vs FUS-200	Manual vs URISED3	FUS-200 vs URISED3
WBC	Slope	2.354 (2.238 to 2.469)	2.946 (2.783 to 3.109)	1.186 (1.155 to 1.217)
	Intercept	-9.821 (-20.695 to 1.053)	-18.482 (-33.822 to 3.141)	-4.159 (-10.257 to 1.940)
	Correlation coefficient	0.861	0.818	0.961
	Bias*	24.358	30.645	5.208
RBC	Slope	2.069 (2.045 to 2.093)	1.362 (1.326 to 1.398)	0.643 (0.626 to 0.661)
	Intercept	8.311 (4.582 to 12.041)	2.793 (-2.737 to 8.322)	-2.148 (-7.451 to 3.155)
	Correlation coefficient	0.993	0.962	0.961
	Bias* (counts/pl)	28.446	9.616	-18.503

* Mean difference between the cell counts

The concordance between the manual method and automated analyzers was also comparatively evaluated (Tables 4, 5, 6, 7, and 8). When we evaluated the results according to positive and negative groups, the concordance rates between the manual method and automated analyzers are summarized in Table 9.

Table 4

Comparison of FUS-200, Urised 3 and manual WBC counts.

	FUS-200 ( cells/HPF)						URISED3 (cells/HPF)
Manual (cells/HPF)	0-5	6-10	11-20	21-50	>50	Total	0-5	6-10	11-20	21-50	>50	Total
0-5	344	13	3	2	0	362	330	19	11	1	1	362
6-10	9	12	7	4	0	32	8	9	6	9	0	32
11-20	3	1	4	7	1	16	1	1	4	8	2	16
21-50	1	2	2	5	11	21	1	1	1	8	10	21
>50	0	0	0	0	7	7	0	0	0	0	7	7
Total	357	28	16	18	19	438	340	30	22	26	20	438

Table 5

Comparison of FUS-200, Urised 3 and manual RBC counts.

	FUS-200(cells/HPF)						URISED3 (cells/HPF)
Manual (cells/HPF)	0-5	6-10	11-20	21-50	>50	Total	0-5	6-10	11-20	21-50	>50	Total
0-5	362	34	21	3	1	421	336	3	0	0	0	339
6-10	3	0	3	2	0	8	19	11	1	0	0	31
11-20	0	0	0	5	0	5	2	10	10	0	0	22
21-50	0	0	0	0	0	0	1	0	7	18	0	26
>50	0	0	0	0	4	4	1	0	0	2	17	20
Total	365	34	24	10	5	438	359	24	18	20	17	438

Table 6

Comparison of FUS-200 and Urised 3 WBC.

	UR ISED3 ( cells/HPF)
FUS-200 (cells/HPF)	0-5	6-10	11-20	21-50	>50	Total
0-5	334	19	5	0	0	358
6-10	4	10	11	3	0	28
11-20	2	1	4	10	0	17
21-50	1	1	2	11	3	18
>50	0	0	0	2	17	19
Total	341	31	22	26	20	440

Table 7

Comparison of FUS-200 and Urised 3 RBC.

	UR ISED3 ( cells/HPF)
FUS-200 (cells/HPF)	0-5	6-10	11-20	21-50	>50	Total
0-5	353	7	6	0	0	366
6-10	30	2	3	0	0	35
11-20	13	5	4	2	0	24
21-50	3	2	0	3	2	10
>50	0	0	0	1	4	5
Total	399	16	13	6	6	440

Table 8

Concordance of urinalysis within the same grade.

	WBC		RBC
	Concordance rate (%)	Kappa	Concordance rate (%)	Kappa
Manual vs FUS-200	84.9	m0.53	83.6	p0.17
Manual vs Urised 3	81.7	m0.48	89.5	g0.71
FUS-200 vs Urised 3	85.5	m0.59	83.2	f0.30

p: poor agreement, f: fair agreement, m: moderate agreement, g: good agreement

Table 9

Concordance of urinalysis within the same condition (negative-positive).

	WBC		RBC
	Concordance rate (%)	Kappa	Concordance rate (%)	Kappa
Manual vs FUS-200	92.9	g0.76	85.8	f0.27
Manual vs Urised 3	90.4	g0.70	92.5	f0.39
FUS-200 vs Urised 3	93	g0.79	86.6	m0.42

f: fair agreement, m: moderate agreement, g: good agreement

When the data were analyzed considering the positive and negative results, the FUS-200 and manual method did not differ for WBC counts (McNemar test; P=0.369). Overall, the non-concordant results could have affected 7.08% of all diagnoses. Comparing the WBC counts of the Urised 3 with manual method, there was a significant difference in classification (McNemar test; P<0.001) and 9.59% of the results were non-concordant. Analysis of the data showed 7% non-concordance between automated techniques (McNemar test; P<0.01).

RBC counts differed significantly between the FUS-200 and manual methods considering the positive and negative results (McNemar test; P<0.001); non-concordance was 14.16%. The Urised 3 and manual methods showed a similar pattern (McNemar test P<0.001), and non-concordance was 7.53%. Both automated methods showed 13.4% non-concordance (McNemar test P<0.001). Diagnostic performance parameters were summarized in Table 10.

Table 10

Diagnostic performance of FUS-200 and Urised 3.

Cells	Method	Sensitivity (%)	Specificity (%)	PPV (%)	NPV
WBC	FUS-200	82.9	95	77.8	96.4
	URISED 3	86.8	91.2	67.4	97.1
RBC	FUS-200	82.4	86	19.2	99.2
	URISED 3	70.6	93.4	30	98.7

PPV: Positive Predictive Value, NPV: Negative Predictive Value

Discussion

To our knowledge, a comparison study between the instruments FUS-200 and Urised 3 is not available in the literature. We classified the WBC and RBC counts according to the clinical decision limits to compare manual microscopy with the automated analyzers. In relation to clinically positive and negative results, the concordance between the manual method and automated analyzers ranged from fair to good for WBC and RBC counts.

Each assessment procedure for urine sediment has its own advantages and disadvantages. Although manual microscopy has a lot of methodological problems and many factors reduce its precision and accuracy, it is accepted as the reference method for urine urinalysis[13]. Automated urine analyzers that use several analytic techniques have been developed to deal with disadvantages of manual microscopy.

In accordance with the literature, we found that our results were more accurate in urine sediments having higher cell counts[14-16]. It is accepted that automated analyzers reduce the time spent on manual examination, as reported in some studies[17,18]. A study which was performed with 214 urine samples, conventional microscopy and a flow cytometry-based UF-1000i device showed a nearly perfect concordance[19].

In our study, we found the precision of Urised 3 lower with respect to another study[16]. Our imprecision was higher than the results reported by Bottini et al., Akin et al. and Zaman et al. in previous studies of Urised[9,14,20]. The precision of the FUS-200 in this study was lower than that reported by Ince et. al.[15]; similar to the precision of UX-2000 reported by S. Laiwejpithaya et. al.[16] and precision of Sysmex UF-1000i reported by Lee et al.[21]. Overall, the FUS-200 showed better precision profile than the Urised 3 due to quality-control sample characteristics. However, both analyzers had higher precision when compared to the conventional microscopy with respect to the results reported by Chien et al.[22] and Jiang et al.[23]. Deming regression analyzes of both analyzers showed good correlation ranging from r=0.818 to r=0.993 for RBC and WBC. Although the same urine specimens were evaluated, differences in measured concentration of RBC and WBC between instruments may be due to different analytic technologies. Differences between the results of analyzers don’t have to be disregarded. The slopes and intercepts were outside of the confidence interval of 1 and 0, respectively (Table 3). If both analyzers and the same reference intervals are to be used in the same laboratory, it is necessary to use a conversion factor to eliminate the effects of these differences[24].

We observed better concordance rate within the same grade for WBC compared to the results reported by Ince et al. and similar concordance between the FUS-200 and manual microscopy considering the RBC results. Degree of concordance for positive results was better for WBC and worse for RBC than the results reported by Ince et al.[15]. Between the Urised 3 and manual microscopy, concordance rate within the same grade was lower for WBC and RBC; degree of concordance for positive results was similar for WBC and lower for RBC with respect to the study of S. Laiwejpithaya et al.[16].

Semi-quantitative analyzes of WBC and RBC showed a lower agreement compared to the results reported by Nagy[25]. In terms of semi-quantitative analysis, Urised 3 showed very good agreement rates. Kappa values showed very good agreement for RBC; and moderate agreement for WBC. The FUS-200 also had very good agreement rates, but the kappa values were different. They were fair for RBC; moderate for WBC. NPV of Urised 3,[10,16,26]; and FUS-200[10,15]. for RBC and WBC was good as reported by other studies PPV of the Urised 3 for WBC and RBC was low as reported by Yuksel et al.[10]. PPV of the FUS-200 for RBC and WBC were worse than that was reported by Ince et al.[15] and similar to PPV reported by Yuksel et al.[10]. These results could be due to urine centrifugation procedure before analysis that may form aggregates or produce cell lysis[27]. The NPV s of the devices were similar or better than their PPVs. These results showed that the two devices have low false-negative results but higher false positive results.

The methods gave some inconsistent results that may mislead clinical diagnoses, particularly for urine samples with cell counts close to the cut-off values (6-10 cells/HPF). We observed clinical non-concordance similar to the results reported by Akin et al.[9]. Other studies have also reported similar clinical non-concordant results[11,28].

In summary, Urised 3 and FUS-200 give reproducible results and analyze great numbers of urine samples. Urised 3 was more specific for RBC and more sensitive for WBC than FUS-200. FUS-200 and Urised 3 had lower PPVs for WBC and RBC relative to NPVs. Both analyzers had better PPVs for WBC rather than RBC. FUS-200 had higher PPV for WBC than Urised 3. FUS-200 had lower PPV for RBC than Urised 3. FUS-200 and Urised 3 had almost perfect and similar NPV for both of the cell types. Lesser number of pathological samples is the limitation of this study.

Conclusion

Urised 3 and FUS-200 had almost similar performance rates compared to standardized manual microscopy considering the clinically positive and negative concordance rates for RBC and WBC counts. So it is important to confirm the results by manual sediment analysis, especially for pathological cases based on the clinically decided cut-off limits.