High False Positives and False Negatives in Yeast Parameter in an Automated Urine Sediment Analyzer.

BACKGROUND: Automated urine sediment analyzers have proven their feasibility in medical laboratories. However, editing manual microscopic review of some specimens severely limits the usefulness of such systems. This study aims to give feedback on the practical experience on »Yeast«, which is one of the parameters that compel frequent manual reviews.
METHODS: 5448 freshly collected urine specimens submitted from various departments of our hospital for diagnostic urinalysis were studied by the UriSed® (77 Elektronika, Hungary). A specialist medical doctor inspected every image on-board, and reviewed the ones with a »Yeast« alarm by traditional manual microscopy.
RESULTS: UriSed alarmed in 491 samples (9%) for yeast. In 59 samples (1%) the number of particles exceeded the cut-off and a »positive for yeast« was set. A false positive report of yeast +1 to 3+/HPF was found in 51 samples (0.9%). There were 8 cases with positive for yeast from both microscopic methods. Thirty-three »negative for yeast« samples were corrected as positive after the manual microscopic review.
CONCLUSIONS: We report a high percentage of false positives and negatives in the yeast parameter, in line with other studies on UriSed as well as on other instruments in the market. As an important feedback, our observations showed that the major concern in false results was »the focusing problem«. We believe in the necessity of a focus check and comparison of alarms between images on board.

Introduction

Machines are fast, cost effective and efficient. They do not give coffee breaks, do not get angry or upset (as much as we know). So began their invasion in the medical laboratories. Automation continues to evolve at a rapid pace, and front-end automation calls for human-free laboratories. The last stand seems to be the urine sediment microscopic analysis, where technicians use microscopes as their sole instruments. The traditional method is labor-intensive, time consuming, requires experience and has wide variability. To this end, there have been attempts to automate the process, thereby improving accuracy, precision, and throughput. In the 1980s, the image-based analysis systems were developed. Up-to-date, the main approaches for the autoquantification and classification of urine particles are flow cytometry, and the digital microscopic image based technologies (1, 2).

The UriSed® (77 Elektronika, Hungary) is a new automated urine microscopy analyzer based on digital imaging. The machine acts in quite a humanoid manner. The sediment is visualized by a built-in light microscopy (eye). A predefined number of high-power field (HPF) digital images are then analyzed with a recognition software (central nervous system). This recognition software is defined as a »special neural-network-based image processing algorithm«. The use of the so-called multilevel decision method has been improved by new editions developed by 77 Elektronika (3).

Several studies compared automated analysis systems with manual microscopy (1–11) and most recognized the accuracy and precision of automated systems, as well as their feasibility as routine screening tools (1, 3, 6, 9). For practical reasons these studies spare more effort on parameters like red blood cells, white blood cells, and epithelial cells. Automated urine analyzers including the UriSed have been evaluated for their ability to distinguish urine samples with and without significant bacteriuria. However, the diagnostic accuracy of their performance in predicting urine culture positivity still needs improvement (12–14). As the machines prove their consent in these parameters, it is time to focus on others. We here aim to give laboratory feedback on the performance of UriSed particularly in yeast parameter.

Material and Methods

Specimens and procedures

We studied 5448 freshly collected urine specimens submitted from various departments of our hospital for diagnostic urinalysis. A clean, preservative container was used for urine collection. UriSed® in our laboratory is connected with LabUMat®, the urine chemistry analyzer of the same manufacturer. All urine samples were analyzed using dipstick biochemical tests followed by automated microscopy. All images from all specimens were followed by the same pathologist on-board, and any specimen with a fungus alarm (under or above the cut-off for a positive) was re-evaluated by manual microscopy.

In this study, all images stored by UriSed® were reviewed on the view station. In conflicting cases manual microscopy was accepted as the gold standard.

UriSed

The UriSed operates on the basis of microscopic examination of a urine sample in a special disposable cuvette. During the measurement process, the urine sample is transferred to the cuvette and centrifuged. High resolution complete views of field images are then recorded automatically by a microscope. After centrifuging the preparation for 10 s at 260 g and thereby pelleting the particles, the device analyses a 2.4 μL urine sample by scanning 15 field images. These images are then evaluated by a special algorithm. The sample is evaluated by a special neural-network-based image processing algorithm through the use of the so-called multilevel decision method. Each image is recognized in ‘real time’, while the evaluation procedure is running on the image just after recording. For the UriSed the following diagnostic cut-off values were used: 1+: ≥ 2 p/HPF; 2+: ≥ 3.33 p/HPF; 3+: ≥ 5.33 p/HPF; 4+: ≥ 13.33 p/HPF.

Manual microscopy

After performing the automated sediment testing by the instrument, the residual urine specimens were collected if a manual examination was to be performed. The urine was centrifuged in test tubes at 1500 rpm for 5 min, supernatant was discarded, and the remaining material was placed onto a microscope slide, which was covered with a slide cover. Ten high power fields (HPFs) were assessed and the results were expressed as the mean of the count obtained per HPF. All samples were completely processed within 2 h after receiving.

We report the results as positive when we are sure about any yeast particles. The microscopic examination was performed by using a light microscope (Olympus, CX21FS1) at the magnification of 400×. For the yeast count, we used an ordinal scale result (negative, 1+, 2+, 3+ or 4+) in manual microscopy.

Sensitivity, specificity, negative and positive predictive values were calculated by using Microsoft Excel spreadsheets.

Results

Day-to-day analyses are presented in the Table I. UriSed alarmed in 491 samples (9%) for yeast. In 59 samples (1%) the number of particles exceeded the cut-off and a »positive for yeast« was set (true positives and false positives). All samples with a yeast alarm were re-evaluated through on-board images and manual microscopy. A false report of yeast +1 to 3+/HPF was found in 51 samples (0.9%) (False positives). There were 8 cases with positive for yeast from both microscopic methods (True positives). Thirty-three negative for yeast samples were reported positive after the manual microscopic review (False negatives). The results are expressed as 1+, 2+, 3+, or 4+ in the patient result print. Test method sensitivity and specificity for the UriSed were 19.5% and 99% respectively; positive predictive value 13.5%, and negative predictive value 99.3%, for post-review.

Table I

Day-to-day performance of UriSed in yeast parameter. We observed the yeast parameter performance of the instrument in 23 randomly selected days in a period of 178 days.

	001	013	026	041	047	053	060	062	069	081	082	104	105	125	128	129	130	132	142	145	156	157	178
#	200	131	166	190	162	270	280	210	250	275	270	150	210	184	280	270	300	240	300	280	230	200	400
ALR	12	10	15	17	11	22	21	20	32	27	28	13	18	15	14	28	26	20	31	27	25	16	43
FP		1		2		3	1	1	4	6		1	1	1		4	2	2	5	3	4	2	8
FN	2	1	1	3		2	1	2	1		3				1	2	1	1	4		2	2	4
TP						1		1	1							1	2			1			1

First raw is dates of study. #: number of specimens studied by the instrument in that day; ALR: alarms for yeast particles; FP: false positive; FN: false negative; TP: true positive

Discussion

UriSed® uses a camera to take 10 to 15 digital images, through a built-in bright-field microscope. Interestingly, the machine uses the same objective we use in manual microscopy in our laboratory. The onboard image magnification approximates to 400× enlargement, which is equivalent to the high power visual microscopy magnification of 10×40 of ours. Briefly, UriSed® and its competent (the pathologist) evaluate the same image. So, what we have compared in this study was the level of intelligence; particularly the capability of the machine and human in recognizing defined particles.

Specifically for yeast cells, the results from UriSed® demonstrated a high number of false positives and false negatives. It is apparent that UriSed® detected other formed elements as yeast cells, and failed to recognize some true yeast cells. Yeast particles are known to be a problem in automated urine sediment analysis, irrespective of the technology used (1, 3, 6–8, 10). In studies comparing different instruments with different methods with the gold standard manual microscopy, both Lamchiagdhase et al. (6) and Alves et al. (7) have reported fair agreement in yeast parameter. Studies on the performance of UriSed® are not many (2, 3, 11, 13–16). For practical reasons they focus on parameters like red blood cells, white blood cells or epithelial cells. Zaman et al. (3) gave respectively detailed information about the yeast parameter in Urised®. They reported a rate of 10% for yeast particles, nearly the same as our results, and the correlation between the visual microscopy and the UriSed® results as a moderate one for yeast. They defined the presence of yeast as a limitation to recognition of the erythrocytes, as in these situations the UriSed® misidentified some of the individual yeast cells and the budding yeast cells as RBC. In our study, we also observed such a confusion but most of our false alarms were due to nuclear fragments and bacteria, followed by crystals and epithelial structures. Most importantly, we observed a problem in focusing in almost all of the false positives and in most of the false alarms (Figure 1). In these cases focusing problems did not need to be present and were really not present in all 10 images of one specimen. Ten percent of these false alarms exceeded the threshold for a false positive report. As an operator reviews the stored images before submitting the final report, almost all false positive reports as well as false negatives were corrected.

Figure 1

The built-in optic eye of UriSed is an Olympus objective. The images were quite clear and very much alike to the 400x manual microscopic vision. Each pair (A&B, C&D, E&F) were 2 of 10 HPF images from the same urine specimen. The images A, C, E were clear images without a focusing problem. Their pairs B, D, F were out of focus, causing blurred larger images of bacteria in B, erythrocytes in D, amorphous crystals in F. UriSed gave false alarms for yeast in these out focus images. The yeast particles were recognized by the instrument in G, while they were missed in H.

Currently, there is a consensus in the literature that for casts, non-squamous epithelial cells, bacteria, crystals, and yeast the technologist should visually inspect and edit images on the screen before releasing the result on those samples. The major limitation in reducing the number of manual on-screen reviews is apparently the failure in the discrimination of these particles in all instruments in the market. The manufacturer of the UriSed® has launched a new software version (version 6) with improved quantitative counting of RBC and WBC. Hopefully, the version 6 would improve precision, linearity and method comparisons. However, the company promises results in more than 10 parameters, and a prominent failure in one of the parameters, like yeast, will probably shadow the achievements in major parameters.

The reader should recognize that we here did not compare any instruments, or comment on the performance of UriSed® in general. We did not have clinical follow-ups, or any information about the clinical consequences of urine microscopy results.

In summary, we here report a high percentage of false positives and negatives in the yeast parameter. After our study was finished, we continued to observe the machine and the operators. As the operators spent more time in image-check, they got leery in checking all sample images, which lead to a constant rate of leaks of false results. Most importantly, we here report a problem in image focusing, as a cause of error in many false results. Our review in the literature pertaining to the subject did not meet such an observation. This may be due to the different methodology of the systems examined in some studies. A blurred vision will fail UriSed, no matter how clever new editions will be. We observed that the focusing problem rarely included all images taken from a single sample. In such a case, number of yeast alarms in an out of focus image differed significantly from the number alarms in other well-focused images of the same specimen; thus if the machine software somehow owns the ability to compare numerical differences in a parameter between all images of a sample, discordant results might be an alarm for the instrument about improper focusing.