Analytical performance, agreement and user-friendliness of six point-of-care testing urine analysers for urinary tract infection in general practice.

OBJECTIVE: Various point-of-care testing (POCT) urine analysers are commercially available for routine urine analysis in general practice. The present study compares analytical performance, agreement and user-friendliness of six different POCT urine analysers for diagnosing urinary tract infection in general practice.
SETTING: All testing procedures were performed at a diagnostic centre for primary care in the Netherlands. Urine samples were collected at four general practices. PRIMARY AND SECONDARY OUTCOME MEASURES: Analytical performance and agreement of the POCT analysers regarding nitrite, leucocytes and erythrocytes, with the laboratory reference standard, was the primary outcome measure, and analysed by calculating sensitivity, specificity, positive and negative predictive value, and Cohen's κ coefficient for agreement. Secondary outcome measures were the user-friendliness of the POCT analysers, in addition to other characteristics of the analysers.
RESULTS: The following six POCT analysers were evaluated: Uryxxon Relax (Macherey Nagel), Urisys 1100 (Roche), Clinitek Status (Siemens), Aution 11 (Menarini), Aution Micro (Menarini) and Urilyzer (Analyticon). Analytical performance was good for all analysers. Compared with laboratory reference standards, overall agreement was good, but differed per parameter and per analyser. Concerning the nitrite test, the most important test for clinical practice, all but one showed perfect agreement with the laboratory standard. For leucocytes and erythrocytes specificity was high, but sensitivity was considerably lower. Agreement for leucocytes varied between good to very good, and for the erythrocyte test between fair and good. First-time users indicated that the analysers were easy to use. They expected higher productivity and accuracy when using these analysers in daily practice.
CONCLUSIONS: The overall performance and user-friendliness of all six commercially available POCT urine analysers was sufficient to justify routine use in suspected urinary tract infections in general practice. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

This is the first time that different commercially available point of care testing (POCT) urine analysers have been evaluated on analytical performance, agreement and user-friendliness.

All six POCT urine analysers in our study have good analytical performance.

First-time users indicate that the analysers are easy to use. They expect higher productivity and accuracy when using these analysers in daily practice.

This study does not address whether the use of POCT urine analysers in the primary care setting has added value compared with visual reading. A study on this subject is currently being conducted.

Introduction

With urinary tract infection (UTI) being part of the top 10 diagnoses for which a general practitioner (GP) is consulted in the Netherlands, urine test strip analysis to diagnose UTI is a common diagnostic procedure in general practice. Urine analysis takes place in almost 2% of all consultations.1 Commonly, UTI is diagnosed based on the test results for nitrite, leucocytes and erythrocytes on the test strip, performed at the GP's office by the GP nurse or midwife. Previous studies have shown that, in general practice, negative test strip test results for nitrite and leucocytes are useful in excluding UTI.2 Professional guidelines in the Netherlands recommend performance of a test strip evaluation of urine in case of a clinical suspicion of UTI. In case of a nitrite negative test strip result, the presence of leucocytes and erythrocytes is assessed. If one of these is positive, a (semi)quantitative urine culture needs to be performed for conclusive diagnosis.3

For urine analysis, a test strip for various tests is dipped in urine. After contact with the urine, the coloured pads of the test strip for analysis of nitrite, leucocytes, protein, glucose, ketones, urobilinogen, bilirubin and erythrocytes (dependent of the type of strip used), can be assessed visually by comparing them to a colour chart most frequently printed on the side of the bottle containing the test strips. Visual inspection of test strips is subjective and errors may occur due to a variety of reasons, including poor technique and poor, inadequate or untimely reading of the test results.4 5 It has been suggested that the use of point of care testing (POCT) urine analysers will increase the quality of urinalysis.6 7 With the introduction of commercially available POCT urine analysers, it may be possible to improve the quality of urinalysis in general practice. These analysers, therefore, need to have good analytical performance and agreement with laboratory standards, but data are lacking for the different parameters. We set out to test, as the primary outcome measure, analytical performance and agreement of six POCT urine analysers on nitrite, leucocytes and erythrocytes, compared with laboratory standard reference tests. Furthermore, as secondary outcome measures, user-friendliness, costs and other characteristics of the analysers were compared.

Methods

Setting and samples

All testing procedures were performed at Saltro Diagnostic Centre, an accredited diagnostic centre that provides laboratory and other diagnostic services in routine primary care for approximately 850 GPs, and over one million inhabitants. We collected 77 urine samples of patients who delivered a urine sample for routine investigation at one of the four participating general practices. Samples were stabilised using laboratory tubes containing Stabilur (Greiner BioOne, Kremsmuenster, Austria). Urine samples are transported from various locations (GP offices and blood draw locations) to the central laboratory for analysis. Saltro Diagnostic Centre routinely uses Stabilur to preserve urine samples for particle counting. Stabilur preservative tubes are acceptable for test strip analysis within 8 h on the day of urine collection.8 Before introduction into the Saltro laboratory, validity of these tubes was tested (personal documentation). Urine samples were analysed on six POCT urine analysers and the laboratory reference standards, the Urisys 2400 and the Sedimax urine analyser, by experienced laboratory technicians. All measurements were carried out within 2 h to avoid any incorrect test results due to ageing of the urine samples.

Automatic urine strip POCT analysers

We included six commercially available POCT urine analysers in our study: Uryxxon Relax (Macherey Nagel, Düren, Germany), Urisys 1100 (Roche Diagnostics Ltd, Rotkreuz, Switzerland), Clinitek Status (Siemens Healthcare, Munich, Germany), Aution 11 and Aution Micro (both Menarini Diagnostics, Florence, Italy) and Urilyzer (Analyticon Biotechnologies AG, Lichtenfels, Germany). All analysers qualitatively and semiquantitatively measure blood, urobilinogen, bilirubin, protein, nitrite, ketones, glucose, pH, specific gravity and leucocytes. Additionally, the Clinitek Status, and Aution 11 and Aution Micro, can also measure albumin and creatinine.

The different device characteristics are set out in table 1.

Table 1

Device characteristics

Name	Uryxxon Relax	Urisys 1100	Clinitek Status	Aution 11	Aution Micro	Urilyzer
Brand	Macherey Nagel	Roche	Siemens	Menarini	Menarini	Analyticon
Size (cm) d×w×h	20×16×17.5	29×15×9.5	27.2×17.1×15.8	32.8×21×16.4	8.1×12.4×3.6	26×19×7.7
Weight (kg)	0.71	0.8	1.66	3.6	0.18	1.5
Connectivity	Yes	Yes	Yes	Yes	No	Yes
Touch screen	Yes	No	Yes	Yes	No, touch pad	Yes
Bar code scanner	Yes	Yes	Yes	Yes, handheld scanner possible	No	Yes
Keyboard	Yes	Yes	No (only onboard-touchscreen)	No	No	Yes
Automatic start of testing	Yes	No	Yes	Yes	No	Yes
Print function	Yes	Yes	Yes	Yes	Yes, removable printer	Yes
Storage (results)	200	100	950	520	100	1000
Calibration	Automatic	Yes, automatic reminder	Automatic	No, but aspect correction	No, but aspect correction	No
Simultaneous testing	No	No	No	Yes	No	No
Measurement principle	Reflectance photometry	Reflectance photometry	Reflectance photometry	Reflectance photometry	Reflectance photometry	Reflectance photometry
Price of analyser†	€	€	€	€€€€	€€	€
Price urine sticks/100 sticks†	**	***	***	**	**	*
Test time	1 min	1 min	1 min	1 min	1 min	1 min
Other tests possible	No	No	Yes, albumin-to-creatinine ratio, Hcg	Yes, micro albumin and creatinine (plus ratio)	Yes, micro albumin and creatinine (plus ratio)	No

Price indication per analyser: €=600–800€, €€=800–1500€, €€€€≥5000€.

Price indication urine stick/100 sticks *=25–30€, **=30–50€, ***=50–70€.

†prices may vary per country and therefore no exact values can be given.

Laboratory reference standard

We used the Urisys 2400 automated urine analyser (Roche Diagnostics Ltd, Rotkreuz, Switzerland) as the laboratory reference standard for the six different POCT urine analysers. Performance of the reference standard has previously been proven to be good.9–11 This analyser is routinely used for all (primary care) urine samples at Saltro Diagnostic Centre (Utrecht, the Netherlands). The Urisys 2400 test strips are measured 60 s after pipetting of the sample onto the test areas. Green and orange LEDs are used for the evaluation of the reactive test areas. Compensation of the intrinsic urine colour is carried out using the reflectance value of the compensation pad for the green LED. Urine colour result is determined by using the reflectance values of the compensation pad for three wavelengths (orange, green and blue). Measuring an internal reference plate along with each test strip measurement automatically compensates potential influence of varying ambient temperature on the test results. The Urisys 2400 measures pH, leucocytes, nitrite, protein, glucose, ketones, urobilinogen, bilirubin, blood (erythrocytes/haemoglobin), clarity and specific gravity.

We used the Sedimax urine analyser (Menarini, Florence, Italy) as additional reference standard for quantitative evaluation of leucocytes, erythrocytes and sediments, to verify the quantitative results of the different urine test strip POCT analysers. Analytical and diagnostic performance of the Sedimax urine analyser has previously been proven to be good.12 13

Analytical performance

To assess within-day and between-day performance, a protocol was used based on the clinical and laboratory standards institute EP5 and EP9 guidelines.14 15 For this study, Dipper urine dipstick control (Quantimetrix, California, USA) levels 1 and 2 were used. These are ready-to-use control liquids, intended as a control for urinalysis reagent strips, requiring no reconstitution or dilution. They are prepared from human urine fortified to target levels. Level 1 is used as a negative control and level 2 as a positive control.

The within-day performance of each POCT urine analyser was quantified testing 20 aliquots of level 1 and 20 aliquots of level 2 on all six POCT urine analysers. Results were compared to expected values listed in the manual of the Dipper urine dipstick control levels 1 and 2.

For the between-day performance, 20 aliquots of levels 1 and 2 Dipper urine dipstick control were tested on every device on 10 consecutive days, twice a day.

Agreement

We assessed agreement of the six POCT urine analysers with the laboratory reference standard urine analyser (Urisys 2400). The routinely used cut-off values of 25 leucocytes/μL and 10 erythrocytes/μL of the Urisys 2400 representing a positive test result, were also used for this study. POCT urine test strips were considered positive when the result was at least ‘1+’, or when its quantitative equivalent was present, as reported in the test strip manual. For a full report on qualitative and equivalent quantitative results per test strip, see online supplementary appendix 1.

Additional reference standard tests to verify the test results on the POCT urine analysers in the quantitative evaluation of leucocytes and erythrocytes were performed using the Sedimax urine analyser using the same cut-off values.

Statistical analysis

Analytical performance and agreement of the POCT analysers regarding nitrite, leucocytes and erythrocytes, with the laboratory reference standard, was the primary outcome measure. Analytical performance of the different POCT urine analysers compared with the reference methods was evaluated by calculating sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). Agreement between the POCT urine analysers and the reference methods was analysed by calculation of the Cohen's κ coefficient and its 95% CI. Correlations were ranked as very good: κ=0.81–1.00; good: κ=0.61–0.81; moderate: κ=0.41–0.60; fair: κ=0.21–0.40; poor κ<0.20.16

User-friendliness

User-friendliness of the six POCT urine analysers was assessed in a separate survey, using a standardised questionnaire.17 The questionnaire contained five questions concerning the user-friendliness of the analyser, test procedure and the susceptibility to flaws. A group of seven GP assistants and two midwives unfamiliar with the devices was asked to perform tests on all six POCT urine analysers in random order. GP assistants and midwives were not instructed on the use of the different POCT urine analysers prior to testing. However, they did receive the short written instruction provided by the manufacturers. Immediately after performing each test, the questionnaire was completed. At the end, first-time users were asked which analyser they found to be most user-friendly.

Subsequently, first-time users were asked if a POCT urine analyser was deemed useful in their daily practice and if they thought that using such an analyser would improve their productivity and efficiency, and whether the analyser would make urine analysis more precise.

Additionally, analyser characteristics collected from manufacturers’ information sheets were evaluated. The combined data were used to evaluate user-friendliness.

Results

Within-day and between-day performance was assessed using level 1 (negative) and level 2 (positive) Dipper Urine Dipstick Controls (Quantimetrix). All results were in agreement with the predefined expected values, meaning they showed negative results for level 1 controls and positive results for level 2 controls for nitrite, leucocytes and erythrocytes at the 20 within-day analyses and the 20 between-day analyses.

Agreement with laboratory reference standards

Seventy-seven anonymous patient samples were collected and analysed using the laboratory standard (Urisys 2400).

Eleven samples showed a positive result for nitrite, 32 samples tested positive on leucocytes (>25 leucocytes/μL) and 44 of the 77 samples showed the presence of >10 erythrocytes/μL. Samples were then analysed on the six different POCT urine analysers. Table 2 shows the sensitivity, specificity, PPVs and NPVs, as well as the κ coefficient and its 95% CI for nitrite, leucocytes and erythrocytes per POCT urine analyser using the Urisys 2400 as reference standard.

Table 2

Sensitivity, specificity, PPV, NPV and κ, with 95% CI between all six different urine analysers and the primary laboratory reference test (Urisys 2400)

		Urisys 2400		Sensitivity	Specificity	PPV	NPV	κ
		Negative	Positive
Nitrite
Clinitek Status	Negative	66	0	1 (0.67 to 1)	1 (0.93 to 1)	1 (0.68 to 1)	1 (0.93 to 1)	1
(Siemens)	Positive	0	11
Uryxxon Relax	Negative	66	0	1 (0.67 to 1)	1 (0.93 to 1)	1 (0.68 to 1)	1 (0.93 to 1)	1
(Macherey Nagel)	Positive	0	11
Urisys 1100	Negative	66	0	1 (0.67 to 1)	1 (0.93 to 1)	1 (0.68 to 1)	1 (0.93 to 1)	1
(Roche)	Positive	0	11
Aution micro	Negative	66	0	1 (0.67 to 1)	1 (0.93 to 1)	1 (0.68 to 1)	1 (0.93 to 1)	1
(Menarini)	Positive	0	11
Aution 11	Negative	66	0	1 (0.67 to 1)	1 (0.93 to 1)	1 (0.68 to 1)	1 (0.93 to 1)	1
(Menarini)	Positive	0	11
Urilyzer	Negative	62	0	1 (0.67 to 1)	0.94 (0.84 to 0.98)	0.73 (0.45 to 0.91)	1 (0.93 to 1)	0.816 (0.64 to 0.99)
(Analyticon)	Positive	4	11
Leucocytes
Clinitek Status	Negative	45	14	0.56 (0.38 to 0.73)	1 (0.90 to 1)	1 (0.78 to 1)	0.76 (0.63 to 0.86)	0.6004 (0.41 to 0.79)
(Siemens)	Positive	0	18
Uryxxon Relax	Negative	44	2	0.94 (0.78 to 0.99)	0.98 (0.87 to 1)	0.97 (0.81 to 1)	0.96 (0.84 to 0.99)	0.9194 (0.83 to 1)
(Macherey Nagel)	Positive	1	30
Urisys 1100	Negative	45	7	0.78 (0.60 to 0.90)	1 (0.90 to 1)	1 (0.83 to 1)	0.87 (0.74 to 0.94)	0.8067 (0.67 to 0.94)
(Roche)	Positive	0	25
Aution micro	Negative	44	2	0.94 (0.78 to 0.99)	0.98 (0.87 to 1)	0.97 (0.81 to 1)	0.96 (0.84 to 0.94)	0.9194 (0.83 to 1)
(Menarini)	Positive	1	30
Aution 11	Negative	43	2	0.93 (0.78 to 0.99)	0.96 (0.84 to 0.99)	0.94 (0.78 to 0.99)	0.96 (0.84 to 0.94)	0.8931 (0.79 to 1)
(Menarini)	Positive	2	30
Urilyzer	Negative	45	11	0.66 (0.47 to 0.81)	1 (0.90 to 1)	1 (0.81 to 1)	0.80 (0.67 to 0.89)	0.6905 (0.52 to 0.86)
(Analyticon)	Positive	0	21
Erytrocytes
Clinitek Status	Negative	33	20	0.55 (0.39 to 0.69)	1 (0.87 to 1)	1 (0.83 to 1)	0.62 (0.48 to 0.75)	0.507 (0.32 to 0.69)
(Siemens)	Positive	0	24
Uryxxon Relax	Negative	31	11	0.75 (0.59 to 0.86)	0.94 (0.78 to 0.99)	0.94 (0.79 to 0.99)	0.74 (0.58 to 0.86)	0.667 (0.50 to 0.83)
(Macherey Nagel)	Positive	2	33
Urisys 1100	Negative	33	16	0.64 (0.48 to 0.77)	1 (0.97 to 1)	1 (0.85 to 1)	0.67 (0.52 to 0.80)	0.600 (0.43 to 0.77)
(Roche)	Positive	0	28
Aution micro	Negative	33	20	0.55 (0.39 to 0.69)	1 (0.87 to 1)	1 (0.83 to 1)	0.62 (0.48 to 0.75)	0.507 (0.32 to 0.69)
(Menarini)	Positive	0	24
Aution 11	Negative	33	28	0.36 (0.23 to 0.52)	1 (0.87 to 1)	1 (0.76 to 1)	0.54 (0.41 to 0.67)	0.329 (0.13 to 0.53)
(Menarini)	Positive	0	16
Urilyzer	Negative	32	12	0.73 (0.57 to 0.85)	0.97 (0.82 to 1)	0.97 (0.82 to 1)	0.73 (0.57 to 0.85)	0.669 (0.51 to 0.83)
(Analyticon)	Positive	1	32

NPV, negative predictive value; PPV, positive predictive value.

For nitrite, all test characteristics (sensitivity, specificity, PPV, NPV) were high for all POCT urine analysers. For the leucocyte and erythrocyte test, specificity was higher than sensitivity for all POCT urine analysers. PPV and NPV were high for the leucocyte tests whereas for the erythrocyte test the PPV was high but the NPV was lower.

Agreement of the nitrite test between the various POCT urine analysers compared with the laboratory analyser was very good based on the κ coefficient for all variables. For the leucocytes tests, agreement varied between good to very good, and for the erythrocyte test, between moderate and good (table 2).

Additionally, samples were reanalysed using the Sedimax urine analyser as a secondary reference standard, and control for the primary reference standard (Urisys 2400), to obtain quantitative results concerning the presence of leucocytes and erythrocytes. Of the 77 samples, 27 samples contained more than 25 leucocytes/μL and 12 samples showed the presence of more than 10 erythrocytes/μL. Using the Sedimax analyser as the secondary reference standard, we calculated the sensitivity, specificity, PPVs and NPVs, as well as the κ coefficient and its 95% CI of the different analysers, as shown in table 3.

Table 3

Sensitivity, specificity, PPV, NPV and κ, with 95% CI for the POCT urine analysers, compared with the secondary laboratory reference test (Sedimax)

		Sedimax		Sensitivity	Specificity	PPV	NPV	κ
		Negative	Positive
Leucocytes
Clinitek Status	Negative	49	10	0.63 (0.42 to 0.80)	0.98 (0.88 to 1)	0.94 (0.71 to 1)	0.83 (0.71 to 0.91)	0.6602 (0.47 to 0.85)
(Siemens)	Positive	1	17
Uryxxon Relax	Negative	45	1	0.96 (0.79 to 1)	0.9 (0.77 to 0.96)	0.84 (0.66 to 0.94)	0.98 (0.87 to 1)	0.8345 (0.71 to 0.96)
(Macherey Nagel)	Positive	5	26
Urisys 1100	Negative	49	3	0.89 (0.70 to 0.97)	0.98 (0.88 to 1)	0.96 (0.78 to 1)	0.94 (0.83 to 0.98)	0.8839 (0.77 to 0.99)
(Roche)	Positive	1	24
Aution micro	Negative	45	1	0.96 (0.79 to 1)	0.9 (0.77 to 0.96)	0.84 (0.66 to 0.94)	0.98 (0.87 to 1)	0.8345 (0.71 to 0.96)
(Menarini)	Positive	5	26
Aution 11	Negative	44	1	0.96 (0.79 to 1)	0.88 (0.75 to 0.95)	0.81 (0.63 to 0.92)	0.98 (0.87 to 1)	0.8085 (0.67 to 0.94)
(Menarini)	Positive	6	26
Urilyzer	Negative	49	7	0.74 (0.53 to 0.88)	0.98 (0.88 to 1)	0.95 (0.74 to 1)	0.88 (0.75 to 0.94)	0.7596 (0.60 to 0.92)
(Analyticon)	Positive	1	20
Urisys 2400	Negative	45	0	1 (0.84 to 1)	0.9 (0.77 to 0.96)	0.84 (0.66 to 0.94)	1 (0.90 to 1)	0.8632 (0.75 to 0.98)
(Roche)	Positive	5	27
Erytrocytes
Clinitek Status	Negative	52	1	0.92 (0.60 to 1)	0.80 (0.68 to 0.89)	0.46 (0.26 to 0.67)	0.98 (0.89 to 1)	0.5091 (0.28 to 0.74)
(Siemens)	Positive	13	11
Uryxxon Relax	Negative	42	0	1 (0.70 to 1)	0.65 (0.52 to 0.76)	0.34 (0.20 to 0.52)	1 (0.90 to 1)	0.3627 (0.14 to 0.58)
(Macherey Nagel)	Positive	23	12
Urisys 1100	Negative	49	0	1 (0.70 to 1)	0.75 (0.63 to 0.85)	0.43 (0.25 to 0.63)	1 (0.91 to 1)	0.4884 (0.27 to 0.71)
(Roche)	Positive	16	12
Aution micro	Negative	52	1	0.92 (0.60 to 1)	0.80 (0.68 to 0.89)	0.46 (0.26 to 0.67)	0.98 (0.89 to 1)	0.5091 (0.28 to 0.74)
(Menarini)	Positive	13	11
Aution 11	Negative	59	2	0.83 (0.51 to 0.97)	0.91 (0.80 to 0.96)	0.63 (0.36 to 0.84)	0.97 (0.88 to 0.99)	0.6524 (0.42 to 0.88)
(Menarini)	Positve	6	10
Urilyzer	Negative	44	0	1 (0.70 to 1)	0.68 (0.55 to 0.78)	0.36 (0.21 to 0.55)	1 (0.90 to 1)	0.3951 (0.17 to 0.62)
(Analyticon)	Positive	21	12
Urisys 2400		33	0	1 (0.70 to 1)	0.51 (0.38 to 0.63)	0.27 (0.15 to 0.43)	1 (0.87 to 1)	0.2432 (0.04 to 0.44)
(Roche)		32	12

NPV, negative predictive value; POCT, point of care testing; PPV, positive predictive value.

For the leucocytes test, characteristics (sensitivity, specificity, PPV and NPV) were high for all POCT urine analysers. For erythrocytes, sensitivity was generally higher than specificity. The PPV for the erythrocyte test was generally low and NPV high for all POCT urine analysers.

Agreement of the leucocyte test between the various POCT urine analysers compared with the laboratory analyser was good to very good based on the κ coefficient for all variables. For the erythrocyte test, agreement varied between fair to good (table 3).

The secondary outcome measures were the user-friendliness of the POCT analysers, in addition to the analysers’ other characteristics. The GP practice nurses (all first-time users) found POCT urine analysers easy to use and most frequently did not have problems receiving test results. The susceptibility to flaws, either in preparation of the analyser, performing the analysis or reading the test results was considered lowest for the Uryxxon Relax (Macherey Nagel) and Urisys 1100 (Roche). Overall, the Uryxxon Relax (Macherey Nagel) was found to be most user-friendly by six of the nine first-time users. Six first-time users found that the POCT urine analyser would be useful in their daily practice. The majority of first-time users felt that the POCT urine analysers would improve efficacy and productivity. Seven out of nine first-time users felt that the use of POCT urine analysers would lead to a more accurate evaluation of the urine strip.

Device characteristics differ mainly in size and price of the device. All but the Aution Micro (Menarini) have the ability to connect to the GP's electronic medical record and to the laboratory information system. The Clinitek Status (Siemens), Aution 11 and Aution Micro (Menarini) also make it possible to perform additional tests, but this was not part of our study. All test strips used in this study contain pads for the analysis of additional tests such as glucose and protein. Although these are also used in general practice, we chose to focus on the tests most often used by GPs, that is, nitrite, leucocytes and erythrocytes, for the diagnosis of UTI.

Discussion

This is the first time that different commercially available POCT urine analysers have been evaluated on analytical performance, agreement and user-friendliness. All POCT urine analysers showed perfect analytical performance when evaluating within-day and between-day performance using control materials. Compared with laboratory reference standards, agreement was generally good to very good, but differed per parameter and per analyser. Concerning the nitrite test, all but one showed perfect agreement with the laboratory standard. For leucocytes and erythrocytes, specificity was high, but sensitivity was generally lower, when compared with the reference standard. Users (GP practice nurses and midwives) reported that the different POCT urine analysers were easy to use.

When comparing our study to the currently available literature, it is apparent that this is the first time that POCT urine analysers have been compared with a laboratory standard for use in general practice. Various studies report on the validation of new laboratory urine analysers.12 18–20 Two studies have shown that automatic reading of urine test strips is more accurate than visual reading of the same test strip.6 7 Peele et al already reported on this in 1977 using a semiautomated dipstick reader in a laboratory setting. However, technology has developed since and the described analyser is no longer available.6 Tighe7 performed his study in a laboratory setting, using aqueous spiked samples instead of patient samples, and although they did evaluate a POCT urine analyser, it was not compared with a laboratory reference standard.

Multiple studies, all conducted in a hospital laboratory setting, have shown that automated urinalysis, using different automated urinalysis systems (not for POCT), is acceptable for the screening of urine samples, and thus avoiding unnecessary urine culture.9 21 22 Mayo et al showed that when comparing two different automated test strip analysers in a laboratory setting, agreement was 99% for nitrite, 82% for leucocytes and 91% for erythrocytes. When comparing the Urisys analyser to manual examination of the urine sediment agreement was 90% for nitrite, 71% for leucocytes and 86% for erythrocytes.22 Chien et al compared three different automated test strip analysers, also in a laboratory setting. They found agreement was 97% for nitrite, 97% for leucocytes and 86% for erythrocytes.9 We have studied and compared all commercially available POCT urine analysers in the Netherlands concerning the most clinically relevant parameters for general practice. We found that agreement with the laboratory standard is similar to agreement found between urine analysers in a laboratory setting, and between urine analysers and manual examination of the urine sediment in a laboratory setting, in the above studies. This confirms that the currently tested POCT urine analysers perform sufficiently for use in primary care practices.

According to the guideline for UTI of the Dutch College of General Practitioners,3 nitrite, leucocyte and erythrocyte tests need to be performed for patients with possible UTI. A positive nitrite test is conclusive for a UTI. A negative nitrite test needs to be combined with the results of the leucocyte and/or erythrocyte test. If either the leucocyte and/or erythrocyte test is positive, the test needs to be followed up with (semi) urine culture. Our test results show that the POCT urine analysers, except for one, have excellent agreement for nitrite compared with the reference method. The presence of leucocytes and/or erythrocytes when tested on the POCT urine analysers rules in UTI in most cases (high PPV when compared with the reference method), whereas the absence of erythrocytes or leucocytes does not rule out UTI in all cases (relatively low NPV). The chance of missing a UTI is, however, reduced by the guideline statement that either leucocytes or erythrocytes needs to be positive to follow-up with (semi) urine culture.

Several limitations of this study need to be addressed. We used the Urisys 2400 automated urine analyser, the routine laboratory analyser at Saltro Diagnostic Centre, as our reference standard. Although the quality of this analyser is generally acknowledged, we cannot exclude the possibility that study results would have been different if another laboratory analyser had been used as a reference standard. Therefore, we have chosen to add additional reference standards for the measurement of leucocytes and erythrocytes. When interpreting the results, the differences in these reference standards have to be considered. When using the Sedimax urine analyser for quantitative measurement of leucocytes and erythrocytes, less leucocyte and erythrocyte positive samples were found than when samples were analysed using the laboratory standard Urisys 2400. An explanation for this is that leucocytes and erythrocytes may have lysed, which can occur when there is a renal infection, and may therefore not have been counted with the Sedimax urine analyser. Dipsticks do not have this limitation, which makes them more sensitive, and therefore possibly accounts for the difference in positive samples.

We compared different dipsticks from different manufacturers. Most dipsticks show their results in qualitative (1+, 2+, etc) and quantitative values. However, not all dipsticks use the same cut-off values for displaying their test results. We chose to use the cut-off value of 1+ as cut-off for a positive test because this is most often used in general practice. When translating back to quantitative results, using the tables provided by the manufacturer, values of leucocytes and erythrocytes can be vastly different. For example, 1+ leucocytes on a dipstick manufactured by Siemens translates to a quantitative value of 70 leucocytes/μL. All other dipsticks give a value of 25 leucocytes/μL as 1+. This problem has been addressed before and users need to be aware of the cut-off values as provided by the manufacturer when interpreting the test results of a urine analysis.21

Although our first-time user panel found that all six POCT urine analysers were relatively easy to use, a larger number of GP assistants and midwives performing the analysis in routine practice would have been desirable to draw firm conclusions on user-friendliness. But the simple test procedure (dipping the urine stick into the urine, entering the strip into the device and display reading of the test result) makes it hard to believe that conclusions would have been different.

This study shows that the POCT urine analysers have good analytical performance and their performance is sufficient to support use in general practice. POCT urine analysers may improve the quality of urinalysis in general practice but a direct comparison of visual interpretation versus automatic reading in routine general practice is necessary to judge the additional value of POCT urine analysis. This study is currently being performed at Saltro Diagnostic Centre.

Conclusions

All six POCT urine analysers in our study have good analytical performance for diagnosing UTI (nitrite, leucocytes and erythrocytes). All but one POCT urine analyser showed perfect agreement with the laboratory standard concerning nitrite test results. Concerning leucocytes and erythrocytes most POCT urine analysers showed high specificity, but sensitivity was lower. User-friendliness was judged as good for all analysers.