Intra- and inter-model reliability of Hemocue Hb 201+ and HemoCue Hb 301 devices.

AIM: The study intended to estimate the intra- and inter-model reliability and agreement between the HemoCue Models 201+ and 301 in a blood bank donor screening setting.
MATERIALS AND METHODS: Venous blood samples from 115 blood donors were analyzed by two Hemocue 201+ instruments and two HemoCue 301 instruments. The Lin's concordance correlation coefficient (CCC) as well as the Bland-Altman limits of agreement was measured.
RESULTS: The 95% lower confidence interval (CI) of all the CCCs had a value >0.90. The CCC between instruments belonging to the same model was compared to instruments belonging to different models. The mean of HemoCue 301 was higher than HemoCue 201+ by 3.04 g/L with the 95% CI of the differences (i.e., the limits of agreement) between the two models ranging from -3.35 to 9.44 g/L. The bias between two instruments was lesser, and the limits of agreement were even narrower between instruments belonging to the same model.
CONCLUSION: The inter-instrument reliability and agreement are satisfactory in a blood bank setting.

Introduction

The HemoCue instruments for hemoglobin estimation have been extensively used for hemoglobin screening.[12] The HemoCue Hb201+ system was first released in 1990 and extensively studied for the screening of anemia.[34] More recently, a HemoCue Hb301 model has been released. It is, therefore, of interest to study the inter-instrument reproducibility (including both within-model and between-model reproducibility) of these point-of-care devices. This would help in documenting whether HemoCue instruments of the same or different make can reliably replace another HemoCue instrument. Documentation of the reliability is especially important in a blood bank setting since multiple instruments are in use, and proper exchangeability of the results of the different instruments needs to be confirmed. However, such reproducibility studies are lacking. This study was designed to estimate the within-model and between-model reproducibility of the HemoCue models Hb 201+ and Hb301 in a blood bank donor screening setting.

Materials and Methods

The present study is a part of the routine quality assessment program for HemoCue analyzers in the blood bank of All India Institute of Medical Sciences Rishikesh. Ethical approval for publication of the study was obtained from the Institutional Ethics Committee. Guidelines of the Helsinki Declaration were followed in the conduct of the study.

The present study was done on 115 donors (consisting of 102 males and 13 females) who were found eligible to donate blood as per the national regulatory guidelines. The initial hemoglobin screening method for blood donation was hemoglobin estimation by the capillary method of a HemoCue instrument belonging to either Hb201+ or Hb301. Venous blood from 115 blood donors was collected during blood donation from the initial discarded blood in the diversion pouch in K2 ethylenediaminetetraacetic acid vials. Hemoglobin from each venous blood sample was estimated by two HemoCue Hb201+ and two HemoCue Hb301 devices. The mean hemoglobin of the donors, as estimated by an automated hematology counter (Sysmex XP-100), was 155 g/L with a standard deviation of 13 g/L. The analytical performance of the HemoCue instruments was already verified against the automated hematology counter Sysmex XP-100 when put into use; besides this, regular quality control checks were done to assure quality.

The readings from all the instruments were compared to one another. Reliability was assessed by the Lin's concordance correlation coefficient (CCC).[5] The CCC between each pair of HemoCue instruments was estimated. In addition, the overall CCC between HemoCue Hb201+ and HemoCue Hb301 was also assessed by calculating the CCC for repeated measures.[6]

The Bland–Altman limits of agreement were calculated to see if the measurements of the instruments were comparable.[7] The limits of agreement between each single instrument were measured by the classical method. Limits of agreement were also calculated between the HemoCue 201+ pair of instruments and HemoCue 301 pair of instruments by the method proposed by Zou using method of variance estimates recovery confidence intervals (CIs).[8] The 95% CIs of the mean difference and the limits of agreement were also estimated.

All studies were carried out on NCSS 11 Statistical Software, Kaysville, Utah, USA[9] and “cccrm” package[10] of R-statistical software by R Foundation for Statistical Computing, Vienna, Austria.[11] The study had >90% power of detecting differences from a Lin's CCC of 0.8 at a two-sided alpha of 0.05, with an expected value of Lin's CCC of 0.9.

Results

The results of the Lin's CCC and the Bland–Altman limits of agreement for different pairs of instrument studied are given in Table 1. The Bland plot showing the differences between each pair of measurements versus the mean value of the measurements between HemoCue Hb 201+ and HemoCue Hb 301 is given in Figure 1.

Table 1

Lin's concordance correlation coefficient between the different instruments and the Bland–Altman limits of agreement analysis

Figure 1

Bland plot showing the differences between each pair of measurements versus the mean value of the measurements between HemoCue Hb 201+ and HemoCue Hb 301

Discussion

Inter-instrument reliability and agreement are important quality indices, especially when there are multiple instruments for measuring the same parameter. Unless different instruments show a high inter-instrument reliability and agreement, the interpretation of estimates from the instruments will be limited. Since multiple HemoCue devices are commonly used in a point-of-care setting[12] or in a blood bank (as a part of routine donor assessment),[13] the inter-instrument reliability needs to be properly documented. However, there are very few studies reporting the inter-instrument and intermodel reliability of these devices.

The 95% lower CI of the Lin's CCC was found to be >0.90 in all cases, suggesting that the inter-instrument reliability was satisfactory. It was also observed that the within-model reliability was higher than the between-model reliability. Therefore, ideally, a blood donation center planning to have multiple screening instruments should be to ensure that the instruments are of the same model to ensure harmonization of the results after proper performance verification. However, when all the instruments cannot be from the same model, the harmonization between instruments is impeded though still remaining within satisfactory limits.

The means of the differences of measurements from two instruments of the same model are closer to zero than the differences of measurements from two instruments belonging to different models. Furthermore, the limits of agreement are also narrower for measurements between the same models compared to that between two different models. Therefore, both the bias and random error are lesser between two instruments from the same model as compared to two instruments from different models. Even then, the bias between two instruments of different models is around 3 g/L, which we feel is satisfactory.

We could not find any other study assessing the reproducibility of the different HemoCue models in a blood bank setting. One study compared HemoCue measurements taken on capillary samples in a primary care setting.[14] However, reproducibility measured from capillary sampling is not a good estimate of between-instrument reliability (though it may remain a good estimate of procedure reliability). This is because capillary sampling itself is associated with preanalytical errors (many of which are operator dependent), which obscures the actual performance and reliability of the instrument. Therefore, we obtained narrower limits of agreement as well as a higher Lin's CCC than the aforementioned study.

We have performed the present study in a blood bank setting. The population screened are more likely to be healthy and have higher hemoglobin levels than the general population, with lower hemoglobin levels between donor variability as compared to the general population. Since reliability is attenuated by a narrow range, the reliability of the HemoCue devices likely will be lower in a blood bank setting than a general patient care setting.[15] Therefore, demonstration of high reliability of these instruments is needed specific to a blood bank setting before being put in routine use in the same.

It should be noted that the present study is a reproducibility study and needs to be differentiated with a validation study, such as that previously reported on the same samples.[16] TThis study shows the extent of additional measurement errors when one HemoCue instrument is replaced by another and therefore the interchangeability of results from one instrument to another. A validation study, on the other hand, shows whether the method is suitable to put in routine use when compared to a gold standard. Both these types of studies are important: a validation study to document and confirm the accuracy of the method and a reproducibility study between instruments to document and confirm the interchangeability of results when inevitably there is a failure of one instrument and it needs to be replaced.

The present study just shows that the analytical reliability of the instruments is adequate; however, the reliability of the capillary method has to be assessed separately. We did not attempt to formally study the reproducibility of the capillary method since the reliability of this method is largely operator dependent and therefore not generalizable to an entire setting of the blood banks having heterogeneous levels of technician competence. To ensure reliability of the capillary method, ensuring proper preanalytical quality control by properly training the technical staff and uniform sample collection is required.

Conclusion

We found satisfactory within-model as well as between-model reproducibility for HemoCue Hb201+ and Hb 301 instruments, even in a blood bank setting. The within-model reliability is higher than the between-model reliability, with lower bias as well as random error; even then, the between model reliability is high and should be suitable for routine blood bank use.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.