Diagnostic accuracy of a bovine specific electronic beta-hydroxybutyrate handheld meter in fresh blood and stored serum samples.

This study aims to evaluate the diagnostic accuracy of the bovine specific beta-hydroxybutyrate (BHB) meter Nova Vet (NVET). We evaluated the accuracy and agreement of the NVET in fresh blood and thawed serum with the reference laboratory assay; and the repeatability, the interference by anticoagulants, and the optimum slope calibration factor. Individual blood samples were collected from 200 Holstein and crossbred cows between 3 and 14 days post-calving from 13 dairy herds in Minnesota. Using a laboratory assay with a cut point of 1.2 BHB mmol/L hyperketonemia prevalence was 10.6% (95% CI: 6.7, 15.8). The sensitivity of NVET in blood and serum was 100.0% while the specificity was 98.3 and 97.7% respectively. The agreement between NVET and the laboratory assay was the highest using blood samples (concordance correlation coefficient -CCC = 96.2, 95% CI: 95.0, 97.1. The coefficient of variation including within day (intra-meter), between- days, and -batches was 13.4% when testing blood samples. Minimal interference was observed with the use of anticoagulants (K-EDTA and Li-Heparin, CCC 0.90 and 0.93 respectively) in reference to whole blood without anticoagulant. The best calibration slope factor in serum was 1.0 (Youden's index: 0.98). Results suggest that the NVET device maintained a high accuracy and precision to quantified BHB concentration when applied in fresh blood and thawed serum under field conditions using the default calibration slope (1.0), and with minimal anticoagulant interference when used in whole blood samples.

Introduction

Dairy cows experience a state of negative energy balance as part of a normal peripartum period. To meet the increased energy demand of milk production, dairy cows mobilize fatty acids from the adipose tissue (Bell, 1995). These fatty acids are partially oxidized into ketone bodies, including beta-hydroxybutyrate (BHB), to be used as an energy source by extrahepatic tissues (Drackley, 1999; Grummer, 1993). Elevated levels of BHB in blood have been associated with impaired reproductive performance, reduction in milk yield, and increased metabolic disorders (Duffield, Lissemore, McBride & Leslie, 2009; LeBlanc, Leslie & Duffield, 2005; McArt, Nydam & Oetzel, 2012), hence, early detection of high BHB levels is crucial.

Reference laboratory tests typically use a spectrophotometric quantitative assay following an enzymatic reaction to determine BHB concentration in blood. Concentrations between 1.2 and 1.4 mmol/L are often used as a cut point for screening programs to diagnose hyperketonemia (HYK; Duffield et al., 2009; Oetzel, 2004; Ospina, Nydam, Stokol & Overton, 2010). However, reference techniques tend to be laborious and time-consuming therefore, multiple electronic handheld methods have been developed to test BHB in the field with variable diagnostic accuracies (Carrier, Stewart, Godden, Fetrow & Rapnicki, 2004; Iwersen, Falkenberg, Voigtsberger, Forderung & Heuwieser, 2009; Krogh, Toft & Enevoldsen, 2011). In order to implement adequate screening programs to detect HYK at the herd and individual level, accurate, precise, and practical diagnostics techniques are essential (Gordon, LeBlanc & Duffield, 2013; Tatone et al., 2016). The Nova Vet (NVET - Nova Vet, Nova Biomedical Co., Waltham, MA), is a specific device to measure BHB concentration recommended for its use in whole blood, that includes a calibration feature (i.e., slope calibration factor) intended to adjust for the variation of hematocrit among species. This device showed good diagnostic accuracy in blood BHB at the recommended slope setting in 2015 (Bach, Heuwieser & McArt, 2016). However, the optimum field sampling protocols, which can be advantageous for researchers or consultants (i.e., test performance on whole blood or serum samples, anticoagulant used, and unadjusted slope) remains unclear.

This project aimed to evaluate the test characteristics of the NVET in reference to the laboratory assay, and the best practices to maintain an appropriate diagnostic performance in farm settings. To achieve this aim we performed four specific objectives in two different observational studies. In the first study, (a) we evaluated the diagnostic accuracy of the device using whole blood and thawed serum, and (b) determined the best slope calibration factor for serum samples. In the second study, (c) we evaluated the precision intra-meter and between days and batches, and (d) assessed the effect of anticoagulants on test results.

Material and methods

Study design and data collection – study 1

All animal use procedures in this research were approved by The University of Minnesota Institutional Animal Care and Use Committee (#1406–31607A).

For the first study, a convenience sample of 13 dairies in southeastern Minnesota with a history of HYK in the previous year were selected from a cohort of herds participating in the Minnesota Dairy Herd Improvement Association. Selected farms were visited at least once between June and September 2014. Blood samples (n = 200) were collected from a random sample of cows between 3 and 14 days postpartum in each farm via the coccygeal vessels using 20-G x 2.54-cm blood collection needles and evacuated tubes without anticoagulant (Becton Dickinson; Franklin Lakes, NJ). The sample size was estimated to identify a desired sensitivity (Se) and specificity (Sp) of at least 90% and 85% respectively, with 80% power and an α error level of 0.05 and an expected HYK-prevalence of 30% (Dohoo, 2009). Immediately after sampling, a droplet of blood was taken directly from the tip of the needle to measure BHB concentration using the NVET handheld device at the out of the box slope setting of 1.0. Unlike the study by Bach et al., 2016 the 2013 manual for our meter did not include a recommendation to adjust the slope. Blood samples were allowed to clot in a cooler and centrifuged in the laboratory at 2000-x g for 15 min within 3 h of collection. Two aliquots of serum from each sample were stored in 2.0 mL vials (Sarstedt, Newtonm NC) and frozen at −80 °C for later analysis. One aliquot was shipped on ice to the Marshfield Veterinary Laboratory (Marshfield, WI) for BHB determination using a spectrophotometric quantitative assay, LiquiColor test (EKF Diagnostics-Stanbio, Boerne, TX) on a Beckman Coulter AU5800 (Beckman Coulter, Brea, CA, USA). The inter- and intra-assay coefficient of variation of the reference laboratory test was <1.7 and <5.2%, respectively. After obtaining the laboratory results, the second serum aliquot samples were thawed for one hour at room temperature and tested with NVET device with five different slope calibration factors (0.9, 1.0, 1.1, 1.25, and 1.5 respectively).

Study design and data collection – study 2

For the second study, blood samples were collected from three Holstein dairy cows between 3 and 14 days in milk on one day. Cows were housed at the University of Minnesota Dairy Cattle Teaching and Research Facility. Based on the farm's weekly BHB monitoring routine, we randomly selected one cow that had BHB ≥1.2 mmol/L and two control cows with BHB <1.2 mmol/L. On day one, three blood samples were collected from each cow using 10 mL evacuated tubes containing either potassium EDTA (K-EDTA - Covidien, Mansfield, MA), Lithium heparin (LI-HEP - Becton Dickinson, Franklin Lakes, NJ) or no-anticoagulant (Becton Dickinson; Franklin Lakes, NJ). After collection, blood samples with anticoagulants were gently mixed inverting the tubes 20 times. To assess NVET device repeatability across time and test strip batches we selected samples containing LI-HEP anticoagulant and tested them 20 times using two different testing strips batch dates. After testing, the samples were stored on a blood tube roller (Diagnostic Products Corporation, CA) to prevent clotting, and retested following a similar procedure on day two (n = 80). To assess the potential interference among the anticoagulants, individual blood samples containing either K-EDTA, LI-HEP, or no anticoagulant were tested four times for BHB immediately after collection (n = 24).

Statistical analyses

All statistical analyses were performed using R 3.4.4 software (R. RStudio, Inc., Boston, MA) .1 The test characteristics in study 1 were calculated using the “epiR” package (Stevenson, 2018). The strength of the association and the level of agreement between devices and the reference method was evaluated with Pearson's correlation coefficient and the concordance correlation coefficients (CCC), respectively, using the “DescTools” package (Signorell, 2018). Limits of agreement plot (LOA) was performed to visualize the agreement, using the “BlandAltmanLeh” package (Lehnert, 2015). Receiver operating characteristic curves (ROC) and the area under the ROC curve (AUC) were calculated to determine the most appropriate slope calibration settings for the NVET meter using “pROC” package (Robin et al., 2011). From the ROC curve, the Youden's Index was calculated to set the optimal thresholds based on the highest Se and Sp combined (Dohoo, 2009). For study 2, the coefficient of variation (CV) was determined within and between days, and between batch variations, and LOA plot, and CCC test was used to evaluate the interference of different anticoagulants on BHB measurement.

Results and discussion

The 99.9% (199/200) of the blood samples collected were successfully analyzed. The 29.1% (58/199) of the animals were in the first, 27.6% (55/199) in the second, and 42.7% (86/199) in their third or greater lactation. The true HYK prevalence was 10.6% (95% CI: 6.7, 15.8) based on the reference test. It is worth noting that the prevalence observed was lower than expected when the sample size was calculated resulting in an increase of the standard errors of the estimates. The Se and Sp were similar in blood and serum samples with better Predictive Values and correlation estimates for blood samples (Table 1).

Table 1

Diagnostic test characteristics of NVET BHB meter on blood and thawed serum samples with a BHB threshold of ≥1.2 mmol/L.

Characteristics of the tests	NVET on blood (95% CI)1	NVET on serum (95% CI)1
Apparent prevalence (%)2	12.1 (7.9, 17.4)	12.6 (8.3, 18.0)
Sensitivity	1.00 (0.83, 1.00)	1.00 (0.84, 1.00)
Specificity	0.98 (0.98, 1.00)	0.98 (0.94, 99.3)
Positive predictive value3	0.87 (0.95, 0.99)	0.84 (0.64, 0.95)
Negative predictive value3	1.00 (0.98, 1.00)	1.00 (0.98, 1.00)
CCC4	0.96 (0.95, 0.97)	0.89 (0.86, 0.91)
Pearson's correlation	0.96 (0.95, 0.97)	0.92 (0.89, 0.94)

Compared to the reference test (EKF Diagnostics-Stanbio, Boerne, TX). Estimates include the 95% confidence intervals of the estimate.

The estimated prevalence by the laboratory assay (reference test) was 10.6%.

Predictive values were calculated based on the estimated prevalence.

Concordance Correlation Coefficient between the devices and the reference test.

High diagnostic accuracy of the BHB device is crucial for the detection of HYK. The Se and Sp for the NVET in both blood and serum were the same, 100% and 98% respectively, with overlapping confidence intervals (Table 1). This contrasts to previously reported results for the NVET using an unadjusted slope calibration for whole blood in which a lower Se (64%) but similar Sp (100%) were observed (Bach et al., 2016). Correlation and concordance between the NVET and the laboratory assay were slightly higher in blood (0.96 both) than in serum (0.92 and 0.89, respectively). Similarities in the results indicate a lack of evidence of either a systematic or a proportional bias. Moreover, LOA plots (Fig. 1) show that most of the observations for both blood and serum, in reference to the laboratory assay, laid within the limits of agreements. These results indicate that serum samples collected for other purposes can be tested in batches after separation, improving efficiency, and reducing the lock-up time.

Fig. 1

Bland Altman plot (limits of agreement) representing the differences in measurements of beta-hydroxybutyrate (BHB) concentrations between the reference test and Nova Vet on (A) fresh blood, and (B) thawed serum. The solid line represents no mean difference of BHB concentration between tests. The dashed middle line represents the mean difference. The upper and lower dashed lines represent the 95% limit of agreement. Analysis performed using n = 199 samples from Holstein cows between 3 and 14 days in milk.

The test characteristics of each calibrated slope setting in serum are presented in Table 2. All the slopes had an AUC of 0.99, and 95% confidence intervals ranging from 0.97 to 1.00. Based on Youden's index, the best calibration slope factor with a BHB threshold of 1.2 mmol/L was the default slope calibration factor of 1.0. Therefore, using the default slope of 1.0 is appropriate when using the NVET in serum to measure BHB concentrations. The agreement between serum and whole blood observed in our study suggests that this slope is likely also appropriate for whole blood.

Table 2

Diagnostic test characteristics of Nova Vet (NVET) meter measuring beta-hydroxybutyrate on blood using ≥1.2 mmol/L threshold under different slopes calibration factor.

Slope Calibration Factor		Sensitivity (95% CI)	Specificity (95% CI)	Positive Predictive Value (95% CI)	Negative Predictive Value (95% CI)	Youden's Index1
0.9		0.90 (0.70, 0.99)	0.99 (0.97, 100)	0.95 (0.75, 100)	0.99 (0.96, 1.00)	0.89
1.02		1.00 (0.84, 1.00)	0.98 (0.94, 0.99)	0.84 (0.64, 0.95)	1.00 (0.98, 1.00)	0.98
1.1		1.00 (0.84, 1.00)	0.93 (0.89, 0.96)	0.64 (0.45, 0.80)	1.00 (0.98, 1.00)	0.93
1.2		1.00 (0.84, 1.00)	0.80 (0.74, 0.86)	0.38 (0.25, 0.51)	1.00 (0.97, 1.00)	0.80
1.5		1.00 (0.84, 1.00)	0.63 (0.55, 0.70)	0.24 (0.16, 0.35)	1.00 (0.97, 1.00)	0.63

Youden's Index represents the optimal threshold of a diagnostic test that maximizes overall classification (i.e., higher sensitivity and specificity combined). It is calculated as the sum of sensitivity and specificity minus one).

Default slope calibration factor.

In study 2, the coefficient of variation including different testing strip batches using samples of whole blood with anticoagulant containing LI-HEP was 13.4% on day one and 12.0% on day two. The precision of the NVET meter was lower than previously reported with the use of samples containing heparin (Bach et al., 2016). However, in our study, we assessed repeatability including different batches of strips and testing days mimicking what can be expected in field conditions, which may have played a role in the lower precision estimates observed. A limitation of our precision results is that we tested a limited number of concentrations; however, these concentrations cover the clinically important range. Anticoagulants can interfere with the determination of certain analytes (Sevastos et al., 2006), which can occur through a variety of mechanisms including the chelation of minerals and the influencing of reagent enzymes (Bowen & Remaley, 2014; Tate & Ward, 2004). In our study, the enzymatic reaction between the BHB and the electrodes that take place in the chemistry layer of the testing strip did not appear to be affected by the presence of LI-HEP or K-EDTA. The CCC when using K-EDTA was 0.90 (95% CI: 0.85, 0.94) and a LOA mean difference in BHB concentration was - 0.01 (95% LOA −0.16 and 0.14). Blood samples containing LI-HEP had a CCC of 0.93 (95% CI: 0.86, 0.96) and LOA mean difference of 0.07 (95% LOA −0.06 and 0.19.) Thus, the effect caused by K-EDTA or LI-HEP in the measurement of BHB was minimal compared to results from blood samples without anticoagulants. This suggests that in scenarios where a large number of blood samples need to be collected in a short period to obtain plasma, the use of these samples collected with an anticoagulant can be considered an alternative to immediate testing.

Conclusion

In summary, based on our results the NVET is an acceptable alternative to the laboratory assay to measure BHB in whole blood with or without anticoagulant or serum samples using the default calibration slope setting with acceptable accuracy and precision for screening purposes.

Ethical statement

All animal use procedures in this research were approved by The University of Minnesota Institutional Animal Care and Use Committee (#1406–31607A) in 2014, Saint Paul, MN, United States. This has been explicitly stated in the Materials and Methods section of the submitted manuscript.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. We assessed the performance of a bovine specific beta-hydroxybutyrate (BHB) meter own by Nova Biomedical Co. (Waltham, MA). Nova Biomedical Co. provided the BHB meter devices and test strips for the study. However, as specified in the manuscript, Nova Biomedical Co. was not involved in any phase of data analyses nor interpretation of results.