Evaluation of four commercial tests for detecting ceftiofur in waste milk bulk tank samples.

The objective of this study was to identify factors affecting the accuracy of four commercial tests for ceftiofur drug residue in milk samples from bulk tank waste milk (WM). WM samples were collected from 12 California dairy farms which were initially tested using liquid chromatography (LC-MS/MS) to confirm their negative status for drug residues above the FDA established tolerance/safe levels. The milk samples were also tested for fat, protein, lactose, solids non-fat (SNF), somatic cell count (SCC), coliform count, and standard plate count (SPC). Each WM sample was divided into two aliquots, one labeled as negative for drug residues (WMN) and the second spiked with ceftiofur as positive for ceftiofur residues (WMPos). Both types of WM samples were tested to evaluate the performance of 4 commercially available tests: Penzyme® Milk Test, SNAP® β-lactam, BetaStar® Plus and Delvo SP-NT®. Three assays in triplicates for the WMN and WMPos were conducted for each WM sample. Test were evaluated using sensitivity, specificity, positive predictive value, negative predictive value and positive likelihood ratio. Kruskal-Wallis method was used to evaluate the effect of milk quality parameters on true positive (TP) and false negative (FN) test results. All WMPos samples were identified as positive by all four tests, rendering 100% sensitivity for each test. The specificity for Penzyme, BetaStar, Delvo, and SNAP tests were 59.2, 55.5, 44.4, and 29.6, respectively. Overall, all tests correctly identified samples with ceftiofur residues (WMPos), as shown by 100% sensitivity. Greater variability was observed regarding identification of samples free of any drug residue, with Penzyme and BetaStar having the highest risk for correctly identifying TN samples. Our findings indicate that when selecting commercial tests to detect drug residues in WM, milk quality parameters must be considered if the aim is to reduce FP test results.

Introduction

Antimicrobial resistance is a great concern for human and animal health. Ceftiofur is a third-generation cephalosporin, a drug class of critical importance to both human and veterinary medicine; hence extending bacterial susceptibility to ceftiofur is a priority [1, 2]. Ceftiofur’s broad antimicrobial spectrum explains its high demand in veterinary medicine. Moreover, ceftiofur is available in food animal formulations with no milk withdrawal periods, making it a preferred antimicrobial drug choice to treat dairy cattle infections.

Transition milk from recently calved dairy cows and milk from hospital cows that contains drug residues during treatment regimens are not fit for human consumption. Such milk, commonly identified as waste milk (WM), is harvested separately from saleable milk and utilized as a feed source for newborn calves by one-third of US dairy farms [3] and by 75.2% of CA dairies in a recent survey [4]. Two recent studies identified ceftiofur as the most common drug residue in WM samples from dairy farms in NY and CA [5, 6]. Furthermore, feeding calves WM containing ceftiofur residues at the concentrations observed on dairies has been shown to result in selection of E. coli resistant to ceftiofur and ceftriaxone, as well as other medically important drugs [7].

Current methods available for detecting drug residues in milk include microbial growth inhibition assays, microbial receptor assays, receptor binding assays, immunologic assays, enzymatic assays, and chromatographic analysis [8]. Such commercial tests are intended for use in raw, commingled saleable milk [9]. Although prior studies have evaluated the impact that many factors can have on test ability to correctly identify samples with drug residues (e.g. milk composition, SCC and total bacteria count), no study has evaluated these tests using bulk tank WM [10, 11]. The objective of this study was to identify factors affecting the accuracy of four commercial tests for ceftiofur drug residue in milk samples from bulk tank waste milk (WM).

Material and methods

Sample selection and processing

Waste milk samples were collected from 12 different commercial farms across California between September 2016 and March 2017. Waste milk samples were collected after verbal consent was obtained. Sample collection details are described previously [6]. Samples were initially tested using liquid chromatography–mass spectrometry/mass spectrometry (LC-MS/MS) to evaluate if they were negative for drug residues present above the FDA established tolerance/safe levels. The limits of quantification (LOQ) for the 27 drug residues screened using liquid chromatography–mass spectrometry/mass spectrometry (LC-MS/MS) are reported in . Liquid chromatography is the gold standard for identifying and quantifying antimicrobial drug residues in milk samples (Gibbons-Burgener, et al., 2001). A pitfall of the LC-MS/MS screening panel used in this study is the lack of screening for amoxicillin, one of the drugs that could result in a positive test using the commercial tests being evaluated in the current study. The BetaStar (BetaStar® Plus, Neogen Animal Safety, Lexington, KY) test was used to identify samples that could be positive for amoxicillin. BetaStar was selected for ampicillin screening because of its ability to differentiate between ceftiofur and other beta lactams (amoxicillin, ampicillin, cephapirin, cloxacillin, and penicillin). Twelve milk samples collected in the original study were selected based on negative test results for beta-lactam drug residues in the milk using LC-MS/MS results. From these 12 samples, three were identified as being positive for amoxicillin drug residues using the BetaStar test, and were excluded from the study. Samples size calculation for detecting a difference between two means between false positive and true negative samples for all milk quality parameters was conducted using this function in JMP (SAS Institute Inc., Cary, NC). The parameters and values used for this calculation were based on our current dataset, and output of this analysis is displayed in .

Milk samples were also tested for fat, protein, lactose, solids non-fat (SNF) percent, somatic cell count (SCC, cells/ml), coliform count (CFU/ml), and standard plate count (SPC, CFU/ml) by the Sierra Dairy Laboratory (Tulare, CA). Each WM sample was divided into two aliquots, one labeled as negative for drug residues (WMN) and one as positive for ceftiofur residues (WMPos). The WMPos samples were WMN samples that were spiked with ceftiofur to a final concentration of 100 ppb of ceftiofur, which is the FDA tolerance for ceftiofur in saleable milk and should result in a positive test result using a commercial test for ceftiofur in saleable milk [12]. The samples (9 WMN and 9 WMPos) were tested to evaluate the performance of 4 commercially available tests: Penzyme® milk test (Neogen Animal Safety, Lexington, KY), SNAP β–lactam® (IDEXX Laboratories, Westbrook, ME), BetaStar Plus® (Neogen Animal Safety, Lexington, KY), and Delvo SP-NT® (DSM Food Specialties, South Bend, IN). These commercial test were not developed for screening of drug residues in waste milk samples, and therefore findings of this study should not be extrapolated to conditions to which these test were developed.

Negative and positive controls for ceftiofur residues were tested using pasteurized whole milk. Four assays in triplicate for the WMN and WMPos were conducted for each WM sample. All samples were vortexed for 30 seconds prior to testing. Samples were tested using the 4 commercial assays, following manufacturer’s recommendations. Test results were labeled as true positive (TP) when resulting in a positive test result in WMPos samples, and false negative (FN) when a negative test result was obtained from WMPos samples. Test results were labeled as true negative (TN) when resulting in a negative test result in WMN samples. Samples were labeled as false positive (FP) when a positive test result was observed for WMN samples, indicating that although drug residues for β-lactam drug were below the FDA tolerance or safe levels for milk, the test still indicated that the sample was positive. In addition to the milk characteristics evaluated in the study that could affect test results, a drug residues detection limit below that established by the FDA could also potentially result in FP result. Milk drug residues screening test detection limits for Beta-lactam drugs used the study are shown in .

The assays were tested in triplicates for the WMN and WMPos for each WM sample for each of the four commercial tests, resulting in a total of 216 tests being conducted. Milk samples were thawed the night before analysis in a 4 ºC refrigerator. All samples were vortexed for 30 seconds prior to testing.

Statistical analysis

For each assay, sensitivity, specificity, positive predictive value, negative predictive value and positive likelihood ratio were calculated using standard methods [18]. For the dataset of the WMN samples that were not spiked, the effect of milk quality parameters on the within and between test variation of results indicating false-positive and true-negative test results were evaluated.

For each commercial test, analysis of variance models were created with the following dependent variables: fat percent, protein percent, lactose percent, SNF percent, SCC (cells/ml), coliform count (CFU/ml), and SPC (CFU/ml). The binary categorical variable for a test classified as false-positive or not, or true-negative or not was used as the explanatory variable. Assay number (repetitive measure) was controlled in the models as a random effect. Homoscedasticity was evaluated by plotting the standardized residuals versus predicted values, and supported the assumption for a linear regression model when points were reasonably equally distributed across all values of the independent variables. Normality assumption was evaluated by visual inspection of the data, distribution frequency (histogram), normal quantile plots and also by using the Shapiro-Wilk test. The equal variance assumption was evaluated by Levene and Welch’s tests. When any of these assumptions were violated, the non-parametric Kruskal-Wallis test was used. A variable was considered significant when a P value ≤ 0.05 was observed. All analyses were conducted using JMP (SAS Institute Inc., Cary, NC).

Results

Mean value (95% CI) for WM quality parameters (composition and microbiology) for samples used in the study (n = 9) were: 4.7% (3.4–5.9) fat, 3.7% (3.4–3.9) protein, 4.3% (4.1–4.5) lactose, 8.7% (8.5–8.9) SNF, 2,378 104cells/ml (1,457–3,299) SCC, 293 CFU/ml (81–667) coliform count, and 92,000 CFU/ml (39,382–144,617) SPC ().

All WMPos samples were identified as positive by all four tests, rendering 100% sensitivity. Results for specificity (Sp), positive predictive value (PPV) and positive likelihood ratio (LR+) are presented in . Least square means (LSM) results from linear regression for milk quality parameters by false positive (FP) and true negative (TN) status are depicted in Fig 1. All milk quality parameters rejected the test of normality, and the Kruskal-Wallis nonparametric approach was used. Results for this analysis found to significant difference between FP and TN results, and are summarized in . No false negative results were observed in the study for any of the commercial tests used.

Fig 1

Mean value for fat (%), protein (%), solids-not-fat (%), somatic cell count (CFU/ml), standard plate count (CFU /ml), coliform counts (CFU /ml), and lactose (%) for false positive (FP) and true negative (TN) test results for the four commercial tests.

Error bars correspond to 95% confidence interval. An asterisk represents test for which a significant difference was observed between FP and TN result based on Kruskal-Wallis nonparametric method.

Repeatability within individual tests for each triplicate milk sample tested revealed intra-individual test disagreement for only Penzyme and SNAP tests, with two negative test results and one positive test results for two different milk samples that were not spiked with ceftiofur, and two negative test results and one positive test result for one milk sample that was not spiked with ceftiofur, respectively.

Discussion

Although these four commercial tests for detecting drug residues in milk are not approved for detection of drugs residues in WM samples, our findings support that all four tests detected ceftiofur drug residues with excellent sensitivity in waste milk samples with known added drug concentrations, as reflected by a 100% sensitivity for all four tests in the study samples. When evaluating the within-individual test repeatability, most tests had high specificity, with only Penzyme and SNAP having disagreement between triplicate test results. The most common known use of WM on dairy farms is feeding it to calves, which has been shown to result in significant increases in selection for antimicrobial resistance to multiple drugs including cephalosporins, and increases in multidrug resistance in fecal E. coli [7]. Furthermore, recent research has also shown that feeding WM to calves impacts the development of the enteric microbiota of calves, when compared to calves fed whole milk without antimicrobial drug residues [19]. However WM is not always used as a feed source, and a recent questionnaire by the European Food Safety Authority (EFSA) Panel on Biological Hazards (BIOHAZ) showed that the disposal of WM when not fed to animals in countries in the European Union included incineration, use as an organic fertilizer, mixing in manure and spreading on land without processing, and composting [20]. These alternative approaches for disposal of waste milk could also result in selection of antimicrobial resistance in the environment, and therefore development of more effective methods for removing the undesired effects of antimicrobial residues in waste milk before disposal would be beneficial. Availability of commercial tests to detect WM samples positive for drug residues is an initial step for selecting batches of WM that may need additional treatment for degrading antimicrobial residues prior to being used as a feed source or disposed in the environment.

Delvo was the test most affected by different milk parameters. Milk samples with FP Delvo test results had lower percentages of fat percent, protein percent and SNF, and higher coliform counts when compared to samples with TN results (). Delvo test is a microbial growth inhibition assay with higher risk for FP test results linked to high SCC and the presence of the natural inhibitors (i.e., lysozyme and lactoferrin) in the samples. Furthermore, FP results have been associated with testing milk from individual animals with mastitis [10, 21–23]. Higher coliform counts observed in WM samples could be originating from fecal contamination (e.g. manure) or due to a high number of cows with coliform mastitis being milked into the WM bulk tank. Quarters with coliform bacteria have been shown to be high on lactoferrin [24]. Although in our study we did not measure lactoferrin, one explanation is that it could be responsible for the observed higher chance for FP in WM samples with significantly higher average coliform counts.

Although samples identified as FP using the Delvo test had lower protein, fat and SNF content when compared to TN samples, it must be noted that such parameters were still above the mean value expected to be found in saleable whole milk (). For example, milk fat percent and SNF percent had mean values of 3.8% and 8.9%, respectively, in saleable bulk tank milk samples from CA in a 2017 report [25]. Furthermore, the fact that previous studies [26, 27] did not identify these factors as increasing the chance of FP test results with the Delvo test could be due to the fact that they did not evaluate these parameters within the range observed in WM samples. Furthermore, our study indicated that a higher risk for TP test results for milk protein percent, fat percent and SNF percent was observed at a specific range, which was above expected values for normal milk samples and below mean values observed in WM samples in our study. This is in contrast to expecting a linear continuous increase in FP as values for these milk parameters increase ().

False positive SNAP test results were observed in WM samples with higher SCC (cells/ml) when compared to TN results (). The SNAP β-lactam assay identifies antimicrobial drug residues through the use of an enzyme-linked receptor-binding assay. Andrew et al. (2001) also reported that increasing somatic cells scores (SCS) were associated with an increase in FP outcomes (P value < 0.01) for the CITE SNAP test (IDEXX Laboratories, Westbrook, ME). In the later study, SNAP tests were evaluated using colostrum and transition milk, with mean SCC values for colostrum of 2,458,000 cell counts/ml, which is similar to the mean value observed in WM used in our study.

False positive milk samples identified using the BetaStar test had higher SCC (cells/ml) and coliform counts when compared to TN samples (S4 Table). BetaStar is a selective receptor-based beta-lactam test for penicillin-binding protein [28]. BetaStar test is based on a specific beta-lactam receptor and protein linked to gold particles, and works in a two-step phase: 1) the preliminary incubation of a receptor with milk, will result in an interaction between the receptor and any beta-lactam antibiotics if present; 2) the incubated medium is allowed to migrate up the dipstick, and if the receptor molecule has not been in contact with beta-lactam antibiotics, the dipstick band will capture all the receptor molecules. This will result in a visible red band being formed in the test area, interpreted as a negative test result. If no band is formed, the receptors have been blocked by beta-lactam antibiotics, interpreted as a positive test result.

In our study, our hypothesis is that high SCC could have interfered with the dipstick receptors ability to bind to receptor molecules, resulting in FP results. Specifically, the SCC interfere by reacting with the receptor molecule instead of the beta-lactam molecule. Receptor-based beta-lactam tests have been shown to be affected by high SCC (SCC > 4,000,000 cells/ml) resulting in FP results, which has been believed to be caused by natural inhibitory substances (i.e., lysozyme and lactoferrin) observed at higher concentrations in milk with high SCC [29].

Milk samples with FP test results based on the Penzyme test had lower SPC (CFU/ml) when compared to TN samples (). The Penzyme test uses an enzymatic colorimetric technique based on an enzymatic reduction reaction within the β-lactam ring, and higher SPC could potentially interfere with this reaction [10]. Milk fat has previously been observed to increase FP test results using the SNAP test, by hindering movement of milk through the assay and causing a lack of chemical reaction [11]. Similar findings have been observed for the Penzyme test [27]. Although we did not observe that in our study, milk samples with FP test results using either the SNAP and Penzyme tests had higher mean fat percent compared to TN test results ().

In addition to milk composition characteristic that could interfere with ability of the test to correctly identify positive and negative samples for drug residues, a characteristic of commercial test for drug residues in milk that could result in higher false positive results is the fact that the limit of detection for some of the β-lactam drugs tested is below the FDA tolerance or safe levels for milk (). Currently the FDA has established that penicillin, ceftiofur, cloxacillin, cephapirin, amoxicillin, and ampicillin are the beta-lactam drugs most commonly used to treat disease in lactating dairy cattle, and it recommends the use of a test that has been show to detect at least four of the six beta-lactams be used [15]. Tolerances and safe levels for drug residues in for milk in the U.S. can be found in the 21 code of federal regulation (CFR) 556 [30]. When establishing the limits of detection for new test, the FDA has determined that for acceptance these test shall not detect drug residues at less than 50% of the tolerance level or 25% of the target testing level for individual drugs, with the exception of penicillin G and tetracycline drug tests [9, 31]. Because of these standards for commercial drug residue test in the U.S., a false positive could occur when a drug residues is below the FDA tolerance or safe levels for milk but yet still at or above the limit of detection for the commercial test. In our study, we used the golden standard LC-MS/MS to label waste milk samples from the farm as being negative for drug residues based on FDA tolerance or safe levels for milk cut-off values, and therefore drug residues within the interval tolerance/safe and limit of detection for a test could have been due to presence of drug residue alone. An interesting finding however is that although Delvo test had the lowest detection limit to four of the six β-lactam drug screened when compared to the other three test, it did not have the lowest specificity for detection of waste milk samples without drug residues (Table 2). This supports the argument that false positive test were less likely to be caused by residues within the interval tolerance/safe and limit of detection for a test.

Table 2

Overall specificity (Sp), positive predictive value (PPV), and positive likelihood ratio (LR+) for detection of ceftiofur residues in milk using four commercial assays.

All four tests had 100% sensitivity.

	% Sp (95% CI)1	% PPV (95% CI)1	LR+ (95% CI)1
Penzyme2	59.2 (43.3–75.2)	71.1 (62.8–79.4)	2.5 (1.4–3.5)
BetaStar3	55.5 (N/A)*	69.2 (N/A)*	2.2 (N/A)*
Delvo4	44.4 (N/A)*	64.3 (N/A)*	1.8 (N/A)*
SNAP5	29.6 (13.7–45.5)	58.7 (53.3–64.1)	(1.1–1.7)

1. 95% confidence interval for results from triplicate testing.

2. Penzyme® Milk Test.

3. BetaStar® Plus.

4. Delvo®- SP.

5. SNAP® β–lactam.

* no variability in triplicate testing results (complete agreement between triplicate testing).

Divergent from the U.S. FDA approach for determining acceptable accuracy limits for commercial test screening for drug residues in milk, the European Community utilizes the Council Directive 96/23/EC as a reference for validating approaches to measure and interpret results of residues in milk [32]. Geographical differences between similar commercial tests due to local regulations should be taken into account when interpreting the findings of our study. As an example, the Council Directive 96/23/EC document includes the use of different approaches for test validation, including the use of detection capability (CCβ), which is the smallest content of the substance that may be detected, identified and/or quantified in a sample with an error probability of β, and decision limit (CCα), which is the limit at and above which it can be concluded with an error probability of α that a sample is non-compliant. As outlined in a recently published study conducted in France assessing the risk of positive reaction to widely used rapid screening test for inhibitors in milk from cows treated with a new ceftiofur formulations, CCβ and CCα are not usually clearly indicated by manufacturers of rapid drug residues tests, and because of that the lowest detected concentrations was estimated to be an appropriate threshold for the interpretation of test results [33].

Conclusion

All four commercial tests showed 100% sensitivity for identifying samples with ceftiofur residues in inoculated WM samples. Greater variability for FP results was observed, with Penzyme and BetaStar having the highest sensitivity for identifying TN samples. Each test was significantly affected by at least one milk quality parameter, with Delvo tests TN and FP test results affected by 4 different milk quality variables. Our findings indicate that when selecting commercial tests to detect drug residues in WM, the four commercial tests evaluated milk quality parameters must be considered if the aim is to reduce FP test results.

LC-MS/MS limit of quantification for drug residues in milk.

(DOCX)

Click here for additional data file.

Sample size output for quality parameters for waste milk samples (n = 9).

For all calculation a significant level of α = 0.05 was used.

(DOCX)

Click here for additional data file.

Quality parameters for waste milk samples (n = 9).

(DOCX)

Click here for additional data file.

Results from Kruskal-Wallis nonparametric test evaluating a significant difference between FP and TN results for each variable.

(DOCX)

Click here for additional data file.

13 Aug 2019

PONE-D-19-20423

Evaluation of four commercial tests for detecting ceftiofur in waste milk bulk tank samples

PLOS ONE

Dear Dr. Pereira,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The manuscript should be revised deeply. The main problem found in the manuscript is related to the some aspects of methodology, statistical analysis and redaction style. Please provide more reliable information on the sample collection. The manuscript should be presented according to guidelines for authors of Plos One. Thank you for your hard work.

We would appreciate receiving your revised manuscript by Sep 27 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Arda Yildirim, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Additional Editor Comments:

The study is very well presented, but lacks in the quality of preparation. I agree with reviewers. The main problem found in the manuscript is related to the some aspects of methodology, statistical analysis and redaction style. Please review the referee comments and make your peer revision.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Dear Editor,

Dear Authors,

First of all, I would like to thank you for the opportunity to review this manuscript on the Evaluation of four commercial tests for detecting ceftiofur in waste milk bulk tank samples. This article presents a primary scientific research in veterinary science of some interest in the current context of antimicrobial stewardship improvement.

To the best of my understanding, the objective of this study was to assess the reliability of commercial screening test kits in detecting ceftiofur residues from waste milk (WM). We agree that those tests have not been developed for this objective, but the control of bulk milk tank, supposedly free of a range of antimicrobials (inhibitors). Therefore, the most interesting feature of such tests is their negative predictive value, or their ability to tell us that the tested milk is free of contamination with a high level of confidence (in order to avoid technological and public health issue). Nevertheless, testing waste milk, when WM comes from cows at risk of having been treated with AB and because WM is massively used to feed newborn calves, is an acceptable stance. Thank you for investigating that.

This paper is well written in acceptable American English. The experiment is clearly presented, and the explanations are easy to follow. I support the publication of this article in PLOS One, provided that the authors respond to the following comments and consider revising certain points.

Major revision: Objective of this research (Abstract L21-22, Introduction L65-66)

This research is presented as an attempt to identify the ability of four commercial screening tests for detection of ceftiofur in WM. Since those tests have not been designed for this use, or the use on individual milk, the reader can ask himself about the real need of this research. So this paper is about test accuracy. The clearly exposed consequences of massive use of WM in newborn calves, and the early exposition to sub-therapeutic doses of antimicrobials suggest that it would be better to do not feed these animals with milk testing positive for inhibitors. The last sentence of the conclusion (L294-295) as well as the abstract and the introduction suggest that the objective of this research is to avoid the misclassification of healthy WM that can be discarded whereas it could be fed to calves. In case this objective exists in this research, it must be disclosed and the consequences of discarding FP WM explained. Alternatively, I suggest cutting these two sentences. Please clarify.

Major revision: Methodology, detection limits, Se, Sp

Based on the proposed methodology (true negative samples – HPLC tested, true positive samples – spiked with ceftiofur 100ppb), it is indeed possible to calculate the actual Se and Sp of the four tests. However, in the real world of milk testing, things go differently. Tests have been designed to maximize the detection of a variety of inhibitory substances, practically of all kinds for Delvotest. Many substances can be detected at concentrations far below the applicable MRL. Therefore, Se is generally good, whereas Sp does not make sense really, or it makes sense at the class level (b-lactams). You duly provided the necessary information in Table 1, the so-called detection limits (for b-lactam drugs). I agree that milk-related factors can influence these limits, and I appreciate your work for that.

However, Table 1 raises two questions.

- References are acceptable but they should be discussed, even data referring to official FDA documents. Most of the information available for these screening are provided by manufacturers. Over time, the information regarding detection limits change. There are for example several version of the SNAP test, available in different regions, at the global level. I expect a short discussion of these figures.

- The detection limit is not a totally clear concept. The detection capability (CCβ) and the decision limit (CCα) are two important performance characteristics of confirmatory methods for banned substances. For the EU we refer to Directive 2002/657/EC. Such a guidance does certainly exist for the US. I expect a short discussion of these statistical questions.

May I suggest you read a recently published paper on this matter? > doi:10.1136/vetreco-2018-000329

Minor revision: Figure 1

Figure 1 is poorly presented. It is supposed to present the core results of this study. Title and legend is required.

Typos

- L234: that (duplication)

- L404: SuPPoting information

Reviewer #2: It is unclear whether the sample size represents the population? (power analysis should be done).

The statistical tests used should be explicitly given in the method.

The normal distribution of the data can be determined by the Shapiro Wilk test. (if n<=30).especially in coliforms Count measurements, std error is higher than average, therefore non parametric methods should be used. normal distribution cannot be observed. (Mann Whitney U Test or median test can be used.)

Some typos should be corrected. (In Table S1)

The statistical method described in the abstract and the narrative in the statistical method is not similar. Specially linear regression or multiple linear regression. Which was used?

Linear regression does not evaluate a significant difference. However, it establishes the equation of the relationship between variables.

The number of samples is not sufficient for multiple linear regression. It is more appropriate that the data be at least 5 times the number of variable.

If the model is to be established, the coefficient of determination, constant and regression coefficient of the models should be given.(Table 3).

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Luc DUREL

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

25 Sep 2019

Reviewer #1: Dear Editor,

Dear Authors,

First of all, I would like to thank you for the opportunity to review this manuscript on the Evaluation of four commercial tests for detecting ceftiofur in waste milk bulk tank samples. This article presents a primary scientific research in veterinary science of some interest in the current context of antimicrobial stewardship improvement.

To the best of my understanding, the objective of this study was to assess the reliability of commercial screening test kits in detecting ceftiofur residues from waste milk (WM). We agree that those tests have not been developed for this objective, but the control of bulk milk tank, supposedly free of a range of antimicrobials (inhibitors). Therefore, the most interesting feature of such tests is their negative predictive value, or their ability to tell us that the tested milk is free of contamination with a high level of confidence (in order to avoid technological and public health issue). Nevertheless, testing waste milk, when WM comes from cows at risk of having been treated with AB and because WM is massively used to feed newborn calves, is an acceptable stance. Thank you for investigating that.

This paper is well written in acceptable American English. The experiment is clearly presented, and the explanations are easy to follow. I support the publication of this article in PLOS One, provided that the authors respond to the following comments and consider revising certain points.

AU: The authors thank the reviewer’s efforts during the careful and through review of the manuscript. Comments and suggestions have helped increase clarity and relevance of the manuscript.

Major revision: Objective of this research (Abstract L21-22, Introduction L65-66)

This research is presented as an attempt to identify the ability of four commercial screening tests for detection of ceftiofur in WM. Since those tests have not been designed for this use, or the use on individual milk, the reader can ask himself about the real need of this research. So this paper is about test accuracy. The clearly exposed consequences of massive use of WM in newborn calves, and the early exposition to sub-therapeutic doses of antimicrobials suggest that it would be better to do not feed these animals with milk testing positive for inhibitors. The last sentence of the conclusion (L294-295) as well as the abstract and the introduction suggest that the objective of this research is to avoid the misclassification of healthy WM that can be discarded whereas it could be fed to calves. In case this objective exists in this research, it must be disclosed and the consequences of discarding FP WM explained. Alternatively, I suggest cutting these two sentences. Please clarify.

AU: the author’s goal for this study was to evaluate the accuracy of these tests for labeling a waste milk sample as positive or not for the presence of ceftiofur residues above a tolerance limit. On a more applied and practice side, the expected use of this information would be to identify positive samples for drug residues, and utilize the appropriate intervention to reduce the undesired effect of these antimicrobial residues, be it selection of resistance when feeding to calves, or environmental selection of resistance when discarding waste milk in the fields. Currently, to my knowledge, we do not have any low cost sustainable approach that is being used for processing waste milk samples to, for example, degrade antibiotics to concentrations below those expected to be harmful. Therefore the point made by the reviewer is valid, and addressing this argument, we have added a sentence to the discussion to clarify this point, and how waste milk containing drug residues is a biological waste that, even if not fed to calves could represent a potential source of environmental selection of antimicrobial resistance.

L 219-222- These alternative approaches for disposal of waste milk could also result in selection of antimicrobial resistance in these environments, and therefore more effective methods for removing the undesired effects of antimicrobial residues in waste milk before disposal would be beneficial.

Major revision: Methodology, detection limits, Se, Sp

Based on the proposed methodology (true negative samples – HPLC tested, true positive samples – spiked with ceftiofur 100ppb), it is indeed possible to calculate the actual Se and Sp of the four tests. However, in the real world of milk testing, things go differently. Tests have been designed to maximize the detection of a variety of inhibitory substances, practically of all kinds for Delvotest. Many substances can be detected at concentrations far below the applicable MRL. Therefore, Se is generally good, whereas Sp does not make sense really, or it makes sense at the class level (b-lactams). You duly provided the necessary information in Table 1, the so-called detection limits (for b-lactam drugs). I agree that milk-related factors can influence these limits, and I appreciate your work for that.

However, Table 1 raises two questions.

- References are acceptable but they should be discussed, even data referring to official FDA documents. Most of the information available for these screening are provided by manufacturers. Over time, the information regarding detection limits change. There are for example several version of the SNAP test, available in different regions, at the global level. I expect a short discussion of these figures.

AU: this is a very valid comment. The goal of the authors with the information in Table 1, was to address this point and have a prompt and clear resource for readers to review peculiarities of each test, especially concerning MRL’s. To address this point, the following paragraph with data from FDA documents has been added:

Ln 287-295 Currently the FDA has established that penicillin, ceftiofur, cloxacillin, cephapirin, amoxicillin, and ampicillin are the beta-lactam drugs most commonly used to treat disease in lactating dairy cattle, and it recommends the use of a test that has been show to detect at least four of the six beta-lactams be used [15]. Tolerances and safe levels for drug residues in for milk in the U.S. can be found in the 21 code of federal regulation (CFR) 556 [30]. When establishing the limits of detection for new test, the FDA has determined that for acceptance these test shall not detect drug residues at less than 50% of the tolerance level or 25% of the target testing level for individual drugs, with the exception of penicillin G and tetracycline drug tests [9, 31].

- The detection limit is not a totally clear concept. The detection capability (CCβ) and the decision limit (CCα) are two important performance characteristics of confirmatory methods for banned substances. For the EU we refer to Directive 2002/657/EC. Such a guidance does certainly exist for the US. I expect a short discussion of these statistical questions.

May I suggest you read a recently published paper on this matter? > doi:10.1136/vetreco-2018-000329

AU: that is a good point. Thanks for sharing the Council directive legislation and the manuscript. Both documents were inserted into the discussion of the manuscript and used to indicate that data form our study should be cautiously interpreted given regional differences for validating rapid test for detection of drug residues in milk.

Ln 306-320 Divergent from the U.S. FDA approach for determining acceptable accuracy limits for commercial test screening for drug residues in milk, the European Community utilizes the Council Directive 96/23/EC as a reference for validating approaches to measure and interpret results of residues in milk [32]. Geographical differences between similar commercial tests due to local regulations should be taken into account when interpreting the findings of our study. As an example, the Council Directive 96/23/EC document includes the use of different approaches for test validation, including the use of detection capability (CCβ), which is the smallest content of the substance that may be detected, identified and/or quantified in a sample with an error probability of β, and decision limit (CCα), which is the limit at and above which it can be concluded with an error probability of α that a sample is non-compliant. As outlined in a recently published study conducted in France assessing the risk of positive reaction to widely used rapid screening test for inhibitors in milk from cows treated with a new ceftiofur formulations, CCβ and CCα are not usually clearly indicated by manufacturers of rapid drug residues tests, and because of that the lowest detected concentrations was estimated to be an appropriate threshold for the interpretation of test results [33].

Minor revision: Figure 1

Figure 1 is poorly presented. It is supposed to present the core results of this study. Title and legend is required.

AU: There may be a miscommunication on how the figures are presented due to the format in which Plos One request figures be submitted. According to Plos One submission guidelines: “Figure captions are inserted immediately after the first paragraph in which the figure is cited. Figure files are uploaded separately.”

Therefore the figures title and legend are present in the main manuscript title and follow:

Figure 1. Mean value for fat (%), protein (%), solids-not-fat (%), somatic cell count (CFU/ml), standard plate count (CFU /ml), coliform counts (CFU /ml), and lactose (%) for false positive (FP) and true negative (TN) test results for the four commercial tests. Error bars correspond to 95% confidence interval. An asterisk represents test for which a significant difference was observed between FP and TN result.

We hope this clarifies the comments for this figure.

Typos

- L234: that (duplication)

AU: typo corrected

- L404: SuPPoting information

AU: typo corrected

Reviewer #2:

AU: The authors thank the reviewer’s efforts through review and suggestions and comments provided to improve the quality of the manuscript. We have carefully evaluated and addressed each point.

It is unclear whether the sample size represents the population? (power analysis should be done).

AU: information on sample size calculation was added to the manuscript.

L 86-90 Samples size calculation for detecting a difference between two means between false positive and true negative samples for all milk quality parameters was conducted using this function in JMP (SAS Institute Inc., Cary, NC). The parameters and values used for this calculation were based on our current dataset, and output of this analysis is displayed in S2 Table.

The statistical tests used should be explicitly given in the method.

AU: the statistical method section was thoroughly reviewed.

The normal distribution of the data can be determined by the Shapiro Wilk test. (if n<=30).especially in coliforms Count measurements, std error is higher than average, therefore non parametric methods should be used. normal distribution cannot be observed. (Mann Whitney U Test or median test can be used.)

AU: data was re-evaluated including the Shapiro-Wilk test as well as other assumption test. The data was re-analyzed using the Kruskal-Wallis nonparametric approach, and data is presented in the new manuscript version.

Some typos should be corrected. (In Table S1)

AU: table 1 was reviewed.

The statistical method described in the abstract and the narrative in the statistical method is not similar. Specially linear regression or multiple linear regression. Which was used?

AU: statistical method was reviewed and checked for consistency throughout the manuscript.

Linear regression does not evaluate a significant difference. However, it establishes the equation of the relationship between variables.

AU: as already mentioned, statistical method was reviewed

The number of samples is not sufficient for multiple linear regression. It is more appropriate that the data be at least 5 times the number of variable.

AU: as already mentioned, statistical method was reviewed

If the model is to be established, the coefficient of determination, constant and regression coefficient of the models should be given.(Table 3).

AU: as already mentioned, statistical method was reviewed

Submitted filename: Answer_to_reviewers_Comments.docx

Click here for additional data file.

24 Oct 2019

Evaluation of four commercial tests for detecting ceftiofur in waste milk bulk tank samples

PONE-D-19-20423R1

Dear Dr. Pereira,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Arda Yildirim, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Thank you for taking the time to completely revise the manuscript.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: No other comments. This work addressed a critical point of young calf rearing and led to practical recommendations. I fully support the publication of this manuscript.

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

5 Nov 2019

PONE-D-19-20423R1

Evaluation of four commercial tests for detecting ceftiofur in waste milk bulk tank samples

Dear Dr. Pereira:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Arda Yildirim

Academic Editor

PLOS ONE

Table 1

Milk drug residues screening test detection limits for Beta-lactam drugs.

Antimicrobial drug concentration at which greater than 90% of the replicates tested elicit a positive test response. Value for drugs are in parts per billion (ppb).

TESTS	Drug Concentration (in ppb) 1						References
	Penicillin G	Ampicillin	Amoxicillin	Cloxacillin	Ceftiofur	Cephapirin
Penzyme Milk Test	5	7	6	ND2	80	11.6	[13]
BetaStar Plus	4.7	5.2	5.5	8.2	80	19	[14]
SNAP β–lactam	3	5.8	7.3	ND2	12	11.7	[15]
Delvo SP-NT	2	4	4	20	100	10	[16]
FDA Tolerance/SafeLevels	5	10	10	10	100	20	[17]

1. Antimicrobial drug concentration at which greater than 90% of the replicates tested elicit a positive test response.

2.ND, drug not screened by the test referenced.

Table 3

Least square means value for samples with false positive (FP) and true negative (TN) test results for variables with a significantly effect on test results.

Only variables for which a significant difference was observed was included in the table.

	FP		TN		P-value3
Test/variables	LSM1	SE2	LSM1	SE2
SNAP4
Lactose (%)	4.26	0.06	4.52	0.09	0.037
SCC (Cells/ml)*	2,890,429	202,253	1,160,223	312,535	0.002
Delvo5
Fat (%)	4	0.3	5.5	0.4	0.004
Protein (%)	3.6	0.07	3.9	0.08	0.027
SNF (%)**	8.5	0.05	8.9	0.05	0.002
Coliforms (CFU/ml)	486	113.4	52.2	126.7	0.014
Penzyme6
SPC (CFU/ml)***	53,931	18,484	118,172	15,310	0.013
Betastar7
SCC (Cells/ml)*	3,032,500	302,032	1,854,000	270,146	0.027
Coliforms (CFU/ml)	265.0	143.3	315.8	128.2	0.026

1. Least square means for each variable found to be significantly different between FP and TN results.

2. Standard Error of the mean

3. P-value for Kruskal-Wallis nonparametric test evaluating a significant difference between FP and TN results for each variable.

4. Penzyme® Milk Test

5. Betastar® Plus

6. Delvo® - SP

7. SNAP® β–lactam

* Somatic cell count

** Solids-non-fat

*** Standard plate count