Considerations in evaluating equipment-free blood culture bottles: A short protocol for use in low-resource settings.

Use of equipment-free, "manual" blood cultures is still widespread in low-resource settings, as requirements for implementation of automated systems are often not met. Quality of manual blood culture bottles currently on the market, however, is usually unknown. An acceptable quality in terms of yield and speed of growth can be ensured by evaluating the bottles using simulated blood cultures. In these experiments, bottles from different systems are inoculated in parallel with blood and a known quantity of bacteria. Based on literature review and personal experiences, we propose a short and practical protocol for an efficient evaluation of manual blood culture bottles, aimed at research or reference laboratories in low-resource settings. Recommendations include: (1) practical equivalence of horse blood and human blood; (2) a diverse selection of 10 to 20 micro-organisms to be tested (both slow- and fast-growing reference organisms); (3) evaluation of both adult and pediatric bottle formulations and blood volumes; (4) a minimum sample size of 120 bottles per bottle type; (5) a formal assessment of usability. Different testing scenarios for increasing levels of reliability are provided, along with practical tools such as worksheets and surveys that can be used by laboratories wishing to evaluate manual blood culture bottles.

Introduction

Diagnosis of bloodstream infection (BSI) relies mainly on blood culture, i.e. the culture of large volumes of blood in blood culture bottles (BCB). In low-resource settings (LRS), mainly so-called “manual” blood culture systems are used, as opposed to the automated blood culture systems standardly used in high-resource settings. Manual BCBs are incubated in a conventional incubator, detection of growth is done based on visual signs of growth [1]. According to a market forecasting study, manual blood culture systems will make up roughly two-thirds of the global blood culture market by 2025 [2]. It is therefore surprising that almost no research has been done on manual blood culture systems since the 1990’s, when automated systems were introduced. For a district hospital laboratory in LRS, it is difficult to find scientific guidance when looking for the optimal BCB brand. The use of home-made BCB is widespread in LRS but should be discouraged due to lack of quality assurance of production methods and the omission of necessary additives [1, 3]. Import of commercial BCB into LRS countries is often expensive and logistically difficult. To address these problems, some LRS reference laboratories have started local, centralized production of blood culture media–an example is the Centralized Media Making Laboratory in Cambodia (https://dmdp.org/media-making-lab/). To ensure acceptable performance of such BCBs, and of newly marketed commercial manual BCBs, it is necessary to perform a validation study, in which these BCBs are preferably compared to the “gold standard” automated blood culture system.

BCB performance is evaluated using phase III clinical comparative studies, in which new BCBs are compared with the gold standard in patients requiring blood culture. However, randomization between blood culture systems requires hypothetic equivalence of both systems, and automated systems have long shown to be superior to manual blood culture systems in terms of yield and time-to-detection [1]. Another option is to inoculate half of the patient’s blood in each system, but this may also deprive the patient of an essential diagnosis, as larger volumes of blood increase the yield of blood cultures [4]. Sampling more blood from patients to mitigate this risk also raises ethical issues, especially in children or patients with low hemoglobin levels [5]. Another argument against clinical comparison studies is the long duration of this study type and the low yield of blood cultures; 85–95% of sampled blood cultures will not show growth, therefore clinical studies require very high sample sizes (typically 8.000 to 9.000) to detect a sufficient number of rare pathogens [6]. Laboratory studies, in which BCBs are spiked with blood and microorganisms causing BSI, can be a valuable alternative. These in vitro studies have no implications on patient management and are more efficient; the BCBs can be spiked with a variety of organisms, pre-defined by the researchers, guaranteeing sufficient numbers and inclusion of all desired pathogens.

In this manuscript, we list some “best practices” for simulated blood cultures, based on literature review and lessons learned from studies carried out at our research group from 2017 to 2020. We propose a practical protocol for a short and efficient validation of manual BCBs using simulated blood cultures and an automated “reference” system. This validation assures minimal performance of the BCB under evaluation, for settings where large studies are not feasible.

BCB performance evaluation study: Set-up, considerations and recommendations

Preparing a BCB performance evaluation study

Objectives of the BCB evaluation study

The main objective of the BCB evaluation study is to compare yield of a new manual BCB with an automated reference system. A secondary objective is to compare speed of growth between systems. Other secondary objectives are assessment of inter-lot variability and usability of BCBs. Some definitions used in this protocol are given in Table 1.

Table 1

Definitions of terms used in the protocol proposal.

Term used	Definition
Run	BCB inoculated in triplicate with the same strain simultaneously and with the same bacterial suspension. The same strain can be tested in different runs to assess inter-lot variability.
Growth	BCB showing signs of growth, either visual (e.g. turbidity, hemolysis or gas production) or as flagged by the automate, with confirmation of bacterial growth by subculture on agar (colonies of the inoculated organism) or by microscopy (bacteria visible on the Gram stain). Bottles showing signs of growth that cannot be confirmed by microscopy or subculture (48h incubation in the case of fastidious organisms and yeast), should be regarded as false-positive.
Yield	Percentage of inoculated bottles that show confirmed growth, as defined above
Speed of growth	Defined as time-to-detection in hours for automated systems, and for cumulative yield on day 1 and day 2 of incubation for manual bottles.
Day of incubation	A BCB is inoculated on day 0 of incubation. Day 1 of incubation is the day after first overnight incubation. Day 2 of incubation is after two nights of incubation.
Blind subculture	A subculture of the blood/broth mixture on an agar plate regardless of the presence of visual signs of growth

Abbreviation: BCB = blood culture bottle.

Acceptability criteria need to be determined before starting the experiments

These criteria will depend on practical considerations, such as price and availability of other BCB and expected patient population. We developed our own set of acceptability criteria for a BCB evaluation for the Mini-lab project [7] (Table 2). In addition, Dailey et al. formulated an extensive target product profile for a blood culture system in LRS [3]. The most important acceptability criteria are yield, speed of growth and ease of use of the BCB under evaluation.

Table 2

Criteria for blood culture bottle performance, as defined by the mini-lab project.

Weight of importance	Criteria
50%	The yield in the bottle tested should not be less than 90% of the yield in the reference method (automated blood culture system)
40%	Cumulative % of grown bottles should be at least 75% at day 1 of incubation and 90% at day 2 of incubation
10%	There should be no systematic failure in any of the non-fastidious species tested (systematic failure = growth in less than 75% of BCB, compared to reference method)

Three testing scenario’s: From minimal to optimal

We propose three “scenarios”, depending on how extensive the validation needs to be (Table 3). For a minimal scenario, 10 different bacterial species are tested (Table 4) with 1 strain per species. Both BCB types (BCB under evaluation and reference system) are inoculated in triplicate with the same blood-bacteria suspension (= one run). This is done once for adult formulations, once for pediatric formulations, with corresponding volumes of blood and representative bacterial concentrations. These runs are then repeated for a second lot of BCBs, to assess inter-lot variability. For the optimal scenario, 20 species are tested in total for three lots, for both adult and pediatric formulations. In the intermediate scenario, 20 species are tested for two lots. Table 3 gives an overview of number of BCBs and volume of blood needed for each of these scenarios. For pathogen species showing suboptimal growth in the BCB under evaluation (yield of less than 75%), we recommend testing additional strains of the same species; the extra number of BCBs and volume of blood needed in this case are also given in Table 3.

Table 3

Logistical and statistical implications of three scenarios of blood culture bottle (BCB) validation.

	Optimal scenario	Intermediate scenario	Minimal scenario
Number of strains tested	20	20	10
Number of lots tested	3	2	2
Total number of bottles needed per bottle type	360	240	120
Total number of bottles needed (all bottle types combined)	720	480	240
Total volume of blood needed	4320 ml	2880 ml	1440 ml
Number per bottle type adult	180	120	60
Number of bottle type pediatric	180	120	60
Detectable difference in yield *	5%	7%	10%
Detectable relative yield *	95%	93%	90%
95% confidence interval **	86–93%	85–93%	83–95%
Number of extra bottles needed per extra strain tested per bottle type	18	12	12
Volume of extra blood needed per extra strain tested	216	144	144

Calculation of detectable differences in yield is based on the normal approximation of the binomial distribution (https://www.stat.ubc.ca/~rollin/stats/ssize/b2.html); confidence intervals are calculated based on the binomial distribution (http://vassarstats.net/prop1.html).

* Compared to reference system, assuming 80% power, 95% confidence and 97% yield of the reference system

** Assuming an observed yield of BCB under evaluation of 90%; confidence interval becomes narrower when observed yield of BCB under evaluation is higher. Confidence intervals for proportions are not symmetrical due to binomial distribution; the uncertainty for these proportions is larger on the lower side of the interval than on the higher side.

Table 4

Proposal for species to be tested in a blood culture bottle performance validation study in low-resource settings.

Pathogen group	Species	Speed of growth
Enterobacterales	Escherichia coli	Fast
	Salmonella Typhimurium	Fast
	Klebsiella pneumoniae	Fast
	Salmonella Typhi*	Fast
	Enterobacter cloacae *	Fast
Non-fermenters	Pseudomonas aeruginosa	Intermediate
	Acinetobacter baumannii	Slow
	Burkholderia cepacia *	Slow
Staphylococcus	Staphylococcus aureus	Intermediate
	Staphylococcus epidermidis *	Slow
Streptococcus	Streptococcus pneumoniae	Fast
	Streptococcus pyogenes *	Fast
	Streptococcus anginosus *	Intermediate
	Streptococcus suis *	Intermediate
Fastidious organisms	Haemophilus influenzae	Slow
	Neisseria meningitidis	Slow
Enterococcus	Enterococcus faecalis *	Intermediate
Yeast	Candida albicans	Slow
	Cryptococcus neoformans *	Slow
	Candida tropicalis *	Slow

*only to be included in the “optimal scenario”

We performed basic sample size calculations to estimate the level of differences that can be detected with these testing scenarios (Table 3). For these sample size calculations, we assumed two-sided tests, confidence levels of 95%, power levels of 80%, and a yield of the reference system of 97% (based on unpublished data from our research group and yield of automated systems published in literature) [8-10]). An online calculator was used for the sample size calculation (https://www.stat.ubc.ca/~rollin/stats/ssize/b2.html) and the VassarStats website for calculation of the confidence interval (http://vassarstats.net/prop1.html). To detect a difference in yield of 10% between the BCB under evaluation and the reference system, corresponding to a relative yield of 90% compared to the reference system, a sample size in each group of at least 115 is needed. This would correspond to a “minimal” testing scenario (Table 3). Such a “minimal” scenario avoids unnecessary extensive validation of a BCB which has a clearly suboptimal performance (< 90% of the reference system). Some of the assumptions of these calculations will influence detectable differences. The higher the yield of the reference system, the higher the power to detect small differences. Stratifying results according to strain, adult versus pediatric formulation or lot number, on the other hand, will decrease power. Appropriate data analysis (taking into account the dependence of the data) may further increase power [11].

A negative control for each BCB type should be added to the test panel per run, to ensure sterility of the blood used and of the procedures. The negative control is a BCB filled with non-spiked blood from the same blood bag or lot of horse blood as used for the experiments.

Performance evaluation

The use of horse blood has logistical and ethical advantages compared to human blood

Most simulated BCB studies are done with human banked blood [10, 12, 13–21], because differences in plasma components and red blood cell enzymes between mammal species may affect bacterial growth. In addition, variations in visible signs of growth are possible due to inter-species differences in aggregation characteristics and fragility of red blood cells [22-24]. Despite these advantages, the large volumes of blood needed (> 4 liter, see Table 3) imply ethical and logistic difficulties. The availability of left-over blood in blood banks is limited and variable over time, jeopardizing efficient experiment planning. Left-over blood from multiple volunteers is pooled, compromizing standardization.

An alternative to human blood is defibrinated horse blood [8, 25–31]. Altun et al. claimed that using horse blood does not significantly alter the performance of blood culture systems [8]. However, no evaluations of equivalence between human blood and horse blood used for BCB evaluation have been published. Advantages of horse blood over human blood are availability, logistic ease and standardization, at least in those settings where commercial horse blood is available. To demonstrate equivalence of horse blood for use in BCB evaluation studies, we performed a head-to-head comparison of horse and human blood (S1 File). We demonstrated that horse blood can be used for BCB validation studies. Sheep blood is generally more available in LRS [32]. Few BCB validation studies using sheep blood have been published [33]. Before systematically using sheep blood, a validation study is needed. To ensure equivalent visual and growth characteristics, packed cell volume of both commercial horse and sheep blood should be similar to human blood (35–45%).

A diverse selection of clinically relevant micro-organisms should be tested

To evaluate whether all types of pathogens grow well in the BCB under evaluation, a broad selection of invasive pathogens should be used. Evidently, key BSI pathogens, depending on the setting, should be selected. We propose a list of essential organisms tailored to LRS in Table 4. This list includes all important pathogen “groups” and both fast- and slow-growing microorganisms [17, 34–38]. Fast-growing organisms have a median time-to-positivity in automated systems between approximately 9–15 hours, whereas slow-growing organisms have time-to-positivity > 18 hours. We only recommend testing of aerobic bacteria (and BCB), as anaerobic bacteremia is less relevant in LRS [1].

We advise using certified bacterial strains (e.g., American Type Culture Collection (ATCC)) instead of clinical strains, as these are representative members of their species. Reproducibility and standardization of the study results is therefore expected. However, basic growth characteristics of the strains used should be assured: some ATCC strains are known to grow poorly in certain culture media (e.g., Escherichia coli ATCC 8739, https://www.lgcstandards-atcc.org/products/all/8739.aspx?geo_country=be#culturemethod); the use of such strains is best avoided.

Dilution series can be used to achieve low bacterial concentrations

The concentration of micro-organisms in the blood of adult BSI patients is in the order of <1–10 colony forming units (CFU) per mL [39-42]. In clinical blood cultures, a BCB filled with 10 mL of blood contains on average 1–100 CFUs. This range should be respected in in vitro BCB evaluations. Indeed, most studies using simulated blood cultures add between 5 and 125 CFU per BCB [8, 12–16, 19–21, 29]. A low CFU concentration can be achieved by making serial dilutions from a 0.5 McFarland suspension containing approximately 1.5 x 108 CFU/mL [43], at least for Gram-negative bacteria such as E. coli. This is not true for all microorganisms: e.g. for Candida species, 0.5 McFarland only contains 1–5 x 106 CFU/mL [44]. Dilution series should take this into account [17]. Methods for dilution series are depicted in Fig 1.

Fig 1

Recommended methods for spiking blood culture bottles.

Calibrated pipettes must be used. Strains to be tested undergo two passages on blood or chocolate agar before the suspension is prepared. To confirm the concentration of the final bacterial suspension, 100 μl spread out on a blood agar plate for colony counts the next day (in triplicate). CFU = colony forming units.

Plate colony counts to check bacterial concentrations are common practice, but have a low precision due to sampling variation, especially at low concentrations [45-49]. The mean colony count follows the Poisson distribution, implying that the lower the concentration, the higher the standard deviation relative to the mean, and the less reliable the plate count will turn out to be. The same sampling variation applies to adding colonies to BCBs. Reliably counting (or adding) CFU numbers lower than 25 is challenging; this is considered the “limit of quantification” [45]. To mitigate the effect of the large total error on the estimated bacteria concentration of the suspension, it is generally advised to take the mean of three plate counts of the same suspension [45]. Triplicate plate counts are also advised as a quality check of the dilution series; in case of persistently lower or higher counts than expected, or in case of large variation between the three separate plate counts per dilution, the dilution procedure should be adapted. Plate counts are also useful in the phase of data analysis, as higher initial bacterial inocula could lead to faster growth and must be adjusted for.

Both adult and pediatric formulations should be tested

Bacterial concentrations in children with bacteremia are believed to be higher than in adults, but blood volumes added to the BCB are lower [50, 51]. Testing of both adult and pediatric blood volumes and bacterial concentrations is advised, as different blood-broth ratios can affect performance of the BCBs [52]. Moreover, many manufacturers have different BCBs for adult and pediatric blood cultures. Blood volume added to pediatric BCBs in simulated blood culture studies is usually 4–5 mL [16, 19–21]. This is the upper limit of recommended blood volumes in children, but is not entirely representative for pediatric populations, as sampled blood volumes typically vary between 0.5 mL (for neonates) and 4 mL (for older children) [51]. We therefore recommend using a volume of 2 mL. For adult formulations, a volume between 8 and 10 mL per BCB is often used [8, 12, 13, 16, 19–21, 29] and is indeed recommended in clinical practice [4].

Growth in manual BCBs should be visually assessed twice daily

Before use, each BCB should be checked visually for evident signs of contamination, such as broth turbidity or colonies on agar slant (for biphasic BCBs). After inoculation, the reference BCBs are incubated until the automate detects growth. Manual BCBs are inspected twice daily for visual signs of growth, such as turbidity, hemolysis and gas production [1] (Fig 2). To standardize this visual inspection, it is advisable to use a diffuse light source, such as a lightbox. The cycle of incubation/inspection continues until the broth (or agar slant) shows visual signs of growth, or up to 7 nights of incubation in case of no visible growth. For all manual BCBs, a blind subculture is performed on blood agar on day 1 (20–24 hours after start of incubation). A second subculture is done for manual BCBs with negative day 1 subculture, either upon visual growth (“positivity” subculture) or after 7 nights without visual growth (“terminal” subculture) to exclude growth (Fig 3). Autolysis of bacteria, leading to visual signs of growth but lack of colony growth on subculture, can be detected by performing Gram stain on such BCBs. A daily schedule of activities is presented in Table 5. Worksheet examples to be used during the study can be found in S2 File.

Fig 2

Visual signs of growth in manual blood culture bottles.

(A): pellicle formation on surface; (B) gas production; (C) turbidity (left bottle: no growth; right bottle: turbidity; (D) puff balls. (first published in Frontiers in Medicine (1), reproduced with permission).

Fig 3

Laboratory processing of spiked manual blood culture bottles.

The bottles are incubated on day 0. Blind subculture is performed on day 1, regardless of signs of growth. If blind subculture is negative, another subculture is done when growth is detected visually (in this example, day 3). If none of the subcultures shows growth, a terminal subculture is done on day 7.

Table 5

Daily schedule of activities during a week of BCB evaluation.

Day of incubation	Comment	Inspection/action
0	Day of spiking and incubation of the BCB	Preparation dilution series
		Spiking of blood
		Inoculation of BCB
		Incubation in automated and static incubator (preferably before 12:00 h)
1	After 1 overnight incubation	Morning: inspect manual BCB for signs of growth
		Perform blind subculture of manual bottles on blood agar
		Afternoon: inspect manual BCB for signs of growth
2	After 2x overnight incubation	Morning: inspect manual BCB for signs of growth
		Afternoon: inspect manual BCB for signs of growth
3	After 3x overnight incubation	Inspect for signs of growth
4	After 4x overnight incubation	Inspect for signs of growth
5	After 5x overnight incubation	Inspect for signs of growth
6	After 6x overnight incubation	Inspect for signs of growth
7	After 7x overnight incubation	Inspect for signs of growth
		If no signs of growth and no colonies on blind subculture done at day 1: perform terminal subculture on blood agar

Data analysis should take “clustering” on bacterial strain into account

Statistical methods to compare the yield and speed of growth of the manual BCB under evaluation with the reference system must take dependence of the data into account: the same bacterial suspension is used to inoculate three BCBs of both BCB types. Yield and speed of growth will be more similar among BCBs inoculated with the same suspension. The data is thus essentially “clustered” on the strain tested. Data analysis should take this into account, as this will affect confidence intervals and p-values for hypothesis tests; the direction of this change is dependent on the type of clustered data [11]. Several analysis approaches are possible; we refer to other papers for detailed information and examples [11, 53]. A practical and robust approach, in our opinion, would be to use a random effects logistic regression, as regression models have the ability to incorporate other variables as well [11].

Ease of use assessment of BCB

Ease of use assessment is often overlooked in the evaluation of in vitro diagnostics (IVD), despite its importance in succesfull implementation of diagnostics in remote settings and WHO recommendations for IVD development [54, 55]. Apart from good technical performance, an ideal BCB allows easy sampling, by assuring a strong vacuum, a septum that is not too rigorous and compatibility with adapters for use with butterfly needles. Visual evaluation of growth for manual BCBs is facilitated by transparency of the bottle material, transparent labels and visual aids such as growth indicators. BCBs should allow easy labelling, with sufficient space to provide all necessary information. Lastly, BCB material should be in line with biosafety requirements. This pertains for instance to the possibility to safely autoclave the material before transport for incineration and subsequent disposal, as some types of plastic cannot withstand high temperatures. For more information on ease of use, see S3 File.

Technical assessment of ease of use: A strong vacuum in the BCB facilitates sampling

The BCB vacuum allows for passive blood aspiration into the BCB. This is necessary when using butterfly needles, but will also facilitate sampling with syringe and needle as it obviates the need to put pressure on the syringe, an action that increases risk of needlestick injury and spills. Commonly used BCBs such as BacT/ALERT have a vacuum exceeding 10 mL, which enables easy sampling but entails a risk of overfilling [56]. To exactly measure the vacuum, specialized devices are needed, such as hydrostatic gauges or mercury column manometers [57]. However, this level of detail is not needed to assess ease of blood collection.

We propose a practical vacuum estimation based on passive aspiration of water, using 10 BCBs from at least 2 different lots. For the needle and syringe sampling method, vacuum is tested by connecting a 21-gauge needle to a 10 mL syringe filled with distilled water. When the needle perforates the BCB septum, the water is aspirated into the BCB (without applying pressure!). The volume of water still present in the syringe at the end of the procedure can be used to calculate the volume of water aspirated. For the butterfly needle sampling method, the hub and tube of a 21-gauge butterfly needle are put into a 100 mL measuring glass filled with distilled water. The septum of the BCB is pierced with the butterfly needle while the other end is kept stable in the measuring glass. After filling of the BCB by the vacuum, the difference between start and end volume of water in the measuring glass is measured and calculated. Using these test data, inter-lot and intra-lot coefficients of variation (CV) of BCB vacuum can be calculated, giving an indication of uniformity of production procedures.

Assessing ease of use by the end-user: Survey (S3 File)

In S3 File, we propose a survey for a qualitative assessment of ease of use. More information on essential aspects of ease of use can also be found in this Supporting information.

Limitation of the study design

General limitations of simulated blood cultures apply here; this in vitro approach does not constitute a “real-life” situation. Many important variables that influence growth of bacteria, such as concentration of bacteria and the presence of antibiotics or other microbiocidal factors in the plasma, may substantially differ from clinical conditions. For manual BCB, which are visually inspected daily, there is also an important observer bias related to simulated blood cultures; the laboratory technicians know that BCBs have been inoculated with microorganisms and this can influence their assessment of signs of growth. Blinded assessment is possible but implies inoculating a large number of BCBs with non-spiked blood as well; this would dramatically increase workload and volume of blood needed. Because of these limitations, results from simulated blood culture studies cannot be directly generalized to the field.

Most studies first add a bacterial suspension with a pre-defined number of CFU to the BCB and then add blood. We opted for another approach, adding the micro-organisms to the blood first and inoculating the blood into the BCB after homogenization of the blood-microorganism mixture on a roller-mixer. Introducing the bacterial suspension directly into the BCB requires inoculation of very small volumes (100 μL) with syringe and needle. This is difficult to do reliably, with the risk of considerably increasing variation in CFU between BCBs, especially as relatively low concentrations are used. Moreover, spiking the blood first creates a situation more similar to real-life conditions, as differences in CFU added between BCBs will depend on the sampling variation of the eventual concentration of bacteria in the spiked blood, and less on technical imprecisions during inoculation. However, this approach adds an additional dilution step with its associated additional variation.

Vacuum in BCBs is not static and typically decreases over time. When comparing the vacuum in BCBs from different lot numbers, it is therefore important to take the date of production into account and to compare only BCBs of approximately the same age (inferred from the expiration and production date). Moreover, many BCBs have a vacuum that aspirates more than 10 mL of water. The proposed method is only appropriate to assess whether vacuum is sufficient to easily aspirate blood, not to give reliable means for vacuums exceeding passive aspiration of 10 mL. Lastly, viscosity of water is lower than that of blood; therefore, the volume of water passively aspirated may be higher than the volume of blood passively aspirated.

Conclusion and future research possibilities

This report aims to give practical guidance to laboratories or research facilities that wish to evaluate the performance of (manual) BCB. We hope to encourage researchers and reference laboratories from LRS to validate BCB in their settings, to ensure acceptable quality of the BCBs and to add research on new manual BCB to the existing literature. There are no recent publications on performance of manual BCB, despite favorable market forecasts for these products [2]. To encourage clinical bacteriology at district laboratory level, quality assurance for consumables is of utmost importance [58]. Evaluation of ease of use of BCB and compatibility with common waste disposal practices such as autoclavation is seldom done, if at all. Nonetheless, these characteristics play an important role in successful implementation of blood cultures in LRS.

For many of the recommendations given above, empirical evidence from published studies was scarce. For example, we found no studies that compared horse blood with human blood for validation studies. Another knowledge gap is the effectiveness of dilution series to assure appropriate bacterial concentrations; we found plate colony counts resulting from dilution series to be widely divergent across bacterial species. Studies quantifying differences in bacterial concentrations between species would be welcomed to obtain more precise estimates of number of CFU added to the BCB, as would improvements in the methods for quantification of bacterial concentrations. More research on best practices for performing standardized and efficient validation studies is needed.

Report on head-to-head comparison of horse blood with human blood for blood culture bottle validation studies.

(DOCX)

Click here for additional data file.

Examples of worksheets to be used in blood culture bottle validation study.

(DOCX)

Click here for additional data file.

Ease of use assessment.

(DOCX)

Click here for additional data file.

18 Mar 2022

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Mehmet Demirci, PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

3. Please ensure that you include a title page within your main document. You should list all authors and all affiliations as per our author instructions and clearly indicate the corresponding author

Additional Editor Comments (if provided):

Comments from reviewer 1 (also found below):

The present manuscrpit proposes a short and practical protocol for an efficient evaluation of manual blood cultures bottles. It is well written and the protocol is very clear and detailed which makes it easy to use. Figures 1 and 3, which are well constructed, three "scenarios" as well as worksheets proposed in Supplementary Material 2 were much appreciated.

I have, however, some comments to the manuscript:

1- Table 1: "visual signs of growth" (turbidity, hemolysis and gas production line 191-192) should be detailled in Table 1 for a better understanding.

2- Table 1: Why make a methylene blue stain ? What is the added value compared to the Gram stain ?

3- Could Figures 1 and 2 be presented with better file resolution? Examples of visual signs of growth shown in figure 2 are not easy to see with this file resolution.

4- Figure 1 legend: Please explain why "strains to be tested undergo TWO passages on [...] agar before the suspension is prepared".

5- " In clinical blood cultures, a BCB [...] contains on average 1-100 CFUs. This range should be respected". Why does the method illustrated in Figure 1 recommand the use of 375 CFUs ? Is it comparable to a "real-life" situation ?

6- Line 189: "Before use, each BCB should be checked visually for evident signs of contamination". Is the visual control sufficient? Shouldn't subculture be performed before inoculation of the BCB to ensure their sterility?

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does the manuscript report a protocol which is of utility to the research community and adds value to the published literature?

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the protocol been described in sufficient detail?

Descriptions of methods and reagents contained in the step-by-step protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample sizes and replication needed to ensure that the data are robust and reproducible.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Does the protocol describe a validated method?

The manuscript must demonstrate that the protocol achieves its intended purpose: either by containing appropriate validation data, or referencing at least one original research article in which the protocol was used to generate data.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. If the manuscript contains new data, have the authors made this data fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the article presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please highlight any specific errors that need correcting in the box below.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The present manuscrpit proposes a short and practical protocol for an efficient evaluation of manual blood cultures bottles. It is well written and the protocol is very clear and detailed which makes it easy to use. Figures 1 and 3, which are well constructed, three "scenarios" as well as worksheets proposed in Supplementary Material 2 were much appreciated.

I have, however, some comments to the manuscript:

1- Table 1: "visual signs of growth" (turbidity, hemolysis and gas production line 191-192) should be detailled in Table 1 for a better understanding.

2- Table 1: Why make a methylene blue stain ? What is the added value compared to the Gram stain ?

3- Could Figures 1 and 2 be presented with better file resolution? Examples of visual signs of growth shown in figure 2 are not easy to see with this file resolution.

4- Figure 1 legend: Please explain why "strains to be tested undergo TWO passages on [...] agar before the suspension is prepared".

5- " In clinical blood cultures, a BCB [...] contains on average 1-100 CFUs. This range should be respected". Why does the method illustrated in Figure 1 recommand the use of 375 CFUs ? Is it comparable to a "real-life" situation ?

6- Line 189: "Before use, each BCB should be checked visually for evident signs of contamination". Is the visual control sufficient? Shouldn't subculture be performed before inoculation of the BCB to ensure their sterility?

Reviewer #2: This Lab protocol paper will be an useful guideline for blood culture. My only concern is about your previous publication (PMID: 31275940) which is a more comprehensive paper in this regard. I understand that the submitted paper focus is protocol but it has overlapping with your mentioned publication.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Farzam Vaziri

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

29 Mar 2022

Dear editors,

First of all, we would like to thank the editor and all reviewers for their time and interest in this manuscript, and for the useful suggestions.

We will respond to the comments raised by the reviewers one by one. We have kept the original comments in black and our responses are given in blue.

Comments for first revision, requested 28-03-22 by the editor and reviewers 18-03-22

Reviewer #1:

1- Table 1: "visual signs of growth" (turbidity, hemolysis and gas production line 191-192) should be detailed in Table 1 for a better understanding.

We have indeed added some lines on characteristics of visual signs of growth in Table 1 (line 81 in the revised manuscript).

2- Table 1: Why make a methylene blue stain ? What is the added value compared to the Gram stain ?

We have removed the mention of methylene blue stain in Table 1; this was a practice we did in our laboratory for a specific study but it is indeed not recommended in general for blood culture bottle validation studies.

3- Could Figures 1 and 2 be presented with better file resolution? Examples of visual signs of growth shown in figure 2 are not easy to see with this file resolution.

We have attempted optimal file resolution in our re-submission.

4- Figure 1 legend: Please explain why "strains to be tested undergo TWO passages on [...] agar before the suspension is prepared".

This was done in line with our laboratory procedures for subculturing of quality control strains, to ensure optimal viability and purity of the strain. Sparse growth after subculture from frozen strains and possible contamination by other strains can be overcome by subculturing twice. This procedure was based on recommendations by the Clinical Microbiology Procedures Handbook (Leber et al, 2016) for using frozen cultures as quality control for antibiotic susceptibility testing. We could not find in literature or handbooks whether this step is needed for validation of blood cultures bottles. We can therefore not make firm recommendations but only explained how it was done in our laboratory.

5- " In clinical blood cultures, a BCB [...] contains on average 1-100 CFUs. This range should be respected". Why does the method illustrated in Figure 1 recommend the use of 375 CFUs ? Is it comparable to a "real-life" situation ?

From the suspension of 375 CFU/mL, 1 ml (=375 CFU) is added to 25 mL (pediatric) or 50 mL (adult) volume of blood. Then this volume of blood is further subdivided over the bottles at a volume of 10 mL (adult) or 2 mL (pediatric) per bottle. Therefore, the eventual number of CFU per bottle is 375/50*10 for adults and 375/25*2 for children. This comes down to 75 CFU in total for an adult bottle (concentration 7.5 CFU/mL) and 30 CFU for a child (concentration 15 CFU/mL). Both of these quantities are in the range of 1-100 CFU per bottle.

6- Line 189: "Before use, each BCB should be checked visually for evident signs of contamination". Is the visual control sufficient? Shouldn't subculture be performed before inoculation of the BCB to ensure their sterility?

Sterility control of all blood culture bottles would be time-consuming and labor-intensive and risks actually contaminating the bottle in the process (although that risk is admittedly low). When the bottles are incubated and show growth, subculture is done and the colony aspect and Gram stain are used to ensure the grown micro-organisms are consistent with the spiked pathogens.

In a previous version of this manuscript, we had advised to perform a sterility control on a subset of each lot of bottles to be evaluated by incubating them for 7 days and then performing a subculture of all of these bottles; however, at an expected contamination rate of < 5% of all bottles, the number needed to reliably rule out contamination would be too high to be feasible.

Reviewer #2: This Lab protocol paper will be an useful guideline for blood culture. My only concern is about your previous publication (PMID: 31275940) which is a more comprehensive paper in this regard. I understand that the submitted paper focus is protocol but it has overlapping with your mentioned publication.

The paper you mention is a general review on use of blood cultures, whereas this manuscript specifically looks at validation of (manual) blood culture bottles to ensure their performance. With regards to some aspects (explanation of bacterial concentration, workflow, visual signs of growth), we agree there is some overlap, however we believe the aim and content of both papers is sufficiently different to justify separate publications.

Kind regards,

Liselotte

11 Apr 2022

Considerations in evaluating equipment-free blood culture bottles: a short protocol for use in low-resource settings

PONE-D-21-35594R1

Dear Dr. Hardy ,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Assoc. Prof. Mehmet Demirci, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

13 Apr 2022

PONE-D-21-35594R1

Considerations in evaluating equipment-free blood culture bottles: a short protocol for use in low-resource settings

Dear Dr. Hardy:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Assoc. Prof. Mehmet Demirci

Academic Editor

PLOS ONE