Literature DB >> 35137133

Honey Bees (Hymenoptera: Apidae) Decrease Foraging But Not Recruitment After Neonicotinoid Exposure.

Bradley D Ohlinger1, Roger Schürch1, Sharif Durzi1,2, Parry M Kietzman1,3, Mary R Silliman1, Margaret J Couvillon1.   

Abstract

Honey bees (Linnaeus, Hymenoptera: Apidae) are widely used as commercial pollinators and commonly forage in agricultural and urban landscapes containing neonicotinoid-treated plants. Previous research has demonstrated that honey bees display adverse behavioral and cognitive effects after treatment with sublethal doses of neonicotinoids. In laboratory studies, honey bees simultaneously increase their proportional intake of neonicotinoid-treated solutions and decrease their total solution consumption to some concentrations of certain neonicotinoids. These findings suggest that neonicotinoids might elicit a suboptimal response in honey bees, in which they forage preferentially on foods containing pesticides, effectively increasing their exposure, while also decreasing their total food intake; however, behavioral responses in semifield and field conditions are less understood. Here we conducted a feeder experiment with freely flying bees to determine the effects of a sublethal, field-realistic concentration of imidacloprid (IMD) on the foraging and recruitment behaviors of honey bees visiting either a control feeder containing a sucrose solution or a treatment feeder containing the same sucrose solution with IMD. We report that IMD-treated honey bees foraged less frequently (-28%) and persistently (-66%) than control foragers. Recruitment behaviors (dance frequency and dance propensity) also decreased with IMD, but nonsignificantly. Our results suggest that neonicotinoids inhibit honey bee foraging, which could potentially decrease food intake and adversely affect colony health.
© The Author(s) 2022. Published by Oxford University Press on behalf of Entomological Society of America.

Entities:  

Keywords:  communication; foraging; imidacloprid; neonicotinoid; waggle dance

Mesh:

Substances:

Year:  2022        PMID: 35137133      PMCID: PMC8826047          DOI: 10.1093/jisesa/ieab095

Source DB:  PubMed          Journal:  J Insect Sci        ISSN: 1536-2442            Impact factor:   1.857


In any agricultural system, pest management approaches, such as chemical control methods, can adversely affect nontarget organisms. Neonicotinoids are widely used to control agricultural and household pests and were responsible for 25% of the total global insecticide market in 2012 (Bass et al. 2015). However, recent reports demonstrating the adverse nontarget effects of neonicotinoids have galvanized efforts to reduce their use, with the European Union opting to ban neonicotinoids in open field crops in 2018 (Jactel et al. 2019). Such insecticides are highly selective agonists of insect nicotinic acetylcholine receptors, causing overexcitation of the neurons associated with learning and memory (Palmer et al. 2013). Neonicotinoids are applied via various methods such as seed treatments (Krupke et al. 2012, Huseth and Groves 2014), foliar sprays (Kumar et al. 2012), stem application (Kumar et al. 2012), and soil application (Huseth and Groves 2014). Once applied, neonicotinoids are absorbed into the treated plant and then transported throughout its tissues (Bonmatin et al. 2015), where they protect against damage from chewing and sucking insects (Kumar et al. 2012). Neonicotinoids also migrate to the nectar and pollen of flowering plants, where they can then be ingested by beneficial, nontarget, insects such as pollinators (Krupke et al. 2012, Long and Krupke 2016). Pollinators are exposed to pesticides while visiting treated crops (Pohorecka et al. 2012, Byrne et al. 2014). Additionally, plants, like wildflowers, growing near treated crops often contain neonicotinoids and provide another route of exposure for pollinating insects (Krupke et al. 2012, Botías et al. 2016, David et al. 2016, Long and Krupke 2016). Overall, the nontarget exposure of beneficial insects to pesticides has recently gained the attention of both researchers, who have reported adverse effects on several pollinator taxa (Sandrock et al. 2014, Tan et al. 2015, Arce et al. 2018, Mustard et al. 2020), and the wider public, who are increasingly concerned about the well-documented declines in pollinators (Biesmeijer et al. 2006, Ellis et al. 2010, Potts et al. 2010). Research into the effects of neonicotinoids on pollinators has often focused on honey bees because of their importance to agriculture (Klein et al. 2007, Aizen and Harder 2009) and their tractability as a research organism (Thompson and Pamminger 2019). Honey bees are the most widely managed, versatile, and economically valuable pollinating insects (Klein et al. 2007). Because of their critical use as commercial pollinators (Klein et al. 2007), honey bees are also closely associated with agricultural crops, and they commonly experience chronic exposure to sublethal doses of neonicotinoids (Tsvetkov et al. 2017). The types and severity of effects reported depend on several factors, which we have grouped into extrinsic and behavioral factors. Important extrinsic factors include the duration of exposure (chronic versus acute; (Tosi et al. 2017a)), type of exposure (oral versus topical; (Aliouane et al. 2009)), the dosage (Yang et al. 2008), and co-exposure to other stressors such as poor nutrition (Tosi et al. 2017b) and pathogens (Doublet et al. 2015). Exposure to neonicotinoids can lead to various adverse effects on honey bee behavior (Yang et al. 2008, Eiri and Nieh 2012, Henry et al. 2012, Schneider et al. 2012, Fischer et al. 2014) and cognition (Aliouane et al. 2009, Tan et al. 2015, Wright et al. 2015, Andrione et al. 2016, Mustard et al. 2020). For example, honey bees treated with pesticides take longer to learn olfactory cues (Wright et al. 2015), are less able to discriminate learned odors from novel odors (Mustard et al. 2020) and retain information related to olfactory cues for shorter periods of time (Tan et al. 2015, Wright et al. 2015). Neonicotinoids have also been reported to impair navigational abilities (Fischer et al. 2014, Tison et al. 2016) and to increase rates of homing failure (Henry et al. 2012, Tison et al. 2016). As a result, honey bees treated with pesticides require more time to complete foraging trips (Yang et al. 2008, Schneider et al. 2012), forage less frequently (Yang et al. 2008, Schneider et al. 2012, Tison et al. 2016, Tison et al. 2020) and perform fewer waggle dances (Eiri and Nieh 2012, Tison et al. 2016, Tison et al. 2020), leading to possible colony-level effects (Henry et al. 2012). Additionally, behavioral factors of the honey bees themselves, especially in the field, also play a critical role in determining the level of exposure and the severity of the resultant adverse effects. For example, (1) honey bees could forage preferentially and suboptimally on food sources containing neonicotinoids (Kessler et al. 2015), leading to high exposure, by increasing the rate at which they visit and recruit foragers to neonicotinoid-treated food sources. (2) Honey bees could also forage indiscriminately and neutrally on foods containing neonicotinoids versus those not containing neonicotinoids. In this case, exposure to pesticides is determined by the relative availability and quality of the accessible plants containing versus those not containing pesticides. Finally, (3) honey bees could forage optimally by actively avoiding food-containing pesticides, leading to lower levels of exposure. Given the range of possible responses to neonicotinoids, research looking at the foraging choices of bees between foods containing neonicotinoids and foods not containing neonicotinoids is necessary to more accurately assess exposure and potential risk for honey bees. Lab studies report that both honey bees and bumblebees might actually prefer sucrose solutions containing field-realistic concentrations of neonicotinoids over control solutions (Kessler et al. 2015, Arce et al. 2018). Additionally, despite consuming more of the neonicotinoid-laced solution, neonicotinoid-treated honey bees and bumblebees consumed overall less total solution to some concentrations of certain neonicotinoids (Kessler et al. 2015). These results indicate that particular doses of neonicotinoids elicit preferential feeding, while also inhibiting total foraging activity. Such results are reflected in semifield studies showing that single doses of neonicotinoid decrease foraging motivation in bumblebees (Lämsä et al. 2018) and sucrose responsiveness in honey bees (Aliouane et al. 2009, Eiri and Nieh 2012, Démares et al. 2016, Démares et al. 2018, Jiang et al. 2018) and field studies showing decreased dancing for natural forage (Tison et al. 2016). Taken together, the above effects could potentially lead to increased pesticide exposure and decreased food intake, a foraging scenario that could harm honey bee colonies. Although lab-based assays provide important opportunities to measure responses in a controlled treatment, semifield and field studies are necessary because ecological consequences of treatment might not be immediately obvious in the lab. Additionally, lab-based studies fail to incorporate the complexities of honey bee foraging and provide only limited insight into the responses of honey bees foraging in the field. Instead, semifield studies, such as feeder experiments, include additional factors that better simulate a typical honey bee foraging scenario (Couvillon et al. 2015). Most semifield studies looked at the effects of acute exposure on individual foraging and recruitment behaviors (Yang et al. 2008, Eiri and Nieh 2012, Schneider et al. 2012), with the effects of chronic exposure being less well-studied (Tison et al. 2016, Tison et al. 2020). Therefore, additional investigation into how chronic exposure to field-realistic doses of neonicotinoids affects the foraging choices of honey bees in the field is needed. In this study, we conducted a feeder experiment with freely flying bees to test for the effect of a sublethal, field-realistic, concentration of the neonicotinoid pesticide, imidacloprid (IMD), on foraging and recruitment behaviors. In doing so, we investigated the individual and colony-level behavioral responses to neonicotinoids in a semifield context.

Materials and Methods

Study Organism

We studied sequentially seven queenright honey bee colonies of mixed European race, predominantly Apis mellifera ligustica, each with brood and approximately 5,000 workers. We studied three colonies in the summer of 2018 and four colonies in the summer of 2019. We housed the colonies in Plexiglas-walled observation hives at the bee field laboratory at the Prices Fork Research Center in Blacksburg, Virginia. The hives were each comprised of three American Standard Deep frames and were connected to the outside via a c. 5 cm × 30 cm plastic tube. We worked with one observation hive at a time (2018: Hive A, B, then C; 2019: Hive D, E, F, then G). Colonies were managed throughout the duration of the project to prevent overcrowding and to standardize the number of empty cells. It is important for there to be space available for additional nectar storage. If not, returning foragers could experience a longer wait time in unloading the sugar solution, which decreases their propensity to recruit via the waggle dance (Seeley 1989, Seeley 1995). Data were collected from 26 July 2018 to 12 August 2018 and 15 July 2019 to 26 August 2019 on days with good foraging weather. We chose to work in high summer because it is easier to train and to recruit bees to feeders when there is a relative forage dearth in the landscape, which we had heard occurs in late July and August in the study location (R.D. Fell and J.M. Wilson, personal communication). Colonies were not given supplemental food for the duration of the experiment.

Training Forager Bees to Feeders and Imidacloprid Treatment

Worker honey bees from the observation hives in our study were trained using one of two methods: a ‘box and jump’ (2018) and a step-wise (2019) method. In 2018, foragers were trained using standard procedures (‘box and jump’), which has been described extensively in the literature (Al Toufailia et al. 2013, Schürch et al. 2013, Couvillon et al. 2015, Schürch et al. 2019)). Our bee lab is adjacent to an apiary, so we encountered an unexpected challenge of accidentally recruited bees from colonies other than our experimental colonies during the 2018 field season. Therefore, for the second year (2019), we modified our methodology and used step-wise training on the day prior to the experiment (Day-1). In this alternative, we began by training foragers to visit a feeder containing 2 M scented (10 µl/L, with lavender in Trial 4, lemongrass in Trial 5, peppermint in Trial 6, and linalool in Trial 7) sucrose solution syrup. The feeders were placed on a tripod (1 m in height) that was positioned approximately 1 m away from the hive entrance tube. We then stepped (10–12 times, approximately 10–15 m between steps) the feeder across an adjacent open field to a location 5 m in front of and center to the eventual position of the two experimental feeders (“north” and “south”). The feeder was placed at each successive position until every forager demonstrated that they had learned the current position by visiting the feeder at least three times. Training was conducted by two researchers, one at the feeder and one at the observation hive. The researcher at the feeder marked visiting foragers with numbered plastic discs (Opalithplättchen, Christian Graze KG, Weinstadt-Endersbach, Germany), while the other researcher confirmed whether the marked foragers returned to the colony. To avoid including honey bees from a nearby apiary, we removed the marked foragers that returned to the feeder but failed to return to the observation hive. In each trial, 5–15 foragers were confirmed at the observation hive with this method. We then left the feeder at the final location until at least ten confirmed foragers visited it at least ten times, which usually took until mid or late afternoon on Day-1. The ten visits provided a highly rewarding experience for the foraging bees, making them likely to return to the same feeder the following day (Day 0; (Al Toufailia et al. 2013)). On the following morning (Day 0), we placed two feeders containing a 2 M scented sucrose solution syrup, identical to the solution used on Day-1, side by side on a tripod at the final training location from Day-1. We allowed bees trained from Day-1 to visit the two feeders freely. These foragers were considered ‘committed’ after visiting one of the two feeders at least five times. When several (3–5) committed foragers were observed simultaneously drinking, we carefully moved the feeders, along with the drinking bees, to the experimental feeder locations. One feeder was placed on the “north” tripod and the other was placed on the “south” tripod, and we randomly assigned the color cue background (yellow or blue, alternating between trials). The Day-1 training bees were then allowed to forage and recruit to the experimental feeders freely: their purpose was not to be part of our experimental cohort of bees, but rather to recruit the bees that then would be our experimental bees. Newly recruited foragers (Day 0 foragers) were marked with unique numbered plastic discs at the feeders and their membership to the experimental colony was confirmed at the observation hive. The color of the plastic discs on each Day 0 forager indicated the color of the feeder that they visited and also distinguished them from the Day-1 training foragers. Each visit by the Day 0 foragers was recorded as a Day 0 training visit. Once several Day 0 foragers had begun to visit and recruit consistently, we removed the Day-1 foragers from the experiment. The two training methods (2018 versus 2019) differed in that the step-wise method required an additional day of training prior to the experiment (Day-1) of bees that were not included in the experiment, while the “box and jump” method only required training on the day of the experiment (Day 0). Importantly, the Day-1 addition did not affect the experiences of the Day 0 foragers, which are the bees included in the experiment. Our goal was to train 10–20 marked individuals to return reliably to their designated tripod (“north” versus “south”), by 11:30–13:30, so that we would have sufficient time to complete the treatment phase. However, in several trials, we were unable to recruit the target number in time (see discussion) and instead began the treatment with a smaller number of bees. For the treatment phase, we removed the two identical feeders containing 2 M scented syrup and replaced them with a new feeder containing 1M unscented solution, either with (treatment) or without (control) IMD (Sigma Aldrich, Reagent Powder). We used a concentration of IMD (100 nM, about 26 ppb) that had previously been shown to generate behavioral effects in the lab (Kessler et al. 2015). In particular, honey bees preferred neonicotinoid laced solutions over control solutions in a two-choice test, with the strongest preference being to solutions containing 26 ppb IMD. Importantly, this concentration has been detected in the nectar of treated plants, such as citrus trees, which are valuable honey-producing plants (Byrne et al. 2014), while higher concentrations (60–80 ppb) have been reported in bee-collected pollen from IMD treated cucurbits (Dively and Kamel 2012). Although returning foragers displayed some initial difficulty in orienting towards the new, unscented feeders, they soon began collecting the 1 M solution. We chose 1 M because we knew, at this time of year for our study location (Schürch et al. 2019) that it would cause good, but not maximum, foraging and recruitment (Seeley 1995), which is important if there is to be an observable effect of treatment on behavior.

Confounding Variables of Incorrect Feeder Visits on Day 0

Initially, we planned to conduct the entire experiment only in 2018. However, our preliminary analysis of the experiment revealed that some foragers did not forage exclusively on their assigned feeder, either treatment or control, and were therefore exposed to both the IMD solution and the control solution. Specifically, bees made 15.08% of their total foraging visits to the alternative (incorrect) feeder during the 3 h experimental period on Day 0. This behavioral response was unexpected and would therefore alter the amount and type of exposure to IMD that the treatment bees would experience. We decided to rerun the experiment in 2019, with some adjustments aimed to prevent feeder swapping and therefore cross-exposure to the IMD and control solutions: in 2019, we increased the distance between the treatment and control feeders (interfeeder distance) from two meters to five meters, while still keeping the total distance from the observation hive equal. Despite these adjustments to the experimental design, 7.53% of the visits made by bees during 2019 were still to the alternative (incorrect) feeder (see below for handling issues). To determine whether the IMD treatment contributed to this unexpected switching behavior, we first assigned the treatment groups according to their trained feeders. The trained feeders were designated as the last feeder that each forager visited during the training phase. We decided on this criterion because of the honey bee’s well-documented tendency towards constant foraging, which makes them more likely to visit the same feeder on successive visits than to switch between two different feeders on successive visits (Hill et al. 1997, Grüter et al. 2011). We compared the proportion of total visits that IMD foragers made to their trained feeder (commitment to training feeder) and found that IMD did not affect commitment to the training feeder (mean relative odds: 0.71, 95% CI: [0.33,1.49]). Despite occurring randomly, the switching behavior is a factor that determines the level exposure of individual bees to the treatment and control. We decided to analyze the bees that did not switch at all separately (from here on, the “consistent foragers”) because they lacked cross-exposure to the treatment and control solutions and therefore received the treatments as they were described in our initial experimental design. Additionally, the lack of effect of treatment on commitment to the training feeder suggests that the consistent foragers represent a random sample, which is helpful for the experiment because it limits the potential for bias in the subsample.

Data Collection—Foraging Frequency, Dance Propensity, Dance Frequency, Waggle Run Frequency, Persistency, Site Specificity

Honey bees are exceptionally sensitive to reward, particularly molarity, and will modulate their behaviors accordingly (von Frisch 1967, Seeley 1995, Couvillon 2012, Couvillon et al. 2015). Therefore, foraging and recruitment reflect how profitable an individual bee has assessed resource quality to be (von Frisch 1967). Of course, the two feeders during the experiment phase possessed identical profitability (1M) and energetic cost (equidistant from the hive), so any differences in the measured behaviors between the treatment and control would indicate an effect of IMD treatment. Once the treatment and control feeders were in place, we counted how many experimental visits an individual bee made to her 1M feeder (either with [treatment] or without [control] IMD) during the 3 h experimental period (foraging frequency). For a visit to be recorded, we used a consistent set of criteria. First, the marked bee must land on the feeder, extend its tongue and drink the solution. Second, visits by individual bees were only counted if three minutes had passed from the previous visit. The second criterion was implemented because foraging bees may visit the feeder without returning to the hive to unload food provisions. Previous studies with similar methods estimated that highly motivated foragers need at least three minutes to travel from the feeder to the colony (145 m) and back to the feeder again (Couvillon et al. 2015). Concurrent to the recording of foraging frequency, we video-recorded waggle dances made by the marked bees within the observation hive during the 3 hr experimental phase. At the start of each experimental phase, a researcher began recording waggle dances with two Canon Vixia HF R82 video cameras, each positioned on either side of the glass-walled observation hive. The researcher then focused the view of the camera on the entire area of the frames where waggle dances were observed (typically the bottom two frames of the colony) and adjusted the view of the camera when necessary to ensure that all waggle dances were visible in the video recordings. The videos were recorded to a SanDisk Extreme SD card and then later uploaded to a Google Team Drive (GTD) for analysis. We then converted video files filmed at 30 frames per second to AVI using Ubuntu (v. 2004.2021.222.0) and imported them into ImageJ (version 1.52i) for visual analysis. The waggle dance is a unique behavior in which a returning bee who is foraging at a profitable food source communicates the distance and direction from the hive to the feeding location (von Frisch 1967, Couvillon 2012). Honey bees, with their exceptional sensitivity to reward quality, are more likely to dance and will dance more if the forager is visiting a highly profitable resource (von Frisch 1967, Seeley 1995, Couvillon et al. 2015). For example, a forager may go back and forth between the hive and the forage site to collect a low or medium valued sucrose solution and still not dance, but she is more likely to begin to dance if the solution is replaced with a higher valued reward (von Frisch 1967, Seeley 1995). Therefore, we monitored the videos for several metrics of the dance. Firstly, we determined the proportion of successful foragers that make any dances at all during the entire 3 h experimental phase (dance propensity) compared to the foragers that do not dance. Secondly, once a bee had made a dance, we monitored how often she then repeats the dance (i.e. the number of return trips to the hive that she dances, or dance frequency). Finally, because the waggle dance consists of multiple repeated circuits, each containing an information-rich waggle run phase + return phase, we recorded the number of waggle runs that the experimental bees performed per dance. Each of these three behaviors (dance propensity, waggle dance frequency, and waggle run frequency) are expected to increase with a honey bees’ perception of food quality and/or adjusted by foraging cost (distance; (Seeley 1994, Seeley 1995, Seeley et al. 2000)). For our experiment, quality and cost were equal, so we monitored these responses to determine the effect of IMD. We allowed the foragers to continue to visit their feeders for the entire 3 h experimental period, which ended between 14:30 and 16:30. We then removed the tripod, umbrellas, and feeders, while carefully noting exactly where each tripod was positioned, so the experimental apparatus could be reassembled in the same location on the following morning for the next phase of the experiment that began on Day 1.

Data Collection—Persistency and Site Specificity

Previous research had shown that bees are more persistent (i.e., returning on Day 1 to check if the feeder has become re-rewarding) to a newly unrewarding feeding location if the location had previously been highly rewarding (Al Toufailia et al. 2013), making persistency another honey bee behavior that correlates with the resource quality. Therefore, to determine if there was an effect of IMD on persistency, we set up the tripods and two empty, scent-free feeders on Day 1 at the exact location (“north” versus “south”) as Day 0. We then monitored them for 1–2 d with observation periods coinciding with the start and end of Day 0 (≈ 9:00–16:30). We aimed to monitor persistency for at least 2 d because our previous research had indicated that these days were the most interesting and informative for these variables (Couvillon et al. 2015). Bad weather shortened our observation period to one day for Trial 4 in Year 2. Persistency was monitored by noting bee number and time of landing on the unrewarding feeder on Day 1–2. Our criterion for counting a persistency visit was that a marked bee should contact the feeder. Simply landing on the tripod or flying near the feeder did not count. Additionally, we surveyed the colonies on the mornings of Days 1 and 2 to assess mortality. The surveys revealed no significant effect of IMD on mortality on either day (Day 1, mean relative odds = 1.36, 95% CI [0.52,3.49]; Day 2, mean relative odds = 1.11,95% CI [0.47,2.69]). During the days following the completion of each trial, we installed a new observation hive at our field site, which we used to house a new experimental colony. We then began the subsequent trial by repeating the training protocol with the new colony. A bee, upon discovering that the feeder to which she had been trained was now empty, would sometimes remain in the area to investigate the other feeder (e.g., not the tripod/tripod color/feeder where that bee was trained on Day 0). We noted the time and bee ID of these visits to the “incorrect” feeder, using the same criterion as for a persistency visit. Visiting the opposite feeder demonstrated low site-specificity (the degree to which honey bees continue to visit a reference feeder), whereas only ever persistently investigating one’s own trained feeder shows high site-specificity. We used a different observation hive and training scent for all seven trials. Changing scent reduced the likelihood of attracting robbers from a previously-trained observation hive. Lastly, for each trial, we swapped both tripod location (north vs. south) and color of treatment tripod (blue vs. yellow). We tested only three of the four possible 2 x 2 combinations in the 2018 field season. Ideally, we would have continued onto an additional trial, but the blooming of goldenrod, a highly attractive resource for foraging honey bees, made training impossible during the Year 1 field season.

Statistical Analysis

The two years’ experiments tested the same hypotheses and we increased the distance between the feeders in 2019. However, because we added the second year after the fact, knowing already the broad strokes of the first year’s data that we analyzed with Maximum Likelihood Models, we then decided that a Bayesian statistical approach would be most appropriate for our analysis. By using Bayesian models, we were able to generate statistical inferences according to our experimental observations from both years, without pooling the data produced by two slightly different experimental designs. The analyses from the two different statistical paradigms (Maximum Likelihood and Bayesian) yielded similar results. We will present here the results of the (more philosophically correct) Bayesian models, and the Maximum Likelihood Models are available upon request. The data sets from the two years (2018 + 2019) were assessed together using a Bayesian updating approach, in which the mean parameter values derived from the models of the data from 2018 were used as priors for the models of the data from 2019. Lastly, to address the unexpected issue of foraging bees incorrectly visiting the other feeder (i.e., treatment bees going to control and control bees going to treatment), we decided to include an analysis, which we will call per-protocol, that looked just at “consistent” honey bee foragers, those that visited their correct, trained feeder (either treatment or control) 100% of the time during the 3 h experimental phase (analysis that included all bees, even those that swapped, is referred to as supplementary analysis, see below). We also excluded the data from hive E in the per-protocol analysis: we had begun the experiment the day before and had to stop the trial prematurely. Even though we removed the bees before beginning again, we later considered how the solution collected was likely distributed throughout the colony via unloader bees (Seeley 1989, Seeley 1995) and dance followers (B.D. Ohlinger, personal observation), who might have then visited the feeder on the following day. We report the results of the per-protocol analysis (Control: n = 52, IMD: n = 62) in the main text because those foragers collected either control or treatment under the target experiment conditions (i.e. 100% of the time), while the results of the supplementary analysis (Control: n = 71, IMD: n = 114) can be found in the Supp Material (online only). All data analysis was done in R 3.6.3 (R Core Team 2020). To determine the effect of IMD on the mean number of experimental visits per bee to the feeders during the 3 h experimental window (foraging frequency), we used Poisson Generalized Linear Mixed Models (GLMMs) with log-link, and with treatment (Control solution versus IMD solution) as a fixed effect and random intercept for hive and for individual bees. Additionally, we used binomial GLMMs with logit-link to model treatment as a fixed effect and with random intercepts for hive and for individual bees to determine the effect of IMD on commitment to the training feeder during day 0. To determine the effect of IMD on recruitment, we first analyzed dance propensity with binomial GLMMs (logit-link) on the proportions of dancing to nondancing bees per treatment with treatment as a fixed effect and a random intercept for hive. Then we used Poisson GLMMs (log-link) to determine the effect of IMD on the number of dances per bee per treatment (dance frequency), with a random intercept for hive and for individual bees. Additionally, we used Poisson GLMMs (log-link) to determine the effect of IMD on the number of waggle runs per dance per treatment (waggle run frequency), with a random intercept for hive and for individual bees. To determine the effect of IMD on the number of visits per bee per treatment to the empty feeders during Day 1 and Day 2 feeders (persistency), we used Poisson GLMMs (log-link) with treatment as a fixed effect, random intercepts for hive and for individual bees. Finally, we use binomial GLMMs (logit-link) with treatment as a fixed effect and random intercepts for hive and individual bees to determine the effect of IMD on proportion of persistency visits that were made to the trained feeders (site-specificity). The Bayesian statistical analysis was done using the Rstan (Stan Development Team 2020) and Rethinking packages (McElreath 2018) in R 3.6.3 (R Core Team 2020). The ulam function was used to build the models in R and to run Hamiltonian Monte Carlo sampling of the resultant posterior distributions (McElreath 2018). For the Bayesian models, we report the mean and 95% credibility interval for the mean response and the odds ratio (binomial models) and proportional mean difference for the treatment versus the control (Poisson models). We used the position of the posterior distributions for the odds ratio and the proportional mean difference to indicate the type (significant versus nonsignificant) and direction of the observed effects. Credibility intervals on the response scale containing 1 indicate nonsignificance, while those distributed entirely above 1 indicate a positive treatment effect and those below 1 indicate a negative treatment effect for Poisson and binomial models.

Results

IMD Decreased Foraging Frequency

There was a significant, negative effect of IMD on foraging frequency (Control: mean = 30.94, 95% CI [15.17,57.20]; IMD: mean = 21.98, 95% CI [10.59,40.59]; proportional mean difference = 0.72, 95% CI [0.51,0.95]). In other words, IMD foragers displayed a 28% [-49 to - 5%] decrease in mean foraging frequency compared to control foragers (Fig. 1).
Fig. 1.

IMD decreased foraging frequency between control and treatment bees during the 3 hour experimental period (*). The vertical line represents the 95% credibility intervals sampled from the posterior for the mean foraging frequency, while the white and black points indicate the mean value from the posterior for the control and IMD foragers, respectively. IMD treated bees foraged c. –28% [–49% to –5%] compared to control bees.

IMD decreased foraging frequency between control and treatment bees during the 3 hour experimental period (*). The vertical line represents the 95% credibility intervals sampled from the posterior for the mean foraging frequency, while the white and black points indicate the mean value from the posterior for the control and IMD foragers, respectively. IMD treated bees foraged c. –28% [–49% to –5%] compared to control bees.

IMD Treated Foragers Displayed Consistent Numerical Decreases in Dance Propensity, Dance Frequency, and Waggle Run Frequency

IMD treated per protocol foragers displayed a numerical decrease in dance propensity (Control: mean = 0.79, 95% CI [0.56,0.92], IMD: mean = 0.61, 95% CI [0.28,0.87]; mean relative odds: 0.40, 95% CI [0.16,1.02]), with IMD foragers displaying a 60% [–84 to +2%] decrease in their odds of dancing compared to control foragers (Fig. 2A). Likewise, IMD treated per protocol foragers displayed a numerical decrease in dance frequency (Control: mean = 11.10, 95% CI [4.46,22.94]; IMD: mean = 7.86, 95% CI [3.54,15.41], proportional mean difference = 0.73, 95% CI: 0.50 to 1.04) with IMD foragers displaying a 27% [–50, +4%] decrease in dance frequency compared to control foragers (Fig. 2B). Despite being nonsignificant, these numerical decreases suggest that IMD foragers tended to be less likely to both dance and, if they did dance, to dance less frequently compared to control foragers across the 3 hr experimental phase. Finally, there was a small numerical decrease in waggle run frequency in IMD treated foragers compared to control foragers (Control: mean = 15.44, 95% CI [9.85,21.43]; IMD: mean = 14.42, 95% CI [8.99,20.26]; proportional mean difference = 0.94, 95% CI [0.74,1.17]) of 6% [–26,+17%] (Fig. 2C).
Fig. 2.

IMD foragers displayed numerical decreases in dance propensity (A), dance frequency (B) and waggle run frequency (C). The vertical lines represent the 95% credibility intervals sampled from the posterior for the mean behavioral responses, while the white and black points indicate the mean values from the posteriors for the control and IMD foragers, respectively. Although these results are nonsignificant, they all demonstrate a decrease in recruitment behaviors with IMD exposure.

IMD foragers displayed numerical decreases in dance propensity (A), dance frequency (B) and waggle run frequency (C). The vertical lines represent the 95% credibility intervals sampled from the posterior for the mean behavioral responses, while the white and black points indicate the mean values from the posteriors for the control and IMD foragers, respectively. Although these results are nonsignificant, they all demonstrate a decrease in recruitment behaviors with IMD exposure.

IMD Decreased Foraging Persistency on Day 2, But Not Day 1

There was a significant, negative effect of IMD on foraging persistency on Day 2 (Control: mean = 0.34, 95% CI [0.05,1.07]; IMD: mean = 0.14, 95% CI [0.02,0.49]; proportional mean difference = 0.44, 95% CI [0.18,0.87]), with the IMD foragers displaying a 66% [–82 to –13%] decrease in foraging persistency compared to control foragers (Fig. 3). In contrast, there was no effect of IMD on foraging persistency on Day 1 (Control: mean = 3.41, 95% CI [1.46,6.13]; IMD: mean = 3.25, 95% CI [1.32,6.19]; proportional mean difference = 0.96, 95% CI: [0.63,1.41], Fig. 3).
Fig. 3.

IMD foragers displayed a nonsignificant decrease in persistency on Day 1 and a significant decrease in persistency on Day 2 (*). The vertical lines represent the 95% credibility intervals sampled from the posterior for the mean total persistency visits, while the white and black points indicate the mean values from the posteriors for the control and IMD foragers, respectively. IMD treated bees were c. –4% [–37 to +41%] less persistent on Day 1 and c. –66% [–82 to –13%] less persistent on Day 2 compared to control foragers.

IMD foragers displayed a nonsignificant decrease in persistency on Day 1 and a significant decrease in persistency on Day 2 (*). The vertical lines represent the 95% credibility intervals sampled from the posterior for the mean total persistency visits, while the white and black points indicate the mean values from the posteriors for the control and IMD foragers, respectively. IMD treated bees were c. –4% [–37 to +41%] less persistent on Day 1 and c. –66% [–82 to –13%] less persistent on Day 2 compared to control foragers.

IMD Did Not Affect Site Specificity

There was no significant effect of IMD on site specificity on Day 1 (Control: mean = 0.91, 95% CI [0.79,0.96], IMD: mean = 0.90, 95% CI [0.73,0.96]; mean relative odds: 0.88, 95% CI [0.41, 1.89]), or on Day 2 (Control: mean = 0.87, 95% CI [0.73,0.96], IMD: mean = 0.93, 95% CI [0.80,0.99]; mean relative odds: 2.31, 95% CI [0.65,8.81]; Fig. 4).
Fig. 4.

IMD foragers did not affect site specificity on Day 1 or Day 2. The vertical lines represent the 95% credibility intervals sampled from the posterior for the mean total persistency visits, while the white and black points indicate the mean values from the posteriors for the control and IMD foragers, respectively.

IMD foragers did not affect site specificity on Day 1 or Day 2. The vertical lines represent the 95% credibility intervals sampled from the posterior for the mean total persistency visits, while the white and black points indicate the mean values from the posteriors for the control and IMD foragers, respectively.

Discussion

Here we used freely flying bees in a semifield feeder experiment to determine the effects of a sublethal, field-realistic, concentration of the neonicotinoid imidacloprid (IMD) on honey bee foraging and recruitment. We report significant effects of IMD on foraging frequency and foraging persistency, and nonsignificant, but considerable effects on waggle dance propensity, waggle dance frequency, and waggle run frequency. In all these latter instances, the data are consistent with IMD decreasing the foraging (Yang et al. 2008, Schneider et al. 2012, Tison et al. 2016, Tison et al. 2020) and recruitment behaviors (Eiri and Nieh 2012, Tison et al. 2016, Tison et al. 2020) in bees, though we cannot completely rule out noneffects. This suggests that IMD causes the honey bee foragers to devalue the reward (Seeley 1994), even though both treatment and control solutions were equal in molarity and equidistant from the lab. These effects were limited to the honey bees that foraged exclusively on the treatment solution versus those that foraged only on the control solution. The observed effects support the hypothesis that honey bees might employ an optimal foraging strategy when exposed to sublethal doses of neonicotinoids, where an organism limits exposure to toxic substances in the field by decreasing their foraging and recruitment activity to them (Easton and Goulson 2013). Additionally, our findings must be considered in the broader context of lab, semifield, and field studies to determine whether these results represent a general decrease in motivation to forage and recruit, which we could call a general inhibition, reduced physical (Williamson et al. 2014), sensory (Andrione et al. 2016) or cognitive abilities (Aliouane et al. 2009, Wright et al. 2015), which we could call foraging impairment, or a specific aversion to neonicotinoid-laced foods, which we could call an adaptive aversion. Previous research, both laboratory (Eiri and Nieh 2012, Kessler et al. 2015) and semifield (Yang et al. 2008, Schneider et al. 2012, Tison et al. 2016, Tison et al. 2020) has demonstrated that neonicotinoids inhibit foraging in honey bees. However, Kessler et al. (2015) reported that caged honey bees preferred solution containing a 26 ppb concentration of IMD over control solutions in two-choice assays. Interestingly, Kessler et al. (2015) also report that caged honey bees foraged preferentially on solutions containing a 1000 nM concentration of thiamethoxam or clothianidin over control solutions in the same two-choice assays. Although these bees drank less total solution than those with access to only control solutions. The latter findings suggest that honey bees simultaneously increase their proportional intake of neonicotinoids while decreasing their total solution consumption when exposed to certain concentrations of some neonicotinoids. These seemingly contrasting results probably reflect the differences between their experimental methods and ours: Firstly, our study compares the foraging behavior of honey bees that were trained to visit a treatment or control solution only, while Kessler et al. (2015) compared the choices of honey bees with equal access to treatment and control solutions against those with access to control solutions alone. Therefore, our experimental design was only able to demonstrate the overall decrease in foraging, but neither confirm nor refute the proportional increase in foraging to neonicotinoid treatment. Secondly, we used freely flying foragers in a feeder experiment that more closely simulates the complexities of a typical foraging scenario: our experiment provided a more cognitively and energetically demanding foraging task, in which honey bees were tasked with navigating to and from a feeder positioned 145 m away from their colony. Freely flying foragers might be more susceptible than caged foragers to the reported adverse physiological (Williamson et al. 2014) and cognitive effects (Wright et al. 2015, Mustard et al. 2020), which could have caused the neonicotinoid induced decreases in foraging to occur at lower doses in our semifield experiment than in the lab (Kessler et al. 2015). Lab studies provide a highly controlled system that is well-suited for describing the basic behavioral responses of honey bees to pesticides. Common laboratory methods, such as proboscis extension response experiments, have demonstrated that neonicotinoids decrease sucrose responsiveness (Aliouane et al. 2009, Eiri and Nieh 2012, Démares et al. 2016, Démares et al. 2018, Jiang et al. 2018), as well as learning (Tan et al. 2015, Mustard et al. 2020) and memory capabilities (Tan et al. 2015, Wright et al. 2015) in honey bee foragers. These studies provide foundational insights into the behavioral responses of honey bees. However, honey bee foraging is an extraordinarily complex task, requiring a suite of navigational (Menzel et al. 2005), sensory (Balbuena et al. 2012), memory (Menzel and Müller 1996), and learning (Menzel and Müller 1996) capabilities to make optimal foraging decisions. In our study, we used feeder experiments to incorporate the additional variables that honey bees might encounter when freely foraging in a landscape. In doing so, we demonstrate that neonicotinoid-treated honey bee foragers decrease their foraging frequency to rewarding solutions and persistency to newly unrewarding solutions in a semifield context. These results indicate that the behaviors observed in individual honey bees in the lab are relevant to colony-level foraging and recruitment behaviors in the semifield. The decrease in foraging activity is consistent with various other semifield studies showing that honey bees reduce their foraging (Yang et al. 2008, Schneider et al. 2012) and recruitment (Eiri and Nieh 2012) to even untreated solution after acute exposure to neonicotinoids. Such studies suggest that honey bees respond to neonicotinoids with a general inhibition of foraging (to both treated and untreated food sources), rather than an adaptive aversion to neonicotinoids. Neonicotinoids are often persistent in the environment (Bonmatin et al. 2005), potentially leading to chronic exposure to foraging honey bees. Therefore, the level of exposure and severity of adverse effects from exposure are determined by the behavioral responses of honey bees in the field. Our study elucidates the effects of a 3 h exposure to sublethal concentration of neonicotinoids on honey bees. In doing so, we demonstrate that honey bees do not avoid neonicotinoid-treated foods over the course of a three hr foraging period, but instead reduce their foraging activity. Only a few other studies, most containing low colony-level replication (Tison et al. 2016, Tison et al. 2020), have investigated the behavior of honey bees foraging on feeders containing sublethal doses of neonicotinoids for extended periods of time. These studies report a similar trend for decreased foraging when visiting feeders containing neonicotinoids over several weeks (Tison et al. 2016, Tison et al. 2020). Additionally, the observed decrease in foraging activity is consistent with field studies reporting decreased colony weight gain in colonies located near agricultural lands (Smart et al. 2018), as well as those reporting decreased foraging traffic (Wu-Smart and Spivak 2016), pollen stores (Wu-Smart and Spivak 2016) and decreased colony weight gain (Wood et al. 2018) in colonies fed a similar 20 ppb solution of IMD, inside the hive, over several weeks. Feeder experiments play an important role in revealing the likely responses of honey bees to different substances, such as neonicotinoids, in the field. Furthermore, understanding these behavioral responses is important for assessing the potential risk of honey bees in urban and agricultural settings, where neonicotinoid exposure is common (Wood et al. 2019). However, it is also important to consider the mechanisms that underlay the observed behavioral effects, which could simultaneously drive the preference for neonicotinoid-laced solutions in the lab (Kessler et al. 2015) and decrease in foraging in the semifield (Yang et al. 2008, Schneider et al. 2012, Tison et al. 2016, Tison et al. 2020). For example, honey bees could experience neurochemical effects, such as octopaminergic rewards (Barron et al. 2007) or dopaminergic punishments (Klappenbach et al. 2013), in response to a positive (or negative) sensory experience (Linn et al. 2020). Additionally, honey bees could experience pharmacological effects that increase (or decrease) their perception of reward, or affect their cognitive and motor abilities (Williamson et al. 2014). Kessler et al. (2015) used electrophysiological recordings of both sugar-sensing neurons and “bitter”-sensing neurons to determine that honey bees cannot taste neonicotinoids in nectar. Therefore, pharmacological effects likely explain the proportional increase in IMD treated solution consumption in the lab. However, freely-flying bees, like those in our experiment, might experience additional effects, such as decreased navigational (Henry et al. 2012) and flight (Tosi et al. 2017a) abilities that could further decrease their ability to forage efficiently. Taken together, pharmacological effects that decrease foraging motivation and/or capabilities are the most plausible mechanism underlying our observed decreases in honey bee foraging and recruitment activity. Honey bee foraging and recruitment behaviors are usually highly correlated (Seeley 1995, Couvillon et al. 2015) and should respond similarly to treatment. Honey bees increase their foraging effort on a gradient from individual foraging to lower quality resources (less frequent to more frequent) to individual foraging and colony-level recruitment to higher quality resources (von Frisch 1967, Seeley 1994, Seeley et al. 2000). Feeder experiments have demonstrated that individual foraging adjusts resource exploitation linearly, while recruitment (or diminished recruitment) alters resource exploitation nonlinearly (von Frisch 1967, Seeley 1995). Therefore, we expected recruitment to respond more strongly to treatment than foraging frequency. Indeed, we report significant decreases in foraging frequency and Day 2 persistency and only nonsignificant (but in the same direction) decreases in dance propensity, dance frequency, and waggle run frequency. Why could it be that we saw a large decrease in foraging behavior (Fig. 1) but only a modest decrease in recruitment behaviors (Fig. 2)? Importantly, not every forager performs a waggle dance. Instead, only foragers working the best resources at any given time will dance, and it is possible for a bee to forage back and forth at a resource without recruiting her nestmates (von Frisch 1967, Seeley 1994, Seeley et al. 2000). Such a situation will occur for good but not great resources, and if that resource suddenly decreases in quality, the foraging will likewise decrease, but recruitment, which was already happening at a low level might not display as large an effect. Interestingly, Eiri and Nieh (2012) report the opposite trend for decreased recruitment, but not foraging activity after a treatment with IMD. However, in their experiment, honey bees visited a feeder positioned 1.5 meters away from the colony (Eiri and Nieh 2012), while our bees visited a feeder 145 m from the colony. Clearly, our experiment provides a more challenging task and bees might have simply struggled to navigate efficiently from the colony to the feeder. Our nonsignificant (but in a consistent direction) effect on dance propensity and dance frequency might reflect the conservative statistical approach that we took, in which we used the 2018 experimental observations to produce priors for the 2019 analysis, rather than pulling the data from the two field seasons with slightly different experimental methods. Feeder training is difficult because foragers prefer to visit natural forage over artificial resources (von Frisch 1967, Seeley 1995). We selected late July-early August as our study period because we initially thought that time would constitute a nectar gap; however, despite our best efforts, we struggled to train bees in both years. We now know from a different study that we are doing in our lab that forage is actually readily available in those months, making it a nonideal time for a feeder experiment. Additionally, we experienced the previously mentioned feeder swapping in 12.3% of visits, resulting in cross-exposure to the control and treatment solutions. We addressed this unexpected behavioral response by analyzing just the bees that exclusively visited their trained feeder. Of course, there is always a risk of introducing bias by systematically eliminating data from analyses. For example, we considered the possibility that the 100% consistent foragers represent a physiologically distinct subset within the colony, consisting of bees that are robust against the effects of IMD. However, our analysis of commitment to the training feeder (i.e. incorrect visits during the experimental phase) indicated that the treatment did not affect the odds that a forager would switch or not switch, which gave us reason to believe that the foragers were randomly allocated into the consistent forager subsample. The most biologically relevant experimental observations are those that measure the responses of organisms under field-realistic conditions. We used a 26 ppb concentration of IMD in our experiment because it is within the range of concentrations found in at least some agricultural settings (Byrne et al. 2014) and had previously been shown to elicit behavioral effects in the lab (Kessler et al. 2015). However, it is important to note that the IMD concentrations found in the field are variable, with a range of possible concentrations being reported (Dively and Kamel 2012, Pohorecka et al. 2012, Byrne et al. 2014, Long and Krupke 2016). For example, Byrne et al. (2014) reported that the concentration of IMD found in citrus nectar depended on field site and lower concentrations were found in the crops of freely foraging honey bees. Additionally, lower concentrations have been reported in bee-collected pollen (Long and Krupke 2016) and bee-collected nectar (Dively and Kamel 2012, Pohorecka et al. 2012), with IMD being nearly undetected in bee-collected pollen in rural landscapes during the summer (Long and Krupke 2016). After examining the range of concentrations reported in different field/experimental contexts and the behavioral responses to a 26 ppb concentration of IMD reported in the lab, we decided that this concentration gave us the best opportunity to both observe behavioral effects and to gain field-relevant insights. However, our results only demonstrate the foraging and recruitment responses of Italian honey bees to a single concentration of IMD, while honey bee stocks vary in their sensitivity to pesticides (Laurino et al. 2013, Rinkevich et al. 2015) and are exposed to various neonicotinoids within a range of concentrations (Dively and Kamel 2012, Pohorecka et al. 2012, Byrne et al. 2014, Long and Krupke 2016). Future studies utilizing honey bees with different genetic backgrounds and/or other neonicotinoids across the broad range of concentrations observed in the field are needed to better assess the risk of neonicotinoids to foraging honey bees. In summary, we report novel, biologically relevant, data describing the behavioral responses of freely-flying honey bees to a sublethel, field-realistic, concentration of IMD. Our results add to previous research demonstrating that honey bees reduce foraging to neonicotinoid-laced solutions in the lab and foraging after acute exposure in a semifield context. Unlike several previous studies, honey bees in our experiment were given the option to continuously visit a feeder containing neonicotinoids. We found that they did not actively avoid the treatment, but instead continued to forage and recruit, but at a reduced rate. These results, along with the results of previous studies, suggest that honey bees probably do not employ optimal foraging to avoid pesticides and instead display a general inhibition of foraging to both foods containing pesticides and those not containing pesticides. Such behaviors could possibly lead to an overall decrease in food intake and poorer health outcomes. Click here for additional data file.
  54 in total

1.  Honey bees navigate according to a map-like spatial memory.

Authors:  Randolf Menzel; Uwe Greggers; Alan Smith; Sandra Berger; Robert Brandt; Sascha Brunke; Gesine Bundrock; Sandra Hülse; Tobias Plümpe; Frank Schaupp; Elke Schüttler; Silke Stach; Jan Stindt; Nicola Stollhoff; Sebastian Watzl
Journal:  Proc Natl Acad Sci U S A       Date:  2005-02-14       Impact factor: 11.205

2.  The global stock of domesticated honey bees is growing slower than agricultural demand for pollination.

Authors:  Marcelo A Aizen; Lawrence D Harder
Journal:  Curr Biol       Date:  2009-05-07       Impact factor: 10.834

3.  Incorporating variability in honey bee waggle dance decoding improves the mapping of communicated resource locations.

Authors:  Roger Schürch; Margaret J Couvillon; Dominic D R Burns; Kiah Tasman; David Waxman; Francis L W Ratnieks
Journal:  J Comp Physiol A Neuroethol Sens Neural Behav Physiol       Date:  2013-10-17       Impact factor: 1.836

4.  Comparative chronic toxicity of three neonicotinoids on New Zealand packaged honey bees.

Authors:  Sarah C Wood; Ivanna V Kozii; Roman V Koziy; Tasha Epp; Elemir Simko
Journal:  PLoS One       Date:  2018-01-02       Impact factor: 3.240

5.  Low dose of neonicotinoid insecticide reduces foraging motivation of bumblebees.

Authors:  Juho Lämsä; Erno Kuusela; Juha Tuomi; Sini Juntunen; Phillip C Watts
Journal:  Proc Biol Sci       Date:  2018-07-25       Impact factor: 5.349

6.  Foraging bumblebees acquire a preference for neonicotinoid-treated food with prolonged exposure.

Authors:  Andres N Arce; Ana Ramos Rodrigues; Jiajun Yu; Thomas J Colgan; Yannick Wurm; Richard J Gill
Journal:  Proc Biol Sci       Date:  2018-08-29       Impact factor: 5.349

7.  The neonicotinoid insecticide imidacloprid repels pollinating flies and beetles at field-realistic concentrations.

Authors:  Amy H Easton; Dave Goulson
Journal:  PLoS One       Date:  2013-01-28       Impact factor: 3.240

8.  Cholinergic pesticides cause mushroom body neuronal inactivation in honeybees.

Authors:  Mary J Palmer; Christopher Moffat; Nastja Saranzewa; Jenni Harvey; Geraldine A Wright; Christopher N Connolly
Journal:  Nat Commun       Date:  2013       Impact factor: 14.919

9.  Neonicotinoids interfere with specific components of navigation in honeybees.

Authors:  Johannes Fischer; Teresa Müller; Anne-Kathrin Spatz; Uwe Greggers; Bernd Grünewald; Randolf Menzel
Journal:  PLoS One       Date:  2014-03-19       Impact factor: 3.240

10.  Sub-lethal effects of dietary neonicotinoid insecticide exposure on honey bee queen fecundity and colony development.

Authors:  Judy Wu-Smart; Marla Spivak
Journal:  Sci Rep       Date:  2016-08-26       Impact factor: 4.379

View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.