An adaptable mesocosm platform for performing integrated assessments of nanomaterial risk in complex environmental systems.

Physical-chemists, (micro)biologists, and ecologists need to conduct meaningful experiments to study the environmental risk of engineered nanomaterials with access to relevant mechanistic data across several spatial and temporal scales. Indoor aquatic mesocosms (60L) that can be tailored to virtually mimic any ecosystem appear as a particularly well-suited device. Here, this concept is illustrated by a pilot study aimed at assessing the distribution of a CeO₂-based nanomaterial within our system at low concentration (1.5 mg/L). Physico-chemical as well as microbiological parameters took two weeks to equilibrate. These parameters were found to be reproducible across the 9-mesocosm setup over a 45-day period of time. Recovery mass balances of 115 ± 18% and 60 ± 30% of the Ce were obtained for the pulse dosing and the chronic dosing, respectively. This demonstrated the relevance of our experimental approach that allows for adequately monitoring the fate and impact of a given nanomaterial.

Engineered nanomaterials (ENMs) have become a fast growing economic sector. As a consequence of the many debates concerning their safety, efforts are developed at international and national levels to develop a code of ethics for a safe and responsible development of ENMs1. A sustained growth of the nanotechnology industry will rely heavily on the characterization of risks to the environment (water and soil resources, trophic transfers, biodiversity) and human health that may be posed by ENMs. To date, investigation of the roles of nano-scale objects in the context of evolutionary change, environmental disturbance, ecosystem structure and function are limited23. Moreover, current strategies to assess the environmental safety of ENMs are based on classical ecotoxicology approaches4, which are not always adequate for ENMs. Indeed, while the hazard is extensively investigated, little attention is paid to the exposure to ENMs despite its pivotal role in understanding their environmental risks. Extent and mode of exposure to ENMs is controlled by a number of parameters including aggregation state and sorption of (in)organic substances, redox as well as ecological factors such ecological feeding type and trophic transfer potential5. There is an abundant literature about the effects of all these parameters taken separately. However, for a robust characterization of the exposure, the complex interplay between these parameters in real ecosystems needs to be considered.

Mesocosms are experimental systems designed to simulate ecosystems6 and are an invaluable tool for addressing the complex issue of exposure during nano-ecotoxicological testing. This experimental strategy has already been used to study the behavior or impacts of ENMs78910. A broad diversity of mesocosms design exists in term of dimension, location (indoor, outdoor) and ecosystem simulation type (estuarine, aquatic freshwater, and terrestrial)11121314. A common factor of all these studies is that mesocosms are considered as a small portion of the natural environment that is brought under controlled conditions. In our study we define a mesocosm as an experimental design which is (i) self-sustaining once set-up and acclimation without any additional input of nutrients or resources, and (ii) that allows controlling all (or the maximum of) input and output parameters to draw a real-life mass balance whatever its dimension or location. Mesocosms has already been applied to trace the transfer of gold ENMs from the water column in an estuarine food web7. The authors modeled the edge of a tidal marsh creek using 366 L-indoor mesocosms maintained for 12 days. Clams and biofilms were observed to accumulate most of gold on a per mass basis. The long-term (18 month) distribution and transformation of silver ENMs was also studied in outdoor mesocosms mimicking freshwater emergent wetlands10. Silver sulfidation was demonstrated in the terrestrial soils and subaquatic sediments and a high body burden of Ag was measured in mosquito fish and chironomids10. Another study used indoor estuarine mesocosms to monitor the leaching of Ag from consumer products incorporating ENMs over 60 days. The investigations described in the literature involve rather large facilities (tank size 120 L and above), to reproduce the assimilative capabilities of a larger natural system. However, the need for multiple replicas in biological studies limits the practicality of large mesocosms due to obvious limitations in space and cost. The challenge is thus downsizing for larger setups while avoiding the artifacts of in vitro and high throughput studies (e.g.151617), which do not simulate adequately the complexities and important ecological interactions within an ecosystem. Moreover, there is a need for adaptable and versatile exposure system to allow different real word exposure scenarios to be contrasted (e.g. ENMs runoff rain or vent loading, or a continuous point source discharge as wastewater treatment plant or industrial discharge).

In this study, we conceived a laboratory-scale mesocosm facility to serve as a platform for investigating ENM exposure and impacts. We opted for modular, small size (60 L), indoor mesocosms. Our experimental system allows the simultaneous monitoring of a number of parameters (e.g. aggregation, settling, mass balance, trophic transfer, biotransformation, oxidative stress, microbial diversity) under environmentally meaningful conditions. This experimental design can accommodate several types of ecosystems such as lotic, lentic, estuarine, or lagoon environments, without requiring expensive and/or cumbersome infrastructures. This versatile tool can then be used by a large community of physical-chemists, (micro)biologists, and ecologists to study the exposure and impacts of ENMs (low doses, chronic contamination) as well as the mechanistic concepts at various temporal and spatial scales. Here we show that, with the adequate methodology, using small sized mesocosms is an approach as robust as using large(r) systems.

Mesocosm Description

Design

An experiment performed in this platform involves 2 phases: the acclimation and equilibration of the mesocosms, and the exposure phase of the experiment, per se. The initial setup consists in introducing the sediment and filling the tank (750 × 200 × 600 mm) with water. The water and the sediment are chosen to be close to the solution chemistry of the natural ecosystem of interest. Macro- and micro-organisms are collected from the same natural environment.

During phase I, the particles suspended by the addition of water are given time to settle, the pH, conductivity, O2, and redox potential stabilize, and the primary producers develop. The duration of this phase depends on the target values (and variations around them) defined for each key parameter (e.g. ΔpH, ΔT, turbidity, ammonia). Then the organisms are allowed to acclimatize (e.g. primary, secondary consumers) and the water pumps are turned on. The selected organisms are involved in a real food web and have different habitats and ecological functions in the ecosystems. The density of organisms is adjusted as a function of the natural environment. The duration of the acclimation depends on biological features of the species as growth rate, metabolism activity, life cycle duration.

Phase II corresponds to the ENMs exposure period and ecotoxicity test. It can be either a single pulse or multiple dosing experiments. The treatments are distributed between each mesocosm as a function of the picoplankton and algal abundances. To avoid any variation in term of primary production, the triplicates are selected to have average concentrations of picoplankton and algae as close as possible between the different treatments.

Sampling and analyses

Several physico-chemical, microbial, and biological analyses can be performed to assess both the exposure and impacts of ENMs on a designated trophic link. A number of parameters can be monitored continuously with the appropriate probe (e.g. pH, temperature, Eh). Other parameters (e.g. metal concentration, number of colloids, picoplankton and algae concentrations) require sampling. Using a small mesocosm setup, water, superficial sediments, cores, picoplankton, algae, and macro-invertebrates can be sampled with any desired periodicity. During sampling, special attention must be given (i) to avoid disturbing the sediment and water column properties, and (ii) to keep micro-organism densities and ENMs concentrations constant.

The distribution of the ENMs or their degradation by-products is assessed by measuring their concentration in the water, sediment, biota, etc. using conventional analytical methods (e.g. ICP-MS or ICP-AES). When necessary, the dissolution of ENMs can be measured separately by placing sealed dialysis bags in the mesocosms.

A thorough characterization of the speciation, (bio)transformation, bio(distribution) of the ENMs in the water, sediment, or biota can be performed using X-ray, IR, Raman spectroscopies, Nuclear magnetic resonance, as well as electron- or X ray-based microscopy and tomography. Such an experimental design also allows monitoring the mechanisms of toxicity at the sub-individual scale on the micro-, macro-organisms as well as on microbial communities. For instance, oxidative stress18 can be assessed using ecotoxicological markers and ecophysiological processes19202122.

Pilot Study Using CeO2 ENMs

A pilot study was conducted to assess the evolution of the distribution of a CeO2-based ENM that is included on the OECD list for ENMs requiring (eco)toxicological testing23. Citrate-coated CeO2 nanoparticles (8 nm of hydrodynamic diameter) sold (Nanobyk 3810, Byk24) as long-term UV-stabilizers were used in this work25.

The study proceeded through phase I and II with respective exposure periods of 17, and 28 days (Fig. 1a). Two contamination scenarios were simulated. The response of the mesocosms to a single mass addition (pulse dose) of 69 mg to achieve an initial concentration of 1.5 mg/L of CeO2 ENMs at day 0 was compared to that of resulting from 11 smaller doses (chronic doses) of 5.2 mg administered 3 times per week during 4 weeks to achieve a final concentration at day 28 of 1.4 mg/L of CeO2 ENMs (Fig. 1a).

Figure 1

Description of the experimental design.

(a) Experimental schedule and concentration of CeO2-based ENMs used in the pilot study. The phase I equilibrium period was 2 weeks prior to dosing. Reported CeO2 concentrations are based upon a single pulse exposure (at 69 mg of Ce) versus multiple dose exposure, which are reported as cumulative exposure concentrations with each addition of 5.2 mg of Ce. (b) Schematics of the indoor aquatic mesocosms. Tank of 750 × 200 × 600 mm.

The mesocosm platform was configured to simulate a pond ecosystem using an invertebrate species (Planorbarius corneus L., 1758, commonly named ramshorn snail) and a natural inoculum coming from a non-contaminated pond (43.34361 N, 6.259663 E, and 107 m above sea level). This pond is part of the protected Natura 2000 Reserve Network. Nine mesocosms were setup allowing 3 replicates per experimental dosing regimen (e.g. single pulse versus multiple dosing) condition and the control. Each mesocosm is made of monolithic glass panels of 12 mm-thick. Five holes (ø 15 mm) drilled at mid height of the large panels are connected to a pump using silicon tubes (Fig. 1b). The mesocosms were filled with 5–8 cm of artificial sediment made of 84 ± 5% (dry weight) of quartz (grain size: ~60% from 0.05–0.2 mm, and ~40% from 0.2–2 mm), 15 ± 5% of kaolinite, and ~1% of CaCO3 (adapted from26). Three hundred g (water saturated weight) of a natural inoculum collected in the pond was sieved at 0.2 mm and laid at the surface of the artificial sediment (1 mm-thick). This natural inoculum contained CaCO3, SiO2, and clay minerals (see supporting information) and introduced the primary producers into the mesocosms. Each mesocosm contained 55 L of the commercialized natural Volvic, and eleven adults P. corneus (3 ± 1 cm of diameter). Volvic is a French commercialized natural water with the following composition: pH 7, 11.5 mg/L Ca2+, 13.5 mg/L Cl−, 71 mg/L HCO3−, 8 mg/L Mg2+, 6.3 mg/L NO3−, 6.2 mg/L K+, 11.6 mg/L Na+. The mesocosms were operated 4 days to acclimate the invertebrates to the experimental conditions before the introduction of the CeO2 ENMs suspensions. This few days of acclimation period were based on previous studies working with micro- and macro-invertebrates in mesocosms272829.

Temperature, pH, conductivity, redox potential, and dissolved O2 were measured every 5 min at mid height of the water column using multi-parameter probes (Odeon Open X) and at the water/sediment interface (up to 10 mm below surficial sediment) and mid height of the sediment using platinum-tipped redox probes1030. A day/night cycle of 10 h/14 h was applied using full spectrum light (Viva light T8 tubes 18 W), and room temperature was kept constant.

All data were analyzed for normality and homoscedasticity using the Kolmogorov-Smirnov test and Levene test31. In low sample size comparisons, differences between groups were analyzed using a Mann-Whtiney U test31. When assumptions were met, One-way ANOVA using General Linear Model followed by Tukey Honest Significant Difference test were performed using STATGRAPHICS Centurion XVI.II (Sigma Plus, France) or Statistica 6 (StatSoft Inc., Tulsa, USA)32.

Phase I: acclimation and equilibration

During this phase of acclimation and equilibration, values for each key parameter (pH, conductivity, and dissolved O2 concentration) and their associated variance between the 9 mesocosms were recorded and calculated. The condition that was set for ending phase I (equilibration) was the reduction by ~50% (compared to the beginning of phase I) of the standard deviation corresponding to the average pH, conductivity, and dissolved O2 concentration between the 9 mesocosms. At the end of phase I, ΔpH of 0.5, ΔT of 0.4°C, ΔO2 of 2.5 mg/L, and ΔC of 20 μS/cm were reached (Fig. 2). The conductivity (C) monotonically increased from 250 to 330 μS/cm due to refilling with mineral water to compensate for evaporation. The pH stabilized around 7.9 ± 0.1 (at 20–25°C) and the dissolved O2 concentration reach 8 ± 0.2 mg/L (~90%), which is close to the natural pond water. Both pH and dissolved O2 concentration underwent diurnal variations of 0.35 pH units, and 0.7 mg/L respectively over the day-night cycle. The redox potential in the water column was stable and positive (between +220 and +250 mV) during phase I, whereas in the sediment the redox potential exhibited a positive-negative inversion (down to −330 mV) two days after the filling with water. This indicates that anoxic conditions prevailed in the sediment (Fig. 2).

Figure 2

Evolution of the physico-chemical parameters.

Redox potential in the sediment (A) and in the water column (B), dissolved oxygen (C), pH (D), total organic carbon (E), and conductivity (F) were measured during phases I and II of the pilot study. The grey surface is defined by the maximum and minimum values of each parameter, and the dark line corresponds to the average values of the 9 mesocosms. One measurement was performed every 5 min.

A significant decrease (p < 0.05) of the picoplankton concentration (from 105 to 104 cells/mL) and algae concentration below 105 algae/mL were observed in the water column at the end of phase I (see supporting information). This decrease was concomitant with the settling of suspended particles and a corresponding decrease in concentration from 9 × 105 to 3 × 105 particle/mL (see supporting information). In contrast, at the surface of the sediment, picoplankton and algae concentrations slightly increased during phase I from 106 to 107 cells/mL and from 105 to 106 algae/mL respectively. This is attributed to the development of the picoplankton and algae coming from the natural inoculum at the sediment surface. Pictures of microorganisms that developed in the water column as single cells or small flocs, and as biofilms in the sediment are given in supporting information. Before the introduction of organisms, bacteria formed biofilms of ~100 µm depth aggregating on clays, quartz particles, and algae, which are primer producers for non-phototrophic bacteria.

At the end of phase I, the similarity of microbial community compositions was assessed by pyrosequencing of the 16S rRNA gene (see supporting information). Microbial diversity in the mesocosms was considered in terms of richness and phylogenetic distance. The number of OTUs (Operational Taxonomic Units)33 and the Chao134 estimator best characterize the microbial richness, while the Bray-Curtis dissimilarity describes the phylogenetic distance between two samples. Table 1 summarizes diversity indices that describe water and benthic compartments of the mesocosms at the end of phase II. Based on Kruskall-Wallis test, there was no significant difference (p > 0.05) among the medians of observed OTUs and Chao1 metrics between A, B, and C in the water column or in the sediment. Thus the richness values for the limnetic or the sediment compartments of the three sets of mesocosms were not statistically different. Microbial phylogenetic diversity was examined at the phylum level. Fig. 3 shows the percentage distribution of OTUs in phyla, which accounted for more than 1%. The Bray-Curtis dissimilarity indices were low (<0.3) showing that the diversity of the mesocosms was similar. Proteobacteria, dominated in the water column (86%), with β-Proteobacteria as main class (76%), followed by the Actinobacteria (3.8%). Both of them often prevail in freshwater microbial communities3536. As expected, the diversity in sediment was higher than in the water column37. Sequences assigned to Proteobacteria were more abundant in water column (>80%) than in sediments (46%) while sequences assigned to Actinobacteria were more abundant in sediment (14%) than in water column (3.8%). Low-abundance phyla included Acidobacteria (5.9%) and Cyanobacteria (5.4%), Chloroflexi (3.7%), Firmicutes (3%), Bactereoidetes (2.6%), Planctomycetes (1.6%), and Nitrospira (1.1%). These phyla are generally enriched in freshwater sediments38 and pond freshwater sediments39. All concluded that prior to addition of ENMs, the mesocosms were reasonably similar and environmentally relevant as they reliably showed similar patterns in microbial communities composition.

Table 1

Diversity indexes of the limnetic and benthic compartments for mesocosms (three replicates each from pools of three mesocosms) before contamination. Observed OTUs and Chao1 were determined on a pool of sequences (3815) that were randomly picked (10 iterations) among 4515 to 17761 sequences for the water column and 6110 to 12944 sequences for the sediments. Bray–Curtis dissimilarities were based on abundances of assigned 97% OTUs at phylum level (75% sequence similarity)

	DNA pooled from triplicate mesocosms prior ENMs contamination	Observed OTUs (mean ± SD, n = 3)	Chao1 (mean ± SD, n = 3)	Bray Curtis dissimilarity
Water column	A: control mesocosms	311 ± 8	565 ± 62	A–B		A–C
	B: multiple dosing	350 ± 9	680 ± 77	0.07	B–C	0.26
	C: pulse dosing	365 ± 4	551 ± 12		0.25
Sediments	A: control mesocosms	1995 ± 19	4415 ± 154	A–B		A–C
	B: multiple dosing	1886 ± 17	3995 ± 123	0.11	B–C	0.07
	C: pulse dosing	1998 ± 27	4509 ± 130		0.13

Figure 3

Bacterial community structure at the phylum level in the mesocosms prior ENMs addition.

DNA pools of three mesocosms: (A) (the 3 control mesocosms), (B) (the 3 replicates prior multiple dosing of ENMs), and (C) (the 3 replicates prior a pulse dosing of ENMs). The abundance superior to 1% is presented as % of the total effective bacterial sequences.

Phase II: CeO2 ENMs contamination

During phase II, pH, redox potential, total organic carbon, dissolved O2, picoplankton, and algae concentrations were stable and not different between control and contaminated mesocosms. The recovery mass balance of the total Ce was determined in the sediment, water and organisms. After 28 days, 115 ± 18% of the Ce injected (for the pulse dosing) and 60 ± 30% of the Ce (for the multiple dosing) were recovered (Table 2). These results were not statistically different (p > 0.05). 89.2 ± 5% to 99.2 ± 0.2% of the Ce recovered was found in the surficial sediments (for multiple and single dosing respectively), 10.8 ± 5% to 0.8 ± 0.2% in the water column, and <0.1% in P. corneus (Table 2). As a function of the contamination scenario, different distributions of Ce were observed over time (Fig. 4). Following the single pulse dosing, the total Ce concentration in the water column decreased to 226 ± 86 μg/L at day 7, 96 ± 25 μg/L at day 14, and 14 ± 3 μg/L at day 28. However, multiple dosing of 5.2 mg of Ce 3 times per week, maintained the concentration of Ce in the water column around 50 ± 10 μg/L. It should be noted that the sampling for Ce dosing was performed before the next ENMs injection. Sedimentation rate constants (k) of Ce in the water column were obtained over time. A k of 0.16 per day was calculated from data on the pulse dosing while the multiple dosing yielded a more complex behavior. The multiple dosing in fact consisted of a series of smaller pulse inputs of ENMs to the system. Initially, a higher rate constant (k of 0.34 per day) characterizes the sedimentation of the first input of ENMs as calculated from the data following the first injection, which was treated mathematically as the response to a pulse input. A value of k equal to 0.44 per day is calculated from the longer-term trend in Ce concentrations ( with V the volume and C the concentration at steady-state). Based on the standard deviation value of the concentrations measured in the water column, k, k, and k can not be considered as statistically different. However, the 2 contamination scenarios lead to concentration of Ce in the water column significantly higher following a multiple dosing of CeO2 ENMs than a single pulse dosing.

Table 2

Mass balance and percentage of Ce measured in different compartments of the mesocosms at the end of the experiment (day 28)

	Mass balance	% of Ce in the surficial sediment	% of Ce in the water column	% of Ce in P. corneus
Multiple dosing	60 ± 30%	89 ± 5%	11 ± 5%	<0.1%
Single pulse dosing	115 ± 18%	99.2 ± 0.2%	0.8 ± 0.2%	<0.1%

Figure 4

Distribution of Ce between the aqueous and sedimentary compartments following multiple or pulse dosing of ENMs.

(A) Concentration of Ce in the water column (mg/L). Sampling was performed at 10 cm from the water surface. (B) Ce concentration at the surface of the sediment (mg/kg). Sampling was performed at the sediment surface (0.5 6 0.1 mm depth). Values are means ± standard deviation. Dotted lines are the exponential fits of the experimental data. (***) is statistical difference between multiple and pulse dosing at a given time (p < 0.001).

Discussion

Our pilot study demonstrated that the physical-chemical conditions during 45 days were reproducible between the 9 mesocosms. Two weeks were necessary to reach a state of equilibrium with anoxic conditions in the sediments, the sedimentation of the particles, and the homogenization of the microbial community composition. The 14-day stabilization time is consistent with earlier mesocosm studies for estuarine ecosystems4041. At the end of the contamination, recovery mass balances were about 115 ± 18% of the Ce (pulse dosing) and 60 ± 30% of the Ce (multiple dosing) which is in agreement with 84% of recovery observed in ref. 7 or 68–76% obtained in ref. 10.

This pilot study also highlighted that the exposure of the organisms (benthic vs. planktonic) will strongly depend on the contamination scenario. After a pulse dosing, the ENMs aggregated in 1 week as evidenced by the decreased of the total Ce concentration in the water column (to 15 μg/L) and concomitantly increased in the surficial sediments (to 540 mg/kg). In contrast, after multiple dosing, total Ce concentration in the water column remained almost constant (50 ± 10 μg/L) while slightly increasing in the sediment (100 mg/kg). Sedimentation appears to have favored the ingestion of Ce by P. corneus since at day 28, 104 ± 75 mg/kg and 60 ± 40 mg/kg (dry weight of digestive gland) were assimilated following a pulse versus multiple dosing. Sedimentation of Ce is related to the homo- or heteroaggregation kinetics of the ENMs. In our pilot study, the initial number of colloidal particles (clay, bacteria, etc.) was low (~105 particles/mL). Hence, the addition of 1 mg/L of ENMs corresponding to 6 × 105 particles/mL could lead to both homoaggregation of CeO2 ENMs and to their heteroaggregation with other particles.

According to the ENMs and the contamination scenarios, these experimental systems can operate with different physical and physico-chemical features (e.g. water quality and depth, sediment mineralogy and depth, current velocity, tidal reservoirs, etc.) and biota, owing to a high flexibility. This approach can be adapted in response to ecotoxicological benchmarks. This allows for testing of the Contaminant of Potential Ecological Concern (COPEC) that may be causing risk or adverse effects to biota at a site. Of course, outdoor mesocosms may provide a more representative simulation of natural ecosystems4243 and results obtained with our experimental design should be extrapolated with caution. Variations representative of natural aquatic environment induced by the wind, direct sunlight, drought, rain, spatial migration, flora etc. are not taken into consideration. However, these indoor mesocosms provide both a high degree of complexity in the system and a reduced uncertainty regarding the fate of the ENMs. This platform allows for simultaneous evaluation of physico-chemical properties and their relationship to the biological systems in situ as the ecosystem and ENMs evolve. This platform provides more realistic conditions for considering ENM transformations that have been lacking to date in many studies of ENM impacts on organisms and proven to be critical in assessing the potential hazards presented by ENMs.

The effects on organisms can therefore be evaluated under relatively controlled conditions that, like larger scale mesocosm studies, allow for evaluation of trophic transfer, maternal transfers, predator-prey interactions, competitive effects, resistance and resilience. However, the complexity of food web and the species diversity will be reduced. This will exacerbate the exposure and impacts compared to real natural environments. It will provide the upper limit of the effects expected in the mimicked ecosystems.

This experimental approach requires a strong integration of disciplinary researches. It offers physico-chemists, (micro)biologists, and ecologists the possibility of conceiving robust experiments to study the exposure and impacts of ENMs (low doses, chronic inputs) in a fashion that accommodates the control required to elucidate underlying mechanisms at various time and spatial scales. For instance, this experimental design can be used to simulate an ENMs runoff rain or vent loading (e.g. single pulse) versus a continuous point source discharge such as wastewater treatment plant or industrial discharge (e.g. multiple dosing). Such versatility allows for contrasting different environmentally relevant exposure scenarios as water exposure risks (e.g. multiple dosing) versus sediment exposure (e.g. single pulse dose) exposure conditions. By simultaneously creating representative conditions for environmental transformation and ecosystem exposure, the platform facilitates the integration of the complementary approaches into an environmental risk assessment model related to nanotechnologies based on reliable exposure and impact data.

Author Contributions

M.A. and A.T. conceived the experiment; M.T. and L.B. run the experiment and collected all the samples; M.T. analyzed the physico-chemical behavior of the ENMs; C.P. and L.B. analyzed the picoplankton and algae; C.S. and M.B. analyzed the bacterial diversity; B.E. designed the redox probes; A.T., M.A., L.B., C.P., E.A., M.T., J.I. collected sampled in the pond, M.A. wrote the manuscript with input and approval from all coauthors; J-Y.B., A.T., M.A., C.S., J.R., A.M., M.R.W., W.A. supervised the work.