Rapid mosaic brain evolution under artificial selection for relative telencephalon size in the guppy (Poecilia reticulata).

The mosaic brain evolution hypothesis, stating that brain regions can evolve relatively independently during cognitive evolution, is an important idea to understand how brains evolve with potential implications even for human brain evolution. Here, we provide the first experimental evidence for this hypothesis through an artificial selection experiment in the guppy (Poecilia reticulata). After four generations of selection on relative telencephalon volume (relative to brain size), we found substantial changes in telencephalon size but no changes in other regions. Further comparisons revealed that up-selected lines had larger telencephalon, while down-selected lines had smaller telencephalon than wild Trinidadian populations. Our results support that independent evolutionary changes in specific brain regions through mosaic brain evolution can be important facilitators of cognitive evolution.

INTRODUCTION

The vertebrate brain contains a number of morphologically and functionally distinct regions, linked through intricate connective systems, ranging from short local links between adjacent structures to connections between spatially distant regions through myelinated axons. Although the vertebrate brain is predominantly conserved with regard to the organization of the regions it contains, there is large variation among vertebrate species in the relative size and function of different regions (). One example is the disproportionately large neocortex in large-brained primates such as humans (). However, how has such variation evolved among the vertebrates, and what are the functional consequences of this variation? Despite more than a century of research, these remain to be important questions in the study of brain evolution (). One of the leading brain evolution theories, the mosaic brain evolution hypothesis (), posits that changes in individual regions may occur due to region-specific selection (e.g., according to explicit cognitive or sensory requirements), relatively independent of changes in other regions. If true, then this should result in a heterogeneous pattern of brain evolution (, ). This is in contrast to the concerted brain evolution hypothesis, which states that brain regions evolve in a coordinated manner, possibly due to underlying developmental constraints (, ).

The mosaic brain evolution hypothesis has received ample support mainly from comparative analyses at the interspecific level by means of examining how ecological factors correlate with specific alterations in neural anatomy across species or populations, e.g., (–). One important example of empirical evidence for this hypothesis was provided by Barton and Harvey (), where the authors analyzed the evolutionary changes in brain regions associated with specific functional units in primates and insectivores. The partly autonomous evolution of these functionally linked components with respect to other brain regions and overall brain size provided support for mosaic brain evolution in mammals. Despite a discernable bias toward mammalian studies, support for mosaic brain evolution in various fish and avian species has also been demonstrated (). For example, three-dimensional reconstructions of the brains of African mormyrid fishes that use active electro sensing revealed an enlarged cerebellum in comparison to outgroup species without such a system (). Another well-studied example concerns food caching in birds, where the ability to store food for later retrieval has been found to be positively correlated with hippocampus volume in species performing this cognitively demanding behavior (–).

Recently, intraspecific comparative analyses have complemented existing interspecific studies and shown positive correlations between song complexity and the brain regions governing singing behavior in song birds [HVC (high vocal center) and RA (robust nucleus of the archistriatum)] (). Intraspecific comparisons provide the additional benefit of examining potential phenotypic plasticity effects brought about by different environmental conditions [e.g., ()]. The potential for plastic changes in neural architecture is especially pronounced in teleost fish, given their prolonged neurogenesis throughout their entire lifetime [e.g., (–)]. That such plastic changes in specific brain regions occur in response to a multitude of factors supports the independent capacity of brain components.

Interspecific and intraspecific correlative comparative analyses form an important tool to investigate evolutionary patterns and generate hypotheses (, , ). An experimental approach, on the other hand, enables the targeted modification of specific components while simultaneously allowing the determination of correlated changes in brain morphology, behavior, and physiology [see, for instance, ()]. This is necessary to identify positive and negative genetic correlations, for instance, to reveal potential trade-offs with life history (, ). Experimental data are thus needed to understand the independent evolutionary potential of brain regions and its associated consequences. Artificial selection experiments on mice and fish have revealed that increases in relative brain size can occur quickly and yield important cognitive benefits [increased associative learning (–), more accurate mate preferences (, ), and more effective predator avoidance (, )] but also incur high energetic costs [lower offspring production (), reduced innate immune response (), and shorter intrinsic life span ()]. However, in line with the mosaic brain hypothesis, relative brain size is a rather crude measure of brain morphology, and evolutionary changes in brain morphology in wild populations are unlikely to target the entire brain (, ). Another fitting and important aspect of the mosaic brain hypothesis concerns the costly nature of neural tissue, which should favor evolutionary expansion of only functionally relevant structures, thereby reducing unnecessary energy expenditure (, ). To increase our understanding of brain morphology evolution and to go a step beyond brain size (), artificial selection experiments that target region evolution could be instrumental.

Here, we provide such an experimental test of the mosaic brain evolution hypothesis through an artificial selection experiment that targets relative telencephalon size (i.e., telencephalon volume in relation to the brain remainder) in guppies (Poecilia reticulata). The established role of the telencephalon as a cognitive center in fish (–) and other vertebrates (, ) makes this region an appealing target of artificial selection with highly general implications, especially given the potential for future tests of the implicated costs and cognitive benefits. We chose the guppy as the model for the experiment because it is a live-bearing teleost species with a well-studied, interesting, and variable ecology, e.g., (–), it has a relatively short generation time (), and large populations can be kept in the laboratory with ease. Specifically, we aimed to increase and decrease the relative size of the telencephalon in relation to the brain remainder while examining for concomitant changes in other aspects of brain morphology. We also investigated sex-specific effects. As a measure of potential reproductive costs () associated with changes in telencephalon size, offspring production, one of the best proxies for fitness in the guppy, was quantified during selection. Last, we investigated how the artificial selection lines compared to wild populations by comparing their relative telencephalon size against a large dataset spanning across 16 Trinidadian guppy populations.

RESULTS

After four generations of selection for relative telencephalon size (telencephalon volume in relation to the volume of the brain remainder), there was an average difference of 10.1% between the up-selected and down-selected females and an average difference of 9.5% in males. There was an overall significant difference between selection treatments for both females and males [estimated differences between up-selected and down-selected lines: ß, presented with 95% credible intervals (CI); females: ß = 0.046 [0.0094; 0.089], PMCMC = 0.044; males: ß = 0.022 [0.0052; 0.041], PMCMC = 0.028] (Fig. 1 and Table 1). Comparisons to the unselected control lines revealed nonsignificant trends for selection to proceed in an asymmetrical manner in females (up-selected versus control line: ß = 0.031 [−0.0023; 0.065], PMCMC = 0.052; down-selected versus control line: ß = −0.014 [−0.077; 0.064], PMCMC = 0.50), but no such pattern was detected in males (up-selected versus control line: ß = 0.030 [−0.017; 0.098], PMCMC = 0.18; down-selected versus control line: ß = −0.0094 [−0.064; 0.050], PMCMC = 0.66). The effects of selection were consistent also when controlling for body length (table S1), suggesting that relative telencephalon size, both in relation to the brain remainder and body size, was affected by the directional selection procedure.

Fig. 1.

Response to artificial selection on relative telencephalon size in both females and males.

(A) Relative telencephalon size in response to directional selection over four generations. P indicates the starting parental population. Depicted are the means and SE values (SEM) for standardized residuals of telencephalon volume regressed on the volume of the brain remainder (total brain volume minus telencephalon volume) within each generation and replicate. The figure shows the selection response across the three replicates, with the left and right panels separately illustrating the response in females and males, respectively. (B) Overall change in telencephalon size in generation F4 females and males. Plotted on the y axis are the raw data of telencephalon volume (log-transformed), with brain remainder (log-transformed) on the x axis. Different selection treatments are indicated by the different colored lines and points (down-selected line, yellow; unselected control line, black; and up-selected line, blue).

Table 1.

Model output for relative telencephalon size (relative to the remainder of the brain) in generation F4 females and males, presented with mean estimates and 95 % CI.

The intercept is set to the intercept of the regression of log telencephalon size on log brain remainder (total brain volume minus telencephalon volume) for the down-selected line in females. Mean estimates are indicative of differences between the intercept (down-selected females) and the specified variable (conforming to the default contrast matrix in R). NA, not applicable. ***P < 0.001, **P < 0.01, *P < 0.05.

Adult generation F4
Fixed terms	Estimate (ß)	Lower CI	Upper CI	P MCMC
Intercept	−0.703	−0.901	−0.556	0.004**
Brain remainder	0.892	0.769	1.02	<0.001***
Sex male	−0.0494	−0.112	0.0213	0.144
Unselected control line	0.0168	−0.0219	0.0551	0.272
Up-selected line	0.0457	0.00909	0.0848	0.030*
Rest of brain:sex males	−0.0977	−0.262	0.0624	0.282
Sex males:unselectedcontrol line	−0.00629	−0.0253	0.0110	0.512
Sex males:up-selected line	0.000940	−0.0172	0.0198	0.914
Random terms	Variance	Lower CI	Upper CI	P
Residual	0.00188	0.00166	0.00214	NA
Selectiontreatment:replicate	0.000490	2.30 × 10−6	0.00160	NA
Replicate	0.0633	4.87 × 10−12	0.155	NA

Results presented hereafter are based on comparisons between down-selected and up-selected lines only, unless otherwise stated (for a full summary of results, see Table 2, table S2, and figs. S1 and S2). No significant differences in any of the other brain regions assessed were detected (Table 2, table S2, and figs. S1 and S2). However, brain size (controlling for body length) was found to be larger in the up-selected lines (ß = 0.023 [0.0073; 0.042], PMCMC = 0.022). Further analysis across each sex revealed that this difference in relative brain size was mainly driven by effects in males (males: ß = 0.022 [0.0052; 0.041], PMCMC = 0.028; females: ß = 0.025 [−0.0070; 0.058], PMCMC = 0.098) (table S3). Exclusion of the telencephalon (total brain volume minus telencephalon volume) showed a nonsignificant trend for a difference between selection treatments (ß = 0.017 [−0.0023; 0.036], P = 0.074). Together with the lack of difference in any of the other brain regions, this supports that the trends toward larger overall brain size in the up-selected male lines were mainly the result of the significantly larger telencephalon in these lines. Realized heritabilities of relative telencephalon size were variable but congruent between sexes when comparing the up- and down-selected lines against the control lines (up-selected females: 1.03 [−0.66; 4.35], down-selected females: 0.44 [−1.37; 2.73], up-selected males: 1.42 [−1.78; 7.00], and down-selected males: 0.0078 [−1.92; 1.50]).

Table 2.

Model output for overall changes in brain morphology in generation F4 adults in response to selection for relative telencephalon size, presented with mean estimates and 95% CI.

The intercept is set to the intercept of the regression of log brain region size on log brain remainder (brain volume minus volume of region of interest) for the down-selected line in females. The mean values presented in the table represent differences between the intercept (down-selected females) and the specified variable (conforming to the default contrast matrix in R). ***P < 0.001, **P < 0.01, *P < 0.05.

	Estimate (ß)	Lower CI	Upper CI	P MCMC
Overall brain
Intercept	−0.677	−0.898	−0.445	0.002**
Standard length	0.897	0.776	1.02	<0.001***
Sex male	−0.364	−0.598	−0.126	0.004**
Unselected control	0.00756	−0.00914	0.0260	0.278
Up-selected line	0.0233	0.00730	0.0423	0.022*
Standard length:sex male	0.232	0.0357	0.408	0.016*
Sex male:unselected control	0.000502	−0.0110	0.0129	0.926
Sex male:up-selected line	−0.00136	−0.0139	0.0107	0.842
Optic tectum
Intercept	0.00619	−0.0108	0.0701	0.138
Rest of brain	0.656	0.575	0.737	<0.001***
Sex male	−0.0251	−0.00217	0.0491	0.076
Unselected control	−0.00141	−0.0139	0.0162	0.846
Up-selected line	−0.00606	−0.0225	0.00969	0.370
Rest of brain:sex male	−0.137	−0.246	−0.0256	0.010**
Sex:unselected control	0.00254	−0.0104	0.0168	0.676
Sex:up-selected line	−0.00488	−0.0191	0.00988	0.518
Cerebellum
Intercept	−0.740	−0.858	−0.548	<0.001***
Rest of brain	1.13	0.970	1.29	<0.001***
Sex male	−0.208	−0.294	−0.131	<0.001***
Unselected control	0.00678	−0.0625	0.0539	0.758
Up-selected line	−0.00500	−0.0625	0.0543	0.790
Rest of brain:sex male	0.264	0.0756	0.492	0.010*
Sex:unselected control	−0.0116	−0.0326	0.00984	0.292
Sex:up-selected line	0.00844	−0.0118	0.0316	0.456
Dorsal medulla
Intercept	−0.725	−1.03	−0.454	0.010**
Rest of brain	0.628	0.416	0.817	<0.001***
Sex male	−0.153	−0.262	−0.0368	0.014*
Unselected control	−0.00234	−0.0423	0.0333	0.854
Up-selected line	0.00582	−0.0331	0.0456	0.700
Rest of brain:sex male	0.143	−0.118	0.404	0.312
Sex:unselected control	0.00329	−0.0236	0.0324	0.842
Sex:up-selected line	0.00769	−0.0217	0.0361	0.654
Hypothalamus
Intercept	−0.714	−0.834	−0.610	0.008**
Rest of brain	0.577	0.423	0.751	<0.001***
Sex male	−0.0413	−0.132	0.0529	0.396
Unselected control	0.00802	−0.0191	0.0395	0.542
Up-selected line	0.0110	−0.0163	0.0408	0.442
Rest of brain:sex male	−0.291	−0.517	−0.0966	0.006**
Sex:unselected control	0.00740	−0.0142	0.0312	0.651
Sex:up-selected line	0.0216	−0.00224	0.0449	0.074
Olfactory bulbs
Intercept	−2.27	−2.73	−1.88	<0.001***
Rest of brain	1.02	0.657	1.34	<0.001***
Sex male	−0.0580	−0.280	0.144	0.600
Unselected control	−0.0170	−0.0846	0.0504	0.514
Up-selected line	0.0290	−0.0440	0.103	0.312
Rest of brain:sex male	−0.202	−0.668	0.227	0.386
Sex:unselected control	0.0640	0.0165	0.110	0.006**
Sex:up-selected line	−0.0149	−0.0639	0.0307	0.542

A previous study comparing the brain morphology of wild-caught and laboratory-reared guppies found that the latter had smaller telencephalon and optic tectum in relation to the former after just a single generation (). Here, we compared the mean telencephalon size between selection treatments and 16 Trinidadian wild populations in females and found that the unselected control lines had a telencephalon size that most closely resembled that found in natural populations, while up-selected and down-selected lines had larger and smaller relative telencephalon size than natural populations (Fig. 2). Upon closer examination, it was also found that selection over four generations resulted in a comparable range of relative telencephalon sizes in our selected sample (range, −0.76 to 1.3) in comparison to natural populations (range, −1.2 to 0.67), contradicting inbreeding issues in this trait over several generations of selection in our experiment (fig. S3).

Fig. 2.

Female relative telencephalon size after three rounds of selection (generation F4) in comparison to wild fish obtained from 16 wild populations in Trinidad.

Plotted on the y axis are the estimated marginal means and 95% confidence intervals, obtained from standardized residuals of telencephalon volume regressed on brain remainder (total brain volume minus telencephalon volume). Pairwise comparisons were performed with respect to the wild populations, and P values were calculated following Holm’s adjustment for multiple comparisons. **P < 0.01; *P < 0.05.

As there is a strong correlation between female body size and reproductive output, we first tested for any difference in standard length across selection treatments but found no such effect (up-selected versus down-selected: ß = 0.0070 [−0.019; 0.036], PMCMC = 0.50). The number of offspring produced during the first parturition was thereafter compared across selection treatments to assess for potential reproductive costs implicated in the selection for a larger telencephalon. There was no difference in the number of offspring produced across selection treatments (up-selected versus down-selected: ß = 0.19 [−0.21; 0.59], PMCMC = 0.28; Fig. 3).

Fig. 3.

Reproductive output, as assessed by the number of juveniles produced during the first parturition, over the different selection lines.

Error bars represent SEM. No significant differences were detected in generation F4 (P > 0.2).

DISCUSSION

Our results show that the relative size of a brain region can evolve independently and quickly under strong directional selection. Hence, our results provide experimental support for the mosaic brain evolution hypothesis (, ). This means that brain regions can display rapid and largely independent evolutionary changes, which further imply that selection can alter individual brain regions in response to directed selection pressures, one of which could be specific cognitive demands imposed by the environment. The rate of evolution shown here is similar to that seen when brain size was the target of selection (); a 9% difference in brain mass between the large- and small-brain lines was found after two generations of artificial selection and a 15% difference after five generations of selection (). That such rapid changes in brain morphology, both in terms of overall brain size and brain regions, can occur in response to directional selection is consistent with the assumed high evolvability of the brain ().

In contrast to the aforementioned artificial selection experiment on relative brain size in guppies, which found a clear reduction in fecundity (i.e., lower offspring number) in the large-brain lines (), we did not find any evidence for any reproductive trade-off with telencephalon size. Together, these results indicate that mosaic brain evolution of key regions associated to cognitive demands may be an energy-effective and thus potentially favored way to reduce life history costs during cognitive evolution. As mentioned previously, the comparative and experimental evidence for the costs associated with increased brain size is substantial. However, little is known about the potential costs associated with variation in brain region size. A potential additional route for future research is therefore to investigate whether life history costs are associated also with vertebrate brain region variation in comparative analyses.

We found no evidence for strong associations in the size between different brain regions. Only the individual region targeted by our selection regime, the telencephalon, changed significantly in size during the experiment. This matches a previous study examining phenotypic and genotypic correlations between brain regions in three-spined stickleback (Gasterosteus aculeatus), where Noreikiene et al. () found that brain regions shared relatively low genetic correlations. Moreover, in a large-scale quantitative genetic analysis study comprising approximately 10,000 mice, different brain regions were found to be regulated by distinct nonoverlapping loci, allowing for the selection of individual brain components (). The lack of a link between telencephalon size and other brain regions also speaks against prominent trade-offs between investments into separate regions. This is somewhat unexpected, given that neural tissue is one of the most energetically costly tissues (, , , ). Our experimental design, which targeted relative telencephalon size in relation to the brain remainder, should also have been effective in revealing potential developmental trade-offs with other brain regions. Again, the lack of evidence for any trade-off in this context may stem from high independence and the low genetic correlations between the sizes of different brain regions. Although our setup was designed to select for telencephalon volume in relation to brain remainder, another potentially important predictor of cognitive ability is the relative size of the telencephalon in relation to body size. It is therefore worth noting that the difference in telencephalon size between selection treatments was significant in relation to both brain remainder and body size, demonstrating the robustness of the selection procedure and generality of our results. Note that our animals were kept under benign laboratory conditions, which may have masked potential energetic trade-offs. To fully understand the intricate evolutionary associations between different brain regions will require more work (e.g., rearing of individuals under different food restriction treatments to mimic natural conditions), ideally also into the interconnectivity between regions (). The increased telencephalon size in the up-selected lines led to an increase in relative brain size but mainly in males (the trend, albeit not statistically significant, was in the same direction also in females; fig. S2 and table S2). Substantial changes in the size of specific regions must of course affect also the overall brain size unless there are trade-offs with other regions (). However, comparative analyses have demonstrated that changes in some regions, especially neocortex size and cerebellum size, are linked to changes in brain size [e.g., (, , )]. Although the effect was mostly evident in males after four generations of selection and apparently mainly driven by the larger telencephalon in these lines, our results provide support for a similar pattern also at the intraspecific level. While more work is clearly needed to unveil the developmental pathways and functional implications of the link between telencephalon size and brain size [e.g., ()], we propose that mosaic brain evolution can be an important first step also toward larger brain size.

Comparative studies often find support for a combination of mosaic and concerted evolution (, , ). How do such patterns go together with our experimental results? Artificial selection experiments reveal what is possible under strong directional selection and do not necessarily reflect exactly the patterns and processes that are occurring in wild populations. Hence, while our experiment shows that telencephalon size can evolve highly independently, many opposing selection pressures such as brain cavity space constraints () and/or developmental and energetic constraints (, , ) may impede the signatures of selection for mosaic brain evolution in wild populations. Selection on cognitive ability in wild populations may also target other behavioral and morphological aspects than brain region size (). One possible reason why we found it harder to decrease, rather than increase, the size of the telencephalon could be limitations due to cognitive thresholds, whereby a minimum size requirement of the telencephalon is necessary. As mentioned earlier, the teleost telencephalon has known implications in a wide range of cognitive processes, including, but not limited to, spatial learning and fear conditioning response. It is therefore conceivable that a fundamental size is required for normal daily functioning, hence curbing the extent to which down-selection can progress. This is in accordance with Haller’s rule, whereby smaller animals tend to have relatively larger brains due to the minimum absolute brain size required for performing basic cognitive functions ().

Realized heritabilities in relative telencephalon size were substantial at least in the up-selected treatments in both sexes but showed high variation. This implies that changes in telencephalon size rests on an important heritable genetic background. While heritability in overall brain volume has been found to be high across species [e.g., humans (, ), rhesus monkeys (), and baboons ()], genetic influences on brain region sizes have been shown to vary considerably (–). In the previous artificial selection on brain size in guppies, realized heritability of relative brain size was found to be relatively high (), suggesting that overall brain size is under strong genetic control. This is congruent with results from human studies where brain volume is substantially influenced by genetic factors, while heritability of individual brain components may vary throughout the life span (). The measure of heritability used here estimates the proportion of total variance that can be attributed to genetic effects (). The fact that we found evidence for considerably higher heritability estimates in the up-selected lines could once again be related to the aforementioned cognitive threshold theory, whereby the requirement for a minimum telencephalon size outweighs genetic influences. Although substantial, the values of realized heritability presented here should be interpreted with some caution. For instance, that the values of realized heritability sometimes lay outside of the normal range of 0 to 1 may implicate effects of phenotypic plasticity. Given that both brain size and brain region sizes are highly plastic, it is possible that environmental effects may result in a range of region sizes outside of the norm. In addition, it is worth noting that measurement error is generally larger for brain regions than, for instance, for overall brain size.

Last, our comparison of the telencephalon selection lines in relation to 16 recently sampled wild populations of guppies shows that the control lines are most similar to the wild populations, while up-selected and down-selected lines have larger and smaller telencephali than the wild populations, respectively. Although telencephalon size in our selection lines mostly falls within the range of the wild populations, telencephalon size in one specific replicate of the up-selected line was larger in comparison to the natural range. These results are consistent with the following: (i) Ample genetic variation still exists in the wild-type laboratory strain of guppies used as the basis for this selection experiment (, ), and (ii) the selection procedure used here has mainly acted within the natural range of telencephalon size variation. Further selection for additional generations will be undertaken to test whether artificial selection on a separate brain region can shift its phenotype even further beyond the naturally occurring levels, similar to what has been done for other morphological traits on numerous occasions (–).

In conclusion, our results show that brain morphology can evolve rapidly in a highly independent fashion under strong directional selection. Our study thus provides experimental support for the mosaic brain evolution hypothesis at the intraspecific level, and we propose that mosaic evolution can be an important facilitator of cognitive evolution at the intraspecific level.

MATERIALS AND METHODS

Artificial selection for relative telencephalon size

We selected on telencephalon size relative to the size of the rest of the brain, because the alternative, to select on relative telencephalon size against body size, was likely to create larger and smaller relative brain size due to the strong positive correlation between telencephalon size and brain size in vertebrates (). Guppies (P. reticulata) originating from a high-predation population in Trinidad and subsequently kept in large populations in the laboratory since 1998 were used as breeding stock for the starting population. The fish that formed the base populations for these selection lines were housed in large mixed sex groups in three 200-liter aquaria. One hundred and twenty breeding pairs were netted out and housed in 3-liter aquaria with light-colored gravel, constant aeration, freely floating live or plastic plants, and >4 water snails (Planorbis sp.) to consume organic waste. The laboratory was maintained at 25° ± 2°C with a 12-hour light, 12-hour dark schedule. Fish were fed 6 days a week with a diet of flake food and freshly hatched brine shrimp. Given the cannibalistic nature of female guppies, a plastic net with mesh measuring 0.4 cm by 0.4 cm was placed at the front of the tank, functioning as a safe zone for fry while restricting the access of adult females. Tanks were scanned once every other day for any newborn offspring, which were removed and placed in similar tanks and kept to a maximum of six individuals per tank. Among these offspring, males were separated from females at the first sign of sexual maturity (i.e., visual detection of gonopodium development) and housed in groups of three to four individuals in similar 3-liter aquaria.

At 140 ± 7 (means ± SD) days of age, 450 of these fish were used to establish the starting population (F0) of three independent experimental replicates of 75 breeding pairs each in similar tanks as described above. Since quantification of brain morphology could not be performed in live fish, we allowed the pairs to produce at least two clutches or at least 12 juveniles per pair, before being euthanized for brain morphology quantification. Breeding pairs were euthanized in a water bath containing an overdose of benzocaine (0.4 g/liter) and fixed with 4% formalin in buffered phosphate-buffered saline (PBS) for 7 days. The samples were then washed twice in PBS and stored at 4°C awaiting dissection.

Standard length of all fish was recorded to the nearest 0.01 mm, measured from the tip of the snout to the end of the caudal peduncle using digital calipers. Whole brains were dissected out from each fish under a dissecting microscope (Leica MZFLIII), and each individual brain was photographed from four separate views with a digital camera (Leica DFC 490). On the basis of these separate images, the length, width, and height of six major brain regions (i.e., telencephalon, optic tectum, cerebellum, dorsal medulla, hypothalamus, and olfactory bulbs) were measured using ImageJ software (fig. S4) (). The volume of each of these brain regions was then determined using the ellipsoid model (, )

Correspondingly, total brain volume was derived as the sum of the six measured brain regions. To select for relative telencephalon size, we first extracted the residuals from sex-specific regressions of telencephalon volume on the volume of the brain remainder excluding the telencephalon and standardized these to a mean of 0 and an SD of 1. On the basis of the sum of the standardized residuals of each pair, offspring from the top and bottom 15 pairs (i.e., top and bottom 20%) were chosen to form the next generation (F1) of up-selected and down-selected lines, respectively, across three replicates. Additional offspring from 15 random pairs (F0) were likewise used to form the control lines (see Fig. 4 for a detailed schematic outline of the selection procedure). The procedure was largely adapted from a previously successful artificial selection experiment for relative brain size in guppies (). Each experimental replicate and selection regime combination was treated independently, thus giving a total of nine distinct populations (three replicates of up-selected, down-selected, and control lines, respectively). Within each of the selection lines for each replicate, two females and two males from each of the selected F0 pairs were used to form 30 breeding pairs for the next generation (F1), resulting in a total of 270 pairs. Again, the top and bottom 15 pairs in terms of relative telencephalon size were chosen for up-selected and down-selected lines, while the control lines were paired randomly in each of the following generations. To prevent inbreeding, breeding pairs never consisted of full siblings.

Fig. 4.

Artificial selection procedure for relative telencephalon size.

Different selection lines are indicated by the different colored arrows [i.e., blue for up-selection (U), black for control line (C), and orange for down-selected (D)] within each generation, and three independent replicates of 30 breeding pairs for each selection line were set up.

Similar to the previous generation, F1 breeding pairs were allowed to produce at least two clutches or at least 12 juveniles before whole brains were dissected out and measured. The selection proceeded in an identical manner as generation F0, first by ranking the parents according to the sum of parental residuals and using the respective offspring to propagate the subsequent generations (F2, F3, and F4).

Brain morphology of wild fish populations

To compare the brain morphology of the telencephalon selection lines to that of wild fish, the brain morphology of wild indigenous female guppies (n = 187) that were obtained from 16 different sites in Trinidad () was quantified in an identical manner as described above. These measurements could thereafter be used as a reference to assess the rate of brain evolution in our selection lines in relation to wild populations.

Reproductive costs

The development and maintenance of neural tissue incur high energetic costs (, ). To examine possible reproductive trade-offs involved in evolving a larger telencephalon, we assessed the possible costs in terms of reproductive output across selection lines. For this, we recorded the total number of juveniles and broods produced by all breeding pairs as a measure of reproductive success.

Statistical analysis

All statistical analyses were conducted in R statistical software v3.5.1 (). We adopted a Bayesian approach in the R package MCMCglmm () to analyze for selection response/effects across lines, with flat priors for the fixed effects and locally uninformative priors for the random effects. We ran each model with 1.65 × 106 iterations, with a thinning interval of 1500 and a burnin of 1.5 × 105, resulting in an effective sample size of at least 1000. All autocorrelations were within the range of −0.1 to 0.1 and were hence deemed acceptable. To avoid statistical confounds resulting from the inclusion of the region being analyzed, we fitted telencephalon volume (log-transformed) as the response variable against the brain remainder excluding the telencephalon (also log-transformed) as the covariate. Our model consisted of the following fixed effects: selection treatment (down-selected, control line, and up-selected), brain remainder, and sex, with two-way interactions between brain remainder and sex and between sex and selection treatment. Replicate and the interaction between selection treatment and replicate were included as random factors. The intercept term was set to the regression of telencephalon size on brain remainder for females in the down-selected line.

To test whether selection results stemmed from an indirect selection on overall brain size, we fitted a similar MCMCglmm model with standard length, sex, selection treatment, and two-way interactions between standard length and sex and between sex and selection treatment as fixed effects. Random factors included replicate and the interaction between selection treatment and replicate. Analogous models were used to examine for potential correlated selection between the size of telencephalon and other brain regions in each of the five other unselected regions, i.e., optic tectum, cerebellum, dorsal medulla, hypothalamus, and olfactory bulbs (table S4).

Realized heritability was estimated as the ratio between the mean response to selection and the mean selection differential for each generation, for “up” and “down” selected lines (in comparison to the control lines), and males and females separately. We ran separate models for each generation and sex combination to estimate the response to selection (calculated as the difference between the up or down lines as compared to the control lines). For the models estimating the selection differential from the means of the selected individuals and the population mean (the mean of the guppies in the selection regimes for either smaller or larger telencephalon), we ran models specific to each generation and sex combination. The models estimating the response to selection contained the explanatory variables selection (up or down and “control”) and log brain remainder (total brain volume minus volume of the telencephalon), with replicate and interaction between selection treatment and replicate as random effects. The models estimating the means of the selected individuals contained the explanatory variables selected (“yes” or “no”), log brain remainder as a covariate, and replicate line as random effect. To evaluate main effects independently of the covariate, log brain remainder was standardized to a mean of 0 and an SD of 1. All models were fit using MCMCglmm as previously described. The full posterior distribution was included in all analysis.

Trade-offs, in terms of reproductive output, were analyzed by comparing the number of offspring produced during the first parturition for generation F4. Since female body size is known to be correlated with fecundity (), standard length was included as a covariate in the model. Given the distribution of our response variable (i.e., number of offspring), we fitted a Bayesian model with a Poisson distribution. Hence, our model consisted of the following fixed effects: standard length, selection treatment, and the interaction between the two. Random factors included replicate and the interaction between selection treatment and replicate.