Female brain size affects the assessment of male attractiveness during mate choice.

Mate choice decisions are central in sexual selection theory aimed to understand how sexual traits evolve and their role in evolutionary diversification. We test the hypothesis that brain size and cognitive ability are important for accurate assessment of partner quality and that variation in brain size and cognitive ability underlies variation in mate choice. We compared sexual preference in guppy female lines selected for divergence in relative brain size, which we have previously shown to have substantial differences in cognitive ability. In a dichotomous choice test, large-brained and wild-type females showed strong preference for males with color traits that predict attractiveness in this species. In contrast, small-brained females showed no preference for males with these traits. In-depth analysis of optomotor response to color cues and gene expression of key opsins in the eye revealed that the observed differences were not due to differences in visual perception of color, indicating that differences in the ability to process indicators of attractiveness are responsible. We thus provide the first experimental support that individual variation in brain size affects mate choice decisions and conclude that differences in cognitive ability may be an important underlying mechanism behind variation in female mate choice.

INTRODUCTION

Mate-quality recognition is central in the study of sexual selection (, ), and studies of intersexual selection commonly assume an association between the preference for a mate and the quality of that particular mate. Preference for high-quality mates will lead to important fitness benefits and positive reinforcing selection for the preferred traits associated with quality (). However, in contrast to the common theoretical assumption that preferences for traits associated with quality in the opposite sex should be invariant, variation in trait preference is frequent in nature (, ). For instance, extrinsic factors, such as social information, have been shown to influence mate preference and sometimes even lead to seemingly suboptimal mate choice across fish (, ), amphibians (, ), and birds (). Intrinsic factors, such as condition, experience, and cognitive ability, are also important for decision-making, and previous studies suggest an important role for these factors in mate choice variation (, –).

Cognitive ability is broadly defined as the acquisition, processing, retention, and use of information (). Cognitive ability is a likely prerequisite for accurate mate choice decisions because the choosy sex needs to integrate complex information to compare individuals with differences in one or several sexual signals, such as sexual traits or complex displays. Variation in cognitive ability is often directly influenced by brain size (, ) and has been suggested to strongly influence variation in sexual behavior at the individual and species levels (, ). Therefore, the role of brain size in mate preferences could have important implications for the evolution of mate choice and sexually selected traits (–). Despite this, the association between brain size and preference for sexually selected traits remains empirically unexplored.

Here, we experimentally test the role of brain size in female mate choice, using female guppies (Poecilia reticulata) selected for divergence in relative brain size. Previous tests in these artificially selected lines have demonstrated higher cognitive ability in the large-brained fish (, , ). The differences observed between the brain size lines are not due to hitchhiking of deleterious alleles in the selection process. There are no differences in swimming performance () or condition index (), and some assays even show physiological advantages in the small-brained lines, such as better immune response (), faster early juvenile growth (), and higher fecundity (). Female mating preferences for males with larger amounts of ornamental coloration (, ) and larger tails () are well established across wild populations in the guppy. The expression of these traits in male guppies is tightly linked to foraging ability () and physiological health (, ). Therefore, choosing to mate with males that have greater expression of these traits is highly likely to confer important fitness benefits to females, for instance, by passing on beneficial genes from these males to their offspring (). If the demonstrated differences in cognitive ability influence the ability to process information on attractiveness when comparing between males, we would expect large-brained females to more often choose attractive versus unattractive males. To test this prediction, we measured the preference of large-brained, small-brained, and wild-type female guppies when choosing between attractive (highly colorful with large tails) and unattractive (dull-colored with small tails) males. We show that large-brained and wild-type females show a clear preference for attractive males, whereas small-brained females show no preference for either attractive or unattractive males. Moreover, we show that females from the different lines do not differ in their ability to perceive colors, based on both visual performance examination and expression of multiple opsin genes, suggesting that differences in mate preference arise during the cognitive processing of male attractiveness cues and not only at the early visual acquisition phase.

RESULTS

Preference for attractive versus unattractive males

We quantified female preference in a standard dichotomous choice setup (see Methods for details). During the first 10 min of the trial, when overall preference levels were highest (Fig. 1B and information S1), we found an overall effect of brain size in the preference for attractive males [means ± SE: small-brained: −0.07 ± 0.08, large-brained: 0.24 ± 0.09, wild-type: 0.36 ± 0.12; LMMpreference: brain size: χ2(1) = 7.404, P = 0.006]. Both large-brained and wild-type females displayed significant preference for attractive males compared to unattractive males (Fig. 1). Small-brained females showed no preference for either attractive or unattractive males over the whole duration of the trial (Fig. 1). Preference for attractive males by large-brained and wild-type females differed significantly from small-brained [LMMpreference: small-brained versus large-brained: χ2(1) = 6.952, P = 0.008; LMMpreference: small-brained versus wild-type: χ2(1) = 8.660, P = 0.003]. There was no significant difference in average preference for attractive males between wild-type and large-brained females [LMMpreference: large-brained versus wild-type: χ2(1) = 0.662, P = 0.414].

Fig. 1

Preference for attractive males.

A standardized preference ratio was calculated as the difference in time spent with each male, divided by the total amount of time in any of the choice areas, in a dichotomous choice test performed in small-brained (n = 36), large-brained (n = 36), and wild-type (n = 16) females. The preference ratio takes values between −1 (all time spent with an unattractive male) and 1 (all time spent with an attractive male). (A) Average preference ratio in the first 10 min of the trial. (B) Independent average preference ratio obtained in the three time periods of 5 min that formed the whole trial.

We verified whether the observed differences could be attributed to differences in how females appraised information on the offered males. First, the observed differences could not be attributed to differences in sexual activity levels because females from all groups spent, on average, the same amount of time outside of the defined choice areas containing male pairs [means ± SE: small-brained: 203 ± 19 s, large-brained: 184 ± 26 s, wild-type: 243 ± 34 s; LMMno choice: brain size: χ2(2) = 3.015, P = 0.221] and the total time spent in front of attractive males was significantly higher in large-brained females than in small-brained females [means ± SE: small-brained: 200 ± 28 s, large-brained: 285 ± 38 s; LMMtotal time: small-brained versus large-brained: χ2(1) = 4.870, P = 0.027]. In addition, analyses of absolute preference irrespective of male attractiveness, controlling for individual activity and choice behavior, showed no differences between small-brained, large-brained, and wild-type females (Fig. 2). Second, we found no evidence that large- and small-brained females differed in their opportunity to gather information about unattractive and attractive males because the ratio of visits between choice areas (difference in visits to each choice area, standardized by the total visits) was not different between brain sizes [means ± SE: small-brained: −0.02 ± 0.06, large-brained: 0.08 ± 0.06; LMMvisits ratio: small-brained versus large-brained: χ2(2) = 1.981, P = 0.159].

Fig. 2

Partner preference irrespective of male attractiveness.

Comparison of expected versus observed absolute standardized preference ratios in the dichotomous choice test performed in small-brained (n = 36), large-brained (n = 36), and wild-type (n = 16) females. The absolute preference takes values between 0 (same time spent in each male choice area) and 1 (all time spent in one of the two male choice areas). Vertical lines denote the observed mean absolute preference for the three up- and down-selected lines artificially selected for relative brain size and for wild-type females. To evaluate whether females showed a preference stronger than chance, we compare the mean preference in each group against a simulated null distribution, which constrained the number and durations of visits. Higher preference values than the null distribution therefore indicate that females visited one male disproportionally often and/or for a disproportionally long time. All groups show a highly significant preference for one of the two males (all P values lower than 0.001).

Color perception

We evaluated the possibility that the observed differences in female preference were caused by underlying differences in the way females of the selection lines perceive male color. We initially estimated female sensitivity to orange colors by measuring their optomotor response (see Methods for details). These experiments are based on the fact that fish orient their position using objects as reference and thus rotate to follow a rotating stimulus, usually bands of alternating color. By decreasing or increasing the saturation intensity of the color bands and tracking fish movement, it is possible to test whether fish are able to discern between the different color bands at different levels of contrast. In the first three rotating stimuli, with a higher saturation difference between red and green bands, of the optomotor response test, all female groups could perceive the contrast between bands and spent, on average, significantly more time performing optomotor behaviors during the rotational phase than during the static phase (mean of the differences: 39.130 s, t50 = −27.689, P < 0.001) (Fig. 3). We then decreased the saturation intensity of the red and green stripes used as stimuli and observed no differences among the female groups in the average time spent performing optomotor behaviors during the rotational phase in relation to the static phase (mean of the differences: 0.655 s, t50 = 0.454, P = 0.651) (Fig. 3). In the final rotating stimuli round, with maximum intensity of red and green bands, all female groups again spent significantly more time performing optomotor behaviors during the rotational phase, with similar magnitude as in the first round with high band contrasts (mean of the differences: 38.953 s, t50 = −20.525, P < 0.001). Thus, the lack of difference observed in the rotating stimuli with lower saturation intensity in green and red bands was not likely a consequence of habituation to the test or caused by stress but rather indicates that the females were less able to perceive the contrast between the color stripes. We found that wild-type, small-brained, and large-brained females did not show differences in their sensitivity to color contrasts in either high saturation contrasts [means ± SE: small-brained: 46.72 ± 1.79 s, large-brained: 49.97 ± 1.73 s, wild-type: 47.54 ± 1.84 s; LMMoptomotor response: brain size: χ2(2) = 1.84, P = 0.398] or low saturation contrasts [means ± SE: small-brained: 20.28 ± 2.48 s, large-brained: 17.13 ± 2.40 s, wild-type: 19.68 ± 2.54 s; LMMoptomotor response: brain size: χ2(2) = 0.774, P = 0.679] (Fig. 3). Post hoc comparisons among groups did not show any differences for any of the multiple separate band contrasts tested (table S2).

Fig. 3

Average total optomotor response.

Small-brained (n = 18), large-brained (n = 18), and wild-type (n = 16) females were subjected to a 60-s rotating stimuli of different saturation contrasts of green and red bands (rotation phase), after 60 s of acclimation, where green and red bands that will rotate next were presented to the fish without motion (static phase). No significant differences were observed between groups for any saturation contrast level.

To evaluate another component of the visual system that could have a strong effect on color perception, we measured the expression of opsin genes in females tested for male preference. Opsins are the proteins in the retina that mediate the initial steps of photon capture that lead to vision. Differences in opsin expression have been linked to differences in the light environment occupied by different species (), and opsin expression has also been linked to female preferences in guppies (). We found no significant expression differences between large- and small-brained females for any of the 10 guppy opsin genes (table S3A). We also measured the expression of opsin genes in males from each group and confirmed previous results showing significant sex differences in the expression of short- and long-wavelength opsins (table S3B) (), indicating that our expression assay contains sufficient sensitivity. Together with the results from the optomotor response experiment, our data thus indicate that large- and small-brained females do not differ in these two aspects of the visual system and are therefore unlikely to differ in their physiological ability to perceive the attractiveness in males.

DISCUSSION

Our study shows that large- and small-brained female guppies differ in their preference for male traits. It is unlikely that these effects are driven by differences in color perception because large- and small-brained females showed no significant differences in optomotor response or opsin gene expression. Together, our results therefore suggest that brain size and cognitive ability may play an important role in mate-quality assessment during mate choice.

Cognitive processes are likely involved in effectively processing information to compare and discriminate between potential mates (, ). Here, females could not observe both males simultaneously and needed to remember both males to make comparisons between them. In natural populations, female guppies often face sequential encounters with different males with varying time lags (). These encounters may impose cognitive challenges to females (, ). Moreover, past social interactions with males and other females in their natural environment would likely affect future assessments of male attractiveness. For instance, mate choice copying from experienced females or previous sexual encounters tend to shape future mate choice decisions in this species and across a large variety of taxa (). A recent neuromolecular study in poeciliid fish suggests a shared pathway between sociality and sexual preferences (). The females tested had neither previous sexual experience nor previous social interaction with males, and the effect of cognitive ability on male quality assessment might be weaker after these interactions. Alternatively, the difference in male quality assessment between large- and small-brained females may be reinforced in natural environments where higher degrees of social and environmental complexity are present (). Our findings open up an exciting avenue for future research on how cognition underlies variation in sexual preferences in cognitively challenging social and natural environments.

An innate preference for more attractive males is widely observed across females from different natural and laboratory populations in this species (, ), and we observed concordant preference for more attractive males in both large-brained and wild-type females. This finding leads to interesting questions about why small-brained females do not show preference for attractive males. One possible explanation is that there might be genetic linkage between male coloration and female preference because large-brained male guppies are more colorful (). However, we find this explanation unlikely for two reasons. First, much of the genetic variation in the male traits used for differential attractiveness in the test is Y-linked (), and the remainder is polygenic and unlikely to be located in one region of the genome. Second, this linkage would be expected to produce a Fisherian runaway process and assortative mating for color in different brain-selected lines. Instead, we observe large variation of preference in small-brained females, leading to a balance in preference between attractive and unattractive males. An additional alternative explanation to our results is that physiological and/or behavioral differences after directional selection on brain size could have altered optimal or rational mate choice between large- and small-brained females. Factors, such as social context (, ), experience (), and condition (, ), have been shown to shift optimal mate choice within and between populations in fish species. We cannot completely exclude the influence of these factors on our results, but we think that their potential effect is unlikely to apply for two reasons. First, all females tested were raised in similar social environments with no visual access to males before the experimental test. Second, the attractive males had higher trait expression in multiple important traits, including a larger area of orange coloration, a larger area of total coloration, and a larger tail area. These sexual traits have all repeatedly been found to be preferred by Trinidadian guppy females across populations (, ). Despite ample empirical evidence for polymorphic trait preferences within populations in this species (), it seems unlikely that small-brained females would show adaptive mate choice for a trait that was not quantified between the size-matched attractive and unattractive males in our choice setup.

Several previous studies have addressed the association between brain size, cognitive ability, and mate choice by investigating the role of cognition in how the chosen sex behaves to secure mating opportunities. Individuals who outperform others in cognitively challenging tasks are preferred by the opposite sex in different species, such as crossbills, guppies, and especially humans (, –). However, it remains unclear whether these preferences directly target cognitive abilities in chosen individuals or whether the preference for these individuals is mediated by an enhanced condition acquired through better cognitive abilities (, ). Far fewer studies have explored the role of cognitive abilities in the choosy sex. Selecting a mate is a key decision with important fitness consequences and that likely requires considerable cognitive abilities (, ). However, our results offer the first experimental support for this idea. We observed that females selected for relatively large brains and known to outperform small-brained females in cognitive tests () seem to make far more accurate mate choice decisions. In showing that there are no differences between these females in color discrimination, condition (, ), or swimming ability (), our results indicate that the observed difference in mate choice is most likely driven by differences in cognitive abilities rather than by any physiological or body condition differences affecting the dichotomous choice test results. Finally, our results are not driven by differences in search strategy or motivation to mate because the total number of visits to choice areas, the time spent out of the choice areas, and the absolute preference irrespective of male attractiveness did not differ between the female groups.

Constraints in cognitive abilities might thus limit the capacity of an individual to perform optimal quality assessment when selecting a sexual partner. If there are fitness benefits to the female for mating with attractive males, female cognitive ability might be under considerable selection to ensure optimal decision-making in mate choice. The huge variation in brain anatomy and cognitive abilities found within and across species suggests that these selection pressures for increased cognitive ability could counter the extensive costs associated with growing and maintaining a large brain, and explains the substantial genetic variation observed in brain size among animals. One potential cost is how the energetic demands of producing neural tissue might directly affect traits with important roles in an individual’s fitness, such as the development of other costly tissue in the organism (, , ), juvenile growth (), fecundity (), or innate immunological response (). In addition, brain size and cognitive ability have been suggested to drive differences in behaviors that might indirectly have an important role in an individual’s fitness. For instance, brain size has previously been associated with better predator avoidance (), leading to greater survival in situations with high predation threat (). Hence, variation in ecological factors may greatly influence variation in brain anatomy and cognitive abilities. In light of the substantial variation that exists in brain size and cognitive ability within most species (), our discovery of brain size affecting mate choice leads to the conclusion that brain size and cognitive ability might be key factors in maintaining variation in mate choice and sexual traits.

METHODS

Study system

We studied the preference for colorful males in female guppies from laboratory-reared descendants of Trinidadian guppies from high predation areas of the Quare River. We used wild-type female guppies from this laboratory population and existing artificial selection lines (based on the same population) for small and large relative brain size, with >10% difference in relative brain size (, ). Briefly, the artificial selection experiment was based on indirect selection for parental brain weight data corrected for body size, which was used to generate replicated lines with large and small relative brain size (three replicates for large-brained and three replicates for small-brained; six populations in total). See Kotrschal et al. () for full details on the selection experiment. These selection lines showed an 11% difference in relative brain size in the third generation () and an up to 13.6% difference in the fourth generation (). After the fourth generation, 30 non-sib males and females from each population were paired to generate a fifth generation of brain size–selected offspring.

Here, we used a total of 16 wild-type females and 36 large-brained and 36 small-brained females from the fifth generation of the brain size selection lines (12 individuals from each of the three up- and down-selected lines). These fish had not been used in any experiments before these behavioral tests. The sample sizes were chosen on the basis of previous experience in effect sizes of the different selection lines and due to constraints in the availability of wild-type females. All offspring were removed from their parental tanks after birth. Males were isolated from females as soon as their gonopodia began to develop and before any color pattern appeared on the males. Females were then placed in 12-liter tanks in groups of 10 fish. One week before the experiment, females were isolated in individual tanks, but we allowed visual contact between tanks containing females to avoid social stress from isolation. However, females did not have visual contact with any mature male before the experiment. All virgin females were tested approximately 6 months after birth. The laboratory was maintained at 26°C with a 12-hour light/12-hour dark schedule, which resulted in a water temperature of 25°C. Fish were fed an alternating daily diet of flake food and live brine shrimp.

Attractive versus unattractive males

To present males with large differences in coloration, we used 60 males for which we quantified coloration using image analysis. For this, we first anesthetized the males with a low dose of benzocaine and photographed both sides using a Nikon D5300 camera. We quantified body length, tail area, and area of orange, black, and iridescence coloration using ImageJ software version 1.44 (). To establish that all males were sexually mature, we first housed the males together with nonparticipant females in the experiment for 2 hours. The two males that did not perform sexual displays toward the female were discarded. Of the 58 remaining males, we selected the 8 males with the highest and lowest combined scores of orange, black, and iridescence area, and then we size-matched them to create eight pairs consisting of one highly attractive male and one unattractive male. Males within pairs did not differ in body length (Welch t = −1.16; df = 13.79; P = 0.26) but presented a large difference in average total coloration, with 26% more area of total coloration in attractive males (Welch t = −8.66; df = 13.79; P < 0.001), 10% more orange coloration in attractive males (Welch t = −8.76; df = 13.79; P < 0.001), and 27% larger tails in more attractive males (Welch t = −2.91; df = 13.79; P < 0.001). To ensure that the differences in attractiveness persisted throughout the mate choice trials, we also quantified the differences in male attractiveness after the experiment and found that total and orange coloration differed by 18 and 6%, respectively (Welch t = −4.95; df = 14; P < 0.001 and Welch t = −3.73; df = 14; P = 0.002, respectively).

Preference test

We tested 36 large-brained and 36 small-brained females for their mating preference for attractive males by measuring side-association data in a dichotomous choice setup (, ). Previous studies in this species have shown consistent results between side-association data and direct measurements of mating preference [reviewed by Houde (34)]. The setup consisted of eight plain glass tanks of 42 cm × 20 cm × 20 cm, where females were allowed to observe an attractive versus unattractive male pair for a 15-min period. Females could not observe both males simultaneously because males were presented in adjoining plain glass tanks of 11 cm × 10 cm × 20 cm with a 10-cm separation (fig. S4). Tests took place over nine consecutive days. All fish were netted and placed in their respective experimental tank 24 hours before the test to allow for acclimation and hence avoid potential differences in stress response between the lines. A nontransparent plastic film was placed on the side walls of the male tanks to avoid visual interactions between males. Equal numbers of attractive and unattractive males were presented to the right and left position toward the female tank for every treatment in a randomized order. We found no significant side bias in preference between the wild-type, large-brained, and small-brained fish (table S5). Every trial was broadcasted live using Logitech HD Webcam C615 from a top position and viewed from a distance on a laptop to avoid disturbance. The position of the female was scored by a single observer using the live observation mode in JWatcher software version 1.0 (). The female experimental tank was divided into three zones to determine female position: (i) left choice zone, the area adjacent to the left male tank up to a maximum distance of 10 cm from it; (ii) right choice zone, the area adjacent to the right male tank up to a maximum distance of 10 cm from it; and (iii) no choice zone, the area between the left and right choice zones and all areas further away than 10 cm from the male tanks (fig. S4). Quantification of behaviors was performed blind to the treatment because only running numbers identified females and male position. To control for the total time during the test that every female spent associating with the offered males, we used a preference ratio widely used in dichotomous choice preference tests (). This preference ratio was obtained as the difference in time spent with each male, standardized by the total amount of time in any of the choice areas [(time with colorful male – time with dull male)/(time with colorful male + time with dull male)].

After completion of the preference analyses in large- and small-brained females, we studied the preference for attractive versus unattractive males in 16 wild-type females of the same age. We repeated the procedure used for large- and small-brained females with a new set of four male pairs over two consecutive days.

To test whether females from all groups demonstrated partner preference irrespective of male attractiveness (color traits and tail length), we studied the absolute value of the preference ratio. The absolute preference could take values between 0 and 1 where 0 indicates that a female divided her time evenly between both males, whereas a score of 1 indicates that a female spent time exclusively with one of the males. To determine whether females significantly preferred one male over the other, regardless of their attractiveness levels based on coloration and tail size, we calculated whether their absolute preference was closer to 1 than expected by chance. The expected value in the case of no preference was affected by the activity and choice dynamics of the female. If she only moved a few times between choice zones, the absolute preference was expected to be closer to 1 than if she moved many times. To control for this, we performed Monte Carlo simulations that constrain the number of zone changes. For each female, we took the sequence of zone changes, randomly replaced the identities for the males, and then calculated the absolute preferences after random assignment. For each group (brain size by replicate), we calculated the mean absolute preference and compared it to 10,000 simulated mean absolute preferences. We calculated two-sided P values by counting how many simulations had an equal or more extreme result than our observation. For the small- and large-brained females, we combined the separate results for the three replicated selection lines into a single P value using Stouffer’s z score method (information S6) ().

Data analyses on absolute preference values (information S1) showed a significant decay of absolute preference over the time of the trial [LMMabs. preference: time: χ2(14) = 36.215, P < 0.001]. Because of this, we separated the data into three time periods of 5 min and independently tested these for potential differences in female preference for attractive males using the standardized preference ratio (Fig. 1B). We chose 5-min periods to ensure that enough preference data were available for all females in the trials. Standardized preference ratio indicates the relative proportion of time that a female spent with the attractive male based on coloration and tail size. Standardized preference ratio could take values between −1 (all time spent with the noncolorful/small-tailed unattractive male) and 1 (all time spent with the colorful/large-tailed attractive male). We tested for potential differences in preference for attractive males between the wild-type, large-brained, and small-brained females with a linear mixed-effects model (LMM). Models included brain size treatment as a fixed effect. The random effects portion of the model included a random intercept for each replicate selection line and a random slope for brain size within each replicate. Also, we included male pair and experimental tank as random factors (information S6). Preference differed between female groups in the initial 5 min of the trial and in the second 5-min period [LMMpreference: brain size: χ2(2) = 6.985, P = 0.030 and LMMpreference: brain size: χ2(2) = 9.124, P = 0.010, respectively]. We observed no difference in the third 5-min period [LMMpreference: brain size: χ2(2) = 1.664, P = 0.435] (Fig. 1B). Hence, to investigate potential differences in female preference for attractive males, we focused on the data obtained during the first 10 min of each trial. We then tested for differences in preference between wild-type females and large- or small-brained females with a similar LMM approach. In the case of tests for potential differences in preference between large- and small-brained females, we also included the day of the experiment as a random factor in the LMM (information S6). All statistical analyses were performed in R version 3.2.2 (). The assumptions of normality and equality of variances were confirmed by visual inspection of the residuals.

Optomotor response test

We quantified the visual capacity of sensitivity of females to male coloration using optomotor response tests (). Optomotor responses of several fish species have been studied by rotating an array of stripes to a chosen threshold speed (). This test has also previously been used to assess the visual performance of guppies (), which exhibit several stereotypical behaviors when subjected to a rotational stimulation: (i) optomotor circling, that is, swimming along to follow the stripes in a rotating circular pattern; (ii) compass reaction, that is, rotating of the body around its own midpoint to follow the stripes in a rotating circular pattern; (iii) nystagmic reaction, that is, turning of the head to follow the stripes in a rotating circular pattern followed by a rapid return to the initial position; and (iv) zero motion, that is, swimming without a clear circular motion or remaining static ().

We performed a variant of the optomotor test on the 16 wild-type and on 18 large-brained and 18 small-brained female guppies randomly selected from the previous preference test (six individuals per replicated relative brain size–selected line). We generated a rotating stimulus with stripes of maximum peak transmittance of 550 and 630 nm. We selected these wavelengths to test the sensitivity of these guppies to a color relevant in male nuptial traits. Orange spots of male guppies in our laboratory population have peak transmittances around 630 nm (). Because of this, we generated a video recording with six different rotating stimuli consisting of 16 alternating bands of red and green. These colors showed peak reflectance around 550 and 630 nm, respectively, when projected to a white background with an InFocus IN114 projector. To generate a gradual shift in the visibility of the successive rotating stimulus, we gradually lowered the intensity in red and green bands by 20% until both bands appeared the same (fig. S7). In addition, we added a seventh rotating stimulus with the initial maximum saturation contrast between bands. This final change in the rotating stimulus was added to control for the effect of stress and/or habituation to the stimuli. For each rotating stimulus, a minute of acclimation time was included before the start of the rotation in the video. This acclimation time consisted of a static image of the contrasting bands that would rotate next.

In a restricted randomized order (one female per treatment), females were individually placed in a circular white tank with a diameter of 25 cm. After 2 min of acclimation, the video recording was projected on the inside of the walls of the tank using an InFocus IN114 projector (fig. S7). Behavior of the fish was recorded using a Sony HDR-SR11E camcorder and later scored by a single observer using BORIS version 2.72. We scored the time that each fish spent performing the four stereotypical rotating behaviors for each acclimatization and rotation stimulus phase. Quantification was performed blind to the treatment. We found no difference between treatment groups in the time spent performing optomotor circling, compass reaction, or nystagmic reaction for any analyzed moving or acclimating phase (table S8). Therefore, we created a new variable, total rotating behavior, which combined the time of all three behaviors described above [(i) to (iii)]. We then tested each rotational stimulus independently for potential differences in total rotating behavior between the large-brained, small-brained, and wild-type females with an LMM. Models included brain size treatment as a fixed effect. The random effects portion of the model included a random intercept for each replicate selection line and a random slope for brain size within each replicate. Also, we included optomotor response during the static phase as a covariate in the model (information S6). All statistical analyses were performed in R version 3.2.2 (). The assumptions of normality and equality of variances were confirmed by visual inspection of the residuals.

Opsin expression analysis

We measured the opsin expression of nine wild-type, nine large-brained, and nine small-brained females previously tested on their preference for attractive males. After dispatch, eyes were removed and immediately placed in RNAlater. Total RNA was purified using RNeasy kits (Qiagen) following standard manufacturer protocols. Expression level was quantified using the NanoString nCounter assay (), which involves hybridizing target sequences in the sample by complementary base pairing to both a reporter probe and a capture probe. We designed probes for the cone opsin genes (LWS A180, LWS P180, LWS1, LWS S180, SWS1, SWS2A, SWS2B, RH2-1, and RH2-2) as well as rhodopsin (RH1). Two additional probes were designed in conserved regions shared by LWS duplicates to verify the specificity of probes. All probe sequences can be found in table S9. Hybridizations were carried out according to NanoString standard protocols. After hybridization, expression data were normalized on the basis of manufacturer spike-in controls to account for technical variance and four housekeeping genes (β-actin, COI, Gapdh, and Myosin HC) to control for sample input. We also sequentially removed each housekeeping gene from the normalization to test for outlier normalization effect. No single housekeeping gene substantially affects normalization of target loci.

A given target gene was considered to be significantly expressed if its absolute count exceeded the mean spike-in negative controls by at least 2 SDs, and all of our target genes met this threshold criterion. We analyzed the differences in expression independently for each measured opsin. Models using an LMM approach with a random intercept and slope for brain size for each replicate did not converge because of low sample size for each replicate/brain size combination. Therefore, we determined differential expression after log2 transformation of expression data for our target genes using paired t tests in combined values for each relative brain size treatment.