The importance of making testable predictions: A cautionary tale.

We found a startling correlation (Pearson ρ > 0.97) between a single event in daily sea surface temperatures each spring, and peak fish egg abundance measurements the following summer, in 7 years of approximately weekly fish egg abundance data collected at Scripps Pier in La Jolla California. Even more surprising was that this event-based result persisted despite the large and variable number of fish species involved (up to 46), and the large and variable time interval between trigger and response (up to ~3 months). To mitigate potential over-fitting, we made an out-of-sample prediction beyond the publication process for the peak summer egg abundance observed at Scripps Pier in 2020 (available on bioRxiv). During peer-review, the prediction failed, and while it would be tempting to explain this away as a result of the record-breaking toxic algal bloom that occurred during the spring (9x higher concentration of dinoflagellates than ever previously recorded), a re-examination of our methodology revealed a potential source of over-fitting that had not been evaluated for robustness. This cautionary tale highlights the importance of testable true out-of-sample predictions of future values that cannot (even accidentally) be used in model fitting, and that can therefore catch model assumptions that may otherwise escape notice. We believe that this example can benefit the current push towards ecology as a predictive science and support the notion that predictions should live and die in the public domain, along with the models that made them.

Introduction

To comprehend the population dynamics underpinning biodiversity and essential ecosystem services, a heavy emphasis is placed on driving mechanisms, both biotic and abiotic. In marine environments, where fish stocks are of substantial ecological and economic interest, drivers need to be untangled to inform effective, practical, and sustainable environmental policy. Temperature is a particularly important (and sometimes controversial [1-3]) driver for fish and other marine ectotherm populations. Here we focus on the well-studied relationship between temperature and fish reproduction [4-6].

At the seasonal timescale, trends between water temperature and spawning activity have been observed in many fish species [7-16]. Huber and Bengtson [11] found that the gonads of inland silversides (Menidia beryllina), a summer spawning species, did not mature to a reproductive level in the absence of increasing water temperatures. In yellow perch (Perca flavenscens) both decreasing autumn temperatures and low winter temperatures have been deemed critical in order for gonadal maturation to occur [12, 13]. Additionally, these fish have been manipulated to spawn earlier in the year by increasing the rate of water temperature change [13]. Kayes and Calbert [14] found that for the same yellow perch species, increasing temperature heightened egg production, but even in the absence of a temperature cue endogenous factors could induce spawning. In cyprinid fishes, the initiation of gametogenesis requires low temperatures, but the completion of the process requires increasing temperatures [7]. Other notable studies use degree days, a measure of time based on temperature, to track gonad development from the initiation of vitellogenesis to the onset of spawning [17, 18]. A study done by Henderson et al. [19] demonstrates that the timing of spring transitions and the duration of summer, defined by a temperature threshold, is related to shifts in the center of biomass for multiple species during their seasonal migration to spawning grounds, however, the shifts observed differ by species. These and other varying and apparently complicated specific effects suggest that more general quantitative relationships covering diverse species may be hard to come by.

Despite this, recently a strong quantitative predictive relationship was detected between average winter temperature and average spring-summer egg abundance for a suite of near-shore-spawning species off the coast of southern California [15], which was subsequently supported by out-of-sample data acquired the following year [16]. This relationship is largely explained by colder waters being indicative of large scale upwelling; a process known to supply nutrients to shallower waters [20].

Building on this encouraging result, we re-examined the data of [15] and [16], but now including the additional 2019 and 2020 data that have since become available, and found a true out-of-sample confirmation of that relationship (Fig 1C). To emphasize, that means the original relationship of Ref. [15] which was based on just five years of available data, has reasonably predicted the subsequent three years of egg abundance. The data underlying this relationship, which consist of approximately weekly-sampled, species-identified egg counts of 46 near-shore-spawning species from Scripps Pier since 2013 (see Methods) also contain substantial, and possibly important, fine-timescale information that was not considered in the seasonal relationship described in [15]. Though the statistical seasonal association is compelling, it emerges only in large-scale averages. By coupling the full-resolution fish egg abundance time series (Fig 1A) with daily-averaged sea-surface temperatures (Fig 1B) from the Southern Californian Coastal Observational Ocean Monitoring System (SCCOOS) dataset, we asked whether finer-timescale temperature dynamics provide information about finer-timescale fish egg abundance dynamics.

Fig 1

Is there a fine-time-scale relationship between temperature and eggs?

A) The total egg abundance in each collection (see Methods) shows substantial variability from year to year in both mean and peak levels. Each fish egg collection was made from the Scripps Institution of Oceanography (SIO) Pier. B) The daily averaged sea surface temperature (SST) in °C at the SIO Pier from data taken every 5–10 minutes from the SCCOOS monitoring station. C) Seasonal averaging reveals the strong negative correlation between the average winter (December–February) SST and the average spring and summer (March–August) egg abundance, identified by [15], with additional points for 2018 [16], and now 2019 and 2020. D) The seasonal correlation breaks down at the daily level; there is no similarly strong correlation between daily winter temperatures and daily egg abundances with time delays ranging from 0 to 180 days. E) The S-Map [21] test for nonlinearity shows that forecasts of egg abundance improve (correlation between predictions and observations) as the nonlinearity parameter (θ) is increased, indicating that egg abundance shows nonlinear behavior. F) Convergent cross-mapping (CCM, [22]), shows that when using the egg abundance time series to map onto the temperature time series, predictions improve as library size increases, indicating there is a dynamic causal effect of daily-averaged temperature on egg abundance.

Results

The high time-resolution data (Fig 1A and 1B) did not show a linear cross-correlation between the daily spring temperature and lagged daily egg abundance, with only weak relationships across all delays at this fine daily timescale (Fig 1D). However, in accordance with previous work [23-26], the S-Map test for nonlinearity [21] revealed that the egg abundance is driven by nonlinear processes (forecasts improve as the nonlinear parameter, θ, is increased, Fig 1E). Further, convergent cross-mapping, a tool for detecting nonlinear coupling in dynamical systems [22] suggested that temperature has a nonlinear effect on egg abundance (converges to ρ = 0.58, n = 295, Fig 1F). Thus, we expected that a daily timescale relationship may be detectable, just not with linear correlation.

One type of event that stands out in the egg abundance time series (Fig 1A) is the peak in summer egg abundance. Both the magnitude and timing of the peak egg abundance varies from year to year with no obvious pattern. Previous studies indicate that increasing water temperature may provide a cue for spring and summer spawning species [19, 27–29]. To ascertain whether a relationship exists between spring temperature increase and peak summer egg abundance, we defined a generic spring temperature trigger (STT). Our STT is the maximum of all temperature increases detected within a moving window of length L, as that window moves over the spring season (Fig 2A, see Methods). This returns a single scalar value for the season, corresponding to a single event with an interpretable characteristic timescale (L). We restricted our analysis of temperature to the spring season (i.e. the season preceding the summer peak) following roughly the causal timescale examined in [15]. By examining a range of possible window lengths, we found a robust relationship around the 1 month timescale, between STT and peak summer egg abundance (L between 3 and 5 weeks, ρ > 0.95, Fig 2B). This relationship was so remarkably strong (ρ up to 0.98; Fig 2C), and apparently robust (Fig 2B) that we felt compelled to share this observation, despite the small number of data points involved (n = 7). To mitigate potential overfitting, we offered a prediction for the 2020 peak summer egg abundance that at the time of writing had not yet been measured (Fig 2C) [30]. To examine whether this relationship was caused by a general spring warming trend, we repeated the analysis on increasingly smoothed (time averaged) temperature data. We found that the predictive relationship from STT to peak summer egg abundance decreased markedly as temperature data became increasingly smoothed (Fig 3), which suggested to us that the information was indeed contained in the daily-resolution temperature information, and not in the trend.

Fig 2

An apparently robust temperature trigger for peak summer egg abundance that ultimately failed in true out-of-sample prediction.

A) We defined the spring temperature trigger (STT) as the largest temperature increase (denoted in red) detected within a monthly sliding window (gray area) as it moves in daily increments over the spring season (dashed lines; see Methods). B) The relationship between STT and peak summer eggs was robust to the width of the sliding window (widths that produce a ρ > 0.95 for the data up to and including 2019 are indicated in green). C) The peak correlation between STT and peak summer eggs (June–August) for 2013–2019 (black dots) and predicted value of 801 eggs for 2020 based on the linear regression (red dot), which differs dramatically from the eventually observed peak summer eggs (blue dot).

Fig 3

Time averaging of the 2013–2019 data obscured the original spring temperature trigger, suggesting the predictive information lay in the daily-resolution temperature data, not in the trend.

A) The relationship between spring temperature trigger (STT defined over 27 days) and maximum summer fish egg abundance declined as SST was increasingly smoothed across the x-axis (from daily to monthly averages). B) The daily (blue), weekly averaged (purple), and monthly averaged (orange) sea-surface temperature in 2017. Note how the magnitude of the STT (red bars) declines with averaging. Note that this figure does not include 2020 data.

The failure of our published out-of-sample prediction for 2020 [30] naturally led us to ask whether an exogenous change in conditions had come into play. Indeed, 2020 has been an anomalous year in many ways, and for marine life at Scripps Pier it was most notably seen in a toxic algal bloom (red tide) that was record-breaking both in terms of density of dinoflagellates (9x higher than ever previously recorded) and duration (over a month from early April until mid-May compared to typical 1–2 week blooms see https://sccoos.org/california-hab-bulletin/red-tide). This led not only to extraordinary fish kills (including within experimental and educational aquaria associated with Scripps that became contaminated by seawater intake) but also modified other physical (e.g. optical) and chemical properties of the near-shore marine environment, with a broad impact across that ecosystem. In many respects, it would have been surprising if fish spawning in 2020 was unaffected by this red tide. Moreover, the direction of the error in our prediction was consistent with these changes. The increased absorption of incident light due to the red, visibly opaque water occurred during our STT, and may have enhanced the STT magnitude, leading to an inflated temperature trigger and peak eggs prediction. The summer fish eggs, on the other hand, would be expected to be substantially reduced by the spring fish kills caused by the red tide.

Despite this potential source of error external to our model, any failure of an out-of-sample prediction deserves careful attention (indeed, the prediction from Ref. [15] still holds in 2020). The fact that the peak observed fish egg count in 2020 actually did not occur during the summer (with a substantially larger peak in the spring which again may be related to the red tide; Fig 4A), prompted us to question whether the peak eggs should be defined as a summer event, or as an annual event. Relaxing this timing definition, and instead looking for an annual peak in eggs with a triggering event that occurs at any time before, completely disrupts the temperature trigger to peak eggs relationship, and not just for the 2020 data (Fig 4B; see Methods). This suggests the possibility of overfitting in the base definitions, that is confirmed by allowing the date defining the boundary between spring and summer to vary (Fig 4C). We find that only small variations in the boundary date between spring and summer can substantially reduce the observed correlation (Fig 4C) which strongly suggests that the original observed relationship may have been a case of over-fitting.

Fig 4

The failure of our 2020 prediction and the structural sensitivity of the model.

A) 2020 fish eggs and temperature time series showing the timing of the red tide. The timing of the red tide was estimated based on the duration of anomalous dissolved oxygen levels measured by the Scripps Ocean Acidification Real-time (SOAR) Monitoring Program. B) Correlation between a triggering event that occurs any time before the annual peak eggs, and the annual peak eggs from 2013 to 2020 suggests possible overfitting (see Methods). C) Sensitivity of the original 2013–2019 STT to peak summer eggs relationship to variation in the spring end date and summer start date further suggests overfitting.

Discussion

Many ecological models and relationships are based on a large number of parameters, which renders the task of assessing model robustness in that high-dimensional parameter space generally difficult. However, even more difficult to assess is the robustness of assumptions that underlie the construction of the model itself, from which the model parameters arise. These assumptions may be non-quantitative (e.g. categorical), such as the choice of functional form or the inclusion or exclusion of model elements. Devising appropriate all-encompassing tests across this full space of possible hypotheses is neither feasible nor advisable, and so this source of uncertainty is often overlooked. Often, however, some assessment of the structural stability of models is both possible and highly enlightening (e.g. [31]) and should be encouraged. In the case of the STT to peak summer eggs relationship discussed here, the underlying definition of the spring and summer intervals was quantitative, and so was particularly easy to assess, but it is often the case with assumptions that they seem so obvious as to become invisible.

The process of making true out-of-sample predictions can account for both the difficulty of seeing potential sources of overfitting, and the difficulty of assessing them. Furthermore, publishing predictions before it is possible to assess their accuracy places in clear view any subsequent revisions of the model or relationship to account for new data. This may seem to go against natural intuition: if a model prediction can be tested, should it not be tested before publication, to improve confidence in the model? Certainly, a confidence threshold must be reached in order to justify publication (as it was here, where because of the nearly perfect fit any subsample with 3 or more points would reasonably produce successful out-of-sample predictions). However, we suggest that once that confidence threshold has been reached, leaving a true out-of-sample validation avenue open to future investigation is a valuable standard to aspire to, and we support the push towards quantitative prediction of truly out-of-sample natural data as a validation standard for ecological and earth systems science.

Materials and methods

Convergent cross mapping

Convergent cross mapping was performed using the block_lnlp() function in rEDM v0.7.3 [32]. The embedding was made with the raw egg abundance data and the daily-averaged SCCOOS temperature time series.

When performing CCM, the variable being predicted is the one being tested as a causal driver. As such, we used the egg abundance time series to predict temperature (thus measuring temperature’s effect on egg abundance). Although temperature data existed for every day, eggs were collected at inconsistent intervals, typically ranging from one collection every 2–5 days. Thus, in order to make a proper embedding, we filtered both temperature and egg abundance time series to only include temperature values for which a collection occurred on a given day, 6–8 days prior, and 13–15 days prior as well. This gave us a 3-dimensional embedding for fish eggs, with time lags of about 1 week, with accompanying temperature values.

Because both egg abundance and temperature are strongly seasonally driven, we needed to make sure we were not identifying shared information in the two variables driven by seasonality. To account for this, nearest neighbor selection only considered time points that were within 90 calendar days for our target prediction. Without doing this, increased library size will only increase the amount of seasonal information resolved in the embedding rather than actual causal inference.

Libraries of potential neighbors (points within 90 calendar days of the date of the target) were generated at random for each predicted point. Library sizes ranged from 10–80 points (increasing by increments of 5). Once the library was randomly generated, the nearest 4 neighbors (E+1, see (22)) in state space were selected and used to make a prediction. After a prediction was made on each temperature value, Pearson's correlation was calculated between observed and predicted values. This process was repeated 50 times for each library size.

STT calculation

As described in the text and illustrated in Fig 2, STT was calculated by allowing the last day of the sliding window to move over the spring interval defined as April 1st to June 18th.

Sensitivity test

The annual peak in egg abundance was identified as the maximum egg abundance occurring between January 1st and December 31st within each year. The trigger-like value was found using the sliding window analysis described in the text and illustrated in Fig 2, however the last day of the sliding window was allowed to move from January 28th to the day immediately preceding the annual peak in eggs.

Sample collection

California Department of Fish and Wildlife permit (#4564) was used for the collection of plankton from the MPA’s. Vertical plankton tows (approximately weekly) were conducted off of the Scripps Pier (32.8328° N, -117.2713° W) from 2013 to 2019. A 1-meter diameter net with 505 micron mesh and a bottle attached to the cod end was lowered to the seafloor, approximately 5 meters, and out of the water 4 times, sampling a total of ~16 cubic meters of seawater. The net was then rinsed by lowering it into the water until the top of the net touched the surface and then raised back out. It is worth noting that this method only samples eggs suspended in the water column and does not effectively collect demersal eggs. There is some variation in the volume of water being sampled (e.g., water depth changes with tide), but since all collections went from seafloor to surface we do not expect any significant effect of egg depth profiles to have any significant effect on our measurement peak summer egg abundance. Currents could also affect sample volume but are rarely strong in the summer [33] and are therefore less likely to skew the value of peak eggs. The eggs captured at the pier all originated 0–3 days before the collection occurred since in Southern California water temperatures, most eggs for the fish species found there hatch within 72 hours [34] hence any eggs from a spawning event preceding the collection by up to 3 days could still be represented in our sample, depending on precise spawning location and currents. Using real-time current velocities, retrospective modeling found that most eggs collected at the Pier site likely originated within a few kilometers of the collection site [34]. The contents of the cod end were concentrated through a 330 micron mesh screen and then sorted under a microscope at 10X. The fish eggs were counted, placed in 1.5 mL tubes containing 95% ethanol, and stored at -20°C for at least 24 hours until further processing. At this step, the morphologically distinct eggs of the Northern anchovy (Engraulis mordax) and Pacific sardine (Sardinops sagax) are counted and stored separately because they do not require molecular methods for identification. The remaining eggs are identified through DNA barcoding. Comprehensive species lists are found in S2 and S3 Tables.

Correlation between multiple variables studied.

The strongest correlation we found was between STT and peak egg abundance (maximum correlation of 0.98). However, as found by [15], a strong, negative correlation exists between average winter temperatures and average summer egg abundance. Not surprisingly, there is also a strong correlation between peak egg abundance and average egg abundance for a given summer (A). Due to transitivity, there is also a strong correlation between average summer eggs and spring temperature triggers (B). Weaker correlations also exist between average winter temperatures and the finer scale temperature triggers (C) and peak summer egg abundance (D), however these are much weaker relationships (p> 0.1).

(TIF)

Click here for additional data file.

Inconclusive evidence of between-species synchrony in peak summer egg abundance.

Shannon diversity (base e) appears to be higher for peaks with lower abundance.

(TIF)

Click here for additional data file.

Species composition of the peak summer egg abundance samples.

The proportional contribution that each of the identified species contributes to the annual peak summer egg abundance. The peak samples in each year are dominated by the eggs of a few species, with the dominant species varying from year-to-year.

(DOCX)

Click here for additional data file.

Scripps pier species abundance 2013–2019.

A list of all 46 species identified in the samples from Scripps Pier from 2013–2019 and the number of eggs identified as each of those species within each year. The sampling effort by year is as follows: 2013 = 161, 2014 = 84, 2015 = 51, 2016 = 52, 2017 = 48, 2018 = 75, 2019 = 65.

(DOCX)

Click here for additional data file.

Scripps pier species frequency 2013–2019.

A list of all 46 species identified in the samples from Scripps Pier from 2013–2019 and the proportion of samples they were present in within each year. The sampling effort by year is as follows: 2013 = 161, 2014 = 84, 2015 = 51, 2016 = 52, 2017 = 48, 2018 = 75, 2019 = 65.

(DOCX)

Click here for additional data file.

6 Aug 2020

PONE-D-20-21189

Temperature triggers provide quantitative predictions of multi-species fish spawning peaks

PLOS ONE

Dear Dr. Sugihara,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

In your revision please address all the comments and suggestions made by reviewer #1. In particular also make sure that the data are freely available and that the link provided to access the data works, thank you.

Please submit your revised manuscript by Sep 20 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Andrea Belgrano, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is an interesting analysis. It was a pleasure to read it. I favor publication. I am left with few questions, and a suggestion, which I hope the authors can address in a review.

1. What is the species composition of the egg catch? I am unsure with the statement that eggs are buoyant. Many are not, and it depends on the species.

2. Somewhat related to my previous question, Fig S2 is interesting and it shows decline of egg diversity in relation to peak eggs abundance. This is presumably driven by an increase of dominance of few or a single species. Can you elaborate on the species that dominate the samples, especially when egg abundance increases? At the end of the methods the authors indicate that the morphologically distinct anchovy and sardine eggs were removed, and the rest of the eggs were counted and identified to species using DNA barcoding. Would be great to see the species list.

5. This is a comment/suggestion. I suggest redirecting the focus of the research question toward a more mechanistic relationship between temperature and egg abundance. The author ask whether 'finer-timescale temperature dynamics provide information about finer-timescale fish egg abundance dynamics.' However, the striking relationships that they have uncovered between STT and peak egg abundance, in my view, is still an integrated measure rather than an examination of a finer scale relationship between temperature and eggs. There is still value in this relationship of course, but not of the same type suggested by the author. This analyses reveals potential mechanisms, pointing to the fact that large variations of water temperature during spring, may trigger massive spawning events during summer.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Lorenzo Ciannelli

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

22 Oct 2020

Response to Reveiwer 1

Reviewer’s comment: What is the species composition of the egg catch? I am unsure with the statement that eggs are buoyant. Many are not, and it depends on the species.

The Reviewer raises an important point, and we’re sure many people will be interested in the composition of the samples. To address this, we have included two new supplemental tables. The first of these two tables, Table S2: Scripps Pier Species Abundance 2013–2019, lists all 46 of the species identified from the sampling at Scripps Pier from 2013 to 2019 and the number of eggs identified as each of these species within each year. The second of these two tables, Table S3: Scripps Pier Species Frequency 2013–2019, lists all 46 of the species and the proportion of samples (out of the yearly sampling effort) in which eggs from the species were observed within each year.

With regard to buoyancy, we have added a statement to the manuscript that our methods only effectively sample eggs suspended in the water column; we found very few eggs from species with demersal eggs – those were presumably stirred off the bottom by the net.

Reviewer’s comment: Somewhat related to my previous question, Fig S2 is interesting and it shows decline of egg diversity in relation to peak eggs abundance. This is presumably driven by an increase of dominance of few or a single species. Can you elaborate on the species that dominate the samples, especially when egg abundance increases? At the end of the methods the authors indicate that the morphologically distinct anchovy and sardine eggs were removed, and the rest of the eggs were counted and identified to species using DNA barcoding. Would be great to see the species list.

We thank the Reviewer for highlighting this opportunity for clarification. We have now included a new supplemental table, Table S1: Species composition of the peak summer egg abundance samples, that lists the proportion of the annual summer peak eggs that were identified as each species. This table highlights which species dominate the peak summer egg abundance samples. We have slightly expanded the main text in the discussion of synchrony to accommodate the addition of this table.

Reviewer’s comment: This is a comment/suggestion. I suggest redirecting the focus of the research question toward a more mechanistic relationship between temperature and egg abundance. The author ask whether 'finer-timescale temperature dynamics provide information about finer-timescale fish egg abundance dynamics.' However, the striking relationships that they have uncovered between STT and peak egg abundance, in my view, is still an integrated measure rather than an examination of a finer scale relationship between temperature and eggs. There is still value in this relationship of course, but not of the same type suggested by the author. This analyses reveals potential mechanisms, pointing to the fact that large variations of water temperature during spring, may trigger massive spawning events during summer.

While we greatly appreciate this suggestion, and would like to be able to pursue it, it is not currently possible to redirect our research question to address the finely resolved details of the mechanistic relationship between temperature and egg abundance. This is because we did not specifically measure a variable to demonstrate how temperature is acting to influence fish reproduction. Given that we are measuring a macroscopic output variable, egg abundance, it is difficult to determine whether the STT is having a direct physiological effect on the fish or whether it is related to other more proximate factors that drive increases in egg abundance. These are questions perhaps best answered by experimental manipulation; in the current experimental design of our study we could not discern the finer-scale details of the mechanism at play. The causality detection method (CCM), however, does verify (within these data) that there is a causal link here between temperature and egg abundance insofar as changes in temperature propagate to changes in egg abundance. Moreover, we show that the strength of the relationship identified here is dependent upon fine time scale measurements – both the STT and the peak summer egg abundance are captured through frequent measurements, daily in the case of STT and weekly in the case of peak summer egg abundance. In Figure 3A we showed that in smoothing the daily temperature datum, we lose the signal between the STT and peak summer egg abundance. Therefore, we focused our research question on fine time scale dynamics that are shown to be essential to this analysis, rather than a mechanistic relationship that we are unable to speak to, given the output variable we measured.

Submitted filename: Response to Reviewers_9.01.2020.pdf

Click here for additional data file.

2 Nov 2020

The importance of making testable predictions: a cautionary tale

PONE-D-20-21189R1

Dear Dr. Sugihara,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Andrea Belgrano, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

The revised manuscript addresses All the comments/suggestions made by reviewer #1 including the availability/access to All the data that you have now nicely provided via the GitHub repository. The manuscript makes an interesting and transparent "tale" that I believe will be of great interest to the broad readership of PLOS ONE.

26 Nov 2020

PONE-D-20-21189R1

The importance of making testable predictions: a cautionary tale

Dear Dr. Sugihara:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Andrea Belgrano

Academic Editor

PLOS ONE