Comment on "Application of an in Vitro Assay to Identify Chemicals That Increase Estradiol and Progesterone Synthesis and Are Potential Breast Cancer Risk Factors".

The article by Cardona and Rudel (2021) analyzed ToxCast results for the H295R steroidogenesis assays and reported that “296 chemicals increased [estradiol] (182) or [progesterone] (185), with 71 chemicals increasing both.” This result is both fascinating and rather striking in suggesting that in vitro 32% of chemicals increase estrogen production, 32% increase progesterone production, and 12% increase both. Based on our reanalysis of the data (available on Github: https://github.com/DataSciBurgoon/toxcast_steroidogenesis/blob/main/Raptor_SSI_Steroidogenesis.ipynb) and based on the fact that most ToxCast chemicals have small sample sizes (generally , atypically ) and therefore likely suffer from sampling bias, most if not all of the chemicals identified by Cardona and Rudel (2021) are false positives (Christley 2010; Gelman and Carlin 2014; Lin 2018).

Peculiarly, Cardona and Rudel (2021) did not base their analysis directly on ToxCast data from the U.S. Environmental Protection Agency’s invitrodb database. Rather, they used data from Haggard et al. (2018, 2019); using data from only a single plate for each chemical, ignoring replicate information from other plates. Ignoring available replicates is a violation of common statistical standards and will result in flawed and biased results, mostly due to sampling bias and failure to account for plate effects.

The authors also overlooked data quality issues reported for several of the chemicals in the ToxCast dataset in the National Toxicology Program’s Integrated Chemical Environment (ICE) database (Abedini et al. 2021). Thirty-seven and 42 of the chemicals that Cardona and Rudel (2021) identified as increasing estrogen or progesterone production, respectively, have quality flags that indicate that the chemical used may not be what was intended, or that the concentration may be 30% or less than the intended concentration.

Furthermore, the progesterone data exhibit a major day effect, indicating they are not of suitable quality for analysis. Without additional information, the progesterone data should be removed entirely.

Our reanalysis identified either 17 or 3 chemicals (depending upon stringency) that we allege may be stimulating the production of estrogen in vitro; however, all of those are likely false positives due to small sample sizes and sampling bias. This is far less than the number identified by Cardona and Rudel (2021), suggesting most of the chemicals they identified are false positives. Therefore, we suggest that these 17 or 3 chemicals would be good starting points for further research at human-relevant concentrations, with the understanding that they very well could be false positives due to the limited number of replicates.