Literature DB >> 30998705

Multi-trait multi-environment models in the genetic selection of segregating soybean progeny.

Leonardo Volpato1, Rodrigo Silva Alves2, Paulo Eduardo Teodoro3, Marcos Deon Vilela de Resende4, Moysés Nascimento4, Ana Carolina Campana Nascimento4, Willian Hytalo Ludke1, Felipe Lopes da Silva1, Aluízio Borém1.   

Abstract

At present, single-trait best linear unbiased prediction (BLUP) is the standard method for genetic selection in soybean. However, when genetic selection is performed based on two or more genetically correlated traits and these are analyzed individually, selection bias may arise. Under these conditions, considering the correlation structure between the evaluated traits may provide more-accurate genetic estimates for the evaluated parameters, even under environmental influences. The present study was thus developed to examine the efficiency and applicability of multi-trait multi-environment (MTME) models by the residual maximum likelihood (REML/BLUP) and Bayesian approaches in the genetic selection of segregating soybean progeny. The study involved data pertaining to 203 soybean F2:4 progeny assessed in two environments for the following traits: number of days to maturity (DM), 100-seed weight (SW), and average seed yield per plot (SY). Variance components and genetic and non-genetic parameters were estimated via the REML/BLUP and Bayesian methods. The variance components estimated and the breeding values and genetic gains predicted with selection through the Bayesian procedure were similar to those obtained by REML/BLUP. The frequentist and Bayesian MTME models provided higher estimates of broad-sense heritability per plot (or heritability of total effects of progeny; [Formula: see text]) and mean accuracy of progeny than their respective single-trait versions. Bayesian analysis provided the credibility intervals for the estimates of [Formula: see text]. Therefore, MTME led to greater predicted gains from selection. On this basis, this procedure can be efficiently applied in the genetic selection of segregating soybean progeny.

Entities:  

Mesh:

Year:  2019        PMID: 30998705      PMCID: PMC6472761          DOI: 10.1371/journal.pone.0215315

Source DB:  PubMed          Journal:  PLoS One        ISSN: 1932-6203            Impact factor:   3.240


Introduction

Soybean [Glycine max (L.) Merrill] is the fourth most widely grown crop in the world. This species is originally from China and is the major crop in the USA, Brazil, Argentina, and many other countries [1]. Soybean is currently grown from low to high latitudes, where it is used as a source of oil, protein, biodiesel, etc. [2]. In this scenario, because the genotype × environment (G×E) interaction plays an essential role in genotypic expression, it must be considered in the evaluation and selection of superior genotypes [3-5]. The selection of segregating soybean progeny is a rather complex process because the traits of agronomic importance (e.g., maturity, yield, etc.) are of quantitative nature. Furthermore, because some traits are correlated with each other, selection based on one of them leads to alterations in others [6,7]. In soybean breeding, most of the available methods for the selection of progeny or lines are useful in the analysis of a single trait measured either in a single environment [8,9] or in various environments with the incorporation of the G×E interaction [10-12]. However, researchers often face situations in which multiple traits are measured across multiple environments [13]. Moreover, selection bias may arise when genetic selection is performed based on two or more genetically correlated traits (due to pleiotropism, imbalance in the gametic phase, and/or the common influence of the environment) and these are analyzed individually. This is especially true in sequential selection [13-15]. To reduce selection bias, Henderson et al. [16] proposed the multiple-trait BLUP method. An additional advantage of jointly modeling multiple traits compared to analyzing each trait separately is that the inference process appropriately accounts for the correlation between the traits, which helps to increase prediction accuracy, statistical power, and parameter estimation accuracy [14,16,17]. Despite the improvements provided by the multiple-trait BLUP (Best Linear Unbiased Prediction) method, to the best of our knowledge, there are few studies combining multi-trait models under a multi-environment approach. The multiple-trait BLUP procedure is ideal, as it makes it possible to simultaneously analyze traits that are correlated with each other and that exhibit covariance heterogeneity across experimented environments [18,19]. In this way, a covariance structure is applied to each random factor in the model; e.g., the progeny effects, the G×E interaction effects, and the residual effects [20,21]. Even though data collected in plant breeding studies often present a multi-trait multi-environment structure, as they take into consideration the genetic correlations and the G × E interaction, more complex models are required, rendering the computation process more laborious. Some studies have demonstrated the potential of the Bayesian approach for genetic evaluation in plant breeding considering multi-trait or multi-environment models [22-25]. In this approach, the parameters are interpreted as random variables, following the law of probability, which assumes a priori knowledge [25,26]. In view of the above-described situation, the present study proposes to examine the efficiency and applicability of multi-trait multi-environment (MTME) models in the selection of segregating soybean progeny, using phenotypic data, by the frequentist (FMTME) and Bayesian (BMTME) methodologies. Effective biometric tools were exploited and compared with the usual tools employed in genetic breeding aiming at increasing the predicted genetic gain. In this way, study demonstrated it was possible to attain these goals by selecting soybean genotypes with better genetic potentials (desirable phenotypic traits) in different evaluation environments.

Material and methods

Experimental data

Three populations (Pop) belonging to the Soybean Breeding Program at the Federal University of Viçosa (UFV) were obtained from crosses between divergent inbred lines (Pop1: TMG123RR/M7211RR; Pop2: UFVSCitrinoRR/UFVSTurquezaRR; and Pop3: M7908RR/M7211RR). These lines were classified into different relative maturity groups according to the soybean crop management classification [27], aiming to exploit genetic variability for the selection of productive progeny. Seventy-two, 71, and 60 F2 plants of populations 1, 2, and 3, respectively, were separately bulk-harvested and threshed. One sample was collected from each F2:3 progeny to compose the 203 F2:4 progeny that were used in this study by the within-progeny bulk method [28,29]. To evaluate these progeny, two trials were conducted in the 2016/2017 crop year at the Teaching, Research, and Academic-Extension Units at UFV. One of them took place in Capinópolis—MG, Brazil (18°40'48" S latitude, 49°33'58" W longitude; 530 m altitude) and the other in Viçosa—MG, Brazil (20º45'45" S latitude, 42º49'27" W longitude; 647 m altitude). The experiments were set up as a randomized complete block design with three replicates per site. Plots consisted of two 3.0-m rows spaced 0.5 m apart, with a plant stand density of 13 seeds per meter, totaling a density of 256,000 plants ha–1. All plant management operations were undertaken in accordance with the requirements of the crop in the region [30]. Three target agronomic traits were evaluated, namely: number of days to maturity (DM, days), 100-seed weight (SW, g), and average seed yield per plot (SY, g). The first variable, DM, corresponds to the number of days before 95% of the pods were mature, as indicated by their color [31]. To evaluate the SW and SY traits, the grains were dried to 13% moisture. All data and code to run the main models used in this study are available in S1 Table and S2 Table, respectively.

Frequentist statistical analyses

The Restricted Maximum Likelihood/Best Linear Unbiased Prediction (REML/BLUP) procedure was adopted for statistical analyses under a frequentist approach (Patterson and Thompson [32] and Henderson [33]). The frequentist single-trait multi-environment (FSTME) statistical model associated with the evaluation of segregating progeny in a randomized block design in two environments, with one observation per plot, is given by: where is the vector of phenotypes; is the vector of block effects added to the overall mean (assumed fixed); is the vector of progeny effects (assumed random), in which ; is the vector of the G×E interaction effects (random), in which ; and is the vector of residuals (random), in which . The capital letters (X, Z and W) represent the incidence matrices for the effects of , and , respectively. The vector encompasses all replicates of all locations. In this case, this vector comprises the effects of locations and of replicates within locations. The goodness of fit was obtained using Akaike information criteria (AIC [34]), defined by AIC = −2LogL+2p, in which LogL is the log-likelihood function and p is the number of estimated parameters, according to Little et al. [35]; and the Likelihood Ratio Test (LRT), following Wilks [36], using Chi-square Statistics with one degree of freedom, which is calculated by the following equation: λ = 2[LogL−LogL], in which L and L are the maximum likelihood associated with the full and the reduced models, respectively. In the frequentist multi-trait multi-environment (FMTME) approach, this model was transformed into ~(0,∑⊗), ~(0,∑int⊗) and ~(0,∑⊗), where ∑ is the progeny covariance matrix; ∑int is the G×E interaction covariance matrix; ∑ is the residual covariance matrix; I is an identity matrix with order appropriate to the respective random effect; and ⊗ denotes the Kronecker product. ∑, ∑int, and ∑ are also unstructured covariance structures (US) [18,19]. The following , , and are the vectors of observed responses for each trait (DM, SW, and SY) and (co)variance matrix estimates: To perform the statistical analyses of the FSTME and FMTME models and obtain the variance components and breeding values, we applied the ASReml 4.1 package [21] of integrated R software (Development Core Team—[37]).

Bayesian statistical analyses

The single-trait multi-environment and multi-trait multi-environment models as well as the covariance matrix structures (US) of the frequentist analyses were used according to the Bayesian approach (BSTME and BMTME, respectively) via Markov Chain Monte Carlo (MCMC) to estimate the variance components and genetic parameter. To gather further information about representation of model, matrix, and vectors structures as well as priori probability distributions, we recommend reading Mrode [18] and Gilmour et al. [21] and recent publications Junqueira et al. [23], Torres et al. [25] and Mora and Serra [38]. We assumed that the variance-covariance matrices follow an Inverse-Wishart (IW) distribution and independent Inverse-Gamma (IG) distributions, which were used as a priori to model the variance-covariance matrix [39-41]. The ensuing covariance matrix distribution is such that all standard-deviation parameters have Half-t distributions and the correlation parameters have uniform distributions on (-1,1) for a particular choice of the IW shape parameter [13]. The advantage of this approach is that it allows us to choose the shape and scale parameters that achieve high non-arbitrary information of all standard deviations and correlation parameters [40]. The full (considering the genotype and G × E interaction effects) models were compared with the reduced (disregarding the genotype or G × E interaction effects) models by the deviance information criterion (DIC) proposed by Spiegelhalter et al. [42]: , where is a point estimate of the deviance obtained by replacing the parameters with their posterior mean estimates in the likelihood function and p is the effective number of parameters in the model. Models with lower DIC should be preferred over models with higher DIC. The Bayesian models (BSTME and BMTME) were implemented in the “MCMCglmm” package [43,44] of R software (R Development Core Team—[37]). A total of 1,000,000 samples were generated, and assuming a burn-in period and sampling interval of 500,000 and 5 iterations, respectively, this resulted in a final total of 100,000 samples. The convergence of MCMC was checked by the criterion of Geweke [45], which was performed using the “boa” [46] and “CODA” (Convergence Diagnosis and Output Analysis) [47] R packages.

Genetic and non-genetic components

Broad-sense heritability per plot (or heritability of total effects from progeny) for the frequentist and Bayesian models were computed based on the approximated estimators, as discussed in Piepho et al. [48], using the following expression: where : variance of progeny; : variance of the progeny × environment interaction; : error variance; n: number of locations; and r: number of replicates. For the Bayesian models, the posterior estimates were calculated from the posterior samples of the variance components obtained by the model. The accuracy of progeny selection for the frequentist models (FSTME and FMTME) was estimated based on the following expression [49]: where is the genotypic variance and PEV is the prediction error variance extracted from the diagonal of the generalized inverse of the coefficient matrix of the mixed-model equations. For the Bayesian approach (BSTME and BMTME models), the accuracy of progeny selection was estimated according to Resende et al. [19] from the posterior distribution, given by: where is the standard deviation of the predicted breeding value. According to Resende et al. [19], in Bayesian inference, the variance of the very parameter that is assumed as a random variable is computed. The “boa” [46] and “bayesplot” [50,51] packages of R software were used to calculate and plot the highest posterior density (HPD) intervals for all parameters, respectively. Estimates of the coefficient of experimental variation (CVe) and selective accuracy () were used to evaluate the experimental quality of the models [52].

Genetic correlation

To determine the genetic covariance by the frequentists and Bayesian single-trait models (FSTME and BSTME, respectively), a pairwise analysis of the sum of phenotypic values of the traits was performed. Thus, the covariances were obtained, as proposed by Resende et al. [53], using the following expression: where is the variance of the sum of phenotypic values of traits i and j; is the genotypic variance of trait i; and is the genotypic variance of trait j. Genetic covariances by the multi-trait multi-environment models (FMTME and BMTME models) were obtained directly by the mixed-model output from each applied methodology. The genetic correlation coefficients between the DM, SW, and SY traits were obtained, as suggested by Piepho et al. [54], using the expression below for all models:

Progeny selection

The Spearman rank correlation coefficient was calculated between the BLUP (breeding value) of the progeny from the analyses of FSTME, FMTME, BSTME, and BMTME, and its significance was verified using nonparametric bootstrap in the R package ‘boot’ [55,56]. The agreement between the selected progeny was also checked by the coincidence index (CI) proposed by Hamblin and Zimmermann [57], as shown below: where A is the number of coincident progeny in the two methods, M is the number of selected progeny, and C is the number of progeny coincident due to chance (C = bM, where b is the selection intensity = 0.15; i.e., 15%). Selection gain was predicted for each trait (DM, SW, and SY) based on the expression below: where GV is the predicted genotypic value of progeny i and n is the number of selected progeny (30). In order to perform a simultaneous selection and infer about the efficiency of selection gain for each evaluated trait between the frequentist (FSTME and FMTME) and Bayesian (BSTME and BMTME) models, we applied the additive genetic index using Selegen REML/BLUP software (AGI–[58]). The predicted breeding values of the selected progeny were thus compared by the frequentist and Bayesian approaches. The weights for each trait were defined based on the coefficients of genetic variation [59]. For all traits, the progeny were selected to increase the phenotypic expression, or provide the highest BLUP. After the direction of selection was defined, the genotypic values for each progeny [weighted by the pre-established weights (CV) for each trait] were summed, generating the AGI value. Subsequently, they were organized in descending order.

Results

Analysis of deviance and model fitting

The significance of progeny (G) and G×E interaction effects of the FSTME model were evaluated. Significant G and G×E interaction effects (P ≤ 0.01) were detected in the LRT for the DM, SW, and SY traits (Table 1). According to the AIC from the results obtained with the FSTME model, the model including the G and G×E interaction effects (full model) showed the best fit (lowest AIC value) for all traits (Table 2). Thus, according to the two methodologies, the full FSTME model was the most suitable to estimate the genetic parameters and predict the genotypic values. Likewise, the FMTME model was also appropriate, as it showed the lowest AIC value. In the Bayesian models, all chains achieved convergence by the criterion of Geweke [45]. Overall, the DIC values were smaller when using the full Model (considering genotype and G × E interaction effects), being the difference in relation to full Model higher than 2 (Table 2), which according to Spiegelhalter et al. [42] it’s enough to suggest that the use of full Model can lead to higher accuracy in estimating the parameters (Table 2). Therefore, since this model component is important the “best” genotypes measured in different environments couldn’t the same.
Table 1

Deviance and likelihood ratio test (LRT) for number of days to maturity (DM), 100-seed weight (SW) (grams), and average seed yield per plot (SY) (grams) evaluated in 203 soybean F2:4 progeny for single-trait multi-environment (FSTME) frequentist analysis.

EffectDMSWSY
DevianceLRTDevianceLRTDevianceLRT
Progeny (G)7048.74141.17**2398.0494.15**5837.9715.68**
G×E interaction7010.14102.57**2362.5758.68**5853.3831.09**
Full model6907.572303.895822.29

** Significant at the 0.01 probability level according to the chi-square test.

Table 2

Akaike information criteria for the full model and deviance information criteria for the full (considering genotype and G × E interaction effects) and reduced [disregarding the genotype (−prog) and interaction (−int) effects] models for number of days to maturity (DM), 100-seed weight (SW) (grams), and average seed yield per plot (SY) (grams) via frequentist and Bayesian single-trait multi-environment (FSTME and BSTME) and multi-trait multi-environment (FMTME and BMTME) models.

  Akaike information criteria (AIC)
ModelTraitFull modelReduced model1 (−prog)Reduced model2 (−int)
FMTMEDM, SW, SY14861.2215184.5315051.87
FSTMEDM6913.5697052.7437014.143
FSTMESW2309.8922402.0422366.568
FSTMESY5828.2915841.9695857.377
Deviance information criteria (DIC)
BMTMEDM, SW, SY20577.320694.7121019.89
BSTMEDM8700.4678741.2438938.965
BSTMESW4157.5474198.9514303.262
BSTMESY7890.4097900.897968.122
** Significant at the 0.01 probability level according to the chi-square test. The estimates of genotypic variance and G×E interaction variance were higher and lower, respectively, when obtained via FMTME and BMTME, for all traits. By contrast, the residual variance () and individual phenotypic variance () estimates were similar regardless of the procedure used for the frequentist models, whereas in the Bayesian models these estimates were higher for all traits, even for the genotypic component . However, the and estimates were similar between the BSTME and BMTME models. As a consequence, both the frequentist and Bayesian MTME models led to higher estimates of , CV, and Ac compared with the frequentists and Bayesian single-trait models. The exception was the SW trait, for which no significant differences were observed (Table 3) between single- and multi-trait multi-environment models. For the other components, both procedures led to results proportional to the estimates of and .
Table 3

Estimates of variance components and genetic and non-genetic parameters for number of days to maturity (DM), 100-seed weight (SW) (grams), and average seed yield per plot (SY) (grams) evaluated in 203 soybean F2:4 progeny via frequentist single-trait multi-environment (FSTME) and multi-trait multi-environment (FMTME) and Bayesian single-trait multi-environment (BSTME) and multi-trait multi-environment (BMTME) models.

ComponentFSTMEFMTMEBSTMEBMTME
DMSWSYDMSWSYDMSWSYDMSWSY
σg2128.6251.80458.6171130.9471.808410.4882129.5981.81588.3801131.6161.816111.0459
σint233.92280.624410.072731.67410.62288.299534.22040.627810.283431.29170.62536.9344
σres256.52531.500836.456656.52341.500636.441956.74991.508136.666557.10891.515237.5989
σphen2219.0733.929755.1464219.1443.931855.2297220.5693.951855.3299220.0173.956655.5792
hprog20.82980.76240.43680.83830.76310.5064Table 4aTable 4 aTable 4 aTable 4 aTable 4 aTable 4a
Cint20.15480.15890.18270.14450.15840.15030.15520.15880.18590.14220.1580.1248
CVg0.07910.07290.1410.07980.0730.15550.07940.07320.13900.08010.07320.1596
CVe0.05240.06650.29010.05240.06650.28990.05250.06660.29080.05270.06680.2945
Acprog0.90870.86910.65570.94850.87130.9330.96330.63370.73670.97330.63550.8941

: genotypic variance; : genotype × environment interaction variance; : residual variance; : individual phenotypic variance; : mean broad-sense heritability per plot; : coefficient of determination of the genotype × environment interaction effects; CV: genotypic coefficient of variation; CV: residual coefficient variation; and Ac: mean accuracy of progeny.

aResults showed in Table 4.

: genotypic variance; : genotype × environment interaction variance; : residual variance; : individual phenotypic variance; : mean broad-sense heritability per plot; : coefficient of determination of the genotype × environment interaction effects; CV: genotypic coefficient of variation; CV: residual coefficient variation; and Ac: mean accuracy of progeny. aResults showed in Table 4.
Table 4

Posterior inferences for mode, mean, median, and higher posterior density (HPD) interval of the broad-sense heritability per plot, considering the proposed Bayesian single-trait multi-environment (BSTME) and multi-trait multi-environment (BMTME) models for number of days to maturity (DM), 100-seed weight (SW) (grams), and average seed yield per plot (SY) (grams).

BSTMEBMTME
Traithprog2HPD (95%)hprog2HPD (95%)
ModeMeanMedianLowerUpperModeMeanMedianLowerUpper
DM0.83540.82820.82980.77980.8740.8420.83750.83880.79360.8787
SW0.7650.760.76280.69050.82470.7750.7650.76280.69080.8247
SY0.440.42040.42820.24060.5860.53610.52920.5310.43040.6292
Compared with FSTME, the FMTME model provided increases of the orders of 1.81, 0.22, and 21.72% in the estimate of and reductions of 6.63, 0.26, and 17.61% in the estimate of for the respective traits DM, SW, and SY. As a result, the estimates increased by 1.03, 0.1, and 15.94%; CV, by 0.88, 0.14 and 10.28%; and Ac, by 4.38, 0.25, and 42.29% for the respective traits. Compared with BSTME, the BMTME model led to 1.56, 0.02, and 31.81% higher estimates of and 8.56, 0.4, and 32.57% lower estimates of for the respective traits DM, SW, and SY. The reduced ranging of (lower and upper difference) with Bayesian MTME credible intervals (probability of 95%) were 9.66, 0.22, and 42.44% for the DM, SW, and SY traits, respectively (Table 4). Thus, the mean increased by 1.08, 0.00, and 24.01%; CV increased by 0.88, 0.14, and 10.82%; and Ac increased by 1.04, 0.28, and 21.37% for the respective traits. The posterior density intervals of heritability genetic parameters (Table 4, and Figs 1–3) were accessed to assist in the selection of genotypes in the Bayesian models. Thus, the breeding values and their HPD intervals obtained from the Bayesian STME and MTME models for each trait can be useful tools in progeny selection. Posterior density intervals of estimates of genotypic variance for BSTME and BMTME were showed in S1–S4 Figs.
Fig 1

Posterior density for the Bayesian single-trait multi-environment (BSTME) (top) and multi-trait multi-environment (BMTME) (bottom) models of the broad-sense heritability per plot for number of days to maturity (DM). The solid color represents the posterior density of 95% intervals and the solid vertical line indicates the mean.

Fig 3

Posterior density for the Bayesian single-trait multi-environment (BSTME) (top) and multi-trait multi-environment (BMTME) (bottom) models of the broad-sense heritability per plot for average seed yield per plot (SY; grams). The solid color represents the posterior density of 95% intervals and the solid vertical line indicates the mean.

Posterior density for the Bayesian single-trait multi-environment (BSTME) (top) and multi-trait multi-environment (BMTME) (bottom) models of the broad-sense heritability per plot for number of days to maturity (DM). The solid color represents the posterior density of 95% intervals and the solid vertical line indicates the mean. Posterior density for the Bayesian single-trait multi-environment (BSTME) (top) and multi-trait multi-environment (BMTME) (bottom) models of the broad-sense heritability per plot for 100-seed weight (SW; grams). The solid color represents the posterior density of 95% intervals and the solid vertical line indicates the mean. Posterior density for the Bayesian single-trait multi-environment (BSTME) (top) and multi-trait multi-environment (BMTME) (bottom) models of the broad-sense heritability per plot for average seed yield per plot (SY; grams). The solid color represents the posterior density of 95% intervals and the solid vertical line indicates the mean. Genetic correlations between the DM, SW, and SY traits obtained by the frequentist and Bayesian models are given in Table 5. For the DM-SW and SW-SY pairs, low correlations were detected in every comparison, indicating absence of linear associations. However, a high and positive correlation was found between DM and SY in both methodologies. Additionally, the correlation between this pair of traits estimated via FSTME and BSTME exceeded the parameter space. The same was not true when it was estimated via FMTME and BMTME, which showed similar values within the parameter space and were thus more realistic (not biased).
Table 5

Genetic correlations between number of days to maturity (DM), 100-seed weight (SW) (grams), and average seed yield per plot (SY) (grams) evaluated in 203 soybean F2:4 progeny via frequentist single-trait multi-environment (FSTME) and multi-trait multi-environment (FMTME) and Bayesian single-trait multi-environment (BSTME) and multi-trait multi-environment (BMTME) models.

CorrelationFSTMEFMTMEBSTMEBMTME
ρ (DM,SW)–0.1090–0.1107–0.1048–0.1078
ρ (DM,SY)0.97180.9724
ρ (SW,SY)–0.0153–0.0217–0.0164–0.0296

¶: value higher than unity

¶: value higher than unity The Spearman rank correlations between breeding values via single- and multi-trait multi-environment models (FSTME and FMTME; BSTME and BMTME) were significant in all comparisons. This correlation was medium for SY (79.16 and 70.92) in the frequentist and Bayesian models, respectively, and high for the other comparisons. These results were confirmed by the agreement (Table 6).
Table 6

Predicted selection gain, agreement, and Spearman rank correlation in the selection of the 30 best soybean progeny via frequentist (FSTME and FMTME) and Bayesian (BSTME and BMTME) models for number of days to maturity (DM), 100-seed weight (SW; grams), and average seed yield per plot (SY; grams) evaluated in 203 soybean F2:4 progeny.

TraitPredicted selection gainAgreement (%)
FSTMEFMTMEBSTMEBMTMEFSTME × FMTMEBSTME × BMTME
DM12.7013.2212.6713.3293.30 (0.9349b)93.31 (0.9289)
SW1.921.931.921.92100.00 (0.9963)100.00 (0.9965)
SY3.113.742.993.8446.40 (0.7916)46.60 (0.7992)

b Spearman rank correlation

b Spearman rank correlation In the frequentist and Bayesian analyses, the predicted selection gains were equivalent for all traits. However, the MTME models showed greater gains. For the SW trait, both the frequentist and the Bayesian procedures generated very similar results (Table 6). For DM and SY, however, there was less agreement between the selected progeny, especially for the SY trait, which culminated in greater discrepancy between the gains predicted from selection. For the DM trait, the MTME models led to increased gains predicted from selection: 4.01 and 5.13% for the frequentist and Bayesian methodologies, respectively. Considering the SY trait, for which the agreement between the selected progeny was lower than 50%, the MTME models showed increases of 20.26% and 28.43% (frequentist and Bayesian models, respectively) in the gain predicted from selection, as compared with their respective single-trait models (Table 6). When the AGI was used for the simultaneous selection of the 30 best soybean progeny, higher index gains were found for the FMTME and BMTME models (Tables 7 and 8) compared with the gain of the overall mean of AGI. The gains predicted from selection were similar for both models (FMTME and BMTME), for all traits. Moreover, greater gains were observed for the SY variable (15.03 and 15.71% for FMTME and BMTME, respectively).
Table 7

Order, progeny (Prog), breeding value (u+g), and Additive Genetic Index (AGI) of the 30 progeny selected simultaneously via frequentist single-trait multi-environment (FSTME) and multi-trait multi-environment (FMTME) models for number of days to maturity (DM), 100-seed weight (SW; grams), and average seed yield per plot (SY; grams) evaluated in 203 soybean F2:4 progeny.

  FSTME  FMTME 
OrderProgTraitAGIProgTraitAGI
DMSWSYDMSWSY
1521156.9220.6623.7031.39521158.3120.5425.4929.79
2267148.0620.3626.5131.13550157.5020.2225.1529.47
3550157.0520.3422.9331.02267155.9520.2225.0629.33
4556152.9119.9822.8330.44556153.8719.9024.0928.80
5235152.0819.1325.3130.38235157.9719.0025.2528.72
6545149.8620.1623.0730.36537151.5620.1423.4628.67
7537151.6620.2022.1730.33545151.6520.0823.5528.66
8554151.3820.3221.8330.31562157.6519.0324.8628.66
9520153.1819.7722.4930.24554150.6820.2723.2128.65
10262154.5618.1726.4430.20262162.6817.9926.4328.63
1156154.1519.8021.7930.18520153.6419.6923.9628.63
12562158.5819.1221.9830.1456153.2119.7423.7928.60
13568157.4719.6020.8130.10568154.4119.5523.9828.59
14272154.7019.2422.6530.06567153.7019.6123.7228.54
15515149.7220.6320.3530.04272155.4919.1624.3728.49
16266147.9319.0325.4530.01515146.5520.6222.0428.36
17567158.4419.6319.8629.98215155.1018.9724.3528.33
18215151.3819.0723.9629.97569155.4118.9124.1328.28
19546145.0220.0322.6729.80266154.8018.9124.3928.28
20571154.1519.5020.7729.76269157.1118.4924.6828.23
21269156.0918.5822.6929.75571151.6419.4623.2228.20
22569158.0218.9720.9029.7551155.2618.8024.0528.19
23514151.8019.4921.6029.75239155.6518.6024.3428.13
24531151.5219.6920.9629.71514151.1219.4423.1528.13
25239153.6018.6923.0829.70531149.6619.6522.7428.09
2651158.1618.8320.6129.61546146.9919.9822.2528.02
27212150.1418.0925.5929.60524152.5719.0123.4728.02
28516150.4219.5821.2029.60544149.8719.5122.7128.01
29524153.0419.0721.7029.59512154.7018.5823.9727.99
30544152.9119.5420.3629.59516149.2319.5422.6227.96
Mean of sel.153.1619.5122.5430.08153.8019.4523.9528.48
Overall mean143.4518.4320.8228.21143.4518.4320.8226.49
Gain (%)6.775.858.276.64 7.215.5515.037.50
Table 8

Order, progeny (Prog), breeding value (u+g), and Additive Genetic Index (AGI) of the 30 progeny selected simultaneously via Bayesian single-trait multi-environment (BSTME) and multi-trait multi-environment (BMTME) models for number of days to maturity (DM), 100-seed weight (SW; grams), and average seed yield per plot (SY; grams) evaluated in 203 soybean F2:4 progeny.

  BSTME  BMTME 
OrderProgTraitAGIProgTraitAGI
DMSWSYDMSWSY
1521156.8620.6523.6031.23521158.4920.5325.5829.96
2267148.0620.3726.2830.95550157.6120.2225.2329.64
3550157.0220.3422.8530.86267156.4820.2225.2329.50
4556152.9019.9822.7530.28556154.0019.9024.1628.96
5235152.0819.1325.1430.21235158.3019.0125.4128.90
6545149.8720.1622.9930.20537151.6420.1323.4928.83
7537151.6220.1922.1230.17562157.5919.0324.9228.82
8554151.3820.3221.8030.16545151.8320.0823.6228.82
9520153.1619.7722.4230.09262163.1518.0126.6828.81
1056154.1319.8121.7430.03554150.7320.2623.2428.80
11262154.5518.1826.2430.03520153.7319.6924.0128.79
12562158.5419.1221.9429.9856153.2219.7423.8228.75
13568157.4819.6020.8129.95568154.2819.5423.9928.75
14272154.6819.2522.5829.90567153.4919.6023.7028.70
15515149.6920.6320.3729.90272155.5319.1624.4328.65
16567158.4219.6319.8929.84515146.4920.6022.0028.50
17266147.9219.0325.2729.84215155.3018.9724.4528.50
18215151.3719.0723.8429.81266155.1818.9224.5528.44
19546145.0220.0422.6029.64569155.2618.9124.1528.44
20571154.1019.5020.7729.61269157.1318.5024.7628.40
21569157.9818.9720.9029.60571151.5419.4523.2228.35
22514151.7719.4921.5529.5951155.0618.8024.0628.35
23269156.0718.5922.6129.59239155.7518.6024.4328.30
24531151.5119.6920.9629.56514151.1319.4423.1828.28
25239153.5718.6922.9929.54531149.6119.6522.7428.24
2651158.1518.8320.6229.46524152.5619.0123.5128.17
27516150.4019.5721.1929.45546147.2019.9922.2928.16
28524153.0419.0721.6629.44544149.7719.5022.7028.16
29544152.8919.5420.3629.44512154.6918.5824.0328.15
30212150.1118.0925.4129.43212157.8017.9725.1128.13
Mean of sel.153.1419.5122.4729.93154.1519.4024.0928.64
Overall mean143.4518.4320.8228.06143.4518.4320.8226.63
Gain (%)6.765.867.956.64 7.465.2615.717.56

Discussion

Analysis of deviance

The LRT for the FSTME model revealed that the progeny and G×E interaction effects are significant (P < 0.01) for the DM, SW, and SY traits. Consequently, the respective variance components are significantly different from zero and so are the respective coefficients of determination (Table 1). The fit of the frequentist models was checked by AIC. This criterion indicated the full model as the most suitable to estimate the variance components and predict the genotypic values (Table 2). For the Bayesian approach, the full (considering genotype and G × E interaction effects) and reduced (only genotype or G x E interactions effects) models were compared through DIC (Deviance Information Criterion), which suggests that models with smaller DIC are better supported by the data. According to Spiegelhalter et al. [42], models with differences in DIC values lower than 2 need to be considered as equally well. Therefore, since DIC values obtained were higher than 2, it is possible to indicate the superiority of full model over the restrict models. The generalization of AIC is the most common method of assessing the fit of a statistical model estimated via Bayesian inference (DIC). The effects of Bayesian models can be used as an inference to the test of hypothesis [19]. In Bayesian statistics, the lowest expected deviance has the HPD [60], and this was observed in the present study for the MTME model (Tables 2 and 3). In both criteria (AIC and DIC) for the choice of statistical models, the obtained results revealed that the MTME models showed the best fit, explaining the genetic variability of the experiment and selection considering the genetic (progeny) and environmental interaction effects (Table 2).

Variance components

Variance components are the variances associated with the random effects of a model. Knowing them is of great importance in genetics and breeding, since the population and the breeding method to be used depend on information that can be obtained from these components. The solution of mixed-model equations depends on knowledge of the variance and covariance matrix, whose structure is known, but its components often are not. At present, the standard method for the estimation of variance components is REML, developed by Patterson and Thompson [32]. The BLUP method [33] maximizes the correlation between the predicted and the true genotypic value; i.e., it minimizes the prediction error variance (PEV). Additionally, it is not biased, as we expect the predicted genotypic value to be equal to the true genotypic value [61]. Further, BLUP allows for the simultaneous use of several sources of information as well as information originating from experiments carried out in one or various locations and evaluated in one or various harvests [62]. Although the mixed-model methodology by the frequentist approach has several desirable characteristics [49], the adoption of Bayesian statistical inference for genetic evaluation in the breeding of crop species has shown to be advantageous. Bayesian models have been used since 1986 [63] and further exploited in recent years [23-25,64,65] due to the great computational advancements and new methodological applications and elucidations. Bayesian analysis is based on the knowledge of the posterior distribution of the parameters to be estimated. This allows for the construction of exact credibility intervals for the estimates of random variables, variance components, and fixed effects [66]. Higher values for the interval with 95% credibility of distribution for the broad-sense heritability parameter found in this study (Table 4) were also presented in the study of Torres et al. [25] to estimate genetic parameters for N-uptake efficiency and N-utilization efficiency under contrasting N levels in the soil via BMTME models. The difference between mean, mode, and median of broad-sense heritability estimates (Table 4) reflects some lack of symmetry in the posterior distribution estimates [38]. However, for the SY trait, differences between the BSTME and BMTME are clear when we analyze the posterior densities, mainly because the posterior MTME resulted in a more narrow and symmetric distribution, confirming the increase in precision (Fig 3). When the prior distribution is informative, the credibility interval tends to be narrower than the confidence intervals. When the mixed-model parameters are assigned non-informative distributions, Bayesian and frequentist inferences should be equivalent [67]. Mathew et al. [68] showed that Bayesian inference is superior to frequentist inference when the posterior distribution of a variance component is bimodal. Mathew et al. [68], Waldmann et al. [69], Schenkel et al. [70], and Harville et al. [71] did not find relevant differences between the breeding values predicted by frequentist or Bayesian approach. Schenkel et al. [70] also observed that the breeding values presented the same bias and accuracy. Silva et al. [72] found results from noninformative analyses and results from REML/BLUP analyses (frequentist) for some components of variance and heritability and for breeding values. The specific results obtained by the frequentist and Bayesian approaches were similar (Table 3). This was expected, since non-informative prior distributions were used in Bayesian analysis. The modes of the marginal posterior distributions of the genetic parameters were similar to the corresponding REML estimates. From the Bayesian point-of-view, the estimates obtained via REML correspond to the modes of the combined posterior distributions of the variance components, obtained by Bayesian approach, given the use of uniform priors for the fixed effects and variance components [66]. The frequentist and Bayesian MTME models provided higher estimates and lower estimates, which resulted in higher for all evaluated traits. The highest estimates were found for DM and SW and the lowest for SY, which confirms that DM and SW are less complex traits and are thus less influenced by the environment than SY [73-75]. The genotypic coefficient of variation (CV) quantifies the magnitude of genetic variation available for selection, and thus high values are desirable [76]. In this way, the increase seen in this parameter with the use of the MTME models is important for breeding programs. The residual coefficient of variation (CV) is a measurement of experimental precision of statistical and non-genetic nature. According to Resende and Duarte [52], CV is of moderate magnitude for the SY trait and low magnitude for the DM and SW traits, indicating good experimental precision. Moreover, as expected, there were no alterations in the CV estimates when the FSTME and FMTME models were used (Table 3).

Genetic correlations

Studies on genotypic, phenotypic, and environmental correlations in soybean involve traits that are evaluated from flowering to maturity; notably, yield and its components [77-80]. Our results corroborate those reported by Cober et al. [81], who also obtained a high correlation between DM and SY and no linear associations between the DM-SW and SW-SY pairs. The authors argued that the genes controlling maturity in soybean have pleiotropic effects with grain yield. Ablett et al. [82] and Lee et al. [83] reported that late maturity was associated with high yield. Liu et al. [84] investigated the genetic architecture of three growth period traits and confirmed that the soybean growth stages are highly correlated with grain yield. Li et al. [85] and Zhang et al. [75] observed that the association between the BARC-016957-02165 marker and seed yield was located in the same region as a QTL controlling pod maturity on chromosome 6, which explains the high correlation observed between these traits, as the QTL were closer to each other. According to Pollak et al. [14], selection biases may occur when traits are analyzed individually. This bias was observed in the present study, where the genetic correlation value between the DM and SY traits exceeded the parameter space (value higher than 1) (Table 5). Viana et al. [15] evaluated two traits in popcorn and also found that the genetic correlation obtained with the single-trait model exceeded the parameter space. According to Thompson and Meyer [86], the increase in accuracy obtained with the use of multi-trait BLUP analysis compared with single-trait analysis is proportional to the difference between the genetic and environmental correlations of the analyzed traits. In the context of whole-genome prediction, Jia et al. [87], Guo et al. [88], and Jiang et al. [22] found that joint prediction of multiple traits benefits from genetic correlations between traits and significantly improves prediction accuracy compared to single-trait methods, specifically for low-heritability traits that are genetically correlated with a high-heritability trait. This fact was observed in our study, in which the DM variable showed high heritability and high correlation with SY, consequently generating significant increases in selection accuracy for SY. However, for both methodologies—frequentist and Bayesian—there was no significant increase in selection accuracy for the SW trait, as verified by its low correlation with the other evaluated traits. The observed differences in the genotypic values predicted by the Bayesian and BLUP/REML procedures were small, leading to a slight alteration in the ranking of the progeny selected by both procedures. This finding was confirmed by Rank Spearman correlation (Table 6). However, compared with the FSTME and BSTME models, the respective MTME models showed higher genetic gains for DM and, especially, for SY. According to Resende et al. [66], these are the conditions for there to be correspondence between the frequentist and Bayesian methodologies for the fixed and random effect parameters: attribution of non-informative priori for the fixed effects, normal priori for the random effects, and normal likelihood for the observations vector. These promises were used in the present study, which explains the obtained results. Despite the high agreement between the progeny selected for the DM and SW traits by both procedures, there was little agreement for the SY trait, which resulted in greater gains predicted from selection via MTME. Piepho et al. [3] and Piepho et al. [62] recommended the use of multiple-trait models to predict breeding values in annual crops, because this procedure has the best statistical properties and provides more-accurate results. Resende et al. [19] and Okeke et al. [89] stated that one of the main advantages of using multivariate models is higher selection accuracy. Okeke et al. [89] also reported that multi-environment models were useful for understanding G×E interactions. Higher Ac were observed when obtained via MTME for all traits, with SY standing out with 42.29 and 21.37% increases in the frequentist and Bayesian models, respectively. Greater accuracy and efficiency of multiple-trait models were also reported by Viana et al. [15] in selection among and within half-sib families. However, it must be stressed that the BSTME model obtained superior accuracy in comparison with the FSTME model for DM and SY, despite the similar broad-sense heritability values. This is explained by the use of the estimator of selection accuracy. In this regard, Resende et al. [19] described that when Bayesian accuracy is higher than frequentist accuracy, the distribution of the parameters attributed to Bayesian approach were probably more adequate than those associated with the traditional model. The opposite can be considered true for the SW variable, for which the frequentist models obtained better results due to a better adjustment of the normal distribution of the parameters attributed to the data. These conclusions are also valid for the MTME models, which exhibited different obtained accuracies; however, for the SY variable, the FMTME model showed the best fit according to the mean accuracy of the progeny (Table 3). As can be seen in Tables 7 and 8, desirable gains are obtained in selecting the best progeny for the DM, SW, and SY variables for all models based on the AGI. However, the high positive correlation between the DM and SY traits can favor the selection of high-yielding and late-cycle progeny. In this case, selection indices can help breeders select progeny that exhibit gains for both traits simultaneously [90]. Although similar gains were found for both approaches employed, it should be stressed that there was a slight increase in gain (%) when the 30 best progeny were selected for the DM and SY traits using Bayesian approach. This is a desirable factor that should be taken into account by breeders. According to Silva et al. [1], individual plants or progeny in the F2:3 or F2:4 generations can be selected aiming at the adoption of the recurrent selection method described by Hallauer et al. [91], by exploiting genetic variability among and within progeny. Silva et al. [1] stated that the F2:4 generation is suitable for selection, since 87.5% (1.75) of the total additive genetic variance () that will be available in F∞ is already available in F2:4. Thus, progeny selection in F2:4 through a more precise method is relevant. Early in the generation of the base populations of the soybean breeding programs, many populations are commonly obtained at the expense of the number of progeny to be evaluated; i.e., the evaluation of future lines, be them for high or low heritability, is based on samples with a finite (small) number of progeny. Thus, Bayes' theorem is recommended for those situations, as it gives precise solutions to the problem of finite-size samples, because for each data set—large or small—there is an exact posterior distribution to draw inferences. The MTME models provided better results than the single-trait models using frequentist and Bayesian approach. Therefore, the former procedures can be efficiently applied in the genetic selection of segregating soybean progeny. However, it is necessary to use an adequate statistical tool that provides algorithms and routines to efficiently perform the analyses. Though not necessarily easy, the use of Bayesian inference in quantitative genetics in the breeding of crop species [69,72] is a tendency in breeding programs [5].

Conclusion remarks

For our data set, the average BMTME processing time using an Intel(R) i7-5500U (2.4 GHz) processor with 8 GB of RAM was 1 h 40 min and 35 s, corresponding to approximately 0.006 s for each MCMC iteration. Silva et al. [72] considered this performance plausible, but pointed out that improvements can be obtained using the conditional decompositions proposed by Hallander et al. [92]. For the same purpose, in addition to improving the prior information, Montesinos-López et al. [13] proposed a Bayesian model for analyzing multiple traits and multiple environments for the whole-genome prediction model. The authors also developed an R-software package that offers specialized and optimized routines to efficiently perform the analyses under the proposed model. By contrast, the FMTME model took approximately 14 s to converge. Despite the considerable difference in processing time of the analysis and output size of the results (around 1.03 GB) due to the high number of interactions adopted, the Bayesian model showed to be efficient for the proposed objective. Furthermore, it provided additional results to those obtained by the frequentist approach, with noteworthy credibility intervals. The Bayesian models have desirable potentials when using informative prior distributions, providing parameters with lower standard deviations and/or possible genetic gains. However, the quality of the informative prior may have questionable origins and may not generate considerable advantages. Silva et al. [72] showed it can be advantageous to implement a Bayesian framework for mixed-model analysis in the breeding of crop species using informative priors. However, for potential future studies in plant breeding, the implementation of informative prior fitted to MTME models can be the next step to be assessed.

Data set necessary to replicate the findings of our research.

(TXT) Click here for additional data file.

Scripts to run the multi-trait multi-environment models.

Data set necessary to replicate the findings of our research. (TXT) Click here for additional data file.

On the diagonal, posterior density of 95% intervals of the estimate of variance components via the Bayesian multi-trait multi-environment (BMTME) model; off the diagonal the traits’ correlations by score for number of days to maturity (DM), 100-seed weight (SW; grams), and average seed yield per plot (SY; grams).

(TIFF) Click here for additional data file.

Posterior density for the Bayesian single-trait multi-environment model (BSTME) of the estimate of variance components for maturity (DM).

The solid color represents the posterior distributions of 95% intervals and the solid vertical line indicates the mean for number of days. (TIFF) Click here for additional data file.

Posterior density for the Bayesian single-trait multi-environment model (BSTME) of the estimate of variance components for 100-seed weight (SW).

The solid color represents the posterior distributions of 95% intervals and the solid vertical line indicates the mean. (TIFF) Click here for additional data file.

Posterior density for the Bayesian single-trait multi-environment model (BSTME) of the estimate of variance components for average seed yield per plot (SY).

The solid color represents the posterior distributions of 95% intervals and the solid vertical line indicates the mean. (TIFF) Click here for additional data file.
  27 in total

1.  Best linear unbiased estimation and prediction under a selection model.

Authors:  C R Henderson
Journal:  Biometrics       Date:  1975-06       Impact factor: 2.571

2.  Computing heritability and selection response from unbalanced plant breeding trials.

Authors:  Hans-Peter Piepho; Jens Möhring
Journal:  Genetics       Date:  2007-11       Impact factor: 4.562

3.  Multiple-trait Gibbs sampler for animal models: flexible programs for Bayesian and likelihood-based (co)variance component inference.

Authors:  C P Van Tassell; L D Van Vleck
Journal:  J Anim Sci       Date:  1996-11       Impact factor: 3.159

4.  Identification of quantitative trait loci for plant height, lodging, and maturity in a soybean population segregating for growth habit.

Authors:  S H Lee; M A Bailey; M A Mian; E R Shipe; D A Ashley; W A Parrott; R S Hussey; H R Boerma
Journal:  Theor Appl Genet       Date:  1996-04       Impact factor: 5.699

5.  Efficient Markov chain Monte Carlo implementation of Bayesian analysis of additive and dominance genetic variances in noninbred pedigrees.

Authors:  Patrik Waldmann; Jon Hallander; Fabian Hoti; Mikko J Sillanpää
Journal:  Genetics       Date:  2008-06       Impact factor: 4.562

6.  Comparison between estimation of breeding values and fixed effects using Bayesian and empirical BLUP estimation under selection on parents and missing pedigree information.

Authors:  Flávio S Schenkel; Lawrence R Schaeffer; Paul J Boettcher
Journal:  Genet Sel Evol       Date:  2002 Jan-Feb       Impact factor: 4.297

7.  Joint prediction of multiple quantitative traits using a Bayesian multivariate antedependence model.

Authors:  J Jiang; Q Zhang; L Ma; J Li; Z Wang; J-F Liu
Journal:  Heredity (Edinb)       Date:  2015-04-15       Impact factor: 3.821

8.  The statistical analysis of multi-environment data: modeling genotype-by-environment interaction and its genetic basis.

Authors:  Marcos Malosetti; Jean-Marcel Ribaut; Fred A van Eeuwijk
Journal:  Front Physiol       Date:  2013-03-12       Impact factor: 4.566

9.  Correction: Bayesian Multi-Trait Analysis Reveals a Useful Tool to Increase Oil Concentration and to Decrease Toxicity in Jatropha curcas L.

Authors:  Vinícius Silva Junqueira; Leonardo de Azevedo Peixoto; Bruno Galvêas Laviola; Leonardo Lopes Bhering; Simone Mendonça; Tania da Silveira Agostini Costa; Rosemar Antoniassi
Journal:  PLoS One       Date:  2016-08-08       Impact factor: 3.240

10.  A Genomic Bayesian Multi-trait and Multi-environment Model.

Authors:  Osval A Montesinos-López; Abelardo Montesinos-López; José Crossa; Fernando H Toledo; Oscar Pérez-Hernández; Kent M Eskridge; Jessica Rutkoski
Journal:  G3 (Bethesda)       Date:  2016-09-08       Impact factor: 3.154

View more
  5 in total

1.  Multi-trait and multi-environment Bayesian analysis to predict the G x E interaction in flood-irrigated rice.

Authors:  Antônio Carlos da Silva Júnior; Isabela de Castro Sant'Anna; Michele Jorge Silva Siqueira; Cosme Damião Cruz; Camila Ferreira Azevedo; Moyses Nascimento; Plínio César Soares
Journal:  PLoS One       Date:  2022-05-03       Impact factor: 3.752

2.  Genome-wide association study in hexaploid wheat identifies novel genomic regions associated with resistance to root lesion nematode (Pratylenchus thornei).

Authors:  Deepak Kumar; Shiveta Sharma; Rajiv Sharma; Saksham Pundir; Vikas Kumar Singh; Deepti Chaturvedi; Bansa Singh; Sundeep Kumar; Shailendra Sharma
Journal:  Sci Rep       Date:  2021-02-11       Impact factor: 4.379

3.  Multiple-trait model through Bayesian inference applied to Jatropha curcas breeding for bioenergy.

Authors:  Marco Antônio Peixoto; Jeniffer Santana Pinto Coelho Evangelista; Igor Ferreira Coelho; Rodrigo Silva Alves; Bruno Gâlveas Laviola; Fabyano Fonseca E Silva; Marcos Deon Vilela de Resende; Leonardo Lopes Bhering
Journal:  PLoS One       Date:  2021-03-04       Impact factor: 3.240

4.  Genetic Variation in Flowering Traits of Tasmanian Leptospermum scoparium and Association with Provenance Home Site Climatic Factors.

Authors:  Christopher N Wellington; René E Vaillancourt; Brad M Potts; Dale Worledge; Anthony P O'Grady
Journal:  Plants (Basel)       Date:  2022-04-10

5.  SNP and Haplotype-Based Genomic Selection of Quantitative Traits in Eucalyptus globulus.

Authors:  Paulina Ballesta; Carlos Maldonado; Paulino Pérez-Rodríguez; Freddy Mora
Journal:  Plants (Basel)       Date:  2019-09-05
  5 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.