Literature DB >> 35607882

Change-point analysis of lambda-cyhalothrin efficacy against soybean aphid (Aphis glycines Matsumura): identifying practical resistance from field efficacy trials.

James P Menger1, Arthur V Ribeiro1, Bruce D Potter2, Robert L Koch1.   

Abstract

BACKGROUND: Soybean aphid (Aphis glycines Matsumura) remains the most economically important arthropod pest of soybean in the Upper Midwest Region of the USA. Soybean aphid resistance to the pyrethroid insecticides emerged in 2015; however, the reduction in the efficacy of field applications of pyrethroid insecticides has not been quantified. Based on time-series data from insecticide efficacy trials at two locations, a novel approach of continuous two-phase change point-regression models was used to indicate whether a change in percent control had occurred, and to provide an indication of when and to what degree the percent control had changed.
RESULTS: At both locations examined in this study, a significant change point for percent control of λ-cyhalothrin was detected in 2014, thus marking the onset of practical resistance in the soybean aphid. Percent control decreased at a rate of 4.30% and 19.90% per year at these locations. By contrast, percent control for chlorpyrifos remained high over time with no significant change point.
CONCLUSION: This research demonstrates that retrospective time-series analysis of insecticide efficacy data can identify the onset and magnitude of practical resistance in the field. This further validates and compliments the other lines of evidence related to pyrethroid resistance in soybean aphid.
© 2022 The Authors. Pest Management Science published by John Wiley & Sons Ltd on behalf of Society of Chemical Industry. © 2022 The Authors. Pest Management Science published by John Wiley & Sons Ltd on behalf of Society of Chemical Industry.

Entities:  

Keywords:  efficacy; insecticide; percent control; pest management; practical resistance; pyrethroid

Mesh:

Substances:

Year:  2022        PMID: 35607882      PMCID: PMC9544874          DOI: 10.1002/ps.7006

Source DB:  PubMed          Journal:  Pest Manag Sci        ISSN: 1526-498X            Impact factor:   4.462


INTRODUCTION

Soybean aphid (Aphis glycines Matsumura) (Hemiptera: Aphididae) persists as a significant pest in the Midwest United States (US) more than 20 years after its initial discovery in North America. Despite extensive research into several alternative management tactics, including host‐plant resistance and biological control, the use of foliar insecticides has remained the most effective and economical strategy for managing soybean aphid. , , These foliar applications of insecticides on soybean have relied heavily on organophosphates (Group 1B) and pyrethroids (Group 3A). Insecticide use on soybean in the Midwest increased dramatically after the invasion by soybean aphid. , An overreliance on insecticides can lead to ecological backlash in the form of insecticide resistance. This evolutionary response is frequently associated with the repeated use of insecticides, and resistance typically surfaces as a delayed response after years of seemingly good control. The emergence of insecticide resistance in soybean aphid has posed a significant challenge to soybean production. Reports of field applications of pyrethroid insecticides failing to control soybean aphid were made by growers and consultants in Minnesota in 2015, and in Minnesota and Iowa in 2016. In the following years, reports of such control failures continued in Minnesota and expanded to South Dakota and North Dakota. , In response to these reports, laboratory bioassays were performed and confirmed reduced susceptibility in some field‐collected populations of soybean aphid from Minnesota, North Dakota, South Dakota, Iowa and Manitoba to the pyrethroids λ‐cyhalothrin and bifenthrin, relative to a laboratory‐susceptible population. , More recent work has begun to identify the mechanisms of resistance present in the soybean aphid. Phenotypically resistant populations of soybean aphid with both induced and constitutive overexpression of detoxifying enzyme genes have been reported. Furthermore, point mutations in the voltage‐gated sodium channel genes associated with skdr and kdr have been identified in soybean aphid populations. , It is important to note that such documentation of resistance in the laboratory does not always equate to a reduction in the field efficacy of insecticide applications against the pest. Potential differences in efficacy of insecticides under laboratory and field conditions have been documented elsewhere. , , , Practical resistance is defined as “field‐evolved resistance that reduces pesticide efficacy and has practical consequences for pest control”. More specifically, the Insecticide Resistance Action Committee's definition of resistance specifies “the repeated failure of a product to achieve the expected level of control when used according to the label recommendation for that pest species.” Reductions in the efficacy of field applications of pyrethroid insecticides over time for soybean aphid have not been quantified. However, data to perform such an examination are available from replicated and controlled field experiments (efficacy trials), performed by university researchers at multiple locations and over several years to evaluate the efficacy of various insecticides against soybean aphid. Time‐series analysis can be useful in elucidating underlying trends and systematic patterns in data over time. A class of regression models in which predictors are associated with the outcome in a threshold‐dependent manner by introducing a “change point” provide a simple way to interpret certain types of nonlinear relationships. Change‐point analysis is a distinct form of threshold analysis that is specifically concerned with finding structural changes within a “natural axis” such as time. Data from these insecticide efficacy trials can provide insight into the onset and magnitude of practical resistance of soybean aphid to insecticides. Here, time series were created of data from insecticide efficacy trials performed at two locations in Minnesota spanning 2005–2020. For each location, percent control relative to the untreated control was calculated for a common pyrethroid (λ‐cyhalothrin), an insecticide that soybean aphids have documented resistance towards in laboratory bioassays. In addition, for one location, percent control was calculated for a common organophosphate (chlorpyrifos), an insecticide for which there have been no reports of resistance for soybean aphid. These analyses can indicate whether a change point in percent control has occurred and provide an indication of when and to what degree the percent control has changed.

MATERIALS AND METHODS

Data collection and summary

Data were compiled from insecticide efficacy trials conducted from 2005 to 2020 at the University of Minnesota's Southwest Research and Outreach Center (SWROC) in Lamberton, MN and the University of Minnesota Outreach, Research and Education (UMORE) Park in Rosemount, MN. All the efficacy trials were performed using standard agronomic practices with aphid‐susceptible soybean varieties (that is, not containing Rag genes) adapted to the region (Table 1). Each efficacy trial was conducted as a randomized complete block design with four blocks and multiple insecticide treatments, including foliar application of the pyrethroid λ‐cyhalothrin (Warrior or Warrior II; Syngenta Corp.) applied at a high label rate using standard application practices, and an untreated control treatment (Table 1). In addition, several of the efficacy trials conducted at SWROC included the organophosphate chlorpyrifos (Lorsban 4e or Lorsban Advanced; Dow AgroSciences) applied at a high label rate (Table 1), which were also evaluated as a positive control for insecticide efficacy over time. Over the period that the data were collected, the efficacy trials were conducted by a limited number of people, which minimizes variability among trials. At SWROC, all trials were conducted by B.D. Potter. At UMORE, trials conducted prior to 2013 were performed by D.W. Ragsdale, and trials in 2013 and later were performed by R.L. Koch following the methods used previously at this location.
TABLE 1

Experimental details for insecticide efficacy trials against soybean aphid performed at Lamberton (SWROC) and Rosemount (UMORE), MN from 2005 to 2020

PretreatmentPost‐treatment
LocationYearSample dateOverall density b Treatment date c Sample dateControl density d Treated density e FormulationRate (L ha−1)Volume (L ha−1)Pressure (kPa)Reference
SWROC20052 Aug1172 Aug17 Aug1450Warrior0.23187.08275.8 f
200628 Aug14629 Aug12 Sep2330Warrior0.23187.08275.8 f
0Lorsban 4e1.17
20079 Aug2849 Aug24 Aug3001Warrior0.23187.08275.8 f
200829 Jul95630 Jul12 Aug1894Warrior II0.12187.08275.8 f
20103 Aug1514 Aug19 Aug920Warrior II0.12187.08275.8 f
20113 Aug2723 Aug17 Aug2430Warrior II0.12187.08275.8 f
0Lorsban Advanced1.17
201413 Aug12313 Aug26 Aug561Warrior II0.12140.31241.3 f
201514 Aug46814 Aug28 Aug3742502Warrior II0.12140.31241.3 f
3Lorsban Advanced1.17
201723 Aug28424 Aug6 Sep15312Warrior II0.14140.31241.3 f
12Lorsban Advanced1.17
201815 Aug38916 Aug29 Aug376110Warrior II0.14140.31241.3 f
6Lorsban Advanced1.17
201919 Aug17920 Aug3 Sep850158Warrior II0.14140.31241.3 f
202011 Aug15211 Aug25 Aug15737Warrior II0.14140.31241.3 52
202011 Aug20211Aug25 Aug16348Warrior II0.12140.31241.3 52
UMORE200922 Jul21523 Jul3 Aug178439Warrior0.18187.08 g 53
201312 Aug38213 Aug27 Aug74426Warrior II0.12187.08275.8 48
20141 Aug953 Aug18 Aug44811Warrior II0.12187.08275.8 54
20148 Aug32812 Aug26 Aug52621Warrior II0.12187.08275.8 48
201531 Jul3234 Aug20 Aug977375Warrior II0.12187.08275.8 f
201816 Aug18117 Aug30 Aug83102Warrior II0.12140.31206.8 49
201921 Aug40722 Aug5 Sep634478Warrior II0.12140.31206.8 51

Experiments were performed as randomized complete block designs with four blocks and multiple treatments including foliar application of λ‐cyhalothrin (Warrior or Warrior II), chlorpyrifos (Lorsban 4e or Lorsban Advanced), and an untreated control.

Mean number of aphids per plant across the experimental plots.

Date of foliar application of λ‐cyhalothrin or chlorpyrifos.

Mean number of aphids per plant across the untreated control plots.

Mean number of aphids per plant across the plots treated with λ‐cyhalothrin or chlorpyrifos.

Bruce D. Potter and Robert L. Koch, unpublished data.

Not reported in publication.

Experimental details for insecticide efficacy trials against soybean aphid performed at Lamberton (SWROC) and Rosemount (UMORE), MN from 2005 to 2020 Experiments were performed as randomized complete block designs with four blocks and multiple treatments including foliar application of λ‐cyhalothrin (Warrior or Warrior II), chlorpyrifos (Lorsban 4e or Lorsban Advanced), and an untreated control. Mean number of aphids per plant across the experimental plots. Date of foliar application of λ‐cyhalothrin or chlorpyrifos. Mean number of aphids per plant across the untreated control plots. Mean number of aphids per plant across the plots treated with λ‐cyhalothrin or chlorpyrifos. Bruce D. Potter and Robert L. Koch, unpublished data. Not reported in publication. In insecticide efficacy trials, insecticide applications are not always triggered by densities of pests that would typically require treatment. Sometimes, low pest densities are treated to ensure at least some data will be collected. However, to minimize potential effects of variable aphid densities on the results, data sets were filtered to include only those with pretreatment densities of approximately 100 aphids per plant or more. From each field trial, the efficacy of the targeted insecticides was estimated as percent control measured at approximately 2 weeks after insecticide application (Table 1). Two weeks after treatment falls within the typical range of data collection for soybean aphid efficacy trials, and is a period that has historically represented good control for soybean aphid with foliar applied λ‐cyhalothrin across the region. , , , , , , Percent control was calculated as 100 × (C − T)/C, where C is the mean number of aphids per plant in the untreated control plots, and T is the mean number of aphids per plant in the insecticide‐treated plots. , The efficacy trials conducted at SWROC in 2004 and 2006 had two control treatments per block whose data were averaged prior to calculation of percent control. In addition, the efficacy trial conducted at SWROC in 2018 had two control treatments and two λ‐cyhalothrin treatments per block that were averaged prior to calculation of percent control. One efficacy trial was performed at each location per year, except in 2020 at SWROC and in 2014 at UMORE, where two separate efficacy trials occurred. Percent control was calculated for each trial independently, and individual trial percent control values were averaged to produce a single percent control value in these instances of two trials within a single year at a location, due to trial proximity and the potential lack of independence between aphid populations. These instances of multiple trials at a given location within a year were accounted for by weighting in the analysis (see below).

Data analysis

The resulting time series of percent control for λ‐cyhalothrin at SWROC and UMORE, and for chlorpyrifos at SWROC, were analyzed in R version 4.0.3 (ref. ) and R Studio version 1.3.1093. Each time series was analyzed separately using the chngpt package (code: chngptm) with continuous two‐phase regression models with percent control as the dependent variable and year as the independent variable. In the models, years were weighted for the number of trials within a location. Models were selected by first fitting a “segmented” model (slopes before and after threshold); however, in all cases, the slope before the threshold was not significantly different from 0, which allowed the use of a “hinge” model (slope of 0 before threshold and decreasing slope after threshold). Use of a hinge model is preferred because the model can be estimated with substantially higher precision than that of a segmented model. , Model significance was evaluated using a likelihood ratio test (package: lmtest, code: lrtest) comparing the hinge model to the null (intercept‐only linear) model. Root mean square errors (RMSE) were manually calculated, and change‐point (breakpoint) significance was tested using the chngpt.test function. Estimations of slopes after threshold were derived from the lincomb function.

Literature search

To assess the novelty of using this approach for examining practical resistance, a literature search was conducted on 14 November 2021 to search for the use of similar statistical methods in evaluating field‐evolved resistance and the efficacy of pesticides with the search string (“piecewise regression” OR “threshold regression” OR “broken stick” OR “hockey stick” OR “changepoint” OR “change point”) AND (“insecticide” OR “entomology” OR “insect” OR “resistance” OR “pesticide” OR “fungicide” OR “antibiotic” OR “herbicide”) in Web of Science, CAB Abstracts and Agricola.

RESULTS

SWROC

The time series of percent control for λ‐cyhalothrin at SWROC is presented in Figure 1. For λ‐cyhalothrin, the hinge model provided a better fit than the linear null model (χ2 = 21.03, d.f. = 2, p < 0.001). A significant change point in percent control was identified at 2014 (95% confidence interval: 2007, 2017) (RMSE = 4.50) (χ2 = 22.81, d.f. = 1, p < 0.001). The pre‐change point intercept was 98.57% (95% confidence interval: 96.85, 100.76), for the years 2005–2014. The slope after the change point was −4.30% (95% confidence interval: −25.62, −1.68) per year.
FIGURE 1

Efficacy of λ‐cyhalothrin in field trials in southwest Minnesota. Percent control of soybean aphid populations by foliar application of λ‐cyhalothrin in insecticide efficacy field trials conducted at the University of Minnesota's Southwest Research and Outreach Center (SWROC) in Lamberton, MN from 2005 to 2020. Percent control was calculated relative to the untreated control at approximately 2 weeks after insecticide application. Analysis identified 2014 as the change point in λ‐cyhalothrin efficacy at this location.

Efficacy of λ‐cyhalothrin in field trials in southwest Minnesota. Percent control of soybean aphid populations by foliar application of λ‐cyhalothrin in insecticide efficacy field trials conducted at the University of Minnesota's Southwest Research and Outreach Center (SWROC) in Lamberton, MN from 2005 to 2020. Percent control was calculated relative to the untreated control at approximately 2 weeks after insecticide application. Analysis identified 2014 as the change point in λ‐cyhalothrin efficacy at this location. The time series of percent control for chlorpyrifos at SWROC is presented in Figure 2. For chlorpyrifos, a candidate change point for percent control was detected in 2011; however, it was found to be nonsignificant (χ2 = 2.12, d.f. = 1, p = 0.269), and the fit of the hinge model did not differ from that of the null model (χ2 = 2.12, d.f. = 2, p = 0.3466). An intercept‐only linear model revealed an average (± SEM) percent control of 98.1% ± 1.46%.
FIGURE 2

Efficacy of chlorpyrifos in field trials in southwest Minnesota. Percent control of soybean aphid populations by foliar application of chlorpyrifos in insecticide efficacy field trials conducted at the University of Minnesota's Southwest Research and Outreach Center (SWROC) in Lamberton, MN from 2005 to 2020. Percent control was calculated relative to the untreated control at approximately 2 weeks after insecticide application. Analysis did not identify a significant change point in chlorpyrifos efficacy at this location.

Efficacy of chlorpyrifos in field trials in southwest Minnesota. Percent control of soybean aphid populations by foliar application of chlorpyrifos in insecticide efficacy field trials conducted at the University of Minnesota's Southwest Research and Outreach Center (SWROC) in Lamberton, MN from 2005 to 2020. Percent control was calculated relative to the untreated control at approximately 2 weeks after insecticide application. Analysis did not identify a significant change point in chlorpyrifos efficacy at this location.

UMORE

The time series of percent control for λ‐cyhalothrin at UMORE is presented in Figure 3. For λ‐cyhalothrin, the hinge model provided a better fit than the linear null model (χ2 = 10.27, d.f. = 2, p = 0.006). A significant change point in percent control was identified at 2014 (95% confidence interval: 2009, 2015) (RMSE = 20.33) (χ2 = 10.15, d.f. = 1, p = 0.003). The pre‐change point intercept was 92.90% (95% confidence interval: 85.72, 102.72) for the years 2005–2014, and the slope after the changepoint was −19.90% (95% confidence interval: −35.44, −8.52) per year.
FIGURE 3

Efficacy of λ‐cyhalothrin in field trials in southeast Minnesota. Percent control of soybean aphid populations by foliar application of λ‐cyhalothrin in insecticide efficacy field trials conducted at the University of Minnesota's Outreach, Research and Education (UMORE) Park in Rosemount, MN from 2005 to 2020. Percent control was calculated relative to the untreated control at approximately 2 weeks after insecticide application. Analysis identified 2014 as the change point in λ‐cyhalothrin efficacy at this location.

Efficacy of λ‐cyhalothrin in field trials in southeast Minnesota. Percent control of soybean aphid populations by foliar application of λ‐cyhalothrin in insecticide efficacy field trials conducted at the University of Minnesota's Outreach, Research and Education (UMORE) Park in Rosemount, MN from 2005 to 2020. Percent control was calculated relative to the untreated control at approximately 2 weeks after insecticide application. Analysis identified 2014 as the change point in λ‐cyhalothrin efficacy at this location. Our literature search resulted in 97, 47 and 27 publications from Web of Science, CAB Abstracts and Agricola, respectively. Of the 171 total results, 120 unique publications were found. Within these unique publications, 16 were related to entomology, and 21 involved the use of pesticides. Within our literature search, examples were found of change point and similar statistical methods used in entomology when modeling population dynamics in monarch butterflies (Danaus plexippus), pupation success and behavior of western flower thrips (Frankliniella occidentalis), and the residual efficacy of pesticide‐impregnated ear tags for cattle. Also, similar methods have been used when evaluating the dissipation of pesticides used as seed treatment in soils, examining the behavior and environmental fate of glyphosate in water and sediments, and for determining the effects of defoliation on yield in field beans. However, no examples were found of previous research using change point or similar types of analyses to model time‐series data on insecticide efficacy related to resistance.

DISCUSSION AND CONCLUSION

For both locations examined in this study, a significant change point in percent control for λ‐cyhalothrin was found at 2014; thus, signifying the last year prior to the onset of decreasing control. In the next soybean growing season (2015), reports of control failures and laboratory confirmations of reduced susceptibility began. , In this research, a retrospective time‐series analysis revealed the onset of practical resistance in the field, further validating and complimenting other lines of evidence demonstrating the presence of pyrethroid resistance in soybean aphid. Furthermore, this may be the first time this type of change‐point analysis has been used for retrospective assessment of the onset of practical resistance to insecticides. Insecticides are designed to deliver high levels of consistent control, as seen in the current study from 2005 to 2014. Therefore, the use of hinge modeling (no initial slope) in this system would be expected, which is corroborated by the lack of a significant pre‐change point slope indicated in the segmented models. It is important to note that the pre‐change point intercept does not necessarily represent the average of the pre‐change point observations. For SWROC and UMORE, the pre‐change point intercepts were numerically less than the mean of the pre‐change point observations (SWROC = 99.4%, UMORE = 97.0%). This may occur because the estimated change point candidate is selected considering the overall model with the highest likelihood. A similar effect can also be seen in other publications that use similar analysis. , The post‐change point slopes for the two locations examined in this study demonstrate the importance of conducting pesticide efficacy trials over time using consistent methods to provide such long‐term data sets. At UMORE, the estimated slope of percent control after the change point was approximately five times that of SWROC, although not statistically different because of the wide confidence intervals. The continued evaluation of these insecticides at these locations may refine these estimates as trajectories of change near the terminus of a time series may be difficult to determine. Resistance to pesticides is an ever‐increasing challenge to global agriculture with nearly a 1000 species of pests, including approximately 600 arthropod species, resistant to one or more pesticides. Resistance to a pesticide can be directly documented by demonstrating a reduction in susceptibility over time within a population. However, the impact of resistance on the practical control of a pest can vary because of several factors including frequency of resistance, population density and geographic distribution. , The broader impact of field‐evolved resistance on pest control can vary from insignificant to severe depending on the level of practical resistance realized in the field and the availability of alternative control measures. The current state of pyrethroid resistance in the soybean aphid was likely accelerated through regularly applying a limited number of insecticidal modes of action to manage the insect, and the lack of adoption of other integrated pest management strategies such as host‐plant resistance and biological control. Despite the practical resistance observed for the pyrethroids, the organophosphate chlorpyrifos has remained highly effective. However, on 18 August 2021, the US Environmental Protection Agency released their Final Tolerance Rule for chlorpyrifos revoking all tolerances for the insecticide on food products nationwide, further limiting the products available for the control of soybean aphid. Although some newer, more selective insecticides, such as sulfoxaflor (group 4C), flupyradifurone (group 4D), and afidopyropen (group 9D) are labeled for and effective against soybean aphid, , , , caution should be taken to preserve their efficacy through the use of insecticide resistance management tactics. Regular efficacy monitoring of these alternative chemistries, both in the laboratory and in the field, is necessary to quantify of the onset and magnitude of practical resistance, and to avoid a similar fate as that of the pyrethroids for soybean aphid.
  25 in total

1.  Effects of spinosad, spinosad bait, and chloronicotinyl insecticides on mortality and control of adult and larval western cherry fruit fly (Diptera: Tephritidae).

Authors:  Wee L Yee; Diane G Alston
Journal:  J Econ Entomol       Date:  2006-10       Impact factor: 2.381

Review 2.  Ecology and management of the soybean aphid in North America.

Authors:  David W Ragsdale; Douglas A Landis; Jacques Brodeur; George E Heimpel; Nicolas Desneux
Journal:  Annu Rev Entomol       Date:  2011       Impact factor: 19.686

3.  Efficacy of afidopyropen against soybean aphid (Hemiptera: Aphididae) and toxicity to natural enemies.

Authors:  Robert L Koch; Obiratanea da Silva Queiroz; Rafael Carlesso Aita; Erin W Hodgson; Bruce D Potter; Teresia Nyoike; Christa D Ellers-Kirk
Journal:  Pest Manag Sci       Date:  2019-06-19       Impact factor: 4.845

4.  Defining terms for proactive management of resistance to Bt crops and pesticides.

Authors:  Bruce E Tabashnik; David Mota-Sanchez; Mark E Whalon; Robert M Hollingworth; Yves Carrière
Journal:  J Econ Entomol       Date:  2014-04       Impact factor: 2.381

5.  Resistance of codling moth, Cydia pomonella (L.) (Lepidoptera: Tortricidae), larvae in Michigan to insecticides with different modes of action and the impact on field residual activity.

Authors:  David Mota-Sanchez; John C Wise; Ryan Vander Poppen; Larry J Gut; Robert M Hollingworth
Journal:  Pest Manag Sci       Date:  2008-09       Impact factor: 4.845

6.  Field and laboratory tests of new insecticides against the apple maggot, Rhagoletis pomonella (Walsh) (Diptera: Tephritidae).

Authors:  W Harvey Reissig
Journal:  J Econ Entomol       Date:  2003-10       Impact factor: 2.381

7.  Probability of cost-effective management of soybean aphid (Hemiptera: Aphididae) in North America.

Authors:  Kevin D Johnson; Matthew E O'Neal; David W Ragsdale; Christina D DiFonzo; Scott M Swinton; Philip M Dixon; Bruce D Potter; Erin W Hodgson; Alejandro C Costamagna
Journal:  J Econ Entomol       Date:  2009-12       Impact factor: 2.381

8.  Identification of beneficial and detrimental bacteria impacting sorghum responses to drought using multi-scale and multi-system microbiome comparisons.

Authors:  Mingsheng Qi; Jeffrey C Berry; Kira W Veley; Lily O'Connor; Omri M Finkel; Isai Salas-González; Molly Kuhs; Julietta Jupe; Emily Holcomb; Tijana Glavina Del Rio; Cody Creech; Peng Liu; Susannah G Tringe; Jeffery L Dangl; Daniel P Schachtman; Rebecca S Bart
Journal:  ISME J       Date:  2022-05-06       Impact factor: 11.217

9.  chngpt: threshold regression model estimation and inference.

Authors:  Youyi Fong; Ying Huang; Peter B Gilbert; Sallie R Permar
Journal:  BMC Bioinformatics       Date:  2017-10-16       Impact factor: 3.169

View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.