Literature DB >> 35921355

socialh: An R package for determining the social hierarchy of animals using data from individual electronic bins.

Júlia de Paula Soares Valente1,2, Matheus Deniz2, Karolini Tenffen de Sousa2, Maria Eugênia Zerlotti Mercadante3, Laila Talarico Dias1,2.   

Abstract

Cattle have a complex social organization, with negative (agonistic) and positive (affiliative) interactions that affect access to environmental resources. Thus, the social behaviour has a major impact on animal production, and it is an important factor to improve the farm animal welfare. The use of data from electronic bins to determine social competition has already been validated; however, the studies used non-free software or did not make the code available. With data from electronic bins is possible to identify when one animal takes the place of another animal, i.e. a replacement occurs, at the feeders or drinkers. However, there is no package for the R environment to detect competitive replacements from electronic bins data. Our general approach consisted in creating a user-friendly R package for social behaviour analysis. The workflow of the socialh package comprises several steps that can be used sequentially or separately, allowing data input from electronic systems, or obtained from the animals' observation. We provide an overview of all functions of the socialh package and demonstrate how this package can be applied using data from electronic feed bins of beef cattle. The socialh package provides support for researchers to determine the social hierarchy of gregarious animals through the synthesis of agonistic interactions (or replacement) in a friendly, versatile, and open-access system, thus contributing to scientific research.

Entities:  

Mesh:

Year:  2022        PMID: 35921355      PMCID: PMC9348686          DOI: 10.1371/journal.pone.0271337

Source DB:  PubMed          Journal:  PLoS One        ISSN: 1932-6203            Impact factor:   3.752


Introduction

The automation of production systems has allowed the simultaneous monitoring of various animals’ parameters. For example, sensors have been shown to be useful for monitoring the cows’ location [1], activities (e.g., walking [2, 3]; lying down [4, 5]), and feeding and drinking behaviour (time and duration [6, 7]). Furthermore, electronic feeders and drinkers are also useful for detecting social competition [8-10] since most disputes occur during feed time [11, 12] and at drinkers on hot days [13]. With the data from electronic bins is possible to define a competition; once an animal (the actor) takes the place of the previous animal (reactor) at the bin [8, 10]. Previous studies have applied algorithms for the autonomous detection of replacements using data from individual electronic bins [8–10, 13]. A replacement occurs when an animal that occupied the bin (reactor) is completely withdrawn by another animal that occupied the same bin (actor) in a short interval. However, there are no studies that assessed the general applicability of a replacement detection algorithm developed in open-access systems (e.g., R software) to assess social relationships, such as social hierarchy of farm animals. Thus, our general approach was to develop and to make available a user-friendly R package, named socialh, that can be used to detect replacements using data from individual electronic bins to determine the social hierarchy based on the competition between each pair of animals in the herd [14]. The aim of this study is to describe the socialh package version 0.1.0. Our package is intended for users who wish to implement more flexible (i.e. it is possible include data from electronics bins and data observational) social behaviour analysis since the R environment allows the integration of several functions from different packages. In addition, we provide an overview of socialh and demonstrate its features and applicability using data from electronic feed bins of beef cattle.

Overview of socialh

Our general approach consisted of created a user-friendly R package for competition behaviour analysis. We choose to use the R software [15] because it is an open-access software and that it offers several resources for data analyses. The socialh package is available from the Comprehensive R Archive Network (CRAN) at https://cran.r-project.org/web/packages/socialh/index.html. The work-flow of the socialh package comprises several steps (Fig 1) that can be used sequentially or separately. First, we developed a function to identify replacements using data from electronic bins (feeder or drinker); a replacement occurs when an animal that occupied the bin (reactor) is completely withdrawn by another animal that occupied the same bin (actor) in a short interval. Second, we integrated other functions to determine the social rank and social hierarchy of the herd. The functions of the socialh package are listed in Table 1.
Fig 1

R package flow chart for competition behaviour analysis.

*Format of datasheet: date—dd/mm/yyyy; time—hour:minutes:seconds.

Table 1

Descriptions of functions of the socialh R package.

FunctionDescription
replacementIdentifies replacements between the actor and reactor from electronic bin data.
repByBinIdentifies the frequency of replacements by bin.
freqActorIdentifies the frequency that an animal was actor.
freqReactorIdentifies the frequency that an animal was reactor.
smatrixBuilds a square matrix containing the frequency of competition between each dyad (each pair of animals).
dmatrixDetermines the Sij dyadic dominance relationship from a sociomatrix.
dvalueDetermines the dominance value, social rank and hierarchy from the Sij dyadic relationship matrix.
landau_indexCalculates the linearity index developed by Landau (1951).
improved_indexCalculates the linearity index improved by de Vries (1995).
barDomGenerates a barplot from the variables obtained in the dvalue function (dominance value, social hierarchy and social rank).
bpDomGenerates a boxplot from the variables obtained in the dvalue function (dominance value, social hierarchy and social rank), and variable obtained in the frequency functions (freqActor, and freqReactor).
actorSociogramGenerates a sociogram with actor information.
reactorSociogramGenerates a sociogram with reactor information.

R package flow chart for competition behaviour analysis.

*Format of datasheet: date—dd/mm/yyyy; time—hour:minutes:seconds.

Database

For the present article, we used a database obtained from feed efficiency test of beef cattle to illustrate the functions of the socialh package. The data were provided by the Beef Cattle Research Center, Institute of Animal Science, Sertãozinho, São Paulo State, Brazil. All management procedures followed animal welfare guidelines and were conducted in accordance with State Law No. 11 977 of the State of São Paulo, Brazil. The database used in the examples is available on socialh package and Kaggle repository (https://www.kaggle.com/datasets/juliavalente/data-from-visits-to-the-trough-of-nellore-cattle). The feed efficiency test was conducted in 2021, including 37 Nellore males with a mean age ± SD of 292 ± 26 days and a mean weight of 255.9 ± 44.5 kg. The group was housed for 21 days in a paddock (3138 m2) containing 5 electronic feed bins (GrowSafe System®, Vytelle–Kansas City, Missouri, USA). The Total Mixed Ration (TMR) was offered twice a day (8:00 and 15:00 hours) and the animals had ad libitum access to TMR and water trough. All animals were fitted with an ear tag transponder that allowed the electronic bin system to record the date and time when each animal entered and left the feeder. The database obtained from the electronic feeder consisted of 90,211 lines of feeding events. A feeding event starts when the ear tag of an animal α is recognized by a specific feeder β and ends when the animal α completely withdraws its head from the feeder β or when a different ear tag transponder of another animal is recognized by the feeder.

Specifications and implementation of the functions

Data preparation and import

We would like to highlight that the data preparation depends on the database and the objective of each research; we therefore did not include these functions in the package. The user has two ways to import the database: (1) data from electronic bins; (2) data from animals’ observation. For the analysis of competition behaviour data from individual electronic bins must contain the equipment identification (column name: equip_id), animal identification (column name: animal_id), date and time (dd/mm/yyyy and hour:minutes:seconds) when the animal entered at the electronic bin (column name: IN), and date and time (dd/mm/yyyy and hour:minutes:seconds) when the animal left the electronic bin (column name: OUT). On the other hand, when competition behaviour is analysed using data collected by video or in person, the database must contain two parameters (columns): actor and reactor. In our case, the data obtained by the electronic bins had the following columns: animal_id, equip_id, date and time of the animal entry into the bin, duration (s), and consumption (kg). To use the socialh package, we had to prepare the database. We determined the date and time when the animal left the bin by summing the date and time when the animal entered at the electronic bin and the duration of the feeding event (column “Duration(s)”). Thus, after prepared the database, we kept the columns with animal identification (animal_id), bin identification (equip_id), date and time of the entry into the bin (IN), and date and time of the exit from the bin (OUT).

Replacement identification

The first function of the socialh R package, replacement(), is responsible to identify replacements at the electronic bins. In our package, we used the definition of replacement proposed by Huzzey et al. [8]. A replacement was identified when the algorithm detected a specific time-interval between two different animals that sequentially visited the same electronic bin (feed or water), i.e., the animal that occupied the electronic bin (reactor) was completely withdrawn by the following animal that occupied the same electronic bin (actor). Thus, before the user apply the replacement() function, it is necessary to determine the optimum time-interval to the algorithm identify a replacement. The definition of the optimal time-interval depends on the species and animal category. For example, for lactating Holstein cows, previous studies used data from feeders and water bins to determine different intervals for the identification of a replacement. Huzzey et al. [8] highlighted that, for feed bins, a shorter interval (≤26 s) between successive feeding events of two cows at one feed bin was associated with competitive replacement. On the other hand, for water bins, McDonald et al. [10] found the optimal time-interval for the identification of replacements to be ≤29 s. Combining electronic feed and water bin data, Foris et al. [9] found that a 20 to 30s interval was optimal to identify competitive replacements. For the database used as example in this study, we used the time-interval of 0-10s to identify the competitive replacement. As the objective of this study was to present the socialh package, we did not focus in determine the optimal time-interval for identify competitive replacement of Nellore cattle. When this study was performed, we were only aware of the available dairy cattle literature, so the choice of the range (0-10s) was based on the authors’ previous experience with Nellore cattle. However, we strongly recommend that future studies determine the optimal time-interval to identify replacements in electronic bins of different breeds and categories of cattle. After pre-processed the database from electronic bins and determinate the optimal time-interval to identify a competitive replacement, the user can apply the replacement() function specifying the following parameters: database—file containing four columns (equip_id, animal_id, IN and OUT); and sec—optimal time-interval in seconds, i.e., [replacement(database, sec.)]. In order to identify a replacement, the function replacement() will order the database according to the columns equip_id and IN (Fig 2). If the time interval between two animals is lower than the time specify at the function, the event will be recognized as a competitive replacement. The output data frame of the replacement() function is printed in two columns: actor and reactor. In our database example, the replacement function using the interval of 0-10s found 54,346 competitive replacements.
Fig 2

A sample of the input database with data from the electronic feeding system of Nellore cattle used in the replacement() function.

#First, install and load the socialh R package from CRAN repository > install.packages(“socialh”) > library(socialh) #Load the database > example.data <- read.csv(“behaviour_data.csv”) # Apply the replacement(x, sec) function to create a data table with actor and reactor and save as an object to use later. > replace <- replacement (example.data, 10) > head(replace) actor reactor 336 704 128 336 336 704 704 336 465 836 465 798

Frequency determination

The socialh package provides three functions that return frequency information: repByBin(), freqActor(), freqReactor(). The repByBin() function, like the replacement() function, uses the data from the electronic bin system (equip_id, animal_id, IN, and OUT) and the output is the frequency of replacements that occurred in each bin. The freqActor() and freqReactor() functions return the frequency that each animal was actor and reactor. #Apply the repByBin(x, sec) function to create an output with frequency information of replacements that occurred in each bin. >replacementByBin<- repByBin(example.data, 10) >print(replacementByBin) equip_id replacements % 1 9216 16.94616 2 12217 22.46433 3 11788 21.67549 4 10698 19.67123 5 10465 19.24279 #Apply the freqActor(x) function to create an output with frequency of an animal was actor. >fActor<- freqActor(replace) >head(fActor) animal_id freq_actor % 109 801 1.4738895 117 1302 2.3957605 128 1827 3.3617930 146 2760 5.0785706 181 1285 2.3644794 227 937 1.7241379 #The freqReactor(x) function create an output with frequency of an animal was reactor. >fReactor<- freqReactor(replace) >head(fReactor) animal_id freq_actor % 109 802 1.475729585 117 1269 2.335038457 128 1899 3.494277408 146 2354 4.331505539 181 1558 2.866816325 227 1272 2.340558643

Matrix determination: Sociometric and dyadic

The output data frame of replacement function (actor and reactor) permits to determine the dominance value for each animal. For the socialh package, we chose to use the method proposed by Kondo and Hurnik [14] to determine the dominance value. There are available several R packages and software that can be used to infer the animals’ dominance value, but these software employ different methodologies. For example, the R package named “aniDom” uses the original and the randomized Elo-rating method [16]; the R package “Elorating” calculates David’s scores [17]; the R package “compete” runs the I&SI method [18]; and the R package “steepness” also calculates David’s scores and normalized David’s scores [19]. The Elo-rating method considers the proportions of wins and losses of a dyad, where the rating of the winner of the dispute is increased by an amount that depends on the chance of winning: the amount is small if the chance of winning is high and vice versa [20]. The Elo-rating and David’s score consider that the overall success of an individual is determined by weighting each dyadic success measure by the unweighted estimate of the interactant’s overall success so that relative strengths of the other individuals are considered [21]. While the Kondo and Hurnik [14] method considers all competition than an animal was involved (i.e., as actor or reactor) in relation to other herd members, without ponderations. Other authors also preferred this index for similar reason [11, 12, 22–24]. To our knowledge, socialh is the first R package that adopted the method proposed by Kondo and Hurnik [14]. To obtain the dominance value based on Kondo and Hurnik [14], we developed three functions: smatrix(), dmatrix() and dvalue(). The functions, which are described below, can be used sequentially, or combined with functions from other packages that also determine the dominance value of an animal. We also highlight that the user can apply these three functions to analyse data obtained through the replacement() function or input data from direct (in-person) or indirect (video) observations. The smatrix() function builds a square matrix that contains the frequency of competition (replacements) between each dyad (each pair of animals). The dmatrix() function transforms the smatrix into a dyadic dominance relationship as proposed by Kondo and Hurnik [14]. Therefore, the dyadic dominance relationship of the ith animal relative to the jth animal (Sij) is assessed qualitatively by the sign of the difference between Xij and Xji, which always results in a value of -1, 0 or +1 (Eq 1). The values distinguish four relationships: domination (value +1), subordinations (value -1), tied (i.e., equal numbers of wins for both members of a dyad), and unknown (no data) relationship (both values 0). The output data frame of smatrix() is a square matrix with the actors in the column and reactors in the row (Fig 3A) and the output of dmatrix() is a square matrix containing the dyadic dominance relationship (Fig 3B).
Fig 3

Example of the output data frame of the (a) smatrix() function and (b) dmatrix() function.

Example of the output data frame of the (a) smatrix() function and (b) dmatrix() function. #Use the smatrix() function to create sociomatrix by a replacement data table and save as an object to use later. > social <- smatrix (replace) > head(social) Actor reactor 109 117 128 146 181 227 109 0 1 1 1–1–1 117–1 0 1 1–1–1 128–1–1 0 1–1–1 146–1–1–1 0–1–1 181 1 1 1 1 0 1 227 1 1 1 1–1 0 #Apply the dmatrix() function to transform the sociomatrix in a dyadic dominance relationship matrix and save as an object to use later. > dyadic <- dmatrix (social) > head(dyadic) equip_id replacements % 1 9216 16.94616 2 12217 22.46433 3 11788 21.67549 4 10698 19.67123 5 10465 19.24279 The dvalue() function sums the dyadic dominance relationship by column (actor) as proposed by Kondo and Hurnik [14]. Therefore, the dyadic relationship of the ith animal relative to the jth animal (Sij) is assessed qualitatively according to Eq (2). where Si is the sum of all relationships involving animal i, and n is the number of possible interactions of one animal in the group with the other animals. The social rank (high and low) and social hierarchy (dominant, intermediate, and subordinate) are determined according to dominance value. The choice of dividing the group according to social rank (2 categories) or social hierarchy (3 categories) depends on the study objectives. Both social rank (SR) and social hierarchy (SH) are estimated by the distance between the highest (+ X) and the lowest (- Y) dominance value, plus 1 (corresponds to the dominance value zero), which determines the number of points in the range (Eq 3; see [22]). For social rank, animals with dominance values in the first half, i.e., those with the lowest values including negative ones, are classified as low rank, and animals with dominance values in the second half are classified as high rank. For social hierarchy, animals with dominance values in the first tertile, i.e., those with the lowest values including negative ones, are classified as subordinate. Animals with dominance values in the second tertile are classified as intermediate, and animals with dominance values in the third tertile with higher positive values are classified as dominant. The output data frame of the dvalue() function is printed in four columns: animal_id, dominance_value, social_rank, and social_hierarchy. #Employ the dvalue() function to determine dominance value, social rank and social hierarchy by a dyadic matrix. > dominance <- dvalue (dyadic, hs = TRUE, rs = TRUE) > head(dominance) animal_id dominance_value social_hierarchy social_rank 1: 227 -26 subordinate low 2: 426 -20 subordinate low 3: 757 -18 subordinate low 4: 181 -16 subordinate low 5: 764 -16 subordinate low 6: 975 -16 subordinate low > tail(dominance) animal_id dominance_value social_hierarchy social_rank 1: 288 15 dominate high 2: 737 16 dominate high 3: 980 17 dominate high 4: 146 18 dominate high 5: 787 19 dominate high 6: 834 18 dominate high

Visualization of the results

To visualize the results, the package provides functions based on the “ggplot2” [25] and “circlize” [26] packages. The barDom() function returns to the user a barplot with information related to the values obtained by the dvalue() function. The purpose of this function is graphically demonstrating the number of animals in each social category social hierarchy (dominant, intermediate and subordinate) or social rank (high and low rank)) (Fig 4). To inspect the distribution, outliers, mean and standard deviation of the dominance values, the user can apply the bpDom() function to obtain a boxplot (Fig 4).
Fig 4

Example of barplot (above) and boxplot (below) using barDom and bpDom functions, respectively.

Example of barplot (above) and boxplot (below) using barDom and bpDom functions, respectively. The sociometric matrix can be visualized through sociograms generated by the actorSociogram() and reactorSociogram() functions, which visually display the actor and reactor relationships between the animals within the evaluated group (Fig 5). The animals are represented around a circular plot and are connected by arrows in which the thickness of the lines is proportional to the frequency of interactions between the animals and the arrowheads indicate the direction of the interactions.
Fig 5

Example of sociograms using actorSociogram (above) and reactorSociogram (below) functions, respectively.

Example of sociograms using actorSociogram (above) and reactorSociogram (below) functions, respectively.

Linearity index

In social organization of animals, a hierarchy is considered linear when one animal A dominates all others in the group; while in a circular relationship, the animal A dominates B and C, B dominates C, that dominates D, and D dominates B. Also, unknown relationships can occur in a herd, this happens when no competitive behaviour is observed between a dyad; or when the number of winning and losses of a dyad are equal [27]. The linearity index indicates the number and effects of unknown relationships in a herd; as increase the number of unknown relationships, decreases the calculated linearity index [28]. First, we developed the landau_index() function to determine the linearity index (h; Eq 4) of the group as proposed by Landau [29]. Landau [29] has devised an index that measure the degree of linearity in a set of dominance relationships between the animals in the same group. The Landau index (h) use the concept of ‘number of dominated animals’, where a value of h > 0.9 generally indicates a strongly linear hierarchy [30]. To apply the landau_index() function, the user must specify a dyadic matrix obtained with the dmatrix() function, i.e., [landau_index (dmatrix)]. where h is the linearity index; n is the number of animals in the group, and V is the number of animals that a given animal dominated. #Apply the landau_index() function to determine the linearity index by a dyadic matrix. > landau <- landau_index (dyadic) > print(landau) [ ] 0.318397 Second, we developed the improved_index() function. This function applies the improved linearity test (h’; Eq 5) as described by de Vries [27]. de Vries [27], developed a modified linearity index (h′) that aims to correct for unknown and tied relationships. To apply the improved_index() function, the user must specify a dyadic matrix obtained with the dmatrix function and the sociomatrix obtained with the smatrix function, i.e., [improved_index (dmatrix, smatrix)]. Both indexes range from 0 to 1, with 0 being a non-linear (every animal in the group tends to dominate the same number of other animals) and 1 a perfectly linear hierarchy (an animal dominates all animals ranked below). where h is the Landau linearity index; n is the number of individuals, and u is the number of unknown relationships. #Apply the improved_index() function to determine the improved linearity index by a dyadic matrix and a sociomatrix. > improved_index <- landau_index (dyadic, social) > print(improved_index) [ ] 0.331911

Relationship between different methods for determining the dominance index available in R packages

There are R packages (e.g., “aniDom”, “Eloranting”, “steepness”) that use different methods (Elo Rating, I&SI, and David Score) for determining the dominance index of gregarious animals, but none of these packages present the index proposed by Kondo and Hurnik [14]. Thus, we determined the dominance index from the different methods available in R packages for the same database of Nellore cattle used in the previous examples. Then, we perform Pearson’s correlation analysis with p-value among the results from all methods including the Kondo and Hurnik index values from the socialh packages (Table 2).
Table 2

Correlations among dominance index for a Nellore cattle group obtained from different methods (Elo Rating, I&SI, and David Score) available on R packages.

 Kondo and Hurnik IndexDavid scoreElo ratingI&SI
Kondo and Hurnik Index10.871***0.291*-0.641*
David score10.418***-0.553***
Elo rating1-0.132
I&SI1

*P < 0,10

***P < 0,01

*P < 0,10 ***P < 0,01 The dominance index from the different methods were significantly correlated, except between Elo Rating and ISI. The correlations between the Kondo and Hurnik index and David Score and Elo Rating were positive, i.e., animals with highest values in one of the methods, also present high values in the other method. On the other hand, the I&SI showed negative correlations because the way in which the animals are classified by this method is opposite to that obtained by the other methods. In this case, the main difference between the methods is the interpretation of the results. In some cases, two animals could receive an identical index value, as occurred in the Kondo and Hurnik Index, and David Score. When applying the I&SI method, we noted that each animal obtained a single value which ranges from 1 to the total number of animals in the group (e.g., in our example ranged from 1 to 38). Another characteristic of the I&SI methods is present only positive values, while the other methods (David Score, Elo Rating, and Kondo and Hurnik Index) classify animals with values that vary from negative to positive. These findings indicate that the I&SI is the most discriminative and Kondo and Hurnik Index, and David Score are the least discriminative. Despite the correlations between values, each method has specificities that must be observed at the moment of application. Thus, the researcher should decide which index best suits to the group of animals that will be evaluated.

Final considerations

The socialh R package provides support for researchers to determine the social hierarchy of gregarious animals through the synthesis of competition behaviour in a friendly, versatile, and open access software, thus contributing to scientific research. Limitations of socialh include the fact that the package uses only the method proposed by Kondo and Hurnik [14], which does not mean that researchers cannot adopt other methods or integrate the results obtained with socialh in other packages. Second, the replacement() function uses the interval-time between animals as a parameter to identify replacements at the bin; however, it is up to the user to inform the value that best suits to the specie studied. We encourage further studies to estimate the most adequate replacement times for different animal species and category. Finally, the socialh can help researchers on determining the social hierarchy of gregarious animals using programming languages. 13 Dec 2021
PONE-D-21-29427
socialh : An R package for determining the social hierarchy of cattle using data from individual electronic bins PLOS ONE Dear Dr. Valente, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. The paper is interesting, however considered that is missing information and there are details that need to be clarified. Please submit your revised manuscript by Jan 26 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Arda Yildirim, Ph.D. Academic Editor PLOS ONE Journal requirements: 1. When submitting your revision, we need you to address these additional requirements. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you for stating the following financial disclosure: “We thank the São Paulo Research Foundation (FAPESP -  https://fapesp.br/en; grant #2017/10630-2 and grant #2017/50339-5). JPSV, MD, and KTS acknowledge the Coordination for the Improvement of High Education Personnel (CAPES - https://www.iie.org/programs/capes ) for the scholarship.” Please state what role the funders took in the study.  If the funders had no role, please state: ""The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."" If this statement is not correct you must amend it as needed. Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf. 3.Thank you for stating the following in the Acknowledgments Section of your manuscript: “We thank the São Paulo Research Foundation (FAPESP, grant #2017/10630-2 and grant #2017/50339-5). We thank the staffs of Beef Cattle Research Center, Institute of Animal Science, Sertãozinho, São Paulo State, Brazil for helping in the data acquisition. JPSV, MD, and KTS acknowledge CAPES (Coordination for the Improvement of High Education Personnel) for the scholarship. Finally, we also thank the professor Luiz Carlos Pinheiro Machado Filho, PhD for all the shared knowledge that motivated us to work with social behavior.” We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: “We thank the São Paulo Research Foundation (FAPESP -  https://fapesp.br/en; grant #2017/10630-2 and grant #2017/50339-5). JPSV, MD, and KTS acknowledge the Coordination for the Improvement of High Education Personnel (CAPES - https://www.iie.org/programs/capes ) for the scholarship.” Please include your amended statements within your cover letter; we will change the online submission form on your behalf. 4. In your Data Journal Requirements: Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability. Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized. Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access. We will update your Data Availability statement to reflect the information you provide in your cover letter. 5. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please delete it from any other section. Additional Editor Comments: Dear Authors, The manuscript have certain suggestions from the reviewers that must be addressed before the final decision. Please respond to the comments of the reviewers and upload a revised version of your manuscript. Thank you. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Partly Reviewer #2: Partly ********** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: N/A Reviewer #2: N/A ********** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: No Reviewer #2: No ********** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ********** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The paper presents an interesting approach as an R package to evaluate social hierarchy. However, there are aspects of that could be improved. The example lacks on detail, which makes the reader hard to follow. In addition, the intervals (line 121) seem critical to evaluate the social hierarchy, and the references used are from Holstein dairy cows and the authors mentioned that there is an effect of the species and category, so what happened with the example and the intervals used- where no Holstein cows? what happens when there are more than 2 animals – no dyadic? I suggest adding a section with the limitations of the package. The text will benefit to have a better explanation of the references cited (lines 157-167), is very hard to follow the idea if there is not a little more elaboration of the cited literature. Specific comments: L25: Farm animal welfare instead of the welfare of farm animals. L30: creating instead of crated L55: Replacement need a definition L58: what does flexible means? Table 1: what happens when there are more than 2 animals? L 80: Bins L101-102: sentence is not clear L115: defined – describing L125: This need to be clarified- there is an effect of species and category, but you run an example in Nellore cattle using intervals from Holstein? L157-167: the methods mentioned here need a brief explanation as least L210: to the dominance L214: what does SH stands for? Reviewer #2: The manuscript describes the existence and basic use of the 'socialh' R package. In general, the manuscript is well written. However, there are many minor English language usage issues throughout the text. These could probably be remedied easily by having a native speaker proofread the text. The software seems reasonable as a starting point. It is somewhat rudimentary in its implementation, however, and lacks the following basic pieces: * Included sample data sets * Package vignette * Fleshed-out help functions The package should be considered as alpha-level at this point. In fact, the paper as it currently stands is really only at the level of an first-draft of a package vignette. It does not rise to the level needed to publish in PLOS ONE. To make this paper publishable in PLOS ONE, I think that most of the following are needed: * Demonstrate or add at least some capability to perform visualization of the results. * Demonstrate or add at least some capability to perform statistical analysis. For example, either implement statistical theory around the Landau index or the improved index. This could be done without much effort using the bootstrap, probably, but perhaps there is actually some theoretical work around these statistics. * Perform at least three analyses of data that are novel in some way. This could be analyzing datasets that the authors have available, or data from other researchers interested. * Or, alternatively, using data from the literature, perform analyses that add to the understanding of the data or that amplify, verify, or contradict the already published analyses. * Demonstrate the utility for large databases that is claimed in Line 285. A large database these days is probably on the order of 10 million records or more. * Optionally, add additional linearity evaluations and provide some empirical evaluation of them via simulation, followed by an evaluation of several real data sets. * Optionally, perform a set of simulations that adds to the ability of researchers to meaningfully choose among ranking, or provides some sort of empirical insight into the use of the software, or provides some empirical validation of the interpretation of the outputs. For future work, it might be useful to consider the following: * Generalize the use of methodology to determine dominance. That is, allow the user to specify different methodologies for this step. The authors list several, but hard-code only one. Flexibility could be achieved by using a function name as an input parameter, for example, where the function is expected to follow a specific API. That would allow a user to program their own methodology quite easily. This would avoid the authors' having to provide a hard-coded list of options. R provides other methods for increasing flexibility, this is only one suggestion. This would allow demonstrating empirically that different interpretations using different dominance evaluations, for example. * Similarly, make social rank a user-definable set of cut-points, rather than hard-coded as only 3 values and 3 labels, and at the default cut-points provided by the R cut() function. At a minimum, here, define these in the start of the function. * In general, avoid any hard-coding of parameters inside the functions. Make these user-definable at best or define them using variables at the start of the function at worst. At least the latter prepares the code for future generalization and makes coding decisions more visible. * If visualization methods are constructed, add ggplot interfaces so that users can automatically have access to a flexible graphical presentation system. ********** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. 25 Apr 2022 We have attached the file of "responses of review" with the answer of all comments. Submitted filename: Response to Reviewers.docx Click here for additional data file. 9 May 2022
PONE-D-21-29427R1
socialh : An R package for determining the social hierarchy of animals using data from individual electronic bins
PLOS ONE Dear Dr. Valente, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.
 
I invite you to revise your manuscript based on reviewer#2 feedback. Please note that a reviewer provided extensive editorial corrections and suggestions to the text, and I believe you should take that into account when revising your manuscript. Thanks. Please submit your revised manuscript by Jun 23 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:
If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Arda Yildirim, Ph.D. Academic Editor PLOS ONE Journal Requirements: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Additional Editor Comments: Dear authors; Reviewer#2 have expressed positive feedback and important comments and suggestions on various aspects of your study. I concur that the study has merit, but before a final recommendation by the PLOS ONE can be made, I invite you to revise your manuscript based on reviewer#2 feedback. Please note that a reviewer provided extensive editorial corrections and suggestions to the text, and I believe you should take that into account when revising your manuscript. Thanks. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: (No Response) ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: N/A Reviewer #2: N/A ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: No ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors have addressed all the comments and with that the paper has improve and therefore I considered that can be publish. Reviewer #2: Although the authors have made some additions to their manuscript, the overall level of the manuscript is still somewhat low. While the implementation of the calculations in R is useful, it is not necessarily difficult or groundbreaking; nor do there appear to be any surprising results or new insights shown or described. A quick glance at the following shows that they may be more appropriate places to publish for the current level of the manuscript: * The Journal of Statistical Software * The R Journal These journals are specifically geared toward providing a place to publish these types of implementations. The Journal of Statistical Software arguably tends toward more sophisticated analyses than the R Journal. It still seems necessary to demonstrate some overall usefulness of this new package in at least two more situations. An alternative would be some simulation studies using the package that could shed light on the capabilities or limitations of the methodology. It would be quite simple to implement bootstrap statistical tests to produce confidence limits and/or tests of point hypotheses. To summarize: * The package implementation is useful, but on its own is more suited to another journal setting. * One example of an analysis is interesting but does not seem to yield any insight or new results. * Perhaps performing two or more analyses could allow the authors to draw some conclusion about the methodology. * Perhaps using simulation studies the authors could derive some conclusion about the methodology. * It would be easy to implement some simple statistical methods --- maybe these would produce some interesting results for either this data set or others. * Perhaps there is deeper statistical theory already extant that could be incorporated. A few minor comments: * Line 345-346 Again, these are simply not large databases, especially for the simple calculations performed by the software. * Understanding that the parameters are already defined for the functions, it is still best to make them user-definable in general. This can be easily accomplished using default values, so the user never needs to set them if that is preferred. ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.
23 Jun 2022 We thank the editor and the reviewer for all contributions to our manuscript. We are grateful for the reviewer’ feedback, which helped improve the quality of the manuscript. We have answered to all comments and requests made by the reviewer. The changes in the manuscript are highlighted in yellow. The details of the responses are given bellow each comment in the "Response to Reviewers" file. Submitted filename: Response to Reviewers.docx Click here for additional data file. 29 Jun 2022 socialh : An R package for determining the social hierarchy of animals using data from individual electronic bins PONE-D-21-29427R2 Dear Dr. Valente, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Arda Yildirim, Ph.D. Academic Editor PLOS ONE Additional Editor Comments (optional): Many thanks for sincerely and thoroughly considering and attending to the comments and concerns. Regards, Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #2: All comments have been addressed ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #2: (No Response) ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #2: (No Response) ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #2: (No Response) ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #2: (No Response) ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #2: The authors have clarified some points, have provided several extensions to the analysis, and have provided more context in the form of real-world data analysis that should make it more suitable for the PLOS ONE audience. My apologies for missing that the data were in fact available. ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #2: No ********** 5 Jul 2022 PONE-D-21-29427R2 socialh: An R package for determining the social hierarchy of animals using data from individual electronic bins Dear Dr. Valente: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Prof. Dr. Arda Yildirim Academic Editor PLOS ONE
  12 in total

1.  Short communication: Automatic detection of social competition using an electronic feeding system.

Authors:  J M Huzzey; D M Weary; B Y F Tiau; M A G von Keyserlingk
Journal:  J Dairy Sci       Date:  2014-03-13       Impact factor: 4.034

2.  Technical note: Using an electronic drinker to monitor competition in dairy cows.

Authors:  Paige V McDonald; Marina A G von Keyserlingk; Daniel M Weary
Journal:  J Dairy Sci       Date:  2019-02-01       Impact factor: 4.034

3.  circlize Implements and enhances circular visualization in R.

Authors:  Zuguang Gu; Lei Gu; Roland Eils; Matthias Schlesner; Benedikt Brors
Journal:  Bioinformatics       Date:  2014-06-14       Impact factor: 6.937

4.  A practical guide for inferring reliable dominance hierarchies and estimating their uncertainty.

Authors:  Alfredo Sánchez-Tójar; Julia Schroeder; Damien Roger Farine
Journal:  J Anim Ecol       Date:  2017-11-27       Impact factor: 5.091

5.  Automatic detection of feeding- and drinking-related agonistic behavior and dominance in dairy cows.

Authors:  B Foris; A J Thompson; M A G von Keyserlingk; N Melzer; D M Weary
Journal:  J Dairy Sci       Date:  2019-08-07       Impact factor: 4.034

6.  Factors associated with estrous expression and subsequent fertility in lactating dairy cows using automated activity monitoring.

Authors:  C M Tippenhauer; J-L Plenio; A M L Madureira; R L A Cerri; W Heuwieser; S Borchardt
Journal:  J Dairy Sci       Date:  2021-03-02       Impact factor: 4.034

Review 7.  Review: Environmental impact of livestock farming and Precision Livestock Farming as a mitigation strategy.

Authors:  Emanuela Tullo; Alberto Finzi; Marcella Guarino
Journal:  Sci Total Environ       Date:  2018-10-04       Impact factor: 7.963

8.  Dairy cow preference for access to an outdoor pack in summer and winter.

Authors:  A M C Smid; E E A Burgers; D M Weary; E A M Bokkers; M A G von Keyserlingk
Journal:  J Dairy Sci       Date:  2018-12-26       Impact factor: 4.034

9.  Time of Grain Supplementation and Social Dominance Modify Feeding Behavior of Heifers in Rotational Grazing Systems.

Authors:  Gabriela Schenato Bica; Luiz Carlos Pinheiro Machado Filho; Dayane Lemos Teixeira; Karolini Tenffen de Sousa; Maria José Hötzel
Journal:  Front Vet Sci       Date:  2020-03-06
View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.