Validity and reliability of new agility test among elite and subelite under 14-soccer players.

BACKGROUND: Agility is a determinant component in soccer performance. This study aimed to evaluate the reliability and sensitivity of a "Modified Illinois change of direction test" (MICODT) in ninety-five U-14 soccer players.
METHODS: A total of 95 U-14 soccer players (mean ± SD: age: 13.61 ± 1.04 years; body mass: 30.52 ± 4.54 kg; height: 1.57 ± 0.1 m) from a professional and semi-professional soccer academy, participated to this study. Sixty of them took part in reliability analysis and thirty-two in sensitivity analysis.
RESULTS: The intraclass correlation coefficient (ICC) that aims to assess relative reliability of the MICODT was of 0.99, and its standard error of measurement (SEM) for absolute reliability was <5% (1.24%). The MICODT's capacity to detect change is "good", it's SEM (0.10 s) was ≤ SWC (0.33 s). The MICODT is significantly correlated to the Illinois change of direction speed test (ICODT) (r = 0.77; p<0.0001). The ICODT's MDC95 (0.64 s) was twice about the MICODT's MDC95 (0.28 s), indicating that MICODT presents better ability to detect true changes than ICODT. The MICODT provided good sensitivity since elite U-14 soccer players were better than non-elite one on MICODT (p = 0.005; dz = 1.01 [large]). This was supported by an area under the ROC curve of 0.77 (CI 95%, 0.59 to 0.89, p<0.0008). The difference observed in these two groups in ICODT was not statistically significant (p = 0.14; dz = 0.51 [small]), showing poor discriminant ability.
CONCLUSION: MICODT can be considered as more suitable protocol for assessing agility performance level than ICODT in U-14 soccer players.

RCT Entities:   Population Interventions Outcomes

Introduction

Agility is a common term that is used by strength and conditioning practitioners and is often considered the basic element of many sports and activities. A soccer player changes direction every 2–4 seconds and makes 1,200–1,400 changes of direction during a game [1]. Thus agility is a fundamental physical quality for the optimal performance of soccer players [2].

In most research, the term agility has been applied to describe any quick and effective combination of braking, changing direction, and accelerating again while maintaining motor control in either a vertical or horizontal direction in response to a stimulus (i.e., an opposing player’s movements, movement of the ball) [3], [4]. In this context, in order to perform successfully, team sport players need to be good movers in different directions, and often, in confined spaces [5]. Thus, improving the change of direction speed (CODS) of soccer players has become a focus of training programmes and consequently, many studies have been conducted to enhance and assess this athletic quality [6], [7].

It has been well established that fitness testing is crucial for sport performance optimization. It is a feature of most contemporary team sport codes. It quantifies and monitors the discrete training-induced adaptations and effects of training and conditioning program. It is not surprising that a strong interest exists in developing and validating field tests of CODS (e.g., the T-test) [7], the Illinois agility test [8], the 505 test [9], the L-run [10], and the zigzag test [11] in order to allow researchers to effectively measure the specific CODS of team sport athletes. The Illinois CODS test (ICODT) is well accepted as a standard test of change of direction speed in team sports and has been used extensively, particularly in soccer players [12], [13], [14], [15], [16]. The ICODT is a timed task involving straight sprinting and multiple direction changes around obstacles. The generic cues of the ICODT closely replicate the majority of those of the movement pattern of soccer. According to Brughelli et al. [12], the ICODT can be described as a rare CODS test that incorporates as many as 12 changes of direction with a dominant horizontal application of force during the effort. However, the ICODT performance might be heavily influenced by the ability to sprint quickly over short distances [17]. Moreover, the duration of this CODS test is approximately 16–18 seconds, and the total sprinting distance covered is approximately 60 m. However, it is well established that the mean distance and duration of sprints during a match in young soccer players is∼25 m and ∼3s, respectively [18]. Therefore, the ICODT might not represent a sport-specific test for young soccer players. Moreover, this test might be overly strenuous for young players, which might also affect the validity and/or reliability of the test. In order to respond to the need for a more specific CODS test for young soccer athletes, we propose a modified version of the ICODT (i.e., MICODT). In this new version, the same nature of directional changes was maintained, but we reduced the total distance covered (i.e., ∼30 m). Moreover the starting position was modified from laying on the ground in a prone position to a standing position. Indeed, the original starting position was not specific to soccer, as almost all bouts during a match are triggered from a standing position.

Thus, the aims of this study were, firstly to examine the absolute and relative reliability of MICODT and ICODT in U-14 soccer players, and secondly to investigate if a MICODT and ICODT could reveal differences in CODS performance between two U-14 soccer players group of different levels of expertise (i.e., elite and non-elite group).

Methods

Experimental Approach to the Problem

Young athletes are widely involved in soccer practice. A soccer match final result is widely recognized to be mainly function of the high speed action performed. Therefore, assessing the change of direction speed of young athletes seems to be extremely needed. To that end, young soccer players involved within the present study performed the ICOD and MICOD tests with a self-administered start in random order and in two different days (24 hours apart). In the first day athletes performed the MICODT and in the second day they performed again the MICODT and the ICODT. All athletes performed three trials of each tests within the two sessions. Recovery period was 3 minutes between trials and 5 minutes between tests (i.e., MICODT and ICODT) to ensure adequate recovery [17]. The best performance was recorded for each player during the two tests. Test procedures were performed in a synthetic surface at the same time of day (i.e., 7–9 p.m.) and in similar environmental conditions (i.e., 24–26°C, 50–60% humidity). The participants were instructed to maintain consistent dietary and sleeping patterns for 48 hours before each session and to refrain from strenuous activity for 24 hours before each session. Before testing, subjects completed a 15-min warm-up, including jogging, lateral displacements, dynamic stretching and jumping.

Participants

The human subject committee of the local university (i.e., higher institute of sport and physical education Ksar said, Tunis, Tunisia) approved the study in accordance with the 1975 Declaration of Helsinki. Subjects and parental written informed consent was given prior to participation in this investigation. A total of 95 U-14 soccer players (mean ± SD: age: 13.61±1.04 years; body mass: 30.52±4.54 kg; height: 1.57±0.1 m) from a professional and semi-professional soccer academy, took part in this study. Participants trained at least three times per week for two hours per session. All participants had been involved in soccer training regularly for more than four years before the study. All subjects were free from any injury that would prevent maximal effort during performance testing. Moreover, they were frequent participants in experimental studies and were fully familiarized with the test procedures, prior to the onset of the study. For the reliability study of the ICODT and MICODT tests scores, 60 players (out of the 95 subjects) (mean ± SD: age = 13.8±1.5 years; body mass = 33.3±5.2 kg; height = 1.6±0.3 m) took part in this first phase. To investigate the construct validity of the MICODT, a group contained a total of 32 athletes divided to two groups of 16 players based on their expertise level participated to this second phase. Athletes from professional soccer academies (i.e., involved regularly in national level match) were assigned as an elite group (EG) (mean ± SD: age = 13.9±0.7 year; body mass = 34.2±5.2 kg; height = 1.6±0.2 m) whereas those who were from the semi-professional academy (i.e., involved only in local level match) (mean ± SD: age = 13.7±1.4 years; body mass = 33.2±4.3 kg; height = 1.3±0.2 m) were assigned to a sub-elite group (SEG). Non-significant (p>0.05) differences in mean age, height, and body mass were noted among the two groups. Also, there were non-significant (p>0.05) differences in the mean values of the maturation index of the players. This index was calculated using the following equation of Mirwald et al [19]: Maturity offset = −9.236+0.0002708×(Leg Length×Sitting Height)−0.001663×(Age×Leg Length)+0.007216×(Age×Sitting Height)+0.02292×(mass/Height) with R = 0.94, R2 = 0.891 and the standard error of estimate = 0.592. This is a non-invasive procedure of predicting years from peak height velocity as a measure of offset using anthropometric variables. Negative values were interpreted as pre-peak height velocity and positive values were interpreted as post-peak height velocity.

Anthropometric Measurements

All measurements were taken in the morning by the same experienced researcher. Dimensions included stature, body mass and sitting height. Stretch stature was measured with a wall-mounted stadiometer (±0.1 cm, Holtain Ltd, Crosswell, UK), sitting height with a stadiometer mounted on a purpose-built table (±0.1 cm, Holtain Ltd), body mass with a weighing device (+0.1 kg). The intraclass correlation coefficient for test-retest reliability and typical error of measurement for height, body mass and sitting height were 0.99, 0.98, 0.99, and 0.17%, 0.75% and 0.21%, respectively.

Illinois Change of Direction Test

The ICODT test outcomes were recorded using an electronic timing system (Microgate SARL, Italy). Two pairs of the electronic timing system sensors mounted on tripods were set at 1 m above the floor and were positioned 3 m apart facing each other on either side of the starting and finishing lines. To avoid undue switch-on of the timing system, participants had to position the front foot immediately before a line set 0.20 m from the photocell beam. The Illinois CODS test (Figure 1) is set up with four cones forming the agility area. On command, from a standing position athlete sprints 9.2 m, turns and returns back to the starting line, then, he swerves in and out of four markers, completing two 9.2 m sprints to finish the agility course [17]. No technical advice was given as to the most effective movement technique. Athletes were instructed to complete the test as quickly as possible. They were instructed not to cut over the markers but to run around them. If a subject failed to do this, the trial was stopped and re-attempted after the requisite recovery period.

Figure 1

Layout of the Illinois Change of Direction Speed Test (ICODT).

The Modified Illinois Change of Direction Test

The protocol for the MICODT (Figure 2) was the same as of the Illinois CODS test. Only the total distance covered, measures of inter-cones distance, and the number of the cones was modified. Criteria for accepted test trials were the same as those used on the Illinois CODS test.

Figure 2

Layout of the Modified Illinois Change of Direction Speed Test (MICODT).

Statistical Analysis

The statistical analyses were carried out using SPSS 19.0 program for Windows (SPSS, Inc, Chicago, IL, USA). Descriptive statistics were generated for all variables. The significance level considered in the present study was set at p<0.05. The normality assumption was checked using Shapiro-Wilk test and all variables in the two phases of the study showed a normal distribution.

Reliability was verified by calculating both intraclass correlation coefficient (ICC) and standard error of measurement (SEM) and construct validity of the MICODT by the receiver operator characteristic (ROC) curve.

Relative reliability was determined by calculating ICC [20]. We considered an ICC below 0.40 as poor, between 0.40 and <0.70 as fair, between 0.70 and <0.90 as good and ≥0.90 as excellent [21]. To test absolute reliability, SEM expressed as coefficient of variation (CV) was calculated. Heteroscedasticity was revealed by calculating the correlation coefficient between the absolute difference and the average of the test trials. The SEM was calculated by dividing the standard deviation of the difference between scores by √2 [20]. As a complement to SEM, the smallest worthwhile change (SWC) was also used. The SWC is determined by the rearrangement of Cohen’s d effect size calculation, where the smallest worthwhile effect (0.2) is multiplied by the between-subject SD. By comparing SWC with SEM, test usefulness can be determined using the thresholds proposed by Liow and Hopkins [22]. When SEM ≤ SWC the test’s capacity to detect change is considered “good”; when SEM = SWC it is considered “satisfactory”, and when SEM ≥ SWC the test is rated as “marginal” [20]. Effect size () was calculated using GPOWER software (Bonn FRG, Bonn University, Department of Psychology) [23]. The following scale was used for the interpretation of : <0.2, [trivial]; 0.2–0.6, [small]; 0.6–1.2, [moderate]; 1.2–2.0, [large]; and >2.0, [very large] [14], [24]. Pairwise comparisons were applied to determine any learning effect or systematic bias between sample mean scores for test and retest using a paired t test. Minimal detectable change (MDC95%), which indicate the smallest change that is not due to error, was calculated as MDC95 = SEM x√2×1.96 [15].

The criterion related validity has been established by assessing the relationship between MICODT and ICODT using a Pearson product moment correlation coefficient. Vincent [25] has suggested that an absolute correlation coefficient of 0.5–0.7 is considered low, one of 0.7–0.8 is considered moderate, and one of 0.9 or above is considered high.

Unpaired Student t test was used to compare EG and SEG players’ performances in the agility tests. The discriminant ability of the MICODT was analysed using ROC curve. The ability of the new agility test to discriminate young soccer player of different competitive levels was assessed using the area under the ROC curve (AUC) [26], [27]. The ROC curve is a method of assessing the discriminating level of a test used to classify individuals into two groups. It is calculated plotting the sensitivity against 1- specificity, where sensitivity is the percentage of individuals correctly identified by the test (e.g., elite level) and specificity is the percentage of individuals who were sub-elite level and correctly identified by the test. The area under the ROC curve was interpreted as the probability of correctly discriminating between soccer players of different competitive levels. An AUC of 0.5 is interpreted as no discriminatory accuracy and 1.0 as perfect discrimination [27]. As a general rule, the area under the ROC curve >0.70 with a confidence intervals (CI) >0.50 are commonly considered to indicate acceptable discriminative ability [27].

Results

Performances on the ICODT and MICODT Tests and Retests

In the analysis of reliability of the ICODT and MICODT, residual data for the two trials were normally distributed (p values ranged from 0.21 to 0.83). The test of equality of means (dependent test), showed no significant bias between ICODT and MICODT test and retest. The estimated effect size (), however, was interpreted as being more than trivial for all outcomes for the two CODS tests (Table 1).

Table 1

Descriptive statistics and results of relative and absolute reliabilities and MDC95% of ICODT and MICODT in U-14 soccer players.

	Trial 1	Trial 2	p	(dz)	ICC	SEM	SEM as CV	SWC	MDC95
	(Mean ± SD)	(Mean ± SD)				(s)	(%)	(s)	(s)
ICODT	18.62±1.26 s	18.60±1.33 s	0.70	0.01	0.94	0.23	1.24	0.25	0.64
MICODT	12.39±1.6 s s	12.38±1.63 s	0.64	0.006	0.99	0.10	0.81	0.33	0.28

SD: standard deviation; : effect size; ICODT: Illinois Change of Direction Test; MICODT: Modified Illinois Change of Direction Test; ICC: Intraclass Correlation Coefficient; SEM: Standard Error of Measurement; CV: Coefficient of variation; SWC: smallest worthwhile change; MDC

Reproducibility of Performance Scores on the ICODT and MICODT Tests

Both relative and absolute retest reliability of the ICODT and MICODT was excellent (Table 1). The SEM expressed as CV of ICODT and MICODT were within 1%. The ability of the ICODT to detect small performance change was satisfactory, while it was good for the MICODT (Table 1).

Minimal Detectable Change of ICODT and MICODT Tests

The MDC95 for ICODT and MICODT performances are presented in Table 1. The MICODT presents the smallest MDC95 value. The Pearson product moment correlation between ICODT and MICODT revealed a very good correlation (r = 0.77; p<0.001).

Construct Validity of Performance Scores on the ICODT and MICODT Tests

The ICODT and MICODT performances of EG and SEG players are listed in Table 2. Independent sample t-test revealed that EG players had significantly better performances (p = 0.005) than SEG players for MICODT. However, the difference observed in these two groups for ICODT was not statistically significant (p = 0.14), showing poor discriminant ability of the ICODT in U-14 soccer players.

Table 2

ICODT and MICODT score performances within elite and subelite U-14 soccer players.

Test	Elite player	Sub-elite player	p	(dz)
ICODT	19.77±0.85 s	20.16±0.70 s	0.14	0.50 [small]
MICODT	12.86 0.41 s	13.32 0.49 s	0.005	1.01 [large]

ICODT: Illinois Change of Direction Test; MICODT: Modified Illinois Change of Direction Test; dz: Effect size.

The results of the ROC curve analysis was established between EG and SEG U-14 soccer players. The MICODT was considered to have acceptable discriminant ability. The area under the ROC curve was 0.77 (CI 95%: 0.59 to 0.89, p<0.008) (Figure 3).

Figure 3

Receiver operating characteristics (ROC) curve for the MICODT between elite and sub-elite U-14 soccer players.

Discussion

To the authors’ knowledge, this is the first study to analyze the reliability and discriminant validity of a new test of change of direction speed (CODS) in U-14 soccer players. The main finding of this study showed that the MICODT is a more reliable and valid test than the ICODT for assessing CODS in U-14 soccer players.

The reliability of fitness testing is a critical issue. It is relevant in establishing the reproducibility of a test and is a prerequisite to establishing validity in future studies. It has been established that within the current investigation, the relative reliability of performance scores of both ICODT and MICODT was excellent. The ICC across the two trials exceeded 0.90. This is consistent with a recent research by Lockie et al. [28] who found a good relative reliability of the ICODT within a sample of soccer players’ athletes. Additionally, these values were comparable to previously reported results on the relative reliability of other agility tests. In this context, Pauole et al. [7] reported an ICC of 0.98 across three CODS T-test trials in college-aged men and women. Sheppard and Young [29] reported an ICC of 0.87 across two reactive agility test trials in thirty-eight Australian football players. Haj-Sassi et al. [30] revealed an ICC of 0.95 in fifty-two male physical education students in a modified CODS T-test.

Absolute reliability was gathered from repeated performances of both tests (i.e., test-retest) by calculating the SEM. A small SEM expressed as CV has been established for ICODT (1.24%) and MICODT (0.81%) showing good absolute reliability of both tests. Lockie et al. [28] revealed that ICODT presented and SEM of 0.29 s (2.5%), supporting the present results. In this study, the likelihood that the true values of the estimated difference in ICODT and MICODT performances (i.e., larger than the SWC) was verified. Results presented in table 1 showed that MICODT had a better ability to detect small performance change in the U-14 soccer players than the ICODT. Its SEM (0.10 s) was smaller than its SWC (0.33 s), while the capacity of the ICODT to detect small performance changes in CODS within U-14 soccer players can only be described as “satisfactory” considering that its SWC (0.25 s) was approximately equal to its SEM (0.23 s). Lockie et al. [28] reported that the SEM of the ICODT was greater than the SWC which means that the usefulness of this test was rated as “marginal”. These findings showed that the usefulness of the ICODT is a subject of discussion.

The SEM allows the calculation of the MDC95%. According to Lexell and Downham [31] the MDC95% allows the determination of whether a real change has occurred between test and retest and when a change in a test-retest performance is ±MDC95% a true change is indicated. The inspection of the results presented in Table 1 showed significant differences between the ICODT’s and MICODT’s MDC95% values. The MICODT’s MDC95% (0.28 s) was about half the ICODT’s MDC95% (0.64 s) indicating that the MICODT represents a better ability to detect true changes in CODS than the ICODT in U-14 soccer players.

Pearson product moment correlation coefficients were performed between ICODT and MICODT. The MICODT was significantly associated with the ICODT. The coefficients of determination showed that the MICODT and ICODT shared 60% common variance in U-14 soccer players. These results indicate that the MICODT could be used to evaluate CODS. Even if there is no widely recognised gold-reference agility test, this strong correlation could provide criterion validity to the MICODT as the ICODT has been widely used as an agility test in soccer players.

One of the prerequisites of test applicability in sport science is its construct validity. Construct validity is commonly established by testing differences between groups of subjects of different competitive level [32]. In the research design of the second phase of this study we operated control over construct (i.e., competitive level) supporting variables testing two groups of U-14 soccer players of different competitive level (i.e., EG and SEG). Results showed significant differences in performances scores in the MICODT between the two groups (i.e., EG vs. SEG) (Table 2). The presence of competitive-level sensitivity of the MICODT was further supported by ROC analysis. Recently, this statistical test has gained popularity [13]. It is currently used to study the discriminant validity of physiological and performance testing [27], [32]. Thus, a test was considered as discriminant if its area under the ROC curve was equal to or greater than 0.70 [26], [27]. In the present study, the area under the ROC curve represented the probability of correctly discriminating elite from sub-elite U-14 soccer player using the MICODT. Results showed that the area under the ROC curve was above 0.70 (0.77; CI 95%: 0.59–0.89) and that the test score able to distinguish between the two different competitive levels group was 12.78 s. This cut-off value gives a “true-positive rate” (sensitivity) of 88.2% and a “false-positive rate” (1-specificity) of 55.6% (Figure 3). Thus, ROC curve analysis suggests that the MICODT has acceptable discriminant ability if used to differentiate between elite and sub-elite U-14 soccer players. Additionally, such a test can guide coaches more effectively through conditioning and training programs across a season and also can help coaches to better understand differences between players in talent selection and identification. These results are consistent with previous findings within rugby players [33]. Authors revealed that the academy group was significantly faster in CODS performance compared to a club level group. Another study by Farrow et al. [34] reported that highly-skilled net ball players group was faster than the lesser-skilled ones in CODS performance. However, this kind of discriminative ability is absent for the ICODT, in view of the fact that no significant difference was detected between mean performances relative to the ICODT across U-14 soccer players’ levels of expertise (Table 2).

Overall, it seems that by keeping the same nature of displacement and reducing the distance to be covered (i.e., from 60 m to ∼30 m) and by modifying the starting position (i.e., from a prone position to a standing one) during the novel proposed CODS test (i.e., MICODT) make it more specific and less strenuous within U-14 soccer players when compared to the old version. This is supported by the better relative and absolute reliability as well as validity results of the MICODT compared to ICODT in U-14 soccer players.

Conclusion

Results from the present investigation suggest that the MICODT is a reliable and valid CODS test that can discriminate between U-14 soccer players of varying playing abilities. The use of the MICODT can be favoured instead of the ICODT since it could provide an alternative, more valid assessment of CODS. It can be used regularly, because of its simplicity and low cost, to monitor training programs intervention directed toward improving CODS performance within U-14 soccer players. Additionally, from the results presented above it seems that the MICODT has a better ability to detect true change in agility performance when compared to ICODT. This characteristic of the MICODT is of utmost importance to monitor accurately agility’s performance level of U-14 soccer players across the time. However, the main limitation of the current investigation is that the new test (MIOCDT), as with the old version (IOCDT), only assesses the physical component of agility (i.e., CODS). As cognitive and perceptual factors contribute a lot to agility performance, future studies testing both the physical and perceptual (i.e., decision making) components of this sports’ critical task (i.e. agility) in young soccer athletes must be conducted. It should also be noted, that in view of the low sample size for the ROC analysis, further studies, on a larger sample, should be conducted to confirm the present findings.