Novel strength test battery to permit evidence-based paralympic classification.

Ordinal-scale strength assessment methods currently used in Paralympic athletics classification prevent the development of evidence-based classification systems. This study evaluated a battery of 7, ratio-scale, isometric tests with the aim of facilitating the development of evidence-based methods of classification. This study aimed to report sex-specific normal performance ranges, evaluate test-retest reliability, and evaluate the relationship between the measures and body mass.Body mass and strength measures were obtained from 118 participants-63 males and 55 females-ages 23.2 years ± 3.7 (mean ± SD). Seventeen participants completed the battery twice to evaluate test-retest reliability. The body mass-strength relationship was evaluated using Pearson correlations and allometric exponents.Conventional patterns of force production were observed. Reliability was acceptable (mean intraclass correlation = 0.85). Eight measures had moderate significant correlations with body size (r = 0.30-61). Allometric exponents were higher in males than in females (mean 0.99 vs 0.30).Results indicate that this comprehensive and parsimonious battery is an important methodological advance because it has psychometric properties critical for the development of evidence-based classification. Measures were interrelated with body size, indicating further research is required to determine whether raw measures require normalization in order to be validly applied in classification.

INTRODUCTION

A total of 2.78 million tickets were sold to the London Paralympic Games, making the Paralympic Games the world’s third biggest sporting event, after the Olympic games and the FIFA World Cup. The movement is genuinely global—174 countries have National Paralympic Committees—and participation is increasing, with >6000 internationally registered athletes in the sport of athletics alone.[1]

Classification systems are integral to Paralympic sport, being used to determine eligibility and control for the impact of eligible impairment types on the outcome of competition.[2] Valid classification systems facilitate competition in which the athletes who succeed are not simply those who have less severe impairments than their competitors, but those who have the most favorable combination of athletic attributes and have enhanced them to best effect.

Classification that is not valid or that is not perceived to be valid poses a significant threat to Paralympic sport. At the elite level, the legitimacy of an individual’s competitive success or athletic achievement can be significantly diminished by the perception that they are in the wrong class, with the potential for considerable personal and financial costs, as well as for discrediting the movement. At the grass-roots level, a classification system that is perceived to be unfair will discourage participation among people with disabilities, rather than achieve the goal of increasing it.

Evidence-based decision making in classification is an essential means of enhancing classification validity, but evidence underpinning current methods of classification is weak. In 2007 the International Paralympic Committee (IPC) mandated the development of evidence-based methods of classification for all Paralympic sports[3] and the IPC Position Stand on Classification outlines key requirements for the development of evidence-based systems, including valid, reliable methods for assessing impairment.[2] Research in this area is limited; however, researchers are beginning to investigate the methods that may contribute to evidence-based classification in Paralympic sport.[4,5]

Impairments classified in Paralympic sport include impaired muscle strength, impaired range of movement, and limb deficiency. The focus of this article is impaired muscle strength, which is required in 16 of the 23 summer Paralympic sports, and is a key component of classification in Athletics (ie, track and field), the largest Paralympic sport. Methods for assessing strength have remained essentially unchanged since the first classification system was described by Sir Ludwig Guttman.[6] They are based on manual muscle testing (MMT) methods in which the strength of individual muscle actions (eg, elbow flexion, knee extension) are assigned a grade from 0 (no voluntary muscle contraction) to 5 (normal strength through normal anatomical range of movement) according to their capacity to overcome gravity and/or manual resistance.[7]

For the purposes of classification, MMT has a number of advantages: it is widely understood and utilized in clinical practice and, because it does not require any instrumentation, it is inexpensive and space-efficient. MMT also provides a body-size–independent assessment of strength. One final advantage of using MMT for classification is that the ability to overcome manual resistance is assessed using the break test,[8] an isometric contraction against manual resistance. Isometric assessments are advantageous for classification purposes because they are known to be relatively unresponsive to the high-speed, dynamic strength training regimes required for performance enhancement in athletics.[9,10] As a consequence, athletes who have optimized sports performance through high-speed, dynamic resistance training are less likely to change their classification strength measures and be placed into a class for athletes with less severe impairments.[2]

Unfortunately MMT also has several important disadvantages that make it unsuitable as a method of strength assessment for classification. First, acceptable interrater reliability is difficult to achieve, a problem exacerbated by the wide range of MMT techniques that are described in published literature and used by classifiers from different countries in their clinical practice.[11,12] Secondly, the relationship between muscle grade and activity limitation is weak. For example, an athlete with full passive range of motion but only 15 degrees of active elbow extension against gravity is likely to experience much more activity limitation in the shot put than an athlete with 100 degrees active range, and yet the correct muscle grade for both actions is 2.[12] This feature reduces the validity of MMT for the purposes of classification.

More fundamentally however, MMT methods are problematic because they use ordinal scales. Ordinal measurement scales are unsuitable for research that aims to develop evidence-based methods of classification, as mandated by the International Paralympic Committee.[3] Specifically, evidence-based methods of classification require quantification of the relative importance of different muscle actions in a given sporting movement, and therefore a ratio-scale measure of strength is necessary. When the relative importance of key muscle actions has been quantified using a ratio-scale measure, it will be possible to validly aggregate strength measures of contributing muscle groups in order to obtain an evidence-based estimate of how much activity limitation different strength impairments will cause, regardless of their distribution and severity.[2]

In order to permit the development of evidence-based methods of classification, our research group developed a battery of novel strength tests. Key features of the battery were that measures were isometric and therefore, according to theory, less training responsive than other dynamic strength tests; instrumented, yielding an outcome measure in Newtons (a ratio scale); comprehensive, assessing all muscle actions of importance in the key disciplines of Paralympic athletics (wheelchair racing, running, throwing, and jumping); and parsimonious, by assessing compound (or multijoint actions) and thereby minimizing the number of tests required and ensuring that individual tests accounted for the greatest possible variance in performance.[2]

The aims of this study were 3-fold: to establish normal performance ranges for each of the novel tests in nondisabled participants, to evaluate the reliability of each of the novel tests, and to assess the strength of association between individual test outcomes and body mass. In relation to the final aim, a sufficiently strong relationship with body size would indicate that, prior to applying these methods in classification, it may be necessary to develop body-size scaling methods that can be validly applied to measures obtained from athletes with neuromusculoskeletal impairments.

METHODS

Participants

Participants were 118 nondisabled, males (N = 63) and females (N = 55), ages 18 to 37 (mean ± SD 23.2 ± 3.7) recruited from the University of Queensland and local sports clubs. All were regularly active in competitive sport or engaged in 3 or more vigorous training sessions per week. The study was approved by the Ethics Committee of the School of Human Movement Studies, University of Queensland (number HMS07/0406) and all participants provided written informed consent prior to participation. Participants completed testing in a single session, with the exception of 17 who returned in a minimum of 2 days and a maximum of 14 days from the initial session, so that test–retest reliability could be analyzed. The sample size calculation was based on the requirement for calculation of allometric scaling exponents and a minimum number of 54 participants were required in each group (males and females) at an effect size of 0.15 with power set at 0.8 and probability at 0.05.

Strength Testing and Body Size

All participants completed a battery of 7 isometric strength tests—grip strength and 6 novel tests—which are presented in Table 1. The battery aimed to be both parsimonious and comprehensive, using the smallest possible number of tests to evaluate the strength of those movements considered most important in the key athletic disciplines (ie, running, jumping, throwing, and wheelchair propulsion). The order in which participants completed the tests was fully randomized.

Table 1

Strength and Body Size Test Protocols

Grip strength was assessed using a handheld dynamometer (Smedlay’s Dynamometer, Fabrication Enterprises, White Plains, NY, USA, 100 kg), with values entered directly to Excel 2007 spreadsheet (Microsoft, Redmond, WA, USA) and then converted to Newtons (N). The 6 novel tests were completed with the participant seated in a customized strength rig (Figure 1). The rig comprised a rigid, aluminum rectangular frame with an S-type load cell (Scale Components, Slacks Creek, QLD, Australia) rated to 394 kg (1000 lb) mounted at one end, opposite a rigid seat. An aluminium plate (250 mm × 196 mm × 12 mm) was secured to the load cell and, once seated, participants applied force to the load cell from a seated position by either pushing or pulling on the plate. Three features of the rig permitted positioning of participants so that force was applied to the load cell from anatomically standardized positions: the load cell was adjustable vertically and horizontally to account for individual differences in sitting height and breadth (see Figure 1, Panel B); the seat position was adjustable in the fore-aft direction to account for individual differences in arm and leg length (see Figure 1, Panel A); backrest height was adjustable to permit positioning at the C7 vertebra regardless of the participant’s sitting height.

FIGURE 1

Panel A shows the rigid aluminum frame in which the strength tests were conducted. The arrow indicates the ability of the chair to move in the fore-aft direction. Panel B shows a magnified view of the load cell set up with arrows indicating the ability to move the load cell vertically and horizontally.

The joint angles selected for each testing position aimed to position prime movers so that, as far as possible, length/tension relationships were optimized.[13] To achieve the precise leg angles described in Table 1, an anthropometric marking pen was used to clearly identify the following landmarks: acromion, greater trochanter, mid-point of lateral knee joint line, and the lateral malleolus. A Sony camera, positioned 1.5 m from the rig, in line with the mid-point of the seat surface at 1 m high, fed a live, sagittal view video image of the participant seated in the strength rig into a personal computer, on which Dartfish (version 4.0.9.0; Dartfish, Lausanne, Switzerland) was installed. The Dartfish angle tool was used to draw the required joint angles and the strength rig could then be adjusted until the participant was positioned so that their video image was matched with image created by the angle tool. The live feed permitted monitoring to ensure required positioning was maintained. This method could not be used for the arm angles which are only observable from an overhead view, and these angles were set using a handheld goniometer (SunShine Diagnostic and Measuring Instruments, New Taipei City, Taiwan) and monitored visually by a member of the testing team. Figure 2 shows athletes positioned for an upper body strength measure (Panel A) and a lower body strength measure (Panel B).

FIGURE 2

Panel A illustrates the position that participants were placed into for the bilateral Supported arm push. Panel B illustrates the position that participants were placed into for the leg extensor strength test.

Once positioned, participants performed 3 maximal isometric contractions of 5 seconds duration, each separated by 30 seconds of rest. Valid trials required peak force to be achieved slowly (>2 seconds and <3 seconds) followed by a 3-second hold at peak. To assist participants achieve peak force slowly, 2 submaximal practice trials were performed with real-time visual feedback of the force–time curve so that participants could easily identify when maximum force was achieved either too quickly or too slowly.

For all tests participants were instructed to “push the plate straight back into the wall.” Participants applied force with their hand(s) for upper limb tests and with their foot/feet for lower limb tests, with the rigid backrest permitting exertion of maximal voluntary force. Hands/feet were positioned so that force was directed through the long axis of the load cell and torque was minimized.

Load cell output was captured by Musclelab v4020e (Ergotest, Porsgunn, Norway) at a sampling rate of 100 Hz. Raw isometric strength scores were exported directly into an Excel 2007 spreadsheet from the load cell data acquisition program (Musclelab), and all trials were then processed in SciLab using a custom-written algorithm (Scilab Enterprises, Versailles, France) to acquire the peak isometric force from each trial. For each trial, the isometric force used for further analysis was calculated as the mean force over the 2-second period with the least variability. To ensure a plateau was achieved, a trial was deemed acceptable if the calculated force was a minimum of 95% of the peak force registered during the trial. The best trial was then employed for statistical analysis.

Body size was assessed through the measurement of body mass. Body mass was measured and recorded on each participant to the nearest 0.1 kg using scales (Seca 760 Mechanical Scales, Seca, Hamburg, Germany).

Data Analysis

All data were analyzed using SPSS v16 (SPSS Inc, Chicago, IL). The data were assessed for normality using the Shapiro–Wilk test of normality. Independent t-tests were performed on the strength data of males and females to determine whether strength tests were significantly affected by sex and should be analyzed separately. Test–retest reliability was assessed using dependant t-tests, 2-tailed intraclass correlations (ICC) method 3,1, standard error of the mean (SEM) and Bland–Altman plots.

The relationship between body mass and each of the strength measures was determined by calculating Pearson correlations and allometric scaling exponents. To calculate the latter, a log-linear regression analysis was conducted using strength as the dependent variable and body mass as the independent variable. The slope of the regression line was used as the allometric scaling exponent,[14] and residual plots were assessed to check the fit of each model. Exponents were calculated on the dominant side for the upper body and the right side for the lower body.

RESULTS

Isometric strength test data were normally distributed. Independent t-tests showed significant differences between males and females in all strength measures, and therefore these data were analyzed separately. Table 2 presents descriptive statistics for males and females for isometric strength measures. Overall, both male and female participants were symmetrical, with no significant difference in mean force production between dominant and nondominant arms, left and right leg flexors, and left and right leg extensors. In the lower limbs, extensor strength (male mean right leg: 1822.4N ± 454.6N; female mean right leg: 1193.5N ± 302.1N) was approximately 5 times greater than flexor strength (male mean right leg: 352.9N ± 78.3N, female mean right leg: 237.4N ± 53.6N).

Table 2

Descriptive Statistics for Body Mass and Strength Measure

The results of analyses for test–retest reliability are presented in Table 3. There was no systematic bias in the test–retest results as indicated by a nonsignificant t-test. For all tests, ICCs ranged from 0.71 to 0.95. The mean difference between the test and retest was consistently low for all tests—absolute difference range was 3.7 to 51.0  N, a relative difference of 0.2% to 7.3%. SEM values were also low and considered acceptable for all tests (18.12–117.44).

Table 3

Test–Retest Reliability of Novel Strength Measures (n = 17)

Pearson correlations showing the relationships between individual strength tests and body mass are presented in Table 4 for males, females, and the total sample. Correlations were moderate and significant for 6 of the 7 tests in males (r = 0.43–0.61, P = 0.000) and 2 of the 7 tests in females (r = 0.30–0.33, P = 0.014–0.03).

Table 4

Pearson Correlations and Allometric Scaling for Body Mass and Strength Measures

Allometric exponents were calculated separately for males and females; exponents were higher for males (range = 0.58–1.27) than for females (range = 0.06–0.67). Residual plots showed all models were appropriately fitted.

DISCUSSION

Currently, the method of strength assessment used in Paralympic athletics classification is based on MMT, an ordinal-scale measure. This is problematic because ordinal-scale measures do not permit quantitative evaluation of the impact of impairment on athletic performance, a fundamental requirement for the development of evidence-based methods of classification. The results from this study indicate that the ratio-scale strength assessment battery described could be validly applied to address this important methodological shortcoming and facilitate the development of evidence-based methods of classification. In addition to furnishing a ratio-scale measure of strength, the battery has a number of features that would be advantageous for Paralympic classification, including the measures are isometric and therefore training resistant; the battery is both comprehensive (assessing resultant forces produced by the muscle groups of principal importance in Paralympic Athletics) and parsimonious; and reliability is good to excellent. Furthermore, the sex-specific normal performance ranges that are reported will permit meaningful interpretation of results in athletes with impairments in future studies. The key advantages of the battery evaluated in this study are expanded in the following paragraphs.

The following example illustrates why ratio-scale measurement is of fundamental importance to the development of evidence-based methods of classification. In wheelchair racing, the current class T51 is based on the strength impairment profile of a person with a complete spinal cord injury at neurological level C5–6, including impaired shoulder flexion strength (up to grade 4) and triceps strength grade 0–3.[7] The current class T52 profile is based on the strength impairment profile of a person with complete spinal cord injury at neurological level C7–8, including normal shoulder flexion and triceps strength (ie, grade 5).[7] If an athlete with polio presents with normal shoulder flexion strength but grade 3 triceps, the classifier must decide whether the athlete is best in class T51 (even though they will have more shoulder flexion strength than other athletes in that class) or in class T52 (even though they will have less triceps strength than other athletes in class). In this instance, decision-making should be based on evidence regarding the relative importance of shoulder flexion and triceps extension to wheelchair propulsion. Current, ordinal-scale methods of strength assessment are not suitable for investigating this question.

Two additional advantages conferred by the proposed test battery are that it is both parsimonious and comprehensive. These advantages occur because, rather than assessing individual muscle actions acting over a single joint, a number of the proposed tests assess the resultant force of a number of key muscle groups acting over more than one joint. For example, the leg extension test assesses the combined strength of hip and knee extensors simultaneously, while the single Supported arm push assesses shoulder horizontal flexion and elbow extension simultaneously. Consequently, the test battery described in this study comprises 7 tests, rather than the 20 individual muscle grade tests required to assess the same muscle groups in the current system. In addition to saving time, the proposed test battery would considerably reduce the number of maximum contractions required from each athlete during classification, helping to ensure that fatigue did not the confound outcomes.

Importantly, the significantly reduced number of tests does not result in a less comprehensive battery—all the principal muscle actions required for the activities that are central in Paralympic athletics are evaluated (ie, wheelchair racing, running, jumping, and throws—both standing and seated). More specifically, the tests capture muscle synergies that are required for performance of the activities of interest. For example, the push pull test for the upper body uses the same prime movers as used for seated throwing with a pole: shoulder extension and elbow flexion of the nonthrowing arm simultaneously assessed at the same time as shoulder flexion and elbow extension on the throwing or dominant arm.

The sex-specific isometric strength ranges reported in this paper on nondisabled individuals will allow the interpretation of results obtained from athletes with impairments. In general, the overall pattern of results was consistent with what is known about strength—males were significantly stronger than females on all strength tests; the lower body was stronger than the upper body and the lower limb extensor strength was greater than the flexor strength. More specifically, Grip strength means (males = 526 N; females = 344 N) were comparable with means previously reported for this protocol (males = 523 N; females = 319 N)[15] and the mean bilateral supported arm push for males (N) was similar to that reported by Hortobagyi (9976 N) for a similar protocol performed in supine.[16] However, the battery also extends what is known about the relative strength of different movements—for example, combined hip and knee flexion was only 19.4% of extension in males and 19.9% in females. These percentages are considerably less than those reported in studies of isolated lower limb movements, which indicate that knee flexion strength is between 43% and 90% of knee extension strength.[17,18]

One final feature of the test battery described in this study which is important for the purposes of classification is that the reliability of all strength tests was excellent (ICC > 0.8) for all but one test.[19] The ICC for the leg flexor test was good with an ICC of 0.71,[19] which is acceptable given the low SEM (314 N) and the small mean difference between test one and test 2 (66 N).

As mentioned in the introduction, one advantage that current methods of strength assessment have is that they are independent of body size. This is an important feature because classification aims to control for the impact of strength impairment on athletic performance without controlling for other advantages conferred by body size. For example, 2 throwers with complete spinal cord injuries at T2 should compete in the same class for the discus, regardless of whether one is 2 m tall and weighs 100 kg and the other is 1.5 m tall and 65 kg. It will be important that any new measure of strength used for the purposes of classification is, as far as possible, independent of body size.

Results from this study indicate that a number of measures—1 in males (leg flexor strength) and 5 measures in females (single supported arm push, bilateral supported arm push, unsupported push pull, leg flexor and plantar flexor strength)—were not strongly or significantly related to body mass. This indicates that it is likely that these measures are sufficiently independent of body size to be validly applied in classification. However, 6 tests in males were moderately and significantly related to body mass (r = 0.43–0.61) and 2 tests in females showed a weaker but still significant relationship (0.30–0.33). Allometric exponents were also calculated to give an indication of the slope of the regression line as this would indicate how much change in strength would be expected for a given change in body mass. The allometric exponents that were calculated were much larger for males (mean of 1.06) than for females (mean of 0.3). This indicates that a given change in body size is associated with a larger change in strength in males than in females. It has been suggested by previous research that this may be due to a decrease in range of strength scores seen when males and females are analyzed separately,[16,20] or a result of higher relative percentages of lean body mass in males when compared with females.[15,21] These results indicate that further research is required to determine whether raw measures require normalization in order to be validly applied in classification.

Unfortunately the scaling exponents developed in this study on nondisabled participants based on body mass will not be able to be applied directly to athletes with disabilities. Body measurements such as body mass, height, or limb circumference are inappropriate in athletes with disabilities as neuromusculoskeletal impairment changes the relationship between these measures of body size and strength in ways that are unpredictable. This is because impairments to the central and peripheral nervous system and to the muscle fiber itself, which commonly affect Paralympic athletes, will disrupt the fundamental premise for scaling—that force is primarily determined by muscle cross-sectional area. For example, people affected by spastic paraplegia resulting from UMN injury will retain muscle bulk better than those affected by lower motor neuron injuries due to the presence of intact spinal-level reflexes, although the impairment of structures may result in comparable impairments of strength function.

It is posited that local bony dimensions—for example, humerus length or biacromial width—may be the most appropriate type of anthropometric measure by which to scale strength measures in athletes with neuromusculoskeletal impairment because, compared to body mass or muscle cross-sectional area, they more commonly remain unaffected by neuromusculoskeletal impairments. Research is required that identifies the most appropriate local bony dimensions by which to scale [22] and subsequently evaluates whether normalized or raw scores are more valid for classification purposes.

CONCLUSIONS

Ordinal-scale strength assessment methods are currently used in Paralympic athletics classification, preventing the development of evidence-based classification systems. The results from this study indicate that the ratio-scale strength assessment battery described could be validly applied to address this important methodological shortcoming and facilitate the development of evidence-based methods of classification by allowing a valid and reliable assessment of muscle strength in athletes with neuromusculoskeletal impairments. The battery has a number of other features that are advantageous for classification, including the measures are isometric and therefore training resistant; the battery is both comprehensive (assessing resultant forces produced by the key muscle groups) and parsimonious; and reliability is good to excellent. Furthermore, the sex-specific normal performance ranges that are reported will permit meaningful interpretation of results in athletes with impairments in future studies. The results from this study have implications for the 16 Paralympic sports that assess strength in the classification process.