Validation of a Composite Test for Assessment of Readiness for Return to Sports After Anterior Cruciate Ligament Reconstruction: The K-STARTS Test.

BACKGROUND: : There is limited information on the appropriate timing of return to sports after anterior cruciate ligament (ACL) reconstruction. A composite test was developed to assess the athlete's ability to return to sports after ACL reconstruction: the Knee Santy Athletic Return To Sport (K-STARTS) test. HYPOTHESIS:: The K-STARTS test meets validation criteria for an outcome score assessing readiness for return to sports after ACL reconstruction. STUDY
DESIGN: : Diagnostic study. LEVEL OF EVIDENCE:: Level 3.
METHODS: : A prospective comparative study identified 410 participants: 371 participants who had undergone ACL reconstruction and a control group of 39 healthy participants. The K-STARTS score is calculated as the sum of 7 tests (8 components), for a maximal value of 21 points. Construct validity, internal consistency, discriminant validity, and sensitivity to change were used to validate this new test.
RESULTS: : The K-STARTS assessment showed a high completion rate (100%), high reproducibility (intraclass correlation coefficient, 0.87; coefficient of variation, 7.8%), and high sensitivity to change. There was moderate correlation with the ACL Return to Sports after Injury scale (ACL-RSI) and hop tests. There were no ceiling or floor effects. There was a significant difference between K-STARTS scores assessed at 6 and 8 months postoperatively (11.2 ± 2.7 vs 17.1 ± 3.2; P < 0.001). The K-STARTS score in the control group was significantly higher than that in the ACL reconstruction group (17.3 ± 2.1 and 13.7 ± 3.8, respectively; P < 0.001).
CONCLUSION: : The K-STARTS test is an objective outcome measure for functional improvement after ACL reconstruction. CLINICAL RELEVANCE:: It is important for the clinician to determine when return to sports is optimal after ACL reconstruction to reduce the current high risk of reinjury.

Anterior cruciate ligament (ACL) reconstruction is one of the most commonly performed orthopaedic procedures.[8,10,11,48] A successful ACL reconstruction procedure is well defined: a low rate of reoperation, recovery of physiological knee function, and ability to return to sports at the previous level.[35,42,47] Numerous nonspecific tools have been developed for broad assessment of functional outcomes after knee injuries.[23,32,33,43] Interest in return to sports, however, is quite recent, and these questionnaire-based general outcome scores have not been objectively validated to determine when a patient may get back to the playing field.[3,6,7,22] In a recent survey of surgeons, many admitted to simply applying a time-based “6 months rule” to allow athletes to return to normal activities without the fear of rerupture.[13] Grindem et al[14] recommend waiting at least 9 months prior to return to sports to reduce the reinjury rate. Nagelli and Hewett[39] have suggested that waiting as long as 24 months is advisable. Many studies, however, have questioned the value of a purely time-based criteria for return to sport.[19,27,45]

There has been much published regarding return to sport after ACL reconstruction. A range of nonvalidated, heterogeneous outcome measures have been used to date, including isokinetic strength testing, knee examination, patient-reported outcome measures, neuromuscular control assessments, and psychological tests.[7,17] A 2011 systematic review identified 35 studies that had objective criteria for assessing return to sport.[7] These criteria included muscle strength or thigh circumference (28 studies), general knee examination (15 studies), single-leg hop tests (10 studies), Lachman test (1 study), and validated questionnaires (1 study). Only 2 studies used multiple factors to determine whether return to sport was appropriate. They concluded that the literature showed a general lack of objective assessment regarding appropriate time to return to sport, leading to problematic decision-making. Another systematic review of level 1 randomized controlled trials reporting a minimum 2-year follow-up after ACL reconstruction with return to sport criteria found that 90% of studies failed to use objective criteria.[17] This lack of objective outcome criteria for return to sport may in part explain why ACL rerupture rates of up to 25% have been reported.[14,25,29,30,37,53-55] These are often related to precocious return to sport.[14,25,26,29,30,37,53-55]

The Knee Santy Athletic Return to Sport (K-STARTS) test is a composite test designed to assess fitness for return to sports after ACL reconstruction. It was developed in 2015 at the Santy Orthopaedic Centre in Lyon, France, after a multidisciplinary consultation with orthopaedic surgeons, physical therapists, sports physicians, and rehabilitation exercise specialists. This study aims to statistically validate the K-STARTS score using a population of patients who underwent ACL reconstruction and a healthy control group. We hypothesize that the K-STARTS test meets validation criteria for an outcome score assessing readiness for return to sports after ACL reconstruction.

Methods

Population

A prospective series, between January 2016 and January 2018, included 410 participants. Three groups were created (Table 1). The first group (G6M) included 371 patients who had undergone an ACL reconstruction. The inclusion criteria consisted of an ACL reconstruction of 1 knee and no history of injury on the contralateral knee. These patients underwent a K-STARTS assessment at 6 months postoperatively. Thirty-three patients were extracted from the G6M group and underwent a second K-STARTS assessment at 8 months postoperatively; they were included in a test-retest group. The control group (CG) comprised 39 healthy participants with no history of injury or previous surgery of the lower limbs.

Table 1.

Participant demographics[ ]

	G6M	GTR	CG
Patients, n (male/female)	371 (287/84)	33 (25/8)	39 (29/10)
Age, y	28.0 ± 9.9 [14; 64]	22.9 ± 6.2 [15; 37]	26.5 ± 8.0 [19; 54]
Height, m	1.75 ± 0.09 [1.51; 1.99]	1.74 ± 0.11 [1.55; 2.00]	1.74 ± 0.09 [1.50; 1.90]
Mass, kg	73.6 ± 13.5 [42; 120]	70.6 ± 12.1 [51; 115]	65.9 ± 12.1 [41; 96]***
Body mass index, kg/m2	24.0 ± 3.3 [16.4; 36.4]	23.2 ± 3.0 [19.2; 32.8]	21.6 ± 3.0 [17.0; 30.3]***
Sport involvement[ b ]	59/229/65/18	1/20/11/1	12/18/9/0[§§§]

CG, control group; G6M, anterior cruciate ligament reconstruction group; GTR, test-retest group.

Mean ± standard deviation of demographic characteristics and sport involvement of the 3 groups. Minimum/maximum in brackets, with *** denoting significant differences between CG and G6M (P ≤ 0.001), and §§§ denoting significant difference between G6M and CG at casual level (P < 0.001).

Expressed in numbers for casual (<1 hour per week), regular (2-4 hours per week), intensive (>6 hours per week), or professional level, respectively.

Construction of the K-STARTS Test

Tests were included based on the strength of evidence for their use in the assessment of readiness for return to sport after ACL reconstruction.

All tests were supervised by the same 2 rehabilitation exercise specialists trained in the assessment of each of the components of the test. The K-STARTS score is calculated as the sum of 8 components analyzed through 7 tests, for a maximum of 21 points: ACL Return to Sports after Injury scale (ACL-RSI) (3 points), single-leg landing (0-3 points), dynamic valgus (yes or no; if yes, –3 points), side-to-side difference (%) for the single-hop test (0-3 points), side-to-side difference (%) for the triple-hop test (0-3 points), side-to-side difference (%) for the side-hop test (0-3 points), side-to-side difference (%) during the crossover hop test (0-3 points), and modified Illinois test (0-3 points).

The report generated provides the composite test score as well as the individual component scores so the clinician knows where to target therapy.

ACL-RSI[52]

Points are awarded according to the results on the ACL-RSI questionnaire. Patients with an ACL-RSI mark of 76% or more scored 3 points, between 63% and 76% scored 2 points, between 55% and 63% scored 1 point, and below 55% scored 0 points.

Single-Leg Landing

As per the qualitative analysis of single-leg loading (QASLS),[20] scoring was defined as a 0 for appropriate strategy, with 1 point for each inappropriate movement, as follows: patients with 0 points obtained a score of 3, 1 point obtained a score of 2, 2 points obtained a score of 1, and when pain prevented the patient from attempting the test or the patient scored 3 points or more, a mark of 0 was obtained.

Dynamic Valgus Penalty

A penalty of 3 points was applied when the patient was judged to have dynamic valgus of the limb on jump landing.

Hop Tests

The high number of marks awarded for hop tests in the K-STARTS score (up to 12 points) reflects the large number of published studies[15,24,41,50] that use hop testing as an objective outcome measure.

The limb symmetry index (LSI) is the percentage deficit of the distance hopped on the involved leg compared with the contralateral uninvolved leg.[1,38] To build the K-STARTS score, points were awarded according to the LSI result on each hop test (single, triple, side, and crossover). Participants with 90% or more obtained 3 points, between 80% and 90% scored 2 points, 80% or less had 1 point, and when pain prevented the test, no points were attributed.

Modified Illinois Test

The Modified Illinois Change of Direction Test (MICODT) is a variation of the Illinois Change of Direction Speed test, a well-accepted standardized test of ability to change direction.[16] To build the K-STARTS score, we awarded points according to the average time for the MICODT. An average time of ≤12.5 seconds scored 3 points, 12.5 to 13.5 seconds scored 2 points, >13.5 seconds scored 1 point, and when pain prevented the test, no points were given.

Statistical Analysis

The Consensus-based Standards for Selection of Health Measurement Instruments was used as a guideline while validating the K-STARTS test.[36] All statistical tests were performed using SPSS 11.0 software (IBM Corp), with the level of significance set at P ≤ 0.05. Ceiling or floor effects were considered to be present if more than 15% of respondents achieved the highest or lowest possible score.

Construct Validity

The Pearson coefficient of correlation (r) was calculated to measure the strength of association between the individual components on each other as well as by each component on the total K-STARTS score. The Spearman coefficient of correlations (ρ) was computed to evaluate the amount of variation explained by each component on the total K-STARTS score. Correlations were considered low, moderate, or strong at r or ρ ≥0.3, ≥0.5, and ≥0.7, respectively.

Internal Consistency

The K-STARTS test is represented by 8 components, and consistency is not an objective. Indeed, it is important in this situation to reflect different rather than similar or homogeneous characteristics of the knee function. Higher Cronbach alpha may be attained by narrowness of content that can limit predictive utility.[31] Schmitt[46] cautioned against the view that α should necessarily be >0.70. He also noted that as coefficient alpha is not a measure of unidimensionality, it may underestimate reliability if a scale is multidimensional. The Cronbach alpha coefficient (α) was performed to demonstrate that each component assessed different things. In this context, we looked for 0.60 < α < 0.70.

Discriminant Validity

The K-STARTS score should discriminate between groups with an established difference in injury severity. The known-group difference was tested by comparing the K-STARTS score between the ACL reconstruction group (G6M) and the healthy control group (CG), using the Welch t test for 2 independent samples.

Reproducibility and Sensitivity to Change

Reproducibility of the K-STARTS score was assessed by asking 10 patients to repeat assessment 24 hours later. For the reliability analysis, the intraclass correlation coefficient (ICC) and the standard error of measurement (SEM) for repeated measures and the coefficient of variation (CV) were calculated. The minimal detectable change (MDC) was calculated using the formula MDC = √2 * 1.96 * SEM and used to define the sensitivity to change. Comparing the K-STARTS test at 6 months and at 8 months postoperatively using the Student t test for paired samples provided information on sensitivity to change.

Results

The 3 groups presented similar demographics and sports involvement (Table 1). Mean raw values for the 8 components of the K-STARTS for the 3 groups are presented in Table 2.

Table 2.

Results of the different components of the K-STARTS test[ ]

			GTR
		G6M	6 months	8 months	CG
ACL-RSI	Raw[ b ]	67.7 ± 17.2 [7; 100]	62.0 ± 14.2 [28; 95]	79.4 ± 13.6 [37; 97]	91.4 ± 9.2 [64; 100]
	Score[ c ]	1.7 ± 1.2 [0; 3]	1.8 ± 1.2 [0; 3]	2.5 ± 1.0 [0; 3]	2.9 ± 0.2 [2; 3]
Single-leg landing	Raw[ d ]	91/243/33/4	5/12/1/0	14/17/2/0	8/17/1/0
	Score[ c ]	2.1 ± 0.6 [0; 3]	2.0 ± 0.5 [1; 3]	2.4 ± 0.6 [1; 3]	2.2 ± 0.5 [1; 3]
Valgus	Raw[ e ]	101/270	6/12	13/19	8/18
	Score[ c ]	−2.2 ± 1.3 [–3; 0]	−2.6 ± 1.0 [–3; 0]	−1.8 ± 1.5 [–3; 0]	−2.3 ± 1.3 [–3; 0]
Single-hop test	Raw[ b ]	90.2 ± 9.0 [60.0; 116.0]	84.9 ± 8.8 [60.0; 101.9]	94.8 ± 7.7 [72.0; 104.9]	99.2 ± 5.6 [87.1; 110.0]
	Score[ c ]	2.5 ± 0.7 [1; 3]	2.1 ± 0.7 [1; 3]	2.8 ± 0.5 [1; 3]	2.9 ± 0.2 [2; 3]
Triple-hop test	Raw[ b ]	89.9 ± 8.1 [63.4; 109.4]	85.2 ± 7.1 [66.9; 100.0]	93.2 ± 5.5 [81.3; 102.5]	98.0 ± 5.5 [84.7; 112.0]
	Score[ c ]	2.5 ± 0.7 [1; 3]	2.1 ± 0.7 [1; 3]	2.8 ± 0.4 [2; 3]	3.0 ± 0.2 [2; 3]
Side-hop test	Raw[ b ]	87.5 ± 14.9 [0; 144.7]	82.6 ± 14.1 [40.4; 100.0]	95.6 ± 8.4 [71.4; 117.6]	65.1 ± 9.6 [79.2; 128.6]
	Score[ c ]	2.3 ± 0.8 [0; 3]	2.1 ± 0.8 [1; 3]	2.8 ± 0.6 [1; 3]	2.8 ± 0.4 [2; 3]
Crossover hop test	Raw[ b ]	90.5 ± 10.1 [0; 118.8]	83.3 ± 7.7 [70.7; 118.8]	95.3 ± 6.0 [82.6; 108.3]	100.0 ± 6.2 [82.9; 115.1]
	Score[ c ]	2.5 ± 0.7 [0; 3]	1.9 ± 0.7 [1; 3]	2.8 ± 0.4 [2; 3]	3.0 ± 0.2 [2; 3]
Modified Illinois	Raw[ b ]	12.73 ± 1.16 [10.65; 19.44]	12.74 ± 0.95 [11.55; 15.07]	12.05 ± 0.90 [10.88; 14.59]	11.96 ± 1.07 [10.35; 14.72]
	Score[ c ]	2.3 ± 0.8 [1; 3]	2.3 ± 0.8 [1; 3]	2.7 ± 0.6 [1; 3]	2.7 ± 0.6 [1; 3]
K-STARTS	Score[ c ]	13.7 ± 3.8 [3; 21]***	11.2 ± 2.7 [6; 18]	17.1 ± 3.2 [9; 21][§§§]	17.3 ± 2.1 [14; 21]

ACL-RSI, Anterior Cruciate Ligament Return to Sports after Injury scale; CG, control group; G6M, anterior cruciate ligament reconstruction group; GTR, test-retest group; K-STARTS, Knee Santy Athletic Return To Sport.

Mean ± standard deviation, raw (or numbers), and score values for each component and K-STARTS in the 3 groups. Minimum/maximum in brackets, with *** denoting significant differences between CG and G6M (P ≤ 0.001), and §§§ denoting significant difference between 6 and 8 months postsurgery in GTR (P < 0.001).

Expressed in percentage (%).

Expressed in points.

Expressed in number of compensations (0, 1, 2, 3+, respectively).

Expressed in number absent and present, respectively.

Construct Validity

Pearson coefficient of correlation (r) and Spearman coefficient of correlations (ρ) for all components are presented in Table 3. The K-STARTS score demonstrated low correlation with the single-leg landing and dynamic valgus component and moderate correlation with the ACL-RSI, hop tests, and modified Illinois test. There was no correlation between the individual components of the test except for among the different components of the hop tests.

Table 3.

Coefficients of Correlation[ ]

		ACL-RSI	Single-Leg Landing	Valgus	Single-Hop Test	Triple-Hop Test	Side-Hop Test	Crossover Hop Test	Modified Illinois Test
r	Single-leg landing	−0.07
	Valgus	0.04	–0.74
	Single-hop test	0.28	−0.07	0.06
	Triple-hop test	0.23	−0.05	0.02	0.69
	Side-hop test	0.24	−0.04	0.00	0.46	0.45
	Crossover hop test	0.23	−0.06	0.0E	0.56	0.64	0.39
	Modified Illinois test	–0.14	0.02	−0.03	–0.31	–0.39	–0.32	–0.34
	K-STARTS	0.53	–0.48	0.49	0.65	0.65	0.55	0.60	–0.51
ρ	K-STARTS	0.57	0.47	0.48	0.62	0.62	0.59	0.62	0.51

ACL-RSI, Anterior Cruciate Ligament Return to Sports after Injury scale; K-STARTS, Knee Santy Athletic Return To Sport test.

Significant differences (P < 0.001) are in boldface.

Internal Consistency

The Cronbach alpha coefficient was 0.65, which was lower than 0.7, indicating that there was little internal consistency between the 8 components included in the K-STARTS test.

Discriminant Validity

The Welch t test for 2 independent samples revealed a significant difference between the CG and G6M groups (effect size, 1.72 [large effect]; P < 0.001). The K-STARTS score in the CG group was significantly higher than in G6M (95% CI, 2.8-4.3) (see Table 2).

Reproducibility and Sensitivity to Change

Ten patients repeated the K-STARTS test 24 hours after their initial test. The ICC for the repeated measures was 0.87, the corresponding SEM was 1.2 points, and the CV was 7.8%. The MDC was 3.3 points, which represents the minimum change required to be 95% confident that real clinical change has occurred.

The Student t test for paired samples showed a significant difference between the K-STARTS score assessed at 6 and 8 months postoperatively (effect size, 1.40 [large effect]; P < 0.001). The K-STARTS score was significantly higher at 8 months postoperatively when compared with 6 months postoperatively (95% CI, −7.1 to −4.2) (see Table 2).

Completion Rate

The completion rate was 100%. If a patient was unable to perform part of the test, he or she scored 0 for that section.

Ceiling and Floor Effects

Only 16 of 371 patients (4.3%) had a maximum score. No patients had a minimum score.

Discussion

The main finding of this study is that the K-STARTS test meets the criteria for validation as an objective test for return to sports after ACL reconstruction. It had a high completion rate and high reproducibility. There was moderate correlation with the ACL-RSI and hop tests, which are tests that have demonstrated good evidence for assessment of return to sports after ACL reconstruction.[15,24,28,41,50]

There was little internal consistency between the 8 components included in the K-STARTS test. The intention was to have low internal consistency, as the various components of this composite test are designed to assess different aspects of ACL rehabilitation. For questionnaires in which the items are different aspects of a complex clinical phenomenon that do not have to be correlated, internal consistency is not relevant.[49] The sensitivity to clinically important change is an aspect of great importance. The SEM (1.2) represents the error associated with an individual score taken at any time point. The MDC represents the minimum amount of change required to exceed the measurement variability (to be 95% confident that real clinical change has occurred). In this case, it was 3.3 points. Clinically, this could represent an increase in 1 grade across 3 items of the K-STARTS questionnaire, or even an increase from 0 to 3 points in 1 item of the questionnaire.

As one would expect, the K-STARTS score in the control group (17.1 ± 2.1) was significantly higher than in the ACL reconstruction group (13.7 ± 3.8). There was, however, no ceiling effect demonstrated. It is useful for an assessment tool to be capable of assessing function without having a ceiling effect. Ceiling effects are considered to be present if more than 15% of respondents achieved the highest possible score.[49] Only 16 patients (4.3%) had a maximum score, and no patients had a minimum score, so there was also no floor effect. Interestingly, many participants in the healthy control group did not achieve a maximum K-STARTS score. This suggests that a maximum score may not be required to return to sport. A follow-up study analyzing long-term data on return-to-sport rates with graft rupture and reinjury rates is required to determine the safe score for return to sport.

Return to sport after ACL reconstruction is much lower than would be expected, with only 65% of patients returning to their preinjury level of sport and only 55% to competitive sport.[5] Graft rupture rates of up to 25% have been reported, and these are often related to early return to sport.[14,25,29,30,37,53-55] One study demonstrated that patients who returned to high-level sporting activity had a 4-fold higher rate of reinjury to the knee.[14] This rate was reduced by 50% for each month return to sport was delayed from 6 to 9 months after surgery. This change in patients’ rehabilitation status was reflected in the difference in the K-STARTS score seen at 6 months postoperatively (11.2 ± 2.7) compared with that when repeated at 8 months (17.1 ± 3.2; P < 0.001). Other studies, however, have questioned the value of a purely time-based criteria for return to sport.[19,27,45]

Insufficient neuromuscular control during dynamic movements may be a major factor in both primary and secondary (postoperative) ACL injury risk.[21,44,56] During various landing and cutting exercises, excessive knee abduction moments and frontal plane trunk displacement may be predictive of ACL injury.[56] There is currently, however, some debate about whether increased knee abduction moments are associated with increased risk of ACL injury, with Goetschius et al[12] unable to reproduce the results of the study by Hewett et al[21] showing this association. These movements have also been linked with increased stress loading, leading to potential graft failure in ACL reconstruction patients.[21,56] The qualitative scoring system (QASLS) used to calculate this component of the K-STARTS score had excellent validity when compared with 3-dimensional motion capture kinematics during single-leg landing.[20] In a number of studies, isokinetic quadriceps strength testing has also been used as a factor in determining readiness for return to sports.[7] We decided to exclude strength testing from the K-STARTS score, as we wanted a test that was simple to administer by a clinician without the need for specialized equipment.

Recently, the idea of a barrage of tests that can assess many of these outcomes together has gained popularity.[18,19,26,34,40] Kyritsis et al[26] demonstrated that athletes who did not meet satisfactory scores on 6 tests before returning to professional sport had a 4 times greater risk of ACL graft rupture. Nawasreh et al[40] found an increased rate of return to sport in patients who passed a battery of 4 tests (81% and 84% at 12 and 24 months, respectively) compared with those who had failed (44% and 46%; P = 0.012). Herbst et al[19] evaluated a series of physical assessments, comparing patient results with normative data to matched healthy controls. All the above evaluation systems provide binary pass/fail outcomes for each component, requiring the patient to achieve passes in a minimum number of components to accomplish an overall pass. This differs from the K-STARTS test, which provides a continuous score rather than a binary outcome. This has the advantage of monitoring a patient’s change over time, as well as detecting differences between patients. This also allows the treating physician to adjust the rehabilitation as appropriate. The K-STARTS provides a composite score as well as the individual components so that the clinician can identify a patient’s areas of weakness on a case by case basis.

Physical tests have been the mainstay of evaluation for return to sport, but several studies have identified psychological readiness as an important factor.[2,4,9,51] A fear of reinjury and lack of confidence in the knee may influence return to sport.[2,4] The ACL-RSI scale was developed to measure the psychological impact of returning to sport after ACL reconstruction.[52] Langford et al[28] demonstrated that patients who had returned to competitive sport at 12 months scored significantly higher on the ACL-RSI scale than participants who had not. Ardern et al[4] assessed the results of 7 different knee questionnaires, analyzing all aspects of knee function, in 164 patients after ACL reconstruction. They found that psychological readiness to return to sport measured using the ACL-RSI was most associated with returning to preinjury levels. An advantage of the K-STARTS test is that it includes both physical tests for assessment of return to sports and a psychological assessment (ACL-RSI), to give a more holistic evaluation of the patient’s ability.

Limitations

All patients in the series were referred to an orthopaedic sports center, and the potential for referral bias exists. The number of participants used for the reliability component is small, and the time between measurements (24 hours) may introduce recall bias (for participants answering the questionnaire) or observer bias (of the clinicians rating quality of performance). There are many factors that contribute to a safe return to sport after ACL reconstruction, which may not all be assessed using surrogate outcome measures. The K-STARTS may measure functional improvement, but long-term data are required to see if this translates into a scoring system that assesses readiness for return to sport after ACL reconstruction. A follow-up study will assess this, analyzing return-to-sport rates with graft rupture and reinjury rates.

Conclusion

The K-STARTS test is an objective outcome measure for functional improvement after ACL reconstruction.