A New Rating Scale for the Rapid Evaluation of High-Level Sports Ability.

BACKGROUND: Nearly half of patients with an anterior cruciate ligament (ACL) injury who have returned to sport after ACL reconstruction can return to sports activity at a competitive level, so emphasis should be focused not only on the timing of return to the sport but also on measuring the ability to participate in high-level sports activity.
PURPOSE: To develop and evaluate a new, self-administered rating scale for rapid evaluation of high-level sports ability among community-level athletes who return to sports after ACL reconstruction surgery. STUDY
DESIGN: Cohort study (diagnosis); Level of evidence, 2.
METHODS: We developed the new rating scale-the Knee Stability in Sports/Cutting-Pivoting Ability (KSS/CPA) scale-in 2 stages. Initially, we used a survey and roundtable discussion to achieve an expert consensus for the KSS/CPA scale from a group of independent orthopaedic experts. Next, 77 amateur athletes who underwent ACL reconstruction by a single surgeon were recruited for a background analysis of data to compare the new scale with results from the Marx activity rating scale, Tegner activity scale, International Knee Documentation Committee Subjective Knee Form, and self-assessment of overall knee function.
RESULTS: The KSS/CPA scale was applicable and effective for evaluating the high-level sports ability of community-level athletes who had returned to their sport after ACL reconstruction. Statistical analysis confirmed the test-retest reliability of the new rating scale (intraclass correlation coefficient, 0.85/0.84 postoperatively) as well as its internal consistency (Cronbach alpha coefficient, 0.73 preoperatively and 0.89 postoperatively), construct validity (Spearman correlation coefficient, >0.35 postoperatively), excellent discriminant validity, acceptable responsiveness, and reasonable minimal detectable change (<25).
CONCLUSION: The KSS/CPA scale can act as a supplement to other clinical outcome measures for a more comprehensive evaluation of community-level athletes' cutting-pivoting ability and knee stability.

Anterior cruciate ligament (ACL) tears constitute >50% of all knee injuries.[12] Moreover, 40% of sports injuries are attributed to noncontact mechanisms, including pivoting and cutting activities.[11,19] Younger athletes are at higher risk of multiple injuries of the ACL mainly because they are more likely to return to high-level sports activities too soon.[26]

Numerous knee-specific outcome measures have been reported[23] to evaluate the condition and function of the knee, including the Knee injury and Osteoarthritis Outcome Score, International Knee Documentation Committee (IKDC) score, American Knee Society score, Lysholm score, Cincinnati Knee Rating System, ACL–Quality of Life questionnaire, Tegner activity scale, and Marx activity scale.[3,23] Most of these scales cover several domains, such as symptoms, function, activities of daily life, sports activity, and physical examination. All of these measurements have been validated and are reliable, as described in previous studies.[23] Even 25 years after they were first introduced, the Lysholm and Tegner activity scales have proven to be acceptable for evaluating responsiveness after ACL treatment.[2]

In addition to pain relief and regaining of function, the ultimate anticipation for most patients after ACL injury or surgery is their return to preinjury activity levels and sports participation.[1] Thus, measuring the readiness and ability of patients to participate in sports after surgery is particularly important. Return-to-sport criteria[8,15,20,21,25] have been proposed so that athletes can safely return to their sport and can predict their future participation level.

During postoperative rehabilitation, several objective physical measurements—for example, strength and functional symmetry,[4] single-leg hopping for distance,[26] and complicated video analysis[10]—have been used to assess athletes. Most of the physical tests, however, were developed to use on professional athletes rather than their amateur counterparts, who are also eager to return to their sport. Because nearly half of those with an ACL injury who have returned to sport after ACL reconstruction can return to sports activity at a competitive level,[1] emphasis should be focused not only on the timing of return to sport, but also on measuring the ability to participate in high-level sports activity.

Our study aimed to develop and evaluate a self-administered rating scale for rapid evaluation of high-level sports ability among community-level athletes with an ACL tear and those who had already returned to sports participation after ACL reconstruction surgery. We hypothesized that the new rating scale would present acceptable psychometric parameters as a supplement for other clinical measures.

Methods

This study was approved by an institutional review board, and all patients gave informed consent to participate.

Data Collection

The study included patients ≥18 years of age with a unilateral ACL injury that occurred during participation in a competitive sport and who had undergone ACL reconstruction surgery performed by a senior surgeon (J.Z.) during 2017-2018. The exclusion criteria were the presence of multiligament injury, a history of prior surgery, or injury or workers’ compensation. In all, 28 patients did not return to their sport and 8 patients failed to attempt to perform cutting and pivoting motion. Thus, 77 patients were available for the final analysis (Figure 1). The descriptive characteristics of the study patients are reported in Table 1.

Figure 1.

Flowchart for patient enrollment. ACLR, anterior cruciate ligament reconstruction.

Table 1

Patient Descriptive Characteristics (N = 77)

Characteristic	Value
Sex, male/female, n	64/13
Age at surgery, y, mean ± SD	30.4 ± 7.0
Body mass index, mean ± SD	24.5 ± 3.8
Follow-up time, mo, mean ± SD	18.5 ± 4.6

Preoperatively and postoperatively, all participants completed the Knee Stability in Sports/Cutting-Pivoting Ability (KSS/CPA) scale. They also completed the Marx activity scale, IKDC Subjective Knee Form (sIKDC) (including symptom, function, and sport activities domains), and Tegner activity scale. Although the Marx and Tegner scales are ordinal, they were analyzed as continuous variables (Marx = 0-100; Tegner = 0-10).[2] In addition, patients provided an overall self-assessment evaluation of their knee function (0 = worst, 100 = best) preoperatively and postoperatively.

KSS/CPA Scale Description

The KSS/CPA scale (Appendix Figure A1) was established by a consensus of experts under the chairmanship of a senior surgeon of sports medicine (J.Z.). Validity was assessed by a group of independent experts in orthopaedics. The KSS section of the scale emphasizes the giving way sensation or instability in the knee during cutting-pivoting activities and linear activities (ie, running in a straight direction, including accelerating or decelerating), which comprise relatively lower level motions. The CPA section focuses on the ability and frequency of performing cutting-pivoting activities, as well as discomfort after performance. As it is clinically impossible for a patient who feels frequent knee instability during linear activities to perform cutting-pivoting activities without knee instability, we eliminated such occasions and simplified the options. The items are ultimately graded on a 5-point Likert scale, with response categories consisting of (A) normal, (B) near normal, (C) abnormal, (D) severely abnormal, and (E) dysfunctional (Appendix Table A1).

Figure A1.

The KSS/CPA Scale.

*Cutting and pivoting activities = changing directions or turning your body while your foot planted while playing a sport. Linear activities = running in a straight direction, including accelerating or decelerating.

Table A1

Grading for the KSS/CPA Scale

Knee Stability in Sports (KSS) Section: Giving Way Sensation/Instability From the Knee
During Cutting and Pivoting Activities		During Linear Activities		Gradeb
Never		Never		A (normal)
Occasionally		Never		B (nearly normal)
Frequently		Never		C (abnormal)
Frequently		Occasionally		D (severely abnormal)
Frequently		Frequently		E (dysfunctional)
Cutting-Pivoting Ability (CPA) Section: Cutting-Pivoting Activities
Ability to Complete	Frequency		Discomfort From the Knee After the Activity	Gradeb
Capable	Frequently		No	A (normal)
Capable	Frequently		Yes	B (nearly normal)
Capable	Occasionally		No	C (abnormal)
Capable	Occasionally		Yes	D (severely abnormal)
Incapable (have tried)	NA		Yes	E (dysfunctional)

NA, not applicable.

Grading: A = 100 points; B = 75 points; C = 50 points; D = 25 points; E = 0 points.

Scale Characteristics

Reliability

The measurements were completed via telephone contact initially at least 12 months after the operation and then twice within 4 weeks. The test-retest reliability was determined by the intraclass correlation coefficient (ICC). The internal consistency of the scale was evaluated using Cronbach alpha coefficient before and after the surgery. Values of >0.7 were considered acceptable and those >0.8 were considered good.[14]

Validity

Construct validity is defined as “compared with a gold standard.”[14] However, no gold standard test has been established before and after ACL reconstruction for community-level patients engaged in cutting-pivoting activities. Because there was no such test, we tested the KSS/CPA scale by comparing it with the Marx and Tegner activity scales, the sIKDC, and an overall self-assessment score. The Marx and Tegner activity scales are widely used to evaluate activity level in sports, ranging from recreational sports to competitive sports, involving cutting-pivoting motions.[23] The sIKDC is validated as a tool to determine an athlete’s readiness to return to advanced-level activities.[15]

The community-level athletes were tested preoperatively and postoperatively through use of the Spearman rank correlation coefficient. The relationship was considered strong if the value was >0.5, moderate if 0.35-0.5, and weak if <0.35.[14] Discriminant validity was tested using the Mann-Whitney test. For this test, the patients were placed in 2 groups according to the 5 possible grades (group AB and group CDE) for comparison before and after the surgery.

Responsiveness

Preoperative and postoperative response distributions were calculated. We also tested the floor and ceiling effects by documenting the percentage of patients who rated the lowest and the highest degrees, respectively. If <30% of patients had scored the limits of the scale, the floor or ceiling effect was considered acceptable.[2] With the ordinal scales (KSS/CPA scale) converted to continuous variables (A = 100, B = 75, C = 50, D = 25, E = 0), the effect size in the study was calculated asThe standardized response mean was calculated asModerate effects were defined as >0.50, and large effects were defined as >0.80.[2]

Minimal Detectable Change

Minimal detectable change (MDC) was established using the distribution-based method[13,16] with the conversion scale described above. MDC was defined as the minimum amount of change needed to identify a real health change resulting from the standard measurement error. With CIs of 90% (corresponding to a 10:1 likelihood) applied as a score of 1.64, we calculated the MDC as

where α is the reliability coefficient of the scale from test-retest studies or Cronbach alpha. In the current study, the ICC was chosen for MDC calculation. All analyses were performed using SPSS 24.0 software (IBM). All reported P values are 2-tailed with α = .05.

Results

The pre- and postoperative scores on the Marx activity scale, Tegner activity scale, sIKDC, and overall self-assessment for the 77 study participants are shown in Table 2.

Table 2

Preoperative and Postoperative Outcome Scores

	Preoperative	Postoperative
Marx activity scale	22.1 ± 28.5	48.5 ± 27.3
Tegner activity scale	1.9 ± 1.6	4.0 ± 1.8
sIKDC
Overall	67.2 ± 13.7	87.2 ± 8.8
Symptoms	24.7 ± 5.0	31.6 ± 4.3
Function	5.3 ± 2.6	8.4 ± 5.0
Sport activities	28.3 ± 6.8	36.7 ± 3.7
Overall self-assessment	42.4 ± 26.0	75.5 ± 12.8

Values are expressed as mean ± SD. sIKDC, International Knee Documentation Committee Subjective Knee Form.

Reliability

The ICCs for the KSS/CPA scale were 0.85/0.84 after the surgery. Cronbach alpha coefficients for internal consistency of the new rating scale were 0.73 preoperatively and 0.89 postoperatively.

Validity

The preoperative Spearman correlation coefficient between the CPA section and the overall self-assessment score was 0.52 (P < .001), whereas the preoperative Spearman correlation for the KSS section and the Marx activity scale was 0.57 (P < .001). However, other coefficients showed relatively lower correlations (Table 3). The postoperative Spearman correlation coefficients for the KSS/CPA scale were moderate to strong (>0.35) compared with other clinical parameters, indicating an acceptable correlation (Table 3).

Table 3

Pre- and Postoperative Spearman Rank Correlation Coefficients of the KSS/CPA Scale With Other Outcome Measures

			sIKDC
	Marx Score	Tegner Score	Overall	Symptoms	Function	Sport Activities	Overall Self-Assessment
Preoperative
KSS section	0.57b	0.36b	0.35b	0.32b	0.24c	0.22	0.37b
CPA section	0.36b	0.25c	0.35b	0.26c	0.33b	0.32b	0.52b
Postoperative
KSS section	0.46b	0.44b	0.49b	0.46b	0.38b	0.44b	0.48b
CPA section	0.66b	0.61b	0.57b	0.56b	0.37b	0.49b	0.58b

CPA, cutting-pivoting ability; KSS, knee stability in sports; sIKDC, International Knee Documentation Committee Subjective Knee Form.

Correlation is statistically significant (P < .01; 2-tailed).

Correlation is statistically significant (P < .05; 2-tailed).

The discriminant validity was excellent. Group AB had much higher scores postoperatively than group CDE for the Marx and Tegner activity scales, sIKDC, and overall self-assessment score (P < .05), whereas preoperatively the difference was smaller between groups AB and CDE (Table 4). Also, the patients established a graded response of measurements pre- and postoperatively on the KSS/CPA scale (Figure 2).

Table 4

Comparison of the KSS/CPA Grading Groups With Other Outcome Measures Preoperatively and Postoperatively

			sIKDC
	Marx Scale	Tegner Scale	Overall	Symptoms	Function	Sport Activities	Overall Self-Assessment
Preoperative
KSS section
Group AB	44.6 ± 33.2	2.3 ± 1.3	83.3 ± 11.2	29.6 ± 3.8	8.6 ± 1.8	34.6 ± 4.7	78.6 ± 9.4
Group CDE	19.8 ± 27.3	1.9 ± 1.6	65.6 ± 13.0	24.2 ± 4.9	4.9 ± 2.4	27.7 ± 6.7	39.4 ± 25.0
P	.016	.228	.002	.005	.001	.007	.001
CPA section
Group AB	61.3 ± 31.1	3.4 ± 2.3	76.7 ± 18.3	28.4 ± 6.3	6.4 ± 3.2	32.4 ± 6.6	70.6 ± 18.1
Group CDE	19.4 ± 26.5	1.8 ± 1.5	66.6 ± 13.3	24.4 ± 4.9	5.2 ± 2.5	28.0 ± 6.8	40.5 ± 25.4
P	.004	.057	.168	.149	.369	.188	.012
Postoperative
KSS section
Group AB	55.1 ± 24.5	4.3 ± 1.7	88.7 ± 8.4	32.2 ± 4.3	8.8 ± 5.5	37.1 ± 3.5	78.3 ± 10.9
Group CDE	23.4 ± 23.1	2.6 ± 1.4	81.6 ± 8.0	29.2 ± 3.0	7.1 ± 1.5	34.8 ± 4.1	64.7 ± 14.1
P	<.001	<.001	.005	.010	.013	.015	<.001
CPA section
Group AB	66.6 ± 17.5	5.1 ± 1.5	91.0 ± 7.7	33.3 ± 4.2	8.3 ± 1.3	37.9 ± 3.2	81.6 ± 9.8
Group CDE	30.9 ± 23.4	2.9 ± 1.3	83.6 ± 8.2	29.9 ± 3.7	8.6 ± 7.0	35.5 ± 3.8	69.5 ± 12.7
P	<.001	<.001	<.001	<.001	.027	<.001	<.001

Values are expressed as mean ± SD. CPA, cutting-pivoting ability; KSS, knee stability in sports; sIKDC, International Knee Documentation Committee Subjective Knee Form. Group AB, grade A plus grade B; Group CDE, grade C plus grade D plus grade E.

Figure 2.

Distribution of (A, B) preoperative and (C, D) postoperative outcome scores within the KSS/CPA grading distribution. For each KSS/CPA grade, the outcome scores are stacked in a single column. Each score presents a decreased tendency from KSS/CPA grades A to E. Error bars indicate SD. There are no error bars in grade A on the preoperative KSS section, grade A on the preoperative CPA section, or grade D on the postoperative KSS section, because only 1 patient distributed in each case. CPA, cutting-pivoting ability; KSS, knee stability in sports; Preop, preoperative; Postop, postoperative; sIKDC, International Knee Documentation Committee Subjective Knee Form.

Responsiveness

After ACL surgery, most responses were in the normal or near-normal range. The percentage distribution of responses for each grade of the KSS/CPA scale was collected (Table 5). Floor effects were found in the KSS and CPA sections preoperatively, at 50.65% and 59.74%, respectively, and we found no evidence of unacceptable ceiling effects.

Table 5

Distribution of Responses for Each Grade of the KSS/CPA Scale (N = 77 Participants)

Section and Grade	Preoperative	Postoperative
KSS section
A (normal)	1 (1.30)	15 (19.48)
B (nearly normal)	6 (7.79)	47 (61.04)
C (abnormal)	7 (9.09)	9 (11.69)
D (severely abnormal)	24 (31.17)	1 (1.30)
E (dysfunctional)	39 (50.65)	5 (6.49)
CPA section
A (normal)	1 (1.30)	12 (15.58)
B (nearly normal)	4 (5.19)	26 (33.77)
C (abnormal)	17 (22.08)	31 (40.26)
D (severely abnormal)	9 (11.69)	4 (5.19)
E (dysfunctional)	46 (59.74)	4 (5.19)

Data are reported as n (%) of participants. CPA, cutting-pivoting ability; KSS, knee stability in sports.

For the KSS/CPA scale, a large effect size was seen at 18 months after surgery, at 2.09/1.76. The standardized response means and MDCs were calculated for the KSS and CPA sections. The MDCs were <25 (ie, a difference of 2 adjacent degrees), indicating that our classified gradation was effective to detect change (Table 6).

Table 6

Responsiveness for KSS/CPA Scale

	Effect Size	Standardized Response Mean	Minimal Detectable Change
KSS section	2.09	1.63	22.48
CPA section	1.76	1.29	24.50

CPA, cutting-pivoting ability; KSS, knee stability in sports.

Discussion

This study analyzed a new rating scale (the KSS/CPA) for rapid evaluation of a patient’s ability to participate in high-level sports activities after surgery for an ACL injury. The new measurement focuses on cutting and pivoting activities for patients who sustain an ACL tear or have already undergone ACL reconstruction and have returned to sports. The developed scale was found to have acceptable psychometric parameters, including test-retest reliability, internal consistency, construct and discriminant validity, responsiveness, and MDC.

The KSS/CPA scale presented satisfactory ICCs and internal consistency for reliability. In terms of construct validity, the correlation between KSS/CPA and other outcome measurements was acceptable, especially postoperatively (>0.35). Additionally, the new rating scale demonstrated a graded response of measurements for discrimination. Although floor effects were detected in the KSS and CPA sections preoperatively, the effect size at 18 months after surgery was considered large, along with a reasonable MDC (<25).

The new scale—graded on knee instability (KSS section) and ability to participate in sports, frequency of participation, and discomfort after participating (CPA section)—distinguishes amateur players who are ready to return to high-level sports (eg, soccer, skiing[7]) from those who should not. Patients with an ACL injury or reconstruction graded A or B based on the KSS and CPA sections scored much higher than those with lower grades (C, D, and E). The good identification rate of the KSS/CPA scale showed that patients who felt less giving way sensation or performed better cutting-pivoting motion were more likely to be engaged in higher level sports (Tegner activity scale[2]) more frequently (Marx activity scale[5,18]) with less discomfort (sIKDC[9]). This finding correlates with clinical experience and published data. The Saltin-Grimby Physical Activity Level Scale[7,22] categorizes physical activities into 4 levels, with “higher-level, specific” activities implying that the patient was performing more cutting-pivoting motions more frequently.

We also found that the KSS/CPA scale presented moderate to strong correlation with these measurements, specifically during the postoperative follow-up period. Interestingly, patients with higher overall self-assessment scores were more likely to perform cutting-pivoting sports activities, which is indicative of the possible contextual factors, such as fear of reinjury and/or a lifestyle change, that affect patients after ACL treatment.[1]

The new scale exhibited excellent responsiveness with a large effect size. Furthermore, the MDC of both the KSS and CPA sections was <25, implying that the gradation of the scales is reasonable. However, floor effects were detected preoperatively, mainly because the ACL plays a crucial role in controlling cutting-pivoting motions[11] and an ACL injury would greatly interfere with these motions. Similar floor effects are found in other widely used scales, such as the 36-Item Short Form Health Survey.[17] Our unique measurement scale should take approximately 1 minute to complete. Consequently, it could supplement the use of other clinical measures for more comprehensive evaluation of cutting-pivoting ability and knee stability.

The distribution of responses for each grade showed that more than half of the patients were graded as dysfunctional after an ACL injury, whereas 80.52% of patients (graded A or B on the KSS section) were back to normal or nearly normal, and 49.35% (graded A or B on the CPA section) showed good performance in cutting-pivoting motions. That is, nearly half of the patients who had ACL reconstruction were capable of performing competitive sports, correlating with previously published data.[1] With the new KSS/CPA scale, those patients graded A or B could be classified as “safe to return” and should be encouraged to engage in higher level sports activities during rehabilitation. For those with lower grades, however, attention must be paid to augmenting the strength of muscles around the knee joint. Sports activities involving cutting-pivoting motions should be avoided unless thorough warm-up exercises are performed, including lateral slide, rush to stop within a short distance, and jogging or fast walking along a pathway, such as a Mobius ring, that goes in opposite directions with gradual acceleration and pathway narrowing.

Among community-level athletes who have undergone surgery for an ACL injury, the KSS/CPA scale is the first in the literature to allow self-assessment of the specific skills required for high-level sports activities. The Tegner activity scale is also designed to evaluate activity level, with scores of 1 to 5 representing recreational sports activity and >5 representing higher level or competitive sports activity.[23] However, the Tegner scale stratifies activities into 10 levels based on specific sports rather than skills and therefore poses difficulty for application among all populations participating in different sports.[2,23,24] Although the Marx activity scale places a focus on specific functional activities[5] in terms of running, decelerating, cutting, and pivoting,[5,18] frequency of activity is considered the sole outcome measure, which could be affected not only by function but also by the amount of time a person has available for recreational sports activity. Thus, our scale takes into consideration a patient’s ability to participate in sports activity, the frequency of activity, and subjective knee feelings in order to better assess capability and readiness to return to high-level sports. Furthermore, composed of simply 2 questions for gradation, the newly proposed scale presents minimal responder burden, compared with the sIKDC, although sIKDC scores were reported to be effective for assessment of failure in the Readiness to Return to Activity Criteria test battery.[15] Comparison with other reported scales indicates that for patients with ACL injury who are willing to return to high-level sports activities after surgery, our new rating scale is an effective alternative for quick assessment of high-level skill abilities.

Clinical, objective examinations of knee function have been validated as screening tools for high risk of reinjury after return to sport, whereas the duration of play and patients’ feelings might affect the risk of reinjury.[25] Therefore, we propose that this new scale, which considers ability, frequency, and subjective feelings, could add to the tools that a clinician uses during rehabilitation and in decision making about return to high-level sports. When results are documented as grade A or B on both parts of the KSS/CPA scale, a minor adjustment is required for rehabilitation, and focus could be turned to the quality of specific skill accomplishment; if any result is assessed as grade C, D, or E, competitive or high-level sports activities are not recommended and emphasis should be placed on enhancing knee function and muscle strength. Considering the current findings, we hope that the scale will provide guidance for rehabilitation protocols for amateur athletes to ensure a safe return to high-level sports activities.

Limitations of the study involve the test-retest reliability assessment. We performed the test only postoperatively, and preoperative validation is recommended for better application. Second, we used overall self-assessment scores, which require further validation and evaluation for its general use, to evaluate patients’ overall perceptions regarding the state of the knee for sport functioning. Third, the results of the MDC calculation require comparison with the minimal clinically important difference, because it has been reported that the anchor-based minimal clinically important difference shows more accurate estimates than the MDC throughout all cohorts.[16] Thus, an anchor-based (eg, degree of satisfaction) minimal clinically important difference is required for calculation in further study. Moreover, further investigation is required regarding correlation of the reinjury rates and the grouping of grades (AB vs CDE) based on our scale at the time of return to sport, and specific patients with underlying high risk of reinjury[6] should be included for subgroup analysis. Further prospective clinical studies should be carried out to evaluate the use of this new scale and explore the relation between this scale and the possibility for return to sport.

Conclusion

The new rating technique presented, the KSS/CPA scale, could be a supplement to other clinical measures for comprehensively evaluating cutting-pivoting ability and knee stability.