Cross-cultural adaptation of Kerlan-Jobe Orthopaedic Clinic shoulder and elbow score: Reliability and validity in Turkish-speaking overhead athletes.

OBJECTIVE: Kerlan-Jobe Orthopaedic Clinic Shoulder and Elbow Score (KJOC-SES) is a subjective assessment tool to measure functional status of the upper extremities in overhead athletes. The aim was to translate and culturally adapt the KJOC-SES and to evaluate the psychometric properties of the Turkish version (KJOC-SES-Tr) in overhead athletes.
METHODS: The forward and back-translation method was followed. One hundred and twenty-three overhead athletes completed the KJOC-SES-Tr, the Disabilities of the Arm, Shoulder, and Hand (DASH), and the American Shoulder and Elbow Surgeons Evaluation Form (ASES). Participants were assigned to one of the following subgroups: asymptomatic (playing without pain) or symptomatic (playing with pain, or not playing due to pain). Internal consistency, reliability, construct validity, discriminant validity, and content validity of the KJOC-SES-Tr were tested.
RESULTS: The test-retest reliability of the KJOC-SES-Tr was excellent with an interclass coefficient of 0.93. There was a strong correlation between the KJOC-SES-Tr and the DASH and the ASES, indicating that the construct validity was good for all participants. Results of the KJOC-SES-Tr significantly differed between different subgroups and categories of athletes. The floor and ceiling effects were acceptable for symptomatic athletes.
CONCLUSION: The KJOC-SES-Tr was shown to be valid, reliable tool to monitor the return to sports following injuries in athletes.

Introduction

Overhead athletes are considered a specific group within the population of athletes, who have high demands on the upper extremities due to repetitive shoulder elevation and external rotation. A high incidence of sports-related shoulder and elbow injuries are documented in this group of athletes.1, 2 Therefore, in order to screen accurately sportive performance and functional status, to monitor treatment and rehabilitation effectiveness, and to evaluate return to the respective sport after injuries in overhead athletes, the self-reported outcome score should be adapted to the athletic demands. Clinically, the athletes often do not have restrictions in activities of daily life; however, they often notice the symptoms during high levels of athletic activity such as training or competitions.

Recently, the Kerlan-Jobe Orthopaedic Clinic Shoulder and Elbow Score (KJOC-SES) has been designed specifically as a subjective assessment tool to measure functional status of the upper extremities in overhead athletes. The KJOC-SES has been developed and validated for English-speaking overhead athletes for the assessment of performance and function, and has been used to document treatment efficacy of various disorders in baseball players. Recently, the Italian and Korean version of the score has also validated.4, 5 Research to date has investigated treatment outcomes in labral lesions, shoulder instabilities, and ulnar collateral ligament reconstruction in both professional and recreational athletes using the KJOC-SES.6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 Although the KJOC-SES has been widely used in English-speaking populations, the original language of the questionnaire inherently makes it difficult to apply to the non-English populations. Subjective scoring systems, such as questionnaires, may be used in populations with languages other than the ones in which they were developed. Several health-related tools are currently available for the Turkish-speaking population; some are disease specific and others, as in the case of musculoskeletal issues, are joint specific. Concerning shoulder and elbow issues, the Disabilities of the Arm, Shoulder and Hand (DASH) scoring system and the American Shoulder and Elbow Surgeons Evaluation Form (ASES) have previously been translated into Turkish and validated.17, 18 However, these scales are only used for the general population and do not incorporate the specific needs of athletes.

There is a high demand for internationally adaptable and accessible self-reported scoring systems. Therefore, the primary aim of this study was to translate the KJOC-SES from English to Turkish to enable use of the questionnaires in Turkish overhead athletes. The secondary aim of this study was to examine the reliability and validity of the Turkish version of KJOC-SES (KJOC-SES-Tr).

Materials and methods

Participants

A total of 123 overhead athletes participated in this study. The eligibility criteria were (1) 18 years or older; (2) currently active or not playing sports due to arm trouble; (3) ability to read and write in Turkish. Each participating athlete was asked to check the category that best described their current self-reported functional status according to three subgroups: (1) playing without any arm trouble, (2) playing, but with arm trouble, or (3) not playing due to arm trouble.

All participants provided written informed consent. The study was approved by the Ethics Committee (GO 16/220-14).

Translation and cross-cultural adaptation

Translation and cross-cultural adaptation of the KJOC-SES was performed in five stages according to guidelines proposed by Beaton et al. In the first stage, two Turkish individuals; an informed and an uninformed translator, with a good command of the English language were responsible for the literal and conceptual translation of the KJOC-SES. The informed translator was a physical therapist, and the uninformed translator was a retired athlete. Both translators spoke Turkish as their mother tongue and English fluently. In the second stage, the translations were compared and reviewed by a bilingual individual who highlighted any conceptual errors or inconsistencies in the translations and established the first Turkish translation in a reconciliation meeting. In the third stage, two native English speakers with a good command of Turkish, who were unaware of the purpose of the study and had no access to the original English version, separately translated the finalized Turkish translation back to English. In the fourth stage, the back-translated version of the KJOC-SES-Tr was compared to the original English version of the KJOC-SES by a committee comprised of a methodologist, a language professional, and the four translators. The committee evaluated the four translations and finalized the KJOC-SES-Tr. In the final stage, preliminary testing was performed to determine comprehension of the Turkish version. Preliminary testing of the final version of the KJOC-SES-Tr was conducted in 12 overhead athletes in order to determine if the athletes had any difficulties understanding the questions.

The final version of the KJOC-SES-Tr was accepted and administered to a larger population to assess its validity and reliability. To investigate test–retest reliability, the athletes were asked to complete the KJOC-SES-Tr again 7 days after the first completion. To ensure that the repeated answers regarding upper extremity function remained stable, the participants were also asked, “Has your status changed since filling out the initial questionnaire?” during the second round. Only participants with no change in upper extremity functions during the time interval were included in the analysis.

In this study, construct validity of the KJOC-SES-Tr was tested by calculating the correlation between KJOC-SES-Tr scores and DASH and ASES scores. All questionnaires were administered to the athletes in random order. Additionally, structural validity, content validity, and discriminate validity were evaluated.

Instruments

The original KJOC-SES is a 10-item questionnaire, developed as a self-assessed patient-reported outcome measure. The KJOC-SES questionnaire includes questions on shoulder and elbow function during sportive performance, impairment, activity limitations, and participation restrictions. The KJOC-SES questionnaire contains a demographics cover sheet and the main questionnaire, including questions evaluated using a visual analogue scale (VAS) from 0 to 100 mm. The total score is calculated as an average of the total scores of the 10 questions. Higher scores are indicative of higher functional response.

Reference questionnaires had been previously translated to Turkish. The validated DASH scoring system17, 20 and ASES Form were used. The DASH score was used to measure disability status of the upper extremities and is scored in two components: the disability/symptom section and the high performance sports module. Results were presented in the form of two total scores ranging from 0 to 100; higher scores indicate higher levels of disability. The ASES evaluates subjective shoulder pain and function/disability. The total ASES score is derived from a pain question using the VAS in addition to a 10-item function score. The ASES has two subscores; the pain score and the cumulative activities of daily living score that are weighted equally and combined for a total score. A score of 100 indicates good shoulder function.

Statistical analyses

A sample size estimation of 10 participants per item in the KJOC-SES-Tr was performed in accordance with previous suggestions. Descriptive statistics are reported as mean, standard deviation, median, interquartile change, counts, and percentages. Statistical significance was set to p < 0.05. All analyses were performed using SPSS Statistics, version 21 software (IBM Corporation, Armonk, NY).

Cronbach's-α coefficients were calculated for internal consistency. A Cronbach's-α below 0.6 indicates poor internal consistency, values from 0.6 to 0.7 indicate reasonable consistency, and values from 0.7 to 0.95 are considered adequate consistency.

To determine test–retest reliability of the KJOC-SES-Tr, we calculated the intra-class correlation coefficients (ICCs) with corresponding 95% confidence intervals (CI) between scores of the first and second administration of the KJOC-SES-Tr. An ICC value of 0.4 or greater was considered satisfactory, and a correlation less than 0.2 was described as poor. A correlation greater than 0.6 was considered very good, and greater than 0.8 was considered excellent. Furthermore, the 95% CI for the ICC and the standard-error-of-measurement (SEM) were calculated. Larger SEM values indicate reduced precision of the KJOC-SES-Tr questionnaire. The minimal detectable change (MDC) was determined using the formula 1.96 × √2 × SEM.

Construct validity was evaluated by examining the correlation of KJOC-SES-Tr scores with DASH and ASES scores. Correlation analysis was done using Pearson's correlation coefficient (r), and the validity of the KJOC-SES-Tr was defined as ‘strong’ if r > 0.5, ‘medium’ if 0.5 < r < 0.3, and ‘small’ if r < 0.3. The structural validity of the KJOC-SES-Tr was tested by exploratory factor analysis using principal component analyses with varimax rotation. Content validity was assessed by analysing the score distribution and the occurrence of ceiling and floor effects. The proportion of participants who obtained the lowest or highest score for each item in the KJOC-SES-Tr was documented. Floor and ceiling effects were considered to be present if more than 15% of respondents achieved the lowest or highest possible score for each item. If ≥ 75% of the items did not show floor or ceiling effects, the questionnaire as a whole was considered to have no floor or ceiling effects. Student's t test, Kruskal–Wallis test, and pairwise Mann–Whitney U test were used to assess discriminant validity.

Results

Cross-cultural adaptation

The Turkish translation of the KJOC-SES and subsequent English back-translation did not lead to any major linguistic problems. The expert committee agreed with the translation and back-translation. Preliminary testing did not reveal any difficulties understanding the KJOC-SES-Tr (available as Appendix).

Study participants

A total of 136 athletes were assessed and 125 of these were eligible to be included in this study. Nine athletes were not familiar with the Turkish language, and two athletes declined to participate and were therefore excluded from the study. Two additional participants did not fill out the second questionnaire. Finally, 123 participants were included in this study (Table 1).

Table 1

Demographics of the cohorts.

Characteristics	Value
	All participants	Asymptomatic Athletes	Symptomatic Athletes
			All Symptomatic Athletes	Athletes competing with arm trouble	Athletes not competing due to arm trouble
Number of athletes (n)	123 (100%)	73 (59.3%)	50 (40.6%)	32 (26.01%)	18 (14.6%)
Sex (Female/Male; n)	38/85	28/45	10/40	7/25	3/15
Age (years)	23.1 ± 5.06	22.7 ± 5.04	23.7 ± 5.09	23.4 ± 4.6	24.1 ± 5.9
Years competing (years)	11.5 ± 5.5	11.7 ± 5.8	11.3 ± 5.1	11.4 ± 4.9	11.1 ± 5.4
Sports (n)
Volleyball	39 (31.7%)	27 (36.9%)	12 (24%)	17 (53.1%)	5 (27.7%)
Handball	20 (16.2%)	9 (12.3%)	11 (22%)	8 (25%)	3 (16.6%)
Tennis	17 (13.8%)	7 (9.5%)	10 (20%)	8 (25%)	2 (11.1%)
Basketball	24 (19.5%)	14 (19.1%)	10 (20%)	7 (21.8)	3 (16.6%)
Water Polo	11 (8.9%)	8 (10.9%)	2 (4%)	–	2 (11.1%)
Swimming	12 (9.7%)	8 (10.9%)	5 (10%)	2 (6.2%)	3 (16.6%)
Injury (n)
Rotator Cuff Tendinopathy	6 (33.3%)	N/A	6 (33.3%)	N/A	6 (33.3%)
Labral Lesion	2 (11.1%)		2 (11.1%)		2 (11.1%)
Instability	4 (22.2%)		4 (22.2%)		4 (22.2%)
Fracture	1 (5.5%)		1 (5.5%)		1 (5.5%)
Tennis Elbow	5 (27.7%)		5 (27.7%)		5 (27.7%)

Note. Data are given as mean and standard deviation (for age and years competing), or as counts and percentages (other parameters).

Internal consistency

The internal consistency of KJOC-SES-Tr, evaluated on the basis of the strength of the correlation among the 10 items, was found to be ‘‘excellent’’ with a Cronbach's-α of 0.94.

Reliability

The test–retest assessment indicated excellent reliability, with an ICC of 0.93 (95% CI, 0.90–0.95) for all participants, 0.81 (0.71–0.88) for asymptomatic athletes, and 0.91 (0.85–0.95) for symptomatic athletes.

The SEM was 1.98 points for all participants, 0.91 points for asymptomatic athletes, and 3.16 points for symptomatic athletes for the first completion of the KJOC-SES-Tr. The MDC was calculated as 5.49 points for all participants, 2.54 points for asymptomatic athletes, and 8.76 points for symptomatic athletes.

Construct validity

Principal component analysis showed one underlying factor of the KJOC-SES-Tr with an explained variance of 71.2% and an eigenvalue of 7.1. Furthermore, the KJOC-SES-Tr had a significant negative correlation with the DASH score and a significant positive correlation with the ASES score (Table 2).

Table 2

Correlation between KJOC-SES-Tr scores and other outcome measures.

		KJOC-SES-Tr	DASH	DASH Sports Module	ASES Total	ASES Pain	ASES Function
All Participants	Mean ± SD	80.6 ± 21.9	8.1 ± 14.8	16.27 ± 25.3	83.5 ± 21.01	40.23 ± 13.5	43.51 ± 9.3
N = 123	r	–	−0.645	−0.843	0.831	0.773	0.758
	p	–	<0.001	<0.001	<0.001	<0.001	<0.001
Asymptomatic Athletes	Mean ± SD	93.6 ± 7.8	1.69 ± 5.5	1.8 ± 5.6	97.12 ± 5.8	48.83 ± 3.4	48.23 ± 4.9
N = 73	r	–	−0.247	−0.450	0.314	0.274	0.204
	p	–	0.03	<0.001	0.007	0.01	0.08
Symptomatic Athletes	Mean ± SD	61.7 ± 22.3	17.35 ± 18.8	37.3 ± 28.02	63.6 ± 19.2	27.6 ± 12.9	36.6 ± 10.03
N = 50	r	–	−0.490	−0.726	0.673	0.532	0.682
	p	–	<0.001	<0.001	<0.001	<0.001	<0.001

Note. P-values indicate the results of Pearson's correlation analysis.

Discriminant validity

The KJOC-SES-Tr results were significantly different between asymptomatic participants and symptomatic participants (mean difference: 31.8 points [26.2–37.4]). When the three subgroups of athletes were compared (playing without any arm trouble; playing, but with arm trouble; and not playing due to arm trouble) KJOC-SES-Tr results also significantly differed between subgroups (Table 3).

Table 3

Results of subgroup analysis of Kerlan Jobe Orthopaedic Clinic Shoulder and Elbow Score Turkish version.

Characteristics	Value					P
	All participants	Asymptomatic Athletes	Symptomatic Athletes
			All Symptomatic Athletes	Athletes competing with arm trouble	Athletes not competing due to arm trouble
Number of athletes	123 (100%)	73 (59.3%)	50 (40.6%)	32 (26.01%)	18 (14.6%)
KJOC-SES-Tr (pts.)
-Mean ± SD	80.6 ± 21.9	93.6 ± 7.8	61.7 ± 22.3	73.9 ± 14.4	40.07 ± 16.8	<0.001*
-Median [IQR]	90 [27.2]	96.7 [8.9]	64.2 [29.6]	76.8 [17.8]	46.9 [27.2]	<0.001†
						<0.001‡

Note. *P-values indicate the results of the Student's t test results. †P-values indicate the results of the Kruskal–Wallis test results. ‡P-values indicate the results of the pairwise Mann–Whitney U tests. KJOC-SES-Tr, Kerlan Jobe Orthopaedic Clinic Shoulder and Elbow Score Turkish version; pts., points; SD, standard deviation; IQR, interquartile range.

Floor and ceiling effects

The overall floor and ceiling effects for each question were acceptable for symptomatic athletes; the floor effect, corresponding to the percentage of symptomatic athletes with a score of 0 for each item varied between 2% and 13%; and the ceiling effect, corresponding to the percentage of symptomatic athletes with a score of 100 for each item varied between 2% and 14%. However, a ceiling effect was observed when using the KJOC-SES-Tr for asymptomatic athletes varying between 15.1% and 75.3%. The floor effect for the asymptomatic athletes varied between 1.4% and 2.7%.

Discussion

The findings of this study revealed that the KJOC-SES-Tr was an internally consistent, valid, and reliable questionnaire for Turkish-speaking overhead athletes.

In the present study, we observed excellent correlation among the 10 questions in the KJOC-SES-Tr. A high Cronbach's-α indicates good internal consistency. However, a Cronbach's-α higher than 0.9 could indicate redundancy. To investigate the presence of redundancy, Cronbach's-α was calculated for the questionnaire, leaving out one item for each calculation. Elimination of items resulted in lower internal consistency; therefore, no items were excluded. The KJOC-SES-Tr showed excellent test–retest reliability between repeated measures, and the correlation coefficient was higher than the score for the original KJOC-SES (ICC: 0.93 vs. 0.88 respectively). However, the ICC score was lower than Italian version of the KJOC-SES. Despite the strong correlation with the DASH, the KJOC-SES-Tr showed a relatively lower correlation value than the original version.3, 7 Alberta et al reported that KJOC-SES had a correlation coefficient of 0.84 with the DASH and 0.86 with the DASH sports module. The correlation coefficients calculated for the KJOC-SES-Tr were 0.64 and 0.84, respectively. In the current study, the KJOC-SES-Tr strongly correlated with the ASES pain and function scores, which was consistent with previous findings of Neri et al. These findings were expected since these scoring systems had been previously validated for upper extremity-related functional status/disability outcome measures and demonstrated good psychometric properties.17, 18, 20, 21, 22 While the DASH contains the sports module score, KJOC-SES has previously been recommended for use in athletes, because KJOC-SES outperformed the DASH in test–retest reliability. Moreover, Domb et al previously showed that the KJOC-SES was superior to the DASH sports module in identifying performance-related changes such as athletes' perception of current status performance, endurance, throwing mechanics, and speed.

Our findings indicated that a difference of less than nine points in the test–retest administration of KJOC-SES-Tr may represent an error of testing. On the other hand, previously, Alberta et al administered the KJOC-SES to injured athletes to monitor treatment responsiveness in a 63- to 564-day follow-up and reported that injured athletes, who improved their functional status category to “playing without any arm trouble,” significantly improved their KJOC-SES by a median of 28.2 points, which can be interpreted as a clinically meaningful change.

The KJOC-SES-Tr accurately stratified the study participants by existence of symptoms and by self-reported functional status category. In our study, 59.3% of the athletes were asymptomatic, 26.1% of the athletes were symptomatic but currently playing, and 14.6% of the athletes were symptomatic but currently not playing. The subanalysis of the reliability and validity showed excellent results. Therefore, the Turkish version of the KJOC-SES can be used as a self-reported outcome measure for screening of overhead athletes, monitoring treatment effectiveness, and evaluation of return to the sport. Consistent with findings of Kraeutler et al, it can be concluded that athletes free from arm trouble should have a score of 90-points or more. Health providers should be aware that athletes with a KJOC-SES-Tr result below 90-points may be of potential concern.

There are some limitations of the present study. Since the KJOC-SES was not specific to any particular sports, and the study population was recruited from different sports, the findings of the current study might only present the reliability and validity of the combined group and not be representative for individual sports types. However, there is no evidence in the literature that athletes participating in different sports exhibit different psychological responses in terms of expressing subjective functional status.

Our findings show that the KJOC-SES-Tr is an excellent outcome measurement for shoulder and elbow-related research for overhead athletes. For clinical practise, the KJOC-SES-Tr may help by providing a reliable tool to identify functional impairments in overhead athletes. This score is likely to be very useful in screening athletes for potential functional impairment in the upper extremity.

Conflicts of interest

The authors report no conflicts of interest.