Cross-cultural adaptation, validity and reliability of the Korean version of the Kerlan-Jobe Orthopedic Clinic shoulder and elbow score.

BACKGROUND: The Kerlan-Jobe Orthopedic Clinic shoulder and elbow score (KJOC score) was developed for assessing performance and function in overhead athletes with shoulder and elbow injury and recently adapted as the standard score for overhead athletes in Major League Baseball. However, the Korean version of the KJOC score was not developed in the literature. The aim of the current study was to adapt the English version of the KJOC score to develop a Korean version (K-KJOC) and to evaluate its validity and reliability.
METHODS: A total of 52 professional baseball players in the Korean Baseball League completed the K-KJOC at two-week intervals during the off-season. The QuickDASH (11-point Disabilities of the Arm, Shoulder and Hand) score was also performed to evaluate the construct validity of the K-KJOC score. The internal consistency of reliability and test-retest reliability were assessed as well.
RESULTS: The K-KJOC score was correlated with the Quick DASH - disability/symptom (-0.309, ~-0.268, p < 0.05), Quick DASH - work (r = -0.721 to -0.671, p = 0.000) and QuickDASH - sports (r = -0.721 to -0.714, p = 0.000). The internal consistency of the K-KJOC score was excellent (Cronbach's α: 0.917-0.966), and the intra-class correlation coefficients of test-retest reliability for the 10 items for the K-KJOC score were fair to excellent (ICC 0.505-0.937, p < 0.05).
CONCLUSION: The K-KJOC score appeared to be a valid and reliable tool for assessing shoulder and elbow injuries in Korean overhead athletes.

Sports-specific instruments for evaluation of performance as well as functional status of overhead athletes are limited and the traditional evaluation for recovery from previous musculoskeletal injuries has been described with the term ‘return to play’.1, 8

The development of instruments to detect the subtle change of sports-related functions in athletes1, 8 might be more difficult than the development of functional outcome instruments in the average population due to the limited knowledge of sports and athletes. Therefore, the commonly used functional outcome instruments in specific diseases, such as the ASES (American Shoulder and Elbow Society) form, the Mayo elbow performance score, and DASH (Disabilities of Arm, Shoulder and Hand), have been used for the functional evaluation of overhead athletes up to date. The other well-known previous functional evaluation tools for overhead athletes were return-to-sport rate, the Conway–Jobe score and the Timmerman–Andrews score.3, 5, 18, 19, 24, 25 The Conway–Jobe score was used to assess outcomes in throwing athletes after ulnar collateral ligament (UCL) repair or reconstruction3, 5, 24 and the Timmerman–Andrews elbow score was developed to evaluate post-traumatic elbow pain and stiffness or outcome assessment after surgery in throwing athletes.5, 24, 25 However, those scoring systems were limited to use only in elbow injuries excluding shoulder injuries and have never been validated and tested for reliability.

Since the development of the Kerlan-Jobe Orthopedic Clinic shoulder and elbow (KJOC) score in 2010, several studies adopted this evaluation instrument and proved it to be valid, reliable, and responsive in shoulder and elbow injuries of various overhead athletes1, 3, 7, 8, 9, 12, 20, 21, 23, 26, 28 Furthermore, the Major League Baseball (MLB) organization adopted this evaluation tool as the standard for the recovery of injuries in MLB players recently. However, the KJOC score has never been translated into other languages before, although the international transfer among overhead athletes occurs in various sports and sports related shoulder and elbow injuries in overhead athletes are prevalent worldwide. We hypothesized that the linguistic and cultural adaptations of the KJOC score into the Korean language would be valid and reliable to administer to Korean overhead athletes.

The aim of this study is to develop the Korean version of the KJOC (K-KJOC) score through cross cultural adaptation and verify validity and reliability in Korean professional baseball league players.

Materials and methods

After IRB approval, consent was obtained from every subject involved in this study. The development of the (K- KJOC) score was allowed by the Kerlan-Jobe Orthopedic Clinic in 2015. The current study was conducted in two phases. The first phase comprised the linguistic and cultural adaptation of the English version of the KJOC score into Korean to develop the Korean version of the KJOC score. The second phase comprised administration of the K-KJOC score to Korean overhead athletes, who were professional baseball players, to verify construct validity and reliability.

Adaptation and development of the K-KJOC score

After the permission of the Kerlan-Jobe Orthopedic Clinic, forward translation into Korean was performed by two bilingual Korean–American translators whose first language was Korean. One translator was registered with a Korean professional baseball team (LG Twins) as an official translator with no medical background and the other was a medical researcher. Those two translations were reviewed and a synthesized edition was created. A couple of discrepancies were resolved by consensus. Subsequently, backward translation into English was performed by two independent translators, whose first language was English and who were completely unknown to this study. The backward translations were reviewed for their linguistic and cultural qualities by members of a committee consisting of health professionals, translators, and linguists. Discrepancies were resolved by consensus to achieve a conceptual equivalence with the original questionnaire. The pre-final version was developed and administered as a pilot study to 10 Korean professional baseball players. Further modifications were then made with the assistance of two athletic trainers. The final version was confirmed by the members of the committee.

Subjects

A total of 52 professional baseball players in the Korean baseball league (age, 24.1 [SD 3.8] years; career, 14.0 [SD 4.0] years; catchers 5, infielders 12, outfielders 11, pitchers 24) were enrolled in this study. Twenty-seven of the 52 (51.9%) had a history of shoulder or elbow injury (shoulder, 10; elbow, 17). Eighteen (34.6%) had a history of surgery and 16 of 18 had undergone elbow surgery (Table I). The athletes completed the K-KJOC questionnaire at two week intervals during off-season. At the first administration of the K-KJOC score, the QuickDASH (11-point Disabilities of the Arm, Shoulder and Hand) score was also administered for the verification of construct validity.

Table I

Summary of baseline characteristics, history of arm injury, and surgery

Age	24.1 (SD 3.8) years
Gender	All male
Career	14.0 (SD 4.0) years
Position	Catcher	5
	Infielder	12
	Outfielder	11
	Pitcher	24
Previous injury	Shoulder (n = 10)	Catcher 0Infielder 2Outfielder 3Pitcher 5
	Elbow (n = 17)	Catcher 1Infielder 3Outfielder 3Pitcher 10
Surgery	MCL reconstruction (n = 9)	Catcher 0Infielder 3Outfielder 1Pitcher 5
	Osteophyte removal (n = 2)	Catcher 0Infielder 1Outfielder 0Pitcher 1
	MCL reconstruction + Osteophyte removal (n = 5)	Catcher 1Infielder 0Outfielder 2Pitcher 2
	SLAP lesion repair (n = 2)	Catcher 0Infielder 1Outfielder 0Pitcher 1

MCL, medial collateral ligament; SLAP, superior labrum anterior and posterior.

The KJOC score was divided into five sections as follows: 1) questions related to the history of injuries to the arm, 2) description of the level of competition in the current sport, 3) description of the current status (three categories [or groups]; group A, playing without any arm trouble; group B, playing, but with arm trouble; group C, not playing due to arm trouble), 4) questions concerning physical functioning under game and practice conditions, and 5) specific questions referring to the level of competition in the sport. Divisions 4) and 5) were composed of 5 items each and each item was scored on a scale of 0 to 10 (10 is the untroubled state).

Assessment of validity and reliability

Construct validity

To measure construct validity, the correlation was assessed with Quick. QuickDASH – disability/symptom had 11 items to measure the physical function and symptoms of subjects with musculoskeletal disorders in the upper limb (score: 0–55, 0 is best and 55 is worst outcome). There were two optional modules in QuickDASH which were modules of athletes, performing artists, or sports/music module (QuickDASH – sports, 4 items) and workers with high degree of physical performance or work module (QuickDASH – work, 4 items).

Reliability

The internal consistency of reliability was assessed for each item of the K-KJOC score by Cronbach's alpha. To prove the test–retest reliability, the K-KJOC score was administered to the same subjects at two week intervals and analyzed by the intra-class correlation coefficient (ICC) for each item.

Sample size calculation

The sample size was determined based on the ICC calculation. During repeated administration, the acceptable reliability (p0) was set to 0.7 and the expected reliability (p1) was set to 0.85. Considering a significance level (α) of 0.05 with 80% power and 10% drop-out, a minimum of 47 samples were required.

Statistical analysis

All statistical analyses were performed with SPSS Statistics for Windows, Version 20.0 (IBM Corp., Armonk, NY, USA), and p values of less than 0.05 were considered statistically significant. For the verification of construct validity of the K-KJOC score, the Pearson correlation coefficient analysis was performed (r: −1.0 to −0.7, strong negative correlation; −0.7 to −0.3, moderate negative correlation; −0.3 to −0.1, weak negative correlation; 0.1–0.3, weak positive correlation; 0.3–0.7, moderate positive correlation; 0.7–1.0 strong positive correlation). Cronbach's alpha was measured to determine the internal consistency (α: >0.9, excellent; 0.8–0.9, good; 0.7–0.8, acceptable; 0.6–0.7, questionable; 0.5–0.6, poor; <0.5, unacceptable) and ICC (Intra-class Correlation Coefficient) with a two-way mixed-effects model was used to verify the test–retest reliability (ICC: >0.75, excellent strength of agreement; 0.75–0.4, agreement between fair and good; <0.4, agreement poor). For the comparison of scores among three different groups of injury, the ANOVA test with Bonferroni post-hoc analysis was performed.

Results

Adaptation

Most of the translations from English to Korean were performed without any problems. However, several linguistic and cultural adaptations were performed. As part of the description of the competition level of the current sport, the English version used the terms ‘major league’ and ‘minor league’. In contrast, there were no equivalent Korean terms in use in the Korean professional leagues. However, terms such as ‘first team (Il-goon)’, ‘second team or farm team (Yi-goon)’ and ‘business team (Sil-up)’ exist, leading us to make some of the linguistic changes on the level of the description of the competition level. The fifth question (item) in English was as follows: ‘How much have arm problems affected your relationship with your coaches, management, and agents?’. Agents are not prevalent in the Korean professional baseball league, so we deleted the word corresponding to ‘agent’ in the Korean version of the question. Furthermore, the answer ‘lost scholarship’ was likely to occur in the English cultural background, but there is no possibility of ‘lost scholarship’ in Korean culture due to the arm troubles of players, leading us to introduce the clause ‘not to be treated fairly due to arm trouble’, which corresponded to the intention of the English version. During the pilot study, every instrument was completed in around 10 minutes. No question was asked during the administration regarding the meaning of each item; however several players were confused about marking ‘x’ in the horizontal bar, which is why we wrote the numbers from 1 to 10 with the same interval below the horizontal bar to enhance the degree of understanding (Supplement 1).

Construct validity

Every item of the K-KJOC score (Table II) was completed without any non-reply and every item was scored from 7.4 (SD 2.7, item 6) to 8.5 (SD2.0, item 1).

Table II

Test-Retest reliability of the Korean version of Kerlan-Jobe Orthopedic Clinic shoulder and elbow (K-KJOC) score

Item number	First administration	Second administration	ICC	95% CI
Item 1	8.1 (SD 2.0)	8.5 (SD 2.0)	0.505	0.138–0.716
Item 2	8.0 (SD 2.3)	7.8 (SD 2.5)	0.691	0.461–0.822
Item 3	7.9 (SD 2.1)	7.9 (SD 2.3)	0.842	0.724–0.909
Item 4	7.6 (SD 2.5)	7.7 (SD 2.5)	0.848	0.736–0.913
Item 5	8.0 (SD 2.7)	8.4 (SD 2.5)	0.85	0.738–0.914
Item 6	7.4 (SD 2.7)	7.6 (SD 2.5)	0.853	0.743–0.915
Item 7	7.5 (SD 2.5)	7.8 (SD 2.4)	0.868	0.771–0.924
Item 8	8.0 (SD 2.5)	8.1 (SD 2.3)	0.782	0.620–0.875
Item 9	7.7 (SD 2.4)	8.0 (SD 2.5)	0.922	0.864–0.955
Item 10	7.5 (SD 2.4)	7.8 (SD 2.2)	0.937	0.890–0.969

ICC, intraclass correlation coefficient; CI, confidence interval; SD, standard deviation.

The lowest score indicated that the respondent was more likely to have trouble with the arm: item 6, which assessed the change of mechanism due to arm injury, revealed the lowest score, resulting in the determination that a change of mechanism during athletic performance was likely to occur. Item 1, which assessed the warm-up before the game and practice, showed the highest score indicating that the respondent was less likely to have trouble during warming up. According to the categories (or group) of the current status of the arm, athletes who were playing without any arm trouble (n = 29, group A) had a mean score of 89.2 to 91.2, athletes playing with arm troubles (n = 20, group B) had a mean score of 65.2 to 68.3 and athletes who could not play due to arm trouble (n = 3, group C) had a mean score of 42.0 to 49.7, with the differences of the K-KJOC score among these three groups being significant (p = 0.000, Fig. 1).

Figure 1

Comparison of mean scores of the Korean version of the Kerlan-Jobe Orthopedic Clinic shoulder and elbow (K-KJOC) score among three categories or groups, which were playing without any arm trouble (Group A, n = 29), playing with arm trouble (Group B, n = 20), and non-playing due to arm trouble (Group C, n = 3).

However, the post-hoc analysis with Bonferroni correction revealed a non-significant difference between groups B and C at first administration (p = 0.163).

Each of the 52 players was evaluated by the QuickDASH – disability/symptom, QuickDASH – work and QuickDASH – sports modules for the correlation analysis with K-KJOC score (Table III).

Table III

The construct validity of the Korean version of the Kerlan-Jobe Orthopedic Clinic shoulder and elbow (K-KJOC) score using Quick DASH – disability/symptom, Quick DASH – work and Quick DASH – sports modules

Quick DASHK-KJOC score	Quick DASH disability/symptom	Quick DASH work	Quick DASH sports
First administration	r = −0.290(p = 0.039)	r = −0.671(p = 0.000)	r = −0.714(p = 0.000)
Second administration	r = −0.309(p = 0.027)	r = −0.721(p = 0.000)	r = −0.721(p = 0.000)

The K-KJOC score was correlated with the QuickDASH – disability/symptom (r = −0.309, ~ −0.290, p < 0.05), QuickDASH – work (r = −0.721 to −0.671, p = 0.000) and QuickDASH – sports (r = −0.721 to −0.714, p = 0.000).

Reliability

The internal consistency of K-KJOC score (items 1–10) was excellent (Cronbach's α: 0.917–0.966) at each administration (Table IV).

Table IV

Internal consistency of the Korean version of the Kerlan-Jobe Orthopedic Clinic shoulder and elbow (K-KJOC) score using Cronbach's alpha

Internal consistencyK-KJOC score	Cronbach's alpha	95% CI
First administration	0.917	0.878–0.947
Second administration	0.955	0.934–0.971

For the test–retest reliability, the retest was performed with a two week interval. The K-KJOC score at first administration had a mean of 77.7 and the K-KJOC score of retest had a mean of 79.6 (Table V).

Table V

Average scores of the Korean version of the Kerlan-Jobe Orthopedic Clinic shoulder and elbow (K-KJOC) score and QuickDASH-disability/symptom, QuickDASH-sports and QuickDASH-work modules

	K-KJOC score (first)	K-KJOC score (second)	Quick DASH – disability/symptom	Quick DASH – sports	Quick DASH – work
Average scores	77.7 (range, 45–100)	79.6 (range, 39–100)	13.8 (range, 11–36)	5.8 (range, 4–15)	6.1 (range, 4–19)

The ICC (Intra-class Correlation Coefficient) of test–retest reliability for 10 items of K-KJOC score (Table II) was fair to excellent (ICC 0.505–0.937, p < 0.05).

Discussion

The current study reports several linguistic and cultural adaptations during the development of the Korean version of the Kerlan-Jobe Orthopedic Clinic overhead athlete shoulder and elbow score (K-KJOC score). The K-KJOC score revealed a weak to moderate negative correlation with the QuickDASH – symptom/disability module, and a moderate to strong negative correlation with the work as well as sports modules of QuickDASH. The internal consistency of the K-KJOC score was excellent and test–retest reliability was also good to excellent.

The ordinary disease-specific instruments for shoulder and elbow were known to be inappropriate during the functional evaluation of overhead athletes, because the overhead athlete who had severe functional impairment in their sports might reveal normal functional outcome in their daily living activities using those instruments. The DASH which could evaluate the general upper extremity function might also be inappropriate to use for overhead athletes due to the involvement of the hand and wrist functional evaluations, although a high correlation between the KJOC score and DASH was observed in a previous study. In fact, there have been no validated overhead athlete-specific scoring systems for the shoulder and elbow until the KJOC score was first introduced by Alberta et al. Several studies with overhead athletes were performed using KJOC score. Domb et al demonstrated that the KJOC score was the most sensitive tool for the detection of a subtle change in sports performance of athletes after ulnar collateral ligament reconstruction. Krauetler et al also reported that the KJOC scores in healthy asymptomatic pitchers are usually above 90 points with a mean of 94.8. The average KJOC scores of overhead athletes who had SLAP repair surgery measured 73.6 to 76.9 at 1 to 3.5 years of follow-up. The average KJOC score of baseball players who returned to previous levels after ulnar collateral ligament reconstruction surgery was known to be 77. According to the current study, baseball players who were playing without any arm trouble scored around 90, players with arm trouble had a mean score around 67, which seemed to be in line with previous studies.

With the perspective of linguistic adaptation, the word ‘major league’ was translated into the ‘first team’ and ‘minor league’ was translated into the ‘second team’. Furthermore, we had the ‘business team’, which was semi-professional. The baseball players who were in the business team were affiliated through their work place, which might be unfamiliar to the North American cultural background. The agent system in the Korean professional baseball league is not prevalent; therefore, the question with ‘whether the problems on shoulder and elbow could affect the relationship with agent or not’ would be inappropriate in the Korean cultural background. The scholarship of athletes seemed to never be lost due to the physical problems of athletes in the Korean cultural background. Therefore, the term ‘lost the scholarship’ was deleted; however, the clause ‘not to be treated fairly due to arm trouble’ was introduced to clarify the meaning. Several players had some difficulty to mark the x on the horizontal line during the pilot administration; thus, we wrote down the numbers from 1 to 10 with equal interval below the horizontal line. During the second administration of the newly developed K-KJOC score, no detailed explanation was needed to clarify the meaning of each item. However, there was the possibility of over- or underestimation on the actual scale because the respondents might have responded using integers without decimals when they marked on the horizontal line.

In the current study, the correlation analysis using QuickDASH was calculated for the verification of construct validity of the K-KJOC score. According to previous literature regarding the development of the KJOC score, the validity of the KJOC score was performed by the DASH. However, we adopted the QuickDASH, because the QuickDASH was advantageous with only 11 items and this might be crucial to administer to the athletes who were reluctant to spend time on completion of questionnaires in comparison with the DASH of 30 items. Furthermore, Radwan et al proved the correlation (r = −0.825, one tailed) between the KJOC score and QuickDASH – sport module in overhead athletes. According to the current study, there was weak to moderate negative correlation between the K-KJOC score and QuickDASH – disability/symptom module and this finding was a little bit different from a previous report with high correlation with DASH. We reasoned that this might be due to the difference of a minimal clinically important difference (MCID) between QuickDASH and DASH. We also observed the moderate to strong negative correlations between the K-KJOC score and work and sport modules of QuickDASH, which was in concordance with the study of Radwan et al. The reliability of the K-KJOC score in the current study was good enough as well. The internal consistency of the K-KJOC score was excellent (Cronbach's alpha 0.917–0.966) and the test–retest reliability of each item was fair to excellent. Although item 1 revealed the lowest ICC of 0.505, it is still in the range of fair agreement. Item 1 is about warm-up and conditioning, and this could be relatively less affected by injury status and more by factors such as daily physical condition, and this could have been the reason for the fair rather than excellent agreement.

There were several limitations. First, we did not study the responsiveness of the K-KJOC score in a certain disease of a sport-specific cohort, so further study of test responsiveness is needed, because the responsiveness reflects the effectiveness of a certain treatment after a certain period of treatment. Second, it would be more appropriate to add other values, such as the isokinetic strength test and maximal grip strength test to QuickDASH during evaluation of the construct validity. Third, there have been no cultural and linguistic adapted versions of the KJOC score before, and the comparison of this study to other similar studies was limited.

Although there are quite a number of professional or semi-professional (business) teams in various sports globally, the absence of validated instruments for functional evaluation in overhead athletes might impede the communication among doctors and researchers in sports medicine fields of non-English language countries. Therefore, the authors expect that this effort to develop the Korean version of the K-KJOC score might enhance communications among doctors and researchers and improve clinical research in sports medicine fields in non-English language countries, including Korea. Furthermore, the cultural and linguistic modifications to functional scores would be more comprehensible for overhead athletes from abroad, and this work might enable physicians to compare the different environments among countries as well as achieve the true globalization of sports medicine. This study is also in line with many published studies in the international journals regarding cultural and linguistic adaptations of functional scores, and this might support the basic concept as well as the importance of our study.2, 11, 13, 15, 16, 17

Conclusion

Cross-cultural and linguistic adaptations were undertaken to construct a valid and reliable Korean version of the K-KJOC. Our study reports K-KJOC scores of fair to excellent when applied to Korean athletes. We hope that the newly developed K-KJOC score will be useful for the assessment of overhead athletes whose first language is Korean.