Norwegian translation, cross-cultural adaptation and validation of the Kerlan-Jobe Orthopaedic Clinic shoulder and elbow questionnaire.

The Kerlan-Jobe Orthopaedic Clinic (KJOC) shoulder and elbow questionnaire, with 10 items and a total score ranging from zero to 100, provides more clinically relevant information about overhead athletes than other shoulder or upper limb patient-reported outcomes.
OBJECTIVES: To translate, cultural adapt and evaluate the measurement properties of the Norwegian version of KJOC shoulder and elbow questionnaire.
METHODS: 33 overhead athletes (age 18.6±4.2, 10 men/23 women) were included in the analysis of face validity and known-group validity, of whom 15 went through cognitive interviews. An electronic version was developed, and six handball players were interviewed to ensure measurement equivalence between the paper-based and electronic version of the questionnaire. Test-retest reproducibility (1-week interval) and concurrent validity with the Disabilities of the Arm, Shoulder and Hand (DASH) questionnaire was investigated in 36 handball players (age 20.7±3.8, 17 men/19 women).
RESULTS: The translation was conducted, and smaller consensus-based adjustments were made. Athletes found the questionnaire easy to understand, with no differences between paper and electronic based version, and preferred the electronic version. The Norwegian electronic version of the KJOC showed excellent internal consistency (Cronbach's α=0.952) and relative test-retest reliability (Intraclass correlation coefficient, ICC=0.967). SEM, minimal detectable change (MDC) and limits of agreement were 3.1, 8.5 and -9.2 to 7.7, respectively. The concurrent validity versus DASH was moderate (Spearman's rho=-0.642). However, KJOC had a wider range in scores than DASH, distinguished better between players playing with and without pain and was more sensitive to capture players playing with pain.
CONCLUSION: This study suggests that the Norwegian version of the KJOC is a reliable and acceptable tool for evaluating shoulder and elbow-related problems in overhead athletes (handball players). © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Initially, a face validation of the Norwegian Kerlan-Jobe Orthopaedic Clinic (KJOC-N) among users was conducted.

A rigorous translation and cultural adaptation process ensure that the questionnaire can be used in multinational research projects.

The use of electronic questionnaires makes easy to collect repeated measures.

Responsiveness of the KJOC-N was not explored.

Mainly handball players were included.

Introduction

Overhead athletes have high prevalence of shoulder and elbow injuries.1–4 Many of these athletes experience no symptoms during activities of daily living, except during training or competition. Furthermore, they often continue to participate in training and competition despite injuries.5 Therefore, commonly used patient-reported outcome measures developed for the normal population, such as DASH questionnaire and American shoulder and elbow surgeons society standardised shoulder assessment form fail to capture overhead athletes’ functional status and changes in performance.6

To address this problem, Alberta et al developed the Kerlan-Jobe Orthopaedic Clinic (KJOC) shoulder and elbow questionnaire; a 10-item questionnaire that uses visual analogue scales (VAS) scales to evaluate the athletes' performance, function and pain.5 The KJOC comprises more questions about throwing-related function and performance than other upper limb questionnaires. It is a valid, reliable and responsive tool in the evaluation of overhead athletes,5 and is more accurate in evaluating outcome of upper limb surgery in overhead athletes than previously used questionnaires.7–9 Additionally, KJOC discriminates between injured and uninjured athletes, and between those competing with and without pain.5 6 10 The KJOC was developed and validated for English-speaking overhead athletes, and has recently been translated to other languages.10–12

According to Beaton,13 if a questionnaire is used across languages or cultures, a rigorous translation and cultural adaptation process are important to maintain the content validity of the instrument at a conceptual level. This will also ensure the questionnaire can be used in multinational research projects.

The aims of the study were to translate, culturally adapt and validate the KJOC for a Norwegian context, to make an electronic version of the Norwegian KJOC (KJOC-N) and to evaluate the measurement properties of KJOC-N.

Materials and methods

We conducted this study in two phases (figure 1). The first phase comprised translation and cultural adaption of the KJOC questionnaire into Norwegian and evaluation of face validity and known-group validity. In the second phase, we adapted the KJOC-N from paper to electronic version and evaluated concurrent validity and test–retest reliability of the electronic version.

Figure 1

Flow chart of the two parts of the study. Part one: translation and cultural adaption of KJOC shoulder and elbow questionnaire into Norwegian. Part two: evaluation of test–retest reliability and construct validity of the KJOC-N versus DASH questionnaire. DASH, Disability of the Arm, Shoulder and Hand; KJOC, Kerlan-Jobe orthopaedic clinic; KJOC-N, Norwegian KJOC.

Questionnaires

The KJOC questionnaire is a self-assessed patient-reported outcome measure to evaluate the shoulder and elbow function, performance and pain in overhead athletes.5 It consists of 10 items, all of which use VAS from zero to 100. The total score is calculated as an average score of the 10 items, ranging from zero to 100. Higher scores indicate higher function.

To assess the concurrent validity of the KJOC-N, we compared it to the Norwegian DASH total and DASH sport/performing arts module.14 DASH is a 30-item self-administered measure of symptoms and functional status.15 Each item has five categorical options, ranging from ‘no difficulty or symptoms’ to ‘unable to perform activity’ or ‘very severe symptoms’. A sum score ranging from zero to 100 is calculated. Higher scores indicate lower function. The DASH sport/performing arts module (also scored 0–100) is a subdivision of DASH, which include four items to capture difficulties related to the athletes’ sports activity.

Translation and cultural adaptation of KJOC

Translation and cultural adaptation of the questionnaire followed the principles of good practice from the International Society of Pharmaecconomics and Outcome research16 and the guidelines by Beaton et al.13 The procedure was as follows:

We obtained permission to translate the KJOC questionnaire from the developer.5

Forward translation: Two independent bilingual Norwegian residents (T1 and T2) with Norwegian as their main language translated the questionnaire into Norwegian. T1 was a physical therapist, while T2 had no medical background.13

Synthesis of the translations: The project manager held separate meetings with each of the translators, to address discrepancies, seek agreement and make a synthesis of the two translations. She also consulted the questionnaire developer about unsolved questions or special phrases.

Back translation: Two independent Norwegian residents with English as their main language (BT1 and BT2) translated the synthesised forward translation back to English. BT1 was a physical therapist. BT2 had no medical background. The two translators were blinded to the original version of the questionnaire.

Harmonisation: The harmonisation group consisted of the forward translators and back translators, and a language-competent research physiotherapist. The project manager communicated with each member of the group, either by separate meetings or by e-mail. We contacted the developer by e-mail for minor questions. The team reached consensus and approved the harmonised version of the KJOC-N questionnaire.

To minimise non-responses and response errors due to misunderstanding of items, 33 overhead athletes completed the harmonised version of KJOC-N and were interviewed if the questionnaire was relevant to their sport, easy to understand and complete. We also interviewed two coaches (handball and tennis), one medical doctor and three physiotherapists, all of whom were involved in overhead sports, about the relevance and interpretation of the questionnaire.

The results from the interviews were reviewed and a final version of the questionnaire was proofread (online supplementary file 1).

Participants and recruitment

Initially, we recruited 33 overhead athletes for face validation and known-group validation of the questionnaire. They were recruited as follows: (1) patients attending the physiotherapy department at the Norwegian Olympic Sports Centre, (2) overhead athletes training at the Norwegian Olympic Sports Centre and (3) handball players from a local handball club. After completing the KJOC-N questionnaire, 15 consecutive athletes were selected for thorough interviews. We used a cognitive interview technique17 to assess how they interpreted each item and if they would like to remove or add any questions. The initial 33 athletes were also asked about age, years of sports participation, type of sport, previous shoulder injury and to assign themselves to one of three categories: (1) playing without pain, (2) playing with pain and (3) not playing due to pain. No personal data were collected. Subsequently, we recruited handball players from two local handball clubs (one man and one woman) for reproducibility and concurrent validity testing. After an information meeting with the coaches and players, 38 players agreed to participate and signed written informed consent. Participant characteristics for the two parts of the study are presented in tables 1 and 2.

Measurement equivalence between the paper-based and electronic version of the KJOC-N questionnaire

When adapting KJOC-N from paper to an electronic version, we made no change in content or meaning of the questionnaire. The main modification was how the VAS was handled. In the electronic version, the respondent uses a slider to indicate their perception of their current state. Since there were only minor modifications, we did not conduct validity testing of the electronic version, but conducted small-scale cognitive interviews and usability testing.18 19 Six players, randomly selected, completed the pen and paper KJOC-N version during a training session and the electronic version online 3–6 days later. Thereafter, they were interviewed about whether the electronic version changed the way they interpreted the questions, decided on an answer or responded.

Concurrent validity (KJOC-N vs DASH) and reproducibility of the electronic version of KJOC-N

We sent the electronic version of KJOC-N and DASH questionnaires to 38 handball players to evaluate the concurrent validity. To evaluate the test–retest reliability, they were asked to fill out KJOC-N again 1 week later. We chose 1-week interval to reduce the likelihood of change of injury status and recall bias. In all, 36 players completed the first questionnaire, of whom 35 returned a second questionnaire 1 week later. They were also asked if their injury status had changed during this week. In all, 33 players had not changed injury status. Both KJOC-N and DASH were completed online, using Infopad (Infopad AS, Svolvaer, Norway). All data were collected and stored in accordance with The Norwegian Personal Data Act §13, Health Register Act §16 and Health Research Act §2.

Patient and public involvement

We had a partial patient and public involvement in the validation process of the translation. As recommended in studies translating questionnaires, we had players, coaches, therapists and a medical doctor to evaluate the questionnaire with regard to the relevance and interpretation of the questionnaire, as described earlier in the method section.

Statistical analysis

Statistical analysis was performed using SPSS V.24 for Windows. Internal consistency was evaluated using Cronbach's alpha, where α=0 indicate no internal consistency and α≥0.9 corresponds to excellent consistency.20 Paired t-test was used to test for systematic differences between the two rounds. Test–retest reliability of the KJOC-N was calculated using a two-way mixed effects model, single measurement and absolute agreement (ICC2,1). Absolute reliability was determined by calculating SE of measurement, SEM=SD*(), minimal detectable change, MDC=SEM*1,96* 2, and Bland and Altman 95% limits of agreement (LOA). Mann–Whitney test was used to examine known-group validity by comparing players with and without a history of shoulder injury and players playing with and without pain. Spearman's correlation test was used to examine the correlation of KJOC-N with DASH total and DASH sports/performing arts scores, and one-sample t-test test was used to compare KJOC-N and DASH total.

The study was approved by the Norwegian Centre for Research Data (59158/2018). The south-east regional committee for medical and health research ethics stated that approval was not necessary since the study did not include an intervention.

Results

Translation and cultural adaptation

During forward translation of KJOC, we mainly discovered discrepancies in choice of synonyms and prepositions. The phrases ‘popping out’ and ‘get loose’ are not commonly used in Norway. The project manager therefore contacted the main author to get a thorough explanation of the meaning. We also needed to describe the ‘level of competition’ in a way that corresponded with the Norwegian system. ‘Professional major league, professional minor league, intercollegiate and high school’ were replaced by ‘International elite level, national elite level, lower levels (please specify)’. Since the questionnaire is intended for use in all kind of overhead athletes, we changed ‘games’ to ‘competitions’ to include those athletes who do not play games, for example, javelin throwers.

In the original version, instruction to the athlete is given both prior to questions 1 and 5: ‘The following questions concern your physical functioning during game and practice conditions’ and ‘The following questions refer to your level of competition in your sport’. To simplify this, we combined the two instructions prior to question one: ‘The following questions refer to your physical function during competition and training, and the consequences of your function’.

When we compared the original with the back translated version of the questionnaire, we found only differences in largely synonymous words and prepositions. Examples are ‘arm trouble’ and ‘how much’ in the original version, compared with ‘arm complaints’ and ‘to what extent’ in the translated version.

The harmonisation group checked all the translations with the original one, and ensured that there were conceptual equivalence between the original and translated versions.

Cognitive interviews/pre-testing

Both athletes, coaches and medical personnel found the questionnaire very relevant and easy to understand and complete. Regarding question 2, ‘How much pain do you experience in your shoulder and elbow?’ with ‘no pain with competition’ as the best result: Some athletes had experienced to be pain free during competition, but the pain arose afterwards. Others had experienced pain during training but not during competition. We therefore changed the best alternative to ‘pain free during and after competition and training’. No one suggested removing or adding any questions. The KJOC-N scores by current participation status and previous injury are shown in tables 3 and 4. Players who were playing with pain had lower scores than those playing without pain (p<0.01). Players with a history of shoulder injury had lower scores than those without a history of shoulder injury (p<0.01).

Measure equivalence of the paper questionnaire and electronic questionnaire

Measurement equivalence, obtained by small-scale cognitive interviews of six players, revealed no change in content or meaning between the paper questionnaire and electronical questionnaire. The players reported the electronical version to be easier to complete than the paper version

Internal consistency and concurrent validity

Cronbach's alpha was 0.952, indicating excellent internal consistency among the 10 items. The correlation between KJOC-N and DASH total scores was −0.642, and between KJOC-N and DASH-sport was −0.790. However, KJOC-N had a wider range in scores than DASH total (60.1 and 23.3, respectively), distinguished better between players playing with and without pain and was more sensitive to capture players playing with pain (table 5).

Table 5

Kerlan-Jobe orthopaedic clinic scores and Disability of Arm Shoulder and Hand scores, comparing groups playing with and without shoulder pain (n=35)

Category	N	KJOC-N	DASH
Playing without pain	19	95.1 (6.2)	2.9 (4.2)
Playing with pain	16	77.8 (19.0)	7.2 (6.9)†
Difference		17.3 (6.9 to 27.8)*	4.4 (-8.2 to −0.5)**

Mean (SD) is presented for the observed values within each group. Mean (95% CI) is presented for the estimated difference between the categories *p<0.001, **p=0.062.

†Difference between KJOC and DASH scores p=0.004.

Test–retest reproducibility

The mean response time between the two assessments was 9 days.

The 33 players who did not change shoulder injury status between the two assessments, had no systematic difference in the mean total score between the two rounds (difference −0.7, 95% CI −2.3 to 0.8). The relative test–retest reproducibility was excellent with an ICC of 0.967 (95%CI 0.935 to 0.984). SEM and MDC were 3.1 and 8.5, respectively. Bland–Altman's 95% LOA ranged from −9.2 to 7.7 and displayed no funnel effect (figure 2).

Figure 2

Bland–Altman plot showing the test–retest results of the KJOC questionnaire (n=33). solid line: mean difference. Dashed lines: upper and lower 95% CIs. KJOC, Kerlan-Jobe Orthopaedic Clinic,

Discussion

In this study, we translated and culturally adapted the KJOC into Norwegian and evaluated its measurement properties. The main findings of the study were that the KJOC-N is a reliable, valid and internally consistent questionnaire for Norwegian overhead athletes.

We experienced no difficulties during the translation and cultural adaptation and made only minor adaptions to reach conceptual equivalence with the original version of KJOC. Overhead athletes thought the questionnaire was easy to understand and fill out, and both athletes, coaches and medical personnel found it relevant. The comparison between the paper and the electronical version yielded no difficulties. The players were able to use the device and software appropriately. In fact, the interviewed players reported the electronical version to be easier to complete.

Our results showed that the Norwegian version of the KJOC has excellent internal consistency (Cronbach’s α=0.952). These results are in accordance with previous translations.10 115

The concurrent validity versus DASH and DASH sport was moderate and strong, respectively. However, in our population of active handball players, KJOC-N had a much wider range in scores than DASH (60.1 and 23.3, respectively), distinguished better between players playing with and without pain (KJOC-N 17.3 points and DASH 4.4 points) and was more sensitive to capture players playing with pain. This suggests that KJOC-N discriminates better between overhead athletes playing with and without pain than the DASH-total does and supports the original idea behind the development of KJOC. The minimum detectable change at the 95% CI level has been reported to be 12.8–17.2 for the DASH.21 Therefore, DASH might not capture the change in scores if a player is developing shoulder pain. Other patient-related outcome measures for the upper limb, such as DASH, assess activities of daily living and do not capture the specific demands of the overhead athlete. Therefore, the KJOC serves as a more precise assessment in this population.

KJOC-N had excellent relative test–retest reliability (ICC=0.967), which is in accordance with previous translations,10 11 and slightly higher than the developer presented.5 Our measures of absolute reliability (SEM=3.05 and MDC=8.45) were slightly higher than other translated versions (Merolla: 0.81/2.42, Turgut: 1.98/5.49). This might be due to the age of the athletes. In our study, the athletes were younger than in previous studies, 20.7 years compared with 23.1–26.6 years. Another explanation could be the difference in sporting levels. Merolla and colleagues included professional athletes only, while in our study, all but one player were amateurs, playing at a national level. An older, professional athlete may be more accurate in everything he or she is doing related to their sport, including replying to questionnaires. Our 95% LOA (−9.2 to 7.7) indicate that, at an individual level, a change of more than 17 points is necessary before measurement error can be ruled out. The KJOC-N showed statistically significant differences between players who were playing with and without pain (median 98 vs 69.3) and players with and without a history of injury (median 99.6 vs 69.3). This is in agreement with other studies, which accurately distinguished between injured and uninjured athletes, and suggested that normal values of overhead athletes should be above 90.5 9 10 22

Strengths

We conducted a rigorous translation and cultural adaptation of the KJOC questionnaire to maintain the content validity of the questionnaire at a conceptual level. This means that the questionnaire can be used in the targeted Norwegian population and can be used in multinational research projects.

Limitations

The small sample size, smaller than recommended in the COSMIN guidelines, is a main limitation of our study. It was a convenient sample, consisting of one male and one female handball team. Handball is the dominant overhead sport in Norway. Therefore, we included handball players only for the reproducibility and concurrent validity measures, which is a limitation of the study. The sample is a random sample of one male and one female handball team and represent the population of interest, where more than half the population have no shoulder problems. Since these athletes often continue to participate in training and competition despite shoulder pain, it is important to have a tool that can distinguish between those with and without a problem. Such a tool can be used for monitoring changes over time in this population. To be used in overhead athletes with moderate to severe shoulder problems, the measurement properties of the questionnaire should be tested out in advance. In the cross-cultural adaption, we also included volleyball and tennis players, as well as coaches and medical personnel from overhead sports.

We did not study the KJOC-N's responsiveness, a psychometric property important for judgement of treatment outcome or monitor changes over time.23 Both the original English and translated Italian version of the questionnaire are previously shown to be responsive in a mix of overhead athletes.5 11 However, since this may vary between populations, it is important to evaluate it in the population of interest.24

Our results are comparable to the original KJOC and previously translated versions, showing that the KJOC-N is a reliable and valid questionnaire in the evaluation of shoulder and elbow-related problems in overhead athletes, and is a better evaluation tool in this population than traditional patient-related outcomes.

Perspectives

Our results suggest that the KJOC-N is a reliable and acceptable tool for evaluating shoulder and elbow-related problems in Norwegian overhead athletes (handball players). A rigorous translation and cultural adaptation process ensure that the questionnaire can be used in multinational research projects. Since KJOC-N is more clinically relevant for overhead athletes than traditional patient-related outcomes, we now have a better tool to evaluate this population. The KJOC is particularly useful for monitoring athletes with overuse injuries, who often play despite pain, and for evaluating symptoms and function, particularly in the late phase of rehabilitation. The use of electronic questionnaires makes is easy to collect repeated measures.

Table 1

Characteristics of the overhead athletes recruited for face validation and known-group validation of the KJOC-N (n=33)

Age (years), mean (SD)	18.6 (4.2)
Gender (n)
Male	10
Female	23
Sport (n)
Handball	23
Volleyball	8
Tennis	2
Sporting level (n)
International elite level	7
National elite level	4
Lower levels	22

KJOC-N, Norwegian version of Kerlan-Jobe Orthopaedic Clinic shoulder and elbow questionnaire.

Table 2

Characteristics of the handball players recruited for reproducibility and concurrent validity testing of KJOC compared with DASH (n=36)

Age (years), mean (SD)	20.7 (3.8)
Gender (n)
Male	17
Female	19
Handball experience (years), mean (SD)*	13.4 (4.3)
Previous shoulder injury (yes/no)†	10/25
Shoulder injury status (n)†
Playing without pain	19
Playing with pain	16
Sporting level (n)†
International elite level	1
National elite level	13
Lower levels	21

*Missing: 1, 2

†Missing: 1.

DASH, Disability of the Arm, Shoulder and Hand questionnaire; KJOC-N, Norwegian version of Kerlan-Jobe Orthopaedic Clinic shoulder and elbow questionnaire.

Table 3

Kerlan-Jobe orthopaedic clinic scores of groups playing with and without shoulder pain (n=33)

Category	N	Min	Q1	Median	Mean	Q3	Max	SD
Playing without pain	21	67	88	98	94	100	100	9.1
Playing with pain	11	31	64	69	71	90	94	17.6
Not playing	1

Q1, first quartile; Q3, third quartile.

Table 4

Kerlan-Jobe orthopaedic clinic scores of groups with and without previous shoulder injury (n=33)

Previous injury	N	Min	Q1	Median	Mean	Q3	Max	SD
No	18	79	89	100	97	100	100	5.3
Yes	15	31	64	69	71	86	90	15.4

Q1, first quartile; Q3, third quartile.