The Dutch version of the American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form is a reliable and valid questionnaire for shoulder problems.

BACKGROUND: The self-assessment section of the American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form (ASESq) is one of the most used patient-reported outcome measures for general shoulder problems. This study was performed to establish a valid Dutch version of the ASESq (ASESq-NL).
MATERIALS AND METHODS: A clinical prospective, nonrandomized study was performed. Translation of the ASESq into Dutch was done following the guidelines of cross-cultural adaptation. Patients older than 17 years of age with shoulder problems were included. Patients who declined to participate or insufficiently completed questionnaires were excluded. For test-retest reliability analysis, the intraclass correlation coefficient (ICC) was calculated and an interval of 7-28 days between test and retest was set. Cronbach alpha was used to determine internal consistency. Dutch validated versions of the Shoulder Pain and Disability Index (SPADI) and 36-Item Short Form Health Survey (SF-36) were completed and compared with the ASESq-NL to evaluate construct validity using a Spearman rank correlation coefficient calculation.
RESULTS: A total of 92 patients were included. Test-retest reliability was excellent with an ICC of 0.82. The mean test-retest interval was 13 days (standard deviation 4.4). Internal consistency was good, with a Cronbach alpha of 0.83. Construct validity of the ASES questionnaire was good. All domains of the ASESq-NL had significant (P < .05) correlations with the domains of the SPADI and the SF-36, except for the SF-36 domains stability with "physical function and energy" and "emotional well-being."
CONCLUSION: The Dutch ASES questionnaire is a valid and reliable tool for the evaluation of shoulder problems and is permissible for implementation into the Dutch health care system.

With health care costs on the rise, national governments and health insurances increasingly demand the evaluation of treatment regimens with valid outcome measures to assess implementation and financial approval to the health care system.14, 17

The use of validated outcome instruments is widely accepted for the assessment of patient outcome to advance medical treatment in the clinical and research setting.18, 24

Outcome measures can be generally divided into clinician-reported and patient-reported outcome instruments. Clinician-reported measures focus on objective outcome assessed by a trained health care professional. In contrast, patient-reported outcome measures (PROMs) specifically evaluate the patient's perspective on health improvement; as a result, PROMs have gained widespread recognition. A range of PROMs exist, from general health questionnaires, like the 36-Item Short Form Health Survey (SF-36), to more disease- and joint-specific questionnaires, like the patient self-evaluation section of the American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form (ASESq).

The ASESq is one of the most used PROMs for shoulder problems.7, 24, 26 It is easy and quick to use and validated for the evaluation of shoulder problems. The ASESq has a broad scientific background, with citations in more than 1000 papers and a good scoring of the AO Handbook on Musculoskeletal Outcome Measures and Instruments.27, 30 Schmidt et al conducted a standardized and systematic review of 11 shoulder-specific PROMs using the Evaluating the Measurement of Patient-Reported Outcomes evaluation tool for PROMs.28, 32 The ASESq obtained the best overall score and was consecutively among the top 3 outcome measures in this study. The ASESq has been translated and validated into many languages, including German, Italian, Spanish, Finnish, Portuguese, Turkish and Arabic.8, 12, 15, 21, 22, 34, 35 A validated Dutch version is currently not available. The multilingual availability of the ASESq makes it particularly suited for clinical research as well for the use in multicultural societies like the Netherlands. This clinical prospective, nonrandomized study was performed in order to cross-culturally adapt and validate the Dutch version of the ASESq (ASESq-NL).

Materials and methods

Questionnaires

The American Shoulder and Elbow Surgeons Standardized Assessment Form was developed during 1990 to 1993 by the ASES to address the need for a state-of-the-art assessment tool for all shoulder patients regardless of diagnosis. The American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form consists of 3 sections: demographic information, patient self-evaluation (ASESq), and physician assessment. We focused on the ASESq, as a PROM, together with the ASES shoulder score index, which is calculated from the items of the ASESq.8, 12, 15, 21, 22, 34, 35 The ASESq contains 18 questions divided over 3 sections: pain, instability and activities of daily living (ADL). The ASES shoulder score index is derived from the visual analog scale (VAS) for pain and the cumulative ADL score. The ADL score consists of 10 questions that assess ADL for both shoulders, graded on a 4-point ordinal scale, from 0 (unable to do) to 3 (not difficult). The shoulder score index (X) has a range from 0 (most disability) to 100 (least disability) and can be calculated with the formula: X = [(10 − VAS pain score) × 5] + [(5/3) × cumulative ADL score].

The validation of the ASESq-NL is achieved by comparing its 3 domains “pain”, “stability” and “activity of daily living” with the corresponding 8 domains of the 36-Item SF-36 and the 2 domains of the Shoulder Pain and Disability Index (SPADI).10, 11, 25 The SF-36 is one of the most used generic patient-reported health measures, but is also specifically validated for shoulder complaints. The SPADI is a tested and widely used self-assessment instrument for the shoulder.7, 25 Both measures were adapted and validated for Dutch language and have both been used for the outcome validation of the original ASESq as well as for the crosscultural adaptation and validation of the ASESq into other languages.8, 12, 22, 24, 30, 34, 35

Translation and linguistic validation method

A forward-backward translation protocol according to the guidelines of cross-cultural adaptation was used following the 5 steps of translation, synthesis, back-translation, expert committee review and pretesting.1, 13 Forward translation was done separately by 2 native Dutch speakers, one acting as an informed translator (orthopedic resident) and the other as an uninformed translator (medical student). A Dutch consensus version of the ASESq was created and checked for cross-cultural differences. Three cross-cultural dissimilarities were identified. Question 4 of the ASESq refers to pain medication and provides example substances aspirin, Advil and Tylenol. The example medication is not typical for the Netherlands and was replaced with paracetamol and ibuprofen, both comparable substances commonly used in the Netherlands. In question 5, narcotic pain medication was translated to medication requiring a doctor's prescription, with the narcotic medication tramadol and codeine as examples. Both questions are not part of the shoulder score index. In the self-evaluation section concerning activities of daily living, the weight in question 7, “lift 10 lbs above the shoulder,” was metrically converted to 5 kg. After completing the Dutch consensus version of the ASESq, a backward translation by a native English speaker not working in the medical field was executed. Both versions were reviewed by an expert committee. Forward and backward translations revealed no severe differences or language difficulties. After approval of the preliminary ASESq-NL a pretest was performed on 20 patients to reveal any problems in handling and understanding. No significant difficulties were reported. As a result, the final Dutch ASES questionnaire (ASESq-NL) was concluded.

Study population

The study was conducted by the Department of Orthopedic Surgery in the Zuyderland Medical Center in the Netherlands. Between October and December 2013, all patients older than 17 years of age who were referred to our clinic with shoulder problems were asked to participate. Patients who delivered incomplete questionnaires or denied participation were excluded. A confirmation letter of the appointment at our outpatient clinic was delivered to the participant by post accompanied by the first set of questionnaires (ASESq-NL, SPADI, SF-36). Participants were instructed to complete the questionnaires unassisted. For test-retest analysis, a second set of questionnaires (ASESq-NL, SPADI, SF-36) was completed at our outpatient clinic before visiting the orthopedic surgeon. Questionnaires with a test-retest interval of less than 1 or more than 4 weeks were excluded, as suggested in the literature.1, 3, 29, 33 Patients who received an intervention during the test-retest interval were also excluded. Interventions were defined as shoulder injections or operations, not counting oral pain medication.

Assessment of psychometric properties

Reliability and validity were assessed to determine the quality of the measurement instrument.

Reliability

Reliability refers to the degree to which results of an instrument can be replicated on recurring measurements across time (test-retest) and among related items on the instrument (internal consistency). It can be expressed by a value from 0, no reliability, to 1, absolute reliability.

The Cronbach alpha was used to calculate internal consistency. It is a widely accepted tool for homogeneity calculation and has been used in most comparable ASESq validation studies.12, 33 Values greater than 0.70 reflected a sufficient correlation between the items of a questionnaire. A result between 0.70 and 0.79 was considered as fair, between 0.80 and 0.89 as good and ≥0.90 as excellent internal consistency. A Cronbach alpha above 0.90 could however imply that items on an instrument are too homogenous and thus redundant. Generally, a maximum alpha value of 0.90 has been recommended.

Test-retest reliability is based on the assumption that 2 separate measurements should be the same if no change occurred. A fitting time interval between test and retest is crucial. A short interval could lead to a memory bias and a long interval to an actual change of status. Following the guidelines of cross-cultural adaptation, a test-retest analysis on 30 or more patients with a time interval from 1 to 4 weeks was performed. The intraclass correlation coefficient (ICC) was calculated to evaluate test-retest reliability. It is a widely accepted tool for the evaluation of test-retest reliability. An ICC of 0 indicated no agreement, and an ICC of 1 absolute agreement, between test and retest. An ICC within 0.60-0.74 was considered good and an ICC >0.74 was considered excellent. Visual presentation of the test-retest reliability was done with a Bland-Altman plot, which allows more insight into the spread of data for test-retest analysis.

Validity

Validity is the degree to which an instrument measures what it is supposed to measure. The 3 domains of the ASESq-NL were compared with the corresponding domains of the SF-36 and SPADI to evaluate construct validity. This was done using the Spearman rank correlation coefficient. Statistical analysis was performed using SPSS 24.0 (IBM, Armonk, NY, USA) and Excel 2010 (Microsoft, Redmond, WA, USA), considering a P ≤ .05 as significant.

Results

A total of 103 patients were asked to participate. Two patients refused to participate and 9 patients did not complete >70% of the questionnaires. The remaining 92 patients were included. The patient characteristics are shown in Table I.

Table I

Patient characteristics

	No. (%)
Sex
Female	47 (51)
Age, yr, mean (SD)	55 (±12)
Affected side
Right shoulder	57 (62)
Left shoulder	29 (31.5)
Bilateral	6 (6.5)
Diagnosis
Tendinitis calcarea	14 (15)
Subacromial impingement	33 (36)
Biceps tendinitis	5 (5)
AC osteoarthritis	9 (10)
SC dislocation	1 (1)
Rotator cuff syndrome	2 (2)
Rotator cuff rupture	8 (9)
Labrum lesion	3 (3)
Multidirectional instability	1 (1)
Glenoid fracture	1 (1)
Omarthrosis	2 (2)
Frozen shoulder	11 (12)
Unknown	2 (2)

SD, standard deviation; AC, acromioclavicular; SC, sternoclavicular

Internal consistency and test-retest reliability

Internal consistency was good, with a Cronbach alpha of 0.83. According to the guidelines of cross-cultural adaptation, a minimum sample size of 30 is required for test-retest reliability analysis. Test-retest was performed in 37 patients. The mean interval between test and retest was 13 days (standard deviation 4.4), with an excellent ICC of 0.82 (95% confidence interval 0.65, 0.91; P < .01) for the total ASESq score. The ICC for the subgroups of the ASESq-NL are shown in Table II. The Bland-Altman plot (Fig. 1) shows good agreement between test and retest, which is consistent with the calculated ICCs.

Table II

Test-retest ASESq

ASESq domains (n = 37)	ICC (95% CI)	P value
Pain	0.80 (0.61, 0.90)	<.01
Stability	0.77 (0.55, 0.88)	<.01
Daily activities	0.84 (0.68, 0.92)	<.01
ASESq score total	0.82 (0.65, 0.91)	<.01

ASESq, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form; ICC, intraclass correlation; CI, confidence interval.

All domains had significant ICCs. The domain Daily activities had the highest correlation and Stability the lowest correlation.

Figure 1

Bland-Altman plot. Top horizontal line: upper limit of agreement, 24; mid horizontal line: mean (bias), –2.28; lower horizontal line: lower limit of agreement, –29. ASESq, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form.

Construct validity

All domains of the ASESq-NL had significant correlations (P < .05) with the domains of the SPADI and the SF-36 (see Tables III and IV), except for the 2 SF-36 domains “stability with physical function and energy” and “emotional well-being.” For the ASESq-NL domain “stability,” only 84 patients completed the stability questions and were available for analysis.

Table III

Correlation of ASESq domains compared with the domains to the domains of the SF-36 questionnaire

SF-36	Physical function	Role limitations– physical	Role limitations– emotional	Energy /Fatigue	Emotional well-being	Social functioning	Pain
ASESq
Activity (n = 92)
Correlation	0.49∗	0.49∗	0.36∗	−0.083	0.17	0.44∗	0.57∗
Significance (2-tailed)	0.000	0.000	0.000	0.43	0.11	0.000	0.003
Pain (n = 92)
Correlation	−0.32∗	−0.35∗	−0.25∗	0.082	−0.13	−0.44∗	−0.61∗
Significance (2-tailed)	0.002	0.001	0.015	0.44	0.24	0.000	0.000
Stability (n = 84)
Correlation	−0.21	−0.22†	−0.22†	0.086	−0.13	−0.30∗	−0.35∗
Significance (2-tailed)	0.054	0.050	0.041	0.44	0.24	0.005	0.001
Total (n = 92)
Correlation	0.53∗	0.52∗	0.35∗	−0.11	0.16	0.50∗	0.73∗
Significance (2-tailed)	0.000	0.000	0.001	0.32	0.13	0.000	0.000

ASESq, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form; SF-36, 36-Item Short Form Health Survey.

The strongest correlation was between ASESq–total and SF-36 Pain, whereas no significant correlation was found for the domains Energy/Fatigue and Emotional well-being, as well as between ASESq stability with physical functioning.

Correlation was significant at the alpha ≤0.01 level (2-tailed).

Correlation was significant at the alpha ≤0.05 level (2-tailed).

Table IV

Correlation of ASESq domains compared with the domains to the domains of the SPADI questionnaire

ASESq	SPADI
	Total score	Pain	Disability
Activity (n = 92)
Correlation	−0.70∗	−0.62∗	−0.71∗
Significance (2-tailed)	0.000	0.000	0.000
Pain (n = 92)
Correlation	0.45∗	0.50∗	0.39∗
Significance (2-tailed)	0.000	0.000	0.000
Stability (n = 84)
Correlation	0.27∗	0.33∗	0.22†
Significance (2-tailed)	0.012	0.002	0.045
Total (n = 92)
Correlation	−0.70∗	−0.70∗	−0.66∗
Significance (2-tailed)	0.000	0.000	0.000

ASESq, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form; SPADI, Shoulder Pain and Disability Index.

All domains of the ASES questionnaire did have statistically significant (P < .05) correlations with the domains of the SPADI. The highest correlation was between the ASESq total score and SPADI Pain, whereas the weakest correlation was between ASESq stability and SPADI disability.

Correlation was significant at the alpha ≤0.01 level (2-tailed).

Correlation was significant at the alpha ≤0.05 level (2-tailed).

Discussion

The most important finding of this study is that the Dutch version of the ASESq is a valid and reliable tool for assessing shoulder problems.

No severe problems were encountered during the translation and adaptation process. A good internal consistency with a Cronbach alpha of 0.83 was found. Alpha values between 0.70 and 0.80 are regarded as acceptable for the comparison of groups. During comparison of our Cronbach alpha with the alpha values of previous adaptation studies, similar results were found, with the exception of the Cronbach alpha described by Goldhahn et al.8, 12, 20, 21, 22, 35 An explanation could be that Goldhahn et al only included patients who underwent a primary total shoulder arthroplasty. Different alpha values could also be due to differences in cohort size, as well as possible variations in the calculation of Cronbach alpha. Considering our good and comparably rated Cronbach alpha, we are confident that a good internal consistency of the Dutch ASES questionnaire was achieved.

Furthermore, an excellent test-retest reliability was found for the Dutch ASES questionnaire with an ICC of 0.82 and a corresponding Bland-Altman plot. Compared with the studies of Celic and Goldhahn, who found an ICC of 0.94 and 0.93, respectively, the ICC of our study was substantially lower.8, 12 A possible explanation for this dissimilarity could be the longer time interval used between test and retest. The mean test-retest interval of this study was 13 (±4.4) days, whereas Celic and Goldhahn had an interval of 3-7 and 7 days, respectively.8, 12 A shorter test-retest interval makes it more likely to get a similar test-result, leading to a higher ICC, because of the carryover effects of patients, who could potentially still remember what they reported in the first questionnaire. During a short interval, it is unlikely that the health status of the shoulder changes considerably, which could lead to a more consistent scoring. In contrast, a long test-retest period could result in significant health change. In this study, a test-retest interval of 1-4 weeks was applied to avoid the risk of memory bias as well as change of health status, as suggested in the literature.1, 3, 29, 33 Piitulainen had a similar test-retest interval of 2 weeks and a comparable ICC of 0.83, supporting the hypothesis mentioned above. In addition, a Bland-Altman plot was drawn for visual presentation and to provide a better insight into the variations between the test and retest compared with the ICC. Although the test-retest interval varies considerably between the ASESq validation studies, the generally produced ICC is similar to this study and consistently very good, indicating an excellent test-retest reliability of the ASESq-NL.8, 20, 21, 22, 35

The domains of the Dutch ASES questionnaire quitu were compared with the corresponding domains of the validated and translated Dutch instruments SF-36 and SPADI to evaluate construct validity. A good construct validity was found for the domains of the SPADI and the SF-36 with the corresponding domains of the ASESq-NL, except for the SF-36 energy, fatigue and emotional well-being. As suggested by Berendes and Piitulainen, it is expected that these SF-36 domains do not correlate linearly with shoulder problems because of their general character.3, 22 In contrast, the ASESq-NL and SPADI had significant correlations in all domains. This was expected because of the specific character of the SPADI, which focuses on shoulder problems.

Although Celik and Piitulainen described a significant correlation for the SF-36 domain “emotional well-being,” Piitulainen suggested that this aberration could be due to “differences in, e.g. sample size, age, reason for shoulder disorder”.8, 22

In addition, the ASESq-NL domain stability showed no significant correlation with the domain “physical function” of the SF-36 health form. Furthermore, the ASESq-NL stability section demonstrated the lowest test-retest agreement in this study. A similar finding was presented in the study of Goldhahn, who found a fair ICC and concluded that the stability section was not suitable for the evaluation of stability in total shoulder arthroplasty patients. Whether this evaluation is true and applicable to other patient groups remains unclear.

A significant correlation was found between the ASESq-NL stability domain and all domains of the shoulder-specific SPADI questionnaire. The ASESq stability section is however, as intended by the original author of the ASESq, not included in the calculation of the shoulder score index.

Strengths of this study are the good representation of the ASESq-NL target population, with an equal distribution of men and women, as well as the wide range of shoulder problems, keeping in mind that the ASESq was designed for shoulder problems.

A limitation of the study is that there is no final consensus on how to cross-culturally adapt and validate health questionnaires. Multiple guidelines are used with significant differences in their approach. As a result, the most respected guidelines were selected. These guidelines were generally also used by the other ASESq adaptation and validation studies, leading to comparable results with confirmation of reliability and construct validity of the ASESq-NL.

Conclusion

The Dutch ASES self-assessment form (Fig. 2) is a valid and reliable tool for the evaluation of shoulder problems and is permissible for implementation into the Dutch health care system.

Figure 2

ASESq-NL: Dutch ASES shoulder form. ASESq, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form.

Disclaimer

The authors, their immediate families and any research foundations with which they are affiliated have not received any financial payments or other benefits from any commercial entity related to the subject of this article.

Acknowledgments

The authors thank Virginia Hoel, Björn Remmerswaal and Mohamed Dinle for their contribution to the forward-backward translation process of this study.