Cross-cultural adaptation and validation of the Musculoskeletal Tumor Society (MSTS) scoring system and Toronto Extremity Salvage Score (TESS) for musculoskeletal sarcoma patients in Greece.

PURPOSE: To perform translation, cross-cultural adaptation, and validation of the Toronto Extremity Salvage Score (TESS) and Musculoskeletal Tumor Society (MSTS) scoring system in Greek patients with lower extremity sarcoma.
METHODS: The Greek version of the MSTS for the lower extremity and TESS questionnaires was developed using previously reported methods. Included were 100 patients with musculoskeletal sarcoma who underwent limb salvage surgery. The test-retest reliability [interclass correlation coefficient (ICC) between 2 different time points], internal consistency (Cronbach's alpha), construct validity (Kaiser's criteria, Eigenvalue > 1 rule), and external validity (Short form-36, Spearman's Rho) were assessed.
RESULTS: The test-retest reliability (ICC was 0.99 for MSTS-LE and 1 for TESS) and internal consistency were high (Cronbach's alpha was 0.763 for MSTS-LE and 0.924 for TESS) for both questionnaires. Based on the Scree plot, the number of factors retained was 1 for MSTS-LE and 2 for TESS. The TESS showed a strong correlation with SF-36 (Spearman's rho = 0.714, p < 0.001), but the correlation between MSTS for lower extremity and SF-36 was weak (Spearman's Rho = 0.313, p = 0.002).
CONCLUSIONS: The Greek version of both the MSTS for lower extremity and TESS questionnaire showed sufficient reliability, internal consistency and good performance using the loading factor analysis when used postoperatively in Greek patients who underwent surgical resection of lower extremity sarcoma. However, only the TESS showed strong correlation with the SF-36, indicating that MSTS for lower extremity was not as powerful for the evaluation of the global health status of these patients.

Introduction

Musculoskeletal sarcomas are malignant tumors that can arise from the bone or soft tissues (fat, muscle, nerve, blood vessels, and connective tissues) [1, 2]. Bone sarcomas are extremely rare types of cancer, accounting for < 0.5% of all malignancies [3]. Bone sarcomas show bimodal distribution during the second decade of life and in ages older than sixty years [3]. Osteosarcoma is the most common type of primary bone tumor, which is usually treated with a combination of surgery and chemotherapy [4]. Soft tissue sarcoma (STS) is a rare type of tumor with a reported incidence of 7000–10,500 new cases per year in the United States [5]. STS more commonly affects patients of male gender and the majority (85%) of tumors occur during adulthood or later in life [6]. Extremities are the most common site of STS [6, 7] Surgical resection and radiation therapy remains the gold standard therapy of the extremity STS, while the use of chemotherapy remain controversial [7-9]. Limb amputation is preserved for cases with inability to resect the tumor with negative margins [10].

In patients with STS or bone sarcoma of the upper or lower extremity who undergo tumor resection and/or amputation procedures, preserving the function of the affected limb is of outmost importance since it significantly affects the patient’s quality of life [11, 12]. With the rapid increase of multi-cultural research projects, the need for the development of methodologies to facilitate the cross-cultural adaptation of health status questionnaires was imminent [13]. In 1998, Beaton et al. [14] published the first guidelines for the cross-cultural adaptation of health status surveys, in order to ensure the maintenance of content validity across different cultures. Apart from the recommendation for linguistic translation of the health status questionnaires, these guidelines aimed to resolve possible cultural adaptation issues with the use of these questionnaires worldwide [14].

The Musculoskeletal Tumor Society (MSTS) scoring system and Toronto Extremity Salvage Score (TESS) are widely accepted scales for the assessment of function in patients with musculoskeletal sarcoma [15]. The MSTS scoring system was originally developed in 1985 and revised in 1993 by Enneking et al. [16] and has been extensively been used in orthopedic oncology. The lower extremity version of MSTS has been translated and validated into multiple languages including Danish, Chinese, Portuguese, and Japanese [12, 17–20]. Similarly, the upper and/or lower extremity versions of the TESS, have been translated and validated in Italian, Dutch, Japanese, Chinese, and Korean [11, 21–25].

To our knowledge, cross-cultural adaptation of the lower extremity versions of MSTS and TESS questionnaires in Greek has not been performed. The purpose of this study was to translate and culturally adapt the lower extremity versions of TESS and MSTS for lower extremity to Greek and to validate the translated version of these two evaluation instruments among patients who underwent limb salvage surgery for lower extremity sarcoma.

Methods

Study population

This study was approved by the local Ethics Committee and all patients participated voluntarily and provided verbal consent. Included in this cross-sectional study were adult patients with a diagnosis of lower extremity sarcoma (STS or bone sarcoma), who had previously undergone one or more tumor resection procedures and were followed up at an outpatient setting. Patients who had diagnosis of lower extremity sarcoma but did not undergo surgical treatment, had diagnosis of dementia (any type) or were in a state of altered metal status, and/or were not fluent in Greek language were excluded.

Translation and cross-cultural adaptation

The translation and cross-cultural adaptation of the TESS and MSTS questionnaires for lower extremity was performed based on the published guidelines by Beaton et al. [14] and Guillemin et al. [26] Two bilingual researchers in orthopedic surgery (IBK, PM) whose mother language was Greek carried out the translation of MSTS and TESS questionnaire used in the Radiation Therapy Oncology Group (RTOG) studies on lower extremity sarcoma [27]. Two bilingual translators whose mother language was English and who did not have medical background translated the Greek version back to English. The review committee, consisted of three orthopedic surgeons who specialize in orthopedic oncology (PJP, ODS, AFM) and who reviewed all versions and components of the original questionnaire and the Greek versions of the MESS and TESS questionnaires.

MSTS, TESS and SF-36 questionnaires

The MSTS is a measure of physical function across 7 items, completed by the physician (preferably by the Orthopedic Surgeon or Surgical Oncologist) or the physician’s designated staff. The 7 items are: pain, range of motion, strength, joint stability, joint deformity, emotional acceptance, and overall function. Each item is scored from 0 to 5 with a maximum possible score of 35, which is converted to a scale from 0 to 100 points. The MSTS has been in use for over 20 years and is a widely recognized and utilized tool used to evaluate physical function [27]. The TESS is a self-administered questionnaire evaluating possible limitations in physical activity. A total of 30 questions are included in the TESS, and the degree of disability is rated from 0 (complete disability) to 5 (no functional impairment) in each item. Similar to MSTS for lower extremity, the final TESS score is converted to a score ranging from 0 to 100 points. If a question in TESS score does not apply to the patient, the last can respond “not applicable.”

The short form 36 (SF-36) is a 36-item health survey status which has been widely accepted as a global measure of health-related quality of life. The SF-36 is composed of the following eight domains: physical functioning, role-physical, bodily pain, general health, vitality, social functioning, role-emotional, and mental health. Each of these domains can be rated from 0 (worst) to 100 (best). The final SF-36 score is converted to a 0–100 points range scale. The vast majority of studies that have validated the MSTS or TESS scores in non-English languages have used the SF-36 to assess the external validity of the translated versions of these questionnaires [11, 17, 20]. The final versions of MSTS (lower extremity) and TESS were administered to 10 volunteers who did not participate in the study, in order to assess the need for additional modification of the questionnaire.

Patient assessment

Eligible patients were retrospectively identified through chart review and invited to participate in the study by the research personnel during their clinical visits and/or by communication via phone or email. The physical examination of the MSTS questionnaire (lower extremity) was filled out by an orthopedic surgery trainee during the postoperative follow-up appointments of the participants. The SF-36 and TESS questionnaires were filled out by patients while waiting for their appointment or by phone with the help of the research staff.

Data collection and statistical analysis

A pre-arranged excel file was designed for data collection and the patient’s answered were recorded. Patients who had responded “not applicable” to more than 50% of the items in any of the questionnaires were excluded. The participants completed the MSTS and TESS questionnaires twice; at time 0 and approximately 2 weeks later. The reliability was tested with test–retest analysis using the interclass correlation coefficient (ICC). Internal consistency (strength of the relationship among the items of the same questionnaire) was assessed using the Cronbach’s a coefficient, and coefficient of > 0.70 was considered acceptable. Construct validity (the degree to which the system assesses the underlying theoretical construct it is supposed to measure) was evaluated by principal component analysis. The number of factors to consider was determined by Kaiser’s criteria (eigenvalue > 1 rule) and a scree plot. In order to calculate each item’s factor loading, repeated varimax rotation was performed. Factor loadings can range from − 1 to 1 and values close to − 1 or 1 indicate that the item strongly affects the variable. Factor loading value above 0.4 is considered acceptable. As a rule-of-thumb factor loading value is considered "weak" if less than 0.4, "strong" if more than 0.6, and "moderate,” it is between 0.4 and 0.6..External validity was assessed by comparing the Greek version of MSTS (lower extremity) and TESS with SF-36 which was already validated in Greek, [28] using Spearman’s rank correlation (Spearman’s Rho). All statistical analyses were performed using SPSS version 18.0 (SPSS Inc, Chicago, IL, USA). The scores were reported as mean values ± SD. The threshold for significance was set at < 0.05.

Results

Translation process

The translators and the review committee only requested minor linguistic corrections to the Greek version of the TESS (lower extremity) and MSTS questionnaires. The translation and cross-cultural adaptation process yielded a Greek version of the MSTS for the lower extremity and TESS questionnaires. (Online Appendix 1 and 2).

Patients and survey results

One-hundred and twenty-five patients agreed to participate in the study. Of those 125 patients, 9 were lost to follow-up, while 16 patients responded “not applicable” to more than 50% of the TESS questionnaire and they were excluded. A total of 100 patients (age 43 ± 19 years, 78% males) were included in the statistical analysis (Table 1). No preoperative scores had been collected. At mean follow-up time of 4 ± 1.2 years (time 2 weeks), the mean TESS was 69.7 ± 14.5 and the mean MSTS for lower extremity score was 62.8 ± 21. No significant difference between the postoperative TESS (Table 2) was found between male and female patients (70.2 and 69.3, respectively, p = 0.766), but the mean MSTS (Table 3) for lower extremity score was significantly higher in females compared to males (69.3 and 59.1, respectively, p < 0.01).

Table 1

Study population characteristics

Patient characteristic	Number of patients (%)
Gender	78 males, 22 females
Age in years (mean, SD)	43 (19)
Follow-up time in years (mean, SD)	4 (1.2)
Location of sarcoma
Hip—proximal thigh	55 (45%)
Distal thigh, knee, and proximal leg	25 (25%)
Lower leg, foot, and ankle	20 (20%)
Sarcoma type
Bone sarcoma	6 (%)
Soft tissue sarcoma	94 (94%)
Total number of patients	100 (100%)

Table 2

Itemized Salvage Score (TESS) as reported in the study population

TESS Item	Mean	Standard deviation	Minimum	Maximum
Question 1	4.6	0.7	2	5
Question 2	4.6	0.7	2	5
Question 3	4.5	0.8	2	5
Question 4	4.6	0.8	2	5
Question 5	4.7	0.7	2	6
Question 6	4.7	0.8	2	6
Question 7	4.8	0.7	2	6
Question 8	4.3	1.1	1	6
Question 9	3.9	1.2	1	6
Question 10	4.5	0.8	2	5
Question 11	4.5	0.8	2	5
Question 12	4.4	0.9	2	5
Question 13	3.2	1.4	1	6
Question 14	3.9	1.2	1	5
Question 15	4.1	1.1	1	5
Question 16	4.1	1.1	1	6
Question 17	5.5	0.8	1	6
Question 18	4.7	0.6	2	5
Question 19	4.3	1.1	1	6
Question 20	4.6	0.7	2	5
Question 21	4.1	1.1	1	6
Question 22	4.5	0.8	2	5
Question 23	4.0	1.2	1	6
Question 24	4.3	1.1	1	6
Question 25	5.4	0.9	2	6
Question 26	4.2	1.1	2	5
Question 27	5.6	0.7	2	6
Question 28	5.2	1.1	1	6
Question 29	4.8	1.1	1	6
Question 30	5.3	1.0	1	6
Question A	4.2	1.1	1	5
Question B	4.2	1.1	1	5
Score	69.8	14.6	22.4	84.2

Table 3

Itemized Musculoskeletal Tumor Society Scale for lower extremity as reported in the study population

MSTS lower extremity domain	Mean	Standard deviation	Range
Motion	3.2	1.5	0–5
Pain	3.5	1.5	0–5
Stability	3.7	1.4	0–5
Deformity	3.2	1.5	0–5
Strength	3.2	1.2	0–5
Functional	2.5	1.4	0–5
Emotional	2.2	1.6	0–5
MSTS-LE score	62.8	21.5	0–5

Reliability and validity

The test–retest reliability was excellent for both the TESS [ICC was 1 (range: 0.99–1] and MSTS-LE score [ICC was 0.997 (range: 0.995–0.998)]. The internal consistency was good with Cronbach’s alpha of R = 0.924 for TESS and R = 0.763 for MSTS (lower extremity). As shown in Fig. 1, the Scree plot for TESS indicated that the appropriate number of factors was two. Figure 2 shows the factor loading values for the 30 items included in the Greek version of TESS questionnaire. Question 29 (ability to socialize with friends) had factor loading value of 0.5 (moderate). According to the Scree plot for MSTS (lower extremity) (Fig. 3), the appropriate number of factors was one. All items had factor loading values of > 0.7 in the MSTS for lower extremity scale. The external validity analysis indicated strong correlation between the SF-36 and TESS (Spearman’s rho = 0.714, p < 0.001) but the correlation between MSTS and SF-36 was weak (Spearman’s Rho = 0.313, p = 0.002).

Fig. 1

Principal factor analysis of the Greek version of TESS: scree plot with eigenvalues of TESS indicating a 2-factor model

Fig. 2

Factor loading values of the 30 items in the Greek version of TESS questionnaire

Fig. 3

Principal factor analysis of the Greek version of MSTS-LE: scree plot with eigenvalues of TESS indicating a 1-factor model

Discussion

The Greek version of TESS and MSTS for lower extremity score showed adequate performance on the loading factor analysis as well as excellent test–retest reliability and internal consistency during the postoperative evaluation of patients with lower extremity sarcoma. The Greek version of TESS was found to be reliable and valid measurement of physical function in patients who undergo surgical treatment for lower extremity sarcoma and showed a strong correlation with the total SF-36. In contrast, the MSTS for lower extremity only showed weak correlation with the total SF-36 score, making it a less suitable functional score to be used for the postoperative evaluation of these patients. The last was expected given the MSTS primarily focuses on physical assessment, while SF-36 is a global health survey which includes mental health evaluation in addition to the evaluation of physical function. No difference in the postoperative TESS scores of female and male patients was detected; however, the mean postoperative MSTS for lower extremity score was significantly higher in female patients.

The English version of the TESS questionnaire had been translated, validated and culturally adapted by other countries including Korea, Japan, Italy, Brazil, and China in patients with sarcoma of the upper and/or lower extremity [11, 21, 23–25]. It is important to appropriately validate one or more patient evaluation instruments in order to perform clinical outcome studies in orthopedic oncology. The TESS is a widely accepted functional evaluation tool in the field of orthopedic oncology worldwide and, thus, its validation in the Greek population will open new horizons for reporting the clinical outcome of patients who received operative therapy for lower extremity musculoskeletal sarcoma. Although the translation and cross-cultural adaptation process was complex, the Greek version of the TESS was found to have adequate reliability, internal consistency, and external validity for use in clinical practice. Previous studies have successfully performed cross-cultural adaptation of this scale in patients with bone and soft tissue tumors of the lower extremity [11, 21, 23–25].

Regarding the construct validity of the Greek version of TESS, one out of the 30 questions were found to have factor loading value of 0.5, which was lower compared to the corresponding value for the remaining 29 questions that had factor loading values > 0.7. As mentioned above, construct validity refers to the degree to which a test measures the intended construct. For an item (question) to be appropriate for the construct, factor loading must be at least 0.4. Factor loading values between 0.4 0.6 are considered “moderate” but acceptable [29, 30]. In the Greek version of TESS, question 29 was assessing the ability of the patient to socialize with his or her friends. This is the first cross-cultural adaptation analysis of the TESS questionnaire identifying a relatively lower factor loading value for this specific question, and therefore, this finding could be attributed to inter-cultural differences related to the perception of having musculoskeletal cancer diagnosis between patients from different countries and its impact on the patients’ social interactions [11, 21, 23–25]. When attempting to validate the TESS in non-English language, researchers should pay attention to this specific question at the translation stage in order to achieve successful cross-cultural adaptation of the TESS.

The MSTS-LE extremity questionnaire has also been validated and cross-culturally adapted into non-English languages for patients with musculoskeletal tumors of the upper and/or lower extremity [17-20]. The majority of these studies used the 6-item MSTS for lower extremity questionnaire [12, 18–20]. This current study used the 7-item MSTE-LE questionnaire that was used in the clinical trial performed by the Radiation Therapy Oncology Group in 2008 in order to assess the outcomes and complications of image-guided radiation therapy in patients with primary STS of the shoulder, arm, hip, or leg [27]. The traditional MSTS questionnaire for the lower extremity is composed of six items including pain, function, emotional acceptance, use of any external support, walking ability, and gait alteration. Each item is rated in a scale of 0–5. The total score ranges from 0 to 30, with higher scores indicating better function. The English version of MSTS for lower extremity that we used included the following 7 items: motion, pain, stability, deformity, strength, functional activity, and emotional acceptance. Similar to the traditional version, each item was rated in a scale of 0 to 5 and the total scored was converted to a scale from 0 to 100. We elected to perform cross-cultural adaptation of this last version due to the inclusion of the emotional acceptance component which, based on our experience with Greek patients, it would be more representative of their overall functional status.

When the external validity of the Greek version of TESS and MSTS for lower extremity score was assessed against the SF-36 survey, TESS showed a strong correlation (Rho = 0.714) but the correlation of MSTS for lower extremity scores with the total SF-36 score was weak (Rho = 0.313). Iwata et al. [19] showed no correlation of the mental health component of SF-36 with the total MSTS score or any of the items of MSTS scoring system. The authors suggested that this finding was due to the MSTS questionnaire not including questions related to mental health [19]. However, this last study found a correlation between the total SF-36 and the 6-item MSTS questionnaire [19]. We adapted the 7-item MSTS for lower extremity scale which included the component of emotional acceptance and, although we expected a strong correlation between the total MSTS (lower extremity) and SF-36 scores, the correlation was found to be weak (Rho = 0.313). Therefore, both the 6-item MSTS and the Greek version of 7-item MSTS for lower extremity questionnaires were mostly representative of the physical capabilities of patients with lower extremity sarcoma and should not be considered equally powerful instruments for the evaluation of global health status.

Our study was limited by several factors. Patients were identified retrospectively using our institutional resources based on diagnosis and we were not able to collect data related to imaging, pathology examination, or intraoperative findings. In addition, some patients filled out the TESS and/or SF-36 questionnaires via telephone communication with the research personnel which might have affected the accuracy of their answers. The Hawthorne effect occurs when research participants behave differently because they know they are being tested, and our study design was vulnerable to this type of bias. A major limitation was that we were not able to validate the validity of MSTS for lower extremity against the various components of the SF-36 survey due to insufficient data; although the total SF-36 were recorded for all patients, the score of the various SF-36 domains were inadequately reported by the research personnel. This was also impacted by ongoing global pandemic (COVID-19), which resulted in cancellation of patient visits and loss of participants. Lastly, the validity of these outcomes might have been affected by the possible presence of selection bias, as most of our patients were seen on an outpatient basis.

Conclusions

The Greek version of both the MSTS for lower extremity and TESS questionnaire showed sufficient reliability, internal consistency and good performance using the loading factor analysis when used postoperatively in Greek patients who underwent surgical resection of lower extremity sarcoma. However, only the TESS showed strong correlation with the SF-36, indicating that MSTS for lower extremity was not as powerful for the evaluation of the global health status of these patients.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 24 KB)

Supplementary file2 (DOCX 36 KB)