Chinese translation and validation of the Sport Concussion Assessment Tool 3 (SCAT3).

OBJECTIVE: The aim of this study was to translate, adapt and validate the Sport Concussion Assessment Tool 3rd edition (SCAT3), a test for assessing athletes for concussion, into the Chinese context.
METHODS: Translation and adaptation were performed in several stages, which included forward translation by two independent teams, translation merging, backward translation, reviews by both native Cantonese-speaking and Mandarin-speaking multidisciplinary expert panels (n=49) for semantic and conceptual equivalence and reviews by pitch-side physiotherapists (n=18) as end-users of the SCAT3 and rugby players (n=11) for face validity. The Serial 3  s subtraction test was used as a substitute for the Months Backward Test (MBT) for measures of concentration in the Standardized Assessment of Concussion subscale. English-speaking and Chinese-speaking rugby players (n=52) were recruited to perform these tests to assess the level of difficulty, time for completion and accuracy. Inter-rater and test-retest reliability were assessed in 33 and 38 healthy young individuals, respectively.
RESULTS: Despite the longer mean completion time (p<0.05) for the Serial 3  s test, no significant difference was found in the percentage accuracy between MBT and the Serial 3  s test. No significant difference was found in either the percentage accuracy or completion time between English-speaking and Cantonese-speaking rugby players. All subscales in the Chinese SCAT3 had excellent levels of inter-rater reliability for all items (ICC2,1 range: 0.96-0.99) but a low to moderate test-retest reliability (ICC3,2 range: 0.32-0.65). The mean completion time of the Chinese SCAT3 was 10.6±1.1  min.
CONCLUSION: Chinese SCAT3 is a valid instrument for pitch-side assessment of concussed Chinese-speaking athletes.

A Chinese version of the Sport Concussion Assessment Tool 3 (SCAT3) has been successfully culturally adapted and validated.

This instrument has wide applicability for both Mandarin-speaking and Cantonese-speaking athletes.

Recently, there has been a large increase in rugby participation in China and Hong Kong.

The Chinese version of SCAT3 developed in this study can be used at the sidelines to evaluate suspected concussion in both Mandarin-speaking and Cantonese-speaking athletes.

Adaptation of the Chinese SCAT3 facilitates comparison of concussion injury surveillance studies performed in different countries.

Introduction

Concussion is a form of traumatic brain injury induced by biomechanical forces leading to the temporary loss of brain function. This injury is one of the most complex of injuries in sports medicine to diagnose, assess and manage.1 Concussion affects athletes in contact and collision sports at all levels. For instance, in 2015, the England Rugby Football Union reported an incidence rate as high as 13.4 concussions per 1000 player-hours among professional rugby players.2

Given the wide variability and subtle clinical manifestations of concussions, the multimodal Sport Concussion Assessment Tool 3rd edition (SCAT3) was recommended as the framework for sideline concussion evaluation.3 This instrument can be adopted for sideline examination to diagnose sports-related concussion and can be used to prevent premature return-to-play decisions, which exposes athletes to heightened risks of further injury and to monitor patient recovery. The test battery includes the Glasgow Coma Scale, Maddocks questions for orientation, a graded symptom checklist, the Standardized Assessment of Concussion (SAC) for cognitive function (orientation, immediate memory, delayed recall and concentration), the modified Balance Error Scoring System (mBESS) for balance examination and coordination examination (the finger-to-nose test). The individual components of the SCAT3 have been validated for use in the assessment of acute concussion, and SCAT3 was endorsed in the 2012 Consensus Statement on Concussion in Sport.3 It is one of the most widely used tools for evaluating sports-related concussions.3

SCAT3 was developed in English, which limits its widespread applicability. There are currently no standardised, cross-culturally adapted and validated concussion assessment tools for Chinese-speaking athletes. In addition, there is an absence of studies supporting the clinical utility of English concussion assessment tools for Chinese-speaking rugby players. Rugby has become increasingly popular among the Chinese. In 2016, there were 14 507 registered rugby players in Hong Kong, 35 361 in China and 5900 in Taiwan.4 Thus, there is a need for a valid and reliable Chinese adaptation of the SCAT3 to help medical professionals evaluate Chinese-speaking athletes for concussion and make appropriate clinical decisions. Mandarin Chinese is the official language in mainland China and Taiwan, whereas Cantonese, a Chinese dialect, is the primary language in Southeast China and Hong Kong. Although mutually unintelligible in their spoken forms, Cantonese and Mandarin are etymologically related and grammatically similar in most other major aspects. They share similar orthographic writing systems with several cognate words.5 6

The aims of this study were to translate and cross-culturally adapt the SCAT3 into a standardised Chinese version (Chinese SCAT3) that can be used by both Cantonese-speaking and Mandarin-speaking individuals and to evaluate the psychometric properties of this adaptation.

Methods

Ethics approval

Informed consent was obtained from all subjects.

Translation and adaptation process

Translation and adaptation procedures were performed in accordance with international translation guidelines for healthcare research.7 8 The process included six stages:

Stage 1: forward translation

In this stage, two independent bilingual teams, each containing two final-year physiotherapy students with Chinese as their native primary language, translated the English SCAT3 version into Chinese. The translation was performed with special attention to cross-cultural equivalence.

Stage 2: synthesis of translations

The two teams and two registered sports physiotherapists met to address discrepancies between the two translated versions. The physiotherapists were not involved in the forward translation and were able to provide impartial advice regarding observed discrepancies. All issues, particularly those related to the terms for symptoms and word lists for immediate memory recall, were resolved by consensus to obtain the first common translation.

Stage 3: backward translation

In this stage, a bilingual native Chinese-speaking university graduate of English language studies was made responsible for back-translating the first common translation from Chinese to English. The translator had no medical background and was blind to the content of the original English version of the SCAT3. This backward translation was compared against the original test to detect translation errors or unexpected interpretations of ambiguous items. Amendments were made accordingly by the two teams and sports physiotherapists to generate the penultimate version.

Stage 4: expert panel appraisal

The penultimate version was appraised by two expert panels. The first panel consisted of experts in Hong Kong, where Cantonese is the main language. It included a neurosurgeon, a neuropsychologist, two clinical psychologists and two sports physiotherapists, all of who were bilingual in English and Chinese. This panel examined the semantic and conceptual equivalence of this version. Modifications were made according to their comments, which were based on knowledge unique to their respective disciplines. This Chinese-modified version was then reviewed by a panel of 43 Mandarin-speaking medical professionals in mainland China (10 psychiatrists, 3 orthopaedic surgeons, 1 neurologist, 1 clinical psychologist and 28 rehabilitation therapists) for inappropriate wording or expressions in the Mandarin context. Each comment or suggestion was discussed, addressed and resolved by consensus.

Stage 5: testing of the penultimate version

The test was assessed by two groups. The first group included bilingual physiotherapists (SCAT3 end-users) (n=18) with more than 1 year of field experience using the original SCAT3. They were asked to rate the whole tool with response options ‘yes/no’ for clarity, comprehensibility, absence of misinterpretations, conceptual equivalence to the original version, grammatical accuracy and coherence. They were asked to make remarks or suggestions regarding this version. The second group included Cantonese-speaking rugby players of the United Services Recreation Club (USRC) Tigers Rugby Football Club, the city’s largest rugby club (n=11). They were asked to comment on clarity and comprehensibility of each test question.

Stage 6: expert panel appraisal

In this last stage, the two expert panels and the research team appraised the final version of the Chinese SCAT3.

Alternative for the Months Backward Test (MBT)

During stages 4 and 5, the expert panels and SCAT3 end-users raised a concern regarding the MBT. MBT is part of the SAC subscale for assessing concentration in the original SCAT3. It is a word sequence production test requiring the respondent to recite months in reverse order starting with ‘December’, ‘November’ and so on, until the individual reaches ‘January’ or is unable to continue.9 However, in the Chinese language context, reciting months in reverse order merely requires ordinal backward counting (ie, from 12, 11 and so on), which is less cognitively demanding and insufficient for assessing a subject’s concentration.

The Serial 3 s test was recommended by the experts as a substitute for MBT. In this test, an individual is asked to successively subtract ‘3’ from ‘100’ continuously until instructed to stop.10 The Serial 3 s test evaluates concentration and attention, as it requires sequential registration, recall and mental manipulation of numbers. It is convenient to administer and easy to grade and was thus considered suitable for use in sideline examination. The instruction for the test was: ‘Now take three from 100 and continue to subtract until I ask you to stop. So, you will say 97, 94…Go ahead’. The sequence was terminated after 12 consecutive subtractions. No time limit was imposed, and similar to the MBT, a score of one point was given for correctly performing the task without error. To evaluate whether the Serial 3 s test was a suitable alternative, we recruited English-speaking (n=24) and Cantonese-speaking (n=28) rugby players. We excluded players with a history of suspected concussion during the past year. English-speaking players performed both MBT and the Serial 3 s test in a randomised order separated by 1 min, whereas Cantonese-speaking players performed the Serial 3 s test only. Accuracy and time to completion were recorded and compared within and across the two player groups. Players were also asked to comment on the difficulty of the tests.

Validation

Internal consistency of the final Chinese SCAT3 version was evaluated. Inter-rater and test–retest reliability was assessed in healthy Cantonese-speaking adults. To establish inter-rater reliability, two assessors with experience using SCAT3 independently scored the Chinese SCAT3; one assessor was randomly chosen as the instructor to read aloud the standardised instructions. Test–retest reliability was assessed by administering the Chinese SCAT3 twice at an interval of 7 days apart. The completion time was recorded to address feasibility.

Statistical analysis

Statistical analyses were performed using SPSS V.22.0. Descriptive statistics were used to report subject demographics. Internal consistency was assessed using Cronbach’s α. Intraclass correlation coefficients were used to assess inter-rater reliability (ICC2,1) and test–retest reliability (ICC3,2). Unpaired t-test was used to compare Serial 3 s test performance between English- and Cantonese-speaking rugby players. Paired t-test was used to compare performance between the Serial 3 s test and MBT in English-speaking rugby players. All values were expressed as mean±SD. Significance was set at α=0.05.

Results

Translation and adaptation

In merging forward translations, agreement between speech habits and preferences was sought. Discrepancies regarding choices of synonyms were resolved. For instance, the Chinese expression for ‘low energy’ used in one forward translation was discarded, as it may be misinterpreted as low energy at only the psychological level. There were also different interpretations. For example, the Maddocks question ‘What team did you play last week/game?’ was interpreted by some study subjects as the team the player belonged to; it was later clarified as the opposing team. Comparing the back-translated version with the original SCAT3, we found discrepancies in synonym choices. The backward translator misinterpreted and translated several terms, such as ‘sense of powerlessness’, ‘state of consciousness’ and ‘shaking’, which corresponded to the original English SCAT3 terms ‘low energy’, ‘mental status’ and ‘unsteadiness’. Translations were therefore modified to convey each term’s original meaning.

Review

The expert panel suggested that English words for testing immediate memory and delayed recall in the SAC subscale be translated into words that are common across Cantonese and Mandarin cultures.

In the context of neuropsychological testing, word lists may contain words from the same semantic categories, and all lists should be synchronised (eg, body part, fruit, vehicle, animal and drink). However, the original English SCAT3 does not exhibit this feature, as the SAC word lists were simply chosen by the instrument developers from a pool of commonly used and phonologically similar English words. Since Chinese is not a phonetic language but logographical (ie, pictorial), native Chinese speakers predominantly rely on written Chinese characters to represent their language. Despite a 70% similarity between written Mandarin and Cantonese,5 11 colloquial Cantonese diverges from Mandarin to a greater extent in its lexical aspects.12 In light of this, substitutions for the SCAT3 English words were made based on three criteria: (1) words that were identically equivalent for both Cantonese and Mandarin in the written form (ie, same Chinese vocabulary); (2) words that were commonly used by native Cantonese and Mandarin speakers (ie, vernacular objects); and (3) words that required disyllabic pronunciation table 1). The chosen words were concrete and relatively simple common nouns in terms of grammatical complexity and level of visual imagery. The list was agreed on and approved by the expert panel.

Table 1

Substitutions of the English words used for assessing immediate memory and delayed recall in the SAC subscale

Words in original SCAT3	Replaced with	Reasons
		Different Chinese vocabulary
Elbow	Wrist (手腕)	Cantonese: 手踭; Mandarin: 手肘
Carpet	Slippers (拖鞋)	Cantonese: 地氈; Mandarin: 地毯
Baby	Student (學生)	Cantonese: 啤啤; Mandarin: 嬰兒
Monkey	Lion (獅子)	Cantonese: 馬騮; Mandarin: 猴子
Blanket	Towel (毛巾)	Cantonese: 毛氈; Mandarin: 毛毯
Iron	Television(電視)	Cantonese: 燙斗; Mandarin: 熨斗
		Not commonly used in Chinese vernacular
Saddle	Subway (地鐵)	馬鞍
Bubble	Softdrink (汽水)	氣泡
Wagon	Train (火車)	馬車
Lemon	Melon (西瓜)	檸檬 (lemon seldom used in North China)
Sandwich	Bread (麵包)	三文治 (three syllables in Cantonese)

In the SCAT3 end-users’ review, Chinese SCAT3 were rated as ‘clear’, ‘easy to understand’, ‘not misinterpretable’, ‘conceptually equivalent to the original version’, ‘grammatically correct’ and ‘coherent’, with 80%–100% agreement for each test characteristic. In the players’ review, all sections of the Chinese SCAT3 were considered ‘clear’ and ‘easy to understand’, except for this Maddocks score question: ‘which half is it today?’. It was wrongly perceived by two players as which half of the court they were at and was subsequently revised to ‘is it the first or the second half of the game now?’.

Serial 3 s test substitution for MBT

As noted previously, the expert panel expressed concerns that direct translation of MBT could reduce the level of difficulty and thereby suggested using the Serial 3 s test instead. Fifteen SCAT3 end-users (85%) thought the two tests reflected similar difficulty. For the English-speaking players, more time was required to complete the Serial 3 s test than MBT (p<0.05), but the percentage accuracy was similar for both tests (table 2). Of the 24 English-speaking players, 13 (54%) perceived Serial 3 s as more difficult, 6 (25%) perceived MBT as more difficult and 5 (21%) considered both tests to be of similar difficulty. There was no significant difference in the percentage accuracy or completion time of the Serial 3 s test between English-speaking and Cantonese-speaking subjects.

Table 2

Completion time and accuracy for Months Backward Test and Serial 3 s test

	English-speaking subjects (n=24)	Cantonese-speaking subjects (n=28)
Gender (M:F)	18:6	24:4
Age (years)	23.4±5.0	25.2±5.8
Education (years)	14.9±2.0	14.8±1.5
Months Backward Test
Time to complete (s)	14.2±7.7	–
Accuracy (%)	100	–
Serial 3 s test
Time to complete (s)	21.4±11.7*	21.1±10.7
Accuracy (%)	98.3±6.0	98.2±4.2

Values are mean±SD.

*p<0.05 versus MBT.

Internal consistency

The Cronbach’s α for the whole SCAT3 was 0.56. The Cronbach’s α for the symptom scale was 0.95, indicating that the measurement was highly reliable. The item–total correlations of the SAC and mBESS ranged from 0.21 to 0.75 and 0.17–0.33, respectively.

Inter-rater and test–retest reliability

Inter-rater reliability of the 33 Cantonese-speaking young adults (11 M:22 F; 21.0±0.8 years; 14.9±0.5 years of education) was excellent in all subscales, with ICC2,1 values of 0.96–1.00.

Test–retest reliability tests were conducted on 38 Cantonese-speaking young adults (13 M:25 F; 21.6±1.7 years; 15.1±0.73 years of education). The results were overall lower than that of the inter-rater results. The symptom scale, SAC and tandem gait showed moderate test–retest results13 (ICC3,2 range: 0.53–0.65). The errors in the balance tests showed a low test–retest value (ICC3,2=0.39) (table 3).

table 3

Inter-rater reliability and test–retest reliability

Variables	Inter-rater reliability ICC2,1 (95% CI)	Test–retest reliability ICC3,2 (95% CI)
Symptom scale
Number of symptoms	0.99 (0.99 to 1.00)	0.64 (0.40 to 0.79)
Symptom severity score	0.99 (0.99 to 1.00)	0.53 (0.26 to 0.73)
SAC: total score	0.98 (0.96 to 0.99)	0.46 (0.17 to 0.68)
mBESS
Number of errors	0.97 (0.94 to 0.99)	0.39 (0.09 to 0.63)
Tandem gait	0.96 (0.91 to 0.98)	0.65 (0.42 to 0.80)
Coordination	1.00	1.00

ICC, intraclass correlation coefficient; SAC, Standardized Assessment of Concussion; mBESS, modified Balance Error Scoring System.

The mean time to complete the Chinese SCAT3 was 10.6±1.1 min; this was comparable with the time required (10 min) for completing the English SCAT3.

Discussion

The English version of SCAT, developed in 2005, is available in Japanese, German and Spanish; however, these versions have not been validated. Because of cultural and linguistic differences, the processes of validation and cross-cultural adaptation are important when an instrument is used in another language. In this study, a standardised Chinese version of SCAT3 was developed through stringent procedures to ensure that the quality of the cross-culturally adapted version would be equivalent to that of the original version.

Semantic and conceptual equivalence

Semantic and conceptual equivalence are crucial for developing congruent translation and adaptation that retain the original purposes and properties of the tool.7 8 Further recommendations from SCAT3 end-users and the rugby players improved the clarity and comprehensibility of the Chinese SCAT3. The translated version was thoroughly evaluated by multiple review sources to ensure conceptual equivalence. Medical professionals from relevant disciplines clarified the concepts regarding various aspects of concussion and contributed knowledge unique to their field, whereas target populations (SCAT3 users and rugby players) provided opinions regarding its practical application.

Serial 3 s test as a substitute for MBT

MBT is designed to include factual knowledge and to involve an automatic word sequence that is well known and universal. As noted by the expert panel, reverse recitation of the months of the year required less concentration to assess this cognitive domain in the Chinese context. The Serial 3 s test is a suitable substitute for MBT as it is a common cognitive test that involves a sequential task and is easy to adminster.10 Although the Serial 3 s test differs from the MBT by requiring foundational subtraction skills, most individuals with an elementary education or above should be capable. We pilot-tested other alternatives, including the seven-item backward reciting of the colours of the rainbow and enumerating the 12-item Chinese zodiac in reverse order but discarded them.

MBT and the Serial 3 s test were assessed in rugby players to determine whether they produced similar results. In 12 consecutive subtractions from ‘100’, there are nine basic subtractions and three regrouping calculations (91–3, 82–3 and 70–3) in the Serial 3 s test. Regrouping calculations require more cognitive function and are thus more challenging than MBT. Despite higher perceived level of difficulty and longer time to complete in the Serial 3 s test, no significant difference was found on accuracy between the MBT and Serial 3 s tests in English-speaking players. Furthermore, the performance between English-speaking and Chinese-speaking players on the Serial 3 s test was not significantly different.

Psychometric properties of SCAT3

The Chinese SCAT3 completion time of approximately 10 min makes it feasible for sideline use. All subscales had excellent inter-rater reliability.

The internal consistency for the whole Chinese SCAT3 is relatively low (0.56). There is a possible explanation of the low value. For a scale where the item scores were summed to calculate a total score, the scale needs to be unidimensional to obtain an interpretable alpha value. The items in the SCAT3, however, cover many different facets of concussion, reflecting the protean nature of its symptoms, and the numerous constructs of this assessment tool may not necessarily be highly inter-related.14 For example, a concussed individual with impaired cognition may not concomitantly have poor balance. For similar reasons subscales like the SAC, it may not be justified to calculate an alpha value for all cognitive domains (‘orientation’, ‘memory’ and ‘concentration’) to produce an overall measure of internal consistency regarding ‘cognition’ in general. In contrast results for the symptom subscale demonstrated high internal consistency (0.96). Similar findings have been reported across collegiate students (0.88–0.94)15 and amateur rugby players (0.98).16 Indeed, for discriminating between concussed and non-concussed athletes, the symptom evaluation subscale exhibited the largest effect size, whereas SAC and mBESS had small to moderate effect sizes.17

Test–retest reliability and discriminative ability of the original SCAT3 have been reported. In high school and collegiate athletes, Chin et al 17 found low–moderate test–retest reliabilities for SCAT3 components in control subjects at the 7-day interval. Lee et al 18 administered SCAT3 twice to male professional rugby and football players 3 months apart and reported moderate test–retest reliabilities for symptom evaluation and SAC subscales but low reliability for mBESS. Our data concur with the literature for test–retest results for mBESS. We noted that the low variance associated with the double-leg stance accounted for the low ICC values as all subjects scored fully for this test. A previous study revealed that overall test reliability improved significantly when the double-leg stance test was removed.19

Concerning the psychometric properties of the SAC subscale, prior reports noted a sensitivity of 0.80–0.94 and specificity of 0.76–0.91.20 Whereas SAC was useful immediately postinjury to differentiate concussed from non-concussed players, its discriminative ability decreased within 5 days postinjury.20 21 Studies are limited regarding the utility of SCAT3 components for tracking recovery in concussed rugby players over an extended time. Previous literature regarding the original SCAT3 might provide hints regarding the properties of the Chinese SCAT3, but actual testing using this translation is recommended.

Areas for further study

This study’s scope did not include evaluating the discriminative ability of the Chinese SCAT3 in concussed and non-concussed rugby players or validating the Serial 3 s test cut-off (12 subtractions) in concussed players. It also did not assess players with different skill levels or athletes participating in other sports. Nonetheless, this study provides a basis for future studies evaluating the Chinese SCAT3 in different target populations.

The Chinese SCAT3 was developed for both the Cantonese and Mandarin context, and it was reviewed by both Cantonese and Mandarin speakers, with face validity established. It was shown to be valid and reliable in Cantonese-speaking rugby players; further studies should evaluate the reliability in Mandarin-speaking rugby players. The latest English version, SCAT5, was recently published in 201722 after the current study was completed. If deemed necessary, the research team will translate the SCAT5 for use in Chinese-speaking athletes.

Conclusion

This study provides a standard, valid and reliable Chinese version of SCAT3 that can be used by medical professionals in sideline concussion screening in native Chinese-speaking rugby players.