Validation and Reliability of the Korean Version of the Sport Anxiety Scale-2.

The main purpose of the present study was to examine the validation and reliability of the Korean version of the Sport Anxiety Scale (SAS-2Kr) by evaluating its factorial invariance across gender. A total of 303 Korean collegiate athletes (198 males and 105 females) from 9 sports participated in the study, and they completed the demographic questionnaire and the SAS-2Kr containing 15 items to measure multidimensional trait anxiety and individual differences in the cognitive and somatic anxiety experienced by athletes. The results of this study indicated that the construct validity in the SAS-2Kr was well established in that the values of the standardized factor loadings, composite reliability, and average variance extracted values were above the recommended cutoff points. The multiple-sample confirmatory factor analyses showed the SAS-2Kr could be generalizable across gender in college samples. The results also indicated that the SAS-2Kr supported the original 3-factor model of SAS-2 in English consisting of somatic anxiety, worry, and concentration disruption, and thus this study provides useful information for researchers to understand the athletes' tendency to experience anxiety reactions in sport situations. Suggestions for future research on competitive trait anxiety are provided in the discussion section.

Introduction

Anxiety is one of the most frequently researched constructs in sport and exercise psychology (Cox et al., 2003; Lundqvist and Hassmén, 2005) because it is a common emotional experience in stressful situations (Spielberger, 1972). Athletes can experience a great deal of performance-related stress in competitive sport. Precompetitive anxiety has been a particular focus of interest in sport and exercise psychology research (Martens et al., 1990b). Anxiety in sports can have detrimental effects on performance (Hayslip et al., 2010; Wilson et al., 2006) and significant relationships have also been reported with perfectionism (i.e., negative reaction to imperfection; Stoeber et al., 2007), injury (Lavallée and Flint, 1991), and burnout in athletes (Cremades et al., 2011). Importantly, athletes with a high level of trait anxiety had a higher level of state anxiety than athletes with a lower level of trait anxiety (Hanton et al., 2002; Martens et al., 1990b). Thus, understanding sport-specific trait anxiety among athletes is important.

In early studies, researchers modified and used general anxiety measures to examine anxiety in sports, but they found sport-specific anxiety measures to be better predictors of athletes’ behavior (Martens et al., 1990b; Smith et al., 1990). Martens (1977) developed the first sport-specific measure for competitive trait anxiety, i.e., the Sports Competition Anxiety Test (SCAT; as cited in Martens et al., 1990b). Although research showed that reliable and valid data could be obtained using the SCAT, its unidimensional nature rendered it inappropriate for examination of cognitive and somatic anxiety subcomponents of competitive trait anxiety. Martens et al. (1990a) introduced a multidimensional construct of anxiety in sport that consists of two components: a cognitive anxiety component and a somatic component. Cognitive anxiety is defined as “the mental component of anxiety and is caused by negative expectations about success or by negative self-evaluation” (Martens et al., 1990b). Somatic anxiety is defined as “the physiological and affective elements of the anxiety experience that develop directly from autonomic arousal” (Martens et al., 1990b). Common somatic anxiety responses include muscle tension, rapid heart rate, clammy hands, and butterflies in the stomach. Each component has a distinct relation to athletic performance. For example, cognitive anxiety has a negative linear relationship with performance, whereas somatic anxiety has an inverted-U (i.e., curvilinear) relationship (Martens et al., 1990b).

The Sport Anxiety Scale (SAS) was originally introduced by Smith et al. (1990) to measure multidimensional trait anxiety and individual differences in the cognitive and somatic anxiety experienced by athletes. The SAS contains 21 items for assessing anxiety response tendencies to sport-specific competitive situations. The three subscales for this scale are worry (7 items), somatic anxiety (8 items), and concentration disruption (5 items). Both subscales of worry and concentration disruption assess cognitive anxiety. Respondents rate their feelings in general before or during a competitive situation using a 4-point Likert scale. Smith et al. (1990) reported obtaining data exhibiting strong reliability and validity with the SAS in rigorous development studies.

The SAS had been used to examine the multidimensional competitive trait anxiety in sport (e.g., Giacobbi and Weinberg, 2000; Jones et al., 2004; Vazou et al., 2006). Smith et al. (2006) subsequently proposed the Sport Anxiety Scale-2. They did so after observing a five-factor solution with several items featuring cross-loadings in data obtained from a sample of children instead of the three-factor structure initially observed in SAS data obtained from older adolescents and adults (Grossbard et al., 2009). As explained by Smith and his colleagues (2006), high school and college athletes had participated in the development of the items in the SAS and hence some original items may have presented difficulties for children in understanding and responding to the items. More specifically, it may have been the case that developmentally “children’s emotional selfperception capabilities do not allow them to differentiate between the three aspects of subjectively experienced anxiety indexed by its items” (Smith et al., 2006). Adapting this notion, Smith and colleagues (2006) obtained 15 items from 30 new or revised items using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) that reproduced the original SAS structure in various age samples. Each of the 3 subscales in the SAS-2 contains five items. The SAS-2 demonstrated acceptable reliability and stronger validity than the original scale.

Competitive anxiety in sports has also been considered one of the important and influential factors on athletic performance in Korea. The Korean version of the CSAI-2 has widely been used for studies related to competitive state anxiety. However, no research has been conducted to translate and test psychometric properties of the SAS-2 in Korean for inquiry on competitive trait anxiety. The absence of a validated Korean language measure of multidimensional trait anxiety has resulted in a situation that researchers used a translated version of the unidimensional SCAT. The use of this unidimensional measure raises questions about the validity and interpretability of associated findings particularly in instances where multidimensional trait anxiety measurement would have been more appropriate. A validated multidimensional trait anxiety measurement in sport was needed so as to distinguish between cognitive and somatic anxiety dimensions. The purpose of this study was therefore to establish the validation and reliability of the Korean version of the SAS-2, and to provide psychometric properties for future research as initial information.

Methods

Participants

A total of 303 collegiate athletes (198 males and 105 females) participated in the study; their age ranged from 19 to 25 years old (M = 20.60, SD = 1.44). The subjects participated in various sports, including handball (16 males and 16 females); field hockey (17 males and 16 females); fencing (23 males and 9 females); taekwondo (26 males and 15 females); gymnastics (15 males and 10 females); track and field (21 males and 7 females); judo (17 males and 21 females); weightlifting (15 males); boxing (33 males and 1 female); and badminton (15 males and 10 females).

Measures

The participants first responded to demographic questions (e.g., gender, age, and sport types) and then the Korean version of the Sport Anxiety Scale-2 (SAS-2Kr). The original Sport Anxiety Scale developed as a sport-specific measure of trait anxiety (Smith et al., 1990) was adapted for a broader population from children to adults (SAS-2; Smith et al., 2006). The SAS-2 consists of 15 items, with five items on each of 3 subscales: worry as cognitive anxiety (e.g., “I worry that I won’t play well”), somatic anxiety (e.g., “My muscles feel tight because I am nervous”), and concentration disruption (e.g., “It is hard for me to focus on what I am supposed to do”). Each item is measured on a four-point Likert scale from 1 (not at all) to 4 (very much so). The SAS-2 was reported to have acceptable reliability (Cronbach’s alpha coefficients between .74 and .92) and greater factorial validity across samples from various age groups than the original SAS (Smith et al., 1990).

In translating the measure of the SAS-2 from English to Korean, the translation-back translation procedures outlined by Vallerand et al. (1992) were used. In brief, this entailed translating the existing, established measure of the core constructs from the SAS-2 into Korean by the first author and two Korean doctoral students who used to be competitive athletes and were studying sport psychology or sport science in the United States. A Korean scholar with approximately ten years of research experience in the field of sport psychology and the first author compared, evaluated and modified the SAS-2 translated into Korean to reconcile any differences based on the scholar’s suggestion. Two Korean scholars familiar with competitive anxiety in sport completed the Korean version of the SAS-2 as a face validity check of items. Finally, two Korean professors and the first author, all of whom earned doctoral degrees in sport and exercise psychology in the United States, translated the Korean version of the SAS-2 back into English without access to the original English version. The Korean professors, second author and first author subsequently discussed the inconsistencies between back-translated versions and the original version until the agreement was achieved to arrive at a final Korean-language version of the instrument for use in this study.

Procedures

After the Institutional Review Board (IRB) approval was obtained, the first author contacted coaches of collegiate sport teams in Korea via phone calls. He explained the purpose of this study to the coaches and asked whether they would forward the email containing the survey link to their teams. The first author sent follow-up emails to coaches agreeing to have their teams participate in the study that included the survey link for forwarding to athletes. Participants receiving the email from the coaches were asked to sign informed consent forms and then complete the demographic information and the SAS-2Kr, respectively.

Statistical Analysis

Descriptive statistics, univariate skewness, univariate kurtosis, and item intercorrelations were calculated using the Statistical Package of the Social Sciences (SPSS 18.0). Absolute values of 2 for skewness and 7 for kurtosis were used for the cut-off criteria of the univariate normality assumption (Curran et al., 1996). The univariate normality was supported in that the absolute value of each item’s skewness was below 2 and kurtosis was below 7. Additionally, Amos 18 (Arbuckle, 2009) was also used to check the multivariate normality assumption. The critical ratio of the Mardia’s coefficient of multivariate kurtosis for the multivariate value was greater than 1.96 (Mardia, 1970). That is, Mardia’s value indicated statistically significant deviation from multivariate normality in the sample. When the multivariate normality assumption is violated (i.e., the data has excessive skewness and/or kurtosis), maximum likelihood estimation of parameters can be problematic in providing biased and/or incorrect estimates (McDonald and Ho, 2002). Robust maximum likelihood estimation was, therefore, used to analyze the multivariate non-normal data. Mplus 6.0 (Muthén and Muthén, 2010) was used for confirmatory factor analysis (CFA) to examine the three-factor model of the SAS-2 because Amos 18 does not provide robust maximum likelihood estimation to analyze data that are not normally distributed.

Several fit indices, as well as the chi-square (χ2) test, were used to assess the model fit to the data because using only the χ2 test is not, in itself, “a reliable guide to model adequacy” (Hu and Bentler, 1998). Because the χ2 is sensitive to sample size and the multivariate normality of the data, especially when sample size is over 200, the normed chi-square (i.e., the ration of the chi-square to the degree of freedom; NC) was also calculated. The NC ranging from 2 to 3 was considered acceptable (as cited in Arbuckle, 2009). The fit indices considered for model comparisons were comparative fit index (CFI), standardized root mean square residual (SRMR), and root mean square error of approximation (RMSEA). According to Hu and Bentler’s (1999) recommendations, a value of .90 for the CFI was regarded as the minimum value for a marginal fit (values of .95 or greater regarded indicative of an excellent fit), and SRMR values of .08 or less being indicative of acceptable fit. RMSEA values below .08 were also considered indicative of acceptable fit (MacCallum et al., 1996).

Factor loadings, composite reliability (CR) and average variance extracted (AVE) were calculated for construct reliability and validity. CR coefficients provide an estimate of the extent to which the set of indicators within the same latent construct is internally consistent, whereas the AVE value is the amount of common variance that indicators in the same latent construct capture (Hair et al., 2010). Hair et al. (2010) suggest that values above 0.7 of CR and 0.5 of AVE indicate adequate convergent validity. The square root of AVE value of a specific factor must be greater than its correlations with other factors for discriminant validity (Fornell and Larcker, 1981). Thus, the CR and AVE values along with the square roots of AVE values compared with correlations among constructs were calculated.

Multiple-sample CFA was conducted to test measurement invariance across gender because there would be a construct bias if the psychometric properties of a measure were different in one group than in another. As suggested by Thompson and Green (2006), a four-step stepwise approach used for this study consisted of hierarchical models of CFA that included increasingly restricted levels of invariance. In Step 1, configural invariance was tested by evaluating the same factor model as the baseline model for further comparisons. Step 2 tested for metric invariance by evaluating the equivalence of factor loadings between athlete gender groups. In Step 3, scalar invariance was examined by evaluating the equivalence of intercepts for the measures between athlete gender groups. Finally, the latent factor mean invariances were evaluated to detect whether the factor means were significantly different across athlete gender groups.

According to Smith et al. (2006), the SAS-2 is able to test a specific subcomponent of performance anxiety by providing scores of each factor (i.e., either one somatic or two cognitive factors), and also examine the global construct of performance anxiety using a total score of all subscales. In line with the study conducted by Smith et al. (2006), a global single-factor model, a three-factor model without the second-order global factor, and a three-factor higher order model were evaluated.

Results

Descriptive Statistics on the SAS-2Kr

A total of 308 participants completed the survey. Five participants had some missing demographic data and also not completed the SAS-2Kr. Thus, their data were removed from the analyses, and the data from 303 participants were used for statistical analyses.

Descriptive statistics, reliability estimates (i.e., Cronbach’s alpha), and correlations for three factors in the SAS-2Kr are presented in Table 1 along with scores of the SAS-2 published by Smith et al. (2006) on the collegiate athletes. Large correlations were observed among all factors in the SAS-2Kr, whereas only one correlation was observed in the SAS-2. The inter-factor correlations in the SAS-2Kr showed the same direction of those in the SAS-2, but the strengths in the SAS-2Kr were different from those in the SAS-2. The correlation between somatic anxiety and concentration disruption was the largest in the SAS-2Kr, whereas the correlation between somatic and worry was the largest in the SAS-2. Values of Cronbach’s alpha for reliability in the SAS-2Kr were .78 for somatic anxiety, .88 for worry, and .83 for concentration disruption.

Table 1

Descriptive Statistics, Cronbach’s Alpha (α), and Correlations for all Factors of the SAS-2Kr and the SAS-2

Factors	N	Mean (SD)	α	Somatic Anxiety	Worry
SAS-2Kr	303
Somatic Anxiety		1.79 (.54)	.78
Worry		2.08 (.68)	.88	.63**
Concentration Disruption		1.74 (.54)	.83	.67**	.59**
Total Score		1.87 (.51)	.91
SAS-2	593
Somatic Anxiety		1.95 (.72)	.89
Worry		2.42 (.77)	.91	.55
Concentration Disruption		1.38 (.47)	.84	.35	.47
Total Score		1.92 (.53)	.91

The values of α in the SAS-2 reported are for college samples.

Correlation is significant at p < .001

Confirmatory Factor Analyses

The CFA for the global single-factor model of the SAS-2Kr was first conducted. The chi-square was significant, χ2 (90) = 460.22, p < .001, and the NC, CFI, RMSEA, and SRMR were 5.11, .80, .117, and .08, respectively. This was consistent with the findings by Smith et al. (2006) that the global single-factor model had a poor fit to the data. The CFA for the three-factor model was conducted. The chi-square statistic was significant, χ2 (87) = 230.08, p < .001, and the NC, CFI, RMSEA, and SRMR were 2.64, .92, .07, and .06, respectively.

The chi-square value and other fit indices showed that the three-factor model had an acceptable fit. The standardized factor loadings, construct reliability, and average variance extracted value for the SAS-2Kr are presented in Table 2. The factor loadings of the SAS-2 are also provided in the table. In the SAS-2Kr, the values of all factor loadings were large (i.e., the ranges of the loadings were from .63 to .85) except Item 2. The three-factor higher order model was also evaluated, and it provided an adequate fit to the data (i.e., χ2 (87) = 230.08, p < .001, NC = 2.64, CFI = .92, RMSEA = .07, and SRMR = .06). The CR values, ranging from .80 to .89, were above the suggested value of .70. The values of the AVE ranged from .45 to .51, with 2 out of 3 exceeding the recommended threshold of .50, indicating modest convergent validity for each construct. For discriminant validity, the square root of the AVE for a particular construct should be greater than its correlations with other constructs (Fornell and Larcker, 1981). The square roots of the AVE value for the SAS-2Kr were .89 for somatic, .94 for worry, and .91 for concentration disruption. All the values were greater than their correlations with other factors.

Table 2

Factor Loadings, Construct Reliability (CR) and Average Variance Extracted (AVE) Value of the original SAS-2 and the SAS-2Kr

Item	Factors and Wordings	Loading		CR	AVE
		SAS-2	SAS-2Kr
	Somatic Anxiety			.80	.45
Item 2	My body feels tense.	.65	.50
Item 6	I feel tense in my stomach.	.74	.69
Item 10	My muscles feel shaky.	.68	.75
Item 12	My stomach feels upset.	.71	.68
Item 14	My muscles feel tight because I am nervous. Worry	.73	.70	.89	.62
Item 3	I worry that I will not play well.	.74	.73
Item 5	I worry that I will let others down.	.68	.75
Item 8	I worry that I will not play my best.	.81	.79
Item 9	I worry that I will play badly.	.81	.85
Item 11	I worry that I will mess up during the game. Concentration Disruption	.74	.80	.84	.51
Item 1	It is hard to concentrate on the game.	.64	.63
Item 4	It is hard for me to focus on what I am supposed to do.	.68	.78
Item 7	I lose focus on the game.	.66	.74
Item 13	I cannot think clearly during the game.	.73	.69
Item 15	I have a hard time focusing on what my coach tells me to do.	.66	.70

All the factor loadings in the SAS-2Kr are significant at p < .001

The results of the measurement invariance tests for the SAS-2Kr are shown in Table 3. Adjusted χ2 difference tests were used for the chi-square difference test (Satorra and Bentler, 2001). The configural model (Model 1) was first tested to confirm whether two groups (i.e., male and female groups) had the same factor structure. The configural model was least constrained. The chi-square value was significant, χ2 (174) = 334.74, p < .001, and the NC, CFI, RMSEA, and SRMR were 1.92, .91, 07, and .06, respectively. The configural model showed an acceptable fit to the data and configural invariance across groups was tenable.

Table 3

Measurement Invariance Tests

	X2S-B	df	Correction Factor	CFI	RMSEA	SRMR
Model 1	334.74*	174	1.14	.92	.08	.07
Model 2	355.43*	186	1.13	.91	.08	.08
Model 3	370.25*	197	1.12	.91	.08	.08
Model 4a	373.91	198	1.12	.91	.08	.08
Model 4b	370.92	198	1.12	.91	.08	.08
Model 4c	371.74	198	1.12	.91	.08	.08

Multiple samples are by gender.

p < .001. Model 1 tested unconstrained (configural) invariance. Model 2 tested metric invariance. Model 3 tested scalar invariance. Model 4a tested factor mean invariance on worry across gender. Model 4b tested factor mean invariance on concentration disruption across gender. Model 4c tested factor mean invariance on somatic anxiety across gender

The metric invariance model (Model 2) constraining the loadings equal across groups indicated an acceptable fit to the data. That is, the chi-square value was significant, χ2 (186) = 355.43, p < .001, and the NC, CFI, RMSEA, and SRMR were 1.91, .91, 07, and .07, respectively. The metric invariance model did not significantly differ from the configural model on the adjusted χ2 difference test, χ2 (12) = 20.53, p > .05. Therefore, the linear relationships between items and factors were equivalent across groups. Subsequently, the scalar invariance (Model 3), constraining the intercepts equal across groups, also indicated an acceptable fit to the data, χ(198) = 384.95, p < .001, and the NC, CFI, RMSEA, and SRMR were 1.94, .90, 07, and .08, respectively. The scalar invariance model did not significantly differ from the metric invariance model on the adjusted χ2 difference test, χ2 (11) = 13.79, p > .05. Thus, tests for equivalence on factor means to test the hypotheses were able to be examined.

Lastly, a mean invariance test was performed by imposing equality constraints on factor means groups. Model 4a, 4b, and 4c indicated factor mean invariance on worry, disruption, and somatic anxiety across gender, respectively. The results of all the mean invariance models indicated an acceptable fit to the data and did significantly differ from the scalar invariance model on the adjusted χ2 difference test.

Discussion

The study examined the validity of the Korean version of the Sport Anxiety Scale-2 to measure Korean athletes’ competitive trait anxiety. The SAS-2 by Smith et al. (2006) was first translated into Korean and then the psychometric properties of the Korean version were examined using confirmatory factor analyses.

The construct validity in the SAS-2Kr was well established in that the values of the standardized factor loadings, CR, AVE values except somatic anxiety were above the suggested cutoff points. Regarding the AVE value for somatic anxiety, Item 2, “my body feels tense”, had a just marginal factor loading value. The item was also related to the model fit increment. Fit indices such as CFI, RMSEA, and SRMR could have improved if the item was removed or had cross loadings to others according to the modification indices provided by Mplus. That is, χ2 (74) = 149.79, p < .001, NC = 2.02, CFI = .96, RMSEA = .06, and SRMR = .04 if Item 2 was removed. Although the AVE value for the somatic anxiety was below the threshold of .50, the CR value was relatively high. More importantly, it was believed that the item was the representative of somatic anxiety (i.e., the item theoretically sounds). In this study, therefore, the authors decided that removing the item to improve the AVE value and overall model fit was not necessary.

The purpose of the multi-sample CFA across gender was to examine whether the results of this study could be generalized. The four-step approach for the measurement invariance test indicated that the SAS-2Kr could be generalizable across gender in college samples. In the study by Smith et al. (2006), the SAS-2, however, was purposely developed to measure competitive trait anxiety in sport for all ages (i.e., from young children to college athletes). Thus, future research should include broad age ranges of Korean samples to examine whether the SAS-2Kr can be used for all ages of athletes.

A limitation of this study might be found in the number of males versus females providing data. It is, however, important to note that uneven ratios of male and female athletes competing in sport are common (Lane et al., 1999). For measurement invariance (i.e., specifically metric invariance), sample sizes of at least 200 per group are recommended for more accurate chi-square statistic and estimated factor loadings (Meade and Bauer, 2007). In this study, the number of male participants was just above the suggested number, whereas the number of females was not. Therefore, recruiting over 200 participants per group to conduct the multi-sample CFA is suggested for future research.

Future research needs to re-examine the construct-related validity such as the convergent and discriminant validity for the SAS-2Kr. Convergent validity means that correlations among the same constructs (i.e., traits) are significantly greater than zero when measurement methods are different, whereas discriminant validity means that correlations among theoretically distinct constructs are near zero regardless of measurement method (Mayes and Ganster, 1983; Muis et al., 2007). Comparing intercorrelations between subscales in the SAS-2kr and anxiety-related or unrelated subscales in other measures could provide more theoretical evidence to evaluate the construct validity as researchers did for the validations of multidimensional trait or state anxiety inventories (Martinent et al., 2010; Smith et al., 1990, 2006). Using a multitrait-multimethod analysis could be another suggestion.

In conclusion, the purpose of this study was to examine the validation of the Korean version of the SAS-2 compared to the original version of the SAS-2. The results of the current study provide useful information on the SAS-2Kr for researchers to understand the Korean athletes’ tendency to experience anxiety reactions in sport situations and also suggestions for future research on development and evaluation of the Korean version of competitive trait anxiety measures.